Computer Science Technical Report Archive

USC Computer Science Technical Reports, no. 727 (2000)

(USC DC Other)

USC Computer Science Technical Reports, no. 727 (2000)

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Systematic T esting of Multicast Routing Proto cols Analysis of F orw ard and Bac kw ard Searc h
T ec hniques
Ahmed Helm y Deb orah Estrin Sandeep Gupta
Univ ersit y of Southern California Los Angeles CA email helm ycenguscedu estrinuscedu sandeepb o oleuscedu
A bstr act The recen t gro wth of the In ternet and its increased het
erogeneit y ha v e increased the complexit y of net w ork proto
col design and testing In addition the adv entof m ultip oin t
m ulticastbased applications has in tro duced new c hallenges
that are qualitativ ely dieren t in nature than the traditional
poin ttop oin t proto cols Multip oin t applications t ypically in
v olv e a group of participan ts sim ultaneously and hence are
inheren tly more complex As more m ultip oin t proto cols are
coming to life the need for a systematic metho d to study
and ev aluate suc h proto cols is b ecoming more apparen t Suc h
metho d aims to exp edite the proto col dev elopmen t cycle and
impro v e proto col robustness and p erformance
In this pap er w e presen t a new metho dology for dev elop
ing systematic and automatic test generation algorithms for
m ultip oin t proto cols These algorithms attempt to syn thesize
net w ork top ologies and sequences of ev en ts that stress the
proto cols correctness or p erformance This problem can b e
view ed as a domainsp ecic searc hproblem thatsuersfrom
the state space explosion problem One goal of this w ork is to
circum v en t the state space explosion problem utilizing kno wl
edge of net w ork and fault mo deling and m ultip oin t proto cols
The t w o approac hes in v estigated in this study are based on for
w ard and bac kw ard searc h tec hniques W e use an extended
nite state mac hine FSM mo del of the proto col The rst
algorithm uses forw ard searc h to p erform reduced reac habil
it y analysis Using domainsp ecic information for m ulticast
routing o v er LANs the algorithm complexit y is reduced from
exp onen tial to p olynomial in the n um b er of routers This ap
proac h ho w ev er do es not fully automate top ology syn thesis
The second algorithm the faultorien ted test generation uses
bac kw ard searc h for top ology syn thesis and uses bac ktrac king
to generate ev en t sequences instead of searc hing forw ard from
initial states
Using these algorithms w eha v e conducted studies for cor
rectness of the m ulticast routing proto col PIM W e prop ose to
extend these algorithms to study endtoend m ultip oin t pro
to cols using a virtual LAN that represen ts dela ys of the un
derlying m ulticast distribution tree
I Intr oduction
Net w ork proto cols are b ecoming more complex with the
exp onen tial gro wth of the In ternet and the in tro duction of
new services at the net w ork transp ort and application lev
els In particular the adv entof IP m ulticast and the MBone
enabled applications ranging from m ultipla y er games to dis
tance learning and teleconferencing among others T o date
little eort has b een exerted to form ulate systematic meth
o ds and to ols that aid in the design and c haracterization of
these proto cols
In addition researc hers are observing new and obscure y et
all to o frequen t failure mo des o v er the in ternets Suc h
failures are b ecoming more frequen t mainly due to the in
creased heterogeneityoftec hnologies in terconnects and con
guration of v arious net w ork comp onen ts Due to the syn
ergy and in teraction b et w een dieren t net w ork proto cols and
comp onen ts errors at one la y er ma y lead to failures at other
la y ers of the proto col stac k F urthermore degraded p erfor
mance of lowlev el net w ork proto cols mayha v e ripple eects
on endtoend proto cols and applications
Net w ork proto col errors are often detected b y application
failure or p erformance degradation Suc h errors are hardest
to diagnose when the b eha vior is unexp ected or unfamiliar
Ev en if a proto col is pro v en to be correct in isolation its
beha vior ma y be unpredictable in an op erational net w ork
where in teraction with other proto cols and the presence of
failures ma y aect its op eration Proto col errors ma y be
v ery costly to repair if disco v ered after deplo ymen t Hence
endea v ors should b e made to capture proto col a ws early in
the design cycle b efore deplo ymen t Topro vide an eectiv e
solution to the ab o v e problems w e presen t a framew ork for
the systematic design and testing of m ulticast proto cols The
framew ork in tegrates test generation algorithms with sim u
lation and implemen tation W e prop ose a suite of practical
metho ds and to ols for automatic test generation for net w ork
proto cols
Man y researc hers ha vedev elop ed proto col v erica
tion metho ds to ensure certain prop erties of proto cols lik e
freedom from deadlo c ks or unsp ecied receptions Muchof
this w ork ho w ev er w as based on assumptions ab out the net
w ork conditions that ma y not alw a ys hold in to da ys In ter
net and hence ma y b ecome in v alid Other approac hes suc h
as reac habilit y analysis attempt to c hec k the proto col state
space and generally suer from the state explosion problem
This problem is exacerbated with the increased complexityof
the proto col Muc h of the previous w ork on proto col v erica
tion targets correctness W e target proto col p erformance and
robustness in the presence of net w ork failures In addition
w e pro vide new metho ds for studying m ulticast proto cols and
top ology syn thesis that previous w orks do not pro vide
W e in v estigate t w o approac hes for test generation The
rst approac h called the faultindep enden t test generation
uses a forw ard searc h algorithm to explore a subset of the
proto col state space to generate the test ev en ts automatically State and fault equiv alence relations are used in this approac h
to reduce the state space The second approac h is called
the faultorien ted test generation and uses a mix of forw ard
and bac kw ard searchtec hniques to syn thesize test ev en ts and
top ologies automatically Weha v e applied these metho ds to m ulticast routing Our
case studies rev ealed sev eral design errors for whic hweha v e
form ulated solutions with the aid of this systematic pro cess
W e further suggest an extension of the mo del to include
endtoend dela ys using the notion of virtual LAN Suc hex tension in conjunction with the faultorien ted test genera
tion canbe usedfor p erformance ev aluation of endtoend
m ultip oin t proto cols
The rest of this do cumen t is organized as follo ws Sec
tion VI presen ts related w ork in proto col v erication con
formance testing and VLSI c hip testing Section II in tro
duces the prop osed framew ork and system denition Sec
tions III IV V presen t the searc h based approac hes and
problem complexit y the faultindep enden t test generation
and the faultorien ted test generation resp ectiv ely Sec
tion VI I concludes b y giving a summary of our w ork and
future researc h
Multicast Routing Ov erview
Multicast proto cols are the class of proto cols that supp ort
group comm unication Multicast routing proto cols include
D VMRP MOSPF PIMDM CBT and PIM
SM Multicast routing aims to deliv er pac k ets ecien tly
to group mem bers b y establishing distribution trees Figure sho ws a v ery simple example of a source S sending to a group
of receiv ers R i S
R1
R2
R3
R4 R5
S: sender to the group
Ri: receiver i of the group
Fig Establishing m ulticast deliv ery tree
Multicast distribution trees ma y be established b y either
broadcastandprune or explicit join proto cols In the former
suchasD VMRP or PIMDM a m ulticast pac k et is broadcast
to all leaf subnet w orks Subnet w orks with no lo cal mem bers
for the group send prune messages to w ards the sources of
the pac k ets to stop further broadcasts Link state proto cols
suc h as MOSPF broadcast mem b ership information to all
no des In con trast in explicit join proto cols suchas CBT
or PIMSM routers send hopb yhop join messages for the
groups and sources for whic h they ha v e lo cal mem bers W e conduct robustness case studies for PIMDM W e are par
ticularly in terested in m ulticast routing proto cols b ecause
they are vulnerable to failure mo des suc h as selectiv e loss
that ha v e not b een traditionally studied in the area of pro
to col design
F or most m ulticast proto cols when routers are connected via
am ultiaccess net w ork or LAN
hopb yhop messages are
m ulticast on the LAN and ma y exp erience selectiv e loss ie
ma y b e receiv ed b y some no des but not others The lik eli
ho o d of selectiv e loss is increased b y the fact that LANs often
con tain h ubs bridges switc hes and other net w ork devices
Selectiv e loss ma y aect proto col robustness
W e use the term LAN to designate a connected net w ork with resp ect to
IPm ulticast This includes shared media suc h as Ethernet or FDDI h ubs
switc hes etc
Similarly endtoend m ulticast proto cols and applications
m ust deal with situations of selectiveloss This dieren tiates
these applications most clearly from their unicast coun ter
parts and raises in teresting robustness questions
Our case studies illustrate wh y selectiv e loss should b e con
sidered when ev aluating proto col robustness This lesson is
lik ely to extend to the design of higher la y er proto cols that
op erate on top of m ulticast and can ha v e similar selectiv e
loss
II Framew ork O ver view
Proto cols maybe ev aluated for correctness or p erformance
W e refer to correctness studies that are conducted in the ab
sence of net w ork failures as v erication In con trast robust
ness studies consider the presence of net w ork failures suc h
as pac k et loss or crashes In general the robustness of a
proto col is its abilit y to resp ond correctly in the face of net
w ork comp onen t failures and pac k et loss This w ork presen ts
a metho dology for studying and ev aluating m ulticast proto
cols sp ecically addressing robustness and p erformance is
sues W e prop ose a framew ork that in tegrates automatic test
generation as a basic comp onentfor proto col design along
with proto col mo deling sim ulation and implemen tation test
ing The ma jor con tribution of this w ork lies in dev eloping
new metho ds for generating stress test scenarios that target
robustness and correctness violation or w orst case p erfor
mance
Instead of studying proto col b eha vior in isolation wein corp orate the proto col mo del with net w ork dynamics and
failures in order to rev eal more realistic b eha vior of proto cols
in op eration
This section presen ts an o v erview of the framew ork and its
constituen t comp onen ts The mo del used to represen t the
proto col and the system is presen ted along with denitions
of the terms used
Our framew ork in tegrates test generation with sim ulation
and implemen tation co de It is used for Systematic Testing
of Robustness by Evaluation of Synthesize d Sc enarios
STRESS As the name implies systematic metho ds for sce
nario syn thesis are a core part of the framew ork W e use
the term scenarios to denote the testsuite consisting of the
top ology and ev en ts
The input to this framew ork is the sp ecication of a pro
to col and a denition of its design requiremen ts in terms of
correctness or p erformance Usually robustness is dened in
terms of net w ork dynamics or fault mo dels A fault mo del
represen ts v arious comp onen t faults suc h as pac k et loss cor
ruption reordering or mac hine crashes The desired output
is a set of testsuites that stress the proto col mec hanisms
according to the robustness criteria
As sho wn in gure the STRESS framew ork includes
test generation detailed sim ulation driv en b y the syn thesized
tests and proto col implemen tation driv en through an em u
lation in terface to the sim ulator In this w ork wefocus on
the test generation TG comp onen t
A T est Gener ation
The core con tribution of our w ork lies in the dev elopmen t
of systematic test generation algorithms for proto col robust
Automatic
Test
Generation (ATG)
Protocol Analysis through
Simulation
Test Patterns and
Scenarios
Emulation
Interface
Design
refinement
Protocol Implementation
Test Signals
Testing
Analysis
and refinement
- Establish a protocol model (e.g. FSM)
- Obtain test sequences to stress certain
(e.g. robustness to message loss, or crashes)
aspects of the model
- Develop detailed protocol simulation
- Study the behavior under the stress
test-suites
- Implement the protocol
- Debug and study behavior using
the simulator output test signals
- Evaluate the test quality (e.g. using
code coverage)
Fig The STRESS framew ork
ness Wein v estigate t wosuc h algorithms eac h using a dif
feren t approac h
In general test generation ma y b e random or deterministic
Generation of random tests is simple but a large set of tests
is needed to ac hiev e a high measure of error co v erage Deter
ministic test generation TG on the other hand pro duces
tests based on a mo del of the proto col The kno wledge built
in to the proto col mo del enables the pro duction of shorter
and higherqualit y test sequences Deterministic TG can b e
a faultindep enden t or b faultorien ted F aultindep enden t
TG w orks without targeting individual faults as dened b y
the fault mo del Suchan approachma y employa forw ard
searc h tec hnique to insp ect the proto col state space or an
equiv alen t subset thereof after in tegrating the fault in to the
proto col mo del In this sense it ma y b e considered a v arian t
of reac habilit y analysis W e use the notion of equiv alence
to reduce the searc h complexit y Section IV describ es our
faultindep enden t approac h
In con trast faultorien ted tests are generated for sp ecied
faults F aultorien ted test generation starts from the fault
eg a lost message and syn thesizes the necessary top ology
and sequence of ev en ts that trigger the error This algorithm
uses a mix of forw ard and bac kw ard searc hes W e presen t
our faultorien ted algorithm in Section V W e conduct case studies for the m ulticast routing proto
col PIMDM to illustrate dierences b et w een the approac hes
and pro vide a basis for comparison
In the remainder of this section w e describ e the system
mo del and denition
B The system mo del
W e dene our target system in terms of net w ork and top ol
ogy elemen ts and a fault mo del
B Elemen ts of the net w ork
Elemen ts of the net w ork consist of m ulticast capable no des
and bidirectional symmetric links No des run same m ulti
cast routing but not necessarily the same unicast routing
The top ology is an N router LAN mo deled at the net w ork
lev el w e do not mo del the MACla y er
F or endtoend p erformance ev aluation the m ulticast dis
tribution tree is abstracted out as dela ys b et w een end systems
and patterns of loss for the m ulticast messages Cascade of
LANs or uniform top ologies are addressed in future researc h
B The fault mo del
W e distinguish b et w een the terms err or and faultAn err or
is a failure of the proto col as dened in the proto col design
requiremen t and sp ecication F or example duplication in
pac k et deliv ery is an error for m ulticast routing A fault is
a lo w lev el eg ph ysical la y er anomalous b eha vior that
ma y aect the beha vior of the proto col under test Note
that a fault ma y not necessarily b e an error for the lo w lev el
proto col
The fault mo del ma y include a Loss of pac k ets suc h
as pac k et loss due to congestion or link failures W e tak e
in to consideration selectiv e pac k et loss where a m ulticast
pac k et ma y b e receiv ed b y some mem b ers of the group but
not others b Loss of state suc has m ulticast andor unicast
routing tables due to mac hine crashes or insucien tmem ory resources c The dela y mo del suc h as transmission
propagation or queuing dela ys F or endtoend m ulticast
proto cols the dela ys are those of the m ulticast distribution
tree and dep end up on the m ulticast routing proto col and
d Unicast routing anomalies suc h as route inconsistencies
oscillations or apping
Usually a fault mo del is dened in conjunction with the
robustness criteria for the proto col under study F or our
robustness studies w e study PIM The designing robustness
goal for PIM is to b e able to reco v er gracefully ie with
out going in to erroneous stable states from single proto col
message loss That is b eing robust to a single message loss
implies that transitions cause the proto col to mo v e from one
correct stable state to another ev en in the presence of se
lectiv e message loss In addition w e study PIM proto col
beha vior in presence of crashes and route inconsistencies
C T est Se quenc e Denition
A fault mo del ma y include a single fault or m ultiple faults
F or our robustness studies w e adopt a singlefault mo del
where only a single fault ma y o ccur during a scenario or a
test sequence
W e dene t w o sequences T e e e n and T
e e e j f e
k
e n where e i is an ev entand f is a
fault Let P q T be the sequence of states and stim uli of
proto col P under test T starting from the initial state q T
is a test sequence if nal P q T
is incorrect ie the
stable state reac hed after the o ccurrence of the fault do es
not satisfy the proto col correctness conditions see Section I I E irresp ectiv e of P q T In case of a faultfree sequence
where T T
the error is attributed to a proto col design
error Whereas when T T
and nal P q T is correct
the error is manifested b y the fault This denition ignores
transien t proto col b eha vior W e are only concerned with the
stable ie nontransien t b eha vior of a proto col
D T est Sc enario
A test scenario is dened b y a sequence of host ev en ts a
top ology and a fault mo del as sho wn in gure Topology
Events
Faults
triggered timed interleaved
LAN
regular topologies
random
packet loss
crashes
routing
anomalies
Fig T est pattern dimensions
The ev en ts are actions p erformed b y the host and act as
input to the system for example join lea v e or send pac k et
The top ology is the routed top ology of set of no des and links
The no des run the set of proto cols under test or other sup
p orting proto cols The links can be either poin ttop oin t
links or LANs This mo del ma y b e extended later to repre
sentv arious dela ys and bandwidths b et w een pairs of no des
b y using a virtual LAN matrix see Section VI IThe fault
mo del used to inject the fault in to the test According to our
singlemessage loss mo del for example a fault ma y denote
the loss of the second message of t yp e pr une tra v ersing a
certain link Kno wing the lo cation and the triggering action
of the fault is imp ortan t in analyzing the proto col b eha vior
E Brief description of PIMDM
F or our robustness studies w e apply our automatic test
generation algorithms to a v ersion of the Proto col Indep en
den t MulticastDense Mo de or PIMDM The description
giv en here is useful for Sections I I I through V
PIMDM uses broadcastandprune to establish the m ulti
cast distribution trees In this mo de of op eration a m ulticast
pac k et is broadcast to all leaf subnet w orks Subnet w orks with
no lo cal mem b ers send prune messages to w ards the sources
of the pac k ets to stop further broadcasts
Routers with new mem b ers joining the group trigger Gr aft
messages to w ards previously pruned sources to reestablish
the branc hes of the deliv ery tree Gr aft messages are ac
kno wledged explicitly at eac h hop using the Gr aftA ck mes
sage
PIMDM uses the underlying unicast routing tables to get
the nexthop information needed for the RPF rev ersepath
forw arding c hec ks This ma y lead to situations where there
are m ultiple forw arders for a LAN The Assert mec hanism
prev en ts these situations and ensures there is at most one
forw arder for a LAN
The correct function of a m ulticast routing proto col in gen
eral is to deliv er data from senders to group mem b ers only
those that ha v e joined the group without an y data loss F or
our metho ds w e only assume that a correctness denition
is giv en b y the proto col designer or sp ecication F or illus
tration w e discuss the proto col errors and the correctness
conditions
E PIM Proto col Errors
In this study w e target proto col design and sp ecication
errors W e are in terested mainly in erroneous stable ie
nontransien t states In general the proto col errors ma y
b e dened in terms of the endtoend b eha vior as functional
correctness requiremen ts In our case for PIMDM an error
ma y manifest itself in one of the follo wing w a ys
black holes consecutiv e pac k et loss b et w een p erio ds of
pac k et deliv ery p acket lo oping the same pac k et tra v erses
the same set of links m ultiple times p acket duplic ation m ultiple copies of the same pac k et are receiv ed b y the same
receiv ers join latency lac k of pac k et deliv ery after a
receiv er joins the group le ave latency unnecessary pac k et
deliv ery after a receiv er lea v es the group
and waste d
b andwidth unnecessary pac k et deliv ery to net w ork links that
do not lead to group mem b ers
E Correctness Conditions
W e assume that correctness conditions are pro vided b y the
proto col designer or the proto col sp ecication These condi
tions are necessary to a v oid the ab o v e proto col errors in a
LAN en vironmen t and include
If one or more of the routers is exp ecting to receiv epac k
ets from the LAN then one other router m ust b e a forw arder
for the LAN Violation of this condition ma y lead to data loss
eg join latency or blac k holes
The LAN m ust ha v e at most one forw arder at a time Vi
olation of this condition ma y lead to data pac k et duplication
The deliv ery tree m ust b e lo opfree
a An y router should accept pac k ets from one incoming in
terface only for eac h routing en try This condition is enforced
b y the RPF Rev erse P ath F orw arding c hec k
b The underlying unicast top ology should b e lo opfree
Violation of this condition ma y lead to data pac k et lo oping
If one of the routers is a forw arder for the LAN then there
m ust b e at least one router exp ecting pac k ets from the LANs
Violation of this condition maylead to lea v e latency III Sear chbased Appr o a ches
The problem of test syn thesis can b e view ed as a searc h
problem By searc hing the p ossible sequences of ev en ts and
faults o v er net w ork top ologies and c hec king for design re
quiremen ts either correctness or p erformance w e can con
struct the test scenarios that stress the proto col Ho w ev er
due to the state space explosion tec hniques m ust be used
to reduce the complexityof the space to be searc hed W e
attempt to use these tec hniques to ac hiev e high test qualit y
and proto col co v erage
F ollo wing w e presen t the GFSM mo del for the case study
proto col PIMDM and use it as an illustrativ e example
Join and lea v e latencies ma y b e considered in other con texts as p erfor
mance issues Ho w ev er in our study w e treat them as errors
These are the correctness conditions for stable states ie not during
transien ts and are dened in terms of proto col states as opp osed to end
pointbeha vior
The mapping from functional correctness requiremen ts for m ulticast rout
ing to the denition in terms of the proto col mo del is curren tly done bythe
designer The automation of this pro cess is part of future researc h
Some esoteric scenarios of route apping ma y lead to m ulticast lo ops in
spite of RPF c hec ks Curren tly our study do es not address this issue as it
do es not p ertain to a lo calized b eha vior
to analyze the complexit y of the state space and the searc h
problem as w ell as illustrate the algorithmic details and prin
ciples in v olv ed in FITG and F OTG
A The Pr oto c ol Mo del
W e represen t the proto col as a nite state mac hine FSM
and the o v erall LAN system b y a global FSM GFSM
I FSM mo del Ev ery instance of the proto col running on
a single router is mo deled b y a deterministic FSM consist
ing of i a set of states ii a set of stim uli causing state
transitions and iii a state transition function or table de
scribing the state transition rules F or a system i this is
represen ted bythe mac hine M i S i i where S is a
nite set of state sym b ols i is the set of stim uli and i is
the state transition function S i S II Glob al FSM mo del The global state is dened as the
comp osition of individual router states The output mes
sages from one router ma y b ecome input messages to other
routers Suchin teraction is captured b y the GFSM mo del in
the global transition table The b eha vior of a system with n
routers ma y b e describ ed b y M G S G G G where S G S S S n is the global state space G n
S
i i is the
set of stim uli and G is the global state transition function
S G G S G The fault mo del is in tegrated in to the GFSM mo del F or
message loss the transition caused b y the message is either
n ullied or mo died dep ending on the selectiv e loss pattern
Crashes ma y b e treated as stim uli causing the routers aected
b y the crash to transit in to a cr ashed state
Net w ork de
la ys are mo deled when needed through the dela y matrix
presen ted in Section VI I
B PIMDM Mo del
F ollo wing is the mo del of a simplied v ersion of PIMDM
B FSM mo del M i S i i i F or a giv en group and a giv en source ie for a sp ecic
sourcegroup pair w e dene the states wrt a sp ecic LAN
to whic h the router R i is attac hed F or example a state
ma y indicate that a router is a forw arder for or a receiv er
exp ecting pac k ets from the LAN
Ba System States S P ossible states in whic h a router
ma y exist are
State Sym bol Meaning
F
i
Router i is a forw arder for the LAN
F
i Timer
i forw arder with Timer
T imer
running
NF
i
Upstream router i a nonforw arder
NH
i
Router i has the LAN as its nexthop
NH
i Timer
same as NH
i
with Timer
T imer
running
NC
i
Router i has a negativ ecac he en try
EU
i
Upstream router i is empt y
ED
i
Do wnstream router i is empt y
M
i
Do wnstream router with attac hed mem ber
NM
i
Do wnstream router with no mem b ers
The p ossible states for upstr e am and downstr e am routers
are as follo ws
The cr ashed state ma yb e one of the states already dened for the pro
to col lik ethe empty state or ma y b e a new state that w as not dened
previously for the proto col
S i fF i F i T imer N F i EU i g if the router is upstream fNH i N H i T imer N C i M i N M i ED i g if the router is do wnstream Bb Stim uli The stim uli considered here include
transmitting and receiving proto col messages timer ev en ts
and external host ev en ts Only stim uli leading to c hange
of state are considered F or example transmitting messages
per se vs receiving messages do es not cause an yc hange of
state except for the Gr af t in whic h case the Rtx timer is
set F ollo wing are the stim uli considered in our study
T ransmitting messages Graft transmission Gr af t Tx
Receiving messages Graft reception Gr af t Rcv Join
reception Join Prune reception Prune Graft Ac kno wl
edgemen t reception GAck Assert reception Asser t and
forw arded pac k ets reception FPkt
Timer ev en ts these ev en ts o ccur due to timer expiration
Exp and include the Graft retransmission timer Rtx the ev en t of its expiration RtxE xp the forw arderdeletion
timer Del and the ev en t of its expiration DelE xp W e
refer to the ev en t of timer expiration as T imer I mpl ication External host ev en ts Ext include host sending pac k
ets SP kt host joining a group HJ oin or HJ and host
lea ving a group Leav e or L fJ oin P rune Graf t Tx Graf t Rcv GAck Asser t F P ktRtxDel SP kt HJ L g B Global FSM mo del
Subscripts are added to distinguish dieren t routers
These subscripts are used to describ e router seman tics and
ho w routers in teract on a LAN An example global state for
a top ology of routers connected to a LAN with router as a forw arder router exp ecting pac k ets from the LAN
and routers and ha v e negativ e cac hes is giv en b y
fF N H N C N C g F or the global stim uli G subscripts
are added to stim uli to denote their originators and recipi
en ts if an y The global transition rules G are extended to
encompass the router and stim uli subscripts
C Dening stable states
W e are concerned with stable state ie nontransien t b e ha vior dened in this section T o obtain erroneous stable
states w e need to dene the transition mec hanisms b et w een
suc h states Wein tro duce the concept of transition classi
cation and completion to distinguish b et w een transientand
stable states
C Classication of T ransitions
W e iden tify t w o t yp es of transitions external ly trigger e d
ET and internal ly trigger edIT transitions The former is
stim ulated byev en ts external to the system eg H J oin or
Leav e whereas the latter is stim ulated byev en ts in ternal to
the system eg FPkt or Gr af t Seman tics of the global stim uli and global transitions will b e describ ed
as needed see Section V
W e note that some transitions ma y b e triggered due to ei
ther in ternal and external ev en ts dep ending on the scenario
F or example a Prune ma y b e triggered due to forw arding
pac k ets byan upstream router FPkt whic his an in ternal
ev en t or a Leav e whic h is an external ev en t
A global state is c hec k ed for correctness at the end of an
externally triggered transition after completing its dep enden t
in ternally triggered transitions
F ollo wing is a table of host ev en ts their dep enden t ET and
IT ev en ts
Host Ev en ts SP kt H J oin Leav e
ET ev en ts FPkt Gr af t P r une
IT ev en ts Asser t P r une GAck J oin
J oin
C T ransition Completion
Toc hec k for the global system correctness all stim ulated
in ternal transitions should b e completed to bring the system
in to a stable state In termediate transien t states should
not b e c hec k ed for correctness since they ma y temp orarily
seem to violate the correctness conditions set forth for sta
ble states and hence maygiv e false error indication The
pro cess of iden tifying complete transitions dep ends on the
nature of the proto col But in general w e ma y iden tify a
complete transition sequence as the sequence of all transi
tions triggered due to a single external stim ulus eg H J oin
or Leav e Therefore w e should b e able to iden tify a tran
sition based up on its stim uli either external or in ternal
A t the end of eac h complete transition sequence the system
exists in either a correct or erroneous stable state Ev en t
triggered timers eg Del Rtx re at the end of a complete
transition
D Pr oblem Complexity
The problem of nding test scenarios leading to proto
col error can b e view ed as a searc h problem of the proto col
state space Con v en tional reac habilit y analysis attempts
to in v estigate this space exhaustiv ely and incurs the state
space explosion problem T o circum v en t this problem w e
use searc h reduction tec hniques using domainsp ecic infor
mation of m ulticast routing
In this section w egiv e the complexit y of exhaustiv e searc h
then discuss the reduction tec hniques w e emplo y based on
notion of equiv alence and giv e the complexit y of the state
space
D Complexit y of exhaustivesearc h
Exhaustiv e searc h attempts to generate all states reac hable
from initial system states Forasystemof n routers where
eachrouterma y exist in an y state s i Sand jS j s states
the n um ber of reac hable states in the system is b ounded b y
s n
With l p ossible transitions w e need l s n
state visits
to in v estigate all transitions F aults suc h as message loss
and crashes increase the branc hing factor l and ma yin tro
duce new states increasing S F or our case study jS j while selectiv e loss and crashes
increase branc hing almost
b y factor of D State reduction through equiv alence
Exhaustiv e searc h has exp onen tial complexit y T o reduce
this complexityw e use the notion of equiv alence In tuitiv ely in m ulticast routing the order in whic h the states are consid
ered is irrelev an t eg if router R or R is a forw arder is
insignican t so long as there is only one forw arder Hence
w e can treat the global state as an unordered set of state
sym bols This concept is called coun ting equiv alence
By
denition the notion of equiv alence implies that byin v esti
gating the equiv alen t subspace w e can test for proto col cor
rectness That is if the equiv alen t subspace is v eried to b e
correct then the proto col is correct and if there is an error in
the proto col then it m ust exist in the equiv alen t subspace
Da Sym b olic represen tation Weuse a sym b olic rep
resen tation as acon v enientformof represen ting the global
state to illustrate the notion of equiv alence and to help in
dening the error and correct states in a succinct manner
In the sym b olic represen tation r routers in state q are rep
resen ted b y q
r
The global state for a system of n routers
is represen ted b y G q
r
q
r
q
r m
m
where m jS j r i n F or sym b olic represen tation of top ologies where n
is unkno wn r i is or more and is or more
T o satisfy the correctness conditions for PIMDM the
correct stable global states are those con taining no for
w arders and no routers exp ecting pac k ets or those con
taining one forw arder and one or more routers exp ecting
pac k ets from the link sym b olically this ma y be giv en b y
G F
N H
N C
and G F
N H

N C

Crashes force an y state to the empt y state
Tw o system states q
q
q n and p
p
p n are strictly
equiv alen t i q
i
p
i
where q
i
p
i
S i n Ho w ev er all
routers use the same deterministic FSM mo del hence all n p erm utations of
q
q
q n are equiv alen t A global state for a system with n routers
ma y b e represen ted as
Q
jS j
i s
k
i
i
where k
i
is the n um b er of routers in state
s
i
S and jS j
i k
i
n F ormally Counting Equivalenc e states that two
system states
Q
jS j
i s
k
i
i
and
Q
jS j
i s
l
i
i
aree quivalent if k
i
l
i
i The notion of coun ting equiv alence also applies to transitions and faults
Those transitions or faults leading to equiv alen t states are considered equiv
alen t

F or con v enience w ema y represen t these t w o states as G
NC

Weuse X to denote any state s i S F or example fX F g
denotes or more states s i S f F g This sym bolic
represen tation is used to estimate the size of the reduced
state space
Db Complexit y of the state space with equiv a
lence reduction Considering coun ting equiv alence nd
ing the n um ber of equiv alen t states b ecomes a problem of
com binatorics The n um ber of equiv alen t states becomes
C n s n
n s n s where n is the n um b er of routers
s is the n um b er of state sym bols and C x y
x y x y
is
the n um ber of y com bination of xset D Represen tation of error and correct states
Dep ending on the correctness denition w ema y get dier
en t coun ts for the n um b er of correct or error states Toget an
idea ab out the size of the correct or error state space for our
case studyw etaket w o denitions of correctness and com
pute the n um b er of correct states F or the correct states of
PIMDM w e either ha v e no forw arders with no routers
exp ecting pac k ets from the LAN or exactly one forw arder
with routers exp ecting pac k ets from the LAN

The correct space and the erroneous space m ust be dis
join t and they m ust b e complete ie add up to the com
plete space otherwise the sp ecication is incorrect See Ap
p endix AA for details
Wepresentt w o correctness denitions that are used in our
case
The rst denition considers the forw arder states as F and
the routers exp ecting pac k ets from the LAN as NH Hence
the sym bolic represen tation of the correct states b ecomes
fX NH F g
or NH F fX F g
and the n um b er of correct states is C n s n C n s n The second denition considers the forw arder states as
fF i F
i Del
g or simply F X and the states exp ecting pac k ets
from the LAN as fNH i N H i Rtx g or simply NH X Hence
the sym bolic represen tation of the correct states b ecomes
fX NH X F X g
or NH X F X fX F X g

and the n um b er of correct states is
C n s n C n s n C n s n Refer to App endix AB for more details on deriving the
n um b er of correct states
and G
F N H

NC
These conditions weha v e found to b e reasonably sucien t to meet the
functional correctness requiremen ts Ho w ev er they ma y not b e necessary hence the searchma y generate false errors Pro ving necessit y is part of future
w ork
In general w e nd that the size of the error state space ac
cording to b oth denitions constitutes the ma jor p ortion of
the whole state space This means that searc h tec hniques
explicitly exploring the error states are lik ely to be more
complex than others W e tak e this in consideration when
designing our metho ds
IV F a ul tindependent Test Genera tion
F aultindep enden t test generation FITG uses the forw ard
searchtec hnique to in v estigate parts of the state space As in
reac habilit y analysis forw ard searc h starts from initial states
and applies the stim uli rep eatedly to pro duce the reac hable
state space or part thereof Con v en tionally an exhaus
tiv e searchis conducted to explore the state space In the
exhaustiv e approac h all reac hable states are expanded un til
the reac hable state space is exhausted W e use sev eral man
ifestations of the notion of coun ting equiv alence in tro duced
earlier to reduce the complexit y of the exhaustiv e algorithm
and expand only equiv alen t subspaces T o examine robust
ness of the proto col w e incorp orate selectiv e loss scenarios
in to the searc h
A R e duction Using Equivalenc es
The searc h pro cedure starts from the initial states

and
k eeps a list of states visited to prev en t lo oping Eac h state
is expanded b y applying the stim uli and adv ancing the state
mac hine forw ard b y implemen ting the transition rules and
returning a new stable state eachtime

W e use the coun t
ing equiv alence notion to reduce the complexit y of the searc h
in three stages of the searc h
The rst reduction w e use is to in v estigate only the equiv
alen t initial states T o ac hiev e this w e simply treat the
set of states constituting the global state as unordered set
instead of ordered set F or example the output of suc h
pro cedure for IS fNM EU g and n w ould be fNM N M g fNM EU g fEU E U g One pro cedure that pro duces suc h equiv alen t initial state
space giv en in App endix BB The complexit y of the this
algorithm is giv en b y C n is n as w as sho wn in Sec
tion I I ID and v eried through sim ulation
The second reduction w e use is during comparison of vis
ited states Instead of comparing the actual states w e com
pare and store equiv alen t states Hence for example the
states fNF N H g and fNH N F g are equiv alen t

F or our case study the routers start as either a nonmem ber NMor
empt y upstream routers EU that is the initial states IS fNM E U g
F or details of the ab o v e pro cedures see App endix BA
A third reduction is made based on the observ ation that
applying iden tical stim uli to dieren t routers in iden tical
states leads to equiv alen t global states Hence w e can elimi
nate some redundan t transitions F or example for the global
state fNH N H F g a Leav e applied to R or R w ould
pro duce the equiv alen t state fNH
N C
F
g T o ac hiev e
this reduction w e add ag c hec k b efore adv ancing the state
mac hine forw ard W e call the algorithm after the third re
duction the reduced algorithm
In all the ab o v e algorithms a forw ard step adv ances the
GFSM to the next stable state This is done b y applying all
the in ternally dep enden t stim uli elicited due to the applied
external stim ulus in addition to an y timer implications if
an y exists Only stable states are c hec k ed for correctness
B Applying the Metho d
In this section w e discuss ho w the faultindep enden ttest
generation can b e applied to the mo del of PIMDM W e ap
ply forw ard searc h tec hniques to study correctness of PIM
DM W e rst study the complexit y of the algorithms without
faults Then w e apply selectiv e message loss to study the pro
to col b eha vior and analyze the proto col errors
B Metho d input
The proto col mo del is pro vided b y the designer or proto col
sp ecication in terms of a transition table or transition rules
of the GFSM and a set of initial state sym b ols The design
requiremen ts in terms of correctness in this case is assumed
to b e also giv en b y the proto col sp ecication This includes
denition of correct states or erroneous states in addition
to the fault mo del if studying robustness F urthermore the
detection of equiv alence classes needs to b e pro vided bythe
designer
Curren tly w e do not automate the detection
of equiv alen t classes Also the n um ber of routers in the
top ology or top ologies to b e in v estigated ie on the LAN
has to b e sp ecied
B Complexityof forw ard searc h for PIMDM
The pro cedures presen ted ab o vew ere sim ulated for PIM
DM to study its correctness This set of results sho ws b eha v
ior of the algorithms without including faults ie when used
for v erication W e iden tied the initial state sym b ols to b e
fNM EU g NM for do wnstream routers and EU for up
stream routers The n um b er of reac hable states visited the
n um ber of transitions and the n um ber of erroneous states
F or our case study the symmetry inheren tinm ulticast o v er LANs w as
used to establish the coun ting equiv alence for states transitions and faults
Expanded States Forwards
Rtrs Exhaustive Reduced Exhaustive Reduced
1 14 9 80 43
2 52 18 537 124
3 178 30 2840 263
4 644 48 14385 503
5 2176 73 63372 881
6 7480 106 271019 1430
7 24362 148 1060120 2187
8 80830 200 4122729 3189
9 259270 263 15187940 4477
10 843440 338 55951533 6092
11 2684665 426 199038216 8079
12 8621630 528 708071468 10483
13 27300731 645 2.461E+09 13353
14 86885238 778 8.546E+09 16738
Fig Sim ulation statistics for forw ard algorithms Exp ande d States
is the n um b er of visited states and F orwards is the n um ber of
forw ard adv ances of the state mac hine
Transitions Errors
Rtrs Exhaustive Reduced Exhaustive Reduced
1 19 11 1 1
2 90 31 7 3
3 343 65 33 6
4 1293 119 191 13
5 4328 197 783 25
6 14962 307 3235 43
7 47915 449 11497 68
8 158913 633 41977 101
9 503860 857 142197 143
10 1638871 1133 491195 195
11 5185208 1457 1625880 258
12 16666549 1843 5441177 333
13 52642280 2285 17751178 421
14 167757882 2799 58220193 523
Fig Sim ulation statistics for forw ard algorithms T r ansitions is
the n um b er of transien t states visited and E r rors is the n um ber
of stable state errors detected
found w ere recorded A summary of the results is giv en in
Figures and The n um b er of expanded states denotes the n um b er of vis
ited stable states The n um b er of forw ards is the n um ber of
times the state mac hine w as adv anced forw ard denoting the
n um b er of transitions b et w een stable states The n um ber of
transitions is the n um b er of visited transien t states and the
n um b er of error states is the n um b er of stable or expanded
states violating the correctness conditions The error condi
tion is giv en as in the second error condition in Section I I I
D Note that eac h of the other error states is equiv alentto
at least one error state detected bythe reduced algorithm
Hence ha ving less n um b er of disco v ered error states byan
algorithm in this case do es not mean losing an y information
or causes of error whic h follo ws from the denition of equiv
alence Reducing the error states means reducing the time
needed to analyze the errors
W e notice that there signican t reduction in the algorithm
complexit y with the use of equiv alence relations In particu
lar the n um b er of transitions is reduced from O
n
for the
exhaustiv e algorithm to O n
for the reduced algorithm
Similar results w ere obtained for the n um ber of forw ards
expanded states and n um ber of error states The reduc
tion gained b y using the coun ting equiv alence is exp onen tial
More detailed presen tation of the algorithmic details and re
sults are giv en in App endix BC
F or robustness analysis vs v erication faults are in
cluded in the GFSM mo del In tuitiv ely an increase in the
o v erall complexit y of the algorithms will be observ ed Al
though weha v e only applied faults to study the b eha vior of
the proto col and not the complexit y of the searc h w ean tici
pate similar asymptotic gains in the reduction using coun ting
equiv alence
B Summary of b eha vioral errors for PIMDM
W e used the ab o v e algorithm to searc h the proto col mo del
for PIMDM Correctness w as c hec k ed automatically bythe
metho d b y c hec king the stable states ie after applying
complete transitions By analyzing the sequence of ev en ts
leading to error wew ere able to reason ab out the proto col b e
ha vior Sev eral PIMDM errors w ere detected b y the metho d some p ertaining to correctness in the absence of message loss
while others w ere only detected in the presence of message
loss Weha v e studied cases of up to router LANs Some
times errors w ere found to o ccur in dieren t top ologies for
similar reasons as will b e sho wn Here w e only discuss results
for the t w o router and router LAN cases for illustration
Only one error w as detected in the t w orouter case With
the initial state fEU E U g ie b oth routers are upstream
routers the system en ters the error state fF N F g where
there is a forw arder for the LAN but there are no routers
exp ecting pac k ets or attac hed mem b ers In this case the
Asser t pro cess c hose one forw arder for the LAN but there
w ere no do wnstream routers to Prune o the extra trac
and so the proto col causes w asted bandwidth
Sev eral errors w ere detected for the router LAN case
Starting from fEU E U EU g the system en ters the error
state fF N F N F g for a similar reason to that giv en ab o v e
Starting from fNM EU EU g the system en ters the er
ror state fNC N F F g By analyzing the trace of ev en ts
leading to the error w e notice that the do wnstream router
NC pruned o one of the upstream routers NF b efore the
Asser t pro cess tak es place to c ho ose a winner for the LAN
Hence the proto col causes w asted bandwidth
Starting from fNM EU EU g the system en ters state
fNH F F g This is due to the transition table rules when
a forw arder sends a pac k et all upstream routers in the EU
state transit in to F state This is not an actual error ho w
ev er since the system will reco v er with the next forw arded
pac k et using Asser t
The detection of this falseerror could
ha vebeen a v oided b y issuing SP kt stim ulus b efore the error
c hec k to see if the system will reco v er with the next pac k et
sen t
With message loss errors w ere detected for Join and
P r une loss When the system is in fNH N H F g state and
one of the do wnstream mem b ers lea v es ie issues L ev en t a
P r une is sen t on the LAN If this Prune is selectiv ely lost b y
the other do wnstream router a J oin will not b e sen t and the
system en ters state fNC N H NF g Similarlyif the Join is
lost the proto col ends up in an error state
C Chal lenges and Limitations
In order to generalize the faultindep enden t test generation
metho d w e need to address sev eral op en researc h issues and
c hallenges
The top ology is an input to the metho d in terms of n um ber
of routers T o add top ology syn thesis to FITG wema y use
the sym b olic represen tation presen ted in Section I I ID where
the use of rep etition constructs

ma y b e used to represen t
the LAN top ology in general A similar principle w as used
in for cac he coherence proto col v erication where the
state space is split using rep etition constructs based on the
correctness denition In Section V w e presen t a new metho d
that syn thesizes the top ology automatically as part of the
searc h pro cess
Equiv alence classes are giv en as input to the metho d In
this study w e ha v e used symmetries inheren t in m ulticast
routing on LANs to utilize equiv alence This symmetry ma y
not exist in other proto cols or top ologies hence the forw ard
searchma y b ecome increasingly complex Automating iden
tication of equiv alence classes is part of future w ork
Other kinds of equiv alence ma y be in v estigated to reduce
complexit y in these cases
Also other tec hniques for
complexit y reduction ma y be in v estigated suc h as statis
tical sampling based on randomization or hashing used in
SPIN Ho w ev er sampling tec hniques do not ac hievefull
co v erage of the state space

This is one case where the correctness conditions for the mo del are su
cien t but not necessary to meet the functional requiremen ts for correctness
th us leading to a false error Suciency and necessit y pro ofs are sub ject of
future w ork

Rep etition constructs include for example the to represen t zero or
more states or the to represen t one or more states t w o or more
so on

An example of another kind of equiv alence is fault dominanc e where a
system is pro v en to necessarily reac h one error b efore reac hing another th us
the former error dominates the latter error
The top ology used in this study is limited to a singlehop
LAN Although w e found it quite useful to study m ulticast
routing o v er LANs the metho d needs to be extended to
m ultihop LAN to b e more general Section VI I in tro duces
the notion of vir tual LAN and future w ork addresses m ulti
LAN top ologies
In sum the faultindep enden t test generation maybeused
for proto col v erication giv en the symmetry inherentin the
system studied ie proto col and top ology F or robustness
studies where the fault mo del is included in the searc h the
complexit y of the searc h gro ws In this approac h w e did
not address address p erformance issues or top ology syn thesis
These issues are addressed in the coming sections Ho w ev er
w e shall reuse the notion of forw ard searc h and the use of
coun ting equiv alence in the metho ds discussed next
V F a ul toriented Test Genera tion
In this section w ein v estigate the faultorien ted test gen
eration F OTG where the tests are generated for sp ecic
faults In this metho d the test generation algorithm starts
from the faults and searc hes for a p ossible error establish
ing the necessary top ology and ev en ts to pro duce the error
Once the error is established a bac kw ard searchtec hnique
pro duces a test sequence leading to the erroneous state if
suc h a state is reac hable Weuse the FSM formalism pre
sen ted in Section I I I to represen t the proto col W e also reuse
some ideas from the FITG algorithm previously presen ted
suc h as forw ard searc h and the notion of equiv alence for
searc h reduction
A F OTG Metho d Overview
F aultorien ted test generation F OTG targets sp ecic
faults or conditions and so is better suited to study ro
bustness in the presence of faults in general F OTG has
three main stages a top ology syn thesis b forw ard im
plication and error detection and c bac kw ard implication
The top ology syn thesis establishes the necessary comp onen ts
eg routers and hosts of the system to trigger the giv en
condition eg trigger a proto col message This leads to
the formation of a global state in the middle of the state
space

F orw ard searc h is then p erformed from that global
state in its vicinit y ie within a complete transition after
applying the fault This pro cess is called forwar d implic a
tion and uses searc h tec hniques similar to those explained
earlier in Section IV If an error o ccurs bac kw ard searc h
The global state from whichF OTG starts is syn thesized for a giv en fault
suc h as a message to b e lost
is p erformed thereafter to establish a v alid sequence lead
ing from an initial state to the syn thesized global state T o
ac hiev e this the transition rules are rev ersed and a searchis
p erformed un til an initial state is reac hed or the syn thesized
state is declared unreac hable This pro cess is called b ackwar d
implic ation Muc h of the algorithmic details are based on condition ef f ect reasoning of the transition rules This reasoning is
emphasized in the seman tics of the transition table used in
the top ology syn thesis and the bac kw ard searc h Section V A describ es these seman tics In Section VB w e describ e the
algorithmic details of F OTG and in Section VC w e describ e
howF OTG w as applies to PIMDM in our case study and
presen t the results and metho d ev aluation Section VD w e
discuss the limitations of the metho d and our ndings
A The T ransition T able
The global state transition ma y be represen ted in sev
eral w a ys Here w ec ho ose a transition table represen tation
that emphasizes the eect of the stim uli on the system and
hence facilitates top ology syn thesis The transition table de
scrib es for eac h stim ulus the conditions of its o ccurrence
A condition is giv en as stim ulus and state or transition de
noted b y stimulusstatetr ans where the transition is giv en
as star tS tate endS tate W e further extend message and router seman tics to cap
ture m ulticast seman tics F ollo wing w e presen t a detailed
description of the seman tics of the transition table then giv e
the resulting transition table for our case study to b e used
later in this section
Aa Seman tics of the transition table In this subsec
tion w e describ e the message and router seman tics pre
conditions and p ostconditions
Stim uli and router seman tics Stim uli are classied based
on the routers aected b ythem Stim uli t yp es include
or ig stim uli or ev en ts o ccurring within the router orig
inating the stim ulus but do not aect other routers and in
clude HJ L SP kt Gr af t Tx Del and Rtx dst messages that are pro cessed b y the destination
router only and include J oin GAck and Gr af t Rcv mcast m ulticast messages that are pro cessed b y all
other routers and include Asser t and FPkt mcastD ow nstr eam m ulticast messages that are pro
cessed b y all other do wnstream routers but only one up
stream router and includes the P r une message
These t yp es are used b y the searc h algorithm for pro cessing
the stim uli and messages According to these dieren tt yp es
of stim uli pro cessing a router maytak e as subscript or ig
dst or other The or ig sym b ol designates the originating
router of the stim ulus or message whereas dst designates
the destination of the message other indicates routers other
than the originator Routers are also classied as upstr eam
or downstream as presen ted in Section I I I PreConditions The preconditions in general are of
the form stimul usstatetr ansition where the transition is
giv en as star tS tate endS tate If there are sev eral pre
conditions then w e can use a logical OR to represen t the
rule Atleast one precondition is necessary to trigger the
stim ulus Example of a stimul usstate condition is the con
dition for J oin message namely P r une
other
N H or ig that is
a Join is triggered b y the reception of a Prune from another
router with the originator of the Join in NH An example
of a stimul ustr ansition condition is the condition for Graft
transmission HJ NC NH ie a host joining and the
transition of the router from the negativecac he state to the
next hop state
P ostConditions A p ostcondition is an ev en t andor
transition that is triggered b y the stim ulus

P ost
conditions ma y be in the form of tr ansition conditiontr ansition conditionstimul us and stimul ustr ansition tr ansition has an implicit condition with whic h it is
asso ciated ie a b means if a GS tate then a b F or example J oin p ostcondition NF
dst
F
dst
means if
NF
dst
GS tate then transition NF F will o ccur
C onditiontr ansition is same as except the condi
tion is explicit

C onditionstimul us if the condition is satised then the
stim ulus is triggered F or example P r une p ostcondition
NH
other
J oin
other
means that for all NH x GS tate
where x is not equal to or ig then ha v e router x trigger
a Join S timul ustr ansition has the transition condition im
plied as in ab o v e F or example Gr af t Rcv p ostcondition
GAck NF
dst
F
dst
means if NF
dst
GS tatethenthe
transition o ccurs and GAck is triggered
If more than one p ostcondition exists then the logical re
lation b et w een them is either an X OR if the router is the
same or an AND if the routers are dieren t F or example
Join p ostconditions are F
dst Del
F
dst
N F
dst
F
dst
Net w ork faults suc h as message loss ma y cause the stim ulus not to
tak e eect F or example losing a Join message will cause the ev entof Join
reception not to tak e eect
This do es not app ear in our case study whic h means F
dst Del
F
dst
X OR NF
dst
F
dst

On the other hand P r une p ostconditions are F
dst
F
dst Del
N H
other
J oin
other
whic h implies that the transi
tion will o ccur if F
dst
GS tate AND a J oin will b e triggered
if NH GS tate F ollo wing is the transition table used in our case study Stim ulus Preconditions P ostconditions
Join P r une
other
N H
or ig
F
dst Del
F
dst
NF
dst
F
dst
Prune LN C F P k tN C F
dst
F
dst Del
NH
other
J oin
other
Gr af t
Tx
HJ NC NH Gr af t
Rcv
NH NH
Rtx
RtxE xp NH
Rtx
NH Gr af t
Rcv
Gr af t
Tx
NH NH
Rtx
GAck NF
dst
F
dst
GAck Gr af t
Rcv
F NH
dst Rtx
NH
dst
Asser t FPkt
other
F
or ig
F
other
NF
other
FP kt SpktF Prune NM NC ED NH M NH EU
other
F
other
F
other
Asser t
Rtx R txExp Gr af t
Tx
NH
or ig Rtx
NH
or ig
Del DelExp F
or ig Del
NF
or ig
SP kt Ext FP kt EU
or ig
F
or ig
HJ oin Ext NM M Gr af t
Tx
NC NH Leav e Ext M N M P r une NH NC Prune NH
Rtx
NC The ab o v e preconditions can be deriv ed automatically
from the p ostconditions In App endix C w e describ e the
PreConditions pro cedure that tak es as input one form of
the con v en tional p ostcondition transition table and pro duces
the precondition seman tics
Ab State Dep endency T able T o aid in test sequence
syn thesis through the bac kw ard implication pro cedure w e
construct what w e call a state dep endency table This table
can b e inferred automatically from the transition table W e
use this table to impro v e the p erformance of the algorithm
and for illustration
F or eac h state the dep endency table con tains the p ossible
preceding states and the stim ulus from whic h the state can b e
reac hed or implied T o obtain this information for a state s the algorithm the p ostconditions of the transition table for
en tries where the endS tate of a transition is s In addition
a state ma y b e iden tied as an initial state IS and hence
can b e readily established without an y preceding states The
dep endencyT able pro cedure in App endix C generates the
dep endency table from the transition table of conditions F or
s IS asym b ol denoting initial state is added to the arra y
en try F or our case study IS fNM EU g Based on
the ab o v e transition table follo wing is the resulting state
dep endency table

There is an implicit condition that can nev er b e satised in b oth state
men ts whic h is the existence of dst in only one state at a time

The p ossible bac kw ard implications are separated b y commas indicat
State P ossible Bac kw ard Implications
F
i
FP kt
other
EU
i
Join
F
i Del
J oin
NF
i
Gr af t
Rcv
NF
i
SP kt
EU
i
F
i Del
P r une
F
i
NF
i
Del
F
i Del
Asser t
F
i
NH
i
RtxGAck
NH
i Rtx
HJ
NC
i
FPkt
M
i
FP kt
ED
i
NH
i Rtx
Gr af t
Tx
NH
i
NC
i
FP kt
NM
i
L
NH
i Rtx
L
NH
i
EU
i
IS ED
i
IS M
i
HJ
NM
i
NM
i
L
M
i
IS In cases where the stim ulus aects more than one router
eg m ulticast P r une m ultiple states need to be sim ul
taneously implied in one bac kw ard step otherwise an IS ma y not b e reac hed T o do this the transitions in the p ost
conditions of the stim ulus are tra v ersed and an y states in
the global state that are endS tates are replaced b y their cor
resp onding star tS tates F or example fM i N M j F
k
g
FPkt
fNH i N C j F
k
g This is tak en care of b y the bac kw ard im
plication section describ ed later
B F OTG details
As previously men tioned our F OTG approac h consists of
three phases I syn thesis of the global state to insp ect I I
forw ard implication and I I I bac kw ard implication These
phases are explained in more detail in this section In Sec
tion VC w e presen t an illustrativ e example for the these
phases
B Syn thesizing the Global State
Starting from a condition eg proto col message or stim u
lus and using the information in the proto col mo del ie
the transition table a global state is syn thesized for in v es
tigation W e refer to this state as the globalstate insp ected
G I and it is obtained as follo ws
The global state is initially empt y and the insp ected stim
ulus is initially set to the stim ulus in v estigated
F or the insp ected stim ulus the states or the
star tS tates of the transition of the p ostcondition are ob
tained from the transition table If these states do not exist
in the global state and cannot b e inferred therefrom then
they are added to the global state
F or the insp ected stim ulus the states or the
endS tates of the transition of the precondition are ob
tained If these states do not exist in the global state and
ing OR relation
cannot be inferred therefrom then they are added to the
global state
Get the stim ulus of the precondition of the insp ected
stim ulus call it new S timul us If new S timul us is not
external Ext then set the insp ected stim ulus to the
new S timul us and go bac kto step The second step considers p ostconditions and adds system
comp onen ts that will b e aected b ythe stim ulus While the
third and forth steps syn thesize the comp onen ts necessary
to trigger the stim ulus The pro cedure giv en in App endix C
syn thesizes minim um top ologies necessary to trigger a giv en
stim ulus of the proto col
Note that there ma y be sev eral preconditions or post conditions for a stim ulus in whic h case sev eral c hoices can b e
made These representbranc hing p oin ts in the searc h space
A t the end of this stage the global state to b e in v estigated
is obtained
B F orw ard Implication
The states follo wing G I ie G I i where i are obtained
through forw ard implication W e simply apply the transi
tions starting from G Ias giv en b y the transition table in
addition to implied transitions suc h as timer implication
F urthermore faults are incorp orated in to the searc h F or
example in the case of a message loss the transition that
w ould ha v e resulted from the message is not applied If more
than one state is aected b y the message then the space is
expanded to include the v arious selectiv e loss scenarios for
the aected routers F or crashes the routers aected b y the
crash transit in to the crashed state as dened b y the ex
panded transition rules as will be sho wn in Section VC
F orw ard implication uses the forw ard searc h tec hniques de
scrib ed earlier in Section IV
According to the transition completion concept see Sec
tion IIIC the prop er analysis of beha vior should start
from externally triggered transitions F or example the anal
ysis should not consider a Join without considering the
P r une triggering it and its eects on the system Th us the
global system state m ust b e rolled bac k to the b eginning of
a complete transition ie the previous stable state b efore
applying the forw ard implication This will b e implied in the
forw ard implication algorithm to simplify the discussion
B Bac kw ard Implication
Bac kw ard implication attempts to obtain a sequence of
ev en ts leading to G I from an initial state I S if sucha
sequence exists ie if G I is reac hable from I S
The state dep endency table describ ed in Section VAb
is used in the bac kw ard searc h
Bac kw ard steps are tak en for the comp onen ts in the global
state G I eac h step pro ducing another global state GS tate F or eac h state in GS tate p ossible bac kw ard implication rules
are attempted to obtain v alid bac kw ard steps to w ard an ini
tial state This pro cess is rep eated for preceding states in a
depth rst fashion A set of visited states is main tained to
a v oid lo oping If all bac kw ard branc hes are exhausted and
no initial state w as reac hed the state is declared unreac hable
T o rewind the global state one step bac kw ard the re
v erse transition rules are applied Dep ending on the stim
ulus t yp e of the bac kw ard rule dieren t states in GS tate
are rolled bac k F or or ig and dst only the originator and
destination of the stim ulus is rolled bac k resp ectiv ely F or
mcast all aected states are rolled bac k except the origina
tor mcastD ow nstr eam is similar to mcast except that all
do wnstream routers or states are rolled bac k while only one
upstream router the destination is rolled bac k App endix C
sho ws pro cedures Bac kw ard and Rewind that implemen t
the ab o v e steps
Note ho w ev er that not all bac kw ard steps are v alid and
bac ktrac king is p erformed when a bac kw ard step is in v alid
Bac ktrac king ma y o ccur when the preceding states con tradict
the rules of the proto col These con tradictions ma y manifest
themselv es as
Src not found sr c is the originator of the stim ulus and the
global state has to include at least one comp onen t to originate
the stim ulus An example of this con tradiction o ccurs for the
Prune stim ulus for a global state fNH F N F g where the
an originating comp onen t of the P r une NC in this case
do es not b elong to the global state
F ailure of minim um top ology c hec k the necessary con
ditions to trigger the stim ulus m ust be presen t in the
global top ology Examples of failing the minim um top ol
ogy c hec k include for instance J oin stim ulus with global
state fNH N F g or Asser t stim ulus with global state
fF N H N C g F ailure of consistency c hec k to main tain consistency of
the transition rules in the rev erse direction wem ust c hec k
that ev ery bac kw ard step has an equiv alentforw ard step T o
ac hievethis w em ust c hec k that there is no transition x y
for the giv en stim ulus suc h that x GS tate Since if x
remains in the preceding global state the corresp onding for
w ard step w ould transform x in to y and the system w ould ex
ist in a state inconsisten t with the initial global state b efore
the bac kw ard step An example of this inconsistency ex
ists when the stim ulus is FPkt and GS tate fF N F E U g where EU F is a post condition for FPkt See Ap
p endix C for the consistency c hec k pro cedure
C Applying The Metho d
In this section w e discuss ho w the faultorien ted test gen
eration can be applied to the mo del of PIMDM Sp eci
cally w e discuss in details the application of F OTG to the
robustness analysis of PIMDM in the presence of single mes
sage loss and mac hine crashes Werstw alk through a sim
ple illustrativ e example Then w e presen t the results of the
case study in terms of correctness violations captured b y the
metho d
C Metho d input
The proto col mo del is pro vided b y the designer or proto
col sp ecication in terms of a transition table
and the
seman tics of the messages In addition a list of faults to b e
studied is giv en as input to the metho d F or example def
inition of the fault as single selectiv e proto col message loss
applied to the list of messages fJ oin P r une Asser t Gr af t g Also a set of initial state sym b ols in our case fNM EU g A
denition of the design requiremen t in this case denition of
correctness is also pro vided b y the sp ecication The rest of
the pro cess is automated
C Illustrativ e example
Figure sho ws the phases of F OTG for a simple example of
a Join loss F ollo wing are the steps tak en for that example
Syn thesizing the Global State
Join star tS tate of p ostcondition is NF
dst
G
I
fNF
k
g
Join state of precondition is NH
i
G
I
fNH
i
NF
k
g goto P r une
Prune star tS tate of p ostcondition is F
k
implied from NF
k
in G
I
P r une state of precondition is NC
j
G
I
fNH
i
NF
k
NC
j
ggoto L
Ext
star tS tate of p ostcondition is NH can b e implied from NC in G
I
F orw ard implication
without loss G
I
fNH
i
NF
k
NC
j
g
Join
G
I
fNH
i
F
k
NC
j
g
loss wrt R
j
fNH
i
NF
k
NC
j
g G
I fNH
i
NF
k
NC
j
g error
Bac kw ard implication
G
I
fNH
i
NF
k
NC
j
g
Prune
G
I fNH
i
F
k
NC
j
g
FP kt
G
I fM
i
F
k
NM
j
g
SP kt
G
I fM
i
EU
k
NM
j
g
HJ
i
G
I fNM
i
EU
k
NM
j
g IS Losing the J oin b y the forw arding router R
k
leads to an
error state where router R i is exp ecting pac k ets from the
LAN but the LAN has no forw arder

The traditional inputoutput transition table is sucien t for our
metho d The prep ostcondition transition table can b e deriv ed automati
cally therefrom
NH
j
NF
k
NC
i
Join
i
Prune
j
.NH
i
Prune
j
Leave
j
.NC
j
Leave
j
Host Event
Stimulus Pre-conditions Post-conditions
NF
k
F
k
(F
k
NF
k
). NH
i
.Join
i
(NH
j
NC
j
).Prune
j
Synthesized
Topology
G
I+1
={NC
j
,NH
i
,F
k
}
No loss of Join
G
I+1
={NC
j
,NH
i
,NF
k
}
Loss of Join
Error state
Prune
j G
I-1
={NC
j
,NH
i
,F
k
}
G
I
={NC
j
,NH
i
,NF
k
}
G
I+
Forward implication G
I-
Backward implication
G
I-2
={NM
j
,M
i
,F
k
}
G
I-3
={NM
j
,M
i
,EU
k
}
G
I-4
={NM
j
,NM
i
,EU
k
}
FPkt
SPkt
HJ
i
Fig Join top ology syn thesis forw ardbac kw ard implication
C Summary of Results
In this section w e briey discuss the results of applying
our metho d to PIMDM The analysis is conducted for single
message loss and momen tary loss of state F or a detailed
analysis of the results see App endix CG
Ca Single message loss Weha v e studied single mes
sage loss scenarios for the J oin P r une Asser t and Gr af t
messages F or this subsection w e mostly consider non
in terlea v ed external ev en ts where the system is stim ulated
only once b et w een stable states The Gr af t message is par
ticularly in teresting since it is ac kno wledged and it raises
timing and sequencing issues that w e address in a later sub
section where w e extend our metho d to consider in terlea ving
of external ev en ts
Our metho d as presen ted here ho w ev er ma y not b e gener
alized to transform anyt yp e of timing problem in to sequenc
ing problem This topic b ears more researc h in the future
Weha v e used the sequences of ev en ts generated automati
cally b y the algorithm to analyze proto col errors and suggest
xes for those errors
Join A scenario similar to that presen ted in Section V
C incurred an error In this case the robustness violation
w as not allo wing another c hance to the do wnstream router
to send a Join A suggested x w ould b e to send another
prune b y F
Del
b efore the timer expires
Prune In the top ology ab o v e an error o ccurs when R i
loses the P r une hence no Join is triggered The x sug
A
B
upstream
downstream
A B
Graft
Graft
GAck
A B
time
Graft
GAck
(I) no loss (II) loss of Graft
A B
t
1
t
2 t
3
t
4
t
5
t
6
Graft
Prune
Graft
GAck
(III) loss of Graft &
interleaved Prune
t
1 t
1
t
2
t
2
t
3
t
3
t
4
Fig Graft ev en t sequencing
gested ab o vetak es care of this case to o
Assert An error in the Asser t case o ccurs with no do wn
stream routers eg G I fF i F j g The design error is the
absence of amec hanism to prev en t pruning pac k ets in this
case One suggested x w ould b e to ha vethe Asser t winner
sc hedule a deletion timer ie becomes F
Del
and ha v e the
do wnstream receiv er if an y send J oin to the Asser t winner
Graft A Gr af t message is ac kno wledged b y GAckhence
the proto col did not incur error when the Gr af t message
w as lost with nonin terlea v ed external ev en ts The proto col
is robust to Gr af t loss with the use of Rtx timer Adv ersary
external conditions are in terlea v ed during the transien t states
and the Rtx timer is cleared suc h that the adv erse ev en t will
not b e o v erridden bythe Rtx mec hanism
T o clear the Rtx timer a transition should b e created from
NH Rtx to NH whic h is triggered bya GAck according to the
state dep endency table NH
GAck
NH Rtx This transition
is then inserted in the ev en t sequence and forw ard and bac k
w ard implications are used to obtain the o v erall sequence of
ev en ts illustrated in gure In the rst and second scenarios
I and II no error o ccurs In the third scenario III when
a Gr af t follo w ed bya P r une is in terlea v ed with the Gr af t
loss the Rtx timer is reset with the receipt of the GAck for
the rst Gr af t and the systems ends up in an error state A
suggested x is to add sequence n um b ers to Gr af ts at the
exp ense of added complexit y Cb Loss of State W e consider momen tary loss of
state in a router A Crashstim ulus transfers the crashed
router from an y state X in to EU or ED Hence w eadd
the follo wing line to the transition table
Stim ulus Precond P ostcond stim ulusstatetrans
Cr ash Ext fNMMNH NCNH
Rtx
g ED fF F
Del
NF g EU
The FSM resumes function immediately after the crash
ie further transitions are not aected W e analyze the
beha vior when the crash o ccurs in an y router state F or ev ery
state a top ology is syn thesized that is necessary to create
that state Welev erage the top ologies previously syn thesized
for the messages F or example state F
Del
maybe created
from state F b y receiving a P r une F
Del
P r une
F Hence w e
ma y use the top ologies constructed for P r une loss to analyze
a crash for F
Del
state
F orw ard implication is then applied and b eha vior after the
crash is c hec k ed for correct pac k et deliv ery Toac hiev e this
host stim uli ie SP kt HJ and L are applied then the
system state is c hec k ed for correctness
In lots of the cases studied the system reco v ered from the
crash ie the system state w as ev en tually correct The re
co v ery is mainly due to the nature of PIMDM where proto
col states are recreated with reception of data pac k ets This
result is not lik ely to extend to proto cols of other natures
eg PIM SparseMo de Ho w ev er in violation with robustness requiremen ts there
existed cases in whic h the system did not reco v er In gure the host joining in I I a did not ha v e the sucien t state to
send a Gr af t and hence gets join latency un til the negativ e
cac he state times out upstream and pac k ets are forw arded
on to the LAN as in I I b
NF NF NF F NF F
NH ED M
NH NM NC
(I)
NH
Crash
ED
(II)
HJ
SPkt
(III)
L
SPkt
Prune
(a) (b)
FPkt FPkt
Fig Crash leading to join latency
In gure I I a the do wnstream router incurs join la
tency due to the crash of the upstream router The state is
not corrected un til the p erio dic broadcast tak es place and
pac k ets are forw arded on to the LAN as in I I b EU F EU NF
NH NH NC NC
(II)
SPkt
(III)
L
Prune
(a) (b)
F EU
NH Rtx NH
(I)
F
Crash
EU
G Tx
G Rcv
GAck
Fig Crash leading to blac k holes
D Chal lenges and Limitations
Although weha v e b een able to apply F OTG to PIMDM
successfully a discussion of the op en issues and c hallenges is
called for In this section w e address some of these issues
The top ologies syn thesized b y the ab o veF OTG study are
only limited to a singlehop LAN with n routers
This
means that the ab o v e F OTG analysis is necessary but not
sucien ttov erify robustness of the endtoend b eha vior of
the proto col in a m ultihop top ology ev en if eac h LAN in the
top ology op erates correctly the in terLAN in teraction ma y
in tro duce erroneous b eha viors Applying F OTG to m ultihop
top ologies is part of future researc h
The analysis for our case studies did not consider net w ork
dela ys In order to study endtoend proto cols net w ork de
la ys m ust be considered in the mo del In Section VI I w e
in tro duce the notion of v ir tual LAN to include endtoend
delayseman tics
Minimal top ologies that are necessary and sucien tto trig ger the stim uli ma y not b e sucien t to capture all correct
ness violations F or example in some cases it ma y require
one mem ber to trigger a Join but t w o mem b ers to exp e
rience an error caused b y J oin loss Hence the top ology
syn thesis stage m ust b e complete in order to capture all p os
sible errors Toac hievethisw e prop ose to use the sym bolic
represen tation F or example to co v er all top ologies with one
or more mem b ers weuse M
In tegration of this notation
with the full metho d is part of future w ork
The eciency of the bac kw ard searchma y b e increased us
ing reduction tec hniques suc h as equiv alence of states and
transitions similar to the ones presen ted in Section IV In
addition the algorithm complexit yma y b e reduced b y utiliz
ing information ab out reac hable states to reduce the searc h
This information could be obtained simply b y storing pre
vious sequences and states visited Alternativ ely the de
signer maypro vide information based on proto colsp ecic
knowledge about reac hable states through a compact rep
resen tation thereof
The top ologies constructed byF OTG are inferred from the
mec hanisms sp ecied b y the transition table of the GFSM
The F OTG algorithm will not construct top ologies resulting
from nonsp ecied mec hanisms F or example if the Asser t
mec hanism that deals with duplicates w as left out due to
a design error the algorithm w ould not construct fF i F j g
top ology Hence F OTG is not guaran teed to detect dupli
cates in this case So F OTG as presen ted here ma y b e used

This limitation is similar to that suered b y FITG in Section IV
to ev aluate b eha vior of sp ecied mec hanisms in the presence
of net w ork failures but is not a general proto col v erication
to ol
The global states syn thesized during the top ology syn the
sis phase are not guaran teed to be reac hable from an ini
tial state Hence the algorithm maybe in v estigating non
reac hable states un til they are detected as unreac hable in the
last bac kw ard searc h phase Adding reac habilit y detection in
the early stages of F OTG is sub ject of future w ork Ho w ev er
statistics collected in our case study see App endix C sho w
that unreac hable states are not the determining factor in the
complexit y of the bac kw ard searc h Hence other reduction
tec hniques ma y be needed to increase the eciency of the
metho d
W e b eliev e that the strength of our faultorien ted metho d
as w as demonstrated lies in its abilit y to construct the nec
essary conditions for erroneous b eha vior b y starting directly
from the fault and a v oiding the exhaustivew alk of the state
space Also con v erting timing problems in to sequencing
problems as w as sho wn for Gr af t analysis reduces the com
plexit y required to study timers F OTG as presentedinthis
c hapter seems best t to study proto col robustness in the
presence of faults F aults presen ted in our studies include
single selectiv e loss of proto col messages and router crashes
VI Rela ted W ork
The related w ork falls mainly in the eld of proto col v er
ication and distributed algorithms In addition some con
cepts of our w ork w ere inspired b y VLSI c hip testing Most
of the literature on m ulticast proto col design addresses ar
c hitecture sp ecication and comparisons b et w een dieren t
proto cols Weare not a w are of an y other w ork to dev elop
systematic metho ds for test generation for m ulticast proto
cols
There is a large b o dy of literature dealing with v erication
of comm unication proto cols Proto col v erication t ypically
addresses w elldened prop erties suc h as safet y and liv eness
prop erties In general the t w o main approac hes for proto col
v erication are theorem pro ving and reac habilit y analysis or
mo del c hec king Theorem pro ving systems dene a
set of axioms and construct relations on these axioms De
sirable prop erties of the proto col are then pro v en mathemat
ically Theorem pro ving includes mo delb ase d and lo gicb ase d
formalisms including rst and higher order logic
Reac habilit y analysis algorithms attempt to gener
ate and insp ect all the proto col states that are reac hable from
giv en initial states Suc h algorithms suer from the state
space explosion problem esp ecially for complex proto cols
T o circum v en t this problem state reduction and con trolled
partial searchtec hniques could b e used These tec h
niques fo cus only on parts of the state space and ma y use
probabilistic random or guided searc hes W ork on distributed algorithms deals with sync hronous
net w orks async hronous shared memory and async hronous
net w ork ed systems Pro ofs can b e established using an
automatatheoretic framew ork Section VIB presen ts w ork
on distributed algorithms and outlines ho w it relates to our
w ork
Conformance testing is used to c heckthatagiv en imple
men tation of a proto col is equiv alen t to its sp ecication It
do es not target design errors or proto col p erformance but
implemen tation errors and uses searc htec hniques to attempt
to co v er the state space of the implemen tation W e discuss
conformance testing in Section VIC
Other related w ork includes a new approac h for v erica
tion of cac he coherence proto cols This recen t study
sho ws ho w reac habilit y analysis complexit y can b e reduced
b y using equiv alence relations and sym b olic represen tation of
states A global FSM nite state mac hine mo del w as used
to c haracterize the proto col b eha vior In our w ork on searc h
algorithms w e adopt some of the principles presen ted in the
ab o v e study namely the global FSM mo del and the notion
of coun ting equiv alence
There is an analogy b et w een our w ork and VLSI c hip test
ing Chip test generation metho ds attempt to generate test
v ectors to rev eal faults in the VLSI fabrication pro cess These
metho ds dene a fault mo del and a circuit mo del for the c hip
under test and usually use searc h algorithms to nd patterns
exp osing exp ected faults VLSI c hip testing sc hemes are dis
cussed in Section VID
A Pr oto c ol V eric ation
Proto col v erication is the problem of ensuring the logi
cal consistency of the proto col sp ecication indep enden tof
an y particular implemen tation Proto col v erication t ypi
cally addresses safety liveness and r esp onsiveness prop er
ties Safet y prop erties include freedom from deadlo c ks
assertion violations improp er terminations and unsp ecied
receptions Liv eness prop erties include detection of accep
tance cycles and absence of nonprogress cycles while re
sp onsiv eness prop erties include timeliness and fault toler
ance whichreco v ers the system to a legal state to resume
normal execution from an illegal state Most proto col v er
ication systems aim to detect violations of these proto col
prop erties
Although w e cannot do justice to the extensiv e body of
w ork in this area w e shall dw ell up on some of the main as
p ects and common approac hes to proto col v erication There
are t w o main approac hes to proto col v erication theorem
pro ving using formal metho ds and reac habilit y analysis
sometimes called mo del c hec king
In theorem pro ving system prop erties are expressed in
logic form ulas dening a set of axioms and rules In con trast
to reac habilit y analysis and mo del c hec king theorem pro v
ing can deal with innite state spaces Ho w ev er in teractiv e
theorem pro v ers require h uman in terv en tion and hence are
slo w and errorprone
Theorem pro ving includes mo delb ase d and lo gicb ase d for
malisms Mo delbased formalisms suc h as Z and Vienna
Dev elopmen t Metho d VDM are suitable for proto col
sp ecications in a succinct manner but lac k the to ol supp ort
for eectiv e pro of of prop erties The use of rst order logic
allo ws the use of theorem pro v ers suc h as the Bo y erMo ore
logic pro v er Nqthm but ma y result in sp ecications
that are dicult to read Higher order logic suchas Pro
tot yp e V erication System PVS pro vides expressiv e
po w er for clear descriptions and pro of capabilities for proto
col prop erties
In general theorem pro ving systems require the denition
of a set of axioms and the construction of relations based
on these axioms The n um b er of axioms and relations gro ws
with the complexit y of the proto col These systems require
strong mathematical bac kground and understanding The
fact that axiomatization and pro ofs dep end largely on h uman
in telligence ma y limit the use of theorem pro ving systems
Theorem pro ving has been used in v erication of dis
tributed algorithms and systems see Section VIB
Sev eral attempts to apply formal v erication to net w ork
proto cols ha v e b een made F or example assertional pro of
tec hniques w ere used to pro v e distance v ector routing path v ector routing and route diusion algorithms and using comm unicating nite state mac hines
An example p oin ttop oin t mobile application w as pro v ed
using assertional reasoning in using UNITY Ax
iomatic reasoning w as used in pro ving a simple transmission
proto col in Algebraic systems based on the calculus of
comm unicating systems CCS ha v e b een used to pro v e
CSMACD F ormal v erication has b een applied to TCP
and TTCP in In all formal v erication metho ds ma y be imp ortan t to
proto col design Ho w ev er they ha v e not b een applied to
widearea m ulticast or complete routing proto cols Webe liev e that theorem pro ving systems will b e ev en more com
plex and p erhaps in tractable in the con text of m ulticast
proto cols
Con v en tional reac habilit y analysis systems are based on
exhaustivereac habilit y analysis T o establish the observ ance
of state in v arian ts it is sucien tto v erify their correctness
with a test for eac h state that is reac hable from a giv en initial
system state The main problem that m ust be addressed
in the design of suc h a system is the state space explosion
problem
V erication of state prop erties includes assertion viola
tions and improp er terminations V erication of sequences of
states includes nonprogress conditions and temp oral claims
Areac habilit y analysis algorithm attempts to generate and
insp ect all the states of a distributed system that are reac h
able from a giv en initial state The three main t yp es of reac h
abilit y analysis algorithms are full searc h con trolled
partial searc h and random sim ulation If full searchex ceeds the memory or time limits it eectiv ely reduces to an
uncon trolled partial searc h and the qualityof the analysis
deteriorates quic kly Con trolled partial searc h attempts to
select a fraction of the full state space that can b e searc hed
within giv en time and space constrain ts Random w alk of the
state space ma y b e used for v ery large state spaces where full
or partial searc h is not feasible
The t ypical measures of reac habilit y analysis qualit y are
state co v erage the fraction of system states tested and error
co v erage the fraction of system errors found This mea
sure represen ts the abilit y to nd errors and is not easily
quan tied since the total n um b er of errors presen t is usually
unkno wn
In practice ho w ev er these measures ma y not b e obtainable
for complex proto cols
In our w ork ho w ev er w e adopt approac hes extending
reac habilit y analysis for m ulticast proto cols Our fault
indep enden t test generation metho d in Section IV is similar
to con trolled partial searc h and uses reduction tec hniques
based on equiv alence relations
B DistributedA lgorithms
There has b een m uchw ork on distributed systems and al
gorithms Distributed algorithms ma y be classied based
on the in terpro cess comm unication metho d or the tim
ing mo del Comm unication metho ds include accessing
shared memory message passing or remote pro cedure calls
With resp ect to timing systems can be sync hronous par
tially sync hronous or async hronous
Sev eral failure mo dels w ere considered in some of the stud
ies on distributed algorithms including message loss or dupli
cation and pro cessor failures suc h as stop or crash failures
transien t failures or b yzan tine failures where failed pro
cessors beha v e arbitrarily Ho w ev er for our target domain
w e do not consider b yzan tine failures
In distributed algorithms are treated in a formal
framew ork using automatatheoretic mo dels and state ma
c hines and results are presen ted in terms of settheoretic
mathematics The formal framew ork is used to presen t pro ofs
or imp ossibilit y results
V erication and pro of metho ds for distributed algorithms
include in v arian t assertions and sim ulation relationships and
are generally pro v ed using induction An in v arian t assertion
is a prop ert y that holds true for all reac hable states of the
system while a sim ulation is a formal relation b et w een an ab
stract solution of the problem and a detailed solution In v ari
antand sim ulation mapping pro ofs maybe c hec k able using
theorempro v ers eg Larc h theorempro v er Async hronous net w ork comp onen ts can b e mo deled as an
inputoutput automata IO automata T o include clo c ks
or timeouts the timedautomata mo dels are used presen ts mo dels for async hronous net w orks that in
clude pro cess and c hannel IO automata mo dels The
c hannel could be poin ttop oin t FIF O queue also called
sendreceiv e c hannel broadcast or m ulticast where only
a set of systems receiv e the messages sen t to the c hannel
In ternet m ultip oin t proto cols that w e address in this study
can b e mo deled as async hronous net w orks with the comp o
nen ts as timedautomata including failure mo dels In fact
the global nite state mac hine GFSM mo del used b y our
searc h algorithms is adopted from async hronous shared mem
ory systems in sp ecic cac he coherence algorithms and
extended with v arious m ulticast and timing seman tics
The transitions of the IO automaton ma y b e giv en in the
form of preconditions and eects This is similar to our
represen tation of the transition table for the faultorien ted
test generation metho d
The com bination of timed automata in v arian ts sim ulation
mappings automaton comp osition and temp oral logic
seem to b e v ery useful to ols for pro ving or dispro ving and
reasoning ab out safet y or liv eness prop erties of distributed al
gorithms It ma y also b e used to establish asymptotic b ounds
on the complexit y of the distributed algorithms
It is not clear ho w ev er ho w theorem pro ving tec hniques
can be used in test syn thesis to construct ev en t sequences
and top ologies that stress net w ork proto cols
In sum w e feel that parts of our w ork dra w from dis
tributed algorithms v erication principles Y et w e feel that
our w ork complemen ts suchw ork as w e fo cus on test syn
thesis problems
C ConformanceT esting
A conformance test is used to c hec k that the external b e ha vior of a giv en implemen tation of a proto col is equiv alen t
to its formal sp ecication A conformance test should fail
only if implemen tation and sp ecication dier By con trast
v erication of the proto col m ust alw a ys rev eal the design er
ror
Giv en an implemen tation under test IUT sequences of
input messages are pro vided and the resulting output is ob
serv ed The test passes only if all observ ed output matc hes
those of the formal sp ecication In eac h state a complete
IUT can accept and resp ond to all input sym b ols from the
complete system v o cabulary The acceptance of an input sig
nal that is outside the ocial input v o cabulary ma y cause a
transition in to a set of states that pro duces erroneous b eha v
ior The series of input sequences used this w ayis called a
conformance test suite The cost of the test can b e expressed
as the length of the test suite ie the total n um b er of mes
sages sen t to the IUT The main problem is to nd an ecien t
pro cedure for generating a conformance test suite for a giv en
proto col
One possible solution is to generate a sequence of state
transitions that passes through ev ery state and ev ery transi
tion at least once also kno wn as a transition tour Ho w ev er in order for this solution to w ork the state of the
mac hine m ust b e c hec k ed after eac h transition since the im
plemen tation ma y b e fault y A Unique InputOutput UIO
sequence is a sequence of transitions that can b e used to de
termine the state of the IUT T obe ableto v erify ev ery state
in the IUT w e m ust b e able to derivea UIO sequence for
ev ery state separately This approac h generally suers from the follo wing dra w
bac ks Not all states of an FSM ha v e a UIO sequence Ev en
if all states in a FSM ha v e a UIO sequence the problem
of deriving UIO sequences has b een pro v ed to b e PSP A CE
complete in ie only v ery short UIO sequences can b e
found in practice
UIO sequences can iden tify states re
liably only in a correct IUT Their b eha vior for faultyIUTs
is unpredictable and they cannot guaran tee that anyt yp e

In a randomized p olynomial time algorithm is presen ted for design
ing UIO c hec king sequences
of fault in an IUT remains detectable Only the presence of
desirable b eha vior can b e tested b y conformance testing not
the absence of undesirable b eha vior
In conclusion conformance testing tec hniques are imp or
tan t for testing proto col implemen tations Ho w ev er it is not
suitable as is to b e used in the design stage of a proto col
W e consider w ork in this area as complemen tary to the fo cus
of our study D VLSI Chip T esting
Chip testing uses a set of w ellestablished approac hes to
generate test v ector patterns generally for detecting ph ysical
defects in the VLSI fabrication pro cess
Common test v ector generation metho ds detect single
stuc k faults where the v alue of a line in the circuit is al
w a ys at logic or T est v ectors are generated based on
a mo del of the circuit and a giv en fault mo del T est v ector
generation can b e faultindep enden t or faultorien ted In a
faultorien ted pro cess test v ectors are generated for sp ecied
faults as dened b y the fault mo del On the other hand a
faultindep enden t pro cess w orks without targeting individual
faults In the faultorien ted pro cess the t w o fundamen tal steps in
generating a test v ector are to activ ate or excite the fault
and to propagate the resulting error to an observ able out
put F ault excitation and error propagation usually in v olv e
a searc h pro cedure with a bac ktrac king strategy to resolveor
undo con tradiction in the assignmen t of line and input v al
ues The line assignmen ts p erformed sometimes determine
or imply other line assignmen ts The pro cess of computing
the line v alues to b e consisten t with previously determined
v alues is referred to as implic ation F orw ard implication is
implying v alues of lines from the fault to w ard the output
while bac kw ard implication is implying v alues of lines from
the fault to w ard the circuit input
Another concept of VLSI testing in whic h w e are in ter
ested is fault equiv alence Tw o faults f and g are said to
be functionally equiv alen t for a circuit C under test x i
C
f
x C g x A test t is said to distinguish b et w een t w o
faults f and g if C
f
t C g t suc h faults are distinguish
able The relation of functional equiv alence partitions the set
of faults in to equiv alence classes F or fault analysis it suces
to consider only one fault from ev ery equiv alen t class
One of the main goals of test v ector generation is to max
imize fault co v erage with a minim um n um b er of inputs The
exp ected output for VLSI c hip testing is fault co v erage vs
test length curv e Our approac hes for proto col testing use some of the ab o v e
principles suc has forw ard and bac kw ard implication fault
indep enden t faultorien ted test generation and fault equiv a
lence
In VLSI c hip testing ho w ev er the test v ectors are pro
duced for a giv en circuit whereas in proto col test generation
the top ology is v ariable A proto col should b e designed to
w ork with arbitrary top ologies whic h adds another dimen
sion to our problem
VI I Summar y and Conclusions
In this study w e ha v e prop osed the STRESS framew ork
that in tegrates test generation in to the proto col design pro
cess Sp ecically w e targeted automatic test generation for
robustness studies of m ulticast routing proto cols Weha v e
adopted a global FSM mo del to representthe m ulticast pro
to cols on a LAN In addition w eha v e used a fault mo del to
represen t pac k et loss and mac hine crashes Weha vein v esti
gated t w o algorithms for test generation namely the fault
indep endenttest generation FITG and the faultorien ted
test generation F OTG Both algorithms w ere used to study
a standard m ulticast routing proto col PIMDM and w ere
compared in terms of errors co v ered and algorithmic com
plexit y F or FITG equiv alence reduction tec hniques w ere
com bined with forw ard searc h to obtain a decrease from ex
p onen tial to p olynomial complexit y for the searc h Ho w ev er
the top ology w as an input to FITG F or F OTG a mix of for
w ard and bac kw ard searchtec hniques allo w ed for automatic
syn thesis of the top ology W e b elieveF OTG is a b etter t for
robustness studies since it targets faults directly The com
plexit y for F OTG w as quite manageable for our case study Corrections to errors captured in the study w ere prop osed
with the aid of our metho d and in tegrated in to the latest
PIMDM sp ecication More case studies are needed to sho w
more general applicabilit y of our metho dology Appendix
I St a te Sp a ce Complexity
In this app endix w e presen t analysis for the state space
complexit y of our target system In sp ecic w e presen t com
pleteness pro of of the state space and the form ulae to com
pute the size of the correct state space
A State Sp ac e Completeness
W e dene the space of all states as X
denoting zero or
more routers in an y state W e also dene the algebraic op er
ators for the space where
X
X
X
X

Y
n
X

Y
n fX Y g
A Error states
In general an error ma y manifest itself as pac k et dupli
cates pac k et loss or w asted bandwidth This is mapp ed
on to the state of the global FSM as follo ws
The existence of t w o or more forw arders on the LAN with
one or more routers exp ecting pac k et from the LAN eg in
the NH X state indicates duplicate deliv ery of pac k ets
The existence of one or more routers exp ecting pac k ets
from the LAN with no forw arders on the LAN indicates a
deciency in pac k et deliv ery join latency or blac k holes
The existence of one or more forw arders for the LAN with
no routers exp ecting pac k ets from the LAN indicates w asted
bandwidth lea v e latency or extra o v erhead
for duplicates one or more NH X with t woormore F X NH X F

X
X
for extra bandwidth one or more F X with zero NH X F X fX NH X g
for blac kholes or pac k et loss one or more NH X with zero
F X NH X fX F X g
A Correct states
As describ ed earlier the correct states can b e describ ed b y
the follo wing rule
exactly one forwar der for the LAN i one or mor e
r outers exp e cting p ackets fr om the LAN zero NH X with zero F X fX NH X F X g
one or more NH X with exactly one F X NH X F X fX F X g
from B and B w e get
NH X F
X
fX F X g
if w e tak e the union of B B and B and apply
B w e get
NH X X

NH
X
fX NH X g
also from B and B w eget F
X
fX NH X F X g

if w e tak e the union of B and B w e get
F
X
fX NH X F X g
fX NH X g

taking the union of B and B w e get
NH
X
fX NH X g
X

whic h is the complete state space
B Numb er of Corr e ct and Err or State Sp ac es
B First case denition
F or the correct states fX NH F g
reduces the sym
bols from whic h to c ho ose the state b y ie yields the
form ula
C n s n C n s n While NH F fX F g
reduces the n um ber of routers
to c ho ose b y and the n um ber of sym bols b y yielding
C n s n C n s n B Second case denition
F or the correct states fX NH X F X g
reduces the
n um b er of states b y yielding
C n s n C n s n While NH X F X fX F X g
reduces the n um ber of
routers to n and the sym b ols to s and yields
C n s n C n s n Weha v e to b e careful here ab out o v erlap of sets of correct
states F or example NH F fX F X g
is equiv alen t to
NH Rtx F fX F X g
when a third router is in NH Rtx in
the rst set and NH in the second set Th us w e need to
remo v e one of the sets NH F N H Rtx fX F X g
whic h
translates in terms of n um b er of states to
C n s n C n s n A similar argumentis giv en when w e replace F ab o veb y
F
Del
th us wem ultiply the n um b er of states to b e remo v ed
b y Th us w e get the total n um ber of equiv alen t correct
states
C n s n C n s n C n s n
T o obtain the E r r or S tates w e can use
E r r or S tates T otal S tates C or r ectS tates
0
10
20
30
40
50
60
70
80
90
100
1 11 21 31 41 51 61 71 81 91
number of routers (n)
Percentage
Error States
Correct States
Fig The p ercen tage of the correct and error states
Figure sho ws the p ercen tage of eac h of the correct and
error state spaces and ho w this p ercen tage c hanges with the
n um b er of routers The gure is sho wn for the second case
error denition Similar results w ere obtained for the rst
case denition
II F or w ard Sear ch Algorithms
This app endix includes detailed pro cedures that imple
men t the forw ard searc h metho d as describ ed in Section IV
It also includes detailed statistics collected for the case study
on PIMDM
A Exhaustive Se ar ch
The ExpandSpace pro cedure giv en b elo w implemen ts an
exhaustiv e searc h where W is the w orking set of states to
b e expanded V is the set of visited states ie already ex
panded and E is the state curren tly b eing explored Ini
tially all the state sets are empt y The nextState function
gets and remo v es the next state from W according to the
searc h strategy if depth rst then W is treated as a stac k
or as a queue if breadth rst
Eac h state is expanded b y applying the stim uli via the
forw ard pro cedure that implemen ts the transition rules and
returns the new stable state New ExpandSpace initGS tate f
add initGS tate to W
while W not empt y f
E nextGState from W add E to V state E
stim applying to state f
New forw ard E stim
if New W or V
add New to W g
g
g
The initial state initGS tate maybe generated using the
follo wing pro cedure that pro duces all p ossible com binations
of initial states I S Init depth GS tate f
state IS f
add state to GS tate depth depth if depth
ExpandSpace GS tate
else
Init depth GS tate
remo v e last elementof GS tate g
g
This pro cedure is called with the follo wing parameters
a n um ber of routers n as the initial depth and b the
empty state as the initial GS tate It is a recursiv e pro ce
dure that do es a tree searc h depth rst with the n um ber of
lev els equal to the n um b er of routers and the branc hing fac
tor equal to the n um b er of initial state sym b ols jI S j is The complexit y of this pro cedure is giv en by is n
B R e duction Using Equivalenc e
W e use the coun ting equiv alence notion to reduce the com
plexit y of the searchin w a ys
The rst reduction w e use is to in v estigate only the equiv
alen t initial states w e call this algorithm Equiv One pro ce
dure that pro duces suchequiv alen t initial state space is the
EquivInit pro cedure giv en b elo w
EquivInit S i GS tate f
state S
for j i to f
New empty state for k to j
add state to New
Expanded States
Rtrs Exhaustive Equiv Equiv+ Reduced Reduction
1 14 10 9 9 1.555556
2 52 24 18 18 2.888889
3 178 52 30 30 5.933333
4 644 114 48 48 13.41667
5 2176 238 73 73 29.80822
6 7480 496 106 106 70.56604
7 24362 1004 148 148 164.6081
8 80830 2037 200 200 404.15
9 259270 4081 263 263 985.8175
10 843440 8198 338 338 2495.385
11 2684665 16386 426 426 6302.031
12 8621630 32810 528 528 16328.84
13 27300731 65574 645 645 42326.71
14 86885238 131180 778 778 111677.7
Fig Sim ulation statistics for forw ard algorithms E xpandedS tates
is the n um b er of visited states
Forwards
Rtrs Exhaustive Equiv Equiv+ Reduced Reduction
1 80 55 51 43 1.860465
2 537 227 177 124 4.330645
3 2840 730 440 263 10.79848
4 14385 2188 970 503 28.59841
5 63372 5829 1923 881 71.9319
6 271019 14863 3491 1430 189.5238
7 1060120 35456 5916 2187 484.7371
8 4122729 82916 9480 3189 1292.797
9 15187940 187433 14523 4477 3392.437
10 55951533 419422 21429 6092 9184.428
11 199038216 921981 30648 8079 24636.49
12 708071468 2013909 42678 10483 67544.74
13 2.461E+09 4355352 58091 13353 184311
14 8.546E+09 9375196 77511 16738 510576.4
Fig Sim ulation statistics for forw ard algorithms F orwards is the
n um b er of calls to f orward
New New GS tate
S trunc S state
if i j ExpandSpace New else
EquivInit
S i j New g
g
This pro cedure is in v ok ed with the follo wing parameters a
the initial set of states I S as S b the n um b er of routers
n as i and c the empty state as GS tate The pro cedure
is recursiv e and pro duces the set of equiv alen t initial states
and in v ok es the ExpandSpace pro cedure for eachequiv alen t
initial state The trunc function truncates S suc h that
!
S
con tains only the state elemen ts in S after the elemen t state F or example trunc fF N M M gF fNM M g The second reduction w e use is during state comparison
Instead of comparing the actual states w e compare and store
equiv alen t states Hence the line if New W or V w ould
c hec k for equiv alen t states W e call the algorithm after this
second reduction Equiv The third reduction is made to eliminate redundan t tran
sitions Toac hiev e this reduction w e add ag c hec k b efore
in v oking forw ard suc h as stateFlag The ag is set to Transitions
Rtrs Exhaustive Equiv Equiv+ Reduced Reduction
1 19 11 11 11 1.727273
2 90 32 31 31 2.903226
3 343 75 65 65 5.276923
4 1293 169 119 119 10.86555
5 4328 347 197 197 21.96954
6 14962 722 307 307 48.73616
7 47915 1433 449 449 106.7149
8 158913 2889 633 633 251.0474
9 503860 5717 857 857 587.9347
10 1638871 11434 1133 1133 1446.488
11 5185208 22715 1457 1457 3558.825
12 16666549 45383 1843 1843 9043.163
13 52642280 90461 2285 2285 23038.2
14 167757882 180794 2799 2799 59934.93
Fig Sim ulation statistics for forw ard algorithms T r ansitions is
the n um b er of transien t states visited
Error States
Rtrs Exhaustive Equiv Equiv+ Reduced Reduction
1 1 1 1 1 1
2 7 3 3 3 2.333333
3 33 7 6 6 5.5
4 191 21 13 13 14.69231
5 783 49 25 25 31.32
6 3235 115 43 43 75.23256
7 11497 239 68 68 169.0735
8 41977 504 101 101 415.6139
9 142197 1012 143 143 994.3846
10 491195 2057 195 195 2518.949
11 1625880 4101 258 258 6301.86
12 5441177 8237 333 333 16339.87
13 17751178 16425 421 421 42164.32
14 58220193 32879 523 523 111319.7
Fig Sim ulation statistics for forw ard algorithms The n um ber of
stable error states reac hed
when the stim uli for that sp ecic state ha v e b een applied
W e call the algorithm after the third reduction the reduced
algorithm
C Complexity analysis of forwardse ar ch for PIMDM
The n um ber of reac hable states visited the n um ber of
transitions and the n um ber of erroneous states found w ere
recorded The result is giv en in gures The
reduction is the ratio of the n um b ers obtained using the ex
haustiv e algorithm to those obtained using the reduced al
gorithm
The n um b er of expanded states denotes the n um b er of vis
ited stable states and is measured simply as the n um ber of
states in the set V in ExpandSpace pro cedure The n um ber
of forw ards is the n um ber of times the forw ard pro cedure
w as called denoting the n um b er of transitions b et w een stable
states The n um ber of transitions is the n um ber of visited
transien t states that are increased with ev ery new state vis
ited in the forw ard pro cedure The n um b er of error states
is the n um ber of stable or expanded states violating the
correctness conditions
The n um b er of transitions is reduced from O
n
for the
exhaustiv e algorithm to O n
for the reduced algorithm
This means that weha v e obtained exp onen tial reduction in
complexit yas sho wn in gure 1.E+0
1.E+1
1.E+2
1.E+3
1.E+4
1.E+5
1 2 3 4 5 6 7 8 9 10 11 12 13 14
number of routers (n)
reduction ratio in states [log]
Fig Reduction ratio from exhaustiv e to the reduced algorithm
III F OTG Algorithms
This app endix includes pseudoco de for pro cedures imple
men ting the faultorien ted test generation F OTG metho d
presen ted in Section V In addition it includes detailed re
sults of our case study to apply F OTG to PIMDM
A Pr eConditions
The pro cedure describ ed b elo w tak es as input the set of
p ostconditions for the FSM stim uli and genrates the set
of preconditions The conds arra y con tains the p ost
conditions ie the eects of the stim uli on the system and
is indexed b y the stim ulus The stim ulus function returns
the stim ulus if an y of the condition The transition func
tion returns the transition or state of the condition
The
preconditions are stored in an arra y pr eC onds indexed b y
the stim ulus
PreConditionsf
stim cond conds stim f
s stim ulus cond t transition cond add tstim to pr eC onds s g
g
B Dep endency T able
The dep endencyT able pro cedure generates the dep en
dency table depT abl e from the transition table of conditions
conds dep endencyT ablef
stim If theres a state in the condition this ma y b e view ed as state state
transition ie transition to the same state
cond conds stim f
endS tate end cond
star tS tate start cond add star tS tatestim to depT able endS tate g
g
F or eac h state s that is endS tate of a transition a set of
star tS tate stimul us pairs leading to the creation of s is
stored in the depT abl e arra y F or s IS asym b ol denoting
initial state is added to the arrayen try F or our case study
IS fNM EU g C T op olo gy Synthesis
The follo wing pro cedure syn thesizes minim um top ologies
necessary to trigger the v arious stim uli of the proto col It
p erforms the third and forth steps of the top ology syn thesis
pro cedure explained in Section VB
buildMinT op os stim f
cond pr eC onds stim f
st end cond
stm stim ulus cond
if t yp e stm or ig
add st to MinT opos stim
else f
if Topo stm buildMinT op os stm topo M inT opos stim add st to M inT opos stim
g
g
g
D BackwardSe ar ch
The Bac kw ard pro cedure calls the Rewind pro cedure to
p erform the bac kw ard searc h A set of visited states V is k ept
to a v oid lo oping F or eac h state in GS tate p ossible bac k
w ard implications are attempted to obtain v alid bac kw ard
steps to w ard initial state Bac kw ard is called recursiv ely
for preceding states as a depth rst searc h If all bac kw ard
branc hes are exhausted and no initial state w as reac hed the
state is declared unreac hable
Bac kw ard GS tate f
if GS tate V
return loop
add GS tate to V
s GS tatef
bk w ds depT able s
bk bkwdsf
New Rewind bk GS tate s if New done
break
else
Bac kw ard New
g
g
if all states are done
return reac hed
else
return unreac hable
g
The Rewind pro cedure tak es the global state one
step bac kw ard b y applying the rev erse transition rules
replace s st GS tate replaces s in GS tate with st and re
turns the new global state Dep ending on the stim ulus
t yp e of the bac kw ard rule bk dieren t states in GS tate are
rolled bac k F or or ig and dst only the originator and des
tination of the stim ulus is rolled bac k resp ectiv ely F or
mcast all aected states are rolled bac k except the origi
nator mcastD ow nstr eam is similar to mcast except that
all do wnstream routers or states are rolled bac k while only
one upstream router the destination is rolled bac k
Rewind bk GS tate s f
if bk IS return done
stim stim ulus bk st start bk if t yp e stim or ig f
New replace s st GS tate return New g
cond pr econds stim
while sr c not found f
str start cond if str GS tate
sr c found
g
if sr c not found
return bac kT rac k
if t yp e stim dst f
New replace s st GS tate if c hec kMinT op o New stim return New else
return bac kT rac k
if not c hec kConsistency stim GS tate return bac kT rac k
New GS tate if t yp e stim mcast
cond conds stim if end cond GS tate not sr c
New replaceendstartGState
if t yp e stim mcastD ow nstr eam
cond conds stim if end cond GS tate not upstr eam
New replaceendstartGState
else if end GS tate upstr eam
New replaceendstartGState once
if c hec kMinT op o New stim return New else
return bac kT rac k
g
Total Average
Backwards Rewinds BackTracks Backwards Rewinds BackTracks
Unreachable (6) 223 586 293 37.16 97.6 48.8
Reachable (16) 23030 61212 31736 1439 3825 1983
Total (22) 23253 61798 32029 1057 2809 1455
Fig Case study statistics for applying F OTG to PIMDM
The follo wing pro cedure c hec ks for consistency of applying
stim to GS tate c hec kConsistency stim GS tate f
cond conds stim cond has transition
if start cond GS tate
return F alse
else
return T rue
g
The follo wing pro cedure c hec ks if GS tate con tains the nec
essary comp onen ts to trigger the stim ulus
c hec kMinT op o GS tate stim f
if M inT opos stim GS tate
return T rue
else
return F alse
g
E Simulation r esults
Weha v e conducted a case study of PIMDM analysis us
ing F OTG A total of top ologies w ere automatically con
structed using as faults the selectiv e loss of JoinPrune
Graft and Assert messages Out of the constructed top olo
gies or global states w ere unreac hable global states and w ere reac hable The statistics for the total and a v erage n um
ber of bac kw ard calls rewind calls and bac ktrac ks is giv en in
gure Although the top ology syn thesis study weha v e presen ted
ab o v e is not complete w e ha v e co v ered a large n um ber of
corner cases using only a manageable n um b er of top ologies
and searc h steps
T o obtain a complete represen tation of the top ologies w e
suggest to use the sym bolic represen tation

presen ted in
Section III Based on our initial estimates w e exp ect the
n um ber of sym b olic top ology representationstobe appro x
imately top ologies ranging from to router LAN
top ologies for the single selectiv e loss and single crash mo d
els

Weha v e used the rep etition constructs
Backwards
total average
all states Reachable Unreachable all states Reachable Unreachable
280 64 216 10.77 7.111 12.71
3965 1056 2909 38.12 37.71 38.28
58996 30694 28302 180.4 383.7 114.6
899274 612009 287265 1021 3255 414.5
Number of calls to Backward()
Rewinds
total average
all states Reachable Unreachable all states Reachable Unreachable
471 116 355 18.12 12.89 20.88
8309 2379 5930 79.89 84.96 78.03
134529 71954 62575 411.4 899.4 253.3
2067426 1414365 653061 2347 7523 942.4
Number of calls to Rewind()
BackTracks
total average
all states Reachable Unreachable all states Reachable Unreachable
163 30 133 6.269 3.333 7.824
3459 946 2513 33.26 33.79 33.07
60321 32684 27637 184.5 408.6 111.9
950421 656028 294393 1079 3490 424.8
Number of back tracks
for Error states
Fig Sim ulation statistics for bac kw ard algorithms
F Exp erimental statistics for PIMDM
Toin v estigate the utilityof F OTG as a v erication to ol w e
ran this set of sim ulations This is not ho w ev er ho wF OTG
is used to study proto col robustness see Section CE for case
study analysis
W e also w an ted to study the eect of unreac hable states on
the complexit y of the v erication The sim ulations for our
case study sho w that unreac hable states do not con tribute
in a signican t manner to the complexit y of the bac kw ard
searc h for larger top ologies Hence in order to use F OTG as
av erication to ol it is not sucien t to add the reac habilit y
detection capabilitytoF OTG
The bac kw ard searchw as applied to the equiv alenterror
states for LANs with to routers connected The sim ula
tion setup in v olv ed a call to a pro cedure similar to EquivInit
in Section BB with the parameter S as the set of state sym
b ols and after an error c heckw as done a call is made to the
Bac kw ard pro cedure instead of ExpandSpace
States w ere classied as reac hable or unreac hable F or the
four top ologies studied LANs with to routers statistics
w ere measured eg max min median a v erage and total
for n um ber of calls to the Bac kw ard and Rewind pro ce
dures and the n um b er of bac kT rac ks w ere measured
As sho wn in gure the statistics sho w that as the top ol
ogy gro ws all the n um b ers for the reac hable states get sig
nican tly larger than those for the unreac hable states as in
gure despite the fact that that the p ercen tage of un
reac hable states increases with the top ology as in gure The reason for suc h b eha vior is due to the fact that when the
state is unreac hable the algorithm reac hes a deadend rela
tiv ely early b y exhausting one branc h of the searc h tree
Ho w ev er for reac hable states the algorithm k eeps on searc h
ing un til it reac hes an initial global state Hence the reac h
able states searc h constitutes the ma jor comp onen t that con
tributes to the complexit y of the algorithm
0
1000
2000
3000
4000
5000
6000
7000
8000
2 3 4 5
number of routers (n)
number of rewinds (avg)
All
Reachable
Unreachable
Fig Complexit y of the F OTG algorithm for error states
0
10
20
30
40
50
60
70
80
90
2 3 4 5
number of routers (n)
percentage
Unreachable
Reachable
Fig P ercen tage of reac hableunreac hable error states using F OTG
G R esults
Weha v e implemen ted an early v ersion of the algorithm in
the NSVINT en vironmentsee h ttpcatarinausceduvin t
and used it to driv e detailed sim ulations of PIMDM therein
to v erify our ndings In this section w e discuss the results of
applying our metho d to PIMDM The analysis is conducted
for single selectiv e message loss
F or the follo wing analyzed messages w e presen t the steps
for top ology syn thesis forw ard and bac kw ard implication
G Join
F ollo wing are the resulting steps for join loss
Syn thesizing the Global State
Set the insp ected message to Join
The star tS tate of the p ostcondition is F
dst Del
G
I
fF
j Del
g
The state of the precondition is NH
i
G
I
fNH
i
F
j Del
g
The stim ulus of the precondition is P r une Set the insp ected message
to Prune
The star tS tate of the p ostcondition is F
j
whic h can b e implied from
F
j Del
in G
I
The state of the precondition is NC
k
G
I
fNH
i
F
j Del
NC
k
g
The stim ulus of the precondition is L Set the insp ected message to L
The star tS tate of the p ostcondition is NH whic h can b e implied from
NC in G
I
The state of the precondition is Ext an external ev en t
F orw ard implication
without loss G
I
fNH
i
F
j Del
NC
k
g
J oin
G
I fNH
i
F
j
NC
k
g
correct state
loss wrt R
j
fNH
i
F
j Del
NC
k
g
Del
G
I fNH
i
NF
j
NC
k
g
error state
Bac kw ard implication
G
I
fNH
i
F
j Del
NC
k
g
P r une
G
I fNH
i
F
j
NC
k
g
FP kt
G
I fM
i
F
j
NM
k
g
SP kt
G
I fM
i
EU
j
NM
k
g
HJ
i
G
I fNM
i
EU
j
NM
k
g IS Losing the Join b y the forw arding router R j leads to an
error state where router R i is exp ecting pac k ets from the
LAN but the LAN has no forw arder
G Assert
F ollo wing are the resulting steps for the Asser t loss
Syn thesizing the Global State
Set the insp ected message to Asser t
The star tS tate of the p ostcondition is F
j
G
I
fF
j
g
The state of the precondition is F
i
G
I
fF
i
F
j
g
Stim ulus of precondition is FP kt
j
Set insp ected message to FPkt
j
The star tS tate of the p ostcondition is EU
i
implied from F
i
in G
i
The state of the precondition is F
j
already in G
I
Stim ulus of precondition is SP kt
j
Set insp ected message to SP kt
j
The star tS tate of the p ostcondition is NF
j
implied from F
j
in G
I
The stim ulus of the precondition is Ext an external ev en t
F orw ard Implication
G
I
fF
i
F
j
g
Asser t
i
G
I fF
i
NF
j
g error
Bac kw ard Implication
G
I
fF
i
F
j
g
FP kt
j
G
I fEU
i
F
j
g
SP kt
j
G
I fEU
i
EU
j
g IS The error in the Asser t case occurs ev en in the absence
of message loss This error o ccurs due to the absence of a
prune to stop the o w of pac k ets to a LAN with no do wn
stream receiv ers This problem occurs for top ologies with
G I fF i F j F
k
g as that sho wn in gure G Graft
F ollo wing are the resulting steps for the Gr af t loss
LAN
Source
Fi Fj
Fk
.. .
Fig A top ology ha ving a fF i F j F
k
g LAN
Syn thesizing the Global State
Set the insp ected message to Gr af t
Rcv
The star tS tate of the p ostcondition is NF G
I
fNF g
the endS tate of the precondition is NH
Rtx
G
I
fNF N H
Rtx
g
The stim ulus of the precondition is Gr af t
Tx
The star tS tate of the p ostcondition is NH implied from NH
Rtx
in G
I
the endS tate of the precondition is NH whichma y b e implied
the stim ulus of the precondition is HJ whic his Ext external
F orw ard Implication
without loss G
I
fNH N F g
Gr af t
Tx
G
I fNH
Rtx
NF g
Gr af t
Rcv
G
I fNH
Rtx
F g
GAck
G
I fNH F g correct state
with loss of Gr af t G
I
fNH N F g
Gr af t
Tx
G
I
fNH
Rtx
NF g
T imer
G
I
fNH N F g
Gr af t
Tx
G
I
fNH
Rtx
NF g
Gr af t
Rcv
G
I
fNH
Rtx
F g
GAck
G
I
fNH F g correct state
W e did not reac h an error state when the Gr af t w as lost
with nonin terlea ving external ev en ts
H Interle aving events and Se quencing
A Gr af t message is ac kno wledged b y the Gr af t Ack
GAck message and if not ac kno wledged it is retransmitted
when the retransmission timer expires In an attempt to cre
ate an erroneous scenario the algorithm generates sequences
to clear the retransmission timer and insert an adv erse ev en t
Since the Gr af t reception causes an upstream router to b e come a forw arder for the LAN the algorithm in terlea v es a
Leav e ev entas an adv ersary ev entto cause that upstream
router to b ecome a nonforw arder
T o clear the retransmission timer the algorithm inserts the
transition NH
GAck
NH Rtxin the ev en t sequence
F orw ard Implication
G I fNH N F g
Gr af t
Tx
G I fNH Rtx N F g
GAck
G I fNH N F g error state
Bac kw ard Implication
Using bac kw ard implication w e can construct a sequence
of ev en ts leading to conditions sucien t to trigger the GAck F rom the transition table these conditions are fNH Rtx F g

G I fNH N F g
HJ
G I fNC N F g
Del
G I fNC F
Del
g
P r une
G I fNC F g
L
G I fNH Rtx F g
W e do not sho w all branc hing or bac ktrac king steps for simplicit y
T o generate the GAck w e con tin ue the bac kw ard implica
tion and attempt to reac h an initial state
G I fNH Rtx F g
Gr af t
Rcv
G I fNH Rtx N F g
Gr af t
Tx
G I fNH N F g
HJ
G I fNC N F g
Del
G I fNC F
Del
g
Prune
G I fNC F g
FPkt
G I fNM F g
SP kt
G I fNM EU g IS Hence when a Gr af t follo w ed bya P r une is in terlea v ed
with the Gr af t loss the retransmission timer is reset with
the receipt of the GAck for the rst Gr af t and the systems
ends up in an error state
References
V P axon EndtoEnd Routing Beha vior in the In ternet IEEEA CM
T r ansactions on Networking V ol No A ne arlier version app e ar edin
Pr o c A CM SIGCOMM Stanfor d CA pages Octob er V P axon EndtoEnd In ternet P ac k et Dynamics A CM SIGCOMM
Septem ber E Clark e and J Wing F ormal Metho ds State of the Art and F u
ture Directions A CM Workshop on Str ate gic Dir e ctions in Computing
R ese ar ch V ol No pages Decem ber A Helm yA Surv ey on Kernel Sp ecication and V erication T e chnic al
R ep ort of the Computer Scienc eDep artment University of South
ern California URL httpwwwusce dudeptcste chnic al r ep ortshtml D W aitzman S Deering C P artridge Distance V ector Multicast Rout
ing Proto col No v em b er RF C
J Mo y Multicast Extension to OSPF Internet Dr aft Septem b er
D Estrin D F arinacci A Helm y V Jacobson and L W ei
Proto col Indep enden t Multicast Dense Mo de PIMDM
Proto col Sp ecication Pr op ose d Exp erimental RF C URL
httpnetwebusce dupimpimdmPIMDM ftxtps ggz Septem ber
A J Ballardie P FF rancis and J Cro w croft Core Based T rees In
Pr o c e e dings of the A CM SIGCOMM San F rancisco D Estrin D F arinacci A Helm y D Thaler S Deering M Handley V Jacobson C Liu P Sharma and L W ei Protocol Independen t
Multicast Sparse Mode PIMSM Motiv ation and Arc hitecture Pr o
p ose d Exp erimental RF C URL httpnetwebusce dupimpimsmPIM
A r ch ftxtps ggz Octob er F Lin P Ch u and M Liu Proto col V erication using Reac habilit y
Analysis Computer Communic ation R eview V ol No T Cormen C Leiserson and R Riv est In tro duction to Algorithms
The MIT Pr ess McGr awHil l F P ong and M Dub ois V erication T ec hniques for Cac he Coherence
Proto cols A CM Computing Surveys V olume No pages March G Holzmann Design and V alidation of Computer Proto cols A TT
Bel l L abs Pr entic e Hal l D Estrin D F arinacci A Helm y D Thaler S Deering M Han
dley V Jacobson C Liu P Sharma and L W ei Proto col In
dep enden t Multicast Sparse Mo de PIMSM Proto col Sp ecica
tion RF C URL httpnetwebusce dupimpimsmPIMSMv
ExpRF C ftxtps ggz Marc h F Lin P Ch u and M Liu Proto col V erication using Reac habilit y
Analysis the state explosion problem and relief strategies Pr o c e e dings
of the A CM SIGCOMM D Probst Using partialorder seman tics to a v oid the state explosion
problem in async hronous systems Pr o c nd Workshop on Computer
A idedV eric ation Springer V erlag New Y ork P Go defroid Using partial orders to impro v e automatic v erication
metho ds Pr o c nd Workshop on ComputerA idedV eric ation Springer
V erlag New Y ork N Maxemc h uc k and K Sabnani Probabilistic v erication of comm u
nication proto cols Pr o c th IFIP WG Int Workshop on Pr oto c ol
Sp e cic ation T esting and V eric ation NorthHol land Publ A mster dam C W est Proto col V alidation b y Random State Exploration Pr o c
th IFIP WG Int Workshop on Pr oto c ol Sp e cic ation T esting and
V eric ation NorthHol land Publ A mster dam
J P ageot and C Jard Exp erience in guiding sim ulation Pr o c VIII
th Workshop on Pr oto c ol Sp e cic ation T esting and V eric ation A tlantic
City NorthHol land Publ A mster dam
Nancy Lync h Distributed Algorithms Mor gan Kaufmann K Saleh I Ahmed K AlSaqabi and A Agarw al Areco v ery ap
proac h to the design of stabilizing comm unication proto cols Journal of
Computer Communic ation V ol No pages April J Spiv ey Understanding Z a Specication Language and its F ormal
Seman tics Cambridge University Pr ess C Jones Systematic Soft w are Dev elopmen t using VDM Pr entic eHal l
Intl R Bo y er and J Mo ore A Computational Logic Handb o ok A c ademic
Pr ess Boston S OwreJRush b yN Shank er and F Henk e F ormal v erication for
faulttoleran tarchitectures Prolegomena to the design of PVS IEEE
T r ansactions on Softwar e Engine ering pages F ebruary
WD T a jibnapis A correctness pro of of a top ology information main
tainence proto col for a distributed computer net w ork Communic ations
of the A CM K Shin and M Chen P erformance analysis of distributed routing
strategies free of pingp ongt yp e lo oping IEEE T r ansactions on Com
puters C F ebruary J Jaee and F Moss A resp onsiv e distributed routing algorithm for
computer net w orks July JJ GarciaLunaAcev es Lo opfree routing using diusing computa
tions F ebruary P M Merlin and A Segall A failsafe distributed routing proto col
COM GC Roman P J McCann and JY Plun Assertional reasoning ab out
pairwise transientin teractions in mobile computing In IEEE edi
tor
th
International Confer enc e on Softwar e Engine ering pages
Marc h KM Chandy and J Misra Par al lel Pr o gr am Design AddisonW esley
Publishing Compan y Inc B Hailp ern A simple proto col whose pro of isnt COM
April R Milner A Calculus of Communic ating Systems Num b er Springer
V erlag
J P arro w V erifying a CSMACDproto col with CCS In VIIIpages
IFIP
M Smith F ormal V erication of Comm unication Proto cols
F OR TEPSTV Confer enc e Octob er Leslie Lamp ort Rob ert Shostak and Marshall P ease The Byzan tine
generals problem A CM T r ansactions on Pr o gr amming L anguages and
Systems Stephen Garland and John Guttag A guide to LP the Larc h Pro v er
R ese ar ch R ep ort Digital Systems R ese ar ch Center Mic hael Merritt F rancesmary Mo dugno and Mark T uttle Time con
strained automata nd International Confer enc e on Concurr ency The
ory
Leslie Lamp ort The temp oral logic of actions A CM T r ansactions on
Pr o gr amming L anguages and SystemsMa y V Klee Com binatorial optimization what is the state of the art!
Math Op er ations R ese ar ch V ol M Y annak akis and D Lee T esting nite state mac hines A CM Sym
p osium on The ory of Computing STOC M Y annak akis and D Lee T esting Finite State Mac hines Journal of
c omputer and systems scienc es JCSS M Abramo vici M Breuer and A F riedman Digital Systems T esting
and T estable Design ATTL abs B Konemann B Bennetts N Jarw ala and B NadeauDostie BuiltIn
SelfT est Assuring System In tegrit y IEEE Computer Magazinepages
No v em ber

Linked assets

Computer Science Technical Report Archive

Conceptually similar

PDF

USC Computer Science Technical Reports, no. 690 (1998)

PDF

USC Computer Science Technical Reports, no. 673 (1998)

PDF

USC Computer Science Technical Reports, no. 696 (1999)

PDF

USC Computer Science Technical Reports, no. 726 (2000)

PDF

USC Computer Science Technical Reports, no. 657 (1997)

PDF

USC Computer Science Technical Reports, no. 674 (1998)

PDF

USC Computer Science Technical Reports, no. 755 (2002)

PDF

USC Computer Science Technical Reports, no. 801 (2003)

PDF

USC Computer Science Technical Reports, no. 644 (1997)

PDF

USC Computer Science Technical Reports, no. 730 (2000)

PDF

USC Computer Science Technical Reports, no. 663 (1998)

PDF

USC Computer Science Technical Reports, no. 809 (2003)

PDF

USC Computer Science Technical Reports, no. 860 (2005)

PDF

USC Computer Science Technical Reports, no. 743 (2001)

PDF

USC Computer Science Technical Reports, no. 811 (2003)

PDF

USC Computer Science Technical Reports, no. 734 (2000)

PDF

USC Computer Science Technical Reports, no. 649 (1997)

PDF

USC Computer Science Technical Reports, no. 608 (1995)

PDF

USC Computer Science Technical Reports, no. 757 (2002)

PDF

USC Computer Science Technical Reports, no. 753 (2002)

Description

Ahmed Helmy, Deborah Estrin, Sandeep Gupta. "Systematic testing of multicast routing protocols: Analysis of forward and backward search techniques." Computer Science Technical Reports (Los Angeles, California, USA: University of Southern California. Department of Computer Science) no. 727 (2000).

Asset Metadata

Creator Estrin, Deborah (author), Gupta, Sandeep (author), Helmy, Ahmed (author)

Core Title USC Computer Science Technical Reports, no. 727 (2000)

Alternative Title Systematic testing of multicast routing protocols: Analysis of forward and backward search techniques (title)

Publisher Department of Computer Science,USC Viterbi School of Engineering, University of Southern California, 3650 McClintock Avenue, Los Angeles, California, 90089, USA (publisher)

Tag OAI-PMH Harvest

Format 28 pages (extent), technical reports (aat)

Language English

Unique identifier UC16269328

Identifier 00-727 Systematic Testing of Multicast Routing Protocols Analysis of Forward and Backward Search Techniques (filename)

Legacy Identifier usc-cstr-00-727

Format 28 pages (extent),technical reports (aat)

Rights Department of Computer Science (University of Southern California) and the author(s).

Internet Media Type application/pdf

Source 20180426-rozan-cstechreports-shoaf (batch), Computer Science Technical Report Archive (collection), University of Southern California. Department of Computer Science. Technical Reports (series)

Access Conditions The author(s) retain rights to their work according to U.S. copyright law. Electronic access is being provided by the USC Libraries, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright.

Repository Name USC Viterbi School of Engineering Department of Computer Science

Repository Location Department of Computer Science. USC Viterbi School of Engineering. Los Angeles\, CA\, 90089

Repository Email csdept@usc.edu

Inherited Values

Title Computer Science Technical Report Archive

Description Archive of computer science technical reports published by the USC Department of Computer Science from 1991 - 2017.

Coverage Temporal 1991/2017

Repository Email csdept@usc.edu

Repository Name USC Viterbi School of Engineering Department of Computer Science

Repository Location Department of Computer Science. USC Viterbi School of Engineering. Los Angeles\, CA\, 90089

Publisher Department of Computer Science,USC Viterbi School of Engineering, University of Southern California, 3650 McClintock Avenue, Los Angeles, California, 90089, USA (publisher)