USC Computer Science Technical Reports, no. 660 (1997)

PDF

Download

Open document

Flip pages

Download a page range

Katia Obraczka and Grig Gheorghiu. "The performance of a service for network-aware applications ." Computer Science Technical Reports (Los Angeles, California, USA: University of Southern California. Department of Computer Science) no. 660 (1997).

Transcript (if available)

Content The P erformance of a Service for Net w orkAw are Applications
Katia Obraczk a and Grig Gheorghiu
Information Sciences Institute
Univ ersit y of Southern California
Admiralt yW a y suite Marina del Rey CA k atia grigisiedu
v oice fax
Abstract
This pap er ev aluates the p erformance of top olo gyd a service for applications that require kno wledge of the
underlying comm unication and computing infrastructure in order to deliv er adequate p erformance T op ology
d estimates the state of the net w ork and net w ork ed resources b y p erio dicall y computing endtoend latency
and a v ailable bandwidth Using its estimates top ologyd computes a fault toleran t high bandwidth lo w dela y
top ology connecting participatin g sites T op ologies are p erio dical ly recomputed to tak ein to accoun t net w ork
and serv er load dynamics Networkawar e application s can then mak e use of top ologyds estimates and logical
top ologies to ensure they get adequate service from the underlying net w ork and computing infrastructure
W e deplo y ed top ologyd on In ternet sites throughout the w orld and collected latency bandwidth and
top ology information for a p erio d of t w o and a half mon ths The results of these In ternetwide exp erimen ts
sho w that top ologyds estimates compare quite w ell with latency and bandwidth measuremen ts from existing
to ols W e also observ e that the logical top ologies computed b y top ologyd are consisten t with curren t latency
and bandwidth estimates The top ologies are stable in the sense that the set of neigh b ors a host sees is small
usually or and sta ys relativ ely constanto v er time On the other hand w eobserv e that top ologies are
resp onsiveto c hanges in net w ork and serv er load as w ell as in group mem b ership
In tro duction
La y eredbased approac hes to system design hide lo w erlev el details from the upp er la y ers The goal is to mak e
the design task easier b y pro viding dieren tlev els of abstraction eac hla y er oers a set of w ell dened services
to the next la y er up in the proto col stac k This approac hw ork ed w ell un til recen tly The gro wth of the In ternet
motiv ated the dev elopmentofav ariet y of more mo dern distributed applications suc h as information dissemina
tion m ultim edia and metacomputing services These applications need to b e networkawar e
to ensure they get
appropriate service from the underlying comm unication and computing infrastructure
While net w ork routing still ensures that pac k ets tak e the b est path b et w een a giv en sourcedestination pair the
application is the one that c ho oses the comm unicatio n endp oin ts F or example a clien ts request to a replicated
information service should b e automatically directed to a closeb y ligh tly loaded serv er Being able to c ho ose
adequate serv ers requires that applications ha v e access to information ab out net w ork and serv er load W e argue
that a service that pro vides kno wledge ab out the dynamics of the underlying comm unication and computing
infrastructure is essen tial for networkawar e applications
This pap er ev aluates the p erformance of top olo gyd a service that estimates the state of net w ork ed resources
b y p erio dically computing the endtoend latency and a v ailable bandwidth among them Since these are endto
end measuremen ts they capture b oth net w ork and serv er load Based on these estimates top ologyd computes
a fault toleran t high bandwidth lo wdela y top ology connecting participating sites T op ologies are p erio dically
recomputed to tak ein to accountnet w ork and serv er load dynamics
W e b orro w ed the term networkawar e from
In the simplest case top ologyds estimates can b e logged for future assessmen ts of the net w ork status Al
ternativ ely top ologyds data can b e either directly consumed b y applications or it can b e p erio dically published
in a directory service for consultation byin terested thirdparties The latter approac h is the one adopted in
the Globus metacomputing infrastructure whic h curren tly publishes top ologyds estimates in its directory
service The data is then in terpreted b y a resource brok er whichuses ittomak e informed decisions ab out the
b est utilization of the a v ailable computational resources W egiv e more details ab out top ologyds in tegration in to
Globus in Section Other p oten tial clien ts of our to ol include replicated databases and distributed information
dissemination services suc h as the W eb and In ternet arc hiv es W ean ticipate that other applications can also
b enet from top ologyds services In ternet cac he hierarc hies suc h as Harv est and Squid can also use
top ologyds net w ork and serv er load information to congure their cac he serv ers T op ologyds logical top ologies
can b e tailored to t the requiremen ts of dieren t distributed applications In the case of replicated information
services top ologyd generates faulttoleran t top ologies that try to minim ize b oth up date propagation cost and
time Logical top ologies for metacomputing systems can simply connect participating sites with a presp ecied
n um b er of closeb yw ellconnected ligh tlyloaded computation resources
In this pap er w e presen t results of In ternetwide exp erimen ts w e conducted to ev aluate the p erformance of
top ologyd The goal of these exp erimen ts w as to v alidate the estimates top ologyd collects and the top ologies
it computes W e describ e our exp erimen ts and results in Sections and Section presen ts an o v erview of
top ologyd More details on top ologyds original motiv ation and design can b e found in whic h also presen ts
preliminary p erformance ev aluation results on a lo calarea net w ork
Related W ork
This section reviews related w ork in the area of In ternet measuremen t It also discusses sev eral net w orka w are
applications and ho w they can mak e use of a service lik e top ologyd As an example w e explain ho w the Globus
metacomputing system is curren tly using top ologyds estimates
In ternet measurem en t eorts
W e start b y briey reviewing some of the eorts in the In ternet measuremen t arena The topic of In ternet
Statistics and Metrics Analysis ISMA has b een so activ e that it deserv ed its o wn w orkshops for the last t w o
y ears The topic is also addressed b y the IP Pro vider Metrics IPPM subgroup of the IETFs Benc hMarking
W orking Group BMW GIPPMs main goalistopro vide and standardize metrics and metho dologies for the
measurementofIn ternet p erformance particularly IP clouds whic h are essen tial for the connectivityof the
In ternet CAID A Co op erativ e Asso ciation for In ternet Data Analysis publishes on the W eb a v ery
thorough MeasurementT ool T axonom y whichsurv eys measuremen t to ols curren tly a v ailable b oth as free and
commercial soft w are
One suc h to ol TReno has b ecome a go o d candidate for IPPMendorsemen t TReno aims at accurately
measuring the bulk transfer capacityof net w ork links b y implemen ting its o wn stateoftheart TCP algorithm
whic h mak es use of SelectiveAc kno wledgemen ts SA CK One of TRenos features is that it measures the through
put of a giv en link indep enden t of the particular TCP implemen tatio n on the end hosts T op ologyd on the other
hand helps end users to kno w what throughput they can actually get using their particular TCP a v or since this
is the p erformance p erceiv ed b y applications F or this reason w e decided to use netperf as the throughput
measuremen t to ol against whichw e compare top ologyds bandwidth estimates to ev aluate their accuracyW e
tried to use TReno to get an upp er b ound for the throughput that can b e exp ected out of a particular net w ork
link but TReno requires ro ot access for installation
W epresen t more details on netperf when w e explain our
exp erimen tal metho dology in Section V an Jacobson the author of traceroute recen tly released pathchar a to ol whic h estimates bandwidth
dela ya v erage queue and loss rate of ev ery hop b et w een a giv en sourcedestination pair of In ternet hosts Pathchar
uses the ICMP proto cols Time Exceeded resp onse to pac k ets whose TTL has expired It is a sophisticated to ol
whichw e unfortunately could not use for the same reason as ab o v e it requires ro ot access for installation
it w as dicult enough to get regular accoun ts on the mac hines w e used in our exp erimen t ro ot priviledges w ere simply imp ossible
to get

A dieren t approac h to measuring net w ork bandwidth is tak en b y bprobe and cprobe These to ols prob e the
net w ork b y sending sev eral pairs bprobe or a short train of pac k ets cprobe They estimate net w ork bandwidth
b y measuring the in terarriv al time b et w een the pac k ets The goal of bprobe is to measure the b ottlenec k
bandwidth along a net w ork path whereas cprobe attempts to estimate the eectiv e bandwidth a v ailable to an
application in the presence of comp eting net w ork trac One adv an tage of these to ols is that they are net w ork
friendly in the sense that they do not load the net w ork with their o wn trac although bprobe do es ha vea
tendency to send pac k ets of sizes up to b ytes for v ery short p erio ds of time Unfortunately at the time w e
conducted our exp erimen ts the to ols w ere only a v ailable for SGI mac hines and based themselv es on v ery accurate
timer mec hanisms a v ailable on this platform
While our to ol pro vides its o wn estimates of net w ork bandwidth and latency it could just as w ell use the
estimates from an IPPMappro v ed to ol whic h for the momen t do es not exist There are curren tly sev eral
applications whic h could b enet from not necessarily v ery accurate estimates of net w ork bandwidth and latency Therefore w een vision our to ol as a middlew are service that pro vides information ab out net w ork and serv er
load to applications whic h dep end on this kind of kno wledge for adequate p erformance Before lo oking at some
examples of suc h applications w e will discuss t w o other existing middlew are services similar in certain asp ects to
our to ol
The Net w ork W eather Service NWS forecasts the endtoend throughput and latency attainable b y
TCPIPbased applications NWS collects data using netperf and then applies to the data a set of forecasting
metho ds including meanbased medianbased and autoregressiv e mo dels NWS then dynamically selects the
b est forecast according to a sp ecied metric whic h can b e based on measuring either the mean square prediction
error or the mean p ercen tage prediction error One dra wbac k of NWS is that it is not faulttoleran t in the sense
that it do es not adjust to failures of the individual mac hines it monitors and it only functions w ell when all the
monitored mac hines are up and running Unlik e NWS top ologyd w as designed to adjust to group mem b ership
dynamics
Due to the explosion of the W orld Wide W eb there is an increasing in terest in measuring p erformance asp ects
of the In ternet sp ecically related to the WWW Suc hW ebbased endtoend measuremen ts seek to determine the
resp onsiv eness of the net w ork bysim ulating user W ebbro wsing sessions F or example Lac hesis w as designed
to assess the p erformance of In ternet Service Pro viders ISPs It b enc hmarks a particular ISP b y taking a list of
socalled landmarks imp ortan t sites across the In ternet including DNS ro ot serv ers w ellkno wn FTP serv ers
and p opular WWW serv ers and measuring the pac k et loss and net w ork latency to eac h landmark Net w ork
throughput is not of concern for Lac hesis since for bro wsing the W eb net w ork delayis m uc h more critical than
throughput Lac hesis is implemen ted as a P erl script around Stanford Univ ersit ys fping program whic hin
its turn uses ICMP Ec hos and ICMP Ec ho Replies to measure net w ork latency Applications
The next paragraphs will discuss some of the applications whic h can b enet from a service lik e the one oered
b y our to ol since go o d measuremen ts and estimates of the net w ork p erformance are essen tial to their prop er
functioning
One imp ortantclien t of our to ol is the resource brok er used in the Globus metacomputing infrastructure Globus is a largescale pro ject whic h aims to enable the deplo ymen t of highp erformance computingin tensiv e
distributed applications b y taking adv an tage of resources suc h as highsp eed net w orks and sup ercomputers The
Globus infrastructure presen ts to its users an uniform in terface to net w ork ed virtual sup ercomputers or meta
computing abstract mac hines Tw oessen tial pieces in the Globus system are the Metacomputing Directory
Service MDS and the resource brok er MDS is a p erv asiv e rep ository of information ab out the curren t state
of the resources information whic hisused b y the resource brok er in order to meet the resource requiremen ts of the
users Our to ol ts in this framew ork as an information service whic h publishes its data eg the logical top ology
together with the bandwidth and R TT estimates in to the MDS and enables the brok er to tak e decisions based
on this information W e should men tion that b oth the in tegration of top ologyd within the complex installation
pro cedure of the Globus soft w are system and the publishing of our to ols data in MDS ha v e b een accomplished
Deplo ymentat n umerous test sites will also start v ery so on
The adv entof Ja va caused a proliferation of clien tside applets used for an extremely large v arietyof
purp oses The smart clien ts of Y oshik a w a et al part of the Berk eley NO W Net w ork Of W orkstations

pro ject are in fact t woJa v a applets whic h tak e a clien tside approac hto pro viding transparen t access to m ulti
serv er In ternet services suchasHTTPand FTP The most in teresting of these applets and the one relev antto
our pap er is the director whic hc ho oses a b est mac hine from a p o ol of serv ers pro viding a particular service
requested b y the clien t The c hoice is transparen t to the user and is based on servicesp ecic information suc has
mac hine load a v ailable net w ork bandwidth net w ork latency or pro cessor sp eed The issue of ho w exactly this
b est serv er is c hosen is someho w skipp ed o v er b y the authors who only men tion the p ossibilit y of emplo ying a
service suc h as the MIDS In ternet W eather Rep ort without actually using it It is in this con text that w e feel
out to ol could b e used with success bypro viding the director all the information it needs including the logical
top ology in order to mak e its c hoice
Ranganathan et al dev elop ed their net w orka w are mobile programs whic h are Ja v a applets that can
adapt to v ariations in net w ork c haracteristics Their mobile In ternet c hat application uses the information ab out
the net w ork conditions to dynamically place the c hat serv er in order to minim ize the resp onse time for the end
users The w a y they monitor the net w ork is b y using Komo do a to ol whic h estimates latency b et w een pairs of
hosts b y sending b yte UDP pac k ets from one host to the other and bac k The authors rep ort some in teresting
observ ations on v ariations in In ternet latency T op ologyd
T op ologyd w as implemen ted to b e fast and robust Consequen tly it is written as a Unix daemon that do es
not fork but instead uses nonblo c king IO for all comm unication The daemon can b e queried in t wow a ys
from a W eb bro wser or via a commandl ine utilit y called tdclient The HTTP in terface is more suitable for
in teractiv e querying sessions a user can see the curren t estimates presen ted in a con v enien t tabular w a yas w ell
as the curren t logical top ology of the group F or example to connect to the master of the group w e are curren tly
runnning go to httpexcaliburu sced u W e are curren tly displa ying the top ology b y sho wing the
domain names of the b est neigh b ors for eachmac hine in the group W eplantoimpro v e the user in terface b y
dra wing the top ology graph using the no des geographical lo cations
The commandline in terface is b etter suited for batc hmo de queries W e wrote a series of P erl scripts as
wrapp ers around this utilityand weha v e used them for all the statistics gathering and parsing that w e needed
T o address scalabilit y and administrativ e decen tralization top ologyd sites can b e organized in m ultiple groups
F or instance a replica of an information service need only store consistency state asso ciated with replicas in its
group instead of the en tire set of service replicas F urthermore allo wing m ultiple groups preserv es autonom y since administrativ e decisions of one group do not aect other groups
T op ologyd allo ws a single mac hine to b e mem ber of m ultiple groups Ov erlapping groups allo w up dates in
one group to mak e their w a y to other groups A designated gr oup master collects the latency and bandwidth
estimates from the other group mem b ers and is resp onsible for the computation and the dissemination of the
logical top ology Estimate Collection and T op ology Generation
Eac h mac hine running top ologyd p erio dically computes a v ailable bandwidth and R TT b et w een itself and all the
other mac hines in the group F or R TT estimates a timestamp ed UDP pac k et is sen t to another mem ber of the
group whic h simply returns it bac k to the originator F or bandwidth estimates eac hmac hine sends a blo ckof
data using TCP the default for the blo c k size is KB Av ailable bandwidth is then computed as
bandw idth by tes sent time
last by te
time
first by te
where the t w o timestamps are tak en b y the destination mac hine When computing the actual estimates to b e
rep orted to the master previous history is tak en in to accoun t in order to a v oid transientc hanges This damping
eect is obtained b y computing
new estimate ol d estimate cur r ent estimate
W e curren tly set to It is imp ortan t to note that top ologyds bandwidth and R TT estimates takein to
accoun t mac hine load While a to ol lik e TReno is more accurate for transp ortla y er measuremen ts top ologyd
tries to measure the actual dela y and bandwidth seen b y an application

P arameter V alue Description
mastersite excaliburuscedu poin ts to group master
pingperiod min latency estimation p erio d
bandwidthperio d min bandwidth estimation p erio d
updateperiod min top ology up date p erio d
estimatesperio d min estimate rep ort p erio d
T able Group parameters and their v alues
The master collects the estimates rep orted b y group mem bers in to a cost matrix for the group whic h the master
uses to compute the groups logical top ology Eac hen try C
ij
in the cost matrix corresp onds to the comm unication
cost b et w een mac hines i and j and is giv en b y B
ij
D
ij
where B
ij
and D
ij
are the estimated bandwidth and
R TT b et w een no des i and j resp ectiv elyT op ologyd then generates the logical top ology b yin v oking a top ology
generator program whic h uses as input the cost matrix and a connectivit y requiremen t k for eac h no de curren tly
weha v e k The top ology generator rst computes a minim um cost spanning tree connecting all the no des
and then for eac h no de whose degree d is less than the required connectivit y k adds the curren tc heap est edge
un til d k The master p erio dically sends the curren t logical top ology to all the mac hines in the group
Group mem be rship
When a mac hine joins a group it sends a join request to the master whic h adds it to the list of kno wn sites
Ho w ev er the mac hine is not ocially part of the group un til the master distributes a new top ology that con tains
the site The rst top ology generated b y the master after a new mem b er joins the group will not b e v ery go o d since the new mac hine has not had time to p erform bandwidth and R TT estimations to the other group mem b ers
Ho w ev er the top ology gets b etter as the estimates impro v e
There is no proto col for lea ving the group Mac hines lea v e a group silen tly and if the master has not heard
from a mem b er after a predetermined p erio d of time it drops the mac hine from the group mem b ership The
silence p erio d is congurable and is curren tly set to hour
T op ologyd w as designed to b e faulttoleran t with resp ect to group mem b ership c hanges A group con tin ues
to function normally as mac hines lea v e or join Ev en if the master is temp orarily disconnected from the rest of
the group the mem b ers will con tin ue to compute the estimates without receiving top ology up dates W e should
p oin t out that in its curren t implemen tation top ologyd do es not include a master bac kup mec hanism in case
the group master b ecomes una v ailable Ho w ev er all top ologyd sites ha v e the same functionalit y and it will b e
quite straigh tforw ard to implemen t a master selection algorithm
Installing top ologyd
T op ologyds installation script is written in P erl and is highly cogurable T op ologyds distribution including
source co de installation script and do cumen tation is publically a v ailable from The to ol w as originally
dev elop ed for SunOS platforms but is also running on Solaris HPUX and SGI Irix W e are curren tly p orting
it to other Unix platforms suc h as IBM AIX
T able describ es the congurable group parameters and sho ws their v alues for the exp erimen ts The master
site parameter p oin ts to the master of the group The ping period and bandwidth period parameters deter
mine ho w often sites estimate latency and bandwidth The update period and estimates period determine
ho w often the logical top ology is up dated and ho w frequest sites rep ort their estimates to the group
Measuremen t strategy
T o conclude this section w e will put our measuremen t strategy in the con text of the framew ork suggested b y
V ern P axson W e should men tion that the IPPM has recen tly pro duced an In ternet Draft whic hdra ws
hea vily from P axsons framew ork

Our metho dology is active in the sense that w e generate new trac as part of the measuremen t pro cess
Examples of other to ols emplo ying an activ e strategy are TReno and pathchar Our metho dology also requires
co op eration among participating hosts who rep ort their estimates so that the groups top ology can b e computed
Co op eration among top ologyd sites is har d in P axsons terms since hosts at b oth ends of a net w ork link m ust
in teract with eac h other as opp osed to soft co op eration whic h is the case of traceroute or pathc har where only the
source host and the in terv ening routers are in v olv ed in pro cessing ICMP messages Our form of hard co op eration
is fault toleran t and suitable for asymmetrical link en vironmen ts lik e the In ternet Finallyw e argue that mac hines running our to ol can b e seen as the co op erating me asur ement platforms en vis
aged in P axsons framew ork as an essen tial part of a necessary measuremen t infrastructure for the p erformance
of IP clouds and bac kb one links
Exp erimen ts
In this section w e presen t the metho dology used in our p erformance study The goal of the In ternetwide exp eri
men ts w e conducted is to v alidate top ologyds bandwidth and latency estimates as w ell as the resulting logical
top ologies
Exp erim en tal T estb ed
W e installed top ologyd at sites scattered throughout the w orldwide In ternet including lo cations in the
US in Europ e in South America in Asia and the South P acic W e congured all participating sites as
mem b ers of a single top ologyd group whose master w as excaliburuscedu T able lists the sites participating in this study and their lo cation Conducting suc h a large scale widely
distributed exp erimen t required considerable eort Getting exp erimen tal accoun ts on all the mac hines
pro v ed
to b e one of the hardest and most time consuming tasks T op ologyds robustness pro v ed to b e in v aluable in suc h
an autonomously decen tralized en vironmen t since it w as quite common that a mac hine got reb o oted or crashed
during the exp erimen ts Other than missing data for the una v ailable sites the exp erimen ts pro ceeded normally These temp orary site failures demonstrated top ologyds abilit y to adjusttomem b ership c hanges caused bya
site v olun tarily lea ving the group and joining in later and b y temp orary failures
As part of the exp erimen t monitoring task w e needed to restart top ologyd whenev er a mac hine crashed and
got reb o oted In pro duction en vironmen ts top ologyd should b e part of the system administration pro cedures
and should b e automatically started at b o ot time or bya cron job
Data Collection
W e sampled top ologyds bandwidth latency and top ology information ev ery hour o v er a p erio d of t w o and a
half mon ths starting b eginning of June and ending in mid Septem ber T ov alidate top ologyds estimates
w e compared them against measuremen ts obtained from running ping and netperf on a subset of the mac hines
in the group W e tried to k eep to a minim um the com binatorial explosion of the N
net w ork paths w e had
a v ailable so w e restricted our ping and netperf measuremen ts to four sites c hosen based on features suchas
their geographic lo cation ho ww ell they w ere connected and ho w stable in terms of uptime they w ere T able lists the four data collection sites W e also limited the data collection p erio d to one w eek during these sev en
da ys w e ran ping and netperf ev ery hour while running top ologyd
Netperf isabenc hmarking to ol that can b e used to measure bulk data transfer p erformance in b oth TCP
and UDP mo des it estimates the a v ailable net w ork throughput b y sending pac k ets of congurable length o v er
a congurable p erio d of time seconds b y default This bandwidth measuring tec hnique is more accurate
than the one used b y top ologyd whic h consists in sending only one TCP pac k et and timing it The dierence
bet w een the t w o metho ds explains wh y netperf rep orts consisten tly higher throughput v alues than top ologyd
The do wnside of netperfs approac h is that it can signican tly load the net w ork with its o wn measuremen t trac
In the case of net w orka w are services whic h do not require extremely accurate measuremen ts w e argue that it is
preferable to use a to ol that generates less net w ork trac
T op ologyd do es not require an y priviledges to install or run so regular user accoun ts suced

Site Lo cation
excaliburuscedu Los Angeles CA USA
sycisiedu Marina del Rey CA USA
infernoin ternetcafecom San ta Barbara CA USA
redondoeceuciedu Irvine CA USA
trekcsorstedu Corv allis OR USA
sahircsumassedu Amherst MA USA
pharosbuedu Boston MA USA
metroisiedu W ashington DC USA
shiv ameecsumic hedu Ann Arb or MI USA
niik aanfdlccm n us Clo quet MN USA
ashok acsumnedu Minneap olis MN USA
bassdcsuky edu Lexington KY USA
marmotcsindi ana edu Blo omington IN USA
pandoracsutsaedu San An tonio TX USA
cosmomcsanl go v Argonne IL USA
shado wcscolum biaedu New Y ork NY USA
exo douprcluedu Rio P edras Puerto Rico USA
itiamathuc hg r Heraklion Greece Europ e
shrewcsuclacuk London UK Europ e
camp erdcuie Dublin Ireland Europ e
b ebacesnetcz Czec h Republic Europ e
nep om uksnic h Switzerland Europ e
garocn ucecnrit Pisa Italy Europ e
fram b o esacetucpucriobr Rio de Janeiro Brazil South America
mirageirdun ussg Singap ore Asia
cosmosk aistackr Seoul South Korea Asia
blueb ottleitssauc klandacnz Auc kland New Zealand
T able P articipating sites
Site Lo cation
cosmomcsanl go v Argonne IL USA
redondoeceuciedu Irvine CA USA
mirageirdun ussg Singap ore Asia
itiamath uc hgr Heraklion Greece Europ e
T able Data collection sites

The fact that our measuremen ts w ere conducted with the same p erio dicit y at all the four data collection sites
ma y lead to sync hronization problems Ho w ev er the data collection sites w ere running top ologyd con tin uously
and w e to ok snapshots of their curren t state ev ery hour T o compare these snapshots against ping and netperf
measuremen ts w e ran them concurren tly with taking top ologyd snapshots Ev en though they ran concurren tly their results w ere spaced in time due to their dieren t execution times In the next set of exp erimen ts w eplan
to sample the data using exp onen tially distributed in terv als eg random arriv als with P oisson distribution as
prop osed b y The output of ping and netperf w as used to v alidate top ologyds latency and bandwidth resp ectiv ely Graphs
sho wing these results are sho wn in Section b elo w W e also ran traceroute at three of the data collection sites
redondoeceucie du cosmomcsanlgov and mirageirdunus sg Traceroute information is useful in
diagnosing transien t problems as w ell as pro viding insighttov alidate the top ologies generated b y top ologyd
Data Analysis
W e automated the pro cess of data collection b y writing P erl scripts wrapp ers around top ologyds tdclient ping and netperf The top ologyd data collection script consists of the follo wing steps It rst queries the master
to get the curren t group mem b ership and the curren t top ology then it queries eac h site from the list returned
b y the master and obtains the estimates to all the other sites A series of P erl scripts then parsed the log les
to obtain summaries and data p oin ts T o plot the data w eusedthe webgraph utilitydev elop ed at In tel and
bundled in the Imeter soft w are distribution Webgraph is a P erl script that op erates on set of datap oin ts It
uses gnuplot and ppmtogif to generate a graph in P ostScript and GIF formats and then app ends the graph to
an HTML do cumen t
T o generate statistics from our data p oin ts w e used the robostats pac k age Robostats tak es in a le
con taining the data p oin ts and generates t w o dieren t sets of statistics One set lists the minim um maxim um mean and standard deviation for the sample p opulation and the other set lists the th and th p ercen tiles the
median and the in terquartile range IQR ie the th p ercen tile min us the th p ercen tile for the same sample
p opulation
It has b een sho wn that the data p opulation obtained b y doing net w ork measuremen ts o v er the In ternet
do es not presen t a normal or Gaussian distribution Hence sho wing the mean and standard deviation are not
meaningful statistics for In ternet measuremen ts F ollo wing the recommendations of the IPPM comm unit yw e
then c hose to presen t the th and th p ercen tiles together with the median and the IQR to summarize our
measuremen ts
Results
The strategy w e used w as to v erify top ologyds latency and bandwidth estimates b y comparing them to mea
suremen ts obtained using other w ellkno wn to ols W e used pings and netperfs measuremen ts of latency and
bandwiidth resp ectiv elyOnce w ev alidate top ologyds estimates of net w ork and serv er load w e use them to
v alidate the logical top ologies top ologyd generates
Latency and Bandwidth
Our results are presen ted in the form of graphs sho wing top ologyd latency in miliseconds and bandwidth in
kilobitssec estimate samples tak en b et w een a giv en pair of hosts o v er time in hours Latency graphs also sho w
the corresp onding ping measuremen ts while bandwidth graphs presen t netperf data
Figure sho w latency R TT and bandwidth BW measuremen ts tak en from redondoeceuciedu to the
three other mac hines in T able W e use redondoeceuci edu s estimates as example measuremen ts tak en
from the other data collection sites presen t similar b eha vior The statistics in Figure summarize measuremen ts
from all data collection mac hines
Figure a sho ws that top ologyds
and pings roundtrip time measuremen ts to cosmomcsanlgov are
Due to c hanges in a data collection sites view of group mem b ershi p top ologyd sometimes rep orts no data for a giv en mac hine
This explains the few o ccurrences of zeros in top ologyds latency and bandwidth estimates

0
1000
2000
3000
4000
5000
6000
7000
8000
9000
0 20 40 60 80 100 120 140 160
RTT (ms)
Timeline (hours)
RTT estimates from redondo to cosmo
"ping.data"
"topology-d.data"
a R TT to cosmomcs an lg ov
0
200
400
600
800
1000
1200
1400
1600
1800
2000
0 10 20 30 40 50 60 70 80 90 100
RTT (ms)
Timeline (hours)
RTT estimates from redondo to itia
"ping.data"
"topology-d.data"
b RTT to itiamcsa nl gov
0
500
1000
1500
2000
2500
0 20 40 60 80 100 120 140 160
RTT (ms)
Timeline (hours)
RTT estimates from redondo to mirage
"ping.data"
"topology-d.data"
c RTT to mirageir du nus s g
0
50
100
150
200
250
300
350
400
0 20 40 60 80 100 120 140 160
Bandwidth (kbit/s)
Timeline (hours)
Bandwidth estimates from redondo to cosmo
"netperf.data"
"topology-d.data"
d BW to cosmomcsa nl gov
0
50
100
150
200
250
0 10 20 30 40 50 60 70 80 90
Bandwidth (kbit/s)
Timeline (hours)
Bandwidth estimates from redondo to itia
"netperf.data"
"topology-d.data"
e BW to itiamcs anl go v
0
20
40
60
80
100
120
140
160
180
200
0 20 40 60 80 100 120 140 160
Bandwidth (kbit/s)
Timeline (hours)
Bandwidth estimates from redondo to mirage
"netperf.data"
"topology-d.data"
f BW to mirageirdu nu ss g
Figure Latency and bandwidth measuremen ts tak en from redondoeceuci edu
v ery similar T op ologyds abilit y to tak e previous history in to accoun t damp ens the eects of transien ts whic h
w ere captured b y pingF or farther hosts connected through b ottlenec k connections suc h as transo ceanic links
b oth top ologyd and ping rep ort higher v ariabilit y in roundtrip time This is the case for itiamathuchgr
and mirageirdunus sg Figures b and c W e notice though that the R TT v aries m uc h more wildly in
the case of the link to mirageirdunuss gT o conrm this v ariabilit yw e insp ected the traceroute logs w e
collected and w e found indeed that there w ere n umerous problems with this net w ork path and m ultiple o ccurrences
of unreac habilit y Generally top ologyds and pings measuremen ts compare quite w ell whic h should b e of no
surprise since they b oth use small UDP pac k ets to estimate the roundtrip time This similarit y is corrob orated
with the statistics presen ted in Figure a
Sites cosmomcsanlgo v itiamathuc hgr mirageirdun ussg redondoeceuciedu
cosmomcsanlgo v NA NA itiamathuc hgr NA NA e mirageirdun ussg e NA e NA redondoeceuciedu NA
e NA
a Latency statistics
Sites cosmomcsanlgo v itiamathuc hgr mirageirdun ussg redondoeceuciedu
cosmomcsanlgo v NA NA itiamathuc hgr NA NA mirageirdun ussg NA e NA redondoeceuciedu NA
NA
b Bandwidth statistics
Figure Statistics summarizing latency and bandwidth measuremen ts Ro ws and columns list source and des
tination mac hines Eac h cell sho ws the th p ercen tilemedianIQRth p ercen tile for our sample p opulation
The n um b ers on the top line refer to top ologyds estimates while the n um b ers on the b ottom line refer to ping
estimates in the case of T able a and to netperf estimates in the case of T able b
T op ologyds and netperfs bandwidth measuremen ts tak en from redondoeceucied u are presen ted in
Figure b and summarized b y the statistics presen ted in Figure b They sho w that although the magnitudes
of the measuremen ts tak en byeac h to ol dier considerably the shap es of the curv es are similar This means that
b oth top ologyd and netperf capture similar trends in bandwidth v ariabilityo v er time although top ologyds
use of damping prev en ts it from capturing transien ts As p oin ted out in Section the discrepancies b et w een
topologyd s and netperfs bandwidth measuremen ts are due to the dieren t metho ds used bythe t w o to ols to
estimate the a v ailable bandwidth Lik einthe R TT measuremen ts w e notice the high v ariabilit y of load in the
transo ceanic link to mirageirdunussg as opp osed to the relativ e stabilit y of the transcon tinen tal path to
cosmomcsanlgov W e also notice from the graphs that there is no clear pattern in net w ork and serv er usage o v er time with
the p ossible exception of the bandwidth graph from redondoeceucie du to mirageirdunuss g This graph

presen ts a certain p erio dicit y with p eaks and slop es roughly ev ery hours
This pattern w as consisten t across
sev eral sets of bandwidth measuremen ts that wetook bet w een these t w o sites and it can also b e observ ed on
the rev erse path from mirageirdunuss g to redondoeceuciedu Ho w ev er the pattern is not apparen t for
the link b et w een cosmomcsanlgo v and mirageirdunussg whic h prompts us to sp eculate that this is an
isolated c haracteristic of the in tercon tinen tal link b et w een the US and Asia F or all the other graphs p eak
and op eak times are indistinguishable due to the fact that our exp erimen ts included sites that span v arious
time zones This curren t trend in net w ork and serv er usage our results captured is driv en b y globally distributed
applications suc h as the W orld Wide W eb
One other observ ation concerns the asymmetry of net w ork paths The fact that net w ork links are generally
asymmetric has b een detected byP axson in and is conrmed b y our statistics
T op ologies
Logical top ologies are recomputed p erio dically and reect c hanges in net w ork and serv er load F or our exp eri
men ts w e congured top ologyd to recompute the groups top ology ev ery hour W e sampled the groups top ology
ev ery hour
There are three observ ations wew anttomak e regarding top ologyds top ologies The rst one concerns the
close correlation b et w een bandwidth and latency estimates and the observ ed top ologies The second observ ation
relates to the top ologys adaptabilit yto c hanges in load captured b y bandwidth and latency estimates as w ell
as c hanges in group mem b ership Finally w e try to quan tify the stabilit y of the top ologies o v er time These
observ ations are discussed in more detail b elo w
Estimates and Logical T op ologies
Wev erify that the logical top ologies the master computes for the group reect indeed the estimates collected b y
the group mem bers F or eac h host w e calculate the median for the R TT and bandwidth estimates tak en from
that mac hine to all the other group mem b ers W e order the resulting list in decreasing order based on the ratio
bandw idthRT T whic h represen ts the cost of a giv en link Also for eac hhostwecounthowman y times an y
giv en mem b er of the group w as seen among the hosts b est neigh b ors and w e order the resulting list in decreasing
order T ables and sho w the top mac hines in these t w o lists for redondoeceuciedu o v er the w eek w e
collected top ologyd ping and netperf data As exp ected there is a strong correlation b et w een the t w o lists
with the same mac hines dominating b oth lists This close correlation b et w een logical top ologies and latency and
bandwidth estimates is observ ed for all the other mac hines in the group
W e notice that the order of the v e b est mac hines according to redondos estimates is the same as the order
in whic h they o ccur in the list of b est neigh b ors with one exception sycisiedu app ears more times in the list of
neigh b ors than excaliburusced u It can also b e observ ed that t w o other mac hines infernointernet cafe com
and mirageirdunussg app ear in the top of the list of neigh b ors although they are not among the top
mac hines according to the estimates This is due to the fact that the curren t top ology computation algo
rithm do es not accoun t for link asymmetrySo if infernointernetcaf eco m and mirageirdunuss g see
redondoeceucie du among their b est neigh b ors then they will app ear in the list of redondos neigh b ors This
sometimes can lead to situations where the top ology resem bles a star ha ving in its cen ter a p o w erful andor
w ell connected mac hine refer to Section for an example W e plan to mo dify the top ology computation
algorithm to takein to accoun t asymetric links W e will also sp ecify a maxim um no de degree
Adaptabili t y
Figure sho ws the star eect men tioned in the previous section Host ashokacsumnedu is connected to
six other hosts Note that the degree parameter the top ology computation algorithm uses sp ecies the minim um
no de degree W e plan to include a maxim um no de degree to limit the n um b er of neigh b ors assigned to a host
and therefore eliminate star top ologies
One asp ect of top ology adaptabilit y concerns c hanges in group mem b ership Figures and sho wt w o
snapshots tak en one hour apart during our exp erimen ts W e observea v ery clear transition When the host
This p erio dicit y is more noticeable for netperfs measuremen ts than for top ologyd s due to the latters use of damping

metro.isi.edu
marmot.cs.indiana.edu
nepomuk.sni.ch
redondo.ece.uci.edu inferno.internet−cafe.com
cosmo.mcs.anl.gov
mirage.irdu.nus.sg
figaro.cnuce.cnr.it
syc.isi.edu
excalibur.usc.edu
trek.cs.orst.edu bass.dcs.uky.edu beba.cesnet.cz
ashoka.cs.umn.edu
niikaan.fdl.cc.mn.us
camper.dcu.ie
Figure Adaptabilit yto c hanges in group mem b ership T op ology snapshot tak en on Sept at PM
P acic Time

marmot.cs.indiana.edu
metro.isi.edu
nepomuk.sni.ch
mirage.irdu.nus.sg beba.cesnet.cz
redondo.ece.uci.edu
inferno.internet−cafe.com
figaro.cnuce.cnr.it bass.dcs.uky.edu
cosmo.mcs.anl.gov
syc.isi.edu
excalibur.usc.edu
trek.cs.orst.edu
niikaan.fdl.cc.mn.us
camper.dcu.ie
Figure Adaptabilit yto c hanges in group mem b ership T op ology snapshot tak en on Sept at PM
P acic Time

Site Median R TT Median BW BWR T
excaliburuscedu ms Kbs
sycisiedu ms Kbs trekcsorstedu ms Kbs ashok acsumnedu ms Kbs niik aanfdlccm n us ms Kbs pharosbuedu ms Kbs T able Estimates for redondoeceuciedu
Site Num b er of o ccurences
sycisiedu infernoin ternetcafecom excaliburuscedu
trekcsorstedu
ashok acsumnedu
mirageirdun ussg
niik aanfdl ccm nus
pharosbuedu
T able T op ology neigh b ors for redondoeceuciedu
ashokacsumnedu w entdo wn the group recongured itself quic kly This top ology c hange illustrates top ology
ds faulttolerance to v olun tary or in v olun tary mem b ership c hanges whic h is extremely v aluable in a distributed
and unpredictible en vironmen t suc h as the In ternet
W e also observ e that top ologies are quite resp onsiveto c hanges in latency and bandwidth estimates
T op ology
snapshots tend to b e v ery similar except for a few links This is a trend w e observ ed for the en tire exp erimen t
It is v ery uncommon to nd t woiden tical top ologies but c hanges from one top ology to the next are v ery small
and generally limited to t w o or three mac hines
Stabili t y
It is not easy to quan tify the stabilit y of logical top ologies As w emen tioned b efore the o v erall graph connecting
the sites w as c hanging con tin uously but in general the c hanges w ere small More noticeable recongurations tak e
place only when a mac hine joins or lea v es the group
W e try to estimate the top ology stabilit y from a sites p oin tofview W e list all the mac hines whichw ere
seen as neigh b ors b y a giv en host in decreasing order of n um b er of o ccurences W e observ e that sites with more
po w erful CPUs andor b etter net w ork connectivit y tend to ha v e larger sets of neigh bors and th us a higher
degree in the graph As previously men tioned this is also due to the fact that top ologyd do es not accoun t for
asymmetric links Mac hines with p o orer connectivit ysuc h as the ones in Europ e and Asia tended to ha vea
m uc h smaller p o ol of neigh bors T o exemplifyw eshowt w o distribution plots for hosts represen ting these t w o
t yp es of mac hines ashokacsumned u in Minnesota USA with v ery go o d connectivit y and a large p o ol of
neigh b ors and itiamathuchgr in Heraklion Greece with a small set of neigh b ors In fact the corresp onding
top ology graphs
sho w that ashoka app ears to b e the ro ot of a subtree although the graph is not a tree with
six or more c hildren while itia lo oks more likealeaf In the distribution plots in Figure the rst data p oin t represen ts the n um b er of o ccurences of the top
mac hine in the ordered list of neigh b ors normalized b y the total n um b er of o ccurences of all mac hines in the list
The second data p oin t represen ts the normalized n um b er of o ccurences of the top t womac hines in the ordered
list of neigh b ors and so on The graphs can b e seen as represen ting the p ercen tage of time a giv en host sees a
Ev en though weha veplen t y of evidence due to space limitations w e do not showan y examples
These graphs are not sho wn due to space limitation

10
20
30
40
50
60
70
80
90
1 2 3 4 5 6 7 8 9 10
Normalized number of occurences
Number of neighbors
Distribution plot for ashoka.cs.umn.edu
"neighbors_pool"
a ashok acsum ne du
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10
Normalized number of occurences
Number of neighbors
Distribution plot for itia.math.uch.gr
"neighbors_pool"
b itiamathuc hg r
Figure Distribution of p o ol of neigh b ors
pool of mac hines as its neigh b ors F or example of the time itiamathuchgr sees neigh b ors among a p o ol
of mac hines while ashokacsumnedu sees neigh b ors among a mac hine pool only of the time W ec hose
these mac hines to represen t the t w o extremes of the sp ectrum but w e found that on a verage a host sees of
the time neigh b ors among a p o ol of mac hines
Wew ould liketo mak e one more remark here there is a tradeo b et w een stabilit y and adaptabilityof the
top ologies that can b e nelytuned b y giving appropriate v alues to the parameters of the group F or example
in our setup estimatesperiod w as set to min utes whic hmean t that eac h site rep orted its estimates to
the master of the group ev ery min utes This is a v alue whic h puts more emphasis on the adaptabilityof the
top ologies to the underlying net w ork conditions and its eect could b e observ ed through the fact that the logical
top ologies usually c hanged from one computation to the other If stabilit y is more imp ortan t than adaptabilit y then estimatesperiod should b e giv en a higher v alue One other parameter whic h inuences this tradeo is
updateperiod the in terv al used b y the master site to compute the top ologies whic hw as min utes in our
setup A higher v alue w ould emphasize again stabilityo v er adaptabilit y Conclusions
The goal of this pap er is t w o fold First w e argued that services lik e top ologyd whic hpro vide kno wledge
ab out the dynamics of the underlying comm unication and computing infrastructure are essen tial for the socalled
networkawar e applications
Second w eev aluated and v alidated the p erformance of top ologyd b y comparing its estimates with the ones
obtained from other w ellkno wn to ols suchas netperf and pingW esho w ed that top ologyds logical top ologies
w ere consisten t with its estimates W e also sho w ed that the top ologies adjust to c hanges in net w ork and serv er
load as w ell as in group mem b ership
AnyIn ternetwide exp erimen t is b ound to generate some in teresting and often times unexp ected conclusions
Our exp erimentw as no exception and b elo ww e list conclusions wedra w ab out the p erformance of top ologyd
as w ell as general lessons learned from conducting our In ternetwide exp erimen t
The conclusions regarding top ologyds p erformance are the follo wing
F aulttolerance is extremely imp ortan t in the constan t presence of reb o ots and sh utdo wns this w as a feature
of our to ol whic h pro v ed to b e essen tial in conducting an In ternetwide exp erimen t
T op ologyds bandwidth estimates compare w ell with netperfs Their absolute v alues dier b ecause of
their dieren t measuremen t strategies but b oth to ols capture similar trends in bandwidth v ariation o v er
time

T op ologyds R TT estimates are v ery similar to pings due to the similarit y of their measuremen t metho ds
T op ologyds damping tec hnique pro v ed to b e v ery useful in eliminating transientbeha vior that the other
to ols captured
T op ologyds logical top ologies are b oth adaptiv e and stable
W e also w anttopoin t out the lessons w e learned from conducting a widearea exp erimen tin v olving sites
scattered across four con tinen ts
Getting regular accoun ts on a signican tn umberofmac hines is v ery hard and timeconsuming
T o ols lik e TReno and pathcharwhic h require ro ot access for installation are hard to deplo y on a large
scale unless they b ecome part of standard UNIX distributions as traceroute did
The N
eect pro duced b y monitoring N sites sending data to eac h other demands automated to ols for
parsing and plotting the log data P erl pro v ed to b e in v aluable
Public domain utilities are v ery handy in pro cessing large amoun ts of data eg webgraph robostats daVinci References
T BernersLee R Cailliau A Luotonen HF Nielsen and A Secret The W orldWide W eb Communic a
tions of the A CM ! August CAID A Caida measuremen t to ols taxonom yAv ailable from h ttpwwwcaidaorgT oo lstaxonom y h tm l R Carter and M Cro v ella Dynamic serv er selection using bandwidth probing in widearea net w orks Boston
Univ ersit y Computer Science Dept TR Marc h A Chankh un tho d P Danzig C Neerdaels M Sc h w artz and K W orrell A hierarc hical in
ternet ob ject cac he Pr o c e e dings of the USENIX Confer enc e pages ! Jan uary h ttpcatarinauscedudanzigcac heps
S Fitzgerald et al A directory service for conguring highp erformance distributed computations Pr o
c e e dings of the th IEEE International Symp osium on High Performanc e Distribute d Computing August
VP axson et al F ramew ork
for ip p erformance metrics In ternet Draft a v ailable at ftpftpadv ancedorgpubIPPMframew ork txt
expiration date Jan uary I F oster and C Kesselman Globus A metacomputing infrastructure to olkit International Journal of
Sup er c omputer Applic ation to app ear Av ailable at Globus w eb page h ttpwwwglobusorg
J Gosling and H McGilton The ja v a language en vironmen ta white pap er Av ailable from
h ttpwwwpascomauj a v a do c
Matrix Information and Directory Services MIDS In ternet w eather rep ort Av ailable from
h ttpwwwmidsorg w eather
ISMA In ternet statistics and metrics analysis Av ailable from h ttpwwwnlarnetISMA
V Jacobson P athc har A to ol to infer c haracteristics of in ternet paths Av ailable from
ftpftpeelblgo vpathc har April R Jones netp erf Av ailable from h ttpwwwcuphpcomnetp erfNetp erfP ageh tml

JSeda y ao Imeter Av ailable from ftpftpin telcom pubi etfippm Im eter targz
D De Lucia top ologyd Av ailable from h ttpwwwisiedup eoplek atiatop ol ogyd targ z
M Mathis and J Madha vi Diagnosting in ternet congestion with a transp ort la y er p erformance to ol Pr o
c e e dings of the INET G Minshall rob ostats Av ailable from h ttpwwwipsiloncom minshallswrob ostats rob ostats
rob ostatsh tml Septem b er K Obraczk a P B Danzig D DeLucia and E Tsai A to ol for massiv ely replicating in ternet arc hiv es
Design implem en tation and exp erience Pr o c e e dings of the th IEEE ICDCS pages ! Ma y V P axson T o w ards a framew ork for dening in ternet p erformance metrics Pr o c e e dings of the INET V P axson Endtoend routing b eha vior in the in ternet Pr o c e e dings of the A CM Sigc omm pages
! August M Ranganathan A Ac hary a S Sharma and J Saltz Net w orka w are mobile programs Pr o c e e dings of the
USENIX T e chnic al Confer enc e RSc hemers fping Av ailable from ftpslapshotstanfordedupub
J Seda y ao and K Akita Lac hesis A to ol for b enc hmarking in ternet service pro viders Pr o c e e dings of the
LISA IX Confer enc e D W essels Squid in ternet ob ject cac he Squid w eb page h ttpsquidnlanrnet July R W olski F orecasting net w ork p erformance to supp ort dynamic sc heduling using the net w ork w eather
service Pr o c e e dings of the th IEEE International Symp osium on High Performanc e Distribute d Computing August C Y oshik a w a B Ch un P Eastham A V ahdat T Anderson and D Culler Using smart clien ts to build
scalable services Pr o c e e dings of the USENIX T e chnic al Confer enc e

Asset Metadata

Creator Gheorghiu, Grig (author), Obraczka, Katia (author)

Core Title USC Computer Science Technical Reports, no. 660 (1997)

Alternative Title The performance of a service for network-aware applications (title)

Publisher Department of Computer Science,USC Viterbi School of Engineering, University of Southern California, 3650 McClintock Avenue, Los Angeles, California, 90089, USA (publisher)

Tag OAI-PMH Harvest

Format 17 pages (extent), technical reports (aat)

Language English

Unique identifier UC16270976

Identifier 97-660 The Performance of a Service for Network-Aware Applications (filename)

Legacy Identifier usc-cstr-97-660

Format 17 pages (extent),technical reports (aat)

Rights Department of Computer Science (University of Southern California) and the author(s).

Internet Media Type application/pdf

Source 20180426-rozan-cstechreports-shoaf (batch), Computer Science Technical Report Archive (collection), University of Southern California. Department of Computer Science. Technical Reports (series)

Access Conditions The author(s) retain rights to their work according to U.S. copyright law. Electronic access is being provided by the USC Libraries, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright.

Repository Name USC Viterbi School of Engineering Department of Computer Science

Repository Location Department of Computer Science. USC Viterbi School of Engineering. Los Angeles\, CA\, 90089

Repository Email csdept@usc.edu

Inherited Values

Title Computer Science Technical Report Archive

Description Archive of computer science technical reports published by the USC Department of Computer Science from 1991 - 2017.

Coverage Temporal 1991/2017

Repository Email csdept@usc.edu

Repository Name USC Viterbi School of Engineering Department of Computer Science

Repository Location Department of Computer Science. USC Viterbi School of Engineering. Los Angeles\, CA\, 90089

Publisher Department of Computer Science,USC Viterbi School of Engineering, University of Southern California, 3650 McClintock Avenue, Los Angeles, California, 90089, USA (publisher)

Linked assets

Computer Science Technical Report Archive