Computer Science Technical Report Archive

USC Computer Science Technical Reports, no. 612 (1995)

(USC DC Other)

USC Computer Science Technical Reports, no. 612 (1995)

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content AN ALGORITHM F OR DISK SP A CE MANA GEMENT
TO MINIMIZE SEEKS
SHAHRAM GHANDEHARIZADEH AND DOUG IERARDI
Abstra ct The past decade has witnessed a proliferation of rep osito
ries whose w orkload consists of queries that retriev e information These
rep ositories pro vide online access to v ast amoun t of data and serveas
an in tegral comp onentof man y application s eg library information
systems scien tic applicatio ns and the en tertainmen t industry Their
storage subsystems are exp ected to b e hierarc hical consisting of mem
ory magnetic disk driv es optical disk driv es and tap e libraries The
database itself resides p ermanen tly on the tap e Files are staged and
selectiv ely cac hed on either the magnetic or optical disk driv es and later
deleted when the a v ailable space of a device is exhausted This b eha v
ior will generally cause fragmen tation of the disk space o v er a p erio d
of time resulting in a noncon tiguous la y out of diskresident les As
a consequence the disk is required to rep osition its read head m ultiple
times incurring seek op erations whenev er a residen t le is retriev ed
in a sequen tial manner th us reducing the o v erall p erformance of the
system
This brief study describ es an algorithm to manage the storage and
placementof les cac hed on mec hanical devices It acts in an online
manner to main tain a dynamicall y c hanging w orking set of diskresiden t
les while guaran teeing b oth maxim um utilizatio n of disk space and
amaxim um of dlg ne seeks for the con tin uous retriev al of eac h disk
residen t le of n blo c ks W e presentt wov arian ts of this algorithm
whic h explore tradeos in reorganization o v erhead and main memory
utilizatio n
Keyw ords data pro cessing le systems cac hing
Date Marc h
Intr oduction
Arecen t trend in the area of databases has b een an increase in the n um ber
of rep ositories whose primary function is to disseminate information These
systems are exp ected to pla y a ma jor role in library information systems
scien tic applications the en tertainmen t industry health care information
systems and kno wledgebased systems These systems t ypically pro vide
online access to v ast amoun t of data The large size of their databases has
led to the use of hierarc hical storage structures consisting of a com bination
of fast and slo w devices the database resides p ermanen tly on the slo w est
device in the hierarc h y and the system con trols the placementof data in
order to hide their high latency using its faster devices suc h as magnetic
disks Data is sw app ed in and out of the disks based on its exp ected future
access patterns with the ob jectiv e of minimizing the frequency of access to
slo w er devices
Ahierarc hical storage structure ma y consist of mec hanical devices suc h
as magnetic disks optical disks and tap e devices These devices exhibit the
follo wing prop erties Data is stored on the medium in a linear manner
The read head of the device ma ybe mo v ed to ph ysical lo cation on the
medium this op eration is termed a se ek Once the seek is complete
a device can read sequen tially from the medium The time required for a
seek on these devices is often substan tial relativ e to the time required to
read data from the medium In this sense seek op erations are w asteful and
should b e minimized in order to maximize the amountofuseful w ork that
is p erformed
This study describ es an algorithm to manage the storage and placementof
ob jects or les on suc h devices The goal of these algorithms is to maximize
the con tiguous la y out of stored ob jects and th us to minimize the n um ber
of seeks p erformed when retrieving an ob ject in a sequen tial manner This
sort of con tin uous retriev al of large ob jects is common in scien tic and m ul
timedia databases Be T o simplify the discussion w e shall assume that
ob jects will b e cac hed on a magnetic disk
All ob jects in the system are assumed to b e readonly A cop yofeachre sides p ermanen tly on the slo w est device in the hierarc h y Since w e fo cus only
on the space managemen t issue w e shall assume that there is an external
mo dule whic h determines the set of ob jects to b e cac hed on the disk This
mo dule issues a p oten tially innite sequence of requests to materialize and
free ob jects on the device See GIZ for details of suc h a mo dule Its
criteria for c ho osing these ob jects to b e cac hed on the disk is unsp ecied
although it is presumably c hosen to t the system and its w orkload W e fur
ther assume that whenev er a request to materialize an ob ject of n blo c ks is
issued there are in fact at least n free blo c ks on the disk th us the external
mo dule has selected victims to b e deleted in order to accommo date the new
ob jects that are to b e materialized and has issues the necessary requests to
main tain its c hosen w orking set in an appropriate order Ho w ev er there are

MINIMIZING SEEKS no further restrictions on this mo dule So if there are a total of c blo c ks on
the disk then it ma yc ho ose to cac he an y set of ob jects of requiring at most
c blo c ks on the disk
Giv en suc h a mo dule it is the dut y of the storage manager to main tain
the requested w orking set on disk Its p erformance will b e judged b yt w o
criteria the time required b y a user pro cess to do a sequen tial read
of an y diskresiden t ob ject in the w orst case and the additional time
required to main tain this disk organization o v er and ab o v e the n um ber
required to materialize les The algorithm of x b elo w guaran tees that an y
diskresiden tobject of n blo c ks can b e retriev ed sequen tially with at most
dlg ne seeks requires few er than n additional disk reads and writes when
materializing an ob ject The t w o algorithms of xand x presen t a tradeo
bet w een the n um b er of additional main tainance op erations incurred and
the complexit y of the disks directory structure a record of the state of the
disk
The algorithm mak es use of w ell kno wn metho ds for managing dynamic
main memory datastructures as in Mel and CLR but applied to
secondary devices These p oin ts are discussed further in x Empirical
results of a comparison b et w een this algorithm and other common strategies
ma y b e found in GIZ
The mo del Assume that the storage medium has b een partitioned
in to physic al blo cks of a xed size where a blo c k is the minim um unit of space
allo cation for the ph ysical device All blo c ks are assumed to ha v e the same
size Let B
B
B
C denote the ordered sequence of ph ysical blo c ks
on the disk where C is the capacit y of the device in blo c ks Assume also a
xed collection of ob jects or les Eac hobject o is comprised of an ordered
sequence ho
o
n
i of n p agesfor some n its size denoted joj Eac h
ph ysical blo c k of the disk can hold the data in exactly one page An ob ject
is diskr esident if all of its pages reside on the disk The assignmen t of pages
to blo c ks will b e giv en b y a partial function that assigns no t w o pages
to the same ph ysical blo c k Let R
b e the set of diskresiden t ob jects for
W e shall assume that ev ery ob ject that is not diskresiden t has no pages
residing on the disk So the algorithm cannot tak eadv an tage of partially
residen t ob jects and whenev er an ob ject is nonresiden t all of its pages
m ust b e materialized on the disk With resp ect to a giv en assignmen t a
disk blo ckis fr e e if do es not assign a page of an y diskresiden tobject to
it
Let o be an y disk residen t ob ject and o
i
apage of o then the retriev al
of o
i
incurs a se ek under the currentla y out if i and its predecessor
o
i is not assigned to the blo c k immediately preceeding o
i
That is in
a con tin uous retriev al of the pages of o in sequence the device is forced to
seek to blo c k o
i
after reading o
i os in ternal fragmen tation under a
giv en la y out is the n um b er of its pages that incur a seek
MINIMIZING SEEKS Basic op erations and their costs Materialization of an ob ject o
on the disk implies that the set of residen t ob jects and their la y out b oth
c hange T o simplify the analysis for this pap er w e assume no cost for
writing a page of o to disk during materialization b ecause an y algorithm
m ust incur this cost Instead w e fo cus on t w o quan tities that the algorithm
in tends to minimize the in ternal fragmen tation of an ob ject or the
n um b er of seeks incurred during a con tin uous retriev al of that ob ject and
the n um b er of pages that are copied from one ph ysical blo c k to another
or the cost of reorganizing or compacting disk space
It is assumed that the v arious datastructures whic h record the state of
the system suc h as directories and the free list are main tained in main
memoryW e do not address issues related to crash reco v ery in this pap er
The Basic Algorithm
T o manage the disks space w e rst imp ose an ordered dary tree structure
on the sequence of ph ysical blo c ks in whic h lea v es corresp ond to blo c ks on
the disk and their order corresp onds to the blo c ks ph ysical sequence T o
simplify the description of the algorithm w etak e d Generalization to
the case d is straigh tforw ard
The tree structure imp osed on the ph ysical blo c ks is built up in the follo w
ing b ottomup manner Let B i j denote the con tiguous sequence of blo c ks
B
i
B
j
F or eachin teger h blg C c and eac h i bC h
c the in terv al
B i h
i h

is the ith se ction of height h The sections of heigh t consisting of single
blo c ks are the lea v es Eac h section of heigh t h con tains exactly t w o
sections of heigh t h whic h are its c hildren By taking these sections
as in ternal no des the blo c ks of the disk are no w organized in to an ordered
forest of complete binary trees Sections that are siblings in this tree also
called buddies Kno So eachcon tiguous section of heigh t h can b e
decomp osed in to a pair of adjacen t sections of heigh t h Con v erselyeac h
pair of buddies of heigh t h can b e com bined to form a single con tiguous
section of heigh t h There is at most one section of eac h heigh t that has no
buddy eac h is the ro ot of a complete binary tree in this forest Belo ww e
call these sections ro ots
Wesa y that a section is o ccupied b y the ob ject o if some subsequence of
os pages are laid out con tiguously in the blo c ks of this section By the
sections of o w e mean the maximal sections o ccupied b y o ie those of
maximal heigh t under con tainmen t
The free list A section is free if ev ery page in that section is free
F or anyla y out the free list is just a list of all maximal free sections ie
those free sections that are not con tained in other free sections The curren t
free list will b e recorded in a memoryresiden t data structure main tained as
MINIMIZING SEEKS a sequence of lists one for eac h p ossible section heightfrom to blg C c The address of eac h maximal section of heigh t h is enqueued in a list that
handles sections of heigh t h only In v arian t prop erties F or an y n nC let n
h
denote the hth
bit in the binary expansion n so that
n blg C c
X
h
n
h
h
In its simplest v ersion the algorithm for managing data on the disk will
main tain the follo wing prop erties
If an ob ject o ccupies a section then all of its pages are stored con tigu
ously in the blo c ks of that section
Let o be an y diskresiden tobject and n joj Then o has exactly n
h
maximal section of heigh t h b lg C c Supp ose that there are f free blo c ks on the disk Then the free list
con tains exactly f
h
maximal free sections of heigh t h for eac h h blg C c F or example supp ose that the disk has a capacit y of blo c ks So the
sections range in heigh t from to There are t w o ro ots one of a complete
binary tree of heigh t and another of heigh t If anobject o of pages is
residen t then the prop erties ab o v e require that the pages of o o ccup y three
sections one of heigh t with blo c ks one section of heigh t with blo c ks and one of heigh t with blo c k A tmost pagesof o will incur
a seek Similarly if the free list con tains free blo c ks then these m ust all
occup y a single section of height T o record the state of the system eg a directory it suces to record
the set of disk residen t ob jects their maximal sections and the sections
on the free list Since eac h maximal section can b e sp ecied b y its starting
address and its heigh t the disp osition of a residen tobject of n blo c ks can b e
recorded in O lg n space and the sections on the free list can b e recorded
in O lg C space So a record of the en tire system requires space at most
X
o R
O lg joj O lg C O jRj lg C Materialization Materialization of an ob ject is handled b y the fol
lo wing algorithm Assume that the prop erties en umerated in the previous
section hold and a request is made to materialize ob ject o of n pages Let
n P
blg C c
h
n
h
h
P artition the blo c ks of o in to in terv als with n
h
in terv als
of size h
for eac h h F or eachheigh t h blg C c if n
h
the follo wing algorithm is
used to allo cate a section of heigh t h If there is a section of heigh t h on the free list then allo cate this section
MINIMIZING SEEKS Otherwise recursiv ely allo cate one section of heigh t h P artition
this in to sections of heigh t h Enqueue one of these on the free list
and return the other
Once all sections are allo cated eac hin terv al of o is copied con tiguously in to
a section of the appropriate heigh t
Lemma The al lo c ation algorithm pr eserves al l the pr op erties of x It r e quir es at most O lg C pr o c essing time and writes exactly n disk blo cks
during the op er ation
Pr o of If there are f free blo c ks and f P
h
f
h
h
then for eac h h there
are f
h
maximal free sections of heigh t h on the free list The algorithm
ab o v e simply mimics the algorithm for computing the dierence of f and n
as binary n umerals The prop erties en umerated ab o v e follo w immediately
from this observ ation
Deletion The follo wing steps are tak en to remo v e the ob ject o from
the diskresiden t set and to reclaim its space First all of the sections of
o are enqueued on the appropriate free lists Then for eac h heigh t h blg C c the space is compacted First in memory a new la y out is
determined using the follo wing algorithm While there are more than d
sections on the list these steps are rep eated
First c ho ose an y of these sections Call these f
and f
F rom these
c ho ose one that is not a ro ot Without loss of generalit y assume that
this is f
and let b
b e its buddy Then
a Remo v e f
from the free list
b Record that the page stored at blo c k b
is to b e copied to f
c Add b
to the free list
No w f
and its buddy are b oth on the free list at heigh t hRemo v e
them from the list merge them and place the resulting section of
heigh t h on the next free list
Once the new la y out is determined it is realized on the disk b ycop ying
pages directly to the nal lo cations that w ere computed b y the algorithm
Of course if the algorithm determined that a page w as to b e copied m ultiple
times rst to one blo c k and from there to another the cop ying phase
merely mo v es it directly from its initial to nal lo cation
Lemma The deletion algorithm pr eserves al l the pr op erties of x The pro of again follo ws from the observ ation that freeing the space allo cated
to an ob ject is formally equiv alen t to adding n to f as binary n umerals
A Lazy V ariant of the Algorithm
The implemen tation of the basic op erations presen ted ab o v e yields a
rather high cost for the deletion of ob jects ev en in an amortized sense
MINIMIZING SEEKS In the w orst case the deletion of an ob ject can cause a cascade of com
pactions in v olving sections of larger heigh t F or example in the case where
eac h of the sublists at heigh ts k has exactly one section freeing a sin
gle blo c k can require cop ying nearly k
additional blo c ks Th us b ecause
the cop ying costs asso ciated with larger sections is also larger the cost of a
deletion cannot b e b ounded immediately b y the size of the ob ject deleted
These costs can b e b ounded b y a simple v arian t of the algorthm ab o vein
whic h the compaction of the deallo cation pro cedure is made lazy In other
w ords once a new la y out for the data on the disk has b een determined after
deallo cation of an ob ject the data is not immediately reorganized Instead
arecordisk ept of the c hanges needed to realized this new organization As
a consequence a deletion will not initiate an y disk activit y and as argued
belo w the additional o v erhead incurred b y materializing an ob ject will b e
prop ortional to its size
T o realize this eac h section on the free list will b e designated either dirty
or cle anIf no ph ysical blo c k of a section con tains v alid data ie a page of a
diskresiden t ob ject that is not stored in some other blo c k then the section
is clean otherwise it is dirt yEac hdirt y section also carries with it a list of
the target blo c ks or sections that are to receiv e the v alid data that its o wn
blo c ks or subsections con tains The materialization and deletion pro cedures
ab o v e are then mo died as follo ws
Deletion Whenev er sections are merged during the deallo cation pro
cedure their resp ectiv e lists are concatenated and the ph ysical mo v emen t
of data to new blo c ks is p ostp oned Hence the in v arian t prop erties and of x ma y infact beviolated on the disk since there ma y b e more than
one uno ccupied section of eac h heighty et the prop ert y is alw a ys main
tained on the memoryresiden t free list Since the deletion pro cedure aects
only these data structures there is no cop ying of diskresiden tdata Materialization When an ob ject is materialized the algorithm of x section is used but with a minor mo dication Before the ob ject is written
to disk the v alid data o ccup ying the sections allo cated to the ob ject are
rst mo v ed to the target lo cations recorded previously Theorem When mo die dasab ove the stor age management algorithm
c an dynamic al ly maintain a diskr esident set of obje cts F or e ach obje ct o
of n blo cks the algorithm guar ante es that while o is diskr esident at
most dlg ne se eks arer e quir e d whenever o is r etr eivedinits entir ety and
that materialization of o on disk r e quir es fewer than n additional disk
r e ads and writes that deletion of o fr om the disk r e quir es no additional
disk activity
Pr o of The rst guaran tee follo ws from the fact that in v ariantproperty
con tin ues to hold b oth in memory and on disk Hence eac hobject o has at
most dlg joje sections The second p oin t follo ws from the observ ation that
the n um b er of blo c ks mo v ed during a write of an ob ject o strictly less than
MINIMIZING SEEKS the n um ber of blo c ks con tained in o itself The last guaran tee is immediate
from the description of the deallo cation pro cedure
Note ho w ev er that cost incurred b y adopting this lazy b eha vior is a larger
size for the memoryresiden t free list
Conclusions
As noted in x the algorithms prop osed ab o v e resem ble main memory
algorithms for dynamic data structures lik e the binomial heap CLR
and those for dealing with decomp osible searc h problems as in Mel It
also has some relation to the buddy system prop osed in Kno LD for
ecien t main memory DRAM storage allo cators Ho w ev er the latter is
required to allo cate a con tiguous c h unk of memory for eac h ob ject materi
alized This results in fragmen tation and and motiv es the need for either a
reorganization pro cess or a garbage collector GR
W e conjecture that the lazy algorithm prop osed ab o veis in factpro vides
an optimal organization for b ounding the n um b er of seeks obtained when
cac hing a w orking set of ob jects on a linear storage medium while b oth
minimizing the o v erhead required to main tain the organization as w orking
set c hanges and maximizing the utilization of the medium the amoun tof
data that is cac hed More precisely Conjecture Under the assumptions of x any stor age management
scheme which guar ante es that every diskr esident obje ct o incurs fewer than
dlg joje se eks for se quential ac c ess either r e quir es morethan joj additional
disk writes for materializing some obje ct oor c annot ful ly utilize the sp ac e
of the devic e fr om some se quenceof r e quests
A ckno wledgements
This researchw as supp orted in part b y the National Science F oundation
under gran ts IRI IRI NYI a w ard CD A and
CCR b y a DOD con tract and b y an unrestricted cashequipmen t
gift from HewlettP ac k ard
References
Be TE Bell Harv esting remotesensing data IEEE Sp e ctrumV olume Num
b er Marc h CLR TH Cormen CE Leiserson and RL Riv est Intr o duction to A lgorithms c hapter pages The MIT Press GIZ S Ghandeharizad eh D Ierardi and R Zimmerman Management of Sp acein
Hier ar chic al Stor age Devic esT ec hnical Rep ort USCCS Univ ersit y
of Southern California Departmen t of Computer Science No v em b er GR J Gra y and A Reuter T r ansaction Pr o c essing Conc epts and T e chniques c hapter pages Morgan Kaufmann Kno K C Kno wlton A fast storage allo cator Communic ations of the A CM Octob er
MINIMIZING SEEKS LD H R Lewis and L Denen b erg Data Structur es Their A lgorithmsc hap
ter pages Harp er Collins Mel K Melhorn Data Structur es and A lgorithms Multidimensio nal Se ar ching
and Computational Ge ometryc hapter VI I pages Springer V erlag Dep ar tment of Computer Science University of Southern Calif ornia Los
Angeles CA

Linked assets

Computer Science Technical Report Archive

Conceptually similar

PDF

USC Computer Science Technical Reports, no. 598 (1994)

PDF

USC Computer Science Technical Reports, no. 628 (1996)

PDF

USC Computer Science Technical Reports, no. 602 (1995)

PDF

USC Computer Science Technical Reports, no. 625 (1996)

PDF

USC Computer Science Technical Reports, no. 610 (1995)

PDF

USC Computer Science Technical Reports, no. 623 (1995)

PDF

USC Computer Science Technical Reports, no. 600 (1995)

PDF

USC Computer Science Technical Reports, no. 584 (1994)

PDF

USC Computer Science Technical Reports, no. 601 (1995)

PDF

USC Computer Science Technical Reports, no. 619 (1995)

PDF

USC Computer Science Technical Reports, no. 650 (1997)

PDF

USC Computer Science Technical Reports, no. 618 (1995)

PDF

USC Computer Science Technical Reports, no. 653 (1997)

PDF

USC Computer Science Technical Reports, no. 622 (1995)

PDF

USC Computer Science Technical Reports, no. 862 (2005)

PDF

USC Computer Science Technical Reports, no. 630 (1996)

PDF

USC Computer Science Technical Reports, no. 627 (1996)

PDF

USC Computer Science Technical Reports, no. 615 (1995)

PDF

USC Computer Science Technical Reports, no. 556 (1993)

PDF

USC Computer Science Technical Reports, no. 666 (1998)

Description

Shahram Ghandeharizadeh, Doug Ierardi. "An algorithm for disk space management to minimize seeks." Computer Science Technical Reports (Los Angeles, California, USA: University of Southern California. Department of Computer Science) no. 612 (1995).

Asset Metadata

Creator Ghandeharizadeh, Shahram (author), Ierardi, Doug (author)

Core Title USC Computer Science Technical Reports, no. 612 (1995)

Alternative Title An algorithm for disk space management to minimize seeks (title)

Publisher Department of Computer Science,USC Viterbi School of Engineering, University of Southern California, 3650 McClintock Avenue, Los Angeles, California, 90089, USA (publisher)

Tag OAI-PMH Harvest

Format 9 pages (extent), technical reports (aat)

Language English

Unique identifier UC16271039

Identifier 95-612 An Algorithm for disk space management to minimize seeks (filename)

Legacy Identifier usc-cstr-95-612

Format 9 pages (extent),technical reports (aat)

Rights Department of Computer Science (University of Southern California) and the author(s).

Internet Media Type application/pdf

Source 20180426-rozan-cstechreports-shoaf (batch), Computer Science Technical Report Archive (collection), University of Southern California. Department of Computer Science. Technical Reports (series)

Access Conditions The author(s) retain rights to their work according to U.S. copyright law. Electronic access is being provided by the USC Libraries, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright.

Repository Name USC Viterbi School of Engineering Department of Computer Science

Repository Location Department of Computer Science. USC Viterbi School of Engineering. Los Angeles\, CA\, 90089

Repository Email csdept@usc.edu

Inherited Values

Title Computer Science Technical Report Archive

Description Archive of computer science technical reports published by the USC Department of Computer Science from 1991 - 2017.

Coverage Temporal 1991/2017

Repository Email csdept@usc.edu

Repository Name USC Viterbi School of Engineering Department of Computer Science

Repository Location Department of Computer Science. USC Viterbi School of Engineering. Los Angeles\, CA\, 90089

Publisher Department of Computer Science,USC Viterbi School of Engineering, University of Southern California, 3650 McClintock Avenue, Los Angeles, California, 90089, USA (publisher)