Page 1 |
Save page Remove page | Previous | 1 of 169 | Next |
|
small (250x250 max)
medium (500x500 max)
large ( > 500x500)
Full Resolution
All (PDF)
|
This page
All
Subset |
PROVENANCE MANAGEMENT FOR DYNAMIC, DISTRIBUTED AND
DATAFLOW ENVIRONMENTS
by
Jing Zhao
A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(COMPUTER SCIENCE)
December 2012
Copyright 2012 Jing Zhao
Object Description
| Title | Provenance management for dynamic, distributed and dataflow environments |
| Author | Zhao, Jing |
| Author email | zhaoj@usc.edu;zhaojing9@gmail.com |
| Degree | Doctor of Philosophy |
| Document type | Dissertation |
| Degree program | Computer Science |
| School | Viterbi School of Engineering |
| Date defended/completed | 2012-07-03 |
| Date submitted | 2012-10-11 |
| Date approved | 2012-10-12 |
| Restricted until | 2012-10-12 |
| Date published | 2012-10-12 |
| Advisor (committee chair) | Prasanna, Viktor |
| Advisor (committee member) |
Nakano, Aiichiro Raghavendra, Raghu Simmhan, Yogesh |
| Abstract | Provenance, the derivation history of data objects, records how, when, and by whom a piece of data was created and modified. Provenance allows users to understand the context of derived data, estimate its quality for use, locate data of interest, and determine datasets affected by erroneous processes. Thus it is playing an important role in scientific experiments and business processes for data quality control, audit trail, and ensuring regulatory compliance. ❧ While most of the previous works only study provenance in a closed and well-controlled environment (e.g., a workflow engine), challenges still exist for holistic provenance management in practical and open environments, where provenance can be distributed, dynamic and diverse. For example, in the Energy Informatics domain, provenance is often collected from large-scale workflows across disciplines and organizations and thus is usually stored in distributed repositories. However, there has been limited research on reconstruction of and query over distributed provenance information. Meanwhile, recurrent and stream processing workflows can generate fine-grained provenance with overwhelming size that can be larger than the original dataset. Provenance storage approaches for efficiently managing such metadata volumes do not have adequate focus in literature. And lastly, the fact that legacy tools without automatic provenance collection functionalities are still widely used leads to the requirement of manual provenance annotation operations, which causes provenance to be incomplete. ❧ In this thesis, by using Energy Informatics as an exemplar domain, we design and develop algorithms and systems for managing provenance in dynamic, distributed and dataflow environments, that are motivated by real world challenges. In particular, we make the following contributions: (1) template-based algorithms that can efficiently store provenance information for dynamic datasets, (2) algorithms for reconstructing and querying provenance graphs from distributed provenance repositories, (3) semantic-based approaches for predicting incomplete provenance. We evaluate our research contributions with use cases from the Energy Informatics domain, including both Smart Oilfield and Smart Grid. The evaluation results demonstrate that our work can achieve efficient and scalable provenance management. As future work, we also discuss key challenges and initial solutions for presenting provenance across different granularities based on its usage context information. |
| Keyword | dataflow; distributed environments; provenance; provenance management; energy informatics |
| Language | English |
| Part of collection | University of Southern California dissertations and theses |
| Publisher (of the original version) | University of Southern California |
| Place of publication (of the original version) | Los Angeles, California |
| Publisher (of the digital version) | University of Southern California. Libraries |
| Provenance | Electronically uploaded by the author |
| Type | texts |
| Legacy record ID | usctheses-m |
| Rights | Zhao, Jing |
| Access conditions | The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given. |
| Repository name | University of Southern California Digital Library |
| Repository address | USC Digital Library, University of Southern California, University Park Campus MC 7002, 106 University Village, Los Angeles, California 90089-7002, USA |
| Repository email | cisadmin@usc.edu |
| Archival file | uscthesesreloadpub_Volume4/etd-ZhaoJing-1246.pdf |
Description
| Title | Page 1 |
| Full text | PROVENANCE MANAGEMENT FOR DYNAMIC, DISTRIBUTED AND DATAFLOW ENVIRONMENTS by Jing Zhao A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (COMPUTER SCIENCE) December 2012 Copyright 2012 Jing Zhao |
Comments
Post a Comment for Page 1
