Page 1 |
Save page Remove page | Previous | 1 of 105 | Next |
|
small (250x250 max)
medium (500x500 max)
Large (1000x1000 max)
Extra Large
large ( > 500x500)
Full Resolution
All (PDF)
|
This page
All
|
NOVEL ALGORITHMS FOR LARGE SCALE SUPERVISED
AND ONE CLASS LEARNING
by
Lingyan Sheng
A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Ful llment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(ELECTRICAL ENGINEERING)
May 2013
Copyright 2013 Lingyan Sheng
Object Description
| Title | Novel algorithms for large scale supervised and one class learning |
| Author | Sheng, Lingyan |
| Author email | lsheng@usc.edu;lingyan.sheng@gmail.com |
| Degree | Doctor of Philosophy |
| Document type | Dissertation |
| Degree program | Electrical Engineering |
| School | Viterbi School of Engineering |
| Date defended/completed | 2012-09-12 |
| Date submitted | 2013-03-18 |
| Date approved | 2013-03-19 |
| Restricted until | 2013-03-19 |
| Date published | 2013-03-19 |
| Advisor (committee chair) | Ortega, Antonio K. |
| Advisor (committee member) |
Kuo, C.-C. Jay Liu, Yan |
| Abstract | Supervised learning is the machine learning task of inferring a function from labeled training data. There have been numerous algorithms proposed for supervised learning, such as linear discriminant analysis (LDA), support vector machine (SVM), decision trees, and etc. However, most of them are not able to handle an increasingly popular type of data, high dimensional data, such as gene expression data, text documents, MRI images, and etc. This phenomenon is often called the curse of dimensionality. Our solution to this problem is an improvement to LDA that imposes a regularized structure on the covariance matrix, so that it becomes block diagonal while feature reduction is performed. The improved method, which we call block diagonal discriminant analysis (BDLDA), effectively exploits the off diagonal information in the covariance matrix without huge computation and memory requirement. BDLDA is further improved by using treelets as a preprocessing tool. Treelets, by transforming the original data by successive local PCA, concentrates more energy near the diagonal items in the covariance matrix, and thus achieves even better accuracy compared to BDLDA. ❧ Supervised learning requires labeled information of all classes. However, since labeled data is often more difficult to obtain than unlabeled data, there is an increasing interest in a special form of learning, namely, one class learning. In one class learning, the training set only has samples of one class, and the goal is to distinguish the class from all other samples. We propose a one class learning algorithm, Graph-One Class Learning (Graph-OCL). Graph-OCL is a two step strategy, where we first identify reliable negative samples, and then we classify the samples based on labeled data and the identified negative samples in the first step. The main novelty is the first step, in which graph-based ranking by learning with local and global consistency (LGC) is used. Graph-based ranking is particularly accurate if the samples and their similarities are well represented by a graph. We also theoretically prove that there is a simple method to select a constant parameter ɑ for LGC, thus eliminating the necessity of model selection by time consuming validation. ❧ Graph-based methods usually scale badly as a function of the sample size. This can be solved by using the Nyström approximation, which samples a few columns to represent the affinity matrix. We propose a new method, BoostNyström, which adaptively samples a subset of columns at each iterative step and updates the sampling probability in the next iterative step. This algorithm is based on a novel perspective, which relates the quality of Nyström approximation with the subspace spanned by the sampled columns. BoostNyström can be potentially applied to Graph-OCL to solve the problem of large data size. |
| Keyword | supervised learning; one class learning; linear discriminant analysis; graph; Nyström approximation |
| Language | English |
| Part of collection | University of Southern California dissertations and theses |
| Publisher (of the original version) | University of Southern California |
| Place of publication (of the original version) | Los Angeles, California |
| Publisher (of the digital version) | University of Southern California. Libraries |
| Provenance | Electronically uploaded by the author |
| Type | texts |
| Legacy record ID | usctheses-m |
| Contributing entity | University of Southern California |
| Rights | Sheng, Lingyan |
| Physical access | The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given. |
| Repository name | University of Southern California Digital Library |
| Repository address | USC Digital Library, University of Southern California, University Park Campus MC 7002, 106 University Village, Los Angeles, California 90089-7002, USA |
| Repository email | cisadmin@dots.usc.edu |
| Archival file | uscthesesreloadpub_Volume7/etd-ShengLingy-1476.pdf |
Description
| Title | Page 1 |
| Contributing entity | University of Southern California |
| Repository email | cisadmin@dots.usc.edu |
| Full text | NOVEL ALGORITHMS FOR LARGE SCALE SUPERVISED AND ONE CLASS LEARNING by Lingyan Sheng A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Ful llment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (ELECTRICAL ENGINEERING) May 2013 Copyright 2013 Lingyan Sheng |
