Page 1 |
Save page Remove page | Previous | 1 of 204 | Next |
|
small (250x250 max)
medium (500x500 max)
large ( > 500x500)
Full Resolution
All (PDF)
|
This page
All
Subset |
ONLINE LEARNING ALGORITHMS FOR NETWORK OPTIMIZATION WITH
UNKNOWN VARIABLES
by
Yi Gai
A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(ELECTRICAL ENGINEERING)
December 2012
Copyright 2012 Yi Gai
Object Description
| Title | Online learning algorithms for network optimization with unknown variables |
| Author | Gai, Yi |
| Author email | yigaiee@gmail.com;ygai@usc.edu |
| Degree | Doctor of Philosophy |
| Document type | Dissertation |
| Degree program | Electrical Engineering |
| School | Viterbi School of Engineering |
| Date defended/completed | 2012-10-17 |
| Date submitted | 2012-11-26 |
| Date approved | 2012-11-26 |
| Restricted until | 2012-11-26 |
| Date published | 2012-11-26 |
| Advisor (committee chair) | Krishnamachari, Bhaskar |
| Advisor (committee member) |
Jain, Rahul Dughmi, Shaddin |
| Abstract | The formulations and theories of multi-armed bandit (MAB) problems provide fundamental tools for optimal sequential decision making and learning in uncertain environments. They have been widely applied to resource allocation, scheduling, and routing in communication networks, particularly in recent years, as the field is seeing an increasing focus on adaptive online learning algorithms to enhance system performance in stochastic, dynamic, and distributed environments. This dissertation addresses several key problems in this domain. ❧ Our first focus is about MAB with linear rewards. As they are fundamentally about combinatorial optimization in unknown environments, one would indeed expect to find even broader use of multi-armed bandits. However, a barrier to their wider application in practice has been the limitation of the basic formulation and corresponding policies, which generally treat each arm as an independent entity. They are inadequate to deal with many combinatorial problems of practical interest in which there are large numbers of arms. In such settings, it is important to consider and exploit any structure in terms of dependencies between the arms. In this dissertation, we show that when the dependencies take a linear form, they can be handled tractably with algorithms that have provably good performance in terms of regret as well as storage and computation. We develop a new class of learning algorithms for different problem settings including i.i.d. rewards, rested Markovian rewards, and restless Markovian rewards, to improve the cost of learning, compared to prior work, for large-scale stochastic network optimization problems. ❧ We then consider the problem of optimal power allocation over parallel channels with stochastically time-varying gain-to-noise ratios for maximizing information rate (stochastic water-filling) with both linear and non-linear multi-armed bandit formulations and propose new efficient online learning algorithms for these. ❧ Finally, we focus on learning in decentralized settings. The desired objective is to develop decentralized online learning algorithms running at each user to make a selection among multiple choices, where there is no information exchange, such that the sum-throughput of all distributed users is maximized. We make two contributions in this problem. First, we consider the setting where the users have a prioritized ranking, such that it is desired for the K-th ranked user to learn to access the arm offering the K-th highest mean reward. For this problem, we present the first distributed algorithm that yields regret that is uniformly logarithmic over time without requiring any prior assumption about the mean rewards. Second, we consider the case when a fair access policy is required, i.e., it is desired for all users to experience the same mean reward. For this problem, we present a distributed algorithm that yields order-optimal regret scaling with respect to the number of users and arms, better than previously proposed algorithms in the literature. |
| Keyword | online learning; network optimization; algorithm design and analysis; machine learning |
| Language | English |
| Part of collection | University of Southern California dissertations and theses |
| Publisher (of the original version) | University of Southern California |
| Place of publication (of the original version) | Los Angeles, California |
| Publisher (of the digital version) | University of Southern California. Libraries |
| Provenance | Electronically uploaded by the author |
| Type | texts |
| Legacy record ID | usctheses-m |
| Rights | Gai, Yi |
| Access conditions | The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given. |
| Repository name | University of Southern California Digital Library |
| Repository address | USC Digital Library, University of Southern California, University Park Campus MC 7002, 106 University Village, Los Angeles, California 90089-7002, USA |
| Repository email | cisadmin@usc.edu |
| Archival file | uscthesesreloadpub_Volume6/etd-GaiYi-1348.pdf |
Description
| Title | Page 1 |
| Full text | ONLINE LEARNING ALGORITHMS FOR NETWORK OPTIMIZATION WITH UNKNOWN VARIABLES by Yi Gai A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (ELECTRICAL ENGINEERING) December 2012 Copyright 2012 Yi Gai |
Comments
Post a Comment for Page 1
