Page 7 |
Save page Remove page | Previous | 7 of 243 | Next |
|
small (250x250 max)
medium (500x500 max)
Large (1000x1000 max)
Extra Large
large ( > 500x500)
Full Resolution
All (PDF)
|
This page
All
|
7 Harmonic Coding: An Optimal Linear Code for Privacy-Preserving Gradient-
Type Computation 79
7.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
7.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
7.3 Achievability Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
7.3.1 Example for K = 2, deg g = 2 . . . . . . . . . . . . . . . . . . . . . . . . . . 84
7.3.2 General Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
7.4 Converse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
7.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
7.5.1 Random Key Access and Extra Computing Power at Master . . . . . . . . . 89
7.5.2 An Optimum Scheme for Char F = deg g . . . . . . . . . . . . . . . . . . . . 89
8 Entangled Polynomial Codes for Secure, Private, and Batch Distributed Matrix
Multiplication: Breaking the “Cubic” Barrier 91
8.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
8.2 Secure, Private, Batch Distributed Matrix Multiplication and Main Results . . . . . 94
8.2.1 Secure Distributed Matrix Multiplication . . . . . . . . . . . . . . . . . . . . 94
8.2.2 Private Distributed Matrix Multiplication . . . . . . . . . . . . . . . . . . . . 95
8.2.3 Batch Distributed Matrix Multiplication . . . . . . . . . . . . . . . . . . . . . 96
8.3 Achievability Schemes for Secure Distributed Matrix Multiplication . . . . . . . . . . 98
8.4 Achievability Schemes for Private Distributed Matrix Multiplication . . . . . . . . . 100
8.5 Achievability Schemes for Batch Distributed Matrix Multiplication . . . . . . . . . . 103
III Optimal Codes for Communication 106
9 Fundamental Limits of Communication for Coded Distributed Computing 107
9.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
9.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
9.3 Converse of Theorem 9.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
9.4 Conclusion and Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
10 The Exact Rate-Memory Tradeoff for Caching with Uncoded Prefetching 114
10.1 System Model and Problem Definition . . . . . . . . . . . . . . . . . . . . . . . . . . 116
10.1.1 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
10.1.2 Problem Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
10.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
10.3 The Optimal Caching Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
10.3.1 Motivating Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
10.3.2 General Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
10.4 Converse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
10.5 Extension to the Decentralized Setting . . . . . . . . . . . . . . . . . . . . . . . . . . 130
10.5.1 System Model and Problem Formulation . . . . . . . . . . . . . . . . . . . . . 130
10.5.2 Exact Rate-Memory Tradeoff for Decentralized Setting . . . . . . . . . . . . . 132
10.6 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
vii
Object Description
| Title | Coded computing: a transformative framework for resilient, secure, private, and communication efficient large scale distributed computing |
| Author | Yu, Qian |
| Author email | qyu880@usc.edu;qianyu0929@gmail.com |
| Degree | Doctor of Philosophy |
| Document type | Dissertation |
| Degree program | Electrical Engineering |
| School | Viterbi School of Engineering |
| Date defended/completed | 2020-03-24 |
| Date submitted | 2020-08-04 |
| Date approved | 2020-08-05 |
| Restricted until | 2020-08-05 |
| Date published | 2020-08-05 |
| Advisor (committee chair) | Avestimehr, Salman |
| Advisor (committee member) |
Luo, HaiPeng Ortega, Antonio Soltanolkotabi, Mahdi |
| Abstract | Modern computing applications often require handling massive amounts of data in a distributed setting, where significant issues on resiliency, security, or privacy could arise. This dissertation presents new computing designs and optimality proofs, that address these issues through coding and information-theoretic approaches. ❧ The first part of this thesis focuses on a standard setup, where the computation is carried out using a set of worker nodes, each can store and process a fraction of the input dataset. The goal is to find computation schemes for providing the optimal resiliency against stragglers given the computation task, the number of workers, and the functions computed by the workers. The resiliency is measured in terms of the recovery threshold, defined as the minimum number of workers to wait for in order to compute the final result. We propose optimal solutions for broad classes of computation tasks, from basic building blocks such as matrix multiplication (entangled polynomial codes), Fourier transform (coded FFT), and convolution (polynomial code), to general functions such as multivariate polynomial evaluation (Lagrange coded computing). We develop optimal computing strategies by introducing a general coding framework called “polynomial coded computing”, to exploit the algebraic structure of the computation task and create computation redundancy in a novel coded form across workers. Polynomial coded computing allows for order-wise improvements over the state of the arts and significantly generalizes classical coding-theoretic results to go beyond linear computations. The encoding and decoding process of polynomial coded computing designs can be mapped to polynomial evaluation and interpolation, which can be computed efficiently. ❧ Then we show that polynomial coded computing can be extended to provide unified frameworks that also enable security and privacy in the computation. We present the optimal designs for three important problems: distributed matrix multiplication, multivariate polynomial evaluation, and gradient-type computation. We prove their optimality by developing information-theoretic and linear-algebraic converse bounding techniques. ❧ Finally, we consider the problem of coding for communication reduction. In the context of distributed computation, we focus on a MapReduce-type framework, where the workers need to shuffle their intermediate results to finish the computation. We aim to understand how to optimally exploit extra computing power to reduce communication, i.e., to establish a fundamental tradeoff between computation and communication. We prove a lower bound on the needed communication load for general allocation of the task assignments, by introducing a novel information-theoretic converse bounding approach. The presented lower bound exactly matches the inverse-proportional coding gain achieved by coded distributed computing schemes, completely characterizing the optimal computation-communication tradeoff. The proposed converse bounding approach strictly improves conventional cut-set bounds and can be widely applied to prove exact optimally results for more general settings, as well as more classical communication problems. We also investigate a problem called coded caching, where a single server is connected to multiple users in a cache network through a shared bottleneck link. Each user has an isolated memory that can be used to prefetch content. Then the server needs to deliver users’ demands efficiently in a following delivery phase. We propose caching and delivery designs that improve the state-of-the-art schemes under both centralized and decentralized settings, for both peak and average communication rates. Moreover, by developing information-theoretic bounds, we prove the proposed designs are exactly optimal among all schemes that use uncoded prefetching, and optimal within a factor of 2.00884 among schemes with coded prefetching. |
| Keyword | coding theory; information theory; distributed computing; security; privacy; matrix multiplication; caching |
| Language | English |
| Part of collection | University of Southern California dissertations and theses |
| Publisher (of the original version) | University of Southern California |
| Place of publication (of the original version) | Los Angeles, California |
| Publisher (of the digital version) | University of Southern California. Libraries |
| Provenance | Electronically uploaded by the author |
| Type | texts |
| Legacy record ID | usctheses-m |
| Contributing entity | University of Southern California |
| Rights | Yu, Qian |
| Physical access | The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given. |
| Repository name | University of Southern California Digital Library |
| Repository address | USC Digital Library, University of Southern California, University Park Campus MC 7002, 106 University Village, Los Angeles, California 90089-7002, USA |
| Repository email | cisadmin@lib.usc.edu |
| Filename | etd-YuQian-8883.pdf |
| Archival file | Volume13/etd-YuQian-8883.pdf |
Description
| Title | Page 7 |
| Full text | 7 Harmonic Coding: An Optimal Linear Code for Privacy-Preserving Gradient- Type Computation 79 7.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 7.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 7.3 Achievability Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 7.3.1 Example for K = 2, deg g = 2 . . . . . . . . . . . . . . . . . . . . . . . . . . 84 7.3.2 General Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 7.4 Converse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 7.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 7.5.1 Random Key Access and Extra Computing Power at Master . . . . . . . . . 89 7.5.2 An Optimum Scheme for Char F = deg g . . . . . . . . . . . . . . . . . . . . 89 8 Entangled Polynomial Codes for Secure, Private, and Batch Distributed Matrix Multiplication: Breaking the “Cubic” Barrier 91 8.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 8.2 Secure, Private, Batch Distributed Matrix Multiplication and Main Results . . . . . 94 8.2.1 Secure Distributed Matrix Multiplication . . . . . . . . . . . . . . . . . . . . 94 8.2.2 Private Distributed Matrix Multiplication . . . . . . . . . . . . . . . . . . . . 95 8.2.3 Batch Distributed Matrix Multiplication . . . . . . . . . . . . . . . . . . . . . 96 8.3 Achievability Schemes for Secure Distributed Matrix Multiplication . . . . . . . . . . 98 8.4 Achievability Schemes for Private Distributed Matrix Multiplication . . . . . . . . . 100 8.5 Achievability Schemes for Batch Distributed Matrix Multiplication . . . . . . . . . . 103 III Optimal Codes for Communication 106 9 Fundamental Limits of Communication for Coded Distributed Computing 107 9.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 9.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 9.3 Converse of Theorem 9.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 9.4 Conclusion and Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 10 The Exact Rate-Memory Tradeoff for Caching with Uncoded Prefetching 114 10.1 System Model and Problem Definition . . . . . . . . . . . . . . . . . . . . . . . . . . 116 10.1.1 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 10.1.2 Problem Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 10.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 10.3 The Optimal Caching Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 10.3.1 Motivating Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 10.3.2 General Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 10.4 Converse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 10.5 Extension to the Decentralized Setting . . . . . . . . . . . . . . . . . . . . . . . . . . 130 10.5.1 System Model and Problem Formulation . . . . . . . . . . . . . . . . . . . . . 130 10.5.2 Exact Rate-Memory Tradeoff for Decentralized Setting . . . . . . . . . . . . . 132 10.6 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 vii |
