Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Application-driven compressed sensing
(USC Thesis Other)
Application-driven compressed sensing
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
Application-driven Compressed Sensing by Sungwon Lee A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulllment of the Requirements for the Degree Doctor of Philosophy (ELECTRICAL ENGINEERING) August 2013 Copyright 2013 Sungwon Lee Dedication To my greatest parents Hunchul Lee Soonok Kim To my loving grandparents Jindoo Lee Jakyoung Huh To my uncle Bonjin Byun To my best friend Ga-Young Suh ii Acknowledgements I would like to express my sincere appreciation and gratitude to my advisor, Prof. Ortega, for his support and patience throughout my graduate studies at the Univer- sity of Southern California. He has been the best advisor who gave me the freedom to explore on my own, and at the same time the guidance to recover when my steps faltered. His philosophy of guidance has trained me as one of researchers who are able to think critically and keep asking myself why. I owe my gratitude to Prof. Krishnamachari and Prof. Neumann for serving as members of my dissertation committee and Prof. Kuo and Prof. Dimakis for serving as committee members of my qualify exam. Their valuable comments and suggestions were really appreciated. I would also like to say thank you to all my colleagues in USC and friends in our research group for valuable discussion and collaboration: Prof. Krishnamachari, Prof. Hashemi, Sundeep Pattem, Chenliang Du, Jaehoon Kim, In Suk Chong, Godwin Shen, Woo-Shik Kim, Hey-Yeon Cheong, Kun-Han Lee, Sean McPherson, Zihong Fan, Greg Lin, Sunil K. Narang, Amin Rezapour, Hilmi E. Egilmez, Akshay Gadde, Yung-Hsuan Chao, Jason Tokayer, Yongzhe Wang, Lingyan Sheng, Gloria Halfacre, Mary Francis, Allan Weber, Seth Scafani, and Talyia Veal. Many friends have supported and encouraged me to keep focused on what I originally dreamed of. Their support and care helped me overcome setbacks and stay focused on my graduate study. I greatly appreciate their friendship; Hoshik iii Kim, Juyong Brian Kim, Euihyun Kang, Soyeon Lee, Hyungil Chae, Christopher S. Chang, SangHyun Chang, Jeehwan Kim, Oh-Hoon Kwon, Jae Kyoung Suh, Jae- sun Seo, Seung-hyun Son, Jeansoo Khim, Jinhwan Lee, Youngsoo Sohn, Andrew Mc Son, Jungwoo Kim, Donnie Hojun Kim, Jaehoon Kim, Ahryon Cho, DGM families, and Gahee Kim. I appreciate Ga-Young Suh her supporting and encouraging me in dicult times. I heartfully thank my uncle, Bonjin Byun, for his constant care and concern from my rst day in USC. Most importantly, none of this would have been possible without the love and support of my parents and grandparents. They to whom this dissertation is dedicated to, has been a constant source of love, concern, support and strength all these years. I would like to express my heart-felt gratitude to my family. iv Table of Contents Dedication ii Acknowledgements iii List of Tables vii List of Figures viii Abstract xii Chapter 1: Introduction 1 1.1 Compressed Sensing (CS) . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Ecient data-gathering in WSN . . . . . . . . . . . . . . . . . . . . 6 1.3 Depth map compression for 3D-TV . . . . . . . . . . . . . . . . . . 10 1.4 Fast localization using UWB radar . . . . . . . . . . . . . . . . . . 12 Chapter 2: Spatially-Localized Compressed Sensing for Ecient Data- gathering in Multi-Hop Sensor Networks 15 2.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.2.1 Low-cost sparse projection based on clustering . . . . . . . . 19 2.2.2 Sparsity-inducing basis and cluster selection . . . . . . . . . 21 2.3 Ecient clustering for spatially-localized CS . . . . . . . . . . . . . 22 2.3.1 Independent vs. Joint reconstruction . . . . . . . . . . . . . 22 2.3.2 Spatially-Localized Projections in CS . . . . . . . . . . . . . 23 2.3.3 Average Energy Overlap . . . . . . . . . . . . . . . . . . . . 25 2.3.4 Maximum Energy Overlap . . . . . . . . . . . . . . . . . . . 27 2.4 Theoretical Result . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.4.1 Denitions and Assumptions . . . . . . . . . . . . . . . . . . 29 2.4.2 Main Result . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.4.3 Proof of Proposition 2.4.1 . . . . . . . . . . . . . . . . . . . 33 2.5 Centralized iterative clustering algorithm . . . . . . . . . . . . . . . 37 2.5.1 Cost function . . . . . . . . . . . . . . . . . . . . . . . . . . 37 v 2.5.2 Algorithm details . . . . . . . . . . . . . . . . . . . . . . . . 39 2.6 Simulation Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 2.6.1 Joint reconstruction vs. independent reconstruction . . . . . 42 2.6.2 SPT-based clustering . . . . . . . . . . . . . . . . . . . . . . 44 2.6.3 Reconstruction accuracy and . . . . . . . . . . . . . . . . 47 2.6.4 Joint optimization . . . . . . . . . . . . . . . . . . . . . . . 50 2.7 Extension to irregularly positioned sensors . . . . . . . . . . . . . . 52 2.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 Chapter 3: Adaptive Compressed Sensing for Depth map Compres- sion Using Graph-based Transform 60 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 3.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 63 3.3 Optimizing GBT for CS . . . . . . . . . . . . . . . . . . . . . . . . 68 3.3.1 Bound on the mutual coherence . . . . . . . . . . . . . . . . 68 3.3.2 Iterative GBT construction for CS . . . . . . . . . . . . . . 72 3.4 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 3.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 Chapter 4: Hardware-driven Compressive Sampling for Fast Target Localization using Single-chip UWB Radar Sensor 82 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 4.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 86 4.2.1 UWB Ranging System and Assumptions . . . . . . . . . . . 86 4.2.2 Window-based Sparse UWB Signal Model . . . . . . . . . . 87 4.2.3 UWB Measurement System and Matrix Formulation . . . . 89 4.3 Proposed Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 4.3.1 LDPC Measurement System . . . . . . . . . . . . . . . . . . 94 4.3.2 Window-based Reweighted L 1 Minimization . . . . . . . . . 96 4.4 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 4.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 Chapter 5: Conclusions and Future Work 107 5.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 5.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 Reference List 110 vi List of Tables 1.1 Choice of sensing / sparsifying basis for each application . . . . . . 2 1.2 Overview of application specic constraints / cost . . . . . . . . . . 3 2.1 Correlation coecient, r2 [1; 1], between M est and M sim . . . . . 48 2.2 Correlation coecient,r2 [1; 1], between maximum energy overlap and MSE for each basis . . . . . . . . . . . . . . . . . . . . . . . . . 49 3.1 For comparison, four dierent methods are considered. First, with H.264/AVC, only 4 4 DCT is enabled, thus no overhead exists. Second, DCT and GBT are enabled in RD optimization and, for GBT, the adjacency matrix,A, is optimized as in [38] with overhead transmitted to the decoder for each block that uses GBT. Third, with the sameA, an additional transform mode, CS, is considered. Lastly, DCT, GBT, and CS are considered in RD optimization as before but the adjacency matrix,A, is optimized as discussed in Section 3.3.2. 75 3.2 BD-PSNR/bitrate results of CS-GBT-2 compared to H.264/AVC. . 76 3.3 BD-PSNR/bitrate results of CS-GBT-2 compared to GBT. . . . . . 77 4.1 Performance evaluation of identication of data support (DS): The acquisition time is reduced to 0.4 compared to sequential sampling scheme, M=N = 0:4, with noisy measurements, N = 30. . . . . . . 102 4.2 Performance evaluation of identication of data support (DS): The acquisition time is reduced to 0.2 compared to sequential sampling scheme, M=N = 0:3, with noisy measurements, N = 20. . . . . . . 103 vii List of Figures 1.1 General matrix formulation of compressed sensing. . . . . . . . . . . 4 2.1 Link between CS measurements and data aggregation in WSN . . . 20 2.2 (a) Illustration of energy overlap for a 44 grid network of 16 sensors. The network is divided into 4 square clusters and 3 basis functions are considered. The bases have spatial resolution of 1, 3 and 9 sen- sors, forB 1 ,B 2 andB 3 , respectively. (b) Permuted sparsifying basis matrix, =P ~ . The entries of each basis function (column vector of ) is lled with colors if non-zero coecients exist and white oth- erwise. Note that 13 more basis functions exist but are omitted here. The maximum energy overlap is 1 in this case, sinceB 1 is completely contained in C 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.3 Illustration of update of edge weights from (a) to (b). There exist 4 square basis functions (B i ) and 5 nodes (v i ) connected by edges, (v i ;v j ), with dierent weights. Assume that the initial node for clus- tering is v 1 . At the i th step, a cluster is formed byfv 1 ;v 2 g because (v 1 ;v 2 ) has the minimum weight of 1. Since the cluster changes, we need to update the weights if necessary. For example, the weight of an edge, (v 1 ;v 3 ), changes from 3 to 5 because v 3 and the clus- ter,fv 1 ;v 2 g is overlapped with the same basis function, B 1 , which increases the with respect to the cluster. . . . . . . . . . . . . . . 39 2.4 Joint optimization of dierent . By running the algorithm with Daubechies 4 basis with 2 nd level of decomposition, 256 sensors are separated into 16 clusters. Dierent choices of generate dierent results; as increases, the edge weights are more sensitive to the change of so that decreases at the cost of increasing D. . . . . . 41 2.5 Independent reconstruction vs. joint reconstruction when the spar- sifying basis is Haar basis with decomposition level of 5. For com- parison, the square-clustering scheme with two dierent number of clusters (4 and 16 clusters) are used. . . . . . . . . . . . . . . . . . 43 viii 2.6 Cost ratio to raw data gathering vs. SNR with dierent number of clusters and clustering schemes when the sparsifying basis is Haar basis with decomposition level of 5. . . . . . . . . . . . . . . . . . . 44 2.7 Performance comparison in terms of cost ratio with respect to raw data gathering vs. SNR for dierent basis functions and 64 SPT- based clusters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 2.8 Performance comparison in terms of cost ratio with respect to raw data gathering vs. SNR for 256 SPT-based clusters with other CS approaches with Haar basis with level of decomposition of 5. . . . . 46 2.9 vs. average number of hops per measurement. Each point corre- sponds to the result of Algorithm 1 with dierent. The points with red circle are chosen for the evaluation of transport cost and MSE in Fig. 2.10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 2.10 Transport cost ratio vs. MSE. The x-axis is the ratio of total trans- port cost of spatially-localized CS to the cost for raw data gathering without any manipulation. We compare performance of results by joint optimization with two dierent 's with that of SPT 64 in [48] 51 2.11 256 sensors in irregular positions and the corresponding graph. The communication range is set as the minimum distance that results in a connected graph. . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 2.12 Compressible WSN data in the GBT. The x-axis shows the indices of GBT coecients and the y-axis shows the cumulated sum of nor- malized energy of GBT basis functions. As shown in Fig. 2.11, the data is generated by a 2 nd AR model with the AR lter H(z) = 1 (1e jw 0z 1 )(1e jw 0z 1 ) , where = 0:99 and w 0 = 359 . . . . . . . . 55 2.13 256 sensors in irregular positions and the corresponding SPT. . . . 57 2.14 Total energy consumption vs. MSE. The x-axis is the total energy consumption in Joules and the y-axis is MSE. The curves are gener- ated by taking averages over 50 realizations of the sensor data. . . . 58 3.1 2-by-2 block examples: we have four pixels (nodes) in the block and assume that there exists one edge between pixels. (a) the edge separates those pixels into two partitions. Since the links between nodes in the graph are assumed not to go across the edge, we have two sub-graphs where 2 nodes are connected to each other by a link. (b) while an edge exists, the edge does not separate pixels into two partitions, generating only one graph. . . . . . . . . . . . . . . . . . 64 ix 3.2 RD performance comparison between i) H.264/AVC, ii) GBT [38], iii) CS-GBT-1, and iv) CS-GBT-2 for dierent sequences: (a) Ballet (b) Newspaper (c) Mobile. . . . . . . . . . . . . . . . . . . . . . . . 79 3.3 Perceptual improvement in Ballet sequence (QP 24): comparison of i) H.264/AVC, ii) GBT, and iii) CS-GBT-2 . . . . . . . . . . . . . . 80 3.4 The absolute dierence between the synthesized view with and with- out depth map compression in Mobile sequence (QP 24): comparison of i) H.264/AVC, ii) GBT, and iii) CS-GBT-2 . . . . . . . . . . . . 81 4.1 Basic motivation for UWB ranging system. After a pulse is trans- mitted, the observation time for the receiver can be divided into non-overlapped windows. Since the pulses and their re ections are narrow, we assume that the re ection localized in one of the win- dows. If a range is chosen, the receiver is able to measure a re ected signal during a specic window corresponding to the roundtrip time for that range. Within each cycle, the receiver only consumes power during a window, leading to low overall power consumption. . . . . 86 4.2 Illustration of the UWB sampling system with the parameters:N C and N W . Throughout N C cycles, one pulse is periodically trans- mitted at the beginning of each cycle which consists of N W non- overlapped windows. After taking summation of the re ections dur- ing N C cycles in analog domain, N S samples are collected in each window. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 4.3 Advantage of HDCS scheme. This shows comparison of two dierent samping schemes formulated as two matrices in (4.8). While Sequen- tial sampling scans a single window during 5 cycles, HDCS collects information from multiple windows during 5 cycles, which could lead to power savings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 4.4 Eect of averaging in the sequential sampling scheme [15]: (a) The red plot indicates the original signal (ground truth) and the blue in- dicates measured signal including the ground truth plus strong noise, SNR=21:5dB (b) Result of sequential sampling after averaging over 500 cycles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 4.5 Eect of averaging in a sequential sampling scheme [15]. SNR in- creases as more samples are averaged (i.e.,the number of cycles in- creases) with high noise level ( N = 30) and system parameters (N S = 16, N W = 155). Note that even after 500 averaging opera- tions, SNR is0:63dB in this example. . . . . . . . . . . . . . . . . 97 x 4.6 Cost ratio vs. MSE: For CS sampling schemes, cost is the total sampling time to collect M measurements. Since we x the number of cycles, N C , for every measurement as 500 in the simulation, the cost ratio is a ratio of the number of measurements to the dimension of signal, M=N. But, for sequential sampling scheme (noted 'SEQ' in the gure), we take N measurements with reduced N C . . . . . . 100 4.7 Performance comparison with respect to maximum mismatch of data support: (a) Fix sampling time ratio as 0:4 and compare performance at dierent noise levels. (b) Fix noise level as 30 and compare per- formance at dierent sampling time ratios. . . . . . . . . . . . . . . 105 4.8 Performance comparison with respect to F-measure: (a) Fix sam- pling time ratio as 0:4 and compare performance at dierent noise levels. (b) Fix noise level as 30 and compare performance at dierent sampling time ratios. . . . . . . . . . . . . . . . . . . . . . . . . . . 106 xi Abstract Traditional compressed sensing (CS) approaches have been focused on the goal of reducing the number of measurements while achieving satisfactory reconstruction. Reducing the number of measurements can directly lead to reductions in costs in some applications, e.g., the scanning time in fast magnetic resonance imaging (MRI) or the sampling rate in analog-to-information conversion. However, in some other applications, minimizing the number of measurements by itself does not provide a better solution in terms of system complexity due to additional application-driven constraints. In general, those constraints aect the choice of either measurement basis or sparsifying basis. For example, if the total cost of collecting measurements is a crucial factor as compared to the reconstruction accuracy, reducing the number of measurements does not guarantee better performance because the increase of measurement cost can exceed the gain achieved by the increase of the number of measurements. Thus, the design of an ecient measurement basis should consider the total cost for measurements as well as the reconstruction accuracy. Also, in coding applications where signals are rst captured and then compressed, better performance can be achieved by adaptively selecting a transform or sparsifying basis and then signaling the chosen transform to the decoder. For instance, for piecewise smooth signals, where sharp edges exist between smooth regions, edge- adaptive transforms can provide sparser representation at the cost of some overhead. xii Thus, the design of sparsifying basis should be optimized with respect to a given measurement basis, while the signaling overhead is minimized. These observations motivated us to investigate ecient design schemes for CS that can provide better reconstruction while minimizing the application-driven costs. In this thesis, we study the optimization of compressed sensing in three dierent applications, each of which imposes a dierent set of constraints: i) ecient data- gathering in wireless sensor networks (WSN), ii) depth map compression using a graph-based transform, and iii) fast target localization using a single-chip ultra- wideband (UWB) radar. Under these application-driven constraints, we study how to minimize application specic costs while minimizing the mutual coherence in order to achieve satisfactory reconstruction using CS. In sensor networks, we take explicitly into consideration the cost of each mea- surement (rather than minimizing the number of measurements), and optimize the measurement matrix that leads to energy ecient data-gathering in WSN. For depth map compression, the constraint to consider is the total bitrate, including both the number of bits for measurements and the bit overhead to code the edge map required for the construction of graph-based transform (GBT). To improve overall performance, we propose a greedy algorithm that selects for each block the GBT that minimizes a metric accounting for both the edge structure of the block and the characteristics of the measurement matrix. For fast localization of objects using a UWB ranging system, we design an ecient measurement system that is constructed using a low-density parity-check (LDPC) matrix, designed to satisfy hardware-driven constraints. To enhance performance, we propose a window-based reweighted L 1 minimization that outperforms other existing algorithms in high noise environments. xiii Chapter 1 Introduction Traditional compressed sensing (CS) approaches have been focused on reducing the number of measurements while achieving satisfactory reconstruction. To achieve satisfactory performance, two main conditions should be satised: (i) sparseness of signal to be reconstructed and (ii) incoherence between measurement (sensing) basis and sparsifying (sparsity-inducing) basis [12,23]. Under these conditions, a reduc- tion in the number of measurements can directly lead to lower complexity in some applications, in terms of application specic metrics such as the scanning time in fast magnetic resonance image (MRI) or the sampling rate in analog-to-information conversion [53,55]. However, in other applications additional constraints should be considered, because fewer measurements do not always provide better performances when the trade-o between complexity and reconstruction quality is considered. In this thesis, we investigate the optimization of compressed sensing under application-driven constraints. There could be many possible applications but we consider three specic examples: i) ecient data-gathering on wireless sensor net- work (constrained sensing matrix), ii) depth map compression using graph-based 1 transforms (adaptive sparsifying basis), and iii) fast target localization using single- chip UWB radar (hardware-driven constrains). Each application has its own con- straints to satisfy and costs to minimize. Each problem can be addressed with a design philosophy satisfying both application-driven and compressed sensing con- straints, while providing satisfactory reconstruction accuracy. While the results in this thesis are presented for specic applications, we expect that this design philosophy can be applied to various scenarios. Table 1.1 summarizes the choices of sensing and sparsifying basis for each ap- plication. The bases to be optimized with respect to the application specic con- straints are highlighted in the table. Table 1.2 describes the application specic constraints or costs to minimize for each application. Table 1.1: Choice of sensing / sparsifying basis for each application Application Sensing Basis () Sparsifying Basis ( ) WSN Sparse block-diagonal random Wavelet or GBT Depth map Hadamard GBT UWB radar Non-negative integer Spatial (Identity) In the WSN application, the transport cost to collect sensor information should be considered for ecient data-gathering of WSN data. Since the total cost of mea- surements in WSN is closely related to the construction of the sensing matrix, the challenge is to design a power-ecient sensing matrix, while preserving incoherence with respect to a given sparsifying basis. For depth map compression, the additional cost to consider is the total number of bits to encode both measurements and edge map, where the edge map is closely related to the construction of the sparsifying basis. In contrast to the WSN appli- cation, the sensing matrix is xed (the Hadamard matrix is chosen in our work) and 2 Table 1.2: Overview of application specic constraints / cost Application Constraints / Cost WSN Transport cost for measurement Depth map Bits for measurements and edge map UWB radar Scanning time under hardware-driven constraints the goal is to optimize the sparsifying basis (graph-based transform) by minimizing the number of bits required for the construction of the transform while achieving satisfactory reconstruction. For fast localization of objects using single-chip UWB ranging system, we need to design an ecient measurement system satisfying hardware-driven constraints: non-negative integer entries in the measurement (sensing) matrix, constant row- wise sum of non-zero entries in the matrix, and a unique structure characterized by Kronecker product. Also, a non-linear reconstruction technique for UWB signal needs to be designed for better reconstruction. Before discussing more details about our design for each application, we rst provide an overview of the basic mathematical results that provide guarantees for robust recovery of sparse signals in CS. 1.1 Compressed Sensing (CS) Compressed Sensing (CS) refers to a general class of systems where the goal is to recover an N-dimensional sample signal (x) having a sparse representation in one basis, the goal is to recover it from a small number of projections (smaller than N) onto a second basis that is incoherent with the rst [11,23]. 3 = M ! 1 measurement y ! a M " N N " 1 K-sparse signal K << M < N # N " N Figure 1.1: General matrix formulation of compressed sensing. More formally, if a signal, x 2 < N , is sparse in a given basis, (i.e., the sparsity inducing basis),x = a;jaj 0 = K, where K N, then theoretically we can reconstruct the original signal withM =O(K logN) measurements by nding the sparsest solution to an under-determined, or ill-conditioned, linear system of equations,y = x = a =Ua; whereU is known as the holographic basis as shown in Fig. 1.1. Reconstruction is possible by solving the convex unconstrained optimization problem: min a 1 2 kyUak 2 2 + kak 1 : (1.1) Multiple results have been derived to establish under what conditions x can be successfully recovered (refer to [1, 6] for more details). Here we summarize some of these results, which will be useful in deriving some of our approaches. As an example, a unique solution can be obtained when and satisfy the restricted isometry property (RIP) [10]. Denition 1.1.1. KRIP [10] An MN matrixU = has the K-restricted isometry property (K-RIP) with constant K 2 [0; 1] if, given an integer K, we have that: (1 K )kak 2 2 kUak 2 2 (1 + K )kak 2 2 4 holds for any K-sparse vectora: The denition of K-RIP indicates that all submatrices ofU of size MK are close to an isometry, and thus distance preserving with a small margin, K . While the design ofU satisfying the K-RIP is an NP-Complete problem, random sensing matrices, , lead toU (= ) satisfying K-RIP with high probability. Random matrices whose entries are i.i.d. Gaussian, Bernoulli, or more generally subgaussian, universally satisfy K-RIP with any choice of orthonormal basis matrix, , leading to perfect reconstruction with high probability when M =O(K log(N=K)) [10,13, 23]. Aside from the random matrix approach, the general problem of nding opti- mal matrices satisfying K-RIP is an NP-Complete problem. Thus, mutual coher- ence [8, 72] has been proposed as an alternative measure to characterize matrices that are likely to meet the RIP condition. Mutual coherence serves as a rough characterization of the degree of similarity between the sparsity and measurement systems and is dened as follows: Denition 1.1.2. Mutual coherence [8] For two matrices with normalized row vectors() and column vectors( ), mutual coherence is dened as (U) = max i;j jU(i;j)j = max i;j j(i) (j)j, where(i) is a row vector of and (j) is a column vector of . Note that (U)2 [0; 1] For to be close to its minimum value, each of the measurement vectors must be spread out in the domain. A small value of (U) indicates that and are incoherent with each other, i.e., no element of one basis ( ) has a sparse representation in terms of the other basis (). In the case when both and are orthogonal, the minimum number of mea- surements for perfect reconstruction can be computed as follows. 5 Theorem 1.1.3. Minimum number of measurements [8] LetU = be an NN orthogonal matrix (U T U =I) withjU(i;j)j(U). For a given signal x = a, suppose that a is supported on a xed (arbitrary) set T with K non-zero entries, and assume that measurements,y = x, are retrieved by random downsampling. Then, L 1 optimization can recover x exactly with high probability if the number of measurements M satises M =O(K 2 (U)N logN) (1.2) The bound on the number of measurements provided by Theorem 1.1.3 de- creases as and become increasingly incoherent, i.e., the minimum number of measurements for perfect reconstruction is determined by for given K and N. In this thesis, we investigate how to reduce while satisfying application specic constraints. In the applications of data-gathering in WSN and depth map com- pression, the orthogonal bases, and , are studied to nd a better solution with measurements obtained by a separate downsampling. But for the UWB applica- tion, we investigate how to nd the actual (rectangular) sensing basis without the separate downsampling for a given =I. 1.2 Ecient data-gathering in WSN In sensor networks, energy ecient data manipulation and transmission is very important for data gathering, due to signicant power constraints on the sensors. This constraint has motivated the study of joint routing and compression for power- ecient data gathering of locally correlated sensor network data. Most of the early 6 works were theoretical in nature and, while providing important insights, ignored the practical details of how compression is to be achieved [19,62,80]. More recently, it has been shown how practical compression schemes such as distributed wavelets can be adapted to work eciently with various routing strategies [63, 75, 76]. The existing transform-based techniques can reduce the number of bits to be transmitted to the sink, thus achieving overall power savings. These transform techniques are essentially critically sampled approaches, so that their cost of gathering scales up with the number of sensors, which could be undesirable when large deployments are considered. Although the number of measurements in CS, M, is also proportional to the dimension of signal,N, the rate of increase is lower than that for the critically sampled approaches because it is a logarithmic function of N, M =O(K logN) . CS has been considered as a potential alternative in this context, as the number of measurements required depends on the characteristics (sparseness) of the signal and also the dimension of the signal [8, 11, 23]. With recent interest in WSN, re- searchers have proposed various ways to apply CS to WSN in order to reduce the gathering costs [26, 52, 60, 69]. But those approaches focused on the total number of measurements rather than the total transport cost of measurements. In [83], it was shown that CS could also operate using sparse random projections (SRP) but this work does not consider transport cost to collect measurements in a multi-hop network. In [47, 68], the potential benets of CS for sensor network applications have been recognized but signicant obstacles remain for it to become competi- tive with more established (e.g., transform-based) data gathering and compression techniques. A primary reason is that CS theoretical developments have focused on minimizing the number of measurements (i.e., the number of samples captured), rather than on minimizing the cost of each measurement. 7 In the WSN data-gathering application, the key observation is that an ecient measurement system needs to be both sparse (few sensors contribute samples to each measurement) and spatially-localized (the sensors that contribute to each mea- surement are close to each other) in order to be competitive in terms of transport cost. In [48], we introduced the use of a cluster-based measurement system for CS; each measurement was obtained by a linear combination of samples captured within a single cluster, and clusters were selected in order to contain sensor nodes that are close to each other. Based on the clustering scheme, we extend this approach by analyzing how the choice of spatial clusters aects the reconstruction accuracy, for a given spatially-localized sparsity basis. Moreover, we propose novel cluster- ing techniques that take into consideration both transport cost and reconstruction quality [45]. More specically, we have two main contributions in this thesis. First, we intro- duce the concept of maximum energy overlap between clusters and basis functions, which we denote. If basis functions and clusters have similar spatial localization, most of the energy of a given basis function is likely to be concentrated in a few clusters, which means that only measurements taken from those clusters are likely to contribute to reconstructing a signal that contains that specic basis function. Since the measurement system is not aware a priori of where signals will be local- ized, it needs to gather enough measurements to reconstruct signals with any spatial localization, and since each cluster overlaps only a few such basis functions, it will need to have a larger number of measurements. Conversely, for the same number of measurements, as the energy of the basis functions is more evenly distributed over clusters (smaller ), this could lead to better reconstruction. To verify this, 8 we provide a proof that the minimum number of measurements for perfect recon- struction is proportional to. Therefore, for given basis functions, we can estimate performance of dierent clustering schemes by computing . Second, we propose a centralized iterative algorithm with a design parameter, , for joint optimization of the energy overlap and distance between nodes in each cluster. A joint optimization is required because there exists a tradeo between and the distance. To achieve smaller (which leads to fewer required measure- ments for a certain level of reconstruction accuracy), each basis function should be overlapped with more clusters. This means that the nodes within a cluster tend to be farther from each other because basis functions are localized in space. Since total transport cost is a function of the number of measurements and transport cost per measurement, the trade-o allows reducing the number of measurements at the cost of increasing transport cost per measurement. By joint optimization using an appropriately chosen, we can achieve a good trade-o between transport cost and reconstruction accuracy. Compared with another CS technique (which showed the best performance in [48]), our simulation results show that our proposed approach with joint optimization achieves almost 50% reduction in transport costs at the same level of mean squared error in the reconstruction. Also, we extend our work to a practical situation where the sensors are randomly deployed. In order to sparsify the sensor data measured at randomly positioned sensors, we propose to use graph-based transform (GBT) that can achieve a sparse representation of the sensor data [78]. We rst represent the random topology as a graph then construct a sparsifying basis by placing eigenvectors of the Laplacian matrix of the graph. With the GBT, we propose a heuristic design of the data- gathering that the aggregations happen at the sensors with fewer neighbors in the graph than a empirically chosen threshold. In our simulations, compared to 9 other methods, our proposed approach shows better performance in the total power consumption at the same level of MSE in the reconstruction. 1.3 Depth map compression for 3D-TV In the problem of depth map compression for 3D-TV, the performance gain with standard image or video coding techniques is limited because the depth map has dierent characteristics from other natural images or standard video sequences. We can characterize a depth map as i) a gray-scale video, ii) containing piecewise smooth signals, and iii) with sharp edges where depth changes across dierent objects or around objects-to-background (background-to-objects) transitions. It has been shown that edges play a crucial role in the rendered views; errors in edge information translate into geometry errors in the interpolated view, which leads to signicant quality degradation [39,40]. These characteristics have motivated edge-adaptive coding tools that apply transforms within smooth regions without ltering across edges. Examples include edge-aware DCTs [31,66,89,90] or wavelets [7,21,65,67,79]. The edge-aware DCT approaches require that the transform block size be adaptive to edge location or the pixel values be rearranged in order to be aligned with the dominant direction of edges in a block. The wavelet-based approaches are not appropriate to block based coding architectures, which are dominant in image and video coding stan- dards such as JPEG and H.264/AVC. CS based methods have also been proposed recently [25,73]. These methods sparsify depth map data using the DCT, which has been widely used for block-based image and video compression, but is inecient for coding blocks containing arbitrary shaped edges. This motivates us to consider similar CS architectures but using better sparsifying basis. 10 To improve the eciency of depth map coding, we propose a new CS framework based on a graph signal representation [46]. The graph based transform (GBT) has been shown to be better for sparse representation of depth map data, especially when arbitrary edges such as diagonal or mixture of horizontal and vertical edges exist in the block [38, 74]. The basic idea is that for each block an edge structure can be identied, and from it an edge adaptive graph transform can be derived. We rst investigate the tradeo between the cost (related to the overhead required to encode edge map) and the reconstruction quality (related to the incoherence between GBT and Hadamard sensing matrix). As more edges are used as a part of the transform, the coherence decreases, so that we can achieve better reconstruction with a xed number of measurements, but the amount of overhead information required to represent the edge map also increases. To nd a better edge map, we propose a greedy algorithm to minimize a cost function that takes into account the mutual coherence. Since computing the mu- tual coherence at every iteration of the algorithm is computationally expensive, we approximate the average mutual coherence between the Hadamard sensing matrix and GBT with a technique that does not require the explicit construction of GBT. At every iteration of the algorithm the cost function can be computed using only the edge map and Hadamard sensing matrix. In our proposed algorithm, the com- plicated construction of the GBT is only required once the optimized edge map is found. Also, since our approach applies to blocks of arbitrary size, we apply our block-based approach to H.264/AVC reference software JM17.1 and compare the performance to the case where H.264/AVC is used for depth map compression. With the proposed method, a 3.7 dB PSNR gain or 38% bitrate saving is observed on average when applied to three depth map sequences. 11 1.4 Fast localization using UWB radar For fast localization of objects using an ultra-wideband (UWB) ranging system, we propose a CS framework based on a recently developed hardware [15]. This hardware utilizes a ranging technique by sending multiple pulses, then averaging the received pulses in short time intervals (windows), each corresponding to a cer- tain roundtrip time of the re ected pulse. Assuming the environment is relatively static, the receiver can localize an object at a specic distance by selecting a cor- responding window and determining if the window contains re ected signal. The averaging within a chosen window provides robustness to noise. It also requires less power consumption, because power is only consumed during the measurement window, which can represent a small percentage of time. However, a limitation of this scheme comes from sequential sampling, i.e., candidate object locations have to be probed in sequence, so that the time required to locate an object will be propor- tional to the number of measurement windows. In this work, we propose techniques that can signicantly reduce the scanning time, with no increase in overall power consumption, and can also operate in high noise environments. The key observa- tion is that in many situations the number of objects that can be localized in the environment is small relative to the number of locations that are probed. In a static environment, this allows us to probe several locations simultaneously, so that each measurement combines re ections at several distances. Processing can then be used to extract the actual position information from the combined observations. In the context of radar applications, many CS-based approaches have been pro- posed that exploit the sparse structure of UWB signals [12,23]. In [2], it was shown that the received signal can be digitized at a rate much lower than the Nyquist rate, but the high noise case scenarios and total power consumption constraints that we 12 mainly focus on in this work are not considered. Also, a precise CS-UWB position- ing system was proposed that exploited the redundancy of UWB signal captured at multiple receivers in order to localize a transmitter [84,85], but the ADC rate is still higher than what can be achieved with the UWB hardware platform we build upon [15]. As a CS approach tightly coupled to hardware, a Random-Modulation Pre-Integrator (RMPI) was proposed to achieve low-rate ADC by random modula- tion in analog domain [86{88] but the random modulation of signals contaminated by powerful noise in analog domain does not provide robust signal recovery. To design a system that is robust to high noise and consumes less power while providing reliable localization, we propose a CS technique [44] tightly coupled to the capabilities of recently developed hardware [15]. First, we observe that the UWB signal is sparse if few object exists since the UWB signal is highly localized in time. Combined with the UWB ranging system, this leads to a special structure where sparse non-zero entries are clustered into a few groups (windows). The number of groups is equal to the number of objects in the region of interest. Second, we design an ecient measurement system subject to several constraints imposed by the hardware. The constraints include (i) non-negative integer entries in the sensing matrix (ii) constant row-wise sum of entries in the matrix (iii) non-zero entries of each row can exist only at the positions with a constant shift, which leads to unique structure characterized by a Kronecker product. Under these constraints, we construct a sensing matrix by using a low-density parity-check (LDPC) matrix that has recently been shown to be a good measurement system in [20,50]. Third, to enhance the localization performance, we propose a window-based reweighted L 1 minimization that shows good performance for aforementioned signal model and measurement system. In our simulations, we compare our proposed method with other existing reconstruction algorithms with respect to several metrics evaluating 13 localization performance. Our simulation results show that our proposed method can achieve reliable target-localization while using only 40% of the sampling time required by the corresponding sequential scanning scheme, even in a highly-noisy environment. The rest of the dissertation is organized as follows. The ecient data-gathering for WSN using spatially-localized CS is described in Chapter 2. The depth map compression exploiting optimized GBT with CS-related constraint is proposed in Chapter 3. The fast localization of UWB radar by CS is presented in Chapter 4. Conclusions and future work are discussed in Chapter 5. 14 Chapter 2 Spatially-Localized Compressed Sensing for Ecient Data-gathering in Multi-Hop Sensor Networks In sensor networks, energy ecient data manipulation / transmission is very im- portant for data gathering, due to signicant power constraints on the sensors. CS has been proposed as a potential solution because it requires capturing a smaller number of samples for successful reconstruction of sparse data. Traditional CS does not take explicitly into consideration the cost of each measurement (it sim- ply tries to minimize the number of measurements), and this ignores the need to transport measurements over the sensor network. In this chapter, we study CS approaches for sensor networks that are spatially-localized, thus reducing the cost of data gathering. In particular, we study the reconstruction accuracy properties of a new distributed measurement system that constructs measurements within spatially-localized clusters. We rst introduce the concept of maximum energy overlap between clusters and basis functions (), and show that can be used to estimate the minimum number of measurements required for accurate reconstruc- tion. Based on this metric, we propose a centralized iterative algorithm for joint 15 optimization of the energy overlap and the distance between sensors in each cluster. Our simulation results show that we can achieve signicant savings in transport cost with small reconstruction error. We also extend our work to the case of sensors placed at random positions. We show that smooth sensor data can be sparsied using a GBT based on the network topology and propose a heuristic design of the sensing matrix that takes into account the number of neighbors of the sensors in the graph. In our simulations, compared to other methods, our proposed approach shows better performance in the total power consumption at the same level of MSE in the reconstruction. The rest of this chapter is organized as follows. We rst introduce previous work related to WSN data gathering in Section 2.1 then formulate it in a CS framework in Section 2.2. Based on this problem formulation, we propose a spatially localized projection scheme associated with an energy overlap between projection basis and sparsifying basis in Section 2.3. Then we provide a mathematical proof that the energy overlap is a good metric to determine the number of measurements for perfect reconstruction in Section 2.4. Techniques to nd a power-ecient clustering scheme are discussed in Section 2.5 and veried by our simulations in Section 2.6. We also extend our work to the case where sensors are placed at random positions and provide preliminary results in Section 2.7. 2.1 Related Work Joint routing and compression has been studied for ecient data gathering of lo- cally correlated sensor network data. Most of the early works were theoretical in nature and, while providing important insights, ignored the practical details of how compression is to be achieved [19, 62, 80]. More recently, it has been shown how 16 practical compression schemes such as distributed wavelets can be adapted to work eciently with various routing strategies [16,63,75,76]. Existing transform-based techniques, including wavelet based approaches [16,75, 76,81] and the distributed KLT [33], can reduce the number of bits to be transmitted to the sink thus achieving overall power savings. These transform techniques are essentially critically sampled approaches, so that their cost of gathering scales up with the number of sensors, which could be undesirable when large deployments are considered. CS has been considered as a potential alternative in this context, as the number of samples required (i.e., number of sensors that need to transmit data), depends on the characteristics (sparseness) of the signal [8,11,23]. In addition, CS is also poten- tially attractive for wireless sensor networks because most computations take place at the decoder (sink), rather than at the encoder (sensors), and thus sensors with minimal computational power can eciently encode data. Also, CS potentially in- creases the security of data transmission because partial measurements intercepted by malicious users are not sucient for decoding (i.e., global information about all measurements is needed). With recent interest in WSN, researchers have proposed various ways to apply CS to WSN. In [69], the authors studied a scenario where ultimately every sen- sor has an approximation of the network data by using gossip algorithms. But, this work assumes a single-hop network communication and does not consider the transmission cost of pre-distribution phase (gossiping phase). Also, an innovative CS approach was proposed that makes it possible to gather sensor data without any inter-sensor communication, based on a distributed compressed sensing ap- proach [4, 26]. But, if a signal does not t the corresponding signal model (joint 17 sparsity model), overall performance may not be good. More specically, the as- sumption that there exists common information such as common data supports (i.e., the locations of the nonzero coecients) among sensor measurements is unrealistic. Some other work exploits the spatial correlations among data measured in each sensor with CS. In [83], it was shown that CS could also operate using sparse random projections (SRP). The implication for a single-hop sensor network is that good eld reconstruction can be obtained even when only a small, randomly chosen fraction of sensors report their measurements. But this work does not consider the transport cost required to collect measurements in a multi-hop network. In [52], CS was proposed to detect abnormal sensor data or uncorrelated data in adjacent neighboring sensors, but the performance of this approach in terms of reconstructing all sensor data is likely to be limited. For the large scale networks, inter- ow network coding was combined to address high link failure in sensor networks [60]. But this approach results in much larger delay in the measurement transmission as compared to other CS approaches. In [47,68], the potential benets of CS for sensor network applications have been recognized but signicant obstacles remain for it to become competitive with more established (e.g., transform-based) data gathering and compression techniques. A primary reason is that CS theoretical developments have focused on minimizing the number of measurements (i.e., the number of samples captured), rather than on minimizing the cost of each measurement. Thus, in this work, we propose to optimize CS to minimize the transport cost of measurements while providing sat- isfactory reconstruction accuracy [45]. 18 2.2 Problem Formulation In this work, we assume thatx2< N is a vector containing measurements obtained by N sensors in a 2D region at a given time and x is K-sparse in a given sparsi- fying basis ~ . The N-sample signal (x) can be recovered from M measurements or projections (M < N) onto a sensing (measurement) basis, if and are incoherent [11,23] as explained in Chapter 1. For ecient data-gathering from sen- sors spread over space to the sink located at the center of the network, we consider distributed measurement strategies that are both sparse and spatially localized. 2.2.1 Low-cost sparse projection based on clustering In order to design distributed measurements strategies that are both sparse and spatially localized, we propose dividing the network into clusters of adjacent nodes and forcing projections to be obtained only from nodes within a cluster. As an ex- ample, we rst consider two simple clustering approaches. Assume that all clusters contain the same number of nodes. When N c clusters are used, each cluster will contain N Nc nodes. In \square clustering", the network is partitioned into a certain number of equal-size square regions. Alternatively, in \SPT-based clustering", we rst construct the shortest path tree (SPT) then, based on that, we iteratively construct clusters from leaf nodes to the sink. These clustering schemes are rst applied to equally spaced sensors on a 2D regular grid, then extended to the sensors positioned at irregular positions at the end of this chapter. Any chosen clustering scheme can be represented in CS terms by generating the corresponding measurement matrix, , and using it to reconstruct the original signal. As shown in Fig. 2.1, each row of represents the aggregation corresponding to one measurement: we place non-zero (e.g., random) coecients in the positions 19 corresponding to sensors that provide their data for a specic measurement and the other positions are set to zero. Thus, the sparsity of a particular measurement in depends on the number of active nodes participating in this aggregation. We expect this approach to be more ecient than traditional CS because the sensors in each cluster are relatively close to each other (leading to a spatially-localized projection) and the number of sensors in each cluster is only a small fraction of the number of sensors in the network (leading to a sparse projection). S1 S6 S2 S5 !"#$ S7 S9 S8 S10 S9 = y ! x 1 2 3 4 5 6 7 8 9 10 Figure 2.1: Link between CS measurements and data aggregation in WSN In order to make the design simpler, we consider non-overlapped clusters, which leads to a block-diagonal structure for . Note that recent work [22,32], seeking to achieve fast CS computation, has also proposed measurement matrices with a block- diagonal structure, with results comparable to those of dense random projections. Our work, however, is motivated by achieving spatially localized projections so that our choice of block-diagonal structure will be constrained by the relative positions of the sensors (each block corresponds to a cluster). 20 2.2.2 Sparsity-inducing basis and cluster selection While it is clear that localized gathering leads to lower communication costs, it is not obvious how it may impact reconstruction quality. Thus, an important goal of this work is to study the interaction between localized gathering and reconstruction. A key observation is that in order to achieve both ecient routing and adequate reconstruction accuracy, the structure of the sparsity-inducing basis should be con- sidered. As an example, consider the case where signals captured by the sensor network can be represented by a \global" basis, e.g., DCT, where each basis spans all the sensors in the network. Then the optimally incoherent measurement matrix will be the identity matrix, I, and thus a good measurement strategy is simply to sample K logN randomly chosen sensors and then forward each measurement directly to the sink (no aggregation). Alternatively, for a completely localized basis, e.g., =I, a dense projection may be best for reconstruction accuracy. However, once the transport costs have been taken into account, the best solution is to just transmit the non-zero samples to the sink via the SPT. In other words, even if CS theory suggests a given type of measurements (e.g., dense projection for the =I case), applying these directly may not lead to an ecient routing and therefore ecient distributed CS may not be achievable. In this work, we rst consider intermediate cases, in particular those where localized bases with dierent spatial resolutions are considered (e.g., wavelets). In [8] it was shown that a partial Fourier measurement matrix is incoherent with wavelet bases at ne scales. However, such a dense projection is not suitable for low- cost data gathering for the reasons discussed above. Next we explore appropriate spatially-localized gathering for data that can be represented in localized bases such as wavelets. 21 2.3 Ecient clustering for spatially-localized CS 2.3.1 Independent vs. Joint reconstruction To help us understand how to select a clustering scheme that is appropriate for CS, we rst compare two types of reconstruction: independent reconstruction and joint reconstruction. Suppose that we construct a series of clusters of nodes and collect a certain number of local measurements from each cluster. As an example, consider 2 clusters and the corresponding localized projections, 1 and 2 , with a given sparsifying basis, : U = = 2 6 4 1 0 0 2 3 7 5 2 6 4 1 2 3 4 3 7 5 ; whereU = 2 6 4 U 1 U 2 3 7 5 = 2 6 4 1 1 1 2 2 3 2 4 3 7 5 : (2.1) Since the two clusters do not overlap with each other, the measurement matrix, in (2.1), has a block diagonal structure. For joint reconstruction, the original sparsifying basis, , is employed. But, for independent reconstruction, data in the rst cluster are reconstructed with partial basis functions, 1 and 2 , and those in the second cluster are with 3 and 4 thus, when N c clusters are involved, independent reconstruction should be performed N c times, once for each cluster. Joint reconstruction is expected to outperform independent reconstruction be- cause measurements collected from a cluster can also convey information about data in other clusters, since basis functions that overlap with more than one clus- ters can be identied with measurements from those clusters. For example, the measurements from the rst cluster can help to reconstruct the signal in the second cluster if there exists a basis function overlapped with both clusters. 22 Based on this observation the key intuition in our work is that clustering schemes that have overlap with more basis functions should be preferred. The degree of over- lap between basis functions and clusters can be measured in many dierent ways. In this work, we propose to use as a metric the energy of the basis functions cap- tured by each cluster. We will discuss the energy overlap in more detail in Section 2.3.3 and verify that joint reconstruction outperforms independent reconstruction through our simulations. 2.3.2 Spatially-Localized Projections in CS An aggregation path in a sensor network can be represented by a row of the measure- ment matrix, . We place non-zero (possibly random) coecients in the positions corresponding to active sensors that provide their data for a specic measurement, while the other positions are set to zero, which means that the sparsity of a partic- ular measurement in the matrix depends on the number of nodes participating in each aggregation. To expressM measurements algebraically, we consider a down-sampling matrix, Q, that chooses M measurements out of N with equal probability. This can be expressed as: y M1 =Q MN NN x N1 (2.2) With respect to standard CS approaches, the actual (rectangular) measurement matrix isQ but, in this work, we explicitly separate the actual measurements as a combination of projections onto an orthogonal matrix, , followed by downsam- pling,Q. This will be needed for our proof in Section 2.4.3. Similarly, the aggregations within a cluster can be expressed as a set of rows of . Since N c non-overlapped clusters are considered, we can express the measurement 23 system as a block diagonal matrix that contains N c square sub-matrices, i on its diagonal, so that i represents the aggregation scheme used for the i th cluster in the network. Therefore, the dimension of i is determined by the number of sensors contained in the i th cluster. Let us denex as an original input signal andx i p as a vector of samples measured in thei th cluster. To associate i withx i p , we introduce a permutation matrix,P , whose output isx p =[x 1 p :::x i p :::x Nc p ] T =Px. Thus, by multiplying with the output ofPx, we have y =QPx; where = 2 6 6 6 6 6 6 6 4 1 2 . . . Nc 3 7 7 7 7 7 7 7 5 : (2.3) The square block-diagonal matrix, , is the measurement matrix and the clusters are uniquely dened by,P , the clustering matrix. A dierent permutationP can be dened for each clustering method, so that in any clustering method we can write the projection as in (2.3). Note that P by itself does not determine clustering, what produces clustering is the block diagonal matrix, . We now discuss how the clustering matrix is related to the sparsifying basis matrix. Since aK-sparse signal is represented byK non-zero coecients in a given basis ~ ,x = ~ a, the measurements are obtained by y =QPx =Q P ~ a =Q a; (2.4) 24 where we dene =P ~ as a permutation of ~ After permutation by the cluster- ing matrix, the measurement matrices for each cluster, i , are correctly associated with data measured by sensors in the corresponding clusters,x i p . In summary, a cluster-based measurement system leads to a block diagonal mea- surement matrix with appropriate permutation,P , related to the physical positions of sensors. Note that recent work [22,32], seeking to achieve fast CS computation, has also proposed measurement matrices with a block-diagonal structure, with re- sults comparable to those of dense random projections. Our work, however, is motivated by goal of achieving spatially localized projections, so that our choice of block-diagonal structure will be constrained by the deterministic positions of the sensors, instead of using a uniformly random permutation as proposed in [22]. 2.3.3 Average Energy Overlap To characterize the distribution of energy of the basis functions with respect to the clusters, we present a metric,E oa , and an analysis of the worst-case scenario. Given =P ~ , for a givenP , suppose that N c is the number of clusters and C i is a set of nodes contained in the i th cluster. The energy overlap between the i th cluster and the j th basis vector, E o (i;j), can be dened as: E o (i;j) = X k2C i (j;k) 2 : (2.5) Then, the average energy overlap per overlapped basis can be a good indicator of distribution of energy of a basis function across multiple clusters. Since the basis 25 functions are normalized, the maximum overlap is 1. For each cluster, E oa (i) is computed as E oa (i) = 1 N N X j=1 E o (i;j);8i2f1; 2; ;N c g; (2.6) E oa (i) is the average energy localized in the i th cluster. Intuitively, this metric shows how much the average energy of the basis functions is localized in each cluster. Thus, as energy of the basis functions is more evenly distributed over the clusters,E oa decreases, which leads to better reconstruction performance with joint reconstruction. While this metric is a good indicator of the average distribution of basis functions, our results indicated that the worst case overlap may be better at predicting reconstruction performance. It would be useful to have a metric to determine the number of projections required from each local cluster in order to achieve a certain level of reconstruction performance. We rst dene what the worst-case is, then propose a method to characterize the `worst-case scenario' performance. If the global sparsity is K, the worst case scenario is when all K basis vectors supporting data are completely contained in a single cluster. In this case, O(K) projections would be required from each cluster. There are two reasons for this. First, since the identity of the cluster where bases are concentrated is not known a priori, it is not possible to concentrate projections within that cluster without measuring information in the others. Second, projections from other clusters not overlapped with those basis vectors do not contribute to reconstruction performance as much as projections from the overlapped cluster. Thus, instead of the average energy overlap, we consider the maximum energy overlap as a metric of reconstruction quality in a spatially localized measurement system. The derivation of the number of measurements for 26 perfect reconstruction in terms of the maximum energy overlap will be discussed in Section 2.4. 2.3.4 Maximum Energy Overlap To quantify the distribution of energy overlap over clusters, we dene the maximum energy overlap for the basis , ( ), as follows: Maximum energy overlap, ( ) ( ) =(P ~ ) = max i;j X l 2 i (l;j); ( )2 [0; 1] represents the maximum amount of energy of a basis functions captured by a single cluster. The matrix i is the rectangular sub-matrix corresponding to the i th cluster. For example, as depicted in Fig. 2.2 (b), we rst compute the sum of squared entries (colored cells) for each pair of (B i ,C j ); For B 1 , energy overlap is 1 withC 1 and zero with the other clusters. Then we take the maximum value of the computed sums. If is 1 (maximum value), it indicates that there exists at least one basis function completely covered by a cluster in space such as the overlap between B 1 and C 1 in Fig. 2.2 (a). In contrast, small means that most basis functions are overlapped with multiple clusters in space. Intuitively, measurements taken from a cluster can also convey information about data in other clusters when basis functions overlap with more than one cluster, e.g., B 2 in Fig. 2.2 (a) can be identied with measurements from those clusters (C 1 and C 2 ). This is one of the reasons why, as will be shown experimen- tally in Section 2.6.1, joint reconstruction outperforms independent reconstruction. 27 C1 C2 C3 C4 (a) overlap in spatial domain C1 C2 C3 C4 B1 B2 B3 (b) overlap in sparsifying basis Figure 2.2: (a) Illustration of energy overlap for a 4 4 grid network of 16 sensors. The network is divided into 4 square clusters and 3 basis functions are considered. The bases have spatial resolution of 1, 3 and 9 sensors, for B 1 , B 2 and B 3 , respec- tively. (b) Permuted sparsifying basis matrix, =P ~ . The entries of each basis function (column vector of ) is lled with colors if non-zero coecients exist and white otherwise. Note that 13 more basis functions exist but are omitted here. The maximum energy overlap is 1 in this case, since B 1 is completely contained in C 1 . If a specic basis function is completely contained within a cluster, e.g. B 1 , then only measurements from C 1 are likely to contribute to reconstructing a signal that contains B 1 . Thus, as discussed in Section 2.3.3, the worst case scenario where all K basis functions supporting data are completely contained in a single cluster, e.g.,B 1 in Fig. 2.2 (a) signicantly increases the number of measurements required to achieve a good reconstruction. Thus, as basis functions are overlapped with more clusters, we will have a potentially higher chance to reconstruct the signal correctly. To further improve localized CS performance, a clustering scheme that minimizes overlap should be chosen. In the next section, we will show how aects reconstruction accuracy and determines the minimum number of measurements for perfect reconstruction. 28 2.4 Theoretical Result As shown in previous sections, the maximum energy overlap, , is determined for given sparsifying basis, ~ , block-diagonal matrix, , and a clustering scheme,P . Here, we show how aects reconstruction accuracy by deriving the minimum number of measurements for perfect reconstruction as a function of . 2.4.1 Denitions and Assumptions We assume that N c non-overlapped clusters contain the same number of sensors. Thus, has N c square sub-matrices with size of N=N c N=N c along its diagonal. Therefore, if each sub-matrix, i , is orthogonal, then is also orthogonal. Based on the problem formulation, our main result is based on three assumptions. First, the sparsifying basis, ~ , is orthogonal. Second, the maximum absolute value of entries in the sparsifying basis is bounded, max i;j j ~ (i;j)j 1= p logN, so that we do not consider degenerate cases, such as the canonical basis in spatial domain ( ~ =I). This assumption is satised by bases such as the DCT or the Daubechies wavelets with a sucient number of levels of decomposition [22]. Lastly, the sub- measurement matrix, i , is an orthogonalized i.i.d. Gaussian matrix: i (j;k) N(0;N c =N). Thus T =I N since the clusters are disjoint. In order to evaluate the coherence between the measurement matrix, , and the permuted sparsifying basis matrix, (=P ~ ), we dene an NN matrixU: 29 U NN = 2 6 6 6 6 6 6 6 4 1 2 . . . Nc 3 7 7 7 7 7 7 7 5 2 6 6 6 6 6 6 6 4 1 2 . . . Nc 3 7 7 7 7 7 7 7 5 ; = 2 6 6 6 6 4 1 1 . . . Nc Nc 3 7 7 7 7 5 ; (2.7) = 2 6 6 6 6 4 U 1 . . . U Nc 3 7 7 7 7 5 : (2.8) By assumption, all square sub-matrices i have the same size, and therefore all theU i also have the same size. But our work can be extended to the case that the clusters have dierent sizes. Moreover U is an orthogonal matrix because and ~ are orthogonal by assumption and the clustering matrix, P , is a permutation matrix, so that: U T U = (P ~ ) T (P ~ ) = ( ~ T P T T )(P ~ ) =I N (2.9) The mutual coherence, , can be computed using Denition 1.1.2 then we can apply it to Theorem 1.1.3 because U is orthogonal. This helps us to derive the minimum number of measurements in terms of the number of clusters,N c , sparsity, K, and the number of sensors, N. 30 2.4.2 Main Result To obtain a bound on the number of measurements, we rst derive an asymptotic upper bound on mutual coherence. With this bound, we can attain the minimum number of measurements for perfect reconstruction by using Theorem 1.1.3 if the aforementioned conditions are satised. Proposition 2.4.1. If all the sub-measurement matrices, i , are orthogonalized i.i.d. Gaussian, N(0; Nc N ), and the orthogonal sparsifying basis, ~ , and the cluster- ing matrix,P , are known a priori, then (U) is asymptotically bounded by Pr " (U)O r N c N logN # = 1O 1 N : (2.10) Proposition 2.4.1 quanties the probability that coherence exceeds a certain bound. The probability that coherence is not bounded by O q Nc N logN is close to 0 as N increases. For the proof, the main technical tools are large deviation in- equalities of sum of independent random variables. Specically, the result is derived from Bernstein's deviation inequality [51] and a union bound for the supremum of a random process. Refer to Section 2.4.3 for details of proof. With aforementioned assumptions and Proposition 2.4.1, we can characterize the impact of on reconstruction accuracy in terms of the number of measurements. 31 Theorem 2.4.2. For a given signal x = a withjaj 0 = K and a clustering scheme with parameter 2 [0; 1], the L 1 optimization can recover x exactly with high probability if the number of measurements M satises M =O(KN c log 2 N): (2.11) Based on the bound on coherence of Proposition 2.4.1, we can derive Theorem 2.4.2, the minimum number of measurements for perfect reconstruction, by using Theorem 1.1.3 and the fact thatU is orthogonal by (2.9). Note that the bound in Theorem 2.4.2 is nearly identical to the bound obtained for the SRM case in [22], where clusters are dened based on random permutation matrices (i.e.,O(KN c log 2 N) vs. O(KN c log 2 N)). Our result shows that random permutation matrices can be further optimized to have smaller , which leads to smaller number of measurements for the same level of reconstruction accuracy. In general, the number of measurements is proportional to the maximum energy overlap because basis functions with more uniformly distributed energy increase the probability of correct reconstruction. Also, the number of measurements is proportional to the number of clusters,N c . This implies that a sparser measurement matrix (largerN c ) requires more measurements for the same level of reconstruction, as also shown in previous work [22,83]. Note that there exists a tradeo between and the distance between nodes belonging to the same cluster. A decrease in can be achieved when each basis function overlaps a larger number of clusters. Since basis functions are spatially localized, that means that nodes within a cluster will tend to be farther from each other, so that they can cover more dierent basis, which in turn leads to an increase 32 of transport cost per measurement. Since total transport cost is a function of both the number of measurements and the transport cost per measurement, this trade- o allows reducing the number of measurements at the cost of increasing transport cost per measurement. In order to construct clusters that take into consideration both reconstruction quality and transport costs, we propose a centralized iterative algorithm, which will be presented in Section 2.5. 2.4.3 Proof of Proposition 2.4.1 In this section, we present the details of the proof of Proposition 2.4.1. The goal is to bound the coherence in terms of with high probability. The sketch of proof is similar to that of SRM in [22] but the details are dierent. First, in our problem, the randomness is due to the coecients of the measurement matrix, (i;j), rather than the uniform permutation as in SRM. Second, we consider an additional quantity, , as well as the number of clusters, N c . Before going into the details of proof, we rst approximate a bound related to E[U 2 i (j;k)]. Basically, U i (j;k) is the inner product between the j th row of i and k th column of as shown in (2.8). Denote X l = i (j;l) i (l;k) so that U i (j;k) = P l i (j;l) i (l;k) = P l X l . Dene = max i;j j (i;j)j 1= p logN by assumption and let (j;k)N(0;N c =N). Therefore,jX l j can be bounded by the product of and three times of the standard deviation of (j;k) with high probability, i.e., 33 jX i j =j i (j;l) i (l;k)j 3 3 s N c N logN : The next approximation is concerned withE[U 2 i (j;k)]. Note thatVar(U i (j;k)) is equal toE[U 2 i (j;k)] becauseE(U i (j;k)) = 0. Var(U i (j;k)) can be approximated as a function of as follows: Var(U i (j;k)) =E[U i (j;k) 2 ] =E 2 4 ( N=Nc X l=1 i (j;l) i (l;k)) 2 3 5 (2.12) =E 2 4 N=Nc X l=1 2 i (j;l) 2 i (l;k) 3 5 = N=Nc X l=1 E[ 2 i (j;l)] 2 i (l;k) (2.13) = N=Nc X l=1 N c N 2 i (l;k) N c N (by denition of ) In (2.12), the cross terms are zero since E( i (j;l)) = 0 and the i (j;l)'s are inde- pendent. Now, we present the details of the proof of Proposition 2.4.1. The main technical tools are large deviation inequalities of sum of independent random variables. More 34 specically, the proof is derived from Bernstein's deviation inequality [51] and a union bound for the supremum of a random process. We rst approximate the probability that mutual coherence is bounded by using a union bound for the supremum of a random process: Pr[] =Pr[max j;k jU(j;k)j] = 1Pr[max j;k jU(j;k)j>] = 1Pr[ [ i (max j;k jU i (j;k)j>)] 1 Nc X i=1 N=Nc X j=1 N X k=1 Pr[jU i (j;k)j>] (by Union Bound) Then we use Bernstein's deviation inequality (Theorem 2.4.3) to approximate the bound of the tail probability ofU i (j;k). Note that theU i (j;k)'s are independent random variables because, by assumption, the (i;j)'s are i.i.d. Gaussian random variables with zero mean. Theorem 2.4.3. Bernstein's inequality [51] IfX 1 ;X 2 ;:::;X n are independent (not necessarily identical) and zero-mean random variables, andjX i jC;8i, then Pr " n X i=1 X i # 2 exp 2 =2 P i=n i=1 E[X 2 i ] +C=3 ! (2.14) 35 Denoting again X l = i (j;l) i (l;k), by the approximation, we have C = 3 s N c N logN (2.15) and X E[X 2 i ] =Var(U i (j;k)) N c N (2.16) By substituting (2.15) and (2.16) in Bernstein's inequality, Pr[] 1 Nc X i=1 N=Nc X j=1 N X k=1 2 exp 2 =2 Var(U i (j;k)) +M=3 (by Berstein's inequality) 1 2N 2 exp 2 =2 (N c =N) + (3 p N c =N logN)=3 ! (by approximation) = 1 exp log 2N 2 2 =2 (N c =N) + ( p N c =N logN) ! We need to nd the minimum, denoted , such that the probability,Pr[ ], asymptotically goes to `1'. To achieve the asymptotic behavior, the 2 nd term (exp term) in the last inequality should be zero with large N, which means that the 2 nd order polynomial inside the exp term should be negative. The polynomial can be expressed as a function of as: f() = 2 =2 + s N c N logN log 2N 2 + N c N log 2N 2 (2.17) 36 To nd for largeN, we rst check the characteristics off(). Sincef(0)> 0, f 0 (0) > 0, and f 00 (0) < 0, the larger root of f() is the minimum such that Pr[ ] = 1 for largeN. Algebraically, from the computation of the larger root of f() in (2.17), we can conclude that Pr[ ] = 1O(1=N); where =O( r N c N logN) With the asymptotic bound of coherence above, we can derive Theorem 2.4.2, the minimum number of measurements for perfect reconstruction, by Theorem 1.1.3. 2.5 Centralized iterative clustering algorithm To achieve a good tradeo between transport cost and reconstruction accuracy, we need to jointly optimize and the distance between nodes in a cluster. This motivated us to design a centralized iterative algorithm that can generate clusters based on a cost function taking into consideration both and distance within cluster. 2.5.1 Cost function For a given undirected graph G = (V;E), we assume that the sparsifying basis, , is known a priori and all the basis functions (columns of ) are normalized to 1 so that 2 [0; 1]. Also, N nodes are placed along a square grid in a eld. 37 In order to construct N c clusters that minimize transport costs, while guar- anteeing perfect reconstruction for sparsity K. To achieve the goal, we design a greedy algorithm based on our propose cost function. The algorithm starts from (i) N c initial nodes, one for each cluster, then (ii) at every iteration, we nd edges connected to each of the clusters and compute the weights, (iii) to nd the edge with the minimum weight, and (iv) we add the edge to the cluster. In short, at each iteration, we want to decide what node to be added to which cluster among all nodes not assigned to one of clusters in previous iterations. We rst assume that the transport cost depends on the distance between nodes and dene the distance in hops as D(e) for an edge, e2E, connecting two nodes, i.e., the smallest number of hops between two nodes in a multi-hop network. But cannot be dened based on just nodes only since it depends on the clusters. Thus, we need to dene (e;C i ) with respect to an edge, e2 E, and a given cluster, C i 2fC 1 ;C 2 ; ;C Nc g. Assume that the edge e is not an edge connecting two nodes in the same cluster, C i . We rst dene a cost function with respect to an edge e and the i th cluster C i as: W (e;C i ) =D(e) +(e;C i ); > 0; (2.18) where (e) is the maximum energy overlap when an edge, e, is connected to the cluster C i . Thus, once a node is added to a cluster, energy overlap of the edges connected to the cluster changes so that the edge weights should also change. As an example, from Fig. 2.3 (a) to (b), the weight of an edge, e = (v 1 ;v 3 ), changes from 3 to 5 because v 3 and the cluster,fv 1 ;v 2 g is overlapped with the same basis function,B 1 , which leads to the increase of the corresponding. The cost function, 38 W (e;C i ), will be used in our proposed algorithm to nd an optimized clusters in Chapter 2.5.2 B2 B3 B1 B4 V1 V2 V3 W=1 V4 V5 W=2 W=3 W=2 W=5 W=4 (a) the i th iteration B2 B3 B1 B4 V1 V2 V3 V4 V5 W=5 W=3 W=2 W=5 W=4 (b) the (i + 1) th iteration Figure 2.3: Illustration of update of edge weights from (a) to (b). There exist 4 square basis functions (B i ) and 5 nodes (v i ) connected by edges, (v i ;v j ), with dierent weights. Assume that the initial node for clustering is v 1 . At thei th step, a cluster is formed byfv 1 ;v 2 g because (v 1 ;v 2 ) has the minimum weight of 1. Since the cluster changes, we need to update the weights if necessary. For example, the weight of an edge, (v 1 ;v 3 ), changes from 3 to 5 because v 3 and the cluster,fv 1 ;v 2 g is overlapped with the same basis function, B 1 , which increases the with respect to the cluster. 2.5.2 Algorithm details The goal of the algorithm is to constructN c clusters that minimize transport costs, while guaranteeing perfect reconstruction for sparsityK. Transport costs depend on the distance between nodes and the number of measurements transmitted, which in turn depends on. Thus we need joint optimization of and the distance between nodes. For joint optimization, our proposed algorithm iteratively grows the clusters by nding an edges with minimum cost and adding to one of clusters. 39 To nd a set of edges to form N c clusters such that the total weight of the edges, W (e;C i )8i, is minimized, we design an algorithm based on a greedy local heuristic. The algorithm starts from N c initial nodes, one for each cluster; we deterministically chose N c nodes located on the grid with equal distance to the adjacent starting nodes. At every iteration, we nd edges connected to each of the clusters and compute the weights,W (e;C i ). Then, the edge with the minimum weight is added to the cluster. This procedure continues until every node is assigned to one of the N c clusters. Refer to Algorithm 1. Algorithm 1 Joint Optimization of and D Given an undirected graph, G(V;E), such thatjVj =N. Assign N c nodes to clusters; one for each cluster, V C i . E C i =?;8i. for k = 1 to NN c do Find E n =f(v 1 ;v 2 )jv 1 2V;v 2 2V C i ;8ig Compute W (e;C i ) =D(e) +(e;C i );8e2E n and8C i (C i min ;e min ) = arg min 8C i ;e2En W (e;C i ) v min = v 1 je min = (v 1 ;v 2 );v 2 2V C i min Add e min to E C i min and v min to V C i min . Remove edges2feje = (v min ;v);8v2V C i 8ig from E. Remove v min from V . end for The algorithm is similar to Prim's algorithm [43] for nding minimum spanning trees (MSTs). Given weights of edges, we choose an edge with minimum weight at every step as in Prim's algorithm. However, we have additional requirements as compared to Prim's algorithm. First, our algorithm ndsN c clusters with minimum total edge weights instead of an MST. Thus, an edge with the minimum weight is added to one of clusters to which the edge is connected rather than to a tree. Second, Prim's algorithm runs under the assumption that the weights of the edges do not change but, in our problem, the edge weights should be updated at every step. 40 0 5 10 15 20 25 30 35 40 45 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 Lambda Beta Lambda vs. Beta 0 5 10 15 20 25 30 35 40 45 21.5 22 22.5 23 23.5 24 24.5 25 25.5 26 Lambda Average D / M Lambda vs. Average hop distance per measurement (a) vs. (b) vs. average D per measurement Figure 2.4: Joint optimization of dierent . By running the algorithm with Daubechies 4 basis with 2 nd level of decomposition, 256 sensors are separated into 16 clusters. Dierent choices of generate dierent results; as increases, the edge weights are more sensitive to the change of so that decreases at the cost of increasing D. Given a sparsifying basis and the positions of sensors, Algorithm 1 generates a set ofN c clusters by minimizing edge weights associated with a. Since edge weight W (e;C i ) =D(e) +(e;C i ), the design parameter controls the balance between two competing terms: (e;C i ) and D(e). As increases, (e;C i ) is a more dom- inant factor than D(e) so that the edges with smaller (e;C i ) have higher chance to be added to a cluster, which means that the spatial extent of clusters increases. As shown in Fig. 2.4, as increases, the nal (Denition 2.3.1) decreases but the average distance per measurement increases. However, it is not clear how to determine the best with respect to both reconstruction accuracy and transport cost. The minimum number of measurements, M, decreases thanks to the decrease of while the distance between nodes within the same cluster, D, increases. Since transport cost is determined by C =MD, dierent aects the overall transport cost and there could exist a (or a range of) good that achieves large energy 41 savings with the similar level of reconstruction accuracy. Thus, given a target reconstruction accuracy, we experimentally search for the minimum transport cost that provides that desired accuracy. 2.6 Simulation Result The simulation consists of four parts. First, we compare performance between inde- pendent reconstruction and joint reconstruction. Second, we compare performance between two dierent routing schemes without the optimization of the maximum energy overlap, then compare the performance of SPT-based clustering to other CS techniques. Third, we verify Theorem 2.4.2 by examining the correlation be- tween the estimated M est by Theorem 2.4.2 and the minimum M sim measured by simulation. Fourth, we evaluate the performance of the joint optimization in terms of transport cost and reconstruction quality. The details of dierent simulation environments will be described in following subsections. 2.6.1 Joint reconstruction vs. independent reconstruction For the simulation, we used 500 realizations generated with sparsityK = 55, where non-zero coecients were randomly selected. In the network, 1024 nodes are de- ployed on the square grid and error free communication is assumed. We do not assume any priority is given to specic clusters for measurements, i.e., we collect the same number of localized measurements for each cluster. With localized pro- jection within each cluster, data is reconstructed jointly and independently with Gradient Pursuit for Sparse Reconstruction (GPSR) [29]. To evaluate performance, SNR is used to evaluate reconstruction accuracy. In this simulation, we focus on the reconstruction accuracy with increasing measurements. 42 200 300 400 500 600 700 800 900 1000 1100 0 10 20 30 40 50 60 70 80 number of measurements (M) SNR (dB) Joint 4 Joint 16 DRP Indep 4 Indep 16 Figure 2.5: Independent reconstruction vs. joint reconstruction when the spar- sifying basis is Haar basis with decomposition level of 5. For comparison, the square-clustering scheme with two dierent number of clusters (4 and 16 clusters) are used. To compare independent reconstruction with joint reconstruction, we used a square-clustering scheme with two dierent number of clusters (4 and 16 clusters) and data sparse on the Haar basis with 5 levels of decomposition. In Fig. 2.5, dense random projection (DRP) corresponds to the case where 256 global measurements (i.e., each measurement is an aggregate of data from all the nodes in the network) are transmitted to the sink and then data is reconstructed using joint reconstruc- tion. Other curves are generated from localized measurements in each cluster and the two types of reconstruction are applied. Fig. 2.5 shows that joint reconstruction outperforms independent reconstruction as expected based on the discussion of Section 2.3.1. In all following simulations only joint reconstruction is used. 43 2.6.2 SPT-based clustering For the simulation, we used the same setup as in Section 2.6.1. But, in this sim- ulation, we consider two types of clustering with dierent number of clusters (16, 64, and 256 clusters): square-clustering and SPT-based clustering. For cost eval- uation, transmission cost is estimated as P (bit) (distance) 2 , as was done in [75,76], although this could be extended to use more realistic cost metrics. We plot reconstruction SNR as a function of the rate of the transport cost of each scheme with respect to the transport cost in the raw data gathering without compression. While we use the cost ratio for evaluation, this simulation is mainly focused on the comparison between square and SPT-based comparison. 0 2 4 6 8 10 12 0 10 20 30 40 50 60 70 80 Cost ratio to Raw data transmission SNR (dB) Square 16 SPT 16 Square 64 SPT 64 Square 256 SPT 256 Figure 2.6: Cost ratio to raw data gathering vs. SNR with dierent number of clusters and clustering schemes when the sparsifying basis is Haar basis with de- composition level of 5. 44 With joint reconstruction and Haar basis, Fig. 2.6 shows that SPT-based clus- tering outperforms square clustering for dierent number of clusters (N c ). As N c increases, reconstruction accuracy decreases because the measurement matrix be- comes sparser as network is separated into more equal-size clusters. However, once the transport costs have been taken into account, more clusters show better perfor- mance because cost per measurement decreases and SPT-based clustering always outperform square clustering. Since we also observed this trend for dierent bases, we will focus on SPT-based clustering in following simulation. 0.8 1 1.2 1.4 1.6 1.8 2 0 10 20 30 40 50 60 70 Cost ratio to Raw data transmission SNR (dB) SPT 64 Haar5 SPT 64 Haar3 SPT 64 DB6 3 SPT 64 DCT Figure 2.7: Performance comparison in terms of cost ratio with respect to raw data gathering vs. SNR for dierent basis functions and 64 SPT-based clusters. In order to investigate eects of the spatial localzation of signals (determined by the spatial extent of basis functions), we generate data with the same sparsity in dierent bases and x SPT-based clustering with 64 clusters. For comparison, we consider three dierent cases: (i) DCT basis, where each basis vectors have 45 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 0 10 20 30 40 50 60 70 80 Cost ratio to Raw data transmission SNR (dB) SPT 256 APR SRP 2 SRP 4 Figure 2.8: Performance comparison in terms of cost ratio with respect to raw data gathering vs. SNR for 256 SPT-based clusters with other CS approaches with Haar basis with level of decomposition of 5. high overlaps in energy which distributed throughout the network, (ii) Haar basis, where the basis vectors have less overlap and the energy distribution of each basis varies from being very localized to being spread out over the whole eld for dierent basis functions at a level, and (iii) Daubechies (DB6) basis, where the overlaps and distribution are intermediate between DCT and Haar. The result in Fig. 2.7 shows that, for the same clustering scheme and data with the same sparsity in a given basis, the gains from joint reconstruction depend on how\well-spread" the energy in the basis vectors is, that is, results are better when data is sparse in more \global" bases. Fig. 2.8 shows that our approach outperforms other CS approaches [68,83]. APR corresponds to a scheme where aggregation occurs along the shortest path to the 46 sink and all the sensors on the paths provide their data for measurements [68]. SRP with dierent parameter, s 0 =sn, represents a scheme that randomly chooses s 0 nodes without considering routing, then transmits data to the sink via SPT with opportunistic aggregation [83]. In the comparison, SRP performs worse than the others because, as we ex- pected, taking samples from random nodes for each measurement signicantly in- creases total transmission cost. Our approach and APR are comparable in terms of transmission cost but our approach shows better reconstruction. 2.6.3 Reconstruction accuracy and In this simulation, we consider 3 dierent 2-D Daubechies basis with 2 levels of decomposition: DB4, DB6, and DB8. For each of the basis, we generate 1000 real- izations with three dierent sparsity levels (K): 20, 38, and 55. In each realization, K basis functions are chosen at random and assigned random coecients. In the network, 1024 nodes are deployed on a square grid in a region of interest and a sink is located at center of the eld collecting measurements from sensors with error free communication. With M measurements, data is jointly reconstructed using the GPSR [29]. To verify Theorem 2.4.2, we rst measure the minimum number of measure- ments, M sim , required for perfect reconstruction in our simulation. To measure M sim , we rst evaluate reconstruction accuracy using the perfect reconstruction rate (Prr). We consider that for a given realization perfect reconstruction is achieved if maxjx ^ xj< 10 3 . For 1000 synthesized sparse data, Prr is dened as a ratio of the number of data perfectly reconstructed to 1000. The minimumM sim for perfect 47 Table 2.1: Correlation coecient, r2 [1; 1], between M est and M sim N = 1024;N c = 16 DB 4 DB 6 DB 8 K=20 0.49 0.44 0.50 K=38 0.59 0.52 0.61 K=55 0.68 0.57 0.63 reconstruction is the smallest M that satises a perfect reconstruction rate larger than 0.99. To collect spatially-localized projections, N sensors are separated into N c non- overlapped clusters with the same size; every cluster contains N=N c sensors. We consider 20 dierent clustering schemes. For each clustering scheme, we divide N sensors in the eld into 16 localized clusters with radial shape going from the sink to the boundary of the network. To generate 20 dierent clustering schemes, we rotate the 16 clusters with a certain angle for each clustering scheme. To estimate M est , for given and K, we rst compute for each clustering scheme so that we have 20 dierent values of , one for each clustering scheme. Then, M est is computed following Theorem 2.4.2. To check if M est and M sim are correlated to each other, we use Pearson's linear correlation coecient, r2 [1; 1], which measures the linear dependence between two variables [64]. As shown in Table 2.1, the correlation value is around 0.55 for dierent K and , which shows that aects reconstruction accuracy in terms of the minimum number of measurements for perfect reconstruction. However, our bound is not tight enough to estimate the exact number of measurements because there exists a gap between M est and M sim . However, is a useful metric because, based on , we can compare dierent clustering schemes and also design a clustering scheme by optimizing as discussed in Section 2.5. 48 Since the maximum energy overlap is a worst case measure, can be mislead- ing. For example, suppose there exists a basis function and cluster for which the energy overlap is 1 (perfect overlap), but the others are relatively small; in this case by denition = 1. However, successful reconstruction is possible with high quality if the basis function associated to the large energy overlap is not in the data support. Therefore, it would be meaningful to examine the impact of maximum energy overlap of each basis function and clusters on the level of error observed for that basis function. For each basis function, we dene i by measuring the maximum energy over- lap between the i th basis function and all clusters. Thus, the i characterizes the distribution of energy of individual basis functions. Table 2.2 shows the average correlation between i and the error related to the corresponding basis function. The simulation results show that basis functions with larger i generate more er- rors. This implies that basis functions with concentrated energy on fewer clusters have lower probability to be identied by spatially-localized measurements. While the results do not show very high correlation; sucient to reliably estimate the number of measurements required for perfect reconstruction, is well estimated, can be a useful metric to nd a better clustering scheme, which will be discussed in Section 2.6.4. Table 2.2: Correlation coecient, r2 [1; 1], between maximum energy overlap and MSE for each basis N = 1024;N c = 16;M = 410 DB 4 DB 6 DB 8 K=20 0.31 0.58 0.52 K=38 0.48 0.62 0.55 K=55 0.47 0.63 0.64 49 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 21 21.5 22 22.5 23 23.5 Beta Average D / M Beta vs. Average hop distance(D) per measurement(M) Figure 2.9: vs. average number of hops per measurement. Each point corresponds to the result of Algorithm 1 with dierent . The points with red circle are chosen for the evaluation of transport cost and MSE in Fig. 2.10 2.6.4 Joint optimization To evaluate the joint optimization, we use mean squared error (MSE) as a metric for reconstruction accuracy. This is because, in practice, we are more interested in the level of error associated to a specic transport cost. Since we allocated the same number of bits for each measurement, transmission cost, P (bit) (number of hops), depends on the product of the number of measurements with the distance in hops. In our simulation, we consider a signal with sparsity K = 38 in 2D Daubechies-4 basis with 2 levels of decomposition. Located at center of the eld, a sink collects M measurements from 64 clusters. For energy eciency, measurements from each cluster are routed to the sink along shortest path. For comparison with other CS 50 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 0 0.2 0.4 0.6 0.8 1 1.2 x 10 −3 Communication Cost Ratio MSE Communication Cost Ratio vs. MSE lambda=8.00, beta=0.42 lambda=16.00, beta=0.30 SPT 64 , beta=0.83 Figure 2.10: Transport cost ratio vs. MSE. The x-axis is the ratio of total trans- port cost of spatially-localized CS to the cost for raw data gathering without any manipulation. We compare performance of results by joint optimization with two dierent 's with that of SPT 64 in [48] approaches, we consider a clustering scheme based on shortest path tree (SPT) that showed the best performance in Section 2.6.2 As discussed in Section 2.5, in general, smaller can be achieved by increasing distance between nodes in the same cluster. The tradeo can also be observed in Fig. 2.9; as increases, we can achieve smaller but this increases hop distance per measurement as discussed in Section 2.5. In addition, Fig. 2.9 shows that, as increases, decreases quickly but becomes saturated at some point. After that, transport cost increases without improvement of. Thus, we can expect that one of 's around the saturation point will correspond to a good operating point showing better performance in terms of total transport cost and reconstruction accuracy. 51 Fig. 2.10 shows the overall performance with dierent . Each curve shows the average MSE over 100 realizations and the variation in reconstruction accuracy is expressed as three times of standard deviation. As expected, a with the value of 16 located at the sharp transition in Fig. 2.9 shows the best performance. With the best and 1024 sensors divided into 64 clusters, we can achieve 40% cost saving with respect to raw data-gathering with small mean squared error ( 1 10 4 ). Compared with SPT-based clustering scheme in [48], our clustering scheme with joint optimization achieves almost 50% reduction in transport costs at the same level of mean squared error in the reconstruction. Also, Fig. 2.10 conrms that for a given transport cost is a good predictor of reconstruction quality; lower leads to better reconstruction quality. The clusters in the SPT-based clustering scheme consume less energy to construct a measurement. However, the savings in transport cost is compensated by a larger number of measurements required for the same level of reconstruction quality which can be explained by large value of (= 0:83). 2.7 Extension to irregularly positioned sensors Up to this point, we have proposed an energy-ecient data gathering scheme in WSN. This approach has two major limitations in practice: (i) regular positions of sensors on the 2D grid and (ii) K-sparse synthesized data in a given sparsifying basis . The regular topology is useful for monitoring buildings, bridges, or power plants but is not appropriate for many other applications such as monitoring of habitat, wild re, or battle eld. This motivate us to study how the CS-based approach can be extended to data-gathering with irregularly positioned sensors. Also, k-sparse data in a given is unlikely to happen in reality. Most signals 52 are compressible in irregular bases, which encourages us to investigate the use of graph-based transforms (GBT) that can provide a sparse representation of realistic sensor data. 100 200 300 400 500 600 100 200 300 400 500 600 Figure 2.11: 256 sensors in irregular positions and the corresponding graph. The communication range is set as the minimum distance that results in a connected graph. In this work, we consider a realistic data model with irregularly positioned sensors. We rst generate data that is independent of the sparsifying basis, using a second order AR model as shown in Fig. 2.11. In the noise-free data, the correlation of data between two nodes increases as the distance between them decreases. On the smooth eld of data, 256 sensors are randomly deployed, and we measure then transmit the data to the sink positioned at the center of the eld along the SPT. 53 Under these realistic assumptions, we use graph-based transforms (GBT) as a sparsifying basis because the transform can be applied to various deployments of sensors if the topology is represented by a graph. For the construction of GBT, we rst represent the WSN as a graph, G(V;E) with nodes (sensors) and links (connections) between sensors as illustrated in Fig. 2.11. Note that the links can exist only if the two sensors are within a specic range. In this work, we set the range as the minimum such that the resulting graph is connected (i.e., there are no disconnected subgraphs) as shown in Fig. 2.11. Since the sensor data is likely to be highly correlated between adjacent sensors, the links between distant sensors that are farther apart can be disconnected for a sparser representation. Note that the GBT is only required at the sink to reconstruct the received signal, so that sensors in the eld do not need to know the complete topology of the network in order to transmit data. In addition, It is possible for nodes to decide locally on how to transmit data, so that centralized coordination is not required. From the graph, the adjacency matrixA is formed, whereA(i;j) =A(j;i) = 1 if the distance between sensori andj is smaller than the minimum communication range. OtherwiseA(i;j) =A(j;i) = 0. Then we dene the degree matrixD, where D(i;i) is the number of links connected to the i th sensor andD(i;j) = 0; 8i6=j. Finally, the Laplacian matrix can be dened as: L =DA = 8 > > > > < > > > > : 1; if (i;j)2E d i ; if i =j 0; otherwise (2.19) After the eigenvalue decomposition, we use the eigenvector matrix as a spar- sifying basis, , whose columns are the eigenvectors of the Laplacian matrix, L. Note that is orthogonal becauseL is symmetric, leading to real eigenvalues and 54 a set of orthogonal eigenvectors (refer to [34,78] for more details). Fig. 2.12 shows the performance of the GBT as a sparsifying basis. Although the sensor data is not perfectly sparse, the GBT shows a good compressibility, i.e., more than 99% of energy is compacted in a few GBT coecients. 0 50 100 150 200 250 300 0.97 0.975 0.98 0.985 0.99 0.995 1 1.005 Normalizaed Cummulative Energy sorted GBT coeff idx Figure 2.12: Compressible WSN data in the GBT. The x-axis shows the indices of GBT coecients and the y-axis shows the cumulated sum of normalized energy of GBT basis functions. As shown in Fig. 2.11, the data is generated by a 2 nd AR model with the AR lter H(z) = 1 (1e jw 0z 1 )(1e jw 0z 1 ) , where = 0:99 and w 0 = 359 For CS-based data-gathering, we consider two approaches: SPT aggregation and GBT-aware aggregation. For the SPT aggregation (same as APR in Section 2.6.2), we randomly choose a certain number of sensors, and aggregate data of all the sensors on the SPT. Then, the linear combinations of data with Gaussian random coecients are transmitted along the SPT [68]. Alternatively, the GBT-aware 55 aggregation selectively chooses the sensors by considering the number of the links connected to the sensors in the graph that is used for GBT construction as in Fig. 2.11. From a certain number of randomly chosen sensors, the aggregation happens along the SPT as in the rst approach. But, an aggregation takes place at a sensor if the number of neighbors connected to the sensor is less than a threshold. Otherwise, the sensor relays the received data to its parent sensor along SPT. Since the GBT can be interpreted as a spectral decomposition over the links, the aggregation over the sensors with fewer neighbors in the graph increases the amount overlap, which leads to better reconstruction as discussed in Section 2.4. The threshold is empirically chosen in this work. Once the threshold is determined, we do not need to update it if the topology remains the same and the sensor data is sparsely represented by the constructed GBT, so that our propose approach does not require a lot of coordination if the topology is known locally. Also, since the aggregation decision depends on a characteristic of local network, this approach can be done in a decentralized way. In this simulation, a second order AR model is used to generate 50 realizations with high spatial data correlation as shown in Fig. 2.13. More specically, the AR lter H(z) = 1 (1e jw 0z 1 )(1e jw 0z 1 ) , where = 0:99 and w 0 = 359. For the simulation, 256 sensors are randomly positioned in the 600 600 grid and the data measured at each sensor is represented using 12 bits. Also, the measurements (or down-sampled data) are transmitted along the SPT as shown in Fig. 2.13. Note that the locations of the sensors do not change throughout our simulations. Also, for measuring energy consumption, we adopt a realistic cost model proposed in [35,82]. Energy in the sensors is dissipated when both transmitting,E T (k;D), and receiving data, E R (k). The energy consumption in k bit transmission over a distance D is 56 100 200 300 400 500 600 100 200 300 400 500 600 Figure 2.13: 256 sensors in irregular positions and the corresponding SPT. E T (k;D) = E elec k +" amp kD 2 Joules and the consumption in k bit reception is E R (k) =E elec k. In our simulation, we compare four dierent approaches: (i) SPT aggregation (CS SPT ) [68], (ii) GBT interpolation(itpl GBT ) [58], (iii) raw transmission without any compression, and (iv) our proposed method (CS GBT ). For itpl GBT , we ran- domly choose a certain number of sensors and transmit the sampled data to the sink along the SPT. Then, data is reconstructed by the graph interpolation tech- nique proposed in [58]. The curve for the raw data transmission is generated with dierent levels of quantization, but the other curves use a xed quantization step size. For CS GBT , we empirically choose the threshold as 5, thus the aggregation happens at the nodes on SPT if the nodes have fewer than 5 neighbors in the graph in Fig. 2.11. The result in Fig. 2.14 shows that our proposed approach outperforms 57 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0 0.005 0.01 0.015 0.02 0.025 0.03 Total Energy Consumption (Joules) MSE MSE vs. Energy Consumption CS GBT CS SPT itpl GBT Raw Data Figure 2.14: Total energy consumption vs. MSE. The x-axis is the total energy consumption in Joules and the y-axis is MSE. The curves are generated by taking averages over 50 realizations of the sensor data. the other methods in terms of the energy consumption and the reconstruction ac- curacy. The GBT interpolation technique shows worse performance because it assumes a bandlimited graph signal supported only at frequencies [0;w], but the cuto frequency,w, in the graph is not small enough with respect to the downsam- pling rate (i.e., the ratio of the number of sensors providing samples with respect to the total number of sensors), so that the reconstruction quality is degraded. 2.8 Conclusion To achieve energy ecient data gathering in WSN, we exploit a sparse and spatially- localized CS measurement system that is aware of transport cost per measurement 58 by constructing measurements within spatially-localized clusters. However, while the spatially-localized measurement system leads to lower transport cost, it is not obvious how it aects reconstruction accuracy. Thus, we rst introduce a metric to measure the maximum energy overlap between clusters and basis functions, . Then we show that the metric has an impact on reconstruction accuracy with respect to the number of measurements for perfect reconstruction. By exploiting the tradeo between and distance between sensors in clusters, we propose a centralized iterative algorithm with a design parameter,, to construct clusters that are jointly aware of energy eciency and reconstruction quality. Our simulation results show that, with an an appropriately chosen , we can achieve signicant savings in transport cost with small reconstruction error. Also, we extend our work to a WSN that consist of sensors in irregular positions. This work shows a preliminary result that the sensor data can be sparsely represented by GBT and our proposed approach shows a promising performance compared to other existing methods. 59 Chapter 3 Adaptive Compressed Sensing for Depth map Compression Using Graph-based Transform 3.1 Introduction Standard compressed sensing (CS) theory prescribes that robust signal recovery is possible when a signal is sparse in a given sparsifying basis. Based on the signal characteristics, the sparsifying basis is often assumed to be known a priori at the decoder. However, for coding applications where signals are rst captured and then compressed, better performance can be achieved by adaptively selecting a transform or sparsifying basis and then signaling the chosen transform to the decoder. For instance, for piecewise smooth signals, where sharp edges exist between smooth regions, edge-adaptive transforms can provide sparser representation at the cost of some overhead. In this work, we consider block-based depth map compression as an example application. Previous work has shown that edge adaptive transforms can be more ecient than standard transforms (e.g., DCT) due to the piecewise smooth nature of these signals [74]. Moreover, correct representation of edges is important be- cause errors in edge information lead to signicant degradation of the quality of 60 interpolated views in 3-D TV applications [39, 40]. For depth map compression, CS-based methods have been recently proposed. CS is applied by either projecting the depth map on a random sensing matrix (Cartesian grid sampling technique) [73] or down-sampling 2D-DCT coecient [25]. However, performance gains achieved by these techniques are limited because the standard DCT is chosen as the spar- sifying basis, which is inecient for coding blocks containing arbitrarily shaped edges (i.e., neither vertical, nor horizontal) separating smooth regions. To preserve the edge information, many researchers have investigated ecient transforms for depth map, that avoid ltering across edges [54,56]. However, these methods have limitations. For example, Platelets [56] have a xed approximation error because depth maps are not exactly piece-wise planar. Shape-Adaptive Wavelets [54] are not amenable to a block based coding architecture, which has been widely adopted in international standards for image and video coding such as H.264/AVC. To overcome those limitations of existing transforms, a graph based transform (GBT) has been proposed [74] that can achieve a sparse representation, even when arbitrary edges exist in a block. It also has the advantage that it can be easily applied within a block-based coding architecture. GBT is based on representing each block as a graph, where each vertex corresponds to a pixel, and vertices are linked only when no strong edges are present between the corresponding pixels. For any given block with arbitrary size, dierent graphs can be chosen, leading to dierent transforms, which depend on the edge structure and therefore require that overhead bits be sent to the decoder. In [74] it was shown that these adaptive GBTs improved performance as compared to DCT-only methods, even when the overhead was taken into account. This work was further extended in [38], which proposed a simple cost function and a search technique to optimize the GBT selection for each 61 block, balancing the increased sparseness achievable if more edges are considered with the added overhead required for transmitting this information to the decoder. In this thesis, we propose a novel CS framework where the adaptive GBT is used as a block-adaptive sparsifying basis. We consider the problem of, given a specic sensing matrix, optimizing the sparsifying basis. In this work, we rst x the sensing matrix as a Hadamard matrix because (i) the retrieval of measurements is computa- tionally simple at the encoder and (ii) the computation of our approximated mutual coherence in Section 3.3 can be greatly simplied. Then, we optimize the choice of GBT by taking into account the reconstruction quality and the overhead required to specify the GBT. Note that the approach in [38] aims at selecting a GBT that provides maximum sparsity for a block, without requiring excessive overhead. A key result in our work is to show that maximum sparsity does not guarantee opti- mal performance when using CS. As studied in [8, 45], CS reconstruction depends not only on the sparsity of signal representation but also on the mutual coherence between sensing matrix and sparsifying basis. Therefore, a GBT providing the sparsest representation of depth data is not necessarily maximally incoherent with a given Hadamard sensing matrix. Thus, joint optimization is required to select the best GBT for a given depth map, taking into account rate overhead (to specify the transform), sparsity of the representation and mutual coherence. We propose a greedy iterative algorithm that evaluates a metric for dierent edge congurations before selecting one. This algorithm uses a low-complexity estimate of the mutual coherence, so that explicit construction of the GBT at the encoder is only required once the edge map has been selected (i.e., it is not required in the iterative process leading to this selection). The proposed block-adaptive CS approach is integrated within an H.264 codec. When evaluating its intra coding performance on three 62 depth map sequences, we observe 3.8 dB PSNR gain in the quality of interpolated views obtained from the decoded depth map, or an average of 39% bitrate savings. The rest of this chapter is organized as follows. We rst formulate the problem in Section 3.2. Then we provide a theoretical result showing that a good GBT can be found by an approximation of mutual coherence in Section 3.3 and the performance is veried by our simulation results in Section 3.4. 3.2 Problem Formulation For the construction of GBT, we rst represent each depth block as a graph,G(V;E) with nodes (pixels) and links (connections) between nodes as illustrated in Fig. 3.1. Note that we use the term \edge" to refer to image edges in order to avoid confusion with the links in the graph. A link is present in the graph only when no edge was selected between the two corresponding pixels. Later in this chapter, we will discuss how edges between pixels will be identied, which is closely related to the optimization of GBT. In this work, we assume 4-neighbor connectivity for the pixels, so that each node, V , can have at most 4 links. From the graph, the adjacency matrix A is formed, where A(i;j) = A(j;i) = 1 if pixel positions i and j are immediate neighbors not separated by an edge. Otherwise A(i;j) = A(j;i) = 0. Then we dene the degree matrixD, whereD(i;i) is the number of links connected to the i th pixel andD(i;j) = 0; 8i6=j. Finally, the Laplacian matrix can be dened as: L =DA = 8 > > > > < > > > > : 1 if (i;j)2E d i if i =j 0 otherwise (3.1) 63 Note thatL is symmetric, leading to real eigenvalues and a set of orthogonal eigen- vectors. Thus we dene the GBT for a given graph as the eigenvector matrix, , whose columns are the eigenvectors of the Laplacian L of the graph. Since is orthogonal, its inverse is T . 1 2 3 4 Edge Link Node (pixel) (a) Example 1 1 2 3 4 Edge Link Node (pixel) (b) Example 2 Figure 3.1: 2-by-2 block examples: we have four pixels (nodes) in the block and assume that there exists one edge between pixels. (a) the edge separates those pixels into two partitions. Since the links between nodes in the graph are assumed not to go across the edge, we have two sub-graphs where 2 nodes are connected to each other by a link. (b) while an edge exists, the edge does not separate pixels into two partitions, generating only one graph. As an example, consider the four pixels separated by an edge as shown in Fig. 3.1. By considering the pixels as nodes and assuming the links between pixels cannot go across an edge, the block can be represented as a graph, G(V;E). To construct the GBT from the graph in Fig. 3.1 (a), we rst construct the adjacency matrix,A 1 , and the degree matrix,D 1 , which has only 1's on its diagonal because each pixel is connected to only one other pixel, due to the existence of an edge: 64 A 1 = 2 6 6 6 6 6 6 6 4 0 1 0 0 1 0 0 0 0 0 0 1 0 0 1 0 3 7 7 7 7 7 7 7 5 ; D 1 = 2 6 6 6 6 6 6 6 4 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 3 7 7 7 7 7 7 7 5 : (3.2) FromD 1 andA 1 , the Laplacian matrix,L 1 , is dened as in (3.1). To construct the GBT, 1 , fromL 1 , we rst apply the eigenvalue decomposition toL 1 then we form 1 with column vectors corresponding to the eigenvectors ofL 1 : L 1 = 2 6 6 6 6 6 6 6 4 1 1 0 0 1 1 0 0 0 0 1 1 0 0 1 1 3 7 7 7 7 7 7 7 5 ; 1 = 2 6 6 6 6 6 6 6 4 1 p 2 0 1 p 2 0 1 p 2 0 1 p 2 0 0 1 p 2 0 1 p 2 0 1 p 2 0 1 p 2 3 7 7 7 7 7 7 7 5 : (3.3) Similarly, the GBT for the example of Fig. 3.1 (b), 2 , can be generated as follows. A 2 = 2 6 6 6 6 6 6 6 4 0 1 0 0 1 0 0 1 0 0 0 1 0 1 1 0 3 7 7 7 7 7 7 7 5 ; D 2 = 2 6 6 6 6 6 6 6 4 1 0 0 0 0 2 0 0 0 0 1 0 0 0 0 2 3 7 7 7 7 7 7 7 5 ; (3.4) L 2 = 2 6 6 6 6 6 6 6 4 1 1 0 0 1 2 0 1 0 0 1 1 0 1 1 2 3 7 7 7 7 7 7 7 5 ; 2 = 2 6 6 6 6 6 6 6 4 0:5 0:65 0:5 0:27 0:5 0:27 0:5 0:65 0:5 0:65 0:5 0:27 0:5 0:27 0:5 0:65 3 7 7 7 7 7 7 7 5 : (3.5) 65 For better visualization, thej th basis function of i , j i , can be shown as 2 2 matrices corresponding to the pixel locations in Fig. 3.1, so that we have: 1 1 = 2 6 4 1 p 2 0 1 p 2 0 3 7 5 ; 2 1 = 2 6 4 0 1 p 2 0 1 p 2 3 7 5 ; (3.6) 3 1 = 2 6 4 1 p 2 0 1 p 2 0 3 7 5 ; 4 1 = 2 6 4 0 1 p 2 0 1 p 2 3 7 5 ; (3.7) 1 2 = 2 6 4 0:5 0:5 0:5 0:5 3 7 5 ; 2 2 = 2 6 4 0:65 0:65 0:27 0:27 3 7 5 ; (3.8) 3 2 = 2 6 4 0:5 0:5 0:5 0:5 3 7 5 ; 4 2 = 2 6 4 0:27 0:27 0:65 0:65 3 7 5 : (3.9) One observation from the two examples in Fig. 3.1 is that the bases (column vectors in 1 and 2 ) can take arbitrary values across an image edge. That is, there is no need for one of the basis functions to provide a smooth approximation to an image discontinuity. Since the edge in the 2 nd example does not separate 4 nodes into more than one partitions, all the basis functions in 2 are global bases that are completely dierent from those in 1 . These show that the dierent edges (edge map) lead to dierent GBTs that provide dierent sparsity in the representation. A spectral decomposition, dened as the projection of a signal onto the eigenvectors of L (equivalently, projection onto t ), can be interpreted as providing the \frequency" contents of the graph signal [34, 78]. Note that in the example of Fig. 3.1(b) the corresponding bases in (3.8) and (3.9) behave as 1-D bases of increasing frequency (more zero crossings) as we follow the links from node 1 to node 3, avoiding the edge between them. 66 In our depth map compression application, block-adaptive GBTs are applied to residual blocks obtained after intra/inter prediction, where the graph from which the GBT is derived is chosen based on the edges present in each residual block. For each block, these edges could be detected by applying a simple threshold to the dierence between neighboring residual pixel values [74]. However, using the same threshold for all blocks does not take into account the overhead required to transmit the chosen edge map to the decoder, which tends to increase with the number of edges. Thus, two blocks may achieve similar levels of sparsity for a given threshold, but the block where more edges are identied may require a higher overall rate. As an alternative, the work in [38] seeks to nd the optimized edge map for each block by considering this overhead. In this work, we also investigate the selection of edge maps that are optimized in order to consume fewer bits, while providing satisfactory reconstruction by compressed sensing. A key observation in our work is that the optimized GBT (which [38] attempts to obtain) may not provide optimal performance if CS is used. This is because performance depends both on the level of sparsity in the representation and on the incoherence between the sparsity basis and the measurement matrix, which is very important for reconstruction, as studied in [8, 45]. For any GBT cho- sen as the sparsifying basis, , we can compute the mutual coherence with a xed Hadamard sensing matrix, , ( ). Based on the mutual coherence, the minimum number of measurements for perfect reconstruction can be computed as M = O(K 2 (U)N logN) [8]. The lower bound on the number of measurements decreases as and become increasingly incoherent. Thus, if we can estimate how varies as a function of the chosen GBT, then we can also compute a bound on the number of measurements needed, which will predict achievable performance. 67 With this, and with an estimate of the cost required to encode the corresponding edge map, it becomes possible to design a good GBT. 3.3 Optimizing GBT for CS 3.3.1 Bound on the mutual coherence We rst derive a bound on the mutual coherence for a given GBT matrix, , and the Hadamard matrix, . Since both matrices are deterministic, the mutual coherence is also deterministic, and could be computed for each candidate GBT. However, GBT construction is a complex operation as it requires nding all the eigenvectors of the Laplacian matrix. The complexity grows as the size of graph (equivalently, the block size in depth map compression) increases. Even if there exist only a few GBTs that are truly useful and for those we could precompute the mutual coherence, the number of useful GBT candidates also increases with the graph size, which leads to larger memory requirements. Thus it would be desirable to avoid having to construct GBTs at every stage of the search for the optimized GBT. In what follows, we derive upper and lower bounds on the mutual coherence then use their average to estimate the mutual coherence of the block. We rst derive the upper bound of the mutual coherence. Theorem 3.3.1. For a given graph G(V;E), the mutual coherence, , between the Hadamard sensing matrix, , and a graph-based transform matrix, , satises (E) r max 8i N G i N ; 68 where N G i denotes the size of the i th sub-graph (equivalently, the number of pixels in the sub-graph). If a graph is connected, then the mutual coherence is bounded by 1 because the DC component of the Hadamard basis is identical to the eigenvector corresponding to the zero eigenvalue of the graph Laplacian. In contrast, a fully disconnected graph where all the pixels are separated by edges can achieve the minimum bound for the mutual coherence. However, this increases the overhead to encode the edge map so that the coding gain is limited. The proof is trivial because all the entries of the Hadamard matrix are1= p N and all the basis functions of GBT (columns of ) are normalized to 1, so that the maximum absolute value of the inner-products is bounded by the maximum size of a group normalized byN. Next, a lower bound on mutual coherence is derived. Theorem 3.3.2. For a given graph G(V;E), mutual coherence, , between an arbitrary sensing matrix, and a graph-based transform, , satises (E) max 8k s P (l;m)2E ((k;l) (k;m)) 2 2jEj ; wherejEj is the total the number of links between nodes. Since we consider 4 4 blocks with 4-neighbor connectivity in this work, the maximumjEj is equal to 24 if G(V;E) is a fully-connected graph. The numerator of the bound is a sum of squared dierences of (i;j) corresponding to connected pixels. The bound indicates that the lower bound of the mutual coherence increases as more pixels corresponding to high variation of are connected. Since is a 69 Hadamard matrix in this work, this corresponds to counting the number of zero crossings in the graph for all Hadamard bases. The proof is based on the fact that x T Lx = P (i;j)2E (x(i)x(j)) 2 ; for any x2< V . Letx T = (i; :). Since L = T , x T Lx = (i; :)( T )(i; :) T (3.10) = X (l;m)2E ((i;l) (i;m)) 2 ; i2f1; 2;:::;Ng (3.11) (3.10) can be rewritten as follows: x T Lx = (i; :)( T )(i; :) T (3.12) =U(i; :)U(i; :) T (3.13) = X j j U(i;j) 2 ; (3.14) whereU = and j is the j th eigenvalue ofL. From (3.11) and (3.14): X (l;m)2E ((i;l) (i;m)) 2 = X j j U(i;j) 2 (3.15) X j j ! max 8j U(i;j) 2 ; (3.16) where P i i =Trace(L) = 2jEj because the total sum of diagonal entries inL is the twice of the total number of links between pixels. From (3.16), we have 70 max 8j jU(i;j)j s P (l;m)2E ((i;l) (i;m)) 2 2jEj : (3.17) Thus, the lower bound of the mutual coherence is derived: (E) = max 8(i;j) jU(i;j)j = max 8i max 8j jU(i;j)j max 8i s P (l;m)2E ((i;l) (i;m)) 2 2jEj : (3.18) Note that both lower bound, lower , and upper bound, upper , can be computed without constructing the GBT. The upper bound is determined by the maximum size of a disconnected sub-graph in the graph and the lower bound by the edge map and the given Hadamard sensing matrix. To approximate the mutual coherence between the two bases for a given graph, G(V;E), we take the average avg (E) = lower (E)+upper(E) 2 . Since the mutual coherence is the maximum correlation between two bases, the mutual coherence can be misleading, especially when only a few correlations are large but the others are small. Thus, instead of looking at the maximum correlation, the average correlation provides a better estimate for the CS performance as studied in [27]. Thus, avg can be used as an alternative metric instead of the original mutual coherence. The averaged mutual coherence will be used to approximate the rate for CS measurements to nd optimized adjacency matrix, which will be covered in the following section. 71 3.3.2 Iterative GBT construction for CS To nd the best sparsifying basis, , we iteratively evaluate a series of adjacency matrices using their average mutual coherence, avg . We assume 4-neighbor con- nectivity in 4 4 block, so that there exist 12 horizontal edges and 12 vertical edges. Instead of searching the whole space of 2 24 possible adjacency matrices, we propose a greedy algorithm to nd an optimized adjacency matrix (Algorithm 2). By dening a cost function, the cost for removing each edge can be calculated. Algorithm 2 Optimization of adjacency matrix,A Given an undirected graph, G(V;E), such thatjVj = 16 andjEj = 0. Construct G 0 (V;E 0 ) such that e i;j 2E 0 ; 8e i;j =f(v i ;v j )jv i ==v j g. E max = 24, E s =EE 0 C min = log 2 ( ( P 8(i;j)2E 0 (v i v j ) 2 ) 2 avg (E 0 ) 2Q 2 ) +m(E 0 ) for k = 1 to E max jE 0 j do e min = arg min 8e=(v i ;v j )2Es jv i v j j Remove e min from E s E k =E k1 Add e min to E k . C k = log 2 ( ( P 8(i;j)2E k (v i v j ) 2 ) 2 avg (E k ) 2Q 2 ) +m(E k ). if C k C min then C min =C k else Remove e min from E k end if end for return E out =E k 72 The basic idea of Algorithm 2 is as follows. In the initial graph, there are only links between pixels that have the same value. This would be equivalent to having a very small threshold to determine whether two pixels have an edge between them. At each iteration, the algorithm nds the link with the minimum pixel dierence among those that have not been examined in previous iterations. This link is added if the updated cost is smaller than the one in the previous iteration. The algorithm repeats the same procedure until all the links have been searched excluding the links in the initial state. Thus, for 16 nodes on the 4 4 grid with 4-neighbor connectivity, the maximum number of iterations is 24 if no link exists at the initial graph. Algorithm 2 optimizes the graph in order to nd a good set of links providing better performance in terms of the number of bits for encoding and the reconstruction quality estimated by the average mutual coherence in Section 3.3. The set of links to be found by the algorithm,E out , can be uniquely represented by adjacency matrix or edge map. The cost function of a given graph, G(V;E), is dened as: C(E) = Cost measurement rate + Cost edge rate = log 2 0 @ P 8(i;j)2E (v i v j ) 2 2 avg (E) 2Q 2 1 A +m(E); (3.19) The edge rate, m(E), is the number of bits required to code E (equivalently, the corresponding adjacency matrix or edge map), which can be represented using 24 bits with 4-neighbor connectivity, and then compressed further using entropy coding. The scaling factor can be applied to control the trade-o between the coecient rate and edge rate, which is empirically determined in our simulation. v is a vector representing the input depth map block and thus v i is the value of 73 pixel i. a ij is the element in the adjacency matrix that correspond to the link between pixels i and j. Thus, P 8(i;j)2E (v i v j ) 2 is the sum of squared dierences between connected pixels, which provides an estimate the cost transmitting the GBT coecients and approximates their sparseness. Then, we divide this with the square of quantization step size, Q. Note that the cost function is identical to the one proposed in [38] except for 2 avg (E). For an intermediate graph, G(V;E k ) in Algorithm 2, the cost function, C(E k ) is dened as: C(E k ) = log 2 0 @ P 8(i;j)2E k (v i v j ) 2 2 avg (E k ) 2Q 2 1 A +m(E k ) (3.20) The average mutual coherence, avg (E), is needed to estimate the rate required to transmit the measurements, since the number of measurements is proportional to K 2 logN as studied in [8]. Note that we can ignore the logN term because total number of pixels in each block does not change during the algorithm. 3.4 Simulation Results The simulation is based on H.264/AVC reference software JM17.1. For simplicity, only a 4 4 blocksize is used in our simulation for the transform, although this can be easily extended to other block sizes. As test sequences in our simulation, we use only intra-frames of depth map sequences Ballet, Newspaper, and Mobile. With RD optimization with respect to H.264/AVC, GBT and CS-GBT, the encoder chooses the best mode and transmits extra bits to signal the transform mode for each block. For CS-GBT, the encoder encodes 4 Hadamard measurements corresponding to the 74 Methods RD Optimization Overhead H.264/AVC DCT None GBT DCT + GBT OptimizedA [38] CS-GBT-1 DCT + GBT + CS OptimizedA [38] CS-GBT-2 DCT + GBT + CS OptimizedA for CS Table 3.1: For comparison, four dierent methods are considered. First, with H.264/AVC, only 4 4 DCT is enabled, thus no overhead exists. Second, DCT and GBT are enabled in RD optimization and, for GBT, the adjacency matrix, A, is optimized as in [38] with overhead transmitted to the decoder for each block that uses GBT. Third, with the same A, an additional transform mode, CS, is considered. Lastly, DCT, GBT, and CS are considered in RD optimization as before but the adjacency matrix,A, is optimized as discussed in Section 3.3.2. 4 lowest frequency bases then scalar quantization is applied to the measurements associated with a QP value. In this work, we do not consider an adaptive choice of measurements because the encoder is required to signal the indices of the cho- sen measurements to the decoder, which increases the overhead bits. It may be possible to optimize the choice of Hadamard projections while taking into account the overhead, but this is left for future work. To reconstruct the depth map from the Hadamard measurements, MOSEK C-library [5] is employed to solve the L 1 minimization, which is then integrated into H.264/AVC reference software JM17.1. For comparison, we construct the GBT matrix using two dierent greedy al- gorithms with dierent cost metric as shown in Table 3.1: i) GBT construction without mutual coherence [38] (GBT and CS-GBT-1) ii) GBT construction with mutual coherence discussed in Section 3.3.2 (CS-GBT-2). The scaling factor in (3.19) is empirically chosen as 0.03 which equals to the one in the cost function of [38] because, in our simulation, the change of the scaling factor does not aect much the overall performance. For both cases, the resulting adjacency matrices 75 are entropy coded and sent to the decoder. The decoder can construct the equiva- lent GBT matrix from the losslessly-encoded adjacency matrix (equivalently, edge map). For CS-GBT-1 and CS-GBT-2, one can choose between DCT, GBT, and CS to achieve the best performance. For example, for each block, the RD cost can be calculated for DCT, GBT, and CS, and the best approach with the smallest RD cost can be selected. The overhead indicating the chosen transform is encoded into the bitstream for each block, and the optimized adjacency matrix is provided only for blocks coded using GBT or CS. Similarly, for GBT, DCT and GBT are considered in the RD optimization and the optimized adjacency matrix is transmitted to the decoder only for blocks coded using GBT. We consider QP values of 24, 28, 32, and 36 to encode depth maps. As a reference, we also compare those approaches to H.264/AVC for the depth map compression. The reconstruction quality is evaluated using PSNR calculated by comparing the ground truth video and the synthesized video using the decoded depth maps. From the RD curves in Fig. 3.2, it is shown that the CS-GBT-2 outperforms H.264/AVC. Also, our proposed approach shows better performance than GBT and CS-GBT-1. This indicates that taking into account explicitly the mutual coherence, as we propose, leads to improvements over simply optimizing the GBT for sparsity (with the metric from [38]). Noticeable PSNR improvements over other methods Sequence BD-PSNR (dB) BD-bitrate (%) Ballet 0.9 dB -49.4 % Newspaper 1.5 dB -26.8 % Mobile 9.2 dB -42.8 % Average 3.9 dB -39.7 % Table 3.2: BD-PSNR/bitrate results of CS-GBT-2 compared to H.264/AVC. 76 Sequence BD-PSNR (dB) BD-bitrate (%) Ballet 0.3 dB -7.8 % Newspaper 0.9 dB -16.1 % Mobile 2.4 dB -9.7 % Average 1.2 dB -11.2 % Table 3.3: BD-PSNR/bitrate results of CS-GBT-2 compared to GBT. are observed because, with our optimized adjacency matrix for CS, more blocks are chosen to be coded using Hadamard measurements. The performance also depends on the number of strong edges in a frame and the level of noise around the edges. Among three sequences in our simulation, the Mobile sequence contains stronger edges along the object boundary with relatively less noise. Thus depth edges are better preserved than those in the other sequences, which leads to the best performance in Fig. 3.4. Also, the perceptual improvement in Ballet sequence is shown in Fig. 3.3. As marked by blue circles, we can notice clear edges reconstructed by our proposed approach. The results for three dierent sequences are shown in Tables 3.2 and 3.3 in terms of BD-PSNR and BD-bitrate. 3.5 Conclusion For depth map compression, we propose a novel CS approach where the adaptive GBT is used as a block-adaptive sparsifying basis. Based on the observation that maximum sparsity does not guarantee optimal performance when using CS, we propose a greedy algorithm that selects for each block a GBT that minimizes a metric that takes into consideration both the edge structure of the block and the 77 characteristics of the CS measurement matrix, using an estimate of average mu- tual coherence. As compared to coding using H.264/AVC, the proposed approach applied to intra-frames shows a signicant gain for interpolated views. 78 32.4 32.6 32.8 33 33.2 33.4 33.6 33.8 34 34.2 50 100 150 200 250 300 350 psnr(dB) kbps Ballet H.264/AVC GBT CS-GBT-1 CS-GBT-2 (a) Ballet 34 36 38 40 42 44 50 100 150 200 250 300 350 400 450 psnr(dB) kbps Newspaper H.264/AVC GBT CS-GBT-1 CS-GBT-2 (b) Newspaper 32 34 36 38 40 42 44 46 20 30 40 50 60 70 80 90 psnr(dB) kbps Mobile H.264/AVC GBT CS-GBT-1 CS-GBT-2 (c) Mobile Figure 3.2: RD performance comparison between i) H.264/AVC, ii) GBT [38], iii) CS-GBT-1, and iv) CS-GBT-2 for dierent sequences: (a) Ballet (b) Newspaper (c) Mobile. 79 (a) H.264/AVC (b) GBT (c) CS-GBT-2 Figure 3.3: Perceptual improvement in Ballet sequence (QP 24): comparison of i) H.264/AVC, ii) GBT, and iii) CS-GBT-2 80 (a) H.264/AVC (b) GBT (c) CS-GBT-2 Figure 3.4: The absolute dierence between the synthesized view with and without depth map compression in Mobile sequence (QP 24): comparison of i) H.264/AVC, ii) GBT, and iii) CS-GBT-2 81 Chapter 4 Hardware-driven Compressive Sampling for Fast Target Localization using Single-chip UWB Radar Sensor 4.1 Introduction Ultra-wideband (UWB) systems have been utilized for important applications such as radar tracking of objects and monitoring breathing or heartbeats of humans [14, 18]. For example, breathing monitoring can be achieved by localizing the subject's chest movement, which is critical for people who are under severe injury or sedation after surgery. This requires a precise and fast localization of objects with high resolution. Compared to available solutions using video camera techniques, UWB provides benets of higher spatial depth resolution [49]. In general, UWB radar sensors employ two types of detection schemes: (i) en- ergy detection [42] or (ii) direct sampling [15, 24]. Energy detection achieves low power consumption and has a simple architecture due to the nature of correlator- based detection circuitry. At the expense of higher power consumption, direct sampling enables reconstruction of the re ected waveform in the whole detection 82 range and therefore provides an opportunity for advanced signal processing to ex- tract additional information [14]. A practical challenge for UWB radar design is to overcome the low SNR from each received pulse due to the UWB emission spectrum mask posed by Federal Communications Commission (FCC) [17]. A recently developed hardware [15, 24] combines the direct sampling approach with a ranging technique (we will call the combined technique sequential sampling in the rest of this chapter). The ranging technique works by sending multiple pulses then averaging the received pulses in short time intervals (windows), each corresponding to a certain roundtrip time of the re ected pulse. Assuming the environment is relatively static, the receiver can localize an object at a specic distance by selecting the window corresponding to that distance and determining if the window contains re ected signal. The averaging within a chosen window provides robustness to noise. It also requires less power consumption, because power is only consumed during the measurement window, which can represent a small percentage of the overall operating time. However, a limitation of this scheme comes from sequential sampling, i.e., candidate object locations have to be probed in sequence, so that the time required to locate an object will be proportional to the number of measurement windows. In this work we propose a novel technique that can signicantly reduce the scanning time, with no increase in overall power consumption. The key observation is that in many situations the number of objects that can be observed is small relative to the number of locations that are probed. This allows us to probe several locations simultaneously, so that each measurement combines re ections at several distances. Processing can then be used extract the actual position information from the combined observations. Our approach is based on applying compressed 83 sensing (CS) principles with a design that is tightly coupled to the UWB hardware platform. In the context of radar applications, many researchers have proposed CS-based approaches that exploit the sparse structure of UWB signals [12, 23]. In [2], au- thors showed that the received signal can be digitized at a rate much lower than the Nyquist rate, without a need for matched lters. But important issues, such as performance in the high noise case and total power consumption are not consid- ered. Similarly, CS was applied to UWB detection applications, but with a mostly theoretical focus [36, 57, 61] or with experiments in a relatively simple environ- ment [77]. Also, a precise CS-UWB positioning system was proposed by exploiting the redundancy of UWB signal captured at multiple receivers to localize a trans- mitter [84, 85]. While this work achieved low ADC sampling rate, the rate is still higher than what can be achieved with the UWB hardware platform we build upon, and its performance is not as robust in high noise environments. As a CS approach tightly coupled to hardware, the Random-Modulation Pre-Integrator (RMPI) was proposed to achieve low-rate ADC by random modulation in analog domain [86{88] but the random modulation of signals contaminated by powerful noise in analog do- main does not provide robust signal recovery. Also, the analog random modulation does not provide the exibility to accommodate dierent sampling algorithms. In this work, we propose a CS technique tightly coupled to the capabilities of recently developed hardware [15,24] with the goal of achieving robustness to noise and low power consumption while providing reliable localization. To the best of our knowledge, this is the rst work that exploits UWB sparsity in the context of a ranging technique for an object localization application. More specically, there are three main contributions in this work. 84 First, we formulate the sparse structure of the signal of interest. The UWB signal is sparse if few objects are present because the UWB signal is highly localized in time. Combined with the UWB ranging system, this leads to a special structure where sparse non-zero entries are clustered into a few groups (windows). The number of windows where non-zero entries occur is equal to the number of objects in the region of interest. More details about our representation of UWB signals will be discussed in Section 4.2.2. Second, we design an ecient measurement system subject to several constraints imposed by the hardware. The constraints include (i) non-negative integer entries in the sensing matrix (ii) constant row-wise sum of entries in the matrix (iii) non-zero entries of each row can exist only at positions with a constant shift, which leads to a unique structure characterized by a Kronecker product. Under these constraints, we construct a sensing matrix by using a low-density parity-check (LDPC) matrix that was recently shown to lead to a good measurement system in [20,50]. Third, in order to enhance the localization performance, we propose a window- based reweightedL 1 minimization and show that it provides good performance for the abovementioned signal model and measurement system. In our simulations, we compare our proposed method with other existing reconstruction algorithms in terms of several metrics for evaluating localization performance. Our simula- tion results show that our proposed method can achieve reliable target-localization while using only 40% of the sampling time required by the corresponding sequential scanning scheme, even in a highly-noisy environment. The rest of this chapter is organized as follows. In Section 4.2, we formulate the UWB ranging system in CS framework and provide how to approximate the total scanning time. Then we describe our proposed approach in Section 4.3 and the result is veried in our simulation in Section 4.4. 85 4.2 Problem Formulation 4.2.1 UWB Ranging System and Assumptions We start by describing the hardware design in [15, 24], which serves as the basis for our design. This system can probe the presence of objects in a certain distance from the receiver as illustrated in Fig. 4.1. In this approach, short-time pulses are transmitted periodically. Assume that we would like to determine whether an object is present at a given distance range from the receiver. Given such a range of distances, we know that the re ected pulse will have a certain roundtrip time that will fall into a short time interval (window). Denoting cycle the interval between successive transmitted pulses, as shown in Fig. 4.1, we divide a cycle into windows, each corresponding to a small distance range. Figure 4.1: Basic motivation for UWB ranging system. After a pulse is transmitted, the observation time for the receiver can be divided into non-overlapped windows. Since the pulses and their re ections are narrow, we assume that the re ection localized in one of the windows. If a range is chosen, the receiver is able to measure a re ected signal during a specic window corresponding to the roundtrip time for that range. Within each cycle, the receiver only consumes power during a window, leading to low overall power consumption. This hardware design for object localization application is based on two assump- tions; (i) the environment and the objects of interest remain stationary within the time that multiple short-time pulses are transmitted, and (ii) the re ected pulses 86 reside within one of the windows. The rst assumption is valid in practice because the hardware operates at picosecond scale for measuring the re ection in each cycle thus scanning more than 100 windows over multiple cycles can nish in the order of nanoseconds [24]. This ne-scale operation in time enables us to apply our pro- posed approach to practical applications such as breathing monitoring. The second assumption is from the observation that the transmitted pulses and their re ections are narrow compared to the time period of each window. Under the aforementioned assumptions, the hardware design has several advan- tages: low power consumption, robustness to noise, and exibility to accommodate dierent sampling algorithms. By selecting a specic range, the system cannot ob- serve objects at other distances, because measurements are performed only within the chosen window. But, as a consequence, power consumption is signicantly re- duced, since no power is consumed during other window intervals, while averaging over multiple cycles increases robustness to noise, e.g., thermal noise from circuits, re ection from objects that are not of interest, etc. Note that noise can be signif- icant in these scenarios. For example, the lower bound of SNR with the hardware design is about21dB based on the discussion in [41]. Note that the system does not consume extra energy when the sampling switches from one window to another. This is an important fact which allows us to combine measurements over multiple windows as we propose next. 4.2.2 Window-based Sparse UWB Signal Model Without noise, the sampled signal,x, will be sparse because of two main reasons. First, UWB pulses are very narrow in time, so that the received signals are them- selves sparse in the time domain, i.e., re ected pulses corresponding to an object 87 of interest are present in a short time interval. Second, the number of objects of interest is small compared to the number of windows. Thus,x has a special struc- ture such that sparse non-zero entries are clustered within a few windows, with the number of windows where re ections are present being equal to the number of objects in the region of interest. In this work, we mainly focus on the second source of sparsity, leading to window-based sparse signal model. Tx Rx window Nw cycle 1 window 1 … window Nw cycle 2 window 1 … … window Nw cycle Nc window 1 … Figure 4.2: Illustration of the UWB sampling system with the parameters:N C and N W . ThroughoutN C cycles, one pulse is periodically transmitted at the beginning of each cycle which consists of N W non-overlapped windows. After taking summa- tion of the re ections duringN C cycles in analog domain,N S samples are collected in each window. Let N S be the number of samples in each window, and assume that the UWB- ranging system hasN W non-overlapped windows in each cycle as shown in Fig. 4.2. Assume that each cycle is long enough that it can capture all re ections of interest. For example, this would mean that, in an indoor environment, the cycle would be long enough to receive a re ection from the furthest point in the environment. To introduce a signal to be reconstructed for the object localization, we rst dene the signal in noise-free environment then we will extend it to the actual measured signal with noise in Section 4.2.3. Suppose that we observe a noise-free signalx i;j in the i th window during the j th cycle, wherex i;j is a vector with N S samples. Under the assumption that the environment and the objects of interest 88 are stationary, we can approximate that the signals (i.e.,x i;j ;8j2f1; 2; ;N C g) observed in the i th window over N C cycles are identical tox[i]: x[i] =x i;1 =x i;2 = =x i;N W ;8i2f1; 2; ;N W g (4.1) Thus, dene a signal,x, by concatenating N W sub-signals,x[i];i2 1;:::;N W : x T = [x 1 ;:::;x N S | {z } x[1] ;:::;x NN S +1 ;:::;x N | {z } x[N W ] ] T (4.2) Since everyx[i] has lengthN S , the dimension ofx isN =N S N W . Our window- based signal model is similar to models such as block-sparsity, cluster-sparsity, or multiple measurement vector (MMV) model [3, 28, 70]. Compared to those signal models, our sparse signal,x, has a common characteristic such that non-zero entries are grouped into a few (sparse) blocks, but the blocks are not overlapped with each other and their sizes are identical. Note that, to the best of our knowledge, this signal model has not been applied to a realistic UWB hardware with ranging capability. 4.2.3 UWB Measurement System and Matrix Formulation In this work, we propose hardware-driven compressive sampling (HDCS) as an alternative to the sequential sampling scheme presented in [15]. The sequential sampling scheme scans the same window in each of theN C cycles. This is repeated for every window until all the windows are scanned. In contrast, the HDCS scheme collects information about multiple windows over the same N C cycles by scanning one window per cycle but switching the window to be scanned over the course of N C cycles. 89 Before deriving a general formulation, we rst consider the sequential sampling scheme. Let N C be the number of cycles over which the receiver integrates before the ADC is activated. After the integrated analog waveform is sampled by the ADC, the measurements obtained from the i th window,y i , can be represented as a linear combination ofx i;j with i.i.d. Gaussian noise,n j : y i = N C X j=1 (x i;j +n i;j ) ;8i2f1; 2; ;N W g (4.3) Since we assume a stationary environment where the signal to be reconstructed does not change over time and the sequential sampling scans each window overN C cycles, we can simplify (4.3) by using (4.1): y i =N C x[i] +n i ; (4.4) wheren i , is a summation of random variables following i.i.d. Gaussian distribution: n i = P N C j=1 n i;j ; n i;j N(0, 2 N ). This process can be interpreted as a diagonal sensing matrix with diagonal term N C by concatenatingy i as follows: y = 2 6 6 6 6 6 6 6 4 y 1 y 2 . . . y N W 3 7 7 7 7 7 7 7 5 = 2 6 6 6 6 6 6 6 4 N C I 0 0 0 N C I 0 . . . . . . . . . . . . 0 0 N C I 3 7 7 7 7 7 7 7 5 NN 2 6 6 6 6 6 6 6 4 x[1] x[2] . . . x[N W ] 3 7 7 7 7 7 7 7 5 + 2 6 6 6 6 6 6 6 4 n 1 n 2 . . . n N W 3 7 7 7 7 7 7 7 5 (4.5) The sequential sampling of (4.5) involves a diagonal matrix (because each win- dow is separately sampled) with equal diagonal terms because all windows are observed the same number of times (N C ). However, with our proposed HDCS scheme, the matrix does not necessarily have zero o-diagonal terms, since one 90 measurement,y i , can include information from multiple windows. Thus, a general HDCS scheme can be represented as follows: y = 2 6 6 6 6 6 6 6 4 y 1 y 2 . . . y M W 3 7 7 7 7 7 7 7 5 = 2 6 6 6 6 6 6 6 4 N (1;1) C I N (1;2) C I N (1;N W ) C I N (2;1) C I N (2;2) C I N (2;N W ) C I . . . . . . . . . . . . N (M W ;1) C I N (M W ;2) C I N (M W ;N W ) C I 3 7 7 7 7 7 7 7 5 MN 2 6 6 6 6 6 6 6 4 x[1] x[2] . . . x[M W ] 3 7 7 7 7 7 7 7 5 + 2 6 6 6 6 6 6 6 4 n 1 n 2 . . . n M W 3 7 7 7 7 7 7 7 5 ; (4.6) where P N W j=1 N (i;j) C =N C ;8i2f1; 2; ;M W g and M W <N W (equivalently, M < N). N (i;j) C represents the number of times information from window j is included in measurement i. In general, M W < N W because we can reduce the number of measurements by aggregating information from multiple windows. Both sampling schemes can be represented by a sensing matrix, and a noise vector,n: y = x +n (4.7) Since the UWB ranging hardware of interest obtains measurements under the as- sumption that the whole re ection is always captured in a window, we can formulate as a matrix containing blocks with dimension N S -by-N S , each corresponding to a specic \ranging" window in the sequential sampling or a specic \measurement" window in the HDCS sampling. Note that there exist two kinds of windows. In HDCS, a \measurement" window aggregates information from multiple \ranging" windows. Also, contains non-negative integer entries indicating the number of cycles to be integrated in order to obtain measurements. More specically, (i;j) indicates the number of cycles over whichx(j) is integrated in order to obtainy(i). 91 Thus, we can easily compute the total scanning time by taking summation of all the entries in : P 8i;j (i;j). W1 W1 W1 W1 W1 W2 ... ... ... W2 W1 W2 W2 W2 W3 W2 W3 W3 W3 W4 ... W4 ... ... ... W4 W1 W3 W4 W4 W4 ... 5 cycles time Sequential sampling Reduced Sampling Time !! ... HDCS sampling Figure 4.3: Advantage of HDCS scheme. This shows comparison of two dierent samping schemes formulated as two matrices in (4.8). While Sequential sampling scans a single window during 5 cycles, HDCS collects information from multiple windows during 5 cycles, which could lead to power savings. The sequential sampling scheme scans a window over N C cycles until all win- dows have been scanned. Thus, this process can be interpreted as a diagonal sensing matrix with the N C on its diagonal as in (4.5) whose total scanning time is N C N(= N C N S N W ). On the contrary, the HDCS scheme collects information about multiple windows from a measurement by scanning a certain combinations of windows during N C cycles. Note that the intuition of HDSC is similar to that in Chapter 2, where energy-ecient data gathering can be achieved by collecting measurements from multiple clusters in a wireless sensor network. Thus, the HDCS can expedite the scanning procedure by reducing the number of measurements as illustrated in Fig. 4.3. For better understanding, we provide an example illustrated in Fig. 4.3, as- suming that we obtain a measurement from 4 windows consisting of 4 samples 92 throughout 5 cycles: N S = 4, N W = 4, and N C = 5. This leads to two sensing matrices, 1 and 2 with sequential sampling and HDCS sampling, respectively: 1 = 2 6 6 6 6 6 6 6 4 5I 0 0 0 0 5I 0 0 0 0 5I 0 0 0 0 5I 3 7 7 7 7 7 7 7 5 1616 ; 2 = 2 6 6 6 6 4 1I 3I 1I 0 0 1I 3I 1I 1I 0 1I 3I 3 7 7 7 7 5 1216 (4.8) Both 1 and 2 contain 4-by-4 identity matrices because we consider 4 samples in each window. As discussed earlier, the sequential sampling scheme generates a diagonal sensing matrix, 1 , with 5 (=N C ) on its diagonal. However, with HDCS scheme, 2 has non-zero o-diagonal terms. If 2 can give us the same level of reconstruction as 1 , we can achieve 0:75 scanning time reduction because, with HDCS, more windows are measured during 5 cycles for each measurement as shown in Fig. 4.3. The decrease of scanning time leads to the same amount of total power reduction. Now, the challenge is how to design a good sensing matrix satisfying the con- straints imposed by the hardware. In other words, arbitrary combinations of win- dows in the HDCS are not guaranteed to provide a reliable localization performance because CS reconstruction performance depends on the coherence of the sensing matrix. For example, there can be a better combination of the windows than the [1; 3; 1] combination in 2 in (4.8). Thus, we propose to optimize a sensing matrix for the HDCS in order to provide both faster and more reliable localization of the objects. The optimization for better performance will be discussed in more detail next. 93 4.3 Proposed Approach 4.3.1 LDPC Measurement System In order to reduce the scanning time, the challenge is how to design a measurement mechanism that can achieve successful reconstruction with fewer measurements. With traditional CS, the random matrices, such as Gaussian random matrix and (uniform randomly) down-sampled Fourier matrix, have been exploited as sensing matrices because they satisfy the restricted isometry property (RIP) with high probability [23]. However, combined with UWB ranging system [15], these popular sensing ma- trices are no longer appropriate due to additional constraints: i) all the entries of the matrix should be non-negative integers because the entries indicate the num- ber of cycles. This condition rules out popular sensing matrices such as random or Fourier matrices that have real entries. ii) the sum of entries in each row is xed as a constant number of cycles,N C . Thus, the scanning time is directly proportional to the number of rows in the sensing matrix. iii) non-zero entries of each row can exist only at the positions with constant shift of N S . Thus, sensing matrix, , can be formulated as a Kronecker product of the identity matrix with a matrix containing coecients at the corresponding positions,A (generating sensing matrix): MN = 2 6 6 6 6 4 a (1;1) I ::: a (1;N W ) I . . . . . . . . . a (M W ;1) I ::: a (M W ;N W ) I 3 7 7 7 7 5 =A M W N W I N S (4.9) For example, the matrices in (4.8) can be represented following (4.9) as: 94 1 = 2 6 6 6 6 6 6 6 4 5 0 0 0 0 5 0 0 0 0 5 0 0 0 0 5 3 7 7 7 7 7 7 7 5 44 I 44 (4.10) 2 = 2 6 6 6 6 4 1 3 1 0 0 1 3 1 1 0 1 3 3 7 7 7 7 5 34 I 44 (4.11) To satisfy these constraints, we propose to adopt low-density parity-check (LDPC) measurement system recently studied in [20,50]. In [20], the authors provide strong theoretical results showing that parity-check matrices corresponding to good chan- nel codes can be used as provably good measurement matrices using basis pursuit reconstruction. In [50], the authors show that LDPC matrices signicantly outper- form other current CS matrices. Thus, by using an LDPC matrix as the generating sensing matrix,A, we can construct a good sensing matrix, , using (4.9), because the coherence of sensing matrix, , is the same as that of the generating sensing matrix,A. This can be easily shown as U = T = (A I) T (A I) =A T A I =U A I Thus, if A is a good CS measurement matrix, then in (4.9) is also a good sensing matrix. Also, since the LDPC matrices,A, have the same number of 1's in each row, the resulting measurement matrices, =A I, also satisfy the second condition. 95 4.3.2 Window-based Reweighted L 1 Minimization As discussed earlier, our goal is to localize the objects in space. Equivalently, this means that we want to identify the data support (DS) which contains the non-zero entries, since this data support directly corresponds to the locations of objects in space. To identify data support with the sequential sampling scheme in [15], we rst reconstruct a signal, ^ x, by dividing integrated measurements by the number of cycles, N C . Then, thresholding is applied with an empirically chosen threshold in order to determine data support. As shown in Fig. 4.4, we can achieve higher SNR by increasing the number of cycles, because the noise can be approximated as i.i.d. Gaussian noise as in [30, 37]. However, a larger number of cycles results in longer acquisition time (or higher power consumption) and lower temporal resolution if objects are moving. 0 500 1000 1500 2000 2500 −50 −40 −30 −20 −10 0 10 20 30 40 50 (a) Without averaging 0 500 1000 1500 2000 2500 −15 −10 −5 0 5 10 15 (b) After 500 iterations Figure 4.4: Eect of averaging in the sequential sampling scheme [15]: (a) The red plot indicates the original signal (ground truth) and the blue indicates measured signal including the ground truth plus strong noise, SNR=21:5dB (b) Result of sequential sampling after averaging over 500 cycles. 96 0 50 100 150 200 250 300 350 400 450 500 −25 −20 −15 −10 −5 0 Number of Cycles (N S ) SNR (dB) Figure 4.5: Eect of averaging in a sequential sampling scheme [15]. SNR increases as more samples are averaged (i.e.,the number of cycles increases) with high noise level ( N = 30) and system parameters (N S = 16, N W = 155). Note that even after 500 averaging operations, SNR is0:63dB in this example. With the HDCS scheme discussed in Section 4.3.1, we propose a two-phase lo- calization process comprising: (i) non-linear signal reconstruction and (ii) thresh- olding. First, we reconstruct signal, ^ x, from M measurements then we identify the data support by a simple thresholding. For signal reconstruction, we solve a non-linear optimization problem to nd a solution to the under-determined system. For thresholding, a window is chosen as one of the possible data supports if the energy of the window is greater than a small value,k^ x[k]k 2 > 0:001. Note that the threshold is xed throughout this work, and is not changed according to dierent noise level or dierent M. For successful reconstruction of the signal, several previous works show promis- ing results (refer to [70] for details). In [28] L 2 =L 1 minimization was based on a 97 block-sparsity model. This approach seeks to minimize the sum of the L 2 norm of the signal over several windows subject to the data-tting constraint: min N W X i=1 kx[i]k 2 s:t: kxyk 2 : Theoretical results in [28] show that signals with block-sparsity can be success- fully reconstructed with a sensing matrix satisfying block RIP. The signal model in [28] can also be applied to our problem because the re ection signal has a few non-zero entries which are clustered within a few windows. However, noisy mea- surements or integer sensing matrices are not considered in [28]. Another approach is iterative reweighted L 1 minimization (RL 1 ) [9, 71]. For the i th iteration, this approach minimizes the L 1 norm of weighted sum of intermediate x i , subject to data-tting constraint: mink N X j=1 W i (j)x i (j)k 1 s:t: kxyk 2 : The weight at (i + 1) th iteration,W i+1 , is computed as the inverse of the absolute value of x i at previous iteration: W i+1 (j) = 1 jx i (j)j + ; j2f1;:::;Ng Here, is a small regularization term that prevents an innite weight term from occurring whenx i is zero. The weight increases as the intermediate result becomes smaller, thus this leads to a solution closer to that of L 0 minimization because large values of x i contribute to the metric as much as smaller values. However, since the weight update is an entry-wise operation, the windows of interest are not successfully identied when noise levels are high. 98 This intuition encouraged us to use an iterative window-based reweighted L 1 minimization (WRL 1 ). The algorithm uses the reweighting technique that has been used in iterative reweightedL 1 minimization with the only dierence being that the weight is computed by window-wise operation; the weights for the entries belonging to the k th window are computed as the L 1 norm of the partial intermediate signal, x[k], within that window. The weight vector of thek th window at thei th iteration, W i [k], is W i [k] = 1 1 kx i1 [k]k 1 + ; k2f1;:::;N W g (4.12) Here 1 is a vector of dimension N S with all entries equal to 1. The weights for the entries in the same window are updated with the same value. This window-based updating scheme was proposed as the adaptive group Lasso algorithm in [91] or reweighted M-Basis Pursuit in [70], but, to the best of our knowledge, we are the rst to apply this reconstruction technique to UWB signal reconstruction. 4.4 Simulation Results In our simulations, we consider realistic parameters such that 155 windows contain- ing 16 samples in each window (N S = 16, N W = 155), which are similar to those used in the design of UWB radar hardware that we take as a starting point [15,24] (16 samples in each of 128 windows). In [15, 24] it was shown that the entire de- tectable range is 15m and the range can be divided to a specic number of windows depending on the requirements of applications. With 155 windows in UWB radar hardware, the maximum spatial depth resolution is 9.6cm, which is desirable for object localization applications in an indoor environment. Also we assume that each measurement is obtained from 500 cycles: N C = 500. In the simulation, the goal is to localize three objects in the region of interest, which 99 is the same as nding three windows in signal, x. We generated a data set of 80 realizations and each data contains three windows with non-zero entries indicating three objects in space; those windows are chosen randomly with a uniform distri- bution and the values of non-zero entries are generated by Gaussian distribution. 0 2 4 6 8 10 0.2 0.4 0.6 MSE Sampling Time Ratio SEQ HDCS_L1 HDCS_L2/L1 HDCS_RL1 HDCS_WRL1 Figure 4.6: Cost ratio vs. MSE: For CS sampling schemes, cost is the total sampling time to collect M measurements. Since we x the number of cycles, N C , for every measurement as 500 in the simulation, the cost ratio is a ratio of the number of measurements to the dimension of signal, M=N. But, for sequential sampling scheme (noted 'SEQ' in the gure), we take N measurements with reduced N C . For the measurement system, we adopt an LDPC matrix as discussed in Sec- tion 4.3.1. For simulation, we rst construct the generating sensing matrix, A, using LDPC matrices with dierent number of rows, M w , by changing the number of 1's in each column from 1 to 3 with that in each row xed as 5. Then, the measurement matrix, , is constructed by = 100A I N S as in (4.9), where the constant multiplier is N C =5 = 100. Also, we consider noisy measurements with three dierent noise levels, N 2 f10; 20; 30g, which generates very low SNR (approximately16:5dB,22:6dB and 26:1dB respectively on average over our data set). For object localization, we 100 rst reconstruct ^ x using L 1 minimization techniques then identify the data sup- port by examining L 2 norm of signals within each window, ^ x[k], as discussed in Section 4.3.2. For reconstruction, we compare window-based reweighted L 1 mini- mization (HDCS WRL 1 ), with three other algorithms: traditional L 1 minimization (HDCS L 1 ), L 2 =L 1 minimization (HDCS L 2 =L 1 ), and reweighted L 1 minimization (HDCS RL 1 ). To evaluate performance, we need to measure localization quality as well as scanning time. The scanning time can be easily computed by counting the num- ber of rows of sensing matrix, , because the number of cycles is the same for all approaches (N C = 500). Thus, the cost ratio is a ratio of the number of measure- ments to the dimension of signal, M=N. To evaluate localization quality, Mean Squared Error (MSE) can be used by measuring the entry-wise dierence of values betweenx and ^ x. Fig. 4.6 shows the comparison of performance between our pro- posed reconstruction technique, HDCS WRL 1 , and other reconstruction techniques, in terms of MSE and scanning time. Note that for the sequential sampling scheme (noted 'SEQ' in Fig. 4.6), we collect N measurements with the reduced number of cycles,N 0 C in order to compare to HDCS sampling at dierent total scanning time. Since the sequential sampling scheme requires to scan all the windows, we reduce the number of cycles, N C , to N 0 C = round(N C M N ) then construct a measurement matrix as: seq =N 0 C 2 6 6 6 6 6 6 6 4 I 0 0 0 I 0 0 . . . 0 . . . . . . 0 0 I 3 7 7 7 7 7 7 7 5 NN (4.13) Since we consider three dierent sampling time ratios, M=N 2f0:2; 0:4; 0:6g, in Fig 4.6, the resultingN 0 C 2f100; 200; 300g. Fig. 4.6 shows that HDCS schemes with 101 dierent reconstruction techniques achieve about ve times better reconstruction quality with similar scanning time. Although MSE is one of the most generic metric for reconstruction evaluation, it can be misleading because smaller MSE does not always guarantee better window identication. For example, perfect identication of data support (DS) can result in large MSE if a large dierence exists betweenx and ^ x within the DS. HDCS L 1 HDCS L 2 =L 1 HDCS RL 1 HDCS WRL 1 avg. mismatch (a) 537.539 132.629 160.562 4.258 avg. no. of DS (b) 26.461 8.124 10.562 3.067 mismatch / DS (a/b) 20.378 16.090 15.065 1.197 max. mismatch 18.371 5.202 5.910 0.989 F-measure 0.206 0.546 0.442 0.943 Table 4.1: Performance evaluation of identication of data support (DS): The acqui- sition time is reduced to 0.4 compared to sequential sampling scheme, M=N = 0:4, with noisy measurements, N = 30. Thus, we consider additional metrics to evaluate mismatch of data support (DS). In the comparisons, we rule out the averaging method in sequential sampling scheme because it requires a good threshold which should be adaptive to param- eters such as the level of noise and number of cycles, N C . First, we compute the number of candidates for DS in terms of the number of windows. As discussed earlier, the windows containing non-zero entries are formed as candidates for the estimated DS. Second, for each candidate, we compute minimum distance to any of the correct DS (ground truth). The distance is computed in terms of the number of windows. Then, the average of minimum distances can be interpreted as a metric to evaluate the error in object location in space. Third, we also consider the maxi- mum mismatch of data support by taking the maximum of the minimum distances 102 HDCS L 1 HDCS L 2 =L 1 HDCS RL 1 HDCS WRL 1 avg. mismatch (a) 345.94 42.28 94.48 5.15 avg. no. of DS (b) 25.30 6.30 10.91 3.21 mismatch / DS (a/b) 13.56 6.74 8.45 1.59 max. mismatch 116 93 95 78 F-measure 0.16 0.23 0.27 0.32 Table 4.2: Performance evaluation of identication of data support (DS): The acqui- sition time is reduced to 0.2 compared to sequential sampling scheme, M=N = 0:3, with noisy measurements, N = 20. between the candidates and the ground truth to evaluate the performance in the worst case. Lastly, we compute F-measure discussed in [70] as 2 jsupp(x)\supp(^ x)j jsupp(x)j +jsupp(^ x)j ; (4.14) where supp(x) =fi2 [1;:::;N W ] :kx[i]k 2 > 0:001g. Note that the F-measure is equal to 1 when the data support of the reconstructed signal coincides exactly with the ground truth. Table 4.1 shows the performance with respect to the metrics evaluating the ability to identify data support (DS). In the result, HDCS L 1 and HDCS L 2 =L 1 ap- proaches are non-iterative methods while the others are iterative algorithms, where the results after three iterations are presented. As shown in Tables 4.1 and 4.2, our proposed reconstruction technique outperforms all the other methods with respect to all the metrics we consider. Especially, the maximum mismatch of data support shows the performance in the worst case. In Table 4.1, HDCS WRL 1 achieves a maximum mismatch of 0.98 windows on average, with only 40% of acquisition time needed for sequential sampling. This means that the average error in location is of the distance represented by a window in the worst case scenario. 103 Figs. 4.7 and 4.8 compare performance with respect to abovementioned two metrics. In Fig. 4.7(a),HDCS WRL 1 shows very small maximum mismatch at every noise level which is almost equal to 1. This indicates that our identied windows are mismatched at most by one window on average over 80 data. Also,HDCS WRL 1 shows very stable performance at dierent noise levels. Fig. 4.7(b) shows that, in the highest level of noise we tested, HDCS WRL 1 shows the best performance and it reaches to almost perfect reconstruction at 0:6 sampling time ratio. Similarly, in Fig. 4.8(a),HDCS WRL 1 shows the highest F-measure at every noise level which is very close to 1. Also, it does not drop as the noise level increases as shown in Fig. 4.8(b). 4.5 Conclusion To design an energy-ecient UWB ranging system, we propose a CS approach combined with a novel hardware architecture. we rst formulate UWB signal rep- resentation with a special structure such that sparse non-zero entries are clustered into a few groups. Also, we design an ecient measurement system that is con- structed by an LDPC matrix, which satises several constraints imposed by the hardware. To enhance performance, we propose a window-based reweighted L 1 minimization which outperforms other existing algorithms in our simulation. The result shows that our proposed method can achieve reliable target-localization while requiring only 40% of sampling time of the sequential sampling scheme in highly- noisy environment. 104 0 5 10 15 20 25 30 35 10 20 30 Max. Mis/Hit Noise Level HDCS_L1 HDCS_L2/L1 HDCS_RL1 HDCS_WRL1 (a) N vs. max. mismatch 0 5 10 15 20 25 30 35 0.2 0.4 0.6 Max. Mis/Hit Sampling Time Ratio HDCS_L1 HDCS_L2/L1 HDCS_RL1 HDCS_WRL1 (b) sampling time ratio vs. max. mismatch Figure 4.7: Performance comparison with respect to maximum mismatch of data support: (a) Fix sampling time ratio as 0:4 and compare performance at dierent noise levels. (b) Fix noise level as 30 and compare performance at dierent sampling time ratios. 105 0.0 0.2 0.4 0.6 0.8 1.0 10 20 30 F-measure Noise Level HDCS_L1 HDCS_L2/L1 HDCS_RL1 HDCS_WRL1 (a) N vs. F-measure 0.0 0.2 0.4 0.6 0.8 1.0 0.2 0.4 0.6 F-measure Sampling Time Ratio HDCS_L1 HDCS_L2/L1 HDCS_RL1 HDCS_WRL1 (b) sampling time ratio vs. F-measure Figure 4.8: Performance comparison with respect to F-measure: (a) Fix sampling time ratio as 0:4 and compare performance at dierent noise levels. (b) Fix noise level as 30 and compare performance at dierent sampling time ratios. 106 Chapter 5 Conclusions and Future Work 5.1 Conclusion We propose to optimize Compressed Sensing (CS) design choices under the con- straints driven by various applications. Unlike in traditional CS approaches, we consider additional constraints driven by dierent application requirements such as transport cost in wireless sensor network, total bitrate in depth map compres- sion, or scanning time in UWB ranging system. In these applications, reducing the number of measurements does not always provide a better solution if application specic constraints are not considered. As we discussed earlier, in the application of WSN, the transport cost to collect the information from the sensors should be considered for the overall performance of data gathering system and this is closely related to the construction of measurement matrix. Thus, we propose to optimize the measurement matrix with respect to the given sparsifying matrix for the joint optimization of the transport cost and the reconstruction accuracy of sensor data for the data gathering. Our proposed approach achieves better performance, as compared to other existing CS techniques related to data gathering of sensor data in wireless sensor network. We also propose 107 a heuristic approach applied to a practical situation where sensors are randomly deployed over a eld of interest. For depth map compression, the additional cost we need to consider is the bit overhead to code the edge map required for the construction of graph-based trans- form (GBT). In contrast to the WSN application, we x the sensing matrix as a Hadamard matrix and we optimize the sparsifying matrix to reduce the number of bits to represent depth map signal while achieving satisfactory reconstruction. We propose a greedy algorithm to optimize the joint optimization of the bit over- head and the reconstruction quality of depth map data and achieve a signicant improvement over H.264/AVC. For fast localization of objects using a UWB ranging system, we propose the design of an ecient measurement system that is constructed using low-density parity-check (LDPC) matrix, designed to satisfy several hardware-related con- straints: non-negative integer entries in measurement (sensing) matrix, constant row-wise sum of non-zero entries in the matrix, and a unique structure character- ized by Kronecker product. To enhance performance, we propose a window-based reweightedL 1 minimization that outperforms other existing algorithms in our sim- ulation. The result shows that our proposed method can achieve reliable target- localization, while using only 40% of the scanning (sampling) time required by the sequential scanning scheme, even in high noise environments. 5.2 Future Work The approaches for WSN and depth map compression are related to optimization of compressed sensing with respect to dierent constraints. Although we extended our work to a more general situation where the sensors are deployed at arbitrary 108 positions, the heuristic solution needs to be further investigated in order to gen- eralize the approach by optimizing spatially-localized projection with respect to a given GBT. Since GBT is a tool to sparsify any generic signal dened on arbitrary graph, the generalized approach can also be applied to simplied cases such as both WSN and depth map compression. Furthermore, the generalized framework can be extended to local construction of GBT as shown in [59]. Instead of global construction of GBT, GBT can be constructed with partial information about neighboring nodes in the graph. Thus, investigating how to optimize spatially-localized projection combined with localized GBT will be meaningful, which will lead to localized optimization of the general framework with less complexity. While UWB radar results are promising, many questions remain. In practice, reconstruction techniques with less complexity will be required, so that they can run in real-time. Also, in order to verify the performance, integration with the hardware will be needed in practice. In addition, a theoretical analysis of the system to evaluate the potential localization performance by this reduced number of measurements or dierent LDPC matrices will be meaningful. 109 Reference List [1] R. Baraniuk. Compressive sensing. IEEE Signal Processing Magazine, 24(4):118{121, Jul. 2007. [2] R. Baraniuk and P. Steeghs. Compressive radar imaging. In IEEE Radar Conf, pages 128{133, 2007. [3] R.G. Baraniuk, V. Cevher, M. Duarte, and C. Hegde. Model-based com- pressive sensing. IEEE Transactions on Information Theory, 56(4):1982{2001, 2010. [4] D. Baron, M. B. Wakin, M. F. Duarte, S. Sarvotham, and R. G. Baraniuk. Distributed compressed sensing. arXiv:0901.3403, Jan. 2009. [5] MOSEK C-library. www.mosek.com. [6] E. Cand es. Compressive sampling. Int. Congress of Mathematics, 3:1433{ 1452, 2006. [7] E. Cand es and D. Donoho. Curvelets - a surprisingly eective nonadaptive representation for objects with edges. Vanderbilt University Press, 1999. [8] E. Cand es and J. Romberg. Sparsity and incoherence in compressive sampling. In Inverse Problems, June 2007. [9] E. J. Cand es, M. B. Wakin, and S. Boyd. Enhancing sparsity by reweighted l1 minimization. Journal of Fourier Analysis and Applications,, 14(5):877{905, Dec. 2008. [10] E.J. Cand es. Compressive sampling. International Congress of Mathemati- cians, 3:14331452, 2006. [11] E.J. Cand es, J. Romberg, and T. Tao. Robust uncertainity principles : exact signal reconstruction from highly incomplete frequency information. In IEEE Transactions on Information Theory, February 2006. [12] E.J. Cand es, J.K. Romberg, and T. Tao. Stable signal recovery from incom- plete and inaccurate measurements. Communications on Pure and Applied Mathematics, 59(8):1207, 2006. 110 [13] E.J. Cand es and T. Tao. Near-optimal signal recovery from random projections and universal encoding strategies. IEEE Trans. Inform. Theory, 52(12):5406 { 5425, 2006. [14] S. Chang, M. Wolf, and J. W. Burdick. Human detection and tracking via ultra-wideband (UWB) radar. IEEE International Conference on Robotics and Automation, May 2010. [15] T. Chu, J. Roderick, S. Chang, T. Mercer, C. Du, and H. Hashemi. A short- range UWB IR CMOS sensor for human feature detection. IEEE International Solid-State Circuits Conference Digest of Technical Papers, pages 294{295, Feb. 2011. [16] A. Ciancio, S. Pattem, A. Ortega, and B. Krishnamachari. Energy-ecient data representation and routing for wireless sensor networks based on a dis- tributed wavelet compression algorithm. In IPSN, April 2006. [17] Federal Communications Commission. First report and order, revision of part 15 of the commission's rules regarding ultra wideband transmission systems. ET Docket, pages 98{153, Feb. 2002. [18] E. Conti, A. Filippi, and S. Tomasin. On the modulation of ultra wide band pulse radar signal by target vital signs. International Symposium on Bioelec- tronics and Bioinformatics, Dec. 2009. [19] R. Cristescu, B. Beferull-Lozano, and M. Vetterli. On network correlated data gathering. In INFOCOM, March 2004. [20] A. G. Dimakis, R. Smarandache, and P. O. Vontobel. LDPC codes for com- pressed sensing. IEEE Transactions on Information Theory, 58(5):3093{3114, May 2012. [21] M. N. Do and M. Vetterli. The contourlet transform: An ecient directional multiresolution image representation. IEEE Trans. Image Proc., 14(12):2091{ 2106, Dec. 2005. [22] T.T. Do, T.D. Tran, and Lu Gan. Fast compressive sampling with structurally random matrices. In ICASSP, April 2008. [23] D. L. Donoho. Compressed sensing. In IEEE Transactions on Information Theory, April 2006. [24] C. Du and H. Hashemi. An UWB CMOS impulse radar. To appear in IEEE Radio Frequency Integrated Circuits Symposium, Jun, 2013. 111 [25] J. Duan, L. Zhang, R. Pan, and Y. Sun. An improved video coding scheme for depth map sequences based on compressed sensing. In International Con- ference on Multimedia Technology (ICMT), Hangzhou, Aug. 2011. [26] M. F. Duarte, M. B. Wakin, D. Baron, and R. G. Baraniuk. Universal dis- tributed sensing via random projections. In IPSN, April 2006. [27] M. Elad. Optimized projections for compressed sensing. IEEE Trans. on Signal Processing, 55(12):5695{5702, 2007. [28] Y. Eldar and M. Mishali. Robust recovery of signals from a structured union of subspaces. IEEE Transactions on Information Theory, 55(11):5302{5316, Nov 2009. [29] M.A.T. Figueiredo, R.D. Nowak, and S.J. Wright. Gradient projection for sparse reconstruction:application to compressed sensing and other inverse problems. In IEEE Journal of Selected Topics in Signal Processing, 2007. [30] J. Foerster. Channel modeling sub-committee report nal. IEEE 802.15.SG3a Study Group, Dec. 2002. [31] J. Fu and B. Zeng. Directional discrete cosine transforms: A theoretical anal- ysis. In Proc. of IEEE Int. Conf. Acoustics, Speech and Signal Proc., ICASSP 2007, volume I, pages 1105{1108, Honolulu, HI, USA, Apr. 2007. [32] L. Gan, T. T. Do, and T. D. Tran. Fast compressive imaging using scrambled block hadamard ensemble. In EUSIPCO, sep 2008. [33] M. Gastpar, P. Dragotti, and M. Vetterli. The distributed Karhunen-Loeve transform. In MMSP, December 2002. [34] D. Hammond, P. Vandergheynst, and R. Gribonval. Wavelets on graphs via spectral graph theory. In Elsevier: Appplied and Computational Harmonic Analysis, volume 30, pages 129{150, Apr. 2010. [35] W. R. Heinzelman, A. Chandrakasan, and H. Balakrishnan. Energy-ecient routing protocols for wireless microsensor networks. In Proc. of Hawaii Intl. Conf. on Sys. Sciences,, Jan. 2000. [36] M. Herman and T. Strohmer. High-resolution radar via compressed sensing. IEEE Transactions on Signal Processing, 57(6):2275{2284, June 2009. [37] E. A. Homier and R. A. Scholtz. Hybrid xed-dwell-time search techniques for rapid acquisition of ultra-wideband signals. International Workshop on Ultra Wideband System, 2003. 112 [38] W.-S. Kim. 3-D video coding system with enhanced rendered view quality. In Ph.D. dissertation, University of Southern California, 2011. [39] W.-S. Kim, A. Ortega, P. Lai, D. Tian, and C. Gomila. Depth map distortion analysis for view rendering and depth coding. In Proc. of IEEE Int. Conf. Image Proc., ICIP 2009, Cairo, Egypt, Nov. 2009. [40] W.-S. Kim, A. Ortega, P. Lai, D. Tian, and C. Gomila. Depth map coding with distortion estimation of rendered view. In Proc. of IS&T/SPIE Electronic Imaging, VIPC 2010, San Jose, CA, USA, Jan. 2010. [41] J. D. Kraus. Antennas. In McGraw-Hill, pages 774{790, 1988. [42] H. Krishnaswamy and H. Hashemi. A 4-channel 4-beam 24-to-26GHz spatio- temporal RAKE radar transceiver in 90nm CMOS for vehicular radar applica- tions. IEEE International Solid-State Circuits Conference Digest of Technical Papers, pages 214{215, 2010. [43] J.B. Kruskal. On the shortest spanning subtree of a graph and the travel- ing salesman problem. In Proceedings of the American Mathematical Society, volume 7, pages 48{50, February 1956. [44] S. Lee, C. Du, H. Hashemi, and A. Ortega. Hardware-driven compressive sampling for fast target localization using single-chip UWB radar sensor. In- ternational Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2013. [45] S. Lee and A. Ortega. Joint optimization of transport cost and reconstruc- tion for spatially- localized compressed sensing in multi-hop sensor networks. APSIPA, Dec. 2010. [46] S. Lee and A. Ortega. Adaptive compressed sensing for depthmap compression using graph-based transform. In Proceedings of International Conference on Image Processing (ICIP), 2012. [47] S. Lee, S. Pattem, M. Sathiamoorthy, B. Krishnamachari, and A. Ortega. Compressed sensing and routing in sensor networks. In USC CENG Technical Report, April 2009. [48] S. Lee, S. Pattem, M. Sathiamoorthy, B. Krishnamachari, and A. Ortega. Spatially-localized compressed sensing and routing in multi-hop sensor net- works. In GSN, June 2009. [49] C. Li, J. Ling, J. Li, and J. Lin. Accurate doppler radar noncontact vital sign detection using the relax algorithm. IEEE Transactions on Instrumentation and Measurement, 59(3):687{695, 2010. 113 [50] W. Lu, K. Kpalma, and J. Ronsin. Sparse binary matrices of LDPC codes for compressed sensing. Data Compression Conference (DCC), 2012. [51] G. Lugosi. Concentration-of-measure inequalities. In Lecture notes, February 2006. [52] C. Luo, F. Wu, J. Sun, and C. W. Chen. Compressive data gathering for large- scale wireless sensor networks. in Proc. of ACM Mobicom09, page 145156, Sep. 2009. [53] M. Lustig, D. Donoho, and J. M. Pauly. Sparse MRI: The application of compressed sensing for rapid mr imaging. In MRM, December 2007. [54] M. Maitre and M. N. Do. Depth and depth-color coding using shape-adaptive wavelets. Visual Communication and Image Representation, Special Issue on Multicamera Imaging:513{522, Jul. 2010. [55] T. Michaeli and Y.C. Eldar. Xampling at the rate of innovation. IEEE Trans- actions on Signal Processing, 60(3), Mar. 2012. [56] Y. Morvan, D. Farin, and P. H. N. de With. Platelet-based coding of depth maps for the transmission of multiview images. In Proc. of SPIE: Stereoscopic Displays and Applications (2006), volume 6055, San Jose, CA, USA, Jan. 2006. [57] F.M. Naini, R. Gribonval, L. Jacques, and P. Vandergheynst. Compressive sampling of pulse trains: Spread the spectrum! International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2009. [58] S. K. Narang, A. Gadde, and A. Ortega. Signal processing techniques for interpolation in graph structured data. International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2013. [59] S. K. Narang and A. Ortega. Local two-channel critically sampled lter-banks on graphs. In Proc. of IEEE Int. Conf. Image Proc., ICIP 2010, Hong Kong, Sep. 2010. [60] N. Nguyen, D. L. Jones, and S. Krishnamurthy. Netcompress: Coupling net- work coding and compressed sensing for ecient data communication in wsns. in IEEE Workshop on SiPS, page 356361, Oct. 2010. [61] JL Paredes, GR Arce, and Z. Wang. Ultra-wideband compressed sensing: Channel estimation. IEEE Journal of Selected Topics in Signal Processing, 1(3):383{395, 2007. [62] S. Pattem, B. Krishnamachari, and R. Govindan. The impact of spatial corre- lation on routing with compression in wireless sensor networks. In IPSN, April 2004. 114 [63] S. Pattem, G. Shen, Y. Chen, B. Krishnamachari, and A. Ortega. Senzip: An architecture for distributed en-route compression in wireless sensor networks. In ESSA, April 2009. [64] K. Pearson. Mathematical contributions to the theory of evolution. iii. regres- sion, heredity and panmixia. In Philos. Trans. Royal Soc. London Ser., volume 187, pages 253{318, 1896. [65] E. Le Pennec and S. Mallat. Sparse geometric image representations with bandelets. IEEE Trans. Image Proc., 14(4):423{438, Apr. 2005. [66] W. Philips. Comparison of techniques for intra-frame coding of arbitrarily shaped video object boundary blocks. IEEE Trans. Circuits Syst. Video Tech- nol., 9(7):1009{1012, Oct. 1999. [67] D. D.-Y. Po and M. N. Do. The contourlet transform: An ecient directional multiresolution image representation. IEEE Trans. Image Proc., 15(6):1610{ 1620, Jun. 2006. [68] G. Quer, R. Masierto, D. Munaretto, M. Rossi, J. Widmer, and M. Zorzi. On the interplay between routing and signal representation for compressive sensing in wireless sensor network. In ITA, February 2009. [69] M. Rabbat, J. Haupt, A. Singh, and R. Nowak. Decentralized compression and predistribution via randomized gossiping. In IPSN, April 2006. [70] A. Rakotomamonjy. Surveying and comparing simultaneous sparse approxi- mation (or group-lasso) algorithms. Signal Processing, 91:1505{1526, 2011. [71] R.Saab, R.Chartrand, and O. Yilmaz. Stable sparse approximations via non- convex optimization. International Conference on Acoustics, Speech, and Sig- nal Processing (ICASSP), 2008. [72] M. Rudelson and R. Vershynin. Sparse reconstruction by convex relaxation: Fourier and gaussian measurements. Annual Conference on Information Sci- ences and Systems, pages 207 { 212, Mar. 2006. [73] M. Sarkis and K. Diepold. Depth map compression via compressed sensing. In Proc. of IEEE Int. Conf. Image Proc., ICIP 2009, Cairo, Egypt, Nov. 2009. [74] G. Shen, W.-S. Kim, S. K. Narang, A. Ortega, J. Lee, and H. Wey. Edge- adaptive transforms for ecient depth-map coding. In Proc. of 28th Picture Coding Symposium, PCS 2010, Nagoya, Japan, Dec. 2010. [75] G. Shen and A. Ortega. Joint routing and 2D transform optimization for irregular sensor network grids using wavelet lifting. In IPSN, April 2008. 115 [76] G. Shen and A. Ortega. Transform-based distributed data gathering. IEEE Transactions on Signal Processing, 58(7), July 2010. [77] G. Shi, J. Lin, X. Chen, F. Qi, D. Liu, and L. Zhang. UWB echo signal detection with ultra-low rate sampling based on compressed sensing. IEEE Transactions on Circuits and Systems II: Express Briefs, 55(4):379{383, 2008. [78] D. I Shuman, S. K. Narang, P. Frossard, A. Ortega, and P. Vandergheynst. Signal processing on graphs: Extending high-dimensional data analysis to net- works and other irregular data domains. SPM, 2013. [79] V. Velisavljevic, B. Beferull-Lozano, M. Vetterli, and P.L. Dragotti. Direc- tionlets: Anisotropic multidirectional representation with separable ltering. IEEE Trans. Image Proc., 15(7):1916{1933, Jul. 2006. [80] P. von Rickenbach and R. Wattenhofer. Gathering correlated data in sensor networks. In DIALM-POMC. ACM, October 2004. [81] R. Wagner, H. Choi, R. Baraniuk, and V. Delouille. Distributed wavelet trans- form for irregular sensor network grids. In SSP, July 2005. [82] A. Wang and A. Chandraksan. Energy-ecient DSPs for wireless sensor net- works. IEEE Signal Processing Magazine, 19(4):68{78, Jul. 2002. [83] W. Wang, M. Garofalakis, and K. Ramchandran. Distributed sparse random projections for renable approximation. In IPSN, April 2007. [84] D. Yang, H. Li, G. D. Peterson, and A. Fathy. UWB signal acquisition in po- sitioning systems: Bayesian compressed sensing with redundancy. Conference on Information Sciences and Systems (CISS), 2009. [85] D. Yang, H. Li, and G.D. Peterson. Compressed sensing TDOA for UWB positioning systems. IEEE Radio and Wireless Symposium (RWS), 2011. [86] J. Yoo, S. Becker, M. Loh, M. Monge, E. Cand es, and A. Emami-Neyestanak. A 100mhz-2ghz 12.5x sub-nyquist rate receiver in 90nm cmos. IEEE Radio Frequency Integrated Circuits Symposium (RFIC), 2012. [87] J. Yoo, S. Becker, M. Loh, M. Monge, E. Cand es, and A. Emami-Neyestanak. Design and implementation of a fully integrated compressed-sensing signal acquisition system. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2012. [88] J. Yoo, C. Turnes, E. Nakamura, C. Le, S. Becker, E. Sovero, M. Wakin, M. Grant, J. Romberg, A. Emami-Neyestanak, and E. Cand es. A compressed sensing parameter extraction platform for radar pulse signal acquisition. IEEE 116 Journal on Emerging and Selected Topics in Circuits and Systems (JETCAS), PP(99):1{13, 2012. [89] B. Zeng and J. Fu. Directional discrete cosine transforms for image coding. In Proc. of IEEE Int. Conf. Multimedia and Expo, ICME 2006, pages 721{724, Toronto, Canada, Jul. 2006. [90] B. Zeng and J. Fu. Directional discrete cosine transforms - a new framework for image coding. IEEE Trans. Circuits Syst. Video Technol., 18(3):305{313, Mar. 2008. [91] H. Zou. The adaptive lasso and its oracle properties. Journal of the American Statistical Association, 101(476):1418{1429, 2006. 117
Abstract (if available)
Abstract
Traditional compressed sensing (CS) approaches have been focused on the goal of reducing the number of measurements while achieving satisfactory reconstruction. Reducing the number of measurements can directly lead to reductions in costs in some applications, e.g., the scanning time in fast magnetic resonance imaging (MRI) or the sampling rate in analog-to-information conversion. However, in some other applications, minimizing the number of measurements by itself does not provide a better solution in terms of system complexity due to additional application-driven constraints. ❧ In general, those constraints affect the choice of either measurement basis or sparsifying basis. For example, if the total cost of collecting measurements is a crucial factor as compared to the reconstruction accuracy, reducing the number of measurements does not guarantee better performance because the increase of measurement cost can exceed the gain achieved by the increase of the number of measurements. Thus, the design of an efficient measurement basis should consider the total cost for measurements as well as the reconstruction accuracy. Also, in coding applications where signals are first captured and then compressed, better performance can be achieved by adaptively selecting a transform or sparsifying basis and then signaling the chosen transform to the decoder. For instance, for piecewise smooth signals, where sharp edges exist between smooth regions, edge-adaptive transforms can provide sparser representation at the cost of some overhead. Thus, the design of sparsifying basis should be optimized with respect to a given measurement basis, while the signaling overhead is minimized. These observations motivated us to investigate efficient design schemes for CS that can provide better reconstruction while minimizing the application-driven costs. ❧ In this thesis, we study the optimization of compressed sensing in three different applications, each of which imposes a different set of constraints: i) efficient data-gathering in wireless sensor networks (WSN), ii) depth map compression using a graph-based transform, and iii) fast target localization using a single-chip ultra-wideband (UWB) radar. Under these application-driven constraints, we study how to minimize application specific costs while minimizing the mutual coherence in order to achieve satisfactory reconstruction using CS. ❧ In sensor networks, we take explicitly into consideration the cost of each measurement (rather than minimizing the number of measurements), and optimize the measurement matrix that leads to energy efficient data-gathering in WSN. For depth map compression, the constraint to consider is the total bitrate, including both the number of bits for measurements and the bit overhead to code the edge map required for the construction of graph-based transform (GBT). To improve overall performance, we propose a greedy algorithm that selects for each block the GBT that minimizes a metric accounting for both the edge structure of the block and the characteristics of the measurement matrix. For fast localization of objects using a UWB ranging system, we design an efficient measurement system that is constructed using a low-density parity-check (LDPC) matrix, designed to satisfy hardware-driven constraints. To enhance performance, we propose a window-based reweighted L₁ minimization that outperforms other existing algorithms in high noise environments.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Lifting transforms on graphs: theory and applications
PDF
Distributed wavelet compression algorithms for wireless sensor networks
PDF
Critically sampled wavelet filterbanks on graphs
PDF
Efficient transforms for graph signals with applications to video coding
PDF
Compilation of data-driven macroprograms for a class of networked sensing applications
PDF
Joint routing and compression in sensor networks: from theory to practice
PDF
Sensing with sound: acoustic tomography and underwater sensor networks
PDF
Compression of signal on graphs with the application to image and video coding
PDF
Efficient coding techniques for high definition video
PDF
On location support and one-hop data collection in wireless sensor networks
PDF
Relative positioning, network formation, and routing in robotic wireless networks
PDF
Design of cost-efficient multi-sensor collaboration in wireless sensor networks
PDF
Gradient-based active query routing in wireless sensor networks
PDF
Modeling and predicting with spatial‐temporal social networks
PDF
Sampling theory for graph signals with applications to semi-supervised learning
PDF
Human motion data analysis and compression using graph based techniques
PDF
Efficient data collection in wireless sensor networks: modeling and algorithms
PDF
Dynamic routing and rate control in stochastic network optimization: from theory to practice
PDF
Data-driven optimization for indoor localization
PDF
Magnetic induction-based wireless body area network and its application toward human motion tracking
Asset Metadata
Creator
Lee, Sungwon
(author)
Core Title
Application-driven compressed sensing
School
Viterbi School of Engineering
Degree
Doctor of Philosophy
Degree Program
Electrical Engineering
Publication Date
08/04/2013
Defense Date
02/07/2013
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
compressed sensing,compressive sampling,data-gathering,depthmap compression,OAI-PMH Harvest,wireless sensor network
Format
application/pdf
(imt)
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Ortega, Antonio K. (
committee chair
), Krishnamachari, Bhaskar (
committee member
), Neumann, Ulrich (
committee member
)
Creator Email
men7ali57@gmail.com,sungwonl@gmail.com
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c3-313894
Unique identifier
UC11293892
Identifier
etd-LeeSungwon-1942.pdf (filename),usctheses-c3-313894 (legacy record id)
Legacy Identifier
etd-LeeSungwon-1942-0.pdf
Dmrecord
313894
Document Type
Dissertation
Format
application/pdf (imt)
Rights
Lee, Sungwon
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Tags
compressed sensing
compressive sampling
data-gathering
depthmap compression
wireless sensor network