University of Southern California Dissertations and Theses

Towards energy efficient mobile sensing

(USC Thesis Other)

Towards energy efficient mobile sensing

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content TOWARDS ENERGY EFFICIENT MOBILE SENSING by Yi Wang A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (ELECTRICAL ENGINEERING) May 2011 Copyright 2011 Yi Wang Dedication To my beloved parents, Wang Yongning and Cai Qingyun. ii Acknowledgements This thesis is certainly not a singular effort. It would not have been completed without the build- ing blocks from those who guided, and collaborated with me in achieving this milestone. Foremost, I would like to express my sincere appreciation to my advisers, Prof. Bhaskar Krishnamachari and Prof. Murali Annavaram. Bhaskar has been providing continuous motivation ever since the first day I was a Ph.D. student. Under Bhaskar’s guidance, I not only developed a comprehensive set of advanced research skills, but also learned what a Ph.D. truly means. I want to thank Bhaskar, for realizing my potentials, for training me in every necessary aspect to help me grow up to be an outstanding researcher, and for sharing his experiences that are outside the campus life. Along with Bhaskar, I also owe my deepest gratitude to my co-adviser, Prof. Murali Annavaram. Murali’s support and encouragement have been accompanying me throughout the last three years of my Ph.D. program. Murali helped me develop rigorous research habits that are extremely important for an outstanding researcher. Besides, Murali provided me a unique opportunity to practice in an industrial research lab, which significantly broadened my views and made me involved in a different research community for the first time. It is my great honor to be a student of Bhaskar and Murali. Their immense knowledge, hard working and enthusiasm in research have been, and will always be inspiring me throughout my lifetime. iii I would like to give my special thanks to my collaborator, Jialiu Lin, a Ph.D. student from Carnegie Mellon University. Without her contribution, the majority of Chapter 3 of this thesis would not have been accomplished. In addition, I’m really grateful to Prof. Qing Zhao for her insightful comments and valuable contributions to my research subjects. Without her guidance, the majority of Chapter 4 of this thesis could not have been accomplished. My sincere thanks also goes to Dr. Quinn Jacobson and Dr. Fan Bai, who were my intern managers at Nokia Research Center and General Motor R&D labs respectively. They not only broadened my perspective on how research is conducted with practical aspects in the industry, but also fought with me together on writing two extremely challenging research papers. I have to thank all my dissertation committee members, Prof. Bhaskar Krishnamachari, Prof. Murali Annavaram, Prof. Urbashi Mitra, Prof. Rahul Jain, and Prof. Gaurav Sukhatme, for their intriguing questions and valuable feedback in order for me to complete this thesis. Besides the individuals mentioned above, I would like to thank all my collaborators, including Akram Ahmed, Dr. Joon Ahn, Dr. Amitabha Ghosh, Prof. Jason Hong, Dr. Shyam Kapadia, Prof. Konstantinos Psounis, Prof. Norman Sadeh, and Prof. Thomas Valente. I could never have completed my Ph.D. unless working together with such outstanding researchers. The members from Autonomous Networks Research Group deserve my heartily thanks as well. It was this group of amazing people that mentored me, accompanied me, and fought with me together day and night. I will never forget the time we spent together. Lastly, I want to thank my beloved fiancee, Rui, who always gives me support uncondition- ally. iv Table of Contents Dedication ii Acknowledgements iii List of Tables viii List of Figures ix Abstract xii Chapter 1: Introduction 1 Chapter 2: Background 6 2.1 User context sensing and applications in mobile devices . . . . . . . . . . . . . . 6 2.2 Energy efficiency research in mobile sensing . . . . . . . . . . . . . . . . . . . . 9 2.3 Information processing in detection oriented systems . . . . . . . . . . . . . . . 12 2.3.1 The Markovian case . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.3.1.1 Hidden Markov models . . . . . . . . . . . . . . . . . . . . . 12 2.3.1.2 The forward-backward algorithm . . . . . . . . . . . . . . . . 14 2.3.1.3 The Viterbi algorithm . . . . . . . . . . . . . . . . . . . . . . 16 2.3.2 The semi-Markovian case . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.3.2.1 Semi-Markov processes . . . . . . . . . . . . . . . . . . . . . 18 2.3.2.2 State estimation in hidden semi-Markov models . . . . . . . . 19 2.3.3 Differences from traditional models . . . . . . . . . . . . . . . . . . . . 21 2.4 Constrained stochastic optimization . . . . . . . . . . . . . . . . . . . . . . . . 22 2.4.1 CMDP definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.4.2 Classes of policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.4.3 Cost criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.4.4 Solving the CMDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Chapter 3: The framework design for an Energy Efficient Mobile Sensing System 28 3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.2 Design methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 3.2.1 Sensor management module . . . . . . . . . . . . . . . . . . . . . . . . 30 3.2.1.1 State and sensor relationship: the XML-format user state de- scriptor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 v 3.2.1.2 Sensor duty cycles . . . . . . . . . . . . . . . . . . . . . . . . 32 3.2.1.3 Generalization of the EEMSS framework . . . . . . . . . . . . 34 3.2.2 Classification module . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3.2.2.1 GPS sensing and mode of travel classification . . . . . . . . . 36 3.2.2.2 WiFi scanning and usage . . . . . . . . . . . . . . . . . . . . 36 3.2.2.3 Real-time motion classification based on accelerometer sensing 37 3.2.2.4 Real-time background sound recognition . . . . . . . . . . . . 39 3.3 Performance evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.3.1 Verifying EEMSS capabilities . . . . . . . . . . . . . . . . . . . . . . . 42 3.3.2 State recognition accuracy . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.3.3 Device lifetime . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 Chapter 4: User state estimation and sensing policy optimization: the Markovian case 52 4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 4.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 4.2.1 Model and assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 53 4.2.2 User state estimation with missing observations . . . . . . . . . . . . . . 54 4.2.3 Expected energy consumption and expected user state estimation error . . 55 4.2.4 The optimization problem . . . . . . . . . . . . . . . . . . . . . . . . . 56 4.3 The Markov-optimal Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.3.1 The CMDP formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.3.2 Finding the Markov-optimal policyu ∗ m . . . . . . . . . . . . . . . . . . 59 4.3.3 A case study: obtainingu ∗ m for sample Markov chains . . . . . . . . . . 63 4.4 Performance comparison on Markov models: Markov-optimal u ∗ m vs. uniform sampling ¯ u . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 4.4.1 Deriving the expected estimation error for uniform sampling . . . . . . . 68 4.4.2 Comparingu ∗ m with ¯ u . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 4.5 Policy comparison on real user state traces . . . . . . . . . . . . . . . . . . . . . 72 4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 Chapter 5: User state estimation and sensing policy optimization: the semi-Markovian case 81 5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 5.2 Semi-Markovian state estimation for missing observations with sojourn time trun- cation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 5.2.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 5.2.2 The forward-backward variables: definition and initialization . . . . . . . 83 5.2.3 State estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 5.2.4 The forward-backward variables: an alternative . . . . . . . . . . . . . . 90 5.2.5 State estimation performance on simulated two-state transition processes 92 5.2.6 State estimation performance on real traces . . . . . . . . . . . . . . . . 98 5.2.6.1 Heavy-tailed state durations: device network connectivity . . . 98 5.2.6.2 Arbitrary state durations: temperature . . . . . . . . . . . . . . 101 5.3 Optimal sensor scheduling with energy constraint . . . . . . . . . . . . . . . . . 102 vi 5.3.1 Obtaining the semi-Markov optimal policyu ∗ s . . . . . . . . . . . . . . . 103 5.3.2 Performance evaluation ofu ∗ s on simulated processes . . . . . . . . . . . 105 5.3.3 Performance ofu ∗ s on heavy-tailed real user state traces . . . . . . . . . . 106 5.4 A novel client-server based system framework for energy efficient mobile sensing 109 5.4.1 System design and implementation . . . . . . . . . . . . . . . . . . . . 109 5.4.2 System performance evaluation . . . . . . . . . . . . . . . . . . . . . . 112 5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Chapter 6: Conclusions and future work 115 References 118 vii List of Tables 3.1 Power and current consumption summary for different sensors on Nokia N95. . . 33 3.2 Sensor duty cycles used in current EEMSS implementation . . . . . . . . . . . . 33 3.3 The computation time and sensor energy cost per sample corresponding to table 3.2. 34 3.4 Standard deviation range of accelerometer magnitude readings for different user activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.5 Classification results based on standard deviation of accelerometer magnitude values. The first column represents the ground truth while the first row represents the classification results based on accelerometer readings. . . . . . . . . . . . . . 39 3.6 Percentage of sound clips classified as “speech” for differentSR thres values. . . 40 3.7 EEMSS confusion matrix of recognizing “Walking”, “Vehicle” and “At some place”. The first column represents the ground truth while the first row represents the recognition results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 4.1 For each data trace, the theoretical gain (obtained based on the Markovian as- sumption) and the experimental gain (obtained by policy application to real traces) are displayed. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 5.1 Four different distribution types and their specific parameters utilized in the sim- ulation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 5.2 Average and maximum percentage gain of semi-Markovian estimation with M1, M2, and M3 over Markovian estimation under different estimation window sizes ranging from 1 to 500. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 5.3 Smartphone battery lifetime and state estimation error comparison among all ex- periments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 viii List of Figures 2.1 The system states (N = 3), outputs (M = 4), transition probabilities, and emis- sion probabilities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.2 The process is sampled at discontiguous time slots, leading to multiple estimation intervals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.1 System architecture of EEMSS implementation on Nokia N95. . . . . . . . . . 31 3.2 The format of XML based state descriptor. . . . . . . . . . . . . . . . . . . . . . 47 3.3 The state transition implication of figure 3.2. . . . . . . . . . . . . . . . . . . . . 47 3.4 Decision tree based background sound classification algorithm. . . . . . . . . . 48 3.5 Shows the frequency domain features after applying FFT to different types of audio signals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.6 Shows the frequency histogram plots after applying SSCH to different types of audio signals shown in figure 3.5. . . . . . . . . . . . . . . . . . . . . . . . . . . 49 3.7 Figure shows the time, location as well as background sound condition detected by EEMSS, for two sample users. (Top: a CMU user. Bottom: a USC user.) . . . 50 3.8 Average N95 lifetime comparison of running EEMSS, CenceMe and the case where all sensors are powered. . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.9 Power usage at a glance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 4.1 Illustration of the Markovian user state sequence, sampled at certain time slots. . 54 4.2 An illustration of the average idle intervals in u ∗ m for different energy budgets. Sub-graphs are positioned according to the six transition matrices given before. I 1 ,I 2 , andI 3 are the average optimal idle interval lengths when state 1, 2, and 3 is observed, respectively. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 ix 4.3 Policy comparison in terms ofE[R] for different energy budgets. . . . . . . . . . 71 4.4 Given different state transition probabilities, this figure shows the average im- provement over different energy budgets, by applying optimal policy as compared to periodic sampling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 4.5 The gain of the optimal policy over uniform sampling (averaged over different en- ergy budgets) vs. the standard deviation of state entropies, in case of{2,3,4,5}- state Markov chains.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 4.6 Markov chains showing two-state transition probabilities for: (1) Stable vs. Mov- ing (2) Not in contact with other users vs. In contact with other users (3) Not in cellular coverage vs. In cellular coverage (4) Not in WiFi coverage vs. In WiFi coverage and (5) No Bluetooth connectivity vs. Bluetooth connectivity. . . . . . 76 4.7 Are the traces strictly Markovian? Figure compares the cdf of state duration in real data (solid curve) to the corresponding geometric distribution (dashed curve) generated with success probability indicated in figure 4.6. . . . . . . . . . . . . 77 4.8 Top graph shows the user motion trace (a solid vertical line means “Moving” and a blank space means “Stable”) and the sampling points of both policies. Bottom graph shows the aggregated state estimation error at each sampling point for both policies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 5.1 The semi-Markov process is sampled at discontiguous time slots, leading to mul- tiple estimation intervals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 5.2 Semi-Markovian estimation vs. Markovian estimation on simulated two-state process where state distributions are deterministic (left) and geometric (right). . 95 5.3 Semi-Markovian estimation vs. Markovian estimation on simulated two-state process where state distributions are binomial. Right figure shows convergence at very large window sizes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 5.4 Semi-Markovian estimation vs. Markovian estimation on simulated two-state process where state sojourn durations follow Zipf’s Law distributions, with dif- ferent tail sizes according to the definitions in table 5.1. . . . . . . . . . . . . . 97 5.5 State duration pmfs for Bluetooth connectivity state trace and the comparison of both estimation algorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 5.6 State duration pmfs for WiFi state trace and the comparison of both estimation algorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 x 5.7 State duration pmfs for temperature connectivity state trace and the comparison of both estimation algorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 5.8 Performance ofu ∗ m vs. u ∗ s , on binomially and Zipf’s Law distributed state pro- cesses. Figure shows the estimation error w.r.t. energy budget (in log scale). . . . 105 5.9 Performance ofu ∗ m vs.u ∗ s , and the genie-aided estimation error on CoSphere net- work connectivity traces. Figure shows the estimation error w.r.t. energy budget (in log scale). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 5.10 System modules and operation flow. . . . . . . . . . . . . . . . . . . . . . . . . 110 xi Abstract Mobile device based urban sensing, participatory sensing, and user activity recognition can pro- vide rich contextual information for mobile applications such as social networking and location- based services. However, as the sensors on mobile devices consume significant amount of energy, the major bottleneck that restricts the continuous functioning of these mobile applications is the limited battery capacity on mobile devices. In this thesis, we first present a novel design framework for an Energy Efficient Mobile Sens- ing System (EEMSS). EEMSS uses hierarchical sensor management strategy to recognize user states as well as to detect state transitions. By powering only a minimum set of sensors and man- aging sensors hierarchically, EEMSS significantly improves device battery life. We present the design, implementation, and evaluation of EEMSS that automatically recognizes a set of users’ daily activities in real time using sensors on an off-the-shelf high-end smart phone. Evaluation of EEMSS with 10 users over one week shows that our approach increases the device battery life by more than 75% while maintaining both high accuracy and low latency in identifying transitions between end-user activities. We then propose a computationally efficient algorithm to obtain the optimal sensor sampling policy under the assumption that the user state transition is Markovian. This Markov-optimal policy minimizes user state estimation error while satisfying a given energy consumption budget. xii The Markov-optimal policy is compared to uniform periodic sensing and performance improve- ment is obtained on both simulated and real user state traces, with approximately 20% average gain on empirically collected user data that pertains to user motion, inter-user contact status, and network connectivity. Finally, we formulate the user state sensing problem as the intermittent sampling of a semi- Markov process, a model that provides more general and flexible capturing of realistic data. We propose (a) a semi-Markovian state estimation mechanism that selects the most likely user state while observations are missing, and (b) a semi-Markov optimal sensing policy u ∗ s which minimizes the expected state estimation error while maintaining a given energy budget. Their performance is shown to significantly outperform Markovian algorithms on simulated two-state processes and real user state traces. In addition, we propose a novel client-server based energy ef- ficient mobile sensing system architecture that automatically learns user dynamics and computes user-specific optimal sensing policy for mobile devices. A system prototype aiming to recognize basic human activity is implemented on Nokia N95 smartphones and desktop computers. We demonstrate the performance benchmark of u ∗ s through a set of experiments, and show that it improves the estimation accuracy by27.8% and48.6% over Markov-optimal policy and uniform sampling, respectively. xiii Chapter 1 Introduction As the number of transistors in unit area doubles every 18 months following Moore’s law, mobile devices such as smartphones are continually integrating more sophisticated functionalities. For example, current generation mobile devices integrate a wide range of sensing and networking features such as GPS, microphone, camera, light sensor, orientation sensor, accelerometer, WiFi, Bluetooth, 3G chips, and so on. These sensors and transceivers collectively enable us to obtain and share user’s ambient context, both unobtrusively and in real time. At the same time, mobile devices have become more powerful and user-friendly clients that support mobile applications and web browsing, through which users are able to conveniently re- ceive desired information and services. In addition, user contextual information can be transferred via the Internet to back-end servers with stronger computation and storage capabilities that are able to deliver desired personalized services back to user, such as location based advertising [7], traffic information reporting [40], social networks [27], and health monitoring [5]. Throughout this thesis, we use the term user state as an important way to represent user context that evolves over time. User state may contain one or a combination of user features including activity, location, and proximity information such as background sound and number 1 of peer devices. As driven by the improving demand of user satisfaction, a growing number of mobile applications and services rely on accurate and automatic detection of user state. For example, it would be much more convenient if our phones can automatically and correctly adjust the ring tone profile to appropriate volume and mode according to the surroundings and the events in which the users are participating. While incorporating user’s contextual information brings mobile application personalization to new levels of sophistication, a major impediment to collecting and determining user context is the limited battery capacity on mobile phones. Since the integrated sensors can consume signifi- cant amount of energy, continuously collecting context information through sensor activation will make even the most compelling mobile application less than desirable to users. For instance, our experiments show that a fully charged battery on Nokia N95 device would be completely drained within six hours if its GPS is operated continuously [90]. As excessive energy consumption may become a major obstacle to the broader acceptance of context-aware mobile applications or ser- vices, energy efficient mobile sensing is critical in order to maintain the lifetime of mobile device battery and to continue to take advantage of mobile applications and services. The first major problem associated with traditional mobile sensing systems is that all the sen- sors on mobile devices are activated continuously to collect user data regardless of user status. For example, the GPS would be turned on continuously no matter where the user is located, yet the ideal case is to shut it down automatically when the user is indoor since GPS cannot communicate with satellites at indoor positions in general. In order to address this problem and provide a more efficient system architecture, we first present in this thesis the design, implementation, and evalu- ation of EEMSS [90], an Energy Efficient Mobile Sensing System that incorporates a hierarchical sensor management scheme to recognize user states pertaining to a comprehensive set of daily 2 activities. EEMSS significantly extends the mobile device battery lifetime by powering a mini- mum set of sensors at any given time and triggering sensors hierarchically. The core component of EEMSS is a sensor management scheme which defines user states and state transition rules by an XML styled state descriptor, which is taken as an input to the system functional block to turn sensors on and off based on the user’s current condition. In addition, user motion and background sound classification algorithms that guarantee real-time system performance have been designed and implemented on EEMSS. A user study has been conducted on two university campuses and the results show that EEMSS is able to detect user states with 92.56% accuracy on average and improves the mobile device battery lifetime by over 75%, compared to a start-of-the-art system (CenceMe [60, 61]). The second major problem in traditional mobile sensing systems (including the EEMSS de- sign) is that sensors adopt fixed and uniform duty cycles whenever activated, which is not adaptive to user behaviors. In general, some user states once entered may stay unchanged for a long while, and if the sensor is sampled too frequently within that state duration, an excessive amount of energy will be consumed. On the other hand, if the user is frequently transiting among different states, it is preferred to sample the sensor more frequently to capture his context, as otherwise it becomes more difficult to estimate the information between two sensor observations. In order to obtain an intelligent sensing policy which is able to assign different frequencies at different user states, two fundamental problems need to be investigated: first, state estimation al- gorithms needs to be designed to provide the “best guess” of user state when the sensor stays idle; second, it is necessary to obtain the optimal sensor sampling policy that leads to the minimum state estimation error while satisfying the energy budget constraint. 3 We first investigate these two problems under the assumption that user state transitions follow a Markov process. We propose state estimation algorithm for missing observations, and formu- late the optimal sensor duty cycle selection problem as an infinite-horizon Constrained Markov Decision Process (CMDP). The expected average cost of the CMDP is considered as the cost criteria to be minimized, with the sensor energy consumption serving as the constraint. Solving such CMDP yields a globally optimal sensor sampling policyu ∗ m which guarantees the minimum expected user state estimation error while satisfying the given energy budget for Markovian user state transitions (we thus name this policy “Markov-optimal”). The Markov-optimal policy is sta- tionary and randomized (shown in [3, 29, 30]) and can be obtained by solving the corresponding Linear Programming (LP) of the CMDP formulation, which requires only polynomial run-time for any number of user state inputs. We show that the Markov-optimal policy always leads to a non-negative gain over uniform sampling and the gain is positively correlated to the standard de- viation of state entropies. We then apply both the Markov-optimal policy and uniform sampling to real user context data traces and show that theu ∗ m leads to approximately 20% improvement over uniform periodic sampling. To generalize the analytical framework, we relax the assumption such that instead of assum- ing geometrically distributed state durations, we consider discrete time semi-Markov processes which allow arbitrary state sojourn time distributions and provide more general and realistic mod- els than Markov processes 1 . We propose a forward-backward state estimation mechanism for semi-Markov processes, and provide a linear programming based approximate sensing policy optimization framework, which is able to efficiently schedule sensor samplings in order to mini- mize state estimation error given an energy constraint. The semi-Markov estimation mechanism 1 Note that a discrete time Markov process is a special case of discrete time semi-Markov process where the sojourn times are geometrically distributed. 4 is evaluated on both simulated state sequences and real user state traces and we demonstrate that it outperforms Markov estimation if the past sojourn time is selected using the correct method. Both the semi-Markov estimation mechanism and the optimization framework can be applied to data traces with any types of state sojourn time distribution, whose probability mass function (pmf) parameters are not necessarily known a priori and can be obtained through the learning of historic data. The approximate semi-Markov optimal policyu ∗ s is examined on simulated as well as real state traces and we show that it obtains performance improvement over the Markov- optimal policyu ∗ m . It is also implemented in accelerometer based activity monitoring system and we demonstrate that the mobile device battery lifetime can be extended to one week from two days by implementingu ∗ s , while the user state estimation error is maintained below 3%. The organization of this thesis is as follows: In Chapter 2, we present relevant works in mobile sensing and the corresponding energy efficiency research, and introduce the background of Hidden Markov Models (HMM), Hidden Semi-Markov Models (HSMM), and Constrained Markov Decision Process (CMDP), which are fundamental tools utilized in this thesis. In Chap- ter 3, we present the design, implementation, and user study result of EEMSS. In Chapter 4 and 5, we investigate state estimation and sensing policy optimization in case of Markovian and semi- Markovian user state processes respectively, and show their evaluation results on simulated and real user state traces, as well as from real system implementations. Finally, in Chapter 6, we con- clude the contributions of this thesis, and present several open problems, which indicate future research directions. 5 Chapter 2 Background 2.1 User context sensing and applications in mobile devices As current generation commodity mobile devices contain a wide range of sensors (such as ac- celerometer, GPS, WiFi, Bluetooth, camera, etc.), and provide significantly batter computation and communication capabilities than traditional wireless sensor devices, a new wave of research has been focusing on the development of people-centric mobile sensing and computing applica- tions. In the context of using mobile device to sense and recognize user state, it is worth noting that Schmidt et al. [74] first propose to incorporate low level sensors to mobile PDAs to demonstrate situational awareness, and Gellersen et al. [37] pointed out the idea that combining a diverse set of sensors that individually captures just a small aspect of an environment may result in a total picture that better characterizes a situation than location or vision based context. A great number of works thereafter utilize commodity cell phones as sensing, computing or application platforms such as [1, 18, 36, 49, 60, 61, 65, 72, 78, 91]. For examples, the authors of [1] and [49] discussed the vision of the sensor network of mobile phones and its potential 6 usages and design challenges. The authors of [65] designed a mobile device based health mon- itoring system that aggregates on-body health related data aiming to provide health alert and emergency response. In [72], a mobile system has been proposed that automatically estimates the demographics of a region based on mobile device population sensing and relevant user in- puts. “CenceMe” [61] is originated from “MetroSense” [26], and uses the integrated as well as external sensors to capture the user’s status such as activity, disposition, and surroundings and enables members of social networks to share their sensing presence. A CenceMe prototype has been made available on Facebook, and the implementation and evaluation of the CenceMe appli- cation has also been discussed [60]. Similarly, “Sensay” [78] is a context-aware mobile phone that uses data from a number of sources to dynamically change cell phone ring tone, alert type, as well as determine users’ “un-interruptible” states. Another example is “MicroBlog” [36], which is a participatory sensing application that allows users with smart phones to generate geo-tagged multimedia based on user location information detected by GPS and WiFi. Motion sensors have been widely used in monitoring and recognizing human activities to provide guidance to specific tasks [4, 56, 82]. For example, in car manufacturing, a context- aware wearable computing system designed by Stiefmeier et al. [82] could support a production or maintenance worker by recognizing the worker’s actions and delivering just-in-time informa- tion about activities to be performed. A common low cost sensor used for detecting motion is the accelerometer. In [33], a software architecture “Satire” was proposed that aims monitor hu- man activity by smart clothes, with the assistance from accelerometer-equipped sensor motes. The authors have shown that it is possible to distinguish human actions based on their unique accelerometer signatures. 7 Like the work in [33], with accelerometer as the main sensing source, activity recognition is usually formulated as a classification problem where the training data is collected with ex- perimenters wearing one or more accelerometer sensors in a certain period. Different kinds of classifiers can be trained and compared in terms of the accuracy of classification [8, 22, 52, 97]. For example, more than 20 human activities including walking, watching TV , running, stretching, etc. can be recognized with fairly high accuracy [8]. Besides motion sensors, researchers from different fields have studied and used a large num- ber of sensors including GPS, Bluetooth, WiFi detector, blood oxygen saturation sensor, ac- celerometer, electrocardiograph sensor, temperature sensor, light sensor, microphone, camera, etc. in projects such as urban/paticipatory sensing [18,54,63,92], activity recognition [5,17,55], and health monitoring [35, 43, 94]. For example, Whitesell et al. [92] have designed and imple- mented a system that analyzes images from air sensors captured from mobile phones and indoor air pollution information has been extracted by comparing the data to a calibrated chart. Target- ing the obesity problem in the health monitoring domain, Annavaram et al. [5] showed that by using data from multiple sensors and applying multi-modal signal processing, seemingly sim- ilar states such as sitting and lying down can be accurately discriminated, while using only a single accelerometer sensor these states can not be easily detected. Wu et al. [94] have designed “SmartCane” system which provides remote monitoring, local processing, and real-time feedback to elder patients in order to assist proper usage of canes to reduce injury and death risks. 8 2.2 Energy efficiency research in mobile sensing Earlier works that explore energy management on mobile devices include [69, 76, 80, 84]. For example, Viredaz et al. [84] surveyed many fundamental but effective methods for saving energy on handheld devices in terms of improving the design and cooperation of system hardware, soft- ware as well as multiple sensing sources. Event driven power-saving method is investigated by Shih et. al. to reduce system energy consumptions [76]. In their work, the authors focused on reducing the idle power, the power a device consumes in a “standby” mode, such that a device turns off the wireless network adaptor to avoid energy waste while not actively used. The device will be powered on only when there is an incoming or outgoing call or when the user needs to use the PDA for other purposes. To further explore the concept of event-driven energy management, a hierarchical power management method was used in [80]. In their demo system “Turdecken”, a mote is used to wake up the PDA, which in turn wakes up the computer by sending a request message. Since the power required by the mote is enough for holding the whole system standby, the power consumption can be saved during system idle time. More recently, the study of energy efficiency in mobile sensing has been conducted in works such as [51], [86], [48], and [2]. Krause et al. [51] investigate the topic of trading off prediction accuracy and power consumption in mobile computing, and the authors showed that even very low sampling rate of the accelerometer can lead to competitive classification results while device energy consumption can be significantly reduced. Wang et al. characterized the trade-off between energy usage and contact missing probability in delay tolerant networks and proposed an opti- mization framework to compute the optimal contact probing interval as a function of the arrival rate. The hierarchical sensor management concept is explored by the “SeeMon” system [48], 9 which achieves energy efficiency by only performing context recognition when changes occur during the context monitoring. Energy management in personal health care system using mobile devices is studied by Aghera et al. [2], where the authors aim to address the tradeoff between energy efficiency and data uploading latency by implementing a task assignment strategy capable of selecting different levels of operation. The authors of [60] are well aware of the battery life constraint of mobile devices and apply duty cycling mechanisms to different physical sensors in order to achieve energy efficiency. However, the lack of intelligent sensor management method still reduces the device lifetime by a significant amount. Energy efficient localization [23, 46, 50, 58, 67, 83, 98] has been extensively studied as well recently, as location is one of the most important types of user context information. For example, Constandache et al. [23] show that human can be profiled based on their mobility patterns and thus location can be predicted. Their “EnLoc” system achieves good localization accuracy with a realistic energy budget. The energy consumption for different localization techniques including GPS, WiFi, and GSM are measured on Nokia N95 device and an energy efficient localization framework named “EnLoc” has been proposed. The authors show that “EnLoc” system achieves good localization accuracy with a realistic energy budget. Park et al. [67] investigate how GPS on mobile device can be activated and duty cycled smartly using the assistance of accelerome- ter, cell tower, and bluetooth, which conduct user movement and velocity estimation, location blacklisting, and location synchronization respectively. Lin et al. [58] studied sensor selection in localization, by comparing the tradeoff between energy consumption and location detection accuracy among different sensors such as GPS, WiFi, and so on. The authors provide an en- ergy optimal sensor selection at any time instance given user’s preference in location detection 10 accuracy. Thiagarajan et al. [83] investigate location estimation for map matching using hidden Markov models, thus providing travel time estimation in intelligent transportation systems. Besides in mobile sensing, energy efficient monitoring and event detection has been widely studied in a much broader context such as communication, data collection, and so on. For ex- ample, Stabellini and Zander [81] propose a new energy efficient algorithm for wireless sensor networks to detect communication interference. Shin et al. [77] propose DEAMON, an energy efficient distributed sensor monitoring algorithm where sensors are controlled using a Boolean expression and energy is conserved by selectively turning on/off the nodes and limiting commu- nication operations. Similarly, Silberstein et al. [79] investigate suppression technique (reducing the cost for reporting changes) and take advantage of temporal and spatial data correlation to reduce data reporting therefore enlarge the lifetime of sensors. The EEMSS framework design presented in Chapter 3 is the first work that systematically investigates energy efficiency issues in mobile sensing systems, which achieves energy efficient sensor management and user state recognition automatically. In particular, we propose a hierar- chical approach for managing sensors, and do so in such a way that still maintains accuracy in sensing the user’s state. Specifically, power hungry sensors are only activated whenever triggered by power efficient ones. By only duty cycling the minimum set of sensors to detect state tran- sition and activating more expensive ones on demand to recognize new state, the device energy consumption can be significantly reduced. However, in EEMSS design, sensors still adopt pre-determined, fixed duty cycles whenever activated, which is not adaptive to different user behaviors. This is also a universal problem in most of the existing sensing systems. We address this problem in Chapter 4 and 5, where we model the user state as a discrete time Markov process and discrete time semi-Markov process 11 respectively, and investigate how sensor duty cycles can be optimized in order to minimize the expected user state estimation error, while maintaining an energy consumption budget. 2.3 Information processing in detection oriented systems Due to the fact that Chapter 4 and 5 of this thesis investigate user state estimation in case of partially observed stochastic process, it is worth introducing the concept of Hidden Markov (semi- Markov) Models and the traditional techniques used for state estimations, such as the forward- backward algorithm [70] and the Viterbi algorithm [85]. We divide the underlying user state process model into two categories, namely, the Markov process model and the semi-Markov process model, and introduce their background respectively in the following. 2.3.1 The Markovian case 2.3.1.1 Hidden Markov models The basic theory of hidden Markov models (HMM) was published by Baum et al. in a series of papers [10–14] in the late 1960s and early 1970s, and its implementation in speech processing has been studied by Baker [6] and Jelinek [44, 45] in the 1970s. More recently, Rabiner [70] provided an overview of the basic theory of HMMs and its applications in speech recognition. An HMM is a discrete Markov process in which the system states cannot be directly observed. Instead, the HMM is observed through a set of outputs, where each output is generated depending on the underlying system state. An HMM is characterized by five elements: • The set of system states is denotes asX ={x 1 ,x 2 ,...,x N }. The state at timet is denoted ass t . 12 • The set of distinct observation symbols, i.e., the outputs, are denoted asY ={y 1 ,y 2 ,...,y M }. The output at timet is denoted aso t . • The state transition probability from system statex i tox j is denoted asa ij , where a ij = Prob[s t+1 =x j |s t =x i ],i,j ∈{1,2,...,N}. (2.1) • The emission probabilityb ik is the probability of detecting outputy k while the system state isx i , i.e., b ik = Prob[o t =y k |s t =x i ],i∈{1,2,...,N},k∈{1,2,...,M}. (2.2) • The initial distributionπ ={π i } where π i = Prob[s 1 =x i ],i∈{1,2,...,N}. (2.3) An illustration of the probability parameters of an HMM is given in Figure 2.1 1 . One typical problem in HMMs is to estimate the most likely underlying system state sequence given the observation sequence. In particular, given the observation sequenceO =o 1 o 2 ···o T and all model parameters, what is the most likely corresponding system state sequenceS =s 1 s 2 ···s T that generates this output sequence and is “optimal” in some sense? Note that since there exist many different optimality criteria, the difficulty usually lies in how to define an “optimal sequence”. In this thesis, we introduce two well-known mechanisms, 1 Source: http://en.wikipedia.org/wiki/Hidden Markov model 13 Figure 2.1: The system states (N = 3), outputs (M = 4), transition probabilities, and emission probabilities. namely, the forward-backward algorithm and the Viterbi algorithm, both attempting to uncover the hidden system state sequence yet aiming to achieve different optimization objectives. 2.3.1.2 The forward-backward algorithm The forward-backward algorithm aims to select an estimated state sequence where the statess t are individually most likely, i.e., it picks the most likely state for each individual time slot. Letλ denote the set of HMM parameters, we define forward variableα t (i) as: α t (i) = Prob[o 1 o 2 ···o t ,s t =x i |λ], (2.4) which is the joint probability of the output sequence till timet and the states t at timet, given the model parameters. The value ofα t (i) can be solved recursively: 14 - Initialization: α 1 (i) =π i b io 1 ,i∈{1,...,N} (2.5) - Induction: α t+1 (i) = N X i=1 α t (i)a ij b jo t+1 ,1≤t≤T −1,1≤j ≤N. (2.6) - Termination: Prob[O|λ] = N X i=1 α T (i). (2.7) Similarly, the backward variableβ t (i) is defined as: β t (i) = Prob[o t+1 o t+2 ···o T |s t =x i ,λ], (2.8) i.e., the probability of the observation sequence from timet+1 till the end, given states t at time t and the model parametersλ. The value ofβ t (i) can be solved recursively as well: - Initialization: β T (i) = 1,i∈{1,...,N} (2.9) - Induction: β t (i) = N X j=1 a ij b jo t+1 β t+1 (j),t =T −1,T −2,...,1,1≤i≤N. (2.10) 15 Now we define the variable γ t (i) = Prob[s t =x i |O,λ] (2.11) i.e., the probability of being in state x i at time t given the entire output sequence and model parameters.γ t (i) can be further written in the following forward-backward form: γ t (i) = α t (i)β t (i) Prob[O|λ] = α t (i)β t (i) P N i=1 α t (i)β t (i) (2.12) Therefore, the most likely states ′ t at timet is selected according to the following: s ′ t = argmax 1≤i≤N [γ t (i)],1≤t≤T = argmax 1≤i≤N [α t (i)β t (i)],1≤t≤T (2.13) because the term P N i=1 α t (i)β t (i) is a normalizing constant. 2.3.1.3 The Viterbi algorithm One problem associated with the forward-backward algorithm is that the resulting estimated state sequence may not be a valid output, due to the fact that Equation 2.13 does not take into con- sideration the occurrence of sequences of states. Unlike the forward-backward algorithm, the Viterbi algorithm finds the best state sequence, i.e., the most likely state sequence within a given estimation interval. 16 Define the quantity δ t (i) = max s 1 ,s 2 ,...,s t−1 Prob[s 1 s 2 ···s t =x i ,o 1 o 2 ···o t |λ] (2.14) which denotes the highest probability along a single path for time t that accounts for all the previous observations and the state at timet. The following relationship holds by induction: δ t+1 (j) = [max i δ t (i)a ij ]·b jo t+1 (2.15) In order to retrieve the most likely state sequence, we need to keep track of all the states that maximize the quantity given by Equation 2.15 , for eacht andi. Therefore, an extra variable τ t (i) is introduced that stores such state information. The complete process for finding the mostly likely state sequence is as follows: - Initialization: δ 1 (i) = π i b io 1 ,i∈{1,...,N} (2.16) τ 1 (i) = 0. (2.17) - Recursion: δ t (j) = max 1≤i≤N [δ t−1 (i)a ij ]b jot , 2≤t≤T,1≤j ≤N, (2.18) and: τ t (j) = argmax 1≤i≤N [δ t−1 (i)a ij ], 2≤t≤T,1≤j ≤N. (2.19) 17 - Boundary condition: s ′ T = argmax 1≤i≤N [δ T (i)]. (2.20) - Constructing state sequence: s ′ t =τ t+1 (s ′ t+1 ),t =T −1,T −2,...,1. (2.21) Note that the termτ t (j) stores the best previous state at timet−1 given thatj is selected at timet, such that the overall path can be constructed by backtracking from the last time slotT . It is important to note that, sinceδ t (j) denotes a “path probability” and this method computes the most likely state sequence instead of estimating the most possible state for each individual time slots, the estimated sequence is considered to be incorrect even if it contains a single bit error. 2.3.2 The semi-Markovian case 2.3.2.1 Semi-Markov processes A semi-Markov process [41] is a generalization of Markov process in which instead of geometri- cally distributed state durations, the state duration can follow any arbitrary distribution type. This allows the Markovian assumption to be relaxed such that analysis can be applied to more general and realistic data. Denote p ij as the probability of state transition from state i to state j. Similar to Markov models, the transition probabilities satisfy p ij ≥ 0,∀i,j ∈{1,2,...,N},i6=j, (2.22) 18 and N X j=1 p ij = 1,∀i∈{1,2,...,N}, (2.23) with no self-transitions, i.e.: p ii = 0,∀i∈{1,2,...,N}. (2.24) However, after the “next state”j has been selected, the process “holds” for a time interval, instead of making the state transition immediately. The corresponding probability mass function (pmf) of the sojourn time of statei is denoted asw i (a) (a ∈ N ∗ ), which is the probability that statei will last fora time slots until the next state transition. w i (a) satisfies 1 ≤ a ≤ A i ≤ ∞, and P a w i (a) = 1, whereA i is the maximum sojourn duration of statei. 2.3.2.2 State estimation in hidden semi-Markov models A hidden semi-Markov model is an extension of the HMM that allows general state duration distributions [31]. In the literature such as [57, 62, 70, 71], HSMM is also named “HMM with variable duration”, “explicit duration HMM”, “generalized HMM”, and so on. In this section, we introduce a forward-backward approach for system state estimation in HSMMs that is originated from [31]. For consistency purpose, we use the same variable definition as in section 2.3.1. The following two forward variables are first defined: α t (i) = Prob[o 1 o 2 ···o t , statei ends att] = A X a=1 α ∗ t−a (i)·w i (a) t Y k=t−a+1 b io k , (2.25) 19 and α ∗ t (i) = Prob[o 1 o 2 ···o t , statei starts att+1] (2.26) = N X j=1 α t (j)·a ji . (2.27) fort = 1,2,...,T . Asα t (i) andα ∗ t (i) are solved recursively, the initial and boundary conditions are given by: α ∗ 0 (i) =π i ,i∈{1,...,N} (2.28) and α ∗ ι (i) = 0,∀ι< 0,i∈{1,...,N}. (2.29) Similarly, the backward variables are defined as follows: β t (i) = Prob[o t o t+1 ···o T | statei starts att] (2.30) = A X a=1 β ∗ t+a (i)·w i (a) t+a−1 Y k=t b io k , (2.31) and β ∗ t (i) = Prob[o t o t+1 ···o T | statei ends att−1] (2.32) = N X j=1 β t (j)·a ij . (2.33) fort =T,T −1,...,1. 20 The boundary conditions are specified as: β ∗ T+1 (i) = 1,∀i∈{1,...,N} (2.34) and β ∗ ι (i) = 0,∀ι>T +1,i∈{1,...,N}. (2.35) The rest is similar to section 2.3.1.2 and the variable γ t (i) = Prob[s t =x i |O,λ] (2.36) is solved for every time slott ∈ {1,...,T}, and the system state at timet is estimated such that γ t (i) is maximized: s ′ t = argmax 1≤i≤N [γ t (i)],1≤t≤T. (2.37) A detailed implementation of the forward-backward based semi-Markovian estimation can be found in Chapter 5 with some modifications in the problem model assumptions. 2.3.3 Differences from traditional models In Chapter 4 and 5 of this thesis, we investigate state estimation in partially observed stochastic processes, i.e., user state information needs to be estimated whenever sensor stays idle. In general, our problem model is different from the traditional HMM (HSMM) model in the following ways: 21 • Sensor observations are assumed to be perfect. Therefore, the output symbol is the same as the underlying system state and the emission probability satisfies: b ik =        1, ifi =k, 0, otherwise. (2.38) • Sampling is carried out intermittently. Therefore, state estimations are conducted for every estimation interval along the sparsely sampled process (see illustration Figure 2.2). Figure 2.2: The process is sampled at discontiguous time slots, leading to multiple estimation intervals. • In case of semi-Markov processes, the state estimation suffers the “unknown past sojourn time” problem, i.e., unlike the traditional assumption in [31], the observed states do not necessarily need to start/finish at the beginning/end of the estimation interval. In Chapter 5 we aim to solve this problem by approximating the past sojourn time in three different ways. 2.4 Constrained stochastic optimization General stochastic optimization tools have been widely used to help design better protocols in mobile operations. For example, in one of the early works, Benini et al. [16] addressed power optimization on battery-operated devices with the assistance of Markov Decision Process (MDP). 22 Cheung et al. [21] propose a MDP based optimization framework to optimize resource usage (e.g., battery-life), such that user-specific reward can be maximized. In most mobile sensing systems, the major design objective is to provide as accurate user context detection as possible; however, turning on the sensor frequently to obtain user status will drain the mobile device battery quickly. In this thesis, we view this as a constrained optimization problem and investigate how sensors could be scheduled optimally such that the expected user state detection error is minimized while satisfying a given energy consumption budget. In this section we introduce a standard optimization tool that is utilized in this thesis to solve the constrained optimization problem - the Constrained Markov Decision Process (CMDP). CMDP [3] is a variant of Markov Decision Processes [15] (MDP) which provides a frame- work for constructing optimal policies for a stochastic process. CMDP considers a situation where one type of cost is to be optimized while keeping others below some pre-defined thresh- olds. For example, Lazar [53] used CMDP to analyze the problem of maximizing throughput subject to delay constraints in telecommunication applications. In traffic applications, Nain and Ross [66] studied the problem where different traffic types compete for the same resource. Some weighted sum of average delays of certain traffic types is to be minimized while some other traf- fic types need to satisfy certain delay constraints. More recent studies of CMDP include [25], where Dolgov and Durfee limit the optimal policy search to stationary deterministic policies cou- pled with a novel problem reduction to mixed integer programming, and the method yields an algorithm that finds computationally feasible policies (i.e., not randomized). 2.4.1 CMDP definition A CMDP is defined by a tuple{X,B,P,c,d} where 23 • X is the state space containing a finite number of states. • B is the finite set of actions. LetK = {(x,b) : x ∈ X,b ∈ B} be the set of state-action pairs. • P are the state transition probabilities, and P ibj is defined as the probability of moving from system statei toj, when actionb is taken. • c :K→R is the type of cost that is to be minimized. • d :K→R K is a vector of immediate costs related toK constraints. 2 2.4.2 Classes of policies The objective of solving a CMDP is to choose a policy which specifies the actions for the con- troller to take at different decision epochs in order to achieve cost optimization under constraints. To define a policy, the history at timet is first defined as a sequence of previous states and actions plus the current state: h t = (x 1 ,b 1 ,...,x t−1 ,b t−1 ,x t ) (2.39) A policyu is a sequenceu = (u 1 ,u 2 ,...) such that if historyh t is observed at timet, the controller will choose actionb with probabilityu t (b|h t ). The following classes of policies are often seen in the literature: • U M (Markov policies):u∈U M ifu t only depends onx t for anyt. • U S (stationary policies):U S is a subset ofU M .w is stationary ifw t does not depend ont. 2 Note that in this thesisK =1 since the only constraint is the sensor energy consumption budget. 24 • U D (stationary deterministic policies): U D is a subset ofU S . g is stationary deterministic if the action chosen at statex is identified by the mapg :X →B. • U R (stationary randomized policies): U R is a subset ofU S . r is stationary randomized if the action at statex is drawn from the setB based on a discrete probability distribution. 2.4.3 Cost criteria Here we introduce two most frequently used cost criteria of CMDP, namely, discounted cost and expected average cost. First, the discounted cost in a finite-horizon CMDP and a infinite-horizon CMDP are defined as follows: C n α (π,u) = (1−α) n X t=1 α t−1 E u π c(X t ,B t ) (2.40) C α (π,u) = (1−α) ∞ X t=1 α t−1 E u π c(X t ,B t ) (2.41) whereπ captures the initial distribution. Note that the discounted cost emphasizes on the near future and gives less importance to the far future. Second, the expected average cost (under finite and infinite horizons respectively) is defined as follows: C n ea (π,u) = P n t=1 E u π c(X t ,B t ) n C ea (π,u) = lim n→∞ C n ea (π,u) DenoteC(π,u) as one general type of cost, the constrained optimization problem (COP) is given as: 25 COP: Find a policy that minimizesC(π,u), s.t.D(π,u)≤V . where bothC(π,u) andD(π,u) stand for one type of the cost defined above. 2.4.4 Solving the CMDP It has been shown by Feinberg and Shwartz [29, 30] that stationary deterministic policies are not guaranteed to be optimal due to the constraints added to the classical MDP model. In order to solve for the optimal stationary randomized policy in polynomial time, Kallenberg [47], and Heyman and Sobel [39], propose the use of LP where adding constraints with the same discount factor does not increase the complexity of the problem. In this section we introduce the use of LP to solve the constrained optimization problems defined in the previous section with infinite horizon and expected average cost. We first need to define a very important concept, namely, the occupation measure corresponding to a policy u, which is the total expected time spent in different state-action pairs. It therefore specifies the probability that a particular state-action pair ever exists in the decision process. For given initial distributionπ and policyu, the occupation measure is defined as: f t ea (π,u;x,b) = 1 t t X s=1 P u π (X s =x,S s =b),b∈B(x) (2.42) The cost can be therefore represented as: C ea (π,u) = X y∈X X b∈B(x) f(y,b)c(y,b) (2.43) 26 Let Q ea (π) be the set of occupation measures that satisfies: Q ea (π) =                        ρ(y,b),y ∈X,b∈B(y) : P y∈X P b∈B(x) ρ(y,b)(δ x (y)−P ybx ) = 0,x∈X P y∈X P b∈B(x) ρ(y,b) = 1 ρ(y,b)≥ 0,∀y,b                        (2.44) It has been shown in [3] that the following LP is equivalent to the COP defined previously: LP(π): Find the infimumC ∗ ofC(ρ) subject to: D k (ρ)≤V k ,k = 1,...,K, andρ∈ Q ea (π) (2.45) whereD k stands for constraint of typek. 27 Chapter 3 The framework design for an Energy Efficient Mobile Sensing System 3.1 Overview In this chapter, we present the framework design, implementation, and evaluation of an Energy Efficient Mobile Sensing System (EEMSS) 1 . EEMSS manages sensors hierarchically by turning on only necessary sensors at any given time and triggering power hungry sensors on demand to achieve automatic user state detection in an energy efficient manner. The benefits of EEMSS’s sensor management methodology are threefold. First, the state description mechanism is a flexible way to add/update user states and their relationship to the sensors. For instance, to account for emerging application needs new states and sensors may be incrementally added to the state description. Second, to achieve energy efficiency, the sensor management methodology assigns the minimum set of sensors and heuristically determines sam- pling lengths and intervals for these set of sensors to detect user’s state as well as transitions to new states. Lastly, our sensor management methodology can be easily extended as a middleware 1 This is based on the work that originally appeared in [90] 28 that manages sensor operations and provides contextual information to higher layer applications with multiple types of devices and sensors involved. EEMSS is currently implemented and evaluated on Nokia N95 devices. In our EEMSS imple- mentation, the state description subsystem currently defines the following states: “Walking”, “Ve- hicle”, “Resting”, “Home talking”, “Home entertaining”, “Working”, “Meeting”, “Office loud”, “Place quiet”, “Place speech” and “Place loud”. All these states are specified as a combination of built-in Nokia N95 sensor readings. The sensors used to recognize these states are accelerometer, WiFi detector, GPS, and microphone. EEMSS incorporates novel and efficient classification al- gorithms for real-time user motion and background sound recognition, which form the foundation of detecting user states. We have also conducted a field user study with 10 users at two different university campuses to evaluate the performance of EEMSS. Our results show that EEMSS is able to detect states with 92.56% accuracy and improves the battery lifetime by over 75%, compared to existing results. Note that although in this chapter we focus only on states that can be detected by integrated sensors on mobile devices, our sensor management methodology is general enough that one can apply our infrastructure to mobile sensing systems that involves more sensors and devices. 3.2 Design methodology The core components of EEMSS can be viewed as a layered architecture that consist of a sensor management module, a classification module, and a sensor control interface. Some other com- ponents have also been incorporated to facilitate developer debugging and evaluation, including 29 real-time user state updates, logging, and user interfaces. Figure 3.1 illustrates the layered de- sign of the system architecture and the interactions among these components: (1) System reads in the XML state descriptor which contains the sensor management scheme. (2) Management module determines the sensors to be monitored based on current user state which is specified by the sensor management scheme. (3) Management module instructs the sensor control interface to turn on/off sensors. (4) Sensor control interface operates individual sensors. (5) Sensor interface reports readings to classification module. (6) Classification module determines the user state. (7) Classification module forwards the intermediate classification result to management module. (8) The user’s state is updated and recorded in real-time. (9) The relevant information is also displayed on the smart phone screen. The sensor control interface contains APIs that provide direct access to the sensors. Through these APIs, application can obtain the sensor readings and instruct sensors to switch on/off for any given duty cycle parameters. We describe the design of the sensor management module and the classification module in more details in the following. 3.2.1 Sensor management module The sensor management module is the major control unit of the system. It first parses a user state description file (in XML format) that describes the sensor management scheme, and then controls the sensors based on the sensing criteria of each user state and state transition conditions by spec- ifying the minimum set of sensors to be monitored under different scenarios (states). The sensor management module configures the sensors in real-time according to the intermediate classifica- tion result acquired from the classification module and informs the sensor control interface what sensors to be turned on and off in the following step. 30 Figure 3.1: System architecture of EEMSS implementation on Nokia N95. 3.2.1.1 State and sensor relationship: the XML-format user state descriptor Sensor assignment is achieved by specifying an XML-format state descriptor as system input that contains all the states to be automatically classified as well as sensor management rules for each state. The system will parse the XML file as input and automatically generate a sensor management module that serves as the core component of EEMSS and controls sensors based on real-time system feedback. In essence, the state descriptor consists of a set of state names, sensors to be monitored, and conditions for state transitions. Figure 3.2 illustrates the general format of a user state descriptor and the corresponding state transition process. It can be seen that a user state is defined between the “<State>” and “</State>” tags. For each state, the sensor(s) to be monitored are specified by “<Sensor>” tags. 31 The hierarchical sensor management is achieved by assigning new sensors based on previous sen- sor readings in order to detect state transition. If the state transition criteria has been satisfied, the user will be considered as entering a new state (denoted by “<NextState>” in the descriptor) and the sensor management algorithm will restart from the new state. For example, based on the sam- ple description in figure 3.2, if the user is at “State2” and “Sensor2” returns “Sensor reading 2” which is not yet sufficient for detecting state transition, “Sensor3” will be turned on immediately to further detect the user’s status in order to identify state transition (see figure 3.3). There are three major advantages of using XML as the format of state descriptor. First, XML is a natural language to represent states in a hierarchical fashion. Second, new state descrip- tors can be added and existing states can be modified with relative ease even by someone with limited programming experience. Finally, XML files are easily parsed by modern programming languages such as Java and Python thereby making the process portable and easy to implement. 3.2.1.2 Sensor duty cycles In EEMSS implementation, sensor duty cycles are arbitrarily chosen based on extensive trials that measure the power consumption of each sensor and computation time based on different sample lengths of data. Table 3.1 summarizes the power consumption of different sensors on Nokia N95 device. Among these sensors, accelerometer consumes the least amount of power compared to other sensors, and fortunately accelerometer is capable of detecting any body movements with a high precision. Hence accelerometer could be first indicator of state transition with high proba- bility if it involves user body motion. In such a case, accelerometer could be sampled periodically as triggers to invoke other sensors if necessary. On the other hand, due to the large power drain 32 and long initialization time, GPS is used only when it is necessary to measure the speed of user’s movement so as to discriminate between modes of travel such as riding in vehicle versus walking. Sensor Power(W) Current(A) Accelerometer 0.0599 0.0150 Microphone 0.2786 0.0707 GPS 0.3308 0.0862 WiFi Scan 1.4260 0.3497 Bluetooth Scan 0.1954 0.0486 Table 3.1: Power and current consumption summary for different sensors on Nokia N95. Table 3.2 summarizes the duty cycle parameters for each of the four sensors used in EEMSS. It is important to note that a particular length of sampling duration is required for a sensor to gather sufficient data. A tradeoff in the selection of sensor idle interval is that longer sleep interval may reduce energy consumption, but the state detection error will be increased since there exist more missing observations. Even though the duty cycle parameters have been refined through extensive empirical tests, the ones finally adopted by EEMSS (as shown in table 3.2) may not be optimal in achieving the best tradeoff between energy consumption and state detection accuracy. In Chapter 4 and Chapter 5, we will systematically investigate this problem and discuss our improved design where we model the energy efficient mobile sensing problem as the question “how a (semi-)Markov Sensor Duty Cycles Accelerometer 6s sensing + 10s sleeping Microphone 4s sensing + 180s sleeping GPS Queries every 20s, timeout in 5 min WiFi scan Event triggered (< 2s to finish)2 Table 3.2: Sensor duty cycles used in current EEMSS implementation 33 Sensor Computation Time/Sample Energy(J)/Sample Accelerometer < 0.1s 0.359 Microphone Quiet:< 0.5s. Loud/Speech:∼ 10s. 1.114 GPS < 0.1s 6.616 WiFi scan < 0.1s 2.852 Table 3.3: The computation time and sensor energy cost per sample corresponding to table 3.2. process can be efficiently sampled?”, and find ways to calculate the sensor duty cycle parameters to achieve the optimal tradeoff between energy consumption and user state estimation error. The computation time (including the time for data processing and classification) and sensor energy consumed per sample based on sensor duty cycle parameters, are summarized in table 3.3. It can be seen that except for loud audio signal processing and classification, which takes approxi- mately 10 seconds to complete (mainly consumed at the FFT stage), all other computations finish almost instantaneously, which enables our system to conduct real-time state recognition. The energy consumption results of each senor are also measured and illustrated in table 3.3. 3.2.1.3 Generalization of the EEMSS framework Various sensors make the user’s contextual information available in multiple dimensions, from which a rich set of user states can be inferred. However, in most cases different users or higher layer applications may only be interested in identifying a small subset of states and exploit the state information for application customization. For example, a ring tone adjustment application, which can automatically adjust the cell phone alarm type, may only need to know the property of background sound in order to infer the current situation. A medical application may require the system to monitor one’s surrounding temperature, oxygen level and the user’s motion such as running and walking to give advise to patient or doctors. In a personal safety application, one 34 factor that one may care is whether the user is riding a vehicle or walking alone such that the mobile client is able to send warning messages to the user when he or she is detected walking in an unsafe area at late night. These are all examples of mobile sensing systems with particular needs, by which our EEMSS framework design can be potentially adopted. New user states can also be added or updated in EEMSS since the XML based state descrip- tion mechanism is highly scalable. A classification algorithm has to be associated with the new state, which takes readings from one or more sensors as input in order to detect the new state. Once the classification algorithm is defined, it needs to be incorporated to the EEMSS classifica- tion module. 3.2.2 Classification module The classification module is the consumer of the sensor raw data. The classification module first processes the raw sensing data into desired format. For example, the magnitude of 3-axis ac- celerometer sensing data is computed, and FFT is performed on sound clips to conduct frequency domain signal analysis. The classification module returns user activity and position feature such as “moving fast”, “walking”, “home wireless access point detected” and “loud environment” by running classification algorithms on processed sensing data. The resulting user activity and position information are both considered as intermediate state which will be forwarded to the sensor management module. The sensor management module then determines whether the sens- ing results satisfy the sensing criteria and decides sensor assignments according to the sensor management algorithm. 35 3.2.2.1 GPS sensing and mode of travel classification In our study, the primary purpose of using GPS is to detect user’s location as well as mode of travel. By combining the instantaneously velocity information and the recent distance of travel measured by comparing current position with previous ones it is possible to distinguish one’s ba- sic mode of travel such as walking or riding a vehicle. For example, if the velocity is greater than 10 mph we consider that the user is using an automotive transport. The classifier is trained by sev- eral location tracking records of user and certain threshold values are identified and implemented into the classification algorithm. GPS can also be used to identify when a user has entered a building or other indoor envi- ronment since a location request timeout will occur since the satellite signals are are likely to be blocked in the indoor environment. It is worth mentioning that from the system implementation point of view, obtaining instant speed as well as the location request timeout functionality are both supported by J2ME API. 3.2.2.2 WiFi scanning and usage The MAC address of visible wireless access points around the user can be inferred by performing a WiFi scan. Since MAC address of each wireless access point is unique, it is possible to tag a particular location by the set of access points in that location. Therefore the mobile device is able to automatically identify its current location by simply detecting nearby wireless access points. For example, it is easy to tell that the user is at home if the WiFi scan result matches his or her home access point set that is pre-memorized by the device. In our current EEMSS implementation, the wireless access points feature of the user’s home and office (if applicable) will be pre-recorded for recognition purpose. While in our current implementation we pre-record 36 the set of access points for each of user’s well defined locations, such as home and office, there are several alternative implementations such as SkyHook [93] that provide the location information by a database table lookup of a set of access points. WiFi scan can also be used to monitor a user’s movement range since a wireless access point normally covers an area of radius 20-30m. Hence if the user moves out of his/her recent range a WiFi scan will detect that the current set of WiFi access points are replaced by a new one. In our system implementation, if the user is detected moving continuously by accelerometer, a WiFi scan will be performed to check whether the user has left his or her recent location and if so, GPS will be turned on immediately to start sampling location information and classify the user’s mode of travel. 3.2.2.3 Real-time motion classification based on accelerometer sensing Activity classification based on accelerometer readings has been widely studied using various machine learning tools [8, 22, 52, 97]. However, in most of the previous works one or more ac- celerometer sensors have to be attached to specific body joints such as knees and elbows. Several data features are then extracted from readings of multiple accelerometers in order to design so- phisticated classifiers to recognize user activities. Most of these classification algorithms are both data and compute intensive and hence are unsuitable for real-time classification using current mobile phone computing capabilities. In EEMSS design, mobile phone is the only source of accelerometer readings. We only make the assumption that the mobile phone is carried by the user at all times without any placement restrictions. Hence, it becomes extremely difficult to perform detailed level motion classification using a single accelerometer. We use only the standard deviation of accelerometer magnitude as 37 Mode STDV Range of Magnitude Still 0 - 1.0334 Walking 9.2616 - 21.3776 Running 35.3768 - 52.3076 Vehicle 4.0204 - 12.6627 Table 3.4: Standard deviation range of accelerometer magnitude readings for different user activ- ities one of the defining features independent of phone placement in order to conduct real-time motion classification, recognizing four human motion modes namely “Still”, “Walking”, “Running”, and “Vehicle”. We have collected accelerometer data in 53 different experiments distributed in two weeks in order to train the classifier. The lengths of experiment vary from several minutes to hours. Within each empirical interval, the person tags the ground truth of his activity information for analysis and comparison purposes. The standard deviation for different activities within each empirical interval is computed off-line. Table 3.4 shows the range of standard deviation distribution based on different data sets collected. It can be found out that there exist certain standard deviation threshold values that could well separate stable, walking, running, and vehicle mode with high accuracy. In order to verify this ob- servation, we have implemented a real-time motion classification algorithm on N95 mobile phone that compares the standard deviation of accelerometer magnitude values with the thresholds in order to distinguish the user’s motion. The device is carried by the user without explicit require- ment of where the phone should be placed. 26 experiments have been conducted each containing a combination of different user motions. The standard deviation of accelerometer magnitude is computed every 6 seconds, and right after which the user’s motion is being classified. Table 3.5 shows the classification results in percentage of recognition accuracy. It can be seen that the al- gorithm works very well for extreme conditions such as stable and running. Furthermore, even 38 Still Vehicle Walking Running Still 99.44% 0.56% 0 0 Vehicle 8.81% 73.86% 16.29% 1.04% Walking 1.18% 10.62% 88.20% 0 Running 0 0 0 100% Table 3.5: Classification results based on standard deviation of accelerometer magnitude values. The first column represents the ground truth while the first row represents the classification results based on accelerometer readings. though the classifier tends to be confused with walking and vehicle mode due to feature overlap, the accuracy is still well maintained above 70%. 3.2.2.4 Real-time background sound recognition This subsection describes the algorithm used for background sound classification. These algo- rithms were coded in Java and run on N95 to classify sound clips recorded using N95. The device records a real time audio clip using microphone sensor and the recorded sound clip will go through two classification steps (figure 3.4). First, by measuring the energy level of the audio signal, the mobile client is able to identify if the environment is silent or loud. Note that the energyE of a time domain signalx(n) is defined byE = P n |x(n) 2 |. Next, if the environment is considered loud, both time and frequency domains of the audio signal are further examined in order to recognize the existence of speech. Specifically, speech signals usually have higher silence ratio (SR) [59] (SR is the ratio between the amount of silent time and the total amount of the audio data) and significant amount of low frequency components. If speech is not detected, the background environment will simply be considered as “loud” or “noisy”. No further classi- fication algorithm is conducted in EEMSS to distinguish audio signals such as music, noise and other types of sound due to their vast variety of the signal features compared to speech. 39 Speech Music Noise SR thres = 0.6 95.31% 18.00% 26.07% SR thres = 0.7 91.14% 16.00% 18.17% SR thres = 0.8 70.91% 10.00% 11.66% SR thres = 0.9 32.64% 8.00% 8.59% Table 3.6: Percentage of sound clips classified as “speech” for differentSR thres values. SR is computed by picking a suitable threshold and then measuring the total amount of time domain signal whose amplitude is below the threshold value. The Fast Fourier Transform (FFT) has been implemented such that the mobile device is also able to conduct frequency domain analysis to the sound signal in real time. Figure 3.5 shows the frequency domain features of four types of audio clips, including a male’s speech, a female’s speech, a noise clip and a music clip. It can be seen clearly that as compared to others, speech signals have significantly more weight on low frequency spectrum from 300Hz to 600Hz. In order to accomplish speech detection in real time, we have implemented the SSCH (Subband Spectral Centroid Histogram) algorithm [32] on mobile devices. Specifically, SSCH passes the power spectrum of the recorded sound clip to a set of highly overlapping bandpass filters and then computes the spectral centroid 2 on each subband and finally constructs a histogram of the subband spectral centroid values. The peak of SSCH is then compared with speech peak frequency thresholds (300Hz - 600Hz) for speech detection purpose. Figure 3.6 illustrates the outputs of applying SSCH algorithm to the same sound clips used in figure 3.5. It can be found out clearly that the histogram peaks closely follow the frequency peaks in the original power spectrum. The classification algorithm is trained and examined on the same data set including 1085 speech clips, 86 music clips and 336 noise clips, all with 4 seconds length which are recorded by Nokia N95 devices. We investigate the effect of different SR thresholds (denoted bySR thres ) 2 The spectral centroidC of a signal is defined as the weighted average of the frequency components with magni- tudes as the weights:C = P f f·X(f) P f X(f) 40 on classification accuracy. The results of speech detection percentage are shown in table 3.6. It can be seen that as SR threshold increases, the number of false positive results are reduced with sacrifice of speech detection accuracy. We choose SR thres = 0.7 in EEMSS implementation which provides more than 90% of detection accuracy and less than 20% false positive results. The above classification results also show that a 4-second sample of audio clip is long enough for the classifier to identify the background sound type. It is also important to note that the complexity of the SSCH algorithm isO(N 2 ) and as the filter overlaps are small, the running time is empirically observed to be close to linear. Empirical results show that on average the overall processing time of a 4 seconds sound clip is lower than 10 seconds on N95 devices. In future as compute capabilities of mobile phones increase we expect the latency of such complex audio processing will be reduced significantly. 3.3 Performance evaluation We have conducted a user trial in November 2008 at University of Southern California and Carnegie Mellon University. The main purpose of the user trial was to test the performance of EEMSS system in a free live setting. We recruited 10 users from both universities including undergraduate and graduate students, faculties and their family. Each participant was provided with a Nokia N95 device with the EEMSS framework pre-installed. Basic operating instructions were provided to the users at the start of the experimental trials. Each user carried the device with EEMSS running for no less than two days. Each participants was requested to fully charge the mobile battery before starting the EEMSS application. EEMSS will continue to run in the background till the battery is completely drained. Participants then fully charge the phone once 41 again before starting the EEMSS application. The cycle of fully charging and discharging con- tinued till the expiration of user’s participation time in the experiments. At the end of the EEMSS evaluation period, we collected more than 260 running hours of data with more than 300 user state transitions detected. 3.3.1 Verifying EEMSS capabilities EEMSS is able to characterize a user’s state by time, location and activities. Besides providing real-time user state update, EEMSS keeps tracking the user’s location by recording the Geo- coordinates of the user when he or she is moving outdoor (recall that GPS is turned on contin- uously and the location information is retrieved every 20 seconds in this scenario). Figure 3.7 visualizes the data collected by EEMSS on 2-D maps. They show the daily traces captured by EEMSS of two different participants from CMU and USC on two campus maps respectively. Within the time frame of these two traces, the EEMSS application keeps running and the phone has not been recharged. Both users have reported that they took buses to school, and the dashed curves which indicate “Vehicle” mode are found to match the bus routes perfectly. The solid curves indicating “Walking” state match the traces that the user is walking between home and bus station, and within university campus. Besides monitoring location change and mode of travel of the user in real-time, EEMSS also automatically detects the surrounding condition when the user is identified to be still at some places in order to infer the user’s activities. In figure 3.7, by probing background sound, the surrounding conditions of the user can be classified as quiet, loud and containing speech. Consequently, the system is able to infer that the user is working, meeting, resting, etc. by combining the detected background condition with location information obtained by WiFi scan. 42 3.3.2 State recognition accuracy We first study the state recognition accuracy of EEMSS for each user. We compared the state predicted by EEMSS with the ground truth state recorded in the user’s diary for every time step. Accuracy is defined as the number of correct predictions over the total number of predictions. It has been found out that the average recognition accuracy over all users is found to be 92.56% with a standard deviation of 2.53%. We also examine the overall state recognition accuracy in terms of the confusion matrix of identifying “Walking”, “Vehicle” and “At some place”. Table 3.7 shows the corresponding con- fusion matrix. The first column represents the ground truth while the first row represents the returned states by EEMSS. It can be seen that the accuracy is very high when the user is staying at some place such as home, office, etc. compared to traveling outdoors. From this table, 12.64% of walking time and 10.59% of vehicle time is categorized as “At some place”. This is because that GPS readings are unavailable due to the device limitations which causes the location timeout and hence the system considers that the user has entered some place. However, this false conclu- sion can be self-corrected since the accelerometer is able to monitor the motion of the user when he or she is considered still and hence GPS will be turned back on immediately as the user keeps moving. The reason that riding a vehicle is recognized as walking is that there exist extreme conditions where the vehicle is moving very slowly and is thus being recognized as the wrong state. 3.3.3 Device lifetime The benefit of low energy consumption of EEMSS has been verified by researchers through lab studies lasting for 12 days. During each day of the study period two researchers have each carried 43 At some place Walking Vehicle At some place 99.17% 0.78% 0.05% Walking 12.64% 84.29% 3.07% Vehicle 10.59% 15.29% 74.12% Table 3.7: EEMSS confusion matrix of recognizing “Walking”, “Vehicle” and “At some place”. The first column represents the ground truth while the first row represents the recognition results. a fully charged N95 smart phone with EEMSS application running on the background and the de- vice lifetime has been examined. The average device lifetime result with EEMSS application running on a fully charged Nokia N95 device is 11.33 hours with regular cell phone functional- ities (as can be seen in figure 3.8). Note that the device lifetime may vary due to different user behaviors. For example, if the user stays at home most of the time with little activities, the device may stay active for much longer time since only accelerometer and microphone will be periodi- cally monitored as compared to the case where one spends most of the time traveling outdoor and the device lifetime will significantly decrease due to extensive GPS usage. We have also examined the device lifetime by turning on GPS, accelerometer and microphone on N95 device with the same sampling frequencies as used in EEMSS, and WiFi scanning is performed every 5 minutes (Note that in EEMSS the WiFi scan is only conducted when the user is leaving or arriving at some places). The device lifetime is found to be less than 5 hours regardless of user activity since no sensor management mechanism is implemented and all the sensors will be periodically sampled until running out of battery, shown as “Without Sensor Management” in figure 3.8. In [60] it has been shown that the CenceMe application can last for around 6.22 hours with no other applications running on the phone. Although the comparison may not be comprehensive it is the only known result so far that describes the device lifetime with an mobile urban sensing application running. Note that there exist some differences between CenceMe 44 and EEMSS implementations. For example, CenceMe adopts an audio sampling duration of 30 seconds and the implements a 60 seconds duty cycle for the microphone, whereas in EEMSS the audio sampling duration is only 4 seconds and the microphone duty cycle is 180 seconds. On the other hand, CenceMe implements an approximately 10 minutes duty cycle for GPS whereas in EEMSS GPS will be turned on continuously when the user is moving outdoors to provide location tracking. Moreover, CenceMe contains data upload cycles and Bluetooth probing that require extra power usage which are not implemented in our system. The results of battery life durations are summarized in figure 3.8, and it can be seen that EEMSS gains more than 75% running hours compared to CenceMe and even larger amount compared to the case where a urban sensing is conducted without sensor management. To visualize the effect of sensor energy management, figure 3.9 illustrates the energy con- sumption model of our system at a glance in a 20 minutes interval when the user walks from his office to a library. It can be seen that at the beginning of the test when the user is sitting in the office, only accelerometer and microphone is being sampled to detect user movement and identify background sound type. When movement is detected, WiFi scanning will be performed to determine whether the user has left the recent location. Note that multiple WiFi scans may be required until the user leaves his previous position. As the user walks towards the library, GPS is turned on in order to provide positioning information and we allow 5 minutes for GPS timeout as the user enters the library where no GPS signal can be received. Finally, a WiFi scan is performed to recognize the current location and accelerometer will be turned back on for user motion detection. 45 3.4 Summary We presented the design, implementation, and evaluation of EEMSS in this chapter. The core component of EEMSS is a sensor management scheme for mobile devices that operates sensors hierarchically, by selectively turning on the minimum set of sensors to monitor user state and triggers new set of sensors if necessary to achieve state transition detection. Energy consumption can be reduced by shutting down unnecessary sensors at any particular time. Our implementation of EEMSS was on Nokia N95 devices that uses our sensor management scheme to manage built- in sensors on the N95, including GPS, WiFi detector, accelerometer and microphone in order to achieve human daily activity recognition. We also proposed and implemented novel classification algorithms for accelerometer and microphone readings that work in real-time and lead to good performance. Finally, we evaluated EEMSS with 10 users from two universities and demonstrated that EEMSS is able to provide a high level of accuracy for state recognition, acceptable state transition detection latency, as well as more than 75% gain on device lifetime compared to a state-of-the-art system. 46 Figure 3.2: The format of XML based state descriptor. Figure 3.3: The state transition implication of figure 3.2. 47 Figure 3.4: Decision tree based background sound classification algorithm. 0 1000 2000 3000 4000 0 2 4 6 8 x 10 7 Magnitude Frequency Speech−Male 0 1000 2000 3000 4000 0 2 4 6 8 10 x 10 7 Magnitude Frequency Speech−Female 0 1000 2000 3000 4000 0 0.5 1 1.5 2 2.5 3 x 10 8 Magnitude Frequency Noise 0 1000 2000 3000 4000 0 0.5 1 1.5 2 x 10 8 Magnitude Frequency Music Figure 3.5: Shows the frequency domain features after applying FFT to different types of audio signals. 48 0 1000 2000 3000 4000 0 2 4 6 8 10 Magnitude Frequency Bins Speech−Male 0 1000 2000 3000 4000 0 2 4 6 8 10 Magnitude Frequency Bins Speech−Female 0 1000 2000 3000 4000 0 2 4 6 8 10 Magnitude Frequency Bins Noise 0 1000 2000 3000 4000 0 2 4 6 8 10 Magnitude Frequency Bins Music Figure 3.6: Shows the frequency histogram plots after applying SSCH to different types of audio signals shown in figure 3.5. 49 Figure 3.7: Figure shows the time, location as well as background sound condition detected by EEMSS, for two sample users. (Top: a CMU user. Bottom: a USC user.) 0 2 4 6 8 10 12 Battery Life (hours) CenceMe EEMSS Without Sensor Management Figure 3.8: Average N95 lifetime comparison of running EEMSS, CenceMe and the case where all sensors are powered. 50 Figure 3.9: Power usage at a glance. 51 Chapter 4 User state estimation and sensing policy optimization: the Markovian case 4.1 Overview As described in Chapter 3, in EEMSS when a sensor is activated it adopts a fixed uniform sensor duty cycle parameter that is not adaptive to user state changes. A large amount of energy is thus wasted by frequently sampling a sensor while the user state stays unchanged. This is in fact a universal problem found in most sensing and signal processing based systems. In order to solve this problem and further improve energy efficiency in mobile sensing, we analytically study the sensor 1 duty cycle selection problem by proposing state estimation mechanism and designing sensing policy optimization framework for Markovian user state processes in this chapter 2 . We assume that time is discretized and the user state evolves as a hidden discrete time Markov chain (DTMC). At each time slot, a sensor can either be sampled to observe user state (assuming 100% detection accuracy) with a unit energy cost associated, or stay idle to save energy. 1 The term “sensor” used here can represent both physical sensors or networking transceivers that can probe the environment by sending packets. 2 This is based on the work that originally appeared in [89] 52 We first propose a state estimation method originated from the forward-backward algorithm that selects the most likely state for each time slot along an estimation interval. We then consider the following constrained optimization question: Given a certain device energy consumption budget, what is the optimal sensor sampling policy that leads to the minimum state estimation error, while satisfying the energy budget? This problem of selecting optimal sampling interval lengths with energy consumption con- straints such that the expected state estimation error can be minimized, is formulated as a CMDP problem. Solving such CMDP yields a stationary randomized policy which has been shown to be the optimal solution among all policies in [3]. Moreover, since the optimal solution of such CMDP can be obtained by solving its corresponding Linear Programming (LP), the computa- tional complexity stays polynomial for any given number of user states. 4.2 Preliminaries 4.2.1 Model and assumptions We assume that time is discretized and the user state evolves as a N-state discrete time Markov chain (DTMC) with transition probabilitiesp ij (i,j ∈{1,2,...,N}) from statei toj. The discrete time horizon is denoted asT ={t,t = 0,1,2,...}. At each time slot during the process, a sensor may be sampled, in which case user state is detected with 100% accuracy and a unit of energy consumption is associated. On the other hand, the sensor can also stay idle to save energy. This process is illustrated in figure 4.1. We defineO as the set of time slots when the sensor makes observation and O is thus a subset of T , as shown by figure 4.1. Ideally, if the sensor can be sampled in each time slot (O =T ), the user state sequence can be obtained perfectly. However, 53 Figure 4.1: Illustration of the Markovian user state sequence, sampled at certain time slots. in general, since sensor duty cycles are required to extend mobile device lifetime, there exist durations where user state needs to be estimated using only observed data and the knowledge of the user state transition dynamics. 4.2.2 User state estimation with missing observations The problem of estimating missing state information between two subsequent observations is formulated as follows: given the fact that one observation detects statei at some timet m and the next observation detects statej at timet n (t m ,t n ∈T andt m <t n ), what is the most likely state sequence that connects them? In this section, we propose a mechanism that estimates the most likely user state for each time slot between two subsequent observations. 54 Given the sensor detects statei at timet m , and detects statej at timet n , the probability of being in statek (k∈ 1,2,...,N) at timet (t m <t<t n ), is given by: p[s t =k|s tm =i,s tn =j] = p[s t =k,s tn =j|s tm =i] p[s tn =j|s tm =i] = p[s t =k|s tm =i]·p[s tn =j|s t =k] p[s tn =j|s tm =i] = P (t−tm) ik ·P (tn−t) kj P (tn−tm) ij (4.1) wheres t denotes the true user state at timet. In order to select the “most likely” state (denoted bys ′ t ) at timet, the quantity given in (4.1) needs to be maximized: s ′ t = argmax k ( p[s t =k|s tm =i,s tn =j] ) s ′ t = argmax k ( P (t−tm) ik ·P (tn−t) kj P (tn−tm) ij ) (4.2) The expected estimation error for time slott, denoted bye t , is defined as the probability of incorrect estimation, which is given by: e t = 1−p[s t =s ′ t |s tm =i,s tn =j] (4.3) = 1−max k ( P (t−tm) ik ·P (tn−t) kj P (tn−tm) ij ) (4.4) 4.2.3 Expected energy consumption and expected user state estimation error Throughout this chapter, we address two important performance metrics: (a) the expected energy consumption and (b) the expected user state estimation error. These two intrinsically conflicting 55 metrics are fundamental performance measures of mobile sensing applications, as the former characterizes the device lifetime and the latter is directly related to the quality of the application in terms of providing high user state detection accuracy. The formal definitions of the two are given as follows: Definition 1. The expected energy consumptionE[C] of the user state sampling process is defined as E[C] = 1 E[I] (4.5) whereE[I] is the expected sensor idle interval, i.e., the average number of time slots the sensor waits before the next sample. Definition 2. The expected user state estimation errorE[R] is defined as the long term average of per-slot estimation error: E[R] = lim n→∞ P n t=1 e t n (4.6) 4.2.4 The optimization problem In practical system operations, a device energy consumption budget is often specified in order to ensure a long enough device lifetime. We defineξ (where 0 ≤ ξ ≤ 1) as the energy budget which is the maximum expected energy consumption allowed. Letu denote a sensor sampling policy, which specifies how sensor duty cycles should be assigned when different user states are detected, i.e., how long the sensor should stay idle after each observation. We investigate the following constrained optimization problem: Considering a long user state evolving process that is Markovian, given a device energy consumption budgetξ and user state transition matrixP , 56 what is the optimal sensor sampling policyu ∗ m such that the expected user state estimation error E[R] is minimized, and the expected energy consumption is maintained below the budget, i.e., E[C]≤ξ? 4.3 The Markov-optimal Policy We formulate the above constrained optimization problem as an infinite-horizon CMDP with ex- pected average cost. Solving such CMDP yields a Markov-optimal policyu ∗ m which is stationary and randomized [3,29,30], i.e.,u ∗ m does not vary over time and decision is randomized over sev- eral actions. Meanwhile, the algorithm that finds the Markov-optimal policyu ∗ m is computation- ally efficient, sinceu ∗ m can be obtained by solving the corresponding LP, which is a polynomial time computation. 4.3.1 The CMDP formulation We use an infinite-horizon CMDP to model the sensor duty cycle selection problem, whose major components are explained as follows: • Decision EpochsO. Decision is made immediately after each sensing observation. The set of decision epochs is defined asO = {d,d = 1,2,...}. Note that this is the sameO defined in section 4.2.1. • System State SpaceX. The system state at decision epochd is the detected user states d . The set of system states is denoted byX ={1,2,...,N}. • Action Space B. The set of actions is denoted by B. An action b ∈ B is the number of time slots the sensor waits until making the next observation. Note that the potential 57 number of actions could be infinity, i.e., B = N ∗ , the set of positive integers. However, in order to facilitate the study, we limit the set of actions to positive integers under some thresholdb m . Asξ decreases,b m needs to be increased in order to meet the energy budget constraint. • System Transition ProbabilitiesP. DefineP ibj as the probability of moving from system statei toj, when actionb is taken. Given the user state transition probability matrixP , it is easy to conclude thatP ibj =P (b) ij . • Intermediate Costc(y,b). The intermediate costc(y,b) is defined as the expected aggre- gate state estimation error when action b is taken on state y, where y ∈ X and b ∈ B. c(y,b) can be calculated as: c(y,b) = X j P (b) yj ·e (b) yj = X j P (b) yj · b−1 X t=1 h 1−max k P (t) yk ·P (b−t) kj P (b) yj i (4.7) wheree (b) yj is the aggregate state estimation error for a length-b observation interval starting at statey and ending at statej. • Expected Average CostE ′ u [R]. Under policyu, the expected average cost of the infinite- horizon CMDP is defined as E ′ u [R] = lim n→∞ P n d=1 E u c(X d ,B d ) n (4.8) 58 wheren is the total number of decision epochs, andE u is the corresponding expectation operator of the intermediate cost, under the policyu. • Intermediate Sampling Intervald(y,b). The intermediate sampling interval when action b is taken on statey, satisfiesd(y,b) =b. • Expected Sampling Interval E u [I]. Similar to E ′ u [R], the expected sampling interval E u [I] is defined as: E u [I] = lim n→∞ P n d=1 E u d(X d ,B d ) n (4.9) • Constraint ξ. The maximum allowed expected energy cost. Therefore E[C] ≤ ξ or E[I]≥ 1/ξ. 4.3.2 Finding the Markov-optimal policyu ∗ m Letρ(y,b) denote the “occupation measure” of statey and actionb, i.e., the probability that such state-action pair ever exists in the decision process. 59 It is shown in [3, 39, 47] that the optimal policy of the CMDP with expected average cost criteria can be obtained by solving the corresponding LP. Inspired by these works, we rewrite the sensor duty cycle optimization problem defined above as the following LP (denoted as LP1): LP1: Find min n X y∈X X b∈B ρ(y,b)c(y,b) o (4.10) subject to: X y∈X X b∈B ρ(y,b)(δ x (y)−P ybx ) = 0,∀x∈X, (4.11) X y∈X X b∈B ρ(y,b) = 1, (4.12) ρ(y,b)≥ 0,∀y,b, and (4.13) X y∈X X b∈B ρ(y,b)(1−b·ξ)≤ 0, (4.14) whereδ x (y) is the delta function ofy concentrated onx. The constraint given in (4.11) empha- sizes the property for ergodic processes and describes that the outgoing and incoming rate for a state need to be the same. The constraints (4.12) and (4.13) defineρ(y,b) as a probability mea- sure. The inequality constraint given in (4.14) guarantees that the expected energy usage is less than the energy constraint valueξ, i.e.: E[Overall Energy Consumed] E[Average Sensing Interval Length] ≤ξ. (4.15) Since the per slot energy consumption is 1 unit, the inequality (4.15) can be further written as: P y P b ρ(y,b) P y P b ρ(y,b)·b ≤ξ, (4.16) 60 which leads to the energy constraint in (4.14). Solving LP1 yields the ideal occupation measure of each state/action pair that minimizes the quantity given in (4.10). The algorithm for selecting the Markov-optimal policyu ∗ m is given as follows: Algorithm 1 Pseudo code for selecting the Markov-optimal sampling policyu ∗ m and calculating the corresponding expected user state estimation errorE u ∗ m [R] 1: Input:ξ,P 2: Output:u ∗ (ξ,P),E u ∗ m [R] 3: Initialize: Select a large enoughb m ∈N ∗ such that the energy constraint can be sufficiently satisfied. LetB ={1,2,...,b m },X ={1,2,...,N} 4: Solve LP1 for the optimal occupation measureρ ∗ (y,b),∀y ∈X,∀b∈B 5: Calculateρ ∗ y = P b∈B ρ ∗ (y,b),∀y ∈X 6: Constructingu ∗ m : Assign ρ ∗ (y,b) ρ ∗ y as the probability of taking actionb at statey,∀y ∈X,b∈B 7: Calculate the expected user state estimation error: E u ∗ m [R] = E ′ u ∗ m [R] 1/ξ = P y∈X P b∈B ρ ∗ (y,b)c(y,b) 1/ξ (4.17) In Algorithm 1, the optimal occupation measure of each state-action pair can be obtained by solving LP1 in step 4. Step 5 of Algorithm 1 calculates the overall probability of seeing a particular statey in the decision process under the optimal policy, and this quantityρ ∗ y is used later in step 6 as the normalization to the optimal occupation measureρ ∗ (y,a) in order to compute 61 the probability of taking actionb whenever statey is detected, in the optimal sampling policyu ∗ m . This normalization step is needed to guarantee that X b ρ ∗ (y,b) ρ ∗ y = 1,∀y ∈X (4.18) In equation (4.17), the optimal expected user state estimation errorE u ∗ m [R] can be calculated once the optimal expected average cost E ′ u ∗ m [R] of the CMDP is obtained. Here we illustrate the relationship betweenE[R] andE ′ [R]. Recall thatE[R] is defined as the long-term, average per-slot error, i.e., the ratio of incorrect estimations over total number of time slots; therefore, under policyu, it can be written as: E u [R] = lim n→∞ P n d=1 E u c(X d ,B d ) P n d=1 E u d(X d ,B d ) (4.19) = lim n→∞ 1 n · P n d=1 E u c(X d ,B d ) 1 n · P n d=1 E u d(X d ,B d ) (4.20) wheren denotes the total number of decision epochs, whereasE u c(X d ,B d ) andE u d(X d ,B d ) denote the aggregated state estimation error and the sampling interval (action) size for the dth decision epoch under policyu, respectively. Considering the Law of Large Numbers, equation (4.20) can be further written as E u [R] = E ′ u [R] E u [I] (4.21) 62 whereE u [I] is the expected sampling interval (action) size under policyu. Since the optimal pol- icyu ∗ m needs to be fully utilizing the energy budget, its expected sampling interval size satisfies: E u ∗ m [I] = 1/ξ. (4.22) It can be seen that solving LP1 to obtain the optimal occupation measures in step 4 is the most time-consuming procedure, as a result, Algorithm 1 is able to return the Markov-optimal policyu ∗ m in polynomial time for any number of state inputs. 4.3.3 A case study: obtainingu ∗ m for sample Markov chains In this section, as a case study, we apply Algorithm 1 to some example Markov chains with 2 and 3 states, and find the optimal policyu ∗ m by solving the corresponding LP in MATLAB. We aim to illustrate the outcome of Algorithm 1 by showing the obtained sensing interval at each state when different energy budgets are specified. The six different user state transition matrices tested are:     0.9 0.1 0.1 0.9         0.9 0.1 0.5 0.5         0.9 0.1 0.9 0.1             0.1 0.7 0.2 0.1 0.7 0.2 0.1 0.7 0.2                 0.4 0.1 0.5 0.3 0.6 0.1 0.2 0.2 0.6                 1/3 1/3 1/3 1/3 1/3 1/3 1/3 1/3 1/3         Different energy budget constraint values ranging from 0 to 1 are specified as different inputs to Algorithm 1. 63 The solution of the LP returns a randomization over all possible state-action pairs, which es- sentially constructs the stationary randomized optimal policyu ∗ m . Since it is difficult to visualize the randomization of multiple actions, we compute the mean value of actions taken at each deci- sion epoch for different user states, i.e., the expected average idle interval at each state according to the optimal policyu ∗ m , and show the result of the average actions in figure 4.2. 0 0.5 1 0 5 10 15 20 25 Energy Budget Value Optimal I 1 and I 2 0 0.5 1 0 5 10 15 20 25 30 Energy Budget Value Optimal I 1 and I 2 0 0.5 1 0 5 10 15 20 25 Energy Budget Value Optimal I 1 and I 2 0 0.5 1 0 5 10 15 20 25 Energy Budget Value Optimal I 1 , I 2 , and I 3 0 0.5 1 0 5 10 15 20 25 30 Energy Budget Value Optimal I 1 , I 2 , and I 3 0 0.5 1 0 5 10 15 20 25 Energy Budget Value Optimal I 1 , I 2 , and I 3 I 1 I 2 I 3 I 1 I 2 I 3 I 1 I 2 I 3 I 1 I 2 I 1 I 2 I 1 I 2 Figure 4.2: An illustration of the average idle intervals inu ∗ m for different energy budgets. Sub- graphs are positioned according to the six transition matrices given before. I 1 ,I 2 , andI 3 are the average optimal idle interval lengths when state 1, 2, and 3 is observed, respectively. When the row elements of the Markov transition matrix are identical, the Markov chain de- generates to an independently and identically distributed (i.i.d.) user state sequence, where the user state transition does not depend on any current or previous state information. In this case, the expected state estimation errorE u ∗ m [R] under the optimal policyu ∗ m can be quantified in terms of user state transition probability matrixP and the energy budgetξ. The following theorem holds. 64 Theorem 1. For Markov chain that degenerates to an i.i.d. user state sequence, given its tran- sition probabilities p ij (i,j ∈ {1,2,...,N}) and the energy budget ξ, the following holds for optimal sampling policyu ∗ m : E u ∗ m [R] = (1−ξ)·(1−max ij {p ij }) (4.23) i.e.,E u ∗ m [R] is linear inξ. Proof. For i.i.d. transition probability matrix the following holds due to identical row elements in the transition matrix: P =P (1) =P (2) =... =P (I) ,∀I ∈N ∗ (4.24) that is, the state transition probability matrix inI steps, denoted byP (I) , is equal to the original state transition probability matrixP , for any positive integerI. Therefore, the terme t in equation (4.4) can be further written as: e t = 1−max k P ik ·P kj P ij (4.25) = 1−max k {P ik } (4.26) i.e., e t stays constant no matter what sample interval is used. This illustrates the fact that the estimation error for each time slot with missing observation is constant. Recall that E[R] is defined as the expected per-slot estimation error, and that the optimal policy utilizes the full 65 energy budget with expected sensor sampling interval 1/ξ, therefore,E u ∗ m [R] can be expressed as: E u ∗ m [R] = 1/ξ−1 1/ξ ·e t = (1−ξ)·(1−max ij {p ij }) (4.27) for optimal policyu ∗ m . It can be concluded from Theorem 1 that, for Markov chains which degenerates to an i.i.d. user state sequence, there exist more than one optimal policy because any randomization of ac- tions that leads to the optimal sampling interval results in the same expected state estimation error. Another observation from figure 4.2 is that, whenever the two-state Markov chain is sym- metric, i.e.,p 12 = p 21 , the average sampling intervals for both state 1 and 2 are equal. In fact, the following theorem describing the property of optimal policy holds for all symmetric two-sate Markov chain: Theorem 2. For two-state symmetric Markov chain where p 12 = p 21 , there exists an optimal sampling policy that makes the same decision, i.e., selects the same randomization of actions at each decision epoch. Proof. LetS = {s i ,i = 1,2,3,...} be the user state sequence in the original decision process, wheres i ∈ {1,2},∀i. DefineT ∗ as the set of time slots where sensing is performed under the optimal policyu ∗ m . We label the original process asS ∗ = {s ∗ i ,i = 1,2,3,...}, in whichs ∗ i is defined as follows: s ∗ i =        s i , if i∈T ∗ 0, if i / ∈T ∗ 66 S ∗ is a process that characterizes the sampled process based on the policyu ∗ m , by retaining the original state sequence when observations are made, and labeling “0” for the rest time slots. Note that the numbers of consecutive zeros in all estimation intervals need not necessarily be the same, due to the fact thatu ∗ m adopts randomized decision at each decision epochu ∗ m . We then define the processS ∗∗ ={s ∗∗ i ,i = 1,2,3,...} where s ∗∗ i =                s ∗∗ i = 2, if s ∗ i = 1 s ∗∗ i = 1, if s ∗ i = 2 s ∗∗ i = 0, if s ∗ i = 0 S ∗∗ is a state sequence containing the “flipped” state from S ∗ for all observed time slots, and zeros otherwise. Since p 12 = p 21 , it immediately follows that P (i) 12 = P (i) 21 ,∀i = 1,2,3.... Recall equation (4.7) where the aggregated error for an observation interval with a certain length is calculated. It can be concluded thate (a) 12 =e (a) 21 ande (a) 22 =e (a) 11 , since the following equalities hold: max k ( P (t) 1k ·P (a−t) k2 P (a) 12 ) = max k ( P (t) 2k ·P (a−t) k1 P (a) 21 ) (4.28) max k ( P (t) 1k ·P (a−t) k1 P (a) 12 ) = max k ( P (t) 2k ·P (a−t) k2 P (a) 22 ) (4.29) where k ∈ {1,2}. It follows that the relabeled process S ∗∗ produces the same expected error as S ∗ , which implies that by swapping all decisions made at state 1 with all decisions made at state 2 in the original decision process, the expected state estimation error remains optimal. Therefore, the optimal policy decision at state 1 is able to be applied to state 2, and vice versa, while maintaining the optimal error. This indicates that for symmetric two-state Markov chain, 67 no matter what state is observed at each decision epoch, there exists an optimal policy that makes the same decision. 4.4 Performance comparison on Markov models: Markov-optimal u ∗ m vs. uniform sampling ¯ u Researchers have proposed many sophisticated sampling schemes in the literature such as [86] and [20], aiming at energy efficient event detection in mobile and wireless networks. However, in the context of mobile sensing for user state detection and estimation, there does not exist an established framework that aims to achieve the best tradeoff between energy consumption and state estimation error and select the best sampling intervals correspondingly. A baseline performance evaluation of the optimal policy u ∗ m , therefore, is to compare it with a uniform policy ¯ u where the sensor simply performs periodic sampling. 4.4.1 Deriving the expected estimation error for uniform sampling We first formally define the uniform policy ¯ u and derive its corresponding expected state estima- tion errorE ¯ u [R]. Since time is discretized, a randomization of actions is required in the uniform policy such that the average idle interval length can achieve 1/ξ, i.e., the energy budget can be fully utilized. Although the potential number of randomization choices could be infinite, we re- strict the randomization to choosing from two neighboring integer values as the sampling interval size. More strictly, the uniform sampling policy ¯ u is defined as follows: after each observation, 68 the sensor waits for ˆ I time slots with probabilityμ, and ˆ I +1 time slots with probability 1−μ, such that the following constraint holds: μ· ˆ I +(1−μ)·( ˆ I +1) = 1 ξ (4.30) where ˆ I ∈N ∗ and0≤μ≤ 1. Equation (4.30) guarantees that the energy budgetξ is fully utilized by randomizing the action over two subsequent integer values. Givenξ, it is straightforward to obtainμ and ˆ I. In particular, ˆ I =⌊ 1 ξ ⌋ andμ =⌈ 1 ξ ⌉− 1 ξ would satisfy equation (4.30). We derive the expected state estimation errorE ¯ u [R] for uniform sampling policy with energy budgetξ. First, letp i denote the steady state probability of detecting a particular statei in any observation. Since the state transition follows the Markovian rule, the current state observation result depends on the previous observed state, and the number of time slots being idle: p i = X j p j ·[μ·P ( ˆ I) ji +(1−μ)·P ( ˆ I+1) ji ] (4.31) Equation (4.31) accounts for all possible cases of last observation that may lead to statei in the current observation. It is obvious that: X i p i = 1 (4.32) Therefore, the results ofp 1 ,p 2 ,..., andp N can be obtained by solving theN + 1 simultaneous equations given in (4.31) and (4.32). 69 The expected state estimation error for uniform sampling policy with energy budgetξ can be expressed as the expected aggregate error per estimation interval divided by the average interval size: E ¯ u [R] = P ij p i ·{μ·P ( ˆ I) ij ·e ( ˆ I) ij +(1−μ)·P ( ˆ I+1) ij ·e ( ˆ I+1) ij } 1/ξ (4.33) wherei,j ∈{1,2,...,N}, ande (·) ij is defined similarly as in equation (4.7). 4.4.2 Comparingu ∗ m with ¯ u We first compare and visualize the performance difference between u ∗ m and ¯ u on two-state Markov chains in terms of their corresponding expected user state estimation error. Figure 4.3 shows the policy comparison result with respect to different energy budgets, for four simple but representative state transition probability matrices (they consist of symmetric, i.i.d., and general Markov chains). To further visualize the condition such that u ∗ m outperforms ¯ u, we show the average policy gain in figure 4.4. Specifically, figure 4.4 shows the average Markov-optimal pol- icy gain (denoted byG u ∗ m \¯ u , which is the ratio of error reduction amount introduced byu ∗ m over the original error obtained by ¯ u, averaged on all the energy budget inputs), on a comprehensive two-state matrix of user state transition probabilities ranging from 0.05 to 0.95, with interval 0.05. It can be observed that u ∗ m does not outperform ¯ u on both symmetric and i.i.d. two-state Markov chains. This is illustrated by both the overlapping curves in figure 4.3 and the zero performance difference on the diagonals of the matrix in figure 4.4. In fact, Theorem 1 and 2 already indicated the fact that there exists an uniform policy which is optimal for symmetric and i.i.d. chains. In all other cases, i.e., in case of Markov chains that are neither symmetric nor i.i.d., 70 0 0.2 0.4 0.6 0.8 1 0 0.1 0.2 0.3 0.4 0.5 p 12 = 0.1 p 21 = 0.1 Energy Budget Value E[R] Comparison 0 0.2 0.4 0.6 0.8 1 0 0.05 0.1 0.15 0.2 p 12 = 0.1 p 21 = 0.5 Energy Budget Value E[R] Comparison 0 0.2 0.4 0.6 0.8 1 0 0.1 0.2 0.3 0.4 0.5 p 12 = 0.5 p 21 = 0.5 Energy Budget Value E[R] Comparison 0 0.2 0.4 0.6 0.8 1 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 p 12 = 0.5 p 21 = 0.9 Energy Budget Value E[R] Comparison Optimal Uniform Optimal Uniform Optimal Uniform Optimal Uniform Figure 4.3: Policy comparison in terms ofE[R] for different energy budgets. u ∗ m always leads to a positive gain except whenξ = 1 (in which case the sensor is sampled in every time slot and no incorrect estimation will occur). To further study the dependency of Markov-optimal policy gain on user state transition dy- namics, we examine the state entropy of given Markov chains. Recall that the entropy of a state measures the uncertainty associated with that state, and for a first-order Markov chain, the entropy of statei is defined as: H i =− X j p ij ·logp ij (4.34) We investigate{2,3,4,5}-state Markov chains and in each case, we randomly generate 100 transition probability matrices on whichu ∗ m and ¯ u are applied and bothG u ∗ m \¯ u and the standard deviation of state entropies are calculated. Figure 4.5 displays the result of the amount of policy gain and the corresponding standard deviation of state entropies, for {2,3,4,5}-state Markov 71 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 p 12 p 21 Gain Figure 4.4: Given different state transition probabilities, this figure shows the average improve- ment over different energy budgets, by applying optimal policy as compared to periodic sampling. chains, respectively. The positive correlation between policy gain and standard deviation of state entropies can be clearly identified. This is because as the deviation of state entropies increases, more frequent sampling is conducted at states with higher entropy to provide lower estimation error and at the same time, longer sampling interval can be assigned at low-entropy states where the state estimation error will not be greatly affected. Unlike the uniform policy, the Markov- optimal policy is able to implement different sampling frequencies at different states in order to achieve better performance. 4.5 Policy comparison on real user state traces The “Markovian user state transition” assumption enables us to provide a tractable state esti- mation mechanism in case of missing observations, a way originated from the forward-backward 72 0 0.2 0.4 0.6 0.8 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 STDV of state entropies Gain 2−state MC 0 0.2 0.4 0.6 0.8 0 0.1 0.2 0.3 0.4 STDV of state entropies Gain 3−state MC 0 0.2 0.4 0.6 0.8 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 STDV of state entropies Gain 4−state MC 0 0.2 0.4 0.6 0.8 1 0 0.1 0.2 0.3 0.4 STDV of state entropies Gain 5−state MC Figure 4.5: The gain of the optimal policy over uniform sampling (averaged over different energy budgets) vs. the standard deviation of state entropies, in case of{2,3,4,5}-state Markov chains.. algorithm that estimates hidden state sequence in traditional Hidden Markov Model (HMM) prob- lems. Moreover, by assuming that the state transition is Markovian, the Markov-optimal sensor sampling policy can be obtained by solving the corresponding CMDP in polynomial time. There certainly are some kinds of user state transitions that have been empirically shown previously to be close to Markovian. A classic example is speech and silence patterns within a monologue [42]. To further consolidate our study and evaluate the performance of Markov- optimal policy on real user data, we collect and study data traces that pertain to user motion change, status of inter-user contact, and status of network connectivity on a broader time scale, and answer the following questions: (a) are these user state transitions strictly Markovian? and 73 (b) Does the Markov-optimal policy perform well? The data collection experiments are explained as follows 3 : User motion: We conducted the user motion trace collection in fall 2009. We are particularly interested in the transition between “Stable” and “Moving”, two motion states that characterize the basic user activity. During the experiment, one participant carried a Nokia N95 device with a python program running in background to automatically detect his activity, using the motion classification technique introduced in [90]. Approximately 40 hours of running data distributed in five different weekdays were collected. The mobile device recorded user motion every 12 seconds, in which, 10 seconds were consumed by the accelerometer for data collection, and the classification algorithm ran at the end of each sample duration. Inter-user contact status: In fall 2006, twenty-five Tmotes [64] with wireless communica- tion capabilities were handed out to an undergraduate class at USC for a week [87]. Each student carried the device during the whole experiment and each mote automatically logged the informa- tion about other devices within communication range. Specifically, the time, duration and node IDs of each contact were stored in the memory for off-line processing. Each node transmitted a contact packet every 0.1 second and the nearby nodes receiving the contact packets recorded the contact information immediately. We view the user as in one of the following two states at any given time: “In contact” or “Not in contact”, since the existence of neighboring devices is of importance in mobile P2P networking. 3 The user motion trace and contact status data set are available at ANRG website (http://anrg.usc.edu/www/downloads), and the CoSphere network connectivity data set is available at CRAW- DAD website (http://crawdad.cs.dartmouth.edu/data.php). 74 Network connectivity: We study CoSphere traces [68], which contain network visibility (Cell, WiFi, and Bluetooth) records on mobile phones carried by 12 participants. The one- month experiment was conducted by researchers at Telematica Instituut in The Netherlands within February/March 2007 time frame. To adapt the data set to our own study, we identify the transition dynamics between “Connected” and “Not connected” in all three cases (Cell, WiFi, and Bluetooth) and evaluate the Markov-optimal sampling mechanism against uniform sampling technique. All the data have been first processed into discrete-time format. We then apply a standard method proposed in [9] to find the probabilities of the user state transition. In particular, defining n ij as the total frequency of state transition fromi toj, the Markov transition probability matrix can be constructed, with p ij =n ij /n j , (4.35) wheren j = P i n ij . We illustrate the resulting Markov chains for several sample participants 4 in figure 4.6. The Markov chains clearly show different user state transition dynamics. We first explore the existence of Markovian property in real data trace, by examining whether the time that the user spent in a state is geometrically distributed. The state duration samples from the trace data are compared to a randomly generated geometrically distributed set of data, with the state transition probabilities shown in figure 4.6 being the success probabilities. Figure 4.7 compares the cu- mulative distribution function (cdf) of state duration in real data, to the corresponding geometric distribution. It can be seen that the state duration in real case tends to show more concentration 4 The data from other participants and from different days show similar transition dynamics in the same set of experiments. 75 Figure 4.6: Markov chains showing two-state transition probabilities for: (1) Stable vs. Moving (2) Not in contact with other users vs. In contact with other users (3) Not in cellular coverage vs. In cellular coverage (4) Not in WiFi coverage vs. In WiFi coverage and (5) No Bluetooth connectivity vs. Bluetooth connectivity. on both small and some extra-large values. This implies that the empirical user data may not strictly follow the Markovian property, as the duration of each state is not strictly geometrically distributed. In fact, in terms of inter-user contact, several previous studies such as [19, 86, 87] have shown that the inter-contact and contact time duration distribution may exhibit heavy-tails in certain time scale rather than memoryless decay. Motivated by the fact that real data traces do not strictly show Markovian properties, in Chap- ter 5, we propose an analytical framework for both user state estimation and sensing policy op- timization under a more generalized model - the semi-Markov process model, which provides more realistic modeling of realistic data by allowing arbitrary state duration distributions. We compare the Markov-optimal policy derived from the transition probabilities shown in figure 4.6 and the uniform policy by applying them to the collected user data, and measure the 76 Figure 4.7: Are the traces strictly Markovian? Figure compares the cdf of state duration in real data (solid curve) to the corresponding geometric distribution (dashed curve) generated with success probability indicated in figure 4.6. expected user state estimation error by comparing the state estimations against the ground truth. Table 4.1 shows the gain on expected state estimation error, by choosing the Markov-optimal policy over uniform sampling, through both numerical calculation and policy application to real state traces. The results shown in table 4.1 indicate that even though the real user state transition is not strictly Markovian, the Markov-optimal sensor sampling policy is still able to provide significant 77 Data Numerical Gain Experimental Gain Motion 19.75% 18.00% Contact 29.55% 27.86% Cellular connectivity 35.31% 36.01% WiFi connectivity 3.93% 4.73% Bluetooth connectivity 11.16% 12.70% Table 4.1: For each data trace, the theoretical gain (obtained based on the Markovian assumption) and the experimental gain (obtained by policy application to real traces) are displayed. improvement over periodic sampling mechanism, and the results match the numerically obtained results quite closely. Figure 4.8: Top graph shows the user motion trace (a solid vertical line means “Moving” and a blank space means “Stable”) and the sampling points of both policies. Bottom graph shows the aggregated state estimation error at each sampling point for both policies. In order to better visualize the benefit of the Markov-optimal policy, we illustrate the sampling points of the Markov-optimal policy as well as uniform sampling on a selected subset of user data trace, and show the corresponding state estimation error for both cases in figure 4.8. In particular, figure 4.8 displays the motion trace (“Stable” vs. “Moving”) of a user within an approximately 1.7-hour interval. The sensor sampling points in case of both policies are marked in the top graph 78 and the aggregated state estimation error at each sampling point is plotted in the bottom graph. The energy budgetξ is set to be 0.04, which corresponds to a 5-minute long sampling interval on average. It can be clearly seen that, as opposed to periodic sampling, the Markov-optimal policy assigns different duty cycles at different states, and therefore is able to reduce the expected state estimation error by imposing more frequent samples when the user state transition is more uncertain (i.e., higher state entropy in Markov case). On the other hand, in order to balance the overall energy consumption, the sampling interval size is increased when state transition is more predictable (as illustrated by the second half of user trace in figure 4.8 when the user tends to be more “stabilized”). 4.6 Summary In this chapter, we study the problem of energy efficient sampling of a discrete, Markovian user state process. In contrast to our prior work [88] where the best stationary deterministic policy is found by exhaustively searching through all sensing intervals, in this chapter, by formulating the constrained optimization problem as an infinite-horizon CMDP with expected average cost, the optimal sensor sampling policy for Markovian user state transitions can be obtained in a com- putationally efficient manner. The resulting Markov-optimal policy achieves the lowest expected user state estimation error, while maintaining the energy consumption below a given budget. The Markov-optimal policy is compared against uniform periodic sampling numerically and we find that the performance gain is positively correlated with the standard deviation of state entropies. We also apply the Markov-optimal policy to real traces and it is shown to be capable of providing 79 significant improvement over periodic sampling, even the real user state transitions are not strictly Markovian. 80 Chapter 5 User state estimation and sensing policy optimization: the semi-Markovian case 5.1 Overview As we demonstrated in the previous chapter, strict Markovian characteristics are rarely seen in real user state traces. In this chapter, we relax the assumption such that the analysis is no longer limited to Markov processes. Instead of assuming geometrically distributed state durations, we consider discrete time semi-Markov processes, which allow arbitrary state sojourn time distri- butions and provide more general and realistic models than Markov processes 1 . We propose a state estimation mechanism for semi-Markov processes, and provide a linear programming based approximate sensing policy optimization framework, which is able to efficiently schedule sensor samplings in order to minimize state estimation error given an energy constraint. Both the semi- Markov estimation mechanism and the optimization framework can be applied to data traces with any types of state sojourn time distribution, whose probability mass function (pmf) parameters are not necessarily known a priori and can be obtained through the learning of historic data. 1 Note that a discrete time Markov process is a special case of discrete time semi-Markov process where the sojourn times are geometrically distributed. 81 5.2 Semi-Markovian state estimation for missing observations with sojourn time truncation 5.2.1 Preliminaries We assume that time is discretized into consecutive time slots and the user state evolves as a N-state discrete time semi-Markov process. Recall that semi-Markov model is similar to Markov model such that state jumps follow a Markov chain with no self-transitions, and is different such that state sojourn times could be random variables with any types of distribution. Similar to the assumption in chapter 4, sensor observations are assumed to be perfect, such that if the sensor is activated in every time slot, the state process could be obtained with 100% accuracy. However, due to energy consumption concern, the sensor needs to perform duty cycling according to certain sensing policy, which leads to a sparsely sampled state sequence with multiple intervals of miss- ing observations (see illustrative figure 5.1). In this thesis, we use the terms “estimation interval” and “observation interval” interchangeably, both denoting the interval of time slots between two consecutive sensor observations. Figure 5.1: The semi-Markov process is sampled at discontiguous time slots, leading to multiple estimation intervals. A complete definition of semi-Markov process can be found in section 2.3.2.1 of Chapter 2. Letp ij andw i (a) be the state transition probability and pmf of state sojourn duration respectively. 82 Recall that that 1 ≤ a ≤ A i < ∞, and P a w i (a) = 1, where A i is the maximum sojourn duration of state i. For practical purposes, we approximate distributions with infinite support with a sufficiently large choice ofA i . LetO t represent the observed state at timet, and we investigate the following state estimation problem: given the state distribution parametersp ij andw i (a), and two state observationsO tm = o 1 andO tn =o 2 , what is the most likely state at any given time slot betweent m andt n ? It is important to note that, unlike the assumptions made in previous works [31, 96], the observed stateso 1 ando 2 do not necessarily need to start at timet m and end at timet n . Instead, the state estimation is conducted in real-time for any given estimation interval, where sensor observations could be made at arbitrary time slots, and the state detection may belong to any instance of the sojourn time duration. We propose a forward-backward based state estimation mechanism in Section 5.2.2 and 5.2.3 that estimates the most likely state for each time slot within an estimation interval. In Sec- tion 5.2.4, we discuss an alternative approach, where the forward and backward variables are defined using the knowledge of the origin time, initial state distribution, and all observations made. This alternative method may lead to better performance but can only be run off-line, and is computationally untractable in most scenarios because it is hard to automatically keep track of when a state enters and finishes its sojourn time. 5.2.2 The forward-backward variables: definition and initialization As the state observationo 1 may truncate the state sojourn time and could belong to any instance of the state duration ofo 1 , we definet ′ m (wheret m −A o 1 <t ′ m ≤t m ) as the time at which stateo 1 is entered, i.e., the observed stateo 1 started its sojourn time at timet ′ m , and lasted through timet m . 83 In real-time system implementation where state durations are probabilistic, the exact value oft ′ m is often unavailable since the sensor makes sparse observation and may not have the knowledge of complete historic state sequence. Therefore, for a state evolving process that is divided into multiple estimation intervals,t ′ m could be obtained through the following alternatives: • Method 1 (M1): Use state estimation result from the previous estimation interval. • Method 2 (M2): Use the expected valueE p of past sojourn time, which can be calculated as follows (for statei): E p (i) = A i X a=1 a X τ=1 w i (a)·τ a . (5.1) • Method 3 (M3): Simply lett ′ m =t m . It can be seen that, M1 conducts state estimation based on previously estimated results, which may cause error propagation for certain types of state distribution, i.e., the previous estimation error may accumulate and lead to undesired performance in future estimations. M2 and M3 do not suffer this problem because they approximate the history of state sojourn durations using expected and zero past sojourn time, respectively. We will study the impact of these three t ′ m selection methods on simulated state sequences in section 5.2.5, under different probabilistic state sojourn time distributions. LetA be the maximum length of state sojourn times, i.e.: A = max i {A i },∀i∈{1,2,...,N}. (5.2) 84 For a given time slott, we define forward variablesf t (i) andf ∗ t (i) as: f t (i) = Prob[o 1 ,t ′ m , statei ends att] = A X a=1 f ∗ t−a (i)·w △ i (a), (5.3) where w △ i (a) =          w i (a), ift−a≥t m , w ′ i (a) = w i (a) P A i k=tm−t ′ m +1 w i (k) , else. (5.4) and f ∗ t (i) = Prob[o 1 ,t ′ m , statei starts att+1] = N X j=1 f t (j)·p ji . (5.5) The reason thatw △ i (a) is defined differently fromw i (a) (as given in equation (5.4)) under the condition thatt−a<t m , is that the observed state at timet m has already spent its sojourn time fort m −t ′ m +1 consecutive time slots such that the rest of the sojourn time distribution needs to be conditioned upon this fact, i.e.,w ′ i (a) = Prob[statei lasts fora slots|a≥t m −t ′ m +1]. In order to computef t (i), the boundary condition forf ∗ t (i) needs to be initialized as follows: f ∗ t (i) =        1, if t =t ′ m −1, andi =o 1 , 0, otherwise. (5.6) 85 and f ∗ tm (i) =p o 1 i ·w o 1 (t m −t ′ m +1),∀i6=o 1 . (5.7) Equation (5.6) emphasizes the fact that the detected stateo 1 has started its sojourn time at time t ′ m timet ′ m whereas equation (5.7) computes the probability that stateo 1 transits to another state in the first time slot of the estimation interval (i.e.,t m +1). To initializef t (i) at timet m , the probability that observationo 1 ends exactly att m needs to be calculated, which satisfies: f tm (i) =        w ′ o 1 (t m −t ′ m +1), ifi =o 1 , 0, ∀i6=o 1 . (5.8) We further define the backward variablesb t (i) andb ∗ t (i) for a given time slott: b t (i) = Prob[o 2 | statei starts att] = A X a=1 b ∗ t+a (i)·w i (a), (5.9) and b ∗ t (i) = Prob[o 2 | statei ends att−1] = N X j=1 b t (j)·p ij . (5.10) Since we do not assume the observed stateo 2 ends att n , the initial and boundary conditions forb ∗ t (i) are therefore given by: b ∗ tn (o 2 ) = 0, (5.11) 86 and b ∗ tn+x (i) =        1,∀x∈{1,2,...,A o 2 }, ifi =o 2 , 0, otherwise. (5.12) Equation (5.12) describes that given the fact that stateo 2 ends at any time slot after timet n , the probability of seeingo 2 at timet n is 1, and correspondingly, the probability of seeing other states att n is 0. The initial conditions forb t (i) need to be specified as: b tn (i) =        P Ao 2 x=1 b ∗ tn+x (o 2 )·w o 2 (x), ifi =o 2 , 0, ∀i6=o 2 . (5.13) in whichb tn (o 2 ) is the average aggregated probability that stateo 2 begins at timet n . 5.2.3 State estimation Once the values of all the forward and backward variables have been obtained by recursion, it is feasible to estimate the state for each time slot betweent m andt n . We define a new forward variable: φ t (i) = Prob[o 1 ,t ′ m ,s t =i]. (5.14) 87 φ t (i) holds the following property: φ t (i) = Prob[o 1 ,t ′ m ,s t−1 =i ands t =i] + Prob[o 1 ,t ′ m ,s t−1 6=i ands t =i] = Prob[o 1 ,t ′ m ,s t−1 =i] − Prob[o 1 ,t ′ m ,s t−1 =i ands t 6=i] + Prob[o 1 ,t ′ m ,s t−1 6=i ands t =i] = φ t−1 (i)−f t−1 (i)+f ∗ t−1 (i). (5.15) It can be seen thatφ t (i) can be calculated by recursion as well. To initialize, we have the following equalities: φ tm (i) =        1, ifi =o 1 , 0, else. (5.16) Next, we define a new backward variable: ψ t (i) = Prob[o 1 ,t ′ m ,o 2 ,s t =i], (5.17) which is the joint probability of o 1 , t ′ m , o 2 , as well as the unknown state at time t. Following the similar relationship illustrated in equation(5.15)-(5.15), ψ t (i) can be further written in the following recursive form: ψ t (i) = ψ t+1 (i)−b t+1 (i)·f ∗ t (i)+f t (i)·b ∗ t+1 (i). (5.18) 88 The initial condition for this backward variableψ t (i) is given by: ψ tn (i) = Prob[o 1 ,t ′ m ,o 2 ,s tn =o 2 ] = Prob[o 1 ,t ′ m ,s tn =o 2 ] = φ tn (i), (5.19) whose value has been stored at the end of the recursion when calculatingφ t (i). Finally, the maximum a posteriori (MAP) estimate of user state at timet is given by: s ′ t = argmax i∈{1,2,...,N} Prob[s t =i|o 1 ,t ′ m ,o 2 ] = argmax i∈{1,2,...,N} Prob[s t =i,o 1 ,t ′ m ,o 2 ] Prob[o 1 ,t ′ m ,o 2 ] = argmax i∈{1,2,...,N} ψ t (i),∀t =t n −1,...,t m +1. (5.20) The following algorithm summarizes the complete procedure of estimating user state with missing observations between two neighboring state detections for a discrete-time, semi-Markov process: It can be seen that in Algorithm 2, solving forf t (i) andb t (i) at any given timet consumes at mostA multiplications, whereas calculatingf ∗ t (i) andb ∗ t (i) requiresN multiplications. There- fore, for an estimation interval with sizeT , the computational complexity of the semi-Markovian estimation algorithm isO(TN(N +A)). In comparison, the Markovian state estimation mech- anism proposed in Chapter 4 needs T matrix multiplications where the matrix size is N ×N. This leads toO(TN 3 ) computational complexity without any speed-up in matrix multiplications. 89 Algorithm 2 An algorithm that estimates missing state information between two observations O tm =o 1 andO tn =o 2 , for a discrete-time, semi-Markov process. 1: Input: p ij , w a (i), A i , and t ′ m (whose value depends on whether M1, M2, or M3 is used), ∀i,j ∈{1,2,...N} 2: Output: State estimation resultS ′ ={s ′ tm+1 ,s ′ tm+2 ,...,s ′ tn−1 } and the corresponding expected error sequencee s ={e tm+1 ,...,e tn−1 } 3: Initialize forward variable f t (i) and f ∗ t (i) using equation (5.6), (5.7), and (5.8). Solve by recursion using equation (5.3) and (5.5). 4: Initialize backward variableb t (i) andb ∗ t (i) using equation (5.11), (5.12), and (5.13). Solve by recursion using equation (5.9) and (5.10). 5: Calculateφ t (i),∀i∈{1,...,N}, and∀t∈ [t m ,t n ], based on equation (5.15) and (5.16). 6: Calculateψ t (i),∀i∈{1,...,N}, and∀t∈ [t m ,t n ], based on equation (5.18) and (5.19). 7: State estimation: s ′ t = argmax i∈{1,2,...,N} ψ t (i),∀t =t n −1,...,t m +1. 8: Expected estimation error: e t =κ· 1− max i∈{1,2,...,N} ψ t (i) ,∀t =t n −1,...,t m +1. whereκ is a normalization factor. Thus, although the complexity of Markovian estimation is not affected by the state sojourn dura- tionA, as the number of states increases, our semi-Markovian approach could potentially become more computationally efficient. 5.2.4 The forward-backward variables: an alternative An alternative way to define the forward and backward variable is to consider not only the neigh- boring observations but the entire observed sequence, assuming the knowledge of (a) the initial state distributionμ, (b) the origin timet 0 that the initial state begins, and (c) the ending timet L where the last state finishes its sojourn time. 90 LetO 0→t denote the observation sequence up to the current timet, whereasO t→L denote the rest of the observation sequence after timet. Then the forward and backward variables could be defined in the following way: f t (i) = Prob[μ,t 0 ,O 0→t , statei ends att] (5.21) f ∗ t (i) = Prob[μ,t 0 ,O 0→t , statei begins att+1], (5.22) and b t (i) = Prob[t L ,O t→L | statei begins att] (5.23) b ∗ t (i) = Prob[t L ,O t→L | statei ends att−1]. (5.24) The forward-backward variables can be initialized according to the specific inputs and state estimation can be conducted by similar recursions introduced in section 5.2.3. Since the above definitions use the exactly information of where the initial state started, it may provide better estimation accuracy as compared to Algorithm 2, which requires an approximation of the past state sojourn time. However, this mechanism only allows off-line estimation since it requires future observation results when applied. Moreover, as the length of semi-Markov process grows, it becomes harder to keep track of all observations and the computationally complexity increases correspondingly. Therefore, in this thesis, we do not provide further analysis of this approach, and we leave the algorithmization and possible optimization of this mechanism as well as its performance evaluation to future work. 91 5.2.5 State estimation performance on simulated two-state transition processes In this subsection, we evaluate the performance of semi-Markovian state estimation mechanism (Algorithm 2) on two-state, simulated state transition processes and explain the observations. In particular, each state sequence is generated based on pre-specified state sojourn time distribu- tions, and is then observed at a constant frequency, leading to consecutive estimation intervals with equal lengths. Within each estimation interval, Algorithm 2 is conducted for state estima- tion. Once state estimation is complete for all the estimation intervals, the overall estimated state sequence is compared to the original sequence and the state estimation error ratio R (which is the total number of incorrect estimations for the entire sequence divided by the length of the state sequence) is calculated. Recall that one of our goals is to design a real-time state estimation mechanism for sparsely sampled real user state processes. In fact, in many human related traces, state distributions have been found to exhibit heavy tails, e.g., Internet file sizes [24], human contact arrival processes [19,86], user motion transitions [89], and so on. These types of traces are often modeled as mem- oryless distributions such as geometric distribution in order to reduce the complexity of study. In this section, we compare the performance of semi-Markovian state estimation to state estimation under Markovian assumption, where, no matter what distribution function does the state duration follow, the state sequence is always assumed to be a Markov process. The Markovian assumption, although not strictly valid in reality, has been widely used to model user related state traces such as speech [42, 70], mobility [38, 75], and activity [34, 73]. 92 Under the stationary Markovian assumption, the state transition probabilities can be estimated in a standard way [9], which is to calculate the ratio of state transition frequencies: p ij =n ij /n j , (5.25) wheren ij is the frequency of state transition fromi toj, andn j = P i n ij . Estimating state information for missing observations under the Markovian assumption has been discussed in [89]. Here we give a brief overview of the mechanism: given that state i is detected at timet m , and statej is detected at timet n (t m ,t n ∈T andt m <t n ), the most likely state (denoted bys ′ t ) at timet is selected as s ′ t = argmax k ( P (t−tm) ik ·P (tn−t) kj P (tn−tm) ij ) . (5.26) i.e., the MAP estimation for Markovian process only requires one equation in matrix multiplica- tion form since Markovian estimation does not suffer the “past sojourn time selection” problem because it implicitly assumes that the underlying state process is memoryless. We first conduct two sanity checks of the correctness of the semi-Markovian estimation by examining its performance on deterministic and geometric distributed state processes. We then select one simple and commonly seen distribution type, namely, binomial distribution, and gen- erate state process and examine the performance of semi-Markovian estimation according to the distribution parameters. For the commonly seen heavy-tail distributions in real user state traces, we choose to investigate Zipf’s Law distribution, which is a discrete, power law, and heavy-tailed distribution type. The pmf parameters are summarized in table 5.1. 93 Deterministic w 1 (a) = 1, ifa = 20, 0, else. w 2 (a) = 1, ifa = 40, 0, else. Geometric w 1 (a) = (1−p 1 ) a−1 ·p 1 , wherep 1 = 0.2 w 2 (a) = (1−p 2 ) a−1 ·p 2 , wherep 2 = 0.6 Binomial w 1 (a) = N 1 a p a 1 (1−p 1 ) N 1 −a ,p 1 = 0.8 andN 1 = 100 w 2 (a) = N 2 a p a 2 (1−p 2 ) N 2 −a ,p 2 = 0.4 andN 2 = 100 Zipf’s Law (smaller tail) w 1 (a) = 1/a s 1 P N 1 n=1 1/n s 1 , wheres 1 = 0.1 andN 1 = 100 w 1 (a) = 1/a s 2 P N 2 n=1 1/n s 2 , wheres 2 = 0.05 andN 2 = 100 Zipf’s Law (larger tail) w 1 (a) = 1/a s 1 P N 1 n=1 1/n s 1 , wheres 1 = 0.01 andN 1 = 100 w 1 (a) = 1/a s 2 P N 2 n=1 1/n s 2 , wheres 2 = 0.005 andN 2 = 100 Table 5.1: Four different distribution types and their specific parameters utilized in the simulation. For each state distribution instance, state estimations are conducted on 10 simulated state processes each containing 1000 state transitions. Specifically, for all non-deterministic state pro- cesses, semi-Markovian estimation mechanism (Algorithm 2) with all variations M1, M2, and M3, as well as Markovian estimation mechanism are conducted and their corresponding error ratioR are compared under different estimation window sizes ranging from 20 to 200 (we also show the result for very large window sizes in case of binomial state distribution which converges slowly). The results and observations are discussed as follows: A. Sanity checks: deterministic and geometric distributions The evaluation of semi-Markovian estimation on deterministic state processes serves as the first sanity check of the correctness of Algorithm 2. As can be seen from figure 5.2 (left por- tion) that, the semi-Markovian estimation mechanism with M1 is able to reconstruct the entire 94 sequence without any error, whereas applying Markovian estimation leads to undesired perfor- mance. Intuitively, when state sojourn durations are fixed, knowing how long a state has already spent its sojourn time should lead to perfect state estimations in each estimation interval. Figure 5.2: Semi-Markovian estimation vs. Markovian estimation on simulated two-state process where state distributions are deterministic (left) and geometric (right). The right portion of figure 5.2 shows that semi-Markovian and Markovian estimation mecha- nisms lead to very close results when state sojourn times are geometrically distributed 2 . Because Markov process is a special case of semi-Markov process, both estimation mechanisms provide very similar results on a state sequence with geometrically distributed state durations. In fact, different versions of semi-Markovian estimation (M1, M2, and M3) provide almost identical re- sults due to the memoryless property of geometric distribution, and this observation serves as the second sanity check of the correctness of semi-Markovian estimation mechanism. B. Binomial It can be seen from figure 5.3 that semi-Markovian estimation algorithm with M1 outper- forms Markovian estimation and provides the best performance, whereas M3 leads to the worst 2 As noted in section 5.2.1, although geometric distribution supports infinite set ofa values, we limit it such that 1≤a≤100 in order to satisfy the running condition of Algorithm 2. 95 performance. Note the similarity between binomial distribution and deterministic distribution, where both pmfs contain peak probability and decay quickly around the concentration. As semi- Markovian estimation mechanism provides state estimations strictly based upon state distribution pmf, the “peak-like” feature greatly reduces the estimation error; therefore, knowing history state sequence information (M1) will always lead to better state estimation results. In addition, it can be found from the right portion of figure 5.3 that the semi-Markovian estimation mechanism provides better performance even under extremely large estimation window sizes, and converges eventually. Figure 5.3: Semi-Markovian estimation vs. Markovian estimation on simulated two-state process where state distributions are binomial. Right figure shows convergence at very large window sizes. C. Zipf’s Law In order to provide a comprehensive performance evaluation of the semi-Markovian estima- tion, we study Zipf’s Law distribution with different tail sizes by varying the distribution param- eters s 1 and s 2 as illustrated in table 5.1. It can be seen from figure 5.4 that semi-Markovian estimation leads to higher performance improvement under larger tail sizes. This is due to the 96 fact that assuming the state process to be Markovian becomes less approximate as the tail size in- creases, therefore, semi-Markovian estimation produces higher gain over Markovian estimation as the tail size of state distribution increases, since it estimates state information based on the exact state distribution functions. Figure 5.4: Semi-Markovian estimation vs. Markovian estimation on simulated two-state process where state sojourn durations follow Zipf’s Law distributions, with different tail sizes according to the definitions in table 5.1. To better understand the results, we summarize in table 5.2 the results of percentage gain over Markovian estimation provided by semi-Markovian estimations corresponding to figure 5.4. The average as well as the maximum gain of all estimation window sizes are shown for methods M1, M2, and M3. While M1 suffers the error propagation problem and M3 simply assumes zero past sojourn time, it can be seen that semi-Markovian estimation with M2 leads to the best average performance among all versions of Algorithm 2. We also observe the fact that for probabilistic state distributions, all estimation mechanisms converge as the estimation window size becomes large enough. This is because as the state pro- cess is sampled extremely sparsely, no matter what estimation mechanism is conducted, it essen- tially selects a state with higher occurrence probability, i.e., steady state probability. Therefore, 97 Smaller tail Larger tail (s 1 = 0.1s 2 = 0.05) (s 1 = 0.01s 2 = 0.005) Avg. Gain Max Gain Avg. Gain Max Gain M1 8.24% 18.21% 11.09% 23.79% M2 9.08% 23.15% 11.37% 19.79% M3 5.87% 17.26% 7.86% 17.61% Table 5.2: Average and maximum percentage gain of semi-Markovian estimation with M1, M2, and M3 over Markovian estimation under different estimation window sizes ranging from 1 to 500. the expected estimation errorR as estimation window size goes to infinity satisfies the following relationship for an N-state semi-Markov process: R T→∞ = 1−max i ( M i P i M i ) , (5.27) whereM i is the mean sojourn time of statei, andi∈{1,...,N}. 5.2.6 State estimation performance on real traces To further strengthen our study, the performance of semi-Markovian estimation mechanism is evaluated on real state traces. We consider two types of data: (a) smartphone network connectivity state and (b) temperature status, in order to examine the estimation algorithm performance on both heavy-tailed state durations and arbitrary state durations that cannot be easily fitted by a certain parametric distribution type. 5.2.6.1 Heavy-tailed state durations: device network connectivity We study traces collected in the CoSphere project [68] 3 conducted by researchers at Telematica Instituut in The Netherlands within February/March 2007 time frame. During the one-month 3 The data is available at CRAWDAD website (http://crawdad.cs.dartmouth.edu/data.php). 98 experiment, twelve participants each carried a smartphone that automatically logged its exposure to cell tower, WiFi access point, and Bluetooth devices. In our study, we pick WiFi and Bluetooth data traces and view the user state as either “Connected” or “Not connected” to infrastructure access points or peer devices, and compare the performance of semi-Markovian estimation and Markovian estimation in terms of the average state estimation error produced. Recall that semi-Markovian state estimation mechanism requires the state distribution param- etersp ij andw a (i) to be estimated a priori based on the collected user state traces. In particular, assuming a stationary process, a standard model parameter estimation mechanism is given by [95] and is summarized as follows: (1) State transition probability p ij is estimated as the ratio between number of transitions from statei toj and the total number of transitions into statej: p ij =n ij /n j , (5.28) where n ij is the frequency of state transition from i to j, and n j = P i n ij . This is similar to equation (5.25) where the Markovian transition probabilities are calculated, except that state transitions only happen whenever a different state is entered. Note that for two-state semi-Markov chain,p 12 =p 21 = 1. (2) State sojourn time distributionw a (i) is estimated as the number of times that statei lasts exactly fora time slots, divided by the number of times that statei is visited: w a (i) =n a (i)/n i ,∀a∈{1,2,...,A}. (5.29) 99 Each user state trace is sampled periodically according to the input estimation window size, and within each observation interval, both the semi-Markovian estimation mechanism and the Markovian estimation mechanisms are executed. We compare the average state estimation error from both estimation mechanisms while varying the observation window size from 20 to 200, and demonstrate both the state duration pmfs and the error comparison result in figure 5.5 and 5.6. Figure 5.5: State duration pmfs for Bluetooth connectivity state trace and the comparison of both estimation algorithms. Figure 5.6: State duration pmfs for WiFi state trace and the comparison of both estimation algo- rithms. It can be seen that semi-Markovian estimation outperforms Markovian estimation on both Bluetooth and WiFi connectivity traces. In particular, the semi-Markovian estimation is able 100 to reduce more than 40% estimation error than Markovian estimation under certain estimation window sizes. This justifies our main statement that, the state sojourn time in real user state traces tend to exhibit heavier tails than memoryless distributions, therefore, modeling the data as Markovian process and using Markovian estimation may lead to undesired state estimation performance; whereas the semi-Markov process is a more general model for real user state traces and the semi-Markovian estimation (Algorithm 2) could provide better performance by estimating state information using the exact state distribution function. 5.2.6.2 Arbitrary state durations: temperature The reason we employ temperature data in our study is to evaluate the performance of semi- Markovian estimation on data traces that demonstrate arbitrary types of state duration distribu- tions (as can be seen in figure 5.7). In particular, the outdoor temperature data of Los Angeles (sampled once an hour) in all Januaries within the past six years 4 has been obtained and divided into “High” and “Low” states according to a pre-selected threshold. The six pieces of data are aggregated and treated as an entire state sequence on which the estimation algorithms are applied. The state estimation result is shown in figure 5.7, from which it can be seen that although the state duration distributions tend to be “bimodal” and may not be easily characterized by a single parametric distribution type, the semi-Markovian estimation algorithm provides lower error by learning the model parameters using equation (5.28) and (5.29) and estimating missing state information using these parameters according to Algorithm 2. 4 The data is available at Weather Underground (http://www.wunderground.com/). 101 Figure 5.7: State duration pmfs for temperature connectivity state trace and the comparison of both estimation algorithms. 5.3 Optimal sensor scheduling with energy constraint In practical system operations, a device energy consumption budget is often specified in order to ensure a long enough device lifetime. Similar to Chapter 4, we defineξ (where0≤ξ ≤ 1) as the budget which is the maximum expected energy consumption allowed, and letu denote a sensor sampling policy, which specifies how sensor duty cycles should be assigned under different user states detections, i.e., how long the sensor should stay idle after each observation. We investigate a similar constrained optimization problem that was first studied in Chapter 4 of this thesis: given the sensor energy consumption budget ξ and state distribution parameters, how should sensor duty cycles be scheduled such that the expected state estimation error can be minimized while the expected energy consumption is maintained below the budget? Feinberg [28] demonstrated the use of LP to solve CSMDP with average rewards. In fact, LP is also the standard tool to solve the above type of constrained optimization problems under Markovian assumption [3] (which can be formulated as a CMDP), and its application on energy efficient mobile sensing can be found in [89]. In this paper, we utilize the LP framework provided 102 in [28, 30] with specific definition of costs (i.e., energy and error) to solve the above constrained optimization problem in order to obtainu ∗ s , the semi-Markov optimal sensing policy, and evaluate its performance. 5.3.1 Obtaining the semi-Markov optimal policyu ∗ s Minimize X y∈X X b∈B ρ(y,b)c s (y,b) (5.30) subject to: X y∈X X b∈B ρ(y,b)(δ x (y)−P s ybx ) = 0,∀x∈X, (5.31) X y∈X X b∈B ρ(y,b) = 1, (5.32) ρ(y,b)≥ 0,∀y,a, and (5.33) X y∈X X b∈B ρ(y,b)(1−b·ξ)≤ 0, (5.34) whereX andB are the set of states and the set of action inputs, respectively. Note that the outcome of the above constrained optimization framework is a stationary ran- domized policy, i.e., the optimal choice of sampling interval is randomized over several integers and the decision does not vary over time. Unlike the original LP framework [28] which yields a policy that does not depend on past sojourn times, the above LP formulation takes into account the previous sojourn time by defin- ing the system state transition probabilityP s ybx and the costc s (y,b) based on the expected past sojourn duration (defined in M2). In particular, given the estimation interval sizeb, starting state 103 y, and ending state x, the interval transition probability P s ybx could be calculated at step 5 of Algorithm 2, i.e.: P s ybx =φ tn (x), (5.35) under appropriate input conditions: o 1 =y,o 2 =x,t n −t m =b, (5.36) and t m −t ′ m = A X a=1 a X τ=1 w y (a)·τ a −1, (5.37) as it is assumed that a detected state y has already spent E p (y) time slots, the expected past sojourn time. Similarly, after step 8 of Algorithm 2, the aggregated estimation error within that estimation interval can be expressed as: e sum = tn X t=tm e t . (5.38) Definee ybx sum as the aggregated estimation error under condition (5.36), therefore, the intermediate costc s (y,b) could be expressed by the following: c s (y,b) = X x P s ybx ·e ybx sum , (5.39) which is the expected, aggregated semi-Markovian state estimation error while statey is detected and sensing interval sizeb is chosen. 104 Once the linear programming is solved for optimal occupation measuresρ ∗ (y,b), the semi- Markov optimal policyu ∗ s can be obtained such that the probability of taking actionb at statey is equal to ρ ∗ (y,b) ρ ∗ y , whereρ ∗ y = P b∈B ρ ∗ (y,b),∀y ∈X. 5.3.2 Performance evaluation ofu ∗ s on simulated processes We first evaluate the performance ofu ∗ s on simulated state processes with binomial and Zipf’s law (with larger tail) state distributions whose parameters are specified in table 5.1. In particular, for each simulated state sequence, the Markov-optimal sensing policyu ∗ m as well asu ∗ s are computed. Both policies are then applied to the state sequence and state estimations are carried out using Markovian and semi-Markovian state estimation respectively. Figure 5.8: Performance ofu ∗ m vs.u ∗ s , on binomially and Zipf’s Law distributed state processes. Figure shows the estimation error w.r.t. energy budget (in log scale). Figure 5.8 indicates that, in case of processes whose state distributions are binomial,u ∗ s pro- vides significant state estimation error reduction as compared tou ∗ m , especially under low energy budget values. When state duration distributions follow Zipf’s law, u ∗ s produces lower estima- tion error thanu ∗ m under small energy budgets, and demonstrates similar performance as energy budget increases. This can be explained by the fact that semi-Markovian estimation mechanism 105 provides significant performance improvement only on a subset of estimation window sizes (as illustrated by figure 5.4), which leads to the corresponding gain byu ∗ s when energy budgets are stringent. In fact, for a given state process, if the semi-Markovian estimation mechanism outperforms Markovian estimation by producing lower estimation error at certain estimation window sizes, u ∗ s should provide better performance thanu ∗ m on the same data set under corresponding energy budgets, since the LP effectively searches for a combination of sensing intervals that lead to the minimum expected estimation error while not exceeding the energy budget. 5.3.3 Performance ofu ∗ s on heavy-tailed real user state traces The performance ofu ∗ s is evaluated on the same set of real user state traces introduced in 5.2.6. Asu ∗ m is optimal for Markov processes, we aim to explore whetheru ∗ s could outperformu ∗ m by achieving a better trade-off between energy consumption and state estimation error, on user state traces that are not strictly Markovian. In particular, the following steps are executed for each user state trace: • Estimate model parameters using (5.28) and (5.29). • Computeu ∗ s using LP (4.10) - (4.14). • Computeu ∗ m using LP1 proposed in [89]. • Apply both policies on real state process and conduct semi-Markovian estimation and Markovian estimation on missing observations, respectively. • Obtain the overall estimation error as well as the energy usage, which is the ratio of the total number of samples over the length of state process. 106 Since the optimization framework is stochastic and is constrained on the expected energy usage, the true energy consumption when executing the policy on a particular trace may not exactly reflect the given energy budget. We thus show the results of estimation error vs. the actual energy usage in figure 5.9 instead of the energy budget input parameter. To better visualize the performance gain, the linear fit of the energy-error pairs under different policies are shown in figure 5.9. It can be seen thatu ∗ s yields a better trade-off between energy consumption and state estimation error as compared tou ∗ m , by producing 12.04% and 14.19% less estimation error on average in each respective case. Figure 5.9: Performance ofu ∗ m vs.u ∗ s , and the genie-aided estimation error on CoSphere network connectivity traces. Figure shows the estimation error w.r.t. energy budget (in log scale). For reference purpose, we also consider an idealized genie-aided mechanism that is able to find the “truly optimal” sampling policy based on complete past and future knowledge of the state sequence. In particular, if the entire state sequence and its length is known a priori, the optimal 107 arrangement of a given number of sample points according to the energy budget can be selected using the following dynamic programming form: f(S,l,s,N,P) = min n f(S,l,s+1,N,P), (5.40) f(S,s,s+1,N −1,P)+e(S,l,s,P) o where f(S,l,s,N,P) corresponds to the minimum state estimation error for sequence S, last observed at timel, current decision epochs, model parametersP, and a total number ofN available sample points, and e(S,l,s,P) measures the semi-Markovian estimation error of the subsequence starting at timel and ending at times. The recursive equation (5.40) can be solved by specifying the following boundary conditions: f(S,l,s,0,P) =e(S,s−1,L,P), (5.41) whereL is the last time slot index of the sequence, and f(S,l,s,N,P) =e(S,l,s,P), ifs =L−N. (5.42) Although the genie-aided mechanism is able to find the lowest estimation error for a given state sequence, it does not provide a general sensing policy guideline because it is trace-specific and requires both future knowledge and complete state record. Nonetheless, it enables us to visu- alize the performance gap between the stochastic optimal policies and the trace-specific “ideal” result. 108 Note that the comparison under large energy budgets is not shown in figure 5.9 becauseu ∗ s and u ∗ m are found to provide comparably low estimation errors when energy budget is not stringent. In fact, this can be easily concluded from figure 5.4 and 5.8 which suggest that the estimation errors are similar under small estimation window sizes. However, real mobile sensing systems often operate under small energy budgets and deal with sparse observations, in which case the semi- Markovian estimation as well as the semi-Markov optimal policy lead to the best improvement and are therefore desirable implementation choices. 5.4 A novel client-server based system framework for energy efficient mobile sensing In this section, we present the design, prototype implementation, and evaluation of a novel client- server based system architecture that conducts user behavior learning, optimal policy calculation, energy efficient sensing, and state estimation in an automated manner. 5.4.1 System design and implementation The high level system design architecture is illustrated in figure 5.10. First, mobile client ex- ecutes the Behavior Learning Module, which collects continuous user state process under full sensor activation and calculates the user state distribution parameters from recorded data trace. This process allows the mobile sensing client to be “user-specific”, i.e., different optimal sensing policies can be derived later on for users with different habits. Once enough data is collected and the dynamics of user state transitions are obtained, the state distribution parameters will be sent 109 to the Optimal Policy Calculation Module on the back-end server in order to compute the user- specificu ∗ s . The optimal sensor duty cycles are transferred back to mobile client to begin energy efficient sensing, which is realized via the Energy Efficient Sensing Module. Eventually, when missing state information needs to be estimated, the intermittently sampled state trace will be sent to the State Estimation Module on the server again for state estimations, and the estimation result could then be transferred back to the client if necessary. It can be seen from figure 5.10 that by sacrificing certain amount of networking cost, inten- sive and occasional computations are shifted to servers with much more powerful computational capabilities for faster and potentially more energy efficient computing. Note that an alternative system design approach, which is to perform all sensing and computations locally on the mobile device, although eliminates the need of communication, suffers several problems such as the lack of mathematical libraries and low computational capabilities on mobile devices. In this paper, we do not further investigate the trade-off between these two design methodologies and leave it to future study. Figure 5.10: System modules and operation flow. 110 We implement a prototype of the system architecture shown in figure 5.10 as an activity recognition system on Nokia N95 devices (mobile client) and desktop PCs (back-end server) using Symbian C++ and Matlab, which communicate to each other via socket connections. The mobile sensing client is designed to differentiate two basic human activity states (“Sta- ble” and “Moving”) based on accelerometer sensing. In particular, the acceleration values on x, y, and z axis of the accelerometer is continuously read for a certain duration, and the standard deviation of the magnitudes of all three axes readings is calculated. The user activity is classified into “Stable” or “Moving” based on whether the standard deviation is below or above some given threshold. Recall that because we assume perfect sensor detections, our goal is not to develop sophisticated activity classification algorithms, but to explore how to derive and implement the semi-Markov optimal policy on the mobile client and whether it is able to provide satisfying performance in real system implementations. In our current system implementation, the user behavior learning phase is conducted for seven consecutive days, during which, the user activity trace is fully detected and logged in the mobile device file system. The symbol “0” and “1” are used to represent different user states so that the state distribution parameters can be easily calculated using equation 5.28 and 5.29. Dur- ing the learning phase, the mobile device is frequently charged to maintain continuous sensing requirement. On the server side, the policy calculation, state estimations, as well as socket connection setup are all accomplished in Matlab which provides standard LP solver. Upon receiving the user state distribution parameters, the server calculates the semi-Markov optimal sensing policy u ∗ s and transfers it back to the client in terms of the average idle interval size under different state detections, such that at each decision epoch (time is virtually divided into discrete slots with 10 111 seconds length each), the mobile sensing client will determine how long to turn off the sensor until the next observation according to the policy. 5.4.2 System performance evaluation As one novelty of this paper is to first implement the stochastic optimal sensing policies in real mobile sensing systems and compare them against the often-used uniform sampling, we focus on providing benchmark performance measurement of the system on the smartphone in terms of the battery lifetime as well as the state estimation error while implementing these policies. Thus, the investigation of the system energy efficiency while carrying out communications and the evaluation of how well the user behavior could be learned through the “full sensor activation” mechanism are currently beyond the scope of this paper, and we leave the potential optimization in the trade-off between communication and computation cost and the investigation of advanced learning techniques such as on-line learning and decision making to future study. Five independent experiments have been conducted and their setups are introduced below. During each experiment, the empirical devices are carried by the participant as his personal cell phone but with no usage other than activity sensing. Exp-1 (ξ = 1): the accelerometer is continuously sampled and user activity is recognized and recorded at each time slot. This experiment sets the benchmark of the device battery lifetime while sensor is fully activated. Exp-2 (ξ = 0): this experiment provides a bottom-line comparison and measures how long the same empirical device would last without any accelerometer sensing. Exp-3,4,5 (ξ = 0.05): the accelerometer on the main empirical smartphone is operated ac- cording to the sensing intervals specified by u ∗ s , u ∗ m , and uniform sampling, respectively. A 112 second smartphone is used throughout these experiments which samples its accelerometer in ev- ery time slot to provide ground truth user state record. This second device is frequently charged to ensure long operating duration. The user behavior learning and optimal policy calculation phases are not considered part of the empirical duration, such that the device battery lifetime is measured from the time the smartphone receives the optimal duty cycle instructions (and the battery is still fully charged) until the battery is completely drained while conducting energy efficient mobile sensing. The partially sampled user state sequences are reconstructed off-line, and are compared to the ground truth recorded by the second N95 device in order to obtain the estimation error. The empirical results are shown in table 5.3. ξ Policy Battery Lifetime Error exp-1 1 Full 52 hours 0 exp-2 0 None 236 hours ∞ exp-3 0.05 u ∗ s 160 hours 1.66% exp-4 0.05 u ∗ m 158 hours 2.30% exp-5 0.05 Uniform 150 hours 3.23% Table 5.3: Smartphone battery lifetime and state estimation error comparison among all experi- ments. First, it is observed that by implementing energy efficient sensing policies with an appropriate energy budget, the smartphone lifetime could be extended significantly with tolerable estimation errors as compared to continuous sensing. The relatively low estimation error is due to the fact that the device is operated continuously to include night hours (i.e., significant idle durations), which is usually formed by a long sequence of consecutive “Stable” states and thus reduces the overall estimation error. Despite this fact, it can be seen from table 5.3 that implementing u ∗ s achieves the longest device battery lifetime among all three energy-saving policies and provides 27.8% and 48.6% gain on estimation accuracy overu ∗ m and uniform sampling respectively. This 113 is not only in line with our results in previous sections, but also validates that the semi-Markov optimal sensing policy is a desirable implementation choice in real systems. 5.5 Summary Modeling real user state traces using Markovian model is often unrealistic and could leads to undesired state estimation performance, as real user state distributions tend to exhibit more un- structured distribution types. In this chapter, we use semi-Markov process to model user state traces, and propose a semi- Markovian state estimation mechanism that can be executed in real-time in order to estimate the most likely user state while sensor observations are missing. A linear programming based con- strained optimization is designed in order to find a tractable approximation to the best sensing policy for semi-Markov processes, which minimizes the expected state estimation error while maintaining an energy consumption budget. Both the state estimation and the approximate op- timal policy have been evaluated on simulated as well as real user state traces and their perfor- mances have been shown to be better than Markovian estimation and Markov-optimal policy. We also design and implement a client-server based, user-specific energy efficient mobile sensing system, and demonstrate through prototype implementation that semi-Markov optimal sensing achieves significantly better performance as compared to Markov-optimal sensing and uniform sensing. 114 Chapter 6 Conclusions and future work Mobile devices such as smart phones are continually integrating more sophisticated sensing and networking capabilities. This makes mobile devices not only communication tools, but also pow- erful sensing and networking entities that can monitor and share user’s ambient context such as activity, location, and proximity information, both unobtrusively and in real time. The ac- curate detection of user context could provide information support to various kinds of mobile applications and services such as location based advertising, traffic information reporting, social networks, and health monitoring. However, due to the significant amount of sensor energy con- sumptions and the limited mobile battery capacity, there is a critical need for energy efficiency in most mobile sensing based applications. In this thesis dissertation, we first developed and evaluated a novel design framework for an Energy Efficient Mobile Sensing System (EEMSS). We utilized the fact that not all the sensors on mobile smart phone need to be activated concurrently to recognize user state, and investigated how the sensors can be automatically turned on/off based upon real-time user status. By specify- ing sensor management scheme in a machine-understandable language (XML), hierarchical sen- sor management is achieved where only a minimum set of sensors is turned on at any given time, 115 and power hungry sensors are triggered by power efficient ones whenever necessary. We also designed user motion and background sound classification algorithms that are light-weight, real- time, but provide high classification accuracies. A prototype of EEMSS has been implemented on an off-the-shelf smart phone, and the evaluation with 10 users over one week demonstrates that the design is able to increase the device battery life by more than 75% as compared to the state-of-the-art system, while maintaining high accuracy in recognizing user states. We then further investigated how sensors should be duty cycled intelligently in order to achieve the best trade-off between two important performance metrics: energy consumption and user state estimation accuracy, under the assumption that the background user state is a Markov process. We designed user state estimation algorithm and proposed computationally efficient constrained optimization framework based on CMDP in order to solve for the Markov-optimal sensing policy, which minimizes the expected user state estimation error while satisfying a given energy budget, for Markovian user state processes. The Markov-optimal sensing policy is evalu- ated on both simulated state processes and real user state traces, and it has been shown to outper- form uniform sampling (which is a popular energy-saving mechanism) significantly. Motivated by the fact that realistic data are seldom strictly Markovian, we developed a more general analytical framework for state estimation as well as sensing policy optimization, for the more generalized semi-Markov processes. Semi-Markov process model allows arbitrary state duration distributions and therefore is more realistic as compared to Markov process model. We demonstrated that the semi-Markovian state estimation algorithm is able to provide better per- formance than Markovian estimation on different types of state duration distribution, as well as on real data pertaining to mobile device’s network connectivity status. We then utilized linear programming to obtain a tractable approximation to the semi-Markov optimal sensing policy, 116 which has also been shown to outperform the Markov-optimal sensing policy on simulated as well as real data traces. Lastly, we designed and implemented a novel client-server based mo- bile sensing system architecture which does user behavior learning, optimal policy calculation, energy efficient sensing, and state estimation automatically. We implemented a prototype of this system as an accelerometer based user activity recognition system on an off-the-shelf smart phone, and as a benchmark result, we demonstrate the performance benchmark ofu ∗ s through a set of experiments, and show that it improves the estimation accuracy by 27.8% and 48.6% over Markov-optimal policy and uniform sampling, respectively. Our research presented in this thesis dissertation provides guidelines for designing energy ef- ficient mobile sensing systems in terms of how different sensors should be better cooperated and how each individual sensor could be optimally duty cycled to achieve the best trade-off between energy consumption and user state estimation accuracy. There exist many possible research di- rections that can succeed the content of this thesis dissertation. For example, how to effectively schedule multiple sensors that could potentially detect the same set of user states with differ- ent detection accuracies and energy costs? How to conduct policy optimization where different weights are associated with different user state detection error, i.e., some states may be more important than others? How to design an energy efficient mobile sensing system where not only sensor cost, but multiple types of energy consumption such as computation and data commu- nication are taken into consideration? How to more efficiently learn the user behavior model in real-time instead of We believe that the design, analysis, and optimization frameworks introduced in this thesis dissertation could serve as significant building blocks for all these kinds of future study in energy efficient mobile sensing. 117 References [1] T. Abdelzaher, Y . Anokwa, P. Boda, J. Burke, D. Estrin, L. Guibas, A. Kansal, S. Madden, and J. Reich. Mobiscopes for human spaces. In IEEE Pervasive Computing, volume 6, pages 20–29, 2007. [2] P. Aghera, D. Fang, T. Simunic Rosing, and K. Patrick. Energy management in wireless healthcare systems. In Proceedings of the IEEE International Conference on Information Processing in Sensor Networks (IPSN), 2009. [3] E. Altman. Constrained Markov Decision Processes. Chapman & Hall/CRC, 1999. [4] A. Andrew, Y . Anokwa, K. Koscher, J. Lester, and G. Borriello. Context to make you more aware. In Proceedings of IEEE International Conference on Distributed Computing Systems Workshops, 2007. [5] M. Annavaram, N. Medvidovic, U. Mitra, S. Narayanan, D. Spruijt-Metz, G. Sukhatme, Z. Meng, S. Qiu, R. Kumar, and G. Thatte. Multimodal sensing for pediatric obesity appli- cations. In UrbanSense Workshop at the ACM Conference on Embedded Networked Sensor Systems (SenSys), 2008. [6] J. K. Baker. The dragon system - an overview. In IEEE Transactions on Acoustic Speech Signal Processing, volume 23, pages 24–29. [7] S. Banerjee and R. R. Dholakia. Does location based advertising work? In International Journal of Mobile Marketing, 2008. [8] L. Bao and S. S. Intille. Activity recognition from user-annotated acceleration data. In Pervasive Computing, pages 1–17, 2004. [9] M. S. Barlett. The frequency goodness of fit test for probability chains. In Mathematical Proceedings of the Cambridge Philosophical Society, volume 47, pages 86–95, 1951. [10] L. E. Baum. An inequality and associated maximization technique in statistical estimation for probabilistic functions of markov processes. In Inequalities, volume 3, pages 1–8, 1972. [11] L. E. Baum and J. A. Egon. An inequality with applications to statistical estimation for probabilistic functions of a markov process and to a model for ecology. In Bulletin of the American Mathemmtical Society, volume 73, pages 360–363, 1967. [12] L. E. Baum and T. Petrie. Statistical inference for probabilistic functions of finite state markov chains. In The Annals of Mathematical Statistics, volume 37, pages 1554–1563, 1966. 118 [13] L. E. Baum, T. Petrie, G. Soules, and N. Weiss. A maximization technique occurring in the statistical analysis of probabilistic functions of markov chains. In The Annals of Mathemat- ical Statistics, volume 41, pages 164–171, 1970. [14] L. E. Baum and G. R. Sell. Growth functions for transformations on manifolds. In Pacific Journal of Mathemmtics, volume 27, pages 211–227, 1968. [15] R. Bellman. Dynamic Programming. Princeton University Press, 1957. [16] L. Benini, A. Bogliolo, G. A. Paleologo, and G. D. Micheli. Markov decision processes for control of a sensor network-based health monitoring system. In IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, volume 18, pages 813–833, 1998. [17] S. Biswas and M. Quwaider. Body posture identification using hidden markov model with wearable sensor networks. In BodyNets Workshop, March 2008. [18] J. Burke, D. Estrin, M. Hansen, A. Parker, N. Ramanathan, S. Reddy, and M. B. Srivastava. Participatory sensing. In WSW Workshop at the ACM Conference on Embedded Networked Sensor Systems (SenSys), 2006. [19] A. Chaintreau, P. Hui, J. Crowcroft, C. Diot, R. Gass, and J. Scott. Pocket switched net- works: Real-world mobility and its consequences for opportunistic forwarding. In Techni- cal Report 617, University of Cambridge, 2005. [20] Z. Charbiwala, Y . Kim, S. Zahedi, J. Friedman, and M. B. Srivastava. Energy efficient sam- pling for event detection in wireless sensor networks. In Proceedings of the International Symposium on Low Power Electronics and Design (ISLPED), 2009. [21] T. L. Cheung, K. Okamoto, F. Maker, X. Liu, and V . Akella. Markov decision process (mdp) framework for optimizing software on mobile phones. In Proceedings of the International Conference on Embedded Software (EMSOFT), 2009. [22] T. Choudhury, G. Borriello, S. Consolvo, D. Haehnel, B. Harrison, B. Hemingway, J. High- tower, P. Klasnja, K. Koscher, An. LaMarca, J. A. Landay, L. LeGrand, J. Lester, A. Rahimi, A. Rea, and D. Wyatt. The mobile sensing platform: An embedded activity recognition sys- tem. In Pervasive Computing, volume 7, pages 32–41, 2008. [23] I. Constandache, S. Gaonkar, M. Sayler, R. R. Choudhury, and L. Cox. Energy-efficient localization via personal mobility profiling. In Proceedings of the International Conference on Mobile Computing, Applications, and Services (MobiCase), 2009. [24] M. E. Crovella and A. Bestavros. Self-similarity in world wide web traffic evidence and possible causes. In IEEE/ACM Transactions on Networking, volume 5, pages 835–846, 1996. [25] D. Dolgov and E. Durfee. Optimal flow control of a class of queuing networks in equilib- rium. In Proceedings of the 19th International Joint Conference on Artificial Intelligence, 2005. 119 [26] S. B. Eisenman, N. D. Lane, E. Miluzzo, R. A. Peterson, G. Ahn, and A. T. Campbell. Metrosense project: People-centric sensing at scale. In Proceedings of WSW Workshop at the ACM Conference on Embedded Networked Sensor Systems (SenSys), 2006. [27] Facebook. http://www.facebook.com. [28] E. A. Feinberg. Constrained semi-markov decision processes with average rewards. In Mathematical Methods of Operations Research, volume 39, pages 257–288, 1994. [29] E. A. Feinberg and A. Shwartz. Markov decision models with weighted discounted criteria. In Math. of Operations Research, volume 19, pages 152–168, 1994. [30] E. A. Feinberg and A. Shwartz. Constrained markov decision models with weighted dis- counted criteria. In Math. of Operations Research, volume 20, pages 302–320, 1995. [31] J. D. Ferguson. Variable duration models for speech. In Symposium on the Application of Hidden Markov Models to Text and Speech, 1980. [32] B. Gajic and K.K. Paliwal. Robust speech recognition in noisy environments based on sub- band spectral centroid histograms. In IEEE Transactions on speech and audio processing, volume 14, pages 600–608, 2006. [33] R. K. Ganti, P. Jayachandran, T. F. Abdelzaher, and J. A. Stankovic. Satire: a software architecture for smart attire. In Proceedings of the International Conference on Mobile Systems, Applications, and Services (MobiSys), 2006. [34] J. Gao, E. G. Hauptmann, A. Bharucha, and H. D. Wactlar. Dining activity analysis us- ing a hidden markov model. In Proceedings of the International Conference on Pattern Recognition (ICPR), 2004. [35] T. Gao, C. Pesto, L. Selavo, Y . Chen, J. Ko, J. Kim, A. Terzis, A. Watt, J. Jeng, B. Chen, K. Lorincz, and M. Welsh. Wireless medical sensor networks in emergency response: Im- plementation and pilot results. In Proceedings of the IEEE Conference on Technologies for Homeland Security (HST), 2008. [36] S. Gaonkar, J. Li, R. R. Choudhury, L. Cox, and A. Schmidt. Micro-blog: sharing and querying content through mobile phones and social participation. In Proceedings of the International Conference on Mobile Systems, Applications, and Services (MobiSys), 2008. [37] H. W. Gellersen, A. Schmidt, and M. Beigl. Multi-sensor context-awareness in mobile devices and smart artifacts. In Mobile Networks and Applications, volume 7, pages 341– 351, 2002. [38] M. Grossglauser and D. Tse. Mobility increases the capacity of ad-hoc wireless networks. In IEEE/ACM Transactions on Networking, volume 10, pages 477–486, 2002. [39] D. P. Heyman and M. J. Sobel. Volume II: Stochastic Models in Operations Research. McGraw-Hill, 1984. 120 [40] B. Hoh, M. Gruteserand R. Herring, J. Ban, D. Work, J. Herrera, A. M. Bayen, M. An- navaram, and Q. Jacobson. Virtual trip lines for distributed privacy-preserving traffic mon- itoring. In Proceedings of the International Conference on Mobile Systems, Applications, and Services (MobiSys), 2008. [41] R. A. Howard. Dynamic Probabilistic Systems. Volume 2- Semi-Markov and decision pro- cesses. Wiley, New York, 1971. [42] J. Jaffe, L. Cassotta, and S. Feldstein. Markovian model of time patterns of speech. In Science, volume 144, pages 884–886, 1964. [43] D. Jea, J. Liu, T. Schmid, and M. Srivastava. Hassle free fitness monitoring. In HealthNet Workshop at the International Conference on Mobile Systems, Applications, and Services (MobiSys), 2008. [44] F. Jelinek. A fast sequential decoding algorithm using a stack. In IBM Journal of Research and Development, volume 13, pages 675–685, 1969. [45] F. Jelinek, L. R. Bahl, and R. L. Mercer. Design of a linguistic statistical decoder for the recognition of continuous speech. In IEEE transactions on Information Theory, volume 21, pages 250–265, 1975. [46] R. Jurdak, P. Corke, D. Dharman, and G. Salagnac. Adaptive gps duty cycling and radio ranging for energy-efficient localization. In Proceedings of the ACM Conference on Em- bedded Networked Sensor Systems (SenSys), 2010. [47] L. C. M. Kallenberg. Linear Programming and Finite Markovian Control Problems. Math- ematical Centrum, 1983. [48] S. Kang, J. Lee, H. Jang, H. Lee, Y . Lee, S. Park, T. Park, and J. Song. Seemon: scalable and energy-efficient context monitoring framework for sensor-rich mobile environments. In Proceedings of the International Conference on Mobile Systems, Applications, and Services (MobiSys), 2008. [49] A. Kansal, M. Goraczko, and F. Zhao. Building a sensor network of mobile phones. In Proceedings of the IEEE International Conference on Information Processing in Sensor Networks (IPSN), 2007. [50] D. H. Kim, Y . Kim, D. Estrin, and M. B. Srivastava. Sensloc: Sensing everyday places and paths using less energy. In Proceedings of the ACM Conference on Embedded Networked Sensor Systems (SenSys), 2010. [51] A. Krause, M. Ihmig, E. Rankin, S. Gupta, D. Leong, D. P. Siewiorek, A. Smailagic, M. Deisher, and U. Sengupta. Trading off prediction accuracy and power consumption for context-aware wearable computing. In Proceedings of the IEEE International Symposium on Wearable Computers (ISWC), 2005. [52] N. D. Lane, H. Lu, S. B. Eisenman, and A. T. Campbell. Cooperative techniques supporting sensor-based people-centric inferencing. In Lecture Notes in Computer Science, volume 5013, pages 75–92, 2008. 121 [53] A. Lazar. Optimal flow control of a class of queuing networks in equilibrium. In IEEE transactions on Automatic Control, volume 28, pages 1001–1007, 1983. [54] J. Lee and M. Kang. Data collection scheme for two-tier vehicular sensor networks. In Lecture Notes in Computer Science, volume 6221, pages 295–298, 2010. [55] J. Lester, T. choudhury, G. Borriello, S. Consolvo, J. Landay, K. Everitt, and I. Smith. Sens- ing and modeling activities to support physical fitness. In Proceedings of the International Conference on Ubiquitous Computing (Ubicomp), Tokyo, Japan, 2005. [56] J. Lester, B. Hannaford, and G. Borriello. Are you with me? - using accelerometers to determine if two devices are carried by the same person. In Pervasive Computing, pages 33–50, 2004. [57] S. E. Levinson. Continuously variable duration hidden markov models for automatic speech recognition. In Computer Speech and Language, volume 1, pages 29–45, 1986. [58] K. Lin, A. Kansal, D. Lymberopoulos, and F. Zhao. Energy-accuracy aware localization for mobile devices. In Proceedings of the International Conference on Mobile Systems, Applications, and Services (MobiSys), 2010. [59] G. Lu and T. Hankinson. A technique towards automatic audio classification and retrieval. In Proceedings of the International Conference on Signal Processing (ICSP), 1998. [60] E. Miluzzo, N. Lane, K. Fodor, R. Peterson, S. Eisenman, H. Lu, M. Musolesi, X. Zheng, and A. Campbell. Sensing meets mobile social networks: The design, implementation and evaluation of the cenceme application. In Proceedings of the ACM Conference on Embedded Networked Sensor Systems (SenSys), 2008. [61] E. Miluzzo, N. D. Lane, S. B. Eisenman, and A. T. Campbell. Cenceme - injecting sensing presence into social networking applications. In EuroSSC - Lecture Notes in Computer Science, volume 4793, pages 1–28, 2007. [62] C. Mitchell, M. Harper, and L. Jamieson. On the complexity of explicit duration hmms. In IEEE Transactions on Speech Audio Processing, volume 3, pages 213–217, 1995. [63] P. Mohan, V . Padmanabhan, and R. Ramjee. Nericell: Rich monitoring of road and traffic conditions using mobile smartphones. In Proceedings of the ACM Conference on Embedded Networked Sensor Systems (SenSys), 2008. [64] Moteiv. Telosb motes datasheet. http://www.xbow.com/Products/Product pdf files/Wireless pdf/TelosB Datasheet.pdf. [65] L. Nachman, J. Huang, R. Kong, R. Shah, J. Shahabdeen, C. Wan, and M. Yarvis. On-body health data aggregation using mobile phones. In Participatory Research Workshop at the ACM Conference on Embedded Networked Sensor Systems (SenSys), 2007. [66] P. Nain and K. W. Ross. Optimal priority assignment with hard constraint. In IEEE trans- actions on Automatic Control, volume 31, pages 883–888, 1986. 122 [67] J. Paek, J. Kim, and R. Govindan. Energy-efficient rate-adaptive gps-based positioning for smartphones. In Proceedings of the International Conference on Mobile Systems, Applica- tions, and Services (MobiSys), 2010. [68] A. Peddemors, H. Eertink, and I. Niemegeers. Density estimation for out-of-range events on personal mobile devices. In Proceedings of MobilityModel, 2008. [69] P. Pillai and K. G. Shin. Real-time dynamic voltage scaling for low-power embedded op- erating systems. In Proceedings of the ACM symposium on Operating systems principles (SOSP), 2001. [70] L. R. Rabiner. A tutorial on hidden markov models and selected applications in speech recognition. In Proceedings of the IEEE, 1989. [71] M. J. Russell and R. K. Moore. Explicit modeling of state occupancy in hidden markov models for automatic speech recognition. In Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1985. [72] A. Ruzzelli, R. Jurdak, and G. OSHare. Managing mobile-based participatory sensing com- munities. In Participatory Research Workshop at the ACM Conference on Embedded Net- worked Sensor Systems (SenSys), 2007. [73] D. Sanchez, M. Tentori, and J. Favela. Hidden markov models for activity recognition in ambient intelligence environments. In Proceedings of the International Conference on Current Trends in Computer Science (ENC), 2007. [74] A. Schmidt, K. A. Aidoo, A. Takaluoma, U. Tuomela, K. V . Laerhoven, and W. V . D. Velde. Advanced interaction in context. In Proceedings of the International Symposium on Handheld and Ubiquitous Computing (HUC), 1999. [75] G. Sharma and R. R. Mazumdar. Scaling laws for capacity and delay in wireless ad hoc networks with random mobility. In Proceedings of the IEEE International Conference on Communications (ICC), 2004. [76] E. Shih, P. Bahl, and M. J. Sinclair. Wake on wireless: an event driven energy saving strategy for battery operated devices. In Proceedings of the International Conference on Mobile Computing and Networking (MobiCom), 2002. [77] M. Shin, P. Tsang, D. Kotz, and C. Cornelius. Deamon: Energy-efficient sensor monitoring. In Proceedings of the IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks (SECON), 2009. [78] D. Siewiorek, A. Smailagic, J. Furukawa, N. Moraveji, K. Reiger, and J. Shaffer. Sensay: a context-aware mobile phone. In Proceedings of the IEEE International Symposium on Wearable Computers (ISWC), 2003. [79] A. Silberstein, R. Braynard, and J. Yang. Constraint chaining: On energy-efficient contin- uous monitoring in sensor networks. In Proceedings of the International Conference on Management of data (SIGMOD), 2006. 123 [80] J. Sorber, N. Banerjee, M. D. Corner, and S. Rollins. Turducken: hierarchical power man- agement for mobile devices. In Proceedings of the International Conference on Mobile Systems, Applications, and Services (MobiSys), 2005. [81] L. Stabellini and J. Zander. Energy efficient detection of intermittent interference in wireless sensor networks. In International Journal of Sensor Networks (IJSNet), volume 8, pages 27–40, 2010. [82] T. Stiefmeier, D. Roggen, G. Troster, G. Ogris, and P. Lukowicz. Wearable activity tracking in car manufacturing. In Pervasive Computing, volume 7, pages 42–50, April-June 2008. [83] A. Thiagarajan, L. S. Ravindranath, K. LaCurts, S. Toledo, J. Eriksson, S. Madden, and H. Balakrishnan. Vtrack: Accurate, energy-aware traffic delay estimation using mobile phones. In Proceedings of the ACM Conference on Embedded Networked Sensor Systems (SenSys), 2009. [84] M. A. Viredaz, L. S. Brakmo, and W. R. Hamburgen. Energy management on handheld devices. In ACM Queue, volume 1, pages 44–52, 2003. [85] A. J. Viterbi. Error bounds for convolutional codes and an asymptotically optimum decod- ing algorithm. In IEEE Transactions on Information Theory, volume 13, pages 260–269, 1967. [86] W. Wang, V . Srinivasan, and M. Motani. Adaptive contact probing mechanisms for delay tolerant applications. In Proceedings of the International Conference on Mobile Computing and Networking (MobiCom), 2007. [87] Y . Wang, B. Krishnamachari, and T. Valante. Findings from an empirical study of fine- grained human social contacts. In Proceedings of the International Conference on Wireless On-demand Network Systems and Services (WONS), 2009. [88] Y . Wang, B. Krishnamachari, Q. Zhao, and M. Annavaram. The tradeoff between energy efficiency and user state estimation accuracy in mobile sensing. In Proceedings of the Inter- national Conference on Mobile Computing, Applications, and Services (MobiCase), 2009. [89] Y . Wang, B. Krishnamachari, Q. Zhao, and M. Annavaram. Markov-optimal sensing pol- icy for user state estimation in mobile devices. In Proceedings of the IEEE International Conference on Information Processing in Sensor Networks (IPSN), 2010. [90] Y . Wang, J. Lin, M. Annavaram, Q. A. Jacobson, J. Hong, B. krishnamachari, and N. Sadeh. A framework of energy efficient mobile sensing for automatic user state recognition. In Proceedings of the International Conference on Mobile Systems, Applications, and Services (MobiSys), 2009. [91] E. Welbourne, J. Lester, A. LaMarca, and G. Borriello. Mobile context inference using low-cost sensors. In Workshop on Location and Context-Awareness (LoCA), 2005. [92] K. Whitesell, B. Kutler, N. Ramanathan, and D. Estrin. Determining indoor air quality from images of an air filter captured on cell phones. In ImageSense Workshop at the ACM Conference on Embedded Networked Sensor Systems (SenSys), 2008. 124 [93] SKYHOOK Wireless. http://www.skyhookwireless.com/. [94] W. H. Wu, L. K. Au, B. Jordan, T. Stathopoulos, M. A. Batalin, W. J. Kaiser, A. Vahdatpour, M. Sarrafzadeh, M. Fang, and J. Chodosh. Smartcane system: An assistive device for geriatrics. In BodyNets Workshop, 2008. [95] S. Yu. Hidden semi-markov models. In Artificial Intelligence, volume 174, pages 215–243, 2009. [96] S. Yu and H. Kobayashi. A hidden semi-markov model with missing data and multiple observation sequences for mobility tracking. In Signal Processing, volume 83, pages 235– 250, 2003. [97] P. Zappi, T. Stiefmeier, E. Farella, D. Roggen, L. Benini, and G. Troster. Activity recogni- tion from on-body sensors by classifier fusion: sensor scalability and robustness. In Pro- ceedings of the International Conference on Sensors, Sensor Networks and Information Processing (ISSNIP), 2007. [98] Z. Zhuang, K. Kim, and J. P. Singh. Improving energy efficiency of location sensing on smartphones. In Proceedings of the International Conference on Mobile Systems, Applica- tions, and Services (MobiSys), 2010. 125

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Crowd-sourced collaborative sensing in highly mobile environments

PDF

Mobility-based topology control of robot networks

PDF

Dynamic routing and rate control in stochastic network optimization: from theory to practice

PDF

Energy efficient design and provisioning of hardware resources in modern computing systems

PDF

A framework for runtime energy efficient mobile execution

PDF

Models and algorithms for energy efficient wireless sensor networks

PDF

Energy-efficient shutdown of circuit components and computing systems

PDF

Compilation of data-driven macroprograms for a class of networked sensing applications

PDF

Energy-efficient computing: Datacenters, mobile devices, and mobile clouds

PDF

Reconfiguration in sensor networks

PDF

Joint routing and compression in sensor networks: from theory to practice

PDF

Energy optimization of mobile applications

PDF

Towards interference-aware protocol design in low-power wireless networks

PDF

A joint framework of design, control, and applications of energy generation and energy storage systems

PDF

Towards green communications: energy efficient solutions for the next generation cellular mobile communication systems

PDF

Automatic image matching for mobile multimedia applications

PDF

Optimizing task assignment for collaborative computing over heterogeneous network devices

PDF

Efficient pipelines for vision-based context sensing

PDF

Energy -efficient information processing and routing in wireless sensor networks: Cross -layer optimization and tradeoffs

PDF

SLA-based, energy-efficient resource management in cloud computing systems

Asset Metadata

Creator Wang, Yi (author)

Core Title Towards energy efficient mobile sensing

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Electrical Engineering

Publication Date 04/01/2011

Defense Date 02/24/2011

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag constrained stochastic optimization,energy efficiency,mobile sensing,OAI-PMH Harvest,state estimation

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Krishnamachari, Bhaskar (committee chair), Annavaram, Murali (committee member), Sukhatme, Gaurav S. (committee member)

Creator Email wangyi@usc.edu,wangyi0511@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-m3713

Unique identifier UC196380

Identifier etd-Wang-4413 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-445526 (legacy record id),usctheses-m3713 (legacy record id)

Legacy Identifier etd-Wang-4413.pdf

Dmrecord 445526

Document Type Dissertation

Rights Wang, Yi

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Repository Name Libraries, University of Southern California

Repository Location Los Angeles, California

Repository Email cisadmin@lib.usc.edu