University of Southern California Dissertations and Theses

Demand response management in smart grid from distributed optimization perspective

(USC Thesis Other)

Demand response management in smart grid from distributed optimization perspective

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content DEMAND RESPONSE MANAGEMENT IN SMART GRID FROM DISTRIBUTED OPTIMIZATION PERSPECTIVE by Zhaohui Zhang A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (CHEMICAL ENGINEERING) May 2017 Copyright 2017 Zhaohui Zhang ii Dedication To my dear family. iii Acknowledgments I would like to thank everyone who supported me, encouraged me and inspired me during my PhD journey at the University of Southern Cal- ifornia. In fact, this doctoral work would not be finished without these peo- ple influencing me in every aspect of my life. Foremost, I would like to offer my sincerest gratitude to my advisor, Professor S. Joe Qin. It is my greatest honor to study and work under his guidance. My doctoral study cannot be completed without his ingenious di- rection and constructive criticism. Throughout my PhD study, he provided me the freedom and support to work on a topic that was of great interest to me, encouraged me to develop new ideas and think innovatively, and as- sisted me with enhancing professional analytical and communication skills. His passion, enthusiasm and belief in science and research have been con- tinuous inspiration and motivation to me and will benefit me through my career. I am grateful for having an exceptional dissertation committee. I appreciate the effort of Dr. Petros Ioannou from Electrical Engineering and Dr. Katherine Shing from the Chemical Engineering Department for serving in my committee. I would like to thank them for their insightful comments iv and constructive advices. The experience with the Qin research group was memorable and en- joyable. I feel very fortunate to have the former members who offered me useful advices and fruitful discussions, including Alan, Yingying, Hu, Jin- gran, Tao, Yu, Johnny, Yining and Wei. I would also like to thank the cur- rent members of the group, Alisha, Qinqin, Yuan and postdoc Gang, visit- ing scholar Juncheng, Lijuan, Ying, Le and Qiang, for their friendliness and consistent help. In particular I appreciate the time spending with Ruilong, Gang and Tao on our joint work. They taught me discipline and approaches in the research process meticulously, and revised my manuscripts over and over again patiently. This dissertation would have not been possible if it were not for their guidance and collaborations as both mentors and friends. I would like to acknowledge Johnson Controls Inc., Texas-Wisconsin- California Control Consortium (TWCCC) and Center for Interactive Smart Oilfield Technologies (Cisoft) for the financial supports they offered during my graduate studies. Finally, I cannot thank them enough, my parents Min and Yongmei, my beloved husband Tao and the rest of my dear family, for their endless love, continuous encouragement and unconditional support. v Table of Contents Dedication ii Acknowledgments iii List of Tables viii List of Figures ix Abstract xi Chapter 1. Introduction 1 1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.1 Smart Grid . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.2 Demand Response Management . . . . . . . . . . . . . . 3 1.1.3 Distributed Optimization Methods . . . . . . . . . . . . 7 1.2 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Chapter 2. Bi-level Game for Demand Response with Information Sharing among Consumers 18 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.2 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.2.1 Two-way Communication Infrastructure . . . . . . . . . 21 2.2.2 Cost Function for Power Provider . . . . . . . . . . . . . 22 2.2.3 Gain Function for Power Consumer . . . . . . . . . . . . 23 2.3 Bi-level Game Formulation . . . . . . . . . . . . . . . . . . . . . 24 2.3.1 Supply Side: Leader-level . . . . . . . . . . . . . . . . . . 24 2.3.2 Demand Side: Follower-level . . . . . . . . . . . . . . . . 25 2.3.3 Leader-Follower Interaction: Stackelberg Game . . . . . 30 2.4 Distributed Algorithms . . . . . . . . . . . . . . . . . . . . . . . 33 2.5 Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . 38 2.5.1 Convergence Analysis . . . . . . . . . . . . . . . . . . . . 38 vi 2.5.2 Equilibrium Analysis . . . . . . . . . . . . . . . . . . . . 42 2.5.3 Performance Comparison . . . . . . . . . . . . . . . . . . 46 2.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 Chapter 3. Sliding Window Games for Cooperative Building Tem- perature Control Using Distributed Learning Method 51 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.2 Centralized Model . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.3 Game Reformulation and Distributed Cooperative Control . . 57 3.3.1 Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 3.3.2 Players . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 3.3.3 Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 3.3.4 Payoff Functions . . . . . . . . . . . . . . . . . . . . . . . 60 3.3.5 Global Payoff . . . . . . . . . . . . . . . . . . . . . . . . . 63 3.3.6 Strategy: A payoff based distributed learning algorithm 63 3.4 Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 3.4.1 Hourly Game . . . . . . . . . . . . . . . . . . . . . . . . . 66 3.4.2 Sliding Window Game . . . . . . . . . . . . . . . . . . . . 70 3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Chapter 4. Distributed Optimization of Multi-building Energy Sys- tem with Spatially and Temporally Coupled Constraints 75 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 4.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . 79 4.3 Dual Decomposition Approach . . . . . . . . . . . . . . . . . . 83 4.3.1 Dual Problem . . . . . . . . . . . . . . . . . . . . . . . . . 84 4.3.2 Dual Solution . . . . . . . . . . . . . . . . . . . . . . . . . 87 4.3.3 Distributed Algorithm . . . . . . . . . . . . . . . . . . . . 89 4.4 Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . 95 4.4.1 Convergence Analysis . . . . . . . . . . . . . . . . . . . . 96 4.4.2 Performance Analysis . . . . . . . . . . . . . . . . . . . . 100 4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 Chapter 5. Indoor Temperature Control of Cost-Effective Smart Buildings via Real-Time Smart Grid Communications 106 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 vii 5.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . 110 5.3 Temporally-Coupled Solution . . . . . . . . . . . . . . . . . . . 114 5.4 Temporally-Decoupled Solution . . . . . . . . . . . . . . . . . . 117 5.5 Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 Chapter 6. Conclusions 128 Bibliography 131 viii List of Tables Table 1.1 A brief comparison between the traditional grid and the smart grid [39]. . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Table 4.1 Summary of notations . . . . . . . . . . . . . . . . . . . . . . 80 Table 5.1 Summary of notations . . . . . . . . . . . . . . . . . . . . . . 111 Table 5.2 Parameter setup . . . . . . . . . . . . . . . . . . . . . . . . . . 124 ix List of Figures Figure 1.1 The concept of the smart grid. . . . . . . . . . . . . . . . . . 1 Figure 1.2 Generation follows load vs. load adapts to generation. . . . 4 Figure 2.1 Two-way communication infrastructure. . . . . . . . . . . . 21 Figure 2.2 Without information sharing among consumers. . . . . . . 25 Figure 2.3 With information sharing among consumers. . . . . . . . . 26 Figure 2.4 With demand information sharing platform. . . . . . . . . . 34 Figure 2.5 Gradient projection solution (2.17) converged to the closed-form solution (2.16). . . . . . . . . . . . . . . . . . . . 39 Figure 2.6 The impact of step size on the convergence rate of the gra- dient projection method. . . . . . . . . . . . . . . . . . . . . 41 Figure 2.7 Initial value does not affect equilibrium point. . . . . . . . . 43 Figure 2.8 Consumer parameter affects equilibrium point. . . . . . . 44 Figure 2.9 Consumer parameter! affects equilibrium point. . . . . . . 45 Figure 2.10 Performance comparison between solution 1 (no) and so- lution 2 (share). . . . . . . . . . . . . . . . . . . . . . . . . . . 47 Figure 2.11 Performance comparison between solution 1 (no: green) and solution 2 (share: red). . . . . . . . . . . . . . . . . . . . 48 Figure 3.1 Global utility distributed (blue) vs. centralized optimum (red). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 x Figure 3.2 Global utility components. . . . . . . . . . . . . . . . . . . . 69 Figure 3.3 Sliding window size = 4 (black) vs. window size = 1 (blue). 71 Figure 3.4 Outdoor (red), indoor (black) temperature and pre-cooling. 72 Figure 4.1 Interaction between information sharing platform and each building based on dual decomposition approach. . . . 91 Figure 4.2 TOU electricity price and the predicted outdoor tempera- ture of Edmonton, AB, Canada on November 18, 2015. . . . 95 Figure 4.3 The impact of step size on the convergence rate of the sub- gradient projection method. . . . . . . . . . . . . . . . . . . . 97 Figure 4.4 The convergence of the dual objective value and the aver- age value of Lagrangian multipliers versus iteration. . . . . 98 Figure 4.5 Constraints violation versus iteration. . . . . . . . . . . . . 99 Figure 4.6 The total energy consumption, total load threshold, and congestion price versus time slot. . . . . . . . . . . . . . . . 100 Figure 4.7 Three buildings’ indoor temperature, comfort zone, and temperature control signals. . . . . . . . . . . . . . . . . . . 102 Figure 5.1 Real-time communications in smart grid for smart buildings.108 Figure 5.2 Illustration of indoor temperature control for smart build- ings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 Figure 5.3 Performance comparison of intuitive and temporally- coupled/decoupled approaches for one zone of a smart building in Edmonton on Jan. 6, 2016. . . . . . . . . . . . . . 121 Figure 5.4 Daily electricity cost reduction by pre-heating in January, 2016. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 xi Abstract Nowadays, by taking advantages of widely deployed smart meters and two-way communication facilities of smart grid, demand response management, aiming to change the electricity usage patterns of end-users with pricing incentives of the supply side, can lead to a significant im- provement of gird reliability and efficiency. The goal of this dissertation is to develop distributed optimization methods under different scenarios to address the challenges caused by hierarchical structure, communication impact and spatially temporally coupled constraints in demand response management of the smart grid system. A novel bi-level demand response game structure is presented and the impact of communication mechanisms among consumers is addressed. The bi-level game interactions are characterized by a follower-level nonco- operative game and a leader-follower Stackelberg game. An information sharing platform is designed and the win-win effectiveness of information sharing among consumers is illustrated. Energy consumers consisting of multiple buildings are also studied, where individual building cooperates to achieve the group-level optimum. Instead of Nash Equilibrium in normal games, Pareto Optimum is achieved xii by leveraging the design of sliding window games and distributed learning algorithm. Each building learns to play a part of the optimal solution and the method can achieve a global sub-optimal solution. Due to distribution infrastructure capacity and heat preservation, spatially and temporally coupled constraints need to be considered. These constraints are sophisticatedly formulated for the primal cost minimization problem, and separable local optimization subproblems are transformed via Lagrangian dual decomposition. With strong duality property and coordination signals, global optimality of the combined local solutions can be guaranteed. In real scenario, the perfect prediction of electricity price and outdoor temperature of the next day is hard to obtain, which will deteriorate perfor- mance of related smart building strategies. A temporally-decoupled algo- rithm for individual building is developed, requiring only the next-hour electricity price. Real-data case studies show that the proposed algorithm could lead to significant economic savings. 1 Chapter 1 Introduction 1.1 Overview 1.1.1 Smart Grid Electricity is the most versatile and widely used form of energy. To meet with the continuously growing global demand, the electrical power system, which was built up over more than one hundred years, needs to undergo significant adjustments [11, 39]. Figure 1.1: The concept of the smart grid. The smart grid is referred to as the next generation of the electrical power system, with the integration of two-way cyber-secure communica- tion technologies and computational intelligence across electricity genera- 2 tion, transmission, substations, distribution and consumption. The aim is to achieve a system that is clean, safe, secure, reliable, resilient, efficient and sustainable [44, 46, 47, 88]. The concept of the smart grid covers the en- tire spectrum of the energy system from the generation to the end points of consumption of the electricity, as illustrated in Fig. 1.1. The ultimate smart grid is a vision with integration of complementary components, sub- systems, functions and services under the pervasive control of highly intel- ligent management and control systems [38]. Table 1.1: A brief comparison between the traditional grid and the smart grid [39]. Traditional grid Smart grid Electromechanical Digital One-way communication Two-way communication Centralized generation Distributed generation Few sensors Sensors throughout Manual monitoring Self-monitoring Manual restoration Self-healing Failures and blackouts Adaptive and islanding Limited control Pervasive control Few customer choices Many customer choices Smart features such as renewable energy generation, advanced me- tering infrastructure (AMI), plug-in hybrid vehicles (PHEVs) and vehicle- to-grid (V2G) capability have been regarded as key components of the smart grid. In addition, by having smart meters installed at users’ premises and 3 two-way communications enabled between the power utility and users, de- mand response (DR) or demand response management (DRM) becomes an essential characteristic of the smart grid, with the ability to shape the users’ electricity loads in an automated and convenient manner [45, 83]. Moreover, with pervasive distributed energy resources (DERs), microgrids, a power subsystem that can operate independently from bulk generation, are be- coming viable nowadays. Two operational modes, grid-connected and is- landed, enable the microgrid being both a generator and consumer in the smart grid [30]. A brief comparison between the traditional grid and the smart grid is summarized in Table 1.1. The benefits associated with the new features of the smart grid include more efficient and reliable power operations and revolutionized energy management, quicker restoration of electricity from power distur- bances or outages, reduced generation cost and reduced electricity price for both utilities and consumers, more flexible demand scheduling with the development of intelligent home management systems, increased integration of distributed and renewable energy generation for cleaner energy systems and less greenhouse emissions. 1.1.2 Demand Response Management Demand response management is an essential component for both utility companies and end-users in the smart grid, which enables the supply 4 and demand side to interact with each other by exchanging the electricity price and demand information, in order to make wise decisions [4, 45]. Figure 1.2: Generation follows load vs. load adapts to generation. Traditionally, due to the lack of communication infrastructures, the generation of power is mostly led by the demand side [3, 81]. With the uncertainty in demand, unpredictable and unavoidable peaks, as well as increasing pressures especially from PHEVs’ charging, the supply side suf- fers from high cost, lack of dependability and low efficiency. Nowadays, taking advantages of widely deployed smart meters and two-way commu- nication facilities of smart grid, the dynamic and market-based electricity prices could serve as the invisible hand to coordinate between the supply side and demand side, such that load can be adapted to generation [94]. As the term indicates, demand response management represents the 5 management of the demand side through demand response approaches, such that electricity usage patterns of end-users change in response to incen- tives from the supply side. The goal of demand response includes shaping the demand profile, reducing the peak demand, and flattening the demand curve [30]. Based on the incentive mechanisms, demand response management programs can be classified into two categories: 1) Incentive-based program, including direct load control (DLC), interruptible load, demand bidding and buyback, and emergency demand reduction [1, 37, 89, 98]. In these programs, the power utilities and consumers 1 reach an agreement that con- sumers are provided with incentive discounts or incentive payments in re- ply to their curtails of electricity usage when the gird reliability is jeopar- dized. 2) Price-based program, including time-of-use (TOU) pricing, critical peak pricing (CPP), real-time pricing (RTP), and inclining block rate (IBR) [6, 14, 26, 51]. These programs are various time-based pricing tariffs that indirectly induce users to dynamically change their energy usage patterns according to the variance of electricity prices, instead of directly controlling their loads. Among aforementioned incentive mechanisms, TOU and RTP are most commonly used incentives for now. In TOU, the electricity prices 1 Throughout the text, the terms “user” and “consumer” are interchangeably used. 6 have three levels: on-peak, mid-peak and off-peak. The electricity price at the on-peak time block is much higher than the price at the mid-peak and off-peak time blocks, in order to induce users to shift their loads to avoid on-peak hours. TOU pricing is usually released far in advance and keeps unchanged for a long time period. In RTP , the electricity price varies with every time interval in a real time manner, usually each hour or each fifteen minutes. Therefore, the RTP pricing is usually released on an hour-ahead or more frequent basis. The most important benefit of demand response management is im- proved resource-efficiency of electricity production due to closer alignment between customers’ electricity prices and the value they place on electricity. This increased efficiency creates a variety of benefits, which fall into four groups [83]: • Participant financial benefits. These are the bill savings and incentive payments earned by customers that adjust their electricity demand in response to time-varying electricity rates or incentive-based pro- grams. • Market-wide financial benefits. According to [40], 6-8% lower energy prices are possible with 1% less peak demand. Lower market prices will result from the more efficient wholesale market. 7 • Reliability benefits. Demand response management lowers the like- lihood and consequences of forced outages that impose financial cost and inconvenience on customers. • Market performance benefits. This refer to the value of demand re- sponse management in mitigating suppliers’ ability to exercise market power by raising power prices significantly above production costs. To sum up, demand response management is an effective means of rescheduling the users’ energy consumption to reduce the operating ex- pense from expensive generators and further to defer the capacity addition in the long run. It will make the power system more reliable, enhance the transparency and efficiency of the electricity market, and lead to mutual financial benefits for both the power utility and all users. Last but not least, it will reduce the generating emissions and alleviate the environmental impacts, by enabling a more efficient utilization of current grid capacity. 1.1.3 Distributed Optimization Methods As the size of power systems expands, more and more distributed generation and flexible demand being involved in smart grid, distributed optimization, as an alternative approach to solve challenges of the central- ized optimization mechanism, has attracted increasing attention recently [100]. 8 According to different origin domain, the majority of the distributed optimization methods can be classified into the following three categories: • Game-based methods (Economics Domain) Game theory is begin- ning to emerge as a powerful tool for the control and optimization of distributed systems [20, 25, 69, 85, 92]. The utilization of game theory involves two steps [59]: firstly, decompose the system to consists multiple self-interested decision makers with corresponding action sets and local objective functions; secondly, specify local algorithms that enable the decision makers to reach a desirable operating point, e.g., Nash Equilibrium or Pareto Optimum of the designed game. There have been extensive researches on simple game models that characterize the behaviors of the power generators and consumers [24, 29, 52, 72, 77]. • Lagrangian-based methods (Optimization Domain) In Lagrangian re- laxation approaches [15, 17, 23, 28, 31, 82], Lagrangian dual problems are defined as a reformulation of the constraint satisfaction problem, and the duality gaps between the primal and dual problems are eval- uated. The distributed algorithms can be devised based on the charac- terization of the saddle points of the Lagrangian function, and involve each agent updating its estimates of the saddle points via a certain di- rectional step, such as a primal or dual projection step, or a subgra- 9 dient or supgradient step, and onto its local constraint set or a com- pact set containing the dual optimal set. Such transformations and representations may reveal hidden decomposability structures of the original problem. • Consensus-based methods (Multi-Agent Systems Domain) In consensus-based methods, multiple agents collectively try to agree on the same state value in an iterative and distributed pattern [21, 79, 80]. There has been interesting researches put forward, such as the flooding based consensus approach proposed in [34, 105, 84], and the incremental cost based consensus method proposed in [12, 107, 108], which were proven to be efficient when neglecting system level coupled constraints, e.g., network constraints and capacity limits. Consensus-based optimization algorithms have the appealing fea- ture that they can operate in a peer-to-peer fashion, with minimal coordination between nodes [97]. The three categories of methods can be compared from several aspects: 1. Selfish vs. Global: Game-based methods are mostly selfish, which focuses on local optimization for each individual player. Meanwhile, Lagrangian-based 10 or Consensus-based methods mostly have global optimization objec- tive, which usually result in better performance but is not as realistic as Game-based methods for motivation in practice. An exception in Game-based methods that can achieve global opti- mization goal is the distributed learning method put forward recently by [70], which teaches each player to play part of the Pareto Optimum instead of the Nash Equilibrium by learning rule design. Our work in Chapter 3 is motivated and based on this distributed learning method. During the sliding window games, each player maxi- mizes its own payoff selfishly, while overall the method can achieve a prob- abilistic global optimal solution. Although the distributed learning method is very interesting and promising, we found several drawbacks of it as follows through practice: 1) The convergence is probabilistic convergence instead of guaranteed convergence. 2) In practice, it requires too many iterations to converge. 3)The probabilistic transition design is hard to handle large feasible region or many players scenarios. 4) Need to discretize feasible region, and thus hard to converge to original continuous optimum. 5) Need real-time payoff feedback to achieve fully-distributed, otherwise is semi-distributed. 2. Handling of Coupled Constraints: 11 Game-based methods and Consensus-based methods are mostly de- signed for games or coordinations with no constraints, while Lagrangian- based methods can deal with constraints better. To realize the capability of constraints handling in games, one option is to incorporate constraints to payoff functions by penalty design, as our work in Chapter 3 where we formulate the payoff function consisting two contributions from constraints. However, it should be noticed that through this method, the transformed problem will not be exactly equal to the orig- inal problem. Another option is a rising direction called ”games with cou- pled constraints” put forward by [58], to realize exact equality of the game to original optimization problem. Lagrangian-based methods are able to handles coupled constraints through dual decomposition, and then design the distributed algorithms to solve the dual problems, the details of which is illustrated in our work in Chapter 4 by carefully formulating and handling of the spatially and temporally coupled constraints. 3. Semi-distributed vs. Fully-distributed: Informally, we call an algorithm fully distributed with respect to a network connectivity graphG if each node ofG operates without using any information beyond that in its local neighborhood inG [74]. Game-based methods can be semi-distributed or fully-distributed, 12 which depend on the mechanism of information sharing. If there is abso- lutely no information sharing, the method can be fully-distributed. Other- wise, the method is semi-distributed and sometimes involves the design of the information sharing mechanism. Lagrangian-based methods are usually semi-distributed, for fully- distributed methods, more advanced optimization theories may be referred to [15, 43, 75]. Consensus-based methods are usually fully-distributed, which only requires neighbors’ information. 4. Periodical vs. Event-triggered Communication: Game-based methods, Lagrangian-based and most Consensus-based methods are periodical, meaning regular time-interval based communica- tions and iterations are necessary. Some of event-triggered consensus algo- rithms have communication design on an irregular bases, and communica- tion will be triggered only when certain conditions are satisfied. To sum up, the Game-based, Lagrangian-based and Consensus-based meth- ods have different advantages and shortcomings comparing with each other from different aspects. The adoption of which method will depend on vari- ous motivations/situations/needs specifically. 13 With the aforementioned methods and achievements, there are still challenges on the following aspects [30]: • Hierarchical game. There have been extensive researches on simple game models that characterize the behaviors of the power providers and consumers [24, 29, 52, 72, 77]. However, hierarchical games are seldom researched on. Hierarchical game structure can characterizes not only the game relationship among the players within each side of the supply or demand, but also the higher-level leader-follower inter- action between the supply side and demand side. • Communication impact. Usually the communications between the power provider and users are assumed to be error-free and cost-free [27, 61], and most studies do not consider the impact of different in- formation sharing mechanisms among the user side. In practice, users may exchange information or form coalitions with other users. Mean- while, fake information during sharing and private information steal- ing are following issues. • Coupled constraints. Demand response management problems usually involve temporally or spatially coupled constraints. For example, users may have certain tasks that need to be finished within a time period, such as dishes to be washed by the end of today. 14 Hence the aggregate energy consumption should be not less than a threshold before a deadline. Besides, some constraints that are looked like hour-by-hour are actually coupled across time horizons. For example, the indoor temperature of the current hour is usually not only determined by current power consumption, but also by the power consumptions in previous time horizons. Before-mentioned are temporally coupled constraints. For a group-consumer with several energy subsystems, the energy consumptions may have spatially coupled constraints among the subsystems. A microgrid that is confined within certain geographical area also has spatially coupled constraints of local generation capacities. Moreover, if energy storage devices such as PHEVs are considered, due to the capacity of batteries and mobility of vehicles, both temporally and spatially coupled constraints are involved. With these coupled constraints, the optimization problem cannot be directly tackled, especially in a distributed manner. 1.2 Outline The dissertation address the aforementioned challenges caused by hierarchical structure, communication impact and spatially temporally cou- pled constraints in demand response management of smart grid system, and focus on the Game-based and Lagrangian-based distributed optimiza- 15 tion methods based on different scenarios. Specifically, Chapter 2 and Chapter 3 focus on the Game-based meth- ods, while Chapter 4 and Chapter 5 utilize the Lagrangian-based methods. Although both using Game-based methods, Chapter 2 focus on selfish goal (Nash Equilibrium), while Chapter 3 aims at global optimization (Pareto Optimum). Meanwhile, Chapter 4 and Chapter 5 address more on the spa- tially or temporally coupled constraints. Chapter 4 handles both spatially and temporally coupled constraints but requires the prediction of the next- day electricity price and outdoor temperatures, while Chapter 5 focus on temporally coupled constraints with only next-hour price required and re- flects the pre-heating effect more clearly. Each chapter of the dissertation is organized as follows: In Chapter 2, the overall demand response problem between the sup- ply side and the demand side is formulated as a bi-level game: a consumer- level noncooperative game and a one-leader-one-follower Stackelberg game between the provider-level and the consumer-level. Each participant focus on selfish goal while consumers share information with each other through a designed information sharing platform for privacy protection. Nash Equi- librium for the noncooperative game and Stackelberg Equilibrium for the Stackelberg game are proved theoretically, and numerical results show that information sharing brings win-win results among all consumers. In Chapter 3, we zoom in to look at a specific consumer, which in 16 practice may consist of multiple residential or commercial buildings, where global optimization of the overall system is what we want to achieve. In- stead of Nash Equilibrium in normal games, Pareto Optimum is achieved through design of a series of sliding window games and based on the dis- tributed learning algorithm. The global utility is decomposed to each build- ing’s payoff, and each building learns to play a part of the optimal solution. Overall, the distributed learning algorithm can achieve a global sub-optimal solution, i.e., the solution converges to the discretized centralized optimal solution with a probability approaching one. In Chapter 4, spatially and temporally coupled constraints are fo- cused on in the multi-building energy management problem. The temper- ature comfort constraints are sophisticatedly converted to temporally cou- pled constraint, and then the primal cost minimization problem is reformu- lated and decoupled into subproblems of each building and each time slot via Lagrangian dual decomposition. The strong duality of the primal and dual problem is proved, and distributed algorithms are proposed based on the subgradient projection method which can achieve the convergence to the global optimum. Chapter 5 address the temporally coupled constraints focusing on the single building scenario, which servers as a practical extension of Chap- ter 4. Instead of the necessity of day-ahead electricity price and outdoor 17 temperature prediction in Chapter 4, which is hard to be accurate in prac- tice, in Chapter 5, the temporally-decoupled algorithm explicitly indicates when to take advantage of pre-heating/cooling for electricity cost reduc- tion, with only the next-hour electricity price required. Compared with the intuitive strategy, real-data case studies demonstrate that our proposed al- gorithm could lead to significant economic savings, which is practically ap- plicable to cost-effective smart buildings. Chapter 6 gives the concluding remarks for the dissertation. 18 Chapter 2 Bi-level Game for Demand Response with Informa- tion Sharing among Consumers 2.1 Introduction By taking advantages of widely deployed smart meters and two-way communication facilities of smart grid, demand response management, aiming to change the electricity usage patterns of end-users with pricing strategies of the supply side, can lead to a significant improvement of gird reliability and efficiency. Demand response management is an effective means for leading load to adapt to generation, and thus reduce the expensive operating cost from standby generators and defers the capacity growth. It will increase the reliability of the power system, enhance the transparency and efficiency of the electricity market, and improve the common financial interests for both the power utility and end-users. Environmentally, by enabling a more efficient utilization of current grid capacity, the generating emissions will be reduced and environmental impacts will be alleviated [30, 94]. Among all the pricing schemes in demand response management, real-time pricing (RTP) has been widely considered to be the most efficient 19 and economic means, where the electricity price varies at every time in- terval (usually every hour or fifteen minutes) [30]. The management goal includes shaping the demand profile, reducing the peak demand, flattening the demand curve, and minimizing the generation and consumption cost. The demand response problem is usually defined and solved within microgrids, which are autonomous subsystems of smart grid within local geographical areas that can operate independently from bulk generation. In such a system, the interactive decision-making process of the power provider and consumer involves a game relationship, i.e., each one’s benefit is dependent on not only his strategy but also all other players’ strategies. Therefore, game theory can be leveraged to model the interactive decision-making process of the power provider and consumer [87]. There have been extensive researches on various single-level game models that characterize the behaviors of the power providers or con- sumers. For example, Ibars et al. [52] and Nguyen et al. [77] studied the demand response game focusing on energy cost minimization and peak-to-average ratio minimization. Mohsenian-Rad et al. [72] and Deng et al. [29] studied the noncooperative game among residential consumers. Chen et al. [24] studied the electricity market from the social perspective, aiming at optimization of the social welfare of the entire system. Wu et al. [103] focused on PHEV charging and discharging scenarios. Maharjan 20 et al. [67] and Chai et al. [22] studied the scenario where multiple power providers interact with multiple consumers. However, hierarchical games are seldom researched on. Hierarchi- cal game structure can characterizes not only the game relationship among the players within each side of the supply or demand, but also the higher- level leader-follower interaction between the supply side and demand side. Besides, there is a lack of study on comparing different information shar- ing mechanisms in the game, while information symmetry and sharing are crucial in game theory. To address the aforementioned problems, in this chapter, we focus on a microgrid system with one power provider and multiple consumers, and model the overall demand response game by a bi-level model, com- prising (1) a consumer-level noncooperative game and (2) a one-leader-one- follower stackelberg game between the provider-level and consumer-level. The structure has good extendability in terms of adding any game on the supply side or changing any game on the demand side, as long as the game satisfies similar good properties. In addition, we have designed an infor- mation sharing platform for consumers and studied the case when all con- sumers share their demand information. Numerical results are provided to illustrate the win-win effectiveness of information sharing among con- sumers. The rest of this chapter is organized as follows. The system model is 21 described in Section 3.2. In Section 2.3, we formulate the demand response problem as a bi-level game and prove the existence of equilibrium. In Sec- tion 2.4, we study the case with information sharing and design distributed algorithms. Numerical results are provided in Section 5.5 and conclusions are drawn in Section 5.6. 2.2 System Model 2.2.1 Two-way Communication Infrastructure Power Provider Smart Meter User 2 Smart Meter User N Smart Meter User 1 Air conditioner Refrigerator Washer Local Area Network (LAN) Power Line … … Figure 2.1: Two-way communication infrastructure. Consider a microgrid system of one power provider and N con- sumers. As shown in Fig. 2.1, the infrastructure of the system contains two layers. On the power layer, each user connects to the power provider via the power line. On the communication layer, users connect with each 22 other and the power provider via the local area network (LAN). Through the LAN, the real-time two-way communication between the power provider and each user becomes feasible. In addition, users can share their information with each other via the LAN conveniently. In smart grid, the computation and optimization is implemented by programmed computers and smart meters. On the supply side, usually the pre-programmed computers are responsible for computing and implement- ing the real-time pricing strategies. On the demand side, each residential consumer is equipped with a smart meter, which can be pre-programmed to do computation in response to the real-time price, and to take automatic control of all household appliances based on the computation results. 2.2.2 Cost Function for Power Provider The cost function C(s) models the expense of supplying s unit of energy by the power provider. Demand response accommodates any form of cost functions as long as they satisfy the following two properties [30]. Property 2.1. Increasing: the cost always increases when the supply amount increases. C(s 1 )<C(s 2 ); 8s 1 <s 2 : (2.1) Property 2.2. Strictly convex: the marginal cost always increases when the 23 supply amount increases. @C(s 1 ) s 1 < @C(s 2 ) s 2 ; 8s 1 <s 2 : (2.2) The piece-wise linear function and quadratic function are two com- mon choices. In this chapter, we consider a quadratic cost function [90]: C(s) =as 2 +bs +c; (2.3) wherea> 0,b 0, andc 0 are three pre-determined parameters. 2.2.3 Gain Function for Power Consumer The gain functionG j (d j ) models the power consumerj’s satisfaction degree obtained by consumingd j unit of energy. The gain function should be nondecreasing and concave, i.e., it is increasing before the energy con- sumption reaches a desired level, and gradually gets saturated when the desired level is satisfied [30]. In this chapter, we consider a quadratic gain function [90]: G j (d j ) = ( ! j d j j 2 d 2 j ; 0d j < ! j j ! j 2 2 j ; d j ! j j : (2.4) where! j > 0 and j > 0 are pre-determined parameters, which vary among consumers, and also vary for the same consumer during different times of a day. We can see this gain function corresponds to a decreasing marginal benefit: @ 2 G j (d j ) @ 2 d j < 0; (2.5) 24 when 0d j < ! j j . 2.3 Bi-level Game Formulation 2.3.1 Supply Side: Leader-level Letl denote the total energy demand from all consumers, i.e., l = X j2M d j ; (2.6) whereM denote the set of all consumers. For the power provider, the profit from supplying s unit of electricity at the price of p is calculated as the revenue minus the cost function, i.e., P p (s;p) =plC(s): (2.7) The local optimization problem at the supply side is: max s;p P p (s;p) (2.8) s:t: sl p 0: It can be seen from (2.7) that once the total demandl is fixed, the increasing of the supply amount s will result in increasing of the cost and thus de- creasing of the profit. Therefore, the provider tends to stay at the minimum supply amount that matches exactly with the demand s =l: (2.9) 25 Substituting (2.9) into (2.7), we have P p (s;p) =psC(s): (2.10) The price is chosen to maximize (2.10): @P p (s;p) @s =p @C(s) @s = 0)p = @C(s) @s s=l : (2.11) Substituting (2.3) into (2.11), we have p(l) = 2al +b: (2.12) The price described in (2.12) is called the market-clearing price [24], which is the local optimal choice of the power provider under the market-clearing condition. 2.3.2 Demand Side: Follower-level Power Provider User 1 User N User j price demand price demand price demand … … Figure 2.2: Without information sharing among consumers. Each consumer can respond to the power provider based on only the price information. As shown in Fig. 2.2, each user only communicates with 26 the power provider to exchange the price and demand information. There is no communication or information sharing among consumers. Power Provider User 1 User N User j price demand price demand price demand … … 1 d j d N d j d 1 d N d Figure 2.3: With information sharing among consumers. In reality, consumers can choose whether to share their demand in- formation with each other or not. In this chapter, we focus on the scenario where all consumers share their demand information and all shared infor- mation are authentic, as shown in Fig. 2.3. More complicated situations such as partial population participation or cheating will be considered as potential directions in our future work. Note that although consumers share demand information, the objec- tive of each individual remains selfish. Each user still aims at maximizing his own gain while minimizing his payment. With the rational and selfish assumption, it is still a noncooperative game among consumers. We can utilize noncooperative game theory for building the model. 27 2.3.2.1 Noncooperative Consumer Game The noncooperative gameG = n M;D;fP j ()g j2M o among con- sumers consists of three components: 1. Players:M =f1; 2; ;Ng is a finite set of players. Each consumer in the system is a player. 2. Strategies:D = M j D j is the strategy space of all players in the game. Each playerj2M chooses a demand strategyd j from his strategy set D j . Letd j = (d 1 ; ;d j1 ;d j+1 ; ;d N ) denote the demand strate- gies of all consumers butj. We can write (d j ;d j ) for the overall de- mand profiled. 3. Payoff functions: the playerj’s payoff is determined by the demand profiled. Each selfish and rational playerj2M choosesd j according to the other players’ strategiesd j to maximizeP j (d j ;d j ). For each consumer, the payoff from consumingd j unit of energy at the price of p is calculated as the gain function minus the payment, i.e., P j (d j ;d j ) =G j (d j )p(l)d j : (2.13) Definition 2.1. Nash Equilibrium (NE) [42]: A strategy profile d = d j ;d j is called NE if and only ifP j (d )P j d j ;d j ,8j2M,8d j 2D j . 28 Definition 2.2. An S-modular game restricts the payoff functionsfP j ()g such that for8j2M either of the following is satisfied [76]: 8 > > < > > : @ 2 P j (d) @d j @d i 0; 8i6=j2M (2.14a) @ 2 P j (d) @d j @d i 0; 8i6=j2M: (2.14b) Lemma 2.1. For the S-modular game, when NE exists and is unique, best response can be used to drive the solution converging to NE [7]. Theorem 2.2. NE of the noncooperative consumer game we formulated in this chapter exists and is unique. Proof. We can prove that the noncooperative consumer game is a strictly concaven-player game, by showing: 1. The strategy setsfD j g j2M are convex, closed, and bounded. This is al- ways true with the bounded strategy sets 0d j d j ;8j2M, where d j denotes the maximum energy consumption level of consumerj. 2. The payoff functionsfP j ()g j2M are continuous ind and concave in d j for a fixed value ofd j . It is easy to see that P j () is continuous and differentiable at every point ofd. The concavity can be shown as follows. Substituting (2.12) into (2.13), we have P j (d) =G j (d j ) (2al +b)d j ; 29 wherel = P j2M d j . Taking its first-order derivative overd j , we have @P j (d) @d j =G 0 j (d j ) [2a(l +d j ) +b]: As aforementioned in Section 2.2.3, the gain function G j () is non- decreasing and concave, i.e., G 0 j (d j ) 0 and G 00 j (d j ) 0. Thus the second-order derivative of the payoff function is @ 2 P j (d) @d 2 j =G 00 j (d j ) 4a< 0: i.e., the payoff function is strictly concave w.r.t. d j . Therefore, the noncooperative consumer game is a strictly concave n-player game and according to [86], NE of the game exists and is unique. Theorem 2.3. NE of the noncooperative consumer game we formulated in this chapter can be achieved by best response. Proof. We have proved in Theorem 2.2 that NE of the noncooperative con- sumer game exists and is unique. Then we calculate the second-order par- tial derivative ofP j (). For8j2M, we have @ 2 P j (d) @d j @d i =2a< 0; 8i6=j2M: Based on Definition 2.2, the game corresponds to the S-modular game. By Lemma 2.1, we can use best response to drive the solution converging to NE. 30 2.3.2.2 Best Response Best response means that at each iteration, each consumer adapts his strategy to the strategies of others to maximize his own payoff. Mathmati- cally, each consumer aims at solving d j = arg max d j 2D j P j (d j ;d j ); (2.15) whereP j () is defined in (2.13). 2.3.3 Leader-Follower Interaction: Stackelberg Game In fact, the whole electricity market is a one-leader multi-follower game, where the power provider leads the game by moving first, i.e., defin- ing the electricity price, and then all consumers make their moves following the price afterwards, i.e., deciding the energy demand, and meanwhile con- sumers may share information with each other on the follower level. The main difference of a leader-follower game from a normal game lies in the situation of asymmetry of information. The leader chooses his strategy in advance, and then the follower makes the move accordingly. Therefore, many concepts and strategies in normal games are no longer suitable. For example, the equilibrium in leader-follower games may not satisfy NE conditions. In this chapter, the overall game between the power provider and all consumers is now modeled by a bi-level game, comprising (1) a follower- 31 level noncooperative game and (2) a one-leader-one-follower Stackelberg game. This model is due to the special structure of the game. On the leader side, the power provider does not care about the detail of how each con- sumer behaves. The provider’s supply and price are only affected by the aggregate behavior of all consumers, as shown in (2.9) and (2.12). There- fore, we can represent the response of the whole follower level by one single follower with strategyl = P j2M d j . Another reason for this model formulation is due to the existing re- search on Stackelberg games. Stackelberg games are sourced from and have been extensively studied on the one-leader-one-follower case. For multi- leader-multi-follower cases, there exists no special focus on the interactions among followers. Since as long as NE exists in the leader-level game, the game structures and equilibrium theories are well-formed. For more com- plicated cases where followers interact/cooperate/share information with each other, even for the single leader case, the equilibrium theories are not trivial and have not been well studied yet. 2.3.3.1 One-Leader-One-Follower Stackelberg Game The Stackelberg demand response game = ;S; P (); L () is modeled as follows: 1. Players: =fP;Lg. The player set has two players: the playerP is 32 the leader, representing the power provider, while the playerL is the follower, which is a virtual player representing all consumers. 2. Strategies: two sets of strategiesP andL. The playerP ’s strategy is the pricep2P, and the playerL’s strategy is the aggregate demand l2L. 3. Payoff functions: two sets of payoff functions P (): PL 7! R and L (): P L 7! R N , where P () is described in (2.10), L () = [P 1 (); ;P N ()] is the payoff vector of all consumers. Definition 2.3. Best Response SetR L (p): The best responsel (p)2R L (p) of the follower after observing the leader’s strategyp isl (p) = P j2M d j (p). Definition 2.4. Stackelberg Equilibrium (SE): A pair of strategies (p S ;l S )2 PL is called SE if l S 2R L (p S ) and P (p;l) P (p S ;l S ) for every pair (p;l) withl2R L (p) [18]. Lemma 2.4. If the setsP andL are compact metric spaces, and the payoff functions P () and L () are continuous, SE always exists [18]. Theorem 2.5. SE of the one-leader-one-follower demand response Stackelberg game we formulated in this chapter exists. Proof. Following Lemma 2.4, we need to prove: 33 1. The setsP andL are compact metric spaces. This is always true by observing that 0 p p (where p denotes the supply capacity of the power provider) and 0 d j d j ;8j2M are closed and bounded subsets of real numbers, thereforeP andD j are compact sets. SinceL is mapping from the compact setsD 1 ; ;D N by the addition opera- tor, it is also compact. 2. The payoff functions P () and L () are continuous. It is obvious that as described in (2.12), P () is continuous. The continuity of L () is not that straightforward, but can be proved by open subset concepts in topology [35]. 2.4 Distributed Algorithms In this section, we focus on the scenario where all consumers share their demand information with each other. However, in practice, it is ob- vious that no one would like to share his information if he can obtain the information of others without sharing his own. Therefore the equilibrium is no one obtains any other information. This is the reason why information sharing is not the usual case. To avoid the above-mentioned situation, an information sharing platform can be set up, as shown in Fig. 2.4, which forces each user to con- 34 Power Provider User 1 User N User j price demand … … Demand Information Sharing Platform contribute demand retrieve aggregate demand Figure 2.4: With demand information sharing platform. tribute his own demand strategy in order to retrieve the aggregate demand information of others. We derive the game solution for the situation that (1) all consumers participate and (2) everyone contributes his real demand strategy, and design distributed algorithms for the supply side and demand side. Specifically, we calculate the best response of the virtual follower in the Stackelberg game by achieving NE of the noncooperative game among consumers, and let the leader choose his optimal action accordingly. NE of the noncooperative consumer game is calculated by deriving the best response strategy of each consumer. The above calculation processes re- peat until the overall demand response game converges. The convergence condition is that the power provider and all consumers do not revise their strategies any more. We begin with solving the best response problem for each consumer 35 as formulated in (2.15), which could be directly solved by letting the first- order derivative equal zero, i.e., @P j (d j ;d j ) @d j =G 0 j (d j ) 2a(l +d j )b = 0 ) d j = arg max d j 2D j P j (d j ;d j ) =d j (l;a;b): (2.16) However, this method may not be applicable in practice since each con- sumer needs to know the power provider’s cost function parametersa and b to solve the problem. In general, such parameter information is private and no power provider would like to reveal it. Therefore, the consumers may not have sufficient information to solve the problem. Due to the strict concavity of the payoff function, a gradient pro- jection method [99] is designed to converge to the closed-form solution of (2.16) in an iterative manner, which does not require power provider to re- veal such private information, i.e., d k+1 j = " d k j + @P j d k j ;d j @d k j # D j = d k j + G 0 j (d k j )p k p l d k j D j ; (2.17) where > 0 is the step size which adjusts the convergence rate, k2 N + denotes the index of iterations, andfg D j denotes the projection onto the strategy setD j . On receiving each consumer’s demand strategy, the information sharing platform computes the aggregate demand by (2.6) and shares it 36 with all consumers as well as the power provider. Based on the aggregate demand, the power provider updates his supply and price by (2.9) and (2.12) respectively, and broadcasts the updated price to all consumers. On receiving the updated price from the power provider and the aggregate demand from the information sharing platform, each consumer updates his demand strategy by (2.17). The iteration processes repeat until the game converges, i.e., no one revises his strategy any more. The algorithms for the supply side and demand side as well as the information sharing platform are summarized as Algorithm 1, 2, and 4, respectively. Algorithm 1: : executed by the power provider. 1 Initialization; 2 repeat 3 Receive demandd k j from all consumersj2M; 4 Update supplys k by (2.9); 5 Update pricep k by (2.12); 6 Broadcast updated pricep k to all consumers; 7 until price does not change; 37 Algorithm 2: : executed by each consumerj2M. 1 Initialization; 2 repeat 3 Receive pricep k from power provider; 4 Receive aggregate demandl k from information platform; 5 ifk> 1 then 6 Record price change p =p k p k1 ; 7 Record aggregate demand change l =l k l k1 ; 8 Compute p=l; 9 else 10 Set p=l = 0; 11 end if 12 Update demandd k+1 j by (2.17); 13 Communicate the updated demandd k+1 j to power provider; 14 Communicate the updated demandd k+1 j to information platform; 15 until demand does not change; Algorithm 3: : executed by the information sharing platform. 1 Initialization; 2 repeat 3 Receive demandd k j from all consumersj2M; 4 Compute aggregate demandl k by (2.6); 5 Broadcast aggregate demandl k to all consumers; 6 until aggregate demand does not change; 38 2.5 Numerical Results In this section, we provide numerical examples to evaluate the per- formance of the proposed bi-level demand response game with information sharing among consumers. For ease of illustration, we consider a simple case for a microgrid system with one power provider and three consumers. It can be extended to more users, with similar results. The simulation parameters for the power provider isa = 0:1,b = 0:5, and c = 0, i.e., the supply cost function is C(s) = 0:1s 2 + 0:5s. For the parameters of the consumer gain functions, we first fix and vary!, then fix ! and vary to show cases of different users. The convergence error tolerance of the demand response game is fixed as = 1e 6 for all following cases. 2.5.1 Convergence Analysis Firstly, the performance of the gradient projection method in Algo- rithm 2 is analyzed. In Fig. 2.5, the solid line represents the iteration process obtained from the gradient projection method in (2.17), while the dotted line denotes the result obtained from the closed-form solution of the best response problem in (2.16). It needs to be noticed that the closed-form so- lution in (2.16) requires the power provider’s function parametersa andb, and thus is not practically applicable. From Fig. 2.5(a), we can see that all demand and price decisions via the gradient projection method converge 39 0 5 10 15 20 0.5 1 1.5 2 2.5 3 3.5 4 Iteration Demand/Price d1 d2 d3 p d1 * d2 * d3 * p * (a) Demand/Price 0 5 10 15 20 −3 −2 −1 0 1 2 3 4 Iteration Payoff P1 P2 P3 P1 * P2 * P3 * (b) Payoff 0 5 10 15 20 −2 0 2 4 6 8 10 12 Iteration Payoff/Profit/Welfare ProviderProfit AvUserPayoff SocialWelfare ProviderProfit * AvUserPayoff * SocialWelfare * (c) Profit/Payoff/Welfare Figure 2.5: Gradient projection solution (2.17) converged to the closed-form solution (2.16). 40 to the closed-form decisions in only a few steps, i.e., the same equilibrium of the demand response game. Correspondingly from 2.5(b) and 2.5(c), the payoff of three users, the profit of the power provider and the social welfare are the same for both methods. Fig. 2.5 is simulated with the configuration parameters 1;2;3 = 0:5;! 1 = 2:5;! 2 = 3;! 3 = 3:5, initial valuesd 1 = 3:0;d 2 = 3:5;d 3 = 4:0;p = 2:6 and step size = 0:5. Similar observations and the same conclusion can be drawn through numerous tests with different configuration parameter settings, i.e., all decisions via the gradient projection method converge to the closed-form ones. Fast convergence can be achieved in reasonable steps with a proper chosen of step size, and we provide the convergence rate analysis with respect to the step size as follows. 41 0 50 100 150 200 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 Iteration Price γ=1.500 γ=0.500 γ=0.100 γ=0.050 γ=0.025 Figure 2.6: The impact of step size on the convergence rate of the gradient projection method. 42 Secondly, we vary the iteration step size = 0:025, 0:050, 0:100, 0:500 and 1:500 respectively, to study the convergence of the gradient projection method under different step size. The result is shown in Fig. 2.6. We can see that as long as the step size is too large, it only affects the convergence speed, but does not impact the converged point. When the step size is small as 0:025, the algorithm needs more than one hundred iterations to converge. As the step size increases, the algorithm converges in much fewer steps. However, when the step size is larger than 1:500, the output will oscillate and finally diverge. In the following simulations, we fix = 0:500 to ensure a fast convergence with 10 20 steps. 2.5.2 Equilibrium Analysis In this section, we would like to analyze the impact factors of the game equilibrium point. In Fig. 2.7, three consumers’ gain function param- eters are fixed at 1;2;3 = 0:5 and! 1 = 2:5,! 2 = 3,! 3 = 3:5, while the initial demand of each consumer is uniformly randomized within the range of d 1 :[1:0; 3:0], d 2 :[2:0; 4:0] and d 3 :[3:0; 5:0], respectively. This parameter setup is to study how the initial demand affects the convergence point. Fig. 2.7 shows that all demand and the price will finally converge to the same equi- librium point regardless of any initial value. This indicates that, as long as the power provider and consumers are fixed, the equilibrium is determined and is not affected by the initial values. 43 0 5 10 15 20 1 1.5 2 2.5 3 Iteration Demand 1 d1 0 =2.3102 d1 0 =1.9967 d1 0 =2.1705 d1 0 =1.5102 d1 0 =2.7818 d1 0 =1.2772 d1 0 =2.6814 (a) Consumer 1 0 5 10 15 20 1.5 2 2.5 3 3.5 4 Iteration Demand 2 d2 0 =2.3252 d2 0 =3.9195 d2 0 =2.4476 d2 0 =3.0119 d2 0 =3.9186 d2 0 =2.2986 d2 0 =2.5086 (b) Consumer 2 0 5 10 15 20 2.5 3 3.5 4 4.5 5 Iteration Demand 3 d3 0 =3.2380 d3 0 =3.6808 d3 0 =4.5025 d3 0 =4.3982 d3 0 =4.0944 d3 0 =3.5150 d3 0 =4.6286 (c) Consumer 3 0 5 10 15 20 1.6 1.8 2 2.2 2.4 2.6 2.8 3 Iteration Price p 0 =2.0747 p 0 =2.4194 p 0 =2.3241 p 0 =2.2841 p 0 =2.6590 p 0 =1.9182 p 0 =2.4637 (d) Power provider Figure 2.7: Initial value does not affect equilibrium point. 44 0 5 10 15 20 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 Iteration Demand 1 ω 1 =2.5, α 1 =0.6 ω 1 =2.5, α 1 =0.5 ω 1 =2.5, α 1 =0.4 (a) Consumer 1 0 5 10 15 20 1.5 1.6 1.7 1.8 1.9 2 2.1 2.2 2.3 Iteration Demand 2 ω 2 =3.0, α 2 =0.6 ω 2 =3.0, α 2 =0.5 ω 2 =3.0, α 2 =0.4 (b) Consumer 2 0 5 10 15 20 1.5 2 2.5 3 Iteration Demand 3 ω 3 =3.5, α 3 =0.6 ω 3 =3.5, α 3 =0.5 ω 3 =3.5, α 3 =0.4 (c) Consumer 3 0 5 10 15 20 1.35 1.4 1.45 1.5 1.55 1.6 1.65 1.7 1.75 1.8 Iteration Price all users α=0.6 all users α=0.5 all users α=0.4 (d) Power provider Figure 2.8: Consumer parameter affects equilibrium point. 45 In Fig. 2.8, we fix three consumers’ functional parameters at! 1 = 2:5, ! 2 = 3, and! 3 = 3:5, while vary 1;2;3 by 0:4 0:6, to study the impact of the parameter on the equilibrium. From the figure, we can see that changing will lead to different equilibria, and is negatively related to the demand. Therefore, a smaller results in a higher energy demand. 0 5 10 15 20 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 2.1 2.2 Iteration Demand 1 α 1 =0.6, ω 1 =2.5 α 1 =0.6, ω 1 =3.0 α 1 =0.6, ω 1 =3.5 (a) Consumer 1 0 5 10 15 20 1.5 1.6 1.7 1.8 1.9 2 2.1 2.2 2.3 2.4 Iteration Demand 2 α 2 =0.5, ω 2 =2.5 α 2 =0.5, ω 2 =3.0 α 2 =0.5, ω 2 =3.5 (b) Consumer 2 0 5 10 15 20 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 Iteration Demand 3 α 3 =0.4, ω 3 =2.5 α 3 =0.4, ω 3 =3.0 α 3 =0.4, ω 3 =3.5 (c) Consumer 3 0 5 10 15 20 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 Iteration Price all users ω=2.5 all users ω=3.0 all users ω=3.5 (d) Power provider Figure 2.9: Consumer parameter! affects equilibrium point. In Fig. 2.9, we fix three consumers’ functional parameters at 1 = 0:6, 46 2 = 0:5, and 3 = 0:4, while vary! 1;2;3 by 2:5 3:5, to study the impact of the parameter ! on the equilibrium. From the figure, we can see that changing! will lead to different equilibria, and! is positively related to the demand. Therefore, a larger! results in a higher energy demand. 2.5.3 Performance Comparison In this section, we provide numerical results for the performance comparison between two scenarios. One is without information sharing among consumers, i.e., each consumer responds to the power provider based on only the price information. The solution to this case is denoted by solution 1 (no). The other is with information sharing among consumers, i.e., each consumer responds to the power provider based on the price information and the aggregate demand information of others. The solution to this case is denoted by solution 2 (share). Figure 2.10 illustrates the performance comparison details. Fig- ure 2.11 shows more simulation results when varies from 0:4 to 0:6, and ! varies from 2:5 to 3:5. In the bar chart, solution 1 (no) is denoted by green bars, while red bars for solution 2 (share). The performance comparison metrics include each user’s energy demand, each user’s payoff, all users’ average payoff, power provider’s profit, and social welfare. All figures show that each consumer’s payoff is improved by infor- mation sharing, while the power provider’s profit drops and the social wel- 47 0 2 4 6 8 10 1 1.5 2 2.5 3 3.5 4 Iteration Demand Demand1(share) Demand2(share) Demand3(share) Demand1(no) Demand2(no) Demand3(no) (a) Consumer demand 0 2 4 6 8 10 −0.5 0 0.5 1 1.5 2 2.5 3 3.5 Iteration Payoff Payoff1(share) Payoff2(share) Payoff3(share) Payoff1(no) Payoff2(no) Payoff3(no) (b) Consumer payoff 0 2 4 6 8 10 0 2 4 6 8 10 12 14 Iteration Payoff/Profit/Welfare ProviderProfit(share) AvUserPayoff(share) SociallWelfare(share) ProviderProfit(no) AvUserPayoff(no) SociallWelfare(no) (c) Power provider profit Figure 2.10: Performance comparison between solution 1 (no) and solution 2 (share). 48 1 2 3 4 5 6 7 8 0 2 4 6 8 10 12 α=0.4; ω 1 =2.5, ω 2 =3, ω 3 =3.5 Demand/Payoff/Social Welfare Payoff1 Payoff3 Social Welfare Demand3 Demand1 Demand2 Payoff2 Provider Profit (a) Fix! and vary 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 ω=2.5; α 1 =0.6, α 2 =0.5, α 3 =0.4 Demand/Payoff/Social Welfare Demand2 Demand3 Payoff1 Payoff2 Payoff3 Provider Profit Social Welfare Demand1 (b) Fix and vary! 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 9 10 α=0.5; ω 1 =2.5, ω 2 =3, ω 3 =3.5 Demand/Payoff/Social Welfare Demand1 Demand2 Demand3 Payoff1 Payoff2 Payoff3 Provider Profit Social Welfare (c) Fix! and vary 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 9 ω=3; α 1 =0.6, α 2 =0.5, α 3 =0.4 Demand/Payoff/Social Welfare Social Welfare Provider Profit Demand1 Demand2 Demand3 Payoff1 Payoff2 Payoff3 (d) Fix and vary! 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 9 α=0.6; ω 1 =2.5, ω 2 =3, ω 3 =3.5 Demand/Payoff/Social Welfare Demand1 Demand2 Demand3 Payoff2 Payoff3 Provider Profit Social Welfare Payoff1 (e) Fix! and vary 1 2 3 4 5 6 7 8 0 2 4 6 8 10 12 14 ω=3.5; α 1 =0.6, α 2 =0.5, α 3 =0.4 Demand/Payoff/Social Welfare Demand1 Demand2 Demand3 Payoff1 Social Welfare Provider Profit Payoff2 Payoff3 (f) Fix and vary! Figure 2.11: Performance comparison between solution 1 (no: green) and solution 2 (share: red). 49 fare slightly decreases. When comparing solution 2 (share) with solution 1 (no), we can see that the information at the supply side remains the same, while each consumer at the demand side gains more information by in- formation sharing. From each consumer’s perspective, by contributing his own demand strategy, he can retrieve the aggregate demand information of others and increase his own payoff. It is actually a win-win situation for all consumers since every user’s payoff is improved. Therefore, we can con- clude that information symmetry and sharing are crucial in game theory and are directly related to solution performance. 50 2.6 Summary In this chapter, we have studied a bi-level game model for the de- mand response problem with one power provider and multiple consumers. The bi-level game consists of (1) a consumer-level noncooperative game and (2) a one-leader-one-follower Stackelberg game between the provider-level and consumer-level. An information sharing platform is designed for the scenario where all consumers share their demand information with each other. We have proved the existence of equilibrium for both games and pro- posed distributed algorithms for the supply side and demand side as well as the information sharing platform. Numerical results are presented to il- lustrate the performance of the proposed algorithms and the effectiveness of information sharing for improving every user’s payoff. The bi-level game model and information sharing mechanism ana- lyzed in this chapter can be further extended to more complicated cases: the scenarios where multiple power providers and multiple consumers have interactions, or the scenarios where different information sharing mecha- nisms are considered such as partial population participation or cheating. 51 Chapter 3 Sliding Window Games for Cooperative Building Temperature Control Using Distributed Learning Method 3.1 Introduction Buildings are among the largest energy consumers in the power grid, which consume almost 70% of the total electricity generated in the US [101]. According to the data of energy consumption in European households [33], 68% of the building energy consumption comes from space heating or cool- ing, 14% from water heating or cooling, and 13% from electric appliances and lighting. Besides the accumulated energy use, buildings tend to have high demand in electricity simultaneously, which causes significant peak demand exertion on the grid [65, 62]. Therefore, under the big picture of demand side management [30, 72, 57, 29, 22], minimization of the energy cost from space heating or cooling in response to the diversity of electricity prices, instead of simply minimizing the energy consumption amount, is one of the main goals of advanced building control systems. Meanwhile, since people spend more than 80% of their time in build- ings, the thermal comfort in a working or living place is strongly related to 52 the occupants’ satisfaction and productivity. Most often, the improvement of the building comfort demands more energy consumption. Hence, one of the most important issues of smart and energy-efficient buildings is to balance the requirement of thermal comfort and energy usage [56]. In the literature of smart building HVAC (heating, ventilation and air conditioning) control system, the supervisory control level usually aims at choosing the optimal energy cost while maintaining a desired indoor comfort level. Recently, distributed temperature control for multiple smart buildings sharing common interest, such as a university or a company, received increasing attention. From a system designer’s perspective, it is desirable to have the group of buildings achieve the global benefit in a distributed cooperative way, such that each building could decide its own set-point temperature or energy consumption which leads to global optimum as a whole. The global benefit is consist of three aspects. Firstly, buildings are anticipated to cooperate with each other such that the total load of all build- ings stays below a certain threshold. The threshold comes from the distri- bution infrastructure capacity, such as the thermal capacity of transformers and feeders within the group. Once it is exceeded, a penalty will be added for compensation and adjustment. Secondly, it is desired to minimize the global energy cost, which is a function of the real-time electricity price and 53 total power load. Electricity price with higher value during peak hours and lower value during off-peak hours can be taken advantage of to reduce the consumption cost. Last but not least, there is a requirement for the real-time indoor temperature comfort level for each building, i.e., the desired tem- perature should not fall below a lower limit or above an upper limit. In this chapter, a weighted average of the aforementioned three aspects of benefit is formulated as the global utility function. The problem of the global utility function maximization can be solved by traditional convex optimization methods in a centralized way, where a central controller can be utilized to handle all the buildings. However, it is of great significance to solve the problem in a distributed manner, which has several advantages over centralized control. Firstly, a distributed control structure makes systems more reliable. In the case that a controller breaks down for one building, other buildings’ performance is not affected or only slightly affected in a distributed system. However, if a centralized controller breaks down, all buildings will be severely affected. Secondly, a distributed control system has a better expandability. The system can be constructed with a large scale and scattered in a large area. It provides a convenient infrastructure when a new building is built and becomes part of the existing control system. Furthermore, the optimization and computation load for each controller would be significantly reduced, 54 which allows for distributed computing and distributed storage. In this chapter, the distributed global utility maximization problem is formulated as a series of sliding window games, with each building be- ing treated as a player, the global utility being decomposed to each player’s payoff, and each game being played over a control horizon. In each game, a newly proposed distributed learning algorithm [70] in game theory is ap- plied, which teaches each player to play part of the Pareto Optimum by states transition. During the games, each player maximizes its own payoff based on the action it played and the payoff it received, without knowing others. Overall, the distributed algorithm can achieve a global sub-optimal solution, i.e., the solution converges to the centralized optimal solution with a probability approaching one. The remainder of this chapter is organized as follows. The central- ized model is described in Section 3.2. In Section 3.3, the multi-building decision making problem is reformulated based on game theory, the payoff functions are designed and the distributed learning algorithm is applied. Case studies are provided in Section 3.4 with three building hourly game and h-hour game respectively and conclusions are drawn in Section 3.5. 3.2 Centralized Model We consider a group of n buildings denoted by the set N = f1; 2;:::;ng and a centralized controller to perform an overall optimum 55 control behavior. The global economic objective function is: (L;P)= n X i=1 24 X t=1 c i (l i (t);p(t)) (3.1) where () denotes the aggregate daily electricity expense of all build- ings, L = L 1 ::: L n T denotes energy consumption matrix for the n buildings, and L i = l i (1) ::: l i (t) ::: l i (24) T denotes energy consumption column vector for building i, where l i (t) is the power con- sumption of building i at the tth time step. P = p(1) ::: p(24) T , where p(t) is the hourly electricity price per energy unit. The function c i (l i (t);p(t)) =c 1 p(t)l i (t) 2 +c 2 p(t)l i (t) +c 3 models the operation cost of buildingi with power loadl i (t) and pricep(t). At every time step, the temperature of each building need to stay within a comfortable range, and the total power load should stay below a threshold: T i lb T i (t)T i ub ; 8i2f1;:::;ng;t2f1;:::; 24g n X i=1 l i (t)L r ;8t2f1;:::; 24g The functional relationship of next time step indoor temperature pre- diction with respect to current time step indoor temperature, outdoor tem- perature and power consumption, can be described as [41] T i (t + 1) ="T i (t) + (1")(T OD (t) Kl i (t)) (3.2) 56 where" is the thermal time constant of the building, is a factor capturing the efficiency of the air conditioning unit,K is a conversion factor andT OD is the outdoor temperature. Since the indoor and outdoor temperature is usually known, once an energy consumption amount is determined, there is a corresponding anticipation of room temperature of next hourT i (t + 1). Therefore, the centralized model is as follows: min L n P i=1 24 P t=1 c i (l i (t);p(t)) s:t: n P i=1 l i (t)L r ;8t2f1;:::; 24g T i lb T i (t)T i ub ;8i2f1;:::;ng;t2f1;:::; 24g (3.3) In the centralized decision making problem, our objective is to minimize the total daily electricity cost of the group of buildings, decision variables are energy consumption amountl i (t) for every buildingi at every hour t, or equivalently temperature set-point for each building in every hour. Meanwhile, we would like each building to satisfy two constraints: the total power load in every hour does not exceed a certain threshold, and the temperature is within a comfort region. The centralized decision making problem can be solved using con- vex optimization techniques. In the problem, p(t), L r and T OD (t) are pre- known parameters. We can see the objective function is the sum of a set of quadratic functions, whose coefficients are usually positive, therefore being a convex objective function. As for the constraints, it is easy to see the total power load constraints are linear. If we transfer the temperature comfort 57 level constraints from functions ofT i (t) to functions ofl i (1);l i (2);:::l i (t), we can see that all constraints are linear. Therefore, many convex optimization solution algorithms such as interior point method can be used. However, due to the drawback of centralized control comparing with distributed control, here we aim at reformulating the problem into multi- building decision making problem. That means, each building can behave as an autonomous agent while they have interactions among the group, and we would like each building learn to play a part of the optimal solution, without having any information of the system as a whole or what the other buildings are doing. This leads us to deterministic game theory. 3.3 Game Reformulation and Distributed Cooperative Con- trol In this chapter, we will first reformulate this multi-building decision making problem as a game, treating each building as a player, and apply a newly proposed payoff based distributed learning algorithm [70] in game theory to solve this problem. The solution would be a near optimal solution, or in another word, the solution converges to the optimal solution with a probability that can be shown to be infinitely close to one. For building sliding window game, we can definey i (t) as y i (t) = [l i (t) l i (t +N p 1)] T (3.4) whereN p is the width of a fixed-width sliding window, and (N p 1) is the 58 length of the prediction horizon. At each time step, although an optimal power consumption strategy for the whole N p time intervals within the sliding window time frame is computed, only the solution of the current time interval should be adopted to output. This is because although the prediction of future inside tempera- ture based on Eq. 5.3 is valid for the next time-step, it will not be as accurate to be used when making a series of predictions. When predicting the in- side temperature of (t + 1)th time step, we would use the updated real-time inside and outside temperature information oftth time step, instead of the predicted inside temperature oftth time step based on (t 1)th time step, and the predicted outside temperature. The optimization procedure will be repeated and similar games will be formulated in subsequent time intervals as the window slides. One point needs to be noticed is, since the game theory tool cannot directly solve con- straint optimal decision making problem, in our game formation of next section, we form the constraints as penalty functions and add a little change. Another point is, since for long time frame and multiple buildings, the deci- sion space for the distributed learning algorithm is too large, in the simula- tion section, we first formulate the problem as hourly game, and then look at a multiple hour sliding window game. Now let’s look at the reformulation of the problem from the perspec- tive of game theory. 59 3.3.1 Games In the multi-building system, there is one game formulated at every time step. The first game of the day begins att = 1, in which each building acts as an autonomous agent and determines its schedule of the power con- sumption for time window fromt = 2 tot =N p + 1, while only the result of t = 2 will be output. Similarly the second game begins att = 2, followed by games repetitively until the last game ends at the end of the day. 3.3.2 Players In each game, a player is a building in the group. More specifically, the local supervisory controller of each building that determines the power consumption or temperature setpoint at every time interval is a player. The players are denoted as a finite set N = f1; 2; ;ng (3.5) 3.3.3 Actions The action of each player is determining a schedule of the power consumption amount for current sliding window time frame. Denote the action of playeri2N in thetth game asa i (t), then a i (t) =y i (t) = [l i (t) l i (t +N p 1)] T (3.6) where l i (t)2L i (3.7) 60 andL i is set of all possible power consumption choices. The action set of playeri is denoted as A i =L i L i | {z } Np (3.8) To make the action set A i finite, we discretize the power consumption set using a minimum threshold l i;max , a maximum threshold l i;max and a step size l i , so that l i = [l i;min : l i :l i;max ] (3.9) The joint action set of all the playersN = f1; 2; ;ng is denoted as A =A 1 A n (3.10) An action profile of all the players is defined as a(t) = (a 1 (t);a 2 (t); ;a n (t))2A (3.11) and a profile of all players’ actions other than playeri is defined as a i (t) = (a 1 (t); ;a i1 (t);a i+1 (t); ;a n (t)) (3.12) Thereforea(t) can also be rewritten as a(t) = (a i (t);a i (t)) (3.13) 3.3.4 Payo Functions In game theory, payoff function represents the benefit a player can obtain as a result of an action profile, and all the players would like to max- imize their payoffs, respectively. 61 In this chapter, we formulate each player’s payoff consisting three parts. Part 1 is the normalized operation cost penalty, which describes the economic benefit of a player. The lower the operation cost, the higher one’s payoff is. Part 2 is the temperature comfort level payoff, which is a refor- mulation of the temperature range constraint, to describe the benefit gains from staying within the comfortable temperature range or loss from falling outside it. Part 3 is the total power load payoff, reformulated from the to- tal power load constraint to introduce the penalty of exceeding the desired maximum peak threshold. Since these three aspects of the payoff function have different units and scales of dollar, kelvin and watt respectively, they are normalized into dimensionless quantities with the same scale of [0; 1]. 3.3.4.1 Part 1: Normalized Operation Cost Penalty u 1i (l i (t);p(t)) = 1 c i (l i (t);p(t)) c max The meaning can be understood as follows. The second term in the equation is current operation cost divided by the maximum possible operation cost, representing a normalized cost with scale [0; 1]. However, since the higher the cost, the lower the payoff, we use 1 minus that term, which is still a dimensionless quantity with scale [0; 1], to define payoff. 62 3.3.4.2 Part 2: Temperature Comfort Level Constraint Penalty u 2i (T i (t 1);T OD (t);l i (t)) = 1 1 + max n 0; T i (t) T ilb +T iub 2 2 T iub T ilb 2 2 o The meaning of the term within max is how far the temperature is from the middle point ofT i lb andT i ub comparing with the half distance T iub T ilb 2 . If the temperature stays within the comfort zone specified by T i lb and T i ub , the value would be 1, otherwise a positive value less than 1. This part has scale [0; 1] as well. 3.3.4.3 Part 3: Total Power Load Constraint Penalty u 3i (l i (t)) = 1 1 + max 0; n P i=1 l i (t)L r The meaning of the term within max is how far the total power load is beyond the constraint L r . If the power load summation stays within the threshold, the value would be 1, otherwise a positive value less than 1. This part has scale [0; 1] as well. 3.3.4.4 Payo Formula: Weighted Average of Three Parts u i (l i (t);p(t);T i (t 1);T OD (t)) =! 1 u 1i (l i (t);p(t))+! 2 u 2i (T i (t 1);T OD (t);l i (t))+! 3 u 3i (l i (t)) 63 3.3.5 Global Payo The global utility/global payoff/social welfare of time slott is usu- ally defined as the summation of all players payoffs at time slott, i.e., W t (L(t);p(t);T (t 1);T OD (t)) = X i2f1;:::;ng u i (l i (t);p(t);T i (t 1);T OD (t)) The daily global utility is defined as W = X t2f1;:::;24g W t Here in order to make all payoffs and utilities have the same scale [0; 1], we define the global utility of time slot as the average payoff of all players, i.e., W t (L(t);p(t);T (t 1);T OD (t)) = 1 n X i2f1;:::;ng u i (l i (t);p(t);T i (t 1);T OD (t)) and the daily global utility as the average hourly global utility over a whole day, i.e., W (L;p;T;T OD ) = 1 24 1 n X t2f1;:::;24g X i2f1;:::;ng u i (l i (t);p(t);T i (t 1);T OD (t)) 3.3.6 Strategy: A payo based distributed learning algorithm Step 1: Initialization Initialize each player with a state [ a i (t); u i (t);m i (t)] where a i (t) is the benchmark action, u i (t) is the benchmark payoff,m i (t) is the mood which can have value of content or discontent. Initially a i (t) is randomly chosen from action set, u i (t) is zero andm i (t) discontent. 64 Step 2: Update each player’s action by different probability functions depends on the player’s mood Ifm i (t) is content, a player chooses a new action according to the following probability distribution: P a i (t) i = c jA i j1 fora i (t)6= a i (t) 1 c fora i (t) = a i (t) (3.14) Ifm i (t) is discontent, a player chooses a new action according to the follow- ing probability distribution: P a i (t) i = 1 jA i j for everya i (t)2A i (3.15) Step 3: Calculate each player’s payoff according to payoff function Step 4: Update each player’s mood Ifm i (t) is content, and the player’s action remains the same, new mood is content; Ifm i (t) is content, but the action changes [ a i (t); u i (t);C] [a i (t);u i (t)] ! [a i (t);u i (t);C] with prob 1u i (t) [a i (t);u i (t);D] with prob 1 1u i (t) (3.16) Ifm i (t) is discontent, [ a i (t); u i (t);D] [a i (k);u i (t)] ! [a i (t);u i (t);C] with prob 1u i (t) [a i (t);u i (t);D] with prob 1 1u i (t) (3.17) 65 Step 5: Convergence condition Define a state profile as all players’ joint states. Count the frequency of each state profile. If the frequency of any state profile is larger than a pre- determined threshold such as 90%, we say the game converges to a global optimal point and stop the algorithm. Otherwise, go back to Step 2 and repeats. Theorem 3.1. Let G be an interdependent n-person game on a finite joint action space. Under the distributed learning algorithm, given any probability p < 1, if the exploration rate> 0 is sufficiently small, then for all sufficiently large timet, a2 arg max a2A W (a) = P i2N u i (a) with at least probability p [70]. 3.4 Case Studies In this section, we provide numerical examples to evaluate the performance of the proposed sliding window game formulation with distributed learning strategy. For ease of illustration, we consider a simple case for a three building energy consumption and temperature comfort cooperation problem. For simplicity but without loss of generality, we take the cooling scenario for example, while the heating scenario follows the similar way. 66 3.4.1 Hourly Game In hourly game, one game is played at the beginning of every hour. In each game, the players only focus on the current time slot and try to maximize the global utility of that time slot, following the strategy stated in last section. In each game, a player’s action is his energy consumption amount in current time slot, i.e., a i (t) =l i (t) His payoff can be calculated as u i (l i (t);p(t);T i (t 1);T OD (t)) =! 1 h 1 c i (l i (t);p(t)) cmax i + ! 2 " 1 1+max 0; T i (t) T ilb +T iub 2 2 T iub T ilb 2 2 # + ! 3 2 4 1 1+max 0; n P i=1 l i (t)Lr 3 5 And the global utility is W t (L(t);p(t);T (t 1);T OD (t)) = 1 n X i2f1;:::;ng u i (l i (t);p(t);T i (t 1);T OD (t)) By running the distributed learning algorithm described in last sec- tion as each player’s strategy, the game can converge to a global optimal point, i.e. an efficient state profile that can achieve the maximum of the global utility of that certain time slot. After convergence, we output the 67 optimal action profile to each corresponding high level controller to imple- ment control performance.The whole process is repeated every hour. The following are several simulation results for hourly optimization. In the figures 3.1 and 3.2, the red line represents the optimal solution ob- tained through exploring the whole solution set using centralized manner and the blue line represents the solution obtained by distributed learning algorithm. 1 2 3 4 5 6 7 8 9 10 11 12 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Global Utility: 12 hrs hours percentage Figure 3.1: Global utility distributed (blue) vs. centralized optimum (red). 68 Figure 3.1 shows that distributed learning algorithm can achieve near optimal global utility maximization solution. The total utility in that figure consists of three parts: The first part is the energy cost, which is a function of hourly price rate and hourly energy consumption amount, and is shown in the top three graphs in Figure 3.2; The second part is the total energy consumption constraint for the three buildings described by penalty function if beyond a threshold set as 1.5kwh, which is illustrated in the third graph of Figure 3.2; The third part is the temperature comfort level, which is described by penalty function if beyond the comfort temperature zone set as [67F, 79F], and is shown in the bottom part in Figure 3.2. 69 1 2 3 4 5 6 7 8 9 10 11 12 0 5 10 Price: 12 hrs hours $/kwh 1 2 3 4 5 6 7 8 9 10 11 12 0 5 10 15 Global Energy Cost: 12 hrs hours $ 1 2 3 4 5 6 7 8 9 10 11 12 0 0.5 1 1.5 Global Energy Consumption: 12 hrs hours kwh 1 2 3 4 5 6 7 8 9 10 11 12 80 85 90 Outdoor Temperature: 12 hrs hours F 1 2 3 4 5 6 7 8 9 10 11 12 65 70 75 80 85 Room Temperature 1: 12 hrs hours F 1 2 3 4 5 6 7 8 9 10 11 12 65 70 75 80 85 Room Temperature 2: 12 hrs hours F 1 2 3 4 5 6 7 8 9 10 11 12 65 70 75 80 85 Room Temperature 3: 12 hrs hours F Figure 3.2: Global utility components. 70 3.4.2 Sliding Window Game In sliding window game, one game is also played at the beginning of every hour. However, in each game, the players focus on an h-hour time horizon, trying to optimize the global utility of that time period. In this case, a player’s action in each game is an energy consumption vector, i.e., a i (t) = l i (t) l i (t + 1) ::: l i (t +h 1) T His payoff can be calculated as u i (l i (t);p(t);T i (t 1);T OD (t)) =! 1 h 1 c i (l i (t);p(t)) cmax i + ! 2 " 1 1+max 0; T i (t) T ilb +T iub 2 2 T iub T ilb 2 2 # + ! 3 2 4 1 1+max 0; n P i=1 l i (t)Lr 3 5 u i (l i (tt +h 1);p(tt +h 1);T i (t 1t +h 2);T OD (tt +h 1)) = 1 h t+h1 P k=t u i (l i (k);p(k);T i (k 1);T OD (k)) And the global utility becomes W tt+h1 (L(tt +h 1);p(tt +h 1);T (t 1t +h 2);T OD (tt +h 1)) = 1 nh P i2f1;:::;ng t+h1 P k=t u i (l i (k);p(k);T i (k 1);T OD (k)) In this sliding window h-horizon game, the global utility over a time period is maximized, so that buildings can perform pre-cooling or pre-heating before the peak-price period. 71 1 2 3 4 5 6 7 8 9 10 11 12 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Global Utility: 12 hrs hours percentage Figure 3.3: Sliding window size = 4 (black) vs. window size = 1 (blue). 72 Figure 3.3 shows a simulation comparison between hourly game and h-horizon game. The blue line represents the optimal solution attained by hourly game, while the black line represents game with a four-hour mov- ing window. In a Monte Carlo simulation of 100 times, four-horizon game achieves an average of 6.03% better daily utility than hourly game. 0 5 10 15 20 25 0 20 40 Temperature Room 1: 24 hrs hours C 0 5 10 15 20 25 0 20 40 Temperature Room 2: 24 hrs hours C 0 5 10 15 20 25 0 20 40 Temperature Room 3: 24 hrs hours C Figure 3.4: Outdoor (red), indoor (black) temperature and pre-cooling. Figure 3.4 shows the room temperature of three buildings under h- horizon game and corresponding distributed learning strategy. The black line represents the room temperatures, the red line represents the outdoor 73 temperature, while the green circle represents the precooling effect in off peak hours. The precooling effect is the result of maximization of payoff for each building, so that each building tends to consume more energy when the electricity price is lower. It is also the result of coordination among buildings, so that their precooling periods tend to offset one another. 3.5 Summary In this chapter, we formulate the distributed global utility maximiza- tion problem as a series of sliding window games, with each building being treated as a player, the global utility being decomposed to each player’s payoff, and each game being played over a control horizon beginning from the current time step. In each game, we apply a newly proposed distributed learning algorithm in game theory such that each building can learn to play a part of the optimal solution. During the games, each player maximizes its own payoff based on the action it played and the payoff it received, with- out knowing others. Overall, the distributed algorithm can achieve a global sub-optimal solution, i.e., the solution converges to the centralized optimal solution with a probability approaching one. The effectiveness of this method is demonstrated by simulation of a three building problem. Sub-optimal convergence property of the method- ology can be illustrated. Moreover, since the method utilizes sliding win- dow games with prediction horizon, pre-heating/cooling effect during off- 74 peak period and autonomous heating/cooling discharging during on-peak period can be observed. 75 Chapter 4 Distributed Optimization of Multi-building Energy System with Spatially and Temporally Coupled Constraints 4.1 Introduction Efficient building temperature control and energy consumption management can have huge impact on energy saving of the whole power grid. There have been extensive researches on energy-efficient building comfort management in a centralized manner. The different approaches could be roughly classified into two categories: conventional methods and computational intelligence methods [33, 93]. The conventional methods include proportional-integrate-derivative (PID) control, optimal control, adaptive control, and model predictive control (MPC). Among them, PID control is proposed to solve the problem of overshoots in thermostats with a dead zone [56]; optimal control is developed to maintain the control performance while further reducing the energy cost [106, 54]; adaptive control is developed to self-regulated and adapt to the climate conditions [71]; MPC is proposed to introduce prediction horizons and models for future disturbances [78, 96, 63, 64, 66]. The computational intelligence 76 methods include neural network approaches, fuzzy logic schemes, and evolutionary algorithms, which are mostly model-free and include user participation in the specification of the desired comfort. For example, Moon et al. [73] designed a thermal control logic framework with four thermal control logics including two predictive and adaptive logics using neural network models. Dounis et al.[32] proposed general guidelines for the design of the fuzzy logic thermal comfort regulator in situations that have multiple input/output controlled systems. In [53, 102], genetic algorithms (GA) based algorithms are proposed which are derivative-free and require minimal specific information. As the rising of distributed control and multi-agent system (MAS) re- search, approaching the original centralized problems in a distributed man- ner becomes a new research direction in the building energy control liter- ature [33]. The reason is that control engineers often face complicated het- erogeneous building environment such as multi-zone buildings and multi- building energy systems. In such systems, each building and each zone can be treated as an autonomous agent that makes its decision based on avail- able local information and behaves based on its own decision, while there is connection/interaction/coordination among the agents. From the system designer’s perspective, it is desirable to have the group of agents cooperate with each other in terms of energy consumption scheduling, such that the 77 global optimal status can be achieved [50, 48]. Among distributed methods, Anandalaskhmi et al. [9] presented peak demand management techniques for a smart community using differ- ent coordination mechanisms including decentralized resource allocation model and Pareto resource allocation model. Scherer et al. [91] developed a distributed model predictive control based energy management scheme, fo- cusing on the scenario with a shared renewable energy source in buildings. Kwak et al. [55] proposed a multi-agent system with user participation in commercial buildings, where agents communicate and negotiate with human occupants to save energy. Forouzandehmehr et al.[41] proposed distributed control of heating, ventilation and air conditioning (HVAC) systems based on game theory and a distributed learning algorithm to achieving Pareto optimality, while the theory guarantees probabilistic convergence as opposed to almost sure convergence. In spite of the variety of aforementioned distributed methods, spa- tially and temporally coupled constraints are seldomly addressed in dis- tributed multi-building energy management. Due to the distribution infras- tructure limit, such as the thermal limit of transformers and feeders within the system, the total load of all buildings needs to be stay below the capac- ity limit. This is a spatially-coupled constraint, i.e., the energy consumption of one building is spatially related to others. Besides, constraints that are 78 looked like hour-by-hour may also be coupled across time horizons. For ex- ample, the indoor temperature of the current hour is usually not only deter- mined by current power consumption, but also by the power consumptions in previous time horizons. This is a temporally-coupled constraint, i.e., in or- der to satisfy the temperature comfort level at a time slot, all of its previous energy consumptions are coupled together. To deal with the spatially and temporally coupled constraints, in this chapter, the temperature comfort constraints are sophisticatedly converted to temporally coupled constraint, and then the primal cost minimization problem is reformulated with spatially and temporally coupled constraints. The primal problem is decomposed via Lagrangian dual decomposition, which results in separable local optimization subproblems of each build- ing at each time slot. Distributed algorithms are derived for each building based on the local information via subgradient projection method, while an information sharing platform is designed for sending, receiving, and updat- ing the coordination signal among autonomous buildings. During the pro- cess, Lagrangian multipliers are serve as coordination signals, such that the global optimality of the combined local solutions can be guaranteed with the iterations of the coordination signals in the right directions. Numerical results are provided to illustrate the convergence of the proposed method and effectiveness of the control performance. 79 The rest of the chapter is organized as follows. The system model and problem formulation is described in Section 5.2. In Section 4.3, separa- ble local subproblems are formulated via the dual decomposition approach, and distributed algorithms are proposed based on the subgradient projec- tion method. Numerical results are provided in Section 5.5 and conclusions are drawn in Section 5.6. 4.2 Problem Formulation We consider a multi-building system consistingN buildings denoted by a setN = f1;:::;i;:::;Ng. The cycle of a day is divided into a set H =f1;:::;h;:::;Hg of time slots. The system is designed to minimize all buildings’ total daily electricity cost. Some important notations used in this chapter are summarized in Table 5.1. The cost minimization objective function is: min X C (X) = min x h i X i;h C h i x h i ; (4.1) where C (X) denotes the aggregate daily electricity expense of all build- ings, C h i () is a increasing, convex and differentiable function that models the operation cost of the HVAC unit of buildingi at time sloth, andx h i is the energy consumption of buildingi at time sloth. Letx i , x 1 i ;:::;x H i de- note the buildingi’s daily energy consumption schedule, andX , [x i ] i2N is all buildings’ daily energy consumption profiles. 80 Table 4.1: Summary of notations Symbol Definition Unit a i,N,N index, number, set of building(s) n/a h (ort),H,H index, number, set of time slot(s) n/a x energy consumption kWh x,x energy consumption lower, upper bound kWh C () cost function n/a l total load threshold kWh T indoor temperature °C T ,T indoor temperature lower, upper bound °C T 0 initial indoor temperature °C T od predicted outdoor temperature °C , ,K thermal constant n/a a,b,c model coefficient n/a f,g, , intermediate variable n/a ,, Lagrangian multiplier n/a x,,, vector ofx,,, n/a X, ,M matrix ofx,, n/a , , step size of,, iteration n/a k index of iteration n/a " error tolerance (stopping criterion) n/a a The unit of a quantity may be omitted in the rest of the chapter if it is specified here. 81 The energy consumption x h i of building i at time slot h is bounded by x h i x h i x h i 8i;h; (4.2) where x h i and x h i denote the lower and upper bounds for the building i’s energy consumption at time sloth, respectively. In addition, we would like all buildings to coordinate with each other such that the total load of all buildings stays below a certain threshold, de- noted byl h at time sloth, i.e., X i x h i l h 8h: (4.3) The threshold comes from the distribution infrastructure limit, such as the thermal limit of transformers and feeders within the group. This is a spatially-coupled constraint, i.e., the energy consumption of one building is spatially related to others. Finally, letT h i denote the temperature inside buildingi at the end of time sloth. We would like the temperature inside each building to always stay within a comfortable range, i.e., T h i T h i T h i 8i;h; (4.4) where T h i and T h i denote the lower and upper bounds for the building i’s indoor temperature at time sloth, respectively. 82 The indoor temperature evolves according to the following exponen- tial decay model [41], i.e., T h i = i T h1 i + (1 i ) T h od i K i x h i ; (4.5) where T h1 i denotes the building i’s indoor temperature at the end of the previous time slot,T h od for the predicted outdoor temperature at the end of the current time slot, andx h i for the buildingi’s energy consumption at the current time slot. The other parameters are the building i’s thermal con- stants: i is a thermal decay factor of buildingi, i is a factor capturing the efficiency of the HVAC unit to heat (+) or cool () the air inside building i, andK i is a conversion factor that is proportional to the performance co- efficient of the HVAC unit of buildingi divided by the total thermal mass. While the exponential decay model relates the constraints (4.4) to the power consumption by continuous controllable HVAC units, the on-off controlled scenarios are omitted here [49]. LetT 0 i denote the initial temperature inside buildingi, and T h od h2H for the predicted outdoor temperature, which are assumed to be known a day ahead. For ease of presentation, we simplify the model coefficients in (4.5) as T h i =a i T h1 i +b i T h od +c i x h i =f h i (x i ) + h i ; (4.6) 83 where 8 < : a i , i b i , 1 i c i , (1 i ) i K i ; and 8 > > < > > : f h i (x i ),c i h P t=1 (a i ) ht x t i h i , (a i ) h T 0 i +b i h P t=1 (a i ) ht T t od : Substitute (4.6) into (4.4), and thus (4.4) is rewritten as T h i f h i (x i ) + h i T h i 8i;h: (4.7) We can notice that the temperature constraint is actually a temporally-coupled constraint. That is, in order to satisfy the temperature comfort level of build- ingi at time sloth, all of its previous energy consumptions x 1 i ;:::;x h1 i are coupled together. Therefore, the global optimization problem is to minimize the aggre- gate daily electricity cost with spatially and temporally coupled constraints. Primal Problem: min X C (X) (4.8) s.t. (4.2); (4.3) and (4.7): 4.3 Dual Decomposition Approach Since the primal problem (4.8) has spatially-coupled constraints (4.3) and temporally-coupled constraints (4.7), which couple the energy 84 consumption of all buildings and across all time horizons, the problem cannot be directly tackled. However, by means of dual decomposition [16], we can decouple (4.8) into separable subproblems. Due to strong duality, the duality gap of the primal problem and the dual problem is zero, and thus solving the primal problem is equivalent to solving the dual problem. We formulate and solve the dual problem as follows. 4.3.1 Dual Problem Define the Lagrangian of the primal problem (4.8) as L (X; ;M;) = X i;h C h i x h i + X h h X i x h i l h ! + X i;h h i T h i f h i (x i ) h i + X i;h h i h f h i (x i ) + h i T h i i ; (4.9) where we relax the constraints (4.3) and (4.7) by introducing the Lagrangian multipliers h 0 and h i ; h i 0, and define, h h2H , , h i i2N;h2H , M , h i i2N;h2H , respectively. Note that (4.9) can be rearranged and simplified as L (X; ;M;) = X i;h C h i x h i + h x h i + X i;h h i h i f h i (x i ) + ; (4.10) 85 where , X h h l h + X i;h h h i T h i h i h i T h i h i i : We can further simplify (4.10) by noticing that X h h i h i f h i (x i ) = X h x h i g h i ( i ; i ); where i , 1 i ;:::; H i and i , 1 i ;:::; H i , if we define a intermediate variable as g h i ( i ; i ),c i T X t=h a th i t i t i ; or recursively, g H i ( i ; i ) =c i H i H i g h i ( i ; i ) =a i g h+1 i ( i ; i ) +c i h i h i : (4.11) Thus, the Lagrangian (4.10) can be further rewritten as L (X; ;M;) = X i;h C h i x h i + h +g h i ( i ; i ) x h i + : The dual objective function is the minimum of the Lagrangian over all buildings’ daily energy consumption profilesX: D (;M;) = min X L (X; ;M;) = X i;h S h i i ; i ; h + ; (4.12) whereS h i i ; i ; h is defined as the (i;h) th subproblem to be solved by buildingi at time sloth. 86 Subproblem:S h i i ; i ; h = min x h i C h i x h i + h +g h i ( i ; i ) x h i (4.13) s.t. x h i x h i x h i : The subproblem functionS h i i ; i ; h is concave as it is the point- wise infimum of affine functions of i ; i ; h [16, Sec. 3.2.3]. Thus, the dual objective functionD (;M;) is also concave. The dual problem is to maximize the dual objective function over the Lagrangian multipliers ;M;. Dual Problem: max ;M; D (;M;) (4.14) s.t. h i ; h i 0 8i;h h 0 8h: The optimal point of the primal problem (4.8) is lower bounded by any feasible value of the dual problem (4.14), i.e., for any ;M; 0 and any feasibleX satisfying (4.2), (4.3) and (4.7), we haveD (;M;) C (X). Theorem 4.1. The primal problem (4.8) has strong duality. Proof. The objective function of the minimization primal problem (4.8) is convex, and all the three inequality constraint functions (4.2), (4.3) and (4.7) are affine. Therefore, according to [16, Sec. 5.3.2], when minimizing over 87 a convex function with differentiable and convex constraints, the duality gap is always zero, provided one of the primal or dual problem is feasible. In this chapter, we only discuss the situation where the primal optimiza- tion problem is feasible, as we can always adjust the temperature comfort requirements (4.4) to meet the physical constraints (4.2) and (4.3). When strong duality holds, the optimal duality gap is zero, i.e., solv- ing the dual will give us the solution back to the primal. From ;M ; , the optimal solution of the dual problem (4.14), we can getX , the optimal solution of the primal problem (4.8), by solving the subproblem (4.13). 4.3.2 Dual Solution From the above, solving the primal problem (4.8) is equivalent to solving its dual problem (4.14). For maximization of the concave and dif- ferentiable dual functionD (;M;), the subgradient projection method can be employed to iteratively converge to the optimal solution. The La- grangian multipliers h i ; h i ; h are updated in the same direction to the sub- gradient of the dual objective function: 8 > > > > > > > < > > > > > > > : h;k+1 i = h;k i + @D( k ;M;) @ h;k i + h;k+1 i = h;k i + @D(M k ;;) @ h;k i + h;k+1 = h;k + @D( k ;;M) @ h;k + ; 88 where ; ; > 0 is the step size which adjusts the convergence rate and k2N + denotes the index of iterations. According to (4.12), the first term of the dual objective function con- sists ofNH independent subproblems. In other words, by dual decompo- sition, the global optimization problem (4.8) has been decoupled into sep- arable local optimization subproblems (4.13) of each building at each time slot. For the local optimization subproblem (4.13) of each building at each time slot, given k i ; k i ; h;k , together with (4.11), due to the convexity of C h i (), the optimal energy consumption is unique and can be derived as ~ x h i ,x h i k i ; k i ; h;k = C h i 0 1 h i g h;k i x h i x h i ; (4.15) where C h i 0 1 () is the inverse function of the first-order derivative of the cost functionC h i (),j b a denotes minfmaxf;ag;bg, andg h;k i , g h i k i ; k i . As mentioned in (4.1), the cost function C h i () is convex, the first-order derivative of which is increasing, and thus its inverse function C h i 0 1 () is also increasing. Under arbitrary k ;M k ; k , the local optimal solution X ( k ;M k ; k ) may not be globally optimal. However, by strong duality, there exists dual optimal ;M ; such that X ( ;M ; ) is the globally optimal solution. From the above, given ;M ; obtained from the dual problem (4.14), each building can solve the subproblem (4.13) distributively without 89 the need to coordinate with other buildings. In this sense, Lagrangian multipli- ers serve as a coordination signal which aligns the local optimality of (4.13) with the global optimality of (4.8). Substituting the subproblem solution (4.15) into the dual objective function (4.12), the subgradient of the dual objective function can be derived as 8 > > > > < > > > > : @D( k ;M;) @ h;k i =T h i f h i (~ x i ) h i @D(M k ;;) @ h;k i =f h i (~ x i ) + h i T h i @D( k ;;M) @ h;k = P i ~ x h i l h : Thus, together with (4.6), we obtain the following Lagrangian multiplier update rule: 8 > > > > > > > < > > > > > > > : h;k+1 i = h h;k i + T h i ~ T h i i + (4.16a) h;k+1 i = h h;k i + ~ T h i T h i i + (4.16b) h;k+1 = " h;k + X i ~ x h i l h !# + : (4.16c) 4.3.3 Distributed Algorithm The distributed algorithm can be approached from two aspects. On one hand, each subproblemS h i i ; i ; h is solved locally at each building i, returning the buildingi’s daily energy consumption schedule ~ x i . These subproblems are defined by (4.13) and solved by (4.15). On the other hand, the dual problemD (;M;) is iteratively solved, returning the dual op- 90 timal Lagrangian multipliers ;M ; . The dual problem is defined by (4.14) and solved by (4.16a)-(4.16c). To implement the above-mentioned two aspects, an information sharing platform is designed to collect the energy consumption from each building and update the Lagrangian multiplier as the coordination signal among buildings. The other two Lagrangian multipliers i ; i are updated locally at each buildingi. The interaction between the information sharing platform and each building based on the dual decomposition approach is illustrated in Fig. 4.1. The coordination signal through the Lagrangian multipliers can be interpreted as follows. Following the classical concepts in economics, the Lagrangian multiplier h is the congestion price at time slot h to balance between the sum of all buildings’ energy consumption and the total load threshold. At time slot h, when the sum of all buildings’ energy consumption P i ~ x h i exceeds the total load threshold l h (aggressive case), the congestion price h will rise (see (4.16c)) to capture the fact that it is expensive to consume energy at that time slot, which will in turn decrease each building’s energy consumption ~ x h i (see (4.15)). The iterative processes repeat until the congestion price converges, and then the global optimal solution is reached. Similarly, the Lagrangian multipliers h i ; h i are the temperature control signals of buildingi to guarantee that the comfortable temperature zone is 91 Information Sharing Platform • Update congestion price: , 1 , (16c) h k h k h h i i xl                 Building • Calculate intermediate variable and Calculate energy consumption: • Calculate indoor temperature and Update temperature control signals:     ,, , 1, , , (11) (11) H H k H k i i i i h k h k h k h k i i i i i i gc g a g c             congestion price daily energy consumption schedule … … Building Building     , 1 , , 1 , (16a) (16b) h k h k h h i i i i h k h k h h i i i i TT TT                                1 , ' (15) h i h i x h h h h k i i i i x x C g      1 (6) h h h h i i i i od i i T aT bT c x     h  , ,1 hH  1, , hH  Figure 4.1: Interaction between information sharing platform and each building based on dual decomposition approach. 92 Algorithm 4: executed by information sharing platform 1 Initialization:k 1, arbitrary nonnegative Lagrangian multipliers h;1 h2H ; 2 repeat 3 Receive daily energy consumption schedule ~ x i from each buildingi2N ; 4 for each time sloth2H do 5 Update congestion price h;k+1 by (4.16c): h;k+1 = h;k + P i ~ x h i l h + ; 6 end for 7 Broadcast congestion price k+1 to all buildings; 8 k k + 1; 9 until h;k h;k1 <";8h; satisfied. Taking the heating scenario for example, where the parameterc i is positive, such that at time sloth the larger the energy consumption ~ x h i , the higher the indoor temperature ~ T h i (see (4.6)). When the energy consump- tion ~ x h i is too large that T h i is exceeded (aggressive case), the temperature control signal h i will rise (see (4.16b)), increasingg h i (see (4.11)), which will in turn decrease the energy consumption ~ x h i (see (4.15)). However, when the energy consumption ~ x h i is too small thatT h i is not reached (conservative case), the temperature control signal h i will drop (see (4.16a)), decreasing g h i (see (4.11)), which will in turn increase the energy consumption ~ x h i (see (4.15)). The iterative processes repeat until the temperature control signals converge, and then the global optimal solution is reached. The distributed algorithms for the information sharing platform and 93 Algorithm 5: executed by each buildingi2N Input: initial indoor temperatureT 0 i , and predicted outdoor temperature T h od h2H Output: optimal daily energy consumption schedulex i 1 Initialization:k 1, arbitrary nonnegative Lagrangian multipliers n h;1 i o h2H ; n h;1 i o h2H ; 2 repeat 3 for each time sloth =H;:::; 1 do 4 Calculate intermediate variableg h;k i by (4.11): 8 < : g H i =c i H;k i H;k i g h;k i =a i g h+1;k i +c i h;k i h;k i ; 5 Calculate energy consumption ~ x h i by (4.15): ~ x h i = C h i 0 1 h i g h;k i x h i x h i ; 6 end for 7 Receive congestion price k+1 from information sharing platform; 8 for each time sloth = 1;:::;H do 9 Calculate indoor temperature ~ T h i by (4.6): ~ T h i =a i ~ T h1 i +b i T h od +c i ~ x h i ; 10 Update temperature control signals h;k+1 i by (4.16a): h;k+1 i = h h;k i + T h i ~ T h i i + and h;k+1 i by (4.16b): h;k+1 i = h h;k i + ~ T h i T h i i + ; 11 end for 12 Send daily energy consumption schedule ~ x i to information sharing platform; 13 k k + 1; 14 until h;k i h;k1 i <"; h;k i h;k1 i <";8h; 94 each building are summarized in Algorithm 4 and Algorithm 5, respec- tively. The parameter " is the convergence error tolerance (stopping crite- rion), and all updates are performed based on local information. The con- gestion price h is updated at the information sharing platform, while the temperature control signals h i ; h i are updated at each building, and each building also calculates its energy consumption ~ x h i and indoor temperature ~ T h i . The information sharing platform will broadcast the congestion price to all buildings, while each building will send its daily energy consumption schedule ~ x i to the information sharing platform. Remark 4.1. Note that the proposed algorithm is not completely distributed since the information sharing platform has to be involved. In this chapter, we nevertheless follow some references to term it as “distributed”, to differ- entiate it from existing centralized methods where the global optimization is directly solve centrally with the local parameters of each building known by the central controller. In our proposed algorithm, the information shar- ing platform only coordinates among buildings instead of determining for each building. Intuitively, as long as there are coupled constraints among buildings, it is hard for them to behave completely independently without minimum information sharing while achieving the global optimal solution. 95 4.4 Numerical Results In this section, we provide numerical examples to illustrate the dis- tributed energy management in multi-building systems, and evaluate the performance of the proposed distributed algorithms based on dual decom- position. For ease of illustration, we consider a simple multi-building sys- tem with 3 smart buildings and 24 time slots. It can be extended to more buildings, with similar results. For simplicity but without loss of generality, we take the heating scenario for example, while the cooling scenario follows the similar way. 0 5 10 15 20 25 0.05 0.1 0.15 0.2 price time slot $/kwh 0 5 10 15 20 25 −14 −12 −10 −8 −6 −4 outdoor temperature time slot o C Figure 4.2: TOU electricity price and the predicted outdoor temperature of Edmonton, AB, Canada on November 18, 2015. The simulation parameters are summarized as follows. The elec- 96 tricity price and outdoor temperature information are based on real data. We adopt the time-of-use (TOU) pricing data in Ontario, Canada for winter weekdays (November 1 - April 30), and take the predicted outdoor temper- ature of Edmonton, AB, Canada on November 18, 2015, as shown in Fig. 4.2. The operation cost function of each HVAC unit isC h i x h i = c h 1;i p h x h i 2 + c h 2;i p h x h i +c h 3;i [41], wherep h is the electricity price at time sloth, andc h 1;i = c h 2;i =c h 3;i = 1 for alli andh. The thermal constants of each building are set asa i =b i = 0:5 andc i = 2 for alli. The lower and upper bounds on energy consumption of each building are set as x h i = 0 and x h i = 15kWh respec- tively for alli andh. The total load threshold is set asl h = 27kWh for all h. The temperature comfort zones for three buildings are set as [22°C,24°C], [23°C,25°C], and [24°C,26°C], respectively. The convergence error tolerance is fixed as" = 1e 6 for all following cases. 4.4.1 Convergence Analysis We firstly vary the iteration step size = 0:003, 0:006, 0:010 and 0:015 respectively, to study the convergence of the subgradient projection method under different step size. The result is shown in Fig. 4.3. We can see that as long as the step size is not too large, it only affects the convergence speed, but does not impact the convergence point. As the step size increases, the algorithm converges in fewer steps. However, when the step size is too large, the output will diverge and oscillate. In the following simulations, 97 0 100 200 300 400 500 540 560 580 600 620 640 660 680 iteration dual objective value γ=0.003, converge in 525 steps γ=0.006, converge in 286 steps γ=0.010, converge in 184 steps γ=0.015, diverge Figure 4.3: The impact of step size on the convergence rate of the subgradi- ent projection method. 98 we fix = 0:01 to ensure a fast convergence. 0 50 100 150 200 0 200 400 600 800 dual objective value iteration dual optimal 0 50 100 150 200 0 0.2 0.4 0.6 0.8 λ iteration 0 50 100 150 200 0 0.5 1 1.5 2 x 10 −3 μ iteration 0 50 100 150 200 0 0.002 0.004 0.006 0.008 0.01 ν iteration Figure 4.4: The convergence of the dual objective value and the average value of Lagrangian multipliers versus iteration. Figure 4.4 illustrate the convergence of the dual objective value and the average value of Lagrangian multipliers versus iteration. The optimal value shown in the figure is the minimum of the primal problem. The av- erage value of Lagrangian multipliers are calculated as = 1 NH P i;h h i , = 1 NH P i;h h i , and = 1 H P h h . The figure indicates that with the it- 99 eration, the slack variables converge to the optimal point such that dual objective value is maximized. The observation that the dual objective value converges to the optimal value of the primal problem verifies the strong duality theory. 0 50 100 150 200 0 2 4 6 total power load violation iteration 0 50 100 150 200 0 2 4 6 temperature upper bound violation iteration 0 50 100 150 200 0 1000 2000 3000 temperature lower bound violation iteration 0 50 100 150 200 0 1000 2000 3000 temperature comfort zone violation iteration Figure 4.5: Constraints violation versus iteration. Figure 4.5 shows the constraints violation over iterations. The four subfigures correspond to the total load violation P h P i x h i l h + , the 100 temperature upper bound violation P i;h T h i T h i + , the temperature lower bound violation P i;h T h i T h i + , and the temperature comfort zone violation P i;h T h i T h i + + T h i T h i + , respectively, where () + denotes maxf; 0g. It is observed that at the beginning, the four constraints are all violated, which means the corresponding primal solution is not feasible. With the convergence, all the constraints are satisfied and thus the corresponding primal point becomes feasible. 4.4.2 Performance Analysis 0 5 10 15 20 25 20 22 24 26 28 time slot kwh total load threshold total power consumption 0 5 10 15 20 25 0 0.01 0.02 time slot congestion price Figure 4.6: The total energy consumption, total load threshold, and conges- tion price versus time slot. Figure 4.6 shows the total energy consumption of all buildings P i x h i , the total load thresholdl h , and the congestion price h , over all time slots. 101 It is observed that the spatially-coupled constraints are satisfied over all time slots, and the congestion prices are always nonnegative. More specif- ically, for all time slots other than time slot h = 6, when the threshold is not reached, the slack variable h = 0. While at time slot h = 6, when the constraint is tight, the slack variable h 6= 0. Such observations verify Karush-Kuhn-Tucker (KKT) conditions at the optimal point, where the mul- tiplication of each inequality constraint with its corresponding slack vari- able should be equal to zero, called complementary slackness. 0 5 10 15 20 25 20 25 time slot o C temperature 1 lower bound 1 upper bound 1 0 5 10 15 20 25 0 1 2 time slot λ 1 0 5 10 15 20 25 −1 0 1 time slot μ 1 (a) Building 1 Figure 4.7(a) shows the indoor temperature T h 1 of building 1, its lower bound T h 1 = 22°C with slack variable h 1 , and its upper bound T h 1 = 24°C with slack variable h 1 . We can see that the upper bound is 102 0 5 10 15 20 25 22 24 26 time slot o C temperature 2 lower bound 2 upper bound 2 0 5 10 15 20 25 0 1 2 time slot λ 2 0 5 10 15 20 25 −1 0 1 time slot μ 2 (b) Building 2 0 5 10 15 20 25 24 26 28 time slot o C temperature 3 lower bound 3 upper bound 3 0 5 10 15 20 25 0 1 2 time slot λ 3 0 5 10 15 20 25 −1 0 1 time slot μ 3 (c) Building 3 Figure 4.7: Three buildings’ indoor temperature, comfort zone, and temper- ature control signals. 103 always slack and its corresponding temperature control signal h 1 = 0 over all time slotsh. Under the heating scenario and with the objective of energy cost minimization, this would be the most common case unless a much tighter temperature upper bound is required or huge pre-heating that reaches the temperature upper bound. On the contrary, the lower bound is almost always tight and its corresponding temperature control signal h 1 6= 0 over all time slots except for time sloth = 6. At time sloth = 6, the temperature lower bound is slack and thus the slack variable h 1 drops to zero. This is the result of pre-heating, caused by the largest electricity price rise of the day at the next time sloth = 7. Figure 4.7(b) and Figure 4.7(c) illustrate the similar results. In three figures, KKT conditions can be verified. The primal feasibility is satisfied as the indoor temperature is always within the temperature comfort zone. The dual feasibility is satisfied as all the temperature control signals are non- negative over all time slots. The complementary slackness is satisfied as the temperature control signal is zero whenever the corresponding temperature bound is tight. 104 4.5 Summary In this chapter, we investigate the distributed energy management problem in multi-building systems. The problem is formulated as a cost minimization problem with spatially and temporally coupled constraints, and then decoupled into subproblems of each building and each time slot via Lagrangian dual decomposition. The strong duality of the primal and dual problem is proved, and distributed algorithms are designed based on the local information via subgradient projection method to converge to the global optimum point. During the process, Lagrangian multipliers are serve as coordination signals, such that the global optimality of the combined local solutions can be guaranteed with the iterations of the coordination signals in the right di- rections. Specifically, the local temperature control signals are designed for handling of the temporally coupled constraints of each building, while an information sharing platform is designed to update and broadcast the con- gestion price to autonomous buildings for handling of the spatially coupled constraints. Numerical results are provided to illustrate the convergence of the proposed method and effectiveness of the control performance. Al- though we target the problem of cost minimization with temperature com- fort zone and total load constraints, the decomposition and recombination framework is general and extendable. 105 This work sheds light on the handling of spatially and temporally coupled constraints in energy management problem, and the design of lo- cal subproblems to achieve the global optimum point through distributed optimization based on local information. In practice, the problem will be more complicated, as there might be situations that one or more local con- trollers break down. In such cases, the stability of the distributed control performance versus the centralized design could be further discussed. 106 Chapter 5 Indoor Temperature Control of Cost-Eective Smart Buildings via Real-Time Smart Grid Com- munications 5.1 Introduction The major energy consumption of buildings comes from the heating, ventilation and air conditioning (HVAC) unit, which raises electricity cost and environmental impact concerns [60]. Besides the accumulated energy usage, buildings tend to have high demand in electricity simultaneously, which causes significant peak load on the power grid [62]. The concept of smart buildings emerges, aiming to make the occupants productive (e.g., illumination, thermal comfort, air quality, physical security, sanitation, and many more) at the lowest electricity cost and environmental impact over the building lifecycle [95, 2]. Since people spend most of their time in build- ings, the indoor thermal comfort in a working or living place is strongly related to the occupants’ satisfaction and productivity [93]. Most often, the improvement of the building comfort demands more energy consumption. Hence, one of the most important issues of smart buildings is to balance the requirement of thermal comfort and electricity cost reduction via indoor 107 temperature control [56]. The traditional power grid is evolving into smart grid [38], where demand response is a key solution to address the ever-increasing peak en- ergy consumption, rather than building extra power plants [30]. Instead of the legacy flat electricity pricing, demand response provides consumers with different electricity prices at different times, which incentivizes load shifting from on-peak to off-peak periods. Various unflat electricity pricing models have been proposed, e.g., time-of-use pricing, critical peak pricing, real-time pricing, etc. Many economists argue that real-time electricity pric- ing is one of the most efficient incentives for competitive electricity mar- kets [13], which will be adopted in this chapter. The real-time electricity price is usually released on an hour-ahead basis and keeps constant during that hour. Under the big picture of demand response, minimization of the electricity cost from space heating/cooling in response to the fluctuation of electricity prices, instead of simply minimizing the energy consumption amount, is one of the main goals of smart buildings. The framework of real-time communications in smart grid for smart buildings is illustrated in Fig. 5.1. There coexist a variety of protocols, stan- dards, and technologies in a local area communication network, e.g., IEEE 802.15.4 (ZigBee), IEEE 802.11 (WiFi), IEEE 802.16m (WiMax), or cellular (GPRS, LTE, 4G). Based on two-way communications, the information of 108 Storage Bulk generation Renewable energy Utility company Electricity Real-time price Energy consumption Smart buildings Local area communication network Figure 5.1: Real-time communications in smart grid for smart buildings. electricity price and energy consumption is exchanged between the utility company and smart buildings in a real-time manner. Due to many random factors, such as the intermittency of renewable energy and the fluctuation of energy consumption, electricity prices are constantly changing. There- fore, real-time communications in smart grid lay a strong foundation for electricity cost reduction in smart buildings. Under the real-time electricity pricing environment in smart grid, building owners are faced with the indoor temperature control problem to minimize the daily electricity cost. It would be inconvenient for building owners to manually keep track of constantly changing electricity prices and take actions accordingly. The indoor temperature controller should be equipped with an automatic price-aware algorithm to control the power level of the HVAC unit. Existing works [41, 8, 10, 68] focused on temporally-coupled solutions based on the assumption of perfect predic- tion of the next-day electricity prices and outdoor temperatures. This may 109 not be practical as it is difficult to predict the exact future electricity prices and outdoor temperatures accurately for a long time. On the other hand, pre-heating/cooling in smart buildings is to pre-heat/cool the indoor space during off-peak periods, storing in the thermal mass, and thereby reducing electricity demand during the on-peak periods [104]. However, when to take advantage of pre-heating/cooling for electricity cost reduction has not been explicitly indicated. This chapter takes into consideration the daily electricity cost and predetermined comfort range of desirable temperatures of a smart building that is subject to indoor temperature control. For the heating scenario, in- tuitively, one may consider to maintain the building’s indoor temperature always at the lower bound of the predetermined comfort range to save the electricity bill. However, this intuition may not be always true, especially with the significant fluctuation of electricity prices. The similar issue ex- ists for the cooling scenario. Although the cost minimization problem can be optimally solved one day ahead in a temporally-coupled manner, the challenge lies in that the building owner needs to know the exact electric- ity prices and outdoor temperatures of the next day. However, such infor- mation cannot be predicted accurately for a long time. Therefore, in this chapter, we equivalently decouple the cost minimization problem into sub- problems at each hour. Each subproblem can be temporally decoupled and 110 optimally solved, only requiring the next-hour electricity price. Besides, the temporally-decoupled algorithm explicitly indicates when to take advan- tage of pre-heating/cooling for electricity cost reduction. Compared with the intuitive strategy, real-data case studies demonstrate that our proposed algorithm could lead to significant economic savings, which is practically applicable to cost-effective smart buildings. The remainder of this chapter is organized as follows. The problem of minimizing the building’s daily electricity cost with the indoor tempera- ture constraint is formulated in Section 5.2. In Section 5.3, we derive the op- timal solution in a temporally-coupled manner, which requires the next-day electricity prices and outdoor temperatures. In Section 5.4, we address the cost minimization problem in a temporally-decoupled manner, only requir- ing the next-hour electricity price. Case studies are provided in Section 5.5 before concluding remarks drawn in Section 5.6. 5.2 Problem Formulation Consider that the cycle of a day is divided into a set H = f1; ;h; ;Hg of time slots. Some important notations used in this chapter are summarized in Table 5.1. Under the real-time electricity pricing environment in smart grid, building owners are faced with the indoor temperature control problem to minimize the daily electricity cost in response to the fluctuation of electricity 111 Table 5.1: Summary of notations Symbol Definition Unit a h (t,),H,H index, number, set of time slot(s) n/a p real-time electricity price $/kWh x energy consumption kWh x daily energy consumption schedule n/a T indoor temperature °C T daily indoor temperature schedule n/a T ,T indoor temperature lower, upper bound °C T 0 initial indoor temperature °C T od predicted outdoor temperature °C thermal decay factor n/a ,K thermal constant n/a b,c model coefficient n/a , intermediate variable n/a a The unit of a quantity may be omitted in the rest of the chapter if it is specified here. 112 prices. The objective function of cost minimization is: min x X h2H p h x h ; (5.1) where p h denotes the electricity price at time slot h, x h is the building’s energy consumption at time slot h, andx , x 1 ; ;x h ; ;x H denotes the building’s daily energy consumption schedule. LetT h denote the temperature inside the building at the end of time sloth. We would like the indoor temperature to always stay within a pre- determined comfort range, i.e., T h T h T h ; (5.2) whereT h andT h denote the lower and upper bounds of the predetermined comfort range for the building’s indoor temperature at time sloth, respec- tively. The indoor temperature evolves according to the following exponen- tial decay model [41]: T h =T h1 + (1) T h od + Kx h ; (5.3) whereT h1 denotes the indoor temperature at the end of the previous time slot, T h od for the outdoor temperature at the beginning of the current time slot, andx h for the building’s energy consumption at the current time slot. The other parameters are the thermal constants of the building: 0 < < 1 113 is a thermal decay factor, is a factor capturing the efficiency of the HVAC unit to heat (+) or cool () the air inside the building, andK is a conversion factor that is proportional to the performance coefficient of the HVAC unit divided by the total thermal mass. The HVAC unit uses powerx h to control the building’s indoor tem- perature at time slot h. In this chapter, our focus is only on the heating scenario as shown in (5.3). The results for the cooling scenario can be eas- ily obtained by changing the sign of . Eq. (5.3) explains how the constraint (5.2) relates to the energy consumed by the HVAC unit. The building owner should determine the power level of the HVAC unit in such a way that the indoor temperature obtained from (5.3) falls within the predetermined com- fort range (5.2) and minimizes the daily electricity cost (5.1). Our goal is to reduce the daily electricity cost by dynamically adjust- ing the power level of the HVAC unit such that the indoor temperature of a smart building is kept within a predetermined comfort range. From the above, the cost minimization problem is to minimize the building’s daily electricity cost with the indoor temperature constraint. Original Problem: min x X h2H p h x h (5.4) s.t. (5.2) and (5.3) 8h2H x h 0 8h2H: 114 5.3 Temporally-Coupled Solution The original problem (5.4) can be viewed as a linear programming problem. Intuitively, one may consider that the solution is to maintain the building’s indoor temperature always at the lower bound of the predeter- mined comfort range. However, this strategy may not always achieve the lowest electricity bill, especially with the significant fluctuation of electricity prices, which will be shown later. LetT 0 denote the building’s initial indoor temperature. For ease of presentation, we simplify the model coefficients in (5.3) and derive T h re- cursively as T h =T h1 +bT h od +cx h =c h X t=1 () ht x t + h ; (5.5) where 8 > > < > > : b, 1 c, (1) K h , () h T 0 +b h P t=1 () ht T t od : From (5.5), the building’s indoor temperature constraint (5.2) is rewritten as T h h c h X t=1 () ht x t T h h : (5.6) Thus, the original problem (5.4) is rewritten as min x X h2H p h x h (5.7) s.t. (5.6) 8h2H x h 0 8h2H: 115 Note that the building’s indoor temperature constraint (5.6) is temporally-coupled on the daily energy consumption schedule x. That is, in order to satisfy the predetermined comfort range at time slot h, all of the building’s current and previous energy consumptions x 1 ; ;x h are coupled together. With the temporally-coupled constraint (5.6), the original problem (5.7) cannot be directly decoupled into a number H of subproblems at each time sloth2H. On the other hand, the original problem (5.7) can be optimally solved in a temporally-coupled manner at the beginning of the day, such that the daily energy consumption schedulex is entirely calculated one day ahead. The challenge lies in that the building owner needs to know the exact elec- tricity prices and outdoor temperatures of the next day. This may not be practical as it is difficult to predict the future electricity prices and outdoor temperatures accurately for a long time. To further mitigate the impact of the prediction error, we propose Algorithm 6 where the building owner can update the current energy con- sumption at the beginning of each time slot in an online manner, based on the actual electricity price and outdoor temperature. Specifically, at the be- ginning of the th ( = 1; ;H) time slot, when the actual electricity price (denoted by p ) is obtained via real-time communications in smart grid, and the actual outdoor temperature (denoted byT od ) is known, the building 116 Algorithm 6: Temporally-Coupled Algorithm 1 for = 1; ;H do 2 At the beginning of the th time slot, when the current electricity price (denoted byp ) is obtained via real-time communications in smart grid, and the current outdoor temperature (denoted byT od ) is known, and the future electricity prices and outdoor temperatures are predicted as ^ p +1 ; ; ^ p H and h ^ T +1 od ; ; ^ T H od i , the building owner can update the current energy consumption by solving min x X h=1 p h x h + H X h=+1 ^ p h x h (5.8) s.t. 8 < : (5.6) 8h2H x h =x h h = 1; ; 1 x h 0 h =; ;H; where h , () h T 0 +b X t=1 () ht T t od +b h X t=+1 () ht ^ T t od ; Let x 1 ; ;x (1) ; ~ x ; ; ~ x H denote the solution, and the building owner takes ~ x as the current energy consumption, i.e.,x ~ x ; 3 end for 117 owner can solve (5.8) to update the current energy consumption. With the more actual information, the building owner can make the wiser decision. Nevertheless, the building owner still needs to predict the future electricity prices ^ p +1 ; ; ^ p H and outdoor temperatures h ^ T +1 od ; ; ^ T H od i . However, such information cannot be predicted accurately, and, to be shown in the following section, is in fact not full required to achieve the optimal solution. 5.4 Temporally-Decoupled Solution In this section, we address the cost minimization problem in a temporally-decoupled manner, only requiring the next-hour electricity price. From (5.5) we have x h = 1 c T h T h1 bT h od : (5.9) Thus, X h2H p h x h = 1 c " H1 X h=1 p h p h+1 T h +p H T H # ; where ,p 1 T 0 +b X h2H p h T h od : Let T , T 1 ; ;T h ; ;T H denote the building’s daily indoor temperature schedule. From the above, the original problem (5.4) can be transformed into an equivalent problem. 118 Transformed Problem: min T 1 c " H1 X h=1 p h p h+1 T h +p H T H # (5.10) s.t. (5.2) 8h2H: That is, the decision variable is transformed from the daily energy con- sumption schedulex into the daily indoor temperature scheduleT . In this way, the former temporally-coupled constraint (5.2) in the original problem (5.4) is no longer coupled in the transformed problem (5.10). Without the coupled constraint, the transformed problem can be equivalently decoupled into a numberH of subproblems at each time sloth2H. SubproblemS h (h = 1; ;H 1): min T h p h p h+1 T h (5.11) s.t. T h T h T h : SubproblemS H : min T H p H T H (5.12) s.t. T H T H T H : Thus, the transformed problem (5.10) can be equivalently rewritten as 1 c " H1 X h=1 S h +S H # : 119 The subproblems (5.11) and (5.12) are simple linear programming problems, which can be easily solved. Thus, the temporally-decoupled al- gorithm for the building’s indoor temperature control is summarized in Al- gorithm 7. Algorithm 7: Temporally-Decoupled Algorithm 1 forh = 1; ;H 1 do 2 At the beginning of theh th time slot, when the current electricity price (denoted byp h ) is obtained via real-time communications in smart grid, and the current outdoor temperature (denoted byT h od ) is known, and the next-hour electricity price is predicted as ^ p h+1 , the building owner can determine the current energy consumption as follows; 3 ifp h < ^ p h+1 then /* Pre-heating */ 4 T h T h ; 5 else 6 T h T h ; 7 end if 8 x h 1 c T h T h1; bT h od ; 9 end for 10 At the beginning of theH th time slot, when the current outdoor temperature (denoted byT H od ) is known, the building owner can determine the current energy consumption as follows; 11 T H T H ; 12 x H 1 c T H T H1; bT H od ; The temporally-decoupled algorithm explicitly indicates that, if the current electricity price is much cheaper than the next-hour price such that p h <p h+1 , the current indoor temperature should be set at the upper bound T h of the predetermined comfort range. Such a case is referred to as pre- 120 heating. Although pre-heating consumes more energy than that with T h , the daily electricity cost is still lower than that of the intuitive strategy, since the current electricity price is much cheaper. The underlying insight can be interpreted as follows. At the next time slot h + 1, if we assume that the HVAC unit needs x energy to raise the indoor temperatureT h+1 by T ; then at the current time sloth, the HVAC unit needs x= energy to raise T h+1 by T (pre-heating), where 0 < < 1 is the building’s thermal decay factor. Since x= > x, pre-hearing consumes more energy. However, when p h < p h+1 , the electricity cost (x=)p h < (x)p h+1 , which means that pre-hearing could reduce the electricity cost. Therefore, when p h < p h+1 , the building owner can take advantage of pre-heating (or pre-cooling in the cooling scenario) for electricity cost reduction. Local area communication network (real-time communications in smart grid) Distribution power line HVAC Temporally- decoupled algorithm Indoor temperature controller Next-hour price prediction Predetermined comfort range Indoor temperature Real-time price Energy consumption Electricity Smart buildings Utility company Figure 5.2: Illustration of indoor temperature control for smart buildings. The temporally-decoupled algorithm is practically applicable, which 121 0:00 4:00 8:00 12:00 16:00 20:00 24:00 0 0.2 0.4 0.6 0.8 Time Real−time price ($/kWh) 0:00 4:00 8:00 12:00 16:00 20:00 24:00 −20 −15 −10 −5 Time Outdoor temperature ( o C) (a) Real-time electricity prices and predicted outdoor temperatures. 0:00 4:00 8:00 12:00 16:00 20:00 24:00 1 2 3 4 5 Time Energy consumption (kWh) intuitive temporally−coupled/decoupled 0:00 4:00 8:00 12:00 16:00 20:00 24:00 19 21 23 25 Time Indoor temperature ( o C) intuitive temporally−coupled/decoupled (b) Energy consumption and indoor temperature schedules. Figure 5.3: Performance comparison of intuitive and temporally- coupled/decoupled approaches for one zone of a smart building in Edmon- ton on Jan. 6, 2016. 122 only requires the next-hour electricity price to make the optimal decision, rather than the next-day electricity prices and outdoor temperatures required by the temporally-coupled algorithm. The framework of indoor temperature control for smart buildings is illustrated in Fig. 5.2. The real-time electricity pricep h is obtained from the utility company via a local area communication network (real-time communications in smart grid). The comfort range h T h ;T h i for the indoor temperature is predetermined by the building owner. The next-hour price prediction functionality is embedded in the indoor temperature controller. How to accurately predict the next-hour price p h+1 from a large number of historical data is considered as our future work. Based on the above information, the temporally-decoupled algorithm embedded in the controller determines the current indoor temperature T h and sends the command to control the power level of the HVAC unit. The corresponding energy consumption amount is reported to the utility company for monitoring and billing via real-time communications in smart grid. Our proposed algorithm outperforms the intuitive strategy and the temporally-coupled algorithm in terms of the solution performance and re- quired information. We can conclude the following observations: 1. The intuitive strategy may not always achieve the lowest electricity bill, but it does not require any information of future electricity prices 123 and outdoor temperatures. 2. The temporally-coupled algorithm and our proposed algorithm can both obtain the optimal solution. 3. The temporally-coupled algorithm requires the next-day electricity prices and outdoor temperatures. 4. Our proposed algorithm only requires the next-hour electricity price. 5.5 Case Studies In this section, we provide case studies from the real data to demon- strate the performance of our proposed algorithm. We consider a smart building in Edmonton, Alberta, Canada, which is divided into multiple in- dependent zones for indoor temperature control. Our focus is only on one zone (e.g., one room) of the smart building. The cycle of a day is divided into 24 time slots (hours). Alberta’s electricity market is an energy-only, real-time market which is cleared every minute and the average of the 60 market clearing prices over an hour, referred to as the Hourly Alberta Pool Price (HAPP), is used as the basis of financial settlements [5]. The predicted outdoor temperatures are taken from [36]. We take the heating scenario as an example, while the results for the cooling scenario are similar. Recom- mended indoor temperatures typically range from 20°C to 23.5°C for the winter [19, Ch. 6]. The thermal constants for one zone of the smart building 124 are set as = 0:7, = 0:9, andK = 15 [41]. The important parameters used in the case studies are summarized in Table 5.2. Table 5.2: Parameter setup Parameter Value Parameter Value H 24 0.7 T;T 20, 23.5 b 0.3 T 0 20 c 4.05 1 6 11 16 21 26 31 0 0.5 1 1.5 2 2.5 3 3.5 Day Daily electricity cost reduction ($) Figure 5.4: Daily electricity cost reduction by pre-heating in January, 2016. The performance comparison of the intuitive and optimal (temporally- coupled/decoupled) approaches is shown in Fig. 5.3 and Fig. 5.4. The 125 real-time electricity prices from HAPP [5] on January 6, 2016 are shown in Fig. 5.3(a) top. We can see the significant fluctuation of electricity prices, especially the two spikes in the early morning and late afternoon, due to the peak demand at those periods. The predicted outdoor temperatures in Edmonton [36] on January 6, 2016 are shown in Fig. 5.3(a) bottom. Based on the real-time electricity prices and predicted outdoor temperatures, the energy consumption and indoor temperature schedules for one zone of the smart building are shown in Fig. 5.3(b). It is observed that, the temporally-coupled algorithm and our proposed algorithm can both obtain the optimal solution. For the intuitive strategy, the indoor temperatures are always maintained at the lower bound of the predetermined comfort range. In such a way, the daily energy consumption is always the lowest, but the daily electricity cost may not always be. The reason is that, at some hours, e.g., 6:00-7:00, 15:00-16:00, and 16:00-17:00, the electricity price is much cheaper than that at the next hour (on-peak periods). Thus, our proposed algorithm, taking advantage of pre-heating, although consumes more energy, can lead to a lower electricity cost than the intuitive strategy. The daily electricity cost reduction by pre-heating in January, 2016 is shown in Fig. 5.4. It shows that pre-hearing could reduce the electricity cost on multiple days of the month. Although the specific economic savings depend on different building thermal parameters, electricity prices, and 126 outdoor temperatures, the case studies demonstrate that our proposed algorithm is practically applicable to cost-effective smart buildings. Note that this is only a daily result for one zone (e.g., one room) of a smart building. Considering the entire lifecycle of a smart building which may have multiple zones, the economic savings would be significant. From a world-wide perspective, the prevalence of such a practically cost-effective algorithm to a huge number of smart buildings will largely benefit the whole community. 5.6 Summary This chapter investigates the indoor temperature control problem to minimize the building’s daily electricity cost via real-time communications in smart grid. Taking the heating scenario as an example, with flat electricity pricing, it is both energy-efficient and cost-effective to maintain the build- ing’s indoor temperature always at the lower bound of the predetermined comfort range. However, under the unflat electricity pricing environment in smart grid, this strategy may not always achieve the lowest electricity bill, especially with the significant fluctuation of electricity prices. The cost min- imization problem can be optimally solved in a temporally-coupled man- ner one day ahead, but the challenge lies in that the building owner needs to know the exact electricity prices and outdoor temperatures of the next day. As such information can hardly be acquired in an accurate way, in this 127 chapter, we equivalently decouple the cost minimization problem into sub- problems at each hour. Each subproblem can be temporally decoupled and optimally solved, only requiring the next-hour electricity price. Besides, the temporally-decoupled algorithm explicitly indicates that, when the current electricity price p h is cheaper than the next-hour price p h+1 multiplied by the building’s thermal decay factor , the building owner can take advan- tage of pre-heating/cooling for electricity cost reduction. It is demonstrated with the real data that our proposed algorithm could result in considerable economic savings compared with the intuitive strategy, paving the way to- wards practically applicable cost-effective smart buildings. 128 Chapter 6 Conclusions This dissertation introduces distributed optimization methods and challenges in demand response management under the big picture of smart grid, presents a novel bi-level game structure for demand response inter- actions, and compares the impact of communication mechanisms among users. The overall demand response game is modeled by a bi-level model, comprising a consumer-level noncooperative game and a one-leader-one- follower Stackelberg game between the provider-level and consumer-level. An information sharing platform is designed and the win-win effectiveness of information sharing among consumers is illustrated. The structure can characterize the higher-level leader-follower interaction between the sup- ply side and demand side, and has good extendability in terms of adding games on the supply side or changing games on the demand side. In practice an energy consumer might represent a set of residential or commercial buildings, therefore, within each set individual units are ex- pected to cooperate to achieve the group-level optimum instead of previous noncooperative game relationship. Through design of a series of sliding window games, each building is treated as a player, the global utility is de- 129 composed to each player’s payoff, and each game is played over a predic- tion horizon. During the games, by applying a newly proposed distributed learning algorithm in game theory, each building learns to play part of the global optimum by states transition. The proposed scheme is applied to a three building case study to demonstrate its effectiveness. Spatially and temporally coupled constraints in the multi-building energy management problem are also addressed. Spatially coupled con- straints are due to the distribution infrastructure limit within the system, while temporally coupled constraints are caused by the heat preservation from previous power consumptions. These constraints are sophisticatedly formulated for the primal cost minimization problem, and separable local optimization subproblems for each building and each time slot are trans- formed via Lagrangian dual decomposition. With strong duality property and the coordination signals based on Lagrangian multipliers, global opti- mality of the combined local solutions can be guaranteed. Although the cost minimization problem can be solved theoretically and numerically, the challenge lies in the requirement of perfect prediction of the electricity prices and outdoor temperatures of the next day, which is hard to to be achieved in practice. A single building temporally-decoupled algorithm is developed, requiring only the next-hour electricity price, it explicitly indicates when to take advantage of pre-heating/cooling for elec- 130 tricity cost reduction. Compared with the intuitive strategy, real-data case studies demonstrate that the proposed algorithm could lead to significant economic savings, which is practically applicable to cost-effective smart buildings. The work could be extended in the following aspects. 1) Consider more complicated consumer communications mechanisms such as partial population participation or cheating in demand response games. 2) How to deal with local controller breaking-down situations under the distributed framework, further analyze the stability of the distributed control per- formance comparing with centralized manner. 3) Add a noncooperative supplier-level game to the hierarchical game model, study the scenario with multiple power providers and multiple consumers. 4) Data driven methods for consumer parameters estimation, consumer type segmentation or consumer demand prediction. 131 Bibliography [1] H. Aalami, M. P . Moghaddam, and G. Yousefi, “Demand response modeling considering interruptible/curtailable loads and capacity market programs,” Applied Energy, vol. 87, no. 1, pp. 243–250, 2010. [2] Y. Agarwal and T. Weng, “From buildings to smart buildings— sensing and actuation to improve energy efficiency,” IEEE Design & Test of Computers, no. 4, pp. 36–44, 2012. [3] M. H. Albadi and E. El-Saadany, “Demand response in electricity mar- kets: An overview,” in IEEE power engineering society general meeting, vol. 2007, 2007, pp. 1–5. [4] M. H. Albadi and E. El-Saadany, “A summary of demand response in electricity markets,” Electric power systems research, vol. 78, no. 11, pp. 1989–1996, 2008. [5] Alberta Electric System Operator (AESO). Hourly Alberta Pool Price (HAPP). [Online]. Available: http://ets.aeso.ca/ [6] H. Allcott, “Real time pricing and electricity markets,” Harvard Uni- versity, vol. 7, 2009. [7] E. Altman and Z. Altman, “S-modular games and power control in wireless networks,” IEEE Transactions on Automatic Control, vol. 48, no. 5, pp. 839–842, 2003. [8] J. ´ Alvarez, J. Redondo, E. Camponogara, J. Normey-Rico, M. Berenguel, and P . Ortigosa, “Optimizing building comfort temperature regulation via model predictive control,” Energy and Buildings, vol. 57, pp. 361–372, 2013. [9] T. Anandalaskhmi, S. Sathiakumar, and N. Parameswaran, “Peak re- duction algorithms for a smart community,” in Proc. IEEE ICEETS, 2013, pp. 1113–1119. 132 [10] M. Avci, M. Erkoc, A. Rahmani, and S. Asfour, “Model predictive HVAC load control in buildings using real-time electricity pricing,” Energy and Buildings, vol. 60, pp. 199–209, 2013. [11] F. Berman, G. C. Fox, and A. J. Hey, “The grid: past, present, future,” 2003. [12] G. Binetti, A. Davoudi, F. L. Lewis, D. Naso, and B. Turchiano, “Distributed consensus-based economic dispatch with transmission losses,” IEEE Transactions on Power Systems, vol. 29, no. 4, pp. 1711– 1720, 2014. [13] E. Bloustein, “Assessment of customer response to real time pricing,” Rutgers-The State University of New Jersey, Tech. Rep, 2005. [14] S. Borenstein, “Equity effects of increasing-block electricity pricing,” Center for the Study of Energy Markets, 2008. [15] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, “Distributed optimization and statistical learning via the alternating direction method of multipliers,” Foundations and Trends® in Machine Learning, vol. 3, no. 1, pp. 1–122, 2011. [16] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge Uni- versity Press, 2004. [17] S. Boyd, L. Xiao, A. Mutapcic, and J. Mattingley, “Notes on decompo- sition methods,” Notes for EE364B, Stanford University, pp. 1–36, 2007. [18] A. Bressan, “Noncooperative differential games. A tutorial,” Depart- ment of Mathematics, Penn State University, Tech. Rep., 2010. [19] H. Burroughs and S. J. Hansen, Managing Indoor Air Quality. The Fairmont Press, Inc. and CRC Press, 2004. [20] E. Campos-Na˜ nez, A. Garcia, and C. Li, “A game-theoretic approach to efficient power management in sensor networks,” Operations Re- search, vol. 56, no. 3, pp. 552–561, 2008. 133 [21] Y. Cao and W. Ren, “Optimal linear-consensus algorithms: an lqr per- spective,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 40, no. 3, pp. 819–830, 2010. [22] B. Chai, J. Chen, Z. Yang, and Y. Zhang, “Demand response manage- ment with multiple utility companies: A two-level game approach,” IEEE Transactions on Smart Grid, vol. 5, no. 2, pp. 722–731, 2014. [23] N. Chatzipanagiotis, D. Dentcheva, and M. M. Zavlanos, “An aug- mented lagrangian method for distributed optimization,” Mathemati- cal Programming, vol. 152, no. 1-2, pp. 405–434, 2015. [24] L. Chen, N. Li, S. H. Low, and J. C. Doyle, “Two market models for demand response in power networks,” Proc. IEEE SmartGridComm, vol. 10, pp. 397–402, 2010. [25] L. Chen, S. H. Low, and J. C. Doyle, “Random access game and medium access control design,” IEEE/ACM Transactions on Network- ing (TON), vol. 18, no. 4, pp. 1303–1316, 2010. [26] S. Datchanamoorthy, S. Kumar, Y. Ozturk, and G. Lee, “Optimal time- of-use pricing for residential load control,” in Smart Grid Communica- tions (SmartGridComm), 2011 IEEE International Conference on. IEEE, 2011, pp. 375–380. [27] R. Deng, J. Chen, X. Cao, Y. Zhang, S. Maharjan, and S. Gjess- ing, “Sensing-performance tradeoff in cognitive radio enabled smart grid,” Smart Grid, IEEE Transactions on, vol. 4, no. 1, pp. 302–310, 2013. [28] R. Deng, G. Xiao, R. Lu, and J. Chen, “Fast distributed demand re- sponse with spatially and temporally coupled constraints in smart grid,” IEEE Transactions on Industrial Informatics, vol. 11, no. 6, pp. 1597–1606, 2015. [29] R. Deng, Z. Yang, J. Chen, N. R. Asr, and M.-Y. Chow, “Residen- tial energy consumption scheduling: A coupled-constraint game ap- proach,” IEEE Transactions on Smart Grid, vol. 5, no. 3, pp. 1340–1350, 2014. 134 [30] R. Deng, Z. Yang, M.-Y. Chow, and J. Chen, “A survey on demand response in smart grids: Mathematical models and approaches,” IEEE Transactions on Industrial Informatics, vol. 11, no. 3, pp. 570–582, 2015. [31] R. Deng, Y. Zhang, S. He, J. Chen, and X. S. Shen, “Maximizing net- work utility of rechargeable sensor networks with spatiotemporally- coupled constraints,” IEEE Journal on Selected Areas in Communications, DOI: 10.1109/JSAC.2016.2520181, to appear. [32] A. Dounis and D. Manolakis, “Design of a fuzzy system for living space thermal-comfort regulation,” Applied Energy, vol. 69, no. 2, pp. 119–144, 2001. [33] A. I. Dounis and C. Caraiscos, “Advanced control systems engineer- ing for energy and comfort management in a building environment— a review,” Renewable and Sustainable Energy Reviews, vol. 13, no. 6, pp. 1246–1261, 2009. [34] W. T. Elsayed and E. F. El-Saadany, “A fully decentralized approach for solving the economic dispatch problem,” IEEE Transactions on Power Systems, vol. 30, no. 4, pp. 2179–2189, 2015. [35] R. Engelking, General topology. Heldermann, 1989. [36] Environment Canada. Weather in Edmonton. [Online]. Available: http://edmonton.weatherstats.ca/ [37] T. Ericson, “Direct load control of residential water heaters,” Energy Policy, vol. 37, no. 9, pp. 3502–3512, 2009. [38] X. Fang, S. Misra, G. Xue, and D. Yang, “Smart gridthe new and im- proved power grid: A survey,” Communications Surveys & Tutorials, IEEE, vol. 14, no. 4, pp. 944–980, 2012. [39] H. Farhangi, “The path of the smart grid,” Power and Energy Magazine, IEEE, vol. 8, no. 1, pp. 18–28, 2010. [40] F. Felder and S. Newell, “Quantifying demand response benefits in pjm,” The Brattle Group, Inc, 2007. 135 [41] N. Forouzandehmehr, S. M. Perlaza, Z. Han, and H. V . Poor, “A sat- isfaction game for heating, ventilation and air conditioning control of smart buildings,” in Proc. IEEE GLOBECOM, 2013, pp. 3164–3169. [42] D. Fudenberg and J. Tirole, Game theory. MIT Press, 1991. [43] E. Ghadimi, A. Teixeira, I. Shames, and M. Johansson, “Optimal pa- rameter selection for the alternating direction method of multipliers (admm): quadratic problems,” IEEE Transactions on Automatic Control, vol. 60, no. 3, pp. 644–658, 2015. [44] H. Gharavi and R. Ghafurian, Smart grid: The electric energy system of the future. IEEE, 2011. [45] J. M. Guerrero, M. Chandorkar, T.-L. Lee, and P . C. Loh, “Advanced control architectures for intelligent microgrids, part i: decentralized and hierarchical control,” IEEE Transactions on Industrial Electronics, vol. 60, no. 4, pp. 1254–1262, 2013. [46] V . C. Gungor, D. Sahin, T. Kocak, S. Ergut, C. Buccella, C. Cecati, and G. P . Hancke, “A survey on smart grid potential applications and communication requirements,” IEEE Transactions on Industrial Infor- matics, vol. 9, no. 1, pp. 28–42, 2013. [47] V . C. Gungor, D. Sahin, T. Kocak, S. Ergut, C. Buccella, C. Cecati, and G. P . Hancke, “Smart grid technologies: Communication technologies and standards,” IEEE transactions on Industrial informatics, vol. 7, no. 4, pp. 529–539, 2011. [48] H. Hao, J. Lian, K. Kalsi, and J. Stoustrup, “Distributed flexibility characterization and resource allocation for multi-zone commercial buildings in the smart grid,” in Proc. IEEE CDC, 2015, pp. 3161–3168. [49] H. Hao, B. M. Sanandaji, K. Poolla, and T. L. Vincent, “Aggregate flexibility of thermostatically controlled loads,” IEEE Transactions on Power Systems, vol. 30, no. 1, pp. 189–198, 2015. 136 [50] H. Hao, Y. Sun, T. E. Carroll, and A. Somani, “A distributed cooper- ative power allocation method for campus buildings,” in Proc. IEEE PES General Meeting, 2015, pp. 1–5. [51] K. Herter, “Residential implementation of critical-peak pricing of electricity,” Energy Policy, vol. 35, no. 4, pp. 2121–2130, 2007. [52] C. Ibars, M. Navarro, and L. Giupponi, “Distributed demand man- agement in smart grid with a congestion game,” in Proc. IEEE Smart- GridComm, 2010, pp. 495–500. [53] D. Kolokotsa, G. Stavrakakis, K. Kalaitzakis, and D. Agoris, “Genetic algorithms optimized fuzzy controller for the indoor environmental management in buildings implemented using plc and local operat- ing networks,” Engineering Applications of Artificial Intelligence, vol. 15, no. 5, pp. 417–428, 2002. [54] M. Kummert, P . Andr´ e, and J. Nicolas, “Optimal heating control in a passive solar commercial building,” Solar Energy, vol. 69, pp. 103–116, 2001. [55] J. Kwak, P . Varakantham, R. Maheswaran, M. Tambe, F. Jazizadeh, G. Kavulya, L. Klein, B. Becerik-Gerber, T. Hayes, and W. Wood, “SAVES: A sustainable multiagent application to conserve building energy considering occupants,” in Proc. AAMAS, 2012, pp. 21–28. [56] G. J. Levermore, Building Energy Management Systems: Applications to Low-Energy HV AC and Natural Ventilation Control. Taylor & Francis, 2000. [57] N. Li, L. Chen, and S. H. Low, “Optimal demand response based on utility maximization in power networks,” in Power and Energy Society General Meeting, 2011 IEEE. IEEE, 2011, pp. 1–8. [58] N. Li and J. R. Marden, “Designing games to handle coupled con- straints,” in Decision and Control (CDC), 2010 49th IEEE Conference on. IEEE, 2010, pp. 250–255. 137 [59] N. Li and J. R. Marden, “Designing games for distributed optimiza- tion,” IEEE Journal of Selected Topics in Signal Processing, vol. 7, no. 2, pp. 230–242, 2013. [60] X. Li and J. Wen, “Review of building energy modeling for control and operation,” Renewable and Sustainable Energy Reviews, vol. 37, pp. 517–537, 2014. [61] H. Liang, B. J. Choi, A. Abdrabou, W. Zhuang, and X. S. Shen, “Decen- tralized economic dispatch in microgrids via heterogeneous wireless networks,” IEEE Journal on Selected Areas in Communications, vol. 30, no. 6, pp. 1061–1074, 2012. [62] J. Ma, J. Qin, T. Salsbury, and P . Xu, “Demand reduction in building energy systems based on economic model predictive control,” Chemi- cal Engineering Science, vol. 67, no. 1, pp. 92–100, 2012. [63] J. Ma, S. J. Qin, and T. Salsbury, “Model predictive control of building energy systems with balanced model reduction,” in American Control Conference (ACC), 2012. IEEE, 2012, pp. 3681–3686. [64] J. Ma, S. J. Qin, and T. Salsbury, “Experimental study of economic model predictive control in building energy systems,” in American Control Conference (ACC), 2013. IEEE, 2013, pp. 3753–3758. [65] J. Ma, S. J. Qin, B. Li, and T. Salsbury, “Economic model predictive control for building energy systems,” in Proc. IEEE ISGT, 2011, pp. 1–5. [66] J. Ma, S. J. Qin, and T. Salsbury, “Application of economic mpc to the energy and demand minimization of a commercial building,” Journal of Process Control, vol. 24, no. 8, pp. 1282–1291, 2014. [67] S. Maharjan, Q. Zhu, Y. Zhang, S. Gjessing, and T. Basar, “Dependable demand response management in the smart grid: A stackelberg game approach,” IEEE Transactions on Smart Grid, vol. 4, no. 1, pp. 120–132, 2013. 138 [68] G. Mantovani and L. Ferrarini, “Temperature control of a commercial building with model predictive control techniques,” IEEE Transactions on Industrial Electronics, vol. 62, no. 4, pp. 2651–2660, 2015. [69] J. R. Marden and M. Effros, “The price of selfishness in network cod- ing,” IEEE Transactions on Information Theory, vol. 58, no. 4, pp. 2349– 2361, 2012. [70] J. R. Marden, H. P . Young, and L. Y. Pao, “Achieving pareto optimality through distributed learning,” SIAM Journal on Control and Optimiza- tion, vol. 52, no. 5, pp. 2753–2770, 2014. [71] K. J. McCartney and J. F. Nicol, “Developing an adaptive control al- gorithm for europe,” Energy and buildings, vol. 34, no. 6, pp. 623–635, 2002. [72] A.-H. Mohsenian-Rad, V . W. Wong, J. Jatskevich, R. Schober, and A. Leon-Garcia, “Autonomous demand-side management based on game-theoretic energy consumption scheduling for the future smart grid,” IEEE Transactions on Smart Grid, vol. 1, no. 3, pp. 320–331, 2010. [73] J. W. Moon and J.-J. Kim, “Ann-based thermal control models for res- idential buildings,” Building and Environment, vol. 45, no. 7, pp. 1612– 1625, 2010. [74] D. Mosk-Aoyama, T. Roughgarden, and D. Shah, “Fully distributed algorithms for convex optimization problems,” in DISC, vol. 7, 2007, pp. 492–493. [75] D. Mosk-Aoyama, T. Roughgarden, and D. Shah, “Fully distributed algorithms for convex optimization problems,” SIAM Journal on Opti- mization, vol. 20, no. 6, pp. 3260–3279, 2010. [76] J. O. Neel, J. H. Reed, R. P . Gilles et al., “Convergence of cognitive radio networks.” in Proc. IEEE WCNC, vol. 4, 2004, pp. 2250–2255. [77] H. K. Nguyen, J. B. Song, and Z. Han, “Demand side management to reduce peak-to-average ratio using game theory in smart grid,” in Proc. IEEE INFOCOM Workshops, 2012, pp. 91–96. 139 [78] F. Oldewurtel, A. Parisio, C. N. Jones, M. Morari, D. Gyalistras, M. Gwerder, V . Stauch, B. Lehmann, and K. Wirth, “Energy efficient building climate control using stochastic model predictive control and weather predictions,” in Proc. IEEE ACC, 2010, pp. 5100–5105. [79] R. Olfati-Saber, J. A. Fax, and R. M. Murray, “Consensus and coop- eration in networked multi-agent systems,” Proceedings of the IEEE, vol. 95, no. 1, pp. 215–233, 2007. [80] R. Olfati-Saber and R. M. Murray, “Consensus problems in networks of agents with switching topology and time-delays,” IEEE Transac- tions on automatic control, vol. 49, no. 9, pp. 1520–1533, 2004. [81] P . Palensky and D. Dietrich, “Demand side management: Demand response, intelligent energy systems, and smart loads,” Industrial In- formatics, IEEE Transactions on, vol. 7, no. 3, pp. 381–388, 2011. [82] D. P . Palomar and M. Chiang, “A tutorial on decomposition methods for network utility maximization,” IEEE Journal on Selected Areas in Communications, vol. 24, no. 8, pp. 1439–1451, 2006. [83] Q. QDR, “Benefits of demand response in electricity markets and rec- ommendations for achieving them,” 2006. [84] N. Rahbari-Asr, U. Ojha, Z. Zhang, and M.-Y. Chow, “Incremen- tal welfare consensus algorithm for cooperative distributed gener- ation/demand response in smart grid,” IEEE Transactions on Smart Grid, vol. 5, no. 6, pp. 2836–2845, 2014. [85] V . Reddy, S. Shakkottai, A. Sprintson, and N. Gautam, “Multipath wireless network coding: A population game perspective,” in INFO- COM, 2010 Proceedings IEEE. IEEE, 2010, pp. 1–9. [86] J. B. Rosen, “Existence and uniqueness of equilibrium points for con- cave n-person games,” Econometrica, vol. 33, no. 3, pp. 520–534, 1965. [87] W. Saad, Z. Han, H. V . Poor, and T. Bas ¸ar, “Game-theoretic methods for the smart grid: An overview of microgrid systems, demand-side 140 management, and smart grid communications,” IEEE Signal Process- ing Magazine, vol. 29, no. 5, pp. 86–105, 2012. [88] A. I. Sabbah, A. El-Mougy, and M. Ibnkahla, “A survey of networking challenges and routing protocols in smart grids,” IEEE Transactions on Industrial Informatics, vol. 10, no. 1, pp. 210–221, 2014. [89] J. Saebi, H. Taheri, J. Mohammadi, and S. S. Nayer, “Demand bid- ding/buyback modeling and its impact on market clearing price,” in Energy Conference and Exhibition (EnergyCon), 2010 IEEE International. IEEE, 2010, pp. 791–796. [90] P . Samadi, A.-H. Mohsenian-Rad, R. Schober, V . W. Wong, and J. Jatskevich, “Optimal real-time pricing algorithm based on utility maximization for smart grid,” in Proc. IEEE SmartGridComm, 2010, pp. 415–420. [91] H. Scherer, M. Pasamontes, J. Guzm´ an, J. ´ Alvarez, E. Camponog- ara, and J. Normey-Rico, “Efficient building energy management us- ing distributed model predictive control,” Journal of Process Control, vol. 24, no. 6, pp. 740–749, 2014. [92] G. Scutari, D. P . Palomar, J.-S. Pang, and F. Facchinei, “Flexible design of cognitive radio wireless systems,” IEEE Signal Processing Magazine, vol. 26, no. 5, pp. 107–123, 2009. [93] P . H. Shaikh, N. B. M. Nor, P . Nallagownden, I. Elamvazuthi, and T. Ibrahim, “A review on optimized control systems for building en- ergy and comfort management of smart sustainable buildings,” Re- newable and Sustainable Energy Reviews, vol. 34, 2014. [94] P . Siano, “Demand response and smart grids—A survey,” Renewable and Sustainable Energy Reviews, vol. 30, pp. 461–478, 2014. [95] J. M. Sinopoli, Smart buildings systems for architects, owners and builders. Butterworth-Heinemann, 2009. 141 [96] J. ˇ Sirok` y, F. Oldewurtel, J. Cigler, and S. Pr´ ıvara, “Experimental analy- sis of model predictive control for an energy efficient building heating system,” Applied Energy, vol. 88, no. 9, pp. 3079–3087, 2011. [97] K. I. Tsianos, S. Lawlor, and M. G. Rabbat, “Consensus-based dis- tributed optimization: Practical issues and applications in large-scale machine learning,” in Communication, Control, and Computing (Aller- ton), 2012 50th Annual Allerton Conference on. IEEE, 2012, pp. 1543– 1550. [98] R. Tyagi and J. W. Black, “Emergency demand response for distribu- tion system contingencies,” in Transmission and Distribution Conference and Exposition, 2010 IEEE PES. IEEE, 2010, pp. 1–4. [99] C. Wang and N. Xiu, “Convergence of the gradient projection method for generalized convex minimization,” Computational Optimization and Applications, vol. 16, no. 2, pp. 111–120, 2000. [100] Y. Wang, S. Wang, and L. Wu, “Distributed optimization approaches for emerging power systems operation: A review,” Electric Power Sys- tems Research, vol. 144, pp. 127–135, 2017. [101] T. Weng and Y. Agarwal, “From buildings to smart buildings— sensing and actuation to improve energy efficiency,” IEEE Design & Test of Computers, vol. 29, no. 4, pp. 36–44, 2012. [102] J. A. Wright, H. A. Loosemore, and R. Farmani, “Optimization of building thermal design and control by multi-criterion genetic algo- rithm,” Energy and Buildings, vol. 34, no. 9, pp. 959–972, 2002. [103] C. Wu, H. Mohsenian-Rad, and J. Huang, “Vehicle-to-aggregator in- teraction game,” IEEE Transactions on Smart Grid, vol. 3, no. 1, pp. 434–442, 2012. [104] P . Xu, P . Haves, M. A. Piette, and L. Zagreus, “Demand shifting with thermal mass in large commercial buildings: Field tests, simulation and audits,” Lawrence Berkeley National Laboratory, 2005. 142 [105] S. Yang, S. Tan, and J.-X. Xu, “Consensus based approach for eco- nomic dispatch problem in a smart grid,” IEEE Transactions on Power Systems, vol. 28, no. 4, pp. 4416–4426, 2013. [106] M. Zaheer-Uddin and G. Zheng, “Optimal control of time-scheduled heating, ventilating and air conditioning processes in buildings,” En- ergy Conversion and Management, vol. 41, no. 1, pp. 49–60, 2000. [107] Z. Zhang and M.-Y. Chow, “Incremental cost consensus algorithm in a smart grid environment,” in Power and Energy Society General Meeting, 2011 IEEE. IEEE, 2011, pp. 1–6. [108] Z. Zhang and M.-Y. Chow, “Convergence analysis of the incremen- tal cost consensus algorithm under different communication net- work topologies in a smart grid,” IEEE Transactions on Power Systems, vol. 27, no. 4, pp. 1761–1768, 2012.

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Integration of energy-efficient infrastructures and policies in smart grid

PDF

Discrete optimization for supply demand matching in smart grids

PDF

The smart grid network: pricing, markets and incentives

PDF

Economic model predictive control for building energy systems

PDF

Distributed adaptive control with application to heating, ventilation and air-conditioning systems

PDF

Utility – customer interface strategy and user application for electric vehicles and distributed generation management

PDF

Electric vehicle integration into the distribution grid: impact, control and forecast

PDF

Residential electricity demand in the context of urban warming: leveraging high resolution smart meter data to quantify spatial and temporal patterns…

PDF

Energy efficient design and provisioning of hardware resources in modern computing systems

PDF

Prediction models for dynamic decision making in smart grid

PDF

Interaction and topology in distributed multi-agent coordination

PDF

I. Asynchronous optimization over weakly coupled renewal systems

PDF

Computational validation of stochastic programming models and applications

PDF

On practical network optimization: convergence, finite buffers, and load balancing

PDF

Optimizing distributed storage in cloud environments

PDF

Novel and efficient schemes for security and privacy issues in smart grids

PDF

User-centric smart sensing for non-intrusive electricity consumption disaggregation in buildings

PDF

Variation-aware circuit and chip level power optimization in digital VLSI systems

PDF

Improving mobility in urban environments using intelligent transportation technologies

PDF

Elements of robustness and optimal control for infrastructure networks

Asset Metadata

Creator Zhang, Zhaohui (author)

Core Title Demand response management in smart grid from distributed optimization perspective

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Chemical Engineering

Publication Date 04/17/2017

Defense Date 03/03/2017

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag building energy saving,demand response,distributed optimization,electric power systems,OAI-PMH Harvest

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Qin, S. Joe (committee chair), Ioannou, Petros (committee member), Shing, Katherine (committee member)

Creator Email happyzzh@gmail.com,zhaohuiz@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c40-355271

Unique identifier UC11256148

Identifier etd-ZhangZhaoh-5185.pdf (filename),usctheses-c40-355271 (legacy record id)

Legacy Identifier etd-ZhangZhaoh-5185.pdf

Dmrecord 355271

Document Type Dissertation

Rights Zhang, Zhaohui

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA