Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Real-time controls in revenue management and service operations
(USC Thesis Other)
Real-time controls in revenue management and service operations
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
REAL-TIME CONTROLS IN REVENUE MANAGEMENT AND SERVICE OPERATIONS by Jeunghyun Kim A Dissertation Presented to the FACULTY OF THE GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (BUSINESS ADMINISTRATION) May 2016 Copyright 2016 Jeunghyun Kim To my parents, children, and wife. ii Acknowledgments This dissertation is one of the biggest accomplishments in my life and I want to take this chance to express my appreciation to people who has been there to lead and encourage me. My biggest appreciation goes to my wife, Jiwon. Without her encouraging spirit, I would not have overcome many of struggles I had during my graduate studies. Her love and dedication make the impossible possible. Also, I want to thank my two children, Gawon and Dowon, for how wonderful children they are (they are too young to read it now, but I will show this dissertation to them when they grow up). They have brought me so much joy. Also, I am really grateful to my parents and parents in law for their unconditional supports. During my graduate studies at the University of Southern California, I have been extremely fortunate to have Professor Amy R. Ward and Professor Ramandeep S. Randhawa as my dissertation co-advisors. They have turned me into a management scientist from a student who was wondering around not knowing what to do. Having worked with them is such a big joy and I hope we can remain not only as colleagues but also as good friends in the future. Also, I want to thank Professors Hamid Nazerzadeh and Michael J. Neely for their guidance in completing my dissertation. Finally, I thank my friends in Bri B6 and ACC 215. It has been my pleasure to share the office with them and discuss things on many topics. My stay at the Marshall School of Business might have not been entertaining without them. I hope I can help them in any forms in their careers. iii Contents Acknowledgments iii Contents iv List of Tables vii List of Figures viii Abstract x 1 Asymptotically Optimal Dynamic Pricing in Queueing Systems 1 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.1 Literature review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.3 Asymptotic Analysis: Preliminaries and Static Pricing . . . . . . . . . . . . . . . . . . . . . 7 1.3.1 Fluid Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.3.2 Refining the Fluid Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.3.3 Static pricing benchmark . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.4 Dynamic Pricing: Asymptotic Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.4.1 The optimal two-price policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 iv 1.4.2 An intuitive argument . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.4.3 An upper bound on the performance of dynamic pricing . . . . . . . . . . . . . . . 15 1.4.4 Computing asymptotically optimal dynamic prices . . . . . . . . . . . . . . . . . . 16 1.4.5 Numerical study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 1.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 1.5.1 Role of uncertainty in customer valuations . . . . . . . . . . . . . . . . . . . . . . 23 1.5.2 Social welfare maximization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 1.5.3 Settings with lead-time quotation and order expediting . . . . . . . . . . . . . . . . 26 1.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2 Dynamic Scheduling of aGI/GI/1+GI Queue with Multiple Customer Classes 32 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.2 The Model and Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 2.2.1 The Heavy Traffic Limit Regime . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 2.3 The Brownian Control Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 2.4 The Brownian Control Problem Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 2.4.1 The Workload Control Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 2.4.2 The Workload Control Problem Solution . . . . . . . . . . . . . . . . . . . . . . . 45 2.5 The Proposed Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 2.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3 Dynamic Scheduling in a Many-Server Multi-Class System: the Role of Customer Impatience in Large Systems 55 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 3.1.1 Literature review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 v 3.2 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 3.3 The Diffusion Control Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 3.3.1 The DCP formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 3.3.2 The DCP solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 3.4 Static Priority and Threshold Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 3.5 A Two-class Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 3.5.1 Optimality of the threshold and static priority . . . . . . . . . . . . . . . . . . . . . 75 3.5.2 Sub-optimality of threshold structure . . . . . . . . . . . . . . . . . . . . . . . . . 76 3.6 Numerical Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 3.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 References 84 A Technical Appendix to Chapter 1 91 A.1 The MDP solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 A.2 Proofs of Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 A.3 Proofs of lemmas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 B Technical Appendix to Chapter 2 124 B.1 Proofs of Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 B.2 Proofs of lemmas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 C Technical Appendix to Chapter 3 156 C.1 Proofs of Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 C.2 Proofs of lemmas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 vi List of Tables 3.1 The role of the safety staffing coefficient (β) in the performance improvement . . . . . . . . 82 vii List of Figures 1.1 Accuracy of drift control problem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 1.2 Performance of two-price (TP) policy relative to static pricing and the DCP-based policy. . . 22 1.3 Change in the optimal refinement function as the system scale grows. . . . . . . . . . . . . 22 1.4 Performance comparison. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.1 TheGI/GI/1+GI queue withN-customer classes, having classk arrival rateλf k , service rateµ k , abandonment distributionF k with associated hazard rate functionh k , and cost per customer abandonmentc k . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.2 (x,w κ (x)) forx≥ 0 andκ∈{κ 1 ,κ 2 ,κ ⋆ ,κ 4 ,κ 5 } whereκ 1 >κ 2 >κ ⋆ >κ 4 >κ 5 . . . . . . 48 2.3 The simulated time average abandonment cost when the system operates under our proposed policy, as compared to the two possible static priority policies, for the system parameters given in the last two paragraphs of Section 2.5. . . . . . . . . . . . . . . . . . . . . . . . . 52 3.1 Description of our proposed policy and threshold policy . . . . . . . . . . . . . . . . . . . . 57 3.2 Visualization of the dimensional reduction. . . . . . . . . . . . . . . . . . . . . . . . . . . 72 3.3 q ⋆ 1 under different patience time distributions. . . . . . . . . . . . . . . . . . . . . . . . . . 77 3.4 Performance comparison. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 viii 3.5 Performance comparison between our proposed policy and FQR control. . . . . . . . . . . . 81 3.6 Performance comparisons between FQR control and threshold control. . . . . . . . . . . . . 82 ix Abstract Operations managers today must solve increasingly difficult resource allocation problems. This is because sophisticated and demanding customers expect ever-increasing product and/or service variety, provided ever- shortening lead-times. Inevitably, stochastic optimization models that represent such complex real-life situ- ations lead to complex policies. The drawback of such policies is that they are often opaque, and, therefore, difficult to explain to a system manager. Furthermore, the exact optimal policy may be very sensitive to the model assumptions and those assumptions are rarely exactly satisfied in reality. This observation has moti- vated my research philosophy devising simple and robust policies to allocate scarce resource with provable performance guarantees. This philosophy has guided my approach to the problems I worked on during my graduate studies, pricing in revenue management and scheduling in service operations. In the following, I summarize my work on each of the aforementioned problems. The first chapter “Asymptotically Optimal Dynamic Pricing in Queueing Systems” is based on a joint work with Professor Ramandeep S. Randhawa. We study optimal dynamic pricing to maximize revenues in queueing systems with price and delay sensitive customers. The system congestion is visible so that upon arrival, customers decide to join the system based on the congestion and the price at that time. We analyze this problem in the typical asymptotic regime of large customer market size and capacity. This asymptotic analysis involves solving a first order or fluid optimization problem that ignores stochastic variability and then refining it by minimizing the revenue loss that occurs due to stochasticity. Denoting the market size byn, one expects the revenue loss due to stochasticity to be on theO( √ n) scale. However, surprisingly, we find that the optimal dynamic pricing leads to an order improvement and the loss due to stochasticity is on theO(n 1/3 ) scale. The corresponding asymptotic control problem also turns out to be non-conventional. We solve this problem to obtain a near-optimal dynamic pricing policy, and further, we show that a simple policy of using only two prices can achieve most of the benefits of dynamic pricing. x The second chapter “Dynamic Scheduling of a GI/GI/1 +GI Queue with Multiclass Customers” is based on a joint work with Professor Amy R. Ward. We consider a dynamic control problem for a GI/GI/1 +GI queue with multiclass customers. The customer classes are distinguished by their inter- arrival time, service time, and abandonment time distributions. There is a cost c k > 0 for every class k∈{1,2,...,N} customer that abandons the queue before receiving service. The objective is to minimize average cost by dynamically choosing which customer class the server should next serve each time the server becomes available (and there are waiting customers at least from two classes). It is not possible to solve this control problem exactly, and so we formulate an approximating Brownian control problem. The Brownian control problem incorporates the entire abandonment distribution of each customer class. We solve the Brownian control problem under the assumption that the abandonment distributions both have an increasing failure rate. We then interpret the solution to the Brownian control problem as a control for the original dynamic scheduling problem. Finally, we perform a simulation study to demonstrate the effectiveness of our proposed control. The solution technique developed in this chapter is generalized into the next chapter in which we consider many-server queueing systems. The third chapter “Dynamic Scheduling in a Many-Server Multi-Class System: the Role of Customer Impatience in Large Systems” is based on a joint work with Professors Amy R. Ward and Ramandeep S. Randhawa. We study optimal scheduling of customers in service systems, such as call centers. In such systems, customers typically hang up and abandon the system after waiting for a long time for their service to commence. Such abandonments are detrimental for the system, and so managers typically use scheduling as a tool to mitigate it. In this paper we study the interplay between customer impa- tience and scheduling decisions when managing heterogeneous customer classes. Specifically, our focus is on general patience distributions, in which a customers instantaneous propensity to abandon may change over time. We find that incorporating these temporal changes into the scheduling decisions can lead to a significant improvement in the system performance. Formally, we solve the underlying Diffusion Control Problem for these systems, and characterize near-optimal scheduling policies. One of our main results is that for a class of parameters, we establish the non-optimality of threshold policies, which are widely used and are near-optimal when temporal changes in customer impatience are ignored. We find it interesting to compare the performance between the fixed-queue-ratio policies under which the ratio of of queue-lengths among classes is maintained at a pre-specified level and the threshold control. In particular, we observe xi that simple fixed-queue-ratio policy is more efficient in managing customer impatience than the threshold control. xii Chapter 1 Asymptotically Optimal Dynamic Pricing in Queueing Systems 1.1 Introduction Time is an important attribute of many products and services with customers valuing quick access at a premium. Given that the ability of a firm to provide quick access changes with the congestion in the system, pricing dynamically as a function of the congestion seems to be a good strategy. Such an approach has been employed by manufacturers who set prices based on lead-times, in road-tolls, where toll-paying single drivers gain access to high occupancy carpool lanes, and more recently by firms such as Uber that use surge pricing to manage congestion between customers and available taxis. The growing prominence of dynamic pricing in practical applications suggests that it may provide significant benefits. Nevertheless, computing and implementing dynamic pricing can be a complex endeavor for the firm. Further, one expects optimal dynamic prices to be intricate functions of the congestion, and this complexity may also not be well received by customers. In this paper, our goal is to understand the value of dynamic pricing in congested systems, and further to investigate if most of this value can be reaped by using simple pricing policies. For our study, we use an analytical framework and consider a monopolistic firm offering a service to price and delay sensitive customers who arrive as a Poisson process. Customers are heterogeneous in their valuation for the service and have linear disutility in their waiting times. At the time of arrival, the customers compare their value for the service with the sum total of the price and waiting costs and then either join if the net value is positive or leave the system otherwise. The firm has limited and fixed capacity and we model the system as a single-server queue. The firm’s decision is to set prices as a function of the congestion or queue-length in order to maximize its long-term average revenue rate. As we will discuss in the literature review section, static versions of this problem, in which the firm sets a single price using steady-state queueing behavior, are well understood. However, the literature on dynamic pricing is quite 1 sparse and treats only some special cases. A priori, one expects some similarity between dynamic pricing and static pricing especially for large systems. This is so because for large systems both pricing methods, loosely speaking, attempt to minimize the system variability, or rather the revenue loss due to variability. Our result shows that this intuition doesn’t quite play out and in fact there is a fundamental difference between the two pricing methods in large systems; dynamic pricing can lead to an “order improvement” in performance relative to static pricing. Formally, we consider the typical large system asymptotics in which the potential customer arrival rate and processing capacity are both large, and increasing without bound. In this regime, static pricing, with linear customer disutility for waits, has been established to follow the conventional square-root behavior, i.e., the loss in revenue due to variability is ofO( √ n), wheren denotes the system size. Based on the literature in this area, one expects dynamic pricing to also result in anO( √ n)- scale loss in revenue, but with a smaller constant. Somewhat surprisingly, we find that dynamic pricing can mitigate the variability to the extent that the queue-lengths, and correspondingly, the loss in revenue is in fact of a smaller order; we identify this order to be the system scale raised to the power one-third, i.e., O(n 1/3 ). Further, we prove that a simple two-price policy, that sets a high price when the queues are large and a low price otherwise can reap most of the benefits of dynamic pricing. In particular, we prove that our proposed two-price policy achieves theO(n 1/3 )-scale up to a logarithmic term. Intuitively, the benefit from dynamic pricing that we find arises because a dynamic pricing policy uses price refinements to maintain congestion at a lower order by increasing prices when queues are long, but at the same time when the queues are short, the prices can be decreased to increase volume, and thus, the revenue. In this sense, dynamic pricing provides an order of magnitude improvement over static pricing. A mathematical explanation for this order improvement is as follows: the optimization problem set up to minimize the revenue loss due to variability trades-off the expected steady-state price refinement with the queue-length scaled by system capacity, n. The conventional logic suggests that both of these should be on the same order, which then would be the square-root order. However, by changing prices dynami- cally especially by introducing negative price corrections, one can maintain the expected steady-state price refinement at a much lower level, and hence balancing the trade-off with the expected queue-length leads to a lower order of revenue loss which is the one-third order. Technically, the optimization problem also contains the second moment of the steady-state price refinement, which is typically of a lower order than the expected steady-state price refinement. However, by changing the price dynamically, the first moment of the steady-state price refinement is lowered to an extent that the second moment of the steady-state price 2 refinement becomes important and balancing this with the scaled expected steady-state queue-length yields the one-third order. Our study sheds deeper managerial insights into the circumstances which make implementing dynamic pricing more appealing. We find that the value of dynamic pricing is strongly tied to how customer hetero- geneity is manifested. In particular, it is heterogeneity in valuations that is the key to deriving value from dynamic pricing. In our study, we find that in settings where customers have the same valuation, there is no order improvement in the revenue loss by dynamic pricing and hence dynamic pricing is less valuable. Our focus in the paper is on dynamic pricing for revenue maximization in settings with observable queue-lengths. However, our key insights extend to other settings as well. In the paper, we discuss two additional settings: one in which the firm operates as a social welfare maximizer, and another in the context of manufacturing, in which the profit maximizing firm quotes lead-times and can expedite orders at a cost. 1.1.1 Literature review This paper studies dynamic pricing to maximize revenues in queueing systems with observable congestion under large market asymptotics. To place this paper’s model and results in perspective, it is useful to view the related literature in terms of three dimensions: (a) Pricing method: do the prices change with congestion (dynamic pricing) or remain fixed (static pricing); (b) Observability of congestion: do the customers observe the congestion at time of joining the system or not; and (c) Mode of analysis: is the analysis exact or asymptotic. The literature that studies static pricing is much more extensive compared with that on dynamic pricing. One of the first papers in this literature is Naor (1969). That paper studies the optimal static price to be set when queues are observable. Another influential paper that considers static pricing is Mendelson and Whang (1990), which studies differentiation between different customer types when maximizing social welfare, but when the queues are not observable. Unobservable queues lend a certain simplicity to the analysis because they allow using steady-state congestion formulas directly, rather than dealing with the underlying Markov chain. More recent papers that study static prices with unobservable queues are Cachon and Feldman (2011), that compares subscription with pay-per-use, and Haviv and Randhawa (2014), that shows that a fixed price can perform very well without any knowledge of the overall demand. All these papers take an exact analysis approach. Maglaras and Zeevi (2003) is the first paper that studied the pricing problem asymptotically and 3 characterized the optimality of the “square-root” regime. That paper considers the case of unobservable queues. It is worth mentioning that in this literature, there are also papers that consider settings in which the queue-length is not directly observable, but the firm can announce lead-times. Plambeck and Ward (2008) considers such a model and uses an asymptotic analysis to characterize the optimal static price and dynamic sequencing policies when lead-times are quoted statically. Asymptotically, quoting lead-times is quite similar to simply making the queue-length visible. One key difference is that in manufacturing settings, expediting orders is allowed to ensure quoted lead-times are always met, which provides some additional flexibility. We elaborate on this distinction in Section 1.5.3. Turning to the literature on dynamic pricing, early papers in this area assume that customers are sensitive to only prices and not delay, and the firm changes prices dynamically because there is a cost to the firm from having high congestion. Examples of such papers are Low (1974) and Paschalidis and Tsitsiklis (2000). These two papers consider finite buffer systems. Low (1974) proves that prices are non-decreasing in the number of customers in the system, whereas Paschalidis and Tsitsiklis (2000) considers a multi-class system and numerically shows that static pricing can perform quite well. More recent papers that study related problems are Yoon and Lewis (2004), Maglaras (2006), and C ¸ il et al. (2011). Yoon and Lewis (2004) assumes a deterministic customer valuation but allows non-stationarity in arrival and service rates and establishes interesting structural properties of the optimal policy, and proposes a practical point-wise stationary approximation. Maglaras (2006) provides insights in pricing and sequencing policies by solving the fluid approximation: pricing and sequencing decisions are separable in large scale systems. C ¸ il et al. (2011) solves the optimal pricing and sequencing rules for a two-class model by solving the MDP and characterizes how the congestion in one class affects the pricing of the other class. In this literature, the papers closest to ours are Chen and Frank (2001) and Ata and Shneorson (2006). Both these papers study optimal dynamic pricing for delay sensitive customers with observable queues using exact analysis (Chen and Frank 2001 considers revenue maximization, whereas Ata and Shneorson 2006 considers social welfare maximization). The main difference between our work and these papers is that we use an asymptotic large system approach which sheds additional insight into the problem and helps characterize both the value of dynamic pricing and the ability of two-price policies in extracting bulk of this value. Papers that perform asymptotic analysis of dynamic pricing are Celik and Maglaras (2008) and Ata and Olsen (2009, 2013). Celik and Maglaras (2008) uses an asymptotic approach to study dynamic pricing to 4 maximize revenues when the firm quotes lead-times and expedites orders to ensure those quoted lead-times to be met. A key difference is that the system in Celik and Maglaras (2008) operates with “ample” capacity. In our work, we focus on settings in which capacity is constraining. We feel that this approach provides flexibility in modeling situations in which capacity decisions are made over a long horizon and further helps study the cases in which there are mismatches between supply and demand. Interestingly, this model change leads to a completely different structure of the asymptotic problem. The paper Ata and Olsen (2009) provides an interesting contrast to our work. That paper focuses on the case of deterministic customer valuations and proves that asymptotically, the optimal dynamic pricing achieves a revenue loss ofO( √ n)-scale. Contrasting this work with our main result demonstrates the fact that considering stochastic valuations for customers (somewhat unexpectedly) fundamentally changes the solution structure compared with the deterministic valuation case. In particular, we find that if customer val- uation is stochastic, then dynamic pricing becomes much more efficient in regulating stochastic variability and leads to an order improvement in the revenue loss, which is lowered fromO( √ n) toO n 1/3 . (Ata and Olsen 2013 is a multi-class extension of Ata and Olsen 2009.) While dynamic pricing is useful in dealing with congestible systems, its usefulness can be enhanced when there is parameter uncertainty or changes in underlying parameters. For instance, Afeche and Ata (2013) uses dynamic pricing to learn customer delay sensitivity and Besbes and Maglaras (2010) studies dynamic pricing when the demand is time-varying. The latter paper uses an observable queue framework with a customer model that is identical to ours. However, the authors use an asymptotic fluid approach to capture the changes in market size, whereas we use a diffusion based approach to capture the changes in congestion. In inventory systems, dynamic pricing has also been studied as a means of learning demand characteristics, see, for instance, Farias and Van Roy (2010), Eren and Maglaras (2010), and Besbes and Zeevi (2012). In our paper, we focus on the value of dynamic pricing when parameters are known but the congestion level varies. One of the down sides to dynamic pricing in such systems is that it leads to continuous price changes. Given that customers feel some distaste when confronted with frequent price changes and that such repug- nance can be a real constraint on markets (cf. Roth 2006), we feel that our finding that a two-price policy can perform extremely well, practically appealing. A related example of dynamic pricing in inventory systems is Netessine (2006) which studies the design of simple price policies when the arrival process is non-stationary by imposing a cost per price change. 5 1.2 Model We model the firm as a single-server queueing system that provides a service to customers who are price and delay sensitive. We assume customer service times are independent and identically distributed according to an exponential distribution with unit mean and that the server processes work at a fixed rate ofn. Customers are differentiated on their service valuation, which we assume is independent and identically distributed across customers with a distribution whose cumulative distribution and density functions are denoted byF andf, respectively. We assume thatf is continuously differentiable and its derivative is denoted byf ′ . We also assume that the customer valuation distribution has a non-decreasing hazard rate, i.e.,H(x) := f(x) 1−F(x) is non-decreasing. Customers are homogeneous on their sensitivity to delay and we useh to denote the per unit time cost of waiting. Upon arrival at timet, a customer observes the posted pricep(t) and the current queue-length Q(t). The customer joins the system if his randomly drawn valuationV exceeds the total expected cost of joining the system. Because the firm’s service rate isn, it follows that a customer arriving at timet joins the queue ifV >p(t)+hQ(t)/n. We assume that potential customers arrive according to a Poisson process with an arrival rate of nλ, which represents the market size. Thus, the effective arrival rate of customers who actually join the system at timet is given by nλP V >p(t)+h Q(t) n =nλ ¯ F p(t)+h Q(t) n , (1.1) where the tail distribution function ¯ F (·) := 1−F (·). The firm’s decision is to select the optimal dynamic pricing strategy, i.e., the functionp(·), in order to maximize the long-term average revenue. For a pricing strategyp, using (1.1), we can write firm’s rate of revenue accrual at timet as p(t)nλ ¯ F p(t)+h Q(t) n . So, the firm’s optimization problem is sup p limsup T→∞ 1 T Z T 0 p(t)nλ ¯ F p(t)+h Q(t) n dt. (1.2) 6 In this paper, we focus on stationary pricing policies so that p(t) is in fact p(Q(t)), and henceforth we interpret the pricing functionp as a function of the queue-length rather than the time explicitly. So, (1.2) can be cast in the following steady-state formulation: sup p∈π R n (p) :=E p(Q)nλ ¯ F p(Q)+h Q n , (1.3) where the expectation is with respect to the steady-state queue-length distribution andπ represents the set of stationary pricing policies that are non-anticipating. We denote the optimal objective function value by R ⋆ n , and the optimizing price function by p ⋆ n . Notice that in this problem, the queue-length’s steady- state distribution is intertwined with the choice of p, which makes solving (1.3) exactly not amenable to generating insights of the nature we seek. Therefore, we perform an asymptotic large system analysis in which the firm’s capacityn is large, and correspondingly, the market sizenλ is also large. 1.3 Asymptotic Analysis: Preliminaries and Static Pricing Our asymptotic approach proceeds in the conventional manner. In Section 1.3.1, We first analyze the system under a fluid approximation by taking a rate-based approach. Then, in Section 1.3.2, we perform some pre- liminary analysis for refining this approximation by incorporating the inter-temporal fluctuations associated with customer arrivals and services. Section 1.3.3 then characterizes the asymptotically optimal static price that also serves as a benchmark for our analysis of dynamic pricing in the next section. 1.3.1 Fluid Analysis In the fluid model of the system, services occur at a fixed rate ofn as long as there is work in the system and at time t, and customers arrive deterministically at the rate of nλ ¯ F p(Q(t))+h Q(t) n . Given the deterministic system behavior, it follows that the optimal pricing strategy for the fluid model is to maintain the queue-length at a zero level by ensuring that the customer arrival rate never exceeds the processing capacity. Thus, optimizing the fluid system entails solving sup p pnλ ¯ F(p) (1.4) s.t.nλ ¯ F(p)≤n. 7 We denote the unconstrained maximizer of the above program by p ⋆ := argmax p p ¯ F(p). Because the customer valuation distribution has a non-decreasing hazard rate, p ⋆ is unique, and further the objective function in (1.4) is concave. Thus, (1.4) is solved by the price ¯ p := max ¯ F −1 1 λ ,p ⋆ . We denote the optimal fluid objective function by ¯ R ⋆ n . We would like to emphasize that ¯ R ⋆ n serves as an important benchmark in our study. Note that ¯ R ⋆ n can be construed as an unachievable ideal revenue because it is obtained using a model that ignores stochasticity. As the ideal revenue, it is intuitive that ¯ R ⋆ n should exceed the revenue achieved under any pricing policy in the actual system in which stochasticity is present. Hence, for any pricing policy, we can interpret the gap in its performance relative to ¯ R ⋆ n as the revenue loss due to stochasticity. The following result formally establishes the intuition by proving that ¯ R ⋆ n bounds the optimal revenue from above. It also shows that the simple strategy of pricing at the fluid optimal price ¯ p, that is,p(q) = ¯ p for allq ≥ 0, lead too(n)-revenue loss. That is, the revenue loss as a fraction of the system scale converges to zero. Proposition 1.3.1. (a) The optimal fluid objective value is an upper bound for the revenue obtained under any pricing policy, i.e., for anyn> 0, we have R ⋆ n ≤ ¯ R ∗ n . (b) The static price ¯ p leads too(n)-revenue loss due to stochasticity, i.e., we have R n (¯ p) = ¯ R ⋆ n −o(n). We would like to highlight that though the naive policy of pricing at the fixed level ¯ p is optimal on the fluid scale, it ignores the queueing aspect of the problem completely. Hence, we only expect this policy to perform well if the system is not capacity constrained or is extremely large in scale. Because we anticipate capacity constraints to arise in practice, we focus on this setting by making the following assumption. Assumption 1.3.1. The system is capacity constrained so that the unconstrained optimizer of the fluid optimization program is not achievable, i.e., we haveλ ¯ F(p ⋆ )> 1. 8 We briefly discuss what happens if this assumption does not hold in Section 1.6 (we would like to point out that Celik and Maglaras 2008 considers the caseλ ¯ F(p ⋆ ) = 1). We analyze price refinements in the next section. 1.3.2 Refining the Fluid Approximation In this section, we refine the fluid optimal price ¯ p by considering prices of the form p n (q) = ¯ p+θ n (q) forq≥ 0, for some functionθ n . Because the static price ¯ p is optimal on the fluid-scale, we expect the refinementθ n to take small values. Thus, we focus on price refinements that are asymptotically zero, i.e.,θ n (q) → 0 as n→∞. We proceed with an informal argument that characterizes the revenue loss due to stochasticity, which will play a key role in our analysis. Our formal results will make this analysis precise. DenotingQ n as the steady- state queue-length, we expect the queue-length to be small relative ton for largen. So, we approximate the expected steady-state revenue by applying the Taylor series expansion to the term ¯ F(p n (Q n )+hQ n /n) around ¯ p as follows: R n (p n ) =E ¯ p+θ n (Q n ) nλ ¯ F ¯ p+θ n (Q n )+h Q n n (a) ≈nλE ¯ p+θ n (Q n ) ¯ F(¯ p)−f(¯ p) θ n (Q n )+h Q n n −f ′ (¯ p) θ n (Q n )+h Qn n 2 2 (b) ≈nλ¯ p ¯ F(¯ p)− −r ′ (¯ p)nλE[θ n (Q n )]+ f (¯ p)+ ¯ pf ′ (¯ p) 2 nλE[θ n (Q n ) 2 ]+h¯ pf(¯ p)λE[Q n ] = ¯ R ⋆ n − αnE[θ n (Q n )]+βnE[θ n (Q n ) 2 ]+γE[Q n ] . (1.5) where r(p) :=p ¯ F(p), α :=−r ′ (¯ p)λ, β := f (¯ p)+ ¯ pf ′ (¯ p)/2 λ, andγ :=h¯ pf(¯ p)λ. (1.6) In (1.5), we obtain(a) using the first two terms of the Taylor series expansion, and we obtain(b) by ignoring the lower order terms, specifically we ignore all terms of the formnE θ n (Q n ) i Qn n j fori+j≥ 2 except the termE[θ n (Q n ) 2 ], which is the second moment of the pricing refinement. In the next section, we will 9 prove that this second moment term (surprisingly) plays an important role in characterizing the optimal dynamic price. The term in parenthesis in (1.5) represents the revenue loss relative to the fluid problem arising due to stochasticity, and so our revenue maximization problem for the refined problem can be restated as minimiz- ing this revenue loss, i.e., we have inf θn αnE[θ n (Q n )]+βnE[θ n (Q n ) 2 ]+γE[Q n ] . (1.7) Thus, this refinement depends on the first and second moments of the steady-state price refinement and the expected steady-state queue-length. We now proceed by solving (1.7). We begin by analyzing the static pricing problem to illustrate the mode of analysis and because it serves as a good benchmark policy to compare dynamic pricing with. 1.3.3 Static pricing benchmark A static or fixed price is independent of the queue-length, and so we abuse notation and set the price p n (q) = ¯ p+θ n for allq≥ 0. Denoting the optimal static price byp ⋆ n,S , we have the following asymptotic characterization. Proposition 1.3.2. If Assumption 1.3.1 holds, then: (a) Static pricing leads toO( √ n)-revenue loss due to stochasticity, i.e., for some constant Π S > 0, we have R n (p ⋆ n,S ) = ¯ R ⋆ n −Π S √ n+o( √ n). Further, the asymptotically optimal static price is p ⋆ n,S = ¯ p+ 1 √ n π S for someπ S ∈R. (b) The expected queue-length under the asymptotically optimal price satisfies: 0< liminf n→∞ E[Q n ] √ n ≤ limsup n→∞ E[Q n ] √ n <∞. (1.8) 10 We next provide an intuitive derivation of this result, which will be useful to illustrate the key value of dynamic pricing in this problem. For this, we only considerθ n > 0. Noting that for largen, the termθ 2 n is much smaller, or more precisely, of a lower order compared withθ n , we can ignore it and this simplifies (1.7) to the following program: inf θn αnθ n +γE[Q n ]. (1.9) Next, consider the expected steady-state queue-length. We can obtain an upper bound for this term by ignoring the customers’ delay sensitivity, i.e., by letting the customer arrival rate depend only on the price. This upper bounding system is anM/M/1 queue with arrival ratenλ ¯ F(¯ p+θ n ). So, we have E[Q n ]≤ n n−nλ ¯ F (¯ p+θ n ) = 1 λf(¯ p)θ n +o 1 θ n . In fact, one can further show thatE[Q n ] = δ θn +o 1 θn , whereδ > 0 is some constant. So, (1.7) reduces to inf θn αnθ n + γδ θ n . Note thatλ ¯ F(p ⋆ )> 1 implies thatα =−r ′ (¯ p)λ> 0. Thus, we obtain thatθ ⋆ n =n − 1 2 q γδ α and the value of the objective function equals2 √ n √ αγδ. 1.4 Dynamic Pricing: Asymptotic Analysis In this section, we analyze dynamic pricing policies. First, in Section 1.4.1, we analyze a simple dynamic pricing policy that utilizes only two price levels (and hence referred to as “two-price” policy) and show that it achieves an order improvement over static pricing. Then, in Section 1.4.2, we provide some intuitive reasoning behind this order-improvement. In Section 1.4.3, we show that the performance of the two-price policy in Section 1.4.1 is indeed close to the optimal performance by computing an asymptotic upper bound on the performance of a general class of dynamic pricing policies. Finally, in Section 1.4.4, we formulate and solve a drift control problem to characterize the asymptotically near-optimal dynamic prices. Section 1.4.5 contains a numerical study that illustrates the key insights of this section. 11 1.4.1 The optimal two-price policy We begin our formal analysis of dynamic pricing by studying a class of policies that uses only two price levels and in this sense we refer to it as two-price (TP) policies. Such a policy sets one price for small queue-lengths and another price for large queue-lengths and can be characterized as follows: p n,TP (q) = ¯ p−θ − n ifq≤τ n , ¯ p+θ + n otherwise, (1.10) for some non-negative constantsθ − n ,θ + n andτ n . Defining φ =β− f ′ (¯ p) 2f (¯ p) α (1.11) and denoting the optimal revenue under two-price policies by R ⋆ n,TP , the following proposition provides an asymptotic characterization of the optimal two-price policy and its performance. It is useful to note that φ = 1 2 H(¯ p)+ H ′ (¯ p) H(¯ p) > 0 whereH denotes the hazard rate function of the valuation distribution Proposition 1.4.1. If Assumption 1.3.1 holds, then: (a) The optimal two-price policy leads toO n 1/3 (logn) 1/3 -revenue loss due to stochasticity, i.e., we have R ⋆ n,TP = ¯ R ⋆ n −n 1/3 (logn) 1/3 Π TP +o n 1/3 (logn) 1/3 , whereΠ TP :=φ 1/3 3h λf(¯ p) 2/3 . (b) The asymptotically optimal two-price policy is: p ⋆ n,TP (q) = ¯ p− logn (nlogn) 1/3 π ifq≤ (nlogn) 1/3 3λf(¯ p)π , ¯ p+ 3 (nlogn) 1/3 π otherwise, whereπ := 1 3 3h λf(¯ p)φ 1/3 . 1. The expected queue-length under the asymptotically optimal two-price policy in(b) is: E[Q n ] =n 1/3 (logn) 1/3 W TP +o n 1/3 (logn) 1/3 , (1.12) 12 whereW TP := 10h 3πλf(¯ p) . This result formally shows that dynamic pricing leads to the order improvement in revenue loss com- pared with static pricing, for which the revenue loss isO(n 1/2 ). Proposition 1.4.1 highlights another impor- tant benefit of using dynamic pricing over static pricing: lowering the congestion in the system while increasing the revenue. According to Proposition 1.4.1, dynamic pricing can achieve both the expected queue-length and the revenue loss of o( √ n). However, for static pricing, in order to lower the expected queue-length too( √ n), revenue is substantially sacrificed in the sense that the revenue loss becomes order of magnitude larger thanO( √ n). We would also like to comment that it is somewhat surprising that the asymptotically optimal two-price policy can be easily characterized, and that it is of such a simple form. In contrast, the asymptotically optimal static pricing cannot be characterized explicitly. This difference arises from the asymptotic analysis of the objective of (1.7). For instance, the expected queue-length term in the objective equals: E[Q n ] = P ∞ i=0 i Q i−1 j=0 λ ¯ F ¯ p+θ n (j)+h j n P ∞ i=0 Q i−1 j=0 λ ¯ F ¯ p+θ n (j)+h j n . (1.13) In the two-price policy, the price refinement is of a larger order relative to the traditional square-root order, implying that the price refinement dominates the queue-length effect and asymptotically the arrival rates take two values: low and high. This makes (1.13) easy to analyze. However, in static pricing, the price refinement and queue-length effect are on the same order, and hence asymptotically, we need to deal with the state dependent argument in the arrival rate, and this precludes a simple explicit characterization. The two-price policy we have identified provides us with a lower bound on the performance of dynamic pricing, and we expect allowing for additional prices will lead to even better performance. This leads to a natural question about how well can dynamic pricing be expected to perform if we allow general pricing policies. It turns out that our proposed two-price policy performs quite close to the optimal dynamic pricing policy from a broad class. Before presenting the formal result, we first discuss the intuition behind the order improvement result. 13 1.4.2 An intuitive argument Clearly, the analysis of dynamic pricing is more complicated than that for static pricing. However, it turns out that beyond complexity of analysis, there is also a fundamental benefit to dynamic pricing that leads to an “order improvement” over static pricing. We proceed by first illustrating this benefit using an informal argument. Unlike static pricing, in which the fixed pricing refinement is positive, under dynamic pricing this refinement can be negative for small queue-lengths and positive for long queue-lengths. If these prices are chosen properly, one can potentially ensure thatnE[θ n (Q n )]≈ 0 so that (1.7) reduces to inf θn βnE[θ n (Q n ) 2 ]+γE[Q n ] . (1.14) Similar to the static pricing case, one expects that the expected steady-state queue-length should be inversely proportional to the price refinement, in this case, the positive price refinement. That is,E[Q n ]≈ ν E[θn(Qn) + ] for someν > 0. Then, (1.14) reduces to inf θn βnE[θ n (Q n ) 2 ]+γ ν E[θ n (Q n ) + ] . (1.15) (It is our convention thata + := max{a,0} anda − :=−min{a,0} for anya∈R.) Expectingθ + n andθ − n functions to be of similar order, we further expect thatE[θ n (Q n ) 2 ] =O E[θ n (Q n ) + ] 2 and hence, ignoring the constants, the trade-off in (1.15) becomes that between the terms nE[θ n (Q n ) + ] 2 and 1 E[θ n (Q n ) + ] . It follows that the optimal refinement should set E[θ n (Q n ) + ] =O n − 1 3 . This choice of price refinement also results in the objective (1.14) beingO(n 1/3 ), i.e., the total revenue under such a dynamic pricing scheme is ¯ R ⋆ n −O(n 1/3 ) as compared with ¯ R ⋆ n −O(n 1/2 ), which one obtains under static pricing. Thus, dynamic pricing is able to mitigate the revenue loss that occurs due to stochasticity by varying the price. In particular, it uses a larger value of price refinement compared with the static pricing and when queue-lengths are small, the refinement is negative to prevent idleness of the server and as the queue-length increases, the refinement becomes positive to lower the queue-length. This leads to lower 14 congestion as well, with the expected steady-state queue-length being ofO n 1/3 under dynamic pricing compared withO n 1/2 under static pricing. Comparing this intuitive reasoning with Proposition 1.4.1, the main difference is that the two price policy achieves theO(n 1/3 ) performance up to logarithmic terms. The reason for this is that discontinuity of the pricing policy. The following section formalizes our intuitive argument for a general class of dynamic pricing policies and proves that their revenue loss due to stochasticity is lower bounded byO(n 1/3 ). 1.4.3 An upper bound on the performance of dynamic pricing We next establish an asymptotic upper bound on optimal revenue achievable using dynamic pricing. For asymptotic analysis, one typically introduces a scale factor that permits scaling of the decision variables as the system scale increases. Under dynamic pricing, there are infinite decision variables and so for making our asymptotic analysis tractable, we will construct a fairly general class of dynamic pricing policies and establish the bound for this class. Dynamic pricing policy class. We focus on a class of sequences of dynamic pricing policies denoted by P such that a sequence{p n }∈P is of the formp n (q) = ¯ p+θ n (q), where θ n (q) = s − n θ q τn ifθ q τn ≤ 0, s + n θ q τn otherwise, (1.16) with the following properties: (a) s − n ,s + n ,τ n are positive constants withlim n→∞ s + n = lim n→∞ s − n = 0. (b) θ is a non-decreasing, bounded function (possibly discontinuous). (c) Ifθ crosses zero, i.e., there exist 0 < x < y <∞ such thatθ(x) < 0 < θ(y), then we defineτ n as the “switch-point” so thatθ( q τn )≤ 0 for allq≤τ n andθ( q τn )≥ 0 otherwise. Equation (1.16) is motivated by the typical approach in asymptotic analysis to separate the scale from the decision variable. In this case, s − n ands + n are the scale parameters that multiply the decision variable θ when it is negative and positive, respectively. In asymptotic analysis, there is typically only one such 15 multiplying scale. However, we have observed in Proposition 1.4.1 that the optimal two-price policy had different scales for the positive and negative price refinements. So, to include such policies, we allow for these different scales. Further, because we expect the queue-length to be asymptotically large, we useτ n to scale down the queue-length in the argument ofθ. Before presenting the result, we would like to point out that the requirements that the price refinement functionθ be non-decreasing and bounded are technical conditions that we require for our analysis to work. Notice that a non-decreasingθ implies that the prices we consider are non-decreasing in the queue-length. Intuitively, one does expect a firm to charge higher prices as the congestion increases, so this require- ment does not seem too restrictive. Nevertheless, in the appendix, we formulate the exact (non-asymptotic) dynamic program that the firm faces and prove that the optimal pricep ⋆ n has the property thatp ⋆ n (q)+h q n , a customer’s total cost of joining the system, is non-decreasing inq (see Lemma A.1.1 in Appendix A.1). Because we expect p ⋆ n (q) = ¯ p +θ ⋆ n (q) with lim n→∞ |θ ⋆ n (q)| q/n = ∞, the optimal prices should indeed be asymptotically non-decreasing. So, restricting attention to this class of policies seems reasonable. We would also like to point out that we do not require any continuity or differentiability conditions onθ. Result. The following result proves that under any dynamic pricing policy from the classP, the smallest possible revenue loss due to stochasticity isO(n 1/3 ), and thus establishes a formal limit on the achievable performance of dynamic pricing. Proposition 1.4.2. If Assumption 1.3.1 holds, then for any sequence of dynamic pricing policy{p n } n≥1 ∈ P, there exists a constantK > 0 such that R n (p n )≤ ¯ R ⋆ n −Kn 1/3 . This result shows that the intuitive reasoning of Section 1.4.2 is tight with respect to the order of optimal- ity and dynamic pricing (within a large class of policies) cannot reduce the revenue loss to a value smaller thanKn 1/3 . 1.4.4 Computing asymptotically optimal dynamic prices In this section, we formulate a drift control problem (DCP) in the diffusion limit to propose the asymptoti- cally optimal pricing policy. 16 Approximating the system dynamics. In order to write out the limiting DCP, we need to approximate the system dynamics with an appropriate diffusion process. To do so, we first write out the exact system dynamics using the following notation: we defineN a andN s as two independent unit rate Poisson processes, I n as the cumulative server idle time process, and we use Q n to denote the queue-length process. Then, noting that the effective arrival rate at timet is Λ n (t) :=nλ ¯ F ¯ p+θ n (Q n (t))+h Q n (t) n and the amount of time the server is busy until timet is T n (t) :=t−I n (t), we can write Q n (t) =N a Z t 0 Λ n (s)ds −N s nT n (t) . We now apply the strong approximation to the arrival and the service completion processes and proceed as in Celik and Maglaras (2008). In particular, we assumeX a andX s are two independent standard Brownian motions so thatN i (t) =t+X i (t)+o( √ t) fori∈{a,s}. Then, we have N a Z t 0 Λ n (s)ds = Z t 0 Λ n (s)ds+ √ nX a Z t 0 Λ n (s) n ds +o √ nt N s (nT n (t)) =nt+ √ nX s (T n (t))−nI n (t)+o √ nt , implying that Q n (t) = Z t 0 (Λ n (s)−n)ds+ √ nX a Z t 0 Λ n (s) n ds − √ nX s (T n (t))+nI n (t)+o √ nt . Define Δ n (q) :=λ ¯ F ¯ p+θ n (q)+h q n − ¯ F (¯ p) , for allq≥ 0, so thatnΔ n (Q n (s)) = Λ n (s)−n is the drift of the system at times. Then, we can write Q n (t) =n Z t 0 Δ n (Q n (s))ds+ √ nX a Z t 0 Λ n (s) n ds − √ nX s (T n (t))+nI n (t)+o √ nt . 17 We next consider the termsX a andX s . Because we expectθ n → 0 andQ n /n→ 0 asn→∞, we have Z t 0 Λ n (s) n ds =t+o(t) and hence X a Z t 0 Λ n (s) n ds =X a (t)+o( √ t). Also, because the server should be busier when operating under this policy than when operating under the optimal static pricing policy, we should haveT n (t) =t 1−o 1 √ n , and so X s (T n (t)) =X s (t)+o n −1/4 √ t . Putting all these components together, we obtain the following approximation of the system dynamics: Q n (t) =n Z t 0 Δ n (Q n (s))ds+ √ 2nX(t)+nI n (t)+o √ nt , (1.17) whereX is another standard Brownian motion. The equation (1.17) motivates the use of a diffusion processZ n to approximate the queue-length process Q n whereZ n is the (weak) solution of the following stochastic differential equation: Z n (t) =n Z t 0 Δ n (Z n (s))ds+ √ 2nB(t)+nL n (t), (1.18) where L n is a non-decreasing process such that R t 0 L n (s)dZ n (s) = 0 for all t > 0, and B is another independent standard Brownian motion. The non-decreasing processL n relates well toI n in (1.17) because at any timet,L n can increase if and only ifZ n (t) = 0 and similarlyI n can increase if and only ifQ n (t) = 0. Solving the limiting DCP. With the state-dependent drift Δ in (1.18) as the decision variable, we now write out the DCP to propose an asymptotically optimal pricing policy. To do so, it will be convenient to rewrite the revenue loss due to stochasticity in terms of the drift rather than the price refinement in (1.5). Straightforward application of the Taylor series expansion yields ¯ R ⋆ n −R n (p n )≈nψE[Δ n (Q n )]+n φ (λf (¯ p)) 2 E h Δ n (Q n ) 2 i +hE[Q n ], (1.19) 18 whereψ :=− α λf(¯ p) andφ is the constant defined in (1.11). By recalling that the process Z n approximates the queue-length, we obtain the following DCP from (1.19): inf Δn nψE[Δ n (Z n )]+n φ (λf (¯ p)) 2 E h Δ n (Z n ) 2 i +hE[Z n ] , (1.20) where the expectation is with respect to the steady-state distribution ofZ n . In solving (1.20), we restrict attention to decreasing drift functions, i.e., Δ n (z) is decreasing in z. This is not restrictive, because Lemma A.1.1 in Appendix A.1 proves that the drift term in the pre-limit queueing system is also decreasing under the optimal pricing policy. We denote the optimizer and optimal objective value of (1.20) byΔ ⋆ n and ˆ R ⋆ n , respectively. The follow- ing result characterizes the solution to the DCP (1.20). Proposition 1.4.3. For eachn≥ 0, there exists a constantκ ⋆ and a continuously differentiable functiong such that the optimal control that solves (1.20) is Δ ⋆ n (q) =− (λf (¯ p)) 2 2φ (g(q)+ψ), for allq≥ 0, where the pair(g,κ ⋆ ) solves ng ′ (q)+hq−n (λf (¯ p)) 2 4φ (g(q)+ψ) 2 =κ ⋆ (1.21) withg(0) = 0 andg ′ (q) > 0 andg ′ (q) ≤ C √ q for allq > 0 for some positive constantC. Further, the optimal objective value of (1.20) ˆ R ⋆ n =κ ⋆ . Notice that for anyκ, (1.21) is an ordinary differential equation that can be solved explicitly. Combining this solution with the conditionsg ′ (q)> 0 andg ′ (q)≤C √ q for allq> 0, allows us to nail down the value ofκ, and thus easily obtain the optimal driftΔ ⋆ n . We would like to point out that unlike most control problems in the literature, the solution of (1.21) depends on the system scale. This occurs because the expected steady-state drift under the optimal solution is of a lower magnitude in order sense than its point-wise value, E[Δ ⋆ n (Zn(t))] Δ ⋆ n (Zn(t)) → 0 asn→∞. 19 Proposed policy. We now use the solution to the asymptotic DCP Δ ⋆ n (q) to propose a pricing policy for the actual system. We first set the drift in the pre-limit system asΔ ⋆ n (q) to obtain the price refinement as θ n (q) =− 1 λf(¯ p) Δ ⋆ n (q) forq≥ 0. So, the proposed pricing policy is to post the following price to a customer who arrives when the queue- length isq: ˆ p ⋆ n (q) = ¯ p− 1 λf(¯ p) Δ ⋆ n (q), (1.22) In the next section, we numerically compare the performance of this policy with that of the asymptotically optimal static price and two-price policies, and also with that of the optimal dynamic price obtained from exact analysis. 1.4.5 Numerical study We use numerical experiments to illustrate three points. First, we show that the solution obtained using the approximate DCP in Section 1.4.4 well approximates the exact optimal solution obtained by solving the underlying Markov Decision Process (MDP). Second, we compare the performance of the asymptotically optimal static and two-price policies with the approximating DCP solution. We verify that static pricing exhibits a greater (and on a higher order) loss in revenue compared with other dynamic pricing schemes, and further that the two-price scheme has near-optimal performance. Finally, we discuss how the optimal pricing policy from MDP varies as the system scale grows to illustrate the asymptotic difference between dynamic and static policies. To illustrate our numerical observations, we pick one set of parameters, in particular, we fix the cus- tomer valuation distribution as a unit mean exponential, the customer delay sensitivity as h = 1 and the potential demandλ = 2e so that the “load” on the systemλ ¯ F(p ⋆ ) = 2. We would like to point out that we tried various other parameters as well: λ ¯ P(p ⋆ ) = 1.1,2,5, and customer valuation distributions: Weibull distribution with shape parameter being 2 and scale parameter being 1 and Uniform [0,1] distribution; the results presented in the following sections are representative of all our numerical experiments. Accuracy of approximating DCP. Figure 1.1a illustrates the accuracy of the DCP in approximating the actual revenue by comparing the scaled revenue loss of the DCP objective (1.20) with the actual scaled 20 revenue loss ¯ R ⋆ n −Rn(ˆ p ⋆ n ) n 1/3 for the price function ˆ p ⋆ n given in (1.22) that solves the DCP. We see that the objective function of our DCP indeed well approximates the actual scaled revenue loss. Figure 1.1b com- pares the scaled revenue loss obtained from implementing the DCP solution ˆ p ⋆ n with the optimal solution obtain from solving the MDP. From the figure, we observe that indeed the price obtained by solving the DCP has excellent performance relative to the optimal. Note that because the complexity of solving the exact MDP increases very quickly with system size, we are only able to solve it exactly for system sizes up ton = 10 5 . System scale,n Scaled revenue loss, ¯ R ⋆ n −Rn(p) n 1/3 Actual objective DCP objective 10 1 10 3 10 5 10 7 10 9 1 1.5 2 2.5 (a) Accuracy of the DCP objective in approximating the actual objective System scale,n Scaled revenue loss, ¯ R ⋆ n −Rn(p) n 1/3 MDP,p ⋆ n DCP, ˆ p ⋆ n 10 1 10 2 10 3 10 4 10 5 1 1.5 2 2.5 (b) Performance of the DCP solution, ˆ p ⋆ n compared with optimal MDP solution Figure 1.1: Accuracy of drift control problem. Performance of two-price policy. We next compare the performance of the asymptotically optimal two- price policy characterized in Section 1.4.1 with the asymptotically optimal static pricing and the solution to the DCP. Figure 1.2 plots the scaled revenue loss for each of these policies for different system sizes. The figure clearly illustrates theO( √ n) revenue loss of static pricing, which when scaled byn 1/3 grows without bound asn increases. We also observe that the two-price policy performs very well and has a very small gap relative to the solution of the DCP. Structure of the optimal dynamic pricing policy. Finally, we illustrate how the optimal dynamic pricing policy from MDP varies as the system scale grows. Denoting the price refinementθ ⋆ n (q) := p ⋆ n (q)− ¯ p for q ≥ 0, Figure 1.3a describes the structure of θ ⋆ n q n 1/3 . We observe two things: (a) the point at which 21 System scale,n Scaled revenue loss, ¯ R ⋆ n −Rn(p) n 1/3 DCP policy TP policy Static pricing 10 1 10 3 10 5 10 7 10 9 0 10 20 30 Figure 1.2: Performance of two-price (TP) policy relative to static pricing and the DCP-based policy. θ ⋆ n changes its sign from negative to positive isO n 1/3 and given byτn 1/3 +o n 1/3 for some constant τ > 0; and (b)θ ⋆ n converges point-wise to 0 asn grows on its domain except for the origin. We further focus on this convergence in Figure 1.3b where we depictn 1/3 θ ⋆ n q n 1/3 for different values ofn. There, we see the convergence ofn 1/3 θ ⋆ n q n 1/3 forq> 0 as the system scalen grows without bound, which is consistent with our results. q θ ⋆ n ( q n 1/3 ) n = 10 5 n = 10 4 n = 10 3 0 0.4 0.8 1.2 1.6 2 0.1 0 −0.1 −0.2 −0.3 −0.4 −0.5 −0.6 (a) Price refinement,θ ⋆ n , as a function of the queue-length. q n 1/3 θ ⋆ n ( q n 1/3 ) n = 10 5 n = 10 4 n = 10 3 0 0.4 0.8 1.2 1.6 2 1 0 −1 −2 −3 −4 −5 (b) Scaled price refinement, n 1/3 θ ⋆ n , as a function of the queue- length. Figure 1.3: Change in the optimal refinement function as the system scale grows. 22 1.5 Discussion In this section, we provide some additional insight into dynamic pricing. In Section 1.5.1, we first discuss the role of uncertainty in customer valuations in the value of dynamic pricing. In Section 1.5.2, we show how our results extend to social welfare optimization, and in Section 1.5.3, we study settings in which the firm quotes lead-times. 1.5.1 Role of uncertainty in customer valuations We have shown that implementing dynamic pricing leads to an order improvement in the revenue loss due to stochasticity, which is reduced from the typicalO( √ n)-scale toO n 1/3 . In light of the paper Ata and Olsen (2009), this result may seem contradictory because that paper uses dynamic pricing and focuses on optimizing anO( √ n)-scale loss in revenue. The key difference here is that Ata and Olsen (2009) assumes a deterministic valuation for the customers, whereas in our model, customers have stochastic valuations for service. Interestingly, this difference leads to a completely different solution structure as we explain next. Let us consider the case of deterministic valuations in our setting, and denote this value by v. Then, the optimal dynamic price that should be charged to a customer arriving whenq customers are waiting is p v (q) = v−h q n . Notice that this price extracts all the customer surplus, and hence, in this setting social welfare maximization is equivalent to revenue maximization. The fluid optimal price here is ¯ p v = v, and hence the price refinements areθ v n (q) = −hq/n. Notice that relative to the fluid price, the dynamic price quoted to each customer is always smaller, i.e.,θ v n (q)≤ 0. This is in contrast with the stochastic valuation case in which the price refinement price can take both negative and positive values. Recall from Section 1.4.2 that it was this flexibility that allowed us to make the first moment of the price refinement extremely small, and lead to to theO(n 1/3 )-result. For deterministic valuations, because the price refinement is always non- positive, the first moment of the price refinement dominates the second moment, and the system operates at a scale governed by the trade-off between the first moment of the price refinement and the queue-length, which yields theO( √ n) scale. To further understand on how the value of dynamic pricing is tied to the valuation heterogeneity, we next consider the case in which customers differ in their delay sensitivity and their valuations are the same. In this case, we find that analogous to the case in Ata and Olsen (2009), dynamic pricing does not lead to 23 an order improvement in the revenue loss. To formally illustrate, suppose that each customer has a deter- ministic valuationv and a random delay sensitivity which is distributed byG. In this case, the mathematical optimization to find the optimal dynamic pricing policy is given by sup pn∈π R n (p n ) :=E p n (Q n )nλP h≤ v−p n (Q n ) Q n /n , (1.23) where the expectation is with respect to the steady-state queue-length distribution andπ is the set of sta- tionary pricing policies that are non-anticipating as in (1.3). Assumeλ > 1 (this is analogous to Assump- tion 1.3.1 in the heterogeneous valuation setting). Under this assumption, it is straightforward to check that the fluid upper bound for the optimal objective value of (1.23) isnv. This is intuitive because the service rate isn and no customer is willing to pay more thanv. Given this fluid upper bound, we numerically find that both static pricing and two-price policy obtain the revenue loss ofO(logn). In this setting, no order improvement is exhibited by the exact same reasoning from the previous paragraph that discusses Ata and Olsen (2009): price refinement only takes non-positive values. 1.5.2 Social welfare maximization We next discuss how our results extend to social welfare maximization. The social welfare here equals the total customer value minus the waiting costs. Therefore, the total long-run average social welfare under a pricing policyp n (q) is given by limsup T→∞ 1 T Z T 0 Z ∞ pn(Qn(t))+h Qn(t) n nλ v−h Q n (t) n f (v)dvdt, and the corresponding optimization problem is sup pn∈π W n (p n ) :=E " Z ∞ pn(Qn)+h Qn n nλ v−h Q n n f (v)dv # (1.24) 24 where the expectation is with respect to the steady-state queue-length distribution andπ is the set of sta- tionary pricing policies that are non-anticipating as in (1.3). As in Section 1.3, we first consider the fluid approximation of (1.24) by ignoring the variability. This optimization problem is ¯ W ⋆ n :=sup p≥0 Z ∞ p nλvf (v)dv (1.25) s.t.nλ ¯ F (p)≤n. The optimal price that solves this program is ¯ p. We next optimize the loss in social welfare as we did for revenue maximization. For this we consider a pricing policyp n (q) = ¯ p+θ n (q) and compute the welfare loss due to variability as ¯ W ⋆ n −W n (p n ) =nλ ( Z ∞ ¯ p vf (v)dv−E " Z ∞ ¯ p+θn(Qn)+h Qn n v−h Q n n f (v)dv #) . (1.26) Arguing as in Section 1.3.2, we can approximate the welfare loss as ¯ W ⋆ n −W n (p n )≈nλ¯ pf (¯ p)E[θ n (Q n )]+nβE h θ n (Q n ) 2 i +(hλ+γ)E[Q n ] for any p n (q) = ¯ p +θ n (q) with θ n (q) = o(1). Thus, the same arguments that we used for the case of revenue maximization can be used to obtain that dynamic pricing leads toO(n 1/3 ) loss in welfare due to variability. We omit the details, but illustrate the result through numerical experiments in Figure 1.4a and 1.4b. Figure 1.4a plots the welfare loss relative to the fluid optimal scaled byn 1/3 under the optimal policy found by solving the MDP and the optimal static pricing for eachn, respectively. As shown in Figure 1.4a, the scaled welfare loss under the MDP-based policy is on the order ofO n 1/3 while the welfare gap is on the order of magnitude greater thanO n 1/3 under the optimal static. Figure 1.4b depicts the welfare loss relative to the fluid optimal scaled by(nlogn) 1/3 under the optimal two-price policy and the optimal static pricing for eachn, respectively. It clearly shows that the welfare loss under two-price policy is on the order ofO (nlogn) 1/3 as for the revenue maximization in Section 1.4.1. (The same parameter from Section 1.4.5 is used to draw Figure 1.4a and 1.4b.) 25 System scale,n Scaled welfare loss, ¯ W ⋆ n −Wn(p) n 1/3 MDP policy Static pricing 10 1 10 2 10 3 10 4 10 5 0 2 4 6 8 10 (a) Performance of the MDP-based policy relative to static pricing. System scale,n Scaled welfare loss, ¯ W ⋆ n −Wn(p) (nlogn) 1/3 TP policy Static pricing 10 1 10 3 10 5 10 7 10 9 0 5 10 15 20 (b) Performance of the two-price policy relative to static pricing. Figure 1.4: Performance comparison. 1.5.3 Settings with lead-time quotation and order expediting We next discuss an extension to settings in which the queue-length is not observable but rather the firm provides a lead-time quotation. In particular, we will focus on the case in which the firm quotes a static lead-time (irrespective of the congestion) and prices dynamically. Further, the firm always meets the lead- time guarantee, by expediting orders at some cost if needed. This setting is a single-class version of the model used in Celik and Maglaras (2008). (We omit discussing the multi-class framework because the essence of the results is the same for the single-class and the multi-class models.) In this setting, we find that if the expediting costs are high, then dynamic pricing exhibits the same O(n 1/3 )-scale asymptotics as we have already observed. For low expediting costs, we find that static pricing itself leads to O(logn)-scale loss in revenue due to variability, and hence dynamic pricing has limited benefits. The optimization problem. The firm’s objective is to select a dynamic pricing strategy,p n (·), and a static lead-time quote,d n , to maximize the long-run average revenue obtained by processing customers minus the long-run average cost of expediting orders. The optimization problem, which is a modification of (1.3) that reflects the long-run average cost of expediting orders, is as follows: sup pn∈π,dn≥0 R n (p n ,d n ) :=E p n (Q)nλ ¯ F (p n (Q)+hd n ) −e lim T→∞ 1 T E[E n (T)], (1.27) 26 wheree is the cost of expediting an order andE n (T) counts the number of expedited orders by timeT , so that the second term in (1.27) is the long-run average cost of expediting orders. Analogous to Section 1.3, our approach is to first solve the fluid approximation of (1.27) and then refine it with diffusion analysis. The fluid approximation of (1.27) is obtained by ignoring the variability due to queueing and is given by ¯ R ⋆ n := sup (p,d)∈R + ×R + pnλ ¯ F(p+hd)−en λ ¯ F(p+hd)−1 + , (1.28) wherepnλ ¯ F(p+hd) anden λ ¯ F(p+hd)−1 + are the rate at which the revenue and the expediting cost are accrued, respectively, in the fluid model. The following result characterizes the solution of (1.28). It is useful to define ¯ p e as the unique solution ofp−e = ¯ F(p) f(p) (the uniqueness follows because the customer valuation distribution has a non-decreasing hazard rate). Proposition 1.5.1. The fluid optimization problem (1.28) has the unique solution(p ⋆ e ,0), where there exists a threshold ¯ e such thatp ⋆ e = ¯ p e ife< ¯ e, andp ⋆ e = ¯ p otherwise. Proposition 1.5.1 proves that the optimal operating regime depends on the magnitude of the expediting cost. In particular, for large expediting costse ≥ ¯ e, the fluid regime is critically loaded withλ ¯ F(p ⋆ e ) = 1 and zero expediting, and for small expediting costse< ¯ e, the fluid regime is overloaded withλ ¯ F(p ⋆ e )> 1, with the arrival rate in excess of capacity satisfied via expediting. As we discuss next, our prior analysis easily extends to the critically loaded regime (e≥ ¯ e) and yields the same performance improvement results for dynamic pricing. In the overloaded regime (e < ¯ e), static pricing itself leads to O(logn)-loss due to variability, and hence there is limited value of implementing dynamic pricing. Critically loaded regime (e≥ ¯ e) In this setting, calculating the near-optimal dynamic pricing policy is slightly different. We consider the price refinementp n (q) = ¯ p+θ n (q) (recall thatp ⋆ e = ¯ p) and a lead-time quotation ofd n . We focus on small 27 values ofθ n andd n in the sense thatθ n (q) =o(1) forq≥ 0 andd n =o(1). Similar to (1.19), we can apply the Taylor series expansion onR n around(¯ p,0) to achieve ¯ R ⋆ n −R n (p n ,d n )≈nψE[Δ e n (Q n ,d n )]+n φ (λf (¯ p)) 2 E h Δ e n (Q n ,d n ) 2 i +nhd n + lim T→∞ 1 T E[E n (T)], (1.29) where Δ e n (q) =λ ¯ F (¯ p+θ n (q)+hd n )− ¯ F (¯ p) . In (1.29), the first three terms on the right hand side are the counterparts of the three terms in (1.19), respectively, whereas the last term is newly added due to the order expediting. Finding Δ e n andd n that jointly minimize the right hand side of (1.29) is not tractable because the state descriptor involves the residual time before being expedited for each job. So, as in Celik and Maglaras (2008), we approximate the optimization by employing an alternative system to the original one. In the alternative system, each arriving customer is quoted the lead-time ofd n . This quotation is not guaranteed and orders are not expedited, but rather customers are rejected when the quotation cannot be met; the firm incurs a cost ofe for each such rejection. By the snapshot principle (see, for example Reiman 1982), such admission control is asymptotically equivalent to not allowing a customer to enter if nd n customers are waiting for service. The asymptotic equivalence between this alternative system and the original system has been observed in Ata (2006). Let ˜ Q n denote the queue-length process in the alternative system. Then, by straightforward application of the strong approximation of Poisson processes, we obtain ˜ Q n (t) =n Z t 0 Δ e n ˜ Q n (s),d n ds+ √ 2n ˜ X(t)+n ˜ I n (t)− ˜ E n (t)+o √ nt , (1.30) where ˜ I n is the cumulative server idle time process, ˜ E n is the cumulative number of rejected customers (number of arrivals when the queue-length process ˜ Q n is at the boundarynd n ), and ˜ X is a standard Brownian motion. Similar to the manner in whichZ n in (1.18) approximatesQ n in (1.17), we use a diffusion process Z e n to approximate ˜ Q n whereZ e n is the (weak) solution of the following stochastic differential equation: Z e n (t) =n Z t 0 Δ e n (Z e n (s),d n )ds+ √ 2nB e (t)+nL e n (t)−E e n (t), (1.31) 28 whereB e is a standard Brownian motion independent of ˜ X,L e n andE e n are non-decreasing processes that satisfy R t 0 Z e n (s)dL e n (s) = 0 and R t 0 (nd n −Z e n (s))dE e n (s) = 0, respectively. ReplacingQ n byZ e n in the right hand side of (1.29), the DCP that we solve to calculate (Δ e⋆ n ,d e⋆ n ) is given by inf Δ e n ,dn nψE[Δ e n (Z e n ,d n )]+n φ (λf (¯ p)) 2 E h Δ e n (Z e n ,d n ) 2 i +nhd n + lim T→∞ 1 T E[E e n (T)] . (1.32) We can solve (1.32) exactly in the same manner as that in the proof of Proposition 1.4.3; for brevity, we omit these calculations. Optimizing (1.32) under the class of two-price policies results in the objective function value ofO n 1/3 (logn) 2/3 . Finding the optimal two-price to solve (1.32) is straightforward as we can write the objective function explicitly. The diffusion process in (1.31) becomes a piece-wise linear diffusion for which the steady-state distribution is given in Browne and Whitt (1995). Also, using that steady-state distribution and the argument in pages 88-90 of Harrison (1985), we can find the closed form expression for the last term in (1.32). Recall that the objective in (1.32) approximates the revenue loss in the actual system. Then, the per- formance of a two-price policy discussed above suggests that the revenue loss under the optimal two-price policy would be ofO n 1/3 (logn) 2/3 not ofO n 1/3 (logn) 1/3 which we saw in the original model. In this extension of lead-time quotation and order expediting, the optimal two-price cannot achieve the revenue loss ofO n 1/3 (logn) 1/3 because each order expediting costs. Overloaded regime (e< ¯ e) For small expediting costs, on the fluid scale, it is better to set the average arrival rate higher than the capacity by charging a price that is lower than the one that matches the demand and the supply. Having λ ¯ F(p ⋆ e )− 1 > 0 when e < ¯ e impacts the value of implementing dynamic pricing. In fact, in this case, it is straightforward to establish that the static policy of charging the price p ⋆ e and quoting a lead-time d e∗ n = 1 2 + 1 λ ¯ F(p ⋆ e )−1 logn n achievesO(logn)-scale loss in revenue, and thus the value of dynamic pricing is quite limited. 29 1.6 Conclusion In this paper, we study optimal dynamic pricing to maximize revenue in a queueing system when the system congestion is observable by arriving customers. We take an asymptotic approach and find that there is a fundamental benefit to dynamic pricing that results in an order improvement in the revenue loss due to stochasticity relative to static pricing. In particular, under the optimal dynamic pricing scheme, the queue- length is maintained atO(n 1/3 )-scale relative to the traditionalO( √ n)-scale that one expects. We formulate an approximating DCP, solving which yields near-optimal performance. Further, we propose a simple two- price policy that sets a low price when system congestion is low and a high price when the congestion is high. We prove that this policy hasO((logn) 1/3 n 1/3 ) revenue loss, i.e., within a logarithm term of the optimal scale. Our numerical experiments show that this policy performs very close to optimal. We also demonstrate that the order improvement in the revenue loss by implementing dynamic pricing policy can be extended to the other settings of social welfare maximization and lead-time quotations with order expediting. Our work also sheds light on scenarios in which dynamic pricing has added value. In particular, we find that dynamic pricing can mitigate the effect of variability much better compared to static pricing when customers differ in valuations. The homogeneity in customer valuation restricts the role of pricing to that of admission control, that is, pricing is used only to turn-on or turn-off the demand. On the other hand, in settings in which valuations are heterogeneous, dynamic pricing has much more flexibility in regulating customers which leads to an order improvement in mitigating stochasticity. In our analysis, we focus on the capacity-constrained setting which is characterized by the condition λ ¯ F(p ⋆ ) > 1, so that the unconstrained optimal price is not optimal for the fluid optimization problem. If this condition does not hold, the value of dynamic pricing is asymptotically quite limited. In particular, if λ ¯ F(p ⋆ ) = 1, then static pricing by itself generates a revenue loss ofO(n 1/3 ). To see this, notice that we would haveα = 0 in the Taylor series expansion (1.5). Hence, the revenue loss of static pricing would be on the same order as that of the corresponding optimal dynamic pricing policy. Finally, ifλ ¯ F(p ⋆ )< 1, then simply pricing atp ⋆ leads to an under-loaded situation, i.e., we obtain anM/M/1 queueing system with utilizationρ =λ ¯ F(p ⋆ )< 1. Hence, asn grows without bound, the queueing fluctuations will be extremely small, in particular ofO(1)-scale, and will effectively play no role in the optimal solution. 30 Our proposed two-price policy is structurally simple and changes prices much less than the overall optimal dynamic pricing policy. An interesting future direction would be to explicitly consider a cost per each price change as in Netessine (2006). 31 Chapter 2 Dynamic Scheduling of aGI/GI/1+GI Queue with Multiple Customer Classes 2.1 Introduction The question of how to schedule customers for service in a single-server queueing system with more than one customer class is a classical problem in the queueing literature. This is because this scheduling problem arises in a wide variety of applications contexts; in particular, the problem arises whenever heterogeneous customers, that can be grouped into classes, must compete to use the same resource for their processing. The focus is on the following scheduling question: when there are customers from different classes waiting and the server becomes free, which class should the server next serve? One very appealing scheduling rule is the well-knowncµ rule. Thecµ rule is a static priority scheduling rule in which it is assumed that each customer classi has a linear cost of delayc i and an average service time 1/µ i , and classes are prioritized in decreasing order of their indexc i µ i . This implies that the server gives priority to customers having small service requirements and a high cost of delay. The cµ rule for scheduling is appealing because it is simple, robust, and has been shown to be optimal (in either a precise or an asymptotic sense) in many settings (see, for example, Smith (1956) and Cox and Smith (1961) for early work, Mieghem (1995) for the generalization to convex delay costs and a more comprehensive literature review, and Mandelbaum and Stolyar (2004) for more recent work). It is also the case that there is much recent work in the queueing literature on queueing models with customer abandonment. The inclusion of customer abandonment in queueing models is important because in real-life impatient customers that are waiting may abandon the queue before receiving their service. The papers Atar et al. (2010, 2011) propose a static priority scheduling rule for a many-server queueing system with multiple customer classes and abandonment that can be viewed as a modifiedcµ rule. More specifically (and with notation modified to be more consistent with this paper), Atar et al. (2010, 2011) 32 consider a M/M/N +M queue with multiple customer classes in which there are linear holding costs h i , abandonment penaltiesp i , an average abandonment time 1/γ i , and an average service time 1/µ i , and propose to prioritize classes in decreasing order of their indexc i µ i /γ i , forc i =h i +p i γ i . Furthermore, the authors show that their proposed modified static prioritycµ rule asymptotically minimizes average cost over a finite time horizon Atar et al. (2010), and over an infinite time horizon Atar et al. (2011), in the overloaded many-server regime. However, static priority rules do not perform well in general in queueing systems with customer aban- donments. For example, Down et al. (2011) requires that the abandonment rates and costs are aligned in a “fortunate” manner in order to prove that a static priority rule is optimal in a M/M/1 +M queueing system with two customer classes. In other parts of the parameter space, Down et al. (2011) find that the consideration of dynamic priority rules can lead to significant improvements over static priority rules. The results in Down et al. (2011) are exact results, found by studying the associated Markov decision process model. The exact results in Down et al. (2011) are consistent with the asymptotic results in other papers on more general parallel server queueing systems with abandonment. For example, Tezcan and Dai (2008, 2010) prove the asymptotic optimality of a static priority rule in the Halfin-Whitt many-server limit regime in more general parallel server queueing systems with abandonment, but only under an assumed condition on the ordering of the abandonment rates and costs. The paper Harrison and Zeevi (2004) shows that the solution to the approximating Brownian control problem that arises for theM/M/N +M queueing sys- tem with multiple customer classes in the Halfin-Whitt many-server regime gives rise to dynamic priority scheduling rules. The paper Rubino and Ata (2009) shows that the solution to the approximating Brownian control problem for a general parallel server queueing system that arises in conventional heavy traffic also gives rise to dynamic priority scheduling rules. The paper Ata and Tongarlak (2013) (that is concurrent with this work) observes that there is no generalizedcµ rule that will be asymptotically optimal in conventional heavy traffic in a multiclass queueing system with abandonments and general delay costs, and shows that the solution to the approximating Brownian control problem again gives rise to dynamic priority scheduling rules, under various assumptions on the delay costs (linear, convex, and convex-concave). The reason static priority rules do not perform well in general in queueing systems with customer aban- donments is well-explained in Ata and Tongarlak (2013). The issue is that static priority rules are “greedy” in the sense that they give priority to the class with the highest immediate cost rate. However, in queueing 33 systems with customer abandonments it can be beneficial to consider “non-greedy” or “forward-looking” policies that trade-off immediate costs with potential future costs caused by abandonments. All of the papers mentioned in the previous three paragraphs assume exponential abandonment times. It is desirable to remove the assumption of exponential abandonment times. Our objective in this paper is to propose a scheduling control for aGI/GI/1+GI queueing system with N-customer classes; that is, we do not assume exponential abandonment times. The methodology we use to derive our proposed scheduling control is to formulate and solve the approximating Brownian control prob- lem that arises in conventional heavy traffic. There are two potential conventional heavy traffic regimes to consider, that in Ward and Glynn (2003, 2005), and that in Reed and Ward (2008) and Lee and Weerasinghe (2011). In the regimes in Ward and Glynn (2003, 2005), the relevant approximating diffusion has linear drift but, as in Zeltyn and Mandelbaum (2005) and Mandelbaum and Momcilovic (2012) in the many-server limit regime, it is only the value of the abandonment density at zero that influences the approximating Brownian control problem. Then, we expect the solution to the approximating Brownian control problem to be as given in Rubino and Ata (2009). In the regimes in Reed and Ward (2008) and Lee and Weerasinghe (2011), the entire abandonment distribution appears in the approximating Brownian control problem, meaning that the approximating Brownian control problem involves a diffusion with non-linear drift. The solution of this Brownian control problem with non-linear drift is challenging, and is the main technical contribution of this paper. In order to solve the approximating Brownian control problem, we consider its “equivalent workload formulation”. The workload control problem solution is specified in terms of the solution to the associated Bellman equations (Theorem 2.4.2). However, the solution to the Bellman equations is not simple, because the relevant diffusion has non-linear, state-dependent drift. We specify the solution to the Bellman equations in terms of a parameterized ordinary differential equation (Theorem 2.4.3). This approach of considering an entire family of differential equations and deriving properties of the differential equation solutions in that family in order to find the Bellman equation solution combines the solution approaches in Rubino and Ata (2009) and Ghosh and Weerasinghe (2007) (who solve Bellman equations that are relevant for a different queueing model, with service rate control and with abandonments). We also note two specific cases in which the Bellman equation solution is simple, and can be specified exactly. The remainder of this paper is organized as follows. Section 2.2 describes our model, formulates the scheduling control problem, and specifies the heavy traffic limit regime. Section 2.3 (informally) derives the 34 Brownian control problem. We solve the Brownian control problem in Section 2.4. We use the Brownian control problem solution to motivate our proposed scheduling control for theGI/GI/1+GI queue with N-customer classes in Section 2.5, and also provide numeric results that compare the performance of our proposed scheduling control to the static priority scheduling rule. We make concluding remarks in Sec- tion 2.6. Finally, the proofs of all results in this chapter can be found in the Appendix, in the order of their appearance. 2.2 The Model and Problem Formulation All random variables and stochastic processes in our model are assumed to be defined on a complete and filtered probability space (Ω,F,{F t },P), where {F t } is a family of sub-σ-algebras of the σ-algebra F indexed byt≥ 0 satisfyingF s ⊂F t whenever0≤s<t<∞. We consider a GI/GI/1 +GI queue with N-customer classes, as shown in Figure 2.1. The model primitives are 3N independent i.i.d. sequences of nonnegative random variables{u k i ,i≥ 1},{v k i ,i≥ 1}, and {a k i ,i ≥ 1}, for k ∈ {1,2,...,N}. We assume that E u k 1 = E v k 1 = 1 and var(u k 1 ) < ∞, var(v k 1 )<∞, fork∈{1,2,...,N}. For a given total arrival rate to the systemλ, fractionsf k ≥ 0 for for k∈{1,2,...,N} and satisfying P N k=1 f k = 1, and service ratesµ k ≥ 0 fork∈{1,2,...,N}, we assume that theith classk∈{1,2,...,N} arrival joins the system at time t k i := i X j=1 u k j f k λ , and will abandon if he does not enter service withina k i time units. Theith classk∈{1,2,...,N} customer to enter service (which is, in general, not the same as theith classk arrival) has service timev k i /µ k . We let F k be the cumulative distribution function of a k 1 , k ∈ {1,2,...,N}. We assume that F k has support on the positive real line, so that F k (x) := 1− exp − Z x 0 h k (u)du , forx≥ 0, 35 Figure 2.1: TheGI/GI/1 +GI queue withN-customer classes, having classk arrival rateλf k , service rateµ k , abandonment distributionF k with associated hazard rate functionh k , and cost per customer aban- donmentc k . whereh k is a nonnegative and continuous function onR + := [0,∞). Then, h k (x) = d dx F k (x) 1−F k (x) is the hazard rate function associated with the abandonment distributionF k . We assumeh k ,k ∈ {1,2,...,N}, are increasing functions and are not the zero function. Then, it is sensible to assume that within each class customers are served in the order of their arrival. Note that our convention is to write increasing (decreasing) to mean non-strictly increasing (decreasing). For a strictly increasing (decreasing) function, we explicitly write “strictly”. Define the renewal processes fork∈{1,2,...,N} and allt≥ 0 A k (t) := sup n i≥ 0 :t k i ≤t o 36 and S k (t) := sup i≥ 0 : i X j=1 v k j µ k ≤t . Then, A k (t) counts the number of arrivals to classk that have occurred in [0,t] andS k (t) represents the number of classk customers that would complete service if the server worked continuously in[0,t]. We let R k (t) be the cumulative number of classk customers that have reneged in [0,t]. We assume that a job in service does not abandon the system. We do not specify the dynamics ofR k precisely because that requires a more complicated, measure-valued state descriptor, similar to the one given in Kang et al. (2010) for a single class, many server queue with abandonment. Instead, in Section 2.3, we motivate a simple approximation forR k . The scheduling control for the system is represented through theN-dimensional non-preemptive service allocation process T(t) := (T 1 (t),T 2 (t),...,T N (t)),t≥ 0. Fork∈{1,2,...,N},T k (t) is the cumulative amount of service time devoted to serving classk in the time interval[0,t]. Then, I(t) :=t− N X k=1 T k (t),t≥ 0, is the cumulative time the server has been idle in [0,t], andS k (T k (t)),k∈{1,2,...,N}, is the number of classk customers served by the server in[0,t]. The number of classk customers in the system at timet> 0 is Q k (t) :=Q k (0)+A k (t)−S k (T k (t))−R k (t). (2.1) The scheduling controlT must satisfy the following properties. (P1) T k (t) isF t -adapted,k∈{1,2,...,N}. (P2) T k (t),k∈{1,2,...,N}, is continuous and increasing withT k (0) = 0. (P3) I(t) is continuous and increasing withI(0) = 0. (P4) Q k (t)≥ 0,k∈{1,2,...,N}, for allt≥ 0. We say a scheduling controlT is admissible if it satisfies (P1)-(P4). 37 There is a costc k > 0 incurred every time a classk∈{1,2,...,N} customer abandons. Our objective is to choose an admissible scheduling controlT that minimizes expected infinite horizon average cost; that is, our objective is to minimizelimsup t→∞ E 1 t C(T 1 ,T 2 ,...,T N )(t) (2.2) over all admissible scheduling controlsT , where C(T 1 ,T 2 ,...,T N ) := N X k=1 c k R k (t). The problem (2.2) is not solvable via an exact analysis. Therefore, we formulate and solve the Brownian control problem that arises as an approximation to (2.2) in heavy traffic (Sections 2.3 and 2.4). We then use the Brownian control problem solution to motivate our proposed scheduling control, and we evaluate its performance using simulation in Section 2.5. 2.2.1 The Heavy Traffic Limit Regime We consider a family of systems indexed by the total arrival rateλ, and letλ → ∞. Our convention is to superscript any parameter or process associated with the system that has total arrival rateλ, and that may vary withλ, byλ. The costsc k ,k ∈ {1,2,...,N}, are held fixed, but the proportion of traffic from each classf λ k , the service ratesµ λ k λ, and the hazard rate functionsh λ k , depend onλ. In particular, theith class k∈{1,2,...,N} arrival joins the system at timet λ,k i := P i j=1 u k j /(f λ k λ), so that A λ k (t) := sup n i≥ 0 :t λ,k i ≤t o . The service time of theith classk∈{1,2,...,N} customer to enter service isv k i /(µ λ k λ), so that the number of classk customers the server could serve if he worked continuously in[0,t] is S λ k (t) := sup i≥ 0 : i X j=1 v k j µ λ k λ ≤t . The following assumption extends the heavy traffic limit regime in Reed and Ward (2008) developed for the single classGI/GI/1+GI queue to theGI/GI/1+GI queue withN-customer classes. Assumption 2.2.1. There are strictly positive finite constantsf k , andµ k fork∈{1,2,...,N} such that 38 (i) P N k=1 f k µ k = 1; (ii) f λ k →f k andµ λ k →µ k asλ→∞; (iii) √ λµ λ k f λ k µ λ k − f k µ k →θ k ∈R asλ→∞; (iv) h λ k (x) :=h k √ λx for eachx≥ 0. Assumption 2.2.1, parts (i)-(iii), are the usual heavy traffic conditions. Assumption 2.2.1 part (iv) is the hazard rate scaling introduced in Reed and Ward (2008) under which the full abandonment distribution appears in the limiting diffusion. Assumption 2.2.1 part (iv) implies that the abandonment time distribution associated with the system that has total arrival rateλ has the representation F λ k (x) := 1−exp − Z x 0 h k √ λy dy , forx≥ 0,k∈{1,2,...,N}. (2.3) The following is useful for the next Section. Fork∈{1,2,...,N} and allt≥ 0, define ˆ A λ k (t) := √ λ 1 λ A λ k (t)−f λ k t ˆ S λ k (t) := √ λ 1 λ S λ k (t)−µ λ k t , and observe that it follows from the functional central limit theorem (see, for example, Whitt (2002)) and the assumed independence of the sequences{u k i ,i≥ 1},{v k i ,i≥ 1}, fork∈{1,2,...,N}, that ˆ A λ 1 , ˆ A λ 2 ,..., ˆ A λ N , ˆ S λ 1 , ˆ S λ 2 ,..., ˆ S λ N ⇒ (2.4) q f 1 var u 1 1 ˆ B A,1 ,..., q f N var u N 1 ˆ B A,N , q µ 1 var v 1 1 ˆ B S,1 ,..., q µ N var v N 1 ˆ B S,N inD, asλ→∞, where ˆ B A,k and ˆ B S,k fork ∈{1,2,...,N} are independent, standard Brownian motions. Next, it follows from (2.1) that for ˆ Q λ k := Q λ k √ λ , ˆ Q λ k (0) := Q λ k (0) √ λ and ˆ R λ k := R λ k √ λ ,k∈{1,2,...,N}, then ˆ Q λ k (t) = ˆ Q λ k (0)+ ˆ A λ k (t)− ˆ S λ k T λ k (t) − ˆ R λ k (t)+ √ λf λ k t− √ λµ λ k T λ k (t),k∈{1,2,...,N}. (2.5) 39 Finally, let ˆ I λ := √ λI λ and ˆ C λ T λ 1 ,T λ 2 := C λ T λ 1 ,T λ 2 √ λ . As is common, the notationD that appears in (2.4) is the set of all functionsω : R + → R m for some appropriate integerm that are right continuous and have left limits, and is endowed with the Skorokhod-J 1 topology. Also, for two stochastic processesX ∈D andX ′ ∈D, we writeX D ≈X ′ to mean thatX andX ′ have approximately the same distribution. 2.3 The Brownian Control Problem Definem : (R + ) N → (R + ) N as m(q) = (m 1 (q 1 ),m 2 (q 2 ),...,m N (q N )) := f 1 Z q 1 /f 1 0 h 1 (u)du,f 2 Z q 2 /f 2 0 h 2 (u)du,...,f N Z q N /f N 0 h N (u)du ! . The Brownian control problem that is relevant for the control problem (2.2) whenλ is large is to minimizelimsup t→∞ E " 1 t Z t 0 N X k=1 c k m k ˆ Q k (s) ds # (2.6) using a control process ˆ Y = ˆ Y 1 , ˆ Y 2 ,..., ˆ Y N such that ˆ Q k (t) := ˆ Q k (0)+ ˆ X k (t)− Z t 0 m k ˆ Q k (s) ds+µ k ˆ Y k (t)≥ 0,t≥ 0,k∈{1,2,...,N} (2.7) ˆ I := N X k=1 ˆ Y k is increasing and has ˆ I(0) = 0, (2.8) where ˆ X = ˆ X 1 , ˆ X 2 ,..., ˆ X N is aN-dimensional Brownian motion with respect to{F t } that starts from the origin, has drift(θ 1 ,θ 2 ,...,θ N ), and has diagonal covariance matrix diag f 1 var u 1 1 + var v 1 1 ,f 2 var u 2 1 + var v 2 1 ,...,f N var u N 1 + var v N 1 . Also, we assume ˆ Q k (0) = ˆ q k ,k∈{1,2,...,N}, for ˆ q k ≥ 0 a deterministic, finite constant. 40 We say the control ˆ Y is admissible if it is F t -adapted, satisfies (2.7)-(2.8), and ˆ Q k , ˆ Y k , ˆ X k ,(Ω,F,{F t },P) is a weak solution (see, for example, Definition 5.3.1 in Karatzas and Shreve (1988)) of (2.7) for k ∈ {1,2,...,N} under ˆ Y . We say the control ˆ Y ⋆ is optimal if it is admissible and for any other admissible ˆ Y ˆ C ˆ Y ⋆ ≤ ˆ C ˆ Y , where ˆ C ˆ Y := limsup t→∞ E " 1 t Z t 0 N X k=1 c k m k ˆ Q k (s) ds # . Our purpose in this Section is to show how the Brownian control problem in (2.6)-(2.8) arises from the control problem (2.2) asλ becomes large. This requires the assumption that the scheduling control achieves the long run rates required for a balanced system in the heavy traffic limit; i.e., that T λ = T λ 1 ,T λ 2 ,...,T λ N ⇒T ⋆ inD, asλ→∞, (2.9) where T ⋆ (t) = (T ⋆ 1 (t),T ⋆ 2 (t),...,T ⋆ N (t)) := f 1 µ 1 t, f 2 µ 2 t,..., f N µ N t for allt≥ 0. Suppose we can show that for largeλ ˆ R λ k (·) D ≈f λ k Z · 0 Z ˆ Q λ k (s)/f λ k 0 h k (y)dy ! ds, fork∈{1,2,...,N}. (2.10) Then, since ˆ C λ T λ 1 ,T λ 2 = P N k=1 c k ˆ R λ k from its definition, the objective function (2.6) for the Brownian control problem follows directly. To motivate the constraints, we first write the evolution equations for ˆ Q k ,k∈{1,2,...,N}, in a more convenient form. For this, from (2.5), ˆ Q λ k (t) = ˆ Q λ k (0)+ ˆ X λ k (t)− ˆ R λ k (t)+µ λ k ˆ Y λ k (t), k∈{1,2,...,N} and for allt≥ 0, (2.11) 41 where ˆ X λ k (t) := ˆ A λ k (t)− ˆ S λ k T λ k (t) + √ λµ λ k f λ k µ λ k − f k µ k t ˆ Y λ k (t) := √ λ T ⋆ k (t)−T λ k (t) . Next, note that ˆ X λ 1 , ˆ X λ 2 ,..., ˆ X λ N ⇒ ˆ X 1 , ˆ X 2 ,..., ˆ X N , asλ→∞, by the observed weak convergence in (2.4), Assumption 2.2.1, the assumed weak limit in (2.9), the con- tinuous mapping theorem, and the random time change theorem. Then, replacing ˆ X λ k ,k ∈ {1,2,...,N}, in (2.11) with its weak limit in the above display, and ˆ R λ k ,k ∈ {1,2,...,N}, with its approximation in (2.10), yields an evolution equation for( ˆ Q λ k , ˆ Y λ k ) that is identical to the constraint (2.7). The constraint (2.8) follows because N X k=1 ˆ Y λ k (t) = √ λ t− N X k=1 T λ k (t) ! = ˆ I λ (t) , for allt≥ 0. It remains to motivate the approximation (2.10). The snapshot principle (a Little’s law that holds for all t ≥ 0 on diffusion scale; see, for example Reiman (1982)) suggests that when λ is large Q λ k (t)−1 + /(f λ k λ) = ˆ Q λ k (t)−1/ √ λ + /(f λ k √ λ) is very close to the virtual waiting time at time t> 0 (that is, the amount of time an infinitely patient classk∈{1,2,...,N} customer that arrives at time t > 0 must wait for service), also noting that in our heavy traffic limit regime the probability a customer abandons becomes small. SinceF λ k (w) is the probability that a classk customer that must waitw> 0 time units for service abandons, it follows that R λ k (·) A λ k (t) D ≈F λ k ˆ Q λ k (·)−1/ √ λ + f λ k √ λ D ≈F λ k ˆ Q λ k (·) f λ k √ λ ! , for sufficiently largeλ. Then, ˆ R λ k (·) D ≈f λ k Z · 0 √ λF λ k ˆ Q λ k (s) f λ k √ λ ! d A λ k (s) f λ k λ . (2.12) 42 The representation forF λ k in (2.3) and a Taylor series expansion suggest that √ λF λ k ˆ Q λ k (s) f λ k √ λ ! = √ λ 1−exp − Z ˆ Q λ k (s)/(f k √ λ) 0 h k ( √ λy)dy !! (2.13) = √ λ 1−exp − 1 √ λ Z ˆ Q λ k (s)/f k 0 h k (y)dy !! D ≈ Z ˆ Q λ k (s)/f λ k 0 h k (y)dy. The approximation (2.10) now follows from (2.12), (2.13), and the fact thatA λ k (t)/(f λ k λ)−t⇒ 0 inD as λ→∞ by the functional strong law of large numbers. 2.4 The Brownian Control Problem Solution We reduce the Brownian control problem to a one-dimensional “equivalent workload formulation” (follow- ing the terminology in Harrison and Van Mieghem (1997)) in Section 2.4.1. We solve the workload control problem in Section 2.4.2. 2.4.1 The Workload Control Problem Motivated by Harrison (2001), we can simplify the Brownian control problem with N-dimensional state descriptor by defining the one-dimensional workload process ˆ W as ˆ W := N X k=1 1 µ k ˆ Q k . (2.14) The term “workload” refers to the fact that ˆ W(t), when multiplied by √ λ, provides an approximation to the expected time it would take to serve all jobs in the system at timet ≥ 0 in theGI/GI/1+GI queueing model withN-customer classes given in Section 2.2. Note that for ˆ ξ := N X k=1 1 µ k ˆ X k 43 a one-dimensional Brownian motion with respect to{F t } that starts from the origin, has drift θ := N X k=1 θ k µ k , and variance σ 2 := N X k=1 1 µ k 2 f k var u k 1 + var v k 1 , it follows that, for ˆ w := P N k=1 ˆ q k /µ k , ˆ W (t) = ˆ w+ ˆ ξ(t)− Z t 0 N X k=1 1 µ k m k ˆ Q k (s) ds+ ˆ I(t), for allt≥ 0. (2.15) The workload control problem is to minimize limsup t→∞ E " 1 t Z t 0 N X k=1 c k m k q k s, ˆ W (s) ds # (2.16) using a control process(q, ˆ I) such that q t, ˆ W(t) = q 1 t, ˆ W(t) ,q 2 t, ˆ W(t) ,...,q N t, ˆ W(t) ∈A ˆ W(t) , for allt≥ 0 (2.17) andA(x) := ( (q 1 ,q 2 ,...,q N ) : N X k=1 q k µ k =x,q k ≥ 0, fork∈{1,2,...,N} ) , forx≥ 0 ˆ W (t) = ˆ w+ ˆ ξ(t)− Z t 0 N X k=1 1 µ k m k q k t, ˆ W (t) ds+ ˆ I(t)≥ 0,t≥ 0. (2.18) ˆ I is increasing and has ˆ I(0) = 0. (2.19) The controlq can be interpreted as a workload configuration that determines how the total workload should be distributed to each customer class. We say that the control (q, ˆ I) is admissible if it isF t -adapted, right- continuous with left limits (RCLL), satisfies (2.17)-(2.19), and under ˆ I, ˆ W, ˆ I, ˆ ξ ,(Ω,F,{F t },P) is a weak solution of (2.18). We say the control (q ⋆ , ˆ I ⋆ ) is optimal if it is admissible and for any admissible (q, ˆ I) ˆ C w q ⋆ , ˆ I ⋆ ≤ ˆ C w (q, ˆ I), 44 where ˆ C w q, ˆ I := limsup t→∞ E " 1 t Z t 0 N X k=1 c k m k q k s, ˆ W (s) ds # . The following Proposition, which is a straightforward adaptation of Propositions 2 and 3 in Rubino and Ata (2009), shows that to solve the Brownian control problem in (2.6)-(2.8), it is sufficient to solve the workload control problem in (2.16)-(2.19). Proposition 2.4.1. The Brownian control problem (2.6)-(2.8) is equivalent to the workload control problem (2.16)-(2.19) in the following sense: • Every admissible control (q, ˆ I) for the workload control problem yields an admissible control ˆ Y for the Brownian control problem, having ˆ C( ˆ Y) = ˆ C w (q, ˆ I). • Similarly, for any admissible control ˆ Y for the Brownian control problem, there exists an admissible control(q, ˆ I) for the workload control problem having ˆ C w (q, ˆ I)≤ ˆ C( ˆ Y). Corollary 2.4.1. It follows from Proposition 2.4.1 that ˆ C( ˆ Y ⋆ ) = ˆ C w (q ⋆ , ˆ I ⋆ ). 2.4.2 The Workload Control Problem Solution The first step in solving the workload control problem is to find a function v : R + → R that is twice- continuously differentiable and a constantκ∈R that solve what is known as the Bellman equation: σ 2 2 v ′′ (x)+φ(x,v ′ (x)) =κ for allx≥ 0 (2.20) and has v ′ (0) = 0,v ′ bounded, and non-negative, (2.21) whereφ :R + ×R + →R is defined as φ(x,v) :=θv + inf q∈A(x) ( N X k=1 m k (q k ) c k − v µ k ) . Remark 2.4.1. Note that conditions in (2.21) are minimum requirements that v function should satisfy. These are different from what we prove later:v ′ (0) = 0,v ′ is monotone increasing, andlim x→∞ v ′ (x) =C for a constantC ∈ (0,∞), which imply (2.21). 45 Next, we show how to construct an optimal control for the workload control problem (2.16)-(2.19) in terms of a solution to the Bellman equations (2.20)-(2.21). Suppose (v,κ ⋆ ) solves (2.20)-(2.21). Define q ⋆ :R + → (R + ) N as q ⋆ (x) := argmin q∈A(x) ( N X k=1 m k (q k ) c k − 1 µ k v ′ (x) ) , (2.22) and let ˆ W ⋆ , ˆ I ⋆ solve ˆ W ⋆ (t) = ˆ w+ ˆ ξ(t)− Z t 0 N X k=1 1 µ k m k q ⋆ k ˆ W ⋆ (s) ds+ ˆ I ⋆ ≥ 0,t≥ 0 (2.23) ˆ I ⋆ is increasing, has ˆ I ⋆ (0) = 0, and Z ∞ 0 ˆ W ⋆ (t)d ˆ I ⋆ (t) = 0. Proposition 2.4.2. The process ˆ W ⋆ in (2.23) is a positive recurrent regenerative process. Furthermore, E h ˆ W ⋆ (t) i /t→ 0 ast→∞. Remark 2.4.2. Intuitively,E h ˆ W ⋆ (t) i /t → 0 ast → ∞ of Proposition 2.4.2 holds because ˆ W ⋆ has the stationary distribution due to the non-linear state-dependent drift term − Z t 0 N X k=1 1 µ k m k q ⋆ k ˆ W ⋆ (s) ds that arises from customer abandonments. To see this, we refer the readers to Proposition 6.1 of Reed and Ward (2008) where the single customer class counterpart of ˆ W ⋆ is shown to have the stationary distribution under some conditions. Those conditions are satisfied, no matter how large θ is, by our assumption of the increasing hazard rate functions of customer abandonments. If one can believe that Proposition 6.1 of Reed and Ward (2008) is generalized to our many customer-class model, so that ˆ W ⋆ has the stationary distribution, then it is easy to accept the last statement of Proposition 4.2 without actually reading its proof. The following Theorem (which uses Proposition 2.4.2 in its proof) shows that(q ⋆ , ˆ I ⋆ ) is an optimal control for the workload control problem and thatκ ⋆ is the minimum cost. Theorem 2.4.2. Assume (v,κ ⋆ ) solves the Bellman equations (2.20)-(2.21),q ⋆ is as defined in (2.22), and ( ˆ W ⋆ , ˆ I ⋆ ) solves (2.23). Then, (i) ˆ C w (q ⋆ , ˆ I ⋆ ) =κ ⋆ ; (ii) q ⋆ , ˆ I ⋆ is optimal for the workload control problem. 46 Remark 2.4.3. The argmin in the definition ofq ⋆ exists and is finite because for any givenx≥ 0 N X k=1 m k (q k ) c k − 1 µ k v ′ (x) is jointly continuous in (q 1 ,q 2 ,...,q N ) and has a domainA(x) which is compact in (R + ) N . In the case that the argmin in the definition ofq ⋆ is not unique,q ⋆ can be defined as anyq ∈ A(x) that achieves the infimum in the definition ofφ. It follows from Theorem 2.4.2 that to solve the workload control problem, it is enough to construct a solution to the Bellman equations (2.20)-(2.21). When either the abandonment distributions are Weibull and c 1 µ 1 =c 2 µ 2 =··· =c N µ N or the abandonment distributions are exponential, the Bellman equations can be solved explicitly for(v,κ ⋆ ) and there are simple, closed-form expressions available for an optimal control q ⋆ and the minimum cost κ ⋆ ; see Sections 2.4.2 and 2.4.2. However, in general, the Bellman equations cannot be solved explicitly. Then, under the assumption thatθ≤ 0, we construct a solution to the Bellman equation by considering, forκ∈R, the initial value problem IVP(κ) κ = 1 2 σ 2 w ′ (x)+φ(x,w(x)) for allx≥ 0 subject to:w(0) = 0. Suppose we can find κ ⋆ such that the function w κ ⋆ : R + → R that solves IVP(κ ⋆ ) is bounded and non-negative. Then, if we set v(x) := Z x 0 w κ ⋆(y)dy for allx≥ 0, (2.24) (v,κ ⋆ ) solves the Bellman equations (2.20)-(2.21). The first step in findingκ ⋆ such that (v,κ ⋆ ) solves the Bellman equations is to define the set U := {κ> 0 :w κ is increasing}. Note that for anyκ∈U,w κ is non-negative. Hence to solve the Bellman equations, it would be sufficient to findκ∈U such thatw κ is also bounded. The following Theorem shows that there exists aκ ⋆ for which w κ ⋆ bounded. 47 Figure 2.2: (x,w κ (x)) forx≥ 0 andκ∈{κ 1 ,κ 2 ,κ ⋆ ,κ 4 ,κ 5 } whereκ 1 >κ 2 >κ ⋆ >κ 4 >κ 5 . Theorem 2.4.3. Assumeθ≤ 0. Letκ ⋆ := infU and definev as in (2.24). Then,κ ⋆ > 0, and(v,κ ⋆ ) solves the Bellman equations (2.20)-(2.21). A Sketch of the proof of Theorem 2.4.3 The purpose of this sketch is to briefly explain the readers how Theorem 2.4.3 is proved before its actual proof found in the appendix. For allx ≥ 0,w κ (x) is continuous inκ and increasing inκ that isw κ (x) ≥ w κ ′(x) for allx≥ 0 ifκ>κ ′ . Furthermore, in can be shown that, forκ / ∈U,w κ (x) first increases and then decreases to−∞ inx. Finally, becausew κ (x) is continuous inκ, we can prove thatκ ⋆ := infU must satisfy w κ ⋆(x) is increasing inx andlim x→∞ w κ ⋆(x)<∞, implying thatw κ ⋆ is bounded. Figure 2.2 summarizes this sketch. Remark 2.4.4. We conjecture that there exists a solution to the Bellman equations in the case thatθ > 0. Furthermore, in in Sections 2.4.2 and 2.4.2, we explicitly construct a solution to the Bellman equations for anyθ ∈ R in the case thatN = 2, and the abandonment distributions are Weibull andc 1 µ 1 = c 2 µ 2 and also in the case thatN = 2, and the abandonment distributions are exponential. 48 The case thatN = 2, the abandonment distributions are Weibull, andc 1 µ 1 =c 2 µ 2 . There is a simple, continuous optimal control that can be written explicitly in the case that the abandonment distributions are Weibull having hazard rate functions h 1 (x) = 2x for allx≥ 0 andh 2 (x) = 3x 2 for allx≥ 0, and the parameters satisfyc 1 µ 1 =c 2 µ 2 . In particular, define q ⋆ (x) := µ 2 1 f 2 2 +3µ 1 µ 2 2 f 1 x− p µ 4 1 f 4 2 +6µ 3 1 µ 2 2 f 1 f 2 2 x 3µ 2 2 f 1 , p µ 2 1 µ 2 2 f 4 2 +6µ 1 µ 4 2 f 1 f 2 2 x−µ 1 µ 2 f 2 2 3µ 2 2 f 1 ! for allx≥ 0, and note thatq ⋆ 1 (x)≥ 0,q ⋆ 2 (x)≥ 0 for allx≥ 0. Then, let ˆ W ⋆ , ˆ I ⋆ solve (2.23), for which it is helpful to note that m(q) = q 2 1 f 1 , q 3 2 f 2 2 for allq 1 ≥ 0 andq 2 ≥ 0. Finally, define κ ⋆ :=c 1 µ 1 lim x→∞ R x 0 g(y)exp 2 σ 2 θy− R y 0 g(z)dz dy lim x→∞ R x 0 exp 2 σ 2 θy− R y 0 g(z)dz dy for g(x) := q ⋆ 1 (x) 2 f 1 µ 1 + q ⋆ 2 (x) 3 f 2 2 µ 2 . Note that the limits in the numerator and denominator in the definition ofκ ⋆ are finite. Proposition 2.4.3. The control(q ⋆ , ˆ I ⋆ ) is optimal for the workload control problem and ˆ C w (q ⋆ , ˆ I ⋆ ) =κ ⋆ > 0. The case thatN = 2 and the abandonment distributions are exponential. When h 1 (x) =γ 1 for allx≥ 0 andh 2 (x) =γ 2 for allx≥ 0, an optimal control either gives static priority to one class or uses a threshold level to decide class priorities dynamically. Without loss of generality, assume that the classes are ordered so that c 1 µ 1 γ 1 > c 2 µ 2 γ 2 . Whether a static priority control or a dynamic priority control is optimal depends on whether the costs, 49 service rates, and drifts have a fortunate alignment. In the following, we specify an optimal controlq ⋆ , and then provide an optimality result. The first step in specifying an optimal control is the following Lemma, which will be used to specify the threshold level whenq ⋆ is a dynamic priority control. Letφ and Φ be the pdf and cdf associated with a standard normal random variable, and, for anyx∈R, letΨ(−x) := Φ(x)/φ(x). SinceΦ(x) = 1−Φ(−x) andφ(x) =φ(−x), we haveΨ(−x) = (1−Φ(−x))/φ(−x). ThenΨ(x) = (1−Φ(x))/φ(x) so that1/Ψ is the hazard rate function associated with a standard normal random variable. Lemma 2.4.1. Assumec 1 µ 1 <c 2 µ 2 . Then, there existsL> 0 such that κ 1 (L) =κ 2 (L)> 0 (2.25) for κ 1 (L) := c 2 µ 2 θ+ 1 q 2 σ 2 γ 2 φ L− θ γ 2 r σ 2 2γ 2 γ 1 (c 1 µ 1 −c 2 µ 2 ) γ 1 −γ 2 +φ − q 2 σ 2 γ 2 θ c 2 µ 2 Φ L− θ γ 2 r σ 2 2γ 2 −Φ − q 2 σ 2 γ 2 θ κ 2 (L) := c 1 µ 1 θ+ 1 q 2 σ 2 γ 1 γ 2 (c 2 µ 2 −c 1 µ 1 ) γ 1 −γ 2 Ψ L− θ γ 1 r σ 2 2γ 1 . We setL ⋆ := inf{L> 0 :κ 1 (L) =κ 2 (L)}, which is non-empty by Lemma 2.4.1. Define, for anyx≥ 0, q ⋆ (x) := (0,xµ 2 ), 0≤x<L ⋆ (xµ 1 ,0), L ⋆ ≤x andκ ⋆ :=κ(L ⋆ )> 0 ifc 2 µ 2 >c 1 µ 1 ; (2.26) q ⋆ (x) := (0,xµ 2 ) for allx≥ 0 andκ ⋆ :=c 2 µ 2 θ+ 1 q 2 σ 2 γ 2 1 Ψ − q 2 σ 2 γ 2 θ otherwise.(2.27) Let ˆ W ⋆ , ˆ I ⋆ solve (2.23), for which it is helpful to note that m(q) = (γ 1 q 1 ,γ 2 q 2 ) for allq 1 ≥ 0 andq 2 ≥ 0. 50 Proposition 2.4.4. The control q ⋆ , ˆ I ⋆ is optimal for the workload control problem and ˆ C w q ⋆ , ˆ I ⋆ = κ ⋆ > 0. 2.5 The Proposed Policy There is a natural translation of the optimal control (q ⋆ , ˆ I ⋆ ) for the workload control problem to a non-preemptive scheduling control for the original system. Specifically, consider the queueing model described in Section 2.2 having arrival rate λ, fraction of class k ∈ {1,2,...,N} customers f λ k , class k ∈ {1,2,...,N} service ratesµ λ k , and hazard rate functions associated with the classk ∈ {1,2,...,N} abandonment distributionh λ k . Then, defineq ⋆ in (2.22) by choosingf k ,µ k , andθ k ,k∈{1,2,...,N}, that is consistent with Assumption 2.2.1 (i)-(iii), and letting h k (x) :=h λ k x √ λ . Also letW be the discrete-time analog of ˆ W defined in (2.14), so that W := N X k=1 Q k µ k . Then, our proposed scheduling control is as follows. Every time the server becomes free at timet≥ 0, and customers at least from two classes are waiting, the server next serves classi if i = min argmax k∈{1,2,...,N} Q k (t) √ λ −q ⋆ k W (t) √ λ ! . If there are only customers from one class waiting, the server next serves a customer from that class. Recall that customers are served FIFO within a class. Finally, when a customer of either class arrives to find the server idle, that customer immediately begins service. We next test the performance of our proposed scheduling control numerically, using simulation, by comparing its simulated infinite horizon average cost to that of the possible static priority policies for this system. We do this under the assumption that the inter-arrival time distribution is exponential, the service time distribution is deterministic,N = 2, and the abandonment distributions are i) Weibull (together with 51 (a) Exponential abandonments (b) Weibull abandonments Figure 2.3: The simulated time average abandonment cost when the system operates under our proposed policy, as compared to the two possible static priority policies, for the system parameters given in the last two paragraphs of Section 2.5. an additional assumption ofc 1 µ 1 = c 2 µ 2 as in Section 2.4.2) and ii) exponential, so thatq ⋆ is as given in Section 2.4.2 and Section 2.4.2 respectively. Figure 2.3 shows the simulated average cost for the queueing model in Section 2.2 is always lower, and sometimes much lower, under our proposed dynamic priority policy than under either of the two possible static priority policies, under the following parameter assumptions. For Weibull abandonments case, we let λ = 100,f λ 1 = f λ 2 = 0.5,h λ 1 (x) = 20x forx ≥ 0 andh λ 2 (x) = 300x 2 forx ≥ 0 so thath λ 1 andh λ 2 are Weibull,c 2 = 400, and varyc 1 ,µ λ 1 , andµ λ 2 . Specifically, we let c 1 = 55+10k fork ={1,2,...,11} and then let µ λ 1 =f λ 1 + c 2 f λ 2 c 1 = 1 2 + 200 55+10k andµ λ 2 = c 1 µ λ 1 c 2 = 1 2 + 55+10k 800 , fork ={1,2,...,11}, so that f λ 1 µ λ 1 + f λ 2 µ λ 2 = 1 andc 1 µ λ 1 =c 2 µ λ 2 . 52 Then,f k = f λ k ,µ k = µ λ k ,c k fork ∈ {1,2,...,N},θ = 0, andh 1 (x) = 2x forx ≥ 0 andh 2 (x) = 3x 2 forx≥ 0 are parameters for the workload control problem that are consistent with Assumption 2.2.1. From Section 2.4.2, q ⋆ 1 (x) = µ 2 1 f 2 2 +3µ 1 µ 2 2 f 1 x− p µ 4 1 f 4 2 +6µ 3 1 µ 2 2 f 1 f 2 2 x 3µ 2 2 f 1 . When abandonments are exponential, we setλ = 1000,f λ 1 =f λ 2 = 0.5 andµ λ 1 =µ λ 2 = 1 so that f λ 1 µ λ 1 + f λ 2 µ λ 2 = 1. Then, we leth λ 1 (x) = 20 forx≥ 0 andh λ 2 (x) = 1 forx≥ 0,c 2 = 400, and varyc 1 by c 1 = 55+10k fork ={1,2,...,11}. Here, we have parametersf k =f λ k ,µ k =µ λ k ,c k fork∈{1,2,...,N},θ = 0, andh 1 (x) = 20(=γ 1 ) for x≥ 0 andh 2 (x) = 1(=γ 2 ) forx≥ 0 for the workload control problem that satisfy Assumption 2.2.1. q ⋆ is determined by (2.26) becausec 2 µ 2 > c 1 µ 1 andc 1 µ 1 γ 1 > c 2 µ 2 γ 2 in the parameter space. Note that for each calculation displayed in Figure 2.3, 20 sample paths are simulated and each simulation is run until the total number of customers to arrive is 1 million. 2.6 Conclusions A classic control question for single server queueing systems with multiple customer classes concerns how to schedule customers for service. We study aGI/GI/1 +GI queue withN-customer classes, in which classes are distinguished by their inter-arrival, service, and abandonment time distributions, and in which the system incurs a class-dependent cost every time a customer abandons. We propose a non-idling, non- preemptive, dynamic scheduling policy that selects which customer class should next be served each time the server becomes free. To do this, we formulate and solve the approximating Brownian control problem for the system that arises in the heavy traffic limit regime in Reed and Ward (2008). Then, our proposed scheduling policy is specified in terms of the Brownian control problem solution. The approximating Brownian control problem is technically challenging because it has non-linear drift which is state-dependent. 53 We evaluate our proposed policy via simulation, by comparing the average simulated cost achieved under our proposed policy to that achieved under the static priority rule that always has the server give priority to a specified class. Our simulation study confirms the intuition we gain from solving the Brownian control problem: in queueing systems with customer abandonments, static priority scheduling policies do not perform well in general, and it is necessary to consider dynamic scheduling policies in order to obtain near minimum cost. There are three immediate interesting directions for future research. The first is to establish that our proposed policy is asymptotically optimal in the heavy traffic limit regime. This problem is technically chal- lenging because we expect that such a proof requires more complicated, measure-valued state-descriptors. The second is to consider delay cost functions that are convex, or ones that are convex-concave, as in Akan et al. (2012) Ata and Olsen (2009, 2013), and Ata and Tongarlak (2013). The third is to consider other, less complicated dynamic priority scheduling policies. While working on our simulation studies, we observed that a simple fixed ratio dynamic priority scheduling control policy sometimes performs almost as well as our proposed policy. It would be interesting to be able to quantify that statement more precisely. 54 Chapter 3 Dynamic Scheduling in a Many-Server Multi-Class System: the Role of Customer Impatience in Large Systems 3.1 Introduction The study of scheduling problems has a long history in the academic literature (Pinedo 2012). One cele- brated result within the context of stochastic scheduling problems is thecµ rule, which is provably optimal (in an exact or asymptotic sense) under a wide variety of assumptions (Smith 1956, Mieghem 1995 and Mandelbaum and Stolyar 2004). However, that rule often fails in the presence of time-sensitive customers (Rubino and Ata 2009, Down et al. 2011), which are ubiquitous in service systems. Our objective in this paper is to identify simple and effective rules to replacecµ in the aforementioned context. Statistically, an important observation about the willingness of a time-sensitive customer to wait for service, termed his patience time, is that it changes over time (Brown et al. 2005, and Mandelbaum and Zeltyn 2013). In particular, as customers wait for their service to begin, some become more patient (for example, due to having already paid a waiting cost) and some become less patient (for example, due to being frustrated from waiting). The novelty in this work is to devise scheduling rules that account for this temporal change in customer patience. Technically, this means not assuming patience times follow an exponential distribution, which is a very common assumption in the literature. We modify a well-studied model for service systems with homogeneous customers, theM/M/N +GI queue (Zeltyn and Mandelbaum 2005), to account for heterogeneous customers. Our multiclassM/M/N+ GI queue distinguishes different customer classes through their patience distribution, their arrival rate, and the cost if a customer from that class’s patience runs out before his service begins. The scheduling decision is how to pair a newly available server with a customer when there are customers from more than one class 55 waiting, in order to minimize long-run average cost. Note that when the costs associated with each customer class is the same, the objective of minimizing long-run average cost exactly coincides with maximizing the throughput rate. The scheduling decision is in general dynamic because it can depend on the relative number of customers from the different classes. The optimization problem to determine optimal scheduling is very complicated and not amenable to exact analysis. This leads us to apply fairly standard approximation techniques used in the extant literature. In particular, we assume that the service system has high customer demand, a large number of servers, and is operating critically loaded. (Formally, we consider the Halfin-Whitt many-server scaling in the quality and efficiency driven regime.) In this asymptotic regime, there arise two potential approximating diffusion control problems (DCPs): one in which the underlying diffusion has linear drift and is based on the value of the patience time density at zero, and one in which the underlying diffusion has nonlinear drift and incorporates the entire patience time density function. It is sufficient to study the solutions to the DCPs because those solutions readily translate to scheduling policies for aM/M/N +GI queue. We solve the DCPs by identifying the Hamilton-Jacobi-Bellman (HJB) equations that characterize an optimal scheduling policy. We show that there exists a unique solution to the HJB equations, implying that an optimal scheduling policy can be computed numerically. We view the HJB equation as a map that connects the patience distribution and the optimal scheduling policy, and we use this map to derive results on the form of an optimal policy. This leads to the following main contributions of this paper. 1. Linear Drift DCP . We show that the DCP solution either leads to a static class ranking or is of threshold form. The key to identifying the optimum thresholds is to first understand which classes will have priority that change in accordance with the system state, and which classes will have a higher priority always. We provide a simple algorithm to do this. 2. General DCP . We show that in a non-trivial portion of the parameter space, the DCP solution does not either lead to a static class ranking or have a threshold form. For a two-class model, we provide conditions on the patience distributions under which (i) the aforementioned forms emerge, or (ii) the optimum policy has a different structure. One novel structure we see emerge is a U-shape, as shown in Figure 3.1, which has rich intuition. Consider a two-class model in which class 1 has a patience time distribution with increasing hazard rate, 56 Number of waiting customers Ideal fraction of class 1 among waiting customers Threshold policy Proposed policy Partial prioritization domain 50 100 150 0% 20% 40% 60% 80% 100% Figure 3.1: Description of our proposed policy and threshold policy class 2 has an exponential patience time distribution, and class 1 has a lower cost if a customers patience runs out before his service begins. When there are few customers in the system, allowing the cheaper class 1 to leave without receiving service is beneficial. As the system becomes more congested, the increasing class 1 hazard rate is evidenced through more and more class 1 customers leaving, resulting in increased cost. That effect is counteracted by giving some priority to the class 1 customers. Eventually, when the system is very congested, the forward-looking system manager returns to fully prioritizing class 2 customers, using the very high abandonment rate of the class 1 customers to trim the long-run average system congestion level. For comparison purposes, Figure 3.1 also shows a threshold structure (the dashed blue line). Even though the threshold structure cannot approximate the U-shape structure, the reader may wonder how dif- ferent the abandonment rates can be. Our numerical results show that the relative difference can be as high as 30%. Equivalently, the absolute difference in abandonment probability is 1.5% - which is large when considering that reasonably managed service systems exhibit abandonment probabilities less than 5% (Mandelbaum and Zeltyn 2013). The remainder of this paper is organized as follows. We end the introduction by reviewing the related literature. Section 3.2 sets up the model and formulates the optimization problem. Section 3.3 formulates the relevant diffusion control problem (DCP), sets up the HJB equations, and proves the existence of a solution to the HJB equations that solves the DCP. We show that the solution to the linear HJB equations either gives a static class ranking or is of threshold form in Section 3.4, but that this result does not extend to the general HJB equations. For a two class-model, Section 3.5 provides conditions on the patience distributions for 57 when the solution to the HJB equations is either a static ranking or has a threshold form, and further shows that the solution has a U-shape when those conditions are not satisfied. We provide numerics in Section 3.6 that shed light on when taking the U-shape structure into account is important. Finally, we make concluding remarks in Section 3.7. All the proofs of technical results can be found in the electronic companion to this paper. 3.1.1 Literature review Our study relates to the literature on scheduling in queuing systems. The most widely known result from this literature is the optimality of policies (including the static priority policy known as the cµ -rule) that greedily prioritize customers in the order of customers’ instantaneous costs. Recently, Atar et al. (2010, 2011) extended the optimality of thecµ -rule to a system with impatient customers modeled as an overloaded multi-classM/M/N +M system. However, the optimality of greedy policies in the presence of customer abandonment does not extend in general. Instead, scheduling policies that dynamically prioritize customers have been proposed (Harrison and Zeevi 2004, Atar et al. 2004, Gurvich and Whitt 2009, Rubino and Ata 2009, and Ghamami and Ward 2013 for critically loaded systems and Down et al. 2011 for lightly loaded ones). The aforementioned papers all assume that the patience time distributions are exponential. For reasons of mathematical tractability, assuming exponential patience time is a common practice in papers that aim to provide useful insights on managing service operations (Gurvich et al. 2008, and Gurvich and Whitt 2010). Our paper differs from these papers as we use generally distributed patience time. The departure from the exponential patience time assumption is motivated by evidence found in empirical or experimental research; see Brown et al. (2005) and Mandelbaum and Zeltyn (2013) for empirical evidence and Kort (1983) for experimental evidence. The papers that consider general patience time distributions are recent and there are two streams of works therein. The first stream builds machinery to approximate systems with general patience time distributions. Whitt (2006) proposes the fluid approximation of an overloaded single classG/GI/N +GI system. Kang et al. (2010) and Zhang (2013) prove the convergence to the fluid model proposed in Whitt (2006) under different assumptions on service-time distribution. Atar et al. (2013) considers an overloaded multi-class G/GI/N +GI system operated under thecµ -rule and proves the accuracy of the fluid approximation. In 58 that paper, when the patience time distribution is exponential, the cµ -rule is proven to be asymptotically optimal. Finally, Bassamboo and Randhawa (2010) studies the accuracy of fluid models for a single class M/M/N +GI system and solves a capacity sizing problem. The authors show that the optimal operating regime differs based on the structure of the hazard rate function of the patience time distribution. Moving from fluid to diffusion approximations, Ward and Glynn (2005), Dai and He (2010, 2011), and Mandelbaum and Momcilovic (2012) use the value of the patience time density function at the origin to approximate customer abandonments and prove its accuracy. Other papers use the entire distribution to approximate customer abandonments. Reed and Ward (2008) introduce a diffusion approximation of a critically loaded single classGI/GI/1+GI system based on the hazard rate function of the patience time distribution and prove the accuracy of the approximation. Reed and Tezcan (2012) prove a similar hazard rate approximation for a critically loaded single classGI/M/N +GI system. Dai and He (2013) propose a hazard rate based diffusion model for a critically loaded single classGI/Ph/N +GI system and devise an efficient numerical algorithm to calculate its steady state distribution. Weerasinghe (2014) shows that the hazard rate approximation for a critically loaded single classG/M/N +GI system with state-dependent service rates is accurate. Katsuda (2015) relaxes some regularity conditions imposed on the patience time distribution in the literature and proves the accuracy of the hazard approximation for a critically loaded single class G/Ph/N +GI system. Huang et al. (2014) propose a unifying approximation scheme that includes the ones based on the patience time density function considered only at the origin, and these ones that consider the entire hazard rate and show that this unified approximation is accurate for a critically loaded single classG/GI/N +GI system. The second stream of the papers that consider general patience time distributions utilizes the machinery developed in the first stream in order to solve optimization problems for systems with general patience time distributions. In the scheduling literature, there are three such papers. Considering an overloaded single classGI/GI/N +GI system, Bassamboo and Randhawa (2015) show the benefit of prioritizing a priori homogeneous customers based on their actual waiting times instead of serving them in an FCFS manner. Our work differs because we optimize scheduling decisions across different classes (distinguished by patience time distributions and associated abandonment costs) but assume customers are served in the FCFS manner within the same class. Long and Zhang (2015) propose a scheduling policy for a multiclassGI/GI/N+GI system and prove its asymptotic optimality in the fluid scale when the patience time distribution for each 59 class has decreasing hazard rate. In our work, the hazard rate function can take any form and we propose a scheduling policy based on the optimization problem in the diffusion scale. We want to point out that Chapter 2 solve a similar problem to the one considered in this chapter for a single server setting by looking at a critically loaded multiclassGI/GI/1+GI. This chapter is different from Chapter 2 in that we study a many server system (M/M/N +GI) and relax the assumed increasing hazard rate of patience time from Chapter 2. 3.2 Model We model the service system as anM/M/N +GI queue. Customers arrive to this system as a Poisson process with rateλ. The work amount from each customer is exponentially distributed with mean 1 µ and servers are homogeneous in the sense that they all work at rate one. Also, servers are fully flexible; that is, every server can serve any customer. Each customer has a randomly distributed patience time and abandons the system without being served if the service is not commenced within the patience time. Different customers exhibit different impatience behaviors. For example, after having waited a while, some will become less sensitive to the delay and be less likely to abandon. Others will become increasingly frustrated as their wait time increases, and become more likely to abandon. We want to model these different reactions of customers with regard to waiting. In particular, we assume that there areK ≥ 2 different types of patience time distributions, G k for k ∈ K := {1,...,K}, and a class k customer’s patience time is drawn fromG k . We assume that the interior of the domain ofG k is (0,d k ) for somed k ≤∞ andG k has a well-defined density function g k . The rate at which class k customers arrive is given by a k λ for some positive constanta k such that P k∈K a k = 1. All random variables are assumed to be independent of each other. When customers are impatient, customer abandonments indicate customers’ dissatisfaction towards the system. Consequently, each abandonment potentially imposes a cost to the system and the system manager wants to minimize this cost associated with customer abandonments. Given the classification of customers, a relevant decision lever is the scheduling policy that describes which customer an available server will next serve when there are customers from multiple classes waiting. Our goal is to devise a scheduling policy that minimizes the abandonment cost. One intuition to minimize the abandonment cost is to always prioritize the class with the highest instantaneous abandonment cost rate. However, such a policy is myopic. If, for 60 example, each abandonment from that class is relatively cheap, then the system manager may prefer to let that class abandon, and to use those abandonments to trim the overall system congestion, thereby incurring short term pain in exchange for long term benefit. To formally study the design of an optimal scheduling policy, we assume there is a classk abandonment penaltyr k and find a scheduling policyπ that minimizes the long-run average cost c(π) := limsup T→∞ E " X k∈K r k R k (T;π) T # . (3.1) In (3.1),R k (T;π) denotes the cumulative number of classk customers that have abandoned in[0,T] under the scheduling policyπ. Note that whenr k =r for some constantr> 0 and allk∈K, minimizing (3.1) is equivalent to maximizing the throughput rate of the system. In a revenue generating system, the penaltyr k can be interpreted as the revenue lost when a classk customer abandons. The class of scheduling policies we consider are those that (i) do not assume knowledge of the future, (ii) enforce that once a customer enters service, that customers stays in service until completion, (iii) do not allow servers to idle while customers are waiting, and (iv) are stationary. Mathematically, the class of admissible scheduling policiesΠ includes all non-anticipating, non-preemptive, and non-idling policies. We would like to findπ ⋆ ∈ Π such that the long run average cost limit exists and attains the minimum possible cost; that is, c(π ⋆ ) = lim T→∞ E " X k∈K r k R k (T;π ⋆ ) T # = inf π∈Π c(π). (3.2) In general, solving (3.2) is very complicated. This is because writing the state evolution equations must involve measure-valued processes that track the time until each customer’s patience is exhausted; see, for example, Kang et al. (2010) and Zhang (2013). The implication is that calculating the long-run average abandonment rate for any fixed scheduling policy is mathematically not tractable. Our approach is to identify a limit regime under which we expect the solution to (3.2) to be close to the solution to an approximating diffusion control problem (DCP). Then, we solve the DCP, interpret that solution as an admissible policy, and validate its performance numerically. The analytic tractability of the DCP allows us to classify its solution in terms of properties of the patience time distributions, from which follows a connection between the structure of a near-optimal policy and the patience time distribution. 61 We consider large firms, with many servers and much demand. In general, there will be either too much capacity, too little capacity, or the capacity will approximately balance supply and demand. Our focus is the case in which the capacity approximately balances supply and demand. Specifically, we assume N = λ µ +β s λ µ (3.3) for some constantβ ∈ R, so that the system operates in the so-called quality and efficient driven (QED) regime (Halfin and Whitt 1981 and Garnett et al. 2002). Still, by varyingβ in (3.3), our study can provide some understanding of optimal scheduling decisions in overloaded and underloaded regimes as well (see Section 3.6). In the QED regime, the main challenge is to manage the stochastic fluctuations created by the variability in the arrival processes and service times. In the underloaded regime, where there is too much capacity, in the sense thatN = (1 +ǫ) λ µ for some constantǫ > 0 independent ofλ, the scheduling policy becomes irrelevant, because there is very little abandonment under any scheduling policy. On the other hand, in the overloaded regime, where there is too little capacity, in the sense thatN = (1−ǫ) λ µ for some constantǫ> 0 independent of λ, the main challenge to address comes from the imbalance between demand and service capacity. This case makes fluid approximations, such as the one in Atar et al. (2010), relevant. The main message of Atar et al. (2010) is that a static priority policy is optimal with too little capacity. However, as pointed out earlier in the literature review, in the QED regime, static priority has been shown not to be optimal. Notations. For anya∈R, we denote max{a,0} and−min{a,0} bya + anda − , respectively. Also, for an increasing functionh, we leth −1 to be its inverse, that is,h −1 (y) := sup{x≥ 0 :h(x)≤y} fory≥ 0 with the convention thath −1 (y) = 0 for ally<h(0) andh −1 (y) =∞ for ally> sup x≥0 h(x). 3.3 The Diffusion Control Problem In this section, we formulate and solve a DCP that emerges in the QED regime to approximate the opti- mization problem (3.2). In Section 3.3.1, we formulate that DCP by identifying the diffusion process that 62 approximates the number of customers in the system. In Section 3.3.2, we use the HJB equation to solve the DCP, and propose our scheduling policy based on the DCP solution. 3.3.1 The DCP formulation Suppose that there are no stochastic fluctuations and the system staffs under (3.3) with β = 0. Then, no customer will be delayed and no server will idle because demand and supply are perfectly matched. However, with stochastic fluctuations causing inefficiency in the system, customers may be delayed and servers can idle. To formulate the DCP, we capture this inefficiency in a single-dimensional state descriptor. LetX(t;π) be the total number of customers in the system (either being served or waiting) centered by the number of servers at timet and underπ. That is, X(t;π) :=Q(0)+Z(0)−N + X k∈K (A k (t)−S k (t;π)−R k (t;π)), (3.4) whereQ(0) andZ(0) are the number of waiting customers and customers being served att = 0, respectively, andA k andS k are Poisson processes that count the classk arrivals and service completions, respectively. Observe that under any non-idlingπ,[X(t;π)] + and[X(t;π)] − count the number of waiting customers and the number of idling servers, respectively at timet. We now find the diffusion process that approximatesX. By respectively applying the strong approxi- mation toA k andS k in a manner similar to that in Celik and Maglaras (2008), P k∈K (A k (t)−S k (t;π)) in (3.4) is decomposed into three pieces; X k∈K (A k (t)−S k (t;π)) = " X k∈K a k λt− Z t 0 µZ k (s;π)ds # + √ 2λB(t)+ǫ λ (t), (3.5) whereB is a standard Brownian motion. On the right hand side of (3.5), the first term represents the net rate at which customers are added to, or subtracted from, the system. By (3.3) and notingN− P k∈K Z k (s;π) = [X(s;π)] − under any non-idling policyπ, this rate becomes−β √ µλt +µ R t 0 [X(s;π)] − ds. The second term, √ 2λB(t), on the right hand side of (3.5) captures stochastic fluctuations in customer arrivals and service completions. The last term, ǫ λ (t), on the right hand side of (3.5) is an error term from the strong 63 approximation and it is order of magnitude smaller than √ λt. By putting these three pieces together, we reach X(t;π) =X(0;π)−β p µλt +µ Z t 0 [X(s;π)] − ds− X k∈K R k (t;π)+ √ 2λB(t)+ǫ λ (t), (3.6) for anyπ ∈ Π. It now only remains to expressR k in (3.6) as a function ofX to derive the approximating diffusion process. To approximateR k , we need to know how much a classk customer who arrived at timet would wait if that customer is infinitely patient. Our best estimate of this quantity, based on a sample path version of Little’s law known as the Snapshot principle (Reiman 1982), is Q k (t;π) a k λ whereQ k denotes the number of class k customers waiting. A critical premise for the Snapshot principle to hold is that the workload configuration insignificantly changes while a customer waits (from the arrival to the service commencement). The premise holds in the QED regime because a significant change in the workload configuration due to customer arrivals or abandonments needs much longer time than the amount of time a customer spends in the queue waiting for service. In a single class version, Whitt (2005) also used Q k (t;π) a k λ to estimate how much a classk customer would wait and our approximation for R k , (3.9), in the next paragraph, is a continuous version of the equation (3.4) in Whitt (2005). Ifw is the amount of time a classk customer would have to wait before receiving service, thenG k (w) is that customer’s abandonment probability. Hence, the Snapshot principle based approximation from the previous paragraph implies that the probability a classk customer arrived att abandons the system is close to G k Q k (t;π) a k λ . To proceed, note that for any π ∈ Π, there exists a K-dimensional vector of random elements,f, such that for anyt> 0, we haveQ k (t;π) = [X(t;π)] + f k (t;π) where (f 1 (t;π),...,f K (t;π))∈A := ( (q 1 ,...,q K ) : K X k=1 q k = 1,q k ≥ 0 fork = 1,...,K ) , (3.7) for anyt > 0. Therefore, the marginal rate at which classk customers abandon at timet is approximately a k λG k [X(t;π)] + f k (t;π) a k λ and soR k (t;π)≈ R t 0 a k λG k [X(s;π)] + f k (t;π) a k λ ds. Observe that for anyx> 0, G k (x) (a) = 1−exp − Z x 0 h k (y)dy (b) ≈ Z x 0 h k (y)dy, (3.8) 64 where h k denotes the hazard rate function associated with the class k patience time distribution. In the equation (3.8),(a) follows by a property of hazard rate function which isG k (x) = 1−exp − R x 0 h k (y)dy forx≥ 0 and(b) is obtained by1−exp(−x)≈x for smallx by the Taylor series expansion. Using (3.8), we obtain the following approximation for the cumulative classk abandonment up tot: R k (t;π)≈λ Z t 0 m k [X(s;π)] + f k (t;π) λ ds, (3.9) where m k (x) :=a k Z x/a k 0 h k (y)dy. (3.10) The approximation in (3.9) is rigorously justified in single customer class queues in the heavy traffic; see Reed and Ward (2008) for aGI/GI/1 +GI system and Reed and Tezcan (2012) for aGI/M/N +GI system. Replacing R k in (3.6) by the right hand side of (3.9), we are now ready to derive an approximating diffusion process forX. In particular, the approximating diffusion process, which we denote byZ, is given by the (weak) solution of the following stochastic differential equation: Z(t) =Z(0)−β p µλt +µ Z t 0 [Z(s)] − ds−λ X k∈K Z t 0 m k [Z(s)] + q k [Z(s)] + λ ! ds+ √ 2λW (t), (3.11) for some controlq ∈A(∞) :={p :p(t)∈A for allt≥ 0} and a standard Brownian motionW indepen- dent to B. We assume that Z(0) = z for some constant z ∈ R. The control in (3.11) only depends on the system state not on the time explicitly because our focus is on stationary scheduling policies. We want to point out that for eachq ∈ A(∞), there existsd(q) such that the diffusion process underq will never reach states larger than or equal tod(q). Also, we haved(q)≤d :=λ P k∈K a k d k (recall the interior of the domain ofG k is (0,d k )) for anyq ∈A(∞). This is because P k∈A m k ( xq k λ ) =∞ for anyx≥d and any q∈A, implying there is drift of−∞ when the state reachesd. Using the right hand side of (3.9) to replaceR k in (3.2) withX(s;π) andf k (s;π) being respectively substituted byZ(s) andq k ([Z(s)] + ), the objective function in (3.1) is approximated by V (q) := limsup T→∞ E " X k∈K r k λ T Z T 0 m k [Z(t)] + q k ([Z(t)] + ) λ dt # , (3.12) 65 forq∈A(∞). Then, the DCP associated with (3.2) is to findq ⋆ ∈A(∞) such that V (q ⋆ ) = lim T→∞ E " X k∈K r k λ T Z T 0 m k [Z ⋆ (t)] + q ⋆ k ([Z ⋆ (s)] + ) λ ! dt # = inf q∈A(∞) V (q), (3.13) whereZ ⋆ is the (weak) solution of the following stochastic differential equation: Z ⋆ (t) =Z ⋆ (0)−β p µλt +µ Z t 0 [Z ⋆ (s)] − ds−λ X k∈K Z t 0 m k [Z ⋆ (s)] + q ⋆ k ([Z ⋆ (s)] + ) λ ! ds+ √ 2λW ⋆ (t) for a standard Brownian motion W ⋆ that is independent of the other random elements introduced in the paper. An alternate DCP formulation: The exponential approximation. The derivation of the DCP in (3.13) entails the use of the hazard rate function of the patience time distribution. Given that the delay is short in the QED regime, a more straightforward approach would be to use the value of the density functiong k (0), orh k (0) asg k (0) = h k (0), to approximate the cumulative number of abandoned classk customers. This follows because m k (x) =g k (0)x+R k (x), (3.14) whereR k (x) is the residual term from the Taylor series expansion. Under this approach, we would define m k (x) asm k (x) :=g k (0)x instead of the one in (3.10) and (3.9) would have been substituted by R k (t;π)≈ Z t 0 g k (0)[X(s;π)] + f k (s;π)ds, (3.15) and the resulting diffusion process would have linear drift. The approximation in (3.15) for single customer class queues in the heavy traffic is proven to be accurate in the literature. Note that (3.9) and (3.15) are identi- cal if and only if the classk patience time distribution is exponential. Henceforth, we call the approximation in (3.15) “the exponential approximation”. Note that under the exponential approximation the underlying diffusion process does not have upper bound whetherd(q) is finite or not. So, for anyq ∈ A(∞), we set d(q) :=∞ for the exponential approximation. 66 3.3.2 The DCP solution Our solution approach for the DCP in (3.13) is to use stochastic calculus by applying Ito’s lemma on the stochastic processZ in (3.11); see Oksendal (2013). This leads to an optimality equation that we call the Hamilton-Jacobi-Bellman (HJB) equation which is an ordinary differential equation (ODE) given by λv ′ (x)+ √ µ [x] − −β √ λ √ µv (x)+λmin q∈A φ(x,v(x),q) =κ such that sup x<d |v(x)|<∞ (3.16) where φ(x,w,q) := X k∈K (r k −w)m k [x] + q k λ for allx<d, w∈R, andq∈A. (3.17) We call a pair (v,κ) of a continuously differentiable function,v : (−∞,d) → R, and a positive constant, κ, that solves (3.16) a solution of the HJB equation. Using the HJB equation, we derive an optimal control of (3.13) in the following theorem. The theorem also provides some properties of the HJB solution that are useful in characterizing the structure of an optimal control of the DCP in (3.13). Theorem 3.3.1. (i) There exists a continuously differentiable function v ⋆ and a positive constant κ ⋆ such that the pair (v ⋆ ,κ ⋆ ) is a unique solution of the HJB equation in (3.16). Furthermore, sup x<d |v ⋆ (x)| ≤ min{r 1 ,...,r K } andv ⋆ is increasing 1 . (ii) For the DCP in (3.13),κ ⋆ = inf q∈A(∞) V(q) andq ⋆ such thatq ⋆ (x) := argmin q∈A φ(x,v ⋆ (x),q) forx<d is an optimizer. Theorem 3.3.1 connects the structure of an optimal control of the DCP and the underlying patience time distributions through the equation q ⋆ (x) = argmin q∈A ( X k∈K r k m k x + q k λ − X k∈K v ⋆ (x)m k x + q k λ ) . (3.18) We see from (3.18) that the DCP solution balances the short-term and the long-term consequences in making scheduling decisions. The first term on the right hand side of (3.18) is from the integrand of the objective 1 Part (i) generalizes Theorem 2.4.3 for the existence of the solution of the HJB equation. In particular, the current chapter relaxes the assumed increasing hazard rate for patience time in Chapter 2. Furthermore, Theorem 2.4.3 of Chapter 2 requiresβ≥ 0 (θ≤ 0 in Chapter 2) 67 function in (3.13) and it represents the instantaneous (or marginal rate of) abandonment cost. The second term on the other hand reflects future consequences of scheduling decisions made at the present through the value function,v ⋆ . Therefore,q ⋆ is designed to take both the short-term and long-term consequences into consideration. This is an important aspect of an optimal scheduling when catering to impatient customers. In the literature, when the abandonment distribution is exponential, this is the intuition for why a static priority policy does not perform well; see, for example, Rubino and Ata (2009) and Down et al. (2011). Proposed policy. Givenq ⋆ that solves (3.13), we now introduce how we interpret it as a scheduling policy in the actual system. Our policy attempts to maintain the proportion of waiting customers atq ⋆ . To do this, a newly idle server processes a customer from the class that exceeds the most from the amount given byq ⋆ . To formally describe our proposed policy, recallQ k (t;π) is the number of classk customers waiting for the service at timet under a scheduling policyπ∈ Π. Under our proposed policy, an available server at timet serves a customer in the head of the classi queue where i∈ argmax k∈K Q k (t;π p ) P j∈K Q j (t;π p ) −q ⋆ j X j∈K Q j (t;π p ) . (3.19) We expect the proportion of classk customers among all waiting customers to trackq ⋆ k under our proposed policy: For exponential patience time distributions, this point is rigorously shown in Gurvich and Whitt (2009). So, by studying properties ofq ⋆ , we understand how the system configuration will look. 3.4 Static Priority and Threshold Control In this section, we derive a solution of the DCP (3.13) when m k (x) is linear in x for all k ∈ K, which happens under the exponential approximation in (3.15). The optimal control from this linear case will be a benchmark for optimal controls derived under the more general approximation (3.9) (in both performance- wise and structure-wise). In the linear case, the HJB solution takes the form of threshold control which is defined as follows. 68 Definition 3.4.1. Let D ⊆ K, with |D| = J ≥ 2, and let (p 1 ,...,p J ) be a permutation of the class indices inD. A threshold control is defined by a J − 1 dimensional vector L = (L 1 ,...,L J−1 ) having L 0 := 0<L 1 <···<L J−1 <L J :=∞ such that q p j (x) = 1 {L j−1 ≤x<L j } forj∈D and q p j (x) = 0 forj / ∈D. From Definition 3.4.1, we see that the key in the design of an optimal threshold control is to specify a class to be prioritized the least, as a function of the system work load, rather than the full priority ranking among classes. The static version of threshold control is defined by settingJ = 1 in Definition 3.4.1, so that it is characterized byk such thatq k (x) = 1 forx> 0. While this static version does not statically rank all classes, we still call this control static priority because it specifies a class that is to statically receive the least priority, which we expect to be more important than assigning rankings among all classes. For an optimal threshold control, the identification of classes for which there exists x > 0 such that q k (x) = 1 is given in the following algorithm (see Figure 3.2 for the visualization of it). To proceed, let m k (x) = γ k x for some constant γ k > 0 and all x > 0. (Without loss of generality, we assume γ 1 r 1 ≥γ 2 r 2 ≥···≥γ K r K .) Algorithm to find an optimal threshold control. 1. Let J 1 :={k∈K :γ k >γ j for allj∈K such thatj >k}. (Observe thatJ 1 is non-empty becauseK ∈ J 1 .) For anyk ∈ K\J 1 , we setq k (x) = 0 forx > 0. Relabel class indices so thatJ 1 ={1,2,...,J 1 } withγ 1 r 1 ≥ γ 2 r 2 ≥···≥ γ J 1 r J 1 andγ 1 > γ 2 > ···>γ J 1 . Intuition. Fork,j ∈ K, supposek > j andγ k ≥ γ j . When the workload is low, prioritizing class j over k to minimize the instantaneous abandonment cost, which is myopic, is optimal. Doing so minimizes the instantaneous abandonment cost because γ j r j ≥ γ k r k . When the workload is high, we want to trim down the congestion with customer abandonments. Becauseγ k ≥γ j , which implies 69 r j >r k , exploiting classj abandonments is too costly and again it is better to prioritize classj over k. Hence,q ⋆ j (x) = 0 forx> 0. 2. Let C :={(γ 1 ,γ 1 r 1 ),...,(γ J 1 ,γ J 1 r J 1 )}∩ (x,y)∈R×R :y≤ γ 1 r 1 −γ J 1 r J 1 γ 1 −γ J 1 (x−γ J 1 )+γ J 1 r J 1 andJ 2 := {j∈J : (γ j ,γ j r j )∈C}. (Observe thatJ 2 is non-empty because 1,J 1 ∈ J 2 .) For any j∈J 1 \J 2 , we setq j (x) = 0 forx> 0. Intuition. Suppose1,2,3∈J 1 while1,2,∈J 1 and2∈J\J 1 so that γ 1 r 1 −γ 2 r 2 γ 1 −γ 2 < γ 1 r 1 −γ 3 r 3 γ 1 −γ 3 < γ 2 r 2 −γ 3 r 3 γ 2 −γ 3 . (3.20) To understand whyq ⋆ 2 (x) = 0 forx > 0, we want to point out that fork < j, r k γ k −r j γ j γ k −γ j captures the marginal benefit of prioritizing class k customers over class j customers. By serving a class k customer over class j customer, the system can reduce the instantaneous abandonment cost, which is myopic, byγ k r k −γ j r j . On the other hand, by prioritizing classj over classk, the system gains the additional abandonment rate ofγ k −γ j and hence can better trim down the congestion, which is forward-looking. Also, recall from the argument after Theorem 3.3.1 thatv ⋆ (x) adjusts the penalty for each abandonment by incorporating future consequences of the current decision. Therefore, when the workload isx andk<j, ifv(x)< r k γ k −r j γ j γ k −γ j , it is optimal to prioritize classk overj for the myopic benefit of reducing the instantaneous abandonment cost. Otherwise, there is strong incentive to reduce the congestion for the future consequences by letting classk customers abandon. By applying this argument to (3.20), it is straightforward to deduce thatq ⋆ 2 (x) = 0 forx> 0. 3. LetJ 3 :={j∈J 2 : (γ j ,γ j r j ) / ∈ int(conv(C))}, where int(A) and conv(A) denote the interior and the convex hull of the setA, respectively. For anyj ∈J 2 \J 3 , we setq j (x) = 0 forx > 0. Relabel class indices so thatJ 3 :={1,...,J 3 } withγ 1 r 1 ≥γ 2 r 2 ≥···≥γ J 3 r J 3 andγ 1 >γ 2 >···>γ J 3 . Intuition. For anyj∈J 2 \J 3 , we can always findj 1 ,j 2 ∈J 1 such that the(γ j ,γ j r j ) is above the line that connects(γ j 1 ,γ j 1 r j 1 ) and(γ j 1 ,γ j 2 r j 2 ). Hence, we can repeat the same argument from (ii) above and conclude thatq j (x) = 0 forx> 0. 70 4. Optimizing the threshold structure within J 3 . Let {T j } J 3 +1 j=1 be the set of constants where T j := r j−1 γ j−1 −r j γ j γ j−1 −γ j forj∈{2,...J 3 },T J 3 +1 := 0 andT 1 :=∞. Forx> 0, we set q j (x) = 1 {T j+1 <v ⋆ (x)≤T j } . (3.21) It is straightforward to check that q given in this reduction algorithm is threshold control. This is becauseT j is decreasing inj (asJ 2 is the set of vertices of a convex set) andv ⋆ (x) is increasing inx by Theorem 3.3.1. Intuition behind the optimality of the structure given in (3.21). SupposeJ 3 = 3. (The structure ofq ⋆ whenJ 3 > 3 are essentially identical and hence we focus onJ 3 = 3.) We have γ 2 r 2 −γ 3 r 3 γ 2 −γ 3 < γ 1 r 1 −γ 3 r 3 γ 1 −γ 3 < γ 1 r 1 −γ 2 r 2 γ 1 −γ 2 , (3.22) from which we can deduce thatq ⋆ 3 (x) = 1 for smallx,q ⋆ 2 (x) = 1 for intermediatex, andq ⋆ 1 (x) = 1 for largex by applying the argument given in (ii) to (3.22). The optimality of the given threshold control is formalized in the following proposition. Proposition 3.4.1. Supposem k (x) in (3.12) isγ k x for someγ k > 0 and allk ∈ K withγ 1 r 1 ≥ γ 2 r 2 ≥ ···≥ γ K r K . Then, either a static priority or a threshold control is optimal for the DCP (3.13) andq ⋆ (x) forx> 0 satisfies (3.21). When the patience time distributions are all exponential, Proposition 3.4.1 in fact proves the asymptotic optimality of threshold control or static priority. To see this, note that the HJB in this exponential case was studied in Section 5 of Atar et al. (2009) (with a caveat that they considered the infinite horizon discounted cost criterion whereas we look at the long-run average cost criterion). The paper Atar et al. (2009) proves the asymptotic optimality of the HJB solution while not explicitly solving the HJB equation. Hence, the asymptotic optimality of threshold control or static priority follows by combining the result in Atar et al. (2009) and Proposition 3.4.1 in this paper. Even for non-exponential patience time distributions, Proposition 3.4.1, together with the accuracy of the exponential approximation found in the aforementioned single class papers, suggests that threshold control or static priority should perform well. However, there are two major concerns. First, there are many distributions, havingg k (0) = 0 org k (0) = ∞, in which case the exponential approximation is no 71 γ γr 0 2 2 4 4 6 6 8 8 10 10 (γ 1 ,γ 1 r 1 ) (γ 2 ,γ 2 r 2 ) (γ 3 ,γ 3 r 3 ) (γ 4 ,γ 4 r 4 ) (γ 5 ,γ 5 r 5 ) (γ 6 ,γ 6 r 6 ) (a)(γ k ,γ k r k ) fork∈K ={1,...,6} γ γr 0 2 2 4 4 6 6 8 8 10 10 (γ 1 ,γ 1 r 1 ) (γ 2 ,γ 2 r 2 ) (γ 4 ,γ 4 r 4 ) (γ 5 ,γ 5 r 5 ) (γ 6 ,γ 6 r 6 ) (b) Step 1: Removal of(γ3,γ3r3). γ γr 0 2 2 4 4 6 6 8 8 10 10 (γ 1 ,γ 1 r 1 ) (γ 4 ,γ 4 r 4 ) (γ 5 ,γ 5 r 5 ) (γ 6 ,γ 6 r 6 ) (c) Step 2: Removal of(γ2,γ2r2). γ γr 0 2 2 4 4 6 6 8 8 10 10 (γ 1 ,γ 1 r 1 ) (γ 5 ,γ 5 r 5 ) (γ 6 ,γ 6 r 6 ) (d) Step 3: Removal of(γ4,γ4r4). Figure 3.2: Visualization of the dimensional reduction. help. Second, the value of the density function (or its derivative of any order in general) at 0 is not a robust statistics and misestimation of that value can cause a change in the policy structure. Therefore, from the policy design perspective, it is preferable to understand the solution of (3.13) when the entire patience time distribution is included, and ask whether or not the solution is threshold control or static priority. Unfortunately, threshold control or static priority for the DCP (3.13) is not in general optimal. We illustrate this in the example below where we present one of the few special cases in which q ⋆ admits a closed form. 72 Example. For allk∈K, supposer k =r> 0 and the hazard rate of the classk patience time distribution is continuous and strictly increasing without a bound in its domain. Then,q ⋆ is given by q ⋆ (x) = λa 1 x h −1 1 ψ(x)λ x ,..., λa K x h −1 K ψ(x)λ x , (3.23) for allx∈ (0,d) where for eachx∈ (0,d),ψ(x) is a unique constant that satisfies X k∈K λa k x h −1 k ψ(x)λ x = 1. Because the hazard rate function for each class is assumed to be continuous,q ⋆ in (3.23) is continuous inx and hence is neither a threshold control nor a static priority. The fact thatq ⋆ is neither a threshold control nor a static priority does not necessarily imply that threshold control yields the cost that is strictly bigger than the optimal cost (or equivalently, threshold control is sub-optimal). This is because the HJB equation is only a sufficient condition for the DCP solution. In the proof of proposition 3.4.2, we prove that the cost under threshold control or static priority is strictly bigger than the optimal cost whenq ⋆ is neither threshold control nor static priority. In fact, when there are two customer classes with non-exponential patience time distributions, the fol- lowing result shows that neither threshold control nor static priority is optimal for (3.13) in more generality. Proposition 3.4.2. Suppose there existk 1 ,k 2 ∈K such thatg k i (0) = 0 andg k i (x) is continuous atx = 0 fori∈{1,2}. Then, neither threshold control nor static priority is optimal for the DCP (3.13). The stated condition in the proposition excludes situations such thatg k (x) is decreasing inx near the origin for all k ∈ K. In such a situation, we can identify some conditions under which either threshold control or static priority is optimal (see Section 3.5.1). Although static priority and threshold control are not optimal in general, there may still be cases when they are. We would like to (1) understand those conditions, and (2) determine structural results for the optimal policy when those conditions are violated. We next focus on a two-class system which provides us with the analytic tractability to understand these points. 73 3.5 A Two-class Model In a two-class setting, the optimal control in Theorem 3.3.1 reduces to q ⋆ 1 (x) = arg min q 1 ∈[0,1] φ(x,v ⋆ (x),(q 1 ,1−q 1 )), for allx> 0, (3.24) andq ⋆ 2 = 1−q ⋆ 1 . The casex≤ 0 corresponds the system having idle servers, meaning the scheduling policy becomes irrelevant. In particular, whenx≤ 0,φ(x,w,(q 1 ,1−q 1 )) = 0 for anyw∈R andq 1 ∈ [0,1], so in this section we only considerx> 0. Our objective in this section is to understand how the patience time distribution influences the structure of q ⋆ . In particular, we analyze the solution of (3.24) for different types of the class 1 patience time distribution while fixing the class 2 patience time distribution to be exponential with rateγ 2 . First, in Section 3.5.1, we provide conditions under which a decreasing hazard rate function leads to the optimality of the static priority and threshold structures. Then, in Section 3.5.2, we find that without the decreasing hazard rate function, the optimal control has a U-shape structure (and so is neither threshold nor static priority). To state the results in the rest of the paper, we denote threshold controls by q 1 T (x) := (0,1) ifx<T (1,0) ifx≥T , forT ∈ (0,∞), q 2 T (x) := (1,0) ifx<T (0,1) ifx≥T , forT ∈ (0,∞) and static controls by q 1 S (x) := (0,1) for allx> 0, q 2 S (x) := (1,0) for allx> 0, so that underq k S , classk customers are statically prioritized over the other class customers. 74 3.5.1 Optimality of the threshold and static priority Suppose the hazard rate function associated with the class 1 patience time distribution is decreasing. Ifγ 2 is small compared toh 1 (0), then to minimize abandonments, intuition suggests prioritizing class 1 customers when the number of customers in the system is small. This is because the class 2 customers are likely to be patient enough for the few class 1 customers in the system to complete their service. However, as the number of customers increases, depending on the penaltiesr 1 andr 2 , we may want to take advantage of the fact that any class 1 customer that has spent some time waiting becomes even more likely to wait longer, sinceh 1 is decreasing. This suggests that the priority could switch, meaning the structure of the optimal policy will be threshold. If the priority never switches, then the structure is static priority, with class class 1 customers always having priority over class 2 customers. We formalize the intuition for the threshold and static priority structures in the preceding paragraph by providing necessary and sufficient conditions under which these structures emerge. To do this, we first observe that whenh 1 is decreasing, thenφ(x,v ⋆ (x),(q 1 ,(1−q 1 ))) is concave inq 1 ∈ [0,1], which means that the minimum occurs at a corner point 2 . Furthermore, it is straightforward to check that the minimum occurs at q ⋆ (x) = (0,1) if r 1 −v ⋆ (x) r 2 −v ⋆ (x) m 1 (x/λ) x/λ ≥γ 2 , (1,0) otherwise. (3.25) The conditions for the threshold and static priority structures emerge from (3.25) whenr 2 ≥ r 1 , by recog- nizing that r 1 −v ⋆ (x) r 2 −v ⋆ (x) m 1 (x/λ) x/λ is decreasing inx (see the proof of Proposition 3.5.1). Proposition 3.5.1. Supposeh 1 (x)≤h 1 (y) for anyx>y≥ 0 andh 2 (x) =γ 2 forx≥ 0. (i) Supposer 2 >r 1 . Then, there existsT ∈ (0,∞) such thatq ⋆ =q 1 T if and only if (r 1 −v ⋆ (0))h 1 (0)> (r 2 −v ⋆ (0))γ 2 . (3.26) Otherwise, if (3.26) does not hold, thenq ⋆ =q 2 S . 2 Note that the class 1 patience time distribution must have unbounded domain, that is d1 =∞, under the assumed decreasing hazard rate. Therefore, a corner point forq1 is either 0 or 1. We point this out because ifd1 was a finite number, then forx > λa1d1, a corner point forq1 would be either 0 or λa 1 d 1 x . 75 (ii) Supposer 2 =r 1 andh 1 (0)>γ 2 . Then, there existsT ∈ (0,∞)such thatq ⋆ =q 1 T if and only if lim x→∞ m 1 (x) x <γ 2 . (3.27) Otherwise, if (3.27) does not hold , thenq ⋆ =q 1 S . However, the optimality of threshold control does not generalize. Even when the hazard rate function associated with the class 1 patience time distribution is decreasing, ifr 1 > r 2 , it is in general not true that q ⋆ takes threshold structure as when r 2 ≥ r 1 . If r 1 > r 2 , prioritizing class 2 customers by relying on class 1 customers’ increasing patience might not be optimal when the number of waiting customers is large. Especially, if class 1 customers become not sufficiently patient as their waits increase, we can formalize that it is optimal to prioritize class 1 customers who have waited enough. The consequence is the sub- optimality of threshold control even under the same condition, (r 1 −v ⋆ (0))h 1 (0) > (r 2 −v ⋆ (0))γ 2 from Proposition 3.5.1(i). Proposition 3.5.2. Supposer 1 >r 2 andh 1 is a non-constant function withh 1 (x)≤h 1 (y) for anyx>y≥ 0 andh 2 (x) = γ 2 forx≥ 0. If (r 1 −v ⋆ (0))h 1 (0) > (r 2 −v ⋆ (0))γ 2 , and lim x→∞ m 1 (x) x > 0, then there exist constants 0 < T 1 ≤ T 2 < ∞ such thatq ⋆ (x) = (0,1) forx ∈ (0,∞)\[T 1 ,T 2 ), andq ⋆ (x) = (1,0) otherwise. Furthermore,V q i T >κ ⋆ fori∈{1,2} and anyT ∈ (0,∞). Instead, In the statement of Proposition 3.5.2, the condition lim x→∞ m 1 (x) x > 0 corresponds to the situation where class 1 customers become not sufficiently patient as they wait. Note that in Proposition 3.5.2, neither the caseT 1 =T 2 is completely ruled out nor we provide sufficient conditions to do so. This is because when r 1 >r 2 , lim x→∞ m 1 (x) x ≥γ 2 impliesq ⋆ (x) = (0,1) for allx> 0 by (3.25). Also, sufficient conditions to excludeT 1 = T 2 heavily depend on the structural properties ofv ⋆ for which only limited knowledge (the ones given in Part (i) of Theorem 3.3.1) is available. 3.5.2 Sub-optimality of threshold structure When the hazard rate associated with the class 1 patience time distribution is not decreasing, the structure of the optimal control is less straightforward to describe analytically. Therefore, we begin with a numerical study. Figures 3.3a and 3.3b respectively solve for the optimal control when the class 1 hazard rate is Weibull (r 1 = 3,r 2 = 3.3,h 1 (x) = 0.4x 3 ,λ = 100,γ 2 = 0.5,β = 0,µ = 1, anda 1 = 0.2) and also when it is 76 log-normal with mean 1 and standard deviation 0.5 (r 1 = 3,r 2 = 3.3,λ = 100,γ 2 = 0.1,β = 0,µ = 1, anda 1 = 0.6). In both cases, the optimal control has a U-shape structure. x q ⋆ 1 (x) 0 50 100 150 0.2 0.4 0.6 0.8 1 (a) Class 1 patience time distribution is Weibull with increasing hazard rate function (h1(x) = 0.4x 3 ). x q ⋆ 1 (x) 0 50 100 150 200 0.2 0.4 0.6 0.8 1 (b) Class 1 patience time distribution is log-normal with uni-modal hazard rate function (mean 1 and standard deviation 0.5). Figure 3.3:q ⋆ 1 under different patience time distributions. The intuition for the shape ofq ⋆ 1 in Figure 3.3 is as follows. When the number of customers waiting is small (workload,x, near 0), since the class 1 hazard rate is small around 0 (true for both the Weibull and log-normal distributions that we consider for whichh 1 (0) = 0), we can safely prioritize class 2 customers and rely on class 1 customers being patient enough to wait for some class 2 customers to finish their service. However, since the class 1 hazard rate is increasing near the origin as the number of customers waiting increases, class 1 customers lose patience, and begin abandoning in greater numbers. Then, even though class 1 has the lower penalty per abandonment than class 2 does,r 1 < r 2 , as in the setting for Figure 3.3, the cost incurred by class 1 abandonment will be significant. The fix is to not strictly prioritize class 2 customers, and to split priorities between the two classes as the number of waiting customers increases. This explains whyq ⋆ 1 decreases in Figure 3.3. However, consistent with Proposition 3.5.1, eventually we prefer to let class 1 customers abandon becauser 1 < r 2 . If the hazard rate is always increasing (as in the case for the Weibull in Figure 3.3a), we can do this in a manner that continuously balances the instantaneous cost of abandonments and the future benefit. If the hazard rate is decreasing for large workload x (as is the case for the log-normal in Figure 3.3b, which is unimodal), continuously achieving the aforementioned balance is not possible, andq ⋆ 1 becomes discontinuous as in Propositions 3.5.1 and 3.5.2. 77 Figure 3.3, and the intuition behind it, makes it clear that we do not expect the optimal policy to have a threshold structure when the class 1 hazard rate is neither decreasing nor constant. We justify this statement analytically under the assumption that the class 1 hazard rate is increasing in the following proposition. Proposition 3.5.3. Suppose h 1 is a non-constant function and h 1 (x) ≥ h 1 (y) for any x > y ≥ 0 and h 2 (x) =γ 2 forx≥ 0. Then,q ⋆ is given by q ⋆ (x) = min 1, λa 1 x h −1 1 r 2 −v ⋆ (x) r 1 −v ⋆ (x) γ 2 ,1−min 1, λa 1 x h −1 1 r 2 −v ⋆ (x) r 1 −v ⋆ (x) γ 2 (3.28) for x ∈ (0,d), and V q 1 S > κ ⋆ and V q i T > κ ⋆ for i ∈ {1,2} and T ∈ (0,∞). Furthermore, V q 2 S >κ ⋆ if (i) r 1 ≥r 2 or; (ii) r 1 <r 2 andG k has the bounded domain. 3.6 Numerical Studies Thus far, our study has shown that the optimality of either static priority or a threshold control for some class of patience time distributions does not generalize. The main goal of this section is to understand how significant the optimality gap betweenq ⋆ andq ∈T whereT := q i T ,q i S fori∈{1,2} andT ∈ (0,∞) in the two-class example and under which circumstances the gap is more severe. Specifically, we compare the performance ofq ⋆ to that ofq T ⋆ forq T ⋆ := inf q∈T V (q). To ease our notation, we callq inT threshold control becauseq i S =q i T forT =∞. In addition to that, we present numerical results on a class of simple heuristics, (derived from the fixed- queue-ratio policies analyzed in Gurvich and Whitt 2010), which is structured based on the DCP solution, that we find more efficient in managing customer impatience than threshold control. Finally, we use our study to understand an optimal scheduling policy in overloaded and underloaded regimes by varying the safety staffing parameter,β. Performance comparison. We solve the DCP (3.13) for different values of the ratio of class 1 abandon- ment penalty to class 2 abandonment penalty, while fixing other parameters. For each ratio, we simulate the 78 system under (3.19) to obtain a pair of class 1 and class 2 abandonment rates. We call the line that connects the pairs an efficient frontier. We then display our results by comparing the efficient frontier and the fron- tier constructed only by considering threshold controls. In the corresponding figures, two dashed lines that wrap each bold line depict 95% confidence intervals. (To generate one frontier, we repeat 50 simulations to calculate the average of abandonment rate and run each simulation until one million customers arrive.) In the following example, we assume the patience time distribution for class 1 is Beta withα = 4 and β = 0.1 while it is Beta withα = 7 andβ = 0.1 for class 2 (a 1 = 0.5,λ = 30,N = 30,µ = 1). Both g 1 (0) and g 2 (0) are 0, so that the exponential approximation discussed in Section 3.3.1 cannot be used. The comparison between the frontiers achieved under our proposed policy and threshold control is depicted in Figure 3.4. As shown in Figure 3.4, the two frontiers are clearly separated: We observe the relative improvement of more than 30%. For example, our proposed policy can achieve abandonment rate of class 2 at 27 per hour while it is 40 per hour for the optimal threshold control when we fix class 1 abandonment rate at 30 per hour. On the other hand, when class 2 abandonment rate is 40 per hour, our proposed policy can lower class 1 abandonment rate to 18 per hour from 30 per hour achieved under the optimal threshold control. These improvements are equivalent to a decrease in the abandonment probability of one class by an absolute 1.5%-point (not relative 1.5%) without altering the abandonment probability of the other class. This is a significant improvement in the QED regime in which the typical abandonment probability is less than 10%. Class 1 abandonments per hour Class 2 abandonments per hour 0 18 36 54 72 9 27 45 63 81 Proposed policy Threshold control Figure 3.4: Performance comparison. We also find that the behavior of the patience time distribution near the origin heavily affects the opti- mality gap betweenq ⋆ andq T ⋆. In particular, in the two-class model, ifg k (0) = ∞ for allk ∈ K,q ⋆ and 79 q T ⋆ perform almost identically. This is because in such a case, as we discuss in Section 3.5.1, thatq ⋆ (x) must take values in{(1,0),(0,1)} forx> 0, so that even when threshold control is not optimal, we expect there exists a threshold control that performs close toq ⋆ . On the other hand, ifg k (0) is 0 for one class and ∞ for the other class, our proposed policy outperforms threshold control, but the gap is not as significant as we see in Figure 3.4. Therefore, we conclude that as the number ofk ∈K such thatg k (0) = 0 grows, we have a higher incentive to use our proposed policy over threshold control. Fixed-queue-ratio policy. We show in Section 3.4 that finding the cost-minimizing threshold control is straightforward and intuitive when the exponential approximation can be used (g k (0) / ∈ {0,∞} for all k ∈ K). However, for many patience time distributions, the prescription in Section 3.4 that simplifies the search for an optimally configured threshold control cannot be applied. Consequently, the search has to be based on simulations, which can be very inefficient: For a K-class system, there are K! structurally different threshold controls to be tested (recall that threshold control is characterized by a permutation of the class indices). Hence, from a practical point of view, it is necessary to identify policies that can potentially outperform threshold control and are more straightforward to optimize. To this end, we study the class of controls that satisfies q f (x) = q f 1 ,...,q f K for all x > 0 where (q f 1 ,...,q f K ) is aK-dimensional vector of constants such thatq f k ≥ 0 and P k∈K q f k = 1. The implemen- tation of this control is done by substitutingq ⋆ in (3.19) toq f . When implemented, this control prioritizes a class whose proportion among waiting customers positively deviates the most from the specified ratio. Hence, under this control, we expect the proportion of classk among waiting customers will remain close to q f k and call this control fixed-queue-ratio (FQR) 3 . Our motivation behind studying FQR controls is as follows. We notice that the structure of q ⋆ in Figure 3.3a and Figure 3.3b are distinguished from that of threshold control in that they are non-extremal, i.e.,q k (x)∈ (0,1) for somex> 0. So, the idea is to try out the simplest non-extremal control which is FQR. We find that FQR control is efficient in managing customer impatience. To illustrate, let K 0 := {k∈K :g k (0) = 0}. In our numerical studies, we see that ifK 0 = K, then a FQR control outperforms the cost-minimizing threshold control. For example, in the above Beta example, where our proposed policy 3 In the literature, Gurvich and Whitt (2010) extensively studies FQR control and proves its asymptotic optimality for the constrained staffing cost minimization problem when constraints regulate quality-of-service metrics such as the tail probability of waiting time or the abandonment probability for each class. In Gurvich and Whitt (2010), patience time distribution is assumed to be exponential and in our paper, we test effectiveness of FQR control for other types of patience time distribution. 80 significantly outperforms threshold control, FQR control and our proposed policy perform almost identi- cally; see Figure 3.5. Note that in small portion of Figure 3.5, FQR slightly outperforms our proposed policy. This is because our proposed policy is based on the diffusion approximation that can guarantee the near-optimality but not the exact optimality. Class 1 abandonments per hour Class 2 abandonments per hour 0 18 36 54 72 9 27 45 63 81 Proposed policy FQR control Figure 3.5: Performance comparison between our proposed policy and FQR control. WhenK 0 is a proper subset ofK, the FQR and threshold controls perform similarly. Whether FQR control outperforms threshold control or vice versa, the performance gap is small. See Figure 3.6 for our numerical illustrations in the two-class example for which class 1 patience time distribution is Weibull and class 2 patience time distribution is exponential. (For both figures, we assumea 1 = 0.5,µ = 1, andλ = 30.) When it is not clear to know which of the two performs better, we recommend FQR control over threshold control. This is because as mentioned earlier, for a large class of patience time distributions, configuring an optimal control in the class of FQR controls is more efficient than in the class of threshold controls. Connection to the other operating regimes. While we formally study the QED regime, our study can be used to understand the optimal control in underloaded and overloaded regimes as well. We do this by comparing the performance between our proposed policy and the cost-minimizing threshold control for different server utilization levels, i.e., different values of the safety staffing coefficient, β in (3.3). In particular, we consider the same two-class setting used in the previous study (for Figure 3.4) except that we varyβ ∈ n − 4 √ 30 ,− 2 √ 30 ,..., 4 √ 30 o , so that the number of servers, N, and the traffic intensity, λ Nµ , take values in{26,28,...,34} and{1.15,1.0728,...,0.88}, respectively. 81 replacements Class 1 abandonments per hour Class 2 abandonments per hour 0 18 36 54 72 9 27 45 63 FQR control Threshold control (a)h1(x) = 4x 3 andh2(x) = 0.1. Class 1 abandonments per hour Class 2 abandonments per hour 0 9 27 45 63 36 54 72 90 108 126 FQR control Threshold control (b)h1(x) = 4x 3 andh2(x) = 1. Figure 3.6: Performance comparisons between FQR control and threshold control. For each traffic intensity, we simulate the frontiers between the class 1 and class 2 abandonment rates under our proposed policy and the optimal threshold policy, respectively, as we did in the previous numerical study. Then, we measure the distance between the frontiers where the metric are given by averaging out the relative improvement and the absolute improvement in class 2 abandonment rate and class 2 abandonment probability, respectively, for each level of class 1 abandonment rate. In this numerical study, we find that the benefit of fully incorporating the entire patience time distribution in the design of scheduling policy becomes substantial when the traffic intensity is close to 1. For large values of traffic intensity, our proposed policy and the cost-minimizing threshold control have similar performance as indicated by small relative and absolute improvements in Table 3.1. On the other hand, for small values of traffic intensity, we observe that distinguishing our proposed policy and the cost-minimizing threshold control matters less: In this case, the fraction of abandoned customers is really small regardless of the choice of scheduling policy which explains the huge relative improvement with the minor absolute improvement for small traffic intensity in Table 3.1. Traffic intensity ( λ Nµ ) 1.15 1.07 1 0.94 0.88 Relative improvement from threshold control (%) 4.2 21.3 35.2 62 73 Absolute improvement from threshold control (%-point) 0.5 0.9 0.9 0.6 0.2 Table 3.1: The role of the safety staffing coefficient (β) in the performance improvement 82 3.7 Conclusion In this paper, we study the problem of scheduling heterogeneous customers to minimize long-run average abandonment cost in a many-server service system. Such systems have been studied for cases in which customers have exponential patience distributions, and our focus is on the general patience distribution case. We use a diffusion-based approach to characterize the Hamilton-Jacobi-Bellman optimality equations that a near-optimal control should satisfy, and further we propose a near-optimal policy based on the solution to these HJB equations. Even for exponential patience distributions, the structure of the solution to the HJB equations has not been identified in the literature, and our first contribution is to provide this characteri- zation: we formally prove that a threshold control policy is asymptotically optimal. For non-exponential patience time distributions, by analyzing the HJB equations, we find that the optimality of threshold con- trol does not generalize. We use a two-class setting to better understand the structure of the near optimal scheduling policy. We find that under some technical conditions, this policy has a novel U-shaped structure that prioritizes one class for low and high workloads. Further, our numerical studies show that compared with threshold control, our proposed policy is able to reduce abandonment rates of one class by 30% without affecting the abandonment rate of the other class. Using extensive numerical experiments, we also find that in such systems, the fixed queue ratio (FQR) control performs better than threshold control for a large class of patience time distributions. We feel there are several avenues for future work in this area. In the paper, we focus on a two-class setting to generate greater insight into the structure of near optimal scheduling policies for general patience distributions. A natural direction to extend this work would be to consider the generalk-class setting, and characterize the near optimal scheduling policies directly. Our numerical results indicate that the FQR control dominates threshold control, and may have performance that is close to the near-optimal policy for two-classes. Formalizing these observations analytically, and extending them tok-classes would also make for an interesting study. Finally, another avenue worth pursuing would be to study optimal scheduling when customer patience distributions are not a priori known, but estimated in an online manner while scheduling simultaneously. 83 References Afeche, P. and Ata, B. (2013), ‘Bayesian dynamic pricing in queueing systems with unknown delay cost characteristics’, Manufacturing and Service Operations Management 15(2), 292–304. Akan, M., Ata, B. s. and Olsen, T. (2012), ‘Congestion-based lead-time quotation for heterogenous cus- tomers with convex-concave delay costs: Optimality of a cost-balancing policy based on convex hull functions’, Operations Research 60(6), 1505–1519. Armony, M. and Ward, A. R. (2010), ‘Fair dynamic routing in large-scale heterogeneous-server systems’, Operations Research 58(3), 624–637. Ata, B. (2006), ‘Dynamic control of a multiclass queue with thin arrival streams’, Operations Research 54(5), 876–892. Ata, B. and Olsen, T. L. (2009), ‘Near-optimal dynamic lead-time quotation and scheduling under convex- concave customer delay costs’, Operations Research 57(3), 753–768. Ata, B. and Olsen, T. L. (2013), ‘Congestion-based leadtime quotation and pricing for revenue maximization with heterogeneous customers’, Queueing Systems 73(1), 35–78. Ata, B. and Shneorson, S. (2006), ‘Dynamic control of an M/M/1 service system with adjustable arrival and service rates’, Management Science 52(11), 1778–1791. Ata, B. and Tongarlak, M. H. (2013), ‘On scheduling a multiclass queue with abandonments under general delay costs’, Queueing Systems 74(1), 65–104. Atar, R., Giat, C. and Shimkin, N. (2010), ‘The cµ θ rule for many-server queues with abandonment’, Opera- tions Research 58(5), 1427–1439. Atar, R., Giat, C. and Shimkin, N. (2011), ‘On the asymptotic optimality of the cµ θ rule under ergodic cost’, Queueing Systems 67(2), 127–144. 84 Atar, R., Kaspi, H. and Shimkin, N. (2013), ‘Fluid limits for many-server systems with reneging under a priority policy’, Mathematics of Operations Research 39(3), 672–696. Atar, R., Mandelbaum, A. and Reiman, M. I. (2004), ‘Scheduling a multi class queue with many exponential servers: Asymptotic optimality in heavy traffic’, The Annals of Applied Probability 14(3), 1084–1134. Atar, R., Mandelbaum, A. and Shaikhet, G. (2009), ‘Simplified control problems for multiclass many-server queueing systems’, Mathematics of Operations Research 34(4), 795–812. Bassamboo, A. and Randhawa, R. S. (2010), ‘On the accuracy of fluid models for capacity sizing in queueing systems with impatient customers’, Operations Research 58(5), 1398–1413. Bassamboo, A. and Randhawa, R. S. (2015), ‘Scheduling homogeneous impatient customers’. Forthcoming in Management Science. Besbes, O. and Maglaras, C. (2010), ‘Revenue optimization for a make-to-order queue in an uncertain market environment’, Operations Research 57(6), 1438–1450. Besbes, O. and Zeevi, A. (2012), ‘Blind network revenue management’, Operations Research 60(6), 1537– 1550. Brown, L., Gans, N., Mandelbaum, A., Sakov, A., Shen, H., Zeltyn, S. and Zhao, L. (2005), ‘Statistical analysis of a telephone call center: A queueing-science perspective’, Journal of the Americal Statistical Association 100(469), 36–50. Browne, S. and Whitt, W. (1995), ‘Piecewise-linear diffusion processes’, Advances in queueing: Theory, methods, and open problems pp. 463–480. Cachon, G. and Feldman, P. (2011), ‘Pricing services subject to congestion: Charge per-use fees or sell subscriptions?’, Manufacturing and Service Operations Management 13(2), 244–260. Celik, S. and Maglaras, C. (2008), ‘Dynamic pricing and lead-time quotation for a multiclass make-to-order queue’, Management Science 54(6), 1132–1146. Chen, H. and Frank, M. Z. (2001), ‘State dependent pricing with a queue’, IIE Transactions 33(10), 847– 860. C ¸ il, E. B., Karaesmen, F. and ¨ Ormeci, E. L. (2011), ‘Dynamic pricing and scheduling in a multi-class single-server queueing system’, Queueing Systems 67(4), 305–331. Cox, D. and Smith, W. (1961), Queues, Methuen, London. 85 Dai, J. G. and He, S. (2010), ‘Customer abandonment in many-server queues’, Mathematics of Operations Research 35(2), 347–362. Dai, J. and He, S. (2011), ‘Queues in service systems: customer abandonment and diffusion approxima- tions’, Tutorials in Operations Research, INFORMS: Hanover, MD pp. 36–59. Dai, J. and He, S. (2013), ‘Many-server queues with customer abandonment: Numerical analysis of their diffusion model’, Stochastic Systems 3(1), 96–146. Down, D. G., Koole, G. and Lewis, M. (2011), ‘Dynamic control of a single server system with abandon- ments’, Queueing Systems 67(1), 63–90. Eren, S. and Maglaras, C. (2010), ‘Monopoly pricing with limited demand information’, Journal of Revenue and Pricing Management 9, 23–48. Farias, V . and Van Roy, B. (2010), ‘Dynamic pricing with a prior on market response’, Operations Research 58(1), 16–29. Garnett, O., Mandelbaum, A. and Reiman, M. (2002), ‘Designing a call center with impatient customers’, Manfacturing and Service Operations Management 4, 208–227. Ghamami, S. and Ward, A. R. (2013), ‘Dynamic scheduling of a two-server parallel server system with complete resource pooling and reneging in heavy traffic: Asymptotic optimality of a two-threshold policy’, Mathematics of Operations Research 38(4), 761–824. Ghosh, A. and Weerasinghe, A. (2007), ‘Optimal buffer size for a stochastic processing network in heavy traffic’, Queueing System 55, 147–159. Gurvich, I., Armony, M. and Mandelbaum, A. (2008), ‘Service-level differentiation in call centers with fully flexible servers’, Management Science 54(2), 279–294. Gurvich, I. and Whitt, W. (2009), ‘Queue-and-idleness-ratio controls in many-server service systems’, Math- ematics of Operations Research 34(2), 363–396. Gurvich, I. and Whitt, W. (2010), ‘Service-level differentiation in many-server service systems via queue- ratio routing’, Operations Research 58(2), 316–328. Halfin, S. and Whitt, W. (1981), ‘Heavy-traffic limits for queues with many exponential servers’, Operations Research 29, 567–588. Harrison, J. M. (1985), Brownian motion and stochastic flow systems, Wiley New York. 86 Harrison, J. M. (2001), ‘Brownian models of open processing networks: Canonical representation of work- load’, The Annals of Applied Probability 10(1), 75–103. Harrison, J. M. and Van Mieghem, J. A. (1997), ‘Dynamic control of brownian networks: state space col- lapse and equivalent workload formulations’, The Annals of Applied Probability pp. 747–771. Harrison, J. M. and Zeevi, A. (2004), ‘Dynamic scheduling of a multiclass queue in the Halfin-Whitt heavy traffic regime’, Operations Research 51, 243–257. Hartman, P. (1964), Ordinary differential equations, Wiley, New York. Haviv, M. and Randhawa, R. (2014), ‘Pricing in queues without demand information’, Manufacturing and Service Operations Management 16(3), 401–411. Huang, J., Zhang, H. and Zhang, J. (2014), ‘A unified approach to diffusion analysis of queues with general patience-time distributions’. Forthcoming in Mathematics of Operations Research. Kang, W., Ramanan, K. et al. (2010), ‘Fluid limits of many-server queues with reneging’, The Annals of Applied Probability 20(6), 2204–2260. Karatzas, I. and Shreve, S. E. (1988), Brownian Motion and Stochastic Calculus, V ol. 113 of Graduate Texts in Mathematics, Second edn, Springer-Verlag, New York. Katsuda, T. (2015), ‘General hazard-type scaling of abandonment time distribution for a G/Ph/n+GI queue in the halfin–whitt heavy-traffic regime’, Queueing Systems 80(1-2), 155–195. Kim, J. and Ward, A. (2013), ‘Dynamic scheduling of a GI/GI/1+GI queue with multiple customer classes’, Queueing Systems 75(2), 339–284. Kort, B. (1983), ‘Models and methods for evaluating customer acceptance of telephone connections’, Proc. IEEE GLOBECOM’83 pp. 706–714. Lee, C. and Weerasinghe, A. (2011), ‘Convergence of a queueing system in heavy traffic with general patience-time distributions’, Stochastic Processes and their Applications 121(11), 2507–2552. Long, Z. and Zhang, J. (2015), ‘Virtual allocation policies for many-server queues with abandonment’. Working Paper. Low, D. (1974), ‘Optimal dynamic pricing policies for an M/M/s queue’, Operations Research 22(3), 545– 561. Maglaras, C. (2006), ‘Revenue management for a multiclass single-server queue via a fluid model analysis’, Operations Research 54(5), 914–932. 87 Maglaras, C. and Zeevi, A. (2003), ‘Pricing and capacity sizing for systems with shared resources: Approx- imate solutions and scaling relations’, Management Science 49(8), 1018–1038. Mandelbaum, A. and Momcilovic, P. (2012), ‘Queues with many servers and impatient customers’, Mathe- matics of Operations Research 37(1), 41–65. Mandelbaum, A. and Stolyar, A. L. (2004), ‘Scheduling flexible servers with convex delay costs: Heavy- traffic optimality of the generalizedcµ -rule’, Operations Research 52(6), 836–855. Mandelbaum, A. and Zeltyn, S. (2013), ‘Data-stories about (im)patient customers in tele-queues’, Queueing Systems 75, 115–146. Mendelson, H. and Whang, S. (1990), ‘Optimal incentive-compatible priority pricing for the M/M/1 queue’, Operations Research 38(5), 870–883. Mieghem, J. V . (1995), ‘Dynamic scheduling with convex delay costs: The generalizedcµ rule’, Ann. Appl. Probab 5(3), 809–833. Naor, P. (1969), ‘The regulation of queue size by levying tolls’, Econometrica 37(1), 15–24. Netessine, S. (2006), ‘Dynamic pricing of inventory/capacity with infrequent price changes’, European Journal of Operations Research 174, 553–580. Oksendal, B. (2013), Stochastic differential equations: an introduction with applications, Springer Science & Business Media. Paschalidis, I. and Tsitsiklis, J. (2000), ‘Congestion-dependent pricing of network services’, IEEE/ACM Transactions on Networking 8(2), 171–184. Pinedo, M. L. (2012), Scheduling: theory, algorithms, and systems, Springer Science & Business Media. Plambeck, E. and Ward, A. (2008), ‘Optimal control of a high volume assemble-to-order system with lead- time quotation and expediting’, Queueing Systems 60(1), 1–69. Puterman, M. (2009), Markov decision processes: discrete stochastic dynamic programming, V ol. 414, John Wiley & Sons. Reed, J. and Tezcan, T. (2012), ‘Hazard rate scaling of the abandonment distribution for the GI/M/N+ GI queue in heavy traffic’, Operations Research 60(4), 981–995. Reed, J. and Ward, A. R. (2008), ‘Approximating the GI/GI/1+ GI queue with a nonlinear drift diffusion: Hazard rate scaling in heavy traffic’, Mathematics of Operations Research 33(3), 606–644. 88 Reiman, M. I. (1982), The heavy traffic diffusion approximation for sojourn times in jackson networks, in ‘Applied probabilitycomputer science: the interface’, Springer, pp. 409–421. Roth, A. E. (2006), Repugnance as a constraint on markets, Technical report, National Bureau of Economic Research. Rubino, M. and Ata, B. (2009), ‘Dynamic control of a make-to-order, parallel-server system with cancella- tions’, Operations Research 57(1), 94–108. Sigman, K. (2011), ‘Lecture notes on stochastic modeling I’. Available from Karl Sigman’s web site. Smith, W. (1956), ‘Various optimizers for single-stage production’, Naval Research Logistics Quarterly 3, 59–66. Tezcan, T. and Dai, J. G. (2008), ‘Optimal control of parallel server systems with many servers in heavy traffic’, Queueing Systems 59, 95–134. Tezcan, T. and Dai, J. G. (2010), ‘Dynamic control of N-sytems with many servers: Asymptotic optimality of a static priority policy in heavy traffic’, Operations Research 58(1), 94–110. Ward, A. R. and Glynn, P. W. (2003), ‘A diffusion approximation for a Markovian queue with reneging’, Queueing Systems 43(1/2), 103–128. Ward, A. R. and Glynn, P. W. (2005), ‘A diffusion approximation for a GI/GI/1 queue with balking or reneging’, Queueing Systems 50(4), 371–400. Weber, T. (2011), Optimal Control Theory with Applications in Economics, MIT press, Cambridge. Weerasinghe, A. (2014), ‘Diffusion approximations for G/M/n+ GI queues with state-dependent service rates’, Mathematics of Operations Research 39(1), 207–228. Whitt, W. (2002), Stochastic Process Limits, Springer, New York. Whitt, W. (2005), ‘Engineering solution of a basic call-center model’, Management Science 51(2), 221–235. Whitt, W. (2006), ‘Fluid models for multiserver queues with abandonments’, Operations Research 54(1), 37–54. Yoon, S. and Lewis, M. (2004), ‘Optimal pricing and admission control in a queueing system with periodi- cally varying parameters’, Queueing Systems 47, 177–199. Zeltyn, S. and Mandelbaum, A. (2005), ‘Call centers with impatient customers: many-server asymptotics of the m/m/n+ g queue’, Queueing Systems 51(3-4), 361–402. 89 Zhang, J. (2013), ‘Fluid models of many-server queues with abandonment’, Queueing Systems 73(2), 147– 193. The appendix is organized as follows. In Chapter A, we prove the technical results found in Chapter A. 90 Appendix A Technical Appendix to Chapter 1 Here is the outline of this chapter. In Appendix A.1, we formally prove that the solution to the exact MDP satisfiesp ⋆ n (q)+h q n is non-decreasing inq. Appendix A.2 provides the proofs of all the results in the body of the paper. Appendix A.3 contains proofs for technical lemmas that are used in Appendix A.2. A.1 The MDP solution We analyze the exact (non-asymptotic) optimization problem (1.3). Using the uniformization technique (Puterman (2009), p. 562), we can derive the following set of equations from our model: γ n = sup p∈F pλ ¯ F (p)−λ ¯ F (p)y n (0) (A.1) γ n = sup p∈F pλ ¯ F p+h i n −λ ¯ F p+h i n y n (i) +y n (i−1) fori≥ 1, (A.2) whereF is a set of feasible prices, γ n is a “guess” for R ⋆ n n , and y n is an associated value function. The following lemma formalizes the “non-decreasing” property that is mentioned in Section 1.4.3. Lemma A.1.1. IfF = [0,p max ] for somep max ∈ (¯ p,∞) andf(q)> 0 for allq≥ 0, there exists a unique pair ofγ n andy n that jointly solves (A.1) and (A.2). Furthermore, R ⋆ n = nγ n andλ ¯ F p ⋆ n (q)+h q n is non-increasing inq, i.e.,p ⋆ n (q)+h q n is non-decreasing inq. Note that (A.1) and (A.2) are of the same form as (12) and (13) of Ata and Shneorson (2006) with a caveat that (A.2) has an infinite number of equations whereas (13) of Ata and Shneorson (2006) has a finite number of equations. Indeed, the proof of Lemma A.1.1 is quite similar to that of its counterpart in Ata and Shneorson (2006) (a combination between Proposition 1 and Corollary 1 therein), and so is omitted here. 91 A.2 Proofs of Main Results Notation. In the following proofs, we will use the following convention: for any two sequences of real numbers{a n } n≥1 and{b n } n≥1 , we use the notation a n ∼ b n , a n & b n , a n ≫ b n , and a n = Θ(b n ) to represent lim n→∞ a n b n = 1, liminf n→∞ a n b n ≥ 1, lim n→∞ b n a n = 0, and0< liminf n→∞ |a n | |b n | ≤ limsup n→∞ |a n | |b n | <∞, respectively. Also, we usex↓a fora∈R to denote “asx approaches toa from the right”. Finally, it is useful to setP TP as a class of sequences of two-price policies defined in Section 1.4.1. Proof of Proposition 1.3.1: Since part (b) of the proposition is subsumed by the results in Proposition 1.3.2, we only prove part (a) here. Forp≥ 0, letr(p) = pλ ¯ F (p). Let ¯ F −1 be the inverse of ¯ F so thatp = ¯ F −1 (q) for eachq ∈ [0,1]. Let ˜ r(q) =λq ¯ F −1 (q). Because the distribution has an increasing hazard rate function, it is straightforward to check ˜ r ′′ (q)< 0, so that ˜ r is concave inq∈ [0,1] and it is maximized atq = ¯ F (p ⋆ ). Observe that for any pricing policyp n , we have R n (p n ) =nλE p n (Q n ) ¯ F p n (Q n )+h Q n n ≤nλE p n (Q n )+h Q n n ¯ F p n (Q n )+h Q n n (a) ≤ n˜ r E ¯ F p n (Q n )+h Q n n (A.3) where (a) follows by Jensen’s inequality. We first consider the case when ¯ F −1 1 λ ≤ p ⋆ so that ¯ R ⋆ n = nλp ⋆ ¯ F (p ⋆ ). In this case we have R n (p n )≤n˜ r E ¯ F p n (Q n )+h Q n n ≤ ¯ R ⋆ n as desired because ˜ r(q)≤λp ⋆ ¯ F (p ⋆ ) for anyq∈ [0,1]. On the other hand, supposep ⋆ < ¯ F −1 1 λ , so that ¯ R ⋆ n = ¯ pn =n˜ r 1 λ (recall thatλ> 1 becauseλ ¯ F (¯ p) = 1). Because ˜ r(q) is increasing inq∈ 0, ¯ F (p ⋆ ) 92 and 1 λ < ¯ F (p ⋆ ), we have ˜ r(q) ≤ ˜ r 1 λ forq ∈ 0, 1 λ . Observe thatnλE h ¯ F p n (Q n )+h Qn n i is the expected steady-state arrival rate and therefore it cannot exceed the service capacityn, i.e, E ¯ F p n (Q n )+h Q n n ≤ 1 λ . (A.4) Putting (A.3) and (A.4) together, we haveR n (p n )≤n˜ r 1 λ = ¯ R ⋆ n . Proof of Proposition 1.3.2: First, we show that for any static pricep n,S (q) = ¯ p+θ n,S forq≥ 0, whereθ n,S is some constant such that θ n,S ≪ 1, liminf n→∞ ¯ R ⋆ n −R n (p n,S ) √ n > 0. (A.5) We next show that there exists a sequence{p n,S } n≥1 under which limsup n→∞ ¯ R ⋆ n −R n (p n,S ) √ n <∞. (A.6) Then, the first part of the proposition is established by combining (A.5) with (A.6). The second part is established using a portion of the argument used to establish (A.5) because for anyθ n,S ≫ 1 √ n we prove liminf n→∞ ¯ R ⋆ n −R n (p n,S ) √ n =∞. (A.7) The following lemma is useful in establishing (A.5) - (A.7) (and many results in the sequel). Lemma A.2.1. For any sequence of prices{p n } inP orP TP , we have ¯ R ⋆ n −R n (p n )&φnE h θ n (Q n ) 2 i +hE[Q n ], ¯ R ⋆ n −R n (p n )∼αnE[θ n (Q n )]+βnE h θ n (Q n ) 2 i +γE[Q n ]. To show (A.5), first, consider the case 1 √ n ≪θ n,S andθ n,S > 0. By Lemma A.2.1, we have ¯ R ⋆ n −R n (p n,S )∼αnθ n,S +βnθ 2 n,S +γE[Q n ] (a) ∼ αnθ n,S +γE[Q n ]≥αnθ n,S , (A.8) 93 where(a) follows becauseθ n,S ≪ 1. Since 1 √ n ≪θ n,S andθ n,S > 0, (A.8) implies (A.5) by (A.7). Next, consider the case 1 √ n ≪θ n,S andθ n,S < 0. Let ˆ τ n := n|θ n,S| h so thatθ n,S +h ˆ τn n = 0. Consider anM/M/1/ˆ τ n system in which the static price is ¯ p and customers are only price-sensitive. LetQ n,lb be the steady-state queue-length of this system. It is obvious thatQ n,lb ≤ Q n a.s. andE[Q n,lb ] ≤ E[Q n ]. Also by the expected steady-state queue-length formula for anM/M/1 system with a finite buffer, we can easily obtain that √ n≪E[Q n,lb ], so that (A.7) is established by Lemma A.2.1. For the cases ofθ n,S = O 1 √ n , we can prove that √ n = O(E[Q n ]), so that (A.5) holds, by con- sidering anM/M/1/⌊ √ n⌋ system in which the static price is ¯ p+max{0,θ n,S }+h 1 √ n and customers are only price sensitive. By combining this paragraph and the previous paragraph, we establish part (b) of this proposition. Finally, to show that there exists a sequence of static pricing policies under which (A.6) is achieved, setθ n,S = 0. Then, by Lemma A.2.1, we have ¯ R ⋆ n −R n (p n,S ) ∼ γE[Q n ]. So, it suffices to show that E[Q n ] = O( √ n). This can be established by considering anM/M/1 queue with a two-price policy of ¯ p+θ n,TP , whereθ n,TP (q) is 0 ifq≤ √ n andh 1 √ n otherwise and customers are only price-sensitive. Proof of Proposition 1.4.1: DefineP ⋆ TP to be a class of sequences of two-price policies defined in (1.10) such that θ − n > 0, θ + n > 0, 1 √ n ≪ min θ − n ,θ + n , and, τ n ≪ √ n. (A.9) We will use the following three lemmas to prove the proposition. Lemma A.2.2. For any{p n,TP } n≥1 ∈P TP \P ⋆ TP , we have liminf n→∞ ¯ R ⋆ n −R n (p n,TP ) (nlogn) 1/3 =∞. Lemma A.2.3. For any{p n,TP } n≥1 ∈P ⋆ TP , we have ¯ R ⋆ n −R n (p n,TP )∼L n θ − n ,θ + n ,τ n , 94 where L n θ − ,θ + ,τ :=αn −θ − D 1 (θ − ,τ)+θ + D 2 (θ + ) D 1 (θ − ,τ)+D 2 (θ + ) +βn (θ − ) 2 D 1 (θ − ,τ)+(θ + ) 2 D 2 (θ + ) D 1 (θ − ,τ)+D 2 (θ + ) +h λ ¯ F (¯ p−θ − ) −τ (D 1 (θ − ,τ)+D 2 (θ + )) λ ¯ F (¯ p−θ − )−1 2 +τ− 1 λ ¯ F (¯ p−θ − )−1 +D 2 θ + ! (A.10) for D 1 (θ,τ) := 1− λ ¯ F (¯ p−θ) −τ−1 λ ¯ F (¯ p−θ)−1 andD 2 (θ) := 1 1−λ ¯ F (¯ p+θ) . Lemma A.2.4. For any{p n,TP } n≥1 ∈P ⋆ TP , we have liminf n→∞ L(θ − n ,θ + n ,τ n ) (nlogn) 1/3 ≥ Π TP . Furthermore, if we let θ ⋆,− n := logn (nlogn) 1/3 π,θ ⋆,+ n := 3 (nlogn) 1/3 π,τ ⋆ n := (nlogn) 1/3 3λf (¯ p)π , then limsup n→∞ L θ ⋆,− n ,θ ⋆,+ n ,τ ⋆ n (nlogn) 1/3 = Π TP . Given Lemma A.2.2-A.2.4, we now prove the proposition. Lemma A.2.2 proves that the sequence of asymptotically optimal two-price policies lies inP ⋆ TP . Lemma A.2.3 provides a tractable characterization of the revenue gap under a sequence inP ⋆ TP . This characterization is used in Lemma A.2.4 to show that the sequence defined in part (b) of the proposition is asymptotically optimal and its performance and the resulting expected queue-length are as given in part (a) and (c), respectively, of the proposition. Proof of Proposition 1.4.2: The following lemma is useful in the proof. Lemma A.2.5. For any sequence {p n } n≥1 ∈ P such that lim q↓0 θ(q) < 0, we have min{ √ n,τ n } = O(E[Q n ]). 95 The case when lim q↓0 θ(q) < 0. Suppose that θ(q) ≤ 0 for q ≥ 0. Then, by considering an M/M/1/ √ n system with the arrival rate ofnλ ¯ F ¯ p+h 1 √ n and the service rate ofn, we can show that √ n = O(E[Q n ]). Therefore, using Lemma A.2.1, the result follows. To proceed, consider the following three cases:s + n τ n ≪ 1, s + n τ n = Θ(1), ands + n τ n ≫ 1. The case whens + n τ n = Θ(1). We already argued that there must existǫ> 0 such thatθ(1+ǫ)> 0. Observe that E h θ n (Q n ) 2 i ≥ s + n θ(1+ǫ) 2 P(Q n ≥ (1+ǫ)τ n ) E[Q n ]≥ (1+ǫ)τ n P(Q n ≥ (1+ǫ)τ n ). We will next prove liminf n→∞ P(Q n ≥ (1+ǫ)τ n )> 0. (A.11) The proof of the result for this case will then follow by applying Lemma A.2.1 because n s + n 2 +τ n =O φnE h θ n (Q n ) 2 i +hE[Q n ] andn 1/3 =O n s + n 2 +τ n , whens + n τ n = Θ(1). To show (A.11), fixN > 1+ǫ and letQ n,lb be the steady-state queue-length of an M/M/1/Nτ n system with the arrival rate ofnλ ¯ F ¯ p+θ n (Nτ n )+h Nτn n and the service rate ofn. Then, it is straightforward to check P(Q n ≥ (1+ǫ)τ n )≥P(Q n,lb ≥ (1+ǫ)τ n ) and liminf n→∞ P(Q n,lb ≥ (1+ǫ)τ n )> 0, which establishes (A.11), and consequently the result for this case. The case whens + n τ n ≫ 1. Supposelim q↓1 θ(q)> 0. Observe that E h θ n (Q n ) 2 i ≥ s + n θ 1+ 1 τ n 2 P(Q n ≥τ n +1), Suppose we establish that 1 s + n τ n =O(P(Q n ≥τ n +1)). (A.12) 96 Then, becauseθ 1+ 1 τn ≥ lim x↓1 θ(x)> 0 and 1 τn =o(s + n ), using (A.12), we have 1 (τ n ) 2 =O E h θ n (Q n ) 2 i . (A.13) By Lemma A.2.1, Lemma A.2.5, and (A.13), the result is then proved for this case. So, we focus on proving (A.12). To do so, we fix N > 1 and let Q n,lb be the steady-state queue-length of an M/M/1/Nτ n system with the service rate ofn and the arrival rate ofnexp −λf (¯ p)ch τn n , ifq ≤ τ n ,, andnexp −λf (¯ p)c θ n (Nτ n )+h Nτn n , otherwise. for some positive constantc> 0 that satisfies exp −λf (¯ p)ch τ n n ≤λ ¯ F (¯ p) ¯ F ¯ p+h τ n n , and exp −λf (¯ p)c θ n (Nτ n )+h Nτ n n ≤λ ¯ F ¯ p+θ n (Nτ n )+h Nτ n n for sufficiently largen. Then, it is straightforward to check that P(Q n ≥τ n +1)≥P(Q n,lb ≥τ n +1) and 1 s + n τ n =O(P(Q n,lb ≥τ n +1)), establishing (A.12). We now consider the case when lim q↓1 θ(q) = 0. If there exists ζ > 0 such that θ(1+ζ) = 0 but lim q↓1+ζ θ(q) > 0, then we can repeat the same arguments in the previous case to complete the proof of the proposition. Therefore, we only need to consider the case θ(q) > 0 for q > q 0 where q 0 := sup{q≥ 1 :θ(q) = 0} while lim q↓q 0 θ(q) = 0. Observe that we only need to consider the case when q 0 <∞ because otherwise, we already know that √ n =O(E[Q n ]). Letν ≥ 0 be some constant such that q 0 = 1+ν. To proceed, define∇ (1) and∇ (k) fork = 2,3,... , as follows: ∇ (1) (θ)(q) = lim x↓q θ(x)−θ(q) x−q , ∇ (k) (θ)(q) = lim x↓q ∇ k−1 (θ)(x)−∇ k−1 (θ)(q) x−q . Then, we must havek θ <∞ wherek θ := inf k≥ 1 :∇ (k) (θ)(1+ν)> 0 . Let ˆ τ n,1 :=τ n 1+ν + 1 (s + n τn) 1/k θ , By the definition ofk θ and∇ (k θ ) , we have θ n (ˆ τ n,1 ) = Θ 1 τ n = Θ 1 ˆ τ n,1 (A.14) 97 where the last inequality follows becauses + n τ n ≫ 1. Letd n = τ k θ n s + n 1 k θ +1 . Then, becauses + n τ n ≫ 1, we have d n ≪τ n and 1 s n ≪d n . (A.15) Define ˆ τ n,2 :=τ n 1+ν + dn τn . Then, by (A.15) and by the definition ofk θ and∇ (k θ ) , we have ˆ τ n,1 < ˆ τ n,2 for largen and ˆ τ n,2 − ˆ τ n,1 = Θ(d n ) andθ n (ˆ τ n,2 ) = Θ 1 d n . (A.16) Define ˆ τ n,3 :=τ n 1+ν + 2dn τn . Then, we also have ˆ τ n,2 < ˆ τ n,3 for largen and ˆ τ n,3 − ˆ τ n,2 = Θ(d n ) andθ n (ˆ τ n,3 ) = Θ 1 d n (A.17) using the same reasoning as in (A.16). LetQ n,lb be the steady-state queue-length of anM/M/1/ˆ τ n,3 system for which the service rate isn and the arrival rate equalsnexp −λf (¯ p)c θ n (ˆ τ n,i )+h ˆ τ n,i n , when there are q ∈ ˆ τ n,i−1 ,ˆ τ n,i customers are in the system, for i = 1,2,3 and ˆ τ n,0 = −1 where c > 0 is some constant that satisfies exp −λf (¯ p)c θ n (ˆ τ n,i )+h ˆ τ n,i n ≤λ ¯ F ¯ p+θ n (ˆ τ n,i )+h ˆ τ n,i n fori = 1,2,3 and for largen. Then, it is straightforward to check thatP(Q n ≥ ˆ τ n,1 ) ≥ P(Q n,lb ≥ ˆ τ n,1 ) and by (A.14), (A.16), and (A.17), we can derive that P ˆ Q n,2 ≥ ˆ τ n,1 = Θ d n τ n . (A.18) Therefore, by (A.18), we have E h θ n (Q n ) 2 i ≥θ n (ˆ τ n,1 ) 2 P(Q n,lb ≥ ˆ τ n,1 ) = Θ 1 τ n d n . Since 1 τn =o 1 dn , we have 1 τ 2 n =O E h θ n (Q n ) 2 i . So, applying Lemma A.2.1, the result is established for this case. 98 The case whens + n τ n ≪ 1. In this case, ifs + n =O 1 √ n , we can prove that √ n =O(E[Q n ]), which establishes the result by applying Lemma A.2.1. Let us focus on the case 1 √ n ≪s + n . Observe that E h θ n (Q n ) 2 i ≥θ n 1 s + n 2 P Q n ≥ 1 s + n andE[Q n ]≥ 1 s + n P Q n ≥ 1 s + n . Further, there exists ǫ > 0 such that θ(1+ǫ) > 0 and for sufficiently large n, 1 s + n τn ≥ 1 +ε, implying θ n 1 s + n 2 ≥ (s + n ) 2 θ(1+ε). We now establish liminf n→∞ P Q n ≥ 1 s + n > 0 because then the proof is complete by Lemma A.2.1. For this, fix N > 2 and let Q n,lb be the steady-state queue-length of an M/M/1/ N s + n queue with the service rate ofn and the arrival rate ofnexp(−λf (¯ p)cs + n ) for some constant c> 0 that satisfies exp −λf (¯ p)cs + n <λ ¯ F ¯ p+θ n 1 s + n +h 1 ns + n . Such ac must exist becauseθ n 1 s + n +h 1 ns + n = Θ(s + n ) andθ is bounded. It is straightforward to check that P Q n ≥ 1 s + n ≥P Q n,lb ≥ 1 s + n and liminf n→∞ P Q n,lb ≥ 1 s + n > 0. Hence, the result holds for this case. The case when lim q↓0 θ(q)≥ 0. Recall that in the case when lim q↓0 θ(q) < 0, our analysis was focused onθ(q) forq≥ 1. In the current case, the same analysis can be applied with0 replacing the role of1 in the previous case. Proof of Proposition 1.4.3: To begin with, it is useful to defineψ :=−α/(λf(¯ p)) and to note thatψ< 0 andφ =β−f ′ (¯ p)α/(2f(¯ p)) forα andβ defined in (1.6) andφ defined in (1.11). Existence of the solution of the HJB equation. We first prove the existence of a pair(g ⋆ ,κ ⋆ ) that solves (1.21). Similar to Chapter 2, we consider the following family of first order ODEs parameterized byκ≥ 0: ng ′ κ (q)+hq−n (λf (¯ p)) 2 4φ (g κ (q)+ψ) 2 =κ, withg κ (0) = 0. (A.19) 99 With the same argument as in the proof of Lemma reflemma:IVP, we know that for each κ, there exist a unique g κ that solves (A.19). Within this family, we will show that there exists a unique κ ⋆ such that (g κ ⋆,κ ⋆ ) solves (1.21). To that end, for eachκ, letw κ :=g κ +ψ and define q ∞,κ := inf q≥ 0 : lim x↑q w κ (x) =∞ . (A.20) We also define setsL andU that bisect non-negative real numbers: L :={κ≥ 0 :S κ 6=∅} andU :={κ≥ 0 :S κ =∅}, where S κ := q∈ [0,q ∞,κ ] :w ′ κ (q)≤ 0 . Lemma A.2.6. For anyκ 1 <κ 2 , we havew κ 1 (q)<w κ 2 (q) forq∈ [0,min{q ∞,κ 1 ,q ∞,κ 2 }]. Lemma A.2.7. BothL andU are non-empty. Lemma A.2.8. Ifκ∈L, thenw κ (q) is strictly quasi-concave inq andlim q→∞ w κ (q) =−∞. Lemma A.2.9. w κ (q) is jointly continuous inκ≥ 0 andq≥ 0. Lemma A.2.10. Letκ ⋆ := sup L. Then,κ ⋆ <∞ andκ ⋆ ∈U, i.e.,w ′ κ ⋆(q)> 0 forq≥ 0. Lemma A.2.11. For anyκ∈L,w k (q)≤ 2 √ φ λf(¯ p) q κ ⋆ n + (λf(¯ p)) 2 4φ ψ 2 +h q n . To complete the proof, suppose noww κ ⋆ does not satisfy the growth condition so that liminf q→∞ w κ ⋆ (q) √ q =∞. Then, there must exist ˆ q such that w κ ⋆ (ˆ q)≥ 2 √ φ λf (¯ p) s κ ⋆ n + (λf (¯ p)) 2 4φ ψ 2 +h ˆ q n +ǫ, which contradicts Lemma A.2.9 because for anyκ<κ ⋆ , we have w κ (ˆ q)≤ 2 √ φ λf (¯ p) s κ ⋆ n + (λf (¯ p)) 2 4φ ψ 2 +h ˆ q n by Lemma A.2.11. Thus, the existence result follows by settingg ⋆ =w κ ⋆−ψ. 100 Optimality of ˆ Δ ⋆ n . It remains to show that for anyΔ n (z) that is decreasing inz, we have liminf t→∞ 1 t E Z t 0 nψΔ n (Z n (s))+n φ (λf (¯ p)) 2 Δ n (Z n (s)) 2 +hZ n (s) ds ≥κ ⋆ (A.21) and the inequality is replaced by the equality if we replaceΔ n by ˆ Δ ⋆ n . Ifinf z≥0 Δ n (z)≥ 0, then E nψΔ n (Z n (s))+n φ (λf (¯ p)) 2 Δ n (Z n (s)) 2 +hZ n (s) ≥nψΔ n (0)+E[hZ n (s)] becauseψ < 0 andφ> 0. However, in this case liminf t→∞ 1 t E h R t 0 Z n (s)ds i =∞ becauseZ n is lower bounded by a reflected Brownian motion with zero drift. So, (A.21) follows. Suppose now that inf z≥0 Δ n (z) := δ < 0. In this case, there must exist ˆ z <∞ such that Δ n (z)≤ 0 for allz≥ ˆ z. To proceed, letv ⋆ (q) := R q 0 g ⋆ (x)dx. Then, by the properties ofg ⋆ stated in the proposition, we havev ⋆ (0) = 0 andv ⋆ (q)≤Cq 3/2 . Furthermore, it is straightforward to check thatv ⋆ satisfies nv ′′ (q)+hq +ninf Δn v ′ (q)+ψ Δ n + φ (λf (¯ p)) 2 Δ 2 n =κ ⋆ . (A.22) Recall thatZ n is the (weak) solution of Z n (t) =n Z t 0 Δ n (Z(s))ds+ √ 2nB(t)+nL n (t), whereB is a standard Brownian motion andL n is an increasing process such that R t 0 Z n (u)dL n (u) = 0 for allt≥ 0 a.s. By Ito’s lemma, we have v ⋆ (Z n (t))−v ⋆ (Z n (0)) = Z t 0 nv ⋆′ (Z n (s))Δ n (Z n (s))+nv ⋆′′ (Z n (s)) ds + √ 2n Z t 0 v ⋆′ (Z n (s))dB(s)+n Z t 0 v ⋆′ (Z n (s))dL n (s). (A.23) By adding Z t 0 nψΔ n (Z n (s))+n φ (λf (¯ p)) 2 Δ n (Z n (s)) 2 +hZ n (s) ds on both hand sides of (A.23), we obtain 101 E v ⋆ (Z n (t))−v ⋆ (Z n (0)) t + 1 t E Z t 0 nψΔ n (Z n (s))+n φ (λf (¯ p)) 2 Δ n (Z n (s)) 2 +hZ n (s) ds = 1 t E Z t 0 nv ⋆′′ (Z n (s))+hZ n (s)+ v ⋆′ (Z n (s))+ψ nΔ n (Z n (s))+n φ (λf (¯ p)) 2 Δ n (Z n (s)) 2 ds , where the last equality follows because R t 0 v ⋆′ (Z n (s))dB(s) is a martingale andv ⋆′ (Z n (s))dL n (s) = 0 for alls≥ 0 a.s. by the conditionv ⋆′ (0) = 0 andL n only increases whenZ n (s) = 0. Asv ⋆ andκ ⋆ jointly solve (A.22), we have 1 t E Z t 0 nψΔ n (Z n (s))+n φ (λf (¯ p)) 2 Δ n (Z n (s)) 2 +hZ n (s) ds ≥κ ⋆ −E v ⋆ (Z n (t))−v ⋆ (Z n (0)) t for anyt≥ 0 andΔ n such thatinf z≥0 Δ n (z) =δ< 0. Becausev ⋆ (q)≤Cq 3/2 , we have E v ⋆ (Z n (t)) t ≤E " Z n (t) 3/2 t # ≤ 1 t 1/3 E " Z n (t) 2 t #! 3/4 (A.24) by Jensen’s inequality. By Ito’s lemma, we have E h Z n (t) 2 i =E h Z n (0) 2 i +2t+E Z t 0 2Δ(Z n (s))Z n (s)ds ≤E h Z n (0) 2 i +2t+E Z t 0 2Δ(Z n (s))Z n (s)1 {Zn(s)≤ˆ z} ds ≤E h Z n (0) 2 i +2 1+Δ(ˆ z(0)) t, implying that limsup t→∞ E " Z n (t) 2 t # <∞. Therefore, by (A.24), we have limsup t→∞ E v ⋆ (Z n (t)) t = 0, (it is without loss of generality to assumeE h Z(0) 2 i <∞) so that (A.21) follows. To prove that ˆ R ⋆ n =κ ⋆ , letZ ⋆ n be the (weak) solution of Z ⋆ (t) =n Z t 0 ˆ Δ ⋆ n (Z ⋆ n (s))ds+ √ 2nB ⋆ (t)+nL ⋆ n (t), 102 whereB ⋆ is a standard Brownian motion that is independent ofB andL ⋆ n is an increasing process such that Z ⋆ n (s)dL ⋆ n (s) = 0 for alls≥ 0 a.s. Observe that becauseg ⋆ (z) is increasing inz and lim z→∞ g ⋆ (z) = ∞, ˆ Δ ⋆ n (z) is decreasing inz and lim z→∞ ˆ Δ ⋆ n (z) =−∞. Therefore, we can repeat the previous arguments to obtain 1 t E Z t 0 nψ ˆ Δ ⋆ n (Z ⋆ n (s))+n φ (λf (¯ p)) 2 ˆ Δ ⋆ n (Z ⋆ n (s)) 2 +hZ ⋆ n (s) ds =κ ⋆ −E v ⋆ (Z ⋆ n (t))−v ⋆ (Z ⋆ n (0)) t and limsup t→∞ E v ⋆ (Z ⋆ n (t))−v ⋆ (Z ⋆ n (0)) t = 0. Therefore, we have liminf t→∞ 1 t E Z t 0 nψ ˆ Δ ⋆ n (Z ⋆ n (s))+n φ (λf (¯ p)) 2 ˆ Δ ⋆ n (Z ⋆ n (s)) 2 +hZ ⋆ n (s) ds =κ ⋆ , proving the optimality of ˆ Δ ⋆ n . Proof of Proposition 1.5.1: It is straightforward to check that the objective value under a feasible solution that has strictly positive lead- time can be improved by decreasing that lead-time to zero. So, we only focus on the case where zero lead time is quoted. Observe that for eache> 0 pλ ¯ F (p)−e λ ¯ F (p)−1 + = pλ ¯ F (p)−e λ ¯ F (p)−1 if p≤ ¯ p, pλ ¯ F (p) otherwise. Recall that both pλ ¯ F (p)−e λ ¯ F (p)−1 and pλ ¯ F (p) are quasi-concave because F has an increasing hazard rate function. Suppose ¯ p e ≤ ¯ p. Then, for anyp≤ ¯ p, by the definition of ¯ p e , it is straightforward to checkpλ ¯ F (p)−e λ ¯ F (p)−1 is maximized atp = ¯ p e . Furthermore, for anyp≥ ¯ p,pλ ¯ F (p) is maximized atp = ¯ p. Therefore, if ¯ p e ≤ ¯ p, pλ ¯ F (p)−e λ ¯ F (p)−1 + is maximized atp = ¯ p e and it is optimal to charge ¯ p e in the fluid optimization. 103 On the other hand, suppose ¯ p e > ¯ p. Then, forp≤ ¯ p,pλ ¯ F (p)−e λ ¯ F (p)−1 is maximized atp = ¯ p. Therefore, if ¯ p e > ¯ p,pλ ¯ F (p)−e λ ¯ F (p)−1 + is maximized atp = ¯ p and it is optimal to charge ¯ p in the fluid optimization. Finally, we prove that there exists ¯ e such that ¯ p e ≤ ¯ p if and only ife ≤ ¯ e. However, this is obvious because ¯ p e is increasing ine≥ 0 and ¯ p e =p ⋆ < ¯ p whene = 0 and ¯ p e =∞ whene =∞. A.3 Proofs of lemmas Proof of Lemma A.2.1: By the Taylor series expansion on ¯ F around ¯ p, we have ¯ R ⋆ n −R n (p n ) =αnE[δ n (Q n )]+φnE h θ n (Q n ) 2 i +hE[Q n ]+nE[ǫ 1,n +λf (¯ p)ǫ 2,n ], ¯ R ⋆ n −R n (p n ) =nαE[θ n (Q n )]+nβE h θ n (Q n ) 2 i +γE[Q n ] +nλE ¯ pǫ 1,n +θ n (Q n ) h Q n n + f ′ (¯ p) 2f (¯ p) θ n (Q n ) 2 +ǫ 1,n , where δ n (Q n ) := 1 f (¯ p) ¯ F (¯ p)− ¯ F ¯ p+θ n (Q n )+h Q n n , ǫ 1,n := 1 f (¯ p) f ′ (P n )−f ′ (¯ p) 2 θ n (Q n ) 2 + f ′ (P n ) 2 2θ n (Q n )h Q n n + h Q n n 2 !! , ǫ 2,n :=h Q n θ n (Q n ) n + f ′ (¯ p) 2f (¯ p) θ n (Q n ) 3 +ǫ 1,n θ n (Q n ). for some random variableP n ∈ h ¯ p,¯ p+θ n (Q n )+h Qn n i a.s.E[δ n (Q n )]≥ 0 is established in the proof of Proposition 1.3.1 (see the argument before (A.4)). Because|f ′ (P n )−f ′ (¯ p)| =O θ n (Q n )+h Qn n a.s. and sup q≥0 |θ n (q)|≪ 1, it is straightforward to check that both nE[|ǫ 1,n +λf (¯ p)ǫ 2,n |] andnλE ¯ pǫ 1,n +θ n (Q n ) h Q n n + f ′ (¯ p) 2f (¯ p) θ n (Q n ) 2 +ǫ 1,n 104 are O E n|θ n (Q n )| 3 +|θ n (Q n )|Q n + Q 2 n n . Because sup q≥0 |θ n (q)| ≪ 1 for any{p n } inP orP TP , we readily haveE[|θ n (Q n )|Q n ] ≪ E[Q n ] and E h |θ n (Q n )| 3 i ≪E h θ n (Q n ) 2 i . Therefore, it suffices to establish E Q 2 n n ≪E[Q n ] (A.25) The proof for when{p n } ∈ P. Define ˆ τ n := inf q≥ 0 :θ n (q)+h q n ≥ 0 . LetQ n,ub be the steady- state queue-length of anM/M/1 queue for which the service rate isn and the arrival rate when there are q customers in the system isnλ ¯ F (¯ p+θ n (0)) ifq ≤ ˆ τ n + √ n andnλ ¯ F ¯ p+h 1 √ n otherwise. Because the arrival rate at each queue-length in this newly introduced system is higher than the arrival rate in our original system, we have Q n ≤ Q n,ub a.s. On the other hand, let Q n,lb be the steady-state queue-length of an M/M/1/(ˆ τ n + √ n) queue for which the service rate is n and the arrival rate when there are q customers in the system isnλ ¯ F (¯ p) ifq ≤ ˆ τ n andnλ ¯ F ¯ p+h 1 √ n otherwise. Because the arrival rate at each queue-length in this system is lower than the arrival rate in our original system, we haveQ n,lb ≤ Q n a.s. Furthermore, it is straightforward to calculate E Q 2 n,ub n and E[Q n,lb ] and check that E Q 2 n,ub n ≪ E[Q n,lb ], establishing (A.25). The proof for when{p n }∈P TP . For a{p n }∈P TP , if−θ − n ≤θ + n (see Section 1.4.1 to see how a two- price policy is characterized byθ − n ,θ + n , andτ n ), then{p n }∈P. So, we only need to analyze{p n }∈P TP for which−θ − n >θ + n . Among many sub-cases under the case of−θ − n >θ + n , we only present the proof of a sub-case where θ − n < 0, 1 √ n ≪ θ − n , θ + n ≤ 0, 1 √ n =O θ + n , and √ n =O(τ n ) as other sub-cases can be handled with similar arguments. 105 Consider anM/M/1 queue with the service rate ofn and the arrival rate when there areq customers in the system given by nλ ¯ F (¯ p+|θ − n |) if q≤τ n nλ ¯ F (¯ p−|θ + n |) elseif q≤ √ n+n|θ + n| h nλ ¯ F ¯ p+ 1 √ n otherwise and call its steady-state queue-lengthQ n,ub . Also consider anM/M/1/ √ n+n|θ + n| h queue with the service rate ofn and the arrival rate when there areq customers are in the system given by nλ ¯ F ¯ p+|θ − n |+h τn n if q≤τ n nλ ¯ F ¯ p− 1 2 |θ + n | elseif q≤ n|θ + n| 2h nλ ¯ F ¯ p+ 1 √ n otherwise and call its steady-state queue-lengthQ n,lb . It is straightforward to check thatQ n,lb ≤Q n ≤Q n,ub a.s. and E Q 2 n,ub n ≪E[Q n,lb ], which establish (A.25). Proof of Lemma A.2.2: To prove this lemma, we need to consider many cases based on which condition is violated in (A.9). Still, each case can be handled by proving one of the following three claims and then applying Lemma A.2.1: E[θ n (Q n )]> 0, E h θ n (Q n ) 2 i ≪E[θ n (Q n )], and liminf n→∞ nE[θ n (Q n )] (nlogn) 1/3 =∞, (A.26) liminf n→∞ nE h θ n (Q n ) 2 i (nlogn) 1/3 =∞, (A.27) liminf n→∞ E[Q n ] (nlogn) 1/3 =∞, (A.28) The most involved analysis occurs when θ − n < 0, 1 √ n ≪ θ − n , θ + n ≤ 0, 1 √ n =O θ + n , and √ n =O(τ n ) (A.29) as it requires to use all (A.26) - (A.28), whereas (A.27) is not used for all other cases. So, we only consider the case of (A.29) here. 106 Suppose lim n→∞ P(Q n >τ n ) = 0 and|θ − n | = o(|θ + n |). Observe first that because √ n =O(τ n ) and we wantE[Q n ] =O(nlogn) 1/3 by Lemma A.2.1, it is necessary that P(Q n >τ n )≤ E[Q n ] τ n +1 =O (logn) 1/3 n 1/6 ! . (A.30) To proceed, consider anM/M/1/(nlogn) 1/3 system with a static pricing of ¯ p +|θ − n | +h (nlogn) 1/3 n . If we letQ n,0 be the steady-state queue-length of this newly introduced system, it can be easily checked that Q n,0 ≤Q n a.s. because the joining probability of customer is less in the new system for any queue-length less than or equal to(nlogn) 1/3 . By using the formula for the expected queue-length of anM/M/1 system with a finite buffer, we conclude that a necessary condition forE[Q n ] =O(nlogn) 1/3 is 1 (nlogn) 1/3 =O θ − n . (A.31) If|θ + n |P(Q n >τ n ) =o(|θ − n |), then we are done because E[θ n (Q n )]∼ θ − n ≥ Θ 1 n 1/3 (logn) 1/3 ! , E h θ n (Q n ) 2 i = θ − n 2 P(Q n ≤τ n )+ θ + n 2 P(Q n >τ n )≪ θ − n , so that (A.26) can be used to prove the lemma. On the other hand, if θ − n =O θ + n P(Q n >τ n ) , i.e., |θ − n | P(Q n >τ n ) =O θ + n , (A.32) then by (A.30) and (A.31) we must have 1 n 1/6 =O θ + n . (A.33) Observe that by (A.31), (A.32), and (A.33), we have E h θ n (Q n ) 2 i ≥ θ + n θ + n P(Q n >τ n )≥ Θ 1 n 1/2 (logn) 1/3 ! , establishing (A.27) that completes the proof. 107 On the other hand whenlim n→∞ P(Q n ≥τ n +1) = 0 and|θ + n | =O(|θ − n |), we haveE h θ n (Q n ) 2 i ≪ E[θ n (Q n )] and E[θ n (Q n )] = θ − n P(Q n ≤τ n )− θ + n P(Q n ≥τ n +1)∼ θ − n P(Q n ≤τ n )≥ Θ 1 √ n , so (A.26) is established and the proof is complete. Finally, iflim n→∞ P(Q n >τ n )> 0, then the lemma is automatically proved by (A.28) because √ n = O(τ n ) andE[Q n ]≥τ n P(Q n >τ n ). Proof of Lemma A.2.3: Fori∈Z + , the set of non-negative integers, define d n,i (θ) := Π i−1 j=0 λ ¯ F ¯ p+θ(j)+h j n and recall that by Lemma A.2.1, ¯ R ⋆ n −R n (p n,TP )∼αn P ∞ i=0 θ n (i)d n,i (θ n ) P ∞ i=0 d n,i (θ n ) +βn P ∞ i=0 θ n (i) 2 d n,i (θ n ) P ∞ i=0 d n,i (θ n ) +γ P ∞ i=0 id n,i (θ n ) P ∞ i=0 d n,i (θ n ) . We will first show that for any pricing policy inP ⋆ TP the delay sensitivity of customers in our system can be ignored while computing the steady-state performance metrics as follows: P ∞ i=0 θ n (i) 2 d n,i (θ n ) P ∞ i=0 d n,i (θ n ) ∼ P ∞ i=0 θ n (i) 2 ˆ d i (θ) P ∞ i=0 ˆ d i (θ) (A.34) P ∞ i=0 id n,i (θ n ) P ∞ i=0 d n,i (θ n ) ∼ P ∞ i=0 i ˆ d i (θ) P ∞ i=0 ˆ d i (θ) , (A.35) P ∞ i=0 θ n (i)d n,i (θ n ) P ∞ i=0 d n,i (θ n ) ∼ P ∞ i=0 θ n (i)−h i n ˆ d i (θ) P ∞ i=0 ˆ d i (θ) (A.36) where ˆ d i (θ) := Π i−1 j=0 λ ¯ F (¯ p+θ(j)). 108 For what follows, it is important to note that for any policy inP ⋆ TP there exists ˆ τ n such that lim n→∞ θ + n ˆ τ n =∞, ˆ τ n ≫τ n , ˆ τ n ≪ √ n, and ˆ τ n exp −θ + n ˆ τ n ≪τ n (A.37) Let us begin by establishing (A.34). Observe that we can write the denominator on the left hand side of (A.34) as follows: ∞ X i=0 d n,i (θ n ) = τn X i=0 Π i−1 j=0 λ ¯ F ¯ p−θ − n +h j n +Π τn j=0 λ ¯ F ¯ p−θ − n +h j n ∞ X i=τn+1 Π i−1 j=τn+1 λ ¯ F ¯ p+θ + n +h j n . We first show that Π τn j=0 λ ¯ F ¯ p−θ − n +h j n ∼ Π τn j=0 λ ¯ F ¯ p−θ − n . (A.38) To that end, observe that Π τn j=0 λ ¯ F ¯ p−θ − n +h j n = Π τn j=0 λ ¯ F ¯ p−θ − n +Π τn j=0 λ ¯ F ¯ p−θ − n Π τn j=0 ¯ F ¯ p−θ − n +h j n ¯ F ¯ p−θ − n −1 . We further have the following bound for sufficiently largen Π τn j=0 ¯ F ¯ p−θ − n +h j n ¯ F ¯ p−θ − n ≥ ¯ F ¯ p−θ − n +h τn n ¯ F ¯ p−θ − n ! τn+1 ≥ 1− K 1 ¯ F ¯ p−θ − n h τ n n ! τn+1 whereK 1 := sup x∈[¯ p−ǫ,¯ p] f (x)<∞ for some smallǫ> 0. Becauseτ n ≪ √ n, we have lim n→∞ 1− K 1 ¯ F ¯ p−π − n h τ n n ! τn+1 = 1. Therefore, Π τn j=0 ¯ F ¯ p−θ − n +h j n ¯ F ¯ p−θ − n −1≪ 1 (A.39) and (A.38) follows. 109 We next show that ∞ X i=τn+1 Π i−1 j=τn+1 λ ¯ F ¯ p+θ + n +h j n ∼ ∞ X i=τn+1 Π i−1 j=τn+1 λ ¯ F ¯ p+θ + n . (A.40) For eachi =τ n +1,τ n +2,... , let π n,i (θ) := Π i−1 j=τn+1 ¯ F ¯ p+θ+h j n ¯ F (¯ p+θ) (A.41) and observe that ∞ X i=τn+1 Π i−1 j=τn+1 λ ¯ F ¯ p+θ + n +h j n = ∞ X i=τn+1 Π i−1 j=τn+1 λ ¯ F ¯ p+θ + n + ˆ τn X i=τn+1 Π i−1 j=τn+1 λ ¯ F ¯ p+θ + n π n,i θ + n −1 + ∞ X i=ˆ τn+1 Π i−1 j=τn+1 λ ¯ F ¯ p+θ + n π n,i θ + n −1 . (A.42) For anyi =τ n +1,...,ˆ τ n , we have π n,i θ + n −1≤ max i=τn+1,...ˆ τn π n,i θ + n −1 . Because ˆ τ n ≪ √ n, we can use the same argument used to obtain (A.39) to establish that max i=τn+1,...,ˆ τn π n,i θ + n −1 ≪ 1, (A.43) implying ˆ τn X i=τn+1 Π i−1 j=τn+1 λ ¯ F ¯ p+θ + n π n,i θ + n −1 ≪ ∞ X i=τn+1 Π i−1 j=τn+1 λ ¯ F ¯ p+θ + n . (A.44) 110 Also, because|π n,i (θ + n )−1|≤ 1, we have ∞ X i=ˆ τn+1 Π i−1 j=τn+1 λ ¯ F ¯ p+θ + n π n,i θ + n −1 ≤ ∞ X i=ˆ τn+1 Π i−1 j=τn+1 λ ¯ F ¯ p+θ + n = λ ¯ F ¯ p+θ + n ˆ τn−τn ∞ X i=τn+1 Π i−1 j=τn+1 λ ¯ F ¯ p+θ + n . Further, because λ ¯ F (¯ p+θ + n ) ˆ τn−τn ≪ 1 by the first and the second equations in (A.37), we have ∞ X i=ˆ τn+1 Π i−1 j=τn+1 λ ¯ F ¯ p+θ + n π n,i θ + n −1 ≪ ∞ X i=τn+1 Π i−1 j=τn+1 λ ¯ F ¯ p+θ + n . (A.45) Hence, (A.40) follows by using (A.42), (A.44), and (A.45). Using similar arguments, we can also establish τn X i=0 Π i−1 j=0 λ ¯ F ¯ p−θ + n +h j n ∼ τn X i=0 Π i−1 j=0 λ ¯ F ¯ p−θ + n , (A.46) so we omit the steps. By (A.38), (A.40), and (A.46), we obtain ∞ X i=0 d n,i (θ n )∼ ∞ X i=0 ˆ d i (θ n ). (A.47) Becauseθ 2 n takes only two values, we can follow the same steps used to achieve (A.47) to deduce that ∞ X i=0 θ n (i) 2 d n,i (θ n )∼ ∞ X i=0 θ n (i) 2 ˆ d i (θ n ) and therefore E h θ n (Q n ) 2 i = P ∞ i=0 θ n (i) 2 d n,i (θ n ) P ∞ i=0 d n,i (θ n ) ∼ P ∞ i=0 θ n (i) 2 ˆ d i (θ n ) P ∞ i=0 ˆ d i (θ n ) . (A.48) 111 Let us next establish (A.35). By (A.38), we have the following approximation for the numerator of the term on the left hand side of (A.35): ∞ X i=0 id n,i (θ n )∼ τn X i=0 iΠ i−1 j=0 λ ¯ F ¯ p−θ − n +h j n +Π τn j=0 λ ¯ F ¯ p−θ − n ∞ X i=τn+1 iΠ i−1 j=τn+1 λ ¯ F ¯ p+θ + n +h j n . Therefore, to establish (A.35), it suffices to show that τn X i=0 iΠ i−1 j=0 λ ¯ F ¯ p−θ − n +h j n ∼ τn X i=0 iΠ i−1 j=0 λ ¯ F ¯ p−θ − n (A.49) ∞ X i=τn+1 iΠ i−1 j=τn+1 λ ¯ F ¯ p+θ + n +h j n ∼ ∞ X i=τn+1 iΠ i−1 j=τn+1 λ ¯ F ¯ p+θ + n . (A.50) We first show (A.50). Defineη n :=λ ¯ F (¯ p+θ + n ) and recallπ n,i in (A.41). Observe that ∞ X i=τn+1 iΠ i−1 j=τn+1 λ ¯ F ¯ p+θ + n +h j n = ∞ X i=τn+1 iη i−τn n + ˆ τn X i=τn+1 iη i−τn n π n,i θ + n −1 + ∞ X i=ˆ τn+1 iη i−τn n π n,i θ + n −1 (A.51) and hence (A.50) is established by showing that the second and the third terms in the right hand side of (A.51) are dominated by the first term in in the right hand side of (A.51). Observe that by (A.43), ˆ τn X i=τn+1 iη i−τn n (π n,i −1)≪ ∞ X i=τn+1 iη i−τn n (A.52) is readily established. Let us establish ∞ X i=ˆ τn+1 iη i−τn n (π n,i −1)≪ ∞ X i=τn+1 iη i−τn n . (A.53) 112 To that end, observe that we have the following upper bound for the left hand side of (A.52): ∞ X i=ˆ τn+1 iη i−τn n (π n,i −1) ≤ ∞ X i=ˆ τn+1 iη i−τn n |π n,i −1| (a) ≤ ∞ X i=ˆ τn+1 iη i−τn n =η ˆ τn−τn+1 n ∞ X i=0 iη i n +(ˆ τ n −1) ∞ X i=0 η i n ! , where(a) follows because|π n,i −1|≤ 1. Further, the right hand side of (A.51) can be written as ∞ X i=τn+1 iη i−τn n =η n ∞ X i=0 iη i n +τ n ∞ X i=0 η i n ! . Byθ + n ≪ 1 √ n and the first and the second equation in (A.37), we haveη ˆ τn−τn n ≪ 1 and(ˆ τ n −1)η ˆ τn−τn n ≪ τ n , implying that η ˆ τn−τn n ∞ X i=0 iη i n +(ˆ τ n −1) ∞ X i=0 η i n ! ≪ ∞ X i=0 iη i n +τ n ∞ X i=0 η i n and hence (A.53) follows. Therefore, by (A.51), (A.52), and (A.53), we have (A.50). Also, using similar arguments, we can establish (A.49) and therefore E[Q n ] = P ∞ i=0 id n,i (θ n ) P ∞ i=0 d n,i (θ n ) ∼ P ∞ i=0 i ˆ d i (θ n ) P ∞ i=0 ˆ d i (θ n ) . (A.54) Let us next establish (A.36). Observe that P ∞ i=0 θ n (i) ˆ d i (θ n ) P ∞ i=0 ˆ d i (θ n ) =− 1 f (¯ p) P ∞ i=0 ¯ F (¯ p)−f (¯ p)θ n (i)− f ′ (¯ p) 2 θ n (i) 2 ˆ d i (θ n ) P ∞ i=0 ˆ d i (θ n ) + ¯ F (¯ p) f (¯ p) − f ′ (¯ p) 2f (¯ p) P ∞ i=0 θ n (i) 2 ˆ d i (θ n ) P ∞ i=0 ˆ d i (θ n ) . Because by the Taylor series expansion on ¯ F around ¯ p ¯ F (¯ p+θ n (i))− ¯ F (¯ p)−f (¯ p)θ n (i)− f ′ (¯ p) 2 θ n (i) 2 =O θ − n 3 + θ + n 3 113 for alli≥ 0, we have P ∞ i=0 ¯ F (¯ p)−f (¯ p)θ n (i)− f ′ (¯ p) 2 θ n (i) 2 ˆ d i (θ n ) P ∞ i=0 ˆ d i (θ n ) = 1 λ P ∞ i=0 ˆ d i+1 (θ n ) P ∞ i=0 ˆ d i (θ n ) +O θ − n 3 + θ + n 3 , implying that ¯ F (¯ p) f (¯ p) = P ∞ i=0 θ n (i) ˆ d i (θ n ) P ∞ i=0 ˆ d i (θ n ) + 1 λf (¯ p) P ∞ i=0 ˆ d i+1 (θ n ) P ∞ i=0 ˆ d i (θ n ) + f ′ (¯ p) 2f (¯ p) P ∞ i=0 θ n (i) 2 ˆ d i (θ n ) P ∞ i=0 ˆ d i (θ n ) +O θ − n 3 + θ + n 3 . (A.55) Also, observe that P ∞ i=0 θ n (i)+h i n d n,i (θ n ) P ∞ i=0 d n,i (θ n ) =− 1 f (¯ p) P ∞ i=0 ¯ F (¯ p)−f (¯ p) θ n (i)+h i n − f ′ (¯ p) 2 θ n (i) 2 d n,i (θ n ) P ∞ i=0 d n,i (θ n ) + ¯ F (¯ p) f (¯ p) − f ′ (¯ p) 2f (¯ p) P ∞ i=0 θ n (i) 2 d n,i (θ n ) P ∞ i=0 d n,i (θ n ) . Again, by the Taylor series expansion on ¯ F around ¯ p, ¯ F (¯ p) f (¯ p) = P ∞ i=0 θ n (i)+h i n d n,i (θ n ) P ∞ i=0 d n,i (θ n ) + 1 λf (¯ p) P ∞ i=0 d n,i+1 (θ n ) P ∞ i=0 d n,i (θ n ) + f ′ (¯ p) 2f (¯ p) P ∞ i=0 θ n (i) 2 d n,i (θ n ) P ∞ i=0 d n,i (θ n ) +o P ∞ i=0 θ n (i) 2 +h i n d n,i (θ n ) P ∞ i=0 d n,i (θ n ) . (A.56) Using (A.55) and (A.56), we have P ∞ i=0 θ n (i)d n,i (θ n ) P ∞ i=0 d n,i (θ n ) = P ∞ i=0 θ n (i) ˆ d i (θ n ) P ∞ i=0 ˆ d i (θ n ) − h n P ∞ i=0 i ˆ d i (θ n ) P ∞ i=0 ˆ d i (θ n ) + 1 λf (¯ p) P ∞ i=0 ˆ d i+1 (θ n ) P ∞ i=0 ˆ d i (θ n ) − P ∞ i=0 d n,i+1 (θ n ) P ∞ i=0 d n,i (θ n ) ! +o P ∞ i=0 θ n (i) 2 +h i n ˆ d i (θ n ) P ∞ i=0 ˆ d i (θ n ) . (A.57) 114 Observe that P ∞ i=0 ˆ d i+1 (θ n ) P ∞ i=0 ˆ d i (θ n ) − P ∞ i=0 d n,i+1 (θ n ) P ∞ i=0 d n,i (θ n ) = P ∞ i=0 ˆ d i+1 (θ n ) 1−Π i j=0 ¯ F(¯ p+θn(j)+h j n ) ¯ F(¯ p+θn(j)) P ∞ i=0 ˆ d i (θ n ) P ∞ i=0 d n,i (θ n ) = Θ P ∞ i=0 ˆ d n,i+1 (θ n ) 1−Π i j=0 ¯ F(¯ p+θn(j)+h j n ) ¯ F(¯ p+θn(j)) P ∞ i=0 ˆ d n,i (θ n ) 2 . Suppose we can show that P ∞ i=0 ˆ d i+1 (θ n ) 1−Π i j=0 ¯ F(¯ p+θn(j)+h j n ) ¯ F(¯ p+θn(j)) P ∞ i=0 ˆ d i (θ n ) ≪ 1. (A.58) Then, by (A.57) and (A.58), we have E[θ n (Q n )] = P ∞ i=0 θ n (i)d n,i (θ n ) P ∞ i=0 d n,i (θ n ) ∼ P ∞ i=0 θ n (i) ˆ d i (θ n ) P ∞ i=0 ˆ d i (θ n ) − h n P ∞ i=0 i ˆ d n,i (θ n ) P ∞ i=0 ˆ d i (θ n ) +o 1+ P ∞ i=0 θ n (i) 2 +h i n ˆ d i (θ n ) P ∞ i=0 ˆ d i (θ n ) . (A.59) Further by (A.48), (A.54), and (A.59) and noting thatγ−hα =h, we have for anyp n,TP ∈P ⋆ TP ¯ R ⋆ n −R n (p n,TP )∼αn P ∞ i=0 θ n (i) ˆ d i (θ n ) P ∞ i=0 ˆ d i (θ n ) +βn P ∞ i=0 θ n (i) 2 ˆ d i (θ n ) P ∞ i=0 ˆ d i (θ n ) +h P ∞ i=0 i ˆ d i (θ n ) P ∞ i=0 ˆ d i (θ n ) +o 1 P ∞ i=0 ˆ d i (θ n ) ! . Becauseθ n takes only two values, we can explicitly calculate the infinite sums in the above display to reach that ¯ R ⋆ n −R n (p n,TP )∼L n θ − n ,θ + n ,τ n as stated in the lemma forL n defined in (A.10). 115 To complete the proof of lemma, it remains to show (A.58). To that end, fori = 0,1,... , define r n,i (θ n ) := Π i j=0 ¯ F ¯ p+θ n (j)+h j n ¯ F (¯ p+θ n (j)) . Then, we can write the left hand side of (A.58) as follows: P ∞ i=0 ˆ d i (θ n )(1−r n,i (θ n )) P ∞ i=0 ˆ d i (θ n ) = P ˆ τn i=0 ˆ d i (θ n )(1−r n,i (θ n )) P ∞ i=0 ˆ d i (θ n ) + P ∞ i=ˆ τn+1 ˆ d i (θ n )(1−r n,i (θ n )) P ∞ i=0 ˆ d i (θ n ) . Let us first establish P ∞ i=ˆ τn+1 ˆ d i (θ n )(1−r n,i (θ n )) P ∞ i=0 ˆ d i (θ n ) ≪ 1. (A.60) Notice that we have ∞ X i=ˆ τn+1 ˆ d i (θ n )(1−r n,i (θ n )) ≤ ∞ X i=ˆ τn+1 ˆ d i (θ n )|1−r n,i (θ n )| (a) ≤ ∞ X i=ˆ τn+1 ˆ d i (θ n ) (b) ≪ ∞ X i=0 ˆ d i (θ n ), where(a) follows because|1−r n,i (θ n )|≤ 1 and(b) can be verified by a simple algebra of the summation of a geometric series. Therefore, (A.60) holds. Let us now establish P ˆ τn i=0 ˆ d i (θ n )(1−r n,i (θ n )) P ∞ i=0 ˆ d i (θ n ) ≪ 1. (A.61) To this end, observe that fori = 0,...,ˆ τ n , we have r n,i (θ n )≥ ¯ F ¯ p−θ − n +h τn n ¯ F ¯ p−θ − n ! τn+1 ¯ F ¯ p+θ + n +h ˆ τn n ¯ F ¯ p+θ + n ! ˆ τn−τn . Becauseτ n ≪ ˆ τ n ≪ √ n by the second and the third equations in (A.37), we have lim n→∞ ¯ F ¯ p−θ − n +h τn n ¯ F ¯ p−θ − n ! τn+1 ¯ F ¯ p+θ + n +h ˆ τn n ¯ F ¯ p+θ + n ! ˆ τn−τn = 1, implying (A.61). 116 Proof of Lemma A.2.4 We first show that lim n→∞ θ − n τ n =∞ (A.62) is necessary to achieve limsup n→∞ L n (θ − n ,θ + n ,τ n ) (nlogn) 1/3 <∞ for any sequence of two-price policies inP ⋆ TP . Suppose that (A.62) is violated. By Lemma A.2.3 and noting thatγ≥ 0, we have L n θ − n ,θ + n ,τ n &αn −θ − n D 1 (θ − n ,τ n )+θ + n D 2 (θ + n ) D 1 θ − n ,τ n +D 2 θ + n +βn (θ − n ) 2 D 1 (θ − n ,τ n )+(θ + n ) 2 D 2 (θ + n ) D 1 θ − n ,τ n +D 2 θ + n , so that it suffices to show that (nlogn) 1/3 ≪αn −θ − n D 1 (θ − n ,τ n )+θ + n D 2 (θ + n ) D 1 θ − n ,τ n +D 2 θ + n +βn (θ − n ) 2 D 1 (θ − n ,τ n )+(θ + n ) 2 D 2 (θ + n ) D 1 θ − n ,τ n +D 2 θ + n . (A.63) To this end, observe that by Taylor series expansion on ¯ F around ¯ p, D 1 θ − n ,τ n +D 2 θ + n ≤ 1 λ ¯ F ¯ p−θ − n −1 + 1 1−λ ¯ F ¯ p+θ + n ∼ 1 λf (¯ p) 1 θ − n + 1 θ + n , −θ − n D 1 θ − n ,τ n +θ + n D 2 θ + n ∼ λ ¯ F (¯ p−θ − n ) −τn−1 λf (¯ p) − λf ′ (¯ p) 2(λf (¯ p)) 2 θ + n +θ − n , θ − n 2 D 1 θ − n ,τ n + θ + n 2 D 2 θ + n ∼ 1 λf (¯ p) θ − n +θ + n . Therefore, by recalling thatβ−− λf ′ (¯ p) 2λf(¯ p) α =φ> 0, we have αn −θ − n D 1 (θ − n ,τ n )+θ + n D 2 (θ + n ) D 1 θ − n ,τ n +D 2 θ + n +βn (θ − n ) 2 D 1 (θ − n ,τ n )+(θ + n ) 2 D 2 (θ + n ) D 1 θ − n ,τ n +D 2 θ + n ≥αn θ − n θ + n λ ¯ F (¯ p−θ − n ) −τn−1 θ − n +θ + n . Because 1 √ n ≪θ − n and 1 √ n ≪θ + n , we have √ n≪αn θ − n θ + n λ ¯ F (¯ p−θ − n ) −τn−1 θ − n +θ + n when (A.62) is violated, so that (A.63) is established. 117 Given that (A.62) must be satisfied, we can simplifyL n in Lemma A.2.3 by the Taylor series expansion on ¯ F around ¯ p to obtain L n θ − n ,θ + n ,τ n ∼αn 1 λ ¯ F ¯ p−θ − n τn+1 1 θ − n + 1 θ + n +φnθ − n θ + n +h 1 λf (¯ p) 1 θ + n − 1 θ − n +τ n . (A.64) From (A.64), we first prove that (nlogn) 1/3 = O(L(θ − n ,θ + n ,τ n )). Suppose on the other hand that there exists a sequence of two-price policies inP ⋆ TP under which L θ − n ,θ + n ,τ n ≪ (nlogn) 1/3 . (A.65) From the last term in the right hand side of (A.64), it is necessary that τ n ≪ (nlogn) 1/3 and 1 θ + n ≪ (nlogn) 1/3 . (A.66) From (A.62) and the first equation in (A.66), we need 1 θ − n ≪ (nlogn) 1/3 . (A.67) Because ofnθ − n θ + n ≪n 1/3 (logn) 1/3 , the second equation in (A.66), and (A.67), we need to have θ − n ≪ (logn) 2/3 n 1/3 andθ + n ≪ (logn) 2/3 n 1/3 . (A.68) By combining the first equation in (A.66) and first equation in (A.68), we obtainτ n θ − n ≪ logn, implying that λ ¯ F ¯ p−θ − n τn+1 = Θ exp λf (¯ p)τ n θ − n =o n 1/4 . (A.69) Using (A.66)-(A.68), and (A.69), we have αn 1 λ ¯ F ¯ p−θ − n τn+1 1 θ − n + 1 θ + n ≫ n 5/12 (logn) 4/3 ≫n 1/3 (logn) 1/3 , (A.70) contradicting (A.65). Therefore, there is no sequence of two-price policies inP ⋆ TP under which (A.65) is achieved. 118 We now turn to optimizeL n (θ − n ,θ + n ,τ n ). Observe that from the arguments in the previous paragraph, θ + n andτ n must be on the order of 1 (nlogn) 1/3 and(nlogn) 1/3 , respectively forL n (θ − n ,θ + n ,τ n ) to be on the order of (nlogn) 1/3 . Furthermore, (A.70) was obtained becauseτ n θ − n < 1 3 logn. Therefore, we needτ n andθ − n to satisfy τ n θ − n ≥ 1 3λf (¯ p) logn. (A.71) Also for nθ − n θ + n to be of order (nlogn) 1/3 , we need θ − n to be on the order of logn (nlogn) 1/3 because of the condition imposed onθ + n earlier in this paragraph. Therefore, it suffices to set θ − n = logn n 1/3 (logn) 1/3 π − n , θ + n = 1 n 1/3 (logn) 1/3 π + n , andτ n =n 1/3 (logn) 1/3 t n (A.72) for some positive constantsπ − n ,π + n , andt n that are on the order ofO(1) to derive an asymptotically optimal two-price policy inP ⋆ TP . By combining (A.64), (A.71), and (A.72), the optimization to findπ − n ,π + n , andt n is given by min π − n >0,π + n >0,tn≥ 1 3λf(¯ p)π − n ˆ L π − n ,π + n ,t n :=ξ n α π − n π + n π − n + 1 logn π + n +φπ − n π + n +h 1 λf (¯ p) 1 π + n − 1 logn 1 π − n +t n (A.73) where ξ n := n 1/3 (logn) 2/3 λ ¯ F ¯ p− logn n 1/3 (logn) 1/3 π − n n 1/3 (logn) 1/3 tn+1 . Letπ ⋆,− n ,π ⋆,+ n , andt ⋆ n be the optimal solution. Then, it can be easily (because the optimization in (A.73) is convex) checked that lim n→∞ π ⋆,− n =π, lim n→∞ π ⋆,+ n = 3π, lim n→∞ t ⋆ n = 1 3λf (¯ p)π , and lim n→∞ ˆ L π ⋆,− n ,π ⋆,+ n ,t ⋆ n = Π TP as stated in the lemma. By noting thatξ n ≪ 1, it is straightforward to check that E[Q n ]∼W TP n 1/3 (logn) 1/3 forW TP defined in the statement of the part (c) of the proposition. 119 Proof of Lemma A.2.5: Suppose √ n = O(τ n ), i.e., there existsc > 0 suchc √ n ≤ τ n , by considering anM/M/1/c √ n system with the arrival rate ofnλ ¯ F ¯ p+h c √ n and the service rate ofn, we can prove that √ n =O(E[Q n ]). On the other hand, ifτ n ≪ √ n, then the we can prove the lemma by considering anM/M/1/τ n system with the arrival rate ofnλ ¯ F ¯ p+h τn n and the service rate ofn. Preliminaries for the proofs of Lemma A.2.6- A.2.11: In the proofs of Lemma A.2.6- A.2.11 that follow, for notational simplifications, we assume n = 1,h = 1,ψ =−1, λf(¯ p) 4φ = 1. (A.74) We note that (A.74) is without loss of generality. Under (A.74),w κ is the solution of w ′ κ (q)+q−w κ (q) 2 =κ, (A.75) such that w κ (0) = −n 1/3 . It is useful to note that because w ′ κ and w κ are well-defined, w ′′ κ is also well defined and given by w ′′ κ (q) = 2w κ (q)w ′ κ (q)−1. (A.76) Proof of Lemma A.2.6: We show that for anyκ 1 <κ 2 ,w κ 1 (q)<w κ 2 (q) for allq∈ [0,min{q ∞,κ 1 ,q ∞,κ 1 }] (recall the definition of q ∞,κ in (A.20)). Suppose C :={q∈ [0,min{q ∞,κ 1 ,q ∞,κ 2 }] :w κ 1 (q) =w κ 2 (q)} is not empty. Let q 1 := inf C. Because w ′ κ 1 (0) < w ′ κ 2 (0), we haveq 1 > 0. Also, because w κ 2 (q) > w κ 1 (q) for all q ∈ (0,q 1 ), we need to have w ′ κ 1 (q 1 ) > w ′ κ 2 (q 1 ). However, this is impossible because w ′ κ 2 >w ′ κ 1 wheneverw κ 2 =w κ 1 by (A.75). 120 Proof of Lemma A.2.7: To show thatL is not empty, considerκ = 0. We must have sup q≥0 w 0 (q) ≤ 0. Suppose not and there existsq > 0 such thatw 0 (q)> 0. Then, there must exist ˜ q such thatw 0 (˜ q) = 0 andw ′ 0 (˜ q)> 0. This is a contradiction because w ′ 0 (˜ q) =−˜ q +w κ (˜ q) 2 =−˜ q< 0. Because sup q≥0 w 0 (q) ≤ 0, for 0 ∈ U to hold, the only possibility is that w 0 (q) is increasing in q and converges to−ǫ for someǫ≥ 0 asq→∞. However, this is impossible because if so, then we would have w ′ 0 (q) =−q +w κ (q) 2 , where the left hand side converges to 0 and the right hand side converges to negative infinity asq grows. Therefore,0∈L andL is not empty. To show thatU is not empty, we first prove that ¯ U is not empty, where ¯ U :={κ> 0 :w κ (1)≥ 1}. Because w κ (1) =κ− 1 2 −n 1/3 + Z 1 0 w κ (x) 2 dx≥κ− 1 2 −n 1/3 and lim κ→∞ κ− 1 2 −n 1/3 = ∞, there must exist ¯ κ such thatw ¯ κ (1) ≥ 1, proving the claim. Therefore, if we letκ 1 := inf ¯ U, then we obtainκ 1 < ∞. IfU is empty, then for anyκ ≥ κ 1 , there existsq 1,κ > 1 such that w κ (q 1,κ ) = 1 and w ′ κ (q 1,κ ) < 0. For q ∈ [1,q 1,κ ], we have w κ (q) 2 ≥ w κ (q) and therefore, w κ (q)≥w κ (q) wherew κ solves w ′ κ (q)+q−w κ (q) =κ (A.77) wherew κ (1) =w κ (1). It is straightforward to solve (A.77) and its solution is given by w κ (q) = 1−κ+q +(κ−1)exp(q−1). Ifκ ≥ min{κ 1 ,1}, thenw κ (q) > 1 for allq ∈ [1,q 1,κ ]. This contradictsw κ (q 1,κ ) = 1. Therefore,U is not empty and the result follows. 121 Proof of Lemma A.2.8: Becausew ′ κ (0) =κ+w κ (q) 2 > 0, it suffices to prove thatw κ does not have a local minima. From (A.76), we see thatw ′′ κ < 0, wheneverw ′ κ = 0, so that there is no local minima. For the second part of the lemma, it is enough to prove thatw κ (q) is not bounded below by a finite constant. If so, then lim q→∞ w ′ κ (q) = 0 because w κ (q) is monotone decreasing for large q. This is a contradiction by noting that−q diverges to negative infinity whereasw κ (q) 2 converges asq→∞ in (A.75). Proof of Lemma A.2.9 The proof is identical to the Proof of Lemma B.1.2 in Chapter B and is omitted here. Proof of Lemma A.2.10: By Lemma A.2.6 and Lemma A.2.7, we already know that κ ⋆ < ∞. We will now prove that κ ⋆ ∈ U. Towards a contradiction, supposeκ ⋆ ∈L. Then, by Lemma A.2.8, we know there existsq ⋆ , the maxima of w κ ⋆, andw ⋆ :=w κ ⋆(q ⋆ )<∞. For a sufficiently smallǫ> 0 , there existsδ > 0 such thatw κ ⋆ (q ⋆ +ǫ) = w ⋆ −δ again by Lemma A.2.8. However, for allκ>κ ⋆ ,κ∈U by Lemma A.2.6 and A.2.8 and therefore, we havew κ (q ⋆ +ǫ)>w ⋆ becausew κ (q ⋆ )>w κ ⋆(q ⋆ ) andw κ (q) is increasing inq ifκ∈U. This contradicts Lemma A.2.9. Therefore,κ ⋆ ∈U andw κ⋆ (q) is increasing inq. Proof of Lemma A.2.11: Note that we only need to considerκ such that sup q≥0 w κ (q)> 0 because otherwise the lemma automati- cally follows. Suppose that forκ∈L, we can showw κ is concave when it is increasing. Then, forκ∈L, we have w κ (q) 2 =w ′ κ (q)+q−κ≤w ′ κ (0)+q, where the last inequality follows becausew κ is concave when it is increasing. This implies that w κ (q)≤ p w ′ κ (0)+q≤ q κ ⋆ +n 2/3 +q, where the second inequality follows becausew ′ κ (0) =κ+w κ (0) 2 ≤κ ⋆ +n 2/3 for anyκ∈L. 122 It remains to show that w κ is concave when it is increasing. Let q κ := inf{q≥ 0 :w κ (q) = 0} (so thatw ′ κ (q κ ) > 0) and ˆ q κ be a constant such thatw κ (ˆ q κ ) = sup q≥0 w κ (q) for allκ ∈ L. Suppose there exists ¯ q κ ∈ (q κ ,ˆ q κ ) such thatw ′′ κ (¯ q κ ) > 0 and there existsq κ := inf{q∈ (¯ q κ ,ˆ q κ ) :w ′′ κ (q) = 0}. In this circumstance,w ′ κ is increasing inq∈ ¯ q κ ,q κ , so that w κ q κ w ′ κ q κ >w κ (¯ q κ )w ′ κ (¯ q κ ) (A.78) becausew κ (q) is increasing inq∈ [0,ˆ q κ ]. However, (A.78) leads to a contradiction because 0 =w ′′ κ q κ = 2w κ q κ w ′ κ q κ −1> 2w κ (¯ q κ )w ′ κ (¯ q κ )−1 =w ′′ κ (¯ q κ )> 0. Therefore, q κ should not exist, implying that w ′ κ (q) is increasing in q ∈ (q κ ,ˆ q κ ). However, this is a contradiction because0 =w ′ κ (ˆ q κ )>w ′ κ (q 0,κ )> 0. 123 Appendix B Technical Appendix to Chapter 2 B.1 Proofs of Main Results Proof of Proposition 2.4.1: The first step is to define the reduced Brownian control problem, which has the one-dimensional state- descriptor ˆ W . The reduced Brownian control problem is to minimize limsup t→∞ E " 1 t Z t 0 N X k=1 c k m k ˆ Q k (s) ds # (B.1) using a control process ˆ Q 1 , ˆ Q 2 ,..., ˆ Q N , ˆ I such that ˆ Q k (t)≥ 0 for allt≥ 0,k∈{1,2,...,N}, and(2.8),(2.14),(2.15) are satisfied. (B.2) The control ˆ Q 1 , ˆ Q 2 ,..., ˆ Q N , ˆ I is admissible if it is F t -adapted and satisfies (B.2). The control ˆ Q ⋆ 1 , ˆ Q ⋆ 2 ,..., ˆ Q ⋆ N , ˆ I ⋆ is optimal if it is admissible and for any admissible ˆ Q 1 , ˆ Q 2 ,..., ˆ Q N , ˆ I ˆ C R ˆ Q ⋆ 1 , ˆ Q ⋆ 2 ,..., ˆ Q ⋆ N , ˆ I ⋆ ≤ ˆ C R ˆ Q 1 , ˆ Q 2 ,..., ˆ Q N , ˆ I , where ˆ C R ˆ Q 1 , ˆ Q 2 ,..., ˆ Q N , ˆ I = limsup t→∞ E " 1 t Z t 0 N X k=1 c k m k ˆ Q k (s) ds # . The following two Claims, which are Propositions 2 and 3 in Rubino and Ata (2009) adapted to our setting, complete the proof. The claims are stated without proof because the arguments in Rubino and Ata (2009) do not rely on their assumption of linear drift, and so apply to our non-linear drift setting also. Claim B.1.1. (Proposition 2 in Rubino and Ata (2009)) The Brownian control problem (2.6)-(2.8) is equiv- alent to the reduced Brownian control problem (B.1)-(B.2) in the following sense: 124 • Every admissible control ˆ Q 1 , ˆ Q 2 ,..., ˆ Q N , ˆ I for the reduced Brownian control problem yields an admissible control ˆ Y for the Brownian control problem having ˆ C( ˆ Y) = ˆ C R ˆ Q 1 , ˆ Q 2 ,..., ˆ Q N , ˆ I . • Similarly, for any admissible control ˆ Y for the Brownian control problem, there exists an admis- sible control ˆ Q 1 , ˆ Q 2 ,..., ˆ Q N , ˆ I for the reduced Brownian control problem having ˆ C( ˆ Y) ≥ ˆ C R ˆ Q 1 , ˆ Q 2 ,..., ˆ Q N , ˆ I . Claim B.1.2. (Proposition 3 in Rubino and Ata (2009)) The reduced Brownian control problem (B.1)-(B.2) is equivalent to the workload control problem in the following sense: • Every admissible control (q, ˆ I) for the workload control problem yields an admissible control ˆ Q 1 , ˆ Q 2 ,..., ˆ Q N , ˆ I for the reduced Brownian control problem having ˆ C R ˆ Q 1 , ˆ Q 2 ,..., ˆ Q N , ˆ I = ˆ C w (q, ˆ I). • Similarly, for any admissible control ˆ Q 1 , ˆ Q 2 ,..., ˆ Q N , ˆ I for the reduced Brownian control prob- lem, there exists an admissible control for the workload control problem having ˆ C w (q, ˆ I) ≤ ˆ C R ˆ Q 1 , ˆ Q 2 ,..., ˆ Q N , ˆ I . Proof of Corollary 2.4.1: It follows from Proposition 2.4.1 that given the control ˆ Y ⋆ that is optimal for the Brownian control problem (2.6)-(2.8), then there exists an admissible control(q, ˆ I) for the workload control problem having ˆ C w (q, ˆ I)≤ ˆ C( ˆ Y ⋆ ). (B.3) Suppose that(q, ˆ I) is not optimal for the workload control problem. Then, there exists an admissible control (q ⋆ , ˆ I ⋆ ) where ˆ C w (q ⋆ , ˆ I ⋆ )< ˆ C w (q, ˆ I). (B.4) It follows from Proposition 2.4.1 that there exists an admissible control ˆ Y for the Brownian control problem having ˆ C( ˆ Y) = ˆ C w (q ⋆ , ˆ I ⋆ ). (B.5) This is a contradiction, because (B.3)-(B.5) implies ˆ C( ˆ Y)< ˆ C( ˆ Y ⋆ ). 125 Proof of Proposition 2.4.2: The following notation is useful in the proof. For anyw≥ 0, let E w [·] :=E h ·| ˆ W ⋆ (0) =w i . (B.6) Then, since ˆ W ⋆ (0) = ˆ w, for the second statement of the Proposition, we must show thatE ˆ w h ˆ W ⋆ (t) i /t→ 0 as t → ∞ . Note that we explicitly subscript the expectation because it is useful in the proof to also consider what happens when the initial state differs from ˆ w. We observe that the process ˆ W ⋆ is a delayed regenerative process with regeneration point 0. The initial cycle length is T 0 := inf n t≥ 0 : ˆ W ⋆ (t) = 0 o . The remaining i.i.d. cycles are defined as the process starting from 0, reachingw> 0, and returning to 0, and we letτ represent the length of one of those cycles. Suppose we can show that (i) ˆ W ⋆ is positive recurrent; that is, thatE ˆ w [τ]<∞; (ii) E ˆ w h R T 0 +τ T 0 ˆ W ⋆ (s)ds i <∞; (iii) E ˆ w h R T 0 0 ˆ W ⋆ (s)ds i <∞. Then, also noting that R T 0 0 ˆ W ⋆ (s)ds < ∞ almost surely, it follows from the renewal reward theorem for regenerative processes (see, for example, Theorem 2.3 in Section 13 on Regenerative Processes of Sigman (2011)) that E ˆ w h R t 0 ˆ W ⋆ (s)ds i t → E ˆ w h R T 0 +τ T 0 ˆ W ⋆ (s)ds i E ˆ w [τ] ast→∞. (B.7) In the next paragraph, we establish that (B.7) implies E ˆ w h ˆ W ⋆ (t) i t → 0 ast→∞. (B.8) To show (B.8), it is enough to show that E ˆ w h ˆ W ⋆ (t) 2 i t 2 → 0 ast→∞. (B.9) 126 To show (B.9), the first step is to recall that ˆ ξ(·) in (2.15) has the same distribution asθ·+σ ˆ B(·) for ˆ B a standard Brownian motion, and then apply Ito’s Lemma to find ˆ W ⋆ (t) 2 = Z t 0 σ 2 +2 ˆ W ⋆ (s) θ− N X k=1 m k q ⋆ k ˆ W ⋆ (s) µ k ds (B.10) + ˆ W ⋆ (0) 2 +2σ Z t 0 ˆ W ⋆ (s)d ˆ B(s)+2 Z t 0 ˆ W ⋆ (s)d ˆ I ⋆ (s) ≤ ˆ W ⋆ (0) 2 +σ 2 t+2θ Z t 0 ˆ W ⋆ (s)ds+2σ Z t 0 ˆ W ⋆ (s)d ˆ B(s). Since any sample path of ˆ W ⋆ (t) 2 is continuous almost surely, for anyt > 0, R t 0 ˆ W ⋆ (s) 2 ds < ∞ almost surely and ˆ M(t) := R t 0 ˆ W ⋆ (s)d ˆ B(s) is a local martingale. Then, there exists a sequence of stopping times {τ k ,k = 1,2,...} such that τ k < τ k+1 for all k = 1,2,..., τ k → ∞ as k → ∞ almost surely, and ˆ M(t∧τ k ) is a martingale for eachk = 1,2,..., so thatE h ˆ M(t∧τ k ) i = 0. Then, E ˆ w h ˆ W ⋆ (t) 2 i t 2 = E ˆ w liminf k→∞ ˆ W ⋆ (t∧τ k ) 2 t 2 (B.11) ≤ liminf k→∞ E ˆ w h ˆ W ⋆ (t∧τ k ) 2 i t 2 (B.12) ≤ ˆ w 2 +σ 2 t t 2 + 2θ t liminf k→∞ E ˆ w h R t∧τ k 0 ˆ W ⋆ (s)ds i t (B.13) ≤ ˆ w 2 +σ 2 t t 2 + 2θ t E ˆ w h R t 0 ˆ W ⋆ (s)ds i t , (B.14) where • (B.11) follows from the definition of the sequence{τ k ,k = 1,2,...}; • (B.12) follows from Fatou’s Lemma; • (B.13) follows from (B.10) and the fact that ˆ M(t∧τ k ) is a martingale; • (B.14) follows from the monotone convergence theorem. Finally, it follows from (B.11)-(B.14) that to establish (B.9), it is enough to establish (B.7). 127 In summary, the proof is complete once we show (i)-(iii). For this, it is helpful to define the operator (Au)(x) := σ 2 2 u ′′ (x)+g(x)u ′ (x) where g(x) :=θ− N X k=1 m k (q ⋆ k (x)) µ k andu is any twice differentiable function having domainR + . In the following, assumingw > ˆ w, we show that for T w := inf n t≥ 0 : ˆ W ⋆ (t)≥w o T w,0 := inf n t≥T w : ˆ W ⋆ (t) = 0 o , for anyk∈{0,1,2,...} E ˆ w " Z T w 0 ˆ W ⋆ (s) k ds # <∞ (B.15) and E w Z T w,0 0 ˆ W ⋆ (s) k ds <∞. (B.16) This is sufficient to complete the proof because when ˆ w = 0, (B.15) and (B.16) imply (i) and (ii) by letting k = 0 andk = 1, and for ˆ w> 0, E ˆ w Z T 0 0 ˆ W ⋆ (s)ds ≤E ˆ w " Z T w 0 ˆ W ⋆ (s)ds # +E w Z T w,0 0 ˆ W ⋆ (s)ds , which implies (iii) by lettingk = 1. The argument to show (B.15). For anyx≥ 0, let r(x) := Z x 0 − 2 σ 2 y k exp Z y 0 2 σ 2 g(z)dz dyexp − Z x 0 2 σ 2 g(z)dz . Then it is straight forward to check that u(x) := Z x w r(y)dy 128 solves (Au)(x) =−x k , u ′ (0) = 0 andu(w) = 0. Moreover, since r(x) < 0 for x > 0, u(x) is decreasing. This implies that u(x) ≥ 0 for x ∈ [0,w] becauseu(w) = 0. Furthermore, there exists a twice differentiable function ˜ u having domainR + such that ˜ u(x) =u(x) forx∈ [0,w] and ˜ u ′ is bounded. From Ito’s Lemma 1 , by recalling ˆ ξ(·) D =θ·+σ ˆ B(·) for a standard Brownian motion ˆ B(·) with ˆ B(0) = 0, ˜ u ˆ W ⋆ (t) = ˜ u ˆ W ⋆ (0) + Z t 0 (A˜ u) ˆ W ⋆ (s) ds+σ Z t 0 ˜ u ′ ˆ W ⋆ (s) d ˆ B(s)+ Z t 0 ˜ u ′ ˆ W ⋆ (s) d ˆ I ⋆ (s) = ˜ u ˆ W ⋆ (0) − Z t 0 ˆ W ⋆ (s) k ds+σ Z t 0 ˜ u ′ ˆ W ⋆ (s) d ˆ B(s). Since ˜ u ′ is bounded, the stochastic integral in the above is a martingale, and, therefore, so is ˜ u ˆ W ⋆ (t) − ˜ u ˆ W ⋆ (0) + Z t 0 ˆ W ⋆ (s) k ds. Then, ˜ u ˆ W ⋆ (t∧T w ) − ˜ u ˆ W ⋆ (0) + Z t∧T w 0 ˆ W ⋆ (s) k ds is also a martingale, and so E ˆ w " ˜ u ˆ W ⋆ t∧T w + Z t∧T w 0 ˆ W ⋆ (s) k ds− ˜ u ˆ W ⋆ (0) # = 0. Since ˆ W ⋆ t∧T w ∈ [0,w], ˜ u ˆ W ⋆ t∧T w ≥ 0, and so E ˆ w " Z t∧T w 0 ˆ W ⋆ (s) k ds # ≤ ˜ u(ˆ w). 1 Although many statements of Ito’s Lemma require a twice continuously differentiable function, it follows from Problem 3.7.3 in Karatzas and Shreve (1988) that a sufficient condition to apply Ito’s Lemma is that the function is twice differentiable. See also the discussion at the end of Section 4.6 in Harrison (2001). This is important because the function ˜ u may not have a continuous second derivative, because the functiong may not be continuous. 129 Taking the limit ast→∞ in the above and using the monotone convergence theorem implies E ˆ w " Z T w 0 ˆ W ⋆ (s) k ds # ≤ ˜ u(ˆ w)<∞, which establishes (B.15). The argument to show (B.16). For any ˜ w>w, let T ˜ w 0 := inf n t≥T w : ˆ W ⋆ (t)≥ ˜ w or ˆ W ⋆ (t) = 0 o . Next, for anyx≥ 0, let q(x) :=q 0 (˜ w)+r(x) where q 0 (˜ w) := R ˜ w 0 R x 0 2 σ 2 exp R y 0 2 σ 2 g(z)dz dyexp − R x 0 2 σ 2 g(z)dz dx R ˜ w 0 exp − R x 0 2 σ 2 g(z)dz dx . Then, it is straightforward to check that v ˜ w (x) := Z x ˜ w q(y)dy solves (Av ˜ w )(x) =−x k , v ˜ w (0) =v ˜ w (˜ w) = 0. Note that there can be at most onex q > 0 such thatq(x)> 0 for allx∈ [0,x q ) andq(x)< 0 for allx>x q . Thenv ˜ w (0) =v ˜ w (˜ w) = 0 implies that suchx q exists andx q ∈ (0, ˜ w) so thatv ˜ w (x) is strictly increasing for x∈ [0,x q ] and strictly decreasing otherwise . This implies thatv ˜ w (x)≥ 0 for allx∈ [0, ˜ w]. Furthermore, there exists a twice differentiable function ˜ v ˜ w having domainR + such that ˜ v ˜ w (x) =v ˜ w (x) for allx∈ [0, ˜ w] and ˜ v ′ ˜ w is bounded. 130 From Ito’s lemma, by recalling ˆ ξ(·) D =θ·+σ ˆ B(·) for a standard Brownian motion ˆ B(·) with ˆ B(0) = 0, ˜ v ˜ w ˆ W ⋆ (t) = ˜ v ˜ w ˆ W ⋆ (0) + Z t 0 (A˜ v ˜ w ) ˆ W ⋆ (s) ds+σ Z t 0 ˜ v ′ ˜ w ˆ W ⋆ (s) d ˆ B(s)+ Z t 0 ˜ v ′ ˜ w ˆ W ⋆ (s) d ˆ I ⋆ (s) = ˜ v ˜ w ˆ W ⋆ (0) − Z t 0 ˆ W ⋆ (s) k ds+σ Z t 0 ˜ v ′ ˜ w ˆ W ⋆ (s) d ˆ B(s)+ Z t 0 ˜ v ′ ˜ w ˆ W ⋆ (s) d ˆ I ⋆ (s). Since ˜ v ′ is bounded, the stochastic integral in the above is a martingale, and, therefore, so is ˜ v ˜ w ˆ W ⋆ (t) − ˜ v ˜ w ˆ W ⋆ (0) + Z t 0 ˆ W ⋆ (s) k ds− Z t 0 ˜ v ′ ˜ w ˆ W ⋆ (s) d ˆ I ⋆ (s) as well as ˜ v ˜ w ˆ W ⋆ (t∧T ˜ w 0 ) − ˜ v ˜ w ˆ W ⋆ (0) + Z t∧T ˜ w 0 0 ˆ W ⋆k (s)ds− Z t∧T ˜ w 0 0 ˜ v ′ ˜ w ˆ W ⋆ (s) d ˆ I ⋆ (s). If we renew ˆ W ⋆ once the stochastic process reachesw> ˆ w, we have ˆ W ⋆ (s)∈ (0, ˜ w) for alls∈ [0,t∧T ˜ w 0 ). Then, for alls∈ [0,t∧T ˜ w 0 ),d ˆ I ⋆ (s) = 0. Therefore, since ˆ I ⋆ is continuous, it follows that Z t∧T ˜ w 0 0 ˜ v ′ ˜ w ˆ W ⋆ (s) d ˆ I ⋆ (s) = 0. Then, the same argument as in the previous paragraph shows that E w " Z T ˜ w 0 0 ˆ W ⋆ (s) k ds # ≤ ˜ v ˜ w (w)<∞. (B.17) The monotone convergence theorem then implies lim ˜ w→∞ E w " Z T ˜ w 0 0 ˆ W ⋆ (s) k ds # =E w Z T w,0 0 ˆ W ⋆ (s) k ds . (B.18) Finally, to complete the proof, it follows from (B.17) and (B.18) that it is enough to show there existsM > 0 that does not depend on ˜ w such that ˜ v ˜ w (w)<M for all ˜ w>w. (B.19) 131 To prove (B.19), observe that, for any ˜ w>w, we have ˜ v ˜ w (w) ≤ Z ˜ w w Z w 0 2 σ 2 y k exp Z y 0 2 σ 2 g(z)dz dy exp − Z w 0 2 σ 2 g(z)dz dw − Z ˜ w 0 Z w 0 2 σ 2 y k exp Z y 0 2 σ 2 g(z)dz dy exp − Z w 0 2 σ 2 g(z)dz dw R ˜ w x exp − R w 0 2 σ 2 g(z)dz dw R ˜ w 0 exp − R w 0 2 σ 2 g(z)dz dw ≤ R ˜ w 0 R w 0 2 σ 2 y k exp R y 0 2 σ 2 g(z)dz dy exp − R w 0 2 σ 2 g(z)dz dw R ˜ w 0 exp − R w 0 2 σ 2 g(z)dz dw Z w 0 exp − Z w 0 2 σ 2 g(z)dz dw. Sinceg(x)→−∞ asx→∞, we have Z ˜ w 0 exp − Z w 0 2 σ 2 g(z)dz dw→∞ as ˜ w→∞. Also, we argue that Z ˜ w 0 Z w 0 2 σ 2 y k exp Z y 0 2 σ 2 g(z)dz dy exp − Z w 0 2 σ 2 g(z)dz dw→∞ as ˜ w→∞. (B.20) To prove, observe that there exists y ⋆ > 0 such that g(x) ≤ 0 for all x ≥ y ⋆ because g(x) → −∞ as x→∞. Then for any ˜ w≥y ⋆ , we have d d˜ w Z ˜ w 0 Z w 0 2 σ 2 y k exp Z y 0 2 σ 2 g(z)dz dy exp − Z w 0 2 σ 2 g(z)dz dw = Z ˜ w 0 2 σ 2 y k exp − Z ˜ w y 2 σ 2 g(z)dz dy = Z y ⋆ 0 2 σ 2 y k exp − Z ˜ w y 2 σ 2 g(z)dz dy + Z ˜ w y ⋆ 2 σ 2 y k exp − Z ˜ w y 2 σ 2 g(z)dz dy ≥ Z ˜ w y ⋆ 2 σ 2 y k dy, implying (B.20). Then, we use L’Hopital’s rule to conclude limsup ˜ w→∞ v ˜ w (w)≤ Z ∞ 0 2 σ 2 y k exp Z y 0 2 σ 2 g(z)dz dy Z x 0 exp − Z w 0 2 σ 2 g(z)dz dw. 132 Since it is straightforward to see that Z ∞ 0 2 σ 2 y k exp Z y 0 2 σ 2 g(z)dz dy<∞, the proof is completed. Proof of Theorem 2.4.2: △ ˆ I(t) := ˆ I(t)− ˆ I(t−),t> 0, be the jump of ˆ I at timet, and let ˆ I C (t) := ˆ I(t)− X 0≤s≤t △ ˆ I(s),t> 0, be the continuous part of ˆ I. Note that ˆ I is RCLL and has finite variation (since it is increasing). Then, the generalized Ito formula (see, for example, Section 4.7 in Harrison (1985)) implies that forv :R + →R that is twice-continuously differentiable, and ˆ W that satisfies (2.18) under an admissible control(q, ˆ I), v ˆ W(t) = v ˆ W(0) + Z t 0 σ 2 2 v ′′ ˆ W(s) + θ− N X k=1 m k q k s, ˆ W (s) µ k v ′ ˆ W(s) ds + Z t 0 σv ′ ˆ W(s) d ˆ B(s)+ Z t 0 v ′ ˆ W(s) d ˆ I C (s)+ X 0≤s<t △v ˆ W(s) , where ˆ B is a standard Brownian motion so that ˆ ξ(·) has the same distribution asθ·+σ ˆ B(·) and △v ˆ W(t) :=v ˆ W(t) −v ˆ W(t−) fort> 0. Whenv ′ is bounded, the stochastic integral is a martingale, and so taking expectations shows that E h v ˆ W(t) i = E Z t 0 σ 2 2 v ′′ ˆ W(s) + θ− N X k=1 m k q k s, ˆ W (s) µ k v ′ ˆ W(s) ds +E h v ˆ W(0) i +E Z t 0 v ′ ˆ W(s) d ˆ I C (s) +E X 0≤s<t △v ˆ W(s) . 133 Then, E " 1 t Z t 0 N X k=1 c k m k q k s, ˆ W (s) ds # (B.21) = E " 1 t Z t 0 σ 2 2 v ′′ ˆ W(s) +θv ′ ˆ W(s) + N X k=1 m k q s, ˆ W(s) c k − 1 µ k v ′ ˆ W(s) # − 1 t E h v ˆ W(t) i + 1 t E h v ˆ W(0) i +E 1 t Z t 0 v ′ ˆ W(s) d ˆ I C (s) +E 1 t X 0≤s<t △v ˆ W(s) . Proof of (i): Since ˆ I ⋆ is continuous v ′ (0) = 0, and R ∞ 0 ˆ W ⋆ (s)d ˆ I ⋆ (s) = 0, and (v ⋆ ,κ ⋆ ) solves (2.20)-(2.21), it follows from (B.21) that E " 1 t Z t 0 N X k=1 c k m k q ⋆ k ˆ W (s) ds # =κ ⋆ − 1 t E h v ⋆ ˆ W ⋆ (t) i + 1 t E[v(ˆ w)]. (B.22) The fact thatv ′ is bounded implies that there exists constantsa,b such that|v(x)|≤a+bx for allx≥ 0, and so E h v ⋆ ˆ W ⋆ (t) i ≤a+bE h ˆ W ⋆ (t) i . Since by Proposition 2.4.2E h ˆ W ⋆ (t) i /t→ 0, it follows that lim t→∞ 1 t E h v ⋆ ˆ W ⋆ (t) i = 0. SinceE[v(ˆ w)]/t→ 0 ast→∞, it follows from (B.22) that ˆ C w q ⋆ , ˆ I ⋆ =κ ⋆ . Proof of (ii): Whenv ′ is non-negative, since ˆ I C is increasing, Z t 0 v ′ ˆ W(s) d ˆ I C (s)≥ 0, 134 and also △v ˆ W(t) = v ˆ W(t) −v ˆ W(t−) = v ˆ W(t) −v ˆ W(t)−△ ˆ I(t) = Z ˆ W(t) ˆ W(t)−△ ˆ I(t) v ′ (s)ds ≥ 0. Next, note that for anyv : R + → R that is twice-continuously differentiable and hasv ′ bounded, and ˆ W that satisfies (2.18) under an admissible control(q, ˆ I) σ 2 2 v ′′ ˆ W(t) +θv ′ ˆ W(t) + N X k=1 m k q k t, ˆ W (t) c k − v ′ ˆ W (t) µ k ≥ σ 2 2 v ′′ ˆ W(t) +φ ˆ W(t),v ′ ˆ W(t) =κ ⋆ , for allt≥ 0. Therefore, from (B.21), E " 1 t Z t 0 N X k=1 c k m k q k s, ˆ W (s) ds # ≥κ ⋆ − 1 t E h v ˆ W(t) i + 1 t E h v ˆ W(0) i . (B.23) Under the assumption that limsup t→∞ 1 t E h ˆ W(t) i = 0, it follows as in the proof of part (i) that limsup t→∞ 1 t E h v ˆ W(t) i = 0. Then, it follows from (B.23) that ˆ C w (q, ˆ I)≥κ ⋆ . Since ˆ C w q ⋆ , ˆ I ⋆ =κ ⋆ from part (i), we conclude that q ⋆ , ˆ I ⋆ is optimal for the workload control problem. 135 To complete the proof of part (ii), we show that if limsup t→∞ 1 t E h ˆ W(t) i > 0, (B.24) then ˆ C w q, ˆ I =∞. It follows from (B.24) that limsup t→∞ E h ˆ W(t) i =∞, which implies that limsup t→∞ E h q k t, ˆ W(t) i =∞ (B.25) for at least onek∈{1,2,...,N} or for all. Fix one suchk. Then, forx 0 such thath k (x 0 )> 0 (noting that such anx 0 exists becauseh k is increasing and is not the zero function by assumption), ˆ C w (q, ˆ I) ≥ limsup t→∞ 1 t Z t 0 c k f k E " Z q k(s, ˆ W(s))/f k 0 h k (y)dy # ds ≥ limsup t→∞ c k f k h k (x 0 ) 1 t Z t 0 E " q k (s, ˆ W(s)) f k −x 0 ! 1 n q k (s, ˆ W(s))>f k x 0 o # ds ≥ limsup t→∞ c k h k (x 0 ) 1 t Z t 0 E h q k (s, ˆ W(s))1 n q k (s, ˆ W(s))>f k x 0 oi ds−c k f k h k (x 0 )x 0 . Hence ˆ C w (q, ˆ I) =∞ if limsup t→∞ E h q k t, ˆ W(t) 1 n q k t, ˆ W(t) >f k x 0 oi =∞. (B.26) Since (B.26) follows from (B.25), the proof is complete. Proof of Theorem 2.4.3: Define the set D := κ> 0 : there existsx κ > 0 such thatw κ is increasing ifx<x κ and decreasing ifx>x κ . . 136 The intuition behind the definitions of the setsU andD, and what we show in this proof, is thatR + can be divided so that for largeκ > 0, κ ∈ U and the solution to IVP(κ) may become large while for small κ> 0,κ∈D and the solution to IVP(κ) decreases to−∞. Noting that anyκ∈U hasw κ non-negative, this suggests that to findκ ⋆ such that (v,κ ⋆ ) solves the Bellman equations (2.20)-(2.21), we should search for κ∈U such thatw κ is bounded. (This proof idea is similar in spirit to how Ghosh and Weerasinghe (2007) construct a solution to the Bellman equations in their paper, that arise in a different context, by considering a parameterized family of differential equations; see Section 3 of Ghosh and Weerasinghe (2007).) Next, we observe several useful properties of the functionφ and the solution to the IVP(κ)w κ . Lemma B.1.1. For any givenκ∈R, there exists a unique solutionw κ :R + →R of IVP(κ). Lemma B.1.2. w κ (x) is jointly continuous in(x,κ). Lemma B.1.3. Assumeθ≤ 0. For any givenx> 0,φ(x,w) is strictly decreasing inw. Lemma B.1.4. Assumeθ≤ 0. Ifκ 1 >κ 2 , thenw κ 1 (x)>w κ 2 (x) for allx> 0. Lemma B.1.5. Assumeθ ≤ 0. For any givenκ > 0, if there existsx κ > 0 such thatw ′ κ (x κ ) < 0, then κ∈D. Lemma B.1.6. Assumeθ≤ 0. Ifκ∈D, thenw κ (x)→−∞ asx→∞. Lemma B.1.7. Assume θ ≤ 0. For any given κ > 0, if there exists x 0 > 0 such that w κ (x 0 ) ≥ min(c 1 µ 1 ,c 2 µ 2 ), thenκ∈U. We now argue that the setsU andD partition the parameter space; that is, U∩D =∅ andU∪D =R + . (B.27) It is immediate from the definitions ofU andD thatU ∩D = ∅. To see thatU ∪D = R + , fixκ > 0. If there existsx > 0 so thatw ′ κ (x) < 0, thenκ ∈ D by Lemma B.1.5. Otherwise, if no suchx exists, then w ′ κ (x)≥ 0 for allx, which impliesw κ ∈U. It follows from (B.27) that if there existsκ ⋆ such that(v,κ ⋆ ) solves the Bellman equation, eitherκ ⋆ ∈U orκ ⋆ ∈ D. It cannot be the case thatκ ⋆ ∈ D because ifκ ⋆ ∈ D thenw ⋆ κ is unbounded by Lemma B.1.6. 137 Hence if the desired κ ⋆ exists, it must be that κ ⋆ ∈ U. Since w κ 1 is everywhere larger than w κ 2 when κ 1 >κ 2 by Lemma B.1.4, it is sensible to choose κ ⋆ := infU, assuming thatU is non-empty. Note that for anyκ∈U,w κ is non-negative. Then, to complete the proof, it is enough to show that 1. U is non-empty; 2. κ ⋆ ∈U; 3. w κ ⋆ is bounded. The argument thatU is non-empty. Fixx 0 > 0 andκ 0 > 0. We may assume thatw κ 0 (x 0 ) < min(c 1 µ 1 ,c 2 µ 2 ,...,c N µ N ) because other- wiseκ 0 ∈ U by Lemma B.1.7. Ifκ > κ 0 , thenw κ (x) > w κ 0 (x) for allx > 0 by Lemma B.1.4. Then, φ(x,w κ 0 (x))>φ(x,w κ (x)) by Lemma B.1.3. Hence from the definition of IVP(κ) 1 2 σ 2 w κ (x 0 ) =κx 0 − Z x 0 0 φ(x,w κ (x))dx, forκ>κ 0 . Since κx 0 − Z x 0 0 φ(x,w κ (x))dx>κx 0 − Z x 0 0 φ(x,w κ 0 (x))dx→∞, asκ→∞, it follows thatw κ (x 0 ) → ∞ asκ → ∞. Sincew κ (x) is jointly continuous inx andκ by Lemma B.1.2, there must existκ>κ 0 such thatw κ (x 0 )≥ min(c 1 µ 1 ,c 2 µ 2 ,...,c N µ N ). Then,κ∈U by Lemma B.1.7. The argument thatκ ⋆ ∈U. It is helpful to first show that ifκ u ∈U andκ d ∈D, thenκ u ≥κ d . The argument is by contradiction. Suppose there existκ u ∈U andκ d ∈D such thatκ u <κ d . There existsx κ d > 0 such thatw κ d (x κ d ) = 0 andw κ d (x)< 0 for allx>x κ d from Lemma B.1.6 and the definition ofD. Also, the mean value theorem implies that there existsx κu > 0 such thatw κu (x κu )> 0 becausew ′ κu (0)> 0. Lemma B.1.4 implies that w κu (x) < w κ d (x) for allx > 0, so thatw κ d (x) < 0 for allx > x κ d , and it must be thatw κu was strictly decreasing over some interval. This contradicts the fact thatκ u ∈U. 138 To see thatκ ⋆ ∈ U, the argument is also by contradiction. Suppose not. Then, from (B.27),κ ⋆ ∈ D, and so there existsx 0 such thatw κ ⋆(x 0 ) = 0. Supposex κ ⋆ is a maximum ofw κ ⋆ and letǫ :=w κ ⋆(x κ ⋆)/3. Then, by the continuity in Lemma B.1.2, there existsκ>κ ⋆ such thatw κ (x 0 )∈ [0,ǫ]. Also,κ∈D because w κ (x κ ⋆) > w κ ⋆(x κ ⋆) by Lemma B.1.4 andw κ ⋆(x κ ⋆) > w κ (x 0 ) by the definition ofǫ, so thatw κ must be strictly decreasing over some interval. Since κ u ∈ U and κ d ∈ D implies κ u ≥ κ d from the paragraph above,infU ≥ supD by (B.27). HenceinfU ≥κ. Butκ>κ ⋆ which is a contradiction. The argument thatw κ ⋆ is bounded. We first note from the definition ofU that eitherw κ ⋆ increases to∞ or it is bounded. In the case that w κ ⋆ increases to∞, there existsx 0 such thatw κ ⋆(x 0 ) = min(c 1 µ 1 ,c 2 µ 2 ,...,c N µ N ). Then, there exists δ > 0 andx δ > x 0 such thatw κ ⋆(x δ ) ≥ min(c 1 µ 1 ,c 2 µ 2 ,...,c N µ N ) +δ. Also, by the joint continuity in Lemma B.1.2, there existsκ < κ ⋆ such thatw κ (x δ ) > w κ ⋆(x δ )−δ/3 > min(c 1 µ 1 ,c 2 µ 2 ,...,c N µ N ). Then,κ∈U by Lemma B.1.7. This is a contradiction becauseκ<κ ⋆ andκ ⋆ = infU. Proof of Proposition 2.4.3: From Theorem 2.4.2, it is sufficient for the proof to construct v : R + → R that is twice continuously differentiable andκ> 0 that solves the Bellman equation (2.20)-(2.21). Define v(x) := Z x 0 v ′ (y)dy for v ′ (x) := R x 0 2 σ 2 (κ ⋆ −c 1 µ 1 g(y))exp 2 σ 2 θy− R y 0 g(z)dz dy exp 2 σ 2 θx− R x 0 g(y)dy . Then,v ′ solves σ 2 2 v ′′ (x)+θv ′ (x)+(c 1 µ 1 −v ′ (x))g(x) =κ ⋆ . (B.28) The first step in showing that (v,κ ⋆ ) solves the Bellman equation is to show that (2.20) is satisfied. Suppose we can show thatv ′ (x)∈ [0,c 1 µ 1 ] for allx≥ 0. Then, we have q ⋆ (x) = argmin q∈A(x) m 1 (q 1 ) c 1 − 1 µ 1 v ′ (x) +m 2 (q 2 ) c 2 − 1 µ 2 v ′ (x) , 139 for anyx≥ 0. To see this, observe that m 1 (q 1 ) c 1 − 1 µ 1 v ′ (x) +m 2 (q 2 ) c 2 − 1 µ 2 v ′ (x) = c 1 µ 1 −v ′ (x) m 1 (q 1 ) µ 1 + m 2 (q 2 ) µ 2 = c 1 µ 1 −v ′ (x) ( q 2 1 µ 1 f 1 + 1 µ 2 f 2 2 µ 2 x− q 1 µ 1 3 ) , for anyq ∈A(x). Next, note that as a function ofq 1 ,q 2 1 /(µ 1 f 1 )+(µ 2 (x−q 1 /µ 1 )) 3 / µ 2 f 2 2 is convex because d 2 dq 2 1 q 2 1 µ 1 f 1 + 1 µ 2 f 2 2 µ 2 x− q 1 µ 1 3 ! > 0 for allq 1 ∈ [0,µ 1 x]. Therefore, argmin q 1 ∈[0,µ 1 x] n q 2 1 /(µ 1 f 1 )+(µ 2 (x−q 1 /µ 1 )) 3 / µ 2 f 2 2 o , for any given x ≥ 0, is derived by solving d dq 1 q 2 1 µ 1 f 1 + 1 µ 2 f 2 2 µ 2 x− q 1 µ 1 3 ! = 2q 1 µ 1 f 1 − 3µ 2 2 µ 1 f 2 2 x− q 1 µ 1 2 = 0. (B.29) Since the solution of (B.29) isq ⋆ 1 (x) defined in Subsection 2.4.2, we have φ x,v ′ (x) =θv ′ (x)+(c 1 µ 1 −v ′ (x))g(x), for allx≥ 0, which implies by (B.28) that (2.20) is satisfied. The argument that v ′ (x) ∈ [0,c 1 µ 1 ] for all x ≥ 0 is by contradiction. Suppose not. Then, since v ′ (0) = 0 andv ′ (x)→ c 1 µ 1 asx→∞ follows from L’Hopital’s rule, there must existx 1 < x 2 such that v ′ (x 1 ) =v ′ (x 2 )>c 1 µ 1 andv ′′ (x 1 )> 0,v ′′ (x 2 )< 0. From (B.28), σ 2 v ′′ (x 1 )+θv ′ (x 1 )+(c 1 µ 1 −v ′ (x 1 ))g(x 1 ) = κ ⋆ σ 2 v ′′ (x 2 )+θv ′ (x 2 )+(c 1 µ 1 −v ′ (x 2 ))g(x 2 ) = κ ⋆ . Subtracting the bottom equation from the top yields σ 2 2 v ′′ (x 1 )−v ′′ (x 2 ) + c 1 µ 1 −v ′ (x 1 ) (g(x 1 )−g(x 2 )) = 0. 140 This is a contradiction becausev ′′ (x 1 )−v ′′ (x 2 )> 0,c−v ′ (x 1 )< 0, andg(x 1 )−g(x 2 )< 0 sinceg is a strictly increasing function. The second step in showing that (v,κ ⋆ ) solves the Bellman equation is to show that (2.21) is satisfied. It is immediate thatv ′ (0) = 0. The fact thatv ′ is bounded follows from the fact thatv ′ is continuous and v ′ (x) → c 1 µ 1 asx → ∞ by L’Hopital’s rule. To see thatv ′ is non-negative, note that it follows from its definition that eitherv ′ (x) ≥ 0 for allx, in which case the proof is complete, or there existsx 0 such that v ′ (x)> 0 for allx<x 0 andv ′ (x)< 0 for allx>x 0 . This is a contradiction becausev ′ (x)→c 1 µ 1 > 0 asx→∞, and so there cannot exist such anx 0 . Proof of Proposition 2.4.4: From Theorem 2.4.2, it is sufficient for the proof to construct v : R + → R that is twice continuously differentiable andκ > 0 that solves the Bellman equation (2.20)-(2.21). We first do this whenc 1 µ 1 γ 1 > c 2 µ 2 γ 2 and c 1 µ 1 ≥ c 2 µ 2 , so that a static priority control is optimal, and second do this when c 1 µ 1 γ 1 > c 2 µ 2 γ 2 andc 1 µ 1 <c 2 µ 2 , so that a dynamic priority control is optimal. The proof whenc 1 µ 1 γ 1 >c 2 µ 2 γ 2 andc 1 µ 1 ≥c 2 µ 2 . Define v(x) := Z x 0 v ′ (y)dy for v ′ (x) := (κ ⋆ −c 2 µ 2 θ) r 2 σ 2 γ 2 Φ x− θ γ 2 r σ 2 2γ 2 −Φ − q 2 σ 2 γ 2 θ φ x− θ γ 2 r σ 2 2γ 2 +c 2 µ 2 1− φ − q 2 σ 2 γ 2 θ φ x− θ γ 2 r σ 2 2γ 2 for allx≥ 0. Note thatκ ⋆ > 0 because κ ⋆ =c 2 µ 2 θ+ 1 q 2 σ 2 γ 2 1 Ψ − q 2 σ 2 γ 2 θ 141 andΨ − q 2 σ 2 γ 2 θ > 0. Also, v ′ (x) =c 2 µ 2 1− Ψ x− θ γ 2 r σ 2 2γ 2 Ψ − q 2 σ 2 γ 2 θ , for allx≥ 0, (B.30) so that it is straightforward to seev ′ (0) = 0 andv ′ (x) increases toc 2 µ 2 asx→∞. Furthermore,v ′ solves σ 2 2 v ′′ (x)+(θ−γ 2 x)v ′ (x)+c 2 µ 2 γ 2 x =κ ⋆ for allx≥ 0. (B.31) To show that (v,κ ⋆ ) solves the Bellman equations (2.20)-(2.21), first note that it is immediate from the representation ofv ′ in (B.30) and the properties of Ψ that (2.21) is satisfied. To see that (2.20) is also satisfied, we show that φ x,v ′ (x) = θv ′ (x)+ inf q∈A(x) γ 1 µ 1 q 1 c 1 µ 1 −v ′ (x) + γ 2 µ 2 q 2 c 2 µ 2 −v ′ (x) (B.32) = θv ′ (x)+γ 2 x c 2 µ 2 −v ′ (x) . Then, it follows from (B.31) and (B.32) that (2.20) holds; i.e., that σ 2 2 v ′′ (x)+φ(x,v ′ (x)) =κ ⋆ for allx≥ 0. We show that (B.32) holds by considering each of the two possibilities: c 1 µ 1 = c 2 µ 2 orc 1 µ 1 > c 2 µ 2 . In the case thatc 1 µ 1 =c 2 µ 2 , it follows from the fact thatc 1 µ 1 γ 1 >c 2 µ 2 γ 2 thatγ 1 >γ 2 , so that inf q∈A(x) γ 1 µ 1 q 1 (c 1 µ 1 −v ′ (x))+ γ 2 µ 2 q 2 (c 2 µ 2 −v ′ (x)) = (c 2 µ 2 −v ′ (x)) inf q∈[0,xµ 1 ] γ 1 µ 1 q 1 + γ 2 µ 2 µ 2 x− q 1 µ 1 = (c 2 µ 2 −v ′ (x)) γ 2 x+ 1 µ 1 inf q∈[0,xµ 1 ] {(γ 1 −γ 2 )q 1 } = γ 2 x(c 2 µ 2 −v ′ (x)) 142 and (B.32) holds. Otherwise, in the case thatc 1 µ 1 >c 2 µ 2 , first note that to show (B.32) holds it is equivalent to show γ 1 (c 1 µ 1 −v ′ (x))−γ 2 (c 2 µ 2 −v ′ (x))> 0 for allx≥ 0 (B.33) because when (B.33) holds inf q∈A(x) γ 1 µ 1 q 1 (c 1 µ 1 −v ′ (x))+ γ 2 µ 2 q 2 (c 2 µ 2 −v ′ (x)) = inf q 1 ∈[0,xµ 1 ] γ 1 µ 1 q 1 (c 1 µ 1 −v ′ (x))+γ 2 x− q 1 µ 1 (c 2 µ 2 −v ′ (x)) = γ 2 x(c 2 µ 2 −v ′ (x))+ 1 µ 1 inf q 1 ∈[0,xµ 1 ] γ 1 (c 1 µ 1 −v ′ (x))−γ 2 (c 2 µ 2 −v ′ (x)) q 1 = γ 2 x(c 2 µ 2 −v ′ (x)). Then, in the case thatc 1 µ 1 >c 2 µ 2 andγ 1 <γ 2 , (B.33) holds because (B.33) is equivalent to c 1 µ 1 γ 1 −c 2 µ 2 γ 2 γ 1 −γ 2 ≤v ′ (x), which holds because (c 1 µ 1 γ 1 −c 2 µ 2 γ 2 )/(γ 1 −γ 2 ) < 0 and v ′ (x) ≥ 0 for all x ≥ 0 from its explicit expression in (B.30). Finally, in the case thatc 1 µ 1 > c 2 µ 2 andγ 1 ≥ γ 2 , (B.33) holds because (B.33) is equivalent to v ′ (x)< c 1 µ 1 γ 1 −c 2 µ 2 γ 2 γ 1 −γ 2 , which holds becausev ′ (x)≤c 2 µ 2 for allx≥ 0 by (B.30),c 2 µ 2 <c 1 µ 1 by assumption, and c 1 µ 1 γ 1 −c 2 µ 2 γ 2 γ 1 −γ 2 −c 1 µ 1 = γ 2 (c 1 µ 1 −c 2 µ 2 ) γ 1 −γ 2 > 0. The proof whenc 1 µ 1 γ 1 >c 2 µ 2 γ 2 andc 1 µ 1 <c 2 µ 2 is true. Define v ′ 1 (x) := (κ ⋆ −c 2 µ 2 θ) r 2 σ 2 γ 2 Φ x− θ γ 2 r σ 2 2γ 2 −Φ − q 2 σ 2 γ 2 θ φ x− θ γ 2 r σ 2 2γ 2 +c 2 µ 2 1− φ − q 2 σ 2 γ 2 θ φ x− θ γ 2 r σ 2 2γ 2 (B.34) 143 and v ′ 2 (x) := (κ ⋆ −c 1 µ 1 θ) r 2 σ 2 γ 1 Φ x− θ γ 1 r σ 2 2γ 1 −Φ L ⋆ − θ γ 1 r σ 2 2γ 1 φ x− θ γ 1 r σ 2 2γ 1 +c 1 µ 1 + γ 2 (c 1 µ 1 −c 2 µ 2 ) γ 1 −γ 2 φ L ⋆ − θ γ 1 r σ 2 2γ 1 φ x− θ γ 1 r σ 2 2γ 1 (B.35) Then, it is straightforward to verify thatv ′ 1 andv ′ 2 in (B.34) and (B.35) solve σ 2 2 v ′′ 1 (x)+(θ−γ 2 x)v ′ 1 (x)+c 2 µ 2 γ 2 x = κ ⋆ ,0≤x<L ⋆ (B.36) σ 2 2 v ′′ 2 (x)+(θ−γ 1 x)v ′ 2 (x)+c 1 µ 1 γ 1 x = κ ⋆ ,L ⋆ ≤x, with initial conditions v ′ 1 (0) = 0 andv ′ 2 (L ⋆ ) = c 1 µ 1 γ 1 −c 2 µ 2 γ 2 γ 1 −γ 2 and that v ′ 1 (L ⋆ ) =v ′ 2 (L ⋆ ) = c 1 µ 1 γ 1 −c 2 µ 2 γ 2 γ 1 −γ 2 > 0. (B.37) It follows from (B.36) and (B.37) thatv ′′ 1 (L ⋆ ) =v ′′ 2 (L ⋆ ). Therefore, v(x) := R x 0 v ′ 1 (y)dy x<L ⋆ R L ⋆ 0 v ′ 1 (y)dy + R x L ⋆ v ′ 2 (y)dy x≥L ⋆ is twice-continuously differentiable. The first step in showing(v,κ ⋆ ) solves the Bellman equation is to show that (2.20) is satisfied. For this, observe that φ x,v ′ (x) = θv ′ (x)+ inf q∈A(x) γ 1 µ 1 q 1 c 1 µ 1 −v ′ (x) + γ 2 µ 2 q 2 c 2 µ 2 −v ′ (x) = θv ′ (x)+γ 2 x(c 2 µ 2 −v ′ (x)) ifv ′ (x)< c 1 µ 1 γ 1 −c 2 µ 2 γ 2 γ 1 −γ 2 θv ′ (x)+γ 1 x(c 1 µ 1 −v ′ (x)) ifv ′ (x)≥ c 1 µ 1 γ 1 −c 2 µ 2 γ 2 γ 1 −γ 2 . 144 Therefore, if we can show that v ′ 1 (x)≤ c 1 µ 1 γ 1 −c 2 µ 2 γ 2 γ 1 −γ 2 for allx<L ⋆ (B.38) and v ′ 2 (x)> c 1 µ 1 γ 1 −c 2 µ 2 γ 2 γ 1 −γ 2 for allx≥L ⋆ , (B.39) it follows from (B.36) that (2.20) holds; i.e., that σ 2 2 v ′′ (x)+φ x,v ′ (x) =κ ⋆ for allx≥ 0. Furthermore, it also follows that q ⋆ (x) = (0,xµ 2 ), 0≤x<L ⋆ (xµ 1 ,0), x≥L ⋆ To see (B.39) holds, replaceκ ⋆ in (B.35) byκ 2 (L ⋆ ) in Lemma 2.4.1 so that v ′ 2 (x) =c 1 µ 1 −Ψ x− θ γ 1 q σ 2 2γ 1 γ 2 (c 2 µ 2 −c 1 µ 1 ) γ 1 −γ 2 φ L ⋆ − θ γ 1 r σ 2 2γ 1 1−Φ L ⋆ − θ γ 1 r σ 2 2γ 1 . (B.40) Thenv ′ 2 (x) for allx≥L ⋆ is strictly increasing function inx because−Ψ(x) is strictly increasing inx for allx∈R. So (B.39) holds since v ′ 2 (L ⋆ ) = c 1 µ 1 γ 1 −c 2 µ 2 γ 2 γ 1 −γ 2 . The argument that (B.38) holds is by contradiction. Suppose that there existsx 0 <L ⋆ for whichv ′ 1 (x 0 )> (c 1 µ 1 γ 1 −c 2 µ 2 γ 2 )/(γ 1 −γ 2 ). Then, there exists x 1 ∈ (x 0 ,L ⋆ ) such that v ′′ 1 (x 0 ) < 0 and v ′ 1 (x 1 ) = (c 1 µ 1 γ 1 −c 2 µ 2 γ 2 )/(γ 1 −γ 2 ). From (B.36), σ 2 2 v ′′ 1 (x 1 )+(θ−γ 2 x 1 )v ′ 1 (x 1 )+c 2 µ 2 γ 2 x 1 = κ ⋆ σ 2 2 v ′′ 1 (L ⋆ )+(θ−γ 2 L ⋆ )v ′ 1 (L ⋆ )+c 2 µ 2 γ 2 L ⋆ = κ ⋆ , 145 so that, subtracting the bottom equation from the top shows that σ 2 2 v ′′ 1 (x 1 )−v ′′ 1 (L ⋆ ) + γ 1 γ 2 (c 2 µ 2 −c 1 µ 1 )(x 1 −L ⋆ ) γ 1 −γ 2 = 0. This is a contradiction becausev ′′ 1 (L ⋆ ) =v ′′ 2 (L ⋆ )≥ 0 (sincev ′ 2 is a strictly increasing function),v ′′ 1 (x 1 )< 0, andx 1 <L ⋆ implies that σ 2 2 v ′′ 1 (x 1 )−v ′′ 1 (L ⋆ ) + γ 1 γ 2 (c 2 µ 2 −c 1 µ 1 )(x 1 −L ⋆ ) γ 1 −γ 2 < 0. The second step in showing (v,κ ⋆ ) solves the Bellman equation is to show that (2.21) is satisfied; i.e., that (i)v ′ (0) = 0; (ii)v ′ is bounded; (iii)v ′ is non-negative. For (i), recall thatv ′ 1 (0) = 0. For (ii), we can show thatv ′ 2 (x)→c 1 µ 1 asx→∞ by noting thatΨ(x)→ 0 asx→∞ in (B.40). For (iii), by calculus and algebra, it can be shown thatv ′ 1 in (B.34) has the equivalent representation v ′ 1 (x) = R x 0 2 σ 2 (κ ⋆ −c 2 µ 2 γ 2 y)φ y− θ γ 2 r σ 2 2γ 2 dy φ x− θ γ 2 r σ 2 2γ 2 Then, depending on how largeκ ⋆ is,v ′ 1 is either non-negative for allx≥ 0 or there exists a uniquex 0 > 0 such thatv ′ 1 (x)> 0 for allx<x 0 andv ′ 1 (x)< 0 for allx>x 0 . Sincev ′ 1 (L ⋆ ) =v ′ 2 (L ⋆ ) = c 1 µ 1 γ 1 −c 2 µ 2 γ 2 γ 1 −γ 2 > 0,x 0 >L ⋆ if suchx 0 exists. Thereforev ′ 1 is non-negative for allx<L ⋆ . Note thatv ′ 2 (x)> 0 for allx>L ⋆ becausev ′ 2 (L ⋆ )> 0 andv ′ 2 is strictly increasing. Thereforev ′ is non-negative. 146 B.2 Proofs of lemmas Proof of Lemma B.1.1: It is enough to show that the functionφ(x,w) is jointly continuous in(x,w) and Lipschitz continuous inw. This is because the fact that there exists a unique solutionw κ of IVP(κ) on a compact interval ofR + then follows from Picard’s existence theorem (see, for example, Proposition 2.4 of Weber (2011)). The fact that there exists a unique solutionw κ of IVP(κ) onR + follows an iteration argument, which we omit because it is standard. To see thatφ(x,w) is continuous in (x,w), letǫ> 0 be arbitrarily small. We show that, fori∈{1,2}, ifx i ≥ 0 andw i ≥ 0 are such that max(|x 1 −x 2 |,|w 1 −w 2 |)<δ for δ :=ǫ 1 |θ|+x 2 P N k=1 h k µ k x 2 f k + P N k=1 (c k µ k +v 1 )h k µ k max(x 1 ,x 2 ) f k , then |φ(x 1 ,w 1 )−φ(x 2 ,w 2 )|<ǫ. Define q 1⋆ 1 ,q 1⋆ 2 ,...,q 1⋆ N := argmin q∈A(x 1 ) ( N X k=1 m k (q k ) c k − w 1 µ k ) so that φ(x 1 ,w 1 ) =θw 1 + N X k=1 m k q 1⋆ k c k − w 1 µ k . Next define ˜ q 2 1 ,˜ q 2 2 ,...,˜ q 2 N := q 1⋆ 1 −µ 1 (x 1 −x 2 ),q 1⋆ 2 ,...,q 1⋆ N ifx 2 ≥x 1 or x 2 <x 1 andq 1⋆ 1 >µ 1 (x 1 −x 2 ) q 1⋆ 1 −pµ 1 (x 1 −x 2 ),q 1⋆ 2 −(1−p)µ 2 (x 1 −x 2 ),q 1⋆ 3 ,...,q 1⋆ N otherwise, where p :=p(x 1 ,x 2 )∈P := 0, q 1⋆ 1 µ 1 (x 1 −x 2 ) 147 is such that ˜ q 2 k ≥ 0 fork∈{1,2,...,N} so that ˜ q 2 1 ,˜ q 2 2 ,...˜ q 2 N ∈A(x 2 ). Note that such a p ∈ P can be found because for any p ∈ P, ˜ q 2 1 ≥ 0, and, for fixed x 1 and x 2 , ˜ q 2 2 is a continuous function ofp having ˜ q 2 1 (p)→µ 2 x 2 > 0 asp→q 1⋆ 1 /(µ 1 (x 1 −x 2 )). Without loss of generality, assume(x 1 ,w 1 ) and(x 2 ,w 2 ) are such that φ(x 1 ,w 1 )<φ(x 2 ,w 2 ). Then φ(x 2 ,w 2 )−φ(x 1 ,w 1 ) ≤ |θ||w 2 −w 1 |+ N X k=1 m k ˜ q 2 k c k − w 2 µ k − N X k=1 m k q 1⋆ k c i − w 1 µ i = |θ||w 2 −w 1 |+ N X k=1 c k |m k ˜ q 2 k −m k q 1⋆ k | + N X k=1 1 µ k |w 1 m k q 1⋆ k −w 1 m k ˜ q 2 k +w 1 m k ˜ q 2 k −w 2 m k ˜ q 2 k | ≤ |θ|+ N X k=1 1 µ k m k ˜ q 2 k ! |w 2 −w 1 |+ N X k=1 c k + w 1 µ k |m k ˜ q 2 k −m k q 1⋆ k |. Since m k ˜ q 2 k ≤h k µ k x 2 f k µ k x 2 ,k∈{1,2,...,N}, and |m k ˜ q 2 k −m k q 1⋆ k | = f k Z q 1⋆ k /f k ˜ q 2 k /f k h i (u)du ≤h k µ k max(x 1 ,x 2 ) f k µ k |x 1 −x 2 |, where the last inequality follows from the definition of ˜ q 2 1 ,˜ q 2 2 ,...,˜ q 2 N , it follows that φ(x 2 ,w 2 )−φ(x 1 ,w 1 ) ≤ |θ|+x 2 N X k=1 h k µ k x 2 f k ! |w 2 −w 1 | (B.41) + N X k=1 (c k µ k +w 1 )h k µ k max(x 1 ,x 2 ) f k |x 1 −x 2 |, 148 so that |φ(x 2 ,w 2 )−φ(x 1 ,w 1 )|<ǫ. To see thatφ(x,w) is Lipschitz continuous inw, note that it follows from (B.41) that whenx 2 =x 1 = x≥ 0, |φ(x 1 ,w 1 )−φ(x 2 ,w 2 )|≤ |θ|+x 2 N X k=1 h k µ k x 2 f k ! |w 2 −w 1 |. Proof of Lemma 2.4.1: Note thatc 1 µ 1 γ 1 >c 2 µ 2 γ 2 andc 2 µ 2 >c 1 µ 1 implies thatγ 1 >γ 2 , γ 1 (c 1 µ 1 −c 2 µ 2 ) γ 1 −γ 2 +c 2 µ 2 = c 1 µ 1 γ 1 −c 2 µ 2 γ 2 γ 1 −γ 2 > 0, and c 2 µ 2 −c 1 µ 1 γ 1 −γ 2 > 0. Then, lim L↓0 κ 1 (L) = ∞ lim L↑∞ κ 1 (L) = c 2 µ 2 θ+ c 2 µ 2 q 2 σ 2 γ 2 1 Ψ q 2 σ 2 γ 2 θ lim L↓0 κ 2 (L) = c 1 µ 1 θ+ 1 q 2 σ 2 γ 1 γ 2 (c 2 µ 2 −c 1 µ 1 ) γ 1 −γ 2 Ψ − θ γ 1 r σ 2 2γ 1 lim L↑∞ κ 2 (L) = ∞. Sinceκ 1 (L) andκ 2 (L) are continuous inL, we conclude that there existsL> 0 such thatκ 1 (L) =κ 2 (L). Sinceκ 2 (L)> 0 for allL≥ 0, it must be that atL such thatκ 1 (L) =κ 2 (L),κ 1 (L) =κ 2 (L)> 0. 149 It is useful for the proof of Proposition 2.4.4 to note that Ψ ′ (x) = 1−Φ(x) φ(x) ′ = −φ 2 (x)−(1−Φ(x))(−x)φ(x) φ 2 (x) =− (−x)Φ(−x)+φ(−x) φ(−x) < 0 for allx ∈ R (see, for example, Lemma EC.3 in Armony and Ward (2010) to see thatxΦ(x)+φ(x) > 0 for allx∈R), so that Ψ is a strictly decreasing function. Also, Ψ(x)→∞ asx→−∞ and Ψ(x)→ 0 as x→∞. Proof of Lemma B.1.2: We already proved thatφ(x,w) is jointly continuous in (x,w) in the proof of Lemma B.1.1. Therefore, we can apply Theorem 2.1 in Hartman (1964) to conclude thatw κ (x) is jointly continuous in(x,κ). Proof of Lemma B.1.3: Letw 1 >w 2 > 0. Then, recalling thatθ≤ 0, φ(x,w 2 ) = θw 2 + inf q∈A(x) ( N X k=1 m k (q k ) c k − w 2 µ k ) > θw 1 + inf q∈A(x) ( N X k=1 m k (q k ) c k − w 1 µ k ) = φ(x,w 1 ) Proof of Lemma B.1.4: The proof for Lemma B.1.4 is identical to the proof of Lemma 4 in the appendix of Rubino and Ata (2009), and so is omitted. 150 Proof of Lemma B.1.5: Sinceφ(0,0) = 0, it follows from the definition of IVP(κ) that 1 2 σ 2 w ′ κ (0) =κ> 0. Hence, the fact thatw ′ κ is continuous onR + and the existence ofx κ such thatw ′ κ (x κ ) < 0 imply that the set E := x> 0 :x<x κ ,w ′ κ (x) = 0, andw ′ κ (y)< 0 for ally∈ (x,x κ ) is not empty and x 0 := infE <x κ . To complete the proof, we have to show two things; (i) E := {x>x κ :w ′ κ (x) = 0} is empty and (ii) w ′ κ (x)≥ 0 for allx∈ [0,x 0 ]. The argument thatE is empty is by contradiction. Suppose not; thatE is not empty. We can define x κ := infE >x 0 . Then, by the definitions ofx 0 andx κ , we have w κ (x 0 )>w κ (x κ ) (B.42) and w ′ κ (x 0 ) =w ′ κ (x κ ) = 0. which from the definition of IVP(κ) implies that φ(x 0 ,w κ (x 0 )) =φ(x κ ,w κ (x κ )) =κ. (B.43) Assume we can show that φ(x 0 ,w κ (x 0 ))≤φ(x κ ,w κ (x 0 )). (B.44) 151 Then, the contradiction arises because (B.42) and so by Lemma B.1.3 φ(x κ ,w κ (x 0 ))<φ(x κ ,w κ (x κ )). (B.45) In summary, (B.44) and (B.45) imply that φ(x 0 ,w κ (x 0 ))<φ(x κ ,w κ (x κ )), which contradicts (B.43). To establish (B.44), assume we can show that w κ (x 0 )< min(c 1 µ 1 ,c 2 µ 2 ,...,c N µ N ). (B.46) Let q κ⋆ := argmin q∈A(x κ ) ( N X i=1 m i (q i ) c i − 1 µ i w κ (x 0 ) ) so that φ(x κ ,w κ (x 0 )) =θw κ (x 0 )+ N X i=1 m i (q κ⋆ i ) c i − 1 µ i w κ (x 0 ) . Sincex κ > x 0 , there existsq 0 ∈ A(x 0 ) such thatq 0 i ≤ q κ⋆ i for alli ∈ {1,2,...,N} with at least one of the inequalities being strict. This implies thatm i q 0 i ≤m i (q κ⋆ i ) for alli∈{1,2,...,N} becauseh i , the hazard rate function of inter-abandonment time of classi customers, is increasing. Since (B.46) implies that (c i −w κ (x 0 )/µ i )> 0 for alli∈{1,2,...,N}, it follows that φ(x κ ,w κ (x 0 ))≥θw κ (x 0 )+ N X i=1 m i q 0 i c i − 1 µ i w κ (x 0 ) . (B.47) The definition ofφ implies that θw κ (x 0 )+ N X i=1 m i q 0 i c i − 1 µ i w κ (x 0 ) ≥φ(x 0 ,w κ (x 0 )). (B.48) Together, (B.47) and (B.48) show (B.44). 152 Finally, it remains to establish (B.46). First suppose that w κ (x 0 ) ≥ max(c 1 µ 1 ,c 2 µ 2 ,...,c N µ N ). Then,c i −w κ (x 0 )/µ i ≤ 0 for alli∈{1,2,...,N}, so that for anyq∈A(x 0 ) θw κ (x 0 )+ N X i=1 m i (q i ) c i − 1 µ i w κ (x 0 ) ≤ 0, which impliesφ(x 0 ,w κ (x 0 ))≤ 0. This is a contradiction because it follows from IVP(κ) that κ = σ 2 2 w ′ κ (x 0 )+φ(x 0 ,w κ (x 0 )) =φ(x 0 ,w κ (x 0 ))≤ 0, but we have assumed κ > 0. Next, let c i µ i for i ∈ {1,2,...,N} be ordered so that c i 1 µ i 1 ≤ c i 2 µ i 2 ··· ≤ c i N µ i N . We may assume that at least one of inequalities is strict because otherwise we have min(c 1 µ 1 ,c 2 µ 2 ,...,c N µ N ) = max(c 1 µ 1 ,c 2 µ 2 ,...,c N µ N ) and the Proof is completed. Let s ∈ {1,2,...,N−1} be the largest value withc is µ is <c i s+1 µ i s+1 and supposec is µ is ≤w κ (x 0 )<c i s+1 µ i s+1 . Then, c i j µ i j − w κ (x 0 ) > 0 for all j ∈ {s+1,s+2,...,N} and c i j µ i j − w κ (x 0 ) ≤ 0 for all j∈{1,2,...,s} so that, φ(x 0 ,w κ (x 0 ))≤θw κ (x 0 )+ N X i=1 m i (q s i ) c i − 1 µ i w κ (x 0 ) ≤ 0, where q s ∈ A(x 0 ) such that q s i j = 0 for all j ∈ {s+1,s+2,...,N}. This is again a contradiction for the same reason as earlier in this paragraph. If we repeat this argument, we can prove thatw κ (x 0 ) < min(c 1 µ 1 ,c 2 µ 2 ,...,c N µ N ) and this completes the Proof of the first part. To complete the Proof of Lemma, it remains to prove thatw ′ κ (x)≥ 0 for allx∈ [0,x 0 ]. Suppose there existsx 1 ∈ (0,x 0 ) such thatw ′ κ (x 1 )< 0. Then, by applying the same argument in the Proof of the first part tox 1 , we conclude thatw ′ κ (x)< 0 for allx>x 1 . This is contradiction becausew ′ κ (x 0 ) = 0 andx 0 >x 1 . Proof of Lemma B.1.6: Ifκ∈D, then by definition ofD, there existsx κ > 0 such thatw κ (x) is decreasing ifx > x κ . There are two possibilities: (i)w κ (x) is bounded for allx≥ 0, or (ii)w κ (x)→−∞ asx→∞. We show that case (i) leads to a contradiction, which is enough to complete the proof. 153 Supposew κ (x) is bounded forx≥ 0. Then,lim x→∞ w κ (x) exists and is finite, which impliesw ′ κ (x)→ 0 asx→∞, and so, from the definition of IVP(κ), φ(x,w κ (x))→κ asx→∞. (B.49) Next, sinceκ∈D, there existsw such thatw κ (x)<w for allx> 0. Then, by Lemma B.1.3, φ(x,w κ (x))>φ(x,w) for allx> 0. (B.50) Letx>x κ and define q ⋆ (x) := argmin q∈A(x) ( N X k=1 m k (q k ) c k − w µ k ) so that φ(x,w) = θw+ N X k=1 m k (q ⋆ k (x)) c k − w µ k (B.51) θw+ N X k=1 f k µ k Z q ⋆ k (x)/f k 0 h k (y)dy ! (c k µ k −w). It follows from (B.46) in the proof of Lemma B.1.5 that for x 0 such that w ′ κ (x 0 ) = 0, w κ (x 0 ) < min(c 1 µ 1 ,c 2 µ 2 ,...,c N µ N ), so that we may assumew< min(c 1 ,µ 1 ,c 2 µ 2 ,...,c N µ N ). Then,c k µ k −w> 0 for allk∈{1,2,...,N}, and it follows from (B.51) that φ(x,w)→∞ asx→∞, (B.52) because max(q ⋆ 1 (x),q ⋆ 2 (x),...,q ⋆ N (x)) → ∞ asx → ∞ from the fact that P N k=1 q ⋆ k (x)/µ k = x for everyx> 0. Finally, (B.52) and (B.50) imply that φ(x,w κ (x))→∞ asx→∞, which contradicts (B.49). 154 Proof of Lemma B.1.7: Fixκ> 0. First note that if there existsx 0 > 0 such thatw κ (x 0 )≥ min(c 1 µ 1 ,c 2 µ 2 ,...,c N µ N ), then by the same argument as the one in the second to the last paragraph of the proof of Lemma B.1.5, φ(x 0 ,w κ (x 0 ))≤ 0. (B.53) It follows from (B.53) that 1 2 σ 2 w ′ κ (x 0 ) =κ−φ(x 0 ,w κ (x 0 ))> 0. Since w κ is continuous, and the same argument as above shows that w ′ κ (x) > 0 whenever w κ (x) ≥ min(c 1 µ 1 ,c 2 µ 2 ,...,c N µ N ), it follows thatw ′ κ (x)> 0 for allx≥x 0 . Next, we claim that there can exist at most onex> 0 such thatw κ (x) = min(c 1 µ 1 ,c 2 µ 2 ,...,c N µ N ). To see this, note that the argument in the previous paragraph shows thatw ′ κ (x)> 0 for allx≥x. Finally, to complete the proof, we must show thatw κ (x) is increasing on [0,x). This argument is by contradiction. Assume there existsx 1 such thatw ′ κ (x 1 ) < 0. Then, by Lemma B.1.5, κ ∈ D, and there existsx κ such thatw κ is decreasing on[x κ ,∞). This is a contradiction, because the previous two paragraphs show thatw ′ κ (x)> 0 for allx≥x. 155 Appendix C Technical Appendix to Chapter 3 C.1 Proofs of Main Results Proof of Theorem 3.3.1: Proof of part (i). In the case whenβ≥ 0 and the hazard rate function of the patience time distribution of each class is increasing, the existence part of this theorem is equivalent to Theorem 2.4.3 in Chapter 2. A careful examination of the proofs in Chap B, which considers a single server setting and hence the domain of the HJB equation isR + instead of(−∞,d), indicates that that proof does not rely on the structure of the hazard rate function. So, the proof of part (i) forβ≥ 0 can be completed by following the proofs in Chap B and we focus on the case whenβ < 0 for proving the existence of the HJB solution. To prove the existence, consider a family of ODEs indexed byκ≥ 0: λv ′ (x)+ √ µ [x] − −β √ λ √ µv (x)+λmin q∈A φ(x,v(x),q) =κ such that sup x≤0 |v(x)|<∞. (C.1) We obtain (C.1) by modifying the boundary conditionsup x<d |v(x)|<∞ in (3.16). The following lemma proves the existence of a uniquev : (−∞,d)→R that solves (C.1), which is denoted byv κ , and provides useful properties ofv κ . Lemma C.1.1. For eachκ ≥ 0, the ODE (C.1) has a unique solutionv κ . v κ (x) is jointly continuous in (κ,x)∈ R + ×(−∞,d), and is non-negative and strictly increasing inx≤ 0. Further,v κ 1 (x) > v κ 2 (x) for allx∈ (−∞,d) andκ 1 >κ 2 andv ′ κ (x) is continuous inx∈ (−∞,d). To proceed, letr := min{r 1 ,...,r K } and D := ( κ≥ 0 : sup x∈[0,d) v κ (x)<r ) (C.2) 156 andU :=D c ∩R + . Given the properties of{v κ } κ≥0 in Lemma C.1.1, we prove the theorem by showing that there exists a uniqueκ∈ (0,∞) such thatsup x∈(−∞,d) |v κ (x)|≤r andv κ (x) is increasing inx∈ (−∞,d). To do so, we need the following lemmas. Lemma C.1.2. NeitherU norD is empty andsupD = infU ∈ (0,∞). Lemma C.1.3. Suppose there existsx 1 ≥ 0 such thatv ′ κ (x 1 ) < 0. Then,v κ (x) is strictly decreasing in x>x 1 . Lemma C.1.4. lim x→d v κ (x) is either−∞,r, or∞. Lemma C.1.5. Ifκ∈D, thenlim x→d v κ (x) =−∞. Lemma C.1.6. Ifκ∈U, thenlim x→d v κ (x)≥r. Lemma C.1.7. min q∈A φ(x,w,q) is jointly continuous in(x,w)∈ (0,d)×R. By Lemma C.1.2,κ ⋆ := infU ∈ (0,∞). Then, the continuity ofv κ inκ derived in Lemma C.1.1 and Lemmas C.1.5-C.1.6 imply thatκ ⋆ ∈ U. Also, sup x∈[0,d) v κ ⋆ (x) = r follows by the continuity ofv κ in κ. It only remains to show thatv ′ κ ⋆ (x) ≥ 0 for allx ∈ [0,d). Suppose there exists ¯ x ∈ [0,d) such that v ′ κ ⋆ (¯ x)< 0. Becausesup x≥0 v κ ⋆ (x) =r, we know thatv κ ⋆ (¯ x)≤r. This implies thatlim x→d v κ ⋆ (x)<r by Lemmas C.1.3 and C.1.4. However, this contradicts to Lemma C.1.6 becauseκ ⋆ ∈U. Hencev ′ κ ⋆ (x)≥ 0 for allx ∈ [0,d). Finally, we use Lemma C.1.4 to prove the uniqueness of the HJB solution by showing that for any κ > κ ⋆ , lim x→d v κ (x) > r. Suppose on the contrary that there exists κ > κ ⋆ such that lim x→d v κ (x) =r. Then, for thisκ, by Lemma C.1.7 and by notinglim x→d v ′ κ (x) = 0, we establish β p λµr =κ by taking lim x→d on the both hand sides of (C.1). But, this is a contradiction because by repeating the limiting argument we reachβ √ λµr =κ ⋆ butκ ⋆ <κ. Proof of part (ii). The following lemma is useful in this proof. Lemma C.1.8. For the diffusion process,Z, defined in (3.11), we have lim t→∞ E[|Z(t)|] t = 0. 157 To show the optimality ofq ⋆ , we show κ ⋆ = lim T→∞ E " X k∈K r k λ T Z T 0 m k [Z ⋆ (t)] + q ⋆ k [Z ⋆ (t)] + λ ! dt # , (C.3) κ ⋆ ≤ limsup T→∞ E " X k∈K r k λ T Z T 0 m k [Z(t)] + q k [Z(t)] + λ ! dt # , (C.4) for anyq∈A(∞) andZ defined in (3.11). To proceed, letu(x) := R x −∞ v κ ⋆ (y)dy. By using Ito’s lemma onu(Z ⋆ (t)) andu(Z(t)), we achieve κ ⋆ = E[u(Z ⋆ (T))] T +E " X k∈K r k λ T Z T 0 m k [Z ⋆ (t)] + q ⋆ k [Z ⋆ (t)] + λ ! dt # , (C.5) κ ⋆ ≤ E[u(Z(T))] T +E " X k∈K r k λ T Z T 0 m k [Z(t)] + q k [Z(t)] + λ ! dt # . (C.6) Given (C.5) and (C.6), the proof of this theorem is complete by Lemma C.1.8 becauseu(x) grows at most linearly inx as shown in part (i) thatsup x∈(−∞,d) v κ ⋆(x)≤r. Proof of Proposition 3.4.1: Observe that when m k (x) = γ k x for some γ k > 0, the optimization min q∈A φ(x,v ⋆ (x),q) forx > 0 reduces to min q∈A x X k∈K (r k −v ⋆ (x))γ k q k . Hence, q ⋆ k (x) = 1 if and only if (r k −v ⋆ (x))γ k < (r j −v ⋆ (x))γ j for allj6=k, (C.7) which is equivalent to (r k γ k −r j γ j )< (γ k −γ j )v ⋆ (x) for allj 6=k. This implies that whenJ 1 ={K}, q ⋆ k (x) = 0 for anyk≤K−1 andx> 0 (recall we haver 1 γ 1 ≥r 2 γ 2 ≥···≥r K γ K ), so thatq ⋆ is a static priority withq ⋆ K (x) = 1 forx> 0. SupposeJ 1 \{K}6=∅. First, we show that for anyj ∈J 1 \J 2 ,q ⋆ j (x) = 0 forx> 0. To see this note that for suchj, we have r 1 γ 1 −r j γ j γ 1 −γ j < r 1 γ 1 −r J 1 γ J 1 γ 1 −γ J 1 < r j γ j −r J 1 γ J 1 γ j −γ J 1 . (C.8) 158 Then, if v ⋆ (x) ≤ r 1 γ 1 −r j γ j γ 1 −γ j , q ⋆ j (x) cannot be 1 because (r j γ j −r J 1 γ J 1 ) > (γ j −γ J 1 )v ⋆ (x) by (C.8) which is equivalent to (r J 1 −v ⋆ (x))γ J 1 < (r j −v ⋆ (x))γ j . Similarly, we can argue thatq ⋆ j (x) = 0 for x> 0 using (C.8). Next, we show that for any j ∈ J 2 \J 3 , q ⋆ j (x) = 0 for all x > 0. Let V be the set of vertices of conv(C). For suchj, it is straight forward to check that there must exists two points inV, namely (r i ,r i γ i ) and(r n ,r n γ n ), such thati<n r i γ i −r j γ j γ i −γ j < r i γ i −r n γ n γ i −γ n < r j γ j −r n γ n γ j −γ n . Then, by using the same argument given early in this paragraph, we conclude thatq ⋆ j (x) = 0 forx> 0. To complete the proof, note that becauseV is the set of vertices ofconv(C), we have 0 =T J 3 +1 <T J 3 <···<T 2 <T 1 =∞. Supposev ⋆ (x)∈ (T j+1 ,T j ] for somej∈{1,...,J 3 }. Then, for suchx> 0, we have v ⋆ (x)< r k γ k −r j γ j γ k −γ j for allk∈{1,...,j−1}, v ⋆ (x)> r j γ j −r k γ k γ j −γ k for allk∈{j +1,...,J 3 }. From the above display, we can deduceq ⋆ j (x) = 1 by (C.7). Finally, suchq ⋆ is a threshold control or a static priority becausev ⋆ (x) is increasing inx by Theorem 3.3.1. Proof of Proposition 3.4.2: We first show that under the condition stated in the proposition,q ⋆ is neither a static priority nor a threshold control. Then, we complete the proof by establishing that whenq ⋆ is neither a static priority nor a threshold control, the cost achieved under any static priority or threshold control is strictly greater than the optimal costκ ⋆ . For the first part, let class 1 and class 2 have patience time distributions that satisfy conditions in the statement of the proposition. Then, there existsa> 0 such that for anyx∈ (0,a), neitherq ⋆ 1 (x) norq ⋆ 2 (x) 159 can take 0 or 1. This is because we can finda> 0 such that forx∈ (0,a), the optimizer for inf q∈[0,1] z(q) is never 0 or 1 where z(q) := (r 1 −v ⋆ (x))m 1 x(1−q) λ +(r 2 −v ⋆ (x))m 2 xq λ . asz ′ (0)< 0 andz ′ (1)> 0. For the second part, letκ ¯ q be the objective function value for (3.12) andZ ¯ q be the diffusion process in (3.11) under ¯ q which is either a threshold control or static priority. We want to prove thatκ ¯ q >κ ⋆ . Suppose on the contrary thatκ ¯ q = κ ⋆ . Recallu(x) = R x −∞ v κ ⋆ (y)dy. By applying Ito’s lemma onu(Z ¯ q (t)), we have E[u(Z ¯ q (t))−u(Z ¯ q (0))] t +E " X k∈K r k λ t Z t 0 m k [Z ¯ q (s)] + ¯ q k [Z ¯ q (s)] + λ ! ds # =E 1 t Z t 0 H(Z ¯ q (s),¯ q)ds , (C.9) where forz∈ (−∞,d) andq∈A H(z,q) :=λv ⋆′ (z)+v ⋆ (z) −β p µλ +µ [z] − +λ X k∈K (r k −v ⋆ (z))m k [z] + q k [z] + λ ! . By Lemma C.1.8, thelimsup t→∞ of the first term in the left hand side of (C.9) is 0. Also note that the cost under ¯ q isκ ⋆ , so that thelimsup t→∞ of the second term in the left hand side of (C.9) isκ ⋆ . Hence, we have κ ⋆ = limsup t→∞ E 1 t Z t 0 H(Z ¯ q (s),¯ q)ds ≥ liminf t→∞ E 1 t Z t 0 H(Z ¯ q (s),¯ q)ds (a) ≥ E liminf t→∞ 1 t Z t 0 H(Z ¯ q (s),¯ q)ds , (C.10) where(a) follows by Fatou’s lemma. Here, we can use Fatou’s lemma because H(Z ¯ q (s),¯ q) (a) ≥ H(Z ¯ q (s),q ⋆ ) (b) = κ ⋆ (c) > 0. (C.11) In (C.11), (a) follows becauseq ⋆ (x) is an optimizer of argmin q∈A (r k −v ⋆ (x))m k xq k λ forx > 0, (b) follows because(v ⋆ ,κ ⋆ )solves (3.16), and(c) follows because of Lemma C.1.2. 160 To proceed, note that in the proof of Lemma C.1.8 ((C.24) and (C.25)), we establish thatE[T 0 ] < ∞ andE[τ]<∞, whereT 0 := inf{t≥ 0 :Z(t) = 0} andτ is the length of i.i.d regenerative cycles in which the diffusion process departs from 0, reaches some constant greater than 0 but less thand(q), and returns back to 0. Also, note that by the first equality in (C.10), we know that there existsT <∞ such that E 1 t Z t 0 H(Z ¯ q (s),¯ q)ds <κ ⋆ +ǫ fort>T and someǫ∈ (0,∞). (C.12) Jointly, E[T 0 ] < ∞, E[τ] < ∞, and (C.12) are sufficient to satisfy conditions stated in Theorem 2.3 of Sigman (2011). Hence, we can use Theorem 2.5 of Chapter 13 of Sigman (2011) to conclude E liminf t→∞ 1 t Z t 0 H(Z ¯ q (s),¯ q)ds =E H Z ⋆ ¯ q ,¯ q . (C.13) Putting (C.10), (C.12), and(C.13), we have κ ⋆ ≥E H Z ⋆ ¯ q ,¯ q ≥E H Z ⋆ ¯ q ,q ⋆ =κ ⋆ , which implies H Z ⋆ ¯ q ,¯ q = H Z ⋆ ¯ q ,q ⋆ almost surely and hence ¯ q Z ⋆ ¯ q is an optimizer of min z∈A φ Z ⋆ ¯ q ,v ⋆ Z ⋆ ¯ q ,q almost surely. This is contradiction because we already argue in the second paragraph of this proof that ¯ q(x) is not an optimizer for min z∈A φ(x,v ⋆ (x),q) for allx∈ (0,a). There- fore, we must haveκ ¯ q >κ ⋆ . Proof of Proposition 3.5.1: Proof of part (i). Observe that r 1 −v ⋆ (x) r 2 −v ⋆ (x) m 1 (x/λ) x/λ in (3.25) is strictly decreasing inx and converges to 0 asx grows. This is because (i) m 1 (x/λ) x/λ is decreasing inx whenh 1 (x) is decreasing inx, and (ii) r 1 −v ⋆ (x) r 2 −v ⋆ (x) is strictly decreasing inx and converges to 0 asx grows by Theorem 3.3.1. Also, observe that by (3.26), we have lim x→∞ r 1 −v ⋆ (x) r 2 −v ⋆ (x) m 1 (x/λ) x/λ >γ 2 . Therefore, q ⋆ = q 1 T for some T ∈ (0,∞) as there must exists a unique T ∈ (0,∞) such that r 1 −v ⋆ (T) r 2 −v ⋆ (T) m 1 (T/λ) Tλ =γ 2 , r 1 −v ⋆ (x) r 2 −v ⋆ (x) m 1 (x/λ) x/λ >γ 2 forx<T , and r 1 −v ⋆ (x) r 2 −v ⋆ (x) m 1 (x/λ) x/λ <γ 2 otherwise. If (3.26) is not satisfied, then r 1 −v ⋆ (x) r 2 −v ⋆ (x) m 1 (x/λ) x/λ <γ 2 for allx> 0, and thereforeq ⋆ =q 2 S . 161 Proof of part (ii). The proof of this part is identical to part (i) and is omitted. Proof of Proposition 3.5.2: First, observe that lim x→0 r 1 −v ⋆ (x) r 2 −v ⋆ (x) m 1 (x/λ) x/λ >γ 2 because(r 1 −v ⋆ (0))h 1 (0)> (r 2 −v ⋆ (0))γ 2 . Therefore, there must existT 1 ∈ (0,∞) such thatq ⋆ (x) = (0,1) forx∈ (0,T 1 ) by (3.25). Also, observe that lim x→∞ r 1 −v ⋆ (x) r 2 −v ⋆ (x) m 1 (x/λ) x/λ =∞. This is because lim x→∞ m 1 (x/λ) x/λ > 0 and lim x→∞ r 1 −v ⋆ (x) r 2 −v ⋆ (x) =∞ by the conditionr 1 > r 2 which implies lim x→∞ v ⋆ (x) = r 2 by Theorem 3.3.1. Hence, there must existT 2 <∞ such thatT 1 ≤ T 2 andq ⋆ (x) = (0,1) forx∈ [T 2 ,∞) by (3.25). The above paragraph establishes the structure of an optimal control of the DCP (3.13) and also the fact that any threshold control cannot solve the HJB (3.16). Then, using the proof of the second part of Proposition 3.4.2, we can conclude that the objective function value of (3.12) under any threshold control is strictly greater thanκ ⋆ , establishing the sub-optimality of threshold control for the DCP (3.13). Proof of Proposition 3.5.3: Let c(q) := (r 1 −v ⋆ (x))a 1 λ Z xq 1 /(a 1 λ) 0 h 1 (u)du+(r 2 −v ⋆ (x))xγ 2 (1−q) forq∈ [0,1]. It is straightforward to check thatc(q) is convex inq whenh 1 (x) is increasing. Observe that c ′ (q) = (r 1 −v ⋆ (x))xh 1 xq 1 a 1 λ −(r 2 −v ⋆ (x))xγ 2 , so thatq ∈ [0,1] that minimizesc(q) is given by min n 1, λa 1 x h −1 1 r 2 −v ⋆ (x) r 1 −v ⋆ (x) γ 2 o . Therefore,q ⋆ satisfies (3.28). Observe thatq ⋆ 6=q 1 S andq ⋆ / ∈{q i T } fori∈{1,2} andT ∈ (0,∞) because λa 1 x h −1 1 r 2 −v ⋆ (x) r 1 −v ⋆ (x) γ 2 162 takes values greater than 0. In fact the onlyx<∞ such that λa 1 x h −1 1 r 2 −v ⋆ (x) r 1 −v ⋆ (x) γ 2 = 0 is a unique solution (if exists) of r 2 −v ⋆ (x) r 1 −v ⋆ (x) γ 2 =h 1 (0). Under (i), which isr 1 ≥r 2 , λa 1 x h −1 1 r 2 −v ⋆ (x) r 1 −v ⋆ (x) γ 2 is decreasing inx and converges to 0 asx grows by Theorem 3.3.1. Henceq ⋆ 6=q 2 S . Under (ii), which isr 1 ≥ r 2 andG k has the bounded domain, λa 1 x h −1 1 r 2 −v ⋆ (x) r 1 −v ⋆ (x) γ 2 is smaller than 1 for largex. Henceq ⋆ 6=q 2 S . Finally, using the proof of the second part of Proposition 3.4.2, we can conclude that the objective function value of (3.12) under threshold control,q 1 S , andq 2 S (whenr 1 ≥ r 2 for the latter) is strictly greater thanκ ⋆ . C.2 Proofs of lemmas We will use the following result in this section. Lemma C.2.1. min q∈A φ(x,w,q) is decreasing inw∈R for allx∈ (0,d) and is increasing inx∈ (0,d) for allw∈R. We omit the details of the proof of this lemma: The first claim of this lemma is a special case of of Lemma B.1.3 in Chapter B (θ = 0 therein) and the second claim of this lemma can be proved by using the similar argument in the proof of Lemma B.1.3 in Chapter B. Proof of Lemma C.1.1: First, we prove that (C.1) has a unique solution forx≤ 0 and this solution satisfies conditions stated in the lemma. Ifx≤ 0, the ODE in (C.1) becomes λv ′ (x)− √ µx +β √ λ √ µv (x) =κ (C.14) 163 because min q∈A φ(x,v(x),q) = 0 for allx ≤ 0. Up to the value ofv(0), the above ODE has a unique solution v(x) = exp √ µ 2 √ µ λ x 2 +2 β √ λ x v(0)− Z 0 x κ λ exp − √ µ 2 √ µ λ y 2 +2 β √ λ y dy . (C.15) For eachκ, we then use the boundary condition in (C.1) to pin downv(0). For the boundary condition in (C.1) to hold, it is necessary thatlim x→−∞ v(x)<∞, which requires v(0) = Z 0 −∞ κ λ exp − √ µ 2 √ µ λ y 2 +2 β √ λ y dy (C.16) becauselim x→∞ exp √ µ 2 √ µ λ x 2 +2 β √ λ x =∞. Under (C.16),v(x) in (C.15) is given by v(x) = κ λ s λ µ Φ q µ λ x+β q λ µ Φ ′ q µ λ x+β q λ µ , (C.17) where Φ denotes the cdf of the standard normal distribution. It is straightforward to check that the function in the right hand side satisfies conditions stated in the lemma. So, we have now proved that the ODE in (C.1) has a unique solution forx≤ 0 and this solution satisfies conditions stated in the lemma. To complete the proof of this lemma, we now considerx> 0. Note that (C.1) is equivalent to λv ′ (x)−β p λµv (x)+λmin q∈A φ(x,v(x),q) =κ, such thatv(0) = κ λ s λ µ Φ(β) Φ ′ (β) (C.18) for which we know the existence of the unique solution that is jointly continuous in (κ,x)∈R + ×R + by Lemma B.1.1 of Chapter B. By connecting v(x) that solves (C.14) forx ≤ 0 and that solves (C.18) for x ≥ 0, we now have the unique solution of (C.1) for eachκ > 0. Let us call this solutionv κ . Note that v ′ κ (x) is continuous inx∈ (−∞,d):v ′ κ (x) is continuous inx< 0 because of (C.17), is continuous inx> 0 by Lemma C.2.1 and (C.18), andlim x↑0 −v ′ κ (x) = lim x↓0 +v ′ κ (x) by (C.14) and (C.18). To prove thatv κ is increasing inκ, observe from (C.18) that we havev κ 1 (0) > v κ 2 (0) forκ 1 > κ 2 . Suppose {x> 0 :v κ 2 (x) =v κ 1 (x)} is not empty and let y be the infimum of this set. Then, because 164 v κ 2 (x)<v κ 1 (x) for allx∈ (0,y), we must havev ′ κ 1 (y)<v ′ κ 2 (y) and henceλv ′ κ 1 (y)−κ 1 <λv ′ κ 2 (y)− κ 2 . However, this is a contradiction because λv ′ κ 1 (y)−κ 1 =β p λµv κ 1 (y)−λmin q∈A φ(y,v κ 1 (y),q) (i) =β p λµv κ 2 (y)−λmin q∈A φ(y,v κ 2 (y),q) =λv ′ κ 2 (y)−κ 2 , where(i) follows becausev κ 1 (y) =v κ 2 (y) by the definition ofy. Proof of Lemma C.1.2: Recall from the proof of Lemma C.1.1 thatv κ (0) = κ λ q λ µ Φ(β) Φ ′ (β) converges to∞ asκ grows,U is non-empty. This also implies thatinfU <∞ Let us now show that 0 ∈ D and hence D is non-empty. Suppose 0 ∈ U and define x 0 := inf{x≥ 0 :v 0 (x) =r}. To proceed, letk := argmin k∈K {r k } and observe that forx∈ (0,d) min q∈A φ(x,r,q) = min q∈A X k6=k (r k −r)m k xq k λ + r k −r m k xq k λ = 0, (C.19) where the last equality follows by settingq k = 0 fork6=k andq k = 1. Then, by (C.18), we have λv ′ 0 (x 0 ) =β p λµr. Because β < 0, the above equation display implies that v ′ 0 (x 0 ) < 0. This is a contradiction because v 0 (0) = 0 by (C.17) which requiresv ′ 0 (x 0 )≥ 0. To prove supD = infU, it suffices to show that ifκ 1 ∈U, thenκ∈U for allκ > κ 1 and ifκ 1 ∈D, thenκ∈D for allκ<κ 1 . However, this is obvious becausev κ is increasing inκ by Lemma C.1.1. We now show that infU > 0. To see this note that v κ (0) < r for any κ < r √ µλ Φ ′ (β) Φ(β) . Hence, in order for such κ to belong to U, we must have v ′ κ (x r ) > 0 where x r := inf{x> 0 :v κ (x)≥r}. To v ′ κ (x r ) > 0, we must haveκ > −β √ λµr becauseλv ′ κ (x r ) = κ +β √ λµr by (C.18) and (C.19). Since β < 0,infU > 0. 165 Proof of Lemma C.1.3: Ifv κ (x) is not strictly decreasing inx > x 1 , there must existx 1 < ˜ x 1 < ˆ x 1 such thatv κ (˜ x 1 ) = v κ (ˆ x 1 ) andv ′ κ (˜ x 1 )≤ 0≤v ′ κ (ˆ x 1 ). From (C.18), we have λv ′ κ (˜ x 1 ) =κ+β p λµv κ (˜ x 1 )−λmin q∈A φ(˜ x 1 ,v κ (˜ x 1 ),q) (a) > κ+β p λµv κ (ˆ x 1 )−λmin q∈A φ(ˆ x 1 ,v κ (ˆ x 1 ),q) =λv ′ κ (ˆ x 1 ), where(a) follows by Lemma C.2.1. However, this is a contradiction as we already havev ′ κ (ˆ x 1 )≥v ′ κ (˜ x 1 ). Hencev κ (x) is strictly decreasing inx>x 1 . Proof of Lemma C.1.4: We prove lim x→d v κ (x) is either−∞,r or∞. Suppose on the contrary lim x→d v κ (x) is a finite number that is notr. Suppose first lim x→d v κ (x)∈ (−∞,r), (C.20) so that lim x→d (r k − v κ (x)) > 0 for all k ∈ K. Also observe that for any q ∈ A, we have lim x→d m k ( xq k (x) a k λ ) =∞ for at least onek∈K. Therefore, we have lim x→d min q(x)∈A φ(x,v κ (x),q(x)) = lim x→d min q(x)∈A X k∈K (r k −v κ (x))m k xq k (x) λ =∞ However, this is a contradiction because the first two terms in (C.18) converge to finite numbers. Suppose on the other hand, lim x→d v κ (x) ∈ (r,∞). Then, we have lim x→d (r k −v ⋆ (x)) < 0 and similarly to the above, we draw a contradiction because lim x→d min q(x)∈A φ(x,v κ (x),q(x)) =−∞. 166 Proof of Lemma C.1.5: Supposev ′ κ (x)≥ 0 for allx∈ [0,d). Then, by the definition ofD,v κ (x) converges from below to some constantc < r asx → d. But, this is a contradiction with the same reasoning used in the first paragraph of the proof of Lemma C.1.4. Therefore, there must exist ¯ x such thatv ′ κ (¯ x) < 0. Given the existence of such ¯ x, we can use Lemma C.1.3 to conclude that lim x→d v κ (x) =−∞ asv κ (x) cannot converge tor as x grows forκ∈D by the definition ofD. Proof of Lemma C.1.6: DefineU 1 :={κ∈U :v κ (0)<r}. Becausev κ (0) is increasing inκ, we have supU 1 = infU\U 1 . Hence to complete the proof, it suffices to prove thatlim x→d v κ (x)≥r forκ∈U 1 becausev κ (x) is increasing in κ forx by Lemma C.1.1. Fixκ 1 ∈ U 1 and definex 1 := inf{x≥ 0 :v κ 1 (x) =r}. Then, we must havev ′ κ 1 (x 1 ) > 0. Suppose there existsx 2 > x 1 such thatv κ (x 2 ) = r. Since min q∈A φ(x,r,q) = 0 by (C.19), we immediately see from (C.18) that v ′ κ 1 (x 1 ) = v ′ κ 1 (x 2 ) > 0. Therefore, for any x > x 1 , we must have v κ 1 (x) ≥ r and lim x→d v κ 1 (x)≥r as desired. Proof of Lemma C.1.7: This lemma is a special case of the first part of the proof of Lemma B.1.2 in Chapter B (θ = 0 therein). So, we omit the proof. Proof of Lemma C.1.8: The proof of this lemma is similar to that of Proposition 2.4.2. The main difference is that in our paper the diffusion process is not reflected at the origin and can take negative values. To complete the proof of the lemma, it suffices to prove that lim t→∞ E h Z(t) 2 i t 2 = 0, (C.21) 167 because by Jensen’s inequality, we have E[|Z(t)|] t ≤ v u u t E h Z(t) 2 i t 2 . To proceed, note thatZ is delayed regenerative process which is composed of the initial cycle during which the process Z reaches 0 from z (recall that we assume Z(0) = z) that is followed by the i.i.d. regenerative cycles that we describe now. Fix ¯ z∈ ((max{z,0},d(q))) (recall thatd(q) is the upper bound of Z under q ∈ A(∞)). Then, in each of these i.i.d. cycles, the process first hits ¯ z from 0 and reaches back to 0 from ¯ z. We denote the length of the initial cycle and the length of an i.i.d. cycle byT 0 andτ, respectively. Given thatZ is a delayed regenerative process, the argument in the proof of Proposition 2.4.2 shows that it is sufficient to prove E h R t 0 Z(s)ds i t → E h R T 0 +τ T 0 Z(s)ds i E[τ] ast→∞ (C.22) to establish (C.21). To prove (C.22), we first establish (i) T 0 <∞ almost surely; (ii) E[τ]<∞; (iii) E h R T 0 +τ T 0 |Z(s)|ds i <∞; (iv) E h R T 0 0 |Z(s)|ds i <∞. Once (i)-(iv) are established, (C.22) follows from Theorem 2.3 of Chapter 13 of Sigman (2011). This is because (i)-(iv) verify conditions listed in that theorem: (i) and (ii) jointly imply thatZ is positive recurrent, (iii) implies that the condition (which isE h R τ 0 +X 1 τ 0 |f (X(s))|ds i < ∞ in Theorem 2.3 of chapter 13 of Sigman (2011)) is satisfied, and (iv) implies that (7) in Sigman (2011) is satisfied. To establish (i)-(iv) above, letT a,b be the time that it takes for the diffusion processZ to reachb if the process initiated ata fora,b∈ (−∞,d(q)), and define E w [·] :=E[·|Z(0) =w] 168 for anyw∈ (−∞,d(q)). Then, we can prove (i)-(iv) by respectively using E[T 0 ] (a) = E z [T z,0 ] =E z Z T z,0 0 1ds ; E[τ] (b) = E 0 [T 0,¯ z ]+E ¯ z [T ¯ z,0 ] =E 0 Z T 0,¯ z 0 1ds +E ¯ z Z T ¯ z,0 0 1ds ; E Z T 0 +τ T 0 |Z(s)|ds (c) ≤ s E Z T 0 +τ T 0 Z(s) 2 ds (d) = s E 0 Z T 0,¯ z 0 Z(s) 2 ds +E ¯ z Z T ¯ z,0 0 Z(s) 2 ds ; E Z T 0 0 |Z(s)|ds (e) ≤ s E Z T 0 0 Z(s) 2 ds (f) = s E z Z Tz,¯ z 0 Z(s) 2 ds +E ¯ z Z T ¯ z,0 0 Z(s) 2 ds . (C.23) In the above display of equations,(a)-(f) follow because: (a) This is obvious asZ(0) =z. (b) Let T 1 := inf{t≥T 0 :Z(t) = ¯ z} and T 2 = inf{t≥T 1 :Z(t) = 0}. Then, we have τ = (T 1 −T 0 )+(T 2 −T 1 ) whereE[T 1 −T 0 ] =E 0 [T 0,¯ z ] andE[T 2 −T 1 ] =E ¯ z [T ¯ z,0 ]. (c) This is by Jensen’s inequality. (d) We can use a similar argument to that in(b) to establish this. (e) This is by Jensen’s inequality. (f) Let ¯ T 1 := inf{t≥ 0 :Z(t) = ¯ z} and ¯ T 2 := inf t≥ ¯ T 1 :Z(t) = 0 . Then, E Z T 0 0 Z(s) 2 ds ≤E " Z ¯ T 1 0 Z(s) 2 ds # +E " Z ¯ T 2 ¯ T 1 Z(s) 2 ds # =E z Z Tz,¯ z 0 Z(s) 2 ds +E ¯ z Z T ¯ z,0 0 Z(s) 2 ds . 169 Using the right hand sides in (C.23), we can finish the proof the lemma by showing E w Z Tw,¯ z 0 Z(s) k ds <∞ forw∈{0,z} andk∈{0,2}, (C.24) E ¯ z Z T ¯ z,0 0 Z(s) k ds <∞ fork∈{0,2}, (C.25) To show (C.24) and (C.25), it is useful to define an operator Au(x) :=λu ′′ (x)+f (x)u ′ (x), for any twice differentiable functionu : (−∞,d(q))→R, where f (x) :=−β p µλ +µ [x] − −λ X k∈K m k [x] + q k [x] + λ ! forq∈A(∞). Establishing (C.24). Fork∈{0,2}, define forx∈ (−∞,d(q)) r(x) := Z x −∞ − 1 λ y k exp − Z x y 1 λ f (z)dz dy u(x) := Z x ¯ z r(y)dy. Then, it is straightforward to check thatu(x) is the solution of Au(x) =−x k such thatu(¯ z) = 0. By applying Ito’s lemma onu(Z(t)), 1 we have u(Z(t∧T w,¯ z )) =u(Z(0))− Z t∧Tw,¯ z 0 Z(s) k ds+λ Z t∧Tw,¯ z 0 u ′ (Z(s))dW (s). To proceed, we argue thatu(x) ≥ 0 forx ≤ ¯ z and the stochastic integral is a martingale. The former is straightforward becauseu(¯ z) = 0 andu ′ (x) < 0 forx ≤ ¯ z. For the latter, observe that for anyx ≤ 0, 1 The function u might not be twice-continuously differentiable because q is not necessarily continuous. But as discussed in Section 4.7 of Harrison (1985), we can still apply Ito’s lemma. 170 r(0)≤ r(x)≤ 0, which implies thatu ′ (x) is uniformly bounded forx≤ 0 becauser(0) >−∞. Also, because u ′ is continuous, u ′ (x) is bounded for x ∈ [0,¯ z]. Therefore as long as w < ¯ z, |u ′ (Z(s))| is bounded fors∈ [0,t∧T w,¯ z ] andλ R t∧Tw,¯ z 0 u ′ (Z(s))dW (s) is a martingale. So, we have u(w) =E w u(Z(t∧T w,¯ z ))+ Z t∧Tw,¯ z 0 Z(s) k ds ≥E w Z t∧Tw,¯ z 0 Z(s) k ds , forw ∈ {0,z}. By applying the monotone convergence theorem (we can do this becauseZ(s) k ≥ 0 for k∈{0,2}), we reach E w Z Tw,¯ z 0 Z(s) k ds ≤u(w)<∞ forw∈{0,z} andk∈{0,2} and hence establish (C.24). Establishing (C.25). Note that for the expectationE ¯ z ,Z(s)≥ 0 fors∈ [0,T ¯ z,0 ]. Hence, we can repeat the same argument that establishes the equation (49) in the proof of Proposition 4.2 in Kim and Ward (2013) to complete the proof of (C.25). 171
Abstract (if available)
Abstract
Operations managers today must solve increasingly difficult resource allocation problems. This is because sophisticated and demanding customers expect ever-increasing product and/or service variety, provided ever-shortening lead-times. Inevitably, stochastic optimization models that represent such complex real-life situations lead to complex policies. The drawback of such policies is that they are often opaque, and, therefore, difficult to explain to a system manager. Furthermore, the exact optimal policy may be very sensitive to the model assumptions—and those assumptions are rarely exactly satisfied in reality. This observation has motivated my research philosophy—devising simple and robust policies to allocate scarce resource with provable performance guarantees. This philosophy has guided my approach to the problems I worked on during my graduate studies, pricing in revenue management and scheduling in service operations. In the following, I summarize my work on each of the aforementioned problems. ❧ The first chapter ""Asymptotically Optimal Dynamic Pricing in Queueing Systems"" is based on a joint work with Professor Ramandeep S. Randhawa. We study optimal dynamic pricing to maximize revenues in queueing systems with price and delay sensitive customers. The system congestion is visible so that upon arrival, customers decide to join the system based on the congestion and the price at that time. We analyze this problem in the typical asymptotic regime of large customer market size and capacity. This asymptotic analysis involves solving a first order or fluid optimization problem that ignores stochastic variability and then refining it by minimizing the revenue loss that occurs due to stochasticity. Denoting the market size by n, one expects the revenue loss due to stochasticity to be on the O(n^0.5) scale. However, surprisingly, we find that the optimal dynamic pricing leads to an order improvement and the loss due to stochasticity is on the O(n¹⁄³) scale. The corresponding asymptotic control problem also turns out to be non-conventional. We solve this problem to obtain a near-optimal dynamic pricing policy, and further, we show that a simple policy of using only two prices can achieve most of the benefits of dynamic pricing. ❧ The second chapter ""Dynamic Scheduling of a GI/GI/1+GI Queue with Multiclass Customers"" is based on a joint work with Professor Amy R. Ward. We consider a dynamic control problem for a GI/GI/1+GI queue with multiclass customers. The customer classes are distinguished by their inter-arrival time, service time, and abandonment time distributions. There is a cost c_k>0 for every class k in {1,2,...,N} customer that abandons the queue before receiving service. The objective is to minimize average cost by dynamically choosing which customer class the server should next serve each time the server becomes available (and there are waiting customers at least from two classes). It is not possible to solve this control problem exactly, and so we formulate an approximating Brownian control problem. The Brownian control problem incorporates the entire abandonment distribution of each customer class. We solve the Brownian control problem under the assumption that the abandonment distributions both have an increasing failure rate. We then interpret the solution to the Brownian control problem as a control for the original dynamic scheduling problem. Finally, we perform a simulation study to demonstrate the effectiveness of our proposed control. The solution technique developed in this chapter is generalized into the next chapter in which we consider many-server queueing systems. ❧ The third chapter ""Dynamic Scheduling in a Many-Server Multi-Class System: the Role of Customer Impatience in Large Systems"" is based on a joint work with Professors Amy R. Ward and Ramandeep S. Randhawa. We study optimal scheduling of customers in service systems, such as call centers. In such systems, customers typically hang up and abandon the system after waiting for a long time for their service to commence. Such abandonments are detrimental for the system, and so managers typically use scheduling as a tool to mitigate it. In this paper we study the interplay between customer impatience and scheduling decisions when managing heterogeneous customer classes. Specifically, our focus is on general patience distributions, in which a customer’s instantaneous propensity to abandon may change over time. We find that incorporating these temporal changes into the scheduling decisions can lead to a significant improvement in the system performance. Formally, we solve the underlying Diffusion Control Problem for these systems, and characterize near-optimal scheduling policies. One of our main results is that for a class of parameters, we establish the non-optimality of threshold policies, which are widely used and are near-optimal when temporal changes in customer impatience are ignored. We find it interesting to compare the performance between the fixed-queue-ratio policies under which the ratio of of queue-lengths among classes is maintained at a pre-specified level and the threshold control. In particular, we observe that simple fixed-queue-ratio policy is more efficient in managing customer impatience than the threshold control.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Essays on capacity sizing and dynamic control of large scale service systems
PDF
Essays on service systems with matching
PDF
Essays on service systems
PDF
Essays on bounded rationality and revenue management
PDF
Stochastic models: simulation and heavy traffic analysis
PDF
Essays on revenue management with choice modeling
PDF
Essays on dynamic control, queueing and pricing
PDF
Essays on information design for online retailers and social networks
PDF
Essays on revenue management
PDF
Efficient policies and mechanisms for online platforms
PDF
Strategic and transitory models of queueing systems
PDF
Modeling customer choice in assortment and transportation applications
PDF
Essays on information, incentives and operational strategies
PDF
Learning and control in decentralized stochastic systems
PDF
On practical network optimization: convergence, finite buffers, and load balancing
PDF
Train scheduling and routing under dynamic headway control
PDF
Iterative path integral stochastic optimal control: theory and applications to motor control
PDF
Novel queueing frameworks for performance analysis of urban traffic systems
PDF
Statistical learning in High Dimensions: Interpretability, inference and applications
PDF
Dynamic pricing and task assignment in real-time spatial crowdsourcing platforms
Asset Metadata
Creator
Kim, Jeunghyun
(author)
Core Title
Real-time controls in revenue management and service operations
School
Marshall School of Business
Degree
Doctor of Philosophy
Degree Program
Business Administration
Publication Date
04/19/2016
Defense Date
03/23/2016
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
dynamic pricing,dynamic scheduling,OAI-PMH Harvest,queueing theory,revenue management,service operations,stochastic control
Format
application/pdf
(imt)
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Randhawa, Ramandeep S. (
committee chair
), Ward, Amy R. (
committee chair
), Nazerzadeh, Hamid (
committee member
), Neely, Michael J. (
committee member
)
Creator Email
jeunghyun.kim.2015@marshall.usc.edu,jeunghyun.kim@gmail.com
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c40-233499
Unique identifier
UC11278335
Identifier
etd-KimJeunghy-4293.pdf (filename),usctheses-c40-233499 (legacy record id)
Legacy Identifier
etd-KimJeunghy-4293.pdf
Dmrecord
233499
Document Type
Dissertation
Format
application/pdf (imt)
Rights
Kim, Jeunghyun
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Tags
dynamic pricing
dynamic scheduling
queueing theory
revenue management
service operations
stochastic control