University of Southern California Dissertations and Theses

Essays on capacity sizing and dynamic control of large scale service systems

(USC Thesis Other)

Essays on capacity sizing and dynamic control of large scale service systems

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content ESSAYS ON CAPACITY SIZING AND DYNAMIC CONTROL OF LARGE SCALE SERVICE SYSTEMS by Ya sar Levent Ko ca ga A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulllment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (BUSINESS ADMINISTRATION) August 2010 Copyright 2010 Ya sar Levent Ko ca ga ii Dedication En de gerli varl gm olan aileme... F ur meine Familie mit viel Liebe... Levent, Los Angeles, Summer 2010 iii Acknowledgments I would like to express my deepest gratitude to my advisor and dissertation chair Professor Amy Ward and want to thank her for the guidance, patience, and support throughout my doctoral studies at Marshall School of Business. I am also grateful to my dissertation committee members Professor Sampath (Raj) Rajagopalan from the IOM Department and Professor Sheldon Ross from the ISE Department. I beneted greatly from their valuable insight and support. I am also obliged to Professor Alper S en from the IE Department of Bilkent University and Professor Mor Armony from Stern School of NYU for their guidance and mentorship during dierent stages of my graduate studies. I would like to thank to my colleagues in the IOM Department for their friendship and support during my doctoral studies. A special thanks is also due to the good old friends from Bilkent for their friendship throughout the years and to the crew of EuroHaus with whom I enjoyed life at USC and Los Angeles a lot. Finally, I want to thank my dear family; my mother Juliane Sieglinde Ko ca ga, my father Ebubekir Ko ca ga, my brother Hasan Ozg ur Ko ca ga and Yen-Ming (Emily) Lee for their enduring love and support during my long years of education. Without you, I would not be here now... iv Table of Contents Dedication ii Acknowledgments iii List of Tables vi List of Figures vii Abstract viii Chapter 1. Introduction 1 Chapter 2. Literature Survey 8 2.1. Markov Decision Processes (MDP) 8 2.2. Heavy Trac Approximations 9 2.3. Call Center Outsourcing 10 2.4. Stang under Uncertain Arrival Rates 11 Chapter 3. Admission Control for a Multi-server Queue with Abandonment 13 3.1. Model Formulation 13 3.2. The Markov Decision Process (MDP) 15 3.2.1 The Optimality Equations 16 3.2.2 The n-Terminating Problem 17 3.2.3 Policy Computation 19 3.3. The Diusion Control Problem (DCP) 21 3.3.1 The Approximating Control Problem 21 3.3.2 Threshold Policies 22 3.3.3 The Diusion Control Problem Solution 26 3.4. The Performance of the Policy Arising from the Diusion Control Problem 30 v Chapter 4. Stang and Dynamic Outsourcing under Uncertain Arrival Rate 38 4.1. Model Description 38 4.2. Asymptotic Analysis 42 4.2.1 Conventional Variability and QED (c = 1=2) 43 4.2.2 Moderate Variability and QED-c (1=2<c< 1) 46 Chapter 5. Conclusions 51 Bibliography 54 Appendices 59 Appendix A: Proofs 59 Appendix B: Tables 102 vi List of Tables Table 1: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. 33 Table 2: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. 34 Table 3: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. 102 Table 4: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. 103 Table 5: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. 104 Table 6: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. 105 Table 7: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. 106 Table 8: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. 107 Table 9: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. 108 Table 10: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. 109 vii List of Figures Figure 1: The percentage cost improvement of the optimal policy. 3 Figure 2: The minimum achievable cost (l ? ) and associated threshold level l ? as a function of a and c. 30 viii Abstract In this dissertation we present and solve two queueing control problems that are related to capacity sizing and dynamic outsourcing in call centers. First we take the stang decision as given and focus on the dynamic outsourcing problem by solving an admission control problem for a multi-server queue with abandonment. Next, we incorporate the upfront stang decision that must made with partial knowledge of the arrival rate distribution. In a M/M/N+M queue, when there are many customers waiting, it may be preferable to reject (or outsource) a new arrival rather than risk that arrival later abandoning without receiving service. On the other hand, rejecting new arrivals in- creases the percentage of time servers are idle, which also may not be desirable. We ad- dress these trade-os by considering an admission control problem for aM/M/N+M queue when there are costs associated with customer abandonment, server idleness, and turning away customers. First, we formulate the relevant Markov decision pro- cess (MDP), show that the optimal policy is of threshold form, and provide a simple and ecient iterative algorithm that does not presuppose a bounded state-space to compute the minimum innite horizon expected average cost and associated thresh- old level. Under certain conditions, we can guarantee that the algorithm provides an exact optimal solution when it stops; otherwise, the algorithm stops when a provided bound on the optimality gap is reached. Next, we solve the approximating diusion control problem (DCP) that arises in the Haln-Whitt many-server limit regime. This allows us to establish that the ix parameter space has a sharp division. Specically, there is an optimal solution with a nite threshold level when the cost of an abandonment exceeds the cost of rejecting a customer; otherwise, there is an optimal solution that exercises no control. This analysis also yields a convenient analytic expression for the innite horizon expected average cost as a function of the threshold level. Finally, we propose a policy for the original system that is based on the DCP solution, and show that this policy is asymptotically optimal. Our extensive numerical study shows that the control that arises from solving the DCP achieves a very similar cost to the control that arises from solving the MDP, even when the number of servers is small. Our second model capitalizes on our previous results by addressing how to jointly decide on an upfront stang level and a dynamic outsourcing policy for a company which prefers to keep its call center operations in-house but also uses outsourcing to manage time periods of overload; a situation commonly known as co-sourcing. The company must decide an upfront stang level without full knowledge of the arrival rate and can only infer the arrival rate when the call center starts operating at which time the stang level cannot be changed and the company might prefer to leverage the outsourcing option. We address this problem through a stochastic program where the objective is to choose a stang level that minimizes the sum of the linear stang costs and the expected operating cost which constitutes of an abandonment cost and a linear cost of outsourcing. For a stylized Markovian model with a general uncer- tainty structure, we rst study the operating sub-problem corresponding to dierent levels of uncertainty. We model the operating sub-problem as an admission control problem in a M=M=N +M queueing model and obtain asymptotically optimal poli- cies through diusion approximations. We explicitly characterize the approximately x optimal policies as threshold policies and show that as the level uncertainty increases the call center will either outsource all waiting calls or not use outsourcing at all. Finally, we use the explicit characterization of the operating sub-problem to study the upfront capacity sizing problem. 1 Chapter 1. Introduction Call centers have become ubiquitous in business. Today, every Fortune 500 company has at least one call center, and the average Fortune 500 company employs 4500 call center agents (who may be distributed across more than one site around the globe, cf. [30]). The industry has experienced phenomenal growth in the past ten years, and there is every reason to think that growth will continue. For many companies, the call center is a primary point-of-contact between itself and its customers. Hence a well-run call center promotes good customer relations, and a poorly-managed one hurts customer relations. But call center management is dicult. A call center manager faces the classical operational challenge of determining appropriate stang levels throughout the day and week in order to meet a random and time-varying demand for service. When stang levels are too low, customers are put on hold, and many hang up in frustration while waiting for an agent to take their call. But when stang levels are too high, the call center manager nds himself paying sta to be idle. On top of these diculties, the stang decision has to be made upfront, without precise knowledge of the arrival rate. One option is to outsource call center operations. Then, the challenges of call center management can be handled by a rm whose primary focus is call center operations. That rm can pool demand amongst various companies, thereby lowering variability, which should allow for more accurate demand forecasts, and so better stang decisions. Moreover, the outsourcer often has lower stang costs. Since stang costs are the largest component of a call center's overall operating budget, this can result in a lower cost per handled call that is signicant. 2 The value inherent in the outsourcing idea is supported by the numbers. The worldwide market for customer service outsourcing is expected to grow from approx- imately $58 billion in 2006 to $92 billion in 2010. This is respectively 19% and 25% of the total global customer care market, meaning the percentage of the total market attributed to outsourcing is also increasing (cf. [37]). However, it is also true that most companies are reluctant to relinquish control of their call center operations to an outsourcer. This is because callers' perception of service quality directly aects the reputation of the company. In general, companies prefer to co-source; that is, to only use an outsourcer to handle part of their total call volume. This is evidenced by a recent survey of Incoming Call Management Institute ([36]). Among 279 call center professionals only 7.9% of respondents used an outsourcer to handle most or all of their calls (over 80%). The question of interest then is: what drives the outsourcing decision? For many companies, it is the ability to dynamically increase the number of agents during time periods of overload. During such periods, callers must wait for some time before reaching an agent. The service quality, as measured by customer wait times and hang ups during these periods will be low. Furthermore the potential cost savings can be dramatic as stang cost usually constitutes the most signicant part of over- all operating cost for call centers. Hence it is of interest to outsource calls during time periods of overload. In fact, 41.3% of the respondents in the same ICMI survey referenced above cited handling overload as one of the major drivers of the outsourc- ing decision. (The most common reason for outsourcing cited was cost reduction (65.1%), followed by handling over ow (41.3%), and then providing extended hours of operation (27%), and nally tapping into the outsourcer's experience and expertise 3 (27%).) Our numerical study gives further support to this reasoning. In particular, Figure 1 shows the cost improvement over a policy that does not use outsourcing (no outsourcing policy) and a policy that starts to outsource as soon as a queue starts to build up ( uid solution) for various parameter sets. Figure 1: The percentage cost improvement of the optimal policy: The gure illus- trates the percentage improvement in long run average cost with respect to a policy that does not outsource any calls and a policy that outsources immediately whenever all servers are busy. For dierent numbers of servers and costs per customer aban- donment, other parameters are kept xed where the time to serve a call is 1 minute, the mean time to abandon is 0.5 minutes and to outsource a call costs $1. Having justied call center co-sourcing as a viable business model for call centers, we move on to describing the two models that will be studied to answer the aforemen- tioned questions that we have raised. The rst model we consider is the admission 4 control for a multi-server queue with abandonment. While the practical importance of the admission control problem stems from its connection to the call center co-sourcing environment, where rejecting an arrival is analogous to outsourcing a call at the third party outsourcer, the admission control problem is also theoretically important on its own. Therefore we prefer to keep our exposition general and discuss the problem in this context with the implicit understanding that rejecting a customer in the general admission control problem corresponds to outsourcing the customer in the call center co-sourcing environment. In a service environment such as a call center or even in the more conventional manufacturing environment, it is important to realize that customers will not wait an indenite amount of time for service. Hence it may occur that a customer that has already joined a queue for service will leave before this service begins, because he has decided that the wait time for that service is excessive. In an eort to prevent this phenomenon, it may be preferable to turn away a newly arriving customer rather than have that customer join the queue, but possibly abandon the queue later without receiving service. This suggests that it is natural to consider admission control in any queueing model that incorporates customer abandonment. Loosely motivated by the aforementioned call center co-sourcing application, we consider a many server setting and explicitly account for customer impatience through customer abandonments. We formulate an admission control problem for a M/M/N+M queue (The +M species that the abandonment distribution is exponential; in particular, we assume that each customer independently abandons if he is still waiting in queue within an exponentially distributed amount of time.) when there are costs associated with cus- tomer abandonment, server idleness, and turning away customers. Our objective is 5 to minimize the long-run expected average cost. The control problem is a continuous- time Markov Decision Process (MDP). Because we do not assume an upper bound on the state space, the transition rates and customer abandonment costs are potentially unbounded. Hence it is not possible to solve the MDP by constructing a uniformized chain. Therefore, we consider the following alternative approach: Through an itera- tive algorithm that truncates the state space we show that a threshold control policy is optimal for this problem under certain conditions. The algorithm stops either when a local minimum is found or when a provided bound on the optimality gap is reached. When the abandonment costa exceeds the cost of rejecting a customerc, we can guar- antee that nding a local minimum coincides with nding an optimal policy having a nite threshold level. This leads us to conjecture that there is a sharp division of the parameter space. Specically, when a>c, there is an optimal policy that has a nite threshold level, and when a c, there is an optimal policy that exercises no control (i.e., that admits every customer). We show this conjecture is valid by solving the approximating diusion control problem that arises in the Haln-Whitt many-server limit regime. The solution to the approximating diusion control problem (DCP) is also of threshold form, and there is a convenient analytic expression for the long-run expected average cost as a function of the threshold level. We compute the threshold level l ? that minimizes the long-run expected average cost, and propose a threshold admission control policy for the original M/M/N+M system that turns away customers whenever the queue reaches the levelN + h p Nl ? i 1 .We prove that this policy is asymptotically optimal in the Haln-Whitt limit regime. We also provide numerics that show that the control 1 Throughoutthetext, [x]denotesthenearestintegerfunction, thatistheclosestintegertox2< 6 that arises from solving the DCP achieves a cost very similar to that of the optimal control that arises from solving the MDP, even when the number of servers is small. In addition to the stochastic variations in the arrival process, most call centers also face a random arrival rate which further complicates the stang decision. Such randomness can be attributed to forecasting errors or factors that cannot be deter- mined explicitly (see for example [14]). While it might be argued that the arrival rate is also time-varying, we chose to model the arrival rate as a time-homogeneous (constant) random variable with the understanding that the stang decision is made for a time window where the arrival rate is relatively constant (cf. [11] or [47]). Thus the outsourcing option can also be regarded as a hedge against imbalances between capacity and realized demand in addition to inter-temporal variations in the queue size (number of calls waiting to be answered). In our second model we focus on the upfront stang and the real time dynamic control decision that determines whether or not to outsource a call. We propose a control problem that rst determines the upfront stang level and then a dynamic operating policy that trades-o the cost of customer dissatisfaction during time pe- riods of overload with the cost of outsourcing. Again, we model the call center as a M/M/N+M queue and we assume there is a cost per customer abandonment, a cost per call outsourced, and an in-house stang cost that is per agent per unit time. The call center manager rst decides the upfront stang level (if any) and then learns the exact value of the arrival rate and has the option to route a newly arriving call to his own call center, or to outsource it. Our objective is to minimize the long-run expected average cost associated with in-house stang, customer abandonment and call outsourcing. 7 This dissertation is organized as follows: Chapter 2 surveys the relevant literature for the two models described above. In Chapter 3, we present our rst model, the admission control problem for a multi-server queue with abandonments and solve it exactly through MDP and approximately through the QED heavy trac approxima- tion. Chapter 4 focuses on the second model described above, the joint stang and dynamic outsourcing problem under uncertain arrival rate. Conclusions and future research directions are given in Chapter 5. All proofs and additional tables supporting our analytical results can be found in the appendix. 8 Chapter 2. Literature Survey In this section we review the relevant literature. 2.1. Markov Decision Processes There is a signicant body of research on Markov decision process models that ad- dresses admission control problems in queues. We refer the reader to [23] for recent research on how to obtain structural results, such as the optimality of a threshold form. When the transition rates can be bounded from above, one common approach to show the optimality of a threshold policy is to prove the convexity (or concavity) of the value function. However, when there is abandonment, the threshold structure may be dicult to obtain, as stated on pages 50-51 in [43]. For related work that does not use a MDP framework, see, for example, [19]. The work most closely related to our MDP analysis is that of [44]. Motivated by a prot maximizing call center with abandonment, the authors address the decision of determining the optimal number of agents (servers) and the optimal number of extra lines (buer size or waiting room size) when there is a xed reward per call, commu- nication costs per customer in system per unit time, and stang costs per agent per unit time. Under the assumption that the state space is bounded and the average service time does not exceed the average abandonment time, the authors prove a monotonicity condition, provide an ecient algorithm to calculate the optimal num- ber of agents and extra lines, and implement their algorithm in a web-based calculator (www.math.vu.nl/ sapot/software/ErlangProt). In comparison, our algorithm does not require either of the aforementioned assumptions. 9 The algorithm we provide is an iterative approximation scheme that truncates the state space, and converges to the optimal control policy as the truncation level goes to innity. The algorithm extends the work of [2], who consider a problem in which there is both admission control and service rate control, from a single server setting with no customer abandonment to a many server setting with customer abandonment. There is a rich history to their problem, but their motivation mainly comes from the work of [27], who consider a single server queue with service rate control, but leave open the problem of also including admission control. A similar and related extension of George and Harrison's work, that additionally investigates pricing issues, is [9]. 2.2. Heavy Trac Approximations There have been several recent papers that study admission control problems in the conventional heavy trac regime (see, for example, [28] when there is no customer abandonment, and [60] and [29] when there is customer abandonment). However, the only other work we nd that considers an admission control problem in the Haln- Whitt (see [33]) limit regime, extended by [25] to include customer abandonment, is that of [61]. They consider an admission control problem for a GI/M/N+M queue, solve an approximating diusion control problem, and prove asymptotic optimality in the Haln-Whitt limit regime, but under the criterion of expected innite horizon discounted cost, and without any MDP analysis. Finally, our numerics show that our proposed policy based on the DCP solu- tion results in an innite horizon expected average cost that is extremely close to the MDP solution, even for a moderate number of servers. For more general work on comparisons between approximate and exact solutions to control problems, and 10 also on connections to controls found using uid models, we refer the reader to [48] and [22]. 2.3. Call Center Outsourcing There is much literature on call centers, and for this, we refer the reader to the general survey papers of [26] and [5], and the specialized survey papers [42] (for queueing models), [46] (for optimization problems), and [45] and [4] (for multi-skill environments). There is less work on the topic of outsourcing in call centers, and most of this work focuses on contracting issues. [52] show how to design contracts that can coordinate both stang and service quality. [49] show how poorly designed contracts can lead to the outsourcer giving priority to non-contracted calls. [35] and [3] study issues of information asymmetry in contract design. [11] show how an outsourcer should sta in order to maximize revenue subject to meeting service level agreements that specify the percentage of calls that must be answered within a given amount of time. All of the aforementioned works assume that all calls are outsourced. We are interested in the co-sourced setting. [6] consider contracting issues in co- sourcing, but they do not model queueing. The eects of queueing become important at the operational level, when dynamically deciding which calls to route to the out- sourcer. From this perspective the work of [24] is closest to ours. They consider routing schemes for outsourcing when customers do not abandon, and there are two types of calls, high-value and low-value, and high-value calls are never outsourced, but low-value calls may be outsourced. Our model is most similar to the pooled-over ow system they analyze in Section 4, for which they show that the optimal policy to determine when to route low-value calls to the outsourcer, subject to maintaining a 11 given service level constraint for the high-value calls, is a threshold that can be found numerically. The main dierences are that we have only one customer type, and we model customer abandonment. Then, the cost of customer abandonment replaces the need for a service-level constraint. Furthermore, in the QED limit regime, we are able to provide a closed form expression for the average cost, which can be used to nd the optimal threshold level. 2.4. Stang Under Uncertain Arrival Rates Stang decisions under arrival rate uncertainty is still an emerging area of call center research. [31] models arrival rate as a random variable to account for forecast errors. [58] addresses the stang decision for service systems with uncertain arrival rate. [39] consider the Poisson mixture model to address the stang decision of a call center facing a random arrival rate. [21] examine the implications of ignoring arrival rate uncertainty. [55] and [56] consider both time varying and random arrival rates and illustrate how to estimate long run and short run performance measures. [18] provide empirical justication to the possibility of random arrival rates through a detailed analysis of call center data and show that the common assumption of known arrival rates is not valid. [10] consider several possible arrival models and test them empirically. All of the aforementioned work is either empirical or uses models that are exact. Next we explore work that is based on heavy trac approximations. Such approximations are of particular importance for call centers as their accuracy increases as the system scale increases. [54] shows how to adapt the well-known square-root stang rule to account for ar- rival rate uncertainty. Using uid models for multi-server queues with abandonments 12 to capture the queueing dynamics, [34] initiate a stream of research for large scale ser- vice systems with with arrival rate uncertainty. The stang decisions for such models boil down to the well-known newsboy model which trades o capacity costs and costs or lost revenue due to shortages (holding and abandonment costs in this case). In a similar spirit, [64] addresses stang decisions for a more general scenario where both arrival and service rates are uncertain. [13] incorporate dynamic call assignment into the stang model of [34] while [12] consider a data driven approach. Among these papers our paper is most related to [14] where the authors analyze the performance of their newsvendor based stang policy, provide optimality bounds and show that the approximate stang prescription works extremely well for large arrival rates and number of servers. [14] show in particular, when the arrival rate uncertainty domi- nates the variability inherent in the Poisson process itself, the performance gap of the newsvendor based prescription is bounded by a constant resulting in superior perfor- mance when the arrival rate and the number of servers are large. A recent paper by [32] addresses the stang decision of a call center under quality-of-service constraints through chance-constrained optimization. Finally, our work is most similar to [47] where the authors analyze the performance of an M/M/N+G queue under both ran- dom and time varying arrival rates. While our focus is on a Markovian system, we rely on the uncertainty model introduced in [47] to facilitate our analysis. 13 Chapter 3. The Admission Control Problem for Multi-Server Queues with Abandonment 3.1. Model Formulation Our objective is to develop an admission control policy for a multi-server queue with impatient customers that minimizes innite horizon average expected cost. We assume the arrival process is Poisson, service times are exponential, and that each customer independently abandons the system if he has not entered service after an exponential amount of time. The cost per customer rejected is c> 0. Each customer abandonment costs a> 0. Finally, each idle server costs h I 0 per unit time. We capture the control decision by the function : Z + !f0; 1g where Z + = f0; 1; 2;:::g is the set of non-negative integers. The system manager rejects an arriving customer that nds x customers in the system if (x) = 1 and admits the customer otherwise. Note that we are restricting the class of admissable policies to the class of stationary policies. The further restriction to deterministic policies is without loss of generality because we show in Section 3.2.that, under certain conditions, there exists a deterministic optimal policy within the class of stationary policies. We are interested in threshold policies. A threshold policy with threshold level l has (x) = 0 for all x<l and (x) = 1 for all xl. Any policy within the class of deterministic stationary policies is equivalent to a threshold policy when the system starts empty 2 . The threshold level l is exactly the smallest state in which customers 2 Consider a policy that rejects customers in state 1, does not reject customers in states 2 and 3, and rejects customers in states 4 and higher. If the system starts empty, this policy is equivalent to a threshold policy with threshold level 1. However, if the system starts in state 2, it is not. 14 are rejected. When the threshold level is nite, the system dynamics are equivalent to a M/M/N+M queueing model with a nite buer, of size determined by the threshold level. Any policy within this class is trivially of threshold form when the system starts empty. The threshold level is exactly the smallest state for which customers are rejected. We now specify the system evolution equations for the process X that denotes the total number of customers in the system. Let > 0 and > 0 represent the system arrival rate and service rate of an individual server. Let 1= for > 0 be the mean time until a customer abandons. The arrival, service and abandonment (or lost customer) processes are formed from the independent, standard Poisson processes A;S; and L. Then, X(t) :=X(0) +A(t) Z t 0 (X(s))dA(s) (1) S Z t 0 [N^X(s)]ds L Z t 0 [X(s)N] + ds ; t 0: The total system cost up to time t is (t) := Z t 0 h I [NX(s)] + ds+ Z t 0 c (X(s))dA(s)+aL Z t 0 [X(s)N] + ds ; t 0: (2) LetA denote the set of functions dened on the non-negative integers and having rangef0; 1g. Our objective is to nd the policy that achieves the minimum average 15 cost dened as z ? := min 2A lim inf t!1 E [(t)] t : (3) It is useful to dene the holding cost rate function h(x) :=h I [Nx] + +a [xN] + ; when the total number of customers in the system is x, and note that E [(t)] =E Z t 0 h(X(s))ds +cE Z t 0 (X(s))dA(s) : Next, we present the Markovian Decision Process analysis for our model. 3.2. The Markov Decision Process (MDP) Observe that when there is no admission control the process X in (1) is a birth- and-death process with birth rate and state-dependent death rate (n) := (n^N) + [nN] + ; and we can formulate the admission control problem as a Markov decision process. Note however that the state space for the processX in (1) can potentially be countably innite which will result in unbounded death rates. Therefore our model does not t into the standard MDP framework where one can apply uniformization. Instead, we pursue an alternative approach and devise an optimal policy through the following iterative algorithm. The algorithm rst determines the system cost under a threshold policy at level l. At each next step, the algorithm increases the threshold level l by 16 one and calculates the system cost. Under the assumption that c < a, we establish that the rst local minimum reached is a global minimum. In general, there is a stopping criterion for the algorithm that is based on a calculatable optimality gap. We start our analysis by showing in Section 3.2.1. that within the class of station- ary policies there exists a deterministic optimal policy. In Sections 3.2.2. and 3.2.3., we provide the aforementioned iterative algorithm and establish its properties. 3.2.1. The Optimality Equations The rst step is to provide the optimality equations for the relative value function v, and average cost constant z (see, for example, [51]). Standard arguments in MDP analysis show that v and z must satisfy v(n) = min 8 > < > : h(n)z +(n) + (n) +(n) v(n 1) + +(n) v(n + 1); h(n)z +(n) + (n) +(n) v(n 1) + +(n) (v(n) +c) 9 > = > ; ; for n2f1; 2;:::g; and v(1) =v(0)h(0) +z: It is preferable for our analysis to work with the relative cost dierences y(n) :=v(n)v(n 1) for n2f1; 2;:::g: Then, we can re-write the preceding equation that v must satisfy in terms of y as follows h(n)z =(n)y(n) min (y(n + 1);c); for n2f1; 2;:::g; (4) 17 and y(1) =zh(0): (5) Our rst theorem shows that there is no randomized policy that has a lower innite horizon average expected cost than z whenz is nite and (y(1);y(2);:::) is a uniformly bounded sequence. Let S denote the set of stationary policies. A policy in this class S = (p(n) :n2f0; 1;:::g) species the probability p(n) of admitting a customer for each possible system state n. Note that the set S includes both deterministic and randomized policies. Theorem 1 Suppose there existsz <1 and a uniformly bounded sequence (y(1);y(2);:::) satisfying equations (4) and (5). Then ifz S is the average cost associated with a policy S 2 S , z S z: Given that there exists a solution to the optimality equations (4)-(5), one can easily see that the rst level n with y(n + 1)c will be the optimal threshold level. The problem is that we do not know upfront that a solution to equations (4)-(5) exists. 3.2.2. The n-Terminating Problem Unfortunately, it is not obvious how to nd a solution to (4)-(5). The key is to introduce the n-terminating problem. In the n-terminating problem, we would like 18 to nd a constant z n and a vector (y n (1);y n (2);:::y n (n);y n (n + 1)) that satisfy h(k)z n =(k)y n (k)y n (k + 1) for all k2f1;:::;ng; (6) y n (1) =z n h(0) and, (7) y n (n + 1) =c: (8) The interpretation for z n is that z n is the average cost associated with a threshold policy at level n. Also because the n-terminating problem is a Markovian Reward Process with nite state space, the existence of a unique solution to equations (6), (7) and (8) is well established in theory (see, for example, Proposition 8.2.1 and Corollary 8.2.7 in [51]). The following Lemma establishes a useful property of then-terminating problem, that the vector y n is increasing in z n . Lemma 1 Suppose z n 1 > ()z n 2 for some non-negative integers n 1 and n 2 . Then, y n 1 (k)> ()y n 2 (k) for k2f1; 2;:::; min(n 1 ;n 2 ) + 1g: We use the n-terminating problem (6)-(8) to construct a threshold policy that achieves the minimum possible average cost within the class of stationary policies. Specically, we solve a sequence ofn-terminating problems (6)-(8), and search for the rst local minimumz m havingz m <z k for allk2f0; 1;:::;m 1g, withz m+1 z m . Consistent with the intuition that it does not make sense to reject an arriving customer when there is a free server, the following Lemma establishes that we do not expect to nd a local minimum before state N. 19 Lemma 2 The sequence of solutions to the n-terminating problem (6)-(8) has z 0 >z 1 >>z N1 >z N : We end this section with a theorem that shows that nding such az m is equivalent to knowing the z in a solution to the optimality equations (4)-(5). Theorem 2 Let c < a and suppose there exists a sequence of solutions to the n- terminating problem (6)-(8) such that z m <z k for all k2f0; 1;:::;m 1g; and z m+1 z m : Then, if z S is the average cost associated with a policy S 2 S , z S z m : 3.2.3. Policy Computation We are now in a position to construct an algorithm to nd a threshold policy that achieves the minimum possible average cost within the class of stationary policies. The algorithm is similar in spirit to Adusumilli and Hasenbein, but is adapted to our setting. In particular, the proofs supporting the algorithm (Theorem 2 above and Theorems 3 and 4 below) must be changed in a non-trivial fashion to account for the state-space dependence caused by customer abandonments. The algorithm leverages Lemma 2 and starts by setting the threshold level to N. 20 Initialization: Set n =N + 1. Step 1: Solve the n-optimality equations (6)-(8). Step 2: If a solution to then-optimality equations (6)-(8) exists andz n z n1 , then the (n 1)-terminating policy is optimal; that is, if z S is the average cost associated with a policy S 2 S , z S z n1 . Otherwise, increase n by 1 and go to step 1. There are two cases: either there will be a local minimum, or there will not be a local minimum. In the case that there is a local minimum and c < a, Theorem 2 implies we have found an optimal policy. If there is not a local minimum, the sequence of solutions to the n-terminating problem converges to z ? . Theorem 3 If the sequence of solutions to the n-terminating problem is decreasing; i.e., if z 0 >z 1 >z 2 > ; then z ? = lim n!1 z n : If no local minimum has been found after n iterations, the following theorem can be used as a stopping criterion. Theorem 4 If z 0 >z 1 >z n , then z n z ? (cy n (n)): Note we do not know upfront whether the algorithm will stop at an optimal threshold. Moreover there is no sharp condition that determines when it is never optimal to reject a customer. 21 3.3. The Diusion Control Problem We formulate and solve an approximating diusion control problem. Section 3.3.1. states the approximating diusion control problem. In Section 3.3.2 we characterize the long-run average expected cost associated with a threshold policy. Finally, in Section 3.3.3. we show that the policy that minimizes long-run average expected cost among all policies is a threshold policy. 3.3.1. The Approximating Control Problem Let B be a standard Brownian motion, and let ^ U be a process that is adapted to B. The process ^ X(t) = ^ X(0) +B(t) + Z t 0 m ^ X(s) ds ^ U(t); (9) where 2 = 2 and m(x) = 8 > < > : mx x 0 m x x> 0 approximates the centered and scaled number of customers in the system, X(t)N p N , and the process ^ U approximates the scaled cumulative number of rejected customers. The constant m can be used to approximate the capacity imbalance of the system; see Theorem 6 and the discussion in the paragraph following Theorem 7. Let ^ h(x) :=h I x +a x + ; and ^ (t) = Z t 0 h ^ X(s) ds +c ^ U(t): (10) 22 Let ^ A be the set of all non-negative, non-decreasing and RCLL processes that are adapted toB, under which (1) a strong solution to the stochastic equation (9) above exists, and (2) E ^ X(t) =t! 0 as t!1. We will solve min ^ U2 ^ A lim inf t!1 E h ^ (t) i t : (11) The diusion control problem (11) approximates the original control problem (3). 3.3.2. Threshold Policies We initiate our analysis by considering the class of threshold policies. We will show that a policy in this class solves the diusion control problem (11); see Theorem 5 in Section 3.3.3. A threshold policy ^ U T that regulates at the level l<1 has ^ U T (0) = h ^ X(0)l i + (12) ^ X(t)l for all t 0 Z 1 0 h l ^ X(t) i + d ^ U T (t) = 0: The following lemma characterizes the long-run average expected cost associated with a threshold policy. 23 Lemma 3 Suppose there exists a twice continuously dierentiable function V having bounded rst derivative and a nite constant that solve 2 2 V 00 (x) +m(x)V 0 (x) + ^ h(x) =; (13) for x l with the boundary condition V 0 (l) = c. Then, represents the average cost associated with controlling the diusion ^ X in (9) using the threshold policy ^ U T regulating at the level l; i.e. = lim t!1 E h ^ (t) i t : We now construct a solution to (13) that is twice continuously dierentiable on (1;l]. For this, it is sucient to specify the functionV 0 . Let and be respectively the standard normal pdf and cdf. The function V 0 (x) := 8 > < > : V 0 1 (x) x 0 V 0 2 (x) 0<xl (14) for V 0 1 (x) := 2 r ( h I m ) exp( 2 (x + m ) 2 )( p 2 (x + m )) h I (15) V 0 2 (x) := 2 r ( +am) exp( 2 (x + m ) 2 ) ( p 2 (x + m )) ( p 2 (l + m )) +a + (ca) exp( 2 ((l + m ) 2 (x + m ) 2 )); (16) 24 and :=(l) := A(l) B(l) (17) A(l) := h I 1 + 2 r m exp m 2 2 m r 2 +c exp 2 l 2 + 2 m l +a 0 B @ 1 exp 2 l 2 + 2 m l + 2 q m exp m 2 2 m q 2 p 2 l + m 1 C A B(l) := 2 p 0 B @ 1 p exp m 2 2 ( m q 2 ) + 1 p exp m 2 2 p 2 (l + m ) m q 2 1 C A solves (13) and has V 0 (l) =c. Dene V (x) := Z x 0 V 0 (y)dy for xl: (18) Then, the function V is twice continuously dierentiable on (1;l]. To see this, note that it is straightforward to check that V 0 1 (0) =V 0 2 (0) and V 00 1 (0) =V 00 2 (0) after deriving V 00 1 (x) = 2 2 x + m V 0 1 (x) + 2 2 ( +h I x) V 00 2 (x) = 2 2 x + m V 0 2 (x) 2 2 a x + 2 2 : Finally, the function V 0 is bounded because it is continuous and lim x!1 V 0 1 (x) = h I : 25 The following Lemma veries the intuition that is positive, and is increasing in the cost parameters a, h I , and c. Lemma 4 The function :=(l;a;h I ;c) satises > 0 for all l;a;h I ;c 0: Furthermore, @ @c > 0; @ @a > 0; and @ @h I > 0: We view as a function of l, and minimize over l for xed values of the cost parametersa,h I , andc. Our next proposition shows the conditions under which(l) has a minimum that will be attained at some nite l. Proposition 1 If a>c, then there exists a unique l ? that satises (ac) l ? (l ? ) =cm; (19) and (l)(l ? ) for all l 0. Otherwise, 0 (l) 0 for all l 0. The formula (19) has the following intuition. In order that l is a minimum, it is necessary that 0 (l) = 0. We show in the proof of Proposition 1 that in order that 0 (l) = 0, it is necessary that (ac) lcm(l) = 0, which implies thatV 00 2 (l) = 0 (using the expression for V 00 2 (x) that appears below (18)). Then, (19) is obtained by letting x =l in (13). Extend the denition of V 0 in (14) so that V 0 (x) = c for all x l ? . Then, the functionV is twice continuously dierentiable becauseV 00? 2 ) = 0. Hence the conditions 26 of Lemma 4 are satised, and we can conclude that (l ? ) represents the average cost associated with controlling the diusion ^ X in (9) through the threshold policy that regulates at the level l ? . Corollary 1 Let ^ U ? T be the threshold policy that regulates at level l ? , with associated cumulative cost process ^ ? T . Then, (l ? ) = lim t!1 E h ^ ? T (t) i t : 3.3.3. The Diusion Control Problem Solution We show that the threshold policy that regulates at the levell ? solves the diusion control problem whena>c, and that it is optimal to exercise no control whenac. The rst step is to provide a verication Lemma that characterizes the minimum achievable long-run average expected cost. Lemma 5 Suppose there exists a twice continuously dierentiable function V having bounded rst derivative, and a constant that satisfy 2 2 V 00 (x) +m (x)V 0 (x) + ^ h (x); (20) and V 0 (x) c for all x2<. Then, for any admissible control ^ U having associated cumulative cost process ^ in (10), lim inf t!1 E h ^ (t) i t : 27 The next step is to show that the threshold policy that regulates at levell ? attains the minimum achievable long-run average expected cost when a>c, and that exer- cising no control attains the minimum achievable long-run average cost when ac. To do this, it is sucient to show that the function V dened in (18) satises the conditions of Lemma 5. In particular, it is enough to show that V 0 1 and V 0 2 in (15) and (16) are increasing. This is because when a > c, V 0 2 (x) = c for all x l ? by construction, and when a c, V 0 2 (x)! a as x!1. Also, it is straightforward to check that (20) is satised (and we show this explicitly in the proof of Theorem 5). Theorem 5 Let ^ U2 ^ A be an admissible control, and let ^ be the associated cumula- tive cost process in (10). (i) Suppose a > c, and let l ? be dened as in the statement of Proposition 1. Let ^ U ? T be the threshold control at l ? , as dened in (12), and let ^ ? T be the associated cumulative cost process in (10). Then, lim inf t!1 E h ^ (t) i t lim t!1 E h ^ ? T (t) i t =(l ? ): (ii) Suppose a c. Let ^ U 0 (t) = 0 for all t 0, and let ^ 0 be the associated cumulative cost process in (10). Then, lim inf t!1 E h ^ (t) i t lim t!1 E h ^ 0 (t) i t = 0 ; 28 where 0 = lim l!1 (l) = h I 1 + 2 q m exp m 2 2 m q 2 2 p 1 p exp( m 2 2 )( m q 2 ) + 1 p exp( m 2 2 )(1 ( m q 2 )) + a 1 + 2 q m exp m 2 2 m q 2 1 2 p 1 p exp( m 2 2 )( m q 2 ) + 1 p exp( m 2 2 )(1 ( m q 2 )) : It is worthwhile to double-check that the expression for 0 agrees with standard results in the literature. Since ^ U 0 (t) = 0 for all t 0, it follows that lim t!1 E h ^ 0 (t) i t =h I E h ^ X(1) i +a E h ^ X(1) + i ; (21) where ^ X(1) has the steady-state density associated with the process ^ X in (9) under control ^ U 0 . The expressions for E[ ^ X(1)j ^ X(1) 0] and E[ ^ X(1)j ^ X(1) > 0] are given in (18.29) of [20], and the expressions for P ( ^ X(1) 0) andP ( ^ X(1)> 0) are given in (18.5) of this same paper. Hence it follows from [20] that E h ^ X(1) i =E h ^ X(1)j ^ X(1) 0 i P ^ X(1) 0 = p 2 1 p 2 m p m q 2 m q 2 + 1 1 p m q 2 m q 2 + 1 p 1 m q 2 m q 2 ; 29 and E h ^ X(1) + i =E h ^ X(1)j ^ X(1)> 0 i P ^ X(1)> 0 = p 2 1 m p p 2 1 m q 2 m q 2 + 1 1 p m q 2 m q 2 + 1 p 1 m q 2 m q 2 : It is now straightforward to verify that h I E h ^ X(1) i +a E h ^ X(1) + i = lim l!1 (l); by plugging the expressions for E h ^ X(1) i and E h ^ X(1) + i into (21). We end this Section by plotting the minimum achievable average cost, (l ? ), and the associated threshold levell ? as a function of the abandonment costa and cost per customer rejected c. Figure 2 shows that (l ? ) is increasing in both the parameters a and c, while l ? is decreasing in a but increasing in c. We do not include the cases where ac, because in this case l ? =1. 30 0 5 10 15 20 25 30 35 40 45 50 0 10 20 30 40 50 0 5 10 15 20 25 c a optimal DCP cost (a) (l ? ) 0 10 20 30 40 50 0 10 20 30 40 50 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 c a optimal DCP threshold (b) l ? Figure 2: The minimum achievable cost (l ? ) and threshold level l ? . The minimum achievable cost (l ? ) and associated threshold level l ? as a function of a and c. [ = =h I = 1; = p 2;m = 0] 3.4. The Performance of the Policy Arising from the Diusion Control Problem There is a natural translation from the optimal policy for the diusion control problem in Theorem 5 to a policy for the original system. Specically, when there are N servers, let ?;N (x) := 1 n [xN] + h p Nl ? io ; wherel ? satises (19). The following Theorem, which is Theorem 7.6 in [50] re-stated in our setting, justies that this is the right translation. 31 Theorem 6 Consider a sequence of systems havingN servers and arrival rate N = Nm p N +o( p N), for somem2<. LetX N denote the number-in-system process for each system in the sequence, that satises (1). Let l 0 be a xed constant, and suppose each system operates under the threshold admission control policy N (x) := 1 n [xN] + h p Nl io : Let ^ X be the diusion process that satises (9) under threshold control (dened in (12)) at level l. Then, if X N (0)= p N) ^ X(0) as N!1, also X N N p N ) ^ X in D as N!1; where D is the space of right-continuous functions having left limits with values in (1;1), equipped with the Skorokhod J 1 metric. Our nal Theorem proves that the policy ?;N is asymptotically optimal. Theorem 7 Consider a sequence of systems havingN servers and arrival rate N = Nm p N +o( p N), for somem2<. LetX N denote the number-in-system process for each system in the sequence, that satises (1). (i) Under the policy ?;N (x), lim N!1 lim t!1 1 p N E t; ?;N t =(l ? ): 32 (ii) Furthermore under any admissable policy N lim inf N!1 lim t!1 1 p N E t; N t (l ? ): Theorem 7 establishes that the policy ?;N matches the performance of an optimal policy for the system withN servers in the Haln-Whitt limit regime, as the number of servers becomes large. We are also interested in studying the performance of the policy ?;N for a xedN. The numeric results in [25] for a system with no admission control suggest that the diusion ^ X under threshold control at level l can be used to very accurately predict system performance in the pre-limit under the control N in Theorem 6 for a xed N, even when N is relatively small (10 or so). Specically, for a system with arrival rate , service rate , and N servers, Theorem 6 suggests that X N p N ^ X +N; where ^ X is the diusion process that satises (9) under threshold control (dened in (12)) at level l, and we let m = N p N : We would like to also numerically validate the intuition that the performance of the system withN servers under the control ?;N , that resulted from solving the diusion control problem 11, is very close to the performance of the system with N servers under an optimal control obtained via the algorithm in Section 3.2.3. that solves the relevant MDP. 33 Table 1: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. [a = 2;c =h I = = 1; = 50] N n ? z ? N + h p Nl ? i z [N+ p Nl ? ] (z ? z [N+ p Nl ? ] )=z ? 10 0.1 11 40.1746 10 40.4716 0.74% 20 0.1 23 30.3146 22 30.5398 0.74% 30 0.1 35 20.5492 34 20.5813 0.16% 40 0.1 51 11.1105 50 11.1129 0.02% 50 0.1 90 4.0115 90 4.0115 0.00% 60 0.1 122 10.1542 110 10.1542 0.00% 70 0.1 122 20.0034 70 20.1368 0.67% 80 0.1 122 30.0000 80 30.0022 0.01% 90 0.1 122 40.0000 90 40.0000 0.00% 100 0.1 122 50.0000 100 50.0000 0.00% 10 1.0 10 40.4716 10 40.4716 0.00% 20 1.0 21 31.0852 20 31.2095 0.40% 30 1.0 31 21.8803 31 21.8803 0.00% 40 1.0 43 13.4822 43 13.4822 0.00% 50 1.0 57 7.9491 58 7.9507 0.00% 60 1.0 80 10.8509 81 10.8509 0.00% 70 1.0 109 20.0263 94 20.0263 0.00% 80 1.0 117 30.0003 98 30.0003 0.00% 90 1.0 118 40.0000 102 40.0000 0.00% 100 1.0 119 50.0000 105 50.0000 0.00% 10 10 10 40.4716 10 40.4716 0.00% 20 10 20 31.2095 20 31.2095 0.00% 30 10 30 22.4835 30 22.4835 0.00% 40 10 40 14.9792 40 14.9792 0.00% 50 10 51 10.4403 51 10.4403 0.00% 60 10 62 11.9627 62 11.9627 0.00% 70 10 74 20.0991 74 20.0991 0.00% 80 10 86 30.0012 85 30.0012 0.00% 90 10 97 40.0000 90 40.0000 0.00% 100 10 109 50.0000 100 50.0000 0.00% 34 Table 2: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. [a = 2;c =h I = = 1; =N] N n ? z ? N + h p Nl ? i z [N+ p Nl ? ] (z ? z [N+ p Nl ? ] )=z ? 10 0.1 27 1.7881 27 1.7881 0.00% 20 0.1 45 2.5336 45 2.5336 0.00% 30 0.1 61 3.1054 61 3.1054 0.00% 40 0.1 75 3.5871 75 3.5871 0.00% 50 0.1 90 4.0115 90 4.0115 0.00% 60 0.1 103 4.3952 104 4.3952 0.00% 70 0.1 117 4.7479 117 4.7479 0.00% 80 0.1 130 5.0762 130 5.0762 0.00% 90 0.1 143 5.3846 143 5.3846 0.00% 100 0.1 156 5.6763 156 5.6763 0.00% 10 1.0 13 3.4984 13 3.4984 0.00% 20 1.0 24 4.9992 25 4.9992 0.00% 30 1.0 36 6.1388 36 6.1388 0.00% 40 1.0 47 7.1014 47 7.1014 0.00% 50 1.0 57 7.9491 58 7.9507 0.02% 60 1.0 68 8.7122 68 8.7122 0.00% 70 1.0 79 9.4157 79 9.4157 0.00% 80 1.0 90 10.0731 90 10.0731 0.00% 90 1.0 100 10.6863 100 10.6863 0.00% 100 1.0 111 11.2684 111 11.2684 0.00% 10 10 10 4.2916 10 4.2916 0.00% 20 10 20 6.3557 20 6.3557 0.00% 30 10 30 7.9476 30 7.9476 0.00% 40 10 40 9.2925 40 9.2925 0.00% 50 10 51 10.4403 51 10.4403 0.00% 60 10 61 11.4337 61 11.4337 0.00% 70 10 71 12.3550 71 12.3550 0.00% 80 10 81 13.2184 81 13.2184 0.00% 90 10 91 14.0340 91 14.0340 0.00% 100 10 101 14.8091 101 14.8091 0.00% 35 Table 1 comparesz [N+ p Nl ? ] , the average cost of the policy ?;N , to the average cost z ? computed according to the algorithm in Section 3.2.3. The threshold values that determine if an arriving customer will be admitted are h N + p Nl ? i and n ? (dened in Section 3.2.3. as the value at which the algorithm terminates) respectively. Table 1 establishes that the diusion approximation is so good as to be indis- tinguishable from the MDP solution in terms of estimating the minimum achievable innite horizon average expected cost: the percentage dierence in the cost is gen- erally less than 0:005% (which is reported as 0.00%). Note that the parameters (, , N, and ) that are used to generate Table 1 are as in [25]. Specically, cus- tomers arrive at rate = 50 per minute and the average time to serve a customer is 1 minute. The number of servers is increased from 10 to 100 to achieve a broad spectrum of trac intensities, so that =N ranges from 0.5 to 5. Customers can be very patient, average, or very impatient, corresponding to a mean time they will wait before abandoning of 10 minutes ( = 0:1), 1 minute ( = 1), or 0.1 minutes ( = 10) respectively. We assume in Table 1 that the idleness cost is xed at h I =$1 per time unit, the rejection cost is c =$1 per customer, and the cost of a customer abandonment isa =$2. In the Technical Appendix, we show the results of performing the identical numeric experiments with abandonment costs of $1.25, $1.5, and $5; see Tables 3, 4, and 5. All these tables support the stated conclusion that the percentage dierence in cost is generally less than 0.005%. The question that arises is: why can the cost be approximated so well, when the threshold level may not be? This occurs in Table 1 (and in Tables 3, 4, and 5 in the Technical Appendix) when the service rate exceeds the arrival rate, and the mean time a customer will wait before abandoning (1= ) is large enough so that few 36 abandon; i.e., when = 10 andN = 90; 100, when = 1 andN = 70; 80; 90; 100, and when = 0:1 and N = 60; 70; 80; 90; 100. Then, many threshold levels will result in not very many customers abandoning and not too much outsourcing, so that the cost function is at and not very sensitive to the threshold level. Since the cost function is at, the numerical algorithms do not dierentiate very well between the dierent threshold levels. Furthermore, it is also true that when N p N exceeds = 50 by too much, we are far away from the Haln-Whitt limit regime, and we cannot theoretically guarantee that the diusion approximation performs well. It is also of interest to perform a numerical study in which the cost function remains sensitive to the threshold level. Table 2 shows the results of a numerical study identical to that in Table 1, except that we increase the arrival rate as the number of servers increases, so that = N, and there is zero capacity imbalance. Then, the cost function does not become too at. In Table 2, not only is the minimum achievable innite horizon expected average cost approximated very well (with less than 0.005% relative error), but also the threshold levels never dier by more than 1. This is also generally true in Tables 6, 7, and 8 in the Technical Appendix, which compliment Tables 3, 4, and 5 exactly as Table 2 does Table 1. (We say \generally" because in Table 6, when = 0:1, the threshold levels do sometimes dier by more than 1.) Finally, Tables 9 and 10 demonstrate that the value of h I does not aect the insights in the previous two paragraphs. This is because those tables show the results of the exact same numerical study as shown in Tables 1 and 2, except that h I = 0, and we nd that the explanations in the previous two paragraphs apply, with one exception. That exception is that sometimes the cost functions themselves are so 37 small, on the order of 10 6 or smaller, that the relative error in the cost approximation can be large, even though the absolute dierence is tiny. We conclude that the policy arising from the solution to the approximating dif- fusion control problem performs extremely well under a variety of system parameter assumptions. 38 Chapter 4. Stang and Dynamic Outsourcing under Uncertain Arrival Rate 4.1. Model Description We start by dening the notation we will use throughout the rest of the text: : Expected (mean) arrival rate. : Arrival rate. X : Random variable quantifying the variability of random arrival rate around its mean . c : Constant denoting the order of the variability (we assume, without loss of generality, that 0c 1). N : In-house stang level. c 1 : In-house stang cost per agent per unit time. c 2 : Outsourcing cost per call sent to outsourcer. a : Abandonment cost per call abandoning. : Abandonment rate. Y (t) : # of customers in system. A(t) : Poisson Process that counts arrivals by time t. ?(N; ) : Operating cost of the optimal policy ? given a stang level N and realizationof arrival rate as . e C u : Centered and scaled cost under admissable policy u. 39 We consider a call center where calls arrive according to a Poisson process with rate . We assume that the arrival rate itself is a random variable with cdfF . The call center manager has to decide the stang levelN in advance and once decided this stang level cannot be changed. We assume that the random arrival rate is learned on the y once the stang decision is done and the call center starts operating, at which time calls can also be sent to a third party outsourcer. Thus the outsourcing option not only helps to reduce congestion at busy times but also serves as a hedge against the uncertainty in the arrival rate itself and the resulting demand-capacity imbalance. Once the stang level is decided and the arrival is learned instantaneously the call center manager needs to decide on an outsourcing policy. At the time of a call arrival, if all in-house agents are busy, the system manager must decide whether to let an arriving customer wait until an in-house server becomes available, or to outsource that call so that the customer receives immediate service. Here, the cost trade-o is between outsourcing the customer at some cost and the decrease in quality of service due to long waiting times. We capture service quality through customer abandonments and a cost associated with each abandoning customer. The outsourcing decision of the call center manager, which we refer to as the oper- ating or outsourcing sub-problem hereafter, can be formulated as a Markov decision process (MDP) resembling the admission control problem studied here in Chapter 3 and in [40]. [40] show that, under certain conditions, the optimal policy is of threshold form, and provide an algorithm to compute the associated innite horizon expected average cost and the threshold level. Nevertheless, this methodology neither provides a sharp condition to determine when it is never optimal to outsource a customer, nor 40 results in a convenient analytic expression for the optimal policy. Thus [40] also solve the approximating diusion control problem (DCP) that arises in the Haln-Whitt QED limit regime and explicitly characterize the minimum achievable innite horizon average expected cost and optimal threshold level in that regime. Based on the DCP solution, the authors then propose a policy which is also of threshold form and show that it is asymptotically optimal. We follow the same approach for our operating sub-problem by identifying ap- proximately optimal policies that work well for large number of servers and arrival rates while we use MDP to numerically quantify the benets achieved through using a threshold policy for the outsourcing sub-problem. Note however that the heavy trac regime we're in is not necessarily QED and is determined by the order of uncertainty. In fact, for higher uncertainty levels, the DCP is quite dierent from the one that arised in QED. We suppress the contractual negotiation between the client and outsourcing com- pany and assume that outsourcing a call costs c 2 which can also include costs as- sociated with decreased quality of service at the outsourcer. We also assume that whenever a call is sent to the outsourcer it can be answered immediately. The in- house stang cost is assumed to be c 1 per unit time per agent. We assume that service times are exponential and that, without loss of generality, the mean service time of each call is normalized to 1. Calls waiting to be answered abandon at an exponential rate and each abandonment costs the call center manager a. Once the stang level is decided, the only control available to the call center manager is the outsourcing policy. We capture this control decision by the function :Z + !f0; 1g where Z + =f0; 1; 2;:::g is the set of non-negative integers. The system manager 41 outsources an arriving customer that nds y customers in the system if(y) = 1 and keeps the customer in-house otherwise. Observe that, once the random arrival rate is learned, the number of customers in system at timet,Y (t), evolves according to the continuos time Markov chain (CTMC) given by Y (t) := Y (0) +A(t) Z t 0 (Y (s))dA(s) (22) N s Z t 0 [N^X(s)]ds N R Z t 0 [X(s)N] + ds ; t 0: where N S and N R are two independent, standard Poisson processes and A denotes arrival process with for a realized rate . LettingA to denote the set of functions dened on the non-negative integers and having rangef0; 1g, then the cost up to time t under any admissable outsourcing policy will be given as (t) := Z t 0 c 2 (Y (s))dA(s) +aN R Z t 0 [Y (s)N] + ds ; t 0: (23) and given the stang level and a realization of the arrival rate, the objective of the call center manager for the dynamic outsourcing sub-problem will be to nd a policy ? that will achieve the minimum long run expected average cost over all policies dened as ?(N; ) := min 2A lim inf t!1 E [(t)] t (24) 42 Then the only upfront control at the discretion of the call center manager will be the stang level N, which must be decided based on its eect on the expected cost of the optimal outsourcing policy. Therefore, the objective of the call center manager is to nd a stang level N that minimizes the sum of the in-house stang cost and the expected cost of the optimal outsourcing sub-problem through the following stochastic program min N c 1 N +E [ ?(N; )] (25) where ?(N; ) is the cost of the optimal second stage policy, i.e., the optimal out- sourcing policy as a function of the stang decision N and realized arrival rate . Observe that nding an exact analytical solution to this stochastic problem is dicult if not impossible. Among other things, this observation is due to the fact that nd- ing an exact analytical expression for the outsourcing problem is not possible. Thus our approach will be to rst determine a proxy for ? (N; ) for large through a diusion approximation and then nd an approximately optimal stang level. From now on we index system parameters by the mean arrival rate and by denoting the associated quantities with a superscript. We present our asymptotic analysis in the next section. 4.2. Asymptotic Analysis We begin by introducing a stylized uncertainty structure regarding the randomness in the arrival rate to further facilitate our asymptotic analysis. To this end we assume that the constant and random arrival rate has the following form which was rst 43 introduced by [47]: = + c X (26) where c is the uncertainty coecient and X is a random variable with cdf F and E [X] = 0 so that =E [] is the mean arrival rate. Note that the random variable X measures the dispersion of the random arrival rate around its mean while the constant c determines the order of the dispersion. From now on we denote by (X) to stress the dependence of the arrival rate on the random variable X. For the purposes of our analysis it suces to assume that 0 c 1 and we shall do so from now on. Furthermore, the magnitude of c determines the relevant heavy trac regime once the arrival rate (X) is learned. Therefore, depending on the value of c the relevant operating sub-problems will have dierent characteristics. We elaborate on this in the next two sections where we analyze the resulting cases depending on the corresponding c value in detail. 4.2.1. Conventional Variability and QED (c = 1=2) We begin by analyzing the case c = 1=2 in detail. To this end, we consider a sequence of systems indexed by the mean arrival rate and restrict attention to admissable policies where an admissable policy refers to an entire sequence u := (N ; ) that species the upfront stang levelN and a dynamic outsourcing policy for a given realization X =x of the random arrival rate. Next, we dene the centered and scaled cost function for a system that operates under an admissable policy u. 44 Denition 1 For system operating under an admissable policy u, the centered and scaled cost function ~ C u is given by e C u := 1 1=2 n c 1 N c 1 +E X h ;? (N ; (X)) io (27) Finally, we dene an asymptotically optimal policy based on the above diusion scaling. Denition 2 A policy u ? is said to be asymptotically optimal if it is admissable, lim sup !1 ~ C u ? <1 and lim sup !1 ~ C u ? lim inf !1 ~ C u (28) We are now in a position to describe our proposed policy. Let ? solve min fc 1 +E X [b ?(X;l ? (X)) ]g (29) whereb ?(m;l) = A(m;l) B(m;l) as dened in [40] and l ? is the optimal threshold level that solves the associated DCP. Then our proposed policy is to sta according to N ;? :=N ( ? ) = + ? 1=2 (30) and for a realization of the random variable X as x, outsource according to ?; (y) := 1 n [yN ] + p l ? ( ? x) o (31) so that a newly arrived customer that sees y customers in systems is served in-house 45 if [yN ] + p l ? ( ? x) Dene the diusion process b Y (t), according to b Y (t) = b Y (0) + p 2B(t) + Z t 0 ;x b Y (s) ds ^ U(t); (32) where the process ^ U(t) satises ^ U(0) = h b Y (0)l i + (33) b Y (t)l for all t 0 Z 1 0 h l b Y (t) i + d ^ U(t) = 0: and the innitesimal mean ;x is given by ;x (y) = 8 > < > : m(;x)y y 0 m(;x) y y> 0 (34) Note that the constant m approximates the capacity imbalance of the original system, in which case m(;x) := lim !1 N p . Next, we proceed with three propositions that we will be used to establish the main result of this section. Proposition 2 Suppose c Y (0) := Y (0)N p ) b Y (0) as !1. Then, under the proposed policy, for any given realization X =x, c Y (t) := Y (t)N p ) b Y (t) (35) 46 and 1 1=2 n c 1 N c 1 + ;? (N; (X)) o !c 1 ? +b ?( ? x;l ? ( ? x)) (36) Proposition 3 Suppose c Y (0) := Y (0)N p ) b Y (0) as !1. Then, under the proposed policy ~ C ;? u ? := 1 1=2 n c 1 N ;? c 1 +E X h ;? (N; (X)) io !c 1 ? +E X [b ?(X;l ? (X)) ] (37) Proposition 4 For any 2< and d2< such that d6= 1=2, under any admissable policy u that has N = + d , lim !1 ~ C u ? =c 1 ? +E X [b ?( ? X;l ? ( ? X)) ] lim inf !1 ~ C u (38) We are now ready to present the main result in this section which establishes that our proposed policy is asymptotically optimal. Theorem 8 The proposed policy as given in (30) and (31) is asymptotically optimal with respect to Denition 2. 47 4.2.2. Moderate Variability and QED-c (1=2<c< 1) Next we present a partial analysis of the so-called moderate variability case. Throughout this subsection we assume that 1=2 < c < 1 and the following form for the stang level N :=N() = + c +o( p N) (39) As pointed out in [47], it is the scaling given in (39) that results in an expected non-degenerate delay probability together with asymptotically negligible expected waiting times and abandonment probabilities as in the well-known QED regime where the arrival rate is known (cf. Theorem 4.3 in [47]). Thus we will assume the call center will staed according to (39). (We conjecture that this stang policy along with an outsourcing policy that starts outsourcing immediately is asymptotically optimal). Under the stang described by (39), for any realization of X asx, the associated actual service grade coecient will be x. Then, depending on whether x < or x > , we will be in one of the cases highlighted in Theorem 4.1 (positive service grade) and Theorem 4.2 (negative service grade) in [47]. This means that we essentially have two separate diusion approximations for the operating problem depending on x, the realization of X. Case 1: x < In this case we are in a regime that resembles QD where the abandonment probability converges to zero at rate 1 c and the abandonment rate, ( +x c )P (ab), is O( 1c ). Then, if the abandonment cost is scaled by c , the abandonment cost will be asymptotically equal to 0. To see this, note that the scaled abandonment rate at the th system is (+x c )P (ab) c which is O( 12c ) and thus converges to 0 since c> 1=2. 48 Case 2: x> In this case we are in a regime that resembles ED. From Theorem 4.2 in [47] we see that the abandonment probability converges to zero at rate 1 1c . Similarly the abandonment rate isO( c ) and when scaled by c the abandonment cost will converge to a constant. Given these two cases we dene a centered-and-scaled version of the optimization problem (25): min 1 c fc 1 N()c 1 +F ()E X [(N(); (X))j X] + (1F ())E X [(N(); (X))j X >]g Our plan is to approach the problem as follows: 1. For a xed value of and an arbitrary value of x > , nd an asymptot- ically optimal solution, say ? , for the routing problem under case 2. Let ^ ?(; (x)) be the corresponding scaled operating cost, and let ^ ?(;x) = lim !1 ^ ?(; (x)). 2. Optimize the following problem over : min fc 1 + (1F ())E X [^ ?(;X)j X >]g (40) 49 3. Establish that the stang policy given in (39) is asymptotically optimal in the following sense: Denition 3 A stang policy N ? coupled with an operating policy ? is said to be asymptotically optimal if lim sup !1 c 1 N ? c 1 +E X [^ ?(;X) ] c lim inf !1 c 1 Nc 1 +E X [^ (;X) ] c for any other stang policy N and operating policy . Next we present some preliminary results that are part of our ongoing work. Theorem 9 Assume x > and let l (;x) be the cost of the dynamic control out- sourcing at threshold N +l p . Then l (;x) c !c 2 (x) (41) Theorem 10 Assume x > and let l (;x) be the cost of the dynamic control outsourcing at threshold N +l p . Then l (;x) c c 2 (x) p !l(ac 2 ) (42) Theorem 11 Dene b Y := Y N x c p where N = + c +o( 1=2 ) and 1=2 < c< 1 then ^ Y ) b Y where b Y is a diusion process with innitesimal driftm(y) = y and innitesimal variance 2 (y) = 2: 50 Theorem 12 Dene Y := Y N c where N = + c +o( 1=2 ) and 1=2 < c < 1 then Y ) Y where Y is a degenerate diusion process with innitesimal drift m(y) =x y + y and innitesimal variance 2 (y) = 0: 51 Chapter 5. Conclusions Throughout this dissertation we study two queueing problems that are related to stang and dynamic outsourcing for large-scale service systems and in particular call centers. In both problems the call center is modeled as a M/M/N+M queue where +M species that the so-called patience clock or the abandonment distribution is exponential and that each customer in queue independently abandons if he is still waiting in queue within an exponentially distributed amount of time. We measure the quality of service (QoS) by the linear abandonment cost that is incurred for each abandonment. We also assume that the call center company has the option of leveraging a third party outsourcer especially to manage the overload or the peak demand hours; a situation commonly known as co-sourcing. Finally, we assume that calls that are routed to the third party outsourcer for a per call fee can be answered immediately. The rst research article in this dissertation takes the in-house stang decision as given and focuses on the dynamic outsourcing decision. For this particle scenario, the real time control question we address is which calls to keep in-house and which calls to route to the outsourcer. Our main contribution here is to observe a con- nection between the decision of whether or not to outsource an arriving customer, and an admission control problem in a multi-server queue with abandonment. This is important because the admission control problem in a multi-server queue with abandonment can be solved either numerically through MDP analysis, or exactly in the so-called Quality-Eciency-Driven (QED) heavy trac regime. In Chapter 3, we establish that a simple threshold policy is optimal and results in signicant cost 52 savings. Moreover, by comparing the exact optimal policy from the MDP solution to that of the QED heavy trac approximation, we show that the control motivated by the QED heavy trac approximation works extremely well. The next question that arises is how to jointly decide on an upfront stang level and a dynamic outsourcing policy. The second article in this dissertation addresses the joint stang and dynamic outsourcing problem in a co-sourcing environment. We explicitly model the stang decision in addition to the dynamic outsourcing decision of the call center manager. Moreover we make our model more pertinent for the call center co-sourcing environment by assuming that the arrival rate itself is a random variable and requiring the stang decision to be made before this uncertainty is resolved. We assume that the call center company rst decides the in-house stang level which cannot be changed thereafter. Next, the call center begins operating and due to the large call volume, the realization of the random arrival rate can be inferred on the y. Depending on the realization of the arrival rate, the client company leverages the third party outsourcer by following the optimal outsourcing policy (which is the focus of the rst model studied in this dissertation). Since it is very dicult (if not impossible) to analytically characterize the optimal stang policy for this two-stage stochastic program, we focus on a heavy trac approximation that is expected to work well for large call volumes. For a particular uncertainty structure (c = 1=2 in Chapter 4), we characterize the asymptotically optimal stang policy and the asymptotically optimal outsourcing policy (for a given realization of the arrival rate). In particular, we show that the asymptotically optimal outsourcing policy is a threshold policy with a nite threshold level (i.e., co-sourcing) ifa>c 2 whereas the outsourcing option will not be leveraged 53 ifac 2 (i.e., no outsourcing). On the other hand, the asymptotically optimal stang policy is such that there is no in-house capacity investment if c 1 c 2 (i.e., complete outsourcing) whereas a QED stang policy is optimal if c 1 <c 2 . While our analysis in Chapter 4 focuses on the particular case c = 1=2, we conjecture that these results will carry on to case 0<c< 1=2 with appropriate adjustments to the service grade. The problems addressed throughout this dissertation can be extended in several directions. Our conjectures in Chapter 4 for the case 1=2 < c 1 still need to be validated. Another possible avenue for future research is to consider a random and time varying arrival rate or incorporating the learning eects explicitly. Finally, investigating how the client company and the third party outsourcer will design the optimal contract that species the cost and service level parameters which we took as given, is another direction for future research as almost all previous work on call center outsourcing contracts has focused on the complete outsourcing case. 54 Bibliography [1] Abramowitz, M. , I. A. Stegun. 1965. Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. Courier Dover Publications. [2] Adusumilli, K. M., J. J. Hasenbein. 2008. Dynamic Admission and Service Rate Control of a Queue. Working Paper, University of Texas, Austin, Texas. [3] Akan, M., B. Ata, M. A. Lariviere. 2007. Asymmetric Information and Economies of Scale in Service Contracting. Working Paper, Kellogg School of Management, Evanston, Illinois. [4] Aksin, O. Z., F. Karaesmen, L. Ormeci. 2007. A review of workforce cross-training in call centers from an operations management perspective. Workforce Cross Train- ing Handbook. CRC Press [5] Aksin, O. Z., M. Armony, V. Mehrotra. 2007. The Modern Call Center: A Multi-Disciplinary Perspective on Operations Management Research. Production and Oper. Management 16(6) 665-688. [6] Aksin, O. Z., F. de Vericourt, F. Karaesmen. 2008. Call Center Outsourcing Con- tract Design and Choice. Management Sci. 54(2) 354{368. [7] Armony, M., A. R. Ward. 2009. Fair Dynamic Routing in Large-Scale Heteroge- neous Server Systems. Forthcoming in Oper. Res. [8] Ata, B., J. M. Harrison, L. A. Shepp. 2005. Drift Rate Control of a Brownian Processing System. Annals of Applied Probability 15(2) 1145{1160. [9] Ata, B., S. Shneorson. 2006. Dynamic Control of an M/M/1 Service System with Adjustable Arrival and Service Rates. Management Sci. 52(11) 1778{1791. [10] L'Ecuyer, P., Avramidis, A. N. A. Deslauriers. 2004. Modeling Daily Arrivals to a Telephone Call Center. Management Sci. 50(7) 896{908. 55 [11] Baron, O., J. Milner. 2006. Stang to Maximize Prot for Call Centers with Al- ternate Service Level Agreements. Working Paper, University of Toronto, Toronto, Canada. [12] Bassamboo, A., A. Zeevi. 2009. On a Data-Driven Method for Stang Large Call Centers. Oper. Res. 57(3) 714{726. [13] Bassamboo, A., J. M. Harrison, and A. Zeevi. 2006. Design and Control of a Large Call Center: Asymptotic Analysis of an LP-Based Method. Oper. Res. 54(3) 419{435. [14] Bassamboo, A., R. Randhawa, A. Zeevi. 2009. Capacity planning in service sys- tems with arrival rate uncertainty: Safety stang principles revisited. Working Paper. [15] Billingsley, P. 1968. Convergence of probability measures. Wiley, New York. [16] Borovkov, A. A.. 1967. On Limit Laws for Service Processes in Multi-channel Systems. Siberian Math J. 8 746-763 (English Translation). [17] Borst, S., A. Mandelbaum, M.I. Reiman. 2004. Dimensioning Large Call Centers. Oper. Res. 52(1) 17{34. [18] Brown, L., N. Gans, A. Mandelbaum, A. Sakov, H. Shen, S. Zeltyn, L. Zhao. 2005. Statistical Analysis of a Telephone Call Center. Journal of the American Statistical Association 100(469) 36{50. [19] Boxma, O. J., P. R. de Waal. 1994. Multiserver Queues with Impatient Cus- tomers. ITC 14 743-756. [20] Browne, S. , W. Whitt. 1995. Piecewise-linear Diusion Processes Advances in queueing: Theory, methods, and open problems. CRC Press, 463{480. [21] Chen, B. P. K., S. G. Henderson. 2001. Two issues in setting call centre stang levels. Annals of Oper. Res. 108(1) 175{192. [22] Chen, W., D. Huang, A. Kulkarni, J. Unnikrishnan, Q. Zhu, P. Mehta, S. Meyn, and A. Wierman. 2009. Approximate Dynamic Programming using Fluid and Dif- fusion Approximations with Applications to Power Management. Working Paper. [23] Cil, E. B., E. L. Ormeci, F. Karaesmen. 2008. Eects of system parameters on the optimal policy structure in a class of queueing control problems. Queueing Systems 61(4) 273{304. 56 [24] Gans, N., Y. P. Zhou. 2007. Call-Routing Schemes for Call-Center Outsourcing Manufacturing & Service Oper. Management 9(1) 33{50. [25] Garnett, O., A. Mandelbaum, M. Reiman. 2002. Designing a Call Center with Impatient Customers. Manufacturing & Service Oper. Management 4(3) 208{227. [26] Gans, N., G. Koole, A. Mandelbaum. 2003. Telephone Call Centers: Tutorial, Review, and Research Prospects. Manufacturing and Service Oper. Management 5(2) 79{141. [27] George, J. M., J. M. Harrison. 2001. Dynamic Control of a Queue with Adjustable Service Rate. Oper. Res. 49(5) 720{731. [28] Ghosh, A. P., A. P. Weerasinghe. 2007. Optimal Buer Size for a Stochastic Processing Network in Heavy Trac. Queueing Systems 55(3) 147{159. [29] Ghosh, A. P., A. P. Weerasinghe. 2008. Optimal Buer Size and Dynamic Rate Control for a Queueing Network with Impatient Customers in Heavy Trac. Work- ing Paper. [30] Gilson, K. A., D. K. Khandelwal. 2005. Getting more from call centers. The McKinsey Quarterly (April, 2005). [31] Grassmann, W. K. 1988. Finding the Right Number of Servers in Real-World Queuing Systems. Interfaces 18(2) 94{104. [32] Gurvich, I., J. Luedtke, T. Tezcan. 2009. Stang Call Centers with Uncer- tain Demand Forecasts: a Chance-Constrained Optimization Approach. Technical report, Working Paper, Kellogg School of Management. [33] Haln, S., W. Whitt. 1981. Heavy-trac Limits for Queues with Many Expo- nential Servers. Oper. Res. 29(3) 567{588. [34] Harrison, J. M., A. Zeevi. 2005. A Method for Stang Large Call Centers Based on Stochastic Fluid Models. Manufacturing & Service Oper. Management 7(1) 20{ 36. [35] Hasija, S., E. J. Pinker, R. A. Shumsky. 2008. Call Center Outsourcing Contracts Under Information Asymmetry. Management Sci. 54(4) 793{807. [36] Incoming Call Center ManagementInstitute(ICMI). 2007. ICMI's Contact Center Outsourcing Report - Key Findings http://www.callcentermagazine.com/showArticle.jhtml?articleID=201805251 57 [37] Incoming Call Center Management Institute.2007.Baird Report: Investor Inter- est in Outsourcing Thrives. (7/18/2007) [38] Jagerman, D. L. 1974. Some properties of the Erlang loss function. Bell System Tech. Journal 53(3) 525{551. [39] Jongbloed, G., G. Koole. 2001. Managing uncertainty in call centres using Pois- son mixtures. Applied Stochastic Models in Business and Industry 17(4) 307{318. [40] Ko ca ga, Y. L., A. R. Ward. 2010. Admission Control for a Multi-Server Queue with Abandonment. Queueing Systems 65(3) 275{323. [41] Koole, G. 1998. Structural Results for the Control of Queueing Systems using Event-Based Dynamic Programming. Queueing Systems 30(3) 323{339. [42] Koole, G., A. Mandelbaum. 2002. Queueing Models of Call Centers: An Intro- duction. Annals of Oper. Res. 113(1-4) 41{59. [43] Koole, G. 2006. Monotonicity in Markov Reward and Decision Chains: Theory and Applications. Foundation and Trends in Stochastic Systems 1(1) 1{76. [44] Koole, G., A. Pot. 2006. A Note on Prot Maximization and Monotonicity for Inbound Call Centers. Working Paper. Department of Mathematics, Vrije Univer- siteit Amseterdam, The Netherlands. [45] Koole, G., A. Pot. 2006. An overview of routing and stang algorithms in multi- skill customer contact centers. Technical Report, Department of Mathematics, Vrije Universiteit Amseterdam, The Netherlands. [46] L'Ecuyer, P. 2006. Modeling and optmization problems in contact centers. Pro- ceedings of the Third International Conference on the Quantitative Evaluation of Systems (QEST 2006) 145{154 [47] Maman, S. 2009. Uncertainty in the demand for service: The case of call centers and emergency departments. M.S. Thesis. Technion-Israel Institute of Technology. [48] Meyn, S. 2005. Workload Models for Stochastic Networks: Value Functions and Performance Evaluation. IEEE Transactions on Automatic Control 50(8) 1106{ 1122. [49] Milner, J. M., T. Olsen. 2008. Service-level agreements in call centers: Perils and Prescriptions. M anagement Sci. 54(2) 238{252. 58 [50] Pang, G., R. Talreja, W. Whitt. 2007. Martingale proofs of Many-Server Heavy- Trac Limits for Markovian Queues. Probability Surveys 4 193{267. [51] Puterman, M. L. 1994. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc. New York, NY, USA. [52] Ren, Z. J., Y. P. Zhou. 2008. Call Center Outsourcing: Coordinating Stang Level and Service Quality. Management Sci. 54(2) 369{383. [53] Rogers, L. C. G., D.Williams. 2000. Diusions, Markov Processes, and Martin- gales. Volume 2, Ito Calculus. Cambridge University Press. [54] Ross, A. M. 2001 Queueing systems with daily cycles and stochastic demand with uncertain parameters. Ph.D. Dissertation. University of California at Berkeley. [55] Steckley, S. G., S. G. Henderson, V. Mehrotra. Forecast errors in service systems. 2009 Probability in the Engineering and Informational Sciences 23(2) 305{332. [56] Steckley, S. G. , S. G. Henderson, V. Mehrotra. 2009. Service System Planning in the Presence of a Random Arrival Rate. [57] Stone, C. 1963. Limit theorems for random walks, birth and death processes, and diusion processes. Illinois J. Math 7(4) 638{660. [58] Thompson, G. M. 1999. Server stang levels in pure service environments when the true mean daily customer arrival rate is a normal random variate. Unpublished manuscript. [59] Ward, A. R., P. W. Glynn. 2003. Properties of the re ected Ornstein-Uhlenbeck process. Queueing Systems 44(2) 109{123. [60] Ward, A. R., S. Kumar. 2008. Asymptotically Optimal Admission Control of a Queue with Impatient Customers. Math. Oper. Res. 33(1) 167{202. [61] Weerasinghe, A., A. Mandelbaum. 2008. A Many Server Controlled Queuing System with Impatient Customers. Working Paper [62] Whitt, W. 1984. Heavy Trac Approximations for Service Systems with Block- ing. AT&T Bell Lab. Tech. J. 63 689{708. [63] Whitt, W. 2006. Fluid Models for Multiserver Queues with Abandonments. Oper. Res. 54(1) 37{54. [64] Whitt, W. 2006. Stafng a Call Center with Uncertain Arrival Rate and Absen- teeism. Production and Oper. Management 15(1) 88{102. 59 Appendices In this Technical Appendix we provide proofs for the various results stated in the dissertation titled: \Capacity Sizing and Dynamic Outsourcing for Large Scale Ser- vice Systems". The Technical Appendix is divided into two subsections. Section A provides the proofs of the results stated in the main paper body. Tables 3 through 8, that give further supporting numerics for the conclusions drawn in Section 3.4., are found in Section B. Appendix A: Proofs Proof of Theorem 1: Letf i (p) :i2f1; 2;:::gg be a sequence of independent Bernouilli random variables whereP ( i (p) = 1) =p andP ( i (p) = 0) = 1p. Under the policy S =fp(x) :x2 f0; 1;:::gg, the number-in-system process X in (1) evolves according to X(t) :=X(0) +A(t) A(t) X i=1 i (p(X(t i ))) S Z t 0 [N^X(s)]ds L Z t 0 [X(s)N] + ds ; t 0; where t i = P i j=1 u j is the arrival time of the ith customer. The process X is a semi-Markov process, and, for any S 2 S , the system is stable due to customer abandonments. Then, E [(t)] =E Z t 0 h(X(s))ds +cE 2 4 A(t) X i=1 i (p(X(t i ))) 3 5 : 60 Since E h R t 0 h(X(s))ds i t !E [h(X(1))] E h P A(t) i=1 i (p(X(t i ))) i t !E [p(X(1))]; as t!1; it follows that E [(t)] t !E [h(X(1))] +cE [p(X(1))]: We now show that zE [h(X(1))] +cE [p(X(1))]: Letf n ;n2f0; 1;:::gg be the steady-state probabilities that represent the long-run average amount of time X spends in state n when operating under S . These must solve the balance equations (1p(n)) n =(n + 1) n+1 ; n2f0; 1;:::g: (A-43) It follows from (4) that (1p(n)) [(n)y(n)y(n + 1)] (1p(n)) [h(n)z] p(n) [(n)y(n)c]p(n) [h(n)z]; n2f0; 1;:::g; 61 and so (n)y(n) (1p(n))y(n + 1)p(n)ch(n)z; n2f0; 1;:::g: Multiplying by n and using the balance equations (A-43) yields (1p(n1))y(n) n1 (1p(n))y(n+1) n p(n)c n (h(n)z) n ; n2f1; 2;:::g: Summing over n shows that 1 X n=1 (1p(n 1))y(n) n1 (1p(n))y(n + 1) n +z 1 X n=1 n 1 X n=1 n (h(n) +cp(n)): Since the sequencefy(n) :n2f0; 1;:::gg is uniformly bounded by assumption, (1p(n))y(n + 1) n ! 0 as n!1; and so 1 X n=1 (1p(n 1))y(n) n1 (1p(n))y(n + 1) n = (1p(0))y(1) 0 : Hence (1p(0))y(1) 0 +z 1 X n=1 n 1 X n=1 n (h(n) +cp(n)): Note that, since y(1)c, it follows from (5) that zh(0) ((1p(0))y(1) +p(0)c): 62 Hence (zh(0)) 0 p(0)c 0 +z 1 X n=1 n 1 X n=1 n (h(n) +cp(n)); which, using the fact that P 1 n=0 n = 1, implies z 1 X n=0 n (h(n) +cp(n)): Since the n are the steady-state probabilities associated with the policy S , 1 X n=0 n (h(n) +cp(n)) =E [h(X(1))] +cE [p(X(1))]: Proof of Lemma 1: It follows from (7) that y n 1 (1) =z n 1 h(0) y n 2 (1) =z n 2 h(0): Hence y n 1 (1)>y n 2 (1). Next, it follows from (6) that z n 1 =h(1)(1)y n 1 (1) +y n 1 (2) z n 2 =h(1)(1)y n 2 (1) +y n 2 (2): 63 Hence y n 1 (2) > y n 2 (2). Continued iteration of the last two sentences shows that y n 1 (k)>y n 2 (k) for k2f1;:::; min(n 1 ;n 2 ) + 1g. Proof of Lemma 2: Without loss of generality, in this proof only, let = 1. The proof is by induction. For the base case, rst note that it follows from (6)-(8) that z 0 =Nh I +c, and h(1)z 1 =(1)y 1 (1)y 1 (2) y 1 (1) =z 1 h(0) y 1 (2) =c: Solving the above equations for z 1 shows z 1 = h I (N 1) +h I N +c 1 + : Since h I (N 1) +h I N +c 1 + =Nh I +c h I +c 1 + <z 0 ; we conclude that z 1 <z 0 . Next, select any n2f2; 3;:::;N 1g, and suppose that z n <z n1 <z 1 <z 0 : To complete the proof, it is sucient to show that z n+1 < z n . The argument is by contradiction. Suppose not; i.e., suppose that z n+1 z n . It follows from Lemma 1 64 that y n (k)<y n1 (k) for all k2f1; 2;:::;ng; and y n (k)y n+1 (k) for all k2f1; 2;:::;n + 1g: Since from (8), y n1 (n) =y n (n + 1) =c, it follows that y n (n)<c and y n+1 (n + 1)c: Equations (6)-(8) show that z n =h(n)(n)y n (n) +c z n+1 =h(n + 1)(n + 1)y n+1 (n + 1) +c; and so z n >h(n)(n)c +c z n+1 h(n + 1)(n + 1)c +c: The fact that h(n)>h(n + 1) and (n)<(n + 1) implies z n >h(n + 1)(n + 1)c +c: We conclude that z n >z n+1 , which is a contradiction. 65 Proof of Theorem 2: LetmN be such thatz m <z k for allk2f0; 1;:::;m 1g andz m z m+1 . Assume also that a>c. For =z m+1 z m 0 dene the modied problem as ~ (k) = 8 > < > : (k) for k2f0; 1;:::;mg (m + 1) for k2fm + 1;m + 2;:::g and ~ h(k) = 8 > < > : h(k) for k2f0; 1;:::;mg h(m + 1) for k2fm + 1;m + 2;:::g and nally y(k) = 8 > < > : y m (k) for k2f0; 1;:::;mg y m+1 (m + 1) for k2fm + 1;m + 2;:::g The idea is to show that if the threshold levelm is optimal for this modied problem then it is also optimal for the original problem. This will be done by assuming there exists k>m with z k <z m and arriving at a contradiction. It is immediate by Lemma 1 thaty m (k)<c for allk2f0; 1;:::;mg andy m+1 (m+ 1) c. Then letting z = z m , it is straightforward to verify that (z; (y(1);y(2);:::)) satises the optimality equations (4)-(5) for the modied problem and therefore the threshold level m is optimal with associated optimal average cost z m . From this point on, let ~ z k be the long run average expected cost of the policy with threshold level k for the modied problem. Similarly, let ~ y k (:) be the relative cost dierence for the associated modied problem. Note that these values will coincide with those of the original problem for km + 1. 66 Now assume that m is not the optimal threshold level for the original problem. Then there exists nm + 2 such that n = inf km + 2 :z k <z m To proceed it is enough to prove that ~ z n <z n which would imply such ann cannot exist because we knowz m = ~ z m < ~ z n as threshold levelm is optimal for the modied problem. Now we show that ~ z n < z n . First note from Lemma 1, y n (k) < c for all k 2f0; 1;:::;ng. From the n-terminating equation (6) for the original and modi- ed problems we have h(n)z n = (n)y n (n)c and ~ h(n) ~ z n = ~ (n)~ y n (n)c: Now assume ~ z n z n . Then subtracting the above equations from each other ~ z n z n = ~ h(n)h(n) + (n)y n (n) ~ (n)~ y n (n) = (m + 1)a na + (N + (nN) )y n (n) (N + (m + 1N) )~ y n (n) = (nm 1) (ay n (n)) (m + 1)(~ y n (n)y n (n)) < 0 where the last inequality follows since y n (n) < c < a and y n (n) < ~ y n (n). To see 67 how the latter follows, rst note that, under the assumption ~ z n z n , the logic of Lemma 1 applies directly for states k = 1; 2;:::;m + 1 giving y n (k) ~ y n (k). To see that y n (k) ~ y n (k) for km + 2 we rewrite the n-terminating equation (6) for the original and modied problem to get y n (k) = (k 1)y n (k 1) +z n h(k 1) (A-44) and ~ y n (k) = ~ (k 1)~ y n (k 1) + ~ z n ~ h(k 1): (A-45) Hence it follows immediately thaty n (m + 2) ~ y n (m + 2). Next subtracting equation A-45 from equation A-44 with k =m + 3 we get (y n (m + 3) ~ y n (m + 3)) = (m + 2)y n (m + 2) +z n h(m + 2) ~ (m + 2)~ y n (m + 2) ~ z n + ~ h(m + 2) = ( (m + 1) + )y n (m + 2) (m + 1)~ y n (m + 2) +z n ~ z n h(m + 2) +h(m + 1) < (m + 1)(y n (m + 2) ~ y n (m + 2)) + c a < 0 where the inequalities follow since y n (m + 2) ~ y n (m + 2) and y n (m + 2) < c < a. Thusy n (m + 3)< ~ y n (m + 3). An inductive argument similar to Lemma 1 establishes y n (k) < ~ y n (k) for all m + 2 < k n, and so it follows similarly that y n (m +j) < ~ y n (m +j) for all j2f3; 4; 5;:::;nmg. Thus our assumption cannot be correct 68 and it must be true that ~ z n <z n . This in turn implies such an n cannot exist which ultimately shows that z m <z k for all k2f0; 1;:::g. Proof of Theorem 3: Consider the modied holding cost function ^ h(x) = 8 > < > : h(x) for xn h(n) for x>n: Similar to Lemma 2 in [2] there exists (^ z n ; (^ y n (1);:::; ^ y n (n)) that satises the optimal- ity equations and has ^ y n (k) c for all k n. Hence, it is never optimal to reject a customer. Let p n (i) and p 1 (i) be the steady state probability of being at state i under the n-terminating policy and the policy that does not exercise any control respectively. Observe that p n (i) = i Q i j=1 j p n (0) and p 1 (i) = i Q i j=1 j p 1 (0): Using P i p n (i) = 1 we get p n (0) = 1 1 + n X i=1 i i Y j=1 j and p 1 (0) = 1 1 + 1 X i=1 i i Y j=1 j : By construction z n ^ z n +p n (n)c + n X i=1 [p n (i)p 1 (i)]h(i): 69 Furthermore, since ^ h(i)h(i) for all i2f0; 1;:::g, it follows that z n z ? +p n (n)c + n X i=1 [p n (i)p 1 (i)]h(i): It is trivial to see that p n (n)! 0 as n!1. Thus if we can show that n X i=0 [p n (i)p 1 (i)]h(i)! 0 as n!1 (A-46) then lim inf n!1 z n lim sup n!1 z n z ? : Sincez n z ? for eachn and the sequence of solutionsfz n g is decreasing by assump- tion, we can conclude that lim n!1 z n =z ? : We now show that (A-46) is correct. Note that n X i=0 [p n (i)p 1 (i)]h(i) = n X i=0 [p n (0)p 1 (0)] i Q i j=1 j h(i) = p n (0)p 1 (0) p 1 (0) n X i=0 p 1 (i)h(i): Dene i ? = inf n i : i < 1 o and note that i < i for alli>i since i is increasing. Then p 1 (i) = ii Q i j=i +1 j p 1 (i )< i ii p 1 (i ) and so 1 X i=0 p 1 (i)h(i)< i X i=0 p 1 (i)h(i) +p 1 (i ) 1 X i=0 i i h(i +i)<1: 70 Since p n (0)!p 1 (0) as n!1 lim n!1 n X i=0 [p n (i)p 1 (i)]h(i) = lim n!1 p n (0)p 1 (0) p 1 (0) n X i=0 p 1 (i)h(i) = 0: (A-47) We conclude that (A-46) is valid. Proof of Theorem 4 Consider the modied problem with holding costs (h 0 (0);:::;h 0 (n 1);h(n);h(n);:::) with h 0 (k) =h(k) n for k<n where n := (cy n (n)): Observe that the pair z n n ; (y n (1);y n (2);:::;y n (n);y n+1 (n + 1);y n+1 (n + 1);:::) satises the optimality equations for the modied problem with lower holding costs implying that the optimal cost of the original problem cannot be any less, i.e., z n n z ? : Proof of Lemma 3: By (9) and Ito's formula, V ^ X(t) =V ^ X(0) + Z t 0 2 2 V 00 ^ X(s) +m ^ X(s) V 0 ^ X(s) ds + Z t 0 V 0 ^ X(s) dB(s) Z t 0 V 0 ^ X(s) d ^ U(s): 71 Since ^ U satises (33) and V satises (13), it follows that V ^ X(t) =V ^ X(0) +t Z t 0 ^ h ^ X(s) ds + Z t 0 V 0 ^ X(s) dB(s)c ^ U(t): The stochastic integral is a martingale because the function V 0 is bounded, and so E h V ^ X(t) i =E h V ^ X(0) i +tE h ^ (t) i : (A-48) The function V has bounded rst derivative, and so it follows that there exists a constant A such that jV (x)jAjxj for all x2<: Hence EV ^ X(t) E V ^ X(t) AE ^ X(t) : Therefore, if we can show that E ^ X(t) t ! 0 as t!1; (A-49) it follows that EV ^ X(t) t ! 0 as t!1: 72 Then, dividing by t and taking the limit as t!1 in (A-48) shows that lim t!1 E h ^ (t) i t =: To complete the proof, we establish (A-49). For this, it is useful to rst note that ^ X(t) st ^ Z(t) for every t 0, where st represents stochastically less then and ^ Z satises the stochastic equation ^ Z(t) = ^ Z(0) +B(t) + Z t 0 m( ^ Z(s))ds: where ^ Z(0) = ^ X(0). Then, the arguments used to prove Lemma EC.1 in [7] can also be used to show that ^ Z(t) st ^ X OU (t) for every t > 0, where ^ X OU (t) is a regulated Ornstein-Uhlenbeck process on [0;1) with innitesimal driftm x and innitesimal variance 2 . Next, let ^ X 0 (t) satisfy (9) under a threshold policy that regulates at the level l = 0, so that ^ X 0 (t) is a regulated Ornstein-Uhlenbeck process on (1; 0] with innitesimal driftmx and innitesimal variance 2 . Then, ^ X 0 (t) st ^ X(t). We conclude that ^ X(t) ^ X 0 (t) + ^ X OU (t) for every t 0: It is known that a regulated Ornstein-Uhlenbeck process has a nite rst transient moment that converges to its nite steady-state moment as t!1. Hence E ^ X 0 (t) t ! 0 and E ^ X OU (t) t ! 0; as t!1; 73 which implies E ^ X(t) t ! 0; as t!1: Proof of Lemma 4: It is sucient to show that the terms multiplyingc,a, andh I = are all positive. This is because the denominator is non-negative, and does not depend on c, h I , or a. The term multiplying c is exp 2 l 2 + 2 m l , which is positive. For the term multiplying a, 0 B @ 1 exp 2 l 2 + 2 m l + 2 q m exp m 2 2 m q 2 p 2 l + m 1 C A ; rst dene f(l) := m r 2 p 2 l + m + m r 2 m r 2 p 2 l + m ; and note that 0 B @ 1 exp 2 l 2 + 2 m l + 2 q m exp m 2 2 m q 2 p 2 l + m 1 C A = p 2 exp m 2 2 f(l): 74 Since f(0) = 0, and f 0 (l) = 2 2 l p 2 l + m > 0; we conclude that p 2 exp m 2 2 f(l)> 0 for all l 0: Finally, the coecient for h I = is 1 + 2 r m exp m 2 2 m r 2 = p 2 exp m 2 2 m r 2 + m r 2 m r 2 : To see this term is positive, it is sucient to establish that (x) +x(x) 0 for all x2<. This is obvious when x 0. Otherwise, when x < 0, we use relation 7.1.13 from [1] e x 2 Z 1 x e t 2 dt 1 x + q x 2 + 4 ; x 0: (A-50) In this case (x< 0 so thatx> 0), by (A-50), xe x 2 =2 (x) +e x 2 =2 (x) = x p e x 2 =2 Z 1 x= p 2 e y 2 dy + 1 p 2 > x p 1 x p 2 + q x 2 2 + 4 + 1 p 2 = p 2x + x p 2 + q x 2 2 + 4 p 2 x p 2 + q x 2 2 + 4 > 0: 75 We conclude (x) +x(x) 0 for all x2<. Proof of Proposition 1: We begin by deriving 0 (l) and 00 (l). For this, it is useful to rst note that A(l) is equivalently written as A(l) = h I 1 + 2 r m exp m 2 2 m r 2 +a 1 + 2 r m exp m 2 2 m r 2 + p 2 exp m 2 2 (ca) p 2 l + m p 2 am p p 2 l + m ! ; so that A 0 (l) = 2 2 p 2 exp m 2 2 p 2 l + m ((ac) lcm): Also, B 0 (l) = 2 2 p 2 exp m 2 2 p 2 l + m : Then, 0 (l) = B(l)A 0 (l)A(l)B 0 (l) B(l) 2 = A 0 (l)B 0 (l)(l) B(l) ; and substitution yields 0 (l) = 2 p 2 2 exp m 2 2 p 2 l + m B(l) ((ac) lcm(l)): 76 It then follows that 00 (l) = 2 p 2 2 exp m 2 2 p 2 (l + m ) B(l) (ac) 0 (0) 2 2 (ac) lcm(l) B(l) h(l) for h(l) :=B(l)( l +m) + p 2 exp m 2 2 p 2 l + m : We next argue that 0 (0)< 0. Since 0 (0) = p 2 exp m 2 2 exp m 2 2 m q 2 m q 2 ((0) +cm); to see that 0 (0)< 0, it is sucient to show that (0) +cm> 0: It follows from the expression for that (0) = p 2 p c + h I exp m 2 2 m q 2 + h I m ; and so (0) +cm = c + h I p 2 m q 2 m r 2 + m r 2 m r 2 : 77 We conclude that (0) +cm > 0 because (x) +x(x) > 0 for all x2<, as shown in the last paragraph of the proof of Lemma 4. Suppose a>c. Any stationary point l must satisfy 0 (l) = 0, which implies (ac) lcm(l) = 0: (A-51) At the point l, it follows from the above equality that 00 (l) = 2 p 2 2 exp m 2 2 p 2 l + m B(l) (ac) > 0: Hence any point l that satises (A-51) will be a local minimum. To see that such a point exists, because 0 (0)< 0, it is sucient to show that for some l> 0, 0 (l)> 0. This follows from the expression for 0 (l) and the fact that (ac) lcm(l)!1 as l!1; because lim l!1 (l) <1. Note also that this point is unique as we cannot have another local minimum without having a local maximum rst. Thus we dene l ? as the unique point that solves (ac) l ? cm(l ? ) = 0: Suppose a<c. It is sucient to show that no local minima exist. This argument is by contradiction. Let l be such that 0 (l) = 0. Then, it follows from the expression for 00 (l) that 00 (l)< 0, and sol is a local maximum. But since 0 (0)< 0, there must 78 exist l2 (0;l) such that 0 (l) = 0 is a local minimum, and so 00 (l) > 0. This is a contradiction, because the expression for 00 implies that 00 (l)< 0. Finally, in the case that a =c, we show directly that 0 (l) 0 for all l 0. For this, it follows from the expression for 0 (l) that it is sucient to show cm +(l) 0 for all l 0: Since when a =c (l) +cm = 0 B B @ h I 1 + 2 q m exp m 2 2 m q 2 +c 1 +m 2 q m exp m 2 2 m q 2 p 2 l + m +B(l) 1 C C A B(l) ; and we established in the proof of Lemma 4 that 1 + 2 r m exp m 2 2 m r 2 0; it is sucient to show that 1 +m 2 r m exp m 2 2 m r 2 p 2 l + m +B(l) 0: 79 This follows because 1 +m 2 r m exp m 2 2 m r 2 p 2 l + m +B(l) = exp m 2 2 p 2 m r 2 + p 2 p m m r 2 ! ; and (x) +x(x) > 0 for all x2<, as shown in the last paragraph of the proof of Lemma 4. Proof of Corollary 1: This is immediate from Lemma 3 and Proposition 1, because V 00? 2 ) = 0 so that the function V is twice continuously dierentiable. Proof of Lemma 5: Let4U(t) :=U(t)U(t ) and4V ^ X(t) :=V ^ X(t) V ^ X(t ) . Dene ^ U(t) := ^ U(t) X 0st 4 ^ U(s) to be the continuous part of U. Then, by (9) and the generalized Ito formula, V ^ X(t) =V ^ X(0) + Z t 0 2 2 V 00 ^ X(s) +m ^ X(s) V 0 ^ X(s) ds + Z t 0 V 0 ^ X(s) dB(s) Z t 0 V 0 ^ X(s) d ^ U(s) + X 0st 4V ^ X(s) : 80 Dene (t) = Z t 0 V 0 ^ X(s) d ^ U(s) X 0st 4V ^ X(s) c ^ U(t): Then the above equation is equivalent to V ^ X(t) V ^ X(0) +(t) + ^ (t) = Z t 0 2 2 V 00 ^ X(s) +m ^ X(s) V 0 ^ X(s) +h( ^ X(s)) ds + Z t 0 V 0 ^ X(s) dB(s): Hence it follows from (20) that V ^ X(t) V ^ X(0) +(t) + ^ (t)t + Z t 0 V 0 ^ X(s) dB(s): (A-52) Since V 0 (x)c for all x2<, (t) Z t 0 cd ^ U(s) X 0st 4V ^ X(s) c ^ U(t) = X 0st c4 ^ U(s) +4V ^ X(s) : Furthermore, because4V ^ X(t) =V ^ X(t) V ^ X(t) +4 ^ U(t) , 4V ^ X(t) +c4 ^ U(t) = Z ^ X(t)+4 ^ U(t) ^ X(t) (cV 0 (x))dx 0; it follows that (t) 0 for all t 0. Then, it follows from (A-52) that V ^ X(t) V ^ X(0) + ^ (t)t + Z t 0 V 0 ^ X(s) dB(s): 81 The stochastic integral is a martingale because the function V 0 is bounded, and so E h V ^ X(t) i E h V ^ X(0) i +E h ^ (t) i t: (A-53) Since E ^ X(t) =t! 0 as t!1 under an admissible control, and, as in the proof of Lemma 5, there exists a constant A such thatjV (x)j Ajxj for all x 2 <, E[V ( ^ X(t))]=t! 0 as t!1. We conclude that dividing by t in (A-53) yields lim inf t!1 E h ^ (t) i t : Proof of Theorem 5: Proof of (i): It follows from Corollary 1 that lim t!1 E h ^ ? T (t) i t =(l ? ): Hence to prove (i), it is sucient to show that the conditions of Lemma 5 are satised. For this, rst observe that, by construction, the function V in (18) is twice contin- uously dierentiable on [0;l]. Since, also by construction, V 0 (x) = c for x l ? , it is enough to show that V 0 (x) satises (20) also for x>l ? and a>c and the functions V 0 1 and V 0 2 are increasing. We start by showing that V 0 (x) satises (20) for x>l ? . To see this observe that 82 l ? satises (19) and V 00 (x) = 0 for xl ? . Thus for xl ? 2 2 V 00 (x) +m (x)V 0 (x) + ^ h (x) = (m + x)c +a x = mc + (ac)x mc + (ac)l ? = (l ? ); (A-54) where the last equality follows since a>c. Thus V 0 (x) satises (20) for all x2< Next we show that the function V 0 1 is strictly increasing. For this, rst note that the function (x)=(x) is strictly increasing because d dx (x) (x) = (x) +x(x) (x) ; and(x)+x(x)> 0, as shown in the last paragraph of the proof of Lemma 4. Then, since it follows from the expression for V 0 1 in (15) that V 0 1 is equivalently written as V 0 1 (x) = p 2 1 p (l) h I m p 2 x + m p 2 x + m h I ; it is sucient to show (l) h I m > 0 for all l 0 (A-55) to conclude that V 0 1 is strictly increasing. It follows from the expression for in (17) 83 that (l) h I m = 0 B B B B B B B B @ h I 1 2 q m exp m 2 2 p 2 l + m m q 2 +a 0 B @ 1 exp 2 l 2 + 2 m l + 2 q m exp m 2 2 m q 2 p 2 l + m 1 C A +c exp 2 l 2 + 2 m l 1 C C C C C C C C A 2 p 1 p exp m 2 2 m q 2 + 1 p exp m 2 2 p 2 l + m m q 2 : We have already shown in the proof of Lemma 4 that the term multiplying a is positive, and the term multiplying c is clearly positive. It remains to show the term multiplying h I = is positive. Since is increasing, 1 2 r m exp m 2 2 p 2 l + m m r 2 1 2 r m exp m 2 2 1 m r 2 ; and 1 m q 2 = m q 2 by the properties of the normal cdf, it is enough to show that 1 2 r m exp m 2 2 1 m r 2 0: 84 This follows because 1 2 p 1 p exp m 2 2 m r 2 = p 2 exp m 2 2 m r 2 m r 2 m r 2 = p 2 exp m 2 2 m r 2 m r 2 m r 2 0; because (x) +x(x) 0 for all x2<, as shown in the last paragraph of the proof of Lemma 4. We next show the function V 0 2 is strictly increasing. Since V 0 2 satises 2 2 V 00 2 (x) (m + x)V 0 2 (x) +a x = for all 0xl ? ; (A-56) it is sucient to show the more general claim that any solution to (A-56) is increasing (and not only solutions that haveV 0 2 (l ? ) =c). SetW (x) =V 0 2 (x) and take derivatives of both sides of (A-56) to nd 2 2 W 00 (x) (m + x)W 0 (x) W (x) +a = 0: (A-57) The argument is by contradiction. Suppose W (x) 0 for some x2 (0;l ? ). Let 2 (0;l ? ) be dened by W () = min x2(0;l ? ) W (x): 85 Then, W 0 () = 0;W 00 () 0, and W () 0. Plugging into (A-57) shows 2 2 W 00 () (m + )W 0 () W () +a a > 0: This is a contradiction, because the right-hand side of (A-57) is 0. Proof of (ii): We have already shown, at the end of Section 3.3.3., that h I E h ^ X(1) i +a E h ^ X(1) + i = lim l!1 (l): It remains to show lim inf t!1 E h ^ (t) i t 0 : (A-58) For this, rst deneV as in (18), except modify the denition ofV 0 2 in (16) as follows V 0 2 (x) := 2 r 0 +am exp 2 x + m 2 ! p 2 x + m 1 +a: The function V 0 2 and the constant 0 solve the ode 2 2 V 00 (x) +m(x)V 0 (x) + ^ h(x) = for all x2<: Furthermore, the function V 0 is bounded because it is continuous and lim x!1 V 0 2 (x) = h I and lim x!1 V 0 2 (x) =a: Finally, the arguments in (i) establish that V 0 is increasing, and so we can conclude that V 0 (x) c for all x2<. Therefore, the conditions of Lemma 5 are satised, 86 which establishes (A-58). Proof of Theorem 6 To see that we are in the setting of Theorem 7.6 in [50], note that the admission control policy N is such that the queue behaves exactly as a M/M/N+M model with nite buer size p Nl. Proof of Theorem 7 Proof of (i): It follows from (2) that E t; ?;N t = h I E h R t 0 [NX(s)] + ds i t +c N E h R t 0 1 n X(s)N p Nl ? o dA N (s) i A N (t) A N (t) N t +a E h N R R t 0 [X(s)N] + ds i t : Taking the limit as t!1 in the above shows that lim t!1 E t; ?;N t =h I E h N ^ X N (1) i + +c N P ^ X N l ? +a E h ^ X N (1)N i + : Let ^ X ? be the diusion in (9) under the threshold policy ^ U ? T in Theorem 5 that 87 regulates at the level l ? . It follows from Lemma 3 that (l ? ) = lim t!1 E h ^ ? (t) i t : It is straightforward to see that lim t!1 R t 0 E h ^ X(s) i ds t =E h ^ X(1) i and lim t!1 R t 0 E h ^ X(s) + i ds t =E h ^ X(1) + i ; the representation for ^ in (10) implies that it remains to show (a) E[NX N (1)] + p N =E h ^ X N (1) i !E h ^ X (1) i as N!1; (b) E[X N (1)N] + p N =E h ^ X N (1) + i !E h ^ X (1) + i as N!1; (c) N p N P ^ X N (1)l ? ! lim t!1 E h ^ U ? T (t) i t as N!1. Proof of (a) and (b): We rst show that the stationary distribution of the scaled original system con- verges to the stationary distribution of the limiting diusion process, i.e. that, ^ X N (1)) ^ X (1): For this, suppose we can show that the sequence of stationary distributions n ^ X N (1) o is tight. Then n ^ X N (1) o will have a convergent subsequence, say n ^ X N 0 (1) o . Let ^ X N 0 (0) be distributed as ^ X N 0 (1). Then n ^ X N 0 (t);t 0 o is a stationary process. Furthermore, by Theorem 6, ^ X N 0 ) ^ X where ^ X (0) is distributed as the limit of 88 ^ X N 0 (0). But we also know that ^ X is stationary as ^ X N 0 is stationary for allN 0 . Thus ^ X N 0 (1) must be converging to the unique stationary distribution of ^ X. Since every convergent subsequence of n ^ X N o converges to this limit, the sequence n ^ X N o itself must also converge to the same limit. Applying the continuous mapping theorem we get ^ X N (1) ) ^ X (1) and ^ X N (1) + ) ^ X (1) + We now establish the tightness of n ^ X N (1) o . This will be done by bounding the process above and below by two processes with the same arrival and service rates and the same number of servers but dierent threshold levels. Let ^ L N and ^ U N denote the QED-scaled number of customers in an M=M=N=N and M=M=N=1 system respectively. Then P ^ L N (1)x P ^ X N (1)x P ^ U N (1)x for all N and x2<. Also it follows from results in [16] and [33] that ^ L N (1) and ^ U N (1) weakly converge asN!1. Hence both processes are tight due to Theorem 6.2 of Billingsley. We conclude that n ^ X N (1) o is also tight. Next we establish that the processes ^ X N (1) and ^ X N (1) + are uniformly in- tegrable (UI). This will also be done by bounding the processes above and below with UI processes having the same arrival/service rates and number of servers but dierent threshold levels. The uniform integrability of ^ U N (1) and ^ U N (1) + follow from Lemma 1 of [33]. Observe that ^ U N (1) + is the scaled number of customers in queue for an M=M=N=1 system. Since ^ X N (1) + st ^ U N (1) + ; 89 where st denotes stochastically larger, it follows that ^ X N (1) + is UI. Using a similar argument as in Lemma 1 of [33] one can show that E h ^ L N (1) 2 i =E " L N (1)N 2 N # = N 2 1 N 2 +N N (1p N (N)) +N 2 N 1 N p N (N) N =N 1 N 2 + N (1p N (N)) +N N 1 N p N (N) where p N (N) is the blocking probability and N = N N is the utilization of the N th system. We know that p N 1 N ! m due to the QED heavy trac assumption. Moreover p Np N (N) also converges to a constant (see, for example, the appendix of [62]). Thus sup N E h ^ L N (1) i 2 < 1 implying that ^ L N (1) and thus ^ L N (1) is uniformly integrable. Since ^ L N (1) is the scaled number of idle servers in an M=M=N=N system ^ X N (1) st ^ L N (1) : Thus ^ X N (1) is also UI. We conclude that (a) and (b) hold, because we have shown that ^ X N (1) ) ^ X(1) and ^ X N (1) + ) ^ X(1) + as N!1, and that the sequences are UI. Proof of (c): Finally we establish (c). Let := lim t!1 E[ ^ U(t)] t . Then an analysis similar to [8] 90 yields that = (l ? + m ) p p p h (l ? + m ) p m p i +e m 2 2 2 m 2 2 m : We show that N p N P ^ X N l ? ! as N!1: From the ow balance equations for the N th system we get P X N (1) =n :=p n (N) = 8 > < > : p 0 (N) ( N ) n n! n nN p 0 (N) ( N ) n N! N Q nN j=1 (N+j ) n>N implying that P X N (1) =N + h l ? p N i =p 0 (N) ( N ) N+[l p N] N! N Q [l ? p N] j=1 (N +j ) : From the convergence of the steady state distributions P ^ X N (1) 0 !P ^ X(1) 0 as N!1: Since P ^ X N (1) 0 = N X n=0 p 0 (N) ( N ) n n! n ; 91 and from (18.5) and (18.6) in [20] P ^ X(1) 0 = m p p e m 2 2 m 2 2 2 h (l ? + m ) p m p i + m ; we conclude that N X n=0 p 0 (N) ( N ) n n! n ! m p p e m 2 2 m 2 2 2 h (l ? + m ) p m p i + m : (A-59) It is also true that P N n=0 e ( N ) ( N ) n n! n =P (Y N N) whereY N is a Poisson random variable with mean and variance N N = N and P (Y N N) =P Y N N N p N N NN N p N N ! : From the QED assumption NN N p N N ! m ; and so applying the central limit theorem for Poisson random variables shows that N X n=0 e ( N ) ( N ) n n! n ! m as N!1: Hence it follows from (A-59) that e ( N ) p 0 (N)! 1 p p e m 2 2 m 2 2 2 h (l ? + m ) p m p i + m as N!1: (A-60) 92 Next, recalling that N =N N and Stirling's formula N! p 2NN N e N where the symbol denotes asymptotic equality, N p N P ^ X N (1)N p Nl ? = N p N e ( N ) p 0 (N) e ( N ) ( N ) N+[l ? p N] N! N [l ? p n] Y j=1 (N +j ) N p N e ( N ) p 0 (N) ( N ) N+[l ? p N] e N N [l ? p N] Y j=1 1 + j N 1 p 2NN N e N N (N) [l ? p N] = N N e ( N ) N p 0 (N) ( N ) N+[ p Nl ? ] e NN N e log [l ? p N] Y j=1 1 + j N 1 p 2 = N N e ( N ) N p 0 (N) ( N ) N+[ p Nl ? ] e NN N e [l ? p N] X j=1 log 1 + j N p 2 : (A-61) Note also that log 1 + j N = j N O( j 2 2 N 2 2 ); which implies [l ? p N] X j=1 log 1 + j N = h l ? p N i ( h l ? p N i + 1) 2N +O( 1 p N ): 93 Hence ( N ) N+[ p Nl ? ] e NN N e [l ? p N] X j=1 log 1 + j N p 2 e N[1 N +log N ]+[l ? p N] log N [l ? p N]([l ? p N]+1) 2N +O( 1 p N ) p 2 : (A-62) Finally, note that p N(1 N )!m= and N =N! as N!1 and that log N = log 1 (1 N ) =(1 N ) (1 N ) 2 2 +o((1 N ) 2 ): It then follows from (A-60), (A-61), and (A-62) that N p N P ^ X N (1)N p Nl ? ! e m 2 2 2 ml ? (l ? ) 2 2 p 2 p p e m 2 2 m 2 2 2 h (l ? + m ) p m p i + m = e m 2 2 2 m 2 2 e m 2 2 2 ml ? (l ? ) 2 2 p 2 e m 2 2 2 m 2 2 p p e m 2 2 m 2 2 2 h (l ? + m ) p m p i + m = (l ? + m ) p p p h (l ? + m ) p m p i +e m 2 2 2 m 2 2 m =: Proof of (ii): Note rst that any admissable policy N is threshold. For a particular policy N , let l N denote the sequence of thresholds indexed by the number of servers. For any 94 sequence l N there are 3 possible cases: (1) l N p N !l as N!1 where l2< + : (2) l N p N !1 as N!1: (3) l N p N does not converge. Note that for cases (1) and (2), where l N p N !l for 0l1, essentially the same reasoning as in part (i) reveals that lim N!1 lim t!1 1 p N (t; N ) t =(l)(l ? ); where the last inequality is due to Theorem 5 that establishes the optimality of the threshold level l ? for the diusion control problem. For part (3), consider the nontrivial case where lim inf N!1 lim t!1 1 p N (t;) t is nite. Then there exists a subsequence N 0 where the liminf is attained. As before there are three possible cases along this subsequence: (1) l N 0 p N 0 !l as N 0 !1 where l2< + : (2) l N 0 p N 0 !1 as N 0 !1: 95 (3) l N 0 p N 0 does not converge. For cases (1) and (2), it is straightforward to see that lim inf N!1 lim T!1 1 p N (t;) t (l ? ). For case (3), one can take a further subsequence, sayN 00 , where l N 00 p N 00 converges. Since the scaled cost process converges along the sequence N 0 and every convergent subsequence of N 0 must have the same limit, the desired result follows. Proof of Proposition 2: The proof follows from Theorem 7.6 of [50] which established the convergence of the diusion scaled number of customers in system and Theorem 5.2 in [40] which established the convergence of the cost of the outsourcing sub-problem. Proof of Proposition 3: Equation (37) trivially follows from equation (36) provided that the limit and expec- tation in the left-hand-side of equation (37) can be interchanged. Hence we justify this interchange by showing that the random variable ;? (N;(X)) p is uniformly in- tegrable. We do so by showing that 0 (N;(X)) p , the random variable denoting the diusion scaled cost of the policy that does not outsource any calls, is UI. Then, the optimality of the policy ? will imply that ;? (N;(X)) p 0 (N;(X)) p and the uniform integrability of ?(N; (X)) will follow. First observe that 0 (N; (X)) only includes the outsourcing cost so that the diusion scaled cost can be expressed as 96 0 (N; (X)) p = a E[Y (1) + ] p = a E[ b Y (1) + ]: Also for a given realization of the arrival rate as+x 1=2 and the steady state balance equation for abandonment rate we have ( +x 1=2 )P ab (x) p = E[ b Y (1) + ]: where P ab (x) is the probability of abandonment as a function of the realized arrival rate. Hence it suces to establish the uniform integrability of +x 1=2 p P ab (x). We also know that, for a given realization of the arrival rate as +x 1=2 , P ab (x) (x) p where the function is dened as in Theorem 4 in Garnett et al. (with a service grade of x instead of ) so that E X ( +x 1=2 )P ab (x) p = E X h p P ab (X) +XP ab (X) i E X [(X)] +E X X (X) p = E X [(X)] + 1 p E X [X(X)] 97 which yields lim !1 E X ( +x 1=2 )P ab (x) p = E X [(X)] = E X lim !1 ( +x 1=2 )P ab (x) p and shows that the positive random variable (+x 1=2 )P ab p is UI(cf. second part of Theorem 5.4 on page 32 of [15]). Proof of Proposition 4: Let us rst consider the case c 1 > c 2 and e C 0 ; the cost of the policy that sets the in-house stang levelN = 0 and outsources the complete call volume. The long run average expected cost of the operating subproblem will then be ;? (N ; (X)) = c 2 E X [(X)] =c 2 . Then, it is easy to see that this policy is asymptotically optimal according to Denition 2. This can easily be seen from as centered and scaled cost as given in equation (27) e C 0 = 1 1=2 (c 1 +c 2 )!1 and thus satisfying equation (28). From here on we assume thatc 1 c 2 . We also assume that under the admissable policy u, the stang level satises N = + d where 2< and d2< such that d6= 1=2 while the realized arrival rate is still of the form = +x 1=2 . Then, there are two possible cases. Case 1: Assume d < 1=2. Then the service grade for a realization X = x, 98 the realized service grade will be m(;x) = lim !1 N p =x and will yield the same centered and scaled cost as in stang according to N = + 0 1=2 = , that is following QED stang with a service grade = 0. Thus the centered and scaled cost of both policies will have the same asymptotic cost as !1, namely E X [b ?(x;l ? (x)) ] and the result follows from the optimality of ? through (29). Case 2: Assume d> 1=2. Obviously the case with > 0 yields over-stang and thus a centered and scaled cost ~ C u where lim !1 ~ C u !1. Thus we focus on the case < 0 and assume that c 2 >c 1 . First observe that, through ow balance equations, a lower bound on the outsourcing and abandonment rates is given by outsourcing rate + abandonment rate +x 1=2 d +o( d ) = d +o( d ) which in turn yields the following lower bound on the operating cost ;? (N ; (X)) for a general outsourcing policy ;? (N ; (X))c 2 d +o( d ): (A-63) Thus we see that the centered and scaled cost for this stang policy as dened in equation (27) 99 ~ C u = 1 1=2 n c 1 N c 1 +E X h ;? (N; (X)) io = c 1 d +E X h ;? (N ; (X)) i 1=2 = d (c 2 c 1 ) +o( d ) 1=2 ! 1 establishing the desired result. Proof of Theorem 8: This follows from the fact that any asymptotically optimal policy has stang of the form N = + 1=2 for some 2< and the denitions of ? and l ? . Proof of Theorem 9: We will prove the assertion of Theorem 9 by establishing that c 2 (x) is asymp- totically both a lower and an upper bound for l (;x) c which will imply the required convergence. We start with the more trivial lower bound. First observe that an upper bound on the departure rate (service completions+abandonment) is given by + c + l p . Then by ow balance equations outsourcing rate +x c c l p = (x) c +o( c ) 100 Dividing by c and letting !1 yields l (;x) c c 2 (x) as !1. Next, note that an upper bound for the outsourcing rate is achieved whenl = 0 in which case we have a pure loss system. It follows from [38] that for a stang level ~ N = + ~ 1=2 that server occupancy 1 ~ + 1=2 . Note, however, that the realized arrival rate in our case is given by = +x c and for a stang policy of the form N = + c this implies that N = + c << ~ N = + ~ 1=2 (our shortage of servers is O( c )). Hence we expect the server occupancy in the original system to be even larger which yields that our outsourcing rate is bounded as follows outsourcing rate +x c ( + 1=2 )(1 ~ + 1=2 ) = (x) c +o( c ) Dividing by c and letting!1 yields l (;x) c c 2 (x) as!1 and establishes the result. Proof of Theorem 11: The proof is based on Stone's Theorem (cf. [57] and [33]). Dene the diusion scaled process b Y := Y N x c p and let m (^ y) and 2 (^ y) be the innitesimal mean and 101 variance of the scaled process. Then by denition m (^ y) = lim h!0 E[ b Y (t +h) b Y (t)j b Y (t) = ^ y] h = lim h!0 E[Y (t +h)Y (t)jY (t) = ^ y p + + c + x c ] h = lim h!0 +x c ( + c ) ( x c + ^ y p ) p ! ^ y =m(^ y) Similarly by denition of the innitesimal variance we have 2 (^ y) = lim h!0 E[( b Y (t +h) b Y )(t)) 2 j b Y (t) = ^ y] h = lim h!0 E[(Y (t +h)Y (t)) 2 jY (t) = ^ y p + + c + x c ] h = lim h!0 +x c + ( + c ) + ( x c + ^ y p ) ! 2 = 2 (^ y) and the result follows. Proof of Theorem 12: The proof again follows from Stone's Theorem along the same lines as Theorem 11. 102 Appendix B: Tables Table 3: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. [a = 1:25;c =h I = = 1; = 50] N n ? z ? N + h p Nl ? i z [N+ p Nl ? ] (z ? z [N+ p Nl ? ] )=z ? 10 0.1 12 40.0631 10 40.4716 1.02% 20 0.1 24 30.1150 22 30.2282 0.38% 30 0.1 38 20.2046 36 20.2282 0.12% 40 0.1 57 10.4324 55 10.4414 0.09% 50 0.1 122 3.0483 147 3.0483 0.00% 60 0.1 122 10.1156 117 10.1156 0.00% 70 0.1 122 20.0026 70 20.1368 0.67% 80 0.1 122 30.0000 80 30.0022 0.01% 90 0.1 122 40.0000 90 40.0000 0.00% 100 0.1 122 50.0000 100 50.0000 0.00% 10 1.0 11 40.2976 10 40.4716 0.43% 20 1.0 22 30.5598 21 30.6404 0.26% 30 1.0 34 21.0042 33 21.0066 0.01% 40 1.0 48 12.0108 47 12.0124 0.01% 50 1.0 75 6.3363 75 6.3363 0.00% 60 1.0 118 10.6382 90 10.6382 0.00% 70 1.0 118 20.0198 95 20.0198 0.00% 80 1.0 118 30.0002 99 30.0002 0.00% 90 1.0 118 40.0000 102 40.0000 0.00% 100 1.0 119 50.0000 107 50.0000 0.00% 10 10 10 40.4716 10 40.4716 0.00% 20 10 20 31.2095 20 31.2095 0.00% 30 10 30 22.4835 30 22.4892 0.00% 40 10 41 14.4837 41 14.5574 0.00% 50 10 53 9.5074 53 9.5074 0.00% 60 10 68 11.5887 68 11.5887 0.00% 70 10 86 20.0752 86 20.0752 0.00% 80 10 99 30.0009 89 30.0009 0.00% 90 10 107 40.0000 97 40.0000 0.00% 100 10 112 50.0000 104 50.0000 0.00% 103 Table 4: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. [a = 1:50;c =h I = = 1; = 50] N n ? z ? N + h p Nl ? i z [N+ p Nl ? ] (z ? z [N+ p Nl ? ] )=z ? 10 0.1 12 40.1070 10 40.4716 0.91% 20 0.1 23 30.1961 22 30.2642 0.23% 30 0.1 36 20.3419 35 20.3689 0.13% 40 0.1 54 10.7050 52 10.7187 0.13% 50 0.1 117 3.3862 117 3.3862 0.00% 60 0.1 117 10.1285 115 10.1285 0.00% 70 0.1 117 20.0029 70 20.1368 0.67% 80 0.1 117 30.0000 80 30.0022 0.01% 90 0.1 117 40.0000 90 40.0000 0.00% 100 0.1 117 50.0000 100 50.0000 0.00% 10 1.0 10 60.0000 10 40.4716 0.00% 20 1.0 21 30.7887 21 30.7887 0.00% 30 1.0 32 21.4140 32 21.4140 0.00% 40 1.0 45 12.7144 44 12.7588 0.35% 50 1.0 63 6.9998 64 6.9998 0.00% 60 1.0 101 10.7091 89 10.7091 0.00% 70 1.0 118 20.0219 94 20.0219 0.00% 80 1.0 118 30.0002 99 30.0002 0.00% 90 1.0 118 40.0000 102 40.0000 0.00% 100 1.0 118 50.0000 106 50.0000 0.00% 10 10.0 10 40.4716 10 40.4716 0.00% 20 10.0 20 31.2095 20 31.2095 0.00% 30 10.0 30 22.4835 30 22.4835 0.00% 40 10.0 40 14.9792 40 14.9834 0.00% 50 10.0 52 10.0366 52 10.0366 0.00% 60 10.0 64 11.7529 64 11.7529 0.00% 70 10.0 78 20.0835 77 20.0835 0.00% 80 10.0 92 30.0010 88 30.0010 0.00% 90 10.0 105 40.0000 96 40.0000 0.00% 100 10.0 111 50.0000 103 50.0000 0.00% 104 Table 5: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. [a = 5;c =h I = = 1; = 50] N n ? z ? N + h p Nl ? i z [N+ p Nl ? ] (z ? z [N+ p Nl ? ] )=z ? 10 0.1 11 40.4144 10 40.4716 0.14% 20 0.1 21 30.7209 21 30.7209 0.00% 30 0.1 33 21.2524 32 21.2875 0.17% 40 0.1 45 12.3972 45 12.3978 0.00% 50 0.1 65 6.1094 65 6.1094 0.00% 60 0.1 110 10.3083 108 10.3083 0.00% 70 0.1 110 20.0069 70 20.1368 0.65% 80 0.1 110 30.0001 80 30.0022 0.01% 90 0.1 110 40.0000 90 40.0000 0.00% 100 0.1 110 50.0000 100 50.0000 0.00% 10 1.0 10 40.4716 10 40.4716 0.00% 20 1.0 20 31.2095 20 31.2095 0.00% 30 1.0 30 22.4835 30 23.1001 0.00% 40 1.0 41 14.7506 41 14.7506 0.00% 50 1.0 52 9.7210 52 9.7210 0.00% 60 1.0 65 11.4958 65 11.4958 0.00% 70 1.0 80 20.0523 80 20.0523 0.00% 80 1.0 95 30.0005 94 30.0005 0.00% 90 1.0 109 40.0000 101 40.0000 0.00% 100 1.0 118 50.0000 105 50.0000 0.00% 10 10 10 40.4716 10 40.4716 0.00% 20 10 20 31.2095 20 31.2095 0.00% 30 10 30 22.4835 30 22.4835 0.00% 40 10 40 14.9792 40 14.9792 0.00% 50 10 50 10.4787 50 10.4787 0.00% 60 10 60 12.1668 60 12.4387 0.00% 70 10 71 20.1367 71 20.1367 0.00% 80 10 81 30.0020 81 30.0015 0.00% 90 10 91 40.0000 91 40.0000 0.00% 100 10 102 50.0000 102 50.0000 0.00% 105 Table 6: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. [a = 1:25;c =h I = = 1;m = 0] N n ? z ? N + h p Nl ? i z [N+ p Nl ? ] (z ? z [N+ p Nl ? ] )=z ? 10 0.1 64 1.3606 53 1.36056 0.00% 20 0.1 97 1.9265 81 1.9265 0.00% 30 0.1 124 2.3604 105 2.3604 0.00% 40 0.1 149 2.7261 126 2.7261 0.00% 50 0.1 171 3.0483 147 3.0483 0.00% 60 0.1 193 3.3395 166 3.3395 0.00% 70 0.1 214 3.6073 185 3.6073 0.00% 80 0.1 234 3.8565 202 3.8565 0.00% 90 0.1 253 4.0906 220 4.0906 0.00% 100 0.1 272 4.3120 237 4.3120 0.00% 10 1.0 21 2.8147 21 2.8147 0.00% 20 1.0 35 3.9974 36 3.9974 0.00% 30 1.0 49 4.9026 49 4.9026 0.00% 40 1.0 62 5.6650 62 5.6650 0.00% 50 1.0 75 6.3363 75 6.3363 0.00% 60 1.0 87 6.9431 87 6.9431 0.00% 70 1.0 100 7.5009 100 7.5009 0.00% 80 1.0 112 8.0200 112 8.0200 0.00% 90 1.0 124 8.5075 124 8.5075 0.00% 100 1.0 135 8.9685 135 8.9685 0.00% 10 10 11 4.1180 11 4.1180 0.00% 20 10 22 5.9413 22 5.9413 0.00% 30 10 32 7.3290 32 7.3290 0.00% 40 10 43 8.4855 43 8.4855 0.00% 50 10 53 9.5074 53 9.5074 0.00% 60 10 64 10.4301 64 10.4301 0.00% 70 10 74 11.2738 74 11.2738 0.00% 80 10 84 12.0631 84 12.0631 0.00% 90 10 95 12.8035 95 12.8035 0.00% 100 10 105 13.4997 105 13.4997 0.00% 106 Table 7: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. [a = 1:5;c =h I = = 1;m = 0] N n ? z ? N + h p Nl ? i z [N+ p Nl ? ] (z ? z [N+ p Nl ? ] )=z ? 10 0.1 40 1.5113 40 1.5113 0.00% 20 0.1 62 2.1400 62 2.1400 0.00% 30 0.1 82 2.6220 82 2.6220 0.00% 40 0.1 100 3.0283 100 3.0283 0.00% 50 0.1 117 3.3862 117 3.3862 0.00% 60 0.1 134 3.7097 134 3.7097 0.00% 70 0.1 150 4.0072 150 4.0072 0.00% 80 0.1 165 4.2841 165 4.2841 0.00% 90 0.1 180 4.5442 180 4.5442 0.00% 100 0.1 195 4.7901 195 4.7901 0.00% 10 1.0 16 3.1032 16 3.1032 0.00% 20 1.0 28 4.4118 28 4.4118 0.00% 30 1.0 40 5.4133 40 5.4133 0.00% 40 1.0 52 6.2566 52 6.2566 0.00% 50 1.0 63 6.9998 64 6.9998 0.00% 60 1.0 75 7.6706 75 7.6706 0.00% 70 1.0 86 8.2876 86 8.2876 0.00% 80 1.0 97 8.8620 97 8.8620 0.00% 90 1.0 108 9.4014 108 9.4014 0.00% 100 1.0 119 9.9114 119 9.9114 0.00% 10 10 10 4.2916 10 4.2916 0.00% 20 10 21 6.2258 21 6.2258 0.00% 30 10 31 7.6812 31 7.6812 0.00% 40 10 41 8.9275 41 8.9275 0.00% 50 10 52 10.0367 52 10.0367 0.00% 60 10 62 10.9957 62 10.9957 0.00% 70 10 72 11.8850 72 11.8850 0.00% 80 10 82 12.7192 82 12.7192 0.00% 90 10 92 13.5080 92 13.5080 0.00% 100 10 102 14.2587 102 14.2587 0.00% 107 Table 8: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. [a = 5;c =h I = = 1;m = 0] N n ? z ? N + h p Nl ? i z [N+ p Nl ? ] (z ? z [N+ p Nl ? ] )=z ? 10 0.1 16 2.7037 16 2.7037 0.00% 20 0.1 29 3.8456 29 3.8533 0.00% 30 0.1 41 4.7220 41 4.7220 0.00% 40 0.1 53 5.4593 53 5.4593 0.00% 50 0.1 65 6.1094 65 6.1094 0.00% 60 0.1 76 6.6969 76 6.6969 0.00% 70 0.1 88 7.2376 88 7.2376 0.00% 80 0.1 99 7.7395 99 7.7395 0.00% 90 0.1 110 8.2115 110 8.2115 0.00% 100 0.1 121 8.6582 121 8.6582 0.00% 10 1.0 11 4.2440 11 4.2440 0.00% 20 1.0 21 6.0461 21 6.0461 0.00% 30 1.0 31 7.4991 31 7.4991 0.00% 40 1.0 42 8.6872 42 8.6872 0.00% 50 1.0 52 9.7210 52 9.7210 0.00% 60 1.0 62 10.6749 62 10.6749 0.00% 70 1.0 72 11.5653 72 11.5653 0.00% 80 1.0 83 12.3784 83 12.3784 0.00% 90 1.0 93 13.1261 93 13.1261 0.00% 100 1.0 103 13.8418 103 13.8418 0.00% 10 10 10 4.29165 10 4.2916 0.00% 20 10 20 6.3557 20 6.3557 0.00% 30 10 30 7.9476 30 7.9476 0.00% 40 10 40 9.2925 40 9.2925 0.00% 50 10 50 10.4788 50 10.4788 0.00% 60 10 60 11.5520 60 11.5520 0.00% 70 10 70 12.5395 70 12.5395 0.00% 80 10 80 13.4590 80 13.4590 0.00% 90 10 90 14.3228 90 14.3228 0.00% 100 10 100 15.1401 100 15.1401 0.00% 108 Table 9: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. [a = 2;h I = 0;c = = 1; = 50] N n ? z ? N + h p Nl ? i z [N+ p Nl ? ] (z ? z [N+ p Nl ? ] )=z ? 10 0.1 11 40.1273 10 40.2358 0.27% 20 0.1 22 30.2401 21 30.3001 0.20% 30 0.1 34 20.4255 33 20.4424 0.08% 40 0.1 48 10.8361 47 10.8450 0.08% 50 0.1 74 2.4733 74 2.4733 0.00% 60 0.1 161 0.1028 117 0.1028 0.00% 70 0.1 160 0.0023 89 0.0024 3.04% 80 0.1 83 1.94E-05 80 0.0011 5636% 90 0.1 137 5.83E-08 90 5.24E-06 8894% 100 0.1 129 6.47E-11 100 8.15E-09 12500% 10 1 10 40.2358 10 40.2358 0.00% 20 1 20 30.3001 20 30.3001 0.00% 30 1 31 21.1434 30 21.2418 0.46% 40 1 42 12.1110 41 12.1418 0.25% 50 1 54 4.4637 54 4.4637 0.00% 60 1 70 0.5602 70 0.5602 0.00% 70 1 90 0.0176 89 0.0176 0.00% 80 1 110 0.0002 96 0.0002 0.02% 90 1 124 5.31E-07 128 5.31E-07 0.00% 100 1 125 6.06E-10 132 6.06E-10 0.00% 10 10 10 40.2358 10 40.2358 0.00% 20 10 20 30.6047 20 30.6047 0.00% 30 10 30 21.2418 30 21.2418 0.00% 40 10 40 12.4896 40 12.4896 0.00% 50 10 50 5.2394 50 5.2394 0.00% 60 10 61 1.0669 61 1.0669 0.00% 70 10 72 0.0598 72 0.0598 0.00% 80 10 83 0.0008 83 0.0008 0.00% 90 10 93 3.09E-06 94 3.09E-06 0.00% 100 10 104 3.98E-09 105 3.98E-09 0.00% 109 Table 10: A comparison of the MDP and DCP solutions for increasing system size, and various customer patience levels. [a = 2;h I = 0;c = = 1;m = 0] N n ? z ? N + h p Nl ? i z [N+ p Nl ? ] (z ? z [N+ p Nl ? ] )=z ? 10 0.1 20 1.099467 21 1.099488 0.00% 20 0.1 35 1.5599 35 1.5599 0.00% 30 0.1 49 1.9133 49 1.9133 0.00% 40 0.1 62 2.2111 62 2.2111 0.00% 50 0.1 74 2.4733 74 2.4733 0.00% 60 0.1 87 2.7104 87 2.7104 0.00% 70 0.1 99 2.9283 99 2.9283 0.00% 80 0.1 111 3.1312 111 3.1312 0.00% 90 0.1 123 3.3218 123 3.3218 0.00% 100 0.1 135 3.5021 135 3.5021 0.00% 10 1 11 1.9588 12 1.9637 0.25% 20 1 22 2.7964 22 2.7964 0.00% 30 1 33 3.4399 33 3.4399 0.00% 40 1 43 3.9924 44 3.9929 0.01% 50 1 54 4.4637 54 4.4637 0.00% 60 1 64 4.9011 64 4.9011 0.00% 70 1 75 5.2960 75 5.2960 0.00% 80 1 85 5.6659 85 5.6659 0.00% 90 1 96 6.0183 96 6.0183 0.00% 100 1 106 6.3425 106 6.3425 0.00% 10 10 10 2.1458 10 2.1458 0.00% 20 10 20 3.1778 20 3.1778 0.00% 30 10 30 3.9738 30 3.9738 0.00% 40 10 40 4.6462 40 4.6462 0.00% 50 10 50 5.2394 50 5.2394 0.00% 60 10 60 5.7760 60 5.7760 0.00% 70 10 70 6.2698 70 6.2698 0.00% 80 10 80 6.7295 80 6.7295 0.00% 90 10 90 7.1614 90 7.1614 0.00% 100 10 100 7.5700 100 7.5700 0.00%

Abstract (if available)

Abstract In this dissertation we solve two queueing control problems that are related to capacity sizing and dynamic outsourcing in call centers. First we take the capacity (staffing) decision as given and focus on the dynamic control (outsourcing) problem by solving an admission control problem for a multi-server queue with abandonment. We establish that a threshold policy is optimal through Markov Decision Process (MDP) analysis and present an algorithm to derive the optimal threshold that minimizes the long- run average cost. We also solve the approximating diffusion control problem (DCP) that arises in the Halfin-Whitt many-server limit regime. We show that the optimal policy is again of threshold form and that the parameter space has a sharper division in the sense that there is an optimal solution with a finite threshold level when the cost of an abandonment exceeds the cost of rejecting a customer

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Real-time controls in revenue management and service operations

PDF

Essays on dynamic control, queueing and pricing

PDF

Essays on service systems

PDF

Stochastic models: simulation and heavy traffic analysis

PDF

Essays on service systems with matching

PDF

Provable reinforcement learning for constrained and multi-agent control systems

PDF

Decision-aware learning in the small-data, large-scale regime

PDF

Optimizing healthcare decision-making: Markov decision processes for liver transplants, frequent interventions, and infectious disease control

PDF

Essays on information, incentives and operational strategies

PDF

I. Asynchronous optimization over weakly coupled renewal systems

PDF

Train scheduling and routing under dynamic headway control

PDF

Commercialization of logistics infrastructure as an offline platform

PDF

Analysis, design, and optimization of large-scale networks of dynamical systems

PDF

Queueing loss system with heterogeneous servers and discriminating arrivals

PDF

Learning and control for wireless networks via graph signal processing

PDF

Elements of dynamic programming: theory and application

PDF

Essays on revenue management with choice modeling

PDF

How do acquirers govern the deal-making process? Three essays on U.S. mergers and acquisitions 1994 – 2017

PDF

Essays on bounded rationality and revenue management

PDF

Large-scale multiple hypothesis testing and simultaneous inference: compound decision theory and data driven procedures

Asset Metadata

Creator Koçağa, Yaşar Levent (author)

Core Title Essays on capacity sizing and dynamic control of large scale service systems

School Marshall School of Business

Degree Doctor of Philosophy

Degree Program Business Administration

Publication Date 08/10/2010

Defense Date 06/03/2010

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag call center management,continuous time Markov chains,Markov decision processes,OAI-PMH Harvest,outsourcing,queueing theory,stochastic dynamic programming

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Ward, Amy R. (committee chair), Rajagopalan, Sampath (committee member), Ross, Sheldon M. (committee member)

Creator Email kocaga@usc.edu,Y.Levent.Kocaga.2009@marshall.usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-m3360

Unique identifier UC1199476

Identifier etd-KOCAGA-3842 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-382682 (legacy record id),usctheses-m3360 (legacy record id)

Legacy Identifier etd-KOCAGA-3842.pdf

Dmrecord 382682

Document Type Dissertation

Rights Koçağa, Yaşar Levent

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Repository Name Libraries, University of Southern California

Repository Location Los Angeles, California

Repository Email cisadmin@lib.usc.edu