Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Set values for mean field games and set valued PDEs
(USC Thesis Other)
Set values for mean field games and set valued PDEs
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
Set Values For Mean Field Games and Set Valued PDEs by Melih İşeri A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulllment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (MATHEMATICS) May 2023 Copyright 2023 Melih İşeri Dedication To my father. For his sincereness and endless support, which I lost forever. To my mother. For her love, patience and constant eorts. To my friends Sarp & Doruk. For their invaluable exchanges in life and genuineness. ii Acknowledgements I am beyond grateful to my advisor Prof. Jianfeng Zhang. His knowledge, guidance and attention was priceless. I was quite fortunate to have this expectional opportunity and the trust that he had. It is always a great pleasure to work hard and explore with him. I was also fortunate to have my undergraduate advisor Prof. Muhittin Mungan, as he gave me all the courage in my path. I experienced the joy of research and collabration, and altough not directly to this thesis, I cannot undervalue his overall contributions. I would like to thank a lot to Prof. Sergey Lototsky for his great support and diverse opportunuties that he provided me to improve myself during the PhD. I would also like to thank Prof. Jin Ma for his support throughtout my PhD studies. I always appreciated all the professors that I learned a lot from, and would like to mention their names: Prof. Tuğrul Burak Gürel, Prof. Cem Yalçın Yıldırım, Prof. Veysi Erkcan Özcan, Prof. Haluk Beker, Prof. Atilla Yılmaz, Prof. Ferit Öztürk, Prof. Kenneth Alexander, Prof. Marc Hoyois, Prof. Peter Baxendale. As always, many thanks and love to all the people in my life. iii TableofContents Dedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi Chapter 1: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Chapter 2: Multivariate Control Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.1 Notations and Setting for the Control Problem . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2 Dynamic Programming Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Chapter 3: Set Values for Mean Field Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.2 Mean eld games on nite space with closed loop controls . . . . . . . . . . . . . . . . . . 19 3.2.1 The basic setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.2.2 The raw set valueV 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.2.3 The set valueV state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.3 TheN-player game with homogeneous equilibria . . . . . . . . . . . . . . . . . . . . . . . 27 3.3.1 TheN-player game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.3.2 Convergence of the empirical measures . . . . . . . . . . . . . . . . . . . . . . . . 30 3.3.3 Convergence of the set values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 3.4 Mean eld games on nite space with relaxed controls . . . . . . . . . . . . . . . . . . . . 36 3.4.1 The relaxed set value with path dependent controls . . . . . . . . . . . . . . . . . . 37 3.4.2 Global formulation of mean eld games . . . . . . . . . . . . . . . . . . . . . . . . 43 3.5 TheN-player game with heterogeneous equilibria . . . . . . . . . . . . . . . . . . . . . . . 48 3.5.1 TheN-player game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.5.2 FromN-player games to mean eld games . . . . . . . . . . . . . . . . . . . . . . . 52 3.5.3 From mean eld games toN-player games . . . . . . . . . . . . . . . . . . . . . . . 56 3.6 A diusion model with state dependent drift controls . . . . . . . . . . . . . . . . . . . . . 60 3.6.1 The mean eld game and the dynamic programming principle . . . . . . . . . . . . 60 3.6.2 Convergence of theN-player game . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 3.7 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 3.7.1 The subtle path dependence issue in Remark 3.4.3 . . . . . . . . . . . . . . . . . . . 76 3.7.2 Some technical proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Chapter 4: Set Valued Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 iv 4.1 Set Valued Functions and Intrinsic Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . 89 4.1.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 4.1.2 Relations to the local representation and the signed distance . . . . . . . . . . . . . 97 4.2 The Itô’s formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 4.3 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 4.3.1 Local Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 4.3.2 Postponed proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 Chapter 5: Set Valued HJB Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 5.1 Wellposedness of the HJB Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 5.2 Moving Scalarization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 5.3 Quadratic Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 Chapter 6: Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 v Abstract Set valued functions have been introduced for many applications, such as time inconsistent stochastic optimization problems, multivariate dynamic risk measures, stochastic target problems, and nonzero sum games with multiple equilibria. In this thesis, we study a set valued approach for both multivariate control theory and mean eld games with possibly multiple equilibria. For mean eld games, we introduce the set of values over all equilibria, which we call the set value of the mean eld game, and established two crucial properties: (i) the dynamic programming principle, also called time consistency; and (ii) the convergence of the set values of the correspondingN-player games. To our best knowledge, this is the rst work in the literature which studies the dynamic value of mean eld games without requiring the uniqueness of mean eld equilibria. We emphasize that the set value is very sensitive to the choice of the admissible controls and we illustrate this by investigating three cases, two in nite state space models and the other in a diusion model. Time consistency is the underlying property for the PDE approach, which is available in many set valued settings, and we have also shown for the multivariate control theory. That is, we showed that the set of all attainable values, potentially in a higher dimension, is time consistent. Moreover, we extend the calculus to set valued functions, and obtained the crucial Itô’s formula. This opens the door for the PDE approach in the set valued setting. Indeed, we established the well-posedness of Hamilton-Jacobi-Bellman equation for the multivariate control theory. vi Chapter1 Introduction Set valued functions are elementary objects in mathematics with vast potential of applications. Vari- ous applications include geometric evolution of surfaces Giga [38], stochastic viability problems Aubin- Frankowska [2], multivariate super-hedging problems Bouchard-Touzi [9], multivariate dynamic risk mea- sures Feinstein-Rudlo [30], stochastic target problems Soner-Touzi [61, 62], nonzero sum games with multiple equilibria Feinstein-Rudlo-Zhang [29]. This thesis studies set valued approaches for multivariate control theory and mean-eld game theory. Furthermore, we developed the calculus for set valued functions, which is crucial to obtain the PDE ap- proach. In many formulations, corresponding set value satises so called dynamic programming principle (DPP), or say time consistency, without even structural or regularity assumptions. As the appropriate notion of Itô’s formula exists for standard control theory and mean eld games with unique equilibrium, together with DPP, corresponding PDEs are obtained to characterize the evolution of values. We show that set valued functions also enjoy the Itô’s formula and Hamilton-Jacobi-Bellman (HJB) equation can be formulated for multivariate control theory. Control theory, in general, studies the optimal behaviour of a single agent in a known environment. Agent faces the problem of deciding actions which aects its own dynamics and objective (value or cost). In the standard theory objective is scalar and the agent is trying to maximize (or minimize) the objective. 1 However, in many situations agent actually faces a high dimensional objective, for example well known mean variance problem, portfolio optimization and various other time inconsistent problems in [30] and Karnam-Ma-Zhang [42]. In chapter 2, we introduce the multivariate control problems. The basic idea of the mean-eld game (MFG) theory is to characterize limits of games where large number of players are subjected to a similar environment and utilities, and the main interaction is through their empirical measures of states. Although these states are indeed random, in the spirit of the law of large numbers, limiting distribution evolves in the common environment, interacting with its own (determinis- tic) law. Moreover, as there is no single value to optimize but rather many players to consider, concept of equilibrium plays a vital role for games, for which the signicance of mean-eld concept becomes evident to understand large population behaviors. For general theory of MFGs, see Caines-Huang-Malhame [10], Lasry-Lions [48], Lions [51], Cardaliaguet [11], Bensoussan-Frehse-Yam [8], and Camona-Delarue [16, 17]. Without any structural assumptions, typically there are multiple Mean Field Equilibria (MFE) with possibly dierent values (see, e.g., Bardi-Fischer [3] for some examples). In chapter 3, we study the set of values over all equilibria, which we call the set value of the MFG. When the MFE is unique, typically under certain monotonicity conditions, set value is reduced to the singleton of the standard value function of the game, which solves the so called master equation. We focus on two crucial properties of the set value: • the Dynamic Programming Principle (DPP), or the time consistency; • the convergence of the set values of the correspondingN-player games, which can be viewed as a type of stability result in terms of model perturbation. With the presence of multiple equilibria, to our best knowledge this is the rst work in the literature to study the MFG dynamically and to address the time consistency issue. We emphasize that the set value is very sensitive to the types of admissible controls. To ensure the convergence, one simple but crucial observation is that theN-player game and the MFG should use the 2 "same" type of controls (more precisely, corresponding types of controls in appropriate sense). We illustrate this point by considering two cases. Note that in the standard literature each player is required to use the same closed loop control along an MFE. For the rst case, we will obtain the desired convergence by restricting theN-player game to homogeneous equilibria, namely each player also uses the same closed loop control. In the second case, we remove such restriction and consider heterogeneous equilibria for the N-player games. In this heterogeneous case players with the same state may choose dierent controls, then one can not expect in the limit they will have to use the same control ∗ . Indeed, in this case the limit is characterized by the MFG with closed loop relaxed controls, or say closed loop mixed strategies, which means players with the same state may still have a distribution of controls to choose from. However, since our relax control for MFG is still homogeneous, namely each player uses the same relax control, the controls for N-player game and for MFG appear to be in dierent forms. To clarify this, we demonstrate that the relaxed controls is not the right framework but rather the corresponding set value coincides with so called the global formulation of MFG. This new framework embeds the structure of heterogeneous controls of N-player games into MFG in an unambiguous way. Therefore, beyond demonstrating the convergence, global formulation of MFG also provides an intuitive perspective for relaxed controls. For the homogeneous case, we will investigate both a discrete time model with nite state space and a continuous time diusion model with drift controls. But for the heterogeneous case we will investigate the discrete model only. The diusion model in such case involves some technical challenges for the convergence and we shall leave it for future research. We shall remind that, however, the DPP would hold in much more general models without signicant diculties. In chapter 4, we developed the set valued calculus. Namely, we have introduced the notion of deriva- tive, the fundamental theorem and most importantly the Itô’s formula for functions of the formV(x) = ∗ When the MFE is unique, under appropriate monotonicity conditions, the set value becomes a singleton and it is not sensitive to the type of admissible controls anymore. Consequently, the convergence becomes possible even if theN-player games and the MFG use dierent types of controls, see e.g. [13] 3 [ m k=0 V k (x), where eachV k is ak-dimensional dierentiable manifold inR m . We note thatV =V 0 a sin- gleton is the standard case for functions, and hence it is appropriate to view it as an extension of calculus. Our work is intrinsically connected to the literature of surface evolution equations. This large body of literature, in our notations, analyses the equation @ t V(t;y) =h(t;y;n;@ y n) (1.1) whereh is a given function andn is the normal vector of the surface. Namely, a smooth manifold evolves in time at eachy2V depending on time and the local characterization of the surface aty. Applications include ame propagation, crystal formations, image processing, and notable, well studied case is mean- curvature ows. We remark (1.1) is a rst order ODE, and the HJB equation is a second order PDE. In that sense, our work can be seen as an extension to this literature, which we refer to Sethian [59], Chen-Giga- Goto [21], Evans-Spruck [27], Soner [60], Barles-Soner-Souganidis [4], [38] and references therein. In chapter 5, HJB equation is discussed for multivariate control theory and the well-posedness is shown. We remark that in the scalar case our set value is an interval where two boundary points corresponds to the inmum & supremum (or cost & value). In this case, HJB equations reduces to the standard HJB equations for each boundary point. In higher dimensions, notion of optimality stays to be the boundary of reachable values. As an application, we provide a moving scalarization proposed by [30]. That is, denoting the value Y of the control, the dynamic problem esssupY t is typically time inconsistent for xed2R m . We construct a process such that initially 0 = and the dynamic problem esssup t Y t becomes time consistent. This provides the characterization of optimality in the future of a control that is optimal initially. It is important, as it allows to more thoroughly decide whether or not to implement the initially optimal control. 4 Chapter2 MultivariateControlProblems This chapter introduces the multivariate control problem. That is, objective to optimize is not scalar, but a set valued function. The rst section introduces the notations and the setting of the multivariate control problem. Second section presents the DPP, or time consistency, which is crucial to obtain (together with Itô’s formula) the PDE approach. 2.1 NotationsandSettingfortheControlProblem For any 0 t T 0 T , we consider the canonical space t;T 0 :=fw2 C([t;T 0 ];R d 0 ) : w t = 0g, together with the canonical processB t s , which is a Brownian motion under the Wiener measureP t;T 0 . We denoteB t;s (w) :=fB t r (w) =w r :trsg. LetF t;T 0 =fF t;s :tsT 0 g be the ltration generated byB t s . ∗ Introduce, for 1a;b; L a t;T 0 :=fF t;T 0 measurable :Ejj a <1g L a;b t;T 0 :=fF t;T 0 progressively measurable :E Z T 0 t j s j a ds b=a <1g L a;loc t;T 0 :=fF t;T 0 progressively measurable : Z T 0 t j s j a ds<1 a.s.g ∗ We may drop the subscripts whent = 0;T0 =T . 5 When needed we explicitly indicate the range of functions such asL a t;T 0 (R m ), and the norms are always induced by the trace. Forw2 t;T 0 andv2R d 0 , C([t;T 0 ];R d 0 )3 (w +v) s :=w s +v; tsT 0 We then introduce, forw2 t;T 0 and ~ w2 T 0 ;T , t;T 3 (w T 0 ~ w) s :=1 fts<T 0 g w s +1 fT 0 sTg ( ~ w s +w T 0 ~ w T 0 ) LetAR ` for some` 1 be compact. Introduce the set of admissible controlsA t;T 0 =L 2 t;T 0 (A). For any (t;x)2 [0;T ]R d and2A t;T , introduceX t;x; as the strong solution of SDE X t;x; s =x + Z s t b(r;X t;x; r ; r )dr + Z s t (r;X t;x; r ; r )dB s where (b;)(t;x;a) : [0;T ]R d A!R d ;R dd 0 are bounded, uniformly continuous int;a and uni- formly Lipschitz inx. For anytT 0 T and2L 2 t;T 0 , introduce (Y s ;Z s ) = (Y t;x;;T 0 ; s ;Z t;x;;T 0 ; s ) as the strong solution of BSDE Y s = + Z T 0 s f(r;X t;x; r ; r ;Y r ;Z r )dr Z T 0 s Z r dB r where f(t;x;a;y;z) : [0;T ]R d AR m R md 0 ! R m is bounded, uniformly continuous in t;a, uniformly Lipschitz overx and with bounded derivatives ony;z. DenoteY t;x; s :=Y t;x;;T;g s where g :R d !R m is a given Lipschitz terminal value. Also, set J(t;x;;T 0 ;) :=Y t;x;;T 0 ; t ; J(t;x;) :=Y t;x; t 6 For a given terminalV(T;x)2C 2 (R d ;D m ), introduce the main object of interest, the set value of the control problem as V(t;x) :=cl n J(t;x;;T;) : for all2A t;T ;2L 2 t;T s.t.2V(T;X t;x; T ) a.s. o (2.1) wherecl denotes the closure. Lemma1 (;)7!J(t;x;;T 0 ;) is continuous. Proof Consider ( n ; n )2A t;T 0 L 2 t;T 0 such that E Z T 0 t j n s s j 2 ds ! 0; Ej n j 2 ! 0; asn!1: Let us note that Z s t EjX t;x; n r X t;x; r j 2 drC Z s t Z r t EjX t;x; n v X t;x; v j 2 +w(j v n v j) 2 dvdr for some modulus of continuity w. By Grönwall inequality and Dominated Convergence Theorem, we have that R T 0 t EjX t;x; n s X t;x; s j 2 ! 0 asn!1. Now, from stability estimates of BSDEs, jJ(t;x; n ;T 0 ; n )J(t;x;;T 0 ;)j 2 CEj n j 2 +CE Z T 0 t jf(s;X t;x; n s ; n s ; 0; 0)f(s;X t;x; s ; s ; 0; 0)jds 2 CEj n j 2 +CE Z T 0 t jX t;x; n s X t;x; s j 2 +w(j n s s j) 2 ds ! 0 7 2.2 DynamicProgrammingPrinciple We now introduce the appropriate formulation of DPP for the set value (2.1). Theorem1 LetV be dened as in (2.1). For anytT 0 T, it holds V(t;x) =cl n J(t;x;;T 0 ;) :82A t;T 0 ; 2L 2 t;T 0 s.t.2V(T 0 ;X t;x; T 0 ) a.s. o (2.2) Proof Fix (t;x) = (0;x 0 ) without loss of generality, and we may drop these from superscripts. Let us denote the right hand side as ~ V(0;x 0 ;T 0 ). To show thatV(0;x 0 ) ~ V(0;x 0 ;T 0 ), let" > 0 be arbitrary, takey 0 2V(0;x 0 ) with the corresponding, such thatjY 0;x 0 ;;T; 0 y 0 j < ". For anyw 0;T 0 2 0;T 0 , introduce ^ 2A T 0 ;T , ^ 2L 2 T 0 ;T , and ~ 2L 2 0;T 0 as ^ (t;w T 0 ;T ) :=(t;w 0;T 0 T 0 w T 0 ;T ); ^ (w T 0 ;T ) :=(w 0;T 0 T 0 w T 0 ;T ); ~ (w 0;T 0 ) :=Y T 0 ;X T 0 ;^ ;T; ^ T 0 where we suppressed the dependence onw 0;T 0 in the notations. Since ^ 2A T 0 ;T and ^ 2L 2 T 0 ;T , ~ 2 V(T 0 ;X T 0 ) almost surely by denition. By uniqueness of BSDE, ">jy 0 Y ;T; 0 j =jy 0 Y ;T 0 ; ~ 0 j and since" is arbitraryy 0 2 ~ V(0;x 0 ;T 0 ). To show the other direction, take arbitrary " > 0, y 0 2 ~ V(0;x 0 ;T 0 ), and 2A 0;T 0 , as in the denition of ~ V(0;x 0 ;T 0 ) such that jy 0 Y ;T 0 ; 0 j<": 8 By selection theorem, (See Himmelberg [39], Theorems 3.5, 5.7, 6.4) we have ^ w 2A T 0 ;T ; ^ w 2L 2 T 0 ;T wherew7! (^ w ; ^ w ) isF 0;T 0 -measurable such that j(w)Y T 0 ;X T 0 (w);^ w ;T; ^ w T 0 j" where ^ w 2V(T;X T 0 ;X T 0 (w);^ w T ). Then we concatenate by notingw 0;T =w 0;T 0 T 0 w T 0 ;T , ~ (t;w) :=(t;w)1 ft<T 0 g + ^ w (t;w)1 fT 0 tg 2A 0;T ; ~ (w 0;T ) := ^ w 0;T 0 (w T 0 ;T w T 0 ) And it follows jy 0 Y 0;x 0 ;~ ;T; ~ 0 j" + Y ;T 0 ; 0 Y ;T 0 ;Y ~ ;T; ~ T 0 0 " +CE Y T 0 ;X T 0 ;~ ;T; ~ T 0 2 <C" Since" is arbitrary, we concludey 0 2V(0;x 0 ). Lastly, we explore an important implication of DPP under the caseV =V m1 [V m ,V m is an open set inR m andV m1 is the topological boundary ofV m . To indicate this case, we sayV b is a hypersurface and to simplify notations, we denote the boundary asV b :=V m1 , the interior asV o :=V m . Letr(t;x;y) by the signed distance function associated withV b (t;x), positive fory2V 0 (t;x). Lemma2 LetV(t;x) be dened as in (2.1), and supposeV b is a hypersurface. (i): Supposethereexists2A t;T suchthatY t;x; t 2V b (t;x)forsomex2R d . ThenforanytT 0 T, Y t;x; T 0 2V b (T 0 ;X t;x; T 0 ). In words, optimal controls stays on the boundary. (ii): Givenarbitrary"> 0,consideranycontrol " 2A t;T andaterminal " asinthedenitionof (2.1) such thatr(t;x;Y t;x; " ;T; " t ) < " 2 . Then for anyt T 0 T,P(r(T 0 ;X t;x; " T 0 ;Y t;x; " ;T; " T 0 ) > ")! 0 as "! 0. 9 (iii): Givenarbitrary"> 0,consideranycontrol " 2A t;T andaterminal " asinthedenitionof (2.1) suchthatr(t;x;Y t;x; " ;T; " t )<" 2 . ThenforanytT 0 T andxedc> 0,P(r(T 0 ;X t;x; " T 0 ;Y t;x; " ;T; " T 0 )> c)C" for some constantC whenever" is suciently small. (iv): Given arbitrary" > 0, consider any control " 2A t;T and a terminal " as in the denition of (2.1) such thatr(t;x;Y t;x; " ;T; " t )<" 4 . Then for anytT 0 T,P(r(T 0 ;X t;x; " T 0 ;Y t;x; " ;T; " T 0 )>")" whenever" is suciently small. Proof We will show(ii),(iii) and(iv) by contradiction as(i) follows similarly with simpler arguments. The proof relies on the maximality of the representationV, in a sense that it contains all of the values, combined with the time consistency. Set (t;x) = (0;x 0 ) without loss of generality, and we will drop (0;x 0 ) from superscripts from now on. (ii): Let " t :=E h 1 n r(T 0 ;X " T 0 ;Y " T 0 )>" o jF B t i =1 n r(T 0 ;X " T 0 ;Y " T 0 )>" o Z T 0 t s dB s and by assuming the Lemma is wrong, there exists > 0 such that lim "!0 " 0 > . Here exists from the Martingale Representation Theorem. Introduce the SDE; t :=" 3=2 n(0;x 0 ;(0;x 0 ;Y " 0 )) + Z t 0 s ds + Z t 0 s dB s (2.3) where is the projection associated withV b and we will determine the; to claim ~ Y t :=Y " t + " t t together with ~ Z is a solution to the BSDE ~ Y t =Y " T 0 + T 0 T 0 + Z T 0 t f(s;X " s ; " s ; ~ Y s ; ~ Z s )ds Z T 0 t ~ Z s dB s (2.4) 10 To do so, let us note d ~ Y t = h f(t;X " t ; " t ;Y " t ;Z " t ) + " t t + tr( T t t ) i dt + h Z " t + " t t + t t i dB t (2.5) where takes values in (R 1d 0 ) m , and hence T takes values in (R d 0 d 0 ) m . Appareantly, ~ Z t :=Z " t + " t t + t t : Denote = [ 1 ; ; d 0 ] where i 2 R and = [ 1 ; ; d 0 ] where i 2 R m for alli2f1; ;d 0 g, then it follows tr( T t t ) = 1 t 1 t + + d 0 t d 0 t Dene a lifting of ~ 2R m , call ~ (i), to (R md 0 ) where ~ (i) occupies theith column and all other entries are 0. Set i (~ ) := Z 1 0 ( ^ @ z i f) T (t;X " t ; " t ;Y " t ;Z " t + 1 t ~ (1) + + i1 t ~ (i 1) + i t ~ (i))~ d for alli2f1; ;d 0 g where ^ @ z i f2 (R m1 ) m . Note that i ’s are Lipschitz in ~ as ^ @ z i f’s are assumed to be bounded and we satised tr( T t t ) =f(t;X " t ; " t ;Y " t ;Z " t )f(t;X " t ; " t ;Y " t ;Z " t + ~ t ) Similarly, determine as t (~ ) := Z 1 0 ( ^ @ y f) T (t;X " t ; " t ;Y " t + " t ~ ;Z " t + ~ t + " t t )~ d + Z 1 0 tr ( ^ @ z f) T (t;X " t ; " t ;Y " t + " t ~ ;Z " t + ~ t + " t t ) t d 11 where ^ @ y f2 (R m1 ) m and ^ @ z f2 (R md 0 ) m and similarly t is Lipschitz in ~ by assumptions. Then we satised " t t =f(t;X " t ; " t ;Y " t ;Z " t + ~ t )f(t;X " t ; " t ;Y " t + " t ~ ;Z t + " t t + ~ t ) Since (2.3) is well-posed, we conclude from (2.5) that ( ~ Y; ~ Z) indeed solves (2.4). Lastly, introduce ( ^ Y; ^ Z) the solution of the BSDE ^ Y t =Y T 0 + " T 0 T 0 1 fj T 0 j<"g + Z T 0 t f(s;X " s ; " s ; ^ Y s ; ^ Z s )ds + Z T 0 t ^ Z s dB s By standard BSDE estimates, j ^ Y 0 ~ Y 0 j 2 CE " T 0 T 0 1 fj T 0 j"g 2 C " 2 Ej T 0 j 4 C" 4 Last step follows by using Burkholder-Davis-Gundy (BDG) inequality and Grönwall inequality as follows (a = 4 below, we will not repeat for other parts.) Ej t j a 3 a1 " 3a=2 + 3 a1 E Z t 0 s ( s ) s (0)ds a + 3 a1 E Z t 0 s ( s ) s (0)dB s a 3 a1 " 3a=2 +CT a1 0 Z t 0 Ej s j a ds +CT a 2 1 0 Z t 0 Ej s j a ds Therefore, ^ Y 0 = ~ Y 0 +o(") =Y 0 + " 0 " 3=2 n(0;x 0 ;(0;x 0 ;Y " 0 )) +o(") By construction ^ Y T 0 2V(T 0 ;X " T 0 ), and Theorem 1 implies ^ Y 0 2V(0;x 0 ). This is a contradiction because r(0;x 0 ;Y " 0 )<" 2 and recall that lim "!0 " 0 > > 0. 12 For(iii), we repeat the same arguments where we dene t :=E h 1 n r(T 0 ;X " T 0 ;Y " T 0 )>c o jF B t i =1 n r(T 0 ;X " T 0 ;Y " T 0 )>c o Z T 0 t s dB s and by contradiction for any constantC, for innitely many" with accumulation point at 0, 0 > C" holds. Also, redene the SDE as t :="n(0;x 0 ;(0;x 0 ;Y " 0 )) + Z t 0 s ds + Z t 0 s dB s following the same steps, introduce ( ~ Y; ~ Z) and then ( ^ Y; ^ Z) as ^ Y t = ~ Y T 0 + T 0 T 0 1 fj T 0 j<cg + Z T 0 t f(s;X " s ; " s ; ^ Y s ; ^ Z s )ds + Z T 0 t ^ Z s dB s Similarly, j ^ Y 0 ~ Y 0 j 2 CE T 0 T 0 1 fj T 0 jcg 2 C c 2 Ej T 0 j 4 C" 4 Therefore, ^ Y 0 = ~ Y 0 +w(")" 2 =Y " 0 + 0 "n(0;x 0 ;(0;x 0 ;Y " 0 )) +w(")" 2 for some boundedw. Note that 0 " > C" 2 for any constantC as explained initially, and recall that the assumption isr(0;x 0 ;Y " 0 ) < " 2 . This is a contradiction, as wheneverC is chosen suciently large to dominate the bound ofjwj + 1, ^ Y 0 will be outside ofV(0;x 0 ). Similarly for(iv), we dene " t :=E h 1 n r(T 0 ;X " T 0 ;Y " T 0 )>" o jF B t i =1 n r(T 0 ;X " T 0 ;Y " T 0 )>" o Z T 0 t s dB s 13 and by contradiction, for innitely many" with accumulation point at 0, " 0 >" holds. Also, redene the SDE as t :=" 3 n(0;x 0 ;(0;x 0 ;Y " 0 )) + Z t 0 s ds + Z t 0 s dB s following the same steps, introduce ( ~ Y; ~ Z) and then ( ^ Y; ^ Z) as ^ Y t =Y T 0 + " T 0 T 0 1 fj T 0 j<"g + Z T 0 t f(s;X " s ; " s ; ^ Y s ; ^ Z s )ds + Z T 0 t ^ Z s dB s Same estimates yields, j ^ Y 0 ~ Y 0 j 2 CE T 0 T 0 1 fj T 0 j"g 2 C " 2 Ej T 0 j 4 C" 10 Therefore, ^ Y 0 = ~ Y 0 +o(e 4 ) =Y " 0 + 0 " 3 n(0;x 0 ;(0;x 0 ;Y " 0 )) +o(" 4 ) Note that 0 " 3 >" 4 . This is a contradiction, as we assumedr(0;x 0 ;Y " 0 )<" 4 . 14 Chapter3 SetValuesforMeanFieldGame 3.1 Introduction As the game theory is intrinsically more involved than the control theory, we carry the further discussion to the beginning of this chapter. We remind again that the set value of games relies heavily on the types of admissible controls we use. We shall consider closed loop controls. The open loop equilibria of games are typically time inconsistent, see e.g. Buckdahn’s counterexample in Pham-Zhang [56, Appendix E] for a two person zero sum game, and consequently, the set value of games with open loop controls would violate the DPP. For the MFG, noting that the required symmetry decomposes the game problem into a standard control problem and a xed point problem of measures, and that open loop and closed loop controls yield the same value function for a standard control problem, it is possible that the set value with open loop controls still satises the DPP. Nevertheless, bearing in mind the DPP of the set value for more general (non-symmetric) games, as well as the practical consideration in terms of the information available to the players, we shall focus on closed loop controls. There is also a very subtle path dependence issue. While the game parameters are state dependent, we may consider both state dependent and path dependent controls. For general non- zero sum games (not mean eld type), [30] shows that DPP holds for the set value for path dependent controls, but in general fails for the set value for state dependent controls. For MFGs with closed loop 15 controls, again due to the required symmetric properties, the set values for both state dependent controls and path dependent controls will satisfy the DPP, but they are in general not equal. For MFGs with closed loop relaxed controls, or say closed loop mixed strategies, however, it turns out that the state dependent controls and the path dependent controls induce the same set value which still satises the DPP. We next turn to the convergence result. Let V and V N denote the set values of the MFG and the corresponding N-player games, respectively, under appropriate closed-loop controls. Our convergence result reads roughly as follows (the precise form is dierent): lim N!1 V N (0;~ x) =V(0;); whenever 1 N N X i=1 x i !: (3.1) In the realm of master equations, again under certain monotonicity conditions and hence with unique MFE, one can show that the values of theN-player games converge to the value of the MFG. See Cardaliaguet- Delarue-Lasry-Lions [13], followed by Bayraktar-Cohen [5], Cardaliaguet [12], Cecchin-Pelino [20], Delarue- Lacker-Ramanan [24, 25], Gangbo-Meszaros [37], and Mou-Zhang [53], to mention a few. So (3.1) can be viewed as their natural extension to MFGs without monotonicities. To ensure the convergence, one main feature is that we dene the set value as the limit of the approx- imate set values over approximate equilibria, rather than the true equilibria. We call the latter the raw set value, and both the set value and the raw set value satisfy the DPP. However, the raw set value is extremely sensitive to small perturbations of the game parameters, in fact, in general even its measurability is not clear, so one can hardly expect the convergence for the raw set values. In the standard control theory, the value function is dened as the inmum of controlled values, which is exactly the limit of values over approximate optimal controls, rather than the value over true optimal controls which may not even exist. So our set value, not the raw set value, is the natural extension of the standard value function in control theory. Moreover, since we are considering innitely many players, an approximate equilibrium means it 16 is approximately optimal for most players, but possibly with a small portion of exceptions, as introduced in Carmona [14]. At this point we should mention that, for MFGs without monotonicity conditions, there have been many publications for the convergence ofN-player games, in terms of equilibria instead of values. For open loop controls, we refer to Camona-Delarue [15], Feleqi [31], Fischer [32], Fischer-Silva [33], Lacker [45], Lasry-Lions [48], Lauriere-Tangpi [49], and Nutz-San Martin-Tan [54], to mention a few. In particular, [45] provides the full characterization for the convergence: any limit of approximate Nash equilibria of N-player games is a weak MFE, and conversely any weak MFE can be obtained as such a limit. The work [32] is also in this direction. For closed loop controls, which we are mainly interested in, the situation becomes much more subtle. The seminal paperp Lacker [46] established the following result: {Strong MFEs} {Limits ofN-player approx. equilibria} {Weak MFEs}: (3.2) Here an MFE is strong if it depends only on the state processes, and weak if it allows for additional ran- domness. The left inclusion in (3.2) was known to be strict in general. This work has very interesting further developments recently ∗ by Lacker-Flem [47] and Djete [26]. In particular, [26] shows that the right inclusion in (3.2) is actually an equality. We emphasize again that we are considering the convergence of sets of values, rather than sets of equilibria as in (3.2). For standard control problems, the focus is typically to characterize the (unique) value and to ndone (approximate) optimal control, and the player is less interested in ndingall optimal controls since they have the same value. The situation is quite dierent for games, because dierent equilibria can lead to dierent values. Then it is not satisfactory to nd just one equilibrium (especially if it is not Pareto optimal). However, for dierent equilibria which lead to the same value, the players are indierent on them. So for practical purpose the players would be more interested in nding all possible ∗ These two works [26, 47] were circulated slightly after our present paper. 17 values † and then to nd one (approximate) equilibrium for each value. This is one major motivation that we focus on the set value, rather than the set of all equilibria. We also note that in general the set value could be much simpler than the set of equilibria. For example, in the trivial case that both the terminal and the running cost functions are constants, the set value is a singleton, while the set of equilibria consists of all admissible controls. We should point out that our admissible controls dier from those in [26, 46, 47] in two aspects, due to both practical and technical considerations. First, for theN-player games, [26, 46, 47] use full information controls i (t;X 1 t ; ;X N t ), while we consider symmetric controls i (t;X i t ; N t ), whereX i t is the state of Playeri, and N t := 1 N P N j=1 X j t is the empirical measure of all the players’ states. Note that, as a principle the controls should depend only on the information the players observe. While both settings are very interesting, sinceN is large, the full information may not be available in many practical situations. The second dierence is that we assume each control is Lipschitz continuous in, while [26, 46, 47] allow for measurable controls. We shall emphasize though we allow the Lipschitz constant to depend on the control, and thus our set value does not depend on any xed Lipschitz constant. Roughly speaking, we are considering game values which can be approximated by Lipschitz continuous approximate equilibria. This is typically the case in the standard control theory: even if the optimal control is discontinuous, in most reasonable framework we should be able to nd Lipschitz continuous approximate optimal controls. The situation is more subtle for games. There indeed may exist (closed loop) equilibrium whose value cannot be approximated by any Lipschitz continuous approximate equilibria. While clearly more general and very interesting mathematically, such measurable equilibria are hard to implement in practice, since inevitably we have all sorts of errors in terms of the information, or say, data. Their numerical computation is another serious challenge. For example, in the popular machine learning algorithm, the key idea is to approximate the controls via composition of linear functions and the activation function, then by denition the optimal † Another very interesting question is how to choose one value, which is optimal in appropriate sense, after characterizing the set value. We shall leave this for future research. 18 controls/equilibria provided by these algorithms are (locally) Lipschitz continuous. That is, the game values falling out of our set value are essentially out of reach of these algorithms, see e.g. [52]. Moreover, as a consequence of our constraints, our proof of (3.1) is technically a lot easier than the compactness arguments for (3.2) used in [26, 46, 47]. Finally we would like to mention some other approaches for MFGs with multiple equilibria. One is to add sucient (possibly innitely dimensional) noise so that the new game will become non-degenerate and hence have unique MFE, see e.g. Bayraktar-Cecchin-Cohen-Delarue [6, 7], Delarue [22], Delarue-Foguen Tchuendom [23], Foguen Tchuendom [34]. Another approach is to study a special type of MFEs, see e.g. Cecchin-Dai Pra-Fisher-Pelino [18], Cecchin-Delarue [19], and [23]. Another interesting work is Possamai- Tangpi [57] which introduces an additional parameter function such that the MFE corresponding to any xed is unique and then the desired convergence is obtained. 3.2 Meaneldgamesonnitespacewithclosedloopcontrols In this section we consider an MFG on nite space (both time and state are nite) with closed loop controls, and for simplicity we restrict to state dependent setting. Since the game typically has multiple MFEs which may induce dierent values, see Example 3.7.1 below for an example, we shall introduce the set value of the game over all MFEs. Our goal is to establish the DPP for the MFG set value, and we shall show in the next section that the set values of the correspondingN-player games converge to the MFG set value. 3.2.1 Thebasicsetting LetT :=f0; ;Tg be the set of discrete times;T t :=ft; ;Tg fort2T;S the nite state space ‡ with sizejSj = d;P(S) the set of probability measures onS, equipped with the 1-Wasserstein distanceW 1 . ‡ We may allow the state spaceSt to depend on timet and all the results in this paper will remain true. 19 SinceS is nite,W 1 is equivalent to the total variation distance § which is convenient for our purpose: by abusing the notationW 1 , W 1 (;) := X x2S j(x)(x)j; ;2P(S): (3.3) LetP 0 (S) denote the subset of2P(S) which has full support, namely(x)> 0 for allx2S. Moreover, letAR d 0 be a measurable set from which the controls take values; andq :TSP(S)AS! (0; 1) be a transition probability function: X ~ x2S q(t;x;;a; ~ x) = 1; 8(t;x;;a)2TSP(S)A: We shall use the weak formulation which is more convenient for closed loop controls. That is, we x the canonical space and consider controlled probability measures on it. To be precise, let :=X :=S T +1 be the canonical space; X : T ! S the canonical process: X t (!) = ! t ; F :=fF t g t2T := F X the ltration generated byX; andA state the set of state dependent admissible controls :TS!A. Introduce the concatenation for controls: ( T 0 ~ )(s;x) :=(s;x)1 fs<T 0 g + ~ (s;x)1 fsT 0 g ; ; ~ 2A state : (3.4) It is clear that T 0 ~ 2A state . Given (t;;)2 TP(S)A state , letP t;; denote the probability measure onF T determined recursively by: fors =t; ;T , P t;; X 1 t =; P t;; (X s+1 = ~ xjX s =x) =q(s;x; s ;(s;x); ~ x); where s :=P t;; X 1 s : (3.5) § More precisely, the total variation distance is 1 2 W1 for theW1 in (3.3). 20 We note that :=f s g s2Tt are uniquely determined andX is a Markov chain onT t underP t;; . We also note that depends on (t;) as well, but we omit it for notational simplicity. However, the distribution offX s g s=0;;t1 is not specied and is irrelevant, andf s g 0s<t is also irrelevant. Moreover, given f g :=f s g s2Tt ,x2S, and ~ 2A state , letP fg;t;x;~ denote the probability measure onF T determined recursively by: fors =t; ;T 1, P fg;t;x;~ (X t =x) = 1; P fg;t;x;~ (X s+1 = xjX s = ~ x) =q(s; ~ x; s ; ~ (s; ~ x); x): (3.6) As in the standard MFG literature, here we are assuming that the population uses the common control while the individual player is allowed to use a dierent control ~ . We remark that, since we assumeq > 0, then for any (t;) and, s 2P 0 (S) for alls > t. For the convenience of presentation, in this section we shall restrict our discussion to the case2P 0 (S). The general case that the initial measure is not fully supported can be treated fairly easily, as we will do in Section 3.6 below. The situation with degenerateq, however, is more subtle and we shall leave for future research. We nally introduce the cost functional for the MFG: for the =f g in (3.5), J(t;;;x; ~ ) :=J( ;t;x; ~ ); v(f g;s;x) := inf ~ 2Astate J(f g;s;x; ~ ); where J(f g;s;x; ~ ) :=E P fg;s;x;~ h G(X T ; T ) + T1 X r=s F (r;X r ; r ; ~ (r;X r )) i : (3.7) Here, sinceT andS are nite,F andG are arbitrary measurable functions satisfying inf a2A F (t;x;;a)>1 for all (t;x;): 21 We remark that herev(f g;;) is the value function of a standard stochastic control problem with parameterf g. In particular, in continuous time models, andv( ;;) will satisfy the Fokker-Planck equation and the HJB equation, respectively. Denition3.2.1 Given (t;)2TP 0 (S),wesay 2A state isastatedependentMFEat (t;),denoted as 2M state (t;), if J(t;; ;x; ) =v( ;t;x); for allx2S: (3.8) In this and the next section, we will use the following conditions. Assumption3.2.2 (i)qc q for some constantc q > 0; (ii)q is Lipschitz continuous in (;a), with a Lipschitz constantL q ; (iii)F;GareboundedbyaconstantC 0 anduniformlycontinuousin (;a),withamodulusofcontinuity function. 3.2.2 TherawsetvalueV 0 We introduce the raw set value for the MFG over all state dependent MFEs: V 0 (t;) := n J(t;; ;; ) : 2M state (t;) o L 0 (S;R): (3.9) Here the elements of V 0 (t;) are functions from S to R, which coincide with R d by identifying ' 2 L 0 (S;R) with ('(x) :x2S)2R d . We callV 0 (t;) the raw set value and we will introduce the set value V(t;) of the MFG in the next subsection. 22 Next, for anyT 0 2T t , 2L 0 (SP 0 (S);R), we introduce the MFG onft; ;T 0 g: J(T 0 ; ;t;;;x; ~ ) :=E P ;t;x;~ h (X T 0 ; T 0 ) + T 0 1 X s=t F (s;X s ; s ; ~ (s;X s )) i : (3.10) In the obvious sense we dene 2M state (T 0 ; ;t;) by: for anyx2S, J(T 0 ; ;t;; ;x; ) =v(T; ; ;t;x) := inf ~ 2Astate J(T; ;t;; ;x; ~ ): (3.11) At below we will repeatedly use the following simple fact due to the tower property of conditional expec- tations: J(t;;;x; ~ ) =J(T 0 ; ;t;;;x; ~ ); where (y;) :=J(T 0 ;;;y; ~ ): (3.12) The following time consistency of MFE is the essence of the DPP for the raw set value. Proposition3.2.3 Fix 0t<T 0 T and2P 0 (S). For any ; ~ 2A state , denote ^ := T 0 ~ and (y;) := J(T 0 ;; ~ ;y; ~ ). Then ^ 2M state (t;) if and only if 2M state (T 0 ; ;t;) and ~ 2M state (T 0 ; T 0 ). Proof (i) We rst prove the if part. Let 2M state (T 0 ; ;t;) and ~ 2M state (T 0 ; T 0 ). For arbitrary 2A state andx2S, by (3.12) we have J(t;; ^ ;x;) =E P ;t;x; h J(T 0 ; T 0 ; ~ ;X T 0 ;) + T 0 1 X s=t F (s;X s ; s ;(s;X s )) i E P ;t;x; h J(T 0 ; T 0 ; ~ ;X T 0 ; ~ ) + T 0 1 X s=t F (s;X s ; s ;(s;X s )) i =E P ;t;x; h (X T 0 ; T 0 ) + T 0 1 X s=t F (s;X s ; s ;(s;X s )) i =J(T 0 ; ;t;; ;x;)J(T 0 ; ;t;; ;x; ) =J(t;; ^ ;x; ^ ); 23 where the rst inequality is due to ~ 2 M state (T 0 ; T 0 ) and the second inequality is due to 2 M state (T 0 ; ;t;). Then ^ 2M state (t;). (ii) We now prove the only if part. Let ^ 2M state (t;). For any2A state , we have T 0 ~ 2 A state . Then, since ^ 2M state (t;), for anyx2S, by (3.12) we have J(T 0 ; ;t;; ;x; ) =J(t;; ^ ;x; ^ )J(t;; ^ ;x; T 0 ~ ) =J(T; ;t;; ;x;): This implies that 2M state (T 0 ; ;t;). Moreover, note that T 0 2A state and again since ^ 2M state (t;), we have E P ;t;x; h J(T 0 ; T 0 ; ~ ;X T 0 ; ~ ) + T 0 1 X s=t F (s;X s ; s ; (s;X s )) i =J(t;; ^ ;x; ^ )J(t;; ^ ;x; T 0 ) =E P ;t;x; h J(T 0 ; T 0 ; ~ ;X T 0 ;) + T 0 1 X s=t F (s;X s ; s ; (s;X s )) i : This implies that, recalling thev in (3.7) and by the standard stochastic control theory, E P ;t;x; h J(T 0 ; T 0 ; ~ ;X T 0 ; ~ ) i inf 2Astate E P ;t;x; h J(T 0 ; T 0 ; ~ ;X T 0 ;) i = E P ;t;x; h v( ^ ;T 0 ;X T 0 ) i : (3.13) On the other hand, by denitionv( ^ ;T 0 ; ~ x)J(T 0 ; T 0 ; ~ ; ~ x; ~ ) for all ~ x2S. Then J(T 0 ; T 0 ; ~ ;X T 0 ; ~ ) =v( ^ ;T 0 ;X T 0 ); P ;t;x; -a.s. Since q > 0, then clearly P ;t;x; (X T 0 = ~ x) > 0 for all ~ x 2 S. Thus J(T 0 ; T 0 ; ~ ; ~ x; ~ ) = v( ^ ;T 0 ; ~ x), for all ~ x2S. This implies that ~ 2M state (T 0 ; T 0 ). 24 We then have the following DPP. Theorem3.2.4 For any 0t<T 0 T, and2P 0 (S), we have V 0 (t;) := n J(T 0 ; ;t;; ;; ) : for all 2L 0 (SP 0 (S);R) and 2A state such that (; T 0 )2V 0 (T 0 ; T 0 ) and 2M state (T 0 ; ;t;) o : (3.14) Proof Let ~ V 0 (t;) denote the right side of (3.14). First, for anyJ(T 0 ; ;t;; ;; )2 ~ V 0 (t;) with desired ; as in (3.14). Since (; T 0 )2 V 0 (T 0 ; T 0 ), there exists ~ 2M state (T 0 ; T 0 ) such that (; T 0 ) = J(T 0 ; T 0 ; ~ ;; ~ ). By Proposition 3.2.3 we have ^ := T 0 ~ 2M state (t;). Then, by (3.12),J(T 0 ; ;t;; ;; ) =J(t;; ^ ;; ^ )2V 0 (t;), and thus ~ V 0 (t;)V 0 (t;). On the other hand, let J(t;; ;; )2 V 0 (t;) with 2M state (t;). Introduce (x;) := J(T 0 ;; ;x; ). By Proposition 3.2.3 again we see that 2M state (T 0 ; ;t;) and 2M state (T 0 ; T 0 ), and the latter implies further that (; T 0 ) 2 V 0 (T 0 ; T 0 ). Then by the denition of ~ V 0 (t;) that J(t;; ;; ) =J(T 0 ; ;t;; ;; )2 ~ V 0 (t;). That is,V 0 (t;) ~ V 0 (t;). 3.2.3 ThesetvalueV state While Theorem 3.2.4 is elegant, the raw set valueV 0 (t;) is very sensitive to small perturbations of the coecientsF;G and the variable. Indeed, even the measurability of the subsetV 0 (t;)R d and the measurability of the mapping 7! V 0 (t;) are not clear to us. Moreover, in general it does not look possible to have the convergence of the raw set value of the correspondingN-player games toV 0 (t;). Therefore, in this subsection we shall modifyV 0 (t;) and introduce the set valueV state (t;) of the MFG as follows. 25 Denition3.2.5 (i) For any (t;)2TP 0 (S) and"> 0, letM " state (t;) denote the set of 2A state such that J(t;; ;x; )v( ;t;x) +"; for allx2S: (3.15) (ii) The set value of the MFG at (t;) is dened as: V state (t;) := \ ">0 V " state (t;); where (3.16) V " state (t;) := n '2L 0 (S;R) :k'J(t;; ;; )k 1 " for some 2M " state (t;) o : Recall (3.7), then (3.15) and (3.16) imply that 0J(t;; ;x; )v( ;t;x)"; k'v( ;t;)k 1 2": (3.17) So we may alternatively deneV " state (t;) by usingk'v( ;t;)k 1 ". Remark3.2.6 (i) In the case that there is only one player, namelyq;F;G do not depend on,P ;t;x; = P t;x; does not depend on and . Let V (t;x) := inf 2Astate E P t;x; h G(X T ) + T1 X s=t F (s;X s ;(s;X s )) i denotethevaluefunctionofthestandardstochasticcontrolproblem. Onecaneasilyseethat,whenthereexists an optimal control ,V 0 (t;) = V state (t;) =fV (t;)g. However, when there is no optimal control, we still haveV state (t;) =fV (t;)g butV 0 (t;) =;. So the natural extension of the value functionV is the set valueV state , notV 0 . 26 (ii)Weremarkthat T ">0 M " state (t;) =M state (t;),however,ingeneralitispossiblethatV state (t;) is strictly larger thanV 0 (t;). Indeed,V state (t;) can be even larger than the closure ofV 0 (t;), where the latter is still empty when there is no optimal control. Similarly, givenT 0 and ,M " state (T 0 ; ;t;) denotes the set of 2A state such that J(T 0 ; ;t;; ;x; ) inf 2Astate J(T 0 ; ;t;; ;x;) +"; 8x2S: (3.18) The DPP remains true forV state after appropriate modications as follows. Theorem3.2.7 Under Assumption 3.2.2 (i), for any 0t<T 0 T and2P 0 (S), V state (t;) := \ ">0 n '2L 0 (S;R) :k'J(T 0 ; ;t;; ;; )k 1 " for some 2L 0 (SP 0 (S);R) and 2A state such that (; T 0 )2V " state (T 0 ; T 0 ); 2M " state (T 0 ; ;t;) o : (3.19) This theorem can be proved by modifying the arguments in Theorem 3.2.4 and Proposition 3.2.3. How- ever, since the proof is very similar to that of Theorem 3.4.2 below, except that the latter is in the more complicated path dependent setting, we thus postpone it to Appendix. 3.3 TheN-playergamewithhomogeneousequilibria In this section we study theN-player game whose set value will converge toV state . 27 3.3.1 TheN-playergame Set N := X N with canonical processes ~ X = (X 1 ; ;X N ), whereX i stands for the state process of Playeri. The empirical measure of ~ X is denoted as: with the Dirac measure , N t := N ~ Xt where N ~ x := 1 N N X i=1 x i 2P(S); for~ x = (x 1 ; ;x N )2S N : (3.20) The playeri will have control i . In the literature, a closed loop control i may depend on the full in- formation ~ X. However, since we are talking about large N, in practice it may not be feasible for each player to observe all other players’ states individually. Moreover, in the MFG setting the population state is characterized by its distribution, not by each player’s individual state. So in this section we consider only symmetric controls, namely i depends on his/her own stateX i and on the others through the empirical measure N . For technical reasons, we introduce another parameterL 0. Denote A L state := n :TSP(S)!A : (t;x;)(t;x;) LW 1 (;);8t;x;; o ; (3.21) andA 1 state := S L0 A L state . Givent2T,~ x2S N , and~ = ( 1 ; ; N )2 (A 1 state ) N , letP t;~ x;~ denote the probability measure onF ~ X T determined recursively by: fors =t; ;T 1, P t;~ x;~ ( ~ X t =~ x) = 1;P t;~ x;~ ( ~ X s+1 =~ x 00 j ~ X s =~ x 0 ) = N Y i=1 q(s;x 0 i ; N s ; i (s;x 0 i ; N s );x 00 i ); (3.22) and the cost function of Playeri is: J i (t;~ x;~ ) :=E P t;~ x;~ h G(X i T ; N T ) + T1 X s=t F (s;X i s ; N s ; i (s;X i s ; N s )) i : (3.23) 28 Remark3.3.1 (i) It is obvious thatA 0 state =A state for theA state in the previous subsection. For the MFG, thereisnoneedtoconsiderA 1 state . Indeed,given (t;)2TP 0 (S),forany2A 1 state ,letP t;; bedened as in (3.5): again denoting s :=P t;; X 1 s , P t;; X 1 t =; P t;; (X s+1 = ~ xjX s =x) =q(s;x; s ;(s;x; s ); ~ x): Introduce ~ (s;x) :=(s;x; s ). Then ~ 2A state andonecaneasilyverifythat ~ = . Inparticular,the set valueV state (t;) will remain the same by allowing2A 1 state . For theN-player game, however, since N is random, the dependence on N makes the dierence. (ii) In the literature one typically uses N;i t := 1 N1 P j6=i X j t , rather than N t , in (3.22) and (3.23). The convergence results in this section will remain true if we use N;i instead. However, we nd it more convenient to use N t . There is another crucial issue concerning the equilibria. Note that an MFE requires by denition that each player takes the same control . To achieve the desired convergence, for theN-player game it is natural to consider only the homogeneous equilibria: 1 = = N , which we will do in the rest of this section. We note that, for a homogeneous control, theP t;~ x; :=P t;~ x;(;;) in (3.22) andJ i (t;~ x;) := J i (t;~ x; (; ;)) in (3.23) are also symmetric in~ x, or say invariant in terms of its empirical measure: P t;~ x; =P t; N ~ x ; ; J i (t;~ x;) =J N (t;x i ; N ~ x ;): (3.24) Denition3.3.2 For any " > 0;L 0, we say 2A L state is a homogeneous state dependent (";L)- equilibrium of theN-player game at (t;~ x), denoted as 2M N;";L state (t;~ x), if: J i (t;~ x; )v N;L i (t;~ x; ) := inf ~ 2A L state J i (t;~ x; ( ; ~ ) i ) +"; i = 1; ;N; where (; ~ ) i denote the vector~ such that i = ~ and j = for allj6=i: (3.25) 29 In light of (3.24), clearlyM N;";L state (t;~ x) is law invariant: M N;";L state (t;~ x) = M N;";L state (t;~ x 0 ) whenever N ~ x = N ~ x 0 . Thus, by abusing the notation, we may denoteM N;";L state (t;~ x) =M N;";L state (t; N ~ x ) and call a homogeneous state dependent (";L)-equilibrium at (t; N ~ x ). Note again thatq > 0, then similar to Subsection 3.2.1, for convenience in this section we restrict to only those~ x such that N ~ x has full support, and we denote S N 0 := ~ x2S N : N ~ x 2P 0 (S) ; P N (S) := N ~ x :~ x2S N 0 P 0 (S): (3.26) We now dene the set value of the homogeneousN-player game: recalling (3.24), V N state (t;) := \ ">0 V N;" state (t;) := \ ">0 [ L0 V N;";L state (t;);8(t;)2TP N (S); where V N;";L state (t;) := n '2L 0 (S;R) :9 2M N;";L state (t;) s.t.k'J N (t;;; )k 1 " o : (3.27) 3.3.2 Convergenceoftheempiricalmeasures Theorem3.3.3 Let Assumption 3.2.2 (ii) hold. Then, for any L 0, there exists a constant C L , which depends only onT;d;L q , andL such that, for anyt2 T,~ x2 S N 0 ,2P 0 (S),; ~ 2A L state , ands t, i = 1; ;N, E P t;~ x;(;~ ) i W 1 ( N s ; s ) C L N ; where N :=W 1 ( N ~ x ;) + 1 p N ; (3.28) W 1 P t;~ x;(;~ ) i (X i s ) 1 ; P ;t;x i ;~ X 1 s C L N : (3.29) Proof We rst recall Remark 3.3.1 and extend all the notations in Subsection 3.2.1 to those2A L state in the obvious sense. Fixt;i and denoteP N :=P t;~ x;(;~ ) i . 30 Step 1. We rst prove (3.28) fors = t + 1. Note thatX 1 t+1 ; ;X N t+1 are independent underP N . By (3.3), we have E P N W 1 ( N t+1 ; t+1 ) = X ~ x2S E P N j N t+1 (~ x) t+1 (~ x)j X ~ x2S E P N j N t+1 (~ x) t+1 (~ x)j 2 1 2 = X ~ x2S h Var P N N t+1 (~ x) + E P N N t+1 (~ x) t+1 (~ x) 2 i 1 2 (3.30) = X ~ x2S h 1 N 2 N X j=1 Var P N 1 fX j t+1 =~ xg + 1 N N X j=1 P N (X j t+1 = ~ x) t+1 (~ x) 2 i 1 2 C p N + X ~ x2S 1 N N X j=1 P N (X j t+1 = ~ x) t+1 (~ x) : Note that, by the desired Lipschitz continuity ofq in and thatjSj =d is nite, 1 N N X j=1 P N (X j t+1 = ~ x) t+1 (~ x) = 1 N X x2S h X j6=i q(t;x; N ~ x ;(t;x; N ~ x ); ~ x)1 fx j =xg +q(t;x; N ~ x ; ~ (t;x; N ~ x ); ~ x)1 fx i =xg i X x2S q(t;x;;(t;x;); ~ x)(x) 1 N X x2S N X j=1 q(t;x; N ~ x ;(t;x; N ~ x ); ~ x)1 fx j =xg X x2S q(t;x;;(t;x;); ~ x)(x) + 1 N X x2S q(t;x; N ~ x ;(t;x; N ~ x ); ~ x)q(t;x; N ~ x ; ~ (t;x; N ~ x ); ~ x) 1 fx i =xg X x2S q(t;x; N ~ x ;(t;x; N ~ x ); ~ x) N ~ x (x) X x2S q(t;x;;(t;x;); ~ x)(x) + 1 N X x2S h j N ~ x (x)(x)j +C L W 1 ( N ~ x ;)(x) i + 1 N C L N : Then,E P N W 1 ( N t+1 ; t+1 ) C p N +C L N C L N : 31 Step 2. We next prove (3.28) by induction. For anys =t; ;T 1, by Step 1 we have E P N W 1 ( N s+1 ; s+1 ) F ~ X s C L h W 1 ( N s ; s ) + 1 p N i ; P N -a.s. Then E P N W 1 ( N s+1 ; s+1 ) =E P N h E P N W 1 ( N s+1 ; s+1 ) ~ X N s i C L E P N W 1 ( N s ; s ) + C L p N : SinceT is nite, by induction we obtain (3.28) immediately. Step 3. We now prove (3.29). Denote s :=W 1 P N (X i s ) 1 ; P i X 1 s where P i :=P ;t;x i ;~ : Then t = 0, and fors =t; ;T 1, s+1 = X ~ x2S P N (X i s+1 = ~ x)P i (X s+1 = ~ x) = X ~ x2S E P N q(s;X i s ; N s ; ~ (s;X i s ; N s ); ~ x) E P i q(s;X s ; s ; ~ (s;X s ; s ); ~ x) X ~ x2S E P N q(s;X i s ; N s ; ~ (s;X i s ; N s ); ~ x) E P N q(s;X i s ; s ; ~ (s;X i s ; s ); ~ x) + X ~ x2S E P N q(s;X i s ; s ; ~ (s;X i s ; s ); ~ x) E P i q(s;X s ; s ; ~ (s;X s ; s ); ~ x) C L E P N W 1 ( N s ; s ) + X x;~ x2S q(s;x; s ; ~ (s;x; s ); ~ x) P N (X i s =x)P i (X s =x) C L N + s ; where the last inequality thanks to (3.28). Now by induction one can easily prove (3.29). 32 3.3.3 Convergenceofthesetvalues We rst study the convergence of the cost functions. Recall the N in (3.28) and the functionsv in (3.7) andv N;L i in (3.25). Theorem3.3.4 Let Assumption 3.2.2 (ii) and (iii) hold. For anyL 0, there exists a modulus of continuity function L ,whichdependsonlyonT;d;L q ,C 0 ,,andLsuchthat,foranyt2T, N ~ x 2P N (S),2P 0 (S), and any; ~ 2A L state ,i = 1; ;N, J i (t;~ x; (; ~ ) i )J(t;;;x i ; ~ ) + v N;L i (t;~ x;)v( ;t;x i ) L ( N ): (3.31) Proof Clearly the uniform estimates forJ implies that forv, so we shall only prove the former one. Recall (3.23), (3.7), and the notationsP N ,P i in the proof of Theorem 3.3.3. Then J i (t;~ x; (; ~ ) i )J(t;;;x i ; ~ ) I T + T1 X s=t I s ; where I T := E P N G(X i T ; N T ) E P i G(X T ; T ) ; I s := E P N F (s;X i s ; N s ; ~ (s;X i s ; N s )) E P i F (s;X s ; s ; ~ (s;X s ; s )) ; s<T: Note that, fors<T , by (3.29), I s E P N F (s;X i s ; N s ; ~ (s;X i s ; N s )) E P N F (s;X i s ; s ; ~ (s;X i s ; s )) + E P N F (s;X i s ; s ; ~ (s;X i s ; s )) E P i F (s;X s ; s ; ~ (s;X s ; s )) E P N C L W 1 ( N s ; s ) + X x2S F (s;x; s ; ~ (s;x; s )) P N (X i s =x)P i (X s =x) E P N C L W 1 ( N s ; s ) +C L N : 33 Similarly we have the estimate forI T , and thus J i (t;~ x; (; ~ ) i )J(t;;;x i ; ~ ) T X s=t E P N C L W 1 ( N s ; s ) +C L N : This, together with (3.28), implies (3.31) for some appropriately dened modulus of continuity function L . Our main result of this section is the following convergence of the set values. Recall, for a sequence of setsfE N g N1 , lim N!1 E N := \ n1 [ Nn E N , lim N!1 E N := [ n1 \ Nn E N . Theorem3.3.5 Let Assumption 3.2.2 (ii), (iii) hold and N ~ x 2P N (S)!2P 0 (S). Then \ ">0 [ L0 lim N!1 V N;";L state (t; N ~ x )V state (t;) \ ">0 lim N!1 V N;";0 state (t; N ~ x ) (3.32) In particular, since lim N!1 V N;";0 state (t; N ~ x ) [ L0 lim N!1 V N;";L state (t; N ~ x ), actually equalities hold. Note that~ x2S N 0 obviously depends onN, so more rigorously we should write~ x N in the above statements. For notational simplicity we omit thisN here. We also remark that at above we are not able to switch the order of lim N!1 and T ">0 S L0 in the left side, or the order of lim N!1 and T ">0 in the right side. Proof (i) We rst prove the right inclusion in (3.32). Fix'2V state (t;),"> 0, and set" 1 := " 2 . Note thatA state =A 0 state . By (3.16), there exists 2M " 1 state (t;) such thatk'J(t;; ;; )k 1 " 1 : Recall (3.15), we have J(t;; ;x; )v( ;t;x) +" 1 ; for allx2S: 34 For any2A 0 state =A state , by Theorem 3.3.4 we have J i (t;~ x; )J(t;; ;x i ; ) + 0 ( N ) v( ;t;x) +" 1 + 0 ( N )v N;L i (t;~ x; ) +" 1 + 2 0 ( N ): ChooseN large enough such that 0 ( N ) " 4 , thenJ i (t;~ x; ) v N;L i (t;~ x; ) +". This implies that 2M N ";0 (t; N ~ x ). Moreover, k'J N (t;; N ~ x ; )k 1 " 1 + sup i J i (t;~ x; )J(t;; ;x i ; ) " 1 + 0 ( N )" 1 + " 4 ": Then ' 2 V N;";0 state (t; N ~ x ) for all N large enough. That is, ' 2 lim N!1 V N;";0 state (t; N ~ x ). Since ' 2 V state (t;) and"> 0 are arbitrary, we obtain the right inclusion in (3.32). (ii) We next show the left inclusion in (3.32). Fix'2 \ ">0 [ L0 lim N!1 V N;";L state (t; N ~ x ) and"> 0. Then, for " 1 := " 2 > 0, there exist L " > 0 and an innite sequencefN k g k1 such that '2 V N k ;" 1 ;L" state (t; N k ~ x ) for all k 1. Recall (3.27), for each k 1 there exists k 2 M N k ;" 1 ;L" state (t; N k ~ x ) such thatk' J N (t;; N k ~ x ; k )k 1 " 1 . By Denition 3.3.2, we have J i (t;~ x; k ) v N k ;L" i (t;~ x; k ) + " 1 : Similar to (i), by Theorem 3.3.4 we have J(t;; k ;x i ; k )v( k ;t;x i ) +" 1 + 2 L" ( N k )v( k ;t;x i ) +"; fork large enough. That is, k 2M " state (t;). Similar to (i) again, fork large enough we havek' J(t;; k ;; k )k 1 ". Then'2V " state (t;). Since"> 0 is arbitrary, we obtain'2V state (t;), and hence derive the left inclusion in (3.32). 35 Remark3.3.6 (i)FromTheorem3.3.5(i)weseethat,forany 2M " 2 state (t;),wehave 2M N;";0 state (t; N ~ x ) whenN is large enough. Moreover, by (3.28) we have the desired estimate for the approximate equilibrium measureE P t;~ x; W 1 ( N s ; s ) C L N . This veries the standard result in the literature that an approxi- mate MFE is an approximate equilibrium of theN-player game. (ii)FromTheorem3.3.5(ii)weseethat,forany k 2M N k ; " 2 ;L" state (t; N k ~ x ),wehave k 2M " state (t;)when kislargeenough,andweagainhavetheestimatefortheapproximateequilibriummeasureE P t;~ x; k W 1 ( N k s ; k s ) C L N k . ThisisinthespiritthatanylimitpointoftheN-playerequilibriummeasuresisanMFEmeasure. Remark3.3.7 (i)Weshouldpointoutthatthekeytoobtaintheconvergencehereistoconsiderhomogeneous equilibriafortheN-playergames. IfweuseheterogeneousequilibriafortheN-playergames,itturnsoutthat wewillhavethedesiredconvergencewhenweconsiderrelaxedcontrolsfortheMFG,aswewilldointhenext two sections. (ii)AnothertechnicaltrickweareusingistheuniformLipschitzcontinuityrequirementontheadmissible controls. Theconvergenceanalysiswillbecomemoresubtlewhenweremovesuchregularityrequirement,see e.g. [46]. 3.4 Meaneldgamesonnitespacewithrelaxedcontrols In this section we study MFG with relaxed controls, or say mixed strategies. Besides its independent interest, our main motivation is to characterize the limit ofN-player games with heterogeneous equilibria. We shall still consider the nite space in Section 3.2, however, for the purpose of generality in this section we consider path dependent setting. 36 3.4.1 Therelaxedsetvaluewithpathdependentcontrols We start with some notations for the path dependent setting. Forx = (x t ) 0tT 2X, denote byx t^ = (x 0 ; ;x t ;x t ; ;x t ) the path stopping att andX t :=fx t^ : x2 Xg X. Forx; ~ x2 X, we say x = t ~ x ifx t^ = ~ x t^ . DenoteX t;x :=f~ x2X : ~ x = t xg andX t;x s :=X t;x \X s , forst. Introduce the concatenationx t ~ x2X by (x t ~ x) s :=x s 1 fstg + ~ x s 1 fs>tg ; and (x t x) s :=x s 1 fstg +x1 fs>tg ; x2S: For eacht2T, letP(X t ) denote the set of probability measures on ( ;F X t ), equipped with W 1 (;) := X x2Xt j(x)(x)j; 8;2P(X t ); andP 0 (X t ) the subset of2P(X t ) with full supportX t . Again this is just for convenience of presentation. For a measure2P(X) =P(X T ), denote t^ := X 1 t^ 2P(X t ). We remark that, by abusing the notation, here t^ denote the joint law of the stopped processX t^ , while in Section 3.2f g denote the family of marginal laws. For a path dependent function' onTXP(X), we say' is adapted if'(t;x;) ='(t;x t^ ; t^ ). Throughout this section, all the path dependent functions are required to be adapted. In particular, the data of the gameq :TXP(X)AS! (0; 1),F :TXP(X)A!R, andG :XP(X)!R are path dependent withq;F adapted. By adapting to the path dependent setting, we shall still assume Assumption 3.2.2. 37 LetA relax denote the set of path dependent adapted relaxed controls :TX!P(A). Givent2T, 2P(X t ), 2A relax , andx2X t , ~ x2X t;x , ~ 2A relax , we introduce: P t;; X 1 t^ =; P t;; (X s+1 = ~ xjX = s x) = Z A q(s;x; ;a; ~ x) (s;x;da); where s^ :=P t;; X 1 s^ ; st; P ;t;x;~ (X = t x) = 1; P ;t;x;~ (X s+1 = xjX = s ~ x) = Z A q(s; ~ x; ;a; x)~ (s; ~ x;da); J( ;s; ~ x; ~ ) :=E P ;t;x;~ h G(X; ) + T1 X r=s Z A F (r;X; ;a)~ (r;X;da) X = s ~ x i ; J(t;; ;x; ~ ) :=J( ;t;x; ~ ); v( ;s; ~ x) := inf ~ 2A relax J( ;s; ~ x; ~ ): (3.33) Denition3.4.1 (i)Foranyt2T,2P 0 (X t ),and"> 0,letM " relax (t;)denotethesetofrelaxed"-MFE 2A relax such that J(t;; ;x; )v( ;t;x) +"; for allx2X t : (3.34) (ii) The relaxed set value of the MFG at (t;) is dened as: V relax (t;) := \ ">0 V " relax (t;); wherek'k Xt := sup x2Xt j'(x)j; and (3.35) V " relax (t;) := n '2L 0 (X t ;R) :9 2M " relax (t;) s.t.k'J(t;; ;; )k Xt " o : Similarly, givenT 0 and :X T 0 P(X T 0 )!R, as in (3.10) dene J(T 0 ; ;t;; ;x; ~ ) :=E P ;t;x;~ h (X T 0 ^ ; T 0 ^ )+ T 0 1 X s=t Z A F (s;X; ;a)~ (s;X;da) i ; (3.36) 38 and letM " relax (T 0 ; ;t;) denote the set of 2A relax such that,8x2X t , J(T 0 ; ;t;; ;x; )v(T; ; ;s;x) := inf 2A relax J(T 0 ; ;t;; ;x; ) +": (3.37) Note that the tower property in (3.12) remains true for relaxed controls: J(t;; ;x; ~ ) =J(T 0 ; ;t;; ;x; ~ ); where (y;) :=J(T 0 ;; ;y; ~ ): (3.38) The DPP forV relax takes the following form. Theorem3.4.2 Under Assumption 3.2.2 (i), for anyt2T,T 0 2T t , and2P 0 (X t ), V relax (t;) = \ ">0 n '2L 0 (X t ;R) :k'J(T 0 ; ;t;; ;; )k Xt " for some 2L 0 (X T 0 P 0 (X T 0 );R) and 2A relax such that (; T 0 ^ )2V " relax (T 0 ; T 0 ^ ); 2M " relax (T 0 ; ;t;) o : (3.39) Proof We shall follow the arguments in Theorem 3.2.4, in particular, we shall extend Proposition 3.2.3. Let ~ V relax (t;) = T ">0 ~ V " relax (t;) denote the right side of (3.39). (i) We rst prove ~ V relax (t;) V relax (t;). Fix'2 ~ V relax (t;), " > 0, and set" 1 := " 4 . Since '2 ~ V " 1 relax (t;), then k'J(T 0 ; ;t;; ;; )k Xt " 1 for some desirable ; as in (3.39): Since (; T 0 ^ )2V " 1 relax (T 0 ; T 0 ^ ), there exists ~ 2M " 1 relax (T 0 ; T 0 ^ ) such that k (; T 0 ^ )J(T 0 ; T 0 ^ ; ~ ;; ~ )k X T 0 " 1 : 39 As in (3.4) denote ^ := T 0 ~ := 1 fs<T 0 g + ~ 1 fsT 0 g 2A relax . Then, for any x2 X t and 2A relax , similarly to Proposition 3.2.3 (i) we have J(t;; ^ ;x; ) =E P ;t;x; h J(T 0 ; T 0 ^ ; ~ ;X T 0 ^ ; ) + T 0 1 X s=t Z A F (s;X; ;a) (s;X;da) i E P ;t;x; h J(T 0 ; T 0 ^ ; ~ ;X T 0 ^ ; ~ ) + T 0 1 X s=t Z A F (s;X; ;a) (s;X;da) i " 1 E P ;t;x; h (X T 0 ^ ; T 0 ^ ) + T 0 1 X s=t Z A F (s;X; ;a) (s;X;da) i 2" 1 =J(T 0 ; ;t;; ;x; ) 2" 1 J(T 0 ; ;t;; ;x; ) 3" 1 =E P ;t;x; h (X T 0 ^ ; T 0 ^ ) + T 0 1 X s=t Z A F (s;X; ;a) (s;X;da) i 3" 1 E P ;t;x; h J(T 0 ; T 0 ^ ; ~ ;X T 0 ^ ; ~ ) + T 0 1 X s=t Z A F (s;X; ;a) (s;X;da) i 4" 1 =J(t;; ^ ;x; ^ ) 4" 1 =J(t;; ^ ;x; ^ )": That is, ^ 2M " relax (t;). Moreover, note that, by (3.38), k'J(t;; ^ ;; ^ )k Xt " 1 +kJ(T 0 ; ;t;; ;; )J(t;; ^ ;; ^ )k Xt =" 1 + sup x2Xt E P ;t;x; (X T 0 ^ ; T 0 ^ )J(T 0 ; T 0 ^ ; ~ ;X T 0 ^ ; ~ ) 2" 1 <": Then'2V " relax (t;). Since"> 0 is arbitrary, we obtain'2V relax (t;). (ii) We now prove the opposite inclusion. Fix'2V relax (t;) and"> 0. Let" 2 > 0 be a small number which will be specied later. Since'2V " 2 relax (t;), then k'J(t;; ;; )k Xt " 2 for some 2M " 2 relax (t;): 40 Introduce (y;) :=J(T 0 ;; ;y; ) and recall (3.38). Then k'J(T 0 ; ;t;; ;; )k Xt =k'(x)J(t;; ;x; )k Xt " 2 : Moreover, since 2M " 2 relax (t;), for any 2A relax andx2X t , we have J(T 0 ; ;t;; ;x; ) =J(t;; ;x; ) J(t;; ;x; T 0 ) +" 2 =J(T 0 ; ;t;; ;x; ) +" 2 : This implies that 2M " 2 relax (T 0 ; ;t;). We claim further that (; T 0 ^ )2V C" 2 relax (T 0 ; T 0 ^ ); (3.40) for some constantC 1. Then by (3.39) we see that'2 ~ V C" 2 relax (t;) ~ V " relax (t;) by setting" 2 " C . Since"> 0 is arbitrary, we obtain'2 ~ V relax (t;). To see (3.40), recalling (3.33), for any 2A relax we have E P ;t;x; h J(T 0 ; T 0 ^ ; ;X T 0 ^ ; ) i E P ;t;x; h J(T 0 ; T 0 ^ ; ;X T 0 ^ ; ) i =J(t;; ;x; )J(t;; ;x; T 0 )" 2 : Then, by taking inmum over 2A relax , it follows from the standard control theory that E P ;x; h J(T 0 ; T 0 ^ ; ;X T 0 ^ ; ) i E P ;t;x; h v( ;T 0 ;X T 0 ^ ) i +" 2 ; 8x2X t : 41 On the other hand, it is obvious thatv( ;T 0 ; ~ x) J(T 0 ; T 0 ^ ; ; ~ x; ) for all ~ x2 X T 0 . Moreover, sinceqc q , clearlyP ;t;x; (X = T 0 ~ x)c T 0 t q , for any ~ x2X t;x T 0 . Thus, 0 J(T 0 ; T 0 ^ ; ; ~ x; )v( ;T 0 ; ~ x) CE P ;t;x; h J(T 0 ; T 0 ^ ; ;X T 0 ^ ; )v( ;T 0 ;X T 0 ^ ) 1 fX= T 0 ~ xg i CE P ;t;x; h J(T 0 ; T 0 ^ ; ;X T 0 ^ ; )v( ;T 0 ;X T 0 ^ ) i C" 2 ; whereC :=c tT 0 q . This implies that 2M C" 2 relax (T 0 ; T 0 ^ ). Then (3.40) follows directly from (; T 0 ^ ) = J(T 0 ; T 0 ^ ; ;; ), and hence'2 ~ V relax (t;). Remark3.4.3 Consider the setting thatq;F;G are state dependent, as in Section 3.2. There is a very subtle issue between state dependence and path dependence of the controls. (i)Forastandardnon-zerosumgameproblemswheretheplayersmayhavedierentcostfunctionsF i ;G i , ifoneusesstatedependentcontrols,ingeneralthesetvaluedoesnotsatisfyDPP.Seeacounterexamplein[30]. However, with path dependent controls the set value of the game satises the DPP. (ii)InSection3.2,sinceallplayershavethesamecostfunction,aswesawthesetvaluewithstatedependent controls satises DPP. If we consider path dependent controls2A path , the set value will also satisfy DPP. However, the set values in these two settings are in general not equal, see Example 3.7.1 in Appendix for a counterexample. (iii)Forrelaxedcontrols,againrestrictingtostatedependentq;F;G,itturnsoutthatstatedependentand pathdependentcontrolsleadtothesamesetvalue,seeTheorem3.7.4inAppendix. Themainreasonisthatthe convex combination of relaxed controls remains a relaxed control, while the controls in Section 3.2 does not share this property. 42 3.4.2 Globalformulationofmeaneldgames In this subsection we provide an alternative formulation for the MFG. This new formulation is motivated from the heterogenous controls for theN-player games, and thus is crucial for the convergence result in the next section. LetA path denote the set of adapted path dependent controls : TX! A, and for eacht2 T, A t path = ((t;); ;(T 1;)) : 2A path . Denote t :=P(X t A t path ), and for each 2 t , dene recursively: forst,x2X t , and ~ x2X t;x , t^ (x) := (x;A t path ); s^ (~ x) := Z A t path s1 Y r=t q(r; ~ x; ;(r; ~ x); ~ x r+1 )(x;d): (3.41) Here, noting that2A t path can be equivalently expressed asf(s; ~ x) :tsT 1; ~ x2X t;x s g, we are using the following interpretation ond: for any' :A t path !R, Z A t path '()d := Z A Z A ' f(s; ~ x)g T1 Y s=t Y ~ x2X t;x s d(s; ~ x): (3.42) Next, for2P 0 (X t ), denote t () :=f2 t : t^ =g. Moreover, recall (3.33), J(t; ;x;) :=J( ;t;x;); v(t; ;x) :=v( ;t;x); x2X t ;2A t path : (3.43) To simplify the notations, we introduce: Q t s (f g; ~ x;) := s1 Y r=t q(r; ~ x;;(r; ~ x); ~ x r+1 ): (3.44) 43 In particular,Q t t (f g;x;) = 1. Then we have, for any ~ x2X t;x , s (~ x) := Z A t path Q t s ( ; ~ x;)(x;d); P ;t;x; (X = s ~ x) =Q t s ( ; ~ x;): (3.45) Denition3.4.4 For anyt2 T, 2P 0 (X t ), and" > 0, we call 2 t () a global"-MFE at (t;), denoted as 2M " global (t;), if Z A t path [J(t; ;x;)v(t; ;x)] (x;d)"; 8x2X t : (3.46) Note that the above is global in time, so we call a global equilibrium. Moreover, since there are innitely many2A t path , it is hard to requireJ(t; ;x;)v(t; ;x) " for each2A t path , we thus use the aboveL 1 -type of optimality condition. For thex part, however, since there are only nitely manyx and each of them has positive probability, we may require the optimality for eachx. The main result of this subsection is the following equivalence result. Theorem3.4.5 For anyt2T and2P 0 (X t ), we have V relax (t;) =V global (t;) := \ ">0 V " global (t;); where V " global (t;) := n '2L 0 (X t ;R) :9 2M " global (t;) s.t.k'v(t; ;)k Xt " o : (3.47) We shall prove the mutual inclusion of the two sides separately. First, given (t; ), we construct a relaxed control as follows: for anyt2T,x2X t , andst, ~ x2X t;x s , (s; ~ x;da) := 1 s^ (~ x) Z A t path Q t s ( ; ~ x;) (s;~ x) (da)(x;d): (3.48) 44 On the opposite direction, givent2T,2P 0 (X t ), 2A relax , recalling (3.42) we construct (x;d) :=(x) T1 Y s=t Y ~ x2X t;x s (s; ~ x;d(s; ~ x)); 8x2X t ;2A t path : (3.49) In particular, the following calculation implies 2 t (): (x;A t path ) = (x) T1 Y s=t Y ~ x2X t;x s (s; ~ x;A) =(x) T1 Y s=t Y ~ x2X t;x s 1 =(x): Lemma3.4.6 For anyt2T,2P 0 (X t ), and 2 t (), 2A relax , we have = and = . Moreover, J(t;; ;x; ) = 1 (x) Z A t path J(t; ;x;)(x;d); 8x2X t : (3.50) Proof We rst prove s^ = s^ by induction. The cases =t follows from the denitions. Assume it holds for allrs. Fors + 1 and ~ x2X t;x s+1 , by Fubini Theorem we have (s+1)^ (~ x) s^ (~ x s^ ) = Z A q(s; ~ x; ;a; ~ x s+1 ) (s; ~ x;da) = Z A q(s; ~ x; ;a; ~ x s+1 ) 1 s^ (~ x) Z A t path Q t s ( ; ~ x;) (s;~ x) (da)(x;d) = 1 s^ (~ x) Z A t path q(s; ~ x; ;(s; ~ x); ~ x s+1 )Q t s ( ; ~ x;)(x;d) = 1 s^ (~ x) Z A t path Q t s+1 ( ; ~ x;)(x;d) = (s+1)^ (~ x) s^ (~ x) : Then (s+1)^ = (s+1)^ , and we complete the induction argument. 45 We next prove s^ = s^ by induction. Again the cases = t is obvious. Assume it holds for all r 0. (i) We rst proveV global (t;)V relax (t;). Fix'2V global (t;) and" > 0. Let" 1 > 0 be a small number which will be specied later. Since'2 V " 1 global (t;), there exists 2M " 1 global (t;) such that k'v(t; ;)k Xt " 1 . Set := . For anyx2 X t , since = , by (3.33), (3.43) we have v( ;t;x; ) =v(t; ;x), and, by (3.50), (3.46), J(t;; ;x; )v(t; ;x) = 1 (x) Z A t path [J(t; ;x;)v(t; ;x)] (x;d) " 1 c "; provided" 1 > 0 is small enough. This implies 2M " relax (t;). Moreover, it is clear now that, for anyx2X t and for a possibly smaller" 1 , '(x)J(t;; ;x; ) " 1 + v(t; ;x)J(t;; ;x; ) " 1 + " 1 c "; Then'2V " relax (t;), and since"> 0 is arbitrary, we obtain'2V relax (t;). (ii) We next proveV relax (t;) V global (t;). Fix'2 V relax (t;)," > 0, and set" 2 := " 2 . Since '2V " 2 relax (t;), there exists 2M " 2 relax (t;) such thatk'J(t;; ;; )k Xt " 2 . Set := , then = . Since 2M " 2 relax (t;), we have j'(x)v(t; ;x)j =j'(x)v( ;t;x)j 2" 2 "; 8x2X t : 47 Moreover, note that, by (3.50) again, Z A t path [J(t; ;x;)v(t; ;x)] (x;d) =(x)[J(t;; ;x; )v(t; ;x)](x)" 2 " 2 ": (3.51) This implies'2V " global (t;), and hence by the arbitrariness of",'2V global (t;). 3.5 TheN-playergamewithheterogeneousequilibria In this section we drop the requirement 1 = = N for the N-player game, and show that the corresponding set value converges toV relax , which in general is strictly larger thanV state . We note that we shall still use the pure strategies, rather than mixed strategies, for theN-player game. Moreover, since we used path dependent controls in Section 3.4, we shall also use path dependent controls here. 3.5.1 TheN-playergame Let N and ~ X be as in Section 3.3, and denote N t^ := N t; ~ Xt^ ; where N t;~ x := 1 N N X i=1 x i2P(X t ); ~ x = (x 1 ; ;x N )2X N t : (3.52) Similarly to (3.26), for the convenience of the presentation we introduce X N 0;t := n ~ x2X N t : supp ( N t;~ x ) =X t o ; P N (X t ) := n N t;~ x :~ x2X N 0;t o : (3.53) 48 We shall consider path dependent symmetric controls:A t;1 path := S L0 A t;L path , where A t;L path := n :ft; ;T 1gXP(X)!A is adapted and uniformly Lipschitz continuous in (underW 1 ) with Lipschitz constantL o : Givent2T,~ x2X N 0;t , and~ = ( 1 ; ; N )2 (A t;1 path ) N , introduce, forst, P t;~ x;~ ( ~ X = t ~ x) = 1; P t;~ x;~ ( ~ X s+1 =~ x 00 j ~ X = s ~ x 0 ) = N Y i=1 q(s;x 0i ; N ; i (s;x 0i ; N );x 00 i ); J i (t;~ x;~ ) :=E P t;~ x;~ h G(X i ; N ) + T1 X s=t F (s;X i ; N ; i (s;X i ; N )) i ; v N;L i (t;~ x;~ ) := inf ~ 2A t;L path J i (t;~ x;~ i ; ~ ); i = 1; ;N: (3.54) Here (~ i ; ~ ) is the vector obtained by replacing i in~ with ~ . Denition3.5.1 For any" > 0;L 0, we say ~ 2 (A t;L path ) N is an (";L)-equilibrium of theN-player game at (t;~ x), denoted as~ 2M N;";L hetero (t;~ x), if: 1 N N X i=1 J i (t;~ x;~ )v N;L i (t;~ x;~ ) ": (3.55) Here, since there areN players and we will sendN!1, similar to (3.46) we do not require the optimality for each player. In fact, by (3.55) one can easily show that 1 N i = 1; ;N :J i (t;~ x;~ )v N;L i (t;~ x;~ ) p " p ": (3.56) This is exactly the ( p "; p ")-equilibrium in [14]. We then dene the set value of theN-player game with heterogeneous equilibria: 49 V N hetero (t;~ x) := \ ">0 V N;" hetero (t;~ x) := \ ">0 [ L0 V N;";L hetero (t;~ x); where V N;";L hetero (t;~ x) := n '2L 0 (X t ;R) :9~ 2M N;";L hetero (t;~ x) such that max x2Xt min fi: x i =xg '(x)v N;L i (t;~ x;~ ) " o : (3.57) Remark3.5.2 (i) An alternative denition ofV N;";L hetero (t;~ x) is to require' satisfying max i=1;;N '(x i )v N;L i (t;~ x;~ ) = max x2Xt max fi: x i =xg '(x)v N;L i (t;~ x;~ ) ": (3.58) Indeed, the convergence result Theorem 3.5.3 below remains true if we use (3.58). However, in general it is possible thatx i =x j butv N;L i (t;~ x;~ )6=v N;L j (t;~ x;~ ). Then, by xingN and sending"! 0, under (3.58) we would haveV N hetero (t;~ x) := T ">0 V N;" hetero (t;~ x) =;. (ii)Inthehomogeneouscase,v N;L i (t;~ x;~ ) =v N;L j (t;~ x;~ )wheneverx i =x j ,sowedon’thavethisissue in (3.27). (iii) Note that N t;~ x = N t;~ x 0 if and only if ~ x is a permutation of ~ x 0 , and one can easily verify that v N;L i (t;~ x;~ ) = v N;L (i) (t; (x (1) ; ;x (N) ); ( (1) ; ; (N) )) for any permutation onf1; ;Ng, . Then,similartothehomogenouscase,V N;";L hetero (t;~ x)isinvariantin N t;~ x andwewilldenoteisasV N;";L hetero (t; N t;~ x ). The following convergence result of the set value is in the same spirit of Theorem 3.3.5. Theorem3.5.3 Let Assumption 3.2.2 hold and N t;~ x 2P N (X t )!2P 0 (X t ) underW 1 . Then \ ">0 [ L0 lim N!1 V N;";L hetero (t; N t;~ x )V relax (t;) \ ">0 lim N!1 V N;";0 hetero (t; N t;~ x ): (3.59) 50 In particular, since lim N!1 V N;";0 hetero (t; N t;~ x ) [ L0 lim N!1 V N;";L hetero (t; N t;~ x ), actually equalities hold. Unlike Theorem 3.3.5, here theN-player game and the MFG take dierent types of controls~ and , respectively. The key for the convergence is the global formulation in Subsection 3.4.2 for MFG. Indeed, given t 2 T, ~ x 2 X N 0;t , and ~ 2 (A t;L path ) N , the N-player game is naturally related to the following N 2P(X t A t;L path ): N (x;d) := 1 N X i2I(x) i (d); whereI(x) := i = 1; ;N :x i =x ; x2X t : (3.60) By the symmetry of the problem, there exists a functionJ N , independent ofi, such that J i (t;~ x;~ ) =J N ( N ;t;x i ; i ); i = 1; ;N: (3.61) We shall use this and Theorem 3.4.5 to prove Theorem 3.5.3 in the rest of this section. We also make the following obvious observation: N (x;A t path ) = jI(x)j N = N t;~ x (x); 8x2X t : (3.62) Remark3.5.4 (i)InthissectionweareusingsymmetriccontrolsandweobtaintheconvergenceinTheorem 3.5.3. Ifweusefullinformationcontrols i (t; ~ X),asobservedin[46]intermsoftheequilibriummeasure,one may expect the limit set value will be strictly larger thanV relax . It will be interesting to nd an appropriate notionofMFEsothatthecorrespondingMFGsetvaluewillbeequaltotheabovelimit,inthesenseofTheorem 3.5.3. (ii)WhiletheconvergenceinTheorem3.5.3isaboutsetvalues,theproofsintherestofthissectionconrm the convergence of the approximate equilibria as well, exactly in the same manner as in Remark 3.3.6. 51 3.5.2 FromN-playergamestomeaneldgames In this subsection we prove the left inclusion in (3.59). Notice that the N in (3.60) is dened onA t;L path , rather thanA t path =A t;0 path . For this purpose, recall (3.44) and introduce N t^ (x) := N t;~ x (x); N s^ (~ x) := 1 N X i2I(x) Q t s ( N ; ~ x; i (;; N )); x2X t ; ~ x2X t;x s ;st; N (x;d) := (x) jI(x)j X i2I(x) i (d); where i (s; ~ x) := i (s; ~ x; N ): (3.63) Then it is obvious that i 2A t path and N 2 t (). Moreover, when = N t;~ x , by (3.45) and (3.62) it is straightforward to verify by induction that N = N . Theorem3.5.5 Let Assumption 3.2.2 (ii) hold. Then, for anyL 0, there exists a constantC L , depending only onT;d;L q , andL such that, for anyt2T,~ x2X N 0;t ,2P 0 (X t ),~ 2 (A t;L path ) N ; ~ 2A t;L path , and for the N ; N dened in (3.63), we have max 1iN max tsT E P t;~ x;(~ i ;~ ) W 1 ( N s^ ; N s^ ) C L N ; N :=W 1 ( N t;~ x ;) + 1 p N : (3.64) Proof Fixi and denote ~ j := j forj6=i, and ~ i := ~ i . We rst show that s :=E P N W 1 ( N s^ ; N s^ ) C L p N ; where P N :=P t;~ x;(~ i ;~ ) : (3.65) Indeed, for s t, by the conditional independence offX j s+1 g 1jN under P N , conditional onF s , it follows from the same arguments as in (3.30) that s+1 = E P N h E P N Fs W 1 ( N (s+1)^ ; N (s+1)^ ) i C p N +C X x2X s+1 E P N h 1 N N X j=1 P N (X j = s+1 xjF s ) N (s+1)^ (x) i : 52 Note that, 1 N N X j=1 P N (X j = s+1 xjF s ) 1 N N X j=1 1 fX j =sxg q(s;x; N ; j (s;x; N );x s+1 ) = 1 N N X j=1 1 fX j =sxg q(s;x; N ; ~ j (s;x; N );x s+1 )q(s;x; N ; j (s;x; N );x s+1 ) C L W 1 ( N s^ ; N s^ ) + 1 N =C L s + 1 N ; where in the last inequality, the rst term is due to the sum over allj6=i. Then s+1 C L s + C p N +E P N h X x2X s+1 1 N N X j=1 1 fX j =sxg q(s;x; N ; j (s;x; N );x s+1 ) 1 N X j2I(xt^) Q t s ( N ;x; j )q(s;x; N ; j (s;x; N );x s+1 ) i =C L s + C p N +E P N h X x2Xs 1 N N X j=1 1 fX j =sxg 1 N X j2I(xt^) Q t s ( N ;x; j ) i =C L s + C p N +E P N h X x2Xs N s^ (x) N s^ (x) i C L s + C p N : It is obvious that t = 0. Then by induction we obtain (3.65). 53 Next, denote s :=W 1 ( N s^ ; N s^ ). Forst, by (3.63), (3.45), and (3.44), we have s+1 = X x2Xt X ~ x2X t;x s+1 N (s+1)^ (~ x) N (s+1)^ (~ x) = X x2Xt X ~ x2X t;x s+1 1 N X j2I(x) Q t s+1 ( N ; ~ x; j ) (x) jI(x)j X j2I(x) Q t s+1 ( N ; ~ x; j ) = X x2Xt X ~ x2X t;x s+1 h 1 N X j2I(x) Q t s+1 ( N ; ~ x; j )Q t s+1 ( N ; ~ x; j ) + 1 N (x) jI(x)j X j2I(x) Q t s+1 ( N ; ~ x; j ) i C X x2Xt X ~ x2X t;x s+1 h 1 N X j2I(x) s X r=t W 1 ( N r^ ; N r^ ) + 1 N (x) jI(x)j jI(x)j i C s X r=t r +C X x2Xt N t;~ x (x)(x) C s X r=t r : Obviously k t = W 1 ( N t;~ x ;). Then by induction we have sup tsT s CW 1 ( N t;~ x ;). This, together with (3.65), implies (3.64) immediately. Theorem3.5.6 For the setting in Theorem 3.5.5 and assuming further Assumption 3.2.2 (iii), there exists a modulus of continuity function L , depending onT;d;L q ,C 0 ,,L, s.t. J i (t;~ x; (~ i ; ~ ))J(t; N ;x i ; ~ (; N )) + v N;L i (t;~ x;~ )v( N ;t;x i ) L ( N ): (3.66) Moreover, assume~ 2M N;" 1 ;L hetero (t;~ x) for some" 1 > 0, then Z A t path [J(t; N ;x;)v(t; N ;x)] N (x;d)" 1 + 2 L ( N ); 8x2X t : (3.67) In particular, if" 1 + 2 L ( N )", then N 2M " global (t;). 54 Proof First, given Theorem 3.5.5, (3.66) follows from the arguments in Theorem 3.3.4. Then, for~ 2 M N;" 1 ;L hetero (t;~ x) andx2X t , by (3.55) we have Z A t path [J(t; N ;x;)v(t; N ;x)] N (x;d) = 1 N X i2I(x) J(t; N ;x; i )v(t; N ;x) 1 N X i2I(x) h J(t; N ;x i ; i )J i (t;~ x;~ ) + J i (t;~ x;~ )v N;L i (t;~ x;~ ) + v N;L i (t;~ x;~ )v( N ;t;x i ) i L ( N ) +" 1 + L ( N ) =" 1 + 2 L ( N ); completing the proof. ProofofTheorem3.5.3: theleftinclusion. We rst x an arbitrary function '2 T ">0 S L0 lim N!1 V N;";L hetero (t; N t;~ x ), " > 0, and set " 1 := " 2 . Then there exists L " 0 and and a sequenceN k !1 (possibly depending on") such that'2 V N k ;" 1 ;L" 1 hetero (t; N k t;~ x ), for allk 1. Now choose k large enough so that 2 L" ( N k ) " 1 . By (3.57) there exists ~ 2M N k ;" 1 ;L" hetero (t;~ x) such that max x2Xt min i2I(x) j'(x)v N;L i (t;~ x;~ )j " 1 . By Theorem 3.5.6 we see that N k 2M " global (t;) and, by (3.66), k'v( N ;t;)k Xt max x2Xt min i2I(x) h '(x)v N;L i (t;~ x;~ ) + v N;L i (t;~ x;~ )v( N ;t;x) i " 1 + L" ( N )": Then'2V " global (t;). Since"> 0 is arbitrary, by Theorem 3.4.5 we get'2V relax (t;). 55 3.5.3 FrommeaneldgamestoN-playergames We now turn to the right inclusion in (3.59). Fixt2T,~ x2X N 0;t ,2P 0 (X t ), and 2A relax . Our goal is to construct a desired~ 2 (A t;0 path ) N . However, since~ , or equivalently the corresponding N , is discrete, we need to discretize rst. We note that it is slightly easier to discretize than a general 2 t (). First, given"> 0, there exists a partitionA =[ n" k=0 A k withn " depending on" (and ) such that, for some arbitrarily xeda k 2A k ,k = 0; ;n " , (s;x;A 0 )";8s2T t ;x2X s ; and jaa k j";8a2A k ; k = 1; ;n " : (3.68) Denote byA t;" path the subset of2A t;0 path taking values inA " :=fa k :k = 0; ;n " g. Dene " (s;x;da) := n" X k=0 (s;x;A k ) a k (da): (3.69) Recall (3.49), we see that supp ( " (x;d)) =A t;" path A t;0 path for allx2X t . Next, recall (3.62) thatN N t;~ x (x) =jI(x)j is a positive integer for allx2X t . Let " t;~ x 2P(X t A t;" path ) be a modication of " such that, " t;~ x (x;A t;" path ) = N t;~ x (x) andN " t;~ x (x;) is an integer; j " t;~ x (x;) " (x;)j 1 N +j N t;~ x (x)(x)j; 8(x;)2X t A t;" path : (3.70) Note that, sinceA t;" path is nite, such a construction is easy. We now construct~ 2 (A t;" path ) N , which relies on " and hence on". Note that X 2A t;" path [N " t;~ x (x;)] =N " t;~ x (x;A t;" path ) =N N t;~ x (x) =jI(x)j; 56 and eachN " t;~ x (x;) is an integer. LetI(x) =[ 2A t;" path I(x;) be a partition ofI(x) such thatjI(x;)j = N " t;~ x (x;). We then set i :=; i2I(x;); (x;)2X t A t;" path : (3.71) Let N be the one dened by (3.60) corresponding to this~ . It is clear that N = " t;~ x . Theorem3.5.7 (i)LetAssumption3.2.2(ii)hold. ThenthereexistsaconstantC,dependingonlyonT;d;L q , such that, for anyt2T,~ x2X N 0;t ,2P 0 (X t ), 2A relax ," > 0, and for the~ 2 (A t;" path ) N constructed above, we have, for the N in (3.64) and for any ~ 2A t;0 path , max 1iN max tsT E P t;~ x;(~ i ;~ ) W 1 ( N s^ ; s^ ) C" +C " N ; (3.72) whereC " may depend on" as well. (ii)AssumefurtherAssumption3.2.2(iii),thenthereexistsamodulusofcontinuityfunction 0 ,depending only onT;d;L q ,C 0 , and, such that, J i (t;~ x; (~ i ; ~ ))J( ;t;x i ; ~ ) + v N;0 i (t;~ x;~ )v( ;t;x i ) 0 C" +C " N : (3.73) Moreover, assume 2M " relax (t;), then 1 N N X i=1 J i (t;~ x;~ )v N;0 i (t;~ x;~ ) " + 2 0 C" +C " N ; 8x2X t : (3.74) In particular, this means that~ 2M N;~ ";0 hetero (t;~ x) with ~ " :=" + 2 0 C" +C " N . 57 Proof (i) We rst show by induction that s :=W 1 s^ ; " s^ C"; s =t; ;T: (3.75) Indeed, it is obvious that t = 0. Forst, by (3.33), (3.68), and (3.69), we have s+1 = X x2X s+1 (s+1)^ (x) " (s+1)^ (x) = X x2Xs;x2S s^ (x) Z A q(s;x; ;a;x) (s;x;da) " s^ (x) Z A q(s;x; " ;a;x) " (s;x;da) X x2Xs;x2S h s^ (x) " s^ (x) + n" X k=1 Z A k q(s;x; ;a;x)q(s;x; " ;a k ;x) (s;x;da) + Z A 0 q(s;x; ;a;x) (s;x;da) + Z A 0 q(s;x; " ;a;x) " (s;x;da) C s +C": Then by induction we have (3.75). We next show by induction that, recalling (3.63), s :=W 1 N s^ ; " s^ C " N ; s =t; ;T: (3.76) 58 Indeed, t =W 1 ( N t;~ x ;). Forst, noting that i 2A t;" path A t;0 path and recalling from Lemma 3.4.6 that " = " , then by (3.63) and (3.45) that s+1 =W 1 N s+1^ ; " (s+1)^ = X x2Xt X ~ x2X t;x s+1 1 N X 2A t;" path X i2I(x;) Q t s+1 ( N ; ~ x;) Z A t path Q t s+1 ( " ; ~ x;) " (x;d) = X x2Xt X ~ x2X t;x s+1 X 2A t;" path " t;~ x (x;)Q t s+1 ( N ; ~ x;) " (x;)Q t s+1 ( " ; ~ x;) X x2Xt X ~ x2X t;x s+1 X 2A t;" path h " t;~ x (x;) " (x;) Q t s+1 ( N ; ~ x;) + " (x;) Q t s+1 ( N ; ~ x;)Q t s+1 ( " ; ~ x;) i : Then, by (3.70) and noting thatC " :=jA t;" path j is independent ofN, we have s+1 X x2Xt X ~ x2X t;x s+1 X 2A t;" path h N Q t s+1 ( N ; ~ x;) +C " (x;) s X r=t W 1 N r^ ; " r^ i C " N +C s X r=t r : This implies (3.76) immediately. Finally, combining (3.75), (3.76), and (3.64), we obtain (3.72). (ii) First, similar to (3.66), by (3.72) we have (3.73) following from the arguments in Theorem 3.3.4. Next, for 2M " relax (t;), by (3.51) we have 2M " global (t;). Then (3.74) follows from similar arguments as those for (3.67). ProofofTheorem3.5.3: therightinclusion. Fix'2V relax (t;) and"> 0. Let" 1 > 0 be a small number which will be specied later. There exists 2M " 1 relax (t;) such thatk'J(t;; ;; )k Xt " 1 . Let " 1 and~ be constructed as above. By (3.74) we have 1 N N X i=1 J i (t;~ x;~ )v N;0 i (t;~ x;~ ) " 1 + 2 0 C" 1 +C " 1 N ; 8x2X t : 59 Choose " 1 small enough such that " 1 + 2 0 (C" 1 +" 1 ) < ". Then, for all N large enough such that N " 1 C" 1 , we have 1 N P N i=1 J i (t;~ x;~ )v N;0 i (t;~ x;~ ) ". That is,~ 2V N;";0 hetero (t; N t;~ x ) for allN large enough. Then, following the same arguments as those in the proof for the left inclusion, we can easily get'2 V N;";0 hetero (t; N t;~ x ) for allN large enough, and thus'2 lim N!1 V N;";0 hetero (t; N t;~ x ). Since" > 0 is arbitrary, we get the desired inclusion. 3.6 Adiusionmodelwithstatedependentdriftcontrols In this section we study a diusion model with closed loop drift controls, where the laws of the controlled state process are all equivalent. The volatility control case involves mutually singular measures (corre- sponding to degenerateq in the discrete setting) and is much more challenging. We shall leave that for future research. To ensure the convergence, we consider state dependent homogeneous controls for the N-player games, as we did in Section 3. 3.6.1 Themeaneldgameandthedynamicprogrammingprinciple Let T > 0 be a xed terminal time, ( ;F;F = fF t g 0tT ;P) a ltered probability space whereF 0 is atomless; B a d-dimensional Brownian motion; and the set A R d 0 a Borel measurable set. The state processX will also take values inR d . Its law lies in the spaceP 2 :=P 2 (R d ) equipped with the 2-Wasserstein distance W 2 . We remark that in the nite state space case W 1 and W 2 are equivalent, while in diusion models they are not. In fact, at below we shall requireW 1 -regularity, which is stronger than theW 2 -regularity, and obtainW 1 -convergence, which is weaker than theW 2 -convergence. This is not surprising in the mean eld literature, see, e.g. [53]. The main advantage of theW 1 -distance is the following well known representation, see e.g. [16]: for any; ~ 2P 1 (R d ), W 1 (; ~ ) = sup n Z R d '(x)[(dx) ~ (dx)] :'2C Lip (R d ) s.t.j'(x)'(~ x)jjx ~ xj o : (3.77) 60 HereC Lip (R d ) denote the set of uniformly Lipschitz continuous functions' : R d ! R. Moreover, for each (t;)2 [0;T ]P 2 , letL 2 (t;) denote the set ofF t -measurable random variables whose law (under P)L =. We consider coecients (b;f) : [0;T ]R d P 2 A! (R d ;R) andg :R d P 2 !R. Throughout this section, the following assumptions will always be in force. Assumption3.6.1 (i)b;f;g are Borel measurable int and bounded byC 0 (for simplicity); (ii)b;f;gareuniformlyLipschitzcontinuousin (x;;a)withaLipschitzconstantL 0 ,wheretheLipschitz continuity in is underW 1 . LetA cont denote the set of admissible controls : [0;T ]R d ! A which is measurable int and Lipschitz continuous inx, with the Lipschitz constantL possibly depending on. Given (t;)2 [0;T ] P 2 ,2L 2 (t;), and2A cont , consider the McKean-Vlasov SDE: X t;; s = + Z s t b(r;X t;; r ; r ;(r;X t;; r ))dr +B s B t ; s :=L X t;; s : (3.78) By the required Lipschitz continuity, the above SDE is wellposed, and it is obvious that t = and s does not depend on the choice of2L 2 (t;). Then, when only the law is involved, by abusing the notations we may also denoteX t;; asX t;; . Next, for anyx2R d , and ~ 2A cont , we introduce J(t;;;x; ~ ) :=J( ;t;x; ~ ); v( ;s;x) := inf ~ 2Acont J( ;s;x; ~ );st; where X ;s;x;~ r =x + Z r s b(l;X ;s;x;~ l ; l ; ~ (l;X ;s;x;~ l ))dl +B r B s ; rs; J( ;s;x; ~ ) :=E h g(X ;s;x;~ T ; T ) + Z T s f(r;X ;s;x;~ r ; r ; ~ (r;X ;s;x;~ r ))dr i : (3.79) 61 Here we abuse the notations by using the same notations as in the discrete setting. Clearlyu(s;x) := J( ;s;x; ~ ) andv(s;x) := v( ;s;x) satisfy the following linear PDE and standard HJB equation on [t;T ]R d , respectively, with parameter : @ s u(s;x) + 1 2 tr @ xx u(s;x) +b(s;x; s ; ~ (s;x))@ x u(s;x) +f(s;x; s ; ~ (s;x)) = 0; @ t v(s;x) + 1 2 tr @ xx v(s;x) + inf a2A b(s;x; s ;a)@ x v(s;x) +f(s;x; s ;a) = 0; u(T;x) =v(T;x) =g(x; T ): (3.80) Denition3.6.2 Fix (t;)2 [0;T ]P 2 . For any"> 0, we say 2A cont is an"-MFE at (t;), denoted as 2M " cont (t;), if Z R d J(t;; ;x; )v( ;t;x) (dx)": (3.81) Remark3.6.3 Similarto (3.55)and (3.56),herewedonotrequire tobeoptimalforeveryplayerx. Infact, alternatively, we may replace (3.81) with n x : jJ(t;; ;x; )v( ;t;x)j>" o <": (3.82) The intuition is that, since there are innitely many players, we shall tolerate that a small portion of players may not be happy for the , as in [14], and their possible deviation from won’t change the equilibrium measure signicantly. We note that, although (3.82) and (3.81) are not equivalent for xed", they dene the same set value in (3.84) below, and the proofs are slightly easier by using (3.81). However, if we require the "-optimality for -a.e. x, namely the probability in the left side of (3.82) becomes 0, then the set value will be dierent and may not satisfy the DPP. Such dierence would disappear in the discrete model though. 62 To dene the set value, we need the following simple but crucial regularity result, whose proof is postponed to Appendix. Lemma3.6.4 Let Assumption 3.6.1 hold. There exists a constantC > 0, depending only onT;d;C 0 ;L 0 , such that, for anyt;;; ~ andst, J( ; ~ ;s;x)J( ; ~ ;s; ~ x) + v( ;s;x)v( ;s; ~ x) Cjx ~ xj; 8x; ~ x: (3.83) We then dene the set value of the mean eld game: V cont (t;) := \ ">0 V " cont (t;); where V " cont (t;) := n '2C Lip (R d ) : there exists 2M " cont (t;) such that Z R d '(x)J(t;; ;x; ) (dx)" o : (3.84) In particular, sinceJ(t;; ;x; )v( ;t;x), then by (3.83) and (3.81) we see that bothJ(t;; ;; ) andv( ;t;) belong toV cont (t;). Moreover, again due to (3.81), we may replace the inequality in the last line of (3.84) with R R d '(x)v( ;t;x) (dx)". Similarly, givenT 0 and 2C Lip (R d ), we may dene the functionsJ(T 0 ; ;t;;;x; ~ ),J(T 0 ; ; ;s;x; ~ ), v(T 0 ; ; ;s;x), as well as the setsM " cont (T 0 ; ;t;),V " cont (T 0 ; ;t;),V cont (T 0 ; ;t;) in the obvious sense. In particular, we have the following tower property: J(t;;;x; ~ ) =J(T 0 ; ;t;;;x; ~ ); where (x) :=J(T 0 ; T 0 ;;x; ~ ); v( ;t;x) =v(T 0 ; ~ ; ;t;x); where ~ (x) :=v( ;T 0 ;x): (3.85) We now establish the DPP forV cont (t;). 63 Theorem3.6.5 Let Assumption 3.6.1 hold. For any 0tT 0 T and2P 2 , it holds V cont (t;) = ~ V cont (t;) := \ ">0 ~ V " cont (t;); where ~ V " cont (t;) := n '2C Lip (R d ) : Z R d j'(x)J(T 0 ; ;t;; ;x; )j(dx)"; for some ( ; ) satisfying: 2V " cont (T 0 ; T 0 ); 2M " cont (T 0 ; ;t;) o : (3.86) Proof (i) We rst proveV cont (t;) ~ V cont (t;). Fix'2 V cont (t;)," > 0, and set" 1 := " 2 . Since '2V " 1 cont (t;), there exists 2M " 1 cont (t;) satisfying (3.84) for" 1 . Denote (x) :=J(T 0 ; T 0 ; ;x; ); ~ (x) :=v( ;T 0 ;x): By (3.85) we haveJ(T 0 ; ;t;; ;x; ) =J(t;; ;x; ) and thus Z R d '(x)J(T 0 ; ;t;; ;x; ) (dx)" 1 ": We shall show that 2V " cont (T 0 ; T 0 ) and 2M " cont (T 0 ; ;t;). Then'2 ~ V " cont (t;), and therefore, since"> 0 is arbitrary, we have'2 ~ V(t;). Step 1. In this step we show that Z R d J(T 0 ; T 0 ; ;x; )v( ;T 0 ;x) T 0 (dx) = Z R d [ (x) ~ (x)] T 0 (dx)" 1 : (3.87) Then 2M " cont (T 0 ; T 0 ), which, together with the regularity of from Lemma 3.6.4, implies immedi- ately that 2V " cont (T 0 ; T 0 ). 64 To see this, we recall (3.78) with2L 2 (t;). Since 2M " 1 cont (t;), by (3.85) we have " 1 E h J(t;; ;; )v( ;t;) i =E h J(T 0 ; ;t;; ;; )v(T 0 ; ~ ; ;t;) i E h J(T 0 ; ;t;; ;; )J(T 0 ; ~ ;t;; ;; ) i =E h (X t;; T 0 ) ~ (X t;; T 0 ) i : Note thatL X t;; T 0 = T 0 , then this is exactly (3.87). Step 2. It remains to show that 2M " cont (T 0 ; ;t;). By the denition ofv and its regularity from Lemma 3.6.4, there exists ~ 2A cont such that J(T 0 ; ;t;; ;x; ~ )v(T 0 ; ; ;t;x) +" 1 ; 8x2R d : Then, denoting ^ := ~ T 0 2A cont , by (3.85) again we have E h J(T 0 ; ;t;; ;; )v(T 0 ; ; ;t;) i E h J(T 0 ; ;t;; ;; )J(T 0 ; ;t;; ;; ~ ) i +" 1 =E h J(t;; ;; )J(t;; ;; ^ ) i +" 1 E h J(t;; ;; )v( ;t;) i +" 1 " 1 +" 1 ="; This means 2M " cont (T 0 ; ;t;). (ii) We next prove ~ V cont (t;) V cont (t;). Fix '2 ~ V cont (t;), " > 0, and set " 1 := " 4 . Since '2 ~ V " 1 cont (t;), there exist ( ; ) satisfying the desired properties in (3.86) for" 1 . In particular, since 2 V " 1 cont (T 0 ; T 0 ), there exists desired ~ 2M " 1 cont (T 0 ; T 0 ) required in (3.84) for" 1 . Denote ^ := T 0 ~ 2A cont and ^ (x) :=J(T 0 ; T 0 ; ~ ;x; ~ ); ~ (x) :=v( ^ ;T 0 ;x): 65 By (3.86), E h J(T 0 ; ;t;; ;; )J(T 0 ; ^ ;t;; ;; ) i (3.88) =E h (X ;t;; T 0 ) ^ (X ;t;; T 0 ) i = Z R d (x)J(T 0 ; T 0 ; ~ ;x; ~ ) T 0 (dx)" 1 Then, since'2 ~ V " 1 cont (t;) with corresponding ( ; ), by (3.85) and (3.88) we have E h '()J(t;; ^ ;; ^ ) i E h '()J(T 0 ; ;t;; ;; ) i +" 1 2" 1 "; where 2 L 2 (t;). We claim further that ^ 2 M " cont (t;). Then ' 2 V " cont (t;), and thus ' 2 V cont (t;), since"> 0 is arbitrary. To see the claim, since 2M " 1 cont (T 0 ; ;t;), ~ 2M " 1 cont (T 0 ; T 0 ), by (3.85) we have E h J(t;; ^ ;; ^ )v( ^ ;t;) i =E h J(T 0 ; ^ ;t;; ;; )v(T 0 ; ~ ; ;t;) i E h J(T 0 ; ;t;; ;; )v(T 0 ; ~ ; ;t;) i +" 1 E h v(T 0 ; ; ;t;)v(T 0 ; ~ ; ;t;) i + 2" 1 sup ~ 2Acont E h J(T 0 ; ;t;; ;; ~ )J(T 0 ; ~ ;t;; ;; ~ ) i + 2" 1 =E (X t;; T 0 ) ~ (X t;; T 0 ) + 2" 1 E ^ (X t;; T 0 ) ~ (X t;; T 0 ) + 3" 1 " 1 + 3" 1 =": This means ^ 2M " cont (t;), and hence completes the proof. Remark3.6.6 (i) Our set valueV cont (t;) is dened for each (t;) with elements inC Lip (R d ), instead of V(t;x;)Rforeach (t;x;). Thisisconsistentwith (3.9)inthediscretemodel,andisduetothefactthat 66 an"-MFE in Denition 3.6.2 depends on (t;), but is common for all initial statesx. Indeed, if we dene V cont (t;x;) in an obvious manner, it will not satisfy the DPP. (ii) The above observation is also consistent with the fact that the following master equation is local in (t;), but non-local inx due to the term@ x V (t; ~ x;): @ t V (t;x;) + 1 2 tr(@ xx V ) +H(x;;@ x V ) + Z R d 1 2 tr(@ ~ x V (t;x;; ~ x)) +@ p H(~ x;;@ x V (t; ~ x;))@ V (t;x;; ~ x) (d~ x) = 0: (3.89) Underappropriateconditions,inparticularundercertainmonotonicityconditions,theabovemasterequation has a unique solution and we haveV cont (t;) =fV(t;)g is a singleton, whereV(t;)(x) :=V (t;x;) is afunctionofx. Inthisway,wemayalsoview (3.89)asarstorderODEonthespaceC 2 (R d )(theregularity inx is a lot easier to obtain): @ t V(t;) +H(;V(t;)) +M(;V(t;);@ V(t;)) = 0; where H(;v())(x) := 1 2 tr(@ xx v(x)) +H(x;;@ x v(x)); M(;v(); ~ v(;))(x) := Z R d 1 2 tr(@ ~ x ~ v(x; ~ x)) +@ p H(~ x;;@ x v(~ x))~ v(x; ~ x) (d~ x): (3.90) It could be interesting to explore master equations from this perspective as well. 3.6.2 ConvergenceoftheN-playergame By enlarging the ltered probability space ( ;F;F;P), if necessary, we letB 1 ; ;B N be independent d-dimensional Brownian motions on it. SetA 1 cont :=[ L0 A L cont , where, for eachL 0,A L cont denotes the set of admissible controls : [0;T ]R d P 2 !A such that j(t;x;)(t; ~ x; ~ )jL jx ~ xj +LW 1 (; ~ ): 67 Here the Lipschitz constantL may depend on, hence the Lipschitz continuity inx is not uniform in . We emphasize that the Lipschitz continuity in is underW 1 , rather thanW 2 , so that we can use the representation (3.77). Note thatA cont =A 0 cont , and by Remark 3.3.1 (i), all the results in the previous subsection remain true if we replaceA cont withA 1 cont . Givent2 [0;T ],~ x = (x 1 ; ;x N )2R dN and~ = ( 1 ; ; N )2 (A L cont ) N , consider X t;~ x;~ ;i s =x i + Z s t b r;X t;~ x;~ ;i r ; t;~ x;~ r ; i (r;X t;~ x;~ ;i r ; t;~ x;~ r ) dr +B i s B i t ;i = 1; ;N; where t;~ x;~ s := 1 N N X i=1 X t;~ x;~ ;i s ; J i (t;~ x;~ ) :=E h g(X t;~ x;~ ;i T ; t;~ x;~ T ) + Z T t f s;X t;~ x;~ ;i s ; t;~ x;~ s ; i (s;X t;~ x;~ ;i s ; t;~ x;~ s ) ds i ; v N;L i (t;~ x;~ ) := inf ~ 2A L cont J i (t;~ x; (~ i ; ~ )): (3.91) In light of Lemma 3.6.4, the following regularity result is interesting in its own right. However, since it will not be used for our main result, we postpone its proof to Appendix. Proposition3.6.7 Let Assumption 3.6.1 hold. For anyL 0, there exists a constantC L > 0, depending onlyonT;d;C 0 ;L 0 ,andL,suchthat,forany (t;~ x)2 [0;T ]R dN , x; ~ x2R d ,and~ 2 (A L cont ) N ,wehave v N;L i t; (~ x i ; x);~ v N;L i t; (~ x i ; ~ x);~ C L j x ~ xj; i = 1; ;N: (3.92) Given2A L cont , by viewing it as the homogeneous control (; ;), we may use the simplied notationsX t;~ x;;i , t;~ x; ,J i (t;~ x;), andv N;L i (t;~ x;) in the obvious sense. Denition3.6.8 (i) For (t;~ x)2 [0;T ]R dN ," > 0,L 0, we call 2A L cont a homogeneous (";L)- equilibrium of theN-player game at (t;~ x), denoted as 2M N;";L cont (t;~ x), if 1 N N X i=1 J i (t;~ x; )v N;L i (t;~ x; ) ": (3.93) 68 (ii) The set value for theN-player game is dened as: V N cont (t;~ x) := \ ">0 V N;" cont (t;~ x) := \ ">0 [ L0 V N;";L cont (t;~ x); where (3.94) V N;";L cont (t;~ x) := n '2C Lip (R d ) :9 2M N;";L cont (t;~ x) s.t. 1 N N X i=1 j'(x i )J i (t;~ x; )j" o : We remark that, althoughV N;";L cont (t;~ x) involves only the valuesf'(x i )g 1iN , for the convenience of the convergence analysis we consider its elements as'2C Lip (R d ). Remark3.6.9 (i)Recall (3.20). Bytherequiredsymmetry,obviouslythereexistfunctionsJ N ;v N;L : [0;T ] P 2 A L cont R d !R such that J i (t;~ x;) =J N (t; N ~ x ;;x i ); v N;L i (t;~ x;) =v N;L (t; N ~ x ;;x i ); i = 1; ;N: (3.95) Moreover,V N cont (t;~ x) is invariant in N ~ x and thus can be denoted asV N cont (t; N ~ x ). (ii) The required inequalities in Denition 3.6.8 are equivalent to: Z R d [J N v N;L ](t; N ~ x ; ;x) N ~ x (dx)"; Z R d '(x)J N (t; N ~ x ; ;x) N ~ x (dx)": We now turn to the convergence, starting with the convergence of the equilibrium measures. Recall the vector (; ~ ) i introduced in (3.25). Theorem3.6.10 Let Assumption 3.6.1 hold. For anyL 0, there exists a constantC L > 0, depending only onT;d;C 0 ;L 0 , andL, such that, for anyt2 [0;T ],~ x2R dN ,2P 2 ,; ~ 2A L cont , andi = 1; ;N, sup tsT E h W 1 ( t;~ x;(;~ ) i s ; s ) i C L N ; (3.96) where N :=W 1 ( N ~ x ;) +N 1 d_3 k~ xk 2 +N 1 ; k~ xk 2 2 := 1 N N X i=1 jx i j 2 : 69 Proof Recall (3.91) and introduce, forj = 1; ;N, ~ X j s =x j + Z s t b(r; ~ X j r ; r ;(r; ~ X j r ; r ))dr +B j s B j t ; ~ N s := 1 N N X j=1 ~ X j s ; ~ X s = ~ + Z s t b(r; ~ X r ; r ;(r; ~ X r ; r ))dr +B s B t ; where ~ 2L 2 (F 0 ; N ~ x ): (3.97) Note that ~ X 1 ; ; ~ X N are independent. We proceed the rest of the proof in two steps. Step 1. In this step we estimateE W 1 (~ N s ; s ) . First, by [53, Lemma 8.4] we have E W 1 (~ N s ;L ~ Xs ) CN 1 d_3 k~ xk 2 : Next, x an' in (3.77) and letu =u ' denote the solution to the following PDE on [t;s]: @ r u + 1 2 tr @ xx u +b(r;x; s ;(r;x; r ))@ x u = 0; u(s;x) ='(x): (3.98) Applying Lemma 3.6.4 with ~ (r;x) :=(r;x; r ) andf = 0, we see thatu is uniformly Lipschitz contin- uous inx, with a Lipschitz constantC independent of' andL. Thus, E '( ~ X s )'(X s ) =E u(t; ~ )u(t;) CE[j ~ j]: SinceF 0 is atomless, we may choose; ~ such thatE[j ~ j] =W 1 ( N ~ x ;), then (3.77) impliesW 1 (L ~ Xs ; s ) CW 1 ( N ~ x ;): Put together, we have E W 1 (~ N s ; s ) CW 1 ( N ~ x ;) +CN 1 d_3 k~ xk 2 C N ; tsT: (3.99) 70 Step 2. We next estimateE W 1 ( t;~ x;(;~ ) i s ; s ) . Denote i := ~ , j := forj6=i, and j s :=b(s; ~ X j s ; ~ N s ; j (s; ~ X j s ; ~ N s ))b(s; ~ X j s ; s ;(s; ~ X j s ; s )); 1jN M s := Q N j=1 M j s ; M j s := exp R s t j r dB j r 1 2 R s t j j r j 2 dr : Then, by the Girsanov theorem we have E W 1 ( t;~ x;(;~ ) i s ; s ) = E M s W 1 (~ N s ; s ) =E [M s 1]W 1 (~ N s ; s ) +E W 1 (~ N s ; s ) = N X j=1 E h Z s t M r j r dB j r W 1 (~ N s ; s ) i +E W 1 (~ N s ; s ) : (3.100) By the martingale representation theorem, we have W 1 (~ N s ; s ) =E W 1 (~ N s ; s ) + N X j=1 Z s t Z j r dB j r : (3.101) Note that ~ X j are independent. Consider the following linear PDE on [t;s]R dN : @ r u(r;~ x 0 ) + 1 2 N X j=1 tr @ x j x j u(r;~ x 0 ) + N X j=1 b(r;x 0 j ; s ;(r;x 0 j ; r ))@ x j u(r;~ x 0 ) = 0; u(s;~ x 0 ) =W 1 ( N ~ x 0; s ): (3.102) By standard BSDE theory, see e.g. [63, Chapter 5], we haveZ j r = @ x j u(r; ~ X t;~ x r ), whereX t;~ x;j r := x j + B j r B j t . Note that the terminal conditionu(s;~ x 0 ) is Lipschitz continuous inx 0 j with Lipschitz constant 1 N . Then, similarly to (3.98), by Lemma 3.6.4 we see thatjZ j jj@ x j uj C N for some constantC independent of andL. Thus, by (3.100) and (3.101), E h W 1 ( t;~ x;(;~ ) i s ; s )W 1 (~ N s ; s ) i = N X j=1 E h Z s t M r j r Z j r dr i C N N X j=1 E h Z s t M r j j r jdr i : 71 Note thatj i jC and, forj6=i,j j r jC L W 1 (~ N r ; r ). Then, by (3.99), E h W 1 ( t;~ x;(;~ ) i s ; s ) i E W 1 (~ N s ; s ) + C N E h Z s t M r j i r jdr + X j6=i Z s t M r j j r jdr i E W 1 (~ N s ; s ) + C N + C L N X j6=i E h Z s t M r W 1 (~ N r ; r )dr i = C N +C L N C L N ; proving (3.96). Theorem3.6.11 For the setting in Theorem 3.6.10, we have J i (t;~ x; (; ~ ) i )J(t;;;x i ; ~ ) + v N;L i (t;~ x;)v( ;t;x i ) C L 1 4 N : (3.103) Proof Fixi. First, by taking supremum over ~ 2A L cont , the uniform estimate forJ implies that forv immediately. So it suces to prove the former estimate. For this purpose, recall (3.91) and denote ~ J i (t;~ x; (; ~ ) i ) :=E P h g(X t;~ x;(;~ ) i ;i T ; T ) + Z T t f(s;X t;~ x;(;~ ) i ;i s ; s ; ~ (s;X t;~ x;(;~ ) i ;i s ; s ))ds i : Then one can easily see that, by applying Theorem 3.6.10, J i (t;~ x; (; ~ ) i ) ~ J i (t;~ x; (; ~ ) i ) C L sup tsT E W 1 ( t;~ x;(;~ ) i s ; s ) C L N : (3.104) Next, denote X i s :=x i +B i s B i t ; ~ N;i s := 1 N h P j6=i X t;~ x;(;~ ) i ;j s + X i s i ; s :=b(s;X i s ; s ; ~ (s;X i s ; s )); M s := exp R s t r dB i r 1 2 R s t j r j 2 dr ; ~ s :=b(s;X i s ; ~ N;i s ; ~ (s;X i s ; ~ N;i s )); ~ M s := exp R s t ~ r dB i r 1 2 R s t j ~ r j 2 dr : 72 By (3.79) and (3.91), it follows from the Girsanov theorem again that ~ J i (t;~ x; (; ~ ) i )J(t;;;x i ; ~ ) = E h ~ M T M T g(X i T ; T ) + Z T t f(s;X i s ; s ; ~ (s;X i s ; s )ds i CE j ~ M T M T j : (3.105) Denote M s := ~ M s M s , s := ~ s s . Then, sinceb is bounded, E[jM s j 2 ] =E h Z s t [ ~ M r ~ r M r r ]dB i r 2 i =E h Z s t [ ~ M r ~ r M r r ] 2 dr i C Z s t E[jM r j 2 ]dr +CE h Z s t j ~ M r j 2 j r j 2 dr i C Z s t E[jM r j 2 ]dr +CE h Z s t ~ M 3 2 r ~ M 1 2 r j r j 1 2 dr i C Z s t E[jM r j 2 ]dr +C E h Z s t ~ M r j r jdr i 1 2 C Z s t E[jM r j 2 ]dr +C L E h Z s t ~ M r W 1 (~ N;i r ; r )dr i 1 2 =C Z s t E[jM r j 2 ]dr +C L E h Z s t W 1 ( t;~ x;(;~ ) i r ; r )dr i 1 2 C Z s t E[jM r j 2 ]dr +C L 1 2 N ; where the last inequality thanks to Theorem 3.6.10. Then, by the Grownwall inequality we obtainE[jM s j 2 ] C L 1 2 N ; and thus (3.105) implies ~ J i (t;~ x; (; ~ ) i )J(t;;;x i ; ~ ) C L 1 4 N : This, together with (3.104), implies the estimate forJ in (3.103) immediately. 73 Theorem3.6.12 Let Assumption 3.6.1 hold. Assume further that lim N!1 W 1 ( N ~ x ;) = 0, and there exists a constantC > 0 such that ¶ k~ xk 2 C for allN. Then \ ">0 [ L0 lim N!1 V N;";L cont (t; N ~ x )V cont (t;) \ ">0 lim N!1 V N;";0 cont (t; N ~ x ) (3.106) In particular, since lim N!1 V N;";0 cont (t; N ~ x ) [ L0 lim N!1 V N;";L cont (t; N ~ x ), actually equalities hold. Proof (i) We rst prove the right inclusion in (3.106). Fix'2V cont (t;)," > 0, and set" 1 := " 2 . By (3.84) and (3.81), there exists 2M " 1 cont (t;) such that Z R d J(t;; ;x; )v( ;t;x) (dx)" 1 ; Z R d '(x)J(t;; ;x; ) (dx)" 1 : Recall Lemma 3.6.4 and note that'2C Lip (R d ), then by (3.77) we have Z R d J(t;; ;x; )v( ;t;x) N ~ x (dx)" 1 +CW 1 ( N ~ x ;); Z R d '(x)J(t;; ;x; ) N ~ x (dx)" 1 +C ' W 1 ( N ~ x ;); whereC ' may depend on the Lipschitz constant of'. Moreover, by (3.103) we have 1 N N X i=1 J i (t;~ x; )v N;L i (t;~ x; ) 1 N N X i=1 J(t;; ;x i ; )v( ;t;x i ) +C L 1 4 N = Z R d J(t;; ;x; )v( ;t;x) N ~ x (dx) +C L 1 4 N " 1 +C L 1 4 N ; 1 N N X i=1 j'(x i )J i (t;~ x; )j 1 N N X i=1 j'(x i )J(t;; ;x i ; )j +C L 1 4 N = Z R d '(x)J(t;; ;x; ) N ~ x (dx) +C L 1 4 N " 1 +C L;' 1 4 N : ¶ Note again that~ x depends onN. Also, the conditions here are slightly weaker than lim N!1 W2( N ~ x ;) = 0. 74 We emphasize again thatk~ xk 2 C is independent ofN. Then, by choosingN large enough such that C L 1 4 N " 1 ,C L;' 1 4 N " 1 , we obtain 1 N N X i=1 J i (t;~ x; )v N;L i (t;~ x; ) "; 1 N N X i=1 j'(x i )J i (t;~ x; )j": This implies that 2 M N;";0 cont (t;~ x) and ' 2 V N;";0 cont (t; N ~ x ), for all N large enough. That is, ' 2 lim N!1 V N;";0 cont (t;~ x) for any"> 0. (ii) We next show the left inclusion in (3.106). Fix'2 \ ">0 [ L0 lim N!1 V N;";L cont (t; N ~ x ), " > 0, and set " 1 := " 2 . There existL " 0 and an innite sequencefN k g k1 such that'2 V N k ;" 1 ;L" cont (t; N ~ x ) for all k 1. Recall (3.93) and (3.94), there exists k 2A L" cont such that 1 N k N k X i=1 J i (t;~ x; k )v N k ;L" i (t;~ x; k ) " 1 ; 1 N k N k X i=1 j'(x i )J i (t;~ x; k )j" 1 : Note thatL " is xed, in particular it is independent ofk. In light of Remark 3.3.1 (i) and denote ~ k (s;x) := k (s;x; k ), then ~ k = k . Similarly to (i), by (3.103) we have Z R d J(t;; ~ k ;x; ~ k )v( k ;t;x) N k ~ x (dx)" 1 +C L" 1 4 N k ; Z R d '(x)J(t;; ~ k ;x; ~ k ) N ~ x (dx)" 1 +C L" 1 4 N k : Then, by Lemma 3.6.4 and (3.77) we have Z R d J(t;; ~ k ;x; ~ k )v( k ;t;x) (dx)" 1 +C L" 1 4 N k +CW 1 ( N k ~ x ;); Z R d '(x)J(t;; ~ k ;x; ~ k ) (dx)" 1 +C L" 1 4 N k +C ' W 1 ( N k ~ x ;): 75 Now choosek large enough (possibly depending on" and') such that C L" 1 4 N k +CW 1 ( N k ~ x ;)" 1 ; C L" 1 4 N k +C ' W 1 ( N k ~ x ;)" 1 : Then we have Z R d J(t;; ~ k ;x; ~ k )v( k ;t;x) (dx)"; Z R d '(x)J(t;; ~ k ;x; ~ k ) (dx)": This implies that ~ k 2 M " cont (t;) and ' 2 V " cont (t;). Since " > 0 is arbitrary, we obtain ' 2 V cont (t;). 3.7 Appendix 3.7.1 ThesubtlepathdependenceissueinRemark3.4.3 In this subsection we elaborate Remark 3.4.3 (ii) and (iii). Throughout the subsection, q;F;G are state dependent as in Section 3.2. For simplicity, we compare state dependent controls and path dependent controls only for the raw set values. We sett = 0 hence2P 0 (X 0 ) =P 0 (S). We rst provide a counterexample to show that the raw set valueV 0 (0;) is in general not equal to the corresponding raw set valueV 0;path (0;) with controls2A path . Example3.7.1 SetT = 2,S =fx;xg,A = [a 0 ; 1a 0 ] for some 00 ~ V " state (t;) denote the right side of (3.19) in the obvious sense. We shall follow the arguments in Theorem 3.2.4. (i) We rst prove ~ V state (t;)V state (t;). Fix'2 ~ V state (t;)," > 0, and set" 1 := " 4 . Since'2 ~ V " 1 state (t;), there exist desirable and 2M " 1 state (T 0 ; ;t;) as in (3.19), and the property (; T 0 )2 V " 1 state (T 0 ; T 0 ) implies further that there exists ~ 2M " 1 state (T 0 ; T 0 ) such that k'J(T 0 ; ;t;; ;; )k 1 " 1 ; k (; T 0 )J(T 0 ; T 0 ; ~ ;; ~ )k 1 " 1 : ∥ While it is trivial thatA state relax A path relax :=A relax , as stated here, in general it is not trivial thatM state relax M path relax , because for the latter one has to compare with other path dependent relax controls, which is a stronger requirement than that forM state relax . The rest of the proof is exactly to proveM state relax M path relax . 81 Denote ^ := T 0 ~ 2A state . Then, for any2A state andx2 S, similar to the arguments in Proposition 3.2.3 (i), we have J(t;; ^ ;x;) =E P ;t;x; h J(T 0 ; T 0 ; ~ ;X T 0 ;) + T 0 1 X s=t F (s;X s ; s ;(s;X s )) i E P ;t;x; h J(T 0 ; T 0 ; ~ ;X T 0 ; ~ ) + T 0 1 X s=t F (s;X s ; s ;(s;X s )) i " 1 E P ;t;x; h (X T 0 ; T 0 ) + T 0 1 X s=t F (s;X s ; s ;(s;X s )) i 2" 1 =J(T 0 ; ;t;; ;x;) 2" 1 J(T 0 ; ;t;; ;x; ) 3" 1 =E P ;t;x; h (X T 0 ; T 0 ) + T 0 1 X s=t F (s;X s ; s ; (s;X s )) i 3" 1 E P ;t;x; h J(T 0 ; T 0 ; ~ ;X T 0 ; ~ ) + T 0 1 X s=t F (s;X s ; s ; (s;X s )) i 4" 1 =J(t;; ^ ;x; ^ )": That is, ^ 2M " state (t;). Moreover, note that k'J(t;; ^ ;; ^ )k 1 " 1 +kJ(T 0 ; ;t;; ;; )J(t;; ^ ;; ^ )k 1 =" 1 + sup x2S E P ;t;x; h (X T 0 ; T 0 )J(T 0 ; T 0 ; ~ ;X T 0 ; ~ ) i 2" 1 ": Then'2V " state (t;). Since"> 0 is arbitrary, we obtain'2V state (t;). (ii) We now prove the opposite inclusion. Fix '2 V state (t;) and " > 0. Let " 1 > 0 be a small number which will be specied later. Since '2 V " 1 state (t;), there exists 2M " 1 state (t;) such that k'J(t;; ;; )k 1 " 1 . Introduce (x;) :=J(T 0 ;; ;x; ). By (3.12) we have k'J(T 0 ; ;t;; ;; )k 1 =k'J(t;; ;; )k 1 " 1 : 82 Moreover, since 2M " 1 state (t;), for any2A state andx2S, we have J(T 0 ; ;t;; ;x; ) =J(t;; ;x; ) J(t;; ;x; T 0 ) +" 1 =J(T; ;t;; ;x;) +" 1 : This implies that 2M " 1 state (T 0 ; ;t;). We claim further that (; T 0 )2V C" 1 (T 0 ; T 0 ); (3.111) for some constantC 1. Then by (3.19) we see that'2 ~ V C" 1 state (t;) ~ V " state (t;) by setting" 1 " C . Since"> 0 is arbitrary, we obtain'2 ~ V state (t;). To show (3.111), we follow the arguments in Proposition 3.2.3 (ii). Recallv in (3.7) and the standard DPP (3.13) forv, for anyx2S we have E P ;t;x; h J(T 0 ; T 0 ; ;X T 0 ; ) i inf 2Astate E P ;t;x; h J(T 0 ; T 0 ; ;X T 0 ;) i +" 1 = E P ;t;x; h v( ;T 0 ;X T 0 ) i +" 1 ; It is obvious thatv( ;T 0 ;) J(T 0 ; T 0 ; ;; ). Moreover, sinceq c q , clearlyP ;t;x; (X T 0 = ~ x)c T 0 t 0 , for any ~ x2S. Thus, forC :=c tT 0 0 , 0 J(T 0 ; T 0 ; ; ~ x; )v( ;T 0 ; ~ x) CE P ;t;x; h J(T 0 ; T 0 ; ;X T 0 ; )v( ;T 0 ;X T 0 ) 1 fX T 0 =~ xg i CE P ;t;x; h J(T 0 ; T 0 ; ;X T 0 ; )v( ;T 0 ;X T 0 ) i C" 1 : 83 This implies that 2M C" 1 state (T 0 ; T 0 ). Since (; T 0 ) =J(T 0 ; T 0 ; ;; ), we obtain (3.111) imme- diately, and hence'2 ~ V state (t;). ProofoftheclaiminRemark3.4.7. By (3.48) and (3.49) we have ( ) (s; ~ x;da) := 1 s^ (~ x) Z A t path Q t s ( ; ~ x;) (s;~ x) (da) (x;d) = 1 s^ (~ x) Z A Z A s1 Y r=t q(r; ~ x; ;(r; ~ x);x r+1 ) (s;~ x) (da) (x) T1 Y r=t Y x2X t;x s (r; x;d(r; x)) = (x) s^ (~ x) Z A Z A s1 Y r=t q(r; ~ x; ;(r; ~ x);x r+1 ) (r; ~ x;d(r; ~ x)) (s;~ x) (da) (s; ~ x;d(s; ~ x)) Y x2X t;x s nf~ xg (s; x;d(s; x)) s1 Y r=t Y x2X t;x s nf~ xg (r; x;d(r; x)) T1 Y r=s Y x2X t;x s (r; x;d(r; x)) = (x) s^ (~ x) s1 Y r=t Z A q(r; ~ x; ; a;x r+1 ) (r; ~ x;d a) (s; ~ x;da) = (x) s^ (~ x) Q t s ( ; ~ x; ) (s; ~ x;da) = (s; ~ x;da): That is, ( ) = . ProofofLemma3.6.4. Clearly the uniform estimate forJ( ;) implies that forv( ;), so we shall only prove the former one. Fix (t;)2 [0;T ]P 2 and; ~ 2A cont , and denoteu(s;x) :=J( ; ~ ;s;x). By standard PDE theoryu is a classical solution to the linear PDE in (5.1) and we have the following formula: denotingX s;x r :=x +B r B s , @ x u(s;x) =E P h [g(X s;x T ; T )g(x; T )] B T B s Ts + Z T s b(r;X s;x t ; r ; ~ (r;X s;x r ))@ x u(r;X s;x r ) +f(r;X s;x t ; r ; ~ (r;X s;x r )) B r B s rs dr i : 84 Then, by the Lipschitz continuity ofg and the boundedness ofb andf, j@ x u(s;x)j E h L 0 jB T B s j 2 Ts +C 0 Z T s j@ x u(r;X s;x r )j + 1 jB r B s j rs dr i C +C 0 E h Z T s j@ x u(r;X s;x r )j jB r B s j rs dr i : DenoteK s :=e s sup x j@ x u(s;x)j, K := sup tsT K s , for some constant> 0. Then K s Ce s +C 0 Z T s K r e (rs) p rs drCe s +C 0 K Z T s e (rs) p rs dr Ce s +C 0 K Z 1 s e (rs) p rs dr =Ce s +C 0 K Z 1 0 e r p r dr =Ce s + C 0 p K: Thus K C 0 p K +Ce T . Set := 4C 2 0 so that C 0 p = 1 2 , we obtain KC 1 := 2Ce T , which implies the desired estimate immediately. ProofofProposition3.6.7. Fix (t;~ x;~ ; x; ~ x) andi. For any ~ 2A L cont , introduce (s;x;) := ~ (s;x x + ~ x;), and denote X i s := x +B i s B i t ; X j s :=x j +B j s B j t ; j6=i; ; N s := 1 N X i s + X j6=i X j s ; M j s := exp Z s t b j r dB j r 1 2 Z s t j b j r j 2 dr ;j 1; where b i s :=b(s; X i s ; N s ; (s; X i s ; N s )); b j s :=b(s;X j s ; N s ; j (s;X j s ; N s ));j6=i: By the Girsanov Theorem we have J i (t; (~ x i ; x); (~ i ; )) =E h N Y j=1 M j T g( X i T ; N T ) + Z T t f(s; X i s ; N s ; (s; X i s ; N s )) ds i : 85 Similarly dene ~ X i , ~ N , ~ M j , ~ b i , ~ b j corresponding to (~ x; ~ ) in the obvious sense. Then we have a similar expression as above and (s; X i s ;) = ~ (s; ~ X i s ;). Therefore, v N;L i t; (~ x i ; x);~ J i (t; (~ x i ; ~ x); (~ i ; ~ )) J i (t; (~ x i ; x); (~ i ; ))J i (t; (~ x i ; ~ x); (~ i ; ~ ))C N X j=1 K j T +K 0 ; (3.112) where K j s := E h Y k<j M k s Y k>j ~ M k s M j s ~ M j s j i ; j 1; K 0 := E h N Y j=1 M j T jg( X i T ; N T )g( ~ X i T ; ~ N T )j + Z T t jf(s; X i s ; N s ; (s; X i s ; N s ))f(s; ~ X i s ; ~ N s ; ~ (s; ~ X i s ; ~ N s ))jds i : Denote x := x ~ x. Note that X i s ~ X i s = x; W 1 ( N s ; ~ N s ) jxj N ; (s; X i s ; N s ) ~ (s; ~ X i s ; ~ N s ) = ~ (s; ~ X i s ; N s ) ~ (s; ~ X i s ; ~ N s ) L N jxj: (3.113) By the required Lipschitz continuity, we have K 0 CE P h N Y j=1 M j T [1 + 1 N ]jxj + Z T t [1 + L N ]jxjds i Cjxj: (3.114) Next, introduce j s :=E h Y k<j M k s Y k>j ~ M k s M j s j 2 i ; j s :=E h Y k<j M k s Y k>j ~ M k s M j s ~ M j s j 2 i : 86 Note thatB 1 ; ;B N are independent. By applying the Itô formula, we have j s = 1 + Z s t E h Y k<j M k r Y k>j ~ M k r M j r b j r j 2 i dr 1 +C Z s t j r dr; Then j s C. Thus, by applying the Itô formula again we have j s = Z s t E h Y k<j M k r Y k>j ~ M k r M j r b j r ~ M j r ~ b j r ] 2 i dr C Z s t E h Y k<j M k r Y k>j ~ M k r j M j r ~ M j r j + M j r j b j r ~ b j r j 2 i dr C Z s t j r dr +C Z s t E h Y k<j M k r Y k>j ~ M k r M j r j b j r ~ b j r j] 2 i dr: Note that, by (3.113), j b i r ~ b i r j = b(s; X i s ; N s ; ~ (s; ~ X i s ; N s ))b(s; ~ X i s ; ~ N s ; ~ (s; ~ X i s ; ~ N s )) C L jxj j b j r ~ b j r j C L N jxj; j6=i: Then, since j s C, i s C Z s t i r dr +C L jxj 2 ; j s C Z s t j r dr + C L N 2 jxj 2 ; j6=i: and thus i s C L jxj 2 ; K i s jxj 2 + i s 2jxj C L jxj; j s C L N 2 jxj 2 ; K j s jxj 2N + N j s 2jxj C L N jxj; j6=i: (3.115) 87 Then, by (3.112), (3.114) and (3.115) we have v N;L i t; (~ x i ; x);~ J i (t; (~ x i ; ~ x); (~ i ; ~ ))K 0 +CK i s +C X j6=i K j s Cjxj +C L jxj +C L X j6=i jxj N C L jxj: Since ~ 2A L is arbitrary, we obtainv N;L i t; (~ x i ; x);~ v N;L i t; (~ x i ; ~ x);~ C L jxj. Similarly we havev N;L i t; (~ x i ; ~ x);~ v t; (~ x i ; x);~ C L jxj, and hence (3.92). 88 Chapter4 SetValuedCalculus In this chapter we introduce the notion of derivatives for set valued functions. An important denition is the intrinsic derivative of a function taking values on the graph of the set valued function, which will play a key role. After presenting the main framework, we illustrate denitions with some examples. Reader will notice that the rst section is presented in a way to cover the regular PDEs. However, following results will reduce to the case of hypersurfaces. In particular, Itô’s formula and the main theorem in the next chapter will only cover a specic (but important) case. Extension of these results to the general setting in this chapter is crucial to obtain similar results in the game theory. To briey comment, because set values for games are dened over all equilibria, but not as values over all possible controls, typically it does not admit an interior and the boundary is not a hypersurface. 4.1 SetValuedFunctionsandIntrinsicDerivatives We consider the space (R k` ) n with natural numbersk;`;n. Endow this space with the trace norm as if A = (A 1 ; ;A n )2 (R k` ) n andB = (B 1 ; ;B n )2 (R k` ) n dene tr(A;B) := n X i=1 A i :B i 89 where : is the trace operator onR k` . In casek = ` = 1, we identifyR n as a column vector and use for the usual inner product. Furthermore, for any natural numbers k 0 ;` 0 , 2 R `` 0 acts on (R k` ) n componentwise from the right and similarly 2R k 0 k acts componentwise from left. Letfe i g i be the standard basis for real coordinate space with any dimension that is appropriate. We reserve@ for the intrinsic derivative that we will shortly introduce, and ^ @ for derivatives on the standard basis. For a smooth : R d ! R m , ^ @ z (z)2 (R d1 ) m is the Jacobian and ^ @ zz (z)2 (R dd ) m is the Hessian. LetD m 2 denote the set of closedD R m such thatD is composed of C 2 -manifolds admitting dif- ferentiable normal vectors. That is, there exists D 0 ; ;D m such that D = [ m k=0 D k where each D k is either empty or a C 2 -manifold with dimension k. Furthermore, for each nonemptyD k , there exists (n 1 D (y); ;n mk D (y)) : D k ! (R m1 ) mk forming a basis for the normal space at each y, and the exterior derivative@ y n ` D (y) exists for ally2D k and 0`mk. For anyy2D, we denote the tangent space, extended tangent space and the normal space (subspaces ofR m ) as T D (y) :=f 0 (0) :8 smooth :R!D and (0) =yg; N D (y) := (T D (y)) ? T + D (y) :=f 0 (0+) :8 smooth :R + !D and (0) =yg where () ? is the orthogonal complement. Note that T D T + D , and for example if y2 D m1 where D m1 is the topological boundary of the open setD m , thenT + D (y) = T D (y) +fn 1 D (y);8 > 0g is a half space. We sayy2D has dimensionk ifT D (y) has dimensionk. Furthermore, for any ~ y2R m , we denote the projection onto tangent and normal space aty asT D (y) ~ y andN D (y) ~ y respectively ∗ , that is, N D (y) ~ y := mk X `=0 (n ` (y) ~ y)n ` (y); T D (y) ~ y := ~ yN D (y) ~ y: ∗ We will use for the projection in general, however, tangent and normal space plays a central role in this section. 90 We shall equipD m 2 with the distance d(D; ~ D) := max k d(D k ; ~ D k ): whered is the standard Hausdor distance, namely d(D k ; ~ D k ) := inf n "> 0 :D k O " ( ~ D k ) andD k O " ( ~ D k ) o ;O " (D) := n y2R m : inf ~ y2D jy ~ yj" o We now consider mappingsV : R d !D m 2 and introduce notations to decomposeV, together with tangent, extended tangent and normal spaces, V k (x) :=fy2V(x) :y has dimensionkg; 0km T V (x;y) :=T V(x) (y); T + V (x;y) :=T + V(x) (y); N V (x;y) :=N V(x) (y): For anyx2 R d andy2 V k (x), letn V (x;y) = (n 1 V (x;y); ;n mk V (x;y)) denote a basis ofN V (x;y). Later, we will dropV from the subscripts. Let us introduce the graph ofV, G V;k :=f(x;y)jx2R d ;y2V k (x)g; andG V := m [ k=0 G V;k : For any functionf from the graphG V;k to (R R ` ) n for any natural number ;`;n, we sayf is bounded if it is bounded in the trace norm uniformly over all (x;y)2G V;k . Furthermore, we say suchf is Lipschitz inR m , if jf(x;y)f(x; ~ y)jLjy ~ yj;8y; ~ y2V k (x) 91 for someL independent of allx2 R d . Lastly, we sayf is continuous if (x n ;y n )2 G V;k converges to (x;y)2G V;k , thenf(x n ;y n ) converges tof(x;y). Norms are induced by the trace. We sayV : R d !D m is continuous, if it is continuous as a mapping under d. Furthermore, we sayV : R d !D m has globally bounded curvature if @ y n ` (x;y) is bounded in the operator norm (or equivalently in the trace norm) by some constant independent of (x;y)2G V for all 1`mk. Remark4.1.1 Later, we will consider a special case whereV takes values as closures of open sets withC 2 - boundary. That is, V(x) = V m (x)[V m1 (x) whereV m is the interior andV m1 is the C 2 -boundary (hypersurface). In this case,n V will be xed as the outward unit normal vector. Remark4.1.2 ForanyfunctiondenedonG V andLipschitziny,byKirszbraunTheoremforHilbertspaces, there exists a Lipschitz extension toR m . In the proofs, for any such function onG V , we always implicitly consider an arbitrary Lipschitz extension with the same constant without changing the notation. Moreover, if the function is continuous, Lipschitz extension is continuous in the usual sense. We say a continuousV :R d !D m 2 is a dierentiable ow if there exists a mapping x;y = (x;y; ~ x) : G V R d !R m such that, for all (x;y)2G V , [Identies] x;y (x) =y, [Preservesdimension] ify2V k (x), x;y (~ x)2V k (~ x) for all ~ x in a neighborhood ofx, [Dierentiable] x;y (~ x) is dierentiable in ~ x in a neighborhood ofx, [Regular] 0 i (x;y) := ^ @ ~ x i x;y (~ x)j ~ x=x is continuous inx and locally Lipschitz iny for alli = 1;:::;d. Moreover, for any (x;y)2G V;k , there exists a continuousn V (~ x; ~ y) in a neighborhood of (x;y). Given a dierentiable owV :R d !D m 2 , we set the derivative@ x i V :G!R m as @ x i V(x;y) :=N 0 i (x;y) =: [@ x i V 1 @ x i V m ] T 92 for alli2f1;:::;dg and introduce@ x V2R md as @ x V := [@ x 1 V T @ x d V T ] = [@ x V 1 @ x V m ] T Lemma4.1.3 @ x V(x;y) is independent of the choice of the mapping for all (x;y)2G V . Proof of lemma 4.1.3 is in the appendix. Remark4.1.4 One can provide an equivalent denition of a dierentiable ow in terms of local representa- tion, see the appendix of this chapter for the discussion in that direction. We choose to present this denition which denes the derivatives in a naive way, independent of how the dierentiable structure ofV is encoded (by ). Wewillstudytherelationbetweenderivativesandthelocalrepresentationinthenextsection. Results are written for the case of hypersurfaces, but extension to the general case is immediate. Next, we present the main result to dene intrinsic derivatives. It can be viewed as a fundamental theorem, however, presented only for a local duration. Proof is postponed to the appendix. Lemma4.1.5 LetV :R d !D m 2 be a dierentiable ow. SetV i =V x;y;i as the solution of, V i t =y + Z t 0 @ x i V(x +se i ;V i s )ds (4.1) Ify2V k (x), thenV i t 2V k (x +te i ) fort in a neighborhood of 02R andi2f1;:::;dg. Finally, we introduce the intrinsic derivative @ x f of a function f : G V ! R. This will allow us to further dene second order derivatives of the set valued functionV, as well as the derivative of the normal vectors@ x n(x;y) = (@ x n 1 (x;y); ;@ x n mk (x;y)). LetV be a dierentiable ow and for any f :G V !R dene intrinsic partial derivatives@ x i f :G V !R d as @ x i f(x;y) := ^ @ f(x +e i ;V i )j =0 93 for alli2f1;:::;dg if they exist. For vector valued functions, lift the denition componentwise. Notice that the derivative ofV is the derivative of the identity mappingf(x;y) = y. Now, second order derivatives@ x i x j V :G!R m are @ x i x j V(x;y) :=@ x i (@ x j V(x;y)) for alli;j = 1;:::;d and we introduce@ xx V(x;y)2 (R dd ) m as @ xx V(x;y) = [@ xx V 1 (x;y) @ xx V m (x;y)] T where (@ xx V k (x;y)) i;j =@ x i x j V k for alli;j = 1;:::;d andk = 1;:::;m. Remark4.1.6 (i): In general@ xx V is not symmetric, i.e.@ xx V6=@ xx V T . However, it holds n T @ xx V(x;y) =n T @ xx V T (x;y) for all (x;y)2G V . This will be shown later once the local representation of the surface studied. (ii): Forf :G V !R m with any smooth extension ^ f :R d R m !R m , derivatives are related as @ x i f(x;y) = ^ @ x i ^ f(x;y) + ^ @ y ^ f(x;y)@ x i V(x;y) This follows from the denitions as, @ x i f(x;y) := ^ @ f(x +e i ;V i ) j=0 = ^ @ ^ f(x +e i ;V i ) j=0 = ^ @ x i ^ f(x +e i ;V i ) + ^ @ y ^ f(x +e i ;V i ) ^ @ V i j=0 and the denition ofV i ,@ x i V yields the result. 94 4.1.1 Examples Example4.1.7 Fix anyD2D m 2 together with a smooth functiona :R d !R m . Let V(x) :=a(x) +D :=fa(x) +y : 8y2Dg: Then for any (x;y)2G V , we can set x;y (~ x) :=y +a(~ x)a(x) and therefore @ x V(x;y) =N ^ @ x a(x) In particular, ifD = D 0 =fa(x)g then@ x V(x;a(x)) = ^ @ x a(x). Another trivial case is wheny2 D m , sinceN(x;y) = 0 it follows@ x V(x;y) = 0. In words, interior points do not observe the perturbations to the set. Intuition that the derivative has to be continuous iny throughout the graph holds true under appropriate denition. Let (x;y n )2G k convergesto (x;y)2G ` forsome 1`km. (Itisnotpossibletoconverge to a point with larger tangent space). It holds thatN(x;y n ) (@ x V(x;y n )@ x V(x;y))! 0. To demonstrate the decompositionD =D 0 [D 1 [D 2 , andT + , D =f(y 1 ;y 2 )2R 2 :y 1 (1y 1 )y 2 y 1 (1y 1 )g D 0 =f(0; 0)g[f(1; 0)g; D 1 =f0<y 1 < 1; y 2 =y 1 (1y 1 ) ory 2 =y 1 (1y 1 )g; D 2 =f0<y 1 < 1;y 1 (1y 1 )<y 2 <y 1 (1y 1 )g; T((0; 0)) =f(0; 0)g; T + ((0; 0)) =f(0; 0)g[fy 1 > 0; (y 2 =y 1 ) 1g: 95 Example4.1.8 Take smooth functionsa :R d !R m andR :R d !R + and dene V(x) =B(a(x);R(x)) whereB(a;r) is the closed ball ata with radiusR. For any (x;y)2G V;m1 , set x;y (~ x) =a(~ x) +n(x;y)R(~ x) and@ x V follows similarly as @ x V(x;y) =nn T (x;y) ^ @ x a(x) + ^ @ x R(x)n(x;y) Example4.1.9 Letd = 1. Fix a positivek2N. Set V(x) =V 1 (x) := n [1 +x cos(x 1 +k)](cos; sin) : 82 [0; 2] o R 2 ThenV is a continuous mapping, however, we cannot endow it with any dierentiable structure atx = 0. Moreover,n is not continuous. Proof Note that V(0) is the unit ball. Fix the point (1; 0) and suppose we consider any (x) = 0;(1;0) (x) such that (0) = (1; 0) and (x)2V(x). i.e. there exists ~ :R! [0; 2] where (x) = (1 +x cos(x 1 +k ~ (x)))(cos ~ (x); sin ~ (x)) We argue that cannot be dierentiable. Because the derivative of the rst component contains the term x(x 2 +k@ x ~ (x)) sin(x 1 +k ~ (x)) 96 which blows up to innity. If it is the case thatx 1 +k ~ (x)! 0 mod , then@ x ~ (0) cannot exists and note that the second component has sin( ~ (x))@ x ~ (x). We left to shown is not continuous at (0; (1; 0)) to the reader. 4.1.2 Relationstothelocalrepresentationandthesigneddistance This section describes derivatives of V up to second order in terms of the local representation. From now on, we will be mainly interested in the special case of hypersurfaces, namely V = V m1 [V m whereV o := V m is the interior andV b := V m1 is the topological boundary. Altough we present the local representation for hypersurfaces, extends to the general case. We then study relations to the signed distance function. Fix (x 0 ;y 0 )2G V , together with a local representation x 0 ;y 0 (x;z) = (x;z) :R d R m1 !R such that after appropriate translation and rotation,V b (x) is locally represented by (z; (x;z)) aroundz = 0 where (x; 0) = 0 and ^ @ z (x;z)j z=0 = 0. (See appendix for further discussion). Note that n(x; (z; (x;z))) = 1 (1 +j ^ @ z j 2 ) 1=2 2 6 6 4 ^ @ z (x;z) 1 3 7 7 5 It is straightforward to compute the following relations @ x i V(x; (z; (x;z))) = ^ @ x i (1 +j ^ @ z j 2 ) 1=2 n @ x j x i V(x; (z; (x;z))) = ^ @ x j x i (1 +j ^ @ z j 2 ) 1=2 n 1 (1 +j ^ @ z j 2 ) 3=2 ( ^ @ x j ^ @ zx i T ^ @ z + ^ @ x i ^ @ zx j T ^ @ z )n + ^ @ x j ^ @ x i (1 +j ^ @ z j 2 ) 5=2 ( ^ @ z T ^ @ zz ^ @ z )n + @ x i (1 +j ^ @ z j 2 ) 1=2 @ x j n 97 Note that sincen@ x j n = 0 for allj = 1;:::;d,n T @ x j x i V is symmetric but not@ x j x i V. Since@ xx V2 (R d R d ) m , it is useful to write it as (@ xx V) k = n k (1 +j ^ @ z j 2 ) 1=2 ^ @ xx n k (1 +j ^ @ z j 2 ) 3=2 ( ^ @ zx ^ @ z ^ @ x T + ^ @ x ^ @ z T ^ @ xz ) + n k (1 +j ^ @ z j 2 ) 5=2 ( ^ @ z T ^ @ zz ^ @ z ) ^ @ x ^ @ x T + 1 (1 +j ^ @ z j 2 ) 1=2 @ x n k ^ @ x T We also encounter terms related to local surface moves, which is again an elementary calculation, T @ y n = 1 (1 +j ^ @ z j 2 ) 1=2 ( T jm1 ^ @ zz jm1 ) @ x n T = 1 (1 +j ^ @ z j 2 ) 1=2 ^ @ zx jm1 + 1 (1 +j ^ @ z j 2 ) 3=2 ( ^ @ x ^ @ z T ^ @ zz ) jm1 for any =: h jm1 jm i 2R md 0 withn T = 0. Remark4.1.10 Inthenextsection,wepresentaproofofItô’sformulabyusingthesigneddistance. However, let us note that by above relations it is a straightforward computation (slightly tedious) to show the claim. Let us continue with the signed distance function in terms of the local representation near the bound- ary. Let(x;) be the projection in the tubular neighbourhood ofV b (x) ontoV b (x). Distance function r(x;y) satises y +r(x;y)n(x;(x;y)) =(x;y) (4.2) 98 Fixx 0 2R d andy 0 2R m in a tubular neighbourhood ofV b (x 0 ). Consider the local representation with appropriate translation and rotation such thaty 0jm1 = 0 (equivalently, (x 0 ;y 0 ) = 0). Now, the last component of (4.2) reads r(x;y) = 1 +j ^ @ z (x;(x;y) jm1 )j 2 ) 1=2 ( (x;(x;y) jm1 )y m ) and then we can directly compute ^ @ y r(x 0 ;y 0 ) = [0; ; 0;1] T =n(x 0 ;(x 0 ;y 0 )) ^ @ x i r(x 0 ;y 0 ) = ^ @ x i (x 0 ; 0) =n T @ x i V(x 0 ;(x 0 ;y 0 )) ^ @ x j ^ @ x i r(x 0 ;y 0 ) = ^ @ x j x i + ^ @ x i z ^ @ x j jm1 + ^ @ x j z ^ @ x i jm1 + ^ @ x i T jm1 ^ @ zz ^ @ x j jm1 + (y 0jm )( ^ @ x i z + ^ @ zz ^ @ x i jm1 ; ^ @ x j z + ^ @ zz ^ @ x j jm1 ) ^ @ yy r(x 0 ;y 0 ) = 2 6 6 4 ^ @ zz ^ @ y jm1 0 3 7 7 5 y 0jm h ( ^ @ zz ^ @ y i jm1 ; ^ @ zz ^ @ y j jm1 ) i m;m i;j=1 ^ @ x ^ @ y r(x 0 ;y 0 ) = 2 6 6 4 ^ @ zx + ^ @ zz ^ @ x jm1 0 3 7 7 5 y 0jm h ( ^ @ x i z + ^ @ zz ^ @ x i jm1 ; ^ @ zz ^ @ y j jm1 ) i d;m i;j=1 where [] k;` i;j=1 representk` matrix. By (4.2), we can represent the derivatives of computed at (x 0 ;y 0 ) in a recursive way; ^ @ x i (x 0 ;y 0 ) =@ x i V(x 0 ;(x 0 ;y 0 )) +r(x 0 ;y 0 ) ^ @ x i (n(x;(x;y 0 ))) jx=x 0 99 ^ @ y j i (x 0 ;y 0 ) =1 fi=jg 1 fi=j=mg +r(x 0 ;y 0 ) ^ @ y (n i (x;(x;y))) Now, with all these computations and noting that r(x 0 ;y 0 ) = y 0jm , we can write the second order derivatives ofr without keeping track of the terms of orderr. To do so, from now on we will denoteO(r) as a generic term dierent at each line wherejO(r)=rj is bounded. ^ @ xx r(x 0 ;y 0 ) =n T @ xx V(x 0 ;(x 0 ;y 0 )) +O(r) ^ @ yy r(x 0 ;y 0 ) =@ y n(x 0 ;(x 0 ;y 0 )) +O(r) ^ @ xy r(x 0 ;y 0 ) =@ x n(x 0 ;(x 0 ;y 0 )) +O(r) Remark4.1.11 (i): Note that the signed distance is as smooth as the surface is. Namely, as we are working withC 2 -boundaries,r(x;)isalsotwicedierentiableinatubularneighborhoodofV b (x). However,projection is only once dierentiable. See Krantz-Parks [43], Foote [35] and Leobacher-Steinicke [50]. (ii): Dierentiability in term ofx will follow by assumptions on the derivatives ofV. 4.2 TheItô’sformula GivenV : [0;T ]R d !D m 2 , by viewing (t;x)2 R d+1 , we have@ t V, @ x V, and@ xx V. We callV2 C 1;2 ([0;T ]R d ;D m 2 ) if all the above derivatives exist and are continuous. We further require that the derivatives are Lipschitz in R m . We use C 1;2 b to indicate that the derivatives are bounded. Introduce V p;q tb (t;x) as V p;q tb (t;x) :=fy2R m : q<r(t;x;y)<pg which is the tubular neighbourhood ofV b (t;x) with distancep > 0 inside andq > 0 outside. Also, set V p tb (t;x) :=V p;p tb (t;x). 100 We sayV(t;) has globally bounded curvature c t if@ y n(t;x;y) is bounded in the operator norm by c t . We sayV is non-degenerate if sup 0tT c t is bounded and there existsq> 0 such that the signed distance r(t;x;y)2 C 1;2;2 whenevery2V q tb (t;x). With applications in mind, we sayV is non-degenerate until the terminal ifV(T;x) =fg(x)g for some functiong :R d !R m and the restriction ofV on [0;T 0 ] for T 0 <T is non-degenerate. In this case, c t !1 ast!T . Forb2L 1;loc t;T (R d ) and2L 2;loc t;T (R d R d 0 ), deneX t;x s as X t;x s =x + Z s t b s ds + Z s t s dB s ; tsT for any (t;x)2 [0;T ]R d . Furthermore, introduce the correction term := tr T @ x n + 1 2 T @ y n where we omitted the arguments on purpose, as it will vary and is clear in the context. Lastly, L b;; V :=@ t V +@ x Vb + 1 2 tr( T @ xx V) n We now introduce the Itô’s formula. To simplify notations, setX =X 0;x 0 . Theorem2 LetV2C 1;2 ([0;T ]R d ;D m 2 )benon-degenerate. Take j (w;y) : [0;T ] R m !R m for allj2f1; ;d 0 gandset = ( 1 ; ; d 0) : [0;T ] R m !R m R d 0 . Also,consider (w;y) : [0;T ] R m !R m . Werequire and tobeuniformlyLipschitziny and(; 0)2L 2;loc 0;T ; (; 0)2L 1;loc 0;T . For y2R m , dene y as the strong solution of y t =y + Z t 0 L b;; + (s;X s ; y s )ds + Z t 0 @ x V + (s;X s ; y s )dB s 101 Suppose t (~ y)2T V (t;X t ; ~ y) holds for all ~ y2V b (t;X t ) almost surely and for all 0tT. (i): Suppose t (~ y)2 T V (t;X t ; ~ y) holds for all ~ y2V b (t;X t ) almost surely for all 0 t T. Then it holdsf y t : y2 V b (0;x 0 )g V b (t;X t ) almost surely for all 0 t T. IfV b takes values in connected compact sets, then the equality holds. (ii): Suppose t (~ y)2 T + V (t;X t ; ~ y) holds for all ~ y2V b (t;X t ) almost surely for all 0tT. Then it holdsf y t :y2V o (0;x 0 )gV o (t;X t ) almost surely. Remark4.2.1 One can formulate the general case, currently as a conjecture, as follows. Suppose, t (~ y)2T V (t;X t ; ~ y) holds for all ~ y2V k (t;X t ) almost surely for all 0tT, (N ( t L b;; ))(t;X t ; ~ y)2 T + V (t;X t ; ~ y) holds for all ~ y2V ` (t;X t ) almost surely for all 0 t T and 1`<k. Then it holdsf y t : y2 V k (0;x 0 )g V k (t;X t ) almost surely for all 1 k m. This claim reduces to (i); (ii) in the hypersurface case. † Proof Choose > 0 small enough and considerV tb ;V 2 tb where the signed distancer(t;x;y)2C 1;2;2 . Recall the projection ontoV b where it holds (t;x;y) =y +r(t;x;y)n(t;x;(t;x;y)) We introduce stopping times recursively as in 0 := inff0tT : y t 2V tb (t;X t )g^T out n := inff in n 0 is a process dened as t := exp Z t in k 1 s B s Z t in k [b 1 s 2 1 j 1 s j 2 ]ds Now, under the assumption of (i), n T = 0 and r(0;x 0 ;y) = 0. (4.3) concludes r(t;X 0;x 0 t ; y t ) = 0, or y t 2 V b (t;X 0;x 0 t ). Suppose further thatV b takes values in connected, compact sets. Note thaty7! y t is a homeomorphism almost surely (See Kunita [44]). In particular, continuous and locally one-to- one. SinceV b (0;x 0 ) is compact, it is mapped to a closed set inV b (0;x 0 ). By invariance of domains for manifolds without boundaries,y7! y t is an open mapping in relative topologies ofV b (0;x 0 ),V b (t;X 0;x 0 t ). Therefore, V b (0;x 0 ) maps to an open set in V b (t;X 0;x 0 t ). This concludes the equality as we assumed connectedness. Under the assumption of (ii), n T 0. Furthermore, r( in 0 ;X 0;x 0 in 0 ; y in 0 ) > 0. Then (4.3) implies r(t;X 0;x 0 t ; y t ) > 0 for all in 0 t out 0 . By induction,r(t;X 0;x 0 t ; y t ) > 0 for all in k t out k and anyk 1. 4.3 Appendix 4.3.1 LocalRepresentation This appendix is devoted to further understand local representations, and equivalent denition of a dier- entiable ow. 104 LetV :R d !D m 2 and xk;V k . Given anyx 0 2R d ,y 0 2V k (x 0 ), asV k is aC 2 -manifold, there exists a mapping x 0 ;y 0 = ( x 0 ;y 0 1 ; ; x 0 ;y 0 mk ) : R k ! R mk (after an appropriate translation and rotation) such that ~ x 0 ;y 0 (z) := (z; x 0 ;y 0 (z)) is a bijection from a neighborhood of 02R k to a neighborhood ofy 0 2V k (x 0 ) and ^ @ z x 0 ;y 0 ` (0) = 02R k for any`2f1; ;mkg. Moreover, n ` (x 0 ;y) := e k+` ( ^ @ z x 0 ;y 0 ` (y jk ); 0; ; 0) je k+` ( ^ @ z x 0 ;y 0 ` (y jk ); 0; ; 0)j (4.4) forms a basis forN V (x 0 ;y) locally aroundy 0 , wherey jk is the projection ofy onto rstk component. Note that the translation and the rotation is chosen such thatN V (x 0 ;y 0 ) = 0R mk , asn ` (x 0 ;y 0 ) =e k+` . Remark4.3.1 AsweareworkingwithC 2 -manifolds,localrepresentationisalsotwicedierentiable. When k =m 1 (for simplicity), we note that@ y n(x;y) = ^ @ zz x;y (0) up to some rotation factors. By slightly abusing the terminology, we call ; ~ both local representations. Furthermore, as we are interested on varying manifolds, we generalize the local representation at (x 0 ;y 0 ) to a neighborhoodx 0 . That is, we extend the same terminology to actually consider ~ x 0 ;y 0 (x;z) := (z; x 0 ;y 0 (x;z)); x2U x 0 whereU x 0 is a neighborhood ofx 0 and ~ x 0 ;y 0 (x;) is a bijection from a neighborhood of 02 R k to its image inV k (x). Following lemma will study the local representation whenV is a dierentiable ow. Note that given a local representation ~ x;y , one can set (x;y; ~ x) := ~ x;y (~ x;y jk ) wherey jk is the projection on the rst 105 k component (after appropriate translation and rotation) and provide an equivalent denition of dieren- tiable ow. Lemma4.3.2 LetV : R d !D m 2 . For anyk2f1; ;mg and (x 0 ;y 0 )2 G V;k , let x 0 ;y 0 be the local representation. (i): SupposeV is a dierentiable ow. Then there exists a neighborhood ofx 0 2R d , and a neighborhood of 02R k such that the local representation is well-dened and dierentiable inx. Moreover, ^ @ z x 0 ;y 0 (;z) is continuous in a neighborhood ofx 0 . (ii): SupposeV :R d !D m 2 isauniformlycontinuousdierentiableowwithgloballyboundedcurvature. Then there exists a uniformly large neighborhood ofx 0 , and a uniformly large neighborhood of 02R k such that the local representation is well-dened and dierentiable inx. Proof We will argue(ii) as(i) follows by similar arguments, without requiring uniform choices. Take a ballB =B(0;R) of radiusR around 02R k , whereR will be determined later. Letfn ` g 1`mk be a continuous basis for the normal space and without loss of generality assumen ` (x 0 ;y 0 ) =e k+` . For arbitrary 0<< 1= p m, let U (x) :=fy2V k : max 1`mk jn ` (x;y)e k+` jg Sincen ` (x 0 ;y 0 ) =e k+l ,y 0 2U (x 0 ). Let x :U (x)!R k be the projection on the rstk component. We will rst argue, for xedx, x is a dieomorphism. The inverse is exactly ~ x 0 ;y 0 (x;). Moreover, we will show that x 0 (U (x 0 )) contains the ballB for a xed radiusR only depending on the global curvature bound ofV. This further implies, as we assumedn is uniformly continuous and by denition ofU (x), one can nd a uniformly large neighborhood ofx 0 (depending on) such that x (U (x)) always contains B. This concludes that local representation is always well dened inB and uniformly large neighborhood ofx 0 . 106 First of all, we need to restrict our attention to a connected component ofU (x). First, x the connected component ofU (x 0 ) containingy 0 and forU (x), choose the connected component in a way that varies continuously. This is possible asV is continuous. We abuse notations slightly and still useU (x). To argue that x is injective, suppose there exists y 1 ;y 2 2 U (x) where x (y 1 ) = x (y 2 ). Since U (x) is path connected, andV k is dierentiable, let : [0; 1]!V k be a dierentiable curve with (0) = y 1 ; (1) = y 2 . Set ~ (t) = (t)(y 1 ). By construction, ~ (0); ~ (1) is orthogonal toe ` 0 for 1 ` 0 k. By Multidimensional Rolle’s Theorem, there existst 0 such that ~ 0 (t 0 ) = 0 (t 0 ) is also orthogonal toe ` 0 for 1` 0 k. Equivalently, there existsy 3 2U (x) and 06=v2T(x;y 3 ) wherev2 0R mk . Therefore there exists`2f1; ;mkg wheren ` (x;y 3 )e k+` = 0, which contradicts with the denition ofU (x). Next, we need to argue ^ @ y x (y) has full rankmk. Note that ^ @ y x (y) = 2 6 6 6 6 6 6 4 e 1 P mk `=1 (n ` (x;y)e 1 )n ` (x;y) . . . e k P mk `=1 (n ` (x;y)e k )n ` (x;y) 3 7 7 7 7 7 7 5 It is straightforward to check that the rows are linearly independent. But rst, let us note that since ^ @ y x (y) is continuous inx, ^ @ z x 0 ;y 0 (x;z) is also continuous inx. Now, suppose forfc ` 0g 1` 0 k R, k X ` 0 =1 c ` 0e ` 0 = k X ` 0 =1 c ` 0 mk X `=1 (n ` (x;y)e ` 0)n ` (x;y) = mk X `=1 k X ` 0 =1 c ` 0(n ` (x;y)e ` 0) n ` (x;y) From the denition ofU (x), let us argue that there is no linear combination ofn ` ’s that lies inR k 0. By slight abuse of notations, consider mk X `=1 c ` n ` (x;y) = mk X `=1 c ` (n ` (x;y)e k+` ) + mk X `=1 c ` e k+` 107 Distance of the last term toR k 0 is ( P mk l=1 c 2 ` ) 1=2 P mk k=1 jc ` j= p m. First term on the right hand side has a norm bounded by P mk `=1 jc ` j. Therefore, their sum cannot lie inR k 0. We argued that (4.4) is a continuous basis for the normal space. Now, we determine R for which U := x 0 (U (x 0 )) containsB(0;R), whereU (x) is dened for (4.4). Take anyv2R k withjvj = 1. By (4.4) and denition ofU (x 0 ), j ^ @ z x 0 ;y 0 ` (x 0 ;z)vj (1 2 ) 1=2 ; 8z2U By Fundamental Theorem of Calculus, for 0rR, e k+` n ` (x 0 ; ~ x 0 ;y 0 (x 0 ;rv)) 1 = Z r 0 (e k+` ) T @ y n ` (x 0 ; ~ x 0 ;y 0 (x 0 ;sv)) 2 6 4 I kk ^ @z x 0 ;y 0 (x0;sv) 3 7 5 mk vds whereI kk is the identity matrix. If c bounds all@ y n ` uniformly, then we have je k+` n ` (x 0 ; ~ x 0 ;y 0 (x 0 ;rv)) 1j c Z r 0 1 + mk X `=1 ( ^ @ z x 0 ;y 0 ` (x 0 ;sv)v) 2 1=2 ds cr 1 +m 2 1 2 1=2 This concludes that we can chooseR c 1 2 (1 +m 2 =(1 2 )) 1=2 =2. Finally, we will argue that partial derivatives of x 0 ;y 0 (x;z) inx exists. Consider the mapping (~ x) = ( 1 (~ x); ; m (~ x)) = (x; (z; x 0 ;y 0 (x;z)); ~ x) coming from the assumption. Note that x 0 ;y 0 (~ x; (~ x) jk ) = (~ x) jmk 108 because takes values onV k , where jk ; jmk denotes the projection on the rstk and the lastmk components respectively. (Recall, we translated and rotated with respect to (x 0 ;y 0 ).) Then it follows, for 1`mk, 1 t x 0 ;y 0 ` (x +te i ; (x +te i ) jk ) x 0 ;y 0 ` (x +te i ;z) + 1 t x 0 ;y 0 ` (x +te i ;z) x 0 ;y 0 ` (x;z) = 1 t k+` (x +te i ) k+` (x) (4.5) Since limit of the rst term and the last term exists, we conclude the result. 4.3.2 Postponedproofs Proof [Proof of Lemma 4.1.3.] Consider the local representation = x;y from Lemma 4.3.2. We will drop (x;y) from superscripts of too, = x;y . Recall (4.5), 1 t ` (x +te i ; (x +te i ) jk ) ` (x +te i ; 0) + 1 t ` (x +te i ; 0) ` (x; 0) = 1 t k+` (x +te i ) k+` (x) for all 1`mk. Notice that, since ^ @ z x;y ` (x; 0) = 0 the rst term goes to 0 ast! 0. Also, projection onto the normal space is the lastmk component. This concludes N ^ @ ~ x i (x)j ~ x=x = ^ @ ~ x i (~ x; 0)j ~ x=x which is independent of . 109 Proof [Proof of Lemma 4.1.5.] As the integrand of (4.1) is continuous inx and locally Lipschitz iny, it is well-posed at least fort in a neighborhood of 0. Take a partition 0 =t 0 <<t n =t, t j :=t j+1 t j , and consider two discretization as follows; ^ V i 0 :=V i 0 =y; ^ V i j+1 := ^ V i j +N 0 i (x +t j e i ; ^ V i j )t j ~ V i 0 :=y; ~ V i j+1 := ~ x+t j e i ; ~ V i j x +t j+1 e i ; ( ~ V i j ) jk 2V k (x +t j+1 e i ) (Notice, we considered a Lipschitz extension). It is clear that ^ V i n !V i t , and ~ V i n 2V k (x+te i ) by denition. Therefore, we only need to showj ^ V i n ~ V i n j! 0. j ^ V i j+1 ~ V i j+1 jj ^ V i j ~ V i j jj( ^ V i j+1 ^ V i j ) ( ~ V i j+1 ~ V i j )j = N 0 i (x +t j e i ; ^ V i j )t j ~ x+t j e i ; ~ V i j x +t j+1 e i ; ( ~ V i j ) jk ~ V i j = N 0 i (x +t j e i ; ^ V i j )t j ^ @ x i x+t j e i ; ~ V i j x +t j e i ; ( ~ V i j ) jk t j +o(jt j j) = N 0 i (x +t j e i ; ^ V i j )t j N 0 i (x +t j e i ; ~ V i j )t j +o(jt j j) Cj ^ V i j ~ V i j jt j +o(jt j j) This suces to conclude the convergence. 110 Chapter5 SetValuedHJBEquation This chapter introduces the notion of HJB equation. We prove the well-posedness in a special (but impor- tant) case whereV(x) =V m1 (x)[V m (x),V m (x) is an open set inR m andV m1 (x) is the topological boundary ofV m (x). To simplify notations, we denote the boundary asV b := V m1 and the interior as V o :=V m . Also, as the normal space isf0g for the interior, we abuse notations and setG V :=G V;m1 . We will repeatedly use notations similar to f[;z](t;x;y) :=f(t;x; t ;y;z); or f[@ x V](t;x;a;y) :=f(t;x;a;y;@ x V(t;x;y)(t;x;a)) not only forf, but whenever it is convenient to shorten the notations in the absence of confusion. 5.1 WellposednessoftheHJBEquations Introduce and recall the notations L :=@ t V +@ x Vb + 1 2 tr( T @ xx V) n; := tr T @ x n + 1 2 T @ y n 111 We sayV2C 1;2 ([0;T ]R d ;D m 2 ) is the classical solution to the HJB equation if sup a2A;2(T(t;x;y)) d 0 n L +f[@ x V +] (t;x;a;y) = 0; 8(t;x;y)2G V (5.1) Notice that the optimization includes a direction besides the control. Remark5.1.1 (i): Form = 1, note thatV(t;x) = [v (t;x);v + (t;x)] is an interval, n2f1; 1g and 0. Then, (5.1) reduces to the standard HJB equations forv andv + . (ii): Thereisacorrespondencebetween (;)(controlandadirectionasaprocess)and (;)(controland a terminal), in a sense that given one pair, a corresponding pair can be constructed. (iii): For the general case, let LV(t;x;y) :=cl n [L +f[@ x V +]](t;x;a;y) :8a2A;2 (T V (t;x;y)) d 0 o : andwenowconjecturethenotionofthesolution. WesayV2C 1;2 ([0;T ]R d ;D m 2 )istheclassicalsolution to the HJB equation if LV(t;x;y)T + V (t;x;y); andLV(t;x;y)\T V (t;x;y)6=;; 8(t;x;y)2G V : (5.2) Herecl()denotestheclosureandG V =[ k G V;k . Intuitionis,sinceLVisthecollectionofvectorsthatderives V, vectors in it should point in the direction that lies in the set. Moreover, it should contain (at least in the closure) a vector that lies in the tangent space. In the case of hypersufaces, the rst condition of (5.2) means n L +f[@ x V +] (t;x;a;y) 0; 8(t;x;y)2G V;m1 and the second condition implies that supremum achieves 0. 112 (iv): For the interiorV o , tangent space is simply the full spaceR m and hence (5.2) is trivially satised. However, one cannot restrict attention to onlyV b as a solution. In that case,T + =T and (5.2) will fail, as it means (5.1) is satised for alla2A;2 (T(t;x;y)) d 0 but not only for the supremum. Before moving on to the main theorem of this section, we state a certain integrability assumption on the terminal condition. Assumption5.1.2 For any (t;x)2 [0;T ]R d ,2L 2 t;T , and2A t;T where2V(T;X t;x; T ) almost surely, there exists a constantC independent of; such thatEjj 4 C. Note that this is satised if V(T;) is contained in a compact set uniformly. Moreover, if there exists uniformly LipschitzR :R d !R + and :R d !R m whereV(T;x)B((x);R(x)), assumption is also satised. Similarly holds ifV(T;x) =g(x) for some Lipschitz function. Remark5.1.3 (i): Wenotethatforf withlineargrowthonz,anysolutiontoHJBequationhastobeconvex. Toobservethis,forany (t;x;y)2G V ,supposethereexists2 (T(t;x;y)) d 0 suchthat tr( T @ y n)< 0. For > 0, 2 1 2 tr( T @ y n) +tr(@ x n) +f(t;x;a;y;@ x V +) 2 1 2 tr( T @ y n)C!1 as!1 Therefore, tr( T @ y n) 0 for all . Moreover, this observation also implies that we can consider to be contained in a compact set. (ii): See section 5.3 for a non-convex solution to HJB equation wheref is quadratic on (y;z). Theorem5.1.4 Suppose the terminalV(T;)2C(R d ;D m 2 ) satises the assumption 5.1.2. (i): LetV(t;x) be dened as in (2.1). IfV2C 1;2 b ([0;T ]R d ;D m 2 ) and non-degenerate, then it is a classical solution to the HJB equation (5.1). (ii): Suppose ~ V2C 1;2 b ([0;T ]R d ;D m )isnon-degenerateandaclassicalsolutiontotheHJBequation (5.1) with the given terminalV(T;). Then it holdsV(t;x) =V(t;x) for all (t;x)2 [0;T ]R d . 113 If the terminal isV(T;) : R d ! R m , (i),(ii) holds by replacing non-degenerate with non-degenerate until the terminal. Moreover, suppose there existsI : [0;T ]R d R m !A andZ : [0;T ]R d R m !R m such that n L Z +f[@ x V +Z] (t;x;I(t;x;y);y) = 0;Z(t;x;y)2T V (t;x;y); 8(t;x;y)2G V and X t =x 0 + Z t t 0 b(s;X s ;I(s;X s ; s ))ds + Z t t 0 (s;X s ;I(t;X t ; t ))dB s t =y 0 Z t t 0 f[I;@ x V +Z](s;X s ; s )ds + Z t t 0 (@ x V[I] +Z)(s;X s ; s )dB s has a strong solution for any (t 0 ;x 0 ;y 0 )2G V . Then, the optimal control and direction ( ; ) is given by (t;w) :=I(t;X t ; t ); (t;w) :=Z(t;X t ; t ) That is, t =Y ;T; T t 2V b (t;X t ) almost surely for allt 0 tT. Proof (i): Without loss of generality, set (t;x) = (0;x 0 ) and we will drop (0;x 0 ) from superscripts from now on. Consider (0;x 0 ;y 0 )2G V ,2A 0;T , and arbitrary"> 0. Let; j : [0;"] R m !R m and = ( 1 ; ; d 0) be bounded continuous processes uniformly Lipschitz inR m . Moreover,n T (t;X t ;y) = 0 and n T (t;X t ;y) = 0 for all y 2 V(t;X t ) almost surely for all 0 t ". Consider the SDE = y 0 ;; , t =y 0 + Z t 0 (L [] +)(s;X s ; s )ds + Z t 0 (@ x V[] +)(s;X s ; s )dB s 114 and by Itô’s Formula, " 2 V b (";X " ). Consider the BSDE (Y;Z) with the terminal " . Then by DPP, Y 0 =J(0;x 0 ;;"; " )2V(0;x 0 ). Note that, jJ(0;x 0 ;;"; " )y 0 j = E Z " 0 (L [] +)(s;X s ; s ) +f[Z s ;](s;X s ;Y s )ds C" Recall the local representation (z) = 0;x 0 ;y 0 (x 0 ;z) as in section 4.1.2 (with further discussions in the appendix). To shorten the notations, letJ " = J(0;x 0 ;;"; " ) andJ " jm1 be the projection ofJ " onto the tangent spaceT(0;x 0 ;y 0 ). Then, n T (0;x 0 ;y 0 ) J " J " jm1 (J " jm1 ) becauseJ " jm1 + (J " jm1 )n T (0;x 0 ;y 0 )2 V b (0;x 0 ) andJ " 2 V(0;x 0 ). Hence we can argue, since the surface is locally at (or ^ @ z (y 0 ) = 0), n T (0;x 0 ;y 0 )(J(0;x 0 ;;"; " )y 0 ) =n T (0;x 0 ;y 0 )(J " J " jm1 ) (J(0;x 0 ;;"; e )j jm1 ) (y 0 ) =o("); i.e. lim "!0 " 1 n T (J(0;x 0 ;;"; " )y 0 ) 0 (5.3) This is almost what we require, however, we need to take care of thez component off. To do so, note that E h " 1 (J(0;x 0 ;;"; " )y 0 ) Z " 0 Z s jZ s j dB s i = 0 where we set Z s := (@ x V[] +)(s;X s ; s )Z s . More explicitly, E " 1 Z " 0 (L [] +)(s;X 0;s ; s ) +f[Z s ;](s;X s ;Y s )ds Z " 0 Z s jZ s j dB s +E " 1 Z " 0 jZ s jds = 0 115 and since the rst line goes to 0 as"! 0, we obtained lim "!0 E " 1 Z " 0 j(@ x V[] +)(s;X s ; s )Z s jds = 0 This concludes that (5.3) implies lim "!0 " 1 Z " 0 n T L [](s;x 0 ;y 0 ) +f[@ x V +;](s;x 0 ;y 0 ) 0 Now we can consider supremum over deterministic values, ∗ sup a2A;2(T(0;x 0 ;y 0 )) d 0 n T L +f[@ x V +] (0;x 0 ;a;y 0 ) 0 (5.4) To show that the supremum achieves 0, consider any BSDE (Y;Z) = (Y ;T; ;Z ;T; ) and introduce t (y) :=Z t @ x V[](t;X t ;y) t (y) := L [] +f[Z;] (t;X t ;y) and rewrite BSDE in the forward direction as Y ;T; t =Y ;T; 0 + Z t 0 (L [] + )(s;X s ;Y ;T; s )ds + Z t 0 (@ x V[] +)(s;X s ;Y ;T; s )dB s Following the computations in Itô’s formula yields dr(t;X t ;Y t ) = [(n T )[] +b 1 r](t;X t ;Y t )dt + [(n T )[] + 1 r](t;X t ;Y t )dB t ∗ If (5.4) does not hold witha ; ,a and(t;w;y) := (nn T )(t;Xt;y) yields a contradiction. 116 for some processesb 1 ; 1 where 1 is bounded andjb 1 t j C(1 +jZ t j). Introduce the stopping time for "> 0 and some xedq> 0 where the signed distance is regular, := infft> 0 :Y ;T; t = 2V q tb (t;X t )g^" (5.5) Now, we can express r(;X ;Y )r(0;x 0 ;Y 0 ) = + Z 0 [(n T )[] +b 1 r](t;X t ;Y t )dt + Z 0 [(n T )[] + 1 r](t;X t ;Y t )dB t (5.6) Assuming the supremum in (5.4) does not achieve 0, we will arrive at a contradiction by controlling the signed distancer appropriately. To roughly present the idea, if we have ( ; ) wherey 0 = Y ;T; 0 , by Lemma 2,r 0 and (5.6) impliesn T = 0;n T = 0, which suces to conclude. We rst start by collecting necessary bounds and then choose (;) to controlr later. Take expectation of (5.6) to obtain E Z 0 (n T )[](t;X t ;Y t )dt =Er(;X ;Y )r(0;x 0 ;Y 0 )E Z 0 b 1 r(t;X t ;Y t )dt Since we will required to control thez component off, let us rewrite above as, E Z 0 n T (L [] +f[@ x V + t nn T t ;])[](t;X t ;Y t )dt =E Z 0 n T f[Z;]f[@ x V + t nn T t ;] [](t;X t ;Y t )dt +r(0;x 0 ;Y 0 )Er(;X ;Y ) +E Z 0 b 1 r(t;X t ;Y t )dt 117 And we bound as E Z 0 n T (L [] +f[@ x V + t nn T t ;])[](t;X t ;Y t )dt r(0;x 0 ;Y 0 ) +Ejrj(;X ;Y ) +CE Z 0 jn T t j[](t;X t ;Y t )dt +E Z 0 jb 1 rj(t;X t ;Y t )dt (5.7) Let us take care the last term now. Although we don’t keep track in the notations,b 1 ; 1 contains terms computed at bothY t and(t;X t ;Y t ). E Z 0 jb 1 rj(t;X t ;Y t )dt E Z 0 jb 1 j 2 (t;X t ;Y t ) 1=2 E Z 0 jrj 2 (t;X t ;Y t ) 1=2 C E Z " 0 1 +jZ t j 2 1=2 E Z 0 jrj 2 (t;X t ;Y t ) 1=2 C E Z 0 jrj 2 (t;X t ;Y t ) 1=2 (5.8) and by assumption 5.1.2, the constant doesn’t depend on (;). Now, by multiplying (5.6) by R 0 sign(n T )[](t;X t ;Y t )dB t and taking the expectation yields E Z 0 jn T j[](t;X t ;Y t )dt =Er(;X ;Y ) Z 0 sign(n T )[](t;X t ;Y t )dB t +E Z 0 (n T )[](t;X t ;Y t )dt Z 0 sign(n T )[](t;X t ;Y t )dB t +E Z 0 (b 1 r)(t;X t ;Y t )dt Z 0 sign(n T )[](t;X t ;Y t )dB t +E Z 0 sign(n T )[]( 1 r)(t;X t ;Y t )dt 118 Note that is bounded andjjC(1 +jZj 2 ) , and we bound above as follows, E Z 0 jn T j[](t;X t ;Y t )dt " 1=2 Ejrj 2 (;X ;Y ) 1=2 +" 1=2 E Z 0 j1 +Zj 2 dt 2 1=2 +" 1=2 E Z 0 jb 1 rj(t;X t ;Y t )dt 3=2 2=3 +CE Z 0 jrj(t;X t ;Y t )dt Let us take care the term that containsb 1 again, E Z 0 jb 1 rj(t;X t ;Y t )dt 3=2 E Z 0 jb 1 j 2 (t;X t ;Y t )dt 3=4 Z 0 jrj 2 (t;X t ;Y t )dt 3=4 E Z 0 jb 1 j 2 (t;X t ;Y t )dt 3=4 E Z 0 jrj 2 (t;X t ;Y t ) 3 1=4 C E Z " 0 1 +jZ t j 2 dt 3=4 " 1=2 E Z 0 jrj 6 (t;X t ;Y t ) 1=4 C" 1=2 E Z 0 jrj 6 (t;X t ;Y t ) 1=4 and hence we obtained E Z 0 jn T j[](t;X t ;Y t )dt " 1=2 Ejrj 2 (;X ;Y ) 1=2 +C" 1=2 +C" 5=6 E Z 0 jrj 6 (t;X t ;Y t )dt 1=6 +CE Z 0 jrj(t;X t ;Y t )dt (5.9) 119 Using (5.8), (5.9) in (5.7), we get E Z 0 n T (L [] +f[@ x V + t nn T t ;])[](t;X t ;Y t )dt r(0;x 0 ;Y 0 ) +Ejrj(;X ;Y ) +C" 1=2 Ejrj 2 (;X ;Y ) 1=2 +C" 1=2 +C" 5=6 E Z 0 jrj 6 (t;X t ;Y t )dt 1=6 +CE Z 0 jrj(t;X t ;Y t )dt +C E Z 0 jrj 2 (t;X t ;Y t ) 1=2 C" 1=2 +r(0;x 0 ;Y 0 ) +C(1 +" 1=2 ) Ejrj 2 (;X ;Y ) 1=2 +C(" 5=6 +" 1=6 +" 1=3 ) Z " 0 Ejrj 6 (t;X t ;Y t )dt 1=6 (5.10) By denition ofV(0;x 0 ), for" 0 > 0, choose " 0 ; " 0 such thatjY " 0 ;T; " 0 0 y 0 j < " 4 0 . By Lemma 2 (iii,iv), P r(t;X " 0 t ;Y t )>" 0 <" 0 ; andP r(t;X " 0 t ;Y t )>q <C q " 2 0 8t2 [0;"] (5.11) Take a partition 0 =t 0 <<t n =" where p " 0 =2<t k+1 t k < p " 0 for all 1kn and P sup 0t" r(t^;X " 0 t^ ;Y t^ ) =q = n X k=1 P sup 0t" r(t^;X " 0 t^ ;Y t^ ) =q;2 (t k1 ;t k ] n X k=1 P r(t k1 ;X " 0 t k1 ;Y t k1 )>" 0 +P sup t k1 t jr(t^;X " 0 t^ ;Y t^ )r(t k1 ;X " 0 t k1 ;Y t k1 )j>q" 0 2" p " 0 + C q 4 " p " 0 120 and similarly, consider a partition with" 5=6 0 =2<t k+1 t k <" 5=6 0 , P sup 0t" r(t^;X " 0 t^ ;Y t^ )>" 1=6 0 = n X k=1 P sup 0t" r(t^;X " 0 t^ ;Y t^ )>" 1=6 0 ;2 (t k1 ;t k ] n X k=1 P r(t k1 ;X " 0 t k1 ;Y t k1 )>" 0 +P sup t k1 t jr(t^;X " 0 t^ ;Y t^ )r(t k1 ;X " 0 t k1 ;Y t k1 )j>" 1=6 0 " 0 2 " " 5=6 0 " 0 + C" " 5=6 0 1 " 2=3 0 " 5=3 0 C"" 1=6 0 We can now bound the expectations, Ejrj 2 (;X " 0 ;Y ) =q 2 P sup 0t" r(t^;X " 0 t^ ;Y t^ ) =q +Ejrj 2 (";X " 0 " ;Y " )1 n sup 0t" r(t^;X " 0 t^ ;Yt^ )" 1=6 0 o " 0 " +Cq 6 " 2 " 1=6 0 (5.13) Finally, let us assume sup a2A;2(T(0;x 0 ;y 0 )) d 0 n T L +f[@ x V +] (0;x 0 ;a;y 0 )<< 0 121 Since the supremum is taken over a compact set (See Remark 5.1.3), it is continuous in (t;x;y) and hence choose 0 such that sup a2A;2(T(t;x;y)) d 0 n T L +f[@ x V +] (t;x;a;y)<=2 wheneverjtj +jxx 0 j +jyy 0 j< 0 . Note that = nn T , recall (5.10) and we use" 0 on subscripts to indicate the dependence, E Z 0 n T (L " 0 [ " 0 ] +f[@ x V + " 0 t nn T " 0 t ; " 0 ])[](t;X " 0 t ;Y " 0 t )dt >=2 E1 n sup 0t" jX " 0 t x 0 j+jY " 0 t y 0 j> 0 o Z 0 n T (L " 0 [ " 0 ] +f[@ x V + " 0 t nn T " 0 t ; " 0 ])[](t;X " 0 t ;Y " 0 t )dt >=2C P( sup 0t" jX " 0 t x 0 j +jY " 0 t y 0 j> 0 ) 1=2 E Z 0 j1 +Z " 0 j 2 2 1=2 >=2C" 1=2 = 0 To conclude the result, use (5.12) and (5.13), together with (5.10), =2<C " 1=2 0 +C" 1=2 +" 4 0 +C p " 0 (2"q 2 +C"q 2 ) +" 0 +q 2 " 0 1=2 +C " 0 " +q 6 " 2 " 1=6 0 1=6 Note that the time partitions of [0;"] are done with mesh size depending on" 0 and hence send" 0 ! 0, =2<C" 1=2 (1 + 1 0 ) which is a contradiction for" small enough. Remark5.1.5 Before moving on to the proof of (ii), let us discuss an important observation which might also play a role for the general setting in the future. Consider V k (x 0 ) as in the general case of Chapter 122 4 and note thatO " (V k (x 0 )), for suciently small " > 0, has an interior and a smooth boundary, hence contained in the case of hypersurfaces. Lety " 2O " (V k (x 0 )) b , and there existsy 0 2 V k (x 0 ) together with v 0 2 N(x 0 ;y 0 );jv 0 j = 1 such thaty " = y 0 +v 0 ". Given x 0 ;y 0 () as in the denition of the dierentiable ow, one can set x 0 ;y" " (x) := x 0 ;y 0 (x) +v(x; x 0 ;y 0 (x))" wherev(x 0 ;y 0 ) = v 0 ; (N@ x v)(x 0 ;y 0 ) = 0. This concludes, sinceN(x 0 ;y " )N(x 0 ;y 0 ),@ x O " (V k )(x 0 ;y " ) =N(x 0 ;y " )@ x V k (x 0 ;y 0 ). In the case of hypersurfaces,v =n and@ x O " (V)(x 0 ;y " ) =@ x V(x 0 ;y 0 ). Toclarifytheintuition,consideraregularsmoothfunctiona(x) :R d !R m andtheclosedballB(a(x);"). Derivative on the surface ofB(a(x);") is given bynn T (x;y) ^ @ x a(x). (ii): Let ~ r " ; ~ " ; ~ n " be the signed distance function, projection and the unit outward normal vector corresponding to ~ V " :=O " ( ~ V) and set ~ r := ~ r 0 ; ~ := ~ 0 ; ~ n := ~ n 0 . Assume there existsy 0 2V(0;x 0 ) such thaty 0 = 2 ~ V(0;x 0 ), equivalently ~ r(0;x 0 ;y 0 )< 0. Let us rst handle the non-degenerate case. Consider"> 0 small enough such that ~ r2C 1;2;2 inside ~ V C" 1=10 tb where we will determine C. From (2.1), choose2A 0;T and2V(T;X T ) such that jy 0 Y ;T; 0 j<" Introduce the stopping timeT 0 as † T 0 := infft> 0 : ~ r(t;X t ;Y ;T; t ) = 0g: Note thatT 0 T becauseV(T;X T ) = ~ V(T;X T ). We claim that for stopping times; 0 , ~ r 2 " (;X ;Y ;T; )1 f< 0 g 1 f< 0 g C 1 E ~ r 2 " ( 0 ;X 0;Y ;T; 0 ) +1 f< 0 g ^ C C" 5=4 ; if C" 1=4 ~ r " (s;X s ;Y ;T; s ) 0;8s 0 (5.14) † Altough the notation is slightly confusing, we reserve for later, andT0 will act as the terminal time. 123 Here we marked constants C 1 ; ^ C that depends on the bounds of derivatives of ~ V, b;;f; ^ @ y f; ^ @ z f, to distinguish from an arbitrary constantC. Moreover, we now set C = 2 ^ C 1=2 . For arbitrary ~ "> 0, introduce stopping times where we will determine the> 0, ~ " := inf T 0 sj ~ r " (s;X s ;Y ;T; s ) = ^ C 1=2 ~ " 1=4 ^T 0 Consider (5.14) with =T 0 ; 0 = 0 ^ 2" <T 0 (since 0 <T 0 ), ~ r 2 " (T 0 ;X T 0 ;Y ;T; T 0 ) C 1 E ~ r 2 " ( 2" ^ 0 ;X 2" ^ 0 ;Y ;T; 2" ^ 0 ) + 2 ^ C 3=2 " 5=4 which holds onf~ r " (T 0 ;X T 0 ;Y ;T; T 0 ) < 0g. For the rest of the arguments, we always implicitly consider this set, as in the complement we already have the estimate that ~ r(T 0 ;X T 0 ;Y ;T; T 0 )>". Let us take care the rst term, E ~ r 2 " ( 2" ^ 0 ;X 2" ^ 0 ;Y ;T; 2" ^ 0 ) =E ~ r 2 " ( 2" ;X 2" ;Y ;T; 2" )1 f 2" < 0 g = 2 1=2 ^ C" 1=2 E 1 f 2" < 0 g =: 2 1=2 ^ C" 1=2 P ( 2" < 0 ) 2 1=2 ^ C" 1=2 P sup " 2 1=4 ^ C 1=2 " 1=4 E sup " 2" ^ 0 almost surely and since " < 2" by the choice of, we conclude 0 < " < 2" (which holds already if " = 2" =T 0 ). But then (5.14) reads; ~ r 2 " (T 0 ;X T 0 ;Y ;T; T 0 ) 2 ^ C 3=2 " 5=4 which concludesY ;T 0 ; T 0 2 ~ V "+C" 5=8(T 0 ;X T 0 ). Note that we can choose the same to repeat the ar- gument with terminalY ;T; T 0 , and considering ~ r 0 " 0 = ~ r "+C" 5=8 +" 0, we concludeY ;T; 0 2 ~ V "+C" 5=8 = (0;x 0 ) ~ V C" 1=8(0;x 0 ). Finally, 0> ~ r(0;x 0 ;y 0 )jy 0 Y ;T; 0 jC" 1=8 "C" 1=8 Since" is arbitrary, we concludey 0 2 ~ V(0;x 0 ) by this contradiction. 125 To complete the argument, we now argue (5.14). To simplify notations, let us drop the superscripts and introduce Y 0 t := ~ " (t;X t ;Y t ); andY 00 t := ~ 0 (t;X t ;Y 0 t ) As we have chosenT " appropriately, it holds thatY 0 t =Y 00 t +n(t;X t ;Y 00 t )" andn(t;X t ;Y 0 t ) =n(t;X t ;Y 00 t ). Moreover, derivatives of ~ V and ~ V " considered onY 0 andY 00 respectively are equal. As ~ r " (t;X t ;Y t )n ~ V" (t;X t ;Y 0 t ) =: ~ r " (t;X t ;Y t )~ n " (t;X t ;Y 0 t ) =Y 0 t Y t and ~ r 2 " (t;X t ;Y t ) =jY 0 t Y t j 2 , Itô Formula yields, d~ r 2 " (t;X t ;Y t ) = 2~ r " (t;X t ;Y t )~ n(t;X t ;Y 00 t ) h ~ L ; (t;X 0;t ;Y 00 t ) +f[;@ x ~ V +](t;X 0;t ;Y 00 t ) i + h f[;@ x ~ V " +](t;X 0;t ;Y 0 t )f[;@ x ~ V +](t;X 0;t ;Y 00 t ) i + h f[;Z t ](t;X 0;t ;Y t )f[;Z t ](t;X 0;t ;Y 0 t ) i + h f[;Z t ](t;X 0;t ;Y 0 t )f[;@ x ~ V " +](t;X 0;t ;Y 0 t ) i + @ x ~ V " + (t;X 0;t ;Y 0 t )Z t 2 dt + 2~ r " (t;X t ;Y t )~ n T " (t;X t ;Y 0 t ) [@ x ~ V " +](t;X 0;t ;Y 0 t )Z t dB t = 2~ r " (t;X t ;Y t )~ n T (t;X t ;Y 00 t ) h ~ L ; (t;X 0;t ;Y 00 t ) +f[;@ x ~ V +](t;X 0;t ;Y 00 t ) i + h f[;@ x ~ V " +](t;X 0;t ;Y 0 t )f[;@ x ~ V +](t;X 0;t ;Y 00 t ) i + y (Y t Y 0 t ) + tr ([@ x ~ V " +](t;X 0;t ;Y 0 t )Z t )( z ) T + @ x ~ V " + (t;X 0;t ;Y 0 t )Z t 2 dt + 2~ r " (t;X t ;Y t )~ n T " (t;X t ;Y 0 t ) [@ x ~ V " +](t;X 0;t ;Y 0 t )Z t dB t 126 where bounded processes y 2R mm ; z 2 (R md 0 ) m are dened as y t := Z 1 0 ^ @ y f[;Z t ](t;X 0;t ;Y 0 +(YY 0 ))d z t := Z 1 0 ^ @ z f[;@ x ~ V " + +(Z t @ x ~ V " )](t;X 0;t ;Y 0 )d As ~ V solves the HJB equation, together with the condition of the argument, and by noting 2~ r " (t;X t ;Y t )~ n T " (t;X t ;Y 0 t )tr ([@ x ~ V " +](t;X 0;t ;Y 0 t )Z t )( z ) T 2 1 @ x ~ V " + (t;X 0;t ;Y 0 t )Z t 2 +C~ r 2 " (t;X t ;Y t ) we can integrate fromt to 0 to obtain the bound ~ r 2 " (t;X t ;Y t )1 ft< 0 g 1 ft< 0 g E t ~ r 2 " ( 0 ;X 0;Y 0) +C" 1+a +CE t Z 0 t ~ r 2 " (s;X s ;Y s )ds and the claim follows by the stochastic Grönwall Inequality. (See O-Kim-Pak [55].) Lastly, for the non-degenerate until the terminal case, recall thatV(T;x) = g(x). In this case, we cannot guarantee that ~ r (hence ~ r " ) will be dierentiable all the way to the terminalT . Therefore, choose T " T such that ~ r2C 1;2;2 inside ~ V C" 1=10 tb up to timeT " . Note that, as"! 0,T " !T . Choose2A 0;T such that jy 0 Y ;T;g 0 j<" In this case, we simply choose any measurable2 ~ V(T " ;X Te ) because 2 1 Ejg(X T )j 2 Ejg(X T" )j 2 +Ejg(X T )g(X T" )j 2 w(TT " ) 127 for some modulus of continuityw. In fact, since@ t ~ V is bounded, we can take it asC(TT " ). Now the proof follows exactly the same and at the end we obtain 0> ~ r(0;x 0 ;y 0 )jy 0 Y ;T"; 0 jC" 1=8 jy 0 Y ;T"; 0 jjY ;T;g 0 Y ;T"; 0 jC" 1=8 "w(T " T )C" 1=8 Since" is arbitrary, we again concludey 0 2 ~ V(0;x 0 ) by this contradiction. To argue that ~ VV, we will proceed by constructing an" 0 -optimal control and direction. For some q> 0, considerT q T and a tubular neighbourhoodV q tb . Fix (0;x 0 ) andy 0 2 ~ V b (0;x 0 ). Since ~ V satises HJB equation (5.1), choosea 0 2A and 0 2 (T ~ V (0;x 0 ;y 0 )) d 0 such that " 0 < ~ n L +f[@ x ~ V + 0 ] (0;x 0 ;a 0 ;y 0 ) 0 For 0tt 1 , set 1 (t;w) =a 0 and dene X 1 t =x 0 + Z t 0 b(s;X 1 s ; 1 )ds + Z t 0 (s;X 1 s ; 1 )dB s (5.15) Extend 0 as 1 (t;w;y) := 0 ~ n~ n T (t;X 1 t ; ~ (t;X 1 t ;y)) 0 (5.16) which is Lipschitz iny whenevery2 ~ V q tb (t;X 1 t ). Dene, 1 t =y 0 Z t 0 f[ 1 ;@ x ~ V + 1 ](s;X 1 s ; 1 s )ds + Z t 0 [@ x ~ V[ 1 ] + 1 ](s;X 1 s ; 1 s )dB s (5.17) 128 Here we make a particular choice for the Lipschitz extension of @ x ~ V. Namely, we set @ x ~ V(t;x;y) := @ x ~ V(t;x; ~ (t;x;y)) in V q tb (t;x), which has only 2q larger Lipschitz constant. The reason is, then the martingale term ofr(t;X 1 t ; 1 t ) vanishes, that is, d~ r(t;X 1 t ; 1 t ) = [(~ n T 1 )[] +b 1 ~ r](t;X 1 t ; 1 t )dt; 1 (t;w;y) := (t;X 1 t ; 1 ;y; 1 ) (5.18) where (t;x;a;y;) :=(L +f[@ x ~ V +])(t;x;a;y) Hereb 1 ; are bounded. Now, by using the uniform continuity, choose = " 0 such that j(~ n T )[](t;x;a;y;) (~ n T )[](t 0 ;x 0 ;a;y 0 ;)j" 0 (5.19) whenever maxfjtt 0 j;jxx 0 j;jyy 0 jg< " 0 . Let 0 = 0 and introduce the stopping time 1 := infft> 0 : maxfjt 0 j;jX 1 t X 1 0 j;j 1 t 1 0 jg> " 0 g^T q (5.20) Sincer(0;x 0 ;y 0 ) = 0, by Grönwall inequality, (5.19), and (5.20), (5.18) implies jrj( 1 ;X 1; 1 )C Z 1 0 (~ n T 1 )[](t;X 1 t ; 1 t )dtC" 0 1 (5.21) almost surely. In particular, if" 0 < q=(2CT ), 1 1 does not escape ~ V q tb ( 1 ;X 1 1 ) and (5.17) has a strong solution in [0; 1 ]. Now, given ( i i ; i i ;X i i ; i i ) for 1i, we can repeat the same construction. Namely, invoke measurable selection to havea i ; i inL 1 0; i satisfying " 0 < ~ n L i +f[@ x ~ V + i ] ( i ;X i i ;a i ; ~ ( i ;X i i ; i i )) 0; 129 Set i+1 (t;w) = i (w) and deneX i+1 ; i+1 ; i+1 ; i+1 similarly as in (5.15),(5.16), (5.17), (5.20). Now, similar to (5.21), r( i+1 ;X i+1; i+1 ) = i+1 X k=1 Z k k1 [(~ n T k )[] +b k ~ r](t;X k t ; k t )dt jrj( i+1 ;X i+1; i+1 )C i+1 X k=1 Z k k1 (~ n T k )[](t;X k t ; k t )dtC" 0 i+1 Note that i increases to T q almost surely, otherwise it would have an accumulation point with some positive probability contradicting with the continuity ofX t or t . Dene " 0 ;q (t;w) := 1 X i=1 i (t;w)1 f i t< i+1 g ; " 0 ;q (t;w;y) := 1 X i=1 i (t;w;y)1 f i t< i+1 g and we have the correspondingX " 0 ;q Tq ; " 0 ;q Tq , where " 0 ;q Tq 2V q tb (T q ;X " 0 ;q Tq ). For the non-degenerate case, chooseT q =T and set " 0 ;q := " 0 ;q T 1 f2V(T;X " 0 ;q T )g + ~ (t;X " 0 ;q t ; " 0 ;q T )1 f= 2V(T;X " 0 ;q T )g Since " 0 ;q 2A 0;T , " 0 ;q 2V(T;X " 0 ;q T ), jy 0 Y " 0 ;q ;T; " 0 ;q 0 j 2 =jY " 0 ;q ;T; " 0 ;q T 0 Y " 0 ;q ;T; " 0 ;q 0 j 2 CEj " 0 ;q T " 0 ;q j 2 Cq 2 andq is arbitrary, we concludey 0 2V(0;x 0 ). For the non-degenerate until the terminal case, takeT q <T appropriately and there is no choice but " 0 ;q :=V(T;X " 0 ;q T ) 130 Note that altough we can dene " 0 ;q untilT q ,X " 0 ;q T is well dened. Similarly, jy 0 Y " 0 ;q ;T; " 0 ;q 0 j 2 =jJ(0;x 0 ; " 0 ;q ;T q ; " 0 ;q Tq )J(0;x 0 ; " 0 ;q ;T q ;Y " 0 ;q ;T; " 0 ;q Tq )j 2 CE " 0 ;q Tq " 0 ;q 2 +CE " 0 ;q Y " 0 ;q ;T; " 0 ;q Tq 2 (q +w(TT q )) 2 +C(TT q ) Sinceq is arbitrary andT q !T asq! 0,y 0 2V(0;x 0 ). The last claim is a straightforward application of the Itô’s formula. 5.2 MovingScalarization Given the set valueV and initial parameters (0;x 0 )2 [0;T ]R d , let us consider the problem sup y2V(0;x 0 ) '(y) = sup 2A '(Y 0;x 0 ; 0 ) (5.22) Note that we have a time consistentV, however, this does not immediately imply the construction of a time consistent control. In this section, we will briey discuss the case'(y) = y for some2 R m . Typically, the dynamic problem esssup 2A t;T Y t;X t ; t whereX follows the optimal control in [0;t] is not time consistent. SupposeV takes values in compact convex sets. Then, given there existsy 2V b (0;x 0 ) such thaty achieves (5.22). Furthermore, assume that there exists withy =Y 0;x 0 ; 0 . SetX =X 0;x 0 ; . Then by Lemma 2,Y t;X t ; t 2V b (t;X t ) and by the same observation esssup 2A t;T Y t;X t ; t n(t;X t ;Y 0;x 0 ; t ) =Y t;X t ; t n(t;X t ;Y 0;x 0 ; t ) 131 This concludes that is optimal in a time consistent way forn(t;X t ;Y 0;x 0 ; t ) wheren(0;x 0 ;y ) =. The importance is, even if the agent initially wants to be optimal for the weightening, by computing n(t;X t ;Y 0;x 0 ; t ) to observe what the initial control will be optimal for in the future, agent can better asses whether or not to use the initially optimal control. 5.3 QuadraticExample Here we will note downf = (f 1 ;f 2 ) = f(a;y;z) that depends quadratically on (y;z) and yields a set V(t) which is not convex. Set the terminalV(T ) = 0 and letA = B(0; 1) be the closed unit ball. Set d 0 = 1. f 1 (a 1 ;a 2 ;y 1 ;y 2 ;z 1 ;z 2 ) =a 1 ; f 2 (a 1 ;a 2 ;y 1 ;y 2 ;z 1 ;z 2 ) = 1 2 1 +y 2 1 2y 1 y 2 a 1 + (2 1 +y 2 1 ) 2 a 2 + 2y 1 z 1 z 2 y 2 z 2 1 + 4 y 2 1 y 2 z 2 1 2 1 +y 2 1 We will explicitly verify that solution to (5.1) is given by V b (t) = n (Tt)[cos; (2 1 + (Tt) 2 cos 2 ) sin] :82 [0; 2] o (5.23) Let ^ n(t;y) := (2 1 +y 2 1 )y 1 y 1 y 2 2 (2 1 +y 2 1 ) 2 ; y 2 (2 1 +y 2 1 ) and note thatn(t;y) = ^ n(t;y)=j^ n(t;y)j. Moreover, ^ @ y n(t;y) = 1 j^ n(t;y)j 2 6 6 4 (2 1 +y 2 1 ) + 2y 2 1 2 y 2 2 (2 1 +y 2 1 ) 2 + 8 y 2 1 y 2 2 (2 1 +y 2 1 ) 3 4 y 1 y 2 (2 1 +y 2 1 ) 2 2 y 1 y 2 (2 1 +y 2 1 ) 2 1 (2 1 +y 2 1 ) 3 7 7 5 132 Notice that we do not need to compute ^ @ y j^ n(t;y)j as that terms kernel is the tangent space. Of course, as @ y n(t;y) acts onT(t;y), we do not need to worry about ^ @ y n(t;y) or@ y n(t;y). Any2T(t;y) is of the form (t;y) = 0 (t;y) y 2 (2 1 +y 2 1 ) ; (2 1 +y 2 1 )y 1 y 1 y 2 2 (2 1 +y 2 1 ) 2 ; 0 (t;y)2R It is straighforward to compute2 1 T @ y n +n T f, and the terms that contains turns out to be 2 0 (Tt) 2 2 2 + 2 1 (Tt) 2 cos 2 0 hence supremum over always yields 0. ‡ Therefore, (5.1) becomes sup a2B(0;1) n T (@ t V +f[0;a])(t;y) = 0;8y2V b (t) Recall the denition of@ t V and notice that xing in (5.23) yields one such . So, we can taket derivative directly and project on to the normal direction to get @ t V = 1 (Tt)j^ nj h (2 1 +y 2 1 )y 2 1 + y 2 2 (2 1 +y 2 1 ) i n(t;y) Finally, let us also computen T f withz = 0, n T f = 1 j^ nj a 1 (2 1 +y 2 1 )y 1 +a 2 y 2 which is linear in (a 1 ;a 2 ), hence maximum is on the boundary ofB(0; 1). It is easy to obtain the maximum, and using the fact that (y 2 2 + (2 1 +y 2 1 ) 2 y 2 1 ) 1=2 = (Tt)(2 1 +y 2 1 ) onV b , we concludeV satises the HJB equation. ‡ Since Yt = R T t f(t;Yt;Zt) R T t ZtdBt, for deterministic controls Z = 0 is adapted and hence observing = 0 is expected. This shows optimal controls are in fact deterministic. 133 Let us further present the optimal trajectories. Since@ x V = 0; = 0, martingale term vanishes for the forward SDE (hence ODE). Let us concentrate ony 1 ;y 2 0 and note that sup a2B(0;1) f[0;a](y) = 1 (Tt) y 1 ; 2y 2 1 y 2 (2 1 +y 2 1 ) +y 2 and the optimal trajectory satises y 0 t =y 0 Z t 0 sup a2B(0;1) f[0;a]( y 0 s )ds Solution is given by y 0 t = Tt T y 0 1 ; y 0 2 (2 1 + (y 0 1 ) 2 ) 2 1 + (Tt) 2 (y 0 1 ) 2 T 2 One can further compute@ t n(t; y 0 t ) to observe that@ t n(t; y 0 t )6= 0 for a generaly 0 (except for example y 0 1 = 0 ory 0 2 = 0), hence we have also presented an example for moving scalarization. (Note that whenT is small,V is in fact convex. Otherwise, one needs to choosey 0 as an extremal point to be optimal.) 134 Chapter6 Bibliography [1] Abreu, D.; Pearce, D.; and Stacchetti, E.,TowardaTheoryofDiscountedRepeatedGameswithImperfect Monitoring, Econometrica, 58 (1990), 1041-1063. [2] Aubin, J.-P. and Frankowska, H., Set-Valued Analysis. Birkhauser, Boston, MA, (2009). [3] Bardi, M. and Fischer, M., On non-uniqueness and uniqueness of solutions in nite-horizon Mean Field Games, ESAIM: Control, Optimization and Calculus of Variations, 25 (2019), 44. [4] Barles, G.; Soner H. M.; Souganidis, P. E., Front propagation and phase eld theory, SIAM Journal on Control and Optimization, Vol. 31 No. 2, (1993). [5] Bayraktar, E. and Cohen, A., Analysis of a nite state many player using its master equation, SIAM J. Control Optim. 56 (2018), 3538-3568. [6] Bayraktar, E.; Cecchin, A.; Cohen, A.; and Delarue, F.,FinitestatemeaneldgameswithWright-Fisher common noise, J. Math. Pures et Appliquées, 147 (2021), 98-162. [7] Bayraktar, E.; Cecchin, A.; Cohen, A.; and Delarue, F.,FinitestatemeaneldgameswithWright-Fisher common noise as limits ofN-player weighted games, Mathematics of Operations Research, accepted, arXiv:2012.04845. [8] Bensoussan, A.; Frehse, J.; and Yam, S. C. P., Mean Field Games and Mean Field Type Control Theory, (2013). New York: Springer Verlag. [9] Bouchard, B. and Touzi, N.,Explicitsolutionofthemultivariatesuper-replicationproblemundertrans- action costs., Annals of Applied Probability, 10, 685-708, (2000). [10] Caines, P. E.; Huang, M.; and Malhamé, R. P., Large population stochastic dynamic games: closed loop McKean-VlasovsystemsandtheNashcertaintyequivalenceprinciple,Commun.Inf.Syst., 6 (2006), 221– 251. [11] Cardaliaguet, P., Notes on mean eld games, lectures by P.L. Lions, Collège de France, (2010). [12] Cardaliaguet, P.,Theconvergenceprobleminmeaneldgameswithlocalcoupling,AppliedMathematics & Optimization, 76 (2017), 177-215. [13] Cardaliaguet, P.; Delarue, F.; Lasry, J.M.; and Lions, P.L.,Themasterequationandtheconvergenceprob- lem in mean eld games, Annals of Mathematics Studies, 201. Princeton University Press, Princeton, NJ, (2019). x+212 pp. 135 [14] Carmona, G., Nash Equilibria of Games with a Continuum of Players, Game Theory and Information, 0412009, University Library of Munich, Germany, 2004. [15] Carmona, R. and Delarue, F.ProbabilisticAnalysisofMean-FieldGames,SIAMJournalonControland Optimization, 51 (2013), 2705-2734. [16] Carmona, R. and Delarue, F., Probabilistic theory of mean eld games with applications I - Mean eld FBSDEs, control, and games, Probability Theory and Stochastic Modeling, 83. Springer, Cham, (2018). [17] Carmona, R. and Delarue, F., Probabilistic theory of mean eld games with applications II - Mean eld games with common noise and master equations, Probability Theory and Stochastic Modeling, 84. Springer, Cham, (2018). [18] Cecchin A.; Dai Pra P.; Fischer M.; and Pelino G., On the convergence problem in mean eld games: a two state model without uniqueness, SIAM J. Control Optim. 57 (2019), 2443-2466. [19] Cecchin A. and Delarue F., Selection by vanishing common noise for potential nite state mean eld games, Communications in Partial Dierential Equations, 47 (2022), 89-168. [20] Cecchin, A. and Pelino, G., Convergence, Fluctuations and Large Deviations for nite state Mean Field Games via the Master Equation. Stoch. Process. Appl. 129 (2019), 4510-4555. [21] Chen Y.-G.; Giga Y.; Goto S.,Uniquenessandexistenceofviscositysolutionsofgeneralizedmeancurva- ture ow equations, Journal of Dierential Geometry, 33, 749-786, (1991). [22] Delarue, F., Restoring uniqueness to mean-eld games by randomizing the equilibria, Stochastic and Partial Dierential Equations: Analysis and Computations, 7 (2019), 598-678. [23] Delarue, F. and Foguen Tchuendom, R., Selection of equilibria in a linear quadratic mean-eld game, Stochastic Process. Appl. 130 (2020), 1000-1040. [24] Delarue, F.; Lacker D.; and Ramanan, K., From the master equation to mean eld game limit theory: Large deviations and concentration of measure. Ann. Probab. 48 (2020), 211-263. [25] Delarue, F.; Lacker D.; and Ramanan, K., From the master equation to mean eld game limit theory: A central limit theorem. Electron. J. Probab. 24 (2019), 1-54. [26] Djete, M. F.,Largepopulationgameswithinteractionsthroughcontrolsandcommonnoise: convergence results and equivalence between open-loop and closed-loop controls, preprint, arXiv:2108.02992. [27] Evans L. C. and Spruck J., Motion of level sets by mean curvature. I, Journal of Dierential Geometry, 33, 635-681, (1991). [28] Feinstein, Z., Continuity and Sensitivity Analysis of Parameterized Nash Games. Econ Theory Bull (2022). https://doi.org/10.1007/s40505-022-00228-0. [29] Feinstein, Z.; Rudlo, B.; and Zhang, J.Dynamicsetvaluesfornonzerosumgameswithmultipleequi- libria, Mathematics of Operations Research, 47 (2022), 616-642. [30] Feinstein, Z. and Rudlo, B., Multi-portfolio time consistency for set-valued convex and coherent risk measures, Finance and Stochastic, 19, 67-107, (2015). [31] Feleqi, E., The Derivation of Ergodic Mean Field Game Equations for Several Populations of Players. Dynamic Games and Applications, 3 (2013), 523-536. 136 [32] Fischer, M., On the connection between symmetric N-player games and mean eld games. The Annals of Applied Probability, 27 (2017), 757-810. [33] Fischer, M. and Silva, F.J., On the Asymptotic Nature of First Order Mean Field Games Applied Mathe- matics & Optimization (2020), 1432-0606. [34] Foguen Tchuendom, R., Uniqueness for linear-quardratic mean eld games with common noise, Dy- namic Games and Applications 8 (2018), 199-210. [35] Foote, R. L., Regularity of the distance function, Proceedings of the American Mathematical Society, Volume 92, Number 1, (1984). [36] Furi, M. and Martelli M. A Multidimensional Version of Rolle’s Theorem The American Mathematical Monthly, Vol. 102, No. 3 (1995), 243-249. [37] Gangbo, W. and Mészáros, A.R., Global well-posedness of master equations for determin- istic displacement convex potential mean eld games, Comm. Pure Appl. Math., (2022), https://doi.org/10.1002/cpa.22069. [38] Giga Y., Surface evolution equations: A level set approacy, Monographs in Mathematics, Birkhäuser Basel, (2006). [39] Himmelberg, C. Measurable Relations, Fundamenta Mathematicae, 87 (1975), 53-72. [40] İşeri M. and Zhang J., Set values for mean eld games, ArXiv:2107.01661, 2021. [41] İşeri M. and Zhang J., Set Valued HJB Equations, working paper. [42] Karnam C.; Ma J.; Zhang J.,Dynamicapproachesforsometime-inconsistentoptimizationproblems, The Annals of Applied Probability, Vol. 27 No. 6, (2017). [43] Krantz, S. G. and Parks H. R.,DistancetoC k hypersurfaces, Journal of Dierential Equations, 40 (1981), 116-120. [44] Kunita, H., Stochastic dierential equations and stochastic ows of dieomorphisms, Springer Berlin Heidelberg (1984), 143–303. [45] Lacker, D., A general characterization of the mean eld limit for stochastic dierential games, Probab. Th. Rel. Fields 165 (2016), 581-648. [46] Lacker, D., On the convergence of closed loop Nash equilibria to the mean eld game limit, Ann. Appl. Probab. 30 (2020), 1693-1761. [47] Lacker, D. and Flem, L.L., Closed loop convergence for mean eld games with common noise, preprint, arXiv:2107.03273. [48] Lasry, J.-M. and Lions, P.-L., Mean eld games, Jpn. J. Math. 2 (2007), 229–260. [49] Lauriere, M. and Tangpi, L.Convergenceoflargepopulationgamestomeaneldgameswithinteraction through the controls, SIAM Journal on Mathematical Analysis, 54 (2022), 10.1137/22M1469328. [50] Leobacher G., Steinicke A., Existence, uniqueness and regularity of the projection onto dierentiable manifolds, Annals of Global Analysis and Geometry, 60 (2021), 559-587. [51] Lions, P.-L., Cours au Collège de France, www.college-de-france.fr. 137 [52] Ma, J., Zhang, J., and Zhang, Z. Deep learning methods for set valued PDEs, working paper. [53] Mou, C. and Zhang, J., Wellposedness of second order master equations for mean eld games with non- smooth data, Memoirs of the AMS, accepted, arXiv:1903.09907. [54] Nutz, M.; San Martin, J.; and Tan, X.,ConvergencetotheMeanFieldGameLimit: ACaseStudy,Annals of Applied Probability, 30 (2020), 259-286. [55] O H.; Kim M.-C.; Pak C.-K.,AstochasticGrönwallinequalityinrandomtimehorizonanditsapplication to BSDE, Journal of Inequalities and Applications, 31, (2020). [56] Pham, T. and Zhang, J.,Twopersonzero–sumgameinweakformulationandpathdependentBellman- Isaacs equation. SIAM Journal on Control and Optimization, 52 (2004), 2090-2121. [57] Possamai, D. and Tangpi, L.,Non-asymptoticconvergenceratesformean-eldgames: weakformulation and McKean–Vlasov BSDEs, preprint, arXiv:2105.00484. [58] Sannikov, Y.,GameswithImperfectlyObservableActionsinContinuousTime,Econometrica, 75 (2007), 1285-1329. [59] Sethian J. A., Curvature and the evolution of fronts, Communications in Mathematical Physics, 101, 487-499, (1985). [60] Soner H. M., Motion of a set by the curvature of its boundary, Journal of Dierential Equations, 101, 313-372, (1993). [61] Soner H. M. and Touzi N., Dynamic programming for stochastic target problems and geometric ows, Journal of the European Mathematical Society, 4, 201-236 (2002). [62] Soner H. M. and Touzi N., A stochastic representation for mean curvature type geometric ows, The Annals of Probability, Vol. 31 No. 3, (2003). [63] Zhang, J.,BackwardStochasticDierentialEquations–fromlineartofullynonlineartheory, Probability Theory and Stochastic Modeling 86, Springer, New York, 2017. 138
Abstract (if available)
Abstract
Set valued functions have been introduced for many applications, such as time inconsistent stochastic optimization problems, multivariate dynamic risk measures, stochastic target problems, and nonzero sum games with multiple equilibria.
In this thesis, we study a set valued approach for both multivariate control theory and mean field games with possibly multiple equilibria. For mean field games, we introduce the set of values over all equilibria, which we call the set value of the mean field game, and established two crucial properties: (i) the dynamic programming principle, also called time consistency; and (ii) the convergence of the set values of the corresponding N -player games. To our best knowledge, this is the first work in the literature which studies the dynamic value of mean field games without requiring the uniqueness of mean field equilibria. We emphasize that the set value is very sensitive to the choice of the admissible controls and we illustrate this by investigating three cases, two in finite state space models and the other in a diffusion model.
Time consistency is the underlying property for the PDE approach, which is available in many set valued settings, and we have also shown for the multivariate control theory. That is, we showed that the set of all attainable values, potentially in a higher dimension, is time consistent. Moreover, we extend the calculus to set valued functions, and obtained the crucial Itô’s formula. This opens the door for the PDE approach in the set valued setting. Indeed, we established the well-posedness of Hamilton-Jacobi-Bellman equation for the multivariate control theory.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Conditional mean-fields stochastic differential equation and their application
PDF
Topics on set-valued backward stochastic differential equations
PDF
Pathwise stochastic analysis and related topics
PDF
Dynamic approaches for some time inconsistent problems
PDF
Controlled McKean-Vlasov equations and related topics
PDF
Equilibrium model of limit order book and optimal execution problem
PDF
Zero-sum stochastic differential games in weak formulation and related norms for semi-martingales
PDF
Elements of dynamic programming: theory and application
PDF
New methods for asymmetric error classification and robust Bayesian inference
PDF
Deterministic evolutionary game theory models for the nonlinear dynamics and control of chemotherapeutic resistance
PDF
Sequential Decision Making and Learning in Multi-Agent Networked Systems
PDF
Information design in non-atomic routing games: computation, repeated setting and experiment
PDF
Learning and decision making in networked systems
PDF
Elements of robustness and optimal control for infrastructure networks
PDF
Linear quadratic control, estimation, and tracking for random abstract parabolic systems with application to transdermal alcohol biosensing
PDF
Novel techniques for analysis and control of traffic flow in urban traffic networks
PDF
Correction, coregistration and connectivity analysis of multi-contrast brain MRI
PDF
Machine learning-driven deformation prediction and compensation for additive manufacturing
PDF
Shrinkage methods for big and complex data analysis
PDF
Physics-informed machine learning techniques for the estimation and uncertainty quantification of breath alcohol concentration from transdermal alcohol biosensor data
Asset Metadata
Creator
İşeri, Melih
(author)
Core Title
Set values for mean field games and set valued PDEs
School
College of Letters, Arts and Sciences
Degree
Doctor of Philosophy
Degree Program
Mathematics
Degree Conferral Date
2023-05
Publication Date
05/09/2023
Defense Date
05/09/2023
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
closed loop controls,dynamic programming principle,Hamilton Jacobi Bellman equation,mean field equilibria,mean field games,moving scalarization,multivariate control problems,OAI-PMH Harvest,relaxed controls,set valued Ito's formula,set valued PDEs,set values
Format
theses
(aat)
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Zhang, Jianfeng (
committee chair
), Lototsky, Sergey (
committee member
), Ma, Jin (
committee member
), Tong, Xin (
committee member
)
Creator Email
melih.iseri@gmail.com,melihise@usc.edu
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-oUC113120605
Unique identifier
UC113120605
Identifier
etd-eriMelih-11813.pdf (filename)
Legacy Identifier
etd-eriMelih-11813
Document Type
Dissertation
Format
theses (aat)
Rights
İşeri, Melih
Internet Media Type
application/pdf
Type
texts
Source
20230511-usctheses-batch-1041
(batch),
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright.
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Repository Email
cisadmin@lib.usc.edu
Tags
closed loop controls
dynamic programming principle
Hamilton Jacobi Bellman equation
mean field equilibria
mean field games
moving scalarization
multivariate control problems
relaxed controls
set valued Ito's formula
set valued PDEs
set values