University of Southern California Dissertations and Theses

Controlled McKean-Vlasov equations and related topics

(USC Thesis Other)

Controlled McKean-Vlasov equations and related topics

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content CONTROLLED MCKEAN-VLASOV EQUATIONS AND RELATED TOPICS by Cong Wu A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulllment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (APPLIED MATHEMATICS) August 2017 Copyright 2017 Cong Wu Acknowledgments My deepest gratitude goes to my doctoral advisor, Professor Jianfeng Zhang, whose patient teachings and guidance throughout my study and research have made my academic experi- ence at USC in the last ve years enjoyable and unforgettable. Through our conversations, Professor Zhang has also played the role of being a valuable mentor of my whole life. It is wonderful to learn from a mathematician and a mentor like him. I am also indebted to Professor Jin Ma, whose various supports and advices give me the feel of family. Besides, I want to thank professors Jinchi Lv, Peter Baxendale, Remigijus Mikulevicius for their service in the committee of my qualifying exam and dissertation defense, and professors Igor Kukavica, Larry Goldstein, Aaron Lauda and Richard Arratia for their teachings in the graduate courses I have enrolled. I also owe my special thanks to Rentao Sun, Fei Wang, Jian Wang, Yongjian Kang, Zimu Zhu, Jie Ruan, Pengbin Feng, Giuseppe Martone, Bin Tian, Zheng Dai and other friends within the math department, and past Ph.D.'s like Weisheng Xie, Xiaojing Xing, Chandrasekhar Karnam, Zemin Zheng, Jia Zhuo, Tian Zhang, and my amazing oce mates Daniel Douglas, Jiyeon Park, Michael Earnest, Hyun-Jung Kim and Chukiat Phonsom, whom I have received a lot of help from and had a lof of fun with. I also received a lot of supports from the sta at Department of Mathematics at USC, before I came to the U.S. and during my study here, and they denitely deserve my sincere appreciation. Lastly, it is impossible for me to express my full gratitude to my wife, Mingjin He, whose sel ess love was an indispensable part of my life in the last ve years. I am also greatly indebted to my parents for everything they have done for me in my whole life. ii Table of Contents Acknowledgments ii Chapter 0: Preliminaries 1 0.1 Wasserstein space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 0.2 Skorohod space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Chapter 1: Introduction 4 1.1 Mean eld games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.2 McKean-Vlasov control problem . . . . . . . . . . . . . . . . . . . . . . . . 6 Chapter 2: Stochastic dierential equation of McKean-Vlasov type 8 2.1 Wellposedness of strong solution . . . . . . . . . . . . . . . . . . . . . . . . 8 2.2 Partical approximations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.3 Strong solution of path dependent case . . . . . . . . . . . . . . . . . . . . . 13 2.4 McKean-Vlasov BSDEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Chapter 3: Generalized It^ o's formula 16 3.1 Dierentiation in L 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.2 Dierentiation inP 2 (R d ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.3 It^ o formula:P 2 (R d ) dependence . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.4 It^ o formula: L 2 dependence . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.5 It^ o formula: functional dependence . . . . . . . . . . . . . . . . . . . . . . . 32 Chapter 4: Classical solution of master equation 43 4.1 Representations of derivatives of u . . . . . . . . . . . . . . . . . . . . . . . 43 4.1.1 L 2 -Derivatives of X t; . . . . . . . . . . . . . . . . . . . . . . . . . . 43 4.1.2 Derivatives of value function . . . . . . . . . . . . . . . . . . . . . . 45 4.2 Wellposedness of classical solution . . . . . . . . . . . . . . . . . . . . . . . 53 Chapter 5: Controlled McKean-Vlasov SDEs and master equation 54 5.1 Problem formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 5.1.1 A subtle ltration issue . . . . . . . . . . . . . . . . . . . . . . . . . 55 5.2 Regularity of value function . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 5.2.1 Regularity of b V . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 5.2.2 V = b V . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 iii 5.3 Dynamic programming principle . . . . . . . . . . . . . . . . . . . . . . . . 69 5.4 Master equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 Chapter 6: Viscosity solution of master equation 72 6.1 Denition of viscosity solutions . . . . . . . . . . . . . . . . . . . . . . . . . 72 6.2 Existence of viscosity solutions . . . . . . . . . . . . . . . . . . . . . . . . . 73 6.3 Partial comparison principle . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 6.4 Classical solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 6.5 Remarks on future directions . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Bibliography 79 iv Chapter 0 Preliminaries 0.1 Wasserstein space Given a Polish metric space E, letP(E) denote the set of Borel probability measures on E. Suppose the distance onE is given byd(;), then setP 2 (E) consists of the measures in P(E) such that R d(x;x 0 ) 2 dx<1 for some (and hence for any) x 0 2E. Denition 0.1.1 (Wasserstein metric) For any;2P 2 (E), the Wasserstein distance of order 2 is dened as W 2 (;) = inf 2(;) Z d(x;y) 2 (dx;dy) 1 2 ; (0.1) where (;) is the set of joint probability measures on EE with marginals equal to and . We say is a coupling of and if2 (;) andP 2 (E) is called the Wasserstein space of order 2. Proposition 0.1.2 (Properties of Wasserstein space, see [5], [9]) (i) W 2 metrizes weak convergence in P 2 (E): k ! under W 2 i k ) and R d(x 0 ;x) 2 k (dx)! R d(x 0 ;x) 2 (dx). (ii) (P 2 (E);W 2 ) is a Polish space if (E;d) is Polish.P 2 (E) is compact if E is compact. (iii) For any ;2P 2 (E), inmum in the denition of W 2 (;) is achieved by at least one coupling 2 (;), i.e. optimal couplings always exist. 1 (iv) Suppose E =R d , and d is the Euclidean distance. If is absolutely continuous with respect to the Lebesgue measure, then there exists a convex map :R d !R such that (Id R d;D)# is the unique optimal coupling for W 2 (;) and hence = D#. Conversely, if convex map : R d ! R satises = D#, then (Id R d;D)# is the unique optimal coupling for ;. Remark 0.1.3 Note that Wasserstein metric and space can be dened for any order p2 [1;1). Results (i)-(iii) of the above proposition still hold in this case. Remark 0.1.4 It is clear that for any given random variablesX;Y , we haveW 2 (L X ;L Y ) kd(X;Y )k 2 . 0.2 Skorohod space The results in this section are from Section 12 of [2]. Let D[0;T ] (or simply D) denote the set of cadlag functions (i.e. right continuous with left limits) over time interval [0;T ]. For a function ! on [0;T ] and I [0;T ], let m ! (I) := sup s;t2I j! s ! t j. The following result gives the uniform property of cadlag functions. Lemma 0.2.1 For every x2D and "> 0, there exists 0 =t 0 <t 1 <<t n =T so that m ! ([t i ;t i+1 ))<" for i = 0; 1; ;n 1. We have the following corollary from this lemma. Corollary 0.2.2 Let !2D. (i) There are at most nitely many points t at which the jumpj! t ! t j exceeds a given positive number, hence ! has at most countably many discontinuities; (ii) ! is bounded; (iii) ! can be uniformly approximated by cadlag, piecewise constant functions, so it is Borel measurable. 2 Letk!k := sup t2[0;T ] j! t j and be the set of strictly increasing, continuous bijections from [0;T ] to itself, then we can dene Skorohod metric on D: d SK (!;! 0 ) := inf 2 maxfkIdk;k!! 0 kg; (0.2) where Id is the identity mapping from [0;T ] to itself. It can be shown that d SK is indeed a metric in D, so (D;d SK ) is called the Skorohod space and the topology generated by d SK is called Skorohod topology. Theorem 0.2.3 (Property of space D) (i) (D;d SK ) is separable. (ii) D is not complete under d SK , but there exists an equivalent metric d 0 SK such that D is complete under d 0 SK . (iii) The space C of continuous functions endowed with uniform metric is a subspace of (D;d SK ). In particular, if ! n converges to a continuous function ! under d SK , then ! n converges to ! uniformly. Remark 0.2.4 If we endow D with the uniform normkk, then we get a Banach space, however this uniform topology is nonseparable and hence too strong. 3 Chapter 1 Introduction The topic of this research was initially motivated by the fast growing eld of mean eld game (MFG) theory. MFG theory was developed independently by J.-M. Lasry and P.-L. Lions in their papers [16], [17], [18], and Huang-Cainses-Malham e [13], around the same time. The term "mean eld" was inherited from the so called mean eld models in physics, which studies the behaviour of systems with many identical particles that are interacting with each other. As a result of Snitzman [21], when the number of particles goes to innity, all particles behave like they are independent and their distribution is described by a common SDE of McKean-Vlasov type, where the law of the state process comes into play in the drift and volatility. The main objective of this research is to provide some answers to control problems of McKean-Vlasov dynamics (or called mean eld control problems), which is closely related to the MFG problem. To solve a classical stochastic control problems, the usual strategy is to rst study the characterization of the value function V (t;x) through the HJB equation, and then solve the optimization problem associated with each possible time-state pair (t;x). The connection between value function and HJB equation relies crucially on Dynamic Pro- gramming Principle, so the HJB equation is also sometimes called dynamic programming equation. We argue that in the case of mean eld control problems, as will be pointed out in later chapters, even though some special subtleties do exist, weak formulation turns out to be the right framework. 4 1.1 Mean eld games In mean eld games, players are assumed to be similar in behaviors since it is usually assumed that the number of players is large and no single player can exert a big impact on the whole group. Intuitively speaking, the dynamics between dierent players are related to each other through statistical data of the whole group. Mathematically speaking, suppose there are N players, then their dynamics, denoted as X i , are given by dX i t =b(t;X i t ; N t ; i t )dt +(t;X i t ; N t ; i t )dB i t (1.1) where N t = 1 N P N i=1 X i t and i is the controll process for playeri. The players are thought of as similar since functions b; are not indexed by i. If the objectives of dierent players are also assumed similar, say, player i needs to minimize his costs J i ( 1 ; ; N ) = Z T 0 f(t;X i t ; N t ; i t )dt +g(X i T ; N T ) (1.2) Due to symmetry, it's reasonable to search for a symmetric Nash equilibrium of the game. When such a Nash equilibrium exists, the statistical distribution of the dynamics of each player should be the same and as the number of players is large, their empirical distribution should be nearly independent of the control of any single player. This provides the insight of how to solve the mean eld game problems. As the rst step, we view the measures f t g 0tT as xed and try to solve a standard control problem: minimize objective E Z T 0 f(t;X t ; t ; t )dt +g(X T ; T ) (1.3) subjective to constraint dX t =b(t;X t ; t ; t )dt +(t;X t ; t ; t )dB t : (1.4) 5 In the next step, we should choosef t g 0tT being equal to the limit of empirical measures N t , which is again equal to the marginal distributions of X by some form of law of large numbers, see Section 2.2. This two-step strategy becomes solving a xed point problem. In the existing literature, process X is viewed as the dynamics of the representative agent of the game. To justify the validity of this two-step strategy, it has been rigorously proved in [6], under some conditions, that if a solution has been found in the above two-step process and the optimal control is of feedback form ? t (X t ; t ), then ( ? t (X 1 t ; t ); ; ? t (X N t ; t )) will provide an approximate Nash equilibrium of the N-player mean eld game. 1.2 McKean-Vlasov control problem In this section, we introduce the control problem of McKean-Vlasov SDEs dX t =b(t;X t ;L(X t ); t )dt +(t;X t ;L(X t ); t )dB t (1.5) This problem is closed related to the mean eld game problem. Indeed, as we will see in Section , (1.5) could be viewed as the limit of (1.1), i.e., when N is large, the dynamics X i of player i should be close to dynamics X. Therefore, each player is faced with the same optimization problem: minimize objective E Z T 0 f(t;X t ;L(X t ); t )dt +g(X T ;L(X T )) ; (1.6) subject to (1.5). Such a control problem is not standard anymore. It has been shown that the optimal control to this non-standard problem also gives equilibriums for N-player games. For the precise statements of results in this direction, see [7]. All the existing results in the literature have been focused on the setting of strong formulation, that is, the optimal control is searched throughout the set of open loop controls. In my thesis, the main topic is to solve this control problem under the weak formulation, that is, the controls we will 6 search through are automatically of closed loop form, which we believe is more natural in applications. 7 Chapter 2 Stochastic dierential equation of McKean-Vlasov type Given a ltered probability space ( ;F;F;P) equipped with F-Brownian motion B and T > 0, let (b;) : [0;T ]R d P(R d ) !R d R dd . For any2F 0 that is independent of B, we are interested in the property of the solution of equation dX t =b(t;X t ;L(X t ))dt +(t;X t ;L(X t ))dB t ; X 0 = (2.1) 2.1 Wellposedness of strong solution We will show that equation (2.1) admits a unique solution under the following usual assump- tions. Assumption 2.1.1 (i) 2L 2 (F 0 );b(; 0; 0 )2H 2;d (F);(; 0; 0 )2H 2;dd (F); (ii) 9L> 0, such that,8t2 [0;T ];x;y2R d ;;2P 2 (R d ), jb(t;x;)b(t;y;)j +j(t;x;)(t;y;)jL(jxyj +W 2 (;)): (2.2) Theorem 2.1.2 Under Assumption 2.1.1, equation (2.1) admits a unique solution in H 2;d (F), which satises estimate E sup 0tT jX t j 2 cI 2 0 (2.3) 8 where c is a constant that depends only on L;T and I 2 0 := E[jj 2 + ( R T 0 jb(t; 0; 0 )jdt) 2 + R T 0 j(t; 0; 0 )j 2 dt]. If b; are deterministic functions, then this unique solution is a strong solution in the sense that X is adapted to ltration F B _(). Proof First we prove uniqueness, suppose X; e X are two solutions of (2.1) in H 2;d . By taking the dierence, we have jX t e X t j 2 2 Z t 0 b(s;X s ;L Xs )b(s; e X s ;L e Xs )ds 2 +2 Z t 0 (s;X s ;L Xs )(s; e X s ;L e Xs )dB s 2 so sup 0st jX s e X s j 2 C Z t 0 jb(s;X s ;L Xs )b(s; e X s ;L e Xs )j 2 ds +2 sup 0st Z s 0 (r;X r ;L Xr )(r; e X r ;L e Xr )dB r 2 Note that the stochastic integral in the second term on the right hand side is a martingale due to Assumption 2.1.1. Then by BDG inequality, we can get E sup 0st jX s e X s j 2 CE Z t 0 jX s e X s j 2 +W 2 (L Xs ;L e Xs ) 2 ds +2E sup 0st Z s 0 (r;X r ;L Xr )(r; e X r ;L e Xr )dB r 2 CE Z t 0 jX s e X s j 2 +W 2 (L Xs ;L e Xs ) 2 ds +CE Z t 0 j(s;X s ;L Xs )(s; e X s ;L e Xs )j 2 ds CE Z t 0 jX s e X s j 2 +W 2 (L Xs ;L e Xs ) 2 ds C Z t 0 E sup 0rs jX r e X r j 2 ds Then it follows from Gronwall's inequality that X; e X are indistinguishable. 9 To prove existence, dene a sequence of processesfX k g k0 by X 0 , and Picard iteration X k+1 t = + Z t 0 b(s;X k s ;L X k s )ds + Z t 0 (s;X k s ;L X k s )dB s : (2.4) By BDG inequality and Lipschitz continuity of b;, it is easy to derive E sup 0st jX k+1 s X k s j 2 C Z t 0 E sup 0rs jX k r X k1 r j 2 ds; (2.5) for some constant C which depends only on L;T . We can show by induction that this inequality further implies that for all k, E sup 0st jX k+1 s X k s j 2 (Ct) k k! E sup 0st jX 1 s X 0 s j 2 ; 8 0tT: (2.6) This is trivial if k = 0. Now let's assume the above inequality holds when k = i, then by (2.4), we have E sup 0st jX i+2 s X i+1 s j 2 C Z t 0 E sup 0rs jX i+1 r X i r j 2 ds C Z t 0 (Cs) i i! dsE sup 0st jX 1 s X 0 s j 2 C i+1 Z t 0 s i i! dsE sup 0st jX 1 s X 0 s j 2 (Ct) i+1 (i + 1)! E sup 0st jX 1 s X 0 s j 2 : So (2.6) is proved. By Borel-Cantelli lemma, we have P sup 0sT jX k+1 s X k s j> 1 2 k i.o. = 0: (2.7) 10 HencefX n g converges uniformly on [0;T ] to a continuous processX almost surely. To show the limit X is indeed a solution of (2.1), note thatfX n g is Cauchy under normkk 1;2 , so E sup 0tT jX n t X t j 2 = E sup 0tT lim m!1 jX n t X m t j 2 E lim m!1 sup 0tT jX n t X m t j 2 lim m!1 E sup 0tT jX n t X m t j 2 ! 0 as n!1 Then it follows from Assumption 2.1.1 that Z t 0 b(s;X n s ;L X n s )ds! Z t 0 b(s;X s ;L Xs )ds almost surely and Z t 0 (s;X n s ;L X n s )dB s ! Z t 0 (s;X s ;L Xs )dB s in probability. Thus X is a solution of (2.1) by taking k!1 in (2.4). If b; are deter- ministic, then it is clear from its denition that X n is adapted to F B _(), hence so is X. To get estimate (2.16), note that kX n X 0 k 1;2 n1 X k=0 kX k+1 X k k 1;2 n1 X k=0 (CT ) k k! 1 2 kX 1 X 0 k 1;2 ckX 1 X 0 k 1;2 : Letting n!1, kXX 0 k 1;2 ckX 1 X 0 k 1;2 : 11 Since kX 1 X 0 k 2 1;2 cE jj 2 + Z 0 jb(s;;L )jds 2 + Z 0 j(s;;L )j 2 ds cI 2 0 ; we have proved (2.16). We remark that Theorem 2.1.2 remains true if (2.1) becomes dX t =b(t;X t ;X t ())dt +(t;X t ;X t ())dB t ; X 0 = (2.8) and (b;) : [0;T ]R d L 2 ( ) !R d R dd satises Assumption 2.1.3 (i) 2L 2 (F 0 );b(; 0; 0)2H 2;d (F);(; 0; 0)2H 2;dd (F); (ii) 9L> 0, such that,8t2 [0;T ];x;y2R d ;;2L 2 ( ), jb(t;x;)b(t;y;)j +j(t;x;)(t;y;)jL(jxyj +kk 2 ): (2.9) 2.2 Partical approximations In this section, we will prove that the solution to MKV SDE can be approximated pathwise by N-particle systems when N is large. Suppose ( i ;F i ;P i ) is a sequence of independent probability spaces. Let ( i ;W i ) be independent copies of each other which are dened on i , and process X i be the solution to dX i t =b(t;X i t ;L(X i t ))dt +(t;X i t ;L(X i t ))dB i t ; X i 0 = i (2.10) For each N, letfX i;N g 1iN be the solution to particle system dX i;N t =b(t;X i;N t ; N t )dt +(t;X i;N t ; N t )dB i t ; X i;N 0 = i (2.11) 12 with N t being the empirical measure 1 N P N i=1 X i;n t . So the particles X i;N are interacting with each other through the empirical measure and clear they have common distribution by symmetry of the particle system. Note that (2.11) is usually a system of classical SDEs. For example, for the case of mean-eld interaction b(t;x;) = R ~ b(t;x; ~ x)(d~ x), the corresponding termb(t;X i;N t ; N t ) of (2.11) becomes 1 N P N j=1 ~ b(t;X i;N t ;X j;N t ). The following theorem describes how the particles X i;N can provide an approximation to the solution of MKV SDEs (2.10), see [8]. Theorem 2.2.1 Under the assumption of Theorem 2.1.2, we have lim N!1 sup 1iN E sup 0tT jX i;N t X i t j 2 = 0: (2.12) The theorem says that the law of (X 1;N ; ;X k;N ) converges weakly to law of (X 1 ; ;X k ) for any xedk asN!1, so the dependence between particles (X 1;N ; ;X k;N ) becomes weak as N is large. 2.3 Strong solution of path dependent case The same argument in Theorem 2.1.2 of previous section can actually be applied to prove a more general result, i.e. we can prove wellposedness results in the path depent case. Let's assume (b;) : [0;T ]C([0;T ];R d )P(C([0;T ];R d )) ! R d R dd , and (2.1) then becomes dX t =b(t;X ^t ;L X^t )dt +(t;X ^t ;L X^t )dB t ; X 0 = (2.13) We assume the space C([0;T ];R d ) is endowed with supremum norm (or uniform norm), then the corresponding Wasserstein metric is dened as W 2 (;) := inf 2(;) Z sup 0tT j! t e ! t j 2 (d!;de !) 1 2 : (2.14) Note that W 2 (L X ;L e X )kX e Xk 1;2 . Similarly, Assumption 2.1.1 now becomes 13 Assumption 2.3.1 (i) 2L 2 (F 0 );b(; 0; 0 )2H 2;d (F);(; 0; 0 )2H 2;dd (F); (ii) 9L> 0, such that,8t2 [0;T ];!;e !2C([0;T ];R d );;2P 2 (C([0;T ];R d )), jb(t;!;)b(t;e !;)j +j(t;!;)(t;e !;)jL(j!e !j 1 +W 2 (;)): (2.15) Then we have Theorem 2.3.2 Under Assumption 2.3.1, equation (2.13) admits a unique strong solution in H 2;d , which satises estimate E sup 0tT jX t j 2 cI 2 0 (2.16) where c is a constant that depends only on L;T and I 2 0 := E[jj 2 + ( R T 0 jb(t; 0; 0 )jdt) 2 + R T 0 j(t; 0; 0 )j 2 dt]. If b; are deterministic, this solution is strong. 2.4 McKean-Vlasov BSDEs In this section, we proof the wellposedness of the following McKean-Vlasov BSDE dY t =E[f(t; ~ Y t ; ~ Z t ;Y t ;Z t )]dt +Z t dB t ; Y T = (2.17) where f(t;!; ~ y; ~ z;y;z) : [0;T ] R n R mn R n R mn ! R n and B is a Brownian motion on ltered probability space ( ;F;P). Assumption 2.4.1 (i) For any ~ y; ~ z;y;z, (t;!)7!f(t;!; ~ y; ~ z;y;z) is F-progressively measurable; (ii) There exist C > 0, so that for all ~ y 1 ; ~ y 2 ; ~ z 1 ; ~ z 2 ;y 1 ;y 2 ;z 1 ;z 2 , that jf(t; ~ y 1 ; ~ z 1 ;y 1 ;z 1 )f(t; ~ y 2 ; ~ z 2 ;y 2 ;z 2 )jC(j~ y 1 ~ y 2 j +j~ z 1 ~ z 2 j +jy 1 y 2 j +jz 1 z 2 j); 14 (iii) f(; 0; 0; 0; 0)2H 2;n . Theorem 2.4.2 If f satises Assumption 2.4.1, then for any square integrable 2F T , equation (2.17) admits a unique adapted solution (Y;Z)2S 2;n H 2;mn . This result is not used in this thesis but we list it here for completeness. For a proof of this result, see [8], Section 2.2. 15 Chapter 3 Generalized It^ o's formula Suppose ( ;F;P) is a ltered probability space and X t is a continuous semimartingale, the classical It^ o's formula in stochastic calculus says that f(t;X t ) is again a continuous semimartingale if function [0;T ]R d 3 (t;x)7!f(t;x)2R d belongs toC 1;2 . In particular, ifX has decompositionX t =M t +A t , whereM is a continuous local martingale andA has nite variation, then f(t;X t ) = f M t + e A t with f M t = f(0;X 0 ) + Z t 0 r x f(s;X s )dM s e A t = Z t 0 @ t f(s;X s )ds + Z t 0 r x f(s;X s )dA s + 1 2 Z t 0 r 2 x f(s;X s ) :dhMi s In order to investigate problems of McKean-Vlasov type, we want to extend this result to functions f dened on [0;T ]P 2 (D[0;T ]). 3.1 Dierentiation in L 2 Let ( ;F;P) be a probability measure space. Denition 3.1.1 Let f :L 2 ( )!R. (i) We say f2C 1 if f is Frechet dierentiable and Df :L 2 ( )!L 2 ( );7!Df() is continuous. (ii) We say f2C 2 if f2C 1 and second order derivative D 2 f()f 1 ; 2 g = lim !0 E[Df( + 1 ) 2 Df() 2 ] . exists for any; 1 ; 2 2L 2 ( ) and functionD 2 f()f;g : (L 2 ( )) 3 !R is continuous. 16 (iii) We say f2C 2 b if Df;D 2 f are also bounded. Remark 3.1.2 Unlike the usual notation, here f2C 2 doesn't imply that the second order Frechet derivative D 2 f of f exists. Actually, it only implies that second order Geateaux derivative exists. The motivation of above denition is that existence of second order Frechet derivative is a very strong restriction. The following example shows that D 2 f doesn't exist even for very "good" functions, see Example 2.3 of [4]. Example 3.1.3 Letf() =E[sin], thenDf() = cos. If second order Fr echet derivative D 2 f exists, then we would have D 2 f()f 1 ; 2 g = E[(sin) 1 2 ]. Let 1 A be independent of , then the remainder of Taylor expansion is R = f( + 1 A )f()E[Df()1 A ] 1 2 D 2 f()f1 A ; 1 A g = E h sin( + 1 A ) sin cos 1 A + 1 2 sin 1 2 A i = E h sin( + 1) 1 2 sin cos i P(A) which is of order O(P(A)) = O(k1 A k 2 L 2 ), not o(k1 A k 2 L 2 ) as P(A)! 0, so f is not twice Fr echet dierentiable at . Fortunately, the second order Gaeteaux derivative is still a bilinear form, as expected, if f is C 2 as in Denition 3.1.1. Proposition 3.1.4 (Section 3.5, [12]) If f :L 2 ( )!R is C 2 in the sense of Denition 3.1.1, then for any , D 2 f()f 1 ; 2 g is symmetric and bilinear in 1 and 2 . 3.2 Dierentiation inP 2 (R d ) We can study dierentiability of a function f :P 2 (R d )! R by lifting it to a function F onL 2 ( ) such that F () =f(L ), for any xed probability space ( ;F;P). If F is Fr echet dierentiable, then DF () could be identied as an element of L 2 ( ). Moreover, we will show that DF ()2() if ( ;F;P) is a rich enough Polish space. 17 Lemma 3.2.1 Let ( ;F;P) be any probability space and is a random variable. If A2F satises E[jA 1 ] =E[jA 2 ]; 8A 1 ;A 2 A with positive measure then is constant in A. Proof SupposeP(A)> 0. LetA 1 =f>E[jA]g\A;A 2 =f<E[jA]g\A. If is not constant in A, we have P(A 1 )> 0;P(A 2 )> 0, then E[jA]<E[jA 1 ] =E[jA 2 ]<E[jA], contradiction. Lemma 3.2.2 Let ( ;F;P) be any probability space that supports a random variable U U(0; 1) and is a random variable. If A2F satises E[1 A 1 ] =E[1 A 2 ]; 8A 1 ;A 2 A with P(A 1 ) =P(A 2 ) then is constant in A. Proof Suppose P(A) > 0. Let A 1 =f > E[jA]g\A;A 2 =f < E[jA]g\A. If is not constant in A, we have P(A 1 ) > 0;P(A 2 ) > 0. Suppose P(A 1 ) P(A 2 ). Since (x) := P(A 1 \fU < xg) is continuous, there exists x 0 such that (x 0 ) = P(A 2 ). Let A 0 1 :=A 1 \fU <x 0 g, thenP(A 0 1 ) =P(A 2 )> 0, but E[jA]P(A 0 1 )<E[1 A 0 1 ] =E[1 A 2 ]< E[jA]P(A 2 ), contradiction. Remark 3.2.3 Given ( ;F;P), if there exists a random variable U U(0; 1) on it, then ( ;F;P) must be atomless. Moreover, since any distribution inP(R) can be realized by a random variable on ([0; 1]; B([0; 1]);L) as its distribution using inverse transform (where B([0; 1]) is the completed-algebra ofB([0; 1]) with respect to Lebesgue measureL), they can also be realized by a random variable on ( ;F;P). In this sense, we can say that ( ;F;P) is also rich enough. Remark 3.2.4 The probability space ([0; 1]; B([0; 1]);L) is indeed a rich enough space. This may seem counterintuitive at rst sight because U : [0; 1] ! R;x 7! x is U(0; 1) and 18 generates the sigma algebraB([0; 1]). In other words, any random variable 2 (U), and hence we cannot nd any other non-constant random variable that is independent from U. Nevertheless, we can still construct a sequence of i.i.d. U(0; 1) random variables on ([0; 1]; B([0; 1]);L), which is usually what we need when we want a rich enough probability space. This result is from the theory of standard probability spaces, see [Rokhlin 1952]. Standard probability spaces are dened as spaces that are isomorphic modulo 0 to (i) unit interval with Lebesgue measure, (ii) a nite or countable set of atoms, or (iii) a combination of both. It has been proved that every probability measure on every Polish space turns that space into a standard probability space. Since R 1 (with product topology) is Polish, for every probability measure on it, by isomorphism, we can construct a sequence of random variables on ([0; 1]; B([0; 1]);L) such that their joint distribution is equal to . It is even possible to construct a Brownian motion as a measurable function from [0; 1] to C[0;1), see [1], Theorem 37.1 or Problem 37.2. It follows from the previous two remarks that for any given probability space ( ;F;P), saying that it is rich enough is equivalent to saying that it supports a U(0; 1) random variable. If is Polish, they are also equivalent to saying that it is atomless by above remark. See [22], Theorem 3.4.23 (the isomorphism theorem for measure spaces). Theorem 3.2.5 Let f :P 2 (R d )!R and ( ;F;P) be a rich enough Polish space. Dene the lifting F :L 2 ( )!R off asF () =f(L ). If F is continuously Fr echet dierentiable, then there exists a deterministic function g :P 2 (R d )R d !R d such that DF () =g(L ;);P-a.s.82L 2 ( ): (3.1) Moreover, function g is independent of the choice of space ( ;F;P). Proof This result has been proved previously by Lions in [5] by using some advanced techniques, but we want to provide a more elementary proof here. First, let's suppose that can only take countably many values, i.e. = P m i=1 c i 1 E i (m can be innity), where E i 's are disjoint. Then, for any A 1 ;A 2 E i for some i, and 19 P(A 1 ) =P(A 2 ), we can easily verify that +c1 A 1 (d) = +c1 A 2 , for any c2R d . Therefore, we have E[DF () k 1 A 1 ] = lim !0 F ( +1 A 1 e k )F () = lim !0 F ( +1 A 2 e k )F () = E[DF () k 1 A 2 ] where DF () k is the k-th component of DF (). Thus, by Lemma 3.2.2, DF () is constant on each E i and hence belongs to (). For an arbitrary 2 L 2 ( ), we can dene n := P 1 i=1 i 2 n 1 [i2 n ;(i+1)2 n ) (), then clearlyk n k 2 ! 0. Since DF is continuous,kDF ( n )DF ()k 2 ! 0. Thus we have proved DF ()2 () because DF ( n )2 ( n ) () and () is closed. Hence, there exists Q 2L 2 L (R d ;R d ) such that DF () =Q ();P-a.s. . To show that function Q only depends on the law =L , note that in the discrete case, suppose = P m i=1 c i 1 E i ;DF () = P m i=1 q i 1 f=c i g , then we have q i;k = E[DF () k 1 E i ] P(E i ) = lim !0 F ( +1 f=c i g e k )F () P( =c i ) Thus, it is clear that q i only depends on the law of . If is continuous, let e be another random variable with the same law , and n ; e n be the discretizations of ; e , as dened before. From the argument in the previous case, we can see that there exist common functions g n :P 2 (R d )R d ! R d such that DF ( n ) = g n (;);DF ( e n ) = g n (; e );P-a.s. Since DF ( n )! DF () and DF ( e n )! DF ( e ) in L 2 , there exists a common function g such thatg(;) =DF ();g(; e ) =DF ( e );P-a.s. Since g(;) is independent of the choices of; e , we have shown thatDF () =g(L ;);P-a.s. ;8. 20 Finally, we want to prove that g is also independent of the choice of space ( ;F;P). Suppose ( e ; e F; e P) is another rich enough Polish space and the lifting of f to it is denoted as e F . Let (;)2L 2 ( ) and ( e ;e )2L 2 ( e ) are such thatL (;) =L ( e ;e ) , then lim !0 e F ( e +e ) e F ( e ) = lim !0 f(L e +e )f(L e ) = lim !0 F ( +)F () = E[DF ()] =E[g(L ;)] = e E[g(L e ; e )e ] so D e F ( e ) exists and is equal to g(L e ; e ). It is also easy to see that e F is also continuously Fr echet dierentiable. The proof is now complete. Remark 3.2.6 Usually we don't need to verify directly that F :L 2 ( )!R is continuously Fr echet dierentiable before applying Theorem 3.2.5. As we have seen in the proof, we only need to verify that (i) for all2L 2 ( ), there is another element ofL 2 ( ), which we denote as DF (), such that the limit lim !0 F ( +)F () exists and is equal to E[DF ()] for all 2 L 2 ( ), and (ii) the mapping 7! DF () is continuous. But it can be shown that these two conditions indeed imply the continuous Fr echet dierentiability of F . Actually, it should be noted that the function g in Theorem 3.2.5 is not uniquely deter- mined by f. In fact, let e g be any other function such that for all 2P 2 (R d ), e g(;) = g(;);-a.e. on the support supp() of , then clearly we also have DF () = e g(L ;). Let's denote the set of such functions g's asD f (note this set is independent of or F ). The following two results on regularity of derivatives were proved by Carmona and Delarue in [7]. Proposition 3.2.7 ([7], Lemma 3.1) Under the assumption of Theorem 3.2.5, there exists a function g 2 D f such that g(;x) is jointly measurable in (;x). Moreover, g(;) is continuous in supp() whenever it has a continuous version in supp(). 21 Proposition 3.2.8 ([7], Lemma 3.3) If DF : L 2 ( )! L 2 ( ) is Lipschitz continuous, then exists a function g2D f such that g(;) is Lipschitz continuous in R d for all 2 P 2 (R d ) with the same Lipschitz constant equal to that of DF . Note that last proposition only shows the partial continuity of derivative g in x. The following result addresses the question of uniqueness of a jointly continuous derivative g in D f. Proposition 3.2.9 Let F : L 2 ( )! R be the same as in Theorem 3.2.5, then set D f contains at most one continuous function. Proof Suppose there are two continuous functions g;e g :P 2 (R d )R d !R d inD f. If has (strictly) positive density (with respect to Lebesgue measureL), then clearly g(;) equals to e g(;) almost everywhere, and hence there are equal everywhere by continuity. Otherwise, there is a Borel set S such thatL(S) > 0 and g(;)6=e g(;) on S. Suppose L =, thenE[jg(;)e g(;)j1 S ()] = R S jg(;x)e g(;x)jd(x)> 0, which contradicts with g(;) =e g(;) =DF ();P-a.s. For a general 2P 2 (R d ), since is rich enough, we can nd two independent random variables ;W such thatL = , and W N(0; 1). Dene n := + W n ; n :=L n , then W 2 ( n ;)k n k 2 ! 0, hencekDF ( n )DF ()k 2 ! 0 by continuity of DF . Since ?W , n has positive density, so we can get g(;x) = lim n!1 g( n ;x) = lim n!1 e g( n ;x) =e g(;x); 8x2R d : The proof is now complete. We remark that the results of this proposition has been observed earlier, see Remark 2.1 of [4]. Here we provide another proof which can be adapted to functional case, see Section 3.5 below. Second proof Letfx i g i2N be a countable dense subset ofR d . If = P i p i x i withp i > 0, then the value g(;x i ) is unique for every x i . By continuity in x, function g(;) is unique 22 inR d . It is not hard to show that the set of all measures 2P 2 (R d ) of the form P i p i x i with p i > 0 form a dense subset ofP 2 (R d ). By continuity in , g is unique. Open question: Under what conditions on f doesD f contain a jointly continous func- tion? Denition 3.2.10 Let f :P 2 (R d )!R. (i) If there exists a function g :P 2 (R d )R d !R d such that (3.1) holds, then we say f is dierentiable (everywhere) and we call each function inD f a derivative of f. (ii) If there is a unique continuous function in D f, then we say f is continuously dierentiable, or f2C 1 in short, and call the continuous function the derivative of f, and write it as @ f. By lifting the rst parameter of (;x)7!@ f(;x) toL 2 ( ), we can similarly dene higher order derivatives @ x @ f :P 2 (R d )R d !R dd ; (;x)7!@ x @ f(;x) @ 2 f :P 2 (R d )R d R d !R d ; (;x;y)7!@ 2 f(;x;y) These derivatives together determines the second order Gateaux derivative D 2 F ()f;g : (L 2 ( )) 3 !R. D 2 F ()f 1 ; 2 g = lim !0 E[DF ( + 1 ) 2 DF () 2 ] . = lim !0 E[@ f(L + 1 ; + 1 ) 2 @ f(L ;) 2 ] . = E e E @ 2 f(L ;; e )e 1 2 +E @ x @ f(L ;) 1 2 (3.2) Denition 3.2.11 Let f :P 2 (R d )!R d . (i) We say f2C 2 if f2C 1 and @ x @ f;@ 2 f exist and are continuous. 23 (ii) We say f2C 2 b if all derivatives are also bounded. (iii) We sayf2C 2 UL if all derivatives are uniformly Lipschitz continuous in all parameters. Remark 3.2.12 It is obvious that if f 2 C 2 , then its lift F is also C 2 in the sense of Denition 3.1.1 due to (3.2). Maybe need a proof here. 3.3 It^ o formula: P 2 (R d ) dependence We can prove a similiar It^ o formula for a smooth function f :P 2 (R d )!R. Theorem 3.3.1 (It^ o formula) Iff2C 2 UL , anddX t =b t dt+ t dB t withE[ R T 0 b 2 t + 4 t dt]< 1, then for all t, f(L Xt ) =f(L X 0 ) + Z t 0 E h @ f(L Xs ;X s )b s + 1 2 tr @ x @ f(L Xs ;X s ) ( s T s ) i ds (3.3) Note that for f2C 2 UL , @ x @ f is bounded. Proof We assumed = 1 because the following argument generalizes easily to high dimen- sional situation. LetF :L 2 ( )!R be the lift off. Fixt, let n : 0 =t 0 <t 1 <<t n =t be a partition of [0;t]. Denote i := t i+1 t i ;X i := X t i ; X i := X t i+1 X t i ;X i; = X i +X i ;F i :=F t i ;E i [] :=E[jF t i ];j n j = max i i . Sincef2C 2 , apply Taylor's formula (see [12], Theorem 3.5.6), we have f(L X i+1 )f(L X i ) = E[DF (X i )X i ] + Z 1 0 (1)D 2 F (X i; )fX i ; X i gd = E[@ f(L X i ;X i )X i ] + Z t 0 (1)fE[@ x @ f(L X i; ;X i; )jX i j 2 ] +E[ e E[@ 2 f(L X i; ;X i; ; g X i; ) g X i ]X i ]gd = Z t i+1 t i E[@ f(L X i ;X i )b s ]ds + Z t 0 (1)fE[@ x @ f(L X i; ;X i; )jX i j 2 ] +E[ e E[@ 2 f(L X i; ;X i; ; g X i; ) g X i ]X i ]gd (3.4) 24 Denote the sum of the rst term in (3.4) over i as J n 1 , then we have J n 1 Z t 0 E[@ f(L Xs ;X s )b s ]ds n1 X i=0 Z t i+1 t i Ej(@ f(L Xs ;X s )@ f(L X i ;X i ))b s jds E Z t 0 b 2 s ds 1 2 n1 X i=0 Z t i+1 t i Ej@ f(L Xs ;X s )@ f(L X i ;X i )j 2 ds 1 2 Since X is continuous and square integrable, and @ f is Lipschitz continuous, the above sum converges to 0 by dominated convergence theorem. Similarly, denote the sum of the second term in (3.4) over i as J n 2 , then we have J n 2 1 2 Z t 0 E[@ x @ f(L Xs ;X s ) 2 s ]ds n1 X i=0 Z 1 0 (1)E[j@ x @ f(L X i; ;X i; )@ x @ f(L X i ;X i )jjX i j 2 ]d + 1 2 E[@ x @ f(L X i ;X i )jX i j 2 ] Z t i+1 t i E[@ x @ f(L Xs ;X s ) 2 s ]ds n1 X i=0 C(kX i k 3 +EjX i j 3 ) +Cj n jE Z t 0 b 2 s ds +Cj n j 1 2 E Z t 0 b 2 s ds 1 2 E Z t 0 2 s ds 1 2 + 1 2 n1 X i=0 Z t i+1 t i Ej(@ x @ f(L Xs ;X s )@ x @ f(L X i ;X i )) 2 s jds Cj n j 1 2 +C X i Z t i+1 t i E[(kX s X i k +jX s X i j) 2 s ]ds ! 0 25 as n!1. For the sum of the third term in (3.4), we can similarly show that it converges to 0 as n!1 because jE[ e E[@ 2 f(L X i; ;X i; ; g X i; ) g X i ]X i ]j CkX i k 3 +jE[ e E[@ 2 f(L X i ;X i ; f X i ) g X i ]X i ]j CkX i k 3 + E[ e E[@ 2 f(L X i ;X i ; f X i ) Z t i+1 t i e b s ds] Z t i+1 t i b s ds] CkX i k 3 +Cj n jE Z t i+1 t i b 2 s ds +CE jX i j Z t i+1 t i b s ds E Z t i+1 t i b s ds CkX i k 3 +Cj n jE Z t i+1 t i b 2 s ds So the proof is now complete. Remark 3.3.2 Note that derivative @ 2 f is not involved in (3.3). We can actually prove (3.3) under the so called "partial regularity" condition, see Chassagneux, Crisan, Delarue (2014). 3.4 It^ o formula: L 2 dependence In this section, L 2 ( ) is used to denote the Hilbert space L 2 ( ;F B T ;P), where B is a Brownian motion on , and we will assume f is a C 2 function dened on L 2 ( ). For example, letf() =E[B T ]. We want to show that similar results of Theorem 3.2.5 are still valid in current case. Since knowing any random variable 2F B T is the same as knowing the joint lawL (;B) , it is natural to guess that there exists a deterministic function g such thatDf() =g(L (;B) ;;B ). Switching the rst parameter back to a random variable, we get Df() =g(();;B ) 82L 2 ( ) (3.5) 26 for some deterministic function g :L 2 ( )RC[0;T ] ! R (();x;!) ! g(();x;!) where !2C[0;T ] is a Brownian path. This is indeed a valid result, but a few words are required to explain. On the one hand, (3.5) seems to be somehow trivial because Df()2F B T , so it is a Brownian functional by denition. The second parameter x of function g also seems to be unnecessary because it is substituted by composition (B ) in (3.5). On the other hand, when we are trying to dene higher order derivatives during the next stage, dropping this parameter may cause problem as there is no guarantee that (which is arbitrary) is a smooth Brownian functional. Therefore, parameter x is used specically to represent the dependence of DF () on the value of, and functiong is uniquely determined by (3.5) ifg is continuous. We will denote g as @ f from now on. The second order derivatives will look like: @ 2 f :L 2 ( )RC[0;T ]RC[0;T ]!R @ x @ f :L 2 ( )RC[0;T ]!R To dene partial derivative@ ! @ f, we will need to use functional Ito calculus originated from B. Dupire, see Cont [10]. For xed pair (();x)2 L 2 ( )R, if @ f(();x;B )2 L 2 ( ), then we will dene @ ! @ f :L 2 ( )RC[0;T ] [0;T ] ! R (();x;!;t) 7! (r B H ;x ) t (!) 27 where H is the square integrable martingale with H ;x t = E[@ f(();x;B )jF B t ], andr B is the vertical derivative operator. Note that @ ! @ f is adapted with respect to (!;t). Analogous to (3.2), we have D 2 f()f 1 ; 2 g = lim !0 E[@ f( + 1 ; + 1 ;B) 2 @ f(();;B) 2 ] . = E[ e E[@ 2 f( e (); e ; e B;;B)e 1 ] 2 ] +E[@ x @ f(();;B) 1 2 ] (3.6) Denition 3.4.1 Let f :L 2 ( )!R. (i) We say f2 C 1 if it is Frechet dierentiable and @ f : L 2 ( )RC[0;T ]! R is continuous. (ii) We say f2C 2 if f2C 1 and @ x @ f;@ 2 f;@ ! @ f exist and are continuous. Assumption 3.4.2 (i) @ f;@ x @ f;@ ! @ f is uniformly Lipschitz continuous in (();x); (ii) @ 2 f is uniformly Lipschitz continuous in (();e x;e !;x;!). (iii) higher order regularity with respect to path: @ ! @ x @ f;@ ! @ e ! @ 2 f exist and satisfy linear growth condition j@ ! @ e ! @ 2 f(();e x;e !;x;!;t)j +j@ ! @ x @ f(();x;!;t)j C(1 +kk +jxj +je xj + sup 0st j! s j + sup 0st je ! s j) Note that it follows from (i) that @ x @ f is bounded. Theorem 3.4.3 Suppose f : L 2 ( )!R is C 2 and satises Assumption 3.4.2. If process X2F B satises dX t =b t dt + t dB t and E[ R T 0 b 2 t + 4 t dt]<1, then we have f(X t ) =f(X 0 ) + Z t 0 E h @ f(X s )b s +@ ! @ f(X s ) s + 1 2 @ x @ f(X s ) 2 s i ds; (3.7) 28 for all t2 [0;T ], where @ f(X s );@ ! @ f(X s );@ x @ f(X s ) are shorthands for @ f(X s ();X s ; B);@ ! @ f(X s ();X s ;B;s);@ x @ f(X s ();X s ;B) respectively. Proof Fix t, let n : 0 = t 0 < t 1 < < t n = t be a partition of [0;t]. Denote i := t i+1 t i ;X i := X t i ; X i := X t i+1 X t i ;X i; = X i + X i ;F i := F t i ;E i [] := E[jF t i ];j n j = max i i . Since f2C 2 , apply Taylor's formula (see [12], Theorem 3.5.6), we have f(X i+1 )f(X i ) = E[Df(X i )X i ] + Z 1 0 (1)D 2 f(X i; )fX i ; X i gd = Z t i+1 t i E[Df(X i )b s ]ds +E Df(X i ) Z t i+1 t i s dB s + Z 1 0 (1)D 2 f(X i; )fX i ; X i gd = Z t i+1 t i E[Df(X i )b s ]ds +E Df(X i ) Z t i+1 t i s dB s + Z 1 0 (1)E[ e E[@ 2 f( g X i; (); g X i; ; e B;X i; ;B) g X i ]X i ] +(1)E[@ x @ f(X i; ();X i; ;B)(X i ) 2 ]d (3.8) Denote the sum of the rst term in (3.8) over i as J n 1 , then we have J n 1 Z t 0 E[@ f(X s )b s ]ds n1 X i=0 Z t i+1 t i Ej(Df(X s )Df(X i ))b s jds E Z t 0 b 2 s ds 1 2 n1 X i=0 Z t i+1 t i EjDf(X s )Df(X i )j 2 ds 1 2 SinceX is continuous and square integrable, andDf() is continuous, for any xeds2 [0;t], EjDf(X s )Df(X i )j 2 ! 0 asn!1. BecauseDf() is also uniformly Lipschitz continuous, above sum converges to 0 by dominated convergence theorem. Similarly, denote the sum of the second term in (3.8) over i as J n 2 and let H i;s = E[Df(X i )jF B s ] = E[@ f(X i ();X i ;B)jF B s ]. When s > t i , H i;s = 29 E[@ f(X i ();x;B)jF B s ]j x=X i , so (r B H i ) s = (r B H X i ;x ) s j x=X i = @ ! @ f(X i ();X i ;B;s), then we have J n 2 Z t 0 E[@ ! @ f(X s ();X s ;B;s) s ]ds = n1 X i=0 Z t i+1 t i E[(r B H i ) s s ]ds Z t 0 E[@ ! @ f(X s ();X s ;B;s) s ]ds n1 X i=0 Z t i+1 t i Ej(@ ! @ f(X s ();X s ;B;s)@ ! @ f(X i ();X i ;B;s)) s jds ! 0 as n!1 due to uniform Lipschitz continuity of @ ! @ f. Next, since jE[ e E[@ 2 f( g X i; (); g X i; ; e B;X i; ;B) g X i ]X i ]j jE[ e E[@ 2 f( f X i (); f X i ; e B;X i ;B) g X i ]X i ]j +CkX i k 3 E[ e E[@ 2 f( f X i (); f X i ; e B;X i ;B) Z t i+1 t i e s d e B s ] Z t i+1 t i s dB s ] +Cj n jE Z t i+1 t i b 2 s ds +Cj n j 1 2 E Z t i+1 t i b 2 s ds 1 2 E Z t i+1 t i 2 s ds 1 2 +CkX i k 3 Z t i+1 t i Z t i+1 t i E[ e E[j@ ! @ e ! @ 2 f( f X i (); f X i ; e B;e s;X i ;B;s) s e e s j]]dsde s +Cj n jE Z t i+1 t i b 2 s ds +Cj n j 1 2 E Z t i+1 t i b 2 s ds 1 2 E Z t i+1 t i 2 s ds 1 2 +CkX i k 3 Cj n jE Z t i+1 t i 2 s ds +Cj n jE Z t i+1 t i b 2 s ds +Cj n j 1 2 E Z t i+1 t i b 2 s ds 1 2 E Z t i+1 t i 2 s ds 1 2 +CkX i k 3 we have shown that the sum of the third term in (3.8) converges to 0 as n!1. 30 Finally, note that E[@ x @ f(X i ();X i ;B) Z t i+1 t i s dB s 2 ]E[@ x @ f(X i ();X i ;B) Z t i+1 t i 2 s ds ] = 2 E[@ x @ f(X i ();X i ;B) Z t i+1 t i Z s t i u dB u s dB s ] = 2 E Z t i+1 t i @ ! @ x @ f(X i ();X i ;B;s) Z s t i u dB u s ds 2 Z t i+1 t i Ej@ ! @ x @ f(X i ();X i ;B;s)j 2 ds 1 2 Z t i+1 t i E Z s t i u dB u s 2 ds 1 2 C 1 2 i E sup t i st i+1 Z s t i u dB u 2 Z t i+1 t i 2 s ds 1 2 C 1 2 i E Z t i+1 t i 2 s ds 2 1 2 C i E Z t i+1 t i 4 s ds 1 2 so we can similarly prove that the sum of the last term in (3.8) converges to 1 2 Z t 0 E[@ x @ f(X s ();X s ;B) 2 s ]ds: The proof of the theorem is complete. Theorem 3.4.4 Let f : [0;T ]L 2 ( )!R and be adapted. If f2C 1;2 and its derivatives satisfy certain assumptions, then f(t;X t ) = f(0;X 0 ) + Z t 0 @ + t f(s;X s ) +E[@ f(s;X s )b s +@ ! @ f(s;X s ) s + 1 2 @ x @ f(s;X s ) 2 s ]ds (3.9) 31 3.5 It^ o formula: functional dependence Let L 2 ( ) = L 2 ( ;F B T ;P;R d ) be the same as previous section. For simplicity, we assume in this section d = 1 and dene := [0;T ]L 2 ( ;D[0;T ]) Hence, for (t;X)2 , X is a square integrable adapted cadlag process over interval [0;T ]. We say a function f : !R is adapted if f(t;X) =f(t;X ^t ). We want to study how to dierentiate a function f : ! R; (t;X ^t )7! f(t;X ^t ). Actually, we only need to assume that f depends only on ^t =L X^t 2P 2 (D[0;T ]), where D[0;T ] is the Skorokhod space, see Section 0.2. This is because depending on a process X ^t 2 L 2 ( ;D[0;T ])) is equivalent to depending on the joint lawL (X^t;B^t) 2 P 2 (D([0;T ];R 2 )), therefore, an adapted functionf : !R could be identied with another function f : [0;T ]P 2 (D([0;T ];R 2 )), which is also adapted in the sense that f(t;) = f(t; ^t ). So from now on, we will assume f is dened on [0;T ]P 2 (D[0;T ]). The idea of generalizing the notion of dierentiation inP 2 (R d ) to = [0;T ]P 2 (C[0;T ]) comes from functional It^ o calculus. Let f be dened on [0;T ]P 2 (D[0;T ]) such that f(t;) = f(t; ^t ), that is, f is adapted. Note that d SK (!; 0) =k!k, so 2P 2 (D[0;T ]) means R D[0;T ] k!k 2 (d!) <1. Then, for any ltered probability space ( ;F T ;F;P), we can lift f to an adapted function F : [0;T ]L 2 ( ;D[0;T ])!R, where L 2 ( ;D[0;T ]) is the set of cadlag processes X such that E[kXk 2 ]<1. We will assume that ( ;F T ;P) is a rich enough (hence supports U(0; 1) and atomless) Polish space. We can dene horizontal derivative and vertical derivative similarly as the functional It^ o derivatives, see Cont [10]: (horizontal derivative) @ + t F (t;X) = lim !0 + F (t +;X ^t )F (t;X ^t ) (3.10) (vertical derivative) E[@ X F (t;X)] = lim !0 F (t;X ^t +1 [t;T ] )F (t;X ^t ) (3.11) 32 where 2 L 2 (F t ). For the horizontal derivative, since @ + t F (t;X) (d) = @ + t F (t;X 0 ) whenever L X^t =L X 0 ^t , we can dene @ + t f : (t;)3 [0;T ]P 2 (D[0;T ])!R as @ + t f(t;L X ) :=@ + t F (t;X): (3.12) For the vertical derivative, note that @ X f(t;X) is an element of L 2 (F t ). In particular, similar to Theorem 3.2.5, we have Theorem 3.5.1 Suppose ( ;F T ;P) is a rich enough Polish probability space. If the ver- tical derivative @ X F (t;X) exists for every t2 [0;T ] and X, and the mapping @ X F (t;) : L 2 ( ;D[0;T ])! L 2 (F t ) is continuous in the sense that d SK (X n ;X) " n ! 0 implies E[j@ X F (t;X n )@ X F (t;X)j 2 ]! 0, then there exists function g : [0;T ]P 2 (D[0;T ])D[0;T ]3 (t;;!)!R such that @ X F (t;X) =g(t;L X^t ;X ^t ): (3.13) Moreover, g can be chosen to be adapted: g(t;;!) =g(t; ^t ;! ^t ). Proof If X is discrete, we denoteL X^t = P i p i ! i with p i > 0 and ! i 2 D[0;T ], then for an arbitrary A 1 A :=fX ^t =! i g, it's straightforward to check by denition that E[@ X F (t;X)1 A 1 ] = lim "!0 f(t; P j6=i p j ! j +P(A 1 ) ! i +"1 [t;T] + (p i P(A 1 )) ! i )f(t; P i p i ! i ) " so E[@ X F (t;X)1 A 1 ] depends only on P(A 1 ) for A 1 A. By Lemma 3.2.2, @ X F (t;X) is constant onA and hence, by choosingA 1 =A, it follows that (3.13) holds for some function g when X is discrete. In order to prove (3.13) in general, let's x a countable partitionfO n i ji2Ng ofD[0;T ] for everyn 1, such that the diameter of eachO n i under Skorohod metricd SK is less than 2 n . Let's also x an element! n i of eachO n i . For an arbitrary cadlag processX2L 2 ( ;D[0;T ]), 33 we deneX n := P i ! n i 1 O n i (X). Then (3.13) can be proved by following the exact argument of Theorem 3.2.5. Remark 3.5.2 It may not be easy to verify the continuity assumption of mapping@ X F (t;) in above theorem directly. Since D[0;T ] is nonseparable under the stronger topology gener- ated by uniform norm, one cannot only verify the weaker assumption thatE[kX n Xk 2 ]! 0 impliesE[j@ X F (t;X n )@ X F (t;X)j 2 ]! 0. On the other hand, as shown below, we are only interested in the case where X is continuous. We have mentioned earlier that if ! is con- tinuous and cadlag functions ! n converges to ! under Skorohod metric, then they converge uniformly. Indeed, by denition, if d SK (! n ;!) " n ! 0, then there exists a sequence n 2E such thatk! n ! n k 2" n ! 0 andkId n k 2" n ! 0. From [2], page 124, when ! is continuous, we have k! n !kk! n ! n k +m ! (kId n k) 2" n +m ! (2" n ) (3.14) where m ! is the modulus of continuity of !. Therefore, if d SK (X n ;X)" n ! 0 and X is continuous, then E[kX n Xk 2 ]E[(2" n +m X (2" n )) 2 ]: Since m X () 2kXk, by dominated convergence theorem, we get E[kX n Xk 2 ]! 0. This observation will be used in the proof of Theorem 3.5.6 below. Due to above theorem, ifF is the lifting off from [0;T ]P 2 (D[0;T ]), then@ X F (t;X) (d) = @ X F (t;X 0 ) wheneverL X^t =L X 0 ^t . Moreover, since D[0;T ] is Polish, by the second proof of Proposition 3.2.9, among all the functions g that satisfy (3.13), there is at most one continuous function. So we will write this continuous function as @ f if it exists. Second order derivative @ 2 f : [0;T ]P 2 (D[0;T ])D[0;T ]D[0;T ]3 (t;;e !;!)!R 34 such that @ 2 f(t;L X ;X;!) =@ X @ F (t;X;!); 8(t;X;!) can be similarly dened through E[@ X @ F (t;X;!)] = lim !0 @ F (t;X ^t +1 [t;T ] ;! ^t )@ F (t;X ^t ;! ^t ) (3.15) as long as @ X @ F (t;;!) : L 2 ( ;D[0;T ])!L 2 (F t ) is continuous, where @ F is the lifting of @ f. At last, @ ! @ f : [0;T ]P 2 (D[0;T ])D[0;T ]3 (t;;!)!R is dened as the vertical derivative with respect to ! at time t in the functional It^ o sense, i.e. @ ! @ f(t;;!) = lim !0 @ f(t; ^t ;! ^t +1 [t;T ] )@ f(t; ^t ;! ^t ) : (3.16) Clearly, @ + t f;@ 2 f;@ ! @ f are all adapted. Denition 3.5.3 Let f : [0;T ]P 2 (D[0;T ])! R be adapted and suppose F : [0;T ] L 2 ( ;D[0;T ])!R is a lifting of f. (i) We say f 2 C 0;1 if f is continuous; vertical derivative @ X F (t;) dened through (3.11) exists and there exists continuous function g such that (3.13) holds (this unique function g is denoted as @ f). (ii) We say f2 C 1;2 if f2 C 0;1 ; @ X @ F (t;;!) exists and is continuous, for all (t;!); and @ + t f;@ 2 f;@ ! @ f exist and are continuous. (iii) We say f2C 1;2 b if f2C 1;2 and all derivatives are also bounded. (iv) We sayf2C 1;2 UL iff2C 1;2 and all its derivatives are Lipschitz continuous in ;!;e !, uniformly in t. 35 Note that this denition doesn't depend on the choice of space . If f2C 1;2 UL , then @ ! @ f is bounded. Theorem 3.5.4 Suppose f : [0;T ]P 2 (D[0;T ])! R is in C 1;2 UL . For any It^ o process X2L 1;2 such that dX t =b t dt + t dB t and E R T 0 b 2 s + 4 s ds<1, we have f(t; ^t ) = f(0; ^0 ) + Z t 0 @ + t f(s; ^s ) +E @ f(s; ^s ;X ^s )b s + 1 2 tr (@ ! @ f(s; ^s ;X ^s ) ( s T s )) ds (3.17) for all t2 [0;T ], where ^t =L X^t . Proof For simplicity, assume X is one dimensional. We will rst prove the theorem by assuming that b; are bounded. Fix t, let n : 0 = t n 0 < t n 1 < < t n n = t be a sequence of partitions of [0;t] such thatj n j! 0, dene piecewise constant approximations X n := P n1 i=0 X t n i 1 [t i ;t i+1 ) () +X t 1 [t;T ] (); n =L X n. Since X is continuous, d SK (X n ^t X ^t )kX n ^t X ^t k! 0. SincekX n ^t X ^t k 2kX ^t k and X2L 1;2 , by dominated convergence theorem, we have W 2 ( ^s ; n ^s ) 2 W 2 ( ^t ; n ^t ) 2 E[d SK (X n ^t ;X ^t ) 2 ]! 0 and hence n ^s ! ^s inP 2 (D[0;T ]) for all st. 36 Let F be the lifting of f, then F (t;X n ^t )F (0;X ^0 ) = n1 X i=0 [F (t n i+1 ;X n ^t n i+1 )F (t n i ;X n ^t n i )] = n1 X i=0 [F (t n i+1 ;X n ^t n i+1 )F (t n i+1 ;X n ^t n i ) +F (t n i+1 ;X n ^t n i )F (t n i ;X n ^t n i )] = n1 X i=0 [F (t n i+1 ;X n ^t n i+1 )F (t n i+1 ;X n ^t n i ) + Z t n i+1 t n i @ + t F (s;X n ^t n i )ds] = Z t 0 @ + t f(s;L X n ^s )ds + n1 X i=0 [F (t n i+1 ;X n ^t n i + (X t n i+1 X t n i )1 [t n i+1 ;T ] ) F (t n i+1 ;X n ^t n i )] Let H : L 2 (F t n i+1 )!R be dened as H() := F (t n i+1 ;X n ^t n i +1 [t n i+1 ;T ] ), then it is easy to check that H is continuously Fr echet dierentiable with DH() =@ f(t n i+1 ;L X n ^t n i +1 [t n i+1 ;T] ;X n ^t n i +1 [t n i+1 ;T ] ) and its second order derivative is given by D 2 H()f 1 ; 2 g =E[@ ! @ f(t n i+1 ;L X n ^t n i +1 [t n i+1 ;T] ;X n ^t n i +1 [t n i+1 ;T ] ) 1 2 ] +E[ e E[@ 2 f(t n i+1 ;L X n ^t n i +1 [t n i+1 ;T] ; e X n ^t n i + e 1 [t n i+1 ;T ] ;X n ^t n i +1 [t n i+1 ;T ] )e 1 ] 2 ]; 37 and is continuous. Hence, F (t n i+1 ;X n ^t n i + (X t n i+1 X t n i )1 [t n i+1 ;T ] )F (t n i+1 ;X n ^t n i ) = H(X t n i+1 X t n i )H(0) = E[DH(0) (X t n i+1 X t n i )] + Z 1 0 (1)D 2 H((X t n i+1 X t n i ))fX t n i+1 X t n i ;X t n i+1 X t n i gd = E[@ f(t n i+1 ;L X n ^t n i ;X n ^t n i ) (X t n i+1 X t n i )] + Z 1 0 (1)fE[@ ! @ f(t n i+1 ;L X n ^t n i +(X t n i+1 X t n i ) ;X n ^t n i +(X t n i+1 X t n i ))(X t n i+1 X t n i ) 2 ] +E[ e E[@ 2 f(t n i+1 ;L X n ^t n i +(X t n i+1 X t n i ) ; e X n ^t n i +( e X t n i+1 e X t n i ); X n ^t n i +(X t n i+1 X t n i ))( e X t n i+1 e X t n i )](X t n i+1 X t n i )]gd = Z t n i+1 t n i E[@ f(t n i+1 ;L X n ^t n i ;X n ^t n i )b s ]ds + 1 2 fE[@ ! @ f(t n i+1 ;L X n ^t n i ;X n ^t n i )(X t n i+1 X t n i ) 2 ] +E[ e E[@ 2 f(t n i+1 ;L X n ^t n i ; e X n ^t n i ;X n ^t n i )( e X t n i+1 e X t n i )](X t n i+1 X t n i )]g +C(kX t n i+1 X t n i k 3 +EjX t n i+1 X t n i j 3 ) = Z t n i+1 t n i E[@ f(t n i+1 ;L X n ^s ;X n ^s )b s ] + 1 2 E[@ ! @ f(t n i+1 ;L X n ^s ;X n ^s ) 2 s ]ds +Cjt n i+1 t n i j 3 2 Sending n!1, we can immediately get (3.17) by using the regularity property of f. Now let's remove the boundedness requirement of b;. For any m2N, let b m s := (m)_b s ^m; m s := (m)_ s ^m and X m t :=X 0 + Z t 0 b m s ds + Z t 0 m s dB s ; t2 [0;T ]; then (3.17) is satised by ( m ;X m ) with m =L X m. It is easy to verify that W 2 ( m ^s ; ^s ) 2 W 2 ( m ;) 2 E[d(X m ;X) 2 ]E sup 0tT jX m t X t j 2 ! 0 38 for all sT . Note that j@ + t f(s; m ^s )@ + t f(s; ^s )jCW 2 ( m ^s ; ^s ) and Ej@ f(s; m ^s ;X m ^s )b m s @ f(s; ^s ;X ^s )b s j E[j@ f(s; ^s ;X ^s )jjb m s b s j] +CE[(W 2 ( m ^s ; ^s ) + sup 0us jX m u X u j)jb s j] Ckb m s b s k +Ckb s k (W 2 ( m ;) +kX m Xk 1;2 ) Ej@ ! @ f(s; m ^s ;X m ^s )( m s ) 2 @ ! @ f(s; ^s ;X ^s )( s ) 2 j E[j@ ! @ f(s; ^s ;X ^s )jj( m s ) 2 ( s ) 2 j] +CE[(W 2 ( m ^s ; ^s ) + sup 0us jX m u X u j)j s j 2 ] CEj( m s ) 2 ( s ) 2 j +Ck s k 2 W 2 ( m ;) +CE[ sup 0uT jX m u X u jj s j 2 ] By dominated convergence theorem, we have proved that (;X) also satises (3.17). By adapting the argument of the previous theorem slightly by introducing stopping times (see [14], p.150), we can actually prove a more general result. Theorem 3.5.5 Suppose f : [0;T ]P 2 (D[0;T ])! R is in C 1;2 UL . For any continuous semimartingale X =M +A such thatL X 2P 2 (D[0;T ]) and EhMi T <1 (hence M is a square integrable martingale starting at X 0 ), E A 2 T <1 (where A is the total variation of A), we have f(t; ^t ) = f(0; ^0 ) + Z t 0 @ + t f(s; ^s )ds +E Z t 0 @ f(s; ^s ;X ^s )dA s + 1 2 E Z t 0 tr (@ ! @ f(s; ^s ;X ^s )dhMi s ) (3.18) for all t2 [0;T ], where ^t =L X^t and the expectations on the right hand side are well dened (i.e. bounded). 39 Question: For a continuous semimartingaleX =M+A withE sup 0tT jX t j 2 <1, can we prove thatEhMi T <1 andE A 2 T <1? This is clearly true if the drift and volatility of X are bounded, see denition ofP L t in [11]. WhenX is an It^ o process withdX t =b t dt+ t dB t , then these conditions becomeE[ R T 0 2 t dt+( R T 0 jb t jdt) 2 ]<1, i.e. b2L 1;2 and2L 2 , which is a sucient condition for X2L 1;2 . In the generalized It^ o formua, function f : [0;T ]P 2 (D[0;T ])!R and its derivatives are only evaluated at continuous measures and processes (given that X is continuous), so we can propose the denition of regularity of functions dened on = [0;T ]P 2 (C[0;T ]). Due to this motivation and the discussion in Remark 3.5.2, we have the following result. Corollary 3.5.6 Suppose ( ;F T ;P) is a rich enough Polish probability space and let F : [0;T ]L 2 ( ;D[0;T ]) ! R be the lift of f : [0;T ]P 2 (D[0;T ]) ! R. If the vertical derivative @ X F (t;X) exists for every t2 [0;T ] and cadlag process X, and the mapping @ X F (t;) : L 2 ( ;D[0;T ])! L 2 (F t ) is continuous in the sense that X is continuous and E[kX n Xk 2 ]! 0 implies E[j@ X F (t;X n )@ X F (t;X)j 2 ]! 0, then there exists function g : [0;T ]P 2 (D[0;T ])D[0;T ]3 (t;;!)!R such that @ X F (t;X) =g(t;L X^t ;X ^t ) (3.19) when X has continuous paths or takes discrete (cadlag) values. Moreover, g can be chosen to be adapted: g(t;;!) =g(t; ^t ;! ^t ). Proof This result is proved immediately by following the proof of Theorem 3.5.1 and using the results in Remark 3.5.2. Denition 3.5.7 Suppose f : [0;T ]P 2 (C[0;T ])! R is adapted, we say that f2 C 0;1 (or C 1;2 ;C 1;2 b ;C 1;2 UL ) if there exists an extension f of f to [0;T ]P 2 (D[0;T ]) such that f has the corresponding regularity as dened in Denition 3.5.3. 40 Example 3.5.8 Let f(t;L X ) =E R t 0 X t X s ds, then @ + t f(t;L X ) = lim !0 + (f(t +;L X ^(t+) )f(t;L X^t ))= =EX 2 t : Since E[@ X F (t;X)] = lim !0 1 E Z t 0 (X t +)X s dsE Z t 0 X t X s ds =E Z t 0 X s ds ; we have @ f(t;;!) = Z t 0 ! s ds; @ ! @ f(t;;!) 0: Actually, we have f2C 1;2 UL . To see that @ f is Lipschitz continous in !, let !; ~ !2D[0;T ], by denition, there exists n 2E so thatk! ~ ! n k_kId n kd SK (!; ~ !)+n 1 , hence Z t 0 ! s ds Z t 0 ~ ! s ds Z t 0 j! s ~ ! s jds Z t 0 j!(s) ~ !( n (s))j +j~ !( n (s)) ~ !(s)jds (d SK (!; ~ !) +n 1 )t + Z t 0 j~ !( n (s)) ~ !(s)jds Since ~ !( n (s))! ~ !(s) at continuity points (which is countable) of ~ ! and cadlag functions are bounded, the integral in the last line converges to 0 by dominated convergence theorem, so we get j@ f(t;;!)@ f(t;; ~ !)jTd SK (!; ~ !): We can check that Ito formula (3.17) becomes E Z t 0 X t X s ds = Z t 0 EX 2 s +E Z t 0 Z s 0 X u du dA s 41 which is clearly true because E Z t 0 X t X s ds Z t 0 EX 2 s ds = Z t 0 E[(X t X s )X s ]ds =E Z t 0 Z t s X s dA u ds =E Z t 0 Z s 0 X u du dA s Example 3.5.9 (A two dimensional example) Let f(t;L X ) =E[X 1 t X 2 t ], then @ + t f(t;L X ) = 0; @ f(t;;!) = 0 B @ ! 2 t ! 1 t 1 C A; @ ! @ f(t;;!) = 0 B @ 0 1 1 0 1 C A: Even though@ f is not continuous in! under Skorohod metric, Ito formula still holds here. Actually, we can verify that in current case, (3.17) becomes E[X 1 t X 2 t ] = E[X 1 0 X 2 0 ] +E[ Z t 0 (X 2 s ;X 1 s )dA s + 1 2 Z t 0 tr ( 0 B @ 0 1 1 0 1 C AdhMi s )] = E[X 1 0 X 2 0 ] +E Z t 0 X 2 s dA 1 s +X 1 s dA 2 s +dhM 1 ;M 2 i s which agrees with the classical It^ o formula. 42 Chapter 4 Classical solution of master equation Let ( ;F;F;P) be a probability space withF =F B , and dX t; s =b(s;X t; s ())ds +(s;X t; s ())dB s ; X t; t = (4.1) whereb; : [0;T ]L 2 (F T )!R. We want to study under what conditions can we derive the regularity fo value function u(t;) := g(X t; T ()). Even though this chapter mainly follows paper [4], it provides a slight extension of the results proved in [4], due to the generalized It^ o's formula (3.9) in Section 3.4. 4.1 Representations of derivatives of u 4.1.1 L 2 -Derivatives of X t; Lemma 4.1.1 Suppose b; are smooth enough. Let's x t, then for all s 2 [t;T ]; 2 L 2 (F t ), the mapping L 2 (F t )! L 2 (F s );7! X t; s is L 2 -dierentiable in the direction of , i.e. X t;+" s =X t; s +"A +o(") (4.2) for some A2L 2 (F s ) and jjo(")jj " ! 0 as "! 0. If we denote the derivative A asr X t; s (), then processr X t; () is the unique solution to r X t; s () = + Z s t E[@ b(r;X t; r )(r X t; r ())]dr + Z s t E[@ (r;X t; r )(r X t; r ())]dB r ; (4.3) 43 We also have the following estimates E sup s2[t;T ] [jr X t; s ()r X t; 0 s ()j 2 ]CEj 0 j 2 (4.4) Moreover, the mapping 7!r X t; s () is R-linear. Lemma 4.1.2 Suppose b; are smooth enough. For xed t, and all s2 [t;T ];2L 2 (F t ), the mapping L 2 (F t )!L 2 (F s );7!r X t; s () is L 2 -dierentiable in the direction of , and the derivativer X t; s (;) is the unique solution to r X t; s (;) = Z s t @ b(r;X t; r )(r X t; r ();r X t; r ()) +E[@ b(r;X t; r )r X t; r (;)]dr + Z s t @ (r;X t; r )(r X t; r ();r X t; r ()) +E[@ (r;X t; r )r X t; r (;)]dB r Proof Assume b = 0. By denition, we want to show that for any 2 L 2 (F s ), there exists a square integrable random variable A2F s such that r X t;+" s () =r X t; s () +"A +o(") (4.5) 44 whereo(") represents a random variable satisfying jjo(")jj " ! 0 as"! 0. By (4.2), (4.3) and Lipschitz continuity of @ , we have 1 " r X t;+" s ()r X t; s () = Z s t 1 " E[@ (r;X t;+" r )r X t;+" r ()@ (r;X t; r )r X t; r ()]dB r = = Z s t E @ (r;X t;+" r ) 1 " r X t;+" r ()r X t; r () +E 1 " @ (r;X t; r +"r X t; r ())@ (r;X t; r ) r X t; r () +E 1 " @ (r;X t; r +"r X t; s () +o("))@ (r;X t; r +"r X t; s ()) r X t; r () dB r = Z s t E @ (r;X t;+" r ) 1 " r X t;+" r ()r X t; r () +@ (r;X t; r )(r X t; r ();r X t; r ()) +o(1)dB r Then we can prove that E sup tsT 1 " r X t;+" s ()r X t; s () r X t; s (;) 2 ! 0 (4.6) as "! 0. Thus mapping 7!r X t; s () is L 2 -dierentiable in the direction of and the derivative is equal tor X t; s (;). 4.1.2 Derivatives of value function Recall that the value function is u(t;()) := g(X t; T ()). Formally dierentiating u yields the following results: i)82L 2 (F t ), @ u(t;)() = E[@ g(X t; T ())(r X t; T ())] = E[@ g(X t; T ;X t; T ;B )(r X t; T ())] (4.7) 45 We want to discuss how to get expression for @ u(t;) as a random variable. Our objective is to nd Y t 2L 2 (F t ) such that @ u(t;)() =E[Y t ]. Now we can begin by dening an adjoint process dY 2 s = 2 s ds + 2 s dB s ; Y 2 T =@ g(X t; T ()); then E[Y 2 T r X t; T ()Y 2 t ] = E[ Z T t Y 2 s ~ E[@ ~ b(s; ~ X t; ~ s )(r ~ X t; ~ s (~ ))] + 2 s ~ E[@ ~ (s; ~ X t; ~ s )(r ~ X t; ~ s (~ ))] +r X t; s () 2 s ds] = Z T t E[( ~ E[ ~ Y 2 s @ b(s;X t; s ) + ~ 2 s @ (s;X t; s )] + 2 s )r X t; s ()]ds So if we let 2 s = ~ E[ ~ Y 2 s ]@ b(s;X t; s ) ~ E[ ~ 2 s ]@ (s;X t; s ) then (4.7) is equal to E[Y 2 t ] and by denition @ u(t;) = Y t; t , where (Y t; ;Z t; ) solves mean eld BSDE 8 > < > : dY t; s = (@ b(s;X t; s )E[Y t; s ] +@ (s;X t; s )E[Z t; s ])ds +Z t; s dB s Y t; T = @ g(X t; T ) (4.8) Please note that in the above discussion, the coecients and terminal values of BSDE (4.8) actually depend on (t;), and instead of (Y 2 ;Z 2 ), we use (Y t; ;Z t; ) to emphasize such 46 dependence, since we need to let t vary in order to study @ ! @ u. By Proposition 5.3, [15], we can get from (4.8), Z t; s = E s D s (@ g(X t; T )) + Z T s D s (@ b(r;X t; r ))E[Y t; r ] +D s (@ (r;X t; r ))E[Z t; r ]dr = E s @ Bs @ g(X t; T ) + Z T s @ Bs @ b(r;X t; r )E[Y t; r ] +@ Bs @ (r;X t; r )E[Z t; r ]dr +(s;X t; s )E s @ x @ g(X t; T ) + Z T s @ x @ b(r;X t; r )E[Y t; r ] +@ x @ (r;X t; r )E[Z t; r ]dr (4.9) ii) From (4.8), we also have @ u(t;) =E t @ g(X t; T ) + Z T t @ b(r;X t; r )E[Y t; r ] +@ (r;X t; r )E[Z t; r ]dr : (4.10) By viewing both sides as function of (();x;B) and dierentiation with respect to x, we can get @ x @ u(t;) =E t @ x @ g(X t; T ) + Z T t @ x @ b(r;X t; r )E[Y t; r ] +@ x @ (r;X t; r )E[Z t; r ]dr (4.11) This is because @ x X t; s = 1 for any s. iii) To study@ ! @ u, we dierentiate (4.8) with respect toB at timet, but we have to make sure that in this process we are viewing as a constant (see previous remark). By denition @ ! @ u(t;) = lim #0 E t [(@ u(t +;)@ u(t;))(B t+ B t )] = lim #0 E t [Y t+; t+ (B t+ B t )] = lim #0 1 Z t+ t E t D s Y t+; t+ ds: (4.12) 47 We can get D s Y t+; t+ by dierentiating (4.8), i.e. (D s Y t+; ;D s Z t+; ) is solution to 8 > > > > > > > < > > > > > > > : dD s Y t+; r = f(@ x @ b(r;X t+; r ;B )D s X t+; r +@ Bs @ b(r;X t+; r ;B ))E[Y t+; r ] +(@ x @ (r;X t+; r ;B )D s X t+; r +@ Bs @ (r;X t+; r ;B ))E[Z t+; r ]gdr +D s Z t+; r dB r ; t +rT D s Y t+; T = @ x @ g(X t+; T ;B )D s X t+; T +@ Bs @ g(X t+; T ;B ) Sinceb; don't depend onx, by dierentiating the SDE forX t+; , we havedD s X t+; r = 0, hence D s X t+; r =D s = 0 because is viewed as constant. Thus above BSDE becomes 8 > > > > < > > > > : dD s Y t+; r = f@ Bs @ b(r;X t+; r ;B )E[Y t+; r ] +@ Bs @ (r;X t+; r ;B )E[Z t+; r ]gdr +D s Z t+; r dB r ; t +rT D s Y t+; T = @ Bs @ g(X t+; T ;B ) (4.13) So @ ! @ u(t;) = lim #0 1 Z t+ t E t D s Y t+; t+ ds = E t @ Bt @ g(X t; T ) + Z T t @ Bt @ b(s;X t; s )E[Y t; s ] +@ Bt @ (s;X t; s )E[Z t; s ]ds (4.14) iv) For @ + t u, we have lim #0 u(t +;)u(t;) = lim #0 g(X t+; T )g(X t; T ) = lim #0 E[@ g(X t; T )( X T )] + R 1 0 (1)@ g( ; X T )( X T ; X T )d = lim #0 (I 1 +I 2 +I 3 ) (4.15) 48 where I 1 = 1 E[@ g(X t; T )( X T )] I 2 = 1 Z 1 0 (1)E[@ x @ g( ; X T )( X T ) 2 ]d I 3 = 1 Z 1 0 (1)E[ ~ E[@ g( ; ~ X T ; ~ B; ; X T ;B) ~ X T ] X T ]d and ; X s := X t; s +(X t+; s X t; s ); X s := X t+; s X t; s . To show that lim #0 I 1 exists, let's follow a similar approach as in the case of @ u(t;). We dene dY s = s ds + s dB s ; Y T =@ g(X t; T ) (4.16) then d(Y s X s ) = ( s X s +Y s (b(s;X t+; s )b(s;X t; s )) + s ((s;X t+; s )(s;X t; s )))ds +( s X s +Y s ((s;X t+; s )(s;X t; s )))dB s Note that b(s;X t+; s )b(s;X t; s ) = Z 1 0 E[@ b(s; ; X s ) X s ]d (s;X t+; s )(s;X t; s ) = Z 1 0 E[@ (s; ; X s ) X s ]d 49 So d(Y s X s ) = s X s + Z 1 0 Y s E[@ b(s; ; X s ) X s ] + s E[@ (s; ; X s ) X s ]d ds s X s + Z 1 0 Y s E[@ (s; ; X s ) X s ]d dB s If we let s = Z 1 0 @ b(s; ; X s )E[Y s ] +@ (s; ; X s )E[ s ]d (4.17) i.e. if we dene (Y ;Z ) as the solution to mean eld BSDE 8 > < > : dY s = f( R 1 0 @ b(s; ; X s )d)E[Y s ] + ( R 1 0 @ (s; ; X s )d)E[Z s ]gds +Z s dB s Y T = @ g(X t; T ) (4.18) then E[Y T X T ] =E[Y t+ X t+ ] =E[Y t+ (X t; t+ )]: By letting! 0 in (4.18), we recover mean eld BSDE (4.8). Note that Y is only dened on [t +;T ], but we can extend it to [t;t +] by Y s =Y t+ Z t+ s Z r dB r ; tst +: (4.19) Letting Y 0 =Y t; and using the estimate for mean eld BSDE, we can prove that E sup tsT jY s Y 0 s j 2 + Z T t jZ s Z 0 s j 2 ds C (4.20) 50 So I 1 = 1 E[Y t+ (X t; t+ )] = 1 E Z t+ t Y s b(s;X t; s ) +Z s (s;X t; s )ds (4.21) We want to show that the right hand side of the above equation converges to some limit as # 0. It's easy to see that lim #0 1 E R t+ t Y s b(s;X t; s )ds = E[Y 0 t b(t;)]. For the Z term, we need to use its representation. By Malliavin calculus, Z s =E s [D s Y t+ ] for s2 [t;t +], wherefD s Y r g t+rT satises (note D s X t; r =(s;X t; s );D s X t+; r = 0) 8 > > > > > > > < > > > > > > > : D s Y r = R 1 0 @ Bs @ b(r; ; X r ) + (1)@ x @ b(r; ; X r )(s;X t; s )dE[Y r ] + R 1 0 @ Bs @ (r; ; X r ) + (1)@ x @ (r; ; X r )(s;X t; s )dE[Z r ] dr +D s Z r dB r D s Y T = @ Bs @ g(X t; T ) +@ x @ g(X t; T )(s;X t; s ) (4.22) Note that D t B s = 1 fstg 6= 1 fs>tg . Similarly, we also have, Z 0 s = E s @ Bs @ g(X t; T ) +@ x @ g(X t; T )(s;X t; s ) + Z T s @ Bs @ b(r;X t; r ) +@ x @ b(r;X t; r )(s;X t; s ) E[Y 0 r ] + @ Bs @ (r;X t; r ) +@ x @ (r;X t; r )(s;X t; s ) E[Z 0 r ]dr (4.23) Thus, for s2 [t;t +], we get Z s Z 0 s = E s Z T t+ Z 1 0 @ Bs @ b(r; ; X r ) + (1)@ x @ b(r; ; X r )(s;X t; s )d E[Y r ] + Z 1 0 @ Bs @ (r; ; X r ) + (1)@ x @ (r; ; X r )(s;X t; s )d E[Z r ]dr Z T s @ Bs @ b(r;X t; r ) +@ x @ b(r;X t; r )(s;X t; s ) E[Y 0 r ] + @ Bs @ (r;X t; r ) +@ x @ (r;X t; r )(s;X t; s ) E[Z 0 r ]dr 51 from which we can easily conclude that E Z s Z 0 s + 1 2 (s;X t; s )E s Z T s @ x @ b(r;X t; r )E[Y 0 r ] +@ x @ (r;X t; r )E[Z 0 r ]dr C 1 2 (4.24) by using (4.20). Thus second term in the right hand side of (4.21) converges to 1 E Z t+ t Z 0 s (s;X t; s ) 1 2 [(s;X t; s )] 2 E s Z T s @ x @ b(r;X t; r )E[Y 0 r ]+@ x @ (r;X t; r )E[Z 0 r ]dr ds: (4.25) Now we need to consider lim #0 1 E Z t+ t jZ 0 s Z 0 t jds But from (4.23) we can easily see that Z 0 s is right continuous at s =t (actually we should dene Z 0 t as the limit of Z 0 s from right), so the above limit is 0. Thus we have proved that lim #0 I 1 = E Y 0 t b(t;) +Z 0 t (t;) 1 2 ((t;)) 2 Z T t @ x @ b(r;X t; r )E[Y 0 r ] +@ x @ (r;X t; r )E[Z 0 r ]dr : (4.26) Note that Y 0 t =@ u(t;);Z 0 t 6=@ ! @ u(t;). For I 2 , we can apply the same process to show that lim #0 I 2 = Z 1 0 (1)E[@ x @ g(X t; T )((t;)) 2 ]d = 1 2 E[@ x @ g(X t; T )((t;)) 2 ] (4.27) We can also prove lim #0 I 3 = 0: (4.28) 52 Hence we have show that the limit (4.15) exists and @ + t u(t;) =E Y 0 t b(t;) +Z 0 t (t;) 1 2 ((t;)) 2 Z T t @ x @ b(r;X t; r )E[Y 0 r ] +@ x @ (r;X t; r )E[Z 0 r ]dr +@ x @ g(X t; T ) : (4.29) Note that Y 0 t =Y t; t =@ u(t;);Z 0 t =Z t; t . By combining (4.9), (4.11), (4.14) and (4.29), we have proved that u satises PDE @ + t u(t;) +E[@ u(t;)b(t;) +@ ! @ u(t;)(t;) + 1 2 @ x @ u(t;)((t;)) 2 ] = 0 (4.30) 4.2 Wellposedness of classical solution Theorem 4.2.1 Suppose b;;g are smooth enough and X t; is the solution of SDE (4.1). Let u(t;) :=g(X t; T ()), then u2C 1;2 and is the unique classical solution of 8 > < > : @ + t u(t;) +E @ u(t;)b(t;) +@ ! @ u(t;)(t;) + 1 2 @ x @ u(t;)(t;) 2 = 0 u(T;) =g(()) (4.31) Proof As shown in the previous section, we have proved that u(t;) := g(X t; T ()) is a classical solution of (4.31). Uniqueness follows immediately by using Ito's formula (3.9). Indeed, for any function v(t;)2 C 1;2 that solves (4.31), we have dv(s;X t; s ) = 0, hence v(t;) =v(T;X t; T ) =g(X t; T ()) =u(t;). 53 Chapter 5 Controlled McKean-Vlasov SDEs and master equation 5.1 Problem formulation Consider the canonical space := C([0;T ];R), which is equipped with the uniform norm k!k := sup t2[0;T ] j! t j. Here we don't require! 0 = 0. LetX be the canonical process dened on , i.e. X t (!) :=! t , andF t :=F X t be the ltration generated by X. We know thatF X T coincides with the Borel -algebraB( ) generated by uniform metrickk. LetP( ) (or sometimes simply written asP) be the set of probability measures on ( ;F T ) andP 2 ( ) (or simplyP 2 ) denote the subset ofP such that R k!k 2 d(!) +1. The Wasserstein distance W 2 onP 2 ( ) is dened as W 2 (;) := inf P2(;) E P h kX 1 X 2 k 2 i 1 2 where (;) denotes the set of probability measuresP inP( ) such that P(X 1 ) 1 =; P(X 2 ) 1 =; andX 1 ;X 2 are the coordinate processes on . (;) is called the set of couplings of and. Since ( ;kk) is a Polish space, (P 2 ( );W 2 ) is also Polish (see [24]). Moreover, the convergence in metric space (P 2 ( );W 2 ) is equivalent to weak convergence plus convergence of second order moments. 54 For eachP2P 2 , letP [0;t] be the law of stopped process X ^t underP, thenP [0;t] is also inP 2 . SinceF X^t T =F t ,P [0;t] is completely determined by its restriction toF t . Let := [0;T ]P 2 . For (t;); (s;)2 , we shall dene the Wasserstein-2 pseudometric on as W 2 ((t;); (s;)) := jtsj +W 2 [0;t] ; [0;s] 2 1 2 : We can easily verify that W 2 satisfys the triangle inequality. However, for any 2P 2 , we also have W 2 ((t;); (t; [0;t] )) = 0, so W 2 is not a metric on . If a function F : !R is measurable with respect to the -algebra induced by W 2 , then it must be adapted in the sense that F (t;) = F (t; [0;t] );8t;. One can think of these functions as carrying some natural structure of adaptedness similar to that of stochastic processes. 5.1.1 A subtle ltration issue Let AR, and (b;) : A!RR be adapted functions and g :P 2 !R. For each t, letA t denote an appropriate set ofA-valued controls. In this paper, we will mainly focus on the following control problem. Given (t;)2 and a certain probability space with a Brownian motion B, suppose is a random process such thatL ^t = ^t , we dene X s = t + Z s t b(r;L X ^r ; r )dr + Z s t (r;L X ^r ; r )dB r ; (5.1) and V (t;) := sup 2At g(L X [0;T] ): (5.2) At rst sight, despite the law dependence, this problem looks very similar to the standard control problem, but we want to point out that things can become much subtler in our case, especially when we need to choose the appropriate control setA t . The choice ofA t should meet two basic requirements: The value functionV (t;) should not depend on the choice of, but only on =L ^t ; V should satisfy the Dynamic Programming Principle (DPP). 55 If we use open-loop controls, there are two natural choicesA 1 t andA 2 t , whereA 1 t uses shifted ltration of B, i.e. s = (s; (B r B t ) trs ), andA 2 t uses the whole ltration of B, i.e. s = (s; (B r ) 0rs ). For the standard setting, they induce the same value function for control problems. However, in our setting (5.1)-(5.2), we don't know whether sup 2A 1 t g(L X [0;T] ) = sup 2A 2 t g(L X [0;T] ) (5.3) is true. Now let's consider the closed-loop controls. There are also dierent choices. If we use the shifted ltration, i.e. s = (s; (X r ) trs ), DPP would fail. The second choice is to use feedback control s =(s;X s ), which works to certain extent and actually is the main setting in the literature. However, we prefer not to use this for several reasons: In practice it is not natural to assume the players cannot use past information; It seems dicult to have regularity of V (t;) without strong constraint on ; It fails to work in non-Markovian models, which are important in applications. So in our model, will be F X -measurable. In this case, when only measurability of is assumed, SDE (5.1) may not admit any solution in the strong sense, i.e. we may have F X T )F B T . So we will consider its weak solution (X;B); ( ;F X ;P). For a discussion of the wellposedness of this SDE, see [19]. Note that our formulation is slightly dierent from the general weak formulation because the canonical space ( ;X;F X ) is xed. Then given (t;), we need to nd a pair (B ;P ) dened on this space such that B is a (P ;F X )-Brownian motion,P [0;t] = [0;t] and X s =X t + Z s t b(r;P [0;r] ; r )dr + Z s t (r;P [0;r] ; r )dB r ; for all tsT (5.4) holdsP -a.s. Note that the value function can be equivalently written as V (t;) := sup 2At g(P ): (5.5) 56 Clearly our V is an adapted function on . In order to guarantee the uniqueness in law of weak solution of (5.4), see Assumption 5.2.2 in next section. 5.2 Regularity of value function The goal of this section is to prove the regularity of value function. The proof could be more challenging than in strong formulation. Let's introduce a simple example to illustrate why this could happen. Example 5.2.1 Suppose t = 0;T = 1;b 0;(s;;y) = 1 +y 2 ;g(L X ^1 ) = 1 3 E[X 4 1 ] (E[X 2 1 ]) 2 , andA t consists of constant controls, i.e. s (X) = 0 (X 0 );8s 2 [0; 1], then X 1 = X 0 + (1 + 0 (X 0 ) 2 )B 1 . Let 0, and for " > 0, let " be a random variable such that P( " = ") = P( " =") = 1 2 . When X 0 = , 1 + 2 is equal to a constant c, then V (0; 0 ) = sup c2R 1 3 E[c 4 B 4 1 ] (E[c 2 B 2 1 ]) 2 = 0. However, when X 0 = " , if we choose control s (X) = 1 fX 0 >0g , then V (0; 1 2 ( " + " ))g(L X ^1 ) = 1 6 E[(2B 1 +") 4 ] + 1 6 E[(B 1 ") 4 ] ( 1 2 E[(2B 1 +") 2 ] + 1 2 E[(B 1 ") 2 ]) 2 = 9 4 2 3 " 4 . Thus, lim "# 0 V (0; 1 2 ( " + " )) 9 4 > 0 =V (0; 0 ), which means that V is discontinuous at (0; 0 ). This example shows that choosing admissible set of controlsA t properly is important to the regularity of V . The major issue with the example is that the controls are assumed to be globally constant in time, which is too restrictive. When the set is too small, the regularity 57 could fail. However, as we will see next, if we just need controls to be locally (i.e. piecewise) constant in time. Let's dene A t := 9n; and t =t 0 <<t n =T; such that s (X) = n1 X i=0 h i (X [0;t i ] )1 [t i ;t i+1 ) (s) where h i 's are measurable and bounded functions ; (5.6) b A t := 9m;n; and 0s 1 <<s m <t =t 0 <<t n =T; such that s (X) = n1 X i=0 h i (X s 1 ; ;X sm ;X [t 0 ;t i ] )1 [t i ;t i+1 ) (s); where h i 's are measurable and bounded functions : (5.7) and V (t;) := sup 2At g(P ); b V (t;) := sup 2 b At g(P ): (5.8) Since b A t A t ,V (t;) b V (t;). As we will see later, we can actually prove they are indeed equal. In fact, for a general progressively measurable control2F X , it is not guaranteed that SDE (5.4) will admit any weak solution, so assuming piecewise constancy of controls seems to be natural. If is piecewise constant, then we just need to solve (5.4) step by step. For each step, since r is xed, the equation is solvable even in the strong sense. The following usual assumptions are to ensure that (5.4) is uniquely solvable (in the strong and hence weak sense). Assumption 5.2.2 There exists a constant L> 0 such that (i) functions b;;g are bounded by L and 6= 0; (ii) b and are uniform Lipschitz continuous with Lipschitz constant L, i.e. j(b;)(t;;y) (b;)(s;;y 0 )jL(W 2 ((t;); (s;)) +jyy 0 j) (5.9) 58 (iii) g is uniform Lipschitz continuous with Lipschitz constant L. Note that when 6= 0, then for a strong solution X (if it exists) corresponds to given Brownian motion B, we must haveF B =F X , so the strong solution can be carried over to the canonical space under our weak formulation setting. The main purpose of this section is to prove Theorem 5.2.3 Under Assumption 5.2.2, V (t;) is Lipschitz continuous, for any t2 [0;T ]. The next lemma will be useful in the proof of this main result. Lemma 5.2.4 Given any t > 0;; ~ 2P 2 (C[0;t]);2 (; ~ ) and " > 0; > 0, suppose f~ s g 0st is an arbitrary random process, dened on a rich enough probability space, with L ~ = ~ . Then for any 0 s 1 < < s m = t, we can nd another continuous process f s g 0st and Brownian motionf ~ B s g s2[0;] (maybe on some extended probability space) such that: (i) L =, (ii) ~ is independent of ~ B, (iii) s j 2(~ s 1 ;;sm ; ~ B [0;] );8j = 1; ;m, (iv) k s 1 ;;sm ~ s 1 ;;sm k 1;2 j s 1 ;;sm j +", where k s 1 ;;sm ~ s 1 ;;sm k 1;2 := E max j j s j ~ s j j 2 1 2 j s 1 ;;sm j := Z Z max 1jm j! s j ~ ! s j j 2 d(!; ~ !) 1 2 : Proof Case 1: If ~ s 1 ;;sm = (~ x 1 ;;~ xm) for some ~ x2R m , we can pick any independent Brownian motionf ~ B s g s2[0;] , and dene s 1 ;;sm := q(U 1 ;U 2 ; ;U m ), where q() is the 59 multivariate quantile transform of multivariate distribution s 1 ;;sm (see Section 1.3 of [20]) andU 1 ; ;U m are i.i.d. U(0; 1) random variable that are dened through ~ B [0;] . For example, we can letU j := =m ( ~ B j=m ~ B (j1)=m ), where =m () is the CDF ofN(0;=m) random variable. The next step is to extend vector ( s j ) j to a processf s g 0sT such that L =. A simple way to achieve this is to dene 0 on another probability space, and then take the coupled measure on the product space such that s j = 0 s j ;8j. Hence 0 could be viewed as an extension of ( s j ) j to a process, so we just denote it again as . It is easy to check the ~ B and found in this way satisfy the desired requirements (i)-(iv). Case 2: If ~ s 1 ;;sm = P i p i (~ x i 1 ;;~ x i m ) , with p i > 0; P i p i = 1. We rst x a partition fO i g B(R m ) of R such that (~ x i 1 ; ; ~ x i m ) 2 O i , and then dene measures i (A) := s 1 ;;sm (AO i )=p i onR m . Supposef ~ B i g is a mutually independent sequence of Brownian motions with ~ B i ? ~ , let i s 1 ;;sm = q i (U i 1 ; ;U i m ), where U i 1 ; ;U i m are i.i.d. U(0; 1) generated from ~ B i [0;] and q i () is the multivariate quantile transform of i . Let s j = P i i s j 1 O i (~ s 1 ;;sm ); ~ B s = P i ~ B i s 1 O i (~ s 1 ;;sm ). It is easy to check that ~ B is also a Brownian motion and s j 2 (~ s 1 ;;sm ; ~ B [0;] ). Actually, if ~ s 1 ;;sm = ~ x i , then ~ B = ~ B i and s j = i s j = q i j (U i 1 ; ;U i m )2 ( ~ B [0;] ). In order to see that ~ ? ~ B, let A;A 0 be two Borel sets, then P(~ 2A; ~ B2A 0 ) = X i P(~ 2A; ~ s 1 ;;sm 2O i ; ~ B2A 0 ) = X i P(~ 2A; ~ s 1 ;;sm 2O i ; ~ B i 2A 0 ) = X i P(~ 2A; ~ s 1 ;;sm 2O i )P( ~ B i 2A 0 ) = X i P(~ 2A; ~ s 1 ;;sm 2O i )P( ~ B2A 0 ) = P(~ 2A)P( ~ B2A 0 ): 60 Instead of (iv), we can prove a stronger result that the joint distribution of ( s 1 ;;sm ; ~ s 1 ;;sm ) is equal to s 1 ;;sm , which follows from P( s 1 ;;sm 2A; ~ s 1 ;;sm 2 ~ A) = X i P( i s 1 ;;sm 2A; ~ s 1 ;;sm = ~ x i )1 ~ A (~ x i ) = X i p i P( i s 1 ;;sm 2A)1 ~ A (~ x i ) = X i p i i (A)1 ~ A (~ x i ) = X i s 1 ;;sm (AO i )1 ~ A (~ x i ) = s 1 ;;sm (A[ i:~ x i 2 ~ A O i ) = s 1 ;;sm (A ~ A): The last equality holds because ~ s 1 ;;sm is supported on setf~ x i g. Case 3: If ~ s 1 ;;sm is a general probability measure inP 2 (R m ). Suppose is a random process withL (;~ ) = . LetfO i g be a countable partition of R m such that for all i, the diameter of O i is less than "=2 under the Euclidean maximum norm. For each i, choose x i 2 O i ;! i 2 such that ! i s 1 ;;sm = x i , then dene ~ 0 = P i ! i 1 O i (~ s 1 ;;sm ) and let 0 =L (;~ 0 ) . By Case 2, we can nd process with s j = P i i s j 1 O i (~ 0 s 1 ;;sm ) and Brownian motion ~ B := P i ~ B i 1 O i (~ 0 s 1 ;;sm ) such that ~ 0 ? ~ B; s j 2 (~ 0 s 1 ;;sm ; ~ B [0;] ) andL (s 1 ;;sm ;~ 0 s 1 ;;sm ) = 0 s 1 ;;sm . Since 1 O i (~ s 1 ;;sm ) = 1 O i (~ 0 s 1 ;;sm ), we can obtain ~ ? ~ B by similar argument as in Case 2. We also have s j 2 (~ s 1 ;;sm ; ~ B [0;] ) because ~ 0 2(~ s 1 ;;sm ). ClearlyL =L =. (iv) also follows immediately from k s 1 ;;sm ~ s 1 ;;sm k 1;2 k s 1 ;;sm ~ 0 s 1 ;;sm k 1;2 +k~ 0 s 1 ;;sm ~ s 1 ;;sm k 1;2 = k s 1 ;;sm ~ 0 s 1 ;;sm k 1;2 +k~ 0 s 1 ;;sm ~ s 1 ;;sm k 1;2 k s 1 ;;sm ~ s 1 ;;sm k 1;2 + 2k~ 0 s 1 ;;sm ~ s 1 ;;sm k 1;2 k s 1 ;;sm ~ s 1 ;;sm k 1;2 +" = j s 1 ;;sm j +" 61 Remark 5.2.5 Since the multivariate quantile transform doesn't generalize to stochastic processes, the results of the above lemma can't be generalized to the case where m-tuple (s 1 ; ;s m ) is replaced by uncountable interval [0;t]. This is why we need to consider value function b V rst. In order to pass from discrete version to continuous version, let's dene M := 2P 8"> 0;9m2N; 0s 0 <s 1 <<s m =T; such that m1 X j=0 s j 1 [s j ;s j+1 ) () + sm 1 fsmg () 1;2 "; where : (5.10) By dominated convergence theorem, it's not hard to prove thatP 2 M. Actually, for any 2P, then having 2P 2 is equivalent to having 2M with its univariate marginals inP 2 (R). With the aid of following obvious result, we can slightly improve (iv) of Lemma 5.2.4. Lemma 5.2.6 Suppose ; ~ 2P 2 (C[0;t]), then for any " > 0 and 0 s 1 < < s m = t, we can nd 0 s 0 1 < < s 0 m 0 = t such thatfs j g fs 0 j 0 g andk (m 0 ) k 1;2 < ";k~ ~ (m 0 ) k 1;2 < ", where ; ~ ~ , (m 0 ) is the simple process obtained from at timesfs 0 j 0 g and ~ (m 0 ) is dened similarly. Furthermore, for any 2 (; ~ ), we have jjj s 0 1 ;;s 0 m 0 j + 4", wherejj := RR sup 0st j! s ~ ! s j 2 d(!; ~ !) 1 2 . 62 Proof The rst half follows obviously from the previous paragraph. Now supposeL (;~ ) = , by Cauchy-Schwarz inequality, we have k ~ k 2 1;2 k (m 0 ) ~ (m 0 ) k 2 1;2 2k ~ k 1;2 E sup s j s ~ s j max j j s 0 j ~ s 0 j j 2 1=2 2k ~ k 1;2 E sup s j s ~ s jj (m 0 ) s ~ (m 0 ) s j 2 1=2 2k ~ k 1;2 k( (m 0 ) ) (~ ~ (m 0 ) )k 1;2 4k ~ k 1;2 " From this, we immediately getk ~ k 1;2 k (m 0 ) ~ (m 0 ) k 1;2 + 4", which is the desired result. 5.2.1 Regularity of b V Proposition 5.2.7 8t2 [0;T ]; b V (t;) is uniformly Lipschitz continuous in . More pre- cisely, there exists a constant C that only depends on L;T such that j b V (t;) b V (t; ~ )jCW 2 ( [0;t] ; ~ [0;t] ): Proof For any xed pair (; ~ ), let's take a control 2 b A t , then s (X) = h i (X s 1 ; ;X sm ; X [t 0 ;t i ] ), if s 2 [t i ;t i+1 ), for some bounded measurable functions h i . Suppose 2 ( [0;t] ; ~ [0;t] ) such that W 2 ( [0;t] ; ~ [0;t] ) is achieved by this optimal cou- pling and ~ is dened on a rich enough probability space withL ~ = ~ [0;t] . By apply- ing Lemma 5.2.6 and then Lemma 5.2.4, for any "; > 0, we can nd 0 s 0 1 < < s 0 m 0 < t 0 = t, a continuous process and Brownian motion ~ B [0;] such that fs j g fs 0 j 0 g;L = [0;t] ; ~ ? ~ B; s 0 1 ;;s 0 m 0 ;t 0 2 (~ s 0 1 ;;s 0 m 0 ;t 0 ; ~ B [0;] ) and k ~ k 1;2 k (m 0 ) ~ (m 0 ) k 1;2 + 4"j s 0 1 ;;s 0 m 0 ;t 0 j + 5"W 2 ( [0;t] ; ~ [0;t] ) + 5". In order to somplify the notation in the proof, we will just writekk forkk 1;2 , X for X t;; and ~ X ~ for ~ X t;~ ;~ . 63 Let X [0;t] = [0;t] ; ~ X ~ [0;t] = ~ [0;t] . For the next step, we want to nd another control ~ 2A t so thatkX [t;T ] ~ X ~ [t;T ] k is bounded byk ~ k. In the following, we will construct ~ step by step on intervals [t 0 ;t 0 +); [t 0 +;t 1 + ); ; [t n1 +;t n +]. For s2 [t 0 ;t 0 +), we simply choose ~ s 0, and then dene ~ X ~ on [t 0 ;t 0 +] as ~ X ~ s = ~ t 0 + Z s t 0 b(r;L ~ X ~ [0;r] ; 0)dr + Z s t 0 (r;L ~ X ~ [0;r] ; 0)d ~ B rt 0 : Since s 0 j ; t 0 2 (~ s 0 0 ; ; ~ s 0 m 0 ; ~ t 0 ; ~ B [0;] ) and 6= 0, we get s 0 j ; t 0 2 ( ~ X ~ s 0 0 ; ; ~ X ~ s 0 m 0 ; ~ X ~ [t 0 ;t 0 +] ). Then we can let ~ s =h 0 ( s 0 1 ; ; s 0 m 0 ; t 0 ) = s ; s2 [t 0 +;t 1 +): Now we can take an arbitrary independent Brownian motion B 0 and dene X on [t 0 ;t 1 ] as X s = t 0 + Z s t 0 b(r;L X [0;r] ;h 0 ( s 0 1 ; ; s 0 m 0 ; t 0 ))dr + Z s t 0 (r;L X [0;r] ;h 0 ( s 0 1 ; ; s 0 m 0 ; t 0 ))dB rt 0 ; s2 [t 0 ;t 1 ] and ~ X ~ on [t 0 +;t 1 +] as ~ X ~ s+ = ~ X ~ t 0 + + Z s t 0 b(r +;L ~ X ~ [0;r+] ;h 0 ( s 0 1 ; ; s 0 m 0 ; t 0 ))dr + Z s t 0 (r +;L ~ X ~ [0;r+] ;h 0 ( s 0 1 ; ; s 0 m 0 ; t 0 ))dB rt 0 ; s2 [t 0 ;t 1 ] The previous SDEs implyB [0;t 1 t 0 ] 2( ~ X ~ s 0 1 ; ; ~ X ~ s 0 m 0 ; ~ X ~ [t 0 ;t 1 +] ) and henceX [t 0 ;t 1 ] 2( ~ X ~ s 0 1 ; ; ~ X ~ s 0 m 0 ; ~ X ~ [t 0 ;t 1 +] ). This means we can let ~ s :=h 1 ( s 0 1 ; ; s 0 m 0 ;X [t 0 ;t 1 ] ) = s ; s2 [t 1 +;t 2 +) 64 and dene X [t 1 ;t 2 ] ; ~ X ~ [t 1 +;t 2 +] in similar way by using the same Brownian motion B as before. By repeatedly constructing ~ in this manner on all subsequent subintervals, we will get X s = t 0 + Z s t 0 b(r;L X [0;r] ; r )dr + Z s t 0 (r;L X [0;r] ; r )dB rt 0 ; ~ X ~ s+ = ~ X ~ t 0 + + Z s t 0 b(r +;L ~ X ~ [0;r+] ; ~ r+ )dr + Z s t 0 (r +;L ~ X ~ [0;r+] ; ~ r+ )dB rt 0 ; for s2 [t 0 ;t n ]. Since ~ r+ = r , by comparing these two SDEs and using the continuity of b;, we will get standard estimate E sup t 0 rs jX r ~ X ~ r+ j 2 E[j t 0 ~ X ~ t 0 + j 2 ] +C Z s t 0 +W 2 (L X [0;r] ;L ~ X ~ [0;r+] ) 2 dr CE sup 0rt 0 j r ~ r j 2 +C Z s t 0 +W 2 (L X [t 0 ;r] ;L ~ X ~ [t 0 +;r+] ) 2 dr CE sup 0rt 0 j r ~ r j 2 +C(t n t 0 ) +C Z s t 0 E sup t 0 r 0 r jX r 0 ~ X ~ r 0 + j 2 dr By Gronwall's inequality, we have E sup t 0 rtn jX r ~ X ~ r+ j 2 e C(tnt 0 ) CE sup 0rt 0 j r ~ r j 2 +C(t n t 0 ) (5.11) where constant C only depends on L. Then we can easily derive E sup trT jX r ~ X ~ r j 2 C +E sup trT jX r ~ X ~ r+ j 2 CE sup 0rt j r ~ r j 2 +C C(W 2 ( [0;t] ; ~ [0;t] ) 2 + +" 2 ) By choosing " = p =W 2 (; ~ ), we have proved that for any 2 b A t , we can nd another control ~ 2 b A t so that W 2 (L X [0;T] ;L ~ X ~ [0;T] )CW 2 ( [0;t] ; ~ [0;t] ), where C depends on T;L. 65 Since C doesn't depend on the choice of , we can easily deduce Lipschitz continuity of b V (t;) in , so the proof is complete. 5.2.2 V = b V We note that Theorem 5.2.3 follows immediately if we can prove Proposition 5.2.8 V (t;) = b V (t;): Proof The argument is as follows: sup 2At g(L X ) sup 2 b At g(L X ) sup 2 b A c t g(L X ) (i) sup 2A c t g(L X ) (ii) sup 2At g(L X ); (5.12) whereA c t consists of the controls = (X) inA t so that the corresponding h i 's are continuous in X and b A c t is similarly dened. Note that here X is again the shorthand for X t;; . We only need to prove inequalities (i), (ii) in the above chain. Proof of (i): Fix 2 A c t , for each m 1, let's dene control m 2 b A c t , so that m s (X) =h m i (X 0 ;X t 2 m ; ;X2 m 1 2 m t ;X [t;t i ] ) :=h i (X m [0;t i ] ) = s (X m ) for s2 [t i ;t i+1 ), where the continuous path X m is dened as X m s = 8 > < > : X s ; s =jt=2 m ;j = 0; ; 2 m 1; or s2 [t;T ] linear; s2 (jt=2 m ; (j + 1)t=2 m );j = 0; ; 2 m 1 It is clear that X m is uniquely determined by X 0 ; ;X2 m 1 2 m t and X [t;T ] , and since h i is continuous, h m i is also continuous. Let X ;X m be the solutions of (5.1) corresponding to controls ; m respectively with X [0;t] =X m [0;t] , then it's not hard to nd a subsequence of fX m g such thatkX X m k! 0 along this subsequence. To see this, we will prove by induction that, there exist a subsequence m q of m, such that E sup t i st i+1 jX s X mq s j 2 ! 0 as q!1; (5.13) 66 for i = 0; ;n 1. When i = 0, by comparing the SDEs satised by X and X m and apply Gronwall's inequality, we have E sup t 0 st 1 jX s X m s j 2 CE h 0 (X [0;t 0 ] )h 0 ((X m ) m [0;t 0 ] ) 2 = CE h 0 (X [0;t 0 ] )h 0 ((X ) m [0;t 0 ] ) 2 ! 0 as m!1; where the last convergence is due to BCT because sup 0st 0 jX s (X ) m s j osc 1 2 m (X [0;t 0 ] ) ! 0 for any continuous path X [0;t 0 ] (!), and h 0 is bounded and continu- ous. Now assume (5.13) holds for ik 1. When i =k, by following similar argument of comparison, we can get E sup t k st k+1 jX s X m s j 2 CE sup t 0 st k jX s X m s j 2 +CE h k (X [0;t k ] )h k ((X m ) m [0;t k ] ) 2 (5.14) The rst term on the right hand side converges to 0 by induction. For the second term, note that sup 0st k jX s (X m ) m s j sup 0st k jX s (X ) m s j + sup 0st k j(X ) m s (X m ) m s j osc 1 2 m (X [0;t 0 ] ) + sup t 0 st k jX s X m s j Since osc 1 2 m (X [0;t 0 ] )! 0 everywhere and sup t 0 st k jX s X m s j L 2 ! 0 by induction, we have sup 0st k jX s (X m ) m s j p ! 0. So there exists a subsequence m q of m, which we will again denote as m, such that sup 0st k jX s (X m ) m s j! 0 almost surely. By bounded convergence theorem, the second term of (5.14) converges to 0 along this subsequence. Thus, inequality (i) follows from the continuity of function g. 67 Proof of (ii): Fix a control 2A t , then we have s (X) = h i (X [0;t i ] ), where h i is bounded measurable, for s2 [t i ;t i+1 );i = 0; ;n 1. We want to nd a sequence of controls m 2A c t such that kX X m k! 0 as m!1 (5.15) whereX ;X m are solution processes of (5.1) corresponding to controls ; m respectively but with the same initial values, i.e. we also have X s = X m s for s2 [0;t 0 ]. We will construct m step by step. Fix a decreasing sequence of positive numbers " m # 0. First, whens2 [t 0 ;t 1 ), since the processX is xed, by Lusin's theorem, there exists a sequence of continuous functionsh m 0 :C[0;t 0 ]!R such thath m 0 =h 0 on some closed setF 0;m C[0;t 0 ], P(X [0;t 0 ] 2 F c 0;m ) " m and h m 0 is bounded by the same constant as h 0 . Assume that m s (X) =h m 0 (X [0;t 0 ] ) for s2 [t 0 ;t 1 ). Then, by similar arguments as before, we have E sup t 0 st 1 jX s X m s j 2 CEjh 0 (X [0;t 0 ] )h m 0 (X [0;t 0 ] )j 2 C" m ! 0: Therefore, there exists a subsequencefm (1) g offmg such that X m (1) [0;t 1 ] ! X [0;t 1 ] ; P-a.s. Next, for s2 [t 1 ;t 2 ), we can use Lusin's theorem similarly, i.e. we let h m 1 be continuous functions on C[0;t 1 ] such that h m 1 = h 1 on some closed set F 1;m , P(X [0;t 1 ] 2 F c 1;m ) " m andh m 1 is bounded by the same constant ash 1 . By bounded convergence theorem, we have Ejh m 1 (X m (1) [0;t 1 ] )h m 1 (X [0;t 1 ] )j 2 ! 0 asm (1) !1 for eachm. Note thatX m (1) [0;t 1 ] depends only on h m (1) 0 , not on the choice of h m 1 . So we can nd a further subsequence offm (1) g, which 68 will again be denoted asfm (1) g, such thatEjh m 1 (X m (1) [0;t 1 ] )h m 1 (X [0;t 1 ] )j 2 " m . By dening controls m on [t 0 ;t 2 ) through combination of h 1 1 with h 1 (1) 0 , h 2 1 with h 2 (1) 0 , , we can get E sup t 1 st 2 jX s X m s j 2 CE sup t 0 st 1 jX s X m (1) s j 2 +CEjh 1 (X [0;t 1 ] )h m 1 (X m (1) [0;t 1 ] )j 2 C" m (1) +CEjh 1 (X [0;t 1 ] )h m 1 (X [0;t 1 ] )j 2 +CEjh m 1 (X [0;t 1 ] )h m 1 (X m (1) [0;t 1 ] )j 2 C" m (1) +C" m +C" m ! 0 as m!1 By repeating the above process for nite steps on the remaining intervals [t 2 ;t 3 ); [t 3 ;t 4 ); , we can easily construct a sequence of controls m inA c t satisfying (5.15), hence the proof of inequality (ii) is complete. 5.3 Dynamic programming principle Under the weak formulation, the following DPP is quite obvious. Theorem 5.3.1 (DPP) WithA t given by (5.6), we have V (t;) = sup P2Pt() V (t +;P); (5.16) whereP t () :=fP t;; j2A t g, and P t;; is the solution P of (5.4) corresponds to (t;). Proof The key idea is to observe, due to piecewise constancy of controls inA t , that (i) P t;; =P t+;P t;; ;j [t+;T] ; 82A t ; (ii) P t+;P t;; 0 ; =P t;; 0 t+ ; 8 0 2A t ;2A t+ ; where j [t+;T ] is the restriction of after time t +, and concatenation 0 t+ 2A t is the control that is equal to 0 during [t;t +) and after time t +. 69 On the one hand, for any 2A t , by (i), we have g(P t;; )V (t +;P t;; ), hence one direction is proved for (5.16). On the other hand, for any P 0 = P t;; 0 2P t (), by (ii), we have V (t +;P 0 ) = sup 2A t+ g(P t+;P 0 ; ) = sup 2A t+ g(P t;; 0 t+ )V (t;), so the other direction is also proved for (5.16). Proposition 5.3.2 (Regularity in time) The value function V : ! R is uniformly Lipschitz continuous in (t;) under W 2 . Proof For any (t;); (s;)2 , assume t < s, by DPP (5.16) and Theorem 5.2.3, we have jV (t;)V (s;)j = sup P2Pt() V (s;P)V (s;) sup P2Pt() jV (s;P)V (s;)j C sup P2Pt() W 2 (P ^s ; ^s ) C W 2 ( ^t ; ^s ) +jtsj 1 2 = CW 2 ((t;); (s;)): 5.4 Master equation If we assume that the value functionV : !R withA t given by (5.6) isC 1;2 UL , then we can derive a PDE solved by V using DPP (5.16). In particular, by apply It^ o formula (3.17) to V (s;P t;; ), we can get (formally) @ + t V (t;) + sup 2At E @ V (t;;X)b(t;; t ) + 1 2 @ ! @ V (t;;X)(t;; t ) 2 = 0 (5.17) 70 where X, with terminal condition V (T;) =g(). So (5.17) can be rewritten as @ + t V (t;) + sup 2At Z @ V (t;;!)b(t;; t (!)) + 1 2 @ ! @ V (t;;!)(t;; t (!)) 2 d(!) = 0 (5.18) Since the functions appeared in the above bracket are continuous, by measurable selec- tion theorem, see Theorem 5.3.1 of [22], for any xed (t;)2 , there exists a sequence of controls n t 2F t such thatj n t jn and for every !, lim n!1 @ V (t;;!)b(t;; n t (!)) + 1 2 @ ! @ V (t;;!)(t;; n t (!)) 2 = sup t2Ft @ V (t;;!)b(t;; t (!)) + 1 2 @ ! @ V (t;;!)(t;; t (!)) 2 Therefore, we can switch the supremum and integration in the last equation. Then we obtain @ + t V (t;) + Z sup a2R @ V (t;;!)b(t;;a) + 1 2 @ ! @ V (t;;!)(t;;a) 2 d(!) = 0 (5.19) This equation is called the master equation of our McKean-Vlasov-type control problem (5.2). When there is no control, this equation will be reduced to (4.31). However, with controls in presence, the value function V is usually not in C 1;2 (see some examples in Section 3.4 of Touzi [23] for classical stochastic control problems), hence we need to study "weak" solutions of (5.17). This is the topic of the following two chapters. 71 Chapter 6 Viscosity solution of master equation In this chapter, we x the canonical space ( ;F) = (C[0;T ];B(C[0;T ])) endowed with uniform topology, and the canonical process X : (t;!)7! ! t . We want to study the weak solutions of the master equation (5.17). 6.1 Denition of viscosity solutions For (t;)2 , letP L t () denote the set of P2P 2 ( ) such that P [0;t] = [0;t] and X is a P-semimartingale with drift and volatility bounded by L, after time t. Note that X may not be aP-semimartingale before t. Proposition 6.1.1 For every (t;) and L> 0, the setP L t () is weakly compact. Proof See [26], Theorem 3. For a function : !R, we will introduce the following set of test functions: A L (t;) := 2C 1;2 UL ([0;T ]P 2 ( )) :9> 0; such that ( )(t;) = 0 = sup tst+ sup P2P L t () ( )(s;P) (6.1) A L (t;) := 2C 1;2 UL ([0;T ]P 2 ( )) :9> 0; such that ( )(t;) = 0 = inf tst+ inf P2P L t () ( )(s;P) (6.2) Note that C 1;2 UL ([0;T ]P 2 ( )) is dened in Denition 3.5.7. 72 Denition 6.1.2 (Viscosity solution) Suppose is an adapted, bounded and uniformly continuous function on [0;T ]P 2 ( ). (i) is called a L-viscosity subsolution of (??) if for all (t;);t 2 [0;T ) and 2 A L (t;), we haveL (t;) 0. (ii) is called a L-viscosity supersolution if (??) if for all (t;);t 2 [0;T ) and 2 A L (t;), we haveL (t;) 0. (iii) is called a viscosity solution if it is both a L-viscosity subsolution and a L-viscosity supersolution for some L> 0. 6.2 Existence of viscosity solutions Let V 0 : !R denote the value function for the control problem (5:2). Theorem 6.2.1 (Existence) V 0 (t;) is a viscosity solution of (5.17). Proof Let L V (t;) be equal to the left hand side of (5.17) without supremum, i.e. LV = sup L V . Fix any L> 0 that bounds b and . If V 0 is not a L-viscosity subsolution, then there exist (t;) with t2 [0;T ) and 2 A L V 0 (t;), such that c :=L (t;)< 0: (6.3) Hence for any control ,L (t;)c< 0. Note that It^ o formula (3.17) still applies to P t;; after time t, so we have d (s;P t;; ) =L (s;P t;; )ds 73 Let be the constant in (6.1) for . Note thatL (s;P t;; ) is continuous in s. In fact, since 2C 1;2 UL and the drift termb is also bounded,L (s;P t;; ) is uniformly continuous in s, uniformly with respect to control . Thus, from inequality sup 2At L (s;P t;; ) sup 2At L (r;P t;; ) sup 2At L (s;P t;; )L (r;P t;; ) ; we can easily see that sup L (s;P t;; ) is also continuous ins, then there exists ~ 2 (0;) such that sup L (s;P t;; )<c=2 for anys2 [t;t + ~ ]. Here> 0 is the corresponding constant for as dened in (6.1). SinceP t;; 2P L t (), for any , we have c ~ 2 > (t + ~ ;P t;; ) (t;) = (t + ~ ;P t;; )V 0 (t;) by (6.1) V 0 (t + ~ ;P t;; )V 0 (t;) by (6.1) By DPP (5.16) and taking supremum over on the right hand side of above inequality (note the left hand side doesn't depend on ), we will get c ~ 2 V 0 (t;)V 0 (t;) = 0 which is a contradiction. On the other hand, if V 0 is not a L-viscosity supersolution, then there exists (t;) and 2A L V 0 (t;) such that c :=L (t;) > 0. By denition of operatorL , we can nd a control 0 such thatL 0 (t;)>c=2. By Ito's formula (3.17), d (s;P t;; 0 ) =L 0 (s;P t;; 0 )ds: 74 Let be the constant in (6.1) for , sinceL 0 (s;P t;; 0 ) is continuous in s, there exists 0 2 (0;) such thatL 0 (s;P t;; 0 ) > c=4 for any s2 [t;t + 0 ]. Thus we will have the following contradiction c 0 4 < (t + 0 ;P t;; 0 ) (t;) = (t + 0 ;P t;; 0 )V 0 (t;) by (6.2) V 0 (t + 0 ;P t;; 0 )V 0 (t;) by (6.2) V 0 (t;)V 0 (t;) by DPP (5:16) = 0 6.3 Partial comparison principle Theorem 6.3.1 (Partial Comparison Principle) Let V 1 be an L-viscosity subsolution andV 2 be anL-viscosity supersolution of (5.17) for some L> 0. IfV 1 (T;)V 2 (T;) and either V 1 2C 1;2 UL or V 2 2C 1;2 UL , then V 1 V 2 . Proof Assume92P 2 ( ) such that c :=V 1 (0;)V 2 (0;)> 0. Dene c := sup 0tT sup P2P L 0 () (V 1 V 2 )(t;P) c 2T (Tt) : It is clear that c c=2. Let V 12 = V 1 V 2 and a t = sup P2P L 0 () V 12 (t;P), then for any s<t, ja t a s j sup P2P L 0 () V 12 (t;P)V 12 (s;P) sup P2P L 0 () jtsj +W 2 (t;P); (s;P) C(jtsj_jtsj 1 2 ) 75 So a t is continuous in t, and hence we can nd t 2 [0;T ) such that c = sup P2P L 0 () V 12 (t ;P) c 2T (Tt ) (6.4) From Proposition 6.1.1, we know thatP L 0 () is weakly compact, so there existsP 2P L 0 () such that c =V 12 (t ;P ) c 2T (Tt ). Suppose V 2 2C 1;2 UL and let's dene (t;) :=V 2 (t;) +c + c 2T (Tt) then (t ;P ) =V 1 (t ;P ). SinceP L t (P )P L 0 (), for anytt andP2P L t (P ), we have (t;P)V 1 (t;P). This implies that 2A L V 1 (t ;P ). Then we will get 0L (t ;P ) = LV 2 (t ;P ) c 2T c 2T < 0, a contradiction. 6.4 Classical solutions In this section, we want to show that if the value function V 2C 1;2 UL , then it is the unique classical solution of master equation (5.17) in space C 1;2 UL ([0;T ]P 2 ( )). Theorem 6.4.1 (Classical solution) If the value function V given by (5.2) is in C 1;2 UL ([0;T ]P 2 (C[0;T ])), then it is the unique classical solution of master equation (5.17). Proof Since V 2C 1;2 UL is a viscosity solution by Theorem 6.2.1, it is also, by denition, a classical solution. Uniqueness follows immediately from Theorem 6.3.1. Theorem 6.4.2 (Verication theorem) Letv2C 1;2 UL ([0;T ]P 2 (C[0;T ])). Suppose that for each (t;;!), the supremum sup a2R @ v(t;;!)b(t;;a) + 1 2 @ ! @ v(t;;!)(t;;a) 2 is achieved by a =a ? (t;;!); 76 function v solves @ + t v(t;)+ Z @ v(t;;!)b(t;;a ? (t;;!))+ 1 2 @ ! @ v(t;;!)(t;;a ? (t;;!)) 2 d(!) = 0 with terminal values v(T;) =g(); there exists P t;;? 2P 2 (C[0;T ]) so that X s =X t + Z s t b(r;P t;;? ;a ? (r;P t;;? ^r ;X ^r ))dr + Z s t (r;P t;;? ;a ? (r;P t;;? ^r ;X ^r ))dB ? r ; st;P t;;? -a.s. ;B ? is a P t;;? -B.M. and P t;;? X 1 ^t = ^t , then vV and t;;? s (X) :=a ? (s;P t;;? ^s ;X ^s ) is optimal, i.e. g(P t;;? )g(P t;; ); 82A t : (6.5) If t;;? 2A t , then v(t;) =V (t;). Proof For every (t;) and 2A t , we have dv(s;P t;; ) = @ + t v(s;P t;; ) +E P t;; h @ v(s;P t;; ;X)b(s;P t;; ; s ) + 1 2 @ ! @ v(s;P t;; ;X)(s;P t;; ; s ) 2 i ds GlenPark;SanFrancisco;CA 0 sov(t;)v(T;P t;; ) =g(P t;; ). Taking supremum over2A t , we getv(t;)V (t;). On the other hand, by applying It^ o's formula again, we obtain dv(s;P t;;? ) = 0, hence v(t;) =v(T;P t;;? ) =g(P t;;? ), and (6.5) follows. 77 6.5 Remarks on future directions Analysis of a general fully nonlinear path-dependent master equation @ + t V (t;) +F (t;;V;@ V (t;;);@ ! @ V (t;;)) = 0 (6.6) is very challenging and a lot of questions still remain open at this moment. So far, we haven't even given a complete proof of the uniqueness of viscosity solution to equation (5.17). This is due to the fact that (5.17) may be nonlinear in @ V , for example, when b(t;;a) =1_a^ 1,and(t;;a) doesn't depend on a. In literature, the usual technique to prove uniqueness is to combine partial comparison with stability result. Stability result says that viscosity solutions of an equation could be approximated in a certain way by the classical solutions of some perturbation equations. However, when nonlinearity is present, wellposedness of classical solutions to master equations still remains open. 78 Bibliography [1] Patrick Billingsley, Probability and Measure, Third Edition, Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, Inc., 1995 [2] Patrick Billingsley, Convergence of Probability Measures, Second Edition, John Wiley & Sons, Inc., 1999. [3] R. Buckdahn, J. Li, S. Peng. Mean-eld backward stochastic dierential equations and related partial dierential equations, Stochastic Processes and their Applications, Vol. 119, No. 10 (2009), 3133-3154. [4] R. Buckdahn, J. Li, S. Peng, C. Rainer. Mean-eld stochastic dierential equations and associated PDEs. Ann. Probab., Vol. 45, No. 2 (2017), 824-878. [5] Pierre Cardaliaguet. Notes on Mean Field Games (from P.-L. Lions' lectures at College de France), 2013. [6] R. Carmona, F. Delarue. Probabilistic analysis of mean eld games. SIAM Journal on Control and Optimization, Vol. 51, No. 4 (2013), 2705-2734. [7] R. Carmona, F. Delarue. Forwardbackward Stochastic Dierential Equations and Con- trolled McKeanVlasov Dynamics, The Annals of Probability, Vol. 43, No. 5 (2015), 2647-2700. [8] Rene Carmona. Lectures on BSDEs, stochastic control, and stochastic dierential games with nancial applications, SIAM, 2016. [9] Cedric Villani. Optimal transport, old and new, Springer, 2008. [10] R. Cont, D.A. Fourni e. Functional It^ o Calculus and Stochastic Integral Representation of Martingales, The Annals of Probability, Vol. 41, No. 1 (2013), 109-133. [11] Ibrahim Ekren, Nizar Touzi, Jianfeng Zhang, Optimal stopping under nonlinear expec- tation, Stochastic Processes and their Applications, Vol. 124, No. 10 (2014), 3277-3311. [12] Richard Hamilton. The inverse function theorem of Nash and Moser, Bulletin of the American Mathematical Society, Vol. 7, No. 1 (1982), 65-222. 79 [13] M. Huang, R. P. Malham e, P. E. Caines. Large population stochastic dynamic games: Closed-loop McKeanVlasov systems and the Nash certainty equivalence principle. Com- munications in Information and Systems, Vol. 6, No. 3 (2006), 221-252. [14] I. Katatzas, S. E. Shreve. Brownian Motion and Stochastic Calculus, Second Edition, Graduate Texts in Mathematics, Vol. 113, Springer, 1991. [15] N. Karoui, S. Peng, M. C. Quenez. Backward stochastic dierential equations in nance. Mathematical Finance, Vol. 7, No. 1 (January 1997), 1-71. [16] J. M. Lasry, P. L. Lions. Jeux a champ moyen. I. Le cas stationnaire. C. R. Math. Acad. Sci. Paris, Vol. 343 (2006), 619-625. [17] J. M. Lasry, P. L. Lions. Jeux a champ moyen. II. Horizon ni et contr^ ole optimal. C. R. Math. Acad. Sci. Paris, Vol. 343 (2006), 679-684. [18] J. M. Lasry, P. L. Lions. Mean eld games. Japanese Journal of Mathematics, Vol. 2, No. 1 (March 2007), 229-260. [19] Juan Li, Min Hui. Weak Solutions of Mean-Field Stochastic Dierential Equations and Application to Zero-Sum Stochastic Dierential Games, SIAM J. Control Optim., Vol. 54, No. 3 (2016), 1826-1858. [20] Ludger Rschendorf. Mathematical Risk Analysis: Dependence, Risk Bounds, Optimal Allocations and Portfolios, Springer Series in Operations Research and Financial Engi- neering, 2013. [21] A. Snitzman. Topics in propagation of chaos, Ecole d'Et e de Probabilits de Saint-Flour XIX { 1989, pp 165-251, Springer, 1989. [22] S.M. Srivastava. A Course on Borel Sets, Graduate Texts in Mathematics, Vol 180, Springer, 1982. [23] Nizar Touzi. Optimal Stochastic Control, Stochastic Target Problems, and Backward SDE, Fields Institute Monographs, Vol 29, Springer, 2013. [24] Cedric Villani. Optimal transport: old and new, Springer, Berlin, 2009. [25] Jiongmin Yong, Xun Yu Zhou. Stochastic Controls: Hamiltonian Systems and HJB Equations, Applications of Mathematics, Vol 43, Springer, 1999. [26] W. A. Zheng, Tightness results for laws of diusion processes application to stochastic mechanics, Ann. Inst. Henri Poincar e, Vol. 21, No. 2 (1985), 103-124. 80

Abstract (if available)

Abstract The main objective of this research is to provide some answers to control problems of McKean-Vlasov dynamics (or called mean field control problems), which is closely related to the MFG problem. The HJB equation could be derived for the value function, which is defined on the infinite dimensional space of continuous paths. The derivation of HJB equation relies on a generalized form of Ito's formula to the functional case. We argue that weak formulation is the right setting to use in practice and theory.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Pathwise stochastic analysis and related topics

PDF

Dynamic approaches for some time inconsistent problems

$Stochastic differential equations driven by fractional Brownian motion and Poisson jumps$

PDF

Stochastic differential equations driven by fractional Brownian motion and Poisson jumps

PDF

Conditional mean-fields stochastic differential equation and their application

PDF

Optimal dividend and investment problems under Sparre Andersen model

PDF

Equilibrium model of limit order book and optimal execution problem

PDF

Elements of dynamic programming: theory and application

PDF

Optimal and exact control of evolution equations

PDF

Topics on set-valued backward stochastic differential equations

PDF

Path dependent partial differential equations and related topics

PDF

Set values for mean field games and set valued PDEs

PDF

On spectral approximations of stochastic partial differential equations driven by Poisson noise

PDF

Some topics on continuous time principal-agent problem

PDF

Forward-backward stochastic differential equations with discontinuous coefficient and regime switching term structure model

PDF

Reinforcement learning for the optimal dividend problem

PDF

Numerical methods for high-dimensional path-dependent PDEs driven by stochastic Volterra integral equations

PDF

Optimal investment and reinsurance problems and related non-Markovian FBSDES with constraints

PDF

On stochastic integro-differential equations

PDF

On the non-degenerate parabolic Kolmogorov integro-differential equation and its applications

PDF

Tamed and truncated numerical methods for stochastic differential equations

Asset Metadata

Creator Wu, Cong (author)

Core Title Controlled McKean-Vlasov equations and related topics

School College of Letters, Arts and Sciences

Degree Doctor of Philosophy

Degree Program Applied Mathematics

Publication Date 07/11/2017

Defense Date 05/10/2017

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag dynamic programming principle,HJB equation,MKV SDE,OAI-PMH Harvest,stochastic control,viscosity solution

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Zhang, Jianfeng (committee chair)

Creator Email congw@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c40-396999

Unique identifier UC11265653

Identifier etd-WuCong-5498.pdf (filename),usctheses-c40-396999 (legacy record id)

Legacy Identifier etd-WuCong-5498.pdf

Dmrecord 396999

Document Type Dissertation

Rights Wu, Cong

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA