Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Optimal and exact control of evolution equations
(USC Thesis Other)
Optimal and exact control of evolution equations
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
OPTIMALANDEXACTCONTROLOFEVOLUTIONEQUATIONS by QianSong ADissertationPresentedtothe FACULTYOFTHEGRADUATESCHOOL UNIVERSITYOFSOUTHERNCALIFORNIA InPartialFulfillmentofthe RequirementsfortheDegree DOCTOROFPHILOSOPHY (APPLIEDMATHEMATICS) August2008 Copyright 2008 QianSong Acknowledgements First I must address my appreciation to my advisor Sergey Lototsky. I am very fortunate to haveSergeyasmyadvisor-heisnotonlyabrilliantresearcherbutalsoawarmheartedteacher. Withouthispatience,guidance,andencouragement,Icouldnothavemadethisfar. IalsothankYonghengDeng,IgorKukavica,RemigijusMikulevicius,andJianfengZhang forservingasmycommitteemembers. Thisdissertationcouldnothavebeencompletedwith- outtheirsupportandvaluableadvices. Inaddition,IamverythankfultoGaryRosen. Hefirst showedmethebeautyofmathematicsandencouragedmetostartmyjourneyintomathemat- ics. Finally,Iwouldliketothankmyparentsfortheirloveandunconditionalsupport,andmy husband,YangYu,forhislove,passionandoptimism. ii TableofContents Acknowledgements ii Abstract iv Chapter1: Optimalcontrolvs. exactcontrol 1 Chapter2: Ordinarydifferentialequations: Optimalcontrol 4 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 Thedeterministiccase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.2.1 Fromcalculusofvariationstooptimalcontrol . . . . . . . . . . . . . 6 2.2.2 Thelinearquadraticcontrolproblem . . . . . . . . . . . . . . . . . . 17 2.3 Thestochasticcase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.3.1 Themaximumanddynamicprogrammingprinciples . . . . . . . . . 26 2.3.2 Thelinearquadraticcontrolproblem . . . . . . . . . . . . . . . . . . 30 Chapter3: Ordinarydifferentialequations: Exactcontrol 39 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.2 Motivationalexamples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 3.2.1 Open-loopcontrol . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 3.2.2 Feed-backcontrolinthedrift . . . . . . . . . . . . . . . . . . . . . . 45 3.2.3 Feed-backcontrolinthediffusion . . . . . . . . . . . . . . . . . . . 47 3.3 Equationswithsingulardrift . . . . . . . . . . . . . . . . . . . . . . . . . . 49 3.3.1 Thelogarithmictransformation . . . . . . . . . . . . . . . . . . . . 49 3.3.2 Thegeneralcase . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 Chapter4: Pararabolicequations: Optimalcontrol 78 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 4.2 Parabolicregularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 4.3 Thelinearquadraticcontrolproblem . . . . . . . . . . . . . . . . . . . . . . 87 Chapter5: Parabolicequations: Exactcontrol 96 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 5.2 Deterministicparabolicequations . . . . . . . . . . . . . . . . . . . . . . . 98 5.3 Stochasticparabolicequations . . . . . . . . . . . . . . . . . . . . . . . . . 102 References 111 iii Abstract Theobjectiveofoptimalcontrolistooptimize(minimizeormaximize)acertainperformance index. The objective of exact control is to ensure that the solution takes a prescribed value at the prescribed time. This dissertation carries out a comparative analysis of these two control problems and derives several general methods of constricting exact feed-back and open-loop controls for deterministic and stochastic evolution equations. The exact feed-back control is carried out using the new notion of a driving function. Several procedures for the optimal selection of the driving function are discussed. It is proved that the driving function must developanon-integrablesingularityattheterminaltime. iv Chapter1 Optimalcontrolvs. exactcontrol Even though the original meaning of the word control is a duplicate register in accounting (counter-roll ),themostcommoncurrentmeaningisregulationordomination. Accordingly,to control an evolution equation means to choose an appropriate input ensuring that the solution hasspecifiedproperties. Thisinputiscalledthecontrolfunction,orsimplycontrol. The objective of optimal control is to optimize (minimize or maximize) a certain perfor- manceindex(orcriterion). Mathematically,aperformanceindexisafunctionalofthesolution andthecontrolfunction,andisalsoknownasthecost(orloss)functionalwhentheobjective isminimization,andthevaluefunctionalwhentheobjectiveismaximization. Dependenceof the performance index on the control function is crucial, because in applications the control does not come free. The control function that optimizes the performance index is often of feed-back type, that is, at every time moment depends only on the current value of the solu- tion and other inputs in the equation. Chapter 2 summarizes the main facts from the optimal controltheoryandinvestigatesseveralkeyexamplesofthelinearquadraticregulator. The objective of the exact control is to ensure that the solution takes a prescribed value at the prescribed time. While it might also be important to impose further restrictions on the control, the mathematical formulation turns out somewhat different from the optimal control problem. Roughly speaking, the reason is as follows. We know that the solution of every “nice”equationisuniquelydeterminedbytheinitialconditions,andthereforecannotbeforced totakeaprescribedvalueataprescribedtime. Asaresult,givenatypicalcostfunctional,any exact feed-back control will have an infinite cost. For deterministic equations, this difficulty 1 is resolved by an open-loop control, an input that is not related to the current value of the solution. Forstochasticequations,suchanopen-loopcontroloftenrequiresknowledgeofthe completenoisetrajectory,andisthereforehardlyeverimplementable. The main result of the dissertation can be stated as follows: given the terminal condition G,thesolutionoftheevolutionequation ˙ Y(t)+A(t,Y(t)) =− Y(t)−G T −t +f(t) (1.0.1) (linearornonlinear,deterministicorstochastic,undersuitableassumptionsontheoperatorA andthefreetermf)hastheproperty lim t→T − Y(t) =G. (1.0.2) In other words, ¯ u(t,x) = −K(t)(x−G), with K(t) = (T −t) −1 , is an exact feed-back control, driving the solution of ˙ X(t) +A(t,X(t)) = ¯ u(t,X(t)) +f(t) toG at timeT. The followingquestionsarealsoaddressed: 1. Existenceanduniquenessofsolutionof(1.0.1); 2. The precise meaning of convergence (1.0.2) (this question is essential for stochastic equationsandequationsininfinitedimensions,suchaspartialdifferentialequations); 3. PossibilitytoreplaceK(t) = (T −t) −1 withotherfunctionsoftime. 4. Optimalityofthefeed-backcontrol ¯ u. In particular, Section 3.3.2 introduces a new notion of the driving function to describe the class of functions that can be used to implement an exact feed-back control. It is proved that suchfunctionsmusthaveanon-integrablesingularityneartheterminaltime T. 2 Notethatthenon-integrabilityofthedrivingfunction K(t)on[0,T]impliesthattheexact feed-back control is never square-integrable in time. This is consistent with uniqueness of solutionforafist-orderevolutionequationandrequiresanewnotionofoptimalityviaacertain limitingprocedure. Forstochasticordinarydifferentialequations,onemethodistoconsidera sequenceofoptimalcontrolproblemsontheintervals[0,T−ε];theresultingoptimalcontrol is related to the logarithmic transformation and pinned diffusions. Unfortunately, there is no clearanalogueofthisapproachfordeterministicequationsandstochasticparabolicequations, andtheexplicitexpressionfortheresultingcontrolrarelyexistsevenforlinearequations. Analternativemethodforconstructingtheexactfeed-backcontrolthatisoptimalinsome sense is to pass to the limit of vanishing control cost in a linear regulator problem. Despite the obvious draw-back — restriction only to linear problems, this method works for both deterministic and stochastic equations in either finite and infinite dimensions, and usually leads to an explicit form of the control function. In particular, Chapters 4 and 5 study the correspondingcontrolproblemsforlinearparabolicequations. A remarkable fact is that, for the standard Brownian motion, both the logarithmic trans- formationandthelinearquadraticregulatorapproachesleadtothesameresult,theBrownian bridge. The result turns out the same also for the Ornstein-Uhlenbeck process, leading to the notion of the Ornstein-Uhlenbeck bridge. In Chapter 5, the Ornstein-Uhlenbeck bridge is extendedtoinfinitedimensionsusingthelinearregulatormethod. 3 Chapter2 Ordinarydifferentialequations: Optimal control 2.1 Introduction Before there was optimal control, there were two somewhat unrelated branches of mathemat- ics: calculusofvariationsandthecontroltheory. While the foundations of the calculus of variations, Fermat’s principle of least time, the brachistochrone problem, and the minimal surface of revolution problem, go back to 17th century,thenamecalculusofvariationstodescribethecorrespondingbranchofmathematics, was suggested in 1756 by L. Euler, and it took nearly two more centuries to develop this branch. In1900,D.Hilbertincludedcalculusofvariationsinthreeofhisfamous23problems, and23rdproblemwasexplicitlyaskingforfurtherdevelopmentsofthesubject. The control theory started as the study of regulators (governors), such as the centrifugal governor suggested in 1788 by James Watt (1736–1819) to regulate the output of a steam enginebyautomaticallycontrollingthepressureintheboiler. Thefirstmathematicalanalysis ofthisandsimilarregulatorswascarriedoutin1868byJamesClerkMaxwell(1831–1879). Interestingly enough, the optimal control, in which the main problem is minimization or maximization of functionals, is more closely connected with calculus of variations than with thecontroltheory,partlybecauseformanyproblemstheoptimalcontrolisisnotcomputable explicitly, which complicates analysis of stability and other questions studied by the control 4 theory. Still, while many books on the subject of optimal control, such as Cannarsa and Sinestrari[12],FlemingandRishel[21]orYoung[53],startwiththediscussionofcalculusof variations and do not attempt to establish any connection with control theory, others, such as Lee and Markus [33], connect optimal control with control theory without explicit reference tocalculusofvariations. Theoneexampleinwhichtheoptimalcontroliscomputableexplicitly,andofteninclosed form, is the linear equation (either deterministic or stochastic), with quadratic performance index. Fordeterministicequations,thestudyofthislinear-quadraticproblemisthesubjectof manybooks,suchasAndersonandMoore[2],Trentelmanetal. [49],orWonham[50]. Of the two approaches to solving the optimal control problem, via the Hamilton-Jacobi- Bellman(HJB)equationorviathemaximumprinciple,thefirstoneseemstobemorepopular, partly because there is a relatively straightforward formula for the optimal control using the derivatives of the solution of the HJB equation. Since the HJB equation often does not have classicalsolutions,amoresophisticatedanalysisisnecessaryusingthenotionoftheviscosity solution (Bardi and Capuzzo-Dolcetta [4], Cannarsa and Sinestrari [12], Fleming and Soner [23],etc.) The HJB approach seem even more popular in the stochastic setting, where the equation isoftenasemi-linearparabolicwithawell-developedtheory,whereasthemaximumprinciple requires the solution of a large system of backward stochastic differential equations, which is a much less familiar object. While the maximum principle was used by Bismut as early as mid-1970’s [9], the complete statement of the principle was not available until the late 1980sevenfortheItˆ odiffusions(seePeng[36]). Themaindifficultyofthestochasticcontrol problem is the possibility to control the noise intensity, which does not have an analogue in the deterministic setting. Accordingly, open questions remain even for the stochastic linear quadraticproblem(see,forexamplethepapersbyChenandZhou[15,14]). Theworkaimed 5 atunifyingtheHJBandthemaximumprincipleapproachesisalsoongoing(see,forexample Zhou[55,57]andYongandZhou[52]). 2.2 Thedeterministiccase 2.2.1 Fromcalculusofvariationstooptimalcontrol Inthemulti-variablecalculus,oneofthecentralproblemsisfindingtheextremal(minimalor maximal) values of functions of several variables. Calculus of variations addresses the same problem for functionals. A functional is a mapping from a set of functions to a Euclidean space. Forexample,ifL =L(t,y,z) : [0,T]×R n ×R n →R isacontinuousfunction,then, forfixedt 0 ,t 1 ∈ [0,T], J(x(·)) = Z t 1 t 0 L(s,x(s), ˙ x(s))ds, ˙ x(s) = dx(s) ds (2.2.1) is a functional on the space of continuously differentiable, R n -valued functions on [t 0 ,t 1 ]. MinimizingJ(x(·))overallx∈{x(·)∈C 1 ([t 0 ,t 1 ];R n ) : x(t 0 ) =a, x(t 1 ) =b,}givenfixed a andb is known as the simplest problem in calculus of variations [21, Section I.3]; the term the simplest problemwas,infact,usedbyD.Hilbertinthestatementofhis23rdproblem. IfListwicecontinuouslydifferentiableinallitsarguments,theneveryfunctionminimiz- ingormaximizing(2.2.1)mustsatisfytheEuler-Lagrangeequations: d ds (L z i (s,x(s), ˙ x(s)))−L y i (s,x(s), ˙ x(s)) = 0, t 0 ≤s≤t 1 , i = 1,...,n, (2.2.2) 6 whereL y i = ∂L(t,y,z)/∂y i ,L z i = ∂L(t,y,z)/∂z i ; these equations are derived by equating tozerothefirstvariationofJ: d dε J(x(·)+εy(·)) ε=0 = 0 and requiring the equality to hold for all continuously-differentiable functions y =y(t) satis- fyingy(t 0 ) =y(t 1 ) = 0;see[18,Section3.3.1]or[21,SectionI.3]. Forthesimplestproblem in calculus of variations, conditionsx(t 0 ) =a, x(t 1 ) =b make (2.2.2) a nonlinear two-point boundary value problem. This problem can have several solutions, or none at all. Here is one generalresultabouttheregularityofthesolution. 2.2.1Proposition (see [21, Corollary I.3.3]). If the functionL is twice continuously differen- tiable in all it is arguments and the matrix L z i z j , i,j = 1,...,n is positive-definite for all t,y,z,theneverysolutionof (2.2.2)withboundaryconditionsx(t 0 ) =a, x(t 1 ) =bisatwice continuously differentiable function on[t 0 ,t 1 ]. In general, regularity of solutions in the problems of calculus of variations is a difficult question,andwasthesubjectofHilbert’s19thproblem. Itiswell-knownthatsystem(2.2.2)of nsecond-orderequationscanbereducedtoasystem of 2n first-order equations, for example, by setting ˙ x(s) =v(s). In what follows, we will see that a different reduction can be possible, so that the resulting system of 2n equation has specialstructureandsymmetry. GivenaC 2 functionx(·)thatsolvestheEuler-Lagrangeequations,wedefine p(s) =L ˙ x (s,x(s), ˙ x(s)), t 0 ≤s≤t 1 . 7 and suppose that, for all x,p ∈ R n , the equations p i = L z i (s,x,z), i = 1,...,n, can be uniquelysolvedforz asasmoothfunctionofs,pandx: z =q(s,x,p) (2.2.3) Withthefunctionq from(2.2.3),definetheHamiltonianH associatedwithL: H(t,x,p) = n X i=1 p i q i (t,x,p)−L(t,x,q(t,x,p)), p,x∈R n , (2.2.4) andthen,thecorrespondingsystemofHamilton’sequations: ˙ x i (s) =H p i (s,x(s),p(s)), ˙ p i (s) =−H x i (s,x(s),p(s)), i = 1,...,n. (2.2.5) In mechanical applications, the functionsx = x(s) and ˙ x = ˙ x(s) represent the position and velocityofaparticle,andH,thetotalenergy(kineticpluspotential). Thefunctionpisthenthe momentum of the particle, andL is the difference between the kinetic and potential energies; the functionL is often known as the Lagrangian. Hamilton’s principle, or the principle of the leastaction,statesthatthetrajectoryx ∗ =x ∗ (s)oftheparticleminimizestheactionfunctional (2.2.1)forthegivenLagrangianL. The following result provides a correspondence between the solution x(·), the Euler- Lagrangeequations(2.2.2)andthesolutionx(·),p(·)oftheHamiltonequations(2.2.5). 2.2.2 Proposition (see [52, Proposition 5.2.1]). Assume that the function L = L(t,y,z) is twice continuously differentiable and the function q = q(s,x,p) exists and is contin- uously differentiable in x. For a continuously differentiable function x = x(s), define p i (s) = L z i (s,x(s), ˙ x(s)), i = 1,...,n. Then the functions x(·),p(·) satisfy (2.2.5) if and only if the functionx(·) satisfies(2.2.2). 8 Notice that the correspondence between (2.2.2) and (2.2.5) essentially relies on the exis- tence and regularity of the functionq in (2.2.3), although only the HamiltonianH is used in the end. Under certain conditions, the HamiltonianH can be computed from the Lagrangian L via the Legendre transformation. For example, if, for every t and x, the function L is a convexfunctionofz andgrowsfasterthan|z|atinfinity,then H(t,x,p) = sup z∈R n n X i=1 p i z i −L(t,x,z) ! ; (2.2.6) see[18,Section3.3.2]. Next,givenafunctionH =H(t,x,p),considerthenonlinearpartialdifferentialequation u t +H(t,x,u x 1 ,...,u xn ) = 0. (2.2.7) Thecharacteristiccurve(x(s),z(s),p(s),τ(s))forthisequationsatisfies ˙ x i (s) =H p i (s,x(s),p(s)), i = 1,...,n; ˙ z(s) = n X i=1 H p i (s,x(s),p(s))p i (s)−τ(s); ˙ p i (s) =−H x i (s,x(s),p(s)), i = 1,...,n; ˙ τ(s) =−H t (s,x(s),p(s)); (2.2.8) see [18, Sections 3.2.1 and 3.2.5(c)]. In other words, the components x(s) and p(s) of the characteristiccurvefor(2.2.7)satisfyHamilton’ssystem(2.2.5). Nonlinearpartialdifferential equation(2.2.7)iscalledtheHamilton-Jacobiequation. Ingeneral,knowledgeofthecharacteristiccurvesofafirst-orderpartialdifferentialequa- tion leads to the solution of the equation. In the context of (2.2.8) and (2.2.7), this procedure works as follows. Ifu(0,x) = g(x) is the initial condition in (2.2.7), then we solve (2.2.5) 9 with initial conditionsx(0) = y andp i (0) = g x i (y),i = 1,...,n fory ∈R n . After that, we solve the equationx = x(t,y) to gety as a function oft,x: y = φ(t,x), and then, by direct computation,verifythatthefunction u(t,x) =g(y)+ Z t 0 n X i=1 H p i (s,x(s,y),p(s,y))p i (s,y) −H(s,x(s,y),p(s,y)) ! ds, (2.2.9) withy =φ(t,x),isasolutionof(2.2.7);see[52,Theorem5.2.4]fordetails. The transition in the opposite direction, from a solution of (2.2.7) to a solution of (2.2.5) is carried out using the notion of the complete integral. By definition, a complete integral of (2.2.7) is a family of functions u = v(t,x,a) parameterized by a ∈ R n such that v(t,x,a) satisfies (2.2.7) for every fixeda and the determinant of the matrix v a i x j , i,j = 1,...,n is non-zeroforall t,x,a. Thenthefollowingresultholds. 2.2.3Proposition(see[52,Theorem5.2.3]). Ifv(t,x,a)+c is a complete integral of (2.2.7) and, for every(t,a,b)∈R 1+2n , the functionsx(t,a,b), p(t,a,b) exist so that v a i (t,x(t),a) =b i , v x i (t,x(t),a) =p i (t), i = 1,...,n, then, for everya,b, the functionsx(t,a,b), p(t,a,b) satisfy(2.2.5). Tosummarize,theoptimizationproblem(2.2.1)withtheLagrangianLleadstotheEuler- Lagrange equations (2.2.2), and, via the Hamiltonian (2.2.4), to the Hamilton equations (2.2.5), which, in turn, are a component of the characteristic curve for the Hamilton-Jacobi equation (2.2.7). Conversely, a complete integral of (2.2.7) leads to a family of solutions of (2.2.5) via Proposition 2.2.3, and every solution of (2.2.5) leads to a solution of (2.2.2) via Proposition2.2.2. 10 The Euler-Lagrange equations (2.2.2) were first written in 1744 by the Swiss mathemati- cian Leonhard Euler (1707–1783) and further investigated in the early 1750s by the French mathematician Joseph-Louis Lagrange (1736–1813). Following the work of Lagrange, Euler in 1756 introduced the term “calculus of variations”. System of equations (2.2.5) was intro- duced in early 1830s by the Irish mathematician William Rowan Hamilton (1805–1865) as a partofhisnewanalyticalapproachtoclassicalmechanics. Shortlyafterthat,heintroducedthe partial differential equation (2.2.8), which was almost immediately investigated by the Ger- man mathematician Karl Gustav Jacob Jacobi (1804–1851). In particular, Proposition 2.2.3 is known as the Hamilton-Jacobi theorem. The Legendre transformation (2.2.6) appears in many problems beside calculus of variations and is named after the French mathematician Adrien-MarieLegendre(1752–1853). The problem of minimizing (2.2.1) does not directly include any control. A more general optimizationprobleminvolvescontrolandiscalledtheoptimalcontrolproblem. Thisproblem can be stated as follows. LetU be the set of measurable functions from the interval [t 0 ,t 1 ] to aseparablemetricspaceU. ConsiderafamilyofdynamicalsystemsinR n , ˙ x u (t) =f(t,x u (t),u(t)), t 0 <t≤t 1 , x(t 0 ) =x 0 , (2.2.10) indexed by u ∈ U. Assume that the functionf = f(t,x,u) is regular enough for equation (2.2.10) to have a unique global solution for every initial conditionx 0 and everyu∈U. For example,Lipschitzcontinuityandlineargrowthoff inx,uniformlyint,u,isenough: |f(t,x,u)−f(t,y,u)|≤C|x−y|, |f(t,x,u)|≤C(1+|x|), (2.2.11) for all x,y ∈ R n , with C independent of t,u. Given an initial condition x 0 , the trajectory (2.2.10) is completely determined by the function u. It is therefore natural to refer to the 11 functionu as control. A control is called feed-back if u(t) = v(t,x u (t)) for some function v =v(t,x). Acontroliscalledopen-loopif u(t)doesnotexplicitlydependonx(t). LetF = F(t,x,u) be a scalar function with the property (2.2.11) andh = h(x), a scalar Lipschitzcontinuousfunction. Foreverytrajectory(2.2.10),definethecostfunctional J(t 0 ,x 0 ;u(·)) = Z t 1 t 0 F(t,x u (t),u(t))dt+h(x u (t 1 )). (2.2.12) The optimal control problem is to find a functionu so thatJ(t 0 ,x 0 ;u(·)) is as small as possi- ble: J(t 0 ,x 0 ;u(·))→ min, (2.2.13) withminimizationcarriedoutoverthesetU. Thefunctionu ∗ suchthat J(t 0 ,x 0 ;u ∗ (·)) = min u(·)∈U J(t 0 ,x 0 ;u(·)) (2.2.14) is called an optimal control. Existence of u ∗ can be established for a very large class of problems(forexample,see[52,Theorem2.5.2]),buttheseexistenceresultshelpverylittlein findingu ∗ . 2.2.4Definition. (a) The Hamiltonian of the optimal control problem(2.2.13) is the function ¯ H(t,x,u,p) = n X i=1 p i f i (t,x,u)−F(t,x,u); (2.2.15) t∈ [t 0 ,t 1 ], x∈R n , u∈U, p∈R n . (b) The value function of the optimal control problem(2.2.13) is the function V(t,x) = inf u(·)∈U J(t,x;u(·)), x∈R n , t∈ [t 0 ,t 1 ]. (2.2.16) 12 Later,wewilldiscusstheconnectionbetweenthefunction ¯ H in(2.2.15)andtheHamilto- nian(2.2.4). Notethat,bydefinition, V(t 1 ,x) =h(x). (2.2.17) Recallthatanoptimalcontrolu ∗ =u ∗ (t)satisfies J(t 1 ,a;u ∗ (·)) = min u(·)∈U J(t 1 ,a;u(·)). Denote byx ∗ = x ∗ (t) the solution of (2.2.10) corresponding tou = u ∗ , The following result illustratestheconnectionbetweenthefunctions ¯ H,V andtheoptimalcontrolu ∗ . 2.2.5Theorem. (a)[MaximumPrinciple,[52,Theorem3.2.1]] There exists a functionp ∗ = p ∗ (t) with values inR n so that ˙ x ∗ i (t) = ¯ H p i (t,x ∗ (t),u ∗ (t),p ∗ (t)), x ∗ i (t 0 ) =a i , i = 1,...,n; ˙ p ∗ i (t) =− ¯ H x i (t,x ∗ (t),u ∗ (t),p ∗ (t)), p ∗ i (t 1 ) =−h x i (x(t 1 )), i = 1,...,n; ¯ H(t,x ∗ (t),u ∗ (t),p ∗ (t)) = max u∈U ¯ H(t,x ∗ (t),u,p ∗ (t)). (2.2.18) (b) [Dynamic Programming principle, [52, Theorem 4.2.1]] If x u is a trajectory of (2.2.10) withx u (t) =y, then V(t,y) = inf u(·)∈U Z τ t F(s,x u (s),u(s))ds+V(τ,x u (τ)) , t 0 <t≤τ ≤t 1 . (2.2.19) (c) [Verification Principle, [52, Theorem 5.3.1]] Assume that the value function 13 V = V(t,x) is twice continuously differentiable and denote by V x the column vector (V x 1 ,...,V xn ) > . Then V t (t,x ∗ (t)) = ¯ H(t,x ∗ (t),u ∗ (t),−V x (t,x ∗ (t)) = max u∈U ¯ H(t,x ∗ (t),u,−V x (t,x ∗ (t)), V x (t,x ∗ (t)) =−p ∗ (t), (2.2.20) and the optimal controlu ∗ is of feed-back type: u ∗ (t) = ¯ u(t,x ∗ (t)), where ¯ u(t,x) = argmin u∈U F(t,x,u)+V > x (t,x)f(t,x,u)) . (2.2.21) An immediate consequence of (2.2.19) is the Hamilton-Jacobi-Bellman (HJB) equation satisfiedbythevaluefunctionV: −V t +sup u∈U ¯ H(t,x,u,−V x ) = 0, t 0 <t<t 1 , V(t 1 ,x) =h(x), (2.2.22) or V t + inf u∈U f > (t,x,u)V x +F(t,x,u) = 0, t 0 <t<t 1 , V(t 1 ,x) =h(x). (2.2.23) An immediate consequence of (2.2.21) is that the function x ∗ = x ∗ (t), the trajectory of (2.2.10)correspondingtotheoptimalcontrol,satisfies ˙ x ∗ (t) =f(t,x ∗ (t),¯ u(t,x ∗ (t))). (2.2.24) In calculus of variations,U is a subset of measurableR m -valued functions. For example, 14 theproblemofminimizing(2.2.1)canbereducedto(2.2.10),(2.2.12),(2.2.13)byconsidering ann+1dimensionaldynamicalsystem(x u ,y u ),x u ∈R n ,y u ∈R,u∈R n ,sothat ˙ x u (t) =u(t), x u (t 0 ) =a, ˙ y u (t) =L(t,x u (t),u(t)), y u (0) = 0, (2.2.25) withthecostfunctional J(t 0 ,a;u(·)) =y u (t 1 ). (2.2.26) The setU is the collection of piece-wise continuous R n -valued functions on [t 0 ,t 1 ] satisfying R t 1 t 0 u(t)dt =b−a. Inparticular,U =R n ,F = 0,h((x,y)) =y. While we call the problem (2.2.13) the optimal control problem, there are very similar- looking problems that are nonetheless belong to calculus of variation; it can be difficult to draw a precise boundary between calculus of variations and optimal control. As a rule, in the calculusofvariations,dynamicalsystemshaveamorespecialform(suchas(2.2.25))andthe setU of the values of the control is open; in optimal control, there are fewer restrictions on thedynamicalsystemandthesetU isclosed. Let us look at equations (2.2.18) and (2.2.22) for the problem (2.2.25), (2.2.26). First of all,notethat ¯ H(t,(x,y),u,(p,q)) = n X i=1 p i u i +qL(t,x,u) (2.2.27) and V(t,(x,y)) = inf u(·)∈U y u (t 1 )+y. (2.2.28) Using(2.2.26),whichimpliesh((x,y)) =y,and(2.2.27),whichimplies ¯ H y = 0,theequation forq ∗ is ˙ q ∗ (t) = 0, q ∗ (t 1 ) =−1, (2.2.29) 15 andsoq ∗ (t) =−1. Asaresult, ¯ H t,(x ∗ (t),y ∗ (t)),u,(p ∗ (t),q ∗ (t)) = n X i=1 u i p ∗ i (t)−L(t,x ∗ (t),u). (2.2.30) andthus ¯ H t,(x ∗ (t),y ∗ (t)),u ∗ (t),(p ∗ (t),q ∗ (t)) =H(t,x ∗ (t),p ∗ (t)), (2.2.31) where the function H is the Hamiltonian (2.2.6). In particular, the functions (x ∗ (t),p ∗ (t)) satisfyHamilton’sequations(2.2.5). Similarly,(2.2.28)impliesV y = 1,andthen ¯ H t,(x,y),u,(−V x ,−V y ) =− n X i=1 u i V x i (t)−L(t,x,u). By (2.2.22), the function v(t,x) = −V(t,(x,y 0 )) satisfies the Hamilton-Jacobi equation (2.2.7) with terminal conditionv(t 1 ,x) = −y 0 and with functionH given by (2.2.6). Thus, while the Euler-Lagrange equations from calculus of variations do not generalize to the opti- malcontrolproblem(2.2.13),bothHamilton’sequations(2.2.5)andHamilton-Jacobiequation (2.2.7)do,asthemaximumprincipleandtheHamilton-Jacobi-Bellmanequation,respectively. Both the maximum principle and dynamic programming principle are results of a team effort. The maximum principle was announced in 1956 following the work of a group led by LevSemenovichPontryagin(1908–1988)attheSteklovInstituteofMathematicsinMoscow; accordingly, the result is also known as Pontryagin’s maximum principle. The dynamic pro- grammingprinciplewasannouncedin1952followingtheworkagroupofmathematiciansdid at the RAND Corporation in Santa Monica, CA. One of the leaders of the group was Richard Ernest Bellman (1920–1984), who published the result in the Proceedings of the National Academy of Sciences [6]; accordingly, the result is also known as Bellamn’s dynamic pro- grammingprinciple. 16 As stated above, the verification principle is a straightforward consequence of the other two [52, Theorem 5.3.7], but has limited application because the value function is usually never twice continuously differentiable. Extensions of the verification principle rely on the notionsofsub-andsuper-differentialsandviscositysolution[52,Theorem5.3.16]. Aseparate questionisexistenceanduniquenessofsolutionofthecorrespondingequation(2.2.24),which is mostly open even when the value function is smooth. Interestingly enough, this question is easier in the stochastic case, as the conditions for existence of a (weak in the probabilistic sense)solutionaremuchlessrestrictive. 2.2.2 Thelinearquadraticcontrolproblem The main example for which the optimal control problem (2.2.10), (2.2.12), (2.2.13) has an explicitsolutionisthelinearquadraticcontrol(orlinearregulator)problem: Z T 0 x > (t)Q(t)x(t)+2u > (t)S(t)x(t)+u > (t)R(t)u(t) dt+x > (T)Gx(T)→ min (2.2.32) overu(·) ∈ C((0,T);R m ), whereQ = Q(t), S = S(t) ∈ R n×m , R = R(t) ∈ R m×m are bounded measurable matrix-valued functions, G∈R n×n , withQ(t), G, R(t) symmetric for everyt≥ 0,x > meansthetransposeofx,and ˙ x(t) =A(t)x(t)+B(t)u(t)+b(t), x(0) =x 0 , (2.2.33) x(t) ∈ R n , u(t) ∈ R m , A = A(t) ∈ R n×n , B = B(t) ∈ R n×m are bounded measurable matrix-valued functions; b =b(t)∈R n is a known continuous function. Comparing (2.2.33) with (2.2.10), note that we now simplify the notations by no longer writing the super-script u inx. 17 2.2.6 Theorem. Assume that the matrix-valued function R = R(t) is uniformly positive def- inite on [0,T]: there exists aδ > 0 such that the matrixR(t)−δI is positive definite for all t∈ [0,T], whereI is the identity matrix. Then the following results hold. (1)[52,Page294] The matrix differential equation ˙ P(t)+P(t)A(t)+A > (t)P(t)+Q(t) − B > (t)P(t)+S(t) > R −1 B > (t)P(t)+S(t) = 0, P(T) =G, (2.2.34) has at most one solution in the class of continuously differentiableR n×n -valued functions on [0,T]. If exists, the matrixP(t) is symmetric for everyt∈ [0,T]. (2)[52,Theorem6.2.8] IfP =P(t)isthesolutionofequation(2.2.34),thentheoptimal controlu ∗ is of feed-back type u ∗ (t) = ¯ u(t,x ∗ (t)), where ¯ u(t,x) =−R −1 (t) B > (t)P(t)+S(t) x+B > (t)ϕ(t) (2.2.35) andϕ =ϕ(t)∈R n is the solution of ˙ ϕ(t)+ (A(t)−B(t)R −1 (t)S(t)) > −P(t)B(t)R −1 (t)B > (t) ϕ(t) +P(t)b(t) = 0, ϕ(T) = 0; (2.2.36) note thatϕ(t) = 0 ifb(t) = 0. The value function of the problem is V(t,x) =x > P(t)x+2ϕ > (t)x + Z T t 2ϕ > (s)b(s)−ϕ > (s)B > (s)R −1 (s)B(s)ϕ(s) ds. (2.2.37) 18 (3) [52, Corollary 6.2.10] If the matricesG andQ(t)−S(t)R −1 (t)S > (t), t ∈ [0,T], are symmetric and non-negative definite, then equation (2.2.34) has a unique solution and the matrixP(t) is symmetric and non-negative definite for all t∈ [0,T]. Equation(2.2.34)isknownastheRiccatidifferentialequation,aftertheItalianmathemati- cian Jacopo Francesco Riccati (1676–1754), who studied it in some detail, gave solutions for severalparticularcases,andpublishedtheresultsin1724,ofcourse,withoutanyreferenceto optimalcontrol. Examplesofclosed-formsolvableRiccati(andmanyother)equationscanbe foundinthebookbyPolyaninandZaitsev[37]. The solution of the linear regulator problem was obtained in the late 1950s and published in 1960 by Rudolf Emil K´ alm´ an (b. 1930), a Hungarian-American scientist. The maximum principle, dynamic programming principle, and the linear regulator problem are considered by many to be the foundations of the modern theory of optimal control. The corresponding originalreferencescouldbethebooksbyPontryaginetal. [38]andBellman[7],andthepaper byR.Kalman[28]. Below, we look at several examples where equation (2.2.34) has an explicit solution. We startwiththefollowingauxiliaryresult. 2.2.7Lemma. The solution of the equation y 0 (t) =y 2 (t)−2ay(t)−b, 0<t<T, y(T) =c6=a± √ a 2 +b, is y(t) =ρ + + 2ϕ βe 2ϕ(T−t)−1 (2.2.38) where ϕ = √ a 2 +b, ρ + =a+ϕ, β = c−a+ϕ c−a−ϕ . 19 Proof. Wehave dy (y−ρ + )(y−ρ − ) =dt, whereρ ± =a±ϕ. Bypartialfractions, 1 (y−ρ + )(y−ρ − ) = 1 2ϕ 1 y−ρ + − 1 y−ρ − , Afterintegration,takingintoaccounttheterminalconditiony(T) =c, ln (y−ρ + )(c−ρ − ) (y−ρ − )(c−ρ + ) = 2ϕ(t−T). Since y(t) = ρ ± are both solutions, it follows by uniqueness of solution that each of the inequalities for the terminal condition, c > ρ + , ρ − < c < ρ + , c < ρ − , implies the corre- sponding inequality fory(t) for allt<T, which means that the expression under the natural logarithmisalwayspositive. Then y−ρ − y−ρ + =βe 2ϕ(T−t) , and(2.2.38)follows. The main examples when equation (2.2.34) can be solved in closed form are such that m =n = 1andallthecoefficientsdonotdependontime. Consider ˙ x(t) =ax(t)+u(t), Z T 0 x 2 (t)+u 2 (t) dt+x 2 (T)→ min, (2.2.39) 20 where we assumedb(t) = 0, S(t) = 0, B(t) = Q(t) = R(t) = G = 1. Equation (2.2.34) becomes ˙ P(t) =P 2 (t)−2aP(t)−1, P(T) = 1. (2.2.40) Denotebyp 1 ,p 2 therootsoftheequationp 2 −2ap−1 = 0: p 1 =a+r, p 2 =a−r, r = √ a 2 +1. (2.2.41) ByLemma2.2.7wefindϕ =r,ρ + =p 1 ,ρ − =p 2 ,β = (1−p 2 )/(1−p 1 ),and P(t) =p 1 + 2r βe 2r(T−t) −1 . (2.2.42) Theoptimaltrajectorysatisfies ˙ x ∗ (t) = (a−P(t))x ∗ (t). (2.2.43) Accordingto(2.2.37),thevaluefunctionis V(0,x) =P(0)x 2 ; (2.2.44) forlargeT,wehaveP(0)≈p 1 andV(0,x)≈p 1 x 2 . Infact,fortheinfinitehorizonproblem, whentheobjectiveistominimize Z ∞ 0 x 2 (t)+u 2 (t) dt, (2.2.45) we haveV(0,x) = p 1 x 2 and ˙ x ∗ (t) = −rx ∗ (t). This, in particular shows that the control is stabilizing: −r < 0 even ifa > 0. While infinite horizon problems form a special class and, 21 in general, must be studied separately, some times passing to the limit T → ∞ makes the formulaseasiertoanalyze. Asanotherexample,considertheone-dimensionalproblem ˙ x(t) =ax(t)+u(t), R Z T 0 u 2 (t)dt+x 2 (T)→ min. (2.2.46) Inthiscase,Q(t) =S(t) = 0and(2.2.34)becomes ˙ P(t) = P 2 (t) R −2aP(t), P(T) = 1. (2.2.47) Settingq R (t) =P(t)/R,weget ˙ q R (t) = 2aq R (t)−q 2 R (t), q R (T) = 1/R, or,afterintegration, q R (t) = 2a 1−(1−2aR)e −2a(T−t) . Thecorrespondingoptimalfeed-backcontrolis ¯ u R (t,x) =−q R (t)x,theoptimaltrajectoryis ˙ x ∗ R (t) =− a 1+(1−2aR)e −2a(T−t) 1−(1−2aR)e −2a(T−t) x ∗ (t), andthevaluefunctionis V R (0,x) = 2aR 1−(1−2aR)e −2aT . InthelimitR→ 0,problem(2.2.46)becomesminimizing|x(T)|,orequivalently,drivingthe systemtozero. Notethaty(t) = lim R→0 x ∗ R (t),ifexists,mustsatisfy ˙ y(t) =− a(1+e −2a(T−t) ) 1−e −2a(T−t) y(t), y(0) =x(0). (2.2.48) 22 Forthefunctionv(t) =y 2 (t)wefind v(t) =x 2 (0)− Z t 0 2a(1+e −2a(T−s) ) 1−e −2a(T−s) v(s)ds. (2.2.49) Whent≈T, 2a(1+e −2a(T−t) ) 1−e −2a(T−t) ≈ 1 T −t , and so lim t→T −v(t) = 0 for every initial condition x(0) (otherwise, the divergence of the integral R T 0 dt/(T −t) would forcev(t) < 0 fort close toT. For the precise statement and proof,seeLemma3.3.4onpage65. Theresultlim t→T −|y(t)| = 0isconsistentwithlim R→0 V R (0,x) = 0forallx∈R. Thus, ¯ u(t,x) =− 2ax 1−e −2a(T−t) (2.2.50) is,inacertainsense,theoptimalfeedbackcontroldrivingthesystem ˙ x(t) =ax(t)+u(t,x(t)) from an arbitrary initial condition at time 0 to zero at timeT. Note that any feedback control that drives the system to zero at time T must violate conditions for uniqueness of solution by developing a singularity at timeT; otherwise, a two-point boundary value problem for a first-order equation would have no solution. On the other hand, using the above arguments, in particular, equality (2.2.49), we conclude that any control function ¯ u of the form ¯ u(t,x) = −K(t)x, whereK(t) > 0 and lim t→T − R t 0 K(s)ds = +∞, will drive the solution of ˙ x(t) = ax(t)+u(t,x(t))tozeroattimeT. Inthenextchapter,westudythesequestionsindetailfor stochasticordinarydifferentialequations. 23 2.3 Thestochasticcase The stochastic calculus of variations, commonly known as the Malliavin calculus, is outside the scope of the current work. In what follows, we discuss the stochastic optimal control problem as the extension of the deterministic problem (2.2.10), (2.2.13) to the stochastic Itˆ o equations. Let (Ω,F,{F t } t≥0 ,P)beastochasticbasiswiththeusualcondition(completenessofF 0 and right-continuity of the family F t , t ≥ 0). We consider the optimal control problem for thestochasticordinarydifferentialequation dX u (t) =b(t,X u (t),u(t))dt+σ(t,X u (t),u(t))dW(t), t 0 <t<t 1 , X u (t 0 ) =x 0 , (2.3.1) withtheobjectivetominimizethecostfunction J(t 0 ,x 0 ;u(·)) =E Z t 1 t 0 F(s,X u (s),u(s))ds+Eh(x(t 1 )) (2.3.2) overthesetU ofmeasurableF t -adaptedprocesseswithvaluesinametricspace U. In(2.3.1), b is anR d -valued non-random function, σ is anR d×d 1 - valued non-random function, and W is ad 1 -dimensional standard Brownian motion. Since the cost function depends only on the expectedvaluesoftheprocessX andcontrolu,aweak,intheprobabilisticsense,solutionof (2.3.1) is enough. Still, we assume that the functionsb,σ,F andh are sufficiently smooth in allthevariables,tohaveallthepartialderivativeswemightneed. As in the deterministic case, we distinguish between a feed-back (or Markov) control in the form u(t) = v(t,X u (t)), where v = v(t,x) is a deterministic function, and open-loop control that does not explicitly depend onX(t). Note that an open loop control must still be F t -adaptedtobeimplementable. 24 Denotebyu ∗ =u ∗ (t)theoptimalcontrol,thatis, J(t 0 ,x 0 ;u ∗ (·)) = min u(·)∈U J(t 0 ,x 0 ;u(·)), and denote by x ∗ = x ∗ (t) the corresponding solution of (2.3.1) (that is, x ∗ (t) = X u ∗ (t)). Similar to the deterministic case, it is possible to prove existence of u ∗ under very general assumptions about the functions b,σ, and F, but the proof does not produce any practical algorithmforcomputingu ∗ . Similartothedeterministiccase,theoptimalcontrolcanbeobtainedeitherfromthemax- imum principle or from the dynamic programming principle. To state both principles, we definethefollowingthreefunctions: H(t,x,u,p,A) = Tr A > σ(t,x,u) +p > b(t,x,u)−F(t,x,u), (2.3.3) H(t,x,u,p,B) = 1 2 Tr σ > (t,x,u)Bσ(t,x,u) +p > b(t,x,u)−F(t,x,u), (2.3.4) V(t,x) = min u(·)∈U J(t,x;u(·)). (2.3.5) wheret∈ [0,T],x∈R n ,u∈U,A∈R d×d 1 ,B ∈R d×d , B > =B,> mean transposition of a matrix or a vector, and Tr means the trace of the matrix (the sum of the diagonal elements). Note that, ifσ = 0, bothH andH reduce to the deterministic Hamiltonian (2.2.15) on page 12;V isthevaluefunctionoftheproblem. Tosimplifythenotations,wewillusethefollowingconventions: • The star convention: if G = G(t,a(t),b(t),...), then G ∗ (t) = G(t,a ∗ (t),b ∗ (t),...). 25 • The summation convention: a ik b k = X k≥1 a ik b k (thatis,summationovertherepeatedindices). 2.3.1 Themaximumanddynamicprogrammingprinciples 2.3.1Theorem. [StochasticMaximumPrinciple[52,Theorem3.3.3]] Assume that u ∗ is the optimal control for the problem (2.3.1), (2.3.2) and let x ∗ be the corresponding solution of (2.3.1). Then there exists a vector function p ∗ = p ∗ (t) ∈ R n , a matrix-valued function q ∗ = q ∗ (t) ∈ R d×d 1 and symmetric matrix-valued functions P = P(t)∈R d×d andQ (m) =Q (m) (t)∈R d×d ,m = 1,...,d 1 , such that dx ∗ i (t) =H ∗ p i (t)+H ∗ q ij (t)dW j (t), x ∗ i (t 0 ) =x 0i , (2.3.6) dp ∗ i (t) =−H ∗ x i (t)dt+q ∗ ij (t)dW j (t), p ∗ i (t 1 ) =−h x i (x ∗ (T)), (2.3.7) dP ij (t) =− ∂b ∗ k (t) ∂x i P kj (t)+P ik (t) ∂b ∗ k (t) ∂x j + ∂σ ∗ km (t) ∂x i P k` (t) ∂σ ∗ `m (t) ∂x j + ∂σ ∗ km (t) ∂x i Q (m) kj (t)+Q (m) ik (t) ∂σ ∗ km (t) ∂x j + ¯ H ∗ x i x j (t) dt +Q (m) ij (t)dW m (t), P ij (t 1 ) =−h x i x j (x ∗ (t 1 )), (2.3.8) ¯ H(t,x ∗ (t),u ∗ (t)) = max u∈U ¯ H(t,x ∗ (t),u), (2.3.9) 26 where ¯ H(t,x,u) =H(t,x,u,p ∗ (t),P(t))+Tr σ > (t,x,u)(q ∗ (t)−P(t)σ ∗ (t)) (2.3.10) =H(t,x,u,p ∗ (t),q ∗ (t))− 1 2 Tr (σ ∗ ) > (t)P(t)σ ∗ (t) + 1 2 Tr σ(t,x,u)−σ ∗ (t) > P(t) σ(t,x,u)−σ ∗ (t) . (2.3.11) WithouttheBrownianmotion,(2.3.6)and(2.3.7)becomethecorrespondingdeterministic equations from (2.2.18) on page 13. In the stochastic setting, (2.3.7) is a backward equa- tion and requires the extra unknownsq to ensure unique solvability; for the same reason, the functionsQ (m) appear in (2.3.8). The functionsP,Q (m) , known as the second-order adjoint variables,donothavedeterministiccounterparts. There are two general situations when there is no need to solve equations (2.3.8) for the functionsP andQ (m) : • If the diffusion σ does not depend on the control variable u. In this case, σ(t,x ∗ (t),u ∗ (t)) =σ(t,x ∗ (t))sothat ¯ H(t,x ∗ (t),u) =H(t,x ∗ (t),u,p ∗ (t),q ∗ (t))− 1 2 Tr σ > (t,x ∗ (t))P(t)σ(t,x ∗ (t)) and(2.3.9)becomes H(t,x ∗ (t),u ∗ (t),p ∗ (t),q ∗ (t)) = max u∈U H(t,x ∗ (t),u,p ∗ (t),q ∗ (t)). (2.3.12) 27 • IfU is a convex subset of a Euclidean space and all the functions in (2.3.1), (2.3.2) are continuouslydifferentiableinu. Inthiscase,by(2.3.11), ¯ H(t,x ∗ (t),u ∗ (t))− ¯ H(t,x ∗ (t),u) =H(t,x ∗ (t),u,p ∗ (t),q ∗ (t))−H ∗ (t)+o(|u ∗ (t)−u|) = (u ∗ (t)−u) > H ∗ u (t)+o(|u ∗ (t)−u|) and(2.3.9)becomes(u ∗ (t)−u) > H ∗ u (t)≥ 0. Next, recall the value function V defined by (2.3.5). Denote by V x the column vector (V x 1 ,...,V x d ) > andbyV xx ,thematrix(V x i x j , i,j = 1,...,d),assumingthederivativesexist. 2.3.2Theorem. (a) [Stochastic dynamic programming principle [52, Theorem 4.3.3]] IfX u is the trajec- tory of (2.2.10) withX u (t) =y andt 0 ≤t≤τ ≤t 1 , then V(t,y) = inf u(·)∈U E Z τ t F(s,X u (s),u(s))ds+V(τ,X u (τ)) . (2.3.13) (b)[Verificationprinciple[52,Theorem5.4.1]] Assumethat thevaluefunctionV istwice continuously differentiable in(t,x). Then V t (t,x ∗ (t)) =H(t,x ∗ (t),u ∗ (t),−V x (t,x ∗ (t)),−V xx (t,x ∗ (t))) = max u∈U H(t,x ∗ (t),u,−V x (t,x ∗ (t)),−V xx (t,x ∗ (t))), V x (t,x ∗ (t)) =−p ∗ (t), V xx (t,x ∗ (t))σ ∗ (t) =−q ∗ (t), (2.3.14) 28 and the optimal controlu ∗ is of feed-back (or Markov) type: u ∗ (t) = ¯ u(t,x ∗ (t)), where ¯ u(t,x) = argmin u∈U F(t,x,u)+(L u V)(t,x) , (2.3.15) where L u = 1 2 (σσ > ) ij (t,x,u) ∂ 2 ∂x i ∂x j +b i (t,x,u) ∂ ∂x i is the generator of the diffusion processX u from(2.3.1). Aconsequenceof(2.3.13)istheHJBequation[52,Proposition4.3.5]: −V t +sup u∈U H(t,x,u,−V x ,−V xx ) = 0, t 0 <t<t 1 , V(t 1 ,x) =h(x), (2.3.16) orequivalently, V t (t,x)+ inf u∈U (L u V)(t,x)+F(t,x,u) = 0, t 0 <t<t 1 , V(t 1 ,x) =h(x). (2.3.17) Onceasolutionof(2.3.16)isfound,theoptimalcontroliscomputedby(2.3.15). Unlikethedeterministiccase,inwhichthevaluefunctionisasolutionofafullynonlinear first-orderpartialdifferentialequationandisoftennotsmooth,thevaluefunction V =V(t,x) caneasilybetwicecontinuouslydifferentiablein(t,x)becauseequation(2.3.16)isparabolic. Forexample,ifσ doesnotdependonuandthematrixσσ > isuniformlypositivedefinite,then (2.3.16)becomesauniformlyparabolicsemi-linearequation V t + 1 2 (σσ > ) ij V x i V x j +G(t,x,V x ) = 0 withasuitablefunctionG;ifthefunctionsb,σ,haresufficientlysmooth,soisthesolutionV. 29 Comparing the two possible ways of finding the optimal control, via the maximum prin- cipleorthedynamicprogrammingprinciple,wenoticethatthemaximumprincipleapproach doesnotrequiresolvingpartialdifferentialequations,althoughthenumberofthecorrespond- ingordinarydifferentialequationscanbelarge,andsolvingthenonlinearbackwardstochastic equation (2.3.7) requires special numerical methods. The HJB equation (2.3.16) brings ana- lytical and computational difficulties to the next level (from ordinary differential equations to equationswithpartialderivatives),butitisasingledeterministicequation. Absenceofcontrol inthediffusionsimplifiestheanalysisofboth(2.3.6)–(2.3.9)and(2.3.16). 2.3.2 Thelinearquadraticcontrolproblem Similar to the deterministic case, a complete solution of the optimal control problem (2.3.1), (2.3.2)existsinthelinear-quadraticcase: dX(t) = [A(t)X(t)+B(t)u(t)+b(t)]dt +[C(t)X(t)+D(t)u(t)+σ(t)]dW(t), J(u(·)) =E Z T 0 (X > QX)(t)+2(u > SX)(t)+(u > Ru)(t) dt +E(X > GX)(T), (2.3.18) whereW is a one-dimensional standard Brownian motion (finitely many independent Brow- nian motions can also be considered: [52, Theorem 6.7.10]),G is a non-random matrix, and A,B,C,D,b,σ,Q,S,R are bounded measurable deterministic functions (matrix- or vector- valued). Tosimplifythenotations,weomitthedependenceoftheprocessX onthecontrolu. We also assume that the matricesG, Q(t), t ≥ 0, andR(t), t ≥ 0, are symmetric, and the initialconditionX(0)isnon-random. 30 For this problem, the stochastic maximum principle after some computations leads to the systemofequations[52,Proposition6.5.5] dx ∗ (t) = [Ax ∗ (t)+Bu ∗ (t)+b(t)]dt+[Cx ∗ (t)+Du ∗ (t)+σ(t)]dW(t), dp ∗ (t) =−[A > p ∗ (t)+C > q ∗ (t)−Qx ∗ (t)−S > u ∗ (t)]dt+q ∗ (t)dW(t), x ∗ (0) =X(0), p ∗ (T) =−Gx ∗ (T), Ru ∗ (t)+Sx ∗ (t)−B > p ∗ (t)−D > q ∗ (t) = 0; (2.3.19) notethatweeliminatedthematricesP andQ. IfthematrixR(t),t≥ 0,isuniformlypositive definitethen[52,Corollary6.5.7] u ∗ (t) =−R −1 (t)[S(t)x ∗ (t)−B > (t)p ∗ (t)−D(t)q ∗ (t)]. Welookforp ∗ intheformp ∗ (t) =−P(t)x ∗ (t)−ϕ(t)tofindthat ˙ P +PA+A > P +C > PC +Q −(S +B > P +D > PC) > (R+D > PD) −1 (S +B > P +D > PC) = 0, P(T) =G, (2.3.20) ˙ ϕ−[A−B(R+D > PD) −1 (S +B > P +D > PC)] > ϕ +[C−D(R+D > PD) −1 (S +B > P +D > PC)] > Pσ +Pb = 0, ϕ(T) = 0. (2.3.21) Thentheoptimalcontrolu ∗ (t)isoffeed-backtype: u ∗ (t) =u(t,x ∗ (t)),where u(t,x) =−(R+D > PD) −1 (S +B > P +D > PC)x −(R+D > PD) −1 (B > ϕ+D > Pσ). (2.3.22) 31 Notethatifb(t) = 0,σ(t) = 0,thenϕ(t) = 0andu(t,x) = Ψ(t)xforsomematrixΨ(t). Thesameresultfollowsfromthedynamicprogrammingprincipleifwelookforthesolu- tionoftheHJBequationintheform V(t,x) =x > P(t)x+x > ϕ(t)+μ(t); the matrixP(t) will then satisfy (2.3.20) and the vectorϕ(t), (2.3.21). Equation (2.3.20) has auniquesolutionifthematrix-valuedfunctions R =R(t)andD =D(t)arecontinuous. IfC = 0,D = 0,σ = 0,then(2.3.20)becomes(2.2.34)onpage18,and(2.3.21)becomes (2.2.36) — the corresponding equations in the deterministic case. By analogy with (2.2.34), equation (2.3.20) is known as the stochastic Riccati equation, even though the equation itself isdeterministic. 2.3.3Theorem. Assume that 1. the matrix R(t), t ≥ 0 is uniformly positive definite and the matrices G and Q(t)− S(t)R −1 (t)S > (t), t≥ 0 are non-negative definite; 2. the functionsR =R(t) andD =D(t) are continuous. Then • [52, Proposition 6.7.1 and Theorem 6.7.2] There exists a unique solutionP =P(t) of equation (2.3.20), and, for everyt≥ 0, the matrixP(t) is symmetric and non-negative definite. • [52, Corollary 6.5.7] The optimal control is unique with probability one and is given by(2.3.22). 32 • [52,Theorem6.6.1] The value function of the problem is V(t,x) =x > P(t)x+2x > ϕ(t) +E Z T t 2ϕ > (s)b(s)+σ > (s)P(s)σ(s)−ψ > (s) ¯ P(s)ψ(s) ds, where ¯ P(t) =R(t)+D > (t)P(t)D(t), ψ(t) = ¯ P −1 (t) B > (t)ϕ(t)+D > (t)P(t)σ(t) . Letuslookatseveralexampleswhenequation(2.3.20)canbesolvedinclosedform. Asin thedeterministiccase,weassumethatboththestateX andthecontroluareone-dimensional, and the coefficientsA,B,C,D,S,Q,R do not depend on time. IfD 6= 0, then, with no loss ofgenerality,wecanassumeD = 1andre-writeequation(2.3.20)as ˙ P(t) =−(2A+C 2 )P(t)−Q+ S +(B +C)P(t) 2 P(t)+R , P(T) =G. (2.3.23) After a change of variables y(t) = P(T −t) +R, we findP(t) = y(T −t)−R, ˙ P(t) = −˙ y(T −t),and(2.3.23)becomes ˙ y(t) = ay 2 (t)+by(t)+c y(t) , y(0) =G+R, (2.3.24) withsuitablenumbersa,b,c: a = (2A+C 2 )−(B +C) 2 , c =−(R(B +C)−S) 2 , b =Q−R (2A+C 2 )−2(B +C) −2S(B +C). The basic question, non-explosion of the solution of (2.3.24), is answered completely in [52, 33 Theorem6.7.9]. Still,thesolutionofequation(2.3.24)usuallyprovidesanimplicitdescription ofy as a function oft, and a closed-form expression for y (and hence, forP) is not available. For example, choose the numbersA =−1/2,B = 1,C =−1,S = 1,Q = 1,G =R = 1/2 sothata = 0,b = 1,c = 1. Then,withy(0) =G+R = 1,wefind y˙ y = (y +1), y−ln(1+y) =t+(1−ln2). While it is clear thaty(t)> 0 for allt≥ 0 (in fact, ˙ y > 0 fory > 0 and soy is an increasing function of t, it is also clear that there is no closed-form expression of y as an elementary functionoft. Tofindthegeneralsolutionof(2.3.24),notethat Z ydy ay 2 +by +c = 1 2a ln|ay 2 +by +c|− b 2a Z dy ay 2 +by +c , a6= 0, (2.3.25) and Z ydy by +c = by +c−cln|by +c| b 2 , b6= 0. Also,settingr =b 2 −4ac,wefind,fora6= 0, Z dy ay 2 +by +c = 1 √ r ln 2ay +b− √ r 2ay +b+ √ r , r> 0 Z dy ay 2 +by +c = 1 a ln|2ay +b|+ b 2a 2 y +ab , r = 0, Z dy ay 2 +by +c = 2 p |r| arctan 2ay +b p |r| , r< 0. It follows from (2.3.25) that an explicit closed-form solution exists if b = 0. Indeed, if a6= 0,then(2.3.24)implies ln|ay 2 +c| = 2at+ln|a(R+G) 2 +c|; 34 ifa = 0,then y 2 = 2ct+(R+G) 2 . Sincec≥ 0,weseethat,unlessy =R+G(whichhappensifa =b =c = 0),thefunctiony eventuallybecomeshitszero,meaningthatequation(2.3.24)hasnoglobalsolution. IfD = 0, then equation (2.3.20) is essentially the same as equation (2.2.34) in the deter- ministic case. For example, if D = C = S = 0 and b(t) = 0, that is, the state equation is dX = (AX +Bu)dt+σ(t)dW(t), thentheoptimalcontrolisthesameasinthedeterministiccase: ¯ u(t,x) =− B R P(t)x, ˙ P +2AP +Q− B 2 P 2 R = 0. (2.3.26) Similarly,non-zero C,withb = 0,σ = 0,D = 0: dX = (AX +Bu)dt+CXdW(t), hastheeffectofchangingthegaincoefficientAinthecorrespondingdeterministicproblem: ¯ u(t,x) =− B R P(t)x, ˙ P +(2A+C 2 )P +Q− B 2 P 2 R = 0. (2.3.27) Forthestateequation dX =AXdt+(Du+σ(t))dW(t) (B =C = 0), 35 we find that the optimal control is not feed-back, but open-loop, that is, does not explicitly dependonthestatex: ¯ u(t,x) =− DP(t) R+D 2 P(t) σ(t), where ˙ P +2AP +Q = 0. (2.3.28) In particular, if the control is free of charge (R = 0), then the optimal strategy is to eliminate thestochastictermbytakingu ∗ (t) =−σ(t)/D. Finally,letusconsiderastochasticanalogof(2.2.46)onpage22: dX(t) = (AX(t)+u(t))dt+dW(t), R Z T 0 u 2 (t)dt+X 2 (T)→ min. (2.3.29) IfA6= 0,then ˙ P(t) = P 2 (t) R −2AP(t), P(T) = 1. (2.3.30) Settingq R (t) =P(t)/R,weget ˙ q R (t) = 2Aq R (t)−q 2 R (t), q R (T) = 1/R, or,afterintegration, q R (t) = 2A 1−(1−2AR)e −2A(T−t) . Thecorrespondingoptimalfeed-backcontrolis ¯ u R (t,x) =−q R (t)xandtheoptimaltrajectory is dX ∗ R (t) =− A 1+(1−2AR)e −2A(T−t) 1−(1−2AR)e −2A(T−t) X ∗ (t)+dW(t). 36 In the limitR → 0, problem (2.2.46) becomes minimizing|X(T)|, or equivalently, driving thesystemtozero. NotethatY(t) = lim R→0 X ∗ R (t),ifexists,mustsatisfy dY(t) =− A(1+e −2A(T−t) ) 1−e −2A(T−t) Y(t)dt+dW(t). (2.3.31) ForthefunctionZ(t) =EY 2 (t)wefind Z(t) =X 2 (0)− Z t 0 2A(1+e −2A(T−s) ) 1−e −2A(T−s) v(s)ds+ t 2 . (2.3.32) Whent≈T, 2A(1+e −2A(T−t) ) 1−e −2A(T−t) ≈ 1 T −t , and so lim t→T −Z(t) = 0 for every initial conditionX(0) (otherwise, the divergence of the integral R T 0 dt/(T −t) would forceZ(t) < 0 fort close toT. For the precise statement and proof, see Lemma 3.3.4 on page 65. Thus, lim t→T −Y(t) = 0 in the mean square. Further analysis shows that the convergence also holds with probability one; see Theorem 3.3.8 on page71. IfA = 0,thenq R (t) = 1/(T −t+R)andY(t) = lim R→0 X ∗ R (t),ifexists,mustsatisfy dY(t) =− Y(t) T −t dt+dW(t), (2.3.33) which is the equation of the Brownian bridge withY(0) = X(0) andY(T) = 0. Note that (2.3.33) also follows after passing to the limit A → 0 in (2.3.31). By analogy, it is there- fore natural to call (2.3.31) the Ornstein-Uhlenbeck bridge. Using the hyperbolic cotangent function coth(x) = e x +e −x e x −e −x , 37 equation(2.3.31)canbewrittenas dY(t) =−Acoth A(T −t) Y(t)dt+dW(t), (2.3.34) which, in particular, shows thatAcoth A(T −t) < 0 for allA6= 0 andt < T, that is, the driftisstabilizing. 38 Chapter3 Ordinarydifferentialequations: Exact control 3.1 Introduction Exactopen-loopcontrolmeansfindinganinputsothatthesolutionataprescribedtimetakes a prescribed value. As long as the underlying system is controllable, finding such an input relativelyeasy. Forexample,aconstantinputcanoftendothejob. Fordeterministicequations, such a control is admissible from every point of view, and a more interesting question can be posed: to find an exact open loop control minimizing some performance index, for example, the total energy of the control. This question is discussed below for the scalar linear equation (seepage42.) Astatementoftheexactcontrolproblemasanoptimalcontrolproblemisnotsatisfactory, as minimization of a quantity does not ensure that zero will be achieved. In fact, we will see in Section 3.2.3 that for some stochastic linear-quadratic control problems with the cost functionalJ(u) =|X u (T)| 2 , the corresponding optimal control makes no attempt to achieve X ∗ (T) = 0. Exactfeed-backcontroleffectivelymeanssolvingatwo-pointboundaryvalueproblemfor the first-order equation, which is impossible without violating the conditions for uniqueness of solution of the corresponding equation. Not surprisingly, neither the maximum principle nor the HJB equation can produce such a control. For linear equations, one possibility is to 39 take a limit in a suitable linear-quadratic problem. Recall that we already have one example in the deterministic setting (page 22) and an example in the stochastic setting (page 37). The idea was to construct an optimal control for a particular linear-quadratic regulator and then pass to the limit and eliminate any running cost of control. In Section 3.2.2, we do the same forastochasticequationwithmultiplicativenoise,andlaterdiscussanotherpossibleapproach using the idea of a pinned diffusion. Finally, we introduce the notion of the driving function and show how to construct an exact feed-back control for a large class of stochastic ordi- narydifferentialequations. Unlikethelogarithmictransformation,theconstructionextendsto equationswithdegeneratediffusion,includingdeterministicequations. Everywhere in this chapter, we fix (Ω,F,{F t } t≥0 ,P), a stochastic basis satisfying the usual conditions (completeness ofF 0 and the right continuity of the filtrationF,{F t } t≥0 ). BothB =B(t)andW =W(t),t≥ 0,willdenotethestandardBrownianmotion(eitherone- ormulti-dimensional)onthisstochasticbasis. 3.2 Motivationalexamples 3.2.1 Open-loopcontrol Westartwithaone-dimensionaldeterministiclinearequation ˙ x(t) =ax(t)+u(t). (3.2.1) Given initial condition x(0) ∈ R and terminal condition g ∈ R, there are many open-loop controlsu = u(t) ensuring thatx(T) = g. For example, we can takeu(t) = u 0 = const. If a6= 0,then x(t) =x(0)e at + u 0 a e at −1 , 40 anditfollowsthat,tohavex(T) =g,weshouldtake u 0 = a(g−x(0)e aT ) e aT −1 . Inthelimita→ 0,weget u 0 = g−x 0 T , (3.2.2) whichcertainlyensuresthatthesolutionof(3.2.1)witha = 0takesvalueg attimeT: x(t) =x 0 + g−x 0 T t. Next, we ask for an open-loop control that ensures x(T) = g and has the smallest possible norminL 2 ((0,T)). Inotherwords,weintroducethecostfunctional J(u) = Z T 0 u 2 (t)dt. (3.2.3) Ifa6= 0,thesolutionof(3.2.1)is x(t) =x(0)e at + Z t 0 u(s)e a(t−s) ds. Theconditionx(T) =g implies Z T 0 e a(T−s) u(s)ds =g−x(0)e aT , or Z T 0 e −as u(s)ds =ge −aT −x(0). (3.2.4) BytheCauchy-Schwartzinequality Z T 0 e −as u(s)ds 2 ≤ Z T 0 e −2as ds Z T 0 u 2 (s)ds 41 or Z T 0 u 2 (s)ds≥ 2a e 2aT −1 ge −aT −x(0) 2 , and the equality holds if and only ifu(t) = Ce −at for some real numberC. In other words, thecontrolfunctionu(t) =Ce −at minimizesthecost(3.2.3). By(3.2.4), C(1−e −2aT ) = 2a(ge −aT −x(0)) or C = 2a(ge −aT −x(0)) 1−e −2aT =a g−x(0)e aT sinh(aT) . Theoptimalexactcontrolistherefore u(t) =a g−x(0)e aT sinh(aT) e −at . (3.2.5) Inthelimita→ 0,weagaingettheconstantfunction(3.2.2). Next,consideraone-dimensionalstochasticequation dY(t) =aY(t)dt+f(t)dB t +u(t)dt, (3.2.6) wheref =f(t) is a given adapted process, and the controlu =u(t) is a continuous function (not necessarily adapted). We want to findu so thatY(T) = g, whereT > 0 is fixed andg is a real number. Again, assume thatu does not explicitly depend onY, that is, the control is open-loop. Tofindu,notethatthesolutionof(3.2.6)is Y(t) =Y(0)e at + Z t 0 e a(t−s) f(s)dB(s)+ Z t 0 e a(t−s) u(s)ds. (3.2.7) 42 TheconditionY(T) =g resultsintheequation g =Y(0)e aT + Z T 0 e a(T−s) f(s)dB(s)+ Z T 0 e a(T−s) u(s)ds. (3.2.8) Therearecertainlymanyfunctionsusatisfyingthisequation. Forexample,takingu(t) =u 0 , aconstantindependentoft,andassuminga6= 0,wefind g−Y(0)e aT − Z T 0 e a(T−s) dB(s) = u 0 (e aT −1) a (3.2.9) or u 0 = a g−Y(0)e aT − R T 0 e a(T−s) f(s)dB(s) e aT −1 . (3.2.10) Ifa = 0,wefind u 0 = g−Y(0)− R T 0 f(s)dB(s) T , (3.2.11) eitherdirectlyfrom(3.2.8)orbypassingtothelimita→ 0in(3.2.10). Inparticular,assumethata = 0,Y(0) =g = 0,andf(t) = 1forallt. Then Y(t) = Z t 0 u(s)ds+B(t). (3.2.12) If u(t) =u 0 =− B(T) T (3.2.13) then(3.2.12)becomes Y(t) =− B(T) T t+B(t). (3.2.14) 43 Clearly, thecontrolfunctionu isnotF t -adapted, because B(T)ismeasurablewithrespectto F T only. Notealsothat(3.2.12)implies E Z T 0 u(t)dt 2 =EB 2 (T) =T, sothat,bytheCauchy-Schwartzinequality, E Z T 0 u 2 (t)dt≥T. Thus,theconstantcontrol(3.2.11)minimizesthecostJ(u) =E R T 0 u 2 (s)ds. Ontheotherhand,itiswellknownthattheBrownianbridge y(t) = (T −t) Z t 0 dB(s) T −s (3.2.15) or dy(t) =− y(t) T −t dt+dB(t). (3.2.16) satisfiesy(0) = y(T) = 0. In other words, a feed-back control u(t) = −Y(t)/(T −t) also ensuresthatY(0) =Y(T) = 0forthesolutionoftheequationdy =dB(t)+u(t)dt. Therearetwomaindifferencesbetween(3.2.14)andtheBrownianbridge(3.2.16): • In (3.2.16), the control u is F t -adapted; in fact, it is even feed-back (also known as Markov in the stochastic setting), that is, depends only on the value of the controlled processattimet:u(t) = ¯ u(t,Y(t))foranon-randomfunction ¯ u. • In(3.2.16),thecontroluisnotsquare-integrable: E Z T 0 u 2 (t)dt = Z T 0 ds (T −s) 2 = +∞. (3.2.17) 44 Wewillseelaterthatthisexampleillustratesthegeneraleffect: everyfeed-backcontrolmov- ingthesolutionof(3.2.6)toY(T) =g isnotsquareintegrable. Thetimecontinuityofurulesoutacontrolu(t) =−f(t) ˙ B(t)whichremovestherandom perturbationandmakestheequationdeterministic: thiscontrolisclearlynon-implementable. It is also clear that any control that requires explicit knowledge of the random perturbationB will be hard to implement, even if the knowledge is non-anticipating: the realization of B(t) is usually not available directly. Accordingly, when it comes to exact control of stochastic equations,wewillbemostlylookingforMarkovcontrols. 3.2.2 Feed-backcontrolinthedrift Bythegeneraluniquenesstheoremforordinarydifferentialequations,exactfeed-backcontrol must develop singularity near the terminal time. Recall that we already have one example in the deterministic setting (page 22) and an example in the stochastic setting (page 37). The idea was to construct an optimal control for a particular linear-quadratic regulator and then pass to the limit and eliminate any running cost of control. Below, we will do the same for a stochasticequationwithmultiplicativenoise. Considertheequation dX u (t) =AX u (T)dt+CX u (t)dW(t)+u(t)dt, A, C∈R. (3.2.18) The objective is, given the initial conditionX(0) and the terminal timeT, to find a feed-back controlusothatX u (T) = 0. IfweconsiderthisasastandardoptimalcontrolproblemwithU =Randcost J 0 (u) =|X u (T)| 2 , 45 then the solution does not exist. Indeed, by (2.3.17) on page 29, the corresponding HJB equationis V t (t,x)+ inf u∈R (ax+u)V x (t,x)+ 1 2 x 2 V xx (t,x) = 0, and the infinum, for everyx, is equal to−∞. Accordingly, we consider a sequence of linear- quadraticcontrolproblemswiththesamestateequation(3.2.18)andthecostfunctionals J R (u) =R Z T 0 u 2 (t)dt+|X(T)| 2 , R> 0. By(2.3.22)onpage31,theoptimalfeed-backcontrolis u R (t,x) =−q R (t)x, where(withq R (t) =P(t)/R) ˙ q R (t)+(2A+C 2 )q R (t)−q 2 R (t) = 0, q R (T) = 1/R. Aswesawonpage22,ifa = 2A+C 2 6= 0,then q R (t) = 2a 1−(1−2aR)e −2a(T−t) ; if2A+C 2 = 0,thenq R (t) = 1/(T −t+R). InthelimitR→ 0,wegetthecontrol u(t,x) = − 2(2A+C 2 )x 1−e −2(2A+C 2 )(T−t) , 2A+C 2 6= 0, − x T −t , 2A+C 2 = 0, 46 andthecorrespondingtrajectory dY(t) = AY(t)+u(t,Y(t)) dt+CY(t)dW(t). Notethat R T 0 |u(t,x)|dt = +∞foreveryx6= 0. 3.2.3 Feed-backcontrolinthediffusion Considertheproblemofoptimalcontrolfortheone-dimensionalequation dX u (t) = [AX u (t)+Bu(t,x)+b(t)]dt+[CX u (t)+Du(t,x)+σ(t)]dW(t) (3.2.19) withthecostfunctional J(u) =E|X u (T)−β| 2 , β∈R. (3.2.20) The functionsA,B,C,D can depend on time. The control set isU = R and the admissible controlsU aretheadaptedR-valuedprocesses. Thenthevaluefunction, V(x,t) = inf u(·)∈U E |X u (T)−β| 2 X u (t) =x , satisfiestheHJBequation −V t +sup u∈R −(Ax+Bu+b(t))V x − 1 2 (Cx+Du+σ(t)) 2 V xx = 0, (3.2.21) withtheterminalconditionV(T,x) =|x−β| 2 .IfD6= 0,then −(Ax+Bu+b(t))V x − 1 2 (Cx+Du+σ(t)) 2 V xx 47 isquadraticinuandthesupremumisachievedwhen u = −BV x −(Cx+σ)DV xx D 2 V xx . (3.2.22) Welookforthesolutionof(3.2.21)intheform V(t,x) = 1 2 P(t)x 2 +ϕ(t)x+f(t) forsuitablefunctionsP,ϕ,f. ThenV x =P(t)x+q(t),V xx =P(t),and(3.2.22)becomes u =− Bx D 2 − Bϕ(t) D 2 P(t) − Cx+σ D . Substituting the expressions for u and V into (3.2.21) and equating the coefficients of the correspondingpowersofx,weconcludethat ˙ P(t)+[A−(B +CD) 2 D −2 ]P(t)+ 1 2 CP(t) 2 = 0, P(T) = 2, (3.2.23) ˙ ϕ(t)+[A−(B +CD)BD −2 ]ϕ(t)+bP +CσP −D −1 σP = 0, ϕ(T) =−2β. Note that if β = 0, then the same equations forP and ϕ follow from the general result for linear-quadraticcontrollerwith Q =R =S = 0;seepage31. Asaresult,theoptimalcontrol problem for equation (3.2.19) with the cost functional (3.2.20) is solvable whenD 6= 0, and theoptimalcontrolu ∗ (t) =u(t,x)isoffeed-backtypeandisgivenby u(t,x) =− B D 2 x− Bϕ(t) D 2 P(t) − Cx+σ(t) D . Thereisnoguarantee,though,thatthissolutionwillprovidetheexactcontrol,thatis,J(u ∗ ) = 0. In fact, we should expect J(u ∗ ) = V(0,x) to be non-zero, for, example, because, by 48 uniqueness of solution of (3.2.23),P(0)6= 0. Thus, the corresponding optimal trajectoryX ∗ willNOTsatisfyX ∗ (T) =β. Moreover,ifB = 0,thatis,onlythediffusioniscontrolled,we findthat dX ∗ (t) = (AX ∗ (t)+b(t))dt. In other words, the optimal control in this case simply eliminates the stochastic component of the equation and makes no attempt to ensureX ∗ (T) = β. As a result, in our subsequent analysisoftheexactcontrolforstochasticequations,wewillnotcontrolthediffusion. 3.3 Equationswithsingulardrift 3.3.1 Thelogarithmictransformation Let X = X(t) be a diffusion process in R d , starting from X(0) = x 0 and satisfying the stochasticdifferentialequation: dX(t) =b(t,X(t))dt+σ(t,X(t))dW(t). (3.3.1) AssumethatthereexistsaC > 0sothat,forallt∈ [0,T]andx,y∈R d |b(t,x)−b(t,y)|+ d X i=1 d 1 X j=1 |σ ij (t,x)−σ ij (t,y)|≤C|x−y|. Thenequation(3.3.1)hasauniquestrongsolution. ThegeneratorA :f 7→Af ofthediffusionX(t)is Af(x) = d X i=1 b i ∂f ∂x i + d X i,j=1 a i,j ∂ 2 f ∂x i ∂x j , (3.3.2) 49 where2a =σσ > andσ > isthetransposeofσ. FixT > 0 and smooth functiong = g(y), y ∈ R d . We assume that there exist positive numbersc,C sothat0<c<g(y)<C forally. Define P(t,T,x,g) =E X(t) =x|g(X(T)) , 0<t<T. (3.3.3) Asafunctionof(t,x),P satisfiesthebackwardKolmogorovequation ∂P ∂t =AP, P(T,T,x,g) =g(x). (3.3.4) Regularity of solutions of parabolic regularity implies that P is a smooth function of (t,x). Moreover,(3.3.3)impliesthat 0<c<P(t,T,x,g)<C forallt,x. (3.3.5) Next,define V(t,x;g) =−lnP(t,T,x,g) (3.3.6) ThenP(t,T,x,g) =e −V(t,x;g) anddirectcomputationshowthat ∂P ∂t =− ∂V ∂t e −V , AP =−e −V AV −(∇V) > a(∇V) ; ∇V is the gradient (column) vector ofV and (∇V) > is its transpose. As a result,V satisfies thenonlinearparabolicpartialdifferentialequation ∂V ∂t =AV −(∇V) > a(∇V) (3.3.7) 50 withtheterminalcondition V(T,x;g) =−lng(x) (3.3.8) We will now interpret equation (3.3.7) as the HJB equation for a certain optimal control problem. We make an additional assumption that the matrix a invertible and the inverse a −1 is a continuous function of (t,x). Considerthecontrolleddiffusionequation dY(t) = (b(t,Y(t))+u(t))dt+σ(t,Y(t))dW(t) (3.3.9) withthecostfunctional J(t,x;u) =E 1 4 Z T t u > (s)a −1 (s,Y(s))u(s)ds−lng(Y(T)) Y(t) =x (3.3.10) tobeminimized. Bythegeneraltheory, • thevaluefunctionV(t,x) = inf u J(t,x;u)satisfies(3.3.7),sothat(3.3.7)isindeedthe HJBequationforthecontrolproblem(3.3.9),(3.3.10). • theoptimalcontrol ¯ uisafeed-back(orMarkov)controlgivenby ¯ u(t,Y(t)) = argmax u −u > (∇V(t,x))− 1 4 u > a −1 (t,x)u x=Y(t) (3.3.11) thatis, ¯ u(t,Y(t)) =−2a(t,x)(∇V(t,x)) x=Y(t) . (3.3.12) 51 Since we already know that V(t,x) = −lnP(t,T,x,g), we conclude that the optima controlis ¯ u(t,Y(t)) = 2a(t,Y(t))(∇lnP(t,T,Y(t),g)), (3.3.13) andthecorrespondingoptimaltrajectory ¯ Y satisfies d ¯ Y(s) = (b(s, ¯ Y(s))+2a(s, ¯ Y(s))(∇lnP(s,T, ¯ Y(s),g)))dt +σ(s, ¯ Y(s))dW(s), t<s<T; ¯ Y(t) =x. (3.3.14) Assumptions on a, regularity of P, and (3.3.5) imply that the function ¯ u(t,x) = 2a(t,x)(∇lnP(t,T,x,g)) is bounded for all (t,x) and is Lipschitz continuous inx. There- fore,equation(3.3.13)hasauniquestrongsolution. Note that, by the Girsanov theorem, the measure ¯ μ, on the space of continuousR d -valued functions, generated by the process ¯ Y is absolutely continuous with respect to the similar measureμ generated by the processX. To write the expression for the density, introduce the notation U(s,x) =∇lnP(s,T,x,g) (3.3.15) Thenthedensityof ¯ μwithrespecttoμsatisfies d¯ μ dμ (X) = exp Z T t U(s,X(s))dX(s)− Z T t (b > U +U > aU)(s,X(s))ds . (3.3.16) Inotherwords,ifwedenotebyQ(X)theright-handsideof(3.3.15)anddefineanewproba- bilitymeasure ˜ P ontheoriginalprobabilityspaceby ˜ P(A) =E(I(A)Q(X)), (3.3.17) thenthedistributionof ¯ Y underP isthesameasthedistributionofX under ˜ P. 52 Transitionfromequation(3.3.1)toequation(3.3.14)iscalledalogarithmictransformation. Weseethattherearetwowaystointerpretthistransformation: 1. asasolutionoftheoptimalcontrolproblem(3.3.9),(3.3.10); 2. asachangeoftheprobabilitymeasureaccordingto(3.3.17). Next, we investigate the logarithmic transformation when the functiong is non-negative, but notnecessarilysmoothoruniformlyseparatedawayfromzero. Tobegin,letusreplacethefunctiong withtheindicatorfunctionχ M ofasetM ∈R d : χ M (x) = 1, x∈M 0, x / ∈M. (3.3.18) ThenP(t,T,x,g) becomesP(t,T,x,M), the probability that the processX(t) starting from pointx at timet ends up in the setM ⊂R d at timeT. As function oft,x,P(t,T,x,M) still satisfiesbackwardKolmogorov’sequation ∂P ∂t +AP = 0, P(T,T,x,M) =χ M (x). (3.3.19) Undertheassumptiononb,σ,standardresultsonregularityofsolutionsofparabolicequations imply thatP(t,T,x,M) is smooth and positive as a function of (t,x) for allt<T; it is also clearthat,beingaprobability,thefunctionP satisfiesP(t,T,x,M)≤ 1. Thenwecandefine V(t,x) =−lnP(t,T,x,M) (3.3.20) andagainverifythatV satisfiesthenonlinearparabolicpartialdifferentialequation ∂V ∂t =AV −(∇V) > a(∇V) (3.3.21) 53 withtheterminalcondition V(T,x) = lnχ M (x) = 0, x∈M, +∞, x / ∈M. (3.3.22) As before, we consider the problem of optimal control of equation (3.3.9) with the objec- tivetominimizethecostfunctional J(t,x;u) =E 1 4 Z T t u > (s)a −1 (s,x)u(s)dt−lnχ M (x) Y(t) =x . (3.3.23) Noticethattheterminalcost(3.3.22)ensuresthattheprocesshitsthesetM attimeT. Similar to(3.3.13),theoptimalcontrolisfeed-back: ¯ u(t,Y(t)) = 2a(t,Y(t))(∇lnP(t,T,Y(t),M)). (3.3.24) The function ¯ u(t,x) will be uniformly bounded in (t,x) and Lipschitz continuous in x for those sets M for which the probability P(t,T,x,M) is uniformly positive. For those sets, equation (3.3.14) describing the optimal trajectory ¯ Y will have a unique strong solution. For othersetsM,wecanconstructaweaksolutionusingachangeofmeasure(3.3.17). Thus,the distribution of the diffusion process ¯ Y is the same as the distribution of the diffusion process X conditionedonhittingthesetM attimeT. LetusnowassumethattheprobabilitymeasureP(t,T,x,·)hasadensitysothat P(t,T,x,M) = Z R d p(t,T,x,y)χ M (y)dy. (3.3.25) 54 ByshrinkingthesetM toapointy 0 ,weget P(t,T,x,M) =p(t,T,x,y 0 ). (3.3.26) If the functionp is strictly positive and continuously differentiable inx, we can consider the equation d ¯ Y(s) = (b(s, ¯ Y(s))+2a(s, ¯ Y(s))(∇lnp(s,T, ¯ Y(s),y 0 )))dt +σ(s, ¯ Y(s))dW(s), t<s<T; ¯ Y(t) =x. (3.3.27) and argue that the distribution of ¯ Y is the same as the distribution of the processX pinned at pointy 0 attimeT. ThedetailsoftheargumentareinthepaperbyJamison[27]. Forexample,bytakingb = 0,andσ ad×didentitymatrix,wefindthatX =W, p(t,T,x,y 0 ) = 1 p 2π(T −t) exp − (x−y 0 ) 2 2(T −t) (3.3.28) andthen ¯ Y becomesthed-dimensionalBrownianbridgefromthepoint xattimettothepoint y 0 attimeT: d ¯ Y(s) =− ¯ Y(s)−y 0 T −s ds+dW(s), t<s≤T, Y(t) =x (3.3.29) or ¯ Y(s) =x T −s T −t +y 0 1− T −s T −t +(T −s) Z s t dW(r) T −r . (3.3.30) Moregenerally,ifb6= 0,thenwehavetheOrnstein-Uhlenbeckprocess dX(t) =bX(t)dt+dW(t), 55 whosedistributionattimetisGaussianwithvariance σ(t) =E Z t 0 e b(t−s) dW(s) 2 = 1 2b e 2bt −1 . (3.3.31) Therefore, p(t,T,x,y 0 ) = 1 p 2πσ(T −t) exp − (x−y 0 ) 2 2σ(T −t) , (3.3.32) whichmeansthatthecorrespondingprocess ¯ Y istheOrnstein-Uhlenbeckbridgefirstencoun- teredonpage37: d ¯ Y(s) = b ¯ Y(t)− 2b( ¯ Y(t)−y 0 ) e 2b(T−t) −1 dt+dW(t); (3.3.33) ify 0 = 0andb =A,thisisexactly(2.3.31). It is clear that the function p(s,T,x,y) has a singularity when s = T: the measure P(T,T,x,·) is a point mass at x, that is, p(T,T,x,y) is the delta function δ(x−y). The drift in the corresponding equation (3.3.27) will have a singularity as well. For example, in thecaseoftheBrownianbridge(3.3.29),wehave E ¯ Y(s)−y 0 T −s 2 = |x+y 0 | T −t 2 + Z s t dr (T −r) 2 =C(x,y 0 ,t,T)+ 1 T −s . In the time homogeneous case (when b and σ do not depend on time and p(s,T,x,y) = ˜ p(T−s,x,y)),Sheu[41,TheoremB]showedthat,underminimalregularityconditiononthe functionsb,σ, the drift in (3.3.27) will have a singularity at least of the order (T −s) −1/2 as s%T,andhencenotsquareintegrableintimeontheinterval(t,T). Asaresult,therunning cost E Z T t ¯ u > (s, ¯ Y(s))a −1 (s ¯ Y(s))¯ u > (s, ¯ Y(s))ds Y(t) =x , 56 on the optimal trajectory is infinite. Note also that the terminal cost in (3.3.23) is not defined ifχ M is replaced by a delta-function. Thus, the optimal control interpretation of (3.3.27) is not straightforward. Using certain limiting procedures, Fleming and Sheu [22, Theorem 3.1] showthat(3.3.27)islocallyoptimal: itistheoptimalcontrolof(3.3.9)oneverytimeinterval (t,T −ε), ε > 0, with a slightly modified cost functional (3.3.23). Thus, even though the optimal running cost on (t,T) is infinite, it is, in a sense, smaller than for any other Markov controlthatensuresthatthesolutionof(3.3.9)hitsthepointy 0 attimeT. The asymptotic behavior of the density functionp = p(t,T,x,y) ast % T is a delicate question. Here is the most general and explicit result, proved by Bismut [10, Theorem 3.8] under the assumptions that the functions b and σ in (3.3.1) are bounded, infinitely differen- tiable, with all the derivatives bounded, and the matrixa = (1/2)σσ > is invertible. Consider the Riemannian manifold obtained fromR d by replacing the usual Euclidean distance with the metric generated by the matrixa. In other words, ifU,V are two vectors at pointx, then the inner product ofU andV isU > a(t,x)V. Let~ n(x,y) be the initialvelocity of the shortest geodesic (shortest path relative to this new metric) starting at the point x and reaching the pointy inunittime. Then lim s→T − (T −s)∇lnp(s,T,x,y 0 ) =~ n(x,y). (3.3.34) Notethatinthecaseofthed-dimensionalBrownianmotion, (T −s)∇lnp(s,T,x,y 0 ) = (y−x), which is consistent with (3.3.20), because the metric stays Euclidean so that the geodesic goingfromx toy inunittime isx+(y−x)tandthushas velocityy−x. Ingeneralthereis noeasyexpressionfor~ n(x,y)intermsofx,y,b,andσ. 57 3.3.2 Thegeneralcase In the previous section, we saw that if the diffusion process has a transition density, then the logarithmic transformation (3.3.27) ensures that the process hits a prescribed pointy 0 at a prescribed time T. The logarithmic transformation does a lot more than simply move the processfrompointxtopointy 0 : 1. The resulting control is feed-back (or Markov) and therefore adapted and does not requiretheknowledgeofthewholetrajectoryoftheBrownianmotionuptotimeT. 2. TheresultingcontrolislocallyoptimalinthesenseofhavingthesmallestL 2 normover every time interval (0,T −ε) among all feed-back controls that bring the process to pointy 0 attimeT. 3. The distribution of the resulting optimal trajectory (in the space of continuous func- tions) is the same as the distribution of the pinned diffusion (uncontrolled diffusion conditionedonhittingpointy 0 attimeT). AgeneraldiscussionofthelogarithmictransformationisinfirsteditionofthebookbyFlem- ingandSoner[23,ChapterVI]. The main difficulty in using this transformation in practice is that the explicit form of the transition density is usually not available, and thus the corresponding control cannot be implemented. In this section, we present easier ways to find a Markov control that moves a process from pointx to pointy 0 in a fixed time interval (Theorem 3.3.8 on page 71). We will also show that every such control must have a singularity at the terminal time and, in particular,notevenabsolutelyintegrableasafunctionoftime(Theorem3.3.2onpage61). Considerthestochasticordinarydifferentialequation dX(t) =b(t,X(t))dt+σ(t,X(t))dW(t), t> 0, 58 withinitialconditionX(0) =X 0 . Equivalently,intheintegralform, X(t) =X 0 + Z t 0 b(s,X(s))ds+ Z t 0 σ(s,X(s))dW(s). (3.3.35) In (3.3.35)X(t),X 0 andb(t,x) are vectors inR d ,σ(t,x) is a matrix inR d×d 1 , andW is an R d 1 -valued standard Brownian motion. We assume that the vector function b and the matrix functionσ arenon-random. 3.3.1Theorem. Assume that 1. The initial conditionX(0) is independent ofW; 2. For eacht∈ [0,T] and alli,j, the functionsb i (t,x) andσ ij (t,x) are continuous inx; 3. Z T 0 |b(t,x)|dt<∞ for all x∈R d ; (3.3.36) 4. For everyR> 0 there exists a measurable int functionK =K(t,R) such that Z T 0 K(t,R)dt<∞ for allR> 0, and for all|x|<R, |y|<R we have 2 d X i=1 (x i −y i )(b i (t,x)−b i (t,y))+ d X i=1 d 1 X j=1 |σ ij (t,x)−σ ij (s,x)| 2 ≤K(t,R)|x−y| 2 ; (3.3.37) 5. For allx∈R d and almost allt∈ [0,T] we have 2 d X i=1 x i b i (t,x)+ d X i=1 d 1 X j=1 |σ ij (t,x)| 2 ≤K(t,1)(1+|x| 2 ). (3.3.38) 59 Then equation(3.3.35) has a unique strong solution. Proof. This is a particular case of the result proved by Krylov and Rozovskii [32, Theorem 3.1]. Roughly speaking, condition (3.3.37) ensures uniqueness of the solution, and (3.3.38) ensuresglobalexistence(non-explosion). Conditions (3.3.37) and (3.3.38) are satisfied if the functionsb,σ are uniformly Lipschitz continuous: thereexistsaC > 0sothat,forallt∈ [0,T]andx,y∈R d |b(t,x)−b(t,y)|+ d X i=1 d 1 X j=1 |σ ij (t,x)−σ ij (t,y)|≤C|x−y|. Thisalsomeasthatthefunctionsb,σ areoflineargrowthinx: |b(t,x)|≤C|x|+|b(t,0)|, |σ ij (t,x)|≤C|x|+|σ ij (t,0)|. A typical example of a scalar equation satisfying the conditions of the theorem without lineargrowthassumptionis dX(t) = (X(t)−X 3 (t))dt+X(t)dW(t) or,moregenerally, dX(t) = −aX 2n+1 (t)+p 2n (X(t)) dt+(bX(t)+c)dW(t), 60 where a > 0, b,c ∈ R, p 2n = p 2n (x) is a polynomial of degree 2n, n ≥ 0. This example shows that the functions b,σ in (3.3.35) need not be bounded or of linear growth. A more exoticexample,comingfrom[32],is dX(t) =−a|X(t)| p−1 sgn(X(t))dt+b|X(t)| p/2 dW(t), wherep∈ (1,2),a,barerealnumberssatisfying −2(p−1)a+ p 2 b 2 4 ≤ 0, and sgn(x) = 1, if x> 0, 0, if x = 0, −1, if x< 0; this example shows that the functionsb,σ in (3.3.35) need not be even locally Lipschitz con- tinuous. Letusnowconsideracontrolleddiffusionprocess dY(t) = (b(t,Y(t))+u(t,Y(t)))dt+σ(t,Y(t))dW(t). Ifthisequationhasauniquestrongsolutionon [0,T],thenY(T)isarandomvariable,andin general,itisimpossibletoprescribeaspecificvaluetoY(T). Asaresult,toachieveY(T) =β forafixedvectorβ wemustviolatesomeconditionsofTheorem3.3.1. Thisstatementcanbe mademorepreciseundersomeadditionalconditionsonthediffusioncoefficientσ. 3.3.2Theorem. AssumethattheconditionsofTheorem3.3.1holdwithb+uinsteadofb,and also 61 1. eachσ ij is continuous inx uniformly int: lim |x−y|→0 sup 0≤t≤T |σ ij (t,x)−σ ij (t,y)| = 0; 2. For eachx, the matrixσσ > is positive definite uniformly int: inf 0≤t≤T inf |y|=1 X i,j,k σ ik (t,x)σ j,k (t,x)y i y j > 0. Then the solutionX of (3.3.35) satisfies P(X(t) =β) = 0 (3.3.39) for every0<t≤T andβ∈R d . Proof. ItisknownfromaresultbyStroockandVaradhan[42,Corollary10.1.4]thatunderthe conditions of the theorem the distribution of the random variableX(t),t > 0, has a density withrespecttotheLebesguemeasureonR d ,andthen(3.3.39)follows. Itisnaturaltoexpectthattheoriginal(uncontrolled)equationhasauniquestrongsolution, so that the functionsb,σ satisfy the conditions of Theorem 3.3.1. Similarly, we would like to keepuniquenessandnon-explosionforthecontrolledequation,whichimpliesthat Z t 0 |u(s,x)|dt<∞ foreveryx∈R d andt<T,andconditions(3.3.37),(3.3.38)holdwithb+uinsteadofbfor almost allt < T. Thus, the only natural way to violate the conditions of Theorem 3.3.1 is to have Z T 0 |u(t,x)|dt = +∞, (3.3.40) 62 at least for some x ∈ R d . This argument is also supported by the simplest example of an exactlycontrolleddiffusion—theBrownianbridgeinR d dY(t) = β−Y(t) T −t dt+dW(t). Thisequationcanbeeasilysolvedtogive Y(t) =Y(0)(T −t)+βt+(T −t) Z t 0 dW(s) T −s . Notethatsince E Z t 0 dW(t) T −s 2 =d Z t 0 ds (T −s) 2 = td T(T −t) , itfollowsthat lim t→T − (T −t) Z t 0 dW(s) T −s = 0 with probability one andY(T) = lim t→T −Y(t) = β for every initial conditionY(0), either deterministic or random independent ofW. Note that in this exampleX(t) = Y(0)+W(t) and u(t,x) = β−x T −t sothat(3.3.40)holdsforallx6=β. The example of the Brownian bridge also shows that condition (3.3.36) is not necessary forexistenceanduniquenessofsolution. Wewillnowgeneralizethisobservation. 3.3.3Theorem. Consider equation dY(t) = (b(t,Y(t))+u(t,Y(t)))dt+σ(t,Y(t))dW(t),0<t<T. (3.3.41) under the following assumptions: 63 1. Y(0) is independent ofW andE|Y(0)| 2 <∞; 2. The functionsb,σ satisfy the conditions of Theorem 3.3.1; 3. The functionu =u(t,x) has the following properties: • R t 0 |u(s,x)|ds<∞ for allx∈R d andt<T. • For every 0≤t<T there exists a positive numberC such that, for allx,y∈R d |u(t,x)−u(t,y)|≤C|x−y|; • For all0≤t<T andx∈R d , 2 d X i=1 x i u i (t,x)≤K(t,1)(1+|x| 2 ), (3.3.42) where the processK =K(t,R) is from Theorem 3.3.1. Then equation(3.3.41) has a unique strong solution for0≤t≤T and sup t∈(0,T) E|Y(t)| 2 <∞. (3.3.43) Proof. By Theorem 3.3.1 on page 59, equation has a unique strong solutionY n (t) on [0,T − (1/n)] for every positive integer n > 1/T; by uniqueness, Y n (t) = Y m (t) for n > m and t<T −(1/m). WethendefineY(t)bysettingY(t) =Y n (t)fort∈ [0,T −(1/n)]. BytheItˆ oformula,fort<T, |Y(t)| 2 =|Y(0)| 2 + d X i=1 Z t 0 2Y i (s)b i (s,Y(s))ds+ d X i=1 Z t 0 2Y i (s)u i (s,Y(s))ds + d X i=1 d 1 X j=1 Z t 0 σ 2 ij (s,Y(s))ds+ d X i=1 d 1 X j=1 Z t 0 2σ ij (s,Y(s))Y i (s)dW j (s). (3.3.44) 64 LetZ(t) =E|Y(t)| 2 . Usingassumptions(3.3.38)and(3.3.42)ofthetheorem,wefind Z(t)≤C 1 +2 Z t 0 K(s,1)Z(s)ds withC 1 = Z(0)+2 R T 0 K(t,1)dt, and then (3.3.43) follows from Gronwall’s inequality (for completeness,thisinequalityissatedandprovedbelow;seeRemark3.3.6onpage70.) Toseehowcondition(3.3.40)affectsthebehaviorofthesolutionatt =T,letusstartwith deterministicequations. 3.3.4 Lemma. Let C be a real number, F = F(t) a continuous function on [0,T], and let K =K(s), s≥ 0 be a measurable function with the properties 1. liminf t→T −K(t)> 0; 2. Z t 0 |K(s)|ds<∞ for allt<T; 3. lim t→T − Z t 0 K(s)ds = +∞. Ify =y(t), t∈ [0,T] is a non-negative continuous function with the property y(t)≤ Z t 0 (C−K(s))y(s)ds+F(s), (3.3.45) then lim t→T − y(t) = 0. Proof. Withnolossofgenerality,wecantakeC = 0;forC6= 0wereplacey(t)withy(t)e Ct (seeRemark3.3.5followingtheproof). Intuitively, the result is clear, because ify(T) > 0, then R T 0 K(s)y(s)ds = +∞ and then (3.3.45)becomesincompatiblewiththeassumptiony(t)≥ 0. 65 Tocarryoutarigorousargument,wewrite y(t) =− Z t 0 K(s)y(s)ds+F 1 (s) (3.3.46) forasuitablefunctionF 1 ;notethatF 1 (0) =y(0). First,assumethatthefunctionF 1 iscontinuouslydifferentiable: F 1 (t) =y(0)+ Z t 0 f 1 (s)ds foraboundedcontinuousfunctionf 1 . Then y 0 (t) =−K(t)y(t)+f 1 (t) sothat y(t) = exp − Z t 0 K(s)ds y(0)+ Z t 0 f 1 (s)exp Z s 0 K(r)dr ds . (3.3.47) Tosimplifythenotation,let Φ(t) = exp − Z t 0 K(s)ds . Then y(t) =y(0)Φ(t)+Φ(t) Z t 0 f 1 (s) Φ(s) ds. (3.3.48) Assumption liminf t→T −K(t) > 0 implies existence of at 0 ∈ (0,T) with the property that K(t)≥ 0forallt∈ (t 0 ,T). Then,fort 0 <s<t<T, R t s K(r)dr> 0andso Φ(t) Φ(s) ≤ 1. 66 Using(3.3.47)andtheassumptiony(t)≥ 0,foreveryt,t 1 ∈ (t 0 ,T), 0≤y(t)≤ Φ(t) |y(0)|+ Z t 1 0 |f 1 (s)| Φ(s) ds + max t∈[0,T] |f 1 (t)|(t−t 1 ). (3.3.49) Assumptionlim t→T − R t 0 K(s)ds = +∞implies lim t→T − Φ(t) = 0. Passingtothelimitin(3.3.49)ast→T − ,wefind 0≤ limsup t→T − |y(t)|≤ max t∈[0,T] |f 1 (t)|(T −t 1 ). Sincet 1 can be arbitrarily close toT, we conclude that limsup t→T −y(t) = 0. Clearly, 0 ≤ liminf t→T −y(t) ≤ limsup t→T −y(t), and therefore limsup t→T −y(t) = liminf t→T −y(t) = lim t→T −y(t) = 0. Let us now assume that the function F 1 is just continuous and bounded. To prove the statementofthelemma,wewillapproximateF 1 withsmoothfunctions. We extend the functionF 1 toR by settingF 1 (t) = F 1 (0),t < 0;F 1 (t) = F 1 (T),t > T. Theresultwillbeauniformlycontinuousfunction,sothatforeverynwecanfindδ(n)sothat |F 1 (t)−F(s)|< 1/nforallt,s∈Rsatisfying|t−s|<δ(n). Letφ =φ(t)beanon-negative smooth function with compact support in [−1,1] and the property R ∞ −∞ φ(t)dt = 1; for the purposeofthisargument,itisenoughtotake φ(t) = (1+cos(πt))/2, |t|≤ 1 0, |t|> 1. Define φ n (t) = (2/δ(n))φ(2t/δ(n)). Then φ n is supported in [−δ(n)/2,δ(n)/2] and 67 R +∞ −∞ φ n (t)dt = 1. Define F 1,n (t) = R ∞ −∞ F 1 (s)φ n (t−s)ds. This function has the follow- ingproperties: 1. F n (t)iscontinuouslydifferentiableon[0,T]; 2. sup t∈[0,T] |F(t)−F n (t)|≤ 1/n. Indeed, F 0 1 (t) = Z ∞ −∞ F 1 (s)φ 0 n (t−s)ds, and |F(t)−F n (t)|≤ Z t+δ(n)/2 t−δ(n)/2 |F(t)−F(s)|φ n (t−s)ds≤ 1/n becauseofourchoiceofδ(n). Next, definey n according to (3.3.47), withF 1,n instead ofF 1 . SinceF 1,n is continuously differentiable,wehavelim t→T −y n (t) = 0foreveryn. Afterintegrationbypartsin(3.3.47), y(t) =F 1 (t)−Φ(t) Z t 0 F 1 (s)K(s) Φ(s) ds. (3.3.50) Then y(t)≤y n (t)+ 1 n 1+Φ(t) Z t 0 0 |K(s)| Φ(s) ds+Φ(t) Z t t 0 d ds 1 Φ(s) ; recallthatK(t)≥ 0fort>t 0 andΦ 0 (t) =−K(t)Φ(t). Asaresult, y(t)≤y n (t)+ 1 n 1+Φ(t) Z t 0 0 |K(s)| Φ(s) ds+1− Φ(t) Φ(t 0 ) . Passingtothelimitt→T − ,wefind limsup t→T − y(t)≤ 2 n . 68 Sincenisarbitraryandy(t)≥ 0,weconcludethat lim t→T − y(t) = 0. Lemma3.3.4isproved. 3.3.5Remark. If in (3.3.46) we replaceK(t) withK(t)−C then, according to (3.3.50), the solutionoftheequation y(t) = Z t 0 (C−K(s))y(s)ds+F 1 (s) is y(t) =F 1 (s)+e Ct Φ(t) Z t 0 C−K(s) Φ(s) e −Cs F 1 (s)ds Withz(t) =e −Ct y(t)andG(t) =e −Ct F 1 (t),weget z(t) =G(t)−Φ(t) Z t 0 G(s)K(s) Φ(s) ds+Φ(t) Z t 0 CG(s) Φ(s) ds Theproofofthelemmashowsthat lim t→T − G(t)−Φ(t) Z t 0 G(s)K(s) Φ(s) ds = 0 (comparewith(3.3.50)),and lim t→T − Φ(t) Z t 0 CG(s) Φ(s) ds = 0 69 (compare with (3.3.48)), so that lim t→T −z(t) = lim t→T −y(t) = 0. Since C is arbitrary, the condition liminf t→T −K(t) > 0 in the statement of the lemma can be replaced with liminf t→T −K(t)>−∞. 3.3.6 Remark. The proof of Lemma 3.3.4 also establishes the Gronwall inequality: ifC,C 1 are positive real numbers, L = L(t) is a positive function satisfying R T 0 L(s)ds ≤ C and y =y(t)isacontinuousnon-negativefunctionsatisfying y(t)≤ Z t 0 L(s)y(s)ds+C 1 , then sup 0<t<T y(t)≤C 1 (1+Ce C ). (3.3.51) Indeed,y(t) = R t 0 L(t)y(s)ds+F(t), whereF(t) is a continuous function satisfyingF(t)≤ C 1 . Set Ψ(t) = exp Z t 0 L(s)ds . By(3.3.50), y(t) =F(t)+Ψ(t) Z t 0 L(s)F(s) Ψ(s) ds≤C 1 +C 1 e C Z t 0 L(s)ds≤C 1 (1+Ce C ). Inequality(3.3.51),alsoknownasGronwall’slemma,wasfirstpublishedinamorespecial form in 1919 byThomas Hakon Gr¨ onwall (1877–1932), a Swedish-American mathematician [25]; he changed the spelling of his name to Gronwall after coming to the US. In 1943, R. E. Bellmangeneralizedtheresult[5]. 3.3.7 Definition. A measurable function K = K(s), 0 ≤ s < T is called a driving function if it has the following properties: 1. liminf t→T −K(t)> 0; 70 2. Z t 0 |K(s)|ds<∞ for allt<T; 3. lim t→T − Z t 0 K(s)ds = +∞. Atypicalexampleofadrivingfunctionis K(t) = 1 (T −t) p , p≥ 1. NotethatifK =K(t)isadrivingfunction,thensoisCK(t)foreveryC > 0. The previous computations (see, for example, Remark 3.3.5) show that a driving function hasthefollowingproperties: lim t→T − exp − Z t 0 K(s)ds = 0, (3.3.52) lim t→T − Z t 0 f(s)exp − Z t s K(τ)dτ ds = 0, (3.3.53) lim t→T − f(t)− Z t 0 K(s)f(s)exp − Z t s K(τ)dτ ds = 0 (3.3.54) foreveryfunctionf =f(t)thatisboundedandcontinuouson[0,T]. Wecannowpresentthemainresultofthissection. 3.3.8Theorem. Consider a controlled diffusion process dY(t) =b(t,Y(t))dt+u(t,Y(t))dt+σ(t,Y(t))dW(t), 0<t<T. (3.3.55) Letβ be an element ofR d . Assume that 1. The initial conditionY(0) is independent ofW andE|Y(0)| 2 <∞. 2. The functionsb andσ have the following properties: 71 • (3.3.37) holds; • There exists a numberC > 0 such that |b(t,x)| 2 + d X i=1 d 1 X j=1 |σ ij (t,x)| 2 ≤C(1+|x| 2 ) (3.3.56) for allt∈ [0,T]. 3. The control functionu has the following properties: • R t 0 |u(s,x)|ds<∞ for everyx∈R d and allt<T; • There exists a positive measurable functionc =c(t) such that Z t 0 c(s)ds<∞ for allt<T and, for allx,y∈R d |u(t,x)−u(t,y)|≤c(t)|x−y|; (3.3.57) • There exists a driving function K = K(t) (see Definition 3.3.7) such that, for everyx∈R d and all0<t<T, 2 d X i=1 (x i −β i )u i (t,x)≤−|x−β| 2 K(t), (3.3.58) Then equation (3.3.55) has a unique strong solution Y(t), 0 ≤ t < T such that sup 0<t<T E|Y(t)| 2 <∞ and lim t→T − Y(t) =β (3.3.59) in the mean square and with probability one. 72 Proof. WestartbyapplyingTheorem3.3.1onpage59. BytheCauchy-Schwartzinequality, d X i=1 x i b i (t,x) ≤|x||b(t,x)|≤|x| 2 +|b(t,x)| 2 , andtherefore(3.3.56)implies(3.3.38). Next, 2 d X i=1 x i u i (t,x) = 2 d X i=1 (x i −β i )u i (t,x)+2 d X i=1 β i u i (t,x) ≤−|x−β| 2 K(t)+2|β||u(t,x)|. Denotemin(K(t),0)byK − (t). Then 2 d X i=1 x i u i (t,x)≤K − (t)(|x| 2 +|β| 2 )+2|β||u(t,x)|. Byassumption, liminf t→T −K(t)> 0andthereforethereexitsat 0 <T suchthatK − (t) = 0 for t ∈ (t 0 ,T). Since by assumption R t 0 |K(t)|dt < ∞ for all t < T, we conclude that R T 0 |K − (t)|dt <∞. Next, takingy = 0 in the inequality|u(t,x)−u(t,y)|≤ C|x−y|, we get |u(t,x)|≤|u(t,0)|+c(t)|x|. Therefore, 2 d X i=1 x i u i (t,x)≤K − (t)|x| 2 +K − |β| 2 +2|β||u(t,0)|+2c(t)|β||x|. Since|x|< 1+|x| 2 ,wedefine K(t) =K − (t)(1+|β| 2 )+2|β||u(t,0)|+2c(t)|β|+1, 73 then R t 0 K(s)ds<∞forallt<T and 2 d X i=1 x i u i (t,x)≤K(t)(1+|x| 2 ). ByTheorem3.3.1onpage59,equationhasauniquestrongsolutionY n (t)on[0,T−(1/n)]for everypositiveintegern> 1/T;byuniqueness,Y n (t) =Y m (t)forn>mandt<T−(1/m). WethendefineY(t)bysettingY(t) =Y n (t)fort∈ [0,T −(1/n)]. Next,introducethenotations Y(t) =Y(t)−β, Z(t) =E|Y(t)| 2 . BytheItˆ oformula, |Y(t)| 2 =|Y(0)| 2 + d X i=1 Z t 0 2Y i (s)b i (s,Y(s))ds+ d X i=1 Z t 0 2Y i (s)u i (s,Y(s))ds + d X i=1 d 1 X j=1 Z t 0 σ 2 ij (s,Y(s))ds+ d X i=1 d 1 X j=1 Z t 0 2σ ij (s,Y(s))Y i (s)dW j (s) (3.3.60) andthen Z(t) =Z(0)+ d X i=1 Z t 0 2E Y i (s)b i (s,Y(s)) ds+ d X i=1 d 1 X j=1 Z t 0 E σ 2 ij (s,Y(s)) ds + d X i=1 Z t 0 2E Y i (s)u i (s,Y(s)) ds. Inparticular,Z(t)iscontinuouson(0,T). Bytheinequality2|ab|≤a 2 +b 2 ,a, b∈R, d X i=1 2E Y i (s)b i (s,Y(s)) ≤Z(s)+E|b(s,Y(s))| 2 , 74 andby(3.3.56), E|b(s,Y(s))| 2 + d X i=1 d 1 X j=1 E σ 2 ij (s,Y(s)) ≤C(1+E|Y(s)| 2 ). (3.3.61) Usingtheinequality a 2 ≤ 2(a−b) 2 +2b 2 , a, b∈R (3.3.62) whichfollowsfrom a 2 = ((a−b)+b) 2 = (a−b) 2 +2(a−b)b+b 2 ≤ (a−b) 2 +(a−b) 2 +b 2 +b 2 , weconcludefrom(3.3.61)that E|Y(s)| 2 ≤ 2Z(t)+2|β| 2 . (3.3.63) By(3.3.58), 2 d X i=1 E Y i (s)u i (s,Y(s)) ≤−Z(s)K(s). Asaresult, Z(t)≤Z(0)+ Z t 0 (2C−K(s))Z(s)ds+C 1 , whereC 1 = (C +2|β| 2 )T. ByLemma3.3.4onpage65, lim t→T − Z(t) = 0, (3.3.64) or lim t→T −Y(t) = β in the mean square. SinceZ is a continuous function, this also means thatsup 0<t<T Z(t)<∞,which,by(3.3.62)impliessup 0<t<T E|Y(t)| 2 <∞. To establish the almost-sure convergence, we need some additional arguments (recall that 75 convergence in the mean square implies convergence in probability, but does not imply con- vergencewithprobabilityone). Itfollowsfrom(3.3.60)that sup 0<t<T d X i=1 Z t 0 2Y i (s)u i (s,Y(s))ds <∞ with probability one. Therefore, Y(t) is continuous on [0,T] and so lim t→T −(Y(t)−β) = Y(T) exists with probability one. Because sup 0<t<T E|Y(t)| 2 < ∞, the family of random variables|Y(t)|,t∈ (0,T)isuniformlyintegrable. Therefore, 0≤E|Y(T)| = lim t→T − E|Y(t)|≤ lim t→T − p Z(t) = 0. Thus, E|Y(T)| = 0, that is, Y(T) = 0 with probability one. This completes the proof of Theorem3.3.8. Forexample,if X(t) = 1+ Z t 0 X(s)dW(s), t≥ 0, isaone-dimensionalgeometricBrownianmotionand dY(t) =− Y(t)+1 1−t dt+Y(t)dW(t), Y(0) = 1, thenY(1) =−1, that is, we drive the geometric Brownian motion from 1 to−1 in unit time. Noticethatthesameresultisachievedby dY(t) =− Y(t)+1 (1−t) γ dt+Y(t)dW(t), Y(0) = 1, foreveryγ≥ 1. 76 More generally, ifβ is a fixed vector inR d andσ = σ(x) is a Lipschitz continuous func- tion onR d , then, by Theorem 3.3.8 we have Y(T − ) = β for the solutions of the following equations,withanyinitialconditionandanycontinuousfunctionf =f(t): dY(t) = β−Y(t) T −t dt+f(t)dt+σ(Y(t))dW(t); dY(t) = β−Y(t) e T−t −1 dt+f(t)dt+σ(Y(t))dW(t); dY(t) = β−Y(t) (T −t) γ dt+f(t)dt+σ(Y(t))dW(t), γ≥ 1. After encountering the Lipschitz continuity several times, we will note that the term goes back to the German mathematician Rudolf Otto Sigismund Lipschitz (1832–1903). Form the very beginning, the importance of the Lipschitz condition|f(x)−f(y)|≤L|x−y| was that itensureduniquesolvabilityoftheordinarydifferentialequation ˙ x =f(x). 77 Chapter4 Pararabolicequations: Optimalcontrol 4.1 Introduction We start with a technical comment related to some notational complications in the subject of optimalcontrolforpartialdifferentialequations. Forhistoricalreasons,thecontrolfunctionis denotedbyu. Onepossibleexplanationcouldbetheinfluenceoftheearlyworkonthesubject comingfromtheformerSovietUnion,wheretheletteruwasusedtodenotecontrolforavery concrete reason: it is the first letter in the word upravlenije, which is the Russian word for control. On the other hand, the unknown function in a partial differential equation is often denoted by u as well, partly because many other neighboring letters in the Latin alphabet, x,y,t,s,areusedasindependentvariables,andthelettersv,wcandenotemanyotherobjects. In particular, in the previous chapters, we use lower-case letters for the functions solving deterministicequations,andupper-caselettersforfunctionssolvingstochasticequations. With V denoting the value function, and bothw andW denoting the Brownian motion, this rules outv andw asthenotationfortheunknownfunctioninapartialdifferentialequation. One solution is offered in the book by Evans [18], where the unknown function in a PDE isu and the control isa; this also emphasizes that the focus of the book is PDEs rather than control. An alternative, and more attractive, in our opinion, solution, comes from the modern lit- erature on control of infinite-dimensional systems. There, the starting object is not a partial differential equations, but an infinite-dimensional evolution equation. The solution of the 78 equation is an element of a Banach space, and the spatial variable does not appear, which makesitpossibletokeepthelettersxandy forthesolutionoftheequation;thisapproachalso makesiteasiertoseethenumerousanalogiesbetweenthefinite-andinfinite-dimensionalset- tings. In the examples, when a traditional partial differential equations must be written, the spatialvariableisdenotedbyξ. Below,wefollowthisnotationalscheme. In what follows, we study optimal control of parabolic equations. Beside the natural extension of the corresponding problems for ordinary differential equations, the need to con- trol stochastic parabolic equations also appears in the problem of optimal control of partially observed nonlinear diffusion processes [52, 56]. Similar to the finite-dimensional setting, the approach using the HJB equation seems more popular, especially in the stochastic setting, as the recent developments in the infinite-dimensional analysis provide the necessary solv- abilityandregularityresults fortheinfinite-dimensionalsemi-linearparabolicequations; see, for example, the books by Cerrai [13], Krylov et al. [31], Da Prato [16], and Da Prato and Zabczyk [17]. The maximum principle approach is also possible (see, for example, Oksendal [34] or Zhou [56]), as well as other, less general and more problem-dependent, approaches (BondarevandPolshkov[11]). Everywhere in this chapter, we fix (Ω,F,{F t } t≥0 ,P), a stochastic basis satisfying the usual conditions (completeness ofF 0 and the right continuity of the filtrationF,{F t } t≥0 ). Both W = W(t) and w = w(t), t ≥ 0, can denote the standard Brownian motion (either one- or multi-dimensional) on this stochastic basis, while W can also denote a cylindrical BrownianmotiononaHilbertspace(seeDefinition4.2.1onpage80below). 4.2 Parabolicregularity The objective of this section is to describe diagonalizable parabolic equations in a scale of Hilbert spaces. Such equations are reduced to an infinite decoupled system of first-order 79 ordinary differential equations. The motivation is the heat equationU t = U ξξ , t > 0, on the intervalξ ∈ (0,1) with zero boundary conditions: the solution of this equation is written as the Fourier seriesU(t,ξ) = P k≥1 U k (t)h k (ξ), whereh k (ξ) = p 1/2sin(πkξ) and ˙ U k (t) = −π 2 k 2 U k (t). In abstract description, there will be no spatial variableξ, andx rather thanU willdenotetheunknownfunctionwithvaluesinaHilbertspace. LetH be a real separable Hilbert space and Λ, a positive-definite self-adjoint operator on Hwiththefollowingproperties: • ThedomainofΛisdenseinH; • Theeigenfunctionsh k , k≥ 1,ofΛformacompleteorthonormalsysteminH. Denoteby` k , k≥ 1theeigenvaluesofΛ: Λh k =` k h k . For γ ∈ R, define the spaceH γ as the closure of the set of finite linear combinations f = P N k=1 f k h h ,f k ∈Rofh k withrespecttothenorm kfk γ = N X k=1 ` 2γ k f 2 k ! 1/2 . (4.2.1) Then every elementv ∈H γ is identified with a formal seriesv = P k≥1 v k h k ,v k ∈R, such that P k≥1 ` 2γ k v 2 k < ∞. We call the numbers v k , k ≥ 1, the generalized Fourier coefficientsofv. 4.2.1Definition. Assume that X k≥1 ` −2α k <∞ (4.2.2) 80 for some α > 0. A cylindrical Brownian motion W is the element of L 2 (Ω× [0,T];H −α ) with generalize Fourier coefficients equal to w k (t), where{w k (t), k ≥ 1} are independent standard Brownian motions. NotethatifW isacylindricalBrownianmotion,then EkW(t)k 2 −α =t X k≥1 ` −2α k <∞, that is, W(t) ∈ H −α for every t ≥ 0. Note also that condition (4.2.2) is necessary and sufficienttoensurethatthecollection{w k (t),k≥ 1}definesanelementofH −α . Clearly,H 0 =H, and, for v ∈H, v k = (v,h k ) H is the usual Fourier coefficient of v; (·,·) H is the inner product in the spaceH. It is clear that, if sup k ` k <∞, thenH γ =H for everyγ. Wethereforeassumethatlim k→∞ ` k = +∞,thatis,theoperatorΛisunbounded. By construction, eachH γ is a Hilbert space, with the inner product (·,·) γ and the norm k·k γ givenby (f,g) γ = X k≥1 ` 2γ k f k g k , kfk 2 γ = X k≥1 ` 2γ k f 2 k = (f,f) γ . (4.2.3) ThecollectionofspacesH ={H γ , γ∈R}iscalledaHilbert scale(see,forexample, Kreinetal. [30,SectionIV.1.10])andhasthefollowingproperties: 1. Ifγ 1 <γ 2 ,thenH γ 2 isadensesubsetofH γ 1 . 2. The operator Λ is linear and bounded fromH γ ontoH γ−1 for everyγ ∈ R (and thus playstheroleofthefirst-orderdifferentialoperator). Wenowdefineanabstractversionofadifferentialoperatoroforder2m. 4.2.2 Definition. A linear operator A is called an m-operator in a Hilbert scaleH = {H γ , γ∈R} if all of the following hold: 81 1. A is linear and bounded fromH γ+m ontoH γ−m for everyγ and somem; 2. The functions{h k , k≥ 1} are eigenfunction ofA(Ah k =λ k h k ); 3. all eigenvaluesλ k ofA are positive. ItfollowsthatanequivalentnorminH γ isgivenby kvk 2 γ = X k≥1 λ γ/m k |v k | 2 , (4.2.4) wherev k , k≥ 1arethegeneralizedFouriercoefficientsofv. In fact, from now on, we will use (4.2.4) as the definition of the norm in the space H γ . An example of an m-operator is A = Λ 2m . More generally, we can take A = P 2m (Λ), whereP 2m =P 2m (z)isapolynomialofdegree2mandP 2m (z)> 0forz > 0. Asamoreconcreteexample,considerthenegativeoftheLaplaceoperator −Δ =− d X i=1 ∂ 2 ∂x 2 i on a smooth bounded domainG inR d with zero boundary conditions. It is known that this operator has a complete orthonormal system of eigenfunctions inH = L 2 (G) and the eigen- valuesλ k satisfy lim k→∞ k −2/d λ k =c G forsomepositivenumberc G dependingonlyonthedomainG;see,forexample,Safarovand Vassiliev[40]. Thus,A =−Δisa1-operator( m = 1). Notealsothat X k≥1 k −2p/d <∞forp>d/2, 82 so that in this example the space-time white noise exists and is an element of H −α for every α>d/2. WewillnowstudyevolutionequationsintheHilbertscaleH,andstartwithadeterministic evolutionequation ˙ x(t)+Ax(t) =f(t), t> 0. (4.2.5) 4.2.3 Definition. The solution of (4.2.5) is an element of S γ H γ whose generalized Fourier x k (t) coefficients satisfy ˙ x k (t)+λ k x k (t) =f k (t). (4.2.6) 4.2.4 Theorem. If A is an m-operator in the Hilbert scale H, x(0) ∈ S γ∈R H γ and f ∈ L 2 ((0,T);H r ), then the solution of (4.2.5) exists and is unique, andx∈L ∞ ((t 0 ,T);H r+m ) for everyt 0 > 0. Proof. By(4.2.6),thegeneralizedFouriercoefficientsx k =x k (t)ofthesolutionsatisfy x k (t) =x k (0)e −λ k t + Z t 0 e −λ k (t−s) f k (s)ds, andso x 2 k (t)≤ 2x 2 k (0)e −λ k t +2 Z t 0 e −2λ k (t−s) ds Z t 0 f 2 k (s)ds. By assumption, P k≥1 λ γ k x 2 k <∞, for someγ ∈R. Therefore P k≥1 λ (r/m)+1 k e −λ k t <∞ for everyt≥t 0 > 0,becauseλ γ k e −λ k t ≤C forsomeC dependingonlyont 0 andγ. Since R t 0 e −2λ k (t−s) ds = (1−e −2λ k t )/(2λ k )and,byassumption, X k≥1 λ r k Z T 0 f 2 k (t)dt<∞, 83 wehave X k≥1 λ r+1 k x 2 k (t)<∞ andtheresultfollows. 4.2.5Remark. The analysis of the proof shows that ifx(0)∈H r andf ∈L 2 ((0,T);H r−m ), thenx∈L 2 ((0,T);H r ). Next,weconsiderthestochasticevolutionequationwithadditivenoise dX +AXdt =f(t)dt+ X k≥1 g k (t)dW k (t)h k , (4.2.7) wheref,g k are deterministic functions (f taking values in someH r and eachg k , inR) and W k areindependentone-dimensionalstandardBrownianmotions. 4.2.6 Definition. The solution of (4.2.7) is an element of S γ H γ whose generalized Fourier x k (t) coefficients satisfy dX k (t)+λ k X k (t)dt =f k (t)dt+g k (t)dW k (t). (4.2.8) 4.2.7Theorem. Assume thatA is an m-operator in the Hilbert scale H,X(0)∈ S γ∈R H γ , and f ∈L 2 ((0,T);H r ). If X k≥1 λ r/m k Z T 0 g 2 k (t)dt < ∞, then the solution of (4.2.7) exists and is unique, and EkX(t)k 2 r+m <∞ for everyt> 0. If sup k≥1,t∈[0,T] |g k (t)|<∞ and P k≥1 λ −r 1 /m k <∞ for somer 1 > 0, then the solution of (4.2.7)existsandisunique,andEkX(t)k 2 r 0 +m <∞foreveryt> 0,wherer 0 = min(−r 1 ,r). Proof. By(4.2.8),thegeneralizedFouriercoefficientofX satisfy X k (t) =X k (0)e −λ k t + Z t 0 e −λ k (t−s) f(s)ds+ Z t 0 e −λ k (t−s) g k (s)dW k (s), 84 andso EX 2 k (t)≤ 3X 2 k (0)e −λ k t +3 Z t 0 e −λ k (t−s) f k (s)ds 2 +3 Z t 0 e −2λ k (t−s) g 2 k (s)ds. ByTheorem4.2.4,wehave,forallt> 0, X k≥1 λ 1+(r/m) k X 2 k (0)e −λ k t + Z t 0 e −λ k (t−s) f k (s)ds 2 ! <∞. If X k≥1 λ r/m k Z T 0 g 2 k (t)dt<∞, thenEkX(t)k 2 r+m <∞ follows from the inequalitye λ k (t−s) ≤ 1,k≥ 1 andt≥s≥ 0. If sup k≥1,t∈[0,T] |g k (t)|<∞, thenEkX(t)k 2 r 0 +m <∞ follows from the inequality R t 0 e −2λ k (t−s) ds≤ 1/λ k ,k≥ 1, t≥ 0. Finally,weconsiderthestochasticevolutionequationwithmultiplicativenoise dX +AXdt =f(t)dt+B(t,X)dW(t), (4.2.9) wheref is a deterministic functions with values in someH γ ,B is a linear operator such that B(t,h k ) =σ k (t)h k ,k≥ 1,andW isaone-dimensionalstandardBrownianmotions. 4.2.8 Definition. The solution of (4.2.9) is an element of S γ H γ whose generalized Fourier x k (t) coefficients satisfy dX k (t)+λ k X k (t)dt =f k (t)dt+X k (t)σ k (t)dW(t). (4.2.10) Notethat,tohaveequation(4.2.9)diagonalizable,thenoisemustbeintimeonly,whereas 85 in equation (4.2.7), the noise must have spatial component viah k . For (4.2.9) to have a solu- tion in someH γ , the operatorA in the deterministic part must, in some sense, dominate the operatorB inthestochasticpart. Moreprecisely,wedefine μ k (t) = 1 2 Z t 0 σ 2 k (s)ds (4.2.11) and assume that there exists a number δ ∈ [0,1] and a number C ∈ R such that, for all k≥ 1, t≥s≥ 0, (1−δ)λ k (t−s)− μ k (t)−μ k (s) ≥C. (4.2.12) If(4.2.12)holdswithδ = 1,thensup k,t |μ k (t)|<∞. 4.2.9 Theorem. Assume that A is an m-operator in the Hilbert scale H, X(0) ∈ H r 0 and f ∈L 2 ((0,T);H r ). If (4.2.12) holds with δ = 0, then the solution of (4.2.9) exists and is unique, and EkX(t)k 2 α <∞ for allt> 0, whereα = min(r,r 0 ). If (4.2.12) holds with δ > 0, then then the solution of (4.2.9) exists and is unique, and EkX(t)k 2 r+m <∞ for allt> 0. Proof. Define Ψ k (t) = exp −λ k t− 1 2 Z t 0 σ 2 k (s)ds+ Z t 0 σ k (s)dW(s) , F k (t,s) =E Ψ k (t) Ψ k (s) 2 . Then X k (t) =X k (0)Ψ k (t)+Ψ k (t) Z t 0 f k (s) Ψ k (s) ds 86 and EkX(t)k 2 γ ≤ 2 X k≥1 λ γ/m k F k (t,0)X k (0)+ Z t 0 F k (t,s)ds Z t 0 f 2 k (s)ds . Bydirectcomputations, F k (t,s) = exp(−2(λ k (t−s)−(μ k (t)−μ k (s)))). If (4.2.12) holds withδ = 0, thenF k (t,s) is uniformly bounded for allk≥ 1 andt≥s≥ 0. If (4.2.12) holds withδ > 0, then there exists a numberC 0 > 0 such that, for allk ≥ 1 and t≥s≥ 0,F k (t,s)≤C 0 e −δλ k (t−s) . Bothstatementsofthetheoremnowfollow. 4.3 Thelinearquadraticcontrolproblem In general, both the deterministic and stochastic linear quadratic control problems extend to theinfinite-dimensionalsettingbyreplacingthematriceswithoperators,withextraflexibility of choosing various inner products in the cost functional. The main question is then analysis of an operator-valued Riccati equation; see, for example the book by Bensoussan et al. [8]. Below, we consider several specific examples in deterministic and stochastic setting that lead toaclosed-formsolutionanddonotexplicitlyappearin[8]oranyotherexistingworks. Inall theexamples, theoriginalinfinite-dimensionalproblemisreducedtoanuncoupledsystemof one-dimensionallinearquadraticcontrollers. LetAbeanm-operatorinaHilbertscale H ={H γ , γ∈R};Ah k =λ k h k . Asafirstexample,considerthecontrolledparabolicequation ˙ x u (t)+Ax u (t) =u(t) (4.3.1) 87 withx u (0) = x 0 ∈H r andu ∈ L 2 ((0,T);H r ). By Theorem 4.2.4,x(t) ∈H r+m for every t> 0,andthenitisnaturaltoconsiderthecostfunctional J(u) = Z T 0 kx u (t)k 2 r+m +ku(t)k 2 r dt+kx u (T)k 2 r+m . (4.3.2) Recallthatthenormk·k γ isdefinedby kvk 2 γ = X k≥1 λ γ/m k v k . DefinethefamilyoflinearoperatorsP(t), 0≤t≤T,by P(t)h k =p k (t)h k , (4.3.3) where ˙ p k (t) =p 2 k (t)+2λ k p k (t)−λ k , p k (T) =λ k . ByLemma2.2.7onpage19, p k (t) =−λ k +a k + 2a k β k e 2a k (T−t) −1 , (4.3.4) with a k = q λ 2 k +λ k , β k = 2λ k +a k 2λ k −a k ; (4.3.5) ifλ k = 1/3,thenp k (t) = 1/3forallt. Similartothefinite-dimensionalcase,thefunction ¯ u(t,x) =−P(t)x (4.3.6) 88 is the optimal feed-back control minimizing the cost function (4.3.2). More specifically, we havethefollowingresult. 4.3.1Theorem. Consider the evolution equation ˙ x ∗ (t)+(A+P(t))x ∗ = 0, x ∗ (0) =x 0 , (4.3.7) and defineu ∗ (t) =−P(t)x ∗ (t). Thenx ∗ ∈ L 2 ((0,T);H r+m ),u ∗ ∈ L 2 ((0,T);H r ),x ∗ (t)∈ H r+m for allt> 0, andJ(u ∗ )≤J(u) for everyu∈L 2 ((0,T);H r ). Proof. Sincelim k→∞ λ k = +∞,itfollowsthatλ k > 1/3andβ k > 1forallsufficientlylarge k. By(4.3.3)and(4.3.4),wehave ˙ x ∗ k =−a k 1+ 2 β k e 2a k (T−t) −1 x ∗ k , anda k >λ k . Therefore |x ∗ k (t)| 2 ≤|x k (0)| 2 e −2λ k t , (4.3.8) andsincebyassumption X k≥1 λ r/m k |x k (0)| 2 <∞, (4.3.9) itfollowsthatx ∗ ∈L 2 ((0,T);H r+m )andx ∗ (t)∈H r+m forallt> 0. Next, Z T 0 |u ∗ k (t)| 2 dt = Z T 0 |p k (t)x ∗ k (t)| 2 dt. Since lim k→∞ (−λ k +a k ) = lim k→∞ λ k λ k + p λ 2 k +λ k = 1 2 89 and lim k→∞ β k = lim k→∞ 2+ q 1+λ −1 k 2− q 1+λ −1 k = 3, itfollowsthat,forallsufficientlylargek,Then p k (t)< 1+2λ k e 2λ k (t−T) and,by(4.3.8), Z T 0 |p k (t)x ∗ k (t)| 2 dt≤C|x k (0)| 2 forsomeconstantC,implyingthatu ∗ ∈L 2 ((0,T);H r ). Finally,notethat ˙ x u k (t) =−λ k x u k (t)+u k (t) andJ(u) = P k≥1 J k ,where J k =λ r/m k Z T 0 λ k |x u k (t)| 2 +|u k (t)| 2 dt+λ 1+(r/m) k |x u k (T)| 2 . By (2.2.43) on page 21,u ∗ k (t) = −p k (t)x ∗ k (t) minimizesJ k for everyk. This completes the proofofthetheorem. AsT →∞,p k (t)→−λ k + p λ 2 k +λ k for every fixedt<T, which is the eigenvalue of theoperator−A+(A 2 +A) 1/2 . Thisobservationleadstothefollowingresultfortheinfinite horizonproblem: 4.3.2Corollary. The solution of the equation ˙ x ∗ (t)+(A 2 +A) 1/2 x ∗ (t) = 0 90 is the optimal trajectory, minimizing Z +∞ 0 kx u (t)k 2 r+m +ku(t)k 2 r dt, withx u defined by(4.3.1). Asasecondexample,considercontrolledstochasticparabolicequation dX u (t)+AX u (t)dt =u(t)dt+X(t)dW(t) (4.3.10) withx u (0) = x 0 ∈H r andu∈ L 2 ((0,T);H r );W is a one-dimensional standard Brownian motion. By Theorem 4.2.9 on page 86, X(t) ∈H r+m for every t > 0, and then it is natural to considerthecostfunctional J(u) =E Z T 0 kX u (t)k 2 r+m +ku(t)k 2 r dt+EkX u (T)k 2 r+m . (4.3.11) Nowthecorrespondingone-dimensionallinearquadraticproblemsaretominimize λ r/m k E Z T 0 λ k |X u k (t)| 2 +|u k (t)| 2 dt+λ 1+(r/m) k |X u k (T)| 2 given dX u k = (−λ k X u k +u k )dt+X u k dW(t); by(2.3.27)onpage35,theoptimalcontrolforeveryk isu ∗ k (t) =−p k (t)X ∗ (t),where ˙ p k (t) =p 2 k (t)+(2λ k −1)p k (t)−λ k , p k (T) =λ k . 91 Define the operatorP(t) byP(t)h k = p k (t)h k . Then, similar to Theorem 4.3.1, we have the followingresult. 4.3.3Theorem. Consider the stochastic parabolic equation dX ∗ (t)+(A+P(t))X ∗ dt =X ∗ (t)dW(t), x ∗ (0) =x 0 , (4.3.12) anddefineu ∗ (t) =−P(t)X ∗ (t). ThenX ∗ ∈L 2 (Ω×(0,T);H r+m ),u ∗ ∈L 2 (Ω×(0,T);H r ), x ∗ (t) ∈ L 2 (Ω×H r+m ) for all t > 0, and J(u ∗ ) ≤ J(u) for every adapted u ∈ L 2 (Ω× (0,T);H r ). Letw k , k≥ 1, be independent standard Brownian motions. As a third example, consider theequationdrivenbycylindricalBrownianmotionon(0,π): dX u (t,ξ) = (X u ξξ (t,ξ)+u(t,ξ))dt+dW(t,ξ) (4.3.13) with zero initial and boundary conditions. For this equation,H = H 0 = L 2 ((0,π)), A = −∂ 2 /∂ξ 2 withzeroboundaryconditions,h k = p 2/πsin(kx),λ k =k 2 ,andso W(t,ξ) = p 2/π X k≥1 w k (t)sin(kξ) is an element ofH −α for everyα > 1/2; then, by Theorem 4.2.7,X u (t)∈L 2 (Ω;H −α+1 for everyu∈L 2 (Ω×(0,T);H r ,aslongasr≥−α. Letusconsiderthefollowingperformanceindexfor(4.3.13): J(u) =E Z T 0 kX u k 2 0 (t)+kuk 2 0 (t) dt+EkX u k 2 0 (T), (4.3.14) 92 where kfk 2 0 (t) = Z π 0 |f(t,ξ)| 2 dξ. Note that the normk·k 0 in the definition ofJ does not reflect the optimal regularity of the processX u ,andtheobjectiveistoseehowthisselectionoftheperformanceindexaffectsthe resultingoptimalcontrol. DefinethefamilyoflinearoperatorsP(t), 0≤t≤T,by P(t,h k ) =p k (t)h k , (4.3.15) where ˙ p k (t) =p 2 k (t)+2k 2 p k (t)−1, p k (T) = 1 andh k (x) = p 2/π sin(kx), k ≥ 1. In this situation we writeP(t,x) rather thanP(t)x, to avoidpotentialconfusionwiththepoint-wisemultiplication. ByLemma2.2.7onpage19, p k (t) =−k 2 +a k + 2a k β k e 2a k (T−t) −1 , (4.3.16) with a k = √ k 4 +1, β k = 1+k 2 +a k 1+k 2 −a k . (4.3.17) Similartothefinite-dimensionalcase,thefunction ¯ u(t,x) =−P(t,x) (4.3.18) is the optimal feed-back control minimizing the cost function (4.3.14). More specifically, we havethefollowingresult. 93 4.3.4Theorem. Consider the stochastic heat equation dX ∗ (t,ξ) = (X ∗ ξξ +P(t,X ∗ ))dt+dW(t,ξ) (4.3.19) forξ∈ (0,π),with zero initial and boundary conditions, and define u ∗ (t,ξ) =− p 2/π X k≥1 p k (t)X ∗ k (t)sin(kξ). Then, for everyt≥ 0,u ∗ (t,·)∈ L 2 (Ω;H 1 ),X ∗ (t)∈ L 2 (Ω;H) andJ(u ∗ )≤ J(u) for every u∈L 2 (Ω×(0,T);H). Proof. By(4.3.15)and(4.3.16),wehave dX ∗ k =−a k 1+ 2 β k e 2a k (T−t) −1 X ∗ k +dw k (t), anda k >k 2 . Therefore E|X ∗ k (t)| 2 ≤ 1−e −2k 2 t k 2 , (4.3.20) andsoX ∗ (t)∈L 2 (Ω;H), t≥ 0. Next, Z T 0 |u ∗ k (t)| 2 dt = Z T 0 |p k (t)X ∗ k (t)| 2 dt. Since lim k→∞ k 2 (−k 2 + √ k 4 +1) = 2 and lim k→∞ k −2 β k = 1, itfollowsthat,forallsufficientlylargek, p k (t)< 2 k 2 +16e −2k 2 (T−t) 94 and,by(4.3.8), Z T 0 E|p k (t)x ∗ k (t)| 2 dt≤ C k 4 forsomeconstantC,implyingthatu ∗ ∈L 2 (Ω×(0,T);H 1 ). Finally,notethat dX u k (t) = (−k 2 X u k (t)+u k (t))dt+dw k (t) andJ(u) = P k≥1 J k ,where J k =E Z T 0 |X u k (t)| 2 +|u k (t)| 2 dt+E|X u k (T)| 2 . By (2.3.26) on page 35,u ∗ k (t) = −p k (t)x ∗ k (t) minimizesJ k for everyk. This completes the proofofthetheorem. 95 Chapter5 Parabolicequations: Exactcontrol 5.1 Introduction One of the reasons why the analogy between the finite- and infinite-dimensional problems is not complete is the absence of the Lebesgue measure in infinite dimensions. In particular, while the Gaussian measure easily extends to infinite dimensions, the Gaussian density does not. Forexample,considertheOrnstein-Uhlenbeckprocess dX(t) =AX(t)dt+dW. In finite dimensions, whenA is a symmetric negative-definite matrix, the solution is a Gaus- sian process and has an invariant measure that is Gaussian, with a known density. In infinite dimensions,whenAisaself-adjointnegative-definiteoperator,thesolutionisalsoaGaussian process and also has an invariant measure, but the density of this measure cannot be written because of the lack of the reference measure. In fact, the invariant measure of the infinite- dimensional Ornstein-Uhlenbeck process is often used as the reference measure on the cor- responding infinite-dimensional space . This, in particular, means that, in infinite dimensions, there is no clear analog of the logarithmic transformation. On the other hand, we can still implement the exact feed-back control using a driving function. The analysis of the exact feed-backcontrolusingadrivingfunctionfordeterministicandstochasticparabolicequations is the main goal of this chapter. To simplify the analysis, a single driving function is used for alltheFouriercomponentsofthesolution. 96 There is another possible approach to constructing an exact feed-back control — by pass- ing to the limit in a linear-quadratic control problem. Unlike the finite-dimensional case, the analysisofthisapproachmustbecarriedoutseparately,asthedrivingfunctionscanbediffer- entindifferentFouriercomponentsofthesolution. Thegeneralizationofthismethodleadsto thenotionofthe driving operator,althoughwedonotinvestigatethisquestion. Anotherspecialfeatureoftheinfinite-dimensionalexactcontrolproblemsisthevarietyof norms in which the convergence to the prescribed terminal condition can occur. In a Hilbert scale,itisdesirabletohavetheconvergenceinthehighestpossiblenorm. Yetanotherspecialfeatureoftheinfinite-dimensionalproblemsisthatparabolicregularity puts restrictions on the values of the solution given the regularity of the open-loop control. Theorem5.2.1addressesthisquestionfordeterministicparabolicequations. In what follows, the main object is an abstract diagonalizable parabolic equation and the goal is to get a closed-form expression for the exact control. On the other hand most exist- ing works on the subject deal with concrete partial differential equations, both parabolic and hyperbolic. Explicit presence of the spatial variable means a possibility to study more sub- tle questions, such as controllability with control applied on portion of domain and/or its the boundary (Albano [1], Barbu et al. [3], Fern´ ando-Cara and Guerrero [20, 19], Fu et al. [24], Immanuilov and Yamamoto [26], Kim [29], Osses [35], Russell [39], Yin [51], Zhang [54], etc.) The review paper by Zuazua [58] describes the main tools, such as Carleman estimates and microlocal analysis. As a rule, an explicit expression of the control is not an objective. A series of papers by An Ton Bui [46, 48, 47, 43, 45, 44] deals exclusively with more tradi- tional exact controllability of stochastic partial differential equations, although the resulting control is usually not adapted and not explicitly computable. Non-adaptedness is especially unpleasant in equations with multiplicative noise because of the added difficulties related to theinterpretationofthestochasticintegral. 97 Everywhere in this chapter, we fix (Ω,F,{F t } t≥0 ,P), a stochastic basis satisfying the usual conditions (completeness ofF 0 and the right continuity of the filtrationF,{F t } t≥0 ). Both W = W(t) and w = w(t), t ≥ 0, can denote the standard Brownian motion (either one- or multi-dimensional) on this stochastic basis, while W can also denote a cylindrical Brownianmotion(seeDefinition4.2.1onpage80) 5.2 Deterministicparabolicequations LetH ={H γ , γ ∈R}, be a scale of real Hilbert spaces andA, an m-operator on this scale (Definition 4.2.2, page 81). Recall that this means A is linear and bounded fromH γ+m to H γ−m for somem> 0 and everyγ∈R;A has a system of eigenfunctionsh k , k≥ 1, that is orthonormal and complete inH =H 0 , and everyh k belongs to everyH γ ; the corresponding eigenvaluesλ k arepositive. Westartwithanopen-loopexactcontrolandconsidertheevolutionequation ˙ x u (t)+Ax u (t) =u(t), t> 0, (5.2.1) wherex u (0) = x(0) ∈H γ 1 is given, and the objective is to find the controlu so thatx u (T) takesaprescribedvalueg∈H γ 2 . The general parabolic regularity (Theorem 4.2.4 on page 83) puts certain restrictions on where the given controlu can move the system (5.2.1). For example, ifu∈ L 2 ((0,T);H γ ), then x u (t) ∈ H γ+m for every t > 0. In particular, we cannot, in general, move the sys- tem (5.2.1) from an initial position inH γ to another position inH γ with a square-integrable H γ -valued control. The following theorem gives one general result about existence of exact control. 98 5.2.1 Theorem. Assume that A is an m-operator in the Hilbert scale H, x(0) ∈ S γ∈R H γ , g∈H r+m for somer∈R. Then the functionu ∗ (t) = P k≥1 u ∗ k (t)h k with u ∗ k (t) =λ k e λ k t g k −x k (0)e −λ k T sinh(λ k T) has the following properties: • The corresponding solutionx ∗ of (5.2.1) satisfiesx ∗ (T) =g. • The functionu ∗ minimizes the value of R T 0 ku(t)k 2 r dt over all the controlsu that ensure x u (T) =g. Proof. Fromx u (t) = P k≥1 x u k (t)h k and(5.2.1)wefind x u (T) = X k≥1 x k (0)e −λ k T + Z T 0 e −λ k (T−t) u k (t)dt h k . (5.2.2) Ifx u k (t) =g k ,then Z T 0 u k (t)e λ k t dt =e λ k T g k −x k (0)e −λ k T . (5.2.3) BytheCauchy-Schwartzinequality, Z T 0 u k (t)e λ k t dt 2 ≤ Z T 0 u 2 k (t)dt Z T 0 e 2λ k t dt (5.2.4) or Z T 0 u 2 k (t)dt≥ 2λ k e 2λ k T −1 Z T 0 u k (t)e λ k t dt 2 , (5.2.5) andtheequalityholdsifandonlyifu k (t) =a k e λ k t forsomenumbera k . Thus,u ∗ k (t) =a k e λ k t foreveryk≥ 1,where,by(5.2.3), a k Z T 0 e 2λ k t dt =e λ k T g k −x k (0)e −λ k T , (5.2.6) 99 thatis, a k = 2λ k e 2λ k T −1 e −λ k T g k −x k (0)e −λ k T . (5.2.7) Consequently, Z T 0 ku ∗ (t)k 2 r dt = X k≥1 e 2λ k T −1 2λ k a 2 k = X k≥1 λ (r/m)+1 k (g k −e −λ k T x k (0)) 2 e −λ k T sinh(λ k T) . (5.2.8) By assumption, P k≥1 λ (r/m)+1 k g 2 k < ∞, and thus u ∗ ∈ L 2 ((0,T);H r ). Note also that if g / ∈H r+m ,then(5.2.8)implies R T 0 ku(t)k 2 r dt =∞foreveryuensuringx u (T) =g. Next, we consider the feed-back exact control. Similar to the ordinary differential equa- tions, uniqueness of solution requires such a control to develop singularity at the terminal timeT. Accordingly, letK = K(t) be a driving function (Definition 3.3.7 on page 70), for example, K(t) = 1/(T −t). We start with the problem of driving to zero the solution of a homogeneousequation. 5.2.2Theorem. Consider the equation ˙ x(t)+Ax(t)+K(t)x(t) = 0, 0<t<T, Assume that A is an m-operator in the Hilbert scale H (Definition 4.2.2, page 81), x(0) ∈ S γ H γ (that is,x(0)∈H γ forsomeγ∈R), andK =K(t) is a driving function (Definition 3.3.7, page 70). Thenlim t→T −kx(t)k γ = 0 foreveryγ∈R. Proof. Thisfollowsfromtheequality kx(t)k 2 γ =e −2 R t 0 K(s)ds X k≥1 |x k (0)| 2 λ γ/m e −λ k t 100 andtheboundednessofλ r k e −λ k T uniformlyink foreveryr∈R. Recallthat,bythedefinition ofthedrivingfunction,lim t→T −exp − R t 0 K(s)ds = 0. Thesolutionoftheinhomogeneousequationcanalsobedriventozeroinasimilarway,but nowthenorminwhichthezeroisreachedwilldependontheregularityoftheinhomogeneous term. 5.2.3Theorem. Consider the equation ˙ x(t)+Ax(t) =f(t)−K(t)x(t), 0<t<T, Assume thatA is an m-operator in the Hilbert scale H,x(0) ∈ S γ H γ ,f ∈ L 2 ((0,T);H r ) for somer∈R, andK =K(t) is a driving function. Thenlim t→T −kx(t)k r = 0. Proof. Thisfollowsfromtheinequality kx(t)k 2 r ≤ 2e −2 R t 0 K(s)ds X k≥1 |x k (0)| 2 λ r/m e −λ k t +2 X k≥1 λ r/m k Z t 0 f k (s)e −λ k (t−s)− R t s K(τ)dτ ds 2 . Ast→ T − , the first term on the right tends to zero by the previous theorem, and the second termgoestozeroby(3.3.53)onpage71,afterapplyingtheCauchy-Schwartzinequality: Z t 0 f k (s)e −λ k (t−s)− R t s K(τ)dτ ds 2 ≤ Z t 0 f 2 k (s)e −2λ k (t−s) ds Z t 0 e −2 R t s K(τ)dτ ds. UsingTheorem5.2.3,wecannowestablishageneralresultaboutexactfeed-backcontrol ofalinearparabolicequation. 101 5.2.4Theorem. Consider the equation ˙ x(t)+Ax(t)+K(t)(x(t)−g) =f(t), 0<t<T. (5.2.9) Assume that A is an m-operator in the Hilbert scale H (Definition 4.2.2, page 81), x(0) ∈ S γ H γ ,f ∈L 2 ((0,T);H r ) for somer∈R,g∈H r+2m , andK =K(t) is a driving function (Definition 3.3.7, page 70). Thenlim t→T −kx(t)−gk r = 0. Proof. Definey(t) =x(t)−g. Then ˙ y(t)+Ay(t)+K(t)y(t) =f(t)−Ag. SinceAg∈H r , theresultfollowsfromTheorem5.2.3. 5.3 Stochasticparabolicequations Similartostochasticordinarydifferentialequation,wewillconcentrateontheMarkovcontrol, andthentheproblemofexactcontrolisreducedtotheanalysisoftheequation ˙ X(t)+AX(t)+K(t)(X(t)−G) =f(t)+F(t,X(t)) ˙ W(t), (5.3.1) whereA is an m-operator in a Hilbert scale H ={H γ , γ ∈R} (see Definition 4.2.2 on page 81),K is a driving function (Definition 3.3.7 on page 70),G is a fixed element of someH γ , F(t,·) is a linear function and ˙ W(t) is a noise term; the precise description of the functionF andthenoise ˙ W willbegivenlater. ThegoaltostudytheconvergenceofthesolutionX(t)to Gast%T. Thefirsttworesultsareforequationswithadditivenoise: dX +AXdt+K(t)(X(t)−G) =f(t)+ X k≥1 g k (t)dW k (t)h k . (5.3.2) 102 5.3.1 Theorem. Assume that A is an m-operator in the Hilbert scale H, X(0) ∈ S γ∈R H γ , f ∈ L 2 ((0,T);H r ), G ∈ H r+2m for some r ∈ R, and each g k = g k (t) is a continu- ous on [0,T] function. If the series P k≥1 λ r/m k g 2 k (t) converges uniformly int ∈ [0,T], then lim t→T −EkX(t)−Gk 2 r = 0. Proof. Similar to the proof of Theorem 5.2.4, we will assume that g = 0, and the proof of Theorem5.2.4showsthatwecanalsoassumeX(0) = 0,f = 0. Next, we proceed similar to the proof of Theorem 4.2.7 on page 84. Define Φ(t) = exp − R t 0 K(s)ds .Then X k (t) = Z t 0 e −λ k (t−s) g k (s) Φ(t) Φ(s) dW k (s) and EkX(t)k 2 γ = Z t 0 X k≥1 λ γ/m k g 2 k (s)e −2λ k (t−s) ! Φ 2 (t) Φ 2 (s) ds. Clearly, X k≥1 λ γ/m k g 2 k (s)e −2λ k (t−s) ≤ X k≥1 λ γ/m k g 2 k (s). Uniform convergence of P k≥1 λ r/m k g 2 k (t) means that the sum is a continuous on [0,T] func- tion, which, together with property (3.3.53), page 71, of the driving function, implies the first statementofthetheorem. Iftheseries P k≥1 λ r/m k g 2 k (t)doesnotconverge,thenamoredelicateanalysisisnecessary, with more detailed assumptions about the eigenvalues λ k and the driving function K. In particular,weassumethatthereexistapositivenumberc A andapositiveintegernumberdso that lim k→∞ k −2m/d λ k =c A . (5.3.3) 103 Forexample,eigenvaluesofellipticpartialdifferentialoperatorsorder2monad-dimensional manifoldhavethisproperty(seeSafarovandVassiliev[40]). 5.3.2 Theorem. In equation (5.3.2), assume thatA is an m-operator in the Hilbert scale H (Definition 4.2.2, page 81), X(0) ∈ S γ∈R H γ , f ∈ L 2 ((0,T);H r ), G ∈ H r+2m for some r ∈ R. If sup k≥1,t∈[0,T] |g k (t)| < ∞, K(t) = p/(T −t), p > 0, and (5.3.3) holds, then lim t→T −EkX(t)−Gk 2 r 0 = 0 for everyr 0 < min(r,m−(d/2)). Proof. Similar to the proof of Theorem 5.2.4, we will assume that g = 0, and the proof of Theorem5.2.4showsthatwecanalsoassumeX(0) = 0,f = 0. Undertheseassumptions, X k (t) = (T −t) p Z t 0 e −λ k (t−s) g k (s)(T −s) −p dW k (s) and EkX(t)k 2 r = X k≥1 λ r/m k EX 2 k (t) = (T −t) 2p Z t 0 g 2 k (s)e −2λ k (t−s) (T −s) −2p ds. By(5.3.3),andassumptionong k ,thereexistpositivenumbersa, C 0 suchthat EkX(t)k 2 r ≤C 0 (T −t) 2p Z t 0 X k≥1 k 2r/d e −a(t−s)k 2m/d (T −s) −2p ds. If 2r/d < −1 or r < −d/2, then the series on the right-hand side of the last inequality converges, and the statement of the theorem follows from Theorem 5.3.1. If r ≥ −d/2, comparethesumwiththeintegral: X k≥1 k 2r/d e −a(t−s)k 2m/d ≤ Z ∞ 0 x 2r/d e −a(t−s)x 2m/d dx. 104 Bydirectcomputation, Z ∞ 0 x 2r/d e −a(t−s)x 2m/d dx = 1 (a(t−s)) q Γ(q), where q = r+(d/2) m , Γ(t) = Z ∞ 0 y t−1 e −y dy. Asaresult, EkX(t)k 2 r ≤C 1 (T −t) 2p Z t 0 (t−s) −q (T −s) −2p ds for someC 1 > 0. If−d/2 ≤ r < m− (d/2), then 0 ≤ q < 1. We writeq = 1− 2ε−δ, 0<ε≤ min(1/2,2p),δ≥ 0,andnotethat,fors<t<T, (T −t) 2p ≤ (T −t) ε (T −s) 2p−ε , (T −s) −2p ≤ (T −s) −2p+ε (t−s) −ε . Thestatementofthetheoremnowfollows: EkX(t)k 2 r ≤C 1 (T −t) 2p Z t 0 (t−s) −1+2ε (T −s) −2p ds ≤C 1 (T −t) ε Z T 0 (T −s) −1+ε+δ dt≤ C 1 T ε+δ ε+δ (T −t) ε → 0, t→T − . Thenextresultisforequationswithmultiplicativenoise dX(t)+AX(t)dt+K(t)(X(t)−G) =f(t)dt+B(t,X(t))dW(t), (5.3.4) whereK is a driving function,G is a deterministic element of someH γ ,f is a deterministic functionswithvaluesinsomeH γ ,B isalinearoperatorsuchthatB(t,h k ) =σ k (t)h k ,k≥ 1, 105 andW is a one-dimensional standard Brownian motions. To ensure existence of solution of (5.3.4)(seeTheorem4.2.9onpage86),wedefine μ k (t) = 1 2 Z t 0 σ 2 k (s)ds (5.3.5) and assume that there exists a number δ ∈ [0,1] and a number C ∈ R such that, for all k≥ 1, t≥s≥ 0, (1−δ)λ k (t−s)− μ k (t)−μ k (s) ≥C. (5.3.6) 5.3.3 Theorem. Assume that A is an m-operator in the Hilbert scale H (Definition 4.2.2, page 81),X(0)∈H r 0 ,G∈H r+2m ,K is a driving function (Definition 3.3.7, page 70), and f ∈L 2 ((0,T);H r ). If (5.3.6) holds withδ = 0, thenlim t→T −EkX(t)−Gk 2 α = 0, whereα = min(r,r 0 ). If (5.3.6) holds withδ> 0, thenlim t→T −EkX(t)−Gk 2 r+m = 0. Proof. With no loss of generality, assume that G = 0 (see proof of Theorem 5.2.4, page 102). On the other hand, since the fundamental solutions of equations (5.3.4) and (5.2.9) are different,wecannoteasilyputf = 0. Define Ψ k (t) = exp −λ k t− 1 2 Z t 0 σ 2 k (s)ds+ Z t 0 σ k (s)dW(s) , F k (t,s) =E Ψ k (t) Ψ k (s) 2 , and Φ(t) = exp Z t 0 K(s)ds ; bydirectcomputations, F k (t,s) = exp −2 λ k (t−s)− μ k (t)−μ k (s) . 106 Then X k (t) =X k (0)Ψ k (t)Φ(t)+Ψ k (t)Φ(t) Z t 0 f k (s) Φ(s)Ψ k (s) ds and EkX(t)k 2 γ ≤ 2Φ 2 (t) X k≥1 λ γ/m k F k (t,0)X k (0)+ Z t 0 F k (t,s) Φ 2 (s) ds Z t 0 f 2 k (s)ds . We start by looking at the contribution of the inhomogeneous termf(t). Note that (5.3.6) impliesF k (t,s)≤C 0 e −δλ k (t−s) forallk≥ 1, t≥s≥ 0,sothat Φ 2 (t) X k≥1 λ r/m k Z t 0 F k (t,s) Φ 2 (s) ds Z t 0 f 2 k (s)ds ≤C 0 Z T 0 kf(t)k 2 r dt Z t 0 Φ 2 (t) Φ 2 (s) ds ; by (3.3.53) on page 71, the right-hand side of the above inequality converges to zero as t→ T − . Next,welookatthecontributionoftheinitialconditionX(0). Ifδ = 0,then Φ 2 (t) X k≥1 λ r 0 /m k F k (t,0)X k (0)≤C 0 kX(0)k 2 r 0 Φ(t)→ 0, t→T − . Ifδ > 0, then there exists a numberC 1 depending only onδ,r,r 0 ,m andT, such that, for all t>T/2, Φ 2 (t) X k≥1 λ r/m k F k (t,0)X k (0)≤C 0 Φ 2 (t) X k≥1 λ |r 0 −r|/m k e −δλ k t λ r 0 /m k X k (0) ≤C 0 C 1 kX(0)k 2 r 0 Φ(t)→ 0, t→T − . ThiscompletestheproofofTheorem5.3.3. 107 As a final example, we construct an infinite-dimensional analogue of the Ornstein- Uhlenbeck bridge by putting together a collection of independent one-dimensional Ornstein- Uhlenbeckbridges(2.3.34)onpage38. Letw k , k≥ 1,beindependentstandardBrownianmotions. GivenT > 0,definethelinear operatoru ∗ (t,·)onL 2 ((0,π)),0<t<T,by (u ∗ (t,X))(ξ) = p 2/π X k≥1 u ∗ k (t,X)sin(kξ), ξ∈ (0,π), (5.3.7) where u ∗ k (t,X) =− p 2/π 2k 2 e k 2 (T−t) −1 Z π 0 X(ξ)sin(kξ)dξ. In this situation we writeu ∗ (t,X) rather thanu ∗ (t)X, to avoid potential confusion with the point-wisemultiplication. ConsidertheequationdrivenbycylindricalBrownianmotionon(0,π): dY(t,ξ) = (Y ξξ (t,ξ)+u ∗ (t,Y(t,ξ))dt+dW(t,ξ) (5.3.8) with zero initial and boundary conditions. For this equation,H =H 0 = L 2 ((0,π)),A =−∂ 2 /∂ξ 2 with zero boundary conditions,h k = p 2/πsin(kx),λ k =k 2 , and so W(t,ξ) = p 2/π X k≥1 w k (t)sin(kξ) is an element ofH −α for everyα > 1/2. The following result shows why the processY is naturaltocallaninfinite-dimensionalOrnstein-Uhlenbeckbridge. 5.3.4Theorem. Equation(5.3.8) has a unique solution such that 1. for each k, Y k (t) = p 2/π R π 0 Y(t,ξ)sin(kξ)dξ is a one-dimensional Ornstein- Uhlenbeck bridge; 108 2. for all0≤t≤T,E R π 0 |Y(t,ξ)| 2 dξ <∞; 3. lim t→T −EkY(t)k 2 0 = 0. Proof. By(5.3.8), dY k (t) =− k 2 + 2k 2 e k 2 (T−t) −1 Y k (t)dt+dw k (t), or dY k (t) =−k 2 coth k 2 (T −t) Y k (t)dt+dw k (t), (5.3.9) which is the same as (2.3.34) on page 38, withA = −k 2 . This proves the first statement of thetheorem. Next,notethat,similartothetrigonometricfunctions,wehave Z coth(x)dx = ln|sinh(x)|+C, Z dx sinh 2 (x) =−coth(x)+C. Asaresult,keepinginmindthatY k (0) = 0, Y k (t) = sinh k 2 (T −t) Z t 0 dw k (s) sinh k 2 (T −s) and E|Y k (t)| 2 = 1 k 2 sinh k 2 (T −t) cosh k 2 (T −t) −sinh k 2 (T −t) coth k 2 T . Rewritingthehyperbolicfunctionsusingexponentials, E|Y k (t)| 2 = 1 k 2 sinh k 2 (T −t) e −k 2 (T−t) −sinh k 2 (T −t) 2e −k 2 T e k 2 T −e −k 2 T ! . 109 Since,forT > 0andt<T, e −k 2 T e k 2 T −e −k 2 T = e −2k 2 T 1−e −2k 2 T ≤C(T)e −2k 2 (T−t) withC(T)> 0dependingonlyonT,and2e −x sinh(x) = 1−e −2x ,wefind E|Y k (t)| 2 ≤ 1 k 2 1−e −2k 2 (T−t) +C(T) 1−e −2k 2 (T−t) 2 ≤ 1−e −2k 2 (T−t) k 2 (1+C(T)). Note that, for every α > 0, the function f(x) = x −2 (1−e −αx 2 ), x > 0, is monotonically decreasing. Then E Z π 0 Y 2 (t,ξ)dξ = X k≥1 E|Y k (t)| 2 ≤ (1+C(T)) Z ∞ 0 1−e −x 2 (T−t) x 2 dx≤C 1 (T) √ T −t, whichcompletestheproofofthetheorem. 110 References [1] P. Albano and D. Tataru, Carleman estimates and boundary observability for a coupled parabolic-hyperbolic system ,Electron.J.DifferentialEquations(2000),no.22,1–15. [2] B.D.O.AndersonandJ.B.Moore, Linear optimal control,Prentice-HallInc.,1971. [3] V.Barbu,A.R˘ as ¸canu,andG.Tessitore,Carlemanestimatesandcontrollabilityoflinear stochastic heat equations,Appl.Math.Optim.47(2003),no.2,97–120. [4] M.BardiandI.Capuzzo-Dolcetta, Optimal control and viscosity solutions of Hamilton- Jacobi-Bellman equations ,Birkh¨ auserBostonInc.,1997. [5] R. Bellman, The stability of solutions of linear differential equations, Duke Math. J. 10 (1943),643–647. [6] R. Bellman, On the theory of dynamic programming, Proc. Nat. Acad. Sci. U. S. A. 38 (1952),716–719. [7] , Dynamic programming, Dover Publications Inc., Mineola, NY, 2003, Reprint ofthesixth(1972)edition;firstedition—1957,PrinicetonUniversitypress. [8] A.Bensoussan,G.DaPrato,M.C.Delfour,andS.K.Mitter,Representationandcontrol ofinfinite-dimensionalsystems.Vol.II ,Systems&Control: Foundations&Applications, Birkh¨ auserBostonInc.,Boston,MA,1993. [9] J.-M. Bismut, An introductory approach to duality in optimal stochastic control, SIAM Rev.20(1978),no.1,62–78. [10] , Large deviations and the Malliavin calculus,ProgressinMathematics,vol.45, Birkh¨ auserBostonInc.,Boston,MA,1984.MRMR755001(86f:58150) [11] B.V.BondarevandYu.N.Polshkov,Ontheoptimalcontrolofthesolutionofastochastic first-order partial differential equation , Cybernet. Systems Anal.34 (1998), no. 2, 269– 276. [12] P. Cannarsa and C. Sinestrari, Semiconcave functions, Hamilton-Jacobi equations, and optimal control,Birkh¨ auserBostonInc.,2004. [13] S. Cerrai, Second order PDE’s in finite and infinite dimension, Lecture Notes in Mathe- matics,vol.1762,Springer-Verlag,Berlin,2001. 111 [14] S. Chen, X. Li, and X.Y Zhou, Stochastic linear quadratic regulators with indefinite control weight costs,SIAMJ.ControlOptim.36(1998),no.5,1685–1702(electronic). [15] S. Chen and X.Y Zhou, Stochastic linear quadratic regulators with indefinite control weight costs. II,SIAMJ.ControlOptim.39(2000),no.4,1065–1081(electronic). [16] G. Da Prato, An introduction to infinite-dimensional analysis , Universitext, Springer- Verlag,Berlin,2006. [17] G.DaPratoandJ.Zabczyk,SecondorderpartialdifferentialequationsinHilbertspaces, London Mathematical Society Lecture Note Series, vol. 293, Cambridge University Press,Cambridge,2002. [18] L.C.Evans, Partial differential equations,AmericanMathematicalSociety,1998. [19] E. Fern´ andez-Cara and S. Guerrero, Global Carleman estimates for solutions of parabolic systems defined by transposition and some applications to controllability, AMRXAppl.Math.Res.Express(2006),Art.ID75090,31. [20] , Global Carleman inequalities for parabolic systems and applications to con- trollability,SIAMJ.ControlOptim.45(2006),no.4,1399–1446(electronic). [21] W.H.FlemingandR.W.Rishel,Deterministicandstochasticoptimalcontrol,Applica- tionsofMathematics,vol.1,Springer,NewYork,1975. [22] W. H. Fleming and S. J. Sheu, Stochastic variational formula for fundamental solutions of parabolic PDE,Appl.Math.Optim.13(1985),no.3,193–204. [23] W. H. Fleming and H. M. Soner, Controlled Markov processes and viscosity solutions, ApplicationsofMathematics(NewYork),vol.25,Springer-Verlag,NewYork,1993. [24] X. Fu, J. Yong, and X. Zhang, Exact controllability for multidimensional semilinear hyperbolicequations,SIAMJ.ControlOptim.46(2007),no.5,1578–1614(electronic). [25] T. H. Gronwall, Note on the derivative with respect to a parameter of the solutions of a system of differential equations,Ann.ofMath20(1919),292–296. [26] O. Y. Imanuvilov and M. Yamamoto, Carleman inequalities for parabolic equations in Sobolev spaces of negative order and exact controllability for semilinear parabolic equations,Publ.Res.Inst.Math.Sci.39(2003),no.2,227–274. [27] B. Jamison, The Markov processes of Schr¨ odinger, Z. Wahrscheinlichkeitstheorie und Verw.Gebiete32(1975),no.4,323–331. [28] R. E. Kalman, Contributions to the theory of optimal control, Bol. Soc. Mat. Mexicana (2)5(1960),102–119. 112 [29] J. U. Kim, Approximate controllability of a stochastic wave equation, Appl. Math. Optim.49(2004),no.1,81–98. [30] S.G.Kre˘ ın,Yu. ¯ I.Petun¯ ın,andE.M.Sem¨ enov,Interpolationoflinearoperators,Trans- lations of Mathematical Monographs, vol. 54, American Mathematical Society, Provi- dence,R.I.,1982. [31] N.V.Krylov,M.R¨ ockner,andJ.Zabczyk,StochasticPDE’sandKolmogorovequations ininfinitedimensions,LectureNotesinMathematics,vol.1715,Springer-Verlag,Berlin, 1999. [32] N. V. Krylov and B. L. Rozovskii, Stochastic evolution equations, J. Sov. Math. 16 (1981),no.4,1233–1276. [33] E. B. Lee and L. Markus, Foundations of optimal control theory, Robert E. Krieger PublishingCo.Inc.,1986. [34] B. Øksendal, Optimal control of stochastic partial differential equations, Stoch. Anal. Appl.23(2005),no.1,165–179. [35] A. Osses, Four variations in global Carleman weights applied to inverse and controlla- bility problems,Comput.Appl.Math.25(2006),no.2-3,167–185. [36] S.G.Peng,Ageneralstochasticmaximumprincipleforoptimalcontrolproblems,SIAM J.ControlOptim.28(1990),no.4,966–979. [37] A. D. Polyanin and V. F. Zaitsev, Handbook of exact solutions for ordinary differential equations,seconded.,Chapman&Hall/CRC,BocaRaton,FL,2003. [38] L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mishchenko, The mathematical theory of optimal processes, Interscience Publishers John Wiley & Sons, Inc. NewYork-London,1962. [39] D. L. Russell, A unified boundary controllability theory for hyperbolic and parabolic partial differential equations,StudiesinAppl.Math.52(1973),189–211. [40] Yu.SafarovandD.Vassiliev,Theasymptoticdistributionofeigenvaluesofpartialdiffer- entialoperators,TranslationsofMathematicalMonographs,vol.155,AmericanMathe- maticalSociety,Providence,RI,1997. [41] S.J.Sheu,SomeestimatesofthetransitiondensityofanondegeneratediffusionMarkov process,Ann.Probab.19(1991),no.2,538–561. [42] D. W. Stroock and S. R. S. Varadhan, Multidimensional diffusion processes, Springer, Berlin,1979. 113 [43] B. A. Ton, Exact controllability for a semilinear wave equation with both interior and boundary controls,Abstr.Appl.Anal.(2005),no.6,619–637. [44] , On the exact controlability of a nonlinear stochastic heat equation. II, Stoch. Anal.Appl.23(2005),no.5,1071–1086. [45] , On the exact controllability of the wave equation with interior and boundary controls,J.Optim.TheoryAppl.125(2005),no.1,19–35. [46] , Exact controllability for a nonlinear stochastic wave equation, Abstr. Appl. Anal.(2006),Art.ID74264,14. [47] , On an exact control problem for a semilinear wave equation with an unknown source,J.Math.Anal.Appl.317(2006),no.1,286–301. [48] , On the exact controllability of a nonlinear stochastic heat equation, Abstr. Appl.Anal.(2006),Art.ID61203,12. [49] H.L. Trentelman, A.A. Stoorvogel, and M. Hautus, Control theory for linear systems, Springer-VerlagLondonLtd.,2001. [50] W. M. Wonham, Linear multivariable control: a geometric approach, Springer-Verlag, 1985. [51] Z. Yin, Null exact controllability of the parabolic equations with equivalued surface boundary condition,J.Appl.Math.Stoch.Anal.(2006),Art.ID62694,10. [52] J.YongandX.-Yu.Zhou, Stochasticcontrols: Hamiltoniansystems andHJBequations, ApplicationsofMathematics,vol.43,Springer,1999. [53] L. C. Young, Lectures on the calculus of variations and optimal control theory, W. B. SaundersCo.,1969. [54] X. Zhang, Explicit observability inequalities for the wave equation with lower order terms by means of Carleman inequalities, SIAM J. Control Optim. 39 (2000), no. 3, 812–834(electronic). [55] X. Y. Zhou, A unified treatment of maximum principle and dynamic programming in stochastic controls,StochasticsStochasticsRep.36(1991),no.3-4,137–161. [56] , On the necessary conditions of optimal controls for stochastic partial differen- tial equations,SIAMJ.ControlOptim.31(1993),no.6,1462–1478. [57] , Hamiltonian systems, HJB equations, and stochastic controls, Conference on decisionandcontrol,1997. 114 [58] E. Zuazua, Handbook of differential equations: Evolutionary differential equations, iii, ch. Controllability and observability of partial differential equations: some results and openproblems,pp.527–621,ElsevierScience,2006. 115
Abstract (if available)
Abstract
The objective of optimal control is to optimize (minimize or maximize) a certain performance index. The objective of exact control is to ensure that the solution takes a prescribed value at the prescribed time. This dissertation carries out a comparative analysis of these two control problems and derives several general methods of constricting exact feed-back and open-loop controls for deterministic and stochastic evolution equations. The exact feed-back control is carried out using the new notion of a driving function. Several procedures for the optimal selection of the driving function are discussed. It is proved that the driving function must develop a non-integrable singularity at the terminal time.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Second order in time stochastic evolution equations and Wiener chaos approach
PDF
Equilibrium model of limit order book and optimal execution problem
PDF
Generalized Taylor effect for main financial markets
PDF
Credit risk of a leveraged firm in a controlled optimal stopping framework
PDF
Numerical weak approximation of stochastic differential equations driven by Levy processes
PDF
Tamed and truncated numerical methods for stochastic differential equations
PDF
On spectral approximations of stochastic partial differential equations driven by Poisson noise
PDF
Optimizing statistical decisions by adding noise
PDF
Elements of dynamic programming: theory and application
PDF
Conditional mean-fields stochastic differential equation and their application
PDF
Gaussian free fields and stochastic parabolic equations
PDF
On the non-degenerate parabolic Kolmogorov integro-differential equation and its applications
PDF
Set values for mean field games and set valued PDEs
PDF
Reinforcement learning for the optimal dividend problem
PDF
Controlled McKean-Vlasov equations and related topics
PDF
Optimal clipped linear strategies for controllable damping
PDF
High-frequency Kelly criterion and fat tails: gambling with an edge
PDF
Large deviations rates in a Gaussian setting and related topics
PDF
Optimal decisions under recursive utility
PDF
Optimal dividend and investment problems under Sparre Andersen model
Asset Metadata
Creator
Song, Qian
(author)
Core Title
Optimal and exact control of evolution equations
School
College of Letters, Arts and Sciences
Degree
Doctor of Philosophy
Degree Program
Applied Mathematics
Publication Date
07/29/2010
Defense Date
06/16/2008
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
exact control,OAI-PMH Harvest,optimal control
Language
English
Advisor
Lototsky, Sergey V. (
committee chair
), Deng, Yongheng (
committee member
), Kukavica, Igor (
committee member
), Mikulevicius, Remigijus (
committee member
), Zhang, Jianfeng (
committee member
)
Creator Email
qsong@usc.edu
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-m1443
Unique identifier
UC1262243
Identifier
etd-Song-20080729 (filename),usctheses-m40 (legacy collection record id),usctheses-c127-90974 (legacy record id),usctheses-m1443 (legacy record id)
Legacy Identifier
etd-Song-20080729.pdf
Dmrecord
90974
Document Type
Dissertation
Rights
Song, Qian
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Repository Name
Libraries, University of Southern California
Repository Location
Los Angeles, California
Repository Email
cisadmin@lib.usc.edu
Tags
exact control
optimal control