Editing Optimal control (section)

==Linear quadratic control==
A special case of the general nonlinear optimal control problem given in the previous section is the [[Linear-quadratic regulator|''linear quadratic'' (LQ) optimal control problem]].  The LQ problem is stated as follows.  Minimize the ''quadratic'' continuous-time cost functional
<math display="block">J=\tfrac{1}{2} \mathbf{x}^{\mathsf{T}}(t_f)\mathbf{S}_f\mathbf{x}(t_f) + \tfrac{1}{2} \int_{t_0}^{t_f} [\,\mathbf{x}^{\mathsf{T}}(t)\mathbf{Q}(t)\mathbf{x}(t) + \mathbf{u}^{\mathsf{T}}(t)\mathbf{R}(t) \mathbf{u}(t)]\, \mathrm dt</math>

Subject to the ''linear'' first-order dynamic constraints
<math display="block">\dot{\mathbf{x}}(t)= \mathbf{A}(t) \mathbf{x}(t) + \mathbf{B}(t) \mathbf{u}(t), </math>
and the initial condition
<math display="block"> \mathbf{x}(t_0) = \mathbf{x}_0</math>

A particular form of the LQ problem that arises in many control system problems is that of the ''linear quadratic regulator'' (LQR) where all of the matrices (i.e., <math>\mathbf{A}</math>, <math>\mathbf{B}</math>, <math>\mathbf{Q}</math>, and <math>\mathbf{R}</math>) are ''constant'', the initial time is arbitrarily set to zero, and the terminal time is taken in the limit <math>t_f\rightarrow\infty</math> (this last assumption is what is known as ''infinite horizon'').  The LQR problem is stated as follows.  Minimize the infinite horizon quadratic continuous-time cost functional
<math display="block">J= \tfrac{1}{2} \int_{0}^{\infty}[\mathbf{x}^{\mathsf{T}}(t)\mathbf{Q}\mathbf{x}(t) + \mathbf{u}^{\mathsf{T}}(t)\mathbf{R}\mathbf{u}(t)]\, \mathrm dt</math>

Subject to the ''linear time-invariant'' first-order dynamic constraints
<math display="block">\dot{\mathbf{x}}(t) = \mathbf{A} \mathbf{x}(t) + \mathbf{B} \mathbf{u}(t), </math>
and the initial condition
<math display="block"> \mathbf{x}(t_0) = \mathbf{x}_0</math>

In the finite-horizon case the matrices are restricted in that <math>\mathbf{Q}</math> and <math>\mathbf{R}</math> are positive semi-definite and positive definite, respectively.  In the infinite-horizon case, however, the [[matrix (mathematics)|matrices]] <math>\mathbf{Q}</math> and <math>\mathbf{R}</math> are not only positive-semidefinite and positive-definite, respectively, but are also ''constant''.  These additional restrictions on
<math>\mathbf{Q}</math> and <math>\mathbf{R}</math> in the infinite-horizon case are enforced to ensure that the cost functional remains positive.  Furthermore, in order to ensure that the cost function is ''bounded'', the additional restriction is imposed that the pair <math>(\mathbf{A},\mathbf{B})</math> is ''[[Controllability|controllable]]''.  Note that the LQ or LQR cost functional can be thought of physically as attempting to minimize the ''control energy'' (measured as a quadratic form).

The infinite horizon problem (i.e., LQR) may seem overly restrictive and essentially useless because it assumes that the operator is driving the system to zero-state and hence driving the output of the system to zero. This is indeed correct. However the problem of driving the output to a desired nonzero level can be solved ''after'' the zero output one is. In fact, it can be proved that this secondary LQR problem can be solved in a very straightforward manner.  It has been shown in classical optimal control theory that the LQ (or LQR) optimal control has the feedback form
<math display="block">\mathbf{u}(t) = -\mathbf{K}(t)\mathbf{x}(t)</math>
where <math>\mathbf{K}(t)</math> is a properly dimensioned matrix, given as
<math display="block">\mathbf{K}(t) = \mathbf{R}^{-1}\mathbf{B}^{\mathsf{T}}\mathbf{S}(t),</math>
and <math>\mathbf{S}(t)</math> is the solution of the differential [[Riccati equation]].  The differential Riccati equation is given as
<math display="block">\dot{\mathbf{S}}(t) = -\mathbf{S}(t)\mathbf{A}-\mathbf{A}^{\mathsf{T}} \mathbf{S}(t) +\mathbf{S}(t)\mathbf{B}\mathbf{R}^{-1}\mathbf{B}^{\mathsf{T}}\mathbf{S}(t) - \mathbf{Q}</math>

For the finite horizon LQ problem, the Riccati equation is integrated backward in time using the terminal boundary condition
<math display="block">\mathbf{S}(t_f) = \mathbf{S}_f</math>

For the infinite horizon LQR problem, the differential Riccati equation is replaced with the ''algebraic'' Riccati equation (ARE) given as
<math display="block">\mathbf{0} = -\mathbf{S}\mathbf{A}-\mathbf{A}^{\mathsf{T}}\mathbf{S}+\mathbf{S}\mathbf{B}\mathbf{R}^{-1}\mathbf{B}^{\mathsf{T}}\mathbf{S}-\mathbf{Q}</math>

Understanding that the ARE arises from infinite horizon problem, the matrices <math>\mathbf{A}</math>, <math>\mathbf{B}</math>, <math>\mathbf{Q}</math>, and <math>\mathbf{R}</math> are all ''constant''.  It is noted that there are in general multiple solutions to the algebraic Riccati equation and the ''positive definite'' (or positive semi-definite) solution is the one that is used to compute the feedback gain.  The LQ (LQR) problem was elegantly solved by [[Rudolf E. Kálmán]].<ref>Kalman, Rudolf. ''A new approach to linear filtering and prediction problems''. Transactions of the ASME, Journal of Basic Engineering, 82:34–45, 1960</ref>