Editing Optimal control (section)

==Examples==
{{unreferenced section|date=April 2018}}
A common solution strategy in many optimal control problems is to solve for the costate (sometimes called the [[shadow price]]) <math>\lambda(t)</math>. The costate summarizes in one number the marginal value of expanding or contracting the state variable next turn. The marginal value is not only the gains accruing to it next turn but associated with the duration of the program. It is nice when <math>\lambda(t)</math> can be solved analytically, but usually, the most one can do is describe it sufficiently well that the intuition can grasp the character of the solution and an equation solver can solve numerically for the values.

Having obtained <math>\lambda(t)</math>, the turn-t optimal value for the control can usually be solved as a differential equation conditional on knowledge of <math>\lambda(t)</math>. Again it is infrequent, especially in continuous-time problems, that one obtains the value of the control or the state explicitly. Usually, the strategy is to solve for thresholds and regions that characterize the optimal control and use a numerical solver to isolate the actual choice values in time.

===Finite time===
{{confusing|section|reason=the law of evolution mentioned in the example is not mentioned in the article and is probably not the same as [[evolution]]|date=October 2018}}
Consider the problem of a mine owner who must decide at what rate to extract ore from their mine. They own rights to the ore from date <math>0</math> to date <math>T</math>. At date <math>0</math> there is <math>x_0</math> ore in the ground, and the time-dependent amount of ore <math>x(t)</math> left in the ground declines at the rate of <math>u(t)</math> that the mine owner extracts it. The mine owner extracts ore at cost <math>u(t)^2/x(t)</math> (the cost of extraction increasing with the square of the extraction speed and the inverse of the amount of ore left) and sells ore at a constant price <math>p</math>. Any ore left in the ground at time <math>T</math> cannot be sold and has no value (there is no "scrap value"). The owner chooses the rate of extraction varying with time <math>u(t)</math> to maximize profits over the period of ownership with no time discounting.

{{ordered list
| 1 = Discrete-time version
{{pb}}
The manager maximizes profit <math>\Pi</math>:
<math display="block">\Pi = \sum_{t=0}^{T-1} \left[ pu_t - \frac{u_t^2}{x_t} \right] </math>
subject to the law of motion for the state variable <math>x_t</math>
<math display="block">x_{t+1} - x_t = - u_t</math>

Form the Hamiltonian and differentiate:
<math display="block">\begin{align}
H &= pu_t - \frac{u_t^2}{x_t} - \lambda_{t+1} u_t \\
\frac{\partial H}{\partial u_t} &= p - \lambda_{t+1} - 2\frac{u_t}{x_t} = 0 \\
\lambda_{t+1} - \lambda_t &= -\frac{\partial H}{\partial x_t} = -\left( \frac{u_t}{x_t} \right)^2
\end{align}</math>

As the mine owner does not value the ore remaining at time <math>T</math>,
<math display="block">\lambda_T = 0</math>

Using the above equations, it is easy to solve for the <math>x_t</math> and <math>\lambda_t</math> series
<math display="block">\begin{align}
\lambda_t &= \lambda_{t+1} + \frac{\left(p-\lambda_{t+1}\right)^2}{4} \\
x_{t+1} &= x_t \frac{2 - p + \lambda_{t+1}}{2}
\end{align}</math>
and using the initial and turn-T conditions, the <math>x_t</math> series can be solved explicitly, giving <math>u_t</math>.

| 2 = Continuous-time version
{{pb}}
The manager maximizes profit <math>\Pi</math>:
<math display="block">\Pi = \int_0^T \left[ pu(t) - \frac{u(t)^2}{x(t)} \right] dt </math>
where the state variable <math>x(t)</math> evolves as follows:
<math display="block"> \dot x(t) = - u(t) </math>

Form the Hamiltonian and differentiate:
<math display="block">\begin{align}
H &= pu(t) - \frac{u(t)^2}{x(t)} - \lambda(t) u(t) \\
\frac{\partial H}{\partial u} &= p - \lambda(t) - 2\frac{u(t)}{x(t)} = 0 \\
\dot\lambda(t) &= -\frac{\partial H}{\partial x} = -\left( \frac{u(t)}{x(t)} \right)^2
\end{align}</math>

As the mine owner does not value the ore remaining at time <math>T</math>,
<math display="block">\lambda(T) = 0</math>

Using the above equations, it is easy to solve for the differential equations governing <math>u(t)</math> and <math>\lambda(t)</math>
<math display="block">\begin{align}
\dot\lambda(t) &= -\frac{(p-\lambda(t))^2}{4} \\
u(t) &= x(t) \frac{p- \lambda(t)}{2}
\end{align}</math>
and using the initial and turn-T conditions, the functions can be solved to yield
<math display="block">x(t) = \frac{\left(4-pt+pT\right)^2}{\left(4+pT\right)^2} x_0  </math>
}}