Editing Markov decision process (section)

====Continuous space: Hamilton–Jacobi–Bellman equation====
In continuous-time MDP, if the state space and action space are continuous, the optimal criterion could be found by solving [[Hamilton–Jacobi–Bellman equation|Hamilton–Jacobi–Bellman (HJB) partial differential equation]]. In order to discuss the HJB equation, we need to reformulate
our problem
:<math>\begin{align} V(s(0),0)= {} & \max_{a(t)=\pi(s(t))}\int_0^T r(s(t),a(t)) \, dt+D[s(T)] \\
\text{s.t.}\quad & \frac{d s(t)}{dt}=f[t,s(t),a(t)]
\end{align}
</math>

<math>D(\cdot)</math> is the terminal reward function, <math>s(t)</math> is the system state vector, <math>a(t)</math> is the system control vector we try to find. <math>f(\cdot)</math> shows how the state vector changes over time. The Hamilton–Jacobi–Bellman equation is as follows:
:<math>0=\max_u ( r(t,s,a) +\frac{\partial V(t,s)}{\partial x}f(t,s,a)) </math>
We could solve the equation to find the optimal control <math>a(t)</math>, which could give us the optimal [[value function]] <math>V^*</math>