Editing Bellman equation (section)

== Solution methods ==
* The [[method of undetermined coefficients]], also known as 'guess and verify', can be used to solve some infinite-horizon, [[Autonomous system (mathematics)|autonomous]] Bellman equations.<ref>{{cite book |first1=Lars |last1=Ljungqvist |first2=Thomas J. |last2=Sargent |title=Recursive Macroeconomic Theory |publisher=MIT Press |year=2004 |edition=2nd |pages=[https://archive.org/details/recursivemacroec02edljun/page/88 88]–90 |isbn=0-262-12274-X |url=https://archive.org/details/recursivemacroec02edljun |url-access=registration }}</ref>
* The Bellman equation can be solved by [[backwards induction]], either [[Closed-form expression|analytically]] in a few special cases, or [[numerical analysis|numerically]] on a computer. Numerical backwards induction is applicable to a wide variety of problems, but may be infeasible when there are many state variables, due to the [[curse of dimensionality]]. Approximate dynamic programming has been introduced by [[Dimitri Bertsekas|D. P. Bertsekas]] and [[John Tsitsiklis|J. N. Tsitsiklis]] with the use of [[artificial neural network]]s ([[multilayer perceptron]]s) for approximating the Bellman function.<ref name="NeuroDynProg">{{cite book |first1=Dimitri P. |last1=Bertsekas |first2=John N. |last2=Tsitsiklis |title=Neuro-dynamic Programming |year=1996 |publisher=Athena Scientific |isbn=978-1-886529-10-6}}</ref> This is an effective mitigation strategy for reducing the impact of dimensionality by replacing the memorization of the complete function mapping for the whole space domain with the memorization of the sole neural network parameters. In particular, for continuous-time systems, an approximate dynamic programming approach that combines both policy iterations with neural networks was introduced.<ref name="CTHJB">{{cite journal |first1=Murad |last1=Abu-Khalaf |first2=Frank L.|last2=Lewis |title=Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach|year=2005 |journal=Automatica |volume=41 | issue=5 | pages=779–791|doi=10.1016/j.automatica.2004.11.034|s2cid=14757582 }}</ref> In discrete-time, an approach to solve the HJB equation combining value iterations and neural networks was introduced.<ref name="DTHJB">{{cite journal |first1=Asma |last1=Al-Tamimi|first2=Frank L.|last2=Lewis |first3=Murad |last3=Abu-Khalaf |title=Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof|year=2008 |journal=IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics |volume= 38| issue=4 | pages=943–949 |doi= 10.1109/TSMCB.2008.926614|pmid=18632382|s2cid=14202785}}</ref>
* By calculating the first-order conditions associated with the Bellman equation, and then using the [[envelope theorem]] to eliminate the derivatives of the value function, it is possible to obtain a system of [[difference equation]]s or [[differential equation]]s called the '[[Euler–Lagrange equation|Euler equation]]s'.<ref>{{cite book |first=Jianjun |last=Miao |title=Economic Dynamics in Discrete Time |publisher=MIT Press |year=2014 |page=134 |isbn=978-0-262-32560-8 |url=https://books.google.com/books?id=dh2EBAAAQBAJ&pg=PA134 }}</ref> Standard techniques for the solution of difference or differential equations can then be used to calculate the dynamics of the state variables and the control variables of the optimization problem.