Editing Bellman equation (section)

=== A dynamic decision problem ===
Let <math>x_t</math> be the state at time <math>t</math>. For a decision that begins at time 0, we take as given the initial state <math>x_0</math>. At any time, the set of possible actions depends on the current state; we express this as <math> a_{t} \in \Gamma (x_t)</math>, where a particular action <math>a_t</math> represents particular values for one or more control variables, and <math>\Gamma (x_t)</math> is the set of actions available to be taken at state <math>x_t</math>. It is also assumed that the state changes from <math>x</math> to a new state <math>T(x,a)</math> when action <math>a</math> is taken, and that the current payoff from taking action <math>a</math> in state <math>x</math> is <math>F(x,a)</math>. Finally, we assume impatience, represented by a [[discount factor]] <math>0<\beta<1</math>.

Under these assumptions, an infinite-horizon decision problem takes the following form:

:<math> V(x_0) \; = \; \max_{ \left \{ a_{t} \right \}_{t=0}^{\infty} }  \sum_{t=0}^{\infty} \beta^t F(x_t,a_{t}), </math>

subject to the constraints

:<math> a_{t} \in \Gamma (x_t), \; x_{t+1}=T(x_t,a_t), \; \forall t = 0, 1, 2, \dots </math>

Notice that we have defined notation <math>V(x_0)</math> to denote the optimal value that can be obtained by maximizing this objective function subject to the assumed constraints. This function is the ''value function''. It is a function of the initial state variable <math>x_0</math>, since the best value obtainable depends on the initial situation.