Optimal control has the property that whatever the initial state at any step and controls selected in this step, the next administration should be chosen with respect to the optimal condition to which the system will come at the end of this step.
A similar problem was solved in the classical optimization analysis using Lagrange multipliers. In applied problems classical Lagrangian method is generally applicable for many reasons, and above all - because of the dimension. Search absolute maximum of n variables, even if the function is differentiable, it time-consuming. If, moreover, we consider that the extremum can be achieved on the boundary, to the study of stationary points within the area added study of stationary points on the boundary. In practice, one variable x can take discrete values (for example, funds are provided in a multiple of 10.), And the return function fh (Xh) may not be differentiable, or even specifying the table. In all these cases, the classical optimization methods are inapplicable. Solution of the problem you can apply methods of nonlinear programming.
This principle ensures that the controls selected at any step, is not the best locally, and the best in terms of the overall process.
So, if the system is in the early k-step what is in and we choose an arbitrary control of the AI, the system will come to a new state lk = F (lk-h s), and further control uk + u -> dn should be chosen with respect to the optimal state The latter means that these controls are maximized performance indicator in the subsequent steps of the process before the end of k + l.