Kybernetika

This paper deals with Markov Control Processes (MCPs) on Euclidean spaces with an infinite horizon and a discounted total cost. Firstly, MCPs which result from the deterministic controlled systems will be analyzed. For such MCPs, conditions that permit to establish the equation known in the literature of Economy as Euler's Equation (EE) will be given. There will be also presented an example of a Markov Control Process with deterministic controlled system where, to obtain the optimal value function, EE applied to the value iteration algorithm will be used. Secondly, the MCPs which result from the perturbation of deterministic controlled systems with a random noise will be dealt with. There will be also provided the conditions which allow to obtain the optimal value function and the optimal policy of a perturbed controlled system, in terms of the optimal value function and the optimal policy of deterministic controlled system corresponding. Finally, several examples to illustrate the last case mentioned will be presented.