 
Summary: Fuzzy Markov Chains
Kostya E. Avrachenkov \Lambda Elie Sanchez y
Abstract
General ønite state fuzzy Markov chains that are introduced have a ønite
convergence to a stationary (may be periodic) solution. The Cesaro average and
the ffpotential for fuzzy Markov chains are deøned, then it is shown that the
relationship between them corresponds to the Blackwell formula in the classical
theory of Markov decision processes. Furthermore, it is pointed out that recur
rency does not necessarily imply ergodicity. However, if a fuzzy Markov chain is
ergodic, then the rows of its ergodic projection equal the greatest eigen fuzzy set
of the transition matrix. Finally, the fuzzy Markov chain is shown to be a robust
system with respect to small perturbations of the transition matrix, which is not
the case for the classical probabilistic Markov chains.
Keywords: Fuzzy Markov chains, eigen fuzzy sets, ergodicity, robustness.
1 Introduction and basic deønitions
In a seminal paper, Bellman and Zadeh [3] ørst studied a fuzzy decision process. Esog
bue and Bellman [7] explored various kinds of fuzzy dynamic programming with ønite
state spaces and ønite action spaces. Fuzzy dynamic programming is dioeerent from
classical dynamic programming, it uses fuzzy sets instead of reward functions. In [11]
Kruse et al introduced fuzzy Markov chains as ja perception of usual Markov chainsj,
