Singular Perturbation for the Discounted Continuous Control of Piecewise Deterministic Markov Processes
- Escola Politecnica da Universidade de Sao Paulo, Departamento de Engenharia de Telecomunicacoes e Controle (Brazil)
- Universite Bordeaux I, IMB, Institut Mathematiques de Bordeaux, INRIA Bordeaux Sud Ouest, Team: CQFD (France)
This paper deals with the expected discounted continuous control of piecewise deterministic Markov processes (PDMP's) using a singular perturbation approach for dealing with rapidly oscillating parameters. The state space of the PDMP is written as the product of a finite set and a subset of the Euclidean space Double-Struck-Capital-R {sup n}. The discrete part of the state, called the regime, characterizes the mode of operation of the physical system under consideration, and is supposed to have a fast (associated to a small parameter {epsilon}>0) and a slow behavior. By using a similar approach as developed in Yin and Zhang (Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach, Applications of Mathematics, vol. 37, Springer, New York, 1998, Chaps. 1 and 3) the idea in this paper is to reduce the number of regimes by considering an averaged model in which the regimes within the same class are aggregated through the quasi-stationary distribution so that the different states in this class are replaced by a single one. The main goal is to show that the value function of the control problem for the system driven by the perturbed Markov chain converges to the value function of this limit control problem as {epsilon} goes to zero. This convergence is obtained by, roughly speaking, showing that the infimum and supremum limits of the value functions satisfy two optimality inequalities as {epsilon} goes to zero. This enables us to show the result by invoking a uniqueness argument, without needing any kind of Lipschitz continuity condition.
- OSTI ID:
- 22043925
- Journal Information:
- Applied Mathematics and Optimization, Vol. 63, Issue 3; Other Information: Copyright (c) 2011 Springer Science+Business Media, LLC; Country of input: International Atomic Energy Agency (IAEA); ISSN 0095-4616
- Country of Publication:
- United States
- Language:
- English
Similar Records
A Piecewise Deterministic Markov Toy Model for Traffic/Maintenance and Associated Hamilton–Jacobi Integrodifferential Systems on Networks
Efficient analysis of stochastic gene dynamics in the non-adiabatic regime using piecewise deterministic Markov processes