Singular Perturbation for the Discounted Continuous Control of Piecewise Deterministic Markov Processes

Journal Article · Wed Jun 15 04:00:00 EDT 2011 · Applied Mathematics and Optimization

DOI:https://doi.org/10.1007/S00245-010-9124-7· OSTI ID:22043925

Costa, O L. V., ^[1]; Dufour, F ^[2]

---

1. Escola Politecnica da Universidade de Sao Paulo, Departamento de Engenharia de Telecomunicacoes e Controle (Brazil)
2. Universite Bordeaux I, IMB, Institut Mathematiques de Bordeaux, INRIA Bordeaux Sud Ouest, Team: CQFD (France)

This paper deals with the expected discounted continuous control of piecewise deterministic Markov processes (PDMP's) using a singular perturbation approach for dealing with rapidly oscillating parameters. The state space of the PDMP is written as the product of a finite set and a subset of the Euclidean space Double-Struck-Capital-R {sup n}. The discrete part of the state, called the regime, characterizes the mode of operation of the physical system under consideration, and is supposed to have a fast (associated to a small parameter {epsilon}>0) and a slow behavior. By using a similar approach as developed in Yin and Zhang (Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach, Applications of Mathematics, vol. 37, Springer, New York, 1998, Chaps. 1 and 3) the idea in this paper is to reduce the number of regimes by considering an averaged model in which the regimes within the same class are aggregated through the quasi-stationary distribution so that the different states in this class are replaced by a single one. The main goal is to show that the value function of the control problem for the system driven by the perturbed Markov chain converges to the value function of this limit control problem as {epsilon} goes to zero. This convergence is obtained by, roughly speaking, showing that the infimum and supremum limits of the value functions satisfy two optimality inequalities as {epsilon} goes to zero. This enables us to show the result by invoking a uniqueness argument, without needing any kind of Lipschitz continuity condition.

• OSTI ID: 22043925
• Journal Information: Applied Mathematics and Optimization, Journal Name: Applied Mathematics and Optimization Journal Issue: 3 Vol. 63; ISSN 0095-4616
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING
CONTROL
CONVERGENCE
DISTURBANCES
EUCLIDEAN SPACE
MARKOV PROCESS
MATHEMATICS