skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Singular Perturbation for the Discounted Continuous Control of Piecewise Deterministic Markov Processes

Abstract

This paper deals with the expected discounted continuous control of piecewise deterministic Markov processes (PDMP's) using a singular perturbation approach for dealing with rapidly oscillating parameters. The state space of the PDMP is written as the product of a finite set and a subset of the Euclidean space Double-Struck-Capital-R {sup n}. The discrete part of the state, called the regime, characterizes the mode of operation of the physical system under consideration, and is supposed to have a fast (associated to a small parameter {epsilon}>0) and a slow behavior. By using a similar approach as developed in Yin and Zhang (Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach, Applications of Mathematics, vol. 37, Springer, New York, 1998, Chaps. 1 and 3) the idea in this paper is to reduce the number of regimes by considering an averaged model in which the regimes within the same class are aggregated through the quasi-stationary distribution so that the different states in this class are replaced by a single one. The main goal is to show that the value function of the control problem for the system driven by the perturbed Markov chain converges to the value function of this limit control problem asmore » {epsilon} goes to zero. This convergence is obtained by, roughly speaking, showing that the infimum and supremum limits of the value functions satisfy two optimality inequalities as {epsilon} goes to zero. This enables us to show the result by invoking a uniqueness argument, without needing any kind of Lipschitz continuity condition.« less

Authors:
 [1];  [2]
  1. Escola Politecnica da Universidade de Sao Paulo, Departamento de Engenharia de Telecomunicacoes e Controle (Brazil)
  2. Universite Bordeaux I, IMB, Institut Mathematiques de Bordeaux, INRIA Bordeaux Sud Ouest, Team: CQFD (France)
Publication Date:
OSTI Identifier:
22043925
Resource Type:
Journal Article
Resource Relation:
Journal Name: Applied Mathematics and Optimization; Journal Volume: 63; Journal Issue: 3; Other Information: Copyright (c) 2011 Springer Science+Business Media, LLC; Country of input: International Atomic Energy Agency (IAEA)
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICAL METHODS AND COMPUTING; CONTROL; CONVERGENCE; DISTURBANCES; EUCLIDEAN SPACE; MARKOV PROCESS; MATHEMATICS

Citation Formats

Costa, O. L. V., E-mail: oswaldo@lac.usp.br, and Dufour, F., E-mail: dufour@math.u-bordeaux1.fr. Singular Perturbation for the Discounted Continuous Control of Piecewise Deterministic Markov Processes. United States: N. p., 2011. Web. doi:10.1007/S00245-010-9124-7.
Costa, O. L. V., E-mail: oswaldo@lac.usp.br, & Dufour, F., E-mail: dufour@math.u-bordeaux1.fr. Singular Perturbation for the Discounted Continuous Control of Piecewise Deterministic Markov Processes. United States. doi:10.1007/S00245-010-9124-7.
Costa, O. L. V., E-mail: oswaldo@lac.usp.br, and Dufour, F., E-mail: dufour@math.u-bordeaux1.fr. Wed . "Singular Perturbation for the Discounted Continuous Control of Piecewise Deterministic Markov Processes". United States. doi:10.1007/S00245-010-9124-7.
@article{osti_22043925,
title = {Singular Perturbation for the Discounted Continuous Control of Piecewise Deterministic Markov Processes},
author = {Costa, O. L. V., E-mail: oswaldo@lac.usp.br and Dufour, F., E-mail: dufour@math.u-bordeaux1.fr},
abstractNote = {This paper deals with the expected discounted continuous control of piecewise deterministic Markov processes (PDMP's) using a singular perturbation approach for dealing with rapidly oscillating parameters. The state space of the PDMP is written as the product of a finite set and a subset of the Euclidean space Double-Struck-Capital-R {sup n}. The discrete part of the state, called the regime, characterizes the mode of operation of the physical system under consideration, and is supposed to have a fast (associated to a small parameter {epsilon}>0) and a slow behavior. By using a similar approach as developed in Yin and Zhang (Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach, Applications of Mathematics, vol. 37, Springer, New York, 1998, Chaps. 1 and 3) the idea in this paper is to reduce the number of regimes by considering an averaged model in which the regimes within the same class are aggregated through the quasi-stationary distribution so that the different states in this class are replaced by a single one. The main goal is to show that the value function of the control problem for the system driven by the perturbed Markov chain converges to the value function of this limit control problem as {epsilon} goes to zero. This convergence is obtained by, roughly speaking, showing that the infimum and supremum limits of the value functions satisfy two optimality inequalities as {epsilon} goes to zero. This enables us to show the result by invoking a uniqueness argument, without needing any kind of Lipschitz continuity condition.},
doi = {10.1007/S00245-010-9124-7},
journal = {Applied Mathematics and Optimization},
number = 3,
volume = 63,
place = {United States},
year = {Wed Jun 15 00:00:00 EDT 2011},
month = {Wed Jun 15 00:00:00 EDT 2011}
}