Singular Perturbation for the Discounted Continuous Control of Piecewise Deterministic Markov Processes

doi:10.1007/S00245-010-9124-7

Title: Singular Perturbation for the Discounted Continuous Control of Piecewise Deterministic Markov Processes

Full Record
Other Related Research

Abstract

This paper deals with the expected discounted continuous control of piecewise deterministic Markov processes (PDMP's) using a singular perturbation approach for dealing with rapidly oscillating parameters. The state space of the PDMP is written as the product of a finite set and a subset of the Euclidean space Double-Struck-Capital-R {sup n}. The discrete part of the state, called the regime, characterizes the mode of operation of the physical system under consideration, and is supposed to have a fast (associated to a small parameter {epsilon}>0) and a slow behavior. By using a similar approach as developed in Yin and Zhang (Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach, Applications of Mathematics, vol. 37, Springer, New York, 1998, Chaps. 1 and 3) the idea in this paper is to reduce the number of regimes by considering an averaged model in which the regimes within the same class are aggregated through the quasi-stationary distribution so that the different states in this class are replaced by a single one. The main goal is to show that the value function of the control problem for the system driven by the perturbed Markov chain converges to the value function of this limit control problem asmore »« less

Authors:

Costa, O. L. V., E-mail: oswaldo@lac.usp.br ^[1]; Dufour, F., E-mail: dufour@math.u-bordeaux1.fr ^[2]

Escola Politecnica da Universidade de Sao Paulo, Departamento de Engenharia de Telecomunicacoes e Controle (Brazil)
Universite Bordeaux I, IMB, Institut Mathematiques de Bordeaux, INRIA Bordeaux Sud Ouest, Team: CQFD (France)

Publication Date:: Wed Jun 15 00:00:00 EDT 2011

OSTI Identifier:: 22043925

Resource Type:: Journal Article

Resource Relation:: Journal Name: Applied Mathematics and Optimization; Journal Volume: 63; Journal Issue: 3; Other Information: Copyright (c) 2011 Springer Science+Business Media, LLC; Country of input: International Atomic Energy Agency (IAEA)

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICAL METHODS AND COMPUTING; CONTROL; CONVERGENCE; DISTURBANCES; EUCLIDEAN SPACE; MARKOV PROCESS; MATHEMATICS

Citation Formats


                    Costa, O. L. V., E-mail: oswaldo@lac.usp.br, and Dufour, F., E-mail: dufour@math.u-bordeaux1.fr. Singular Perturbation for the Discounted Continuous Control of Piecewise Deterministic Markov Processes.  United States: N. p., 2011. 
        Web.  doi:10.1007/S00245-010-9124-7.

Copy to clipboard


                    Costa, O. L. V., E-mail: oswaldo@lac.usp.br, & Dufour, F., E-mail: dufour@math.u-bordeaux1.fr. Singular Perturbation for the Discounted Continuous Control of Piecewise Deterministic Markov Processes.  United States.  doi:10.1007/S00245-010-9124-7.

Copy to clipboard


                    Costa, O. L. V., E-mail: oswaldo@lac.usp.br, and Dufour, F., E-mail: dufour@math.u-bordeaux1.fr. Wed .  
        "Singular Perturbation for the Discounted Continuous Control of Piecewise Deterministic Markov Processes".  United States.  doi:10.1007/S00245-010-9124-7.

Copy to clipboard


                    
@article{osti_22043925,

  title        = {Singular Perturbation for the Discounted Continuous Control of Piecewise Deterministic Markov Processes},

  author       = {Costa, O. L. V., E-mail: oswaldo@lac.usp.br and Dufour, F., E-mail: dufour@math.u-bordeaux1.fr},

  abstractNote = {This paper deals with the expected discounted continuous control of piecewise deterministic Markov processes (PDMP's) using a singular perturbation approach for dealing with rapidly oscillating parameters. The state space of the PDMP is written as the product of a finite set and a subset of the Euclidean space Double-Struck-Capital-R {sup n}. The discrete part of the state, called the regime, characterizes the mode of operation of the physical system under consideration, and is supposed to have a fast (associated to a small parameter {epsilon}>0) and a slow behavior. By using a similar approach as developed in Yin and Zhang (Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach, Applications of Mathematics, vol. 37, Springer, New York, 1998, Chaps. 1 and 3) the idea in this paper is to reduce the number of regimes by considering an averaged model in which the regimes within the same class are aggregated through the quasi-stationary distribution so that the different states in this class are replaced by a single one. The main goal is to show that the value function of the control problem for the system driven by the perturbed Markov chain converges to the value function of this limit control problem as {epsilon} goes to zero. This convergence is obtained by, roughly speaking, showing that the infimum and supremum limits of the value functions satisfy two optimality inequalities as {epsilon} goes to zero. This enables us to show the result by invoking a uniqueness argument, without needing any kind of Lipschitz continuity condition.},

  doi          = {10.1007/S00245-010-9124-7},

  journal      = {Applied Mathematics and Optimization},

  number       = 3,

  volume       = 63,

  place        = {United States},

  year         = {Wed Jun 15 00:00:00 EDT 2011},

  month        = {Wed Jun 15 00:00:00 EDT 2011}

}

Copy to clipboard

Journal Article:

DOI: 10.1007/S00245-010-9124-7

Other availability

Find in Google Scholar

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Similar records in OSTI.GOV collections:

The Policy Iteration Algorithm for Average Continuous Control of Piecewise Deterministic Markov Processes

Journal Article Costa, O. L. V., E-mail: oswaldo@lac.usp.b ; Dufour, F., E-mail: dufour@math.u-bordeaux1.f - Applied Mathematics and Optimization

The main goal of this paper is to apply the so-called policy iteration algorithm (PIA) for the long run average continuous control problem of piecewise deterministic Markov processes (PDMP's) taking values in a general Borel space and with compact action space depending on the state variable. In order to do that we first derive some important properties for a pseudo-Poisson equation associated to the problem. In the sequence it is shown that the convergence of the PIA to a solution satisfying the optimality equation holds under some classical hypotheses and that this optimal solution yields to an optimal control strategymore »« less
DOI: 10.1007/S00245-010-9099-4
Linearization Techniques for Controlled Piecewise Deterministic Markov Processes; Application to Zubov's Method

Journal Article Goreac, Dan, E-mail: Dan.Goreac@univ-mlv.fr ; Serea, Oana-Silvia - Applied Mathematics and Optimization

We aim at characterizing domains of attraction for controlled piecewise deterministic processes using an occupational measure formulation and Zubov's approach. Firstly, we provide linear programming (primal and dual) formulations of discounted, infinite horizon control problems for PDMPs. These formulations involve an infinite-dimensional set of probability measures and are obtained using viscosity solutions theory. Secondly, these tools allow to construct stabilizing measures and to avoid the assumption of stability under concatenation for controls. The domain of controllability is then characterized as some level set of a convenient solution of the associated Hamilton-Jacobi integral-differential equation. The theoretical results are applied to PDMPsmore »« less
DOI: 10.1007/S00245-012-9169-X
A Piecewise Deterministic Markov Toy Model for Traffic/Maintenance and Associated Hamilton–Jacobi Integrodifferential Systems on Networks

Journal Article Goreac, Dan, E-mail: Dan.Goreac@u-pem.fr ; Kobylanski, Magdalena, E-mail: Magdalena.Kobylanski@u-pem.fr ; Martinez, Miguel, E-mail: Miguel.Martinez@u-pem.fr - Applied Mathematics and Optimization

We study optimal control problems in infinite horizon whxen the dynamics belong to a specific class of piecewise deterministic Markov processes constrained to star-shaped networks (corresponding to a toy traffic model). We adapt the results in Soner (SIAM J Control Optim 24(6):1110–1122, 1986) to prove the regularity of the value function and the dynamic programming principle. Extending the networks and Krylov’s “shaking the coefficients” method, we prove that the value function can be seen as the solution to a linearized optimization problem set on a convenient set of probability measures. The approach relies entirely on viscosity arguments. As a by-product,more »« less
DOI: 10.1007/S00245-015-9319-Z
The Discounted Method and Equivalence of Average Criteria for Risk-Sensitive Markov Decision Processes on Borel Spaces

Journal Article Cavazos-Cadena, Rolando, E-mail: rcavazos@uaaan.m ; Salem-Silva, Francisco, E-mail: frsalem@uv.m - Applied Mathematics and Optimization

This note concerns discrete-time controlled Markov chains with Borel state and action spaces. Given a nonnegative cost function, the performance of a control policy is measured by the superior limit risk-sensitive average criterion associated with a constant and positive risk sensitivity coefficient. Within such a framework, the discounted approach is used (a) to establish the existence of solutions for the corresponding optimality inequality, and (b) to show that, under mild conditions on the cost function, the optimal value functions corresponding to the superior and inferior limit average criteria coincide on a certain subset of the state space. The approach ofmore »« less
DOI: 10.1007/S00245-009-9080-2
Conditions for the Solvability of the Linear Programming Formulation for Constrained Discounted Markov Decision Processes

Journal Article Dufour, F., E-mail: dufour@math.u-bordeaux1.fr ; Prieto-Rumeau, T., E-mail: tprieto@ccia.uned.es - Applied Mathematics and Optimization

We consider a discrete-time constrained discounted Markov decision process (MDP) with Borel state and action spaces, compact action sets, and lower semi-continuous cost functions. We introduce a set of hypotheses related to a positive weight function which allow us to consider cost functions that might not be bounded below by a constant, and which imply the solvability of the linear programming formulation of the constrained MDP. In particular, we establish the existence of a constrained optimal stationary policy. Our results are illustrated with an application to a fishery management problem.
DOI: 10.1007/S00245-015-9307-3

Similar Records