# Impulsive Control for Continuous-Time Markov Decision Processes: A Linear Programming Approach

## Abstract

In this paper, we investigate an optimization problem for continuous-time Markov decision processes with both impulsive and continuous controls. We consider the so-called constrained problem where the objective of the controller is to minimize a total expected discounted optimality criterion associated with a cost rate function while keeping other performance criteria of the same form, but associated with different cost rate functions, below some given bounds. Our model allows multiple impulses at the same time moment. The main objective of this work is to study the associated linear program defined on a space of measures including the occupation measures of the controlled process and to provide sufficient conditions to ensure the existence of an optimal control.

- Authors:

- Bordeaux INP, IMB, UMR CNRS 5251 (France)
- University of Liverpool, Department of Mathematical Sciences (United Kingdom)

- Publication Date:

- OSTI Identifier:
- 22617265

- Resource Type:
- Journal Article

- Resource Relation:
- Journal Name: Applied Mathematics and Optimization; Journal Volume: 74; Journal Issue: 1; Other Information: Copyright (c) 2016 Springer Science+Business Media New York; http://www.springer-ny.com; Country of input: International Atomic Energy Agency (IAEA)

- Country of Publication:
- United States

- Language:
- English

- Subject:
- 97 MATHEMATICAL METHODS AND COMPUTING; LINEAR PROGRAMMING; MARKOV PROCESS; OCCUPATIONS; OPTIMAL CONTROL; OPTIMIZATION; PULSES

### Citation Formats

```
Dufour, F., E-mail: dufour@math.u-bordeaux1.fr, and Piunovskiy, A. B., E-mail: piunov@liv.ac.uk.
```*Impulsive Control for Continuous-Time Markov Decision Processes: A Linear Programming Approach*. United States: N. p., 2016.
Web. doi:10.1007/S00245-015-9310-8.

```
Dufour, F., E-mail: dufour@math.u-bordeaux1.fr, & Piunovskiy, A. B., E-mail: piunov@liv.ac.uk.
```*Impulsive Control for Continuous-Time Markov Decision Processes: A Linear Programming Approach*. United States. doi:10.1007/S00245-015-9310-8.

```
Dufour, F., E-mail: dufour@math.u-bordeaux1.fr, and Piunovskiy, A. B., E-mail: piunov@liv.ac.uk. Mon .
"Impulsive Control for Continuous-Time Markov Decision Processes: A Linear Programming Approach". United States.
doi:10.1007/S00245-015-9310-8.
```

```
@article{osti_22617265,
```

title = {Impulsive Control for Continuous-Time Markov Decision Processes: A Linear Programming Approach},

author = {Dufour, F., E-mail: dufour@math.u-bordeaux1.fr and Piunovskiy, A. B., E-mail: piunov@liv.ac.uk},

abstractNote = {In this paper, we investigate an optimization problem for continuous-time Markov decision processes with both impulsive and continuous controls. We consider the so-called constrained problem where the objective of the controller is to minimize a total expected discounted optimality criterion associated with a cost rate function while keeping other performance criteria of the same form, but associated with different cost rate functions, below some given bounds. Our model allows multiple impulses at the same time moment. The main objective of this work is to study the associated linear program defined on a space of measures including the occupation measures of the controlled process and to provide sufficient conditions to ensure the existence of an optimal control.},

doi = {10.1007/S00245-015-9310-8},

journal = {Applied Mathematics and Optimization},

number = 1,

volume = 74,

place = {United States},

year = {Mon Aug 15 00:00:00 EDT 2016},

month = {Mon Aug 15 00:00:00 EDT 2016}

}