Nearly optimal control of singularly perturbed Markov decision processes in discrete time

Journal Article · Sun Jul 01 00:00:00 EDT 2001 · Applied Mathematics and Optimization

DOI:https://doi.org/10.1007/S00245-001-0016-8· OSTI ID:21067503

Liu, R H; edu, qingz@math uga

This work develops asymptotically optimal controls for discrete-time singularly perturbed Markov decision processes (MDPs) having weak and strong interactions. The focus is on finite-state-space-MDP problems. The state space of the underlying Markov chain can be decomposed into a number of recurrent classes or a number of recurrent classes and a group of transient states. Using a hierarchical control approach, continuous-time limit problems that are much simpler to handle than the original ones are derived. Based on the optimal solutions for the limit problems, nearly optimal decisions for the original problems are obtained. The asymptotic optimality of such controls is proved and the rate of convergence is provided. Infinite horizon problems are considered; both discounted costs and long-run average costs are examined.

• OSTI ID: 21067503
• Journal Information: Applied Mathematics and Optimization, Vol. 44, Issue 2; Other Information: DOI: 10.1007/s00245-001-0016-8; Copyright (c) Inc. 2001 Springer-Verlag New York; Country of input: International Atomic Energy Agency (IAEA); ISSN 0095-4616
• Country of Publication: United States
• Language: English

Related Subjects

71 CLASSICAL AND QUANTUM MECHANICS
GENERAL PHYSICS
ASYMPTOTIC SOLUTIONS
CONTROL THEORY
CONVERGENCE
MARKOV PROCESS
MATHEMATICAL SPACE
OPTIMAL CONTROL