Computing optimal policies for partially observable decision processes using compact representations

Conference · Tue Dec 31 04:00:00 EST 1996

OSTI ID:430799

Poole, D ^[1]

---

1. Univ. of British Columbia, Vancouver, BC (Canada)

Partially-observable Markov decision processes provide a general model for decision theoretic planning problems, allowing trade-offs between various courses of actions to be determined under conditions of uncertainty, and incorporating partial observations made by an agent. Dynamic programming algorithms based on the belief state of an agent can be used to construct optimal policies without explicit consideration of past history, but at high computational cost. In this paper, we discuss how structured representations of system dynamics can be incorporated in classic POMDP solution algorithms. We use Bayesian networks with structured conditional probability matrices to represent POMDPs, and use this model to structure the belief space for POMDP algorithms, allowing irrelevant distinctions to be ignored. Apart from speeding up optimal policy construction, we suggest that such representations can be exploited in the development of useful approximation methods.

• OSTI ID: 430799
• Report Number(s): CONF-960876--; CNN: Grant OGP0121843; Grant OGPOO44121
• Country of Publication: United States
• Language: English

Related Subjects

99 GENERAL AND MISCELLANEOUS
ALGORITHMS
ARTIFICIAL INTELLIGENCE
DECISION MAKING
DECISION TREE ANALYSIS
DYNAMIC PROGRAMMING
MARKOV PROCESS
PLANNING