Issue |
RAIRO-Oper. Res.
Volume 53, Number 5, November-December 2019
|
|
---|---|---|
Page(s) | 1749 - 1761 | |
DOI | https://doi.org/10.1051/ro/2018081 | |
Published online | 09 October 2019 |
A linear programming based approach for composite-action Markov decision processes
Department of Industrial Engineering, Dongguan University of Technology, 523808 Dongguan, PR China
* Corresponding author: stephen1998@gmail.com
Received:
12
December
2017
Accepted:
19
September
2018
We study a time homogeneous discrete composite-action Markov decision process (CMDP) which needs to make multiple decisions at each state. In this particular Markov decision process, the state variables are divided into two separable sets and a two-dimensional composite action is chosen at each decision epoch. To solve a composite-action Markov decision process, we propose a novel linear programming model (Contracted Linear Programming Model, CLPM). We show that the CLPM model obtains the optimal state values of a CMDP process. We analyze and compare the number of variables and constraints of the CLPM model and the Traditional Linear Programming Model (TLPM). Computational experiments compare running times and memory usage of the two models. The CLPM model outperforms the TLPM model in both time complexity and space complexity by theoretical analysis and computational experiments.
Mathematics Subject Classification: 90C40 / 90C05
Key words: Markov decision process / linear programming / optimal state value / action
© EDP Sciences, ROADEF, SMAI 2019
Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.