Solving decentralized POMDP problems using genetic algorithms

摘要

The Decentralized Partially Observable Markov Decision Process (DEC-POMDP) model addresses the multiagent planning problem in partially observable environments. Due to its high computational complexity, in general only very small size problems can be solved exactly and most researchers concentrate on approximate solution algorithms to handle more complex cases. However, many approximate solution techniques can handle large size problems only for small horizons due to their exponential memory requirements for representing the policies and searching the policy space. In this study, we offer an approximate solution algorithm called GA-FSC that uses finite state controllers (FSC) to represent a finite-horizon DEC-POMDP policy and searches the policy space using genetic algorithms. We encode FSCs into chromosomes and we use one exact and one approximate technique to calculate the fitness of the chromosomes. The exact calculation technique helps us to obtain better quality solutions with the cost of more processing time compared to the approximate fitness calculation. Our method is able to replicate the best results reported so far in the literature in most cases and it is also able to extend the reported horizons further in almost all cases when compared to optimal approaches.