Bayesian learning for multi-agent coordination

Allen-Williams, M. (2009) Bayesian learning for multi-agent coordination. PhD thesis, University of Southampton.

Download

Preview

Accepted Version
PDF (PhD thesis: Bayesian learning for multi-agent systems)
1805Kb

Abstract

Multi-agent systems draw together a number of significant trends in
modern technology: ubiquity, decentralisation, openness, dynamism and
uncertainty. As work in these fields develops, such systems face
increasing challenges. Two particular challenges are decision making
in uncertain and partially-observable environments, and coordination
with other agents in such environments. Although uncertainty and
coordination have been tackled as separate problems, formal models for
an integrated approach are typically restricted to simple classes of
problem and are not scalable to problems with tens of agents and
millions of states.

We improve on these approaches by extending a principled Bayesian
model into more challenging domains, using Bayesian networks to
visualise specific cases of the model and thus as an aid in deriving
the update equations for the system. One approach which has been shown
to scale well for networked offline problems uses finite state
machines to model other agents. We used this insight to develop an
approximate scalable algorithm applicable to our general model, in
combination with adapting a number of existing approximation
techniques, including state clustering.

We examine the performance of this approximate algorithm on several
cases of an urban rescue problem with respect to differing problem
parameters. Specifically, we consider first scenarios where agents are
aware of the complete situation, but are not certain about the
behaviour of others; that is, our model with all elements but the
actions observable. Secondly, we examine the more complex case where
agents can see the actions of others, but cannot see the full state
and thus are not sure about the beliefs of others. Finally, we
look at the performance of the partially observable state model when
the system is dynamic or open. We find that our best response
algorithm consistently outperforms a handwritten strategy for the
problem, more noticeably as the number of agents and the number of
states involved in the problem increase.