Optimal Resource Allocation and Policy Formulation in Loosely-Coupled Markov Decision Processes

The problem of optimal policy formulation for teams of resource-limited agents in stochastic environments is composed of two strongly-coupled subproblems: a resource allocation problem and a policy optimization problem. We show how to combine the two problems into a single constrained optimization problem that yields optimal resource allocations and policies that are optimal under these allocations. We model the system as a multiagent Markov decision process (MDP), with social welfare of the group as the optimization criterion. The straightforward approach of modeling both the resource allocation and the actual operation of the agents as a multiagent MDP on the joint state and action spaces of all agents is not feasible, because of the exponential increase in the size of the state space. As an alternative, we describe a technique that exploits problem structure by recognizing that agents are only loosely-coupled via the shared resource constraints. This allows us to formulate a constr...

• Type: Conference
• Year: 2004
• Where: AIPS
• Authors: Dmitri A. Dolgov, Edmund H. Durfee
• Keywords: AIPS 2004, Artificial Intelligence, Constrained Optimization Problem, Optimization Problem, Policy Optimization Problem