Rational Swarms

Individual decisions in cooperative swarms: a puzzle

The key characteristic of swarms is locality of perception: The individual robot can only sense in limited range, and interact with a few others around it.

In cooperative swarms, where all the members of the swarm share a joint goal, this raises an interesting puzzle: How can the single robot choose actions that help the whole swarm, if it cannot know how these actions affect the entire swarm? Obviously, the robot will select what it thinks is the optimal action (it is individually rational), but how can it determine what this action is?

We use distributed, multi-agent reinforcement learning (MARL), focusing on algorithms and reward functions that are grounded in the individual robot’s perception, and are aligned with the goals of the swarm, using game theory. The reward functions we develop do not use any public signals or information that would not be available to a physical robot.

This is on-going research: we are looking for students and post-docs

Distributed MARL leads to Heterogeneous Swarms

We have shown that each individual robot can measure its own time and resources spent on coordination, as opposed to working on the task itself. This investment in coordination overhead results from the coordination algorithm taken, and is minimized by the RL process¹. However, this can also cause robots to take actions that help themselves, but hurt the swarm (think of the Prisoner’s Dilemma game). We therefore investigate methods for aligning the individual reward with that of the collective².

Different robots learn to respond differently to collisions and inter-agent conflicts that require coordination. They become heterogeneous in their decision-making. We have experimented and witnessed this in many environments, with both physical robots and simulated robots (see environments gallery page for videos and media). Please see the papers for details¹,². You may also be interested to learn about how the rational swarm model was developed and evolved through the years.

Video Introduction to Swarm Model (2020)

Kaminka’s invited talk on heterogeneous swarms, at the RSS Workshop on Heterogeneous Multi-Robot Task Allocation and Coordination gives an overview of the swarm model and its characteristics.