Rational Swarms

Individual decisions in cooperative swarms: a puzzle

The key characteristic of swarms is locality of perception: The individual robot can only sense in limited range, and interact with a few others around it. In cooperative swarms, where all the members of the swarm share a joint goal, this raises an interesting puzzle:

How can the single robot choose actions that help the whole swarm, if it cannot know how these actions affect the entire swarm?

We use distributed, multi-agent reinforcement learning (MARL), focusing on algorithms and reward functions that are grounded in the individual robot’s perception, and are aligned with the goals of the swarm, using game theory. The reward functions we develop do not use any public signals or information that would not be available to a physical robot. In MARL terms, we are dealing with Independent Learners.

The following paper, published in the Royal Society Transactions best summarizes the approach and results up to 2025:

Gal A. Kaminka, Swarms can be rational. In Philosophical Transactions of the Royal Society A (2025)

However, this is on-going research: we are looking for students and post-docs

Distributed MARL leads to Heterogeneous Swarms

We have shown that each individual robot can measure its own time and resources spent on coordination, as opposed to working on the task itself. This investment in coordination overhead results from the coordination algorithm taken, and is minimized by the RL process¹. However, this can also cause robots to take actions that help themselves, but hurt the swarm (think of the Prisoner’s Dilemma game). We therefore investigate methods for aligning the individual reward with that of the collective².

Different robots learn to respond differently to collisions and inter-agent conflicts that require coordination. They become heterogeneous in their decision-making. We have experimented and witnessed this in many environments, with both physical robots and simulated robots (see environments gallery page for videos and media). Please see the papers for details¹,². You may also be interested to learn about how the rational swarm model was developed and evolved through the years.