Diversity-aware Crowd Model for Robust Robot Navigation in Human Populated Environment

IEEE Robotics and Automation Letters (RAL)

Jiaxu Wu¹, Yusheng Wang¹, Chen Tong¹, Jun Jiang², Yongdong Wang¹, Qi An¹, Atsushi Yamashita¹

¹The University of Tokyo, ²Woven by Toyota

Abstract

Robot navigation in human-populated environments poses challenges due to the diversity of human behaviors and the unpredictability of human paths. However, existing Reinforcement Learning (RL)-based methods often rely on simulators that lack sufficient diversity in human behavior, resulting in navigation policies that overfit specific human behavior and perform poorly in unseen environments.

To address this, we propose a diversity-aware crowd model based on RL, employing Constrained Variational Exploration (VE) with a Mutual Information (MI)-based auxiliary reward to capture fine-grained behavioral diversity. The proposed model leverages a Centralized Training Decentralized Execution (CTDE) paradigm, which ensures stable exploration under multi-agent settings.

Using the proposed diversity-aware model for training, we obtain robust robot navigation policies capable of handling diverse unseen scenarios. Extensive simulation and real-world experiments demonstrate the superior performance of our approach in achieving diverse crowd behaviors and enhancing robot navigation robustness. These findings highlight the potential of our method to advance safe and efficient robot operations in complex dynamic environments.

Diversity-aware
Crowd Model

Constrained
Variational
Exploration

MI-based
Auxiliary
Reward

CTDE
Framework

Constrained Variational Exploration

Our method employs constrained variational exploration to model a wide range of human behaviors, allowing the crowd simulation to capture diverse behavioral patterns while maintaining realistic human movement dynamics.

MI-based Auxiliary Reward

By incorporating mutual information as an auxiliary reward, our approach encourages the discovery of diverse yet realistic behaviors in simulated humans, enhancing the robustness of robot navigation policies trained in such environments.

CTDE Framework

The Centralized Training Decentralized Execution paradigm allows for effective multi-agent reinforcement learning, ensuring stable exploration of diverse behaviors while maintaining coordination among simulated humans.

Installation

conda create -n diversecrowd python=3.8
conda activate diversecrowd
# Install pytorch>=1.9.0 (CUDA>=11.0) manually
git clone https://github.com/wujiaxu/robust_robot_navi.git
cd robust_robot_navi
pip install -e .

# Solve Dependencies
pip install gym==0.21.0
pip install pyglet==1.5.0
pip install importlib-metadata==4.13.0

Training on Existing Environments

# Create your preferred config
python script/create_config.py

# After modifying train_all.sh or train_select.sh to add path of your config
bash script/train_select.sh

Citation

@article{crowdmodel2025wujiaxu,
  author  = {Jiaxu Wu, Yusheng Wang, Chen Tong, Jun Jiang, Yongdong Wang, Qi An and Atsushi Yamashita},
  title   = {Diversity-aware Crowd Model for Robust Robot Navigation in Human Populated Environment},
  journal = {IEEE Robotics and Automation Letters},
  year    = {2025},
  volume  = {10},
}