Improving Reinforcement Learning Experiments in Unity through Waypoint Utilization


  • Caleb Koresh
  • Volkan Ustun University of Southern California Institute for Creative Technologies
  • Rajay Kumar
  • Tim Aris



Multi-agent System, Deep Reinforcement Learning, Military Simulation, Artificial Intelligence


Multi-agent Reinforcement Learning (MARL) models teams of agents that learn by dynamically interacting with an environment and each other, presenting opportunities to train adaptive models for team-based scenarios. However, MARL algorithms pose substantial challenges due to their immense computational requirements. This paper introduces an automatically generated waypoint-based movement system to abstract and simplify complex environments in Unity while allowing agents to learn strategic cooperation. To demonstrate the effectiveness of our approach, we utilized a simple scenario with heterogeneous roles in each team. We trained this scenario on variations of realistic terrains and compared learning between fine-grained (almost) continuous and waypoint-based movement systems. Our results indicate efficiency in learning and improved performance with waypoint-based navigation. Furthermore, our results show that waypoint-based movement systems can effectively learn differentiated behavior policies for heterogeneous roles in these experiments. These early exploratory results point out the potential of waypoint-based navigation for reducing the computational costs of developing and training MARL models in complex environments. The complete project with all scenarios and results is available on GitHub:




How to Cite

Koresh, C., Ustun, V., Kumar, R., & Aris, T. (2024). Improving Reinforcement Learning Experiments in Unity through Waypoint Utilization. The International FLAIRS Conference Proceedings, 37(1).



Special Track: Artificial Intelligence in Games, Serious Games, and Multimedia