Hierarchical, Discontinuous Agent Reinforcement Learning Rewards in Complex Military-Oriented Environments

Abstract

Artificially intelligent agents are seeing increased adoption in both the video game and simulation industry for training, education, and entertainment purposes. These systems often need realistic and believable opponents that must achieve objectives in the face of competing and contradictory priorities and frequently require the rapid creation of a wide spectrum of agents with disparate behaviors that reflect tactical realism. This in turn drives the need for the dynamic training of such agents from available source data. Approaches to do so have yet to be widely investigated due to the smaller scales of these simulation environments. This paper discusses techniques to quickly design and generate a variety of AI agents that follow desired tactics and procedures, including realistic situations that require trade-off decisions between competing objectives. Techniques described include an investigation into deep reinforcement agents that have separable reward structures and can prioritize and re-prioritize goals based on a hierarchy.