Tradeoffs within Representational Behaviors: Multi-Objective Optimization Strategies for Simulated Military Units

Abstract

Generating representative behaviors for opposing and adjacent forces is critical for training military units in a simulated environment. Humans are often required in-the-loop because automated systems struggle to provide intelligent and adaptive behaviors that represent appropriate military doctrine and can be explained in the After Action Review. Automated solutions often fail to maintain enough diversity to be replayed in successive iterations of the simulated mission without becoming easily predictable. This paper describes the decision-making approach for automated hierarchical planning agents designed to overcome these challenges. Our Mission Command Agents (MCAs) combine constraint optimization for multi-agent goal reasoning and state-space planning for route planning to generate Courses of Action (COAs) for simulated units. We detail extensions in both steps to produce a set of COAs on the Pareto front for multiple practical (e.g., distance), doctrinal (e.g., cover and concealment), and behavioral (e.g., risk-tolerance or aggressiveness) objectives.

                        We compare COAs generated using this new approach to results of our previous MCA that uses a more traditional approach of aggregating objectives or preferences into a single weighted utility function and optimizing with respect to that joint utility to find multiple optimal and near-optimal solutions. In contrast to previously emitted behaviors, which often reflected many small differences in utility or positioning, analyzing the trade-offs among multiple objectives produces more meaningfully diverse solutions. In an experiential training context, this diversity supports replaying scenarios with automated units that have distinct preferences and priorities. We summarize our results with evaluation metrics showing a set of COAs can appropriately represent doctrinal constraints with diverse, reasonable, and explainable trade-offs between objectives.