A Hierarchical Goal-Biased Curriculum for Training Reinforcement Learning
DOI:
https://doi.org/10.32473/flairs.v35i.130720Schlagworte:
planning and learning, goal biased curriculum, reinforcement learning, curriculum learning, hierarchical planningAbstract
Hierarchy and curricula are two techniques commonly used to improve training for Reinforcement Learning (RL) agents. Yet few works have examined how to leverage hierarchical planning to generate a curriculum for training RL Options. We formalize a goal skill that extends an RL Option with state-based conditions that must hold during training and execution. We then define a Goal-Skill Network that integrates a Hierarchical Goal Network, a variant of hierarchical planning, with goal skills as the leaves of the network. An automatically generated plan for a Goal-Skill Network correctly orders goal skills such that (1) it is a Goal-Biased Curriculum for training the goal skills, and (2) it can be executed to achieve top-level goals. In a set of six distinct gridworld environments using up to ten goal skills, we demonstrate that these contributions train nearly perfect policies significantly faster than learning a whole policy from scratch.
Downloads
Veröffentlicht
Zitationsvorschlag
Ausgabe
Rubrik
Lizenz
Copyright (c) 2022 Sunandita Patra, Mark Cavolowsky, Onur Kulaksizoglu, Ruoxi Li, Laura Hiatt, Mark Roberts, Dana Nau
Dieses Werk steht unter der Lizenz Creative Commons Namensnennung - Nicht-kommerziell 4.0 International.