A Hierarchical Goal-Biased Curriculum for Training Reinforcement Learning

Autor/innen

  • Sunandita Patra University of Maryland, College Park
  • Mark Cavolowsky University of Maryland, College Park
  • Onur Kulaksizoglu University of Maryland, College Park
  • Ruoxi Li University of Maryland, College Park
  • Laura Hiatt Naval Research Laboratory
  • Mark Roberts
  • Dana Nau

DOI:

https://doi.org/10.32473/flairs.v35i.130720

Schlagworte:

planning and learning, goal biased curriculum, reinforcement learning, curriculum learning, hierarchical planning

Abstract

Hierarchy and curricula are two techniques commonly used to improve training for Reinforcement Learning (RL) agents. Yet few works have examined how to leverage hierarchical planning to generate a curriculum for training RL Options. We formalize a goal skill that extends an RL Option with state-based conditions that must hold during training and execution. We then define a Goal-Skill Network that integrates a Hierarchical Goal Network, a variant of hierarchical planning, with goal skills as the leaves of the network. An automatically generated plan for a Goal-Skill Network correctly orders goal skills such that (1) it is a Goal-Biased Curriculum for training the goal skills, and (2) it can be executed to achieve top-level goals. In a set of six distinct gridworld environments using up to ten goal skills, we demonstrate that these contributions train nearly perfect policies significantly faster than learning a whole policy from scratch.

Downloads

Veröffentlicht

2022-05-04

Zitationsvorschlag

Patra, S., Cavolowsky, M., Kulaksizoglu, O., Li, R., Hiatt, L., Roberts, M., & Nau, D. (2022). A Hierarchical Goal-Biased Curriculum for Training Reinforcement Learning. The International FLAIRS Conference Proceedings, 35. https://doi.org/10.32473/flairs.v35i.130720

Ausgabe

Rubrik

Special Track: Autonomous Robots and Agents