Unifying Regret and State-Action Space Coverage for Effective Unsupervised Environment Design

Unsupervised Environment Design (UED) employs interactive training between a teacher agent and a student agent to train generally-capable student agents. Existing UED methods primarily rely on regret to progressively introduce curriculum complexity for the student but often overlook the importance of environment novelty - a critical element for enhancing an agent's exploration and generalization capabilities. There is a substantial lack of investigating the effects of environment novelty in UED. This paper addresses this gap by introducing the GMM-based Evaluation of Novelty In Environments (GENIE) framework. GENIE quantifies environment novelty within the UED paradigm by using Gaussian Mixture Models. To assess GENIE's effectiveness in quantifying novelty and driving exploration, we integrate it with ACCEL, the state-of-the-art UED algorithm. Empirical results demonstrate the superior zero-shot performance of this extended approach over existing UED algorithms, including its predecessor. By providing a means to quantify environment novelty, GENIE lays the groundwork for future UED algorithms to unify novelty-driven exploration and regret-driven exploitation in curriculum generation.