(ORCSE4529) Reinforcement Learning Course Project: Rise of the AI Knight

Overview

This project investigates the development of reinforcement learning (RL) agents for the challenging action-platformer games Hollow Knight and Silksong. We explore whether improved performance, robustness, and training efficiency can be achieved through refined state representations and modern RL architectures.

Our objective is to design agents capable of learning navigation, combat, and evasion behaviors directly from raw pixel observations and game-state variables such as player health and boss health.

Technical Approach

We draw inspiration from model-based reinforcement learning (MBRL)—particularly from STORM, a stochastic Transformer-based architecture that represents images using categorical VAEs and predicts future dynamics in latent space.
STORM demonstrates how latent imagination, Transformer sequence modeling, and stochastic world representations can significantly improve sample efficiency and long-horizon reasoning.

Outcomes

Reference

Zhang, W., Wang, G., Sun, J., Yuan, Y., & Huang, G. (2023).
STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learning.
Advances in Neural Information Processing Systems (NeurIPS 2023).
PDF