Liam Stewart

Students: Liam Stewart

Project: Virtual Reality for Spatial Visualization Skills: Using Reinforcement Learning to Generate 3D Shapes

Poster: Vertical (PDF) | Horizontal (PDF)

Institution: Lafayette College

Major: Electrical and Computer Engineering

Advisor: Christian Lopez

Abstract

Introducing a Procedural Content Generation (PCG) method based on Reinforcement Learning (RL) to automatically generate 3D shapes of different complexities is the goal of this work. This goal also includes the objective of adapting the spatial visualization task with a Virtual Reality (VR) program to help students develop spatial skills. It has been shown that within STEM fields spatial skills are often used and are valuable to one’s success. Although there are VR programs designed to teach spatial skills, most of them don’t adapt to the skill level of the user. Studies within the context of PCG, demonstrate that using PCG to generate new content for VR applications has an effective potential. By automatically generating shapes they could be tailored to the level of the student during generation. This work and its results suggest that an RL agent could effectively generate an action policy, which is capable of making shapes that have a complexity close to a given desired complexity. In addition, the results also display that generating shapes in this manner is not a trivial task due to the sparsity in the reward space. Regardless, this work establishes a benchmark to use RL as a tool to automatically generate 3D shapes in a VR environment to enable students to effectively develop spatial skills.