Students: Liam Stewart

Project: Virtual Reality for Spatial Visualization Skills: Using Reinforcement Learning to Generate 3D Shapes

Poster: Vertical (PDF) | Horizontal (PDF)

Institution: Lafayette College

Major: Electrical and Computer Engineering

Advisor: Christian Lopez


Introducing a Procedural Content Generation (PCG) method based on Reinforcement Learning (RL) to automatically generate 3D shapes of different complexities is the goal of this work. This goal also includes the objective of adapting the spatial visualization task with a Virtual Reality (VR) program to help students develop spatial skills. It has been shown that within STEM fields spatial skills are often used and are valuable to one’s success. Although there are VR programs designed to teach spatial skills, most of them don’t adapt to the skill level of the user. Studies within the context of PCG, demonstrate that using PCG to generate new content for VR applications has an effective potential. By automatically generating shapes they could be tailored to the level of the student during generation. This work and its results suggest that an RL agent could effectively generate an action policy, which is capable of making shapes that have a complexity close to a given desired complexity. In addition, the results also display that generating shapes in this manner is not a trivial task due to the sparsity in the reward space. Regardless, this work establishes a benchmark to use RL as a tool to automatically generate 3D shapes in a VR environment to enable students to effectively develop spatial skills.

Liam Stewart

About Liam Stewart

Liam Stewart is a sophomore in the class of 2024 at Lafayette College. his major is Electrical and Computer Engineering. Doing research with Lafayette’s Computer Science Department has given Liam insight into the work done within the computer science field, as a whole I am finding it a great learning opportunity. Liam hopes that he can recreate the experience within other fields in the future as they would help him narrow down the career choices he will soon be making.