Overview

In this course we will examine hardware, software, and cognitive foundations involved in creating state-of-the-art virtual reality (VR). The purpose of this course is two-fold: (a) to provide students with both a deep understanding of the fundamentals of VR and (b) to gain experience with consumer-grade VR system architecture. We will explore the challenges to develop and analyze VR systems that are effective and comfortable, motivated by design goals, and constrained by the real world.

This is a very challenging course. The goal is to provide students with both a deep understanding of the scientific, technical, engineering, and human-facing fundamentals of VR and gain engineering experience. Note: We will not learn about VR game development, but instead study the fundamental building blocks of VR systems and implement them in a simulation context.
Topics include:

• Overviews and perspectives on VR and its relation to Computer Graphics
• Human sensation and perception: visual, audio, vestibular, tactile, perceptual training
• Engineering VR systems using Unity, Device-specific SDKs
• Head-mounted display optics and electronics
• Tracking with inertial measurement units: gyros, accelerometers, magnetometers
• Sensor fusion: complementary filter, Kalman filter

There will be a significant number of hands-on programming assignments to be completed by each student, each assignment is a small simulation of head-mounted display functionality inspired by the Oculus 1. There will also be a term project to be completed by a group of students, designed to afford the exploration of individual interests within virtual reality.

Objectives

In this course, I will cover:

1. Historical and modern overviews and perspectives on virtual reality.
2. Fundamentals of sensation, perception, and perceptual training.
3. The scientific, technical, and engineering aspects of virtual reality systems.
4. Evaluation of virtual reality from the lens of design.

Expected Learning Outcomes

At the end of CS5360, you will be able to:

1. Identify, examine, and develop hardware and software that reflects fundamental techniques for the engineering of VR platforms.
2. Choose, develop, explain, and defend the use of particular designs for VR experiences.
3. Understand the effects of specific VR techniques on the human body.
4. Identify and examine state-of-the-art VR design problems and solutions from the industry and academia.

At the end of CS6360, you will be able to do all of the above and in addition:

1. Summarize, distill, and design a research contribution within VR.

Class Details

Term: Fall 2023

Location: WEB 2250

Date and Time: MW / 04:35PM-05:55PM

Instructor: Rogelio E. Cardona-Rivera, Ph.D.

Website:

Canvas

Syllabus:

Not available publicly.

Prerequisites:

CS 5360. "C-" or better in CS 3500 and Full Major status in (Games or Computer Science or Computer Engineering)

CS 6360. "C-" or better in EAE 6310 or Graduate Standing in Computer Science. (Meets with CS 5360; extra work required for this class.)

Format:

This course will primarily be lecture-based, with in-class discussions around material, assignments, and projects and out-of-class assignments, projects, and one exam.

The primary goal of this class is to understand the fundamental building blocks of virtual reality by building a head-mounted display (HMD) from scratch. Since the class does not have a textbook that costs money, you are expected to spend some money in terms of hardware components in order to participate in this course.

Textbook:

The class is designed around the following (freely-available) textbooks:

• LaValle, Steven M. Virtual Reality. (Self-published), 2017.
• Shirley, Peter, Michael Ashikhmin, and Steve Marschner. Fundamentals of Computer Graphics (3rd ed). A. K. Peters/CRC Press (2009)