CS 5360/6360 - Virtual Reality

Overview

In this course we will examine hardware, software, and cognitive foundations involved in creating state-of-the-art virtual reality (VR). The purpose of this course is two-fold: (a) to provide students with both a deep understanding of the fundamentals of VR and (b) to gain experience with consumer-grade VR system architecture. We will explore the challenges to develop and analyze VR systems that are effective and comfortable, motivated by design goals, and constrained by the real world.

This is a very challenging course. The goal is to provide students with both a deep understanding of the scientific, technical, engineering, and human-facing fundamentals of VR and gain engineering experience. Note: We will not learn about VR game development, but instead study the fundamental building blocks of VR systems and implement all of them.
Topics include:

  • Introduction to computer graphics and the rendering pipeline
  • JavaScript with Three.js, OpenGL, WebGL, and GLSL shader programming
  • Stereoscopic perception and rendering
  • Head mounted display optics and electronics
  • Tracking with inertial measurement units: gyros, accelerometers, magnetometers
  • Sensor fusion: complementary filter, Kalman filter
  • Human perception: visual, audio, vestibular, tactile
    There will be a significant number of hands-on programming assignments to be completed by each student or small team of students. Each assignment is a small piece of a bigger project. The goal of the bigger project is to build a fully functional head-mounted virtual reality display (HMD) from off-the-shelf parts. The HMD we will build is inspired by the Oculus 1.

Objectives

In this course, I will cover:

  1. Historical and modern overviews and perspectives on virtual reality.
  2. Fundamentals of sensation, perception, and perceptual training.
  3. The scientific, technical, and engineering aspects of virtual reality systems.
  4. Evaluation of virtual reality from the lens of design.

Expected Learning Outcomes

At the end of CS5360, you will be able to:

  1. Identify, examine, and develop hardware and software that reflects fundamental techniques for the engineering of VR platforms.
  2. Choose, develop, explain, and defend the use of particular designs for VR experiences.
  3. Understand the effects of specific VR techniques on the human body.
  4. Identify and examine state-of-the-art VR design problems and solutions from the industry and academia.

At the end of CS6360, you will be able to do all of the above and in addition:

  1. Summarize, distill, and design a research contribution within VR.

Class Details

Term: Fall 2021

Location: WEB 2230

Date and Time: MW / 04:35PM-05:55PM

Instructor: Rogelio E. Cardona-Rivera

Website:

TBD

Syllabus:

F21

Prerequisites:

CS 5360. "C-" or better in CS 3500 and Full Major status in (Games or Computer Science or Computer Engineering)

CS 6360. "C-" or better in EAE 6310 or Graduate Standing in Computer Science. (Meets with CS 5360; extra work required for this class.)

Format:

This course will primarily be lecture-based, with in-class discussions around material, assignments, and projects and out-of-class assignments, projects, and one exam.

The primary goal of this class is to understand the fundamental building blocks of virtual reality by building a head-mounted display (HMD) from scratch. The HMD is designed to be educational, accessible, reproducible, and low cost. Since the class does not have a textbook that costs money, you are expected to spend a roughly equivalent amount in terms of hardware components in order to participate in this course. The parts are described in the syllabus.

Textbook:

The class is designed around the following (freely-available) textbooks:

  • LaValle, Steven M. Virtual Reality. (Self-published), 2017.
  • Shirley, Peter, Michael Ashikhmin, and Steve Marschner. Fundamentals of Computer Graphics (3rd ed). A. K. Peters/CRC Press (2009)

Elsewhere on the Web

Address

Merrill Engineering Building, #3153
50 Central Campus Drive
Salt Lake City, UT, 84112, USA