Week 4

The learning goals for Week 4 are

  • Learn world building tools for prototyping
  • Learn canvas and interaction UIs
  • Learn hand and avatar animation
  • Make an escape room experience

3D User Interfaces

With the continuing development of affordable immersive virtual reality (VR) systems, there is now a growing market for consumer content. The current form of consumer systems is not dissimilar to the lab-based VR systems of the past 30 years: the primary input mechanism is a head-tracked display and one or two tracked hands with buttons and joysticks on hand-held controllers. Over those 30 years, a very diverse academic literature has emerged that covers design and ergonomics of 3D user interfaces (3DUIs). However, the growing consumer market has engaged a very broad range of creatives that have built a very diverse set of designs. Sometimes these designs adopt findings from the academic literature, but other times they experiment with completely novel or counter-intuitive mechanisms. Find further details in Directions for 3D User Interface Research from Consumer VR Games

Hand Tracking

The latest VR devices such as the Oculus Quest offer hand tracking functionalities allowing user to interact without the use of controllers. The following video gives an overview on the design considerations on this input modality.

Similar to other hand-based input mechanism, make sure that your experience avoid common design mistakes:

  • Reducing gorilla arm
  • Avoid repetitive strain
  • Consider ergonomic cost

For this week’s assignment, we do not use the hand tracking as the development difficulty is considerably higher.


Hand Space UI

Tilt Brush

Hand Tracking UI

Oculus Quest Menu


Beat Saber

Diegetic UI

Job Simulator

Throwing Hand

Three Totally Creative Uses of Quest Hand-tracking

Gaze-based UI

SightLine: The Chair