Exploiting Spatial and Temporal Similarity for Efficient Immersive Graphics Systems

Virtual reality allows people to explore and interact with digital environments as if they were physically present. However, creating this experience requires a headset to display detailed images for both eyes and update them quickly as the user moves. Meeting these demands is even more difficult for portable devices with limited computing power and energy. Generating images on a remote server reduces computation on the headset but places high demands on the network connection needed to deliver them. Meanwhile, emerging approaches that use neural networks to generate intermediate representations or final images provide rich visual detail but introduce additional computation. Across these settings, much of the information is repeated: the two eyes see overlapping parts of a scene, and successive images show many of the same objects. However, simply copying existing results can produce errors because visibility and appearance change with the viewpoint.

This dissertation investigates how immersive graphics systems can reuse this shared information while accounting for the differences between views. First, it develops a virtual reality streaming system that reconstructs one eye view using information from the other eye and previous images, reducing the amount of data transmitted. It then develops methods that share neural shading computations between the two eyes and reuse previous shading results as the viewpoint moves, while preserving changes in surface appearance. Finally, it reduces the cost of updating reused information and spreads these updates across successive images to avoid sudden increases in computation. Together, these approaches show how similarities across space and time can reduce communication and computation while maintaining visual quality and responsiveness.

 

Share this event:

facebook linked in twitter email

Media Contact: Emmalia Lutz