Driving Virtual Reality from Linux

Driving Virtual Reality from Linux


Keith Packard

Head-Mounted Displays (HMD) are the most common device for presenting
virtual reality to the user. They project independent images to each
eye to construct a stereo view and track the position and orientation
of the head using both inertial measurement and external beacons.

All of the elements in the system introduce varying amounts of delay
between the motion of the head and the perception of the image by the
user. Computing the projected images thus requires predicting where
the user will be facing when the image is finally emitted from the

As the user is strapped into the HMD and their visual field is limited
to that provided by the HMD, timing errors in the VR presentation can
cause disorientation leading to nausea or other effects. This leads to
strong latency bounds through the entire application pipeline.

A traditional window system, running multiple applications, cannot
easily provide the guarantees needed by a VR system. Competing
applications may consume window system resources and cause jitter in
the VR presentation. Separating the VR application from the window
system can allow it to operate without interference from other
applications and improve performance.

Started at LCA last year in Hobart, the new DRM Leasing mechanism
allows applications to take over a set of display resources from the
window system and drive them directly. These changes involved
modifications to the Linux kernel, X window system and Mesa Vulkan
implementation. This new system brings proposed Vulkan extensions for
direct display operation to the DRM environment.

The presentation will discuss the design and implementation of the
system, along with highlights of the development process and a
live demonstration of the resulting system.

All of this free software was developed under contract with Valve, who
are using it to provide support for the HTC VIVE HMD.

This talk was given at Linux.conf.au 2018 (LCA2018) which was held on 22-26 January 2018 in Sydney Australia.

linux.conf.au is a conference about the Linux operating system, and all aspects of the thriving ecosystem of Free and Open Source Software that has grown up around it. Run since 1999, in a different Australian or New Zealand city each year, by a team of local volunteers, LCA invites more than 500 people to learn from the people who shape the future of Open Source. For more information on the conference see

#linux.conf.au #linux #foss #opensource