WebVR coming to Servo: Architecture and latency optimizations
We are happy to announce that the first WebVR patches are landing in Servo.
For the impatients: You can download a highly experimental Servo binary compatible with HTC Vive. Switch on your headset and run
servo.exe --resources-path resources webvr\room-scale.html
The current implementation supports the WebVR 1.2 spec that enables the API in contexts other than the main page thread, such as WebWorkers.
We’ve been working hard on an optimized render path for VR to achieve smooth FPS and the required less than 20ms of latency to avoid motion sickness. This is the overall architecture:
The Rust WebVR implementation is a dependency-free library providing both the WebVR spec implementation and the integration with the vendor specific SDKs (OpenVR, Oculus …). Having it decoupled on its own component comes with multiple advantages:
- Fast develop-compile-test cycle. Compilation times are way faster than developing and testing in a full browser.
- Contributions are easier because developers don’t have to deal with the complexity of a browser code base.
- It can be used on any third party project: Room scale demo using vanilla Rust.
The API is inspired on the easy to use WebVR API but adapted to Rust design patterns. The VRService trait offers an entry point to access native SDKs like OpenVR and Oculus SDK. It allows to perform operations such as initialization, shutdown, event polling and VR Device discovery:
The VRDevice trait provides a way to interact with Virtual Reality headsets:
The integration with vendor specific SDKs (OpenVR, Oculus…) are built on top of the VRService and VRDevice traits. OpenVRService, for instance interfaces with Valve’s OpenVR API. While the code is written in Rust, native SDKs are usually implemented in C or C++. We use Rust FFI and rust-bindgen to generate Rust bindings from the native C/C++ header files.
MockService implements a mock VR device that can be used for testing or developing without having a physical headset available. You will be able to get some code done in the train or while attending a boring talk or meeting ;)
VRServiceManager is the main entry point to the rust-webvr library. It handles the life cycle and interfaces to all available VRService implementations. You can use cargo-features to register the default implementations or manually register your own ones. Here is an example of initialization in a vanilla Rust app:
WebVR integration in Servo
Performance and security are both top priorities in a Web browser. DOM Objects are allowed to use VRDevices but they neither own them or have any direct pointers to native objects. There are many reasons for this:
- WebVR Spec enforces privacy and security guidelines. For example a secondary tab is not allowed to read VRDisplay data or stop a VR presentation while the user is having a VR experience in the current tab.
The WebVRThread is a trusted component that fulfills all the performance and security requirements. It owns native VRDevices, handles their life cycle inside Servo and acts a doorman for untrusted VR requests from DOM Objects. Thanks to using Rustlang the implementation is guaranteed to be safe because ownership and thread safety rules are checked at compile-time. As other Servo components traits are splitted into a separate subcomponent to avoid cyclic dependencies in Servo.
VRCompositor Commands (integration with WebRender)
WebRender handles all the GPU and WebGL rendering work in Servo. Some of the native VR SDK functions need to run in the same render thread to have access to the OpenGL context:
- Submitting pixels for each eye to the headset uses a OpenGL texture which can only be read by the driver from the render thread of the WebGL Context.
- Vsync and Sync poses calls must be done in the same render thread where the pixels are sent.
WebVRThread can’t run functions in the WebGL render thread. Webrender is the only component allowed to run functions in the WebGL render thread. The VRCompositor trait is implemented by the WebVRThread using shared VRDevice instance references. It sets up the VRCompositorHandler instance into Webrender when it’s initialized.
A VRDevice instance can be shared via agnostic traits because Webrender is a trusted component too. Rust’s borrow checker enforces multithreading safety rules making very easy to write secure code. A great thing about the language is that it’s also flexible, letting you circumvent the safety rules when performance is the top priority. In our case we use old school raw pointers instead of Arc<> and Mutex<> to share VRdevices between threads in order to optimize the render path by reducing the levels of indirection and locks. Multithreading won’t be a concern in our case because:
- VRDevice implementations are designed to allow calling compositor functions in another thread by using the Send + Sync traits provided in Rustlang.
- Thanks to the security rules implemented in the WebVRThread, when a VRDisplay is in a presenting loop no other JSContext is granted access to the VRDisplay. So really there aren’t multithreading race conditions.
These are the VR Compositor commands that an active VRDisplay DOM object is able to send to WebRender through the IPC-Channel:
You might wonder why a vector of bytes used to send the VRFrameData. This was a design decision to decouple Webrender and the WebVR implementation. This allows for a quicker pull request cycle and avoids dependency version conflicts. Rust-WebVR and WebVRThread can be updated, even adding new fields to the VRFrameData struct without requiring further changes in Webrender. IPC-Channel messages in Servo need to be serialized using serde-serialization, so the array of bytes is used as forward-serialization solution. Rust-webvr library implements the conversion from VRFrameData to bytes using a fast old school memory transmute memcpy.
DOM Objects (Inside Servo)
- WebVRThread is used via ipc-channels to perform operations such as discovering devices, fetching frame data, requesting or stopping presentation to a headset.
- The optimized Webrender path is used via ipc-channels when the WebVRThread grants present access to a VRDisplay.
A struct definition and trait methods implementation are required for each DOMObject defined in WebIDL files. This is what the struct for the VRDisplay DOMObject looks like:
A struct implementation for a DOMObject needs to follow some rules to ensure that the GC tracing works correctly. It requires interior mutability. The JS<DOMObjectRef> holder is used to store GC managed values in structs. On the other hand, Root<DOMObjectRef> holder must be used when dealing with GC managed values on the stack. These holders can be combined with other data types such as Heap, MutHeap, MutNullableHeap, DOMRefCell, Mutex and more depending on your nullability, mutability and multithreading requirements.
It's been a lot of fun seeing Servo WebVR implementation take shape from the early stage without a WebGL backend until it's able to run WebVR samples at 90 fps and low latency. We found Rustlang to be a perfect language for simplifying the development of a complex parallel architecture while matching the high performance, privacy and memory safety requirements in WebVR. In addition Cargo package manager is great and makes handling optional features and complex dependency trees very straightforward.
For us the next steps will be to implement the GamePad API extensions for tracked controllers and integrate more VR devices while we continue improving the WebVR API performance and stability. Stay tuned!