Samsung Gear VR support lands in Servo

We are happy to announce that Samsung Gear VR headset support is landing in Servo. The current implementation is WebVR 1.1 spec-compliant and supports both the remote and headset controllers available in the Samsung Gear VR 2017 model.

If you are eager to explore, you can download a project template compatible with Gear VR Android phones. Add your Oculus signature file, and run the project to launch the application on your mobile phone.

Alongside the Gear VR support, we worked on other Servo areas in order to provide A-Frame compatibility, WebGL extensions, optimized Android compilations and reduced Servo startup times.

A-Frame Compatibility

Servo now supports Mutation Observers that enables us to polyfill Custom Elements. Together with a solid WebVR architecture and better texture loading we can now run any A-Frame content across mobile (Google Daydream, Samsung Gear VR) and desktop (HTC Vive) platforms. All the pieces have fallen into place thanks to all the amazing work that the Servo team is doing.

WebGL Extensions

WebGL Extensions enable applications to get optimal performance by taking advantage of state-of-the-art GPU capabilities. This is even more important in VR because of the extra work required for stereo rendering. We designed the WebGL extension architecture and implemented some of the extensions used by A-Frame/Three.js such as float textures, instancing, compressed textures and VAOs.

Compiling Servo for Android

Recently, the Rust team changed the default Android compilations targets. They added an armv7-linux-androideabi target corresponding to the armeabi-v7a official ABI and changed the arm-linux-androideabi to correspond to the armeabi official ABI instead of armeabi-v7a.

This could cause important performance regressions on Servo because it was using the arm-linux-androideabi target by default. Using the new armv7 compilation target is easy for pure Rust based crates. It’s not so trivial for cmake or makefile based dependencies because they infer the toolchain and compiler names based on the target name triple.

We adapted all the problematic dependencies. We took advantage of this work to add arm64 compilation support and provided a simple CLI API to select any Android compilation target in Servo.

Reduced startup times

C based libfontconfig library was causing long startup times in Servo for Android. We didn’t find a way to fix the library itself so we opted to get rid of it and implement an alternative way to query Android system fonts. Unfortunately, Android doesn't provide an API to query system fonts until Android O so we were forced to parse the system configuration files and load fonts manually.

Gear VR support on Rust-WebVR Library

We started working on ovr-mobile-sys, the Rust bindings crate for the Oculus Mobile SDK API. We used rust-bindgen to automatically generate the bindings from the C headers but had to manually transpile some of the inline SDK header code since inline functions don’t generate symbols and are not exported by rust-bindgen.

Then we added the SDK integration into the rust-webvr standalone library. The OculusVRService class offers the entry point to access Oculus SDK and handles life-cycle operations such as initialization, shutdown, and VR device discovery. The integration with the headset is implemented in OculusVRDisplay. Gear VR lacks positional tracking, but by using the neck model provided in the SDK, we expose a basic position vector simulating how the human head naturally rotates relative to the base of the neck.

In order to read Gear VR sensor inputs and submit frames to the headset, the Android activity must enter VR Mode by calling vrapi_EnterVrMode() function. Oculus Mobile SDK requires a precise life cycle management and handling some events that may interleave in complex ways. For a correct implementation the Android Activity must enter VR mode in a surfaceChanged() or onResume() event, whichever comes last. And it must leave VR mode in a surfaceDestroyed() or onPause() event, whichever comes first.

In a Glutin based Android NativeActivity, life cycle events are notified using Rust channels. This caused synchronization problems due to non-deterministic event handling in multithreading. We couldn’t guarantee that the vrapi_LeaveVrMode() function was called before NativeActivity’s EGLSurface was destroyed and the app went to background. Additionally, we needed to block the event notifier thread until Gear VR resources are freed, in a different renderer thread, to prevent collisions (e.g. Glutin dropping the EGLSurface at the same time that VR renderer thread was leaving VR mode). We contributed a deterministic event handling implementation to the Rust-android-glue.

Oculus mobile SDK allows to directly send a WebGL context texture to the headset. Despite that, we opted for a triple buffered swap chain recommended in the SDK to avoid potential flickering and performance problems when using the same texture every frame. As we did with the Daydream implementation, we render the VR-ready texture to the current ovrTextureSwapChain using a BlitFramebuffer-based solution, instead of rendering a quad, to avoid implementing the required OpenGL state-change safeguards or context switching.

Oculus Mobile SDK allowed us to directly attach the NativeActivity’s surface to the Gear VR time warp renderer. We were able to run the pure Rust room-scale demo without writing a line of Java. It’s nice that the SDK allows to achieve a java-free integration, but our luck changed when we integrated all this work into a full browser architecture.

Gear VR integration into Servo

Our Daydream integration worked inside Servo almost on a first try after it landed on the rust-webvr standalone library. This was not the case with the Gear VR integration…

First, we had to research and fix up to four specific GPU driver issues with the Mali-T880 GPU used in the Samsung Galaxy S7 phone:

As a result, we were able to see WebGL stereo rendering on the screen but entering VR mode crashed with a JNI assertion failure inside the Oculus VR SDK. This was caused because inside the browser context different threads are used for the rendering and VR device initialization/discovery. This requires the use of different Oculus ovrJava instances for each thread.

The assertion failure was gone but we couldn’t see anything on the screen after calling vrapi_EnterVrMode(). The logcat error messages triggered by the Oculus SDK helped to find the cause of the problem. The Gear VR time warp implementation hijacks the explicitly passed Android window surface pointer. We could use the NativeActivity’s window surface in the standalone room-scale demo. In a full browser architecture, however, there is a fight to take over ownership of the Android surface between time warp thread and the browser compositor. We discarded the idea of directly using the NativeActivity’s window surface and decided to switch to a Java SurfaceView VR backend in order to make both the browser’s compositor and Gear VR’s time warp thread happy.

By this means, the VR mode life cycle fit nicely in the browser architecture. There was one final surprise though. The activity entered VR mode correctly, there were no errors in the logcat, time warp thread was showing correct render stats and the headset pose data was correctly fetched. Nevertheless, the VR scene with lens distortion was not yet visible in the Android view hierarchy. This led to a new instance of spending some hours of debugging to change a single line of code. The Android SurfaceView was being rendered correctly but it was composited below the NativeActivity’s browser window because setZOrderOnTop() is not enabled by default on Android:

After this change everything worked flawlessly and it was time to enjoy running some WebVR experiences on the Gear VR ;)

Conclusion

It's been a lot of fun seeing Gear VR support land in Servo and being able to run A-Frame demos in it. We continue to work hard on squeezing WebGL and WebVR performance and expect to land some nice optimizations soon. We are also working on implementing unique WebVR features that no other browser has yet. More news soon ;) Stay tuned!