The Design of Firefox Reality

For web designers and developers, the arrival of a new platform and a new web browser can be fraught with excitement but also with anxiety. There are new technical constraints, new interaction techniques, and the visual patterns worn smooth by decades of print and web design must be revisited with new eyes. This post summarizes the excitement and anxiety that we face while bringing Firefox Reality to stand-alone augmented and virtual reality headsets.

John Maeda wrote, “Simplicity is about subtracting the obvious and adding the meaningful.” For the coming class of stand-alone headsets, we have revisited the web's core assumptions about what is obvious and what is meaningful. We have found a new set of constraints, goals, and criteria.

Constraints we have in bucketfuls. Headset hardware has limited battery power and computation. Each operating system has unique capabilities and quirks. Some headsets come with hand-held controllers, some recognize hand gestures, and some use gaze controls and touchpads. The size of the user interfaces must be designed to support relatively clumsy input devices (compared to mouse or touch) and so must be larger. Pass-through displays in augmented reality glasses require different rendering techniques than the opaque screens in virtual reality head mounted displays. With the loss of the keyboard and mouse, traditional approaches to core features such as text entry and window management completely fail. Unlike laptop or mobile web devices, VR headsets require users to find a safe location with enough room to move around a virtual scene.

Goals in stand-alone headsets are inherently different than when using a laptop or a mobile device. Headset manufacturers will continue to experiment with comfort, power management, and visual quality, but for the next few years headset sessions will be shorter than the 8 to 16 hours per day that people use laptops and mobiles. Designs for headset software must to be oriented around short sessions with many breaks between intense interactions. User interfaces must be placed to prevent neck strain and repetitive injuries. Laptop and mobile web applications serve many purposes, but the initial wave of headsets will target a few specific domains like on-site industrial use and entertainment experiences like gaming. While immersive experiences are an important new aspect of the web, the vast majority of the web remains in 2D so headsets must enable users smooth transitions between the 2D and 3D content.

Criteria for Firefox Reality are different because stand-alone headsets are essentially different from all other platforms. They are more intimately connected to our bodies, and so must be more personable. They are between our eyes and our environment, and so must be our trusted intermediaries. They can cause nausea and disorientation, and so must be visibly smooth and consistent.

With those constraints, goals, and criteria in mind, here are the major components for the first version of Firefox Reality:

Environments

In augmented reality headsets, the environment in which Firefox Reality is running is actual reality. In VR headsets, it's depressing to work in a vast black void so it's necessary to spruce up VR with virtual environments. While the environment will eventually be fully configurable by users, to get started we will have at least three choices.

Windows

Stand-alone headsets have no fallback screens like on a PC tethered VR system or the screen on a mobile device that can clip into a headset. Therefore, it is important that the existing 2D web work well in Firefox Reality. We've taken the decades of web compatibility investment in Firefox's Gecko renderer and wrapped it in a user interface that is both comfortably familiar and designed specifically for new types of input. We’re experimenting with the usual tabs, URL bars, full-screen mode, and windows that you expect from a browser, but each has been reworked for use in headsets and for both hand and voice controls.

The tray

Where desktop and mobile browsers have top bar menus, Firefox Reality has a button tray. Here you will find all of the application-wide actions, like opening a new window or editing your settings. The tray is like a workbench or a dashboard, providing you with a solid place to orient yourself as you move through virtual and actual environments.

The sigil

Unlike on desktop browsers, immersive web pages using WebVR can take over the entire visible environment. It is possible that a malicious web coder could create something that looks and acts exactly like trusted browser UI and then act maliciously, such as capturing passwords or other private information. To prevent this, we’ve created a new concept: the security sigil. Its shape, color, and pattern are chosen by the user and those choices are not shared with any web page. If the user sees their unique sigil on the tray then they know that they are interacting with the real Firefox Reality instead of a spoofed user interface.

New realities

Firefox was the first web browser to ship virtual reality for the desktop. With Firefox Reality, we're bringing the same capabilities to AR and VR headsets. You will paint and sculpt fantastic landscapes using A-Painter. You will be able to explore new worlds with the games and educational experiences from organizations you already know. All of these will be based on web standards that enable anything you make to work across the entire web.

Future work
The initial design of Firefox Reality focuses on what is possible today, but we are also working on what will be possible tomorrow. Our next-generation rendering engine, Servo, provides us with the ability to use computer vision, parallel processing, and experimental rendering techniques to prototype the future of the immersive web. The Mixed Reality social team is building new ways to bring distributed groups together for work and play. We’re also experimenting with new ways to share links and tabs between all of your devices so that you can move from mobile to desktop to headset in that uniquely web-ish way.

Augmented and virtual reality give us another chance to see better. Firefox Reality will succeed only when it enables us to better see ourselves, each other, and the world. If you would like to see better, you can download an experimental build. If you are a designer or developer, you can help us build Firefox Reality by visiting the GitHub repo or emailing us.

(Thanks to Nadja Haldimann (@thenadj) and Jenni Hoang (@jen_maicha) for their work on the design of Firefox Reality)