Pathfinder: a first look at the best fonts and vector graphics on VR/AR
Second only to watching video, most of the time people spend on computing devices today involves reading text and looking at vector graphics in the toolbars and user interfaces of programs. Over the last 20 years, a great deal of focus has gone into improving the quality of those fonts and graphics: subpixel anti-aliasing, cached font maps, etc.
Unfortunately, as you can see in the left image below, that work results in grainy and jagged text in modern AR headsets. Ideally, we would render text smoothly at all angles, as shown in the image on the right.
The key limitation today is that most vector graphics or font rendering libraries assume that the images are viewed head on at a fixed resolution, rather than having a user walk around the content viewing it from many different angles and distances. Fortunately, Pathfinder fundamentally reimagines this rasterization process, and can render complex images and documents in real time.
The following movie provides a demonstration of both Pathfinder's font and vector graphics rendering.
The main innovation in Pathfinder is to move rendering of curved shapes from the CPU to the GPU, and to support the transforms required for viewing from any angle. Since many modern devices have very powerful GPUs compared to their CPU, this results in a massive performance gain in 3D rendering, allowing graphics and text to be rendered on every frame, rather than rendered once at a fixed resolution. This is done by breaking down complex images into much smaller tiles, many of which are just flat colour, and writing GPU shader code to handle the tiles that are more complicated. (For more details, see A Look at Pathfinder by Nicolas Silva.)
How Pathfinder renders the GhostScript Tiger (from A Look at Pathfinder)
Today, we are providing demos of the technology, available for daydream VR and Magic Leap here! These are provided to give an early glimpse of what Pathfinder is capable of, and not for deploying in production yet! We are working to enable Pathfinder inside of WebRender, which will allow it to be used directly in our new browsers for AR headsets based on Servo and potentially in all Firefox Gecko-based browsers in the future. And we'd like to make it available for use in Unity projects and as a WASM package, which will not have full integration with the operating system but still provide a dramatic improvement in rasterization quality. Please drop by the Github repository to check it out for your own projects and contribute to ensure that our new computing platforms have the best possible readability.