Enabling the Social 3D Web
-
As hinted in our recent announcement of our Mixed Reality program we’d like to share more on our efforts to help accelerate the arrival of the Social 3D Web.
Mixed Reality is going to provide a wide variety of applications that will change the way we live and work. One which stands to be the most transformative is the ability to naturally communicate with another person through the Internet. Unlike traditional online communication tools like text, voice, and video, the promise of Mixed Reality is that you can be present with other people, much like real life, and engage with them in a more interactive way. You can make eye contact, nod your head, high five, dance, and even play catch with anyone around the world, regardless of where you are!
Mozilla’s mission is to ensure the Internet remains a global public resource. As part of this mission, fostering this new and transformative form of Internet-based communication is something we care deeply about. We believe that the web is the platform that will provide the best user experience for Mixed Reality-based communication. It will ensure that people can connect and collaborate in this new medium openly, across devices and ecosystems, free from walled gardens, all while staying in control of their identity. Meeting with others around the world in Mixed Reality should be as easy as sharing a link, and creating a virtual space to spend time in should be as easy as building your first website.
To help realize this vision, we have formed a dedicated team focused on Social Mixed Reality. Today, we’ll be sharing our vision and roadmap for accelerating the arrival of Social MR experiences to the web.
WebVR has come a long way since its very first steps in 2014. The WebVR and A-Frame communities have produced amazing experiences, but multi-user social experiences are still few and far between. Without a shared set of libraries and APIs, Social MR experiences on the web are often inconsistent, with limited support for identity and avatars, and lack basic tools to allow users to find, share, and experience content together.
To address this, we are investing in areas that we believe will help fill in some of the missing pieces of Social MR experiences on the web. To start, we will be delivering open source components and services focused on making it possible for A-Frame developers to deliver rich and compelling Social MR experiences with a few lines of code. In addition, we will be building experimental Social MR products of our own in order to showcase the technology and to provide motivating examples.
Although we plan to share some initial demos in the near future, our work has advanced enough that we wanted to share our github repositories and invite the community to join the conversation, provide feedback, or even begin actively contributing.
We’ll be focused on the following areas:
Avatars & Identity
We want to enable A-Frame (and eventually, other frameworks and game engines) to easily add real-time, natural, human communication to their experiences. This includes efficient networking of voice and avatars, consistent locomotion controls, and mechanisms for self-expression and customization with an identity that you control. We will also be exploring ways for creators to publish and distribute custom avatars and accessories to users of Social MR applications.
Communication
Within a Social MR experience, you’ll often want to find your friends or meet new people, while having the controls you need to ensure your comfort and safety. We are aiming to bring traditional text and voice-based communication, social networking, and cross-app interactions like messaging to Mixed Reality A-Frame apps. This includes traditional features like friending, messaging, and blocking, as well as Mixed Reality-specific features like personal space management.
Entities
Once you are able to be with other people in Mixed Reality, you’ll want to interact with shared 3D objects. You should be able to throw a frisbee to one another, play cards, or even create a sculpture together. Also, objects in the world should be where you left them when you come back later. We’ll make it easy for Social MR apps to support the live manipulation, networking, and persistence of physics-enabled 3D objects.
Tools
There are some things you always want to be able to do in a social setting. In real life, no matter where you are, you can always pull out your phone to snap a photo or send a message. Similarly, we want to provide components and tools that are useful in — and across — all Social MR experiences. Tied together via your identity, these components could allow drawing in 3D, navigating to a web page, sharing a video, or taking a selfie in any app.
Search + Discovery
The MR Web is full of disconnected experiences. How do you find new apps to try, or share one with friends, or even join an app your friends are currently using? How do you meet new people? We’ll improve the discovery and sharing of MR content, and give users ways to meet new and interesting people, all through the web within Mixed Reality. For example, matchmaking for apps that need groups of multiple players, a friends list showing who is online and in which app, or a content feed highlighting the best Social MR experiences on the web.
How to get involved
If you are interested in helping bring Social MR to the web, first and foremost please join the conversation with us in the #social channel on the WebVR Slack.
Additionally, feel free to browse our Github repositories and open an issue if you have questions or feedback. We are just getting started, so there may be quite a bit of churn over the coming months, but we’re excited to share what we’ve done so far.
mr-social-client- A prototype of a social experience, where we’ll be testing out features and services as we build them.janus-plugin-sfu- A Selective-Forwarding Unit plugin for the Janus WebRTC gateway that allows centralized WebRTC message processing, to reduce upstream bandwidth and (eventually) provide authoritative networking services.reticulum- A Phoenix networking and application server that will be the entry point to most of the services we provide.mr-ops- Our infrastructure as code, operations, and deployment scripts.socialmr- A container for tracking GitHub issues we’re working on.
We’ll also be contributing back to other existing projects. For example we recently contributed positional audio support and are working on a Janus networking adapter for the awesome networked-aframe project by Hayden Lee.
FAQ
How long has Mozilla been working on this?
We’ve been exploring this area for over a year, but the formation of a dedicated team (and the repos published above) kicked off over the last several weeks. Although this is relatively new body of work, now that we have put together a roadmap and have some initial efforts in-progress we wanted to share it with the community.
Will you support third-party game engines such as Unity and Unreal Engine?
This is something we will be definitely be exploring down the road. Our near-term goals are focused on developing services for the web and libraries for A-Frame and three.js Social MR applications, but we hope to make these services and libraries more generally available to other platforms. We would be interested in your feedback on this!
Will you be proposing new browser APIs?
As we progress, we may find it makes sense to incorporate some of the functionality we’ve built into the browsers directly. At that point we would propose a solution (similar to how we recently proposed the WebXR API to address extending WebVR to encompass AR concepts) to solicit feedback from developers and other browser vendors.