In 2023, the augmented and virtual reality industry (AR & VR, although collectively referred to as "XR") witnessed a paradigm shift with the rise of mixed reality (MR). This was especially highlighted by the unveiling of the Vision Pro, Apple’s first ever XR headset, as well as by the launch of devices like the Meta Quest 3 headset. However, MR is often misinterpreted or oversimplified, but understanding it in its true essence is pivotal for the wider adoption of the technology and the ultimate growth of the XR industry as a whole.
The Meta Quest 3 was released earlier this year, whereas the Apple Vision Pro, whilst announced back in June 2023, will not be available to consumers until 2024.
Image source: Meta / Apple
The common, but somewhat misleading definition of mixed reality is that of a combination of both VR and AR, wherein MR brings together both digital elements and the real world. However, this definition is slightly misleading, and is not favored among XR purists.
Originally, mixed reality was defined as the entire spectrum from reality to virtuality, with the real world (reality) on one end, and virtual reality (virtuality) on the other. AR then sat somewhere in between.
One big part of why the term is used today is due to the fact that the industry saw what was essentially a shift in language as a result of marketing. With both Microsoft and Magic Leap wanting to differentiate their respective XR devices from Google Glass (which may or may not have given AR a bad rap when it came out) and other head-up display (HUD) type augmented reality devices at the time (which technically weren’t true AR), the two companies landed on the term “mixed reality” for the marketing of the HoloLens and Magic Leap One headsets.
A user in the Magic Leap 2 headset at AWE USA
As it stands, we find that the best definition of augmented reality/AR in the eyes of the purists (AWE’s Ori Inbar being one of them), is in fact Ronald T. Azuma’s definition, which states that AR:
Combines both real and virtual;
Is interactive in real time; and
Is registered in 3-D
This definition of course encompasses what is now currently described as “mixed reality.” In reality (no pun intended), the correct description of what we are seeing in 2023 and will continue to see throughout 2024, is in fact the maturity of AR on VR hardware – or as is technically known: “Video Passthrough AR.”
With the semantics out of the way, we can look at what the rise of mixed reality means for the XR industry as a whole. So far this year, several devices have been announced or released that prioritize MR. Notable advancements have been made with devices like the aforementioned Vision Pro and Quest 3, and there are likely many more on the way in the coming months.
With big players like Samsung, Qualcomm, and Google teaming up to launch a headset that will almost certainly emphasize MR (it will have to in order to be able to compete with Apple’s device), it indicates a broader industry trend and move towards mixed reality devices and experiences.
Apple Vision Pro and Meta Quest 3 headsets launch touting MR applications: These devices mark a pivotal shift in the VR/AR industry, signifying the importance and potential of MR technology.
Anticipated releases by tech giants: The Samsung-Qualcomm-Google collaboration points to an industry-wide focus on MR, suggesting a future rich with mixed reality applications.
Varjo and Lynx contributions have significantly advanced MR tech: Varjo for instance has helped pave the way for technology that provides a level of visual quality and resolution that is almost indistinguishable from the real world.
The realm of MR has been fertile ground for innovation, particularly in gaming and user experience design. Pioneers like Lee Vermuelen and Thomas Van Bouwel have been instrumental in pushing the boundaries of MR. Their work, previously conceptual, is now being realized with the latest hardware advancements.
Major gaming studios such as Resolution Games and Schell Games are leading the way with groundbreaking titles like 'Demeo', 'Spatial Ops', 'Blaston', and 'I Expect You to Die: Home Sweet Home', offering immersive experiences that were previously unimaginable.
There are already some impressive MR gaming titles that have been released on the Quest 3, with titles like ‘Drop Dead: The Cabin — Home Invasion’ allowing players to shoot zombies in their own living rooms.
Several studios are also working on mixed reality titles, including Resolution Games (Demeo, Spatial Ops and Blaston ), and Schell Games (I Expect you to Die: Home Sweet Home) both putting out compelling gaming titles that have kept users returning for more immersive fun over the years.
Individuals such as Thomas Van Bouwel are also leading the gaming charge, with their impressive mixed reality titles including Cubism, Coaster Mania, and Laser Dance.
In the context of UX breakthroughs, the potential of MR extends into realms like education and training, where XR experiences can significantly enhance learning and skill acquisition. This immersive learning approach can revolutionize education and professional training, offering realistic simulations and interactive experiences that traditional methods cannot provide.
Voices like Karl Guttag have been influential in shaping the discourse around MR, highlighting its potential and challenges.
During some of our online events surrounding the Apple Vision Pro, where our panel of industry experts delved into the nuances of video passthrough/AR on VR, offering a glimpse into the future of this technology.
The implications of MR extend far beyond the gaming and entertainment industry. In fields like education, healthcare, and retail, MR is opening new avenues for interactive learning, medical training, and shopping experiences. The technology has the power to completely revolutionize how companies and brands interact with their staff and customers, offering profound and memorable experiences for users.
One great example of how the technology is already being used, in this case in the aerospace industry, would be Boeing, which is employing mixed reality to assist with various aspects of its operations. This includes anything from enhancing its training and simulation with MR, to streamlining the maintenance and repair process, or improving operational efficiency through some of the many benefits that mixed reality technology can offer workers – remote assistance for example.
Another example of the use of mixed reality technologies within a global organization would be Lowe's, which is transforming the retail experience for both shoppers and employees with XR. Lowe's has embraced MR to enhance the customer experience and streamline the design and decision-making processes by enabling customers to visualize home improvement projects in their own homes before making a purchase. Furthermore, the company has been using MR for employee training, enabling Lowe's associates to gain a better understanding of the products, thereby enhancing employee knowledge and ultimately improving customer service, as employees are better equipped to assist customers with their queries and project plans.
2024 is set to be a pivotal year for XR technology, and AWE will be at the forefront of showcasing the best the industry has to offer. While the ultimate goal for many in the AR community is the development of optical passthrough AR glasses, the current focus on video passthrough (AR on VR hardware) is a substantial step forward. This approach is not just a stopgap measure but a crucial phase in perfecting user experiences, developing new tools, and familiarizing the public with spatial interactions in a mixed reality context.
Commenting on this, AWE’s co-founder Ori Inbar, says: “The AR community has been clamoring for optical passthrough AR glasses – lightweight, fashionable and with a wide field of view – for decades, but the hardware is still years away. Until then, video passthrough is a great alternative to keep things moving: perfecting user experiences, building tools, and getting people used to this radical new type of spatial interaction in the real world.”
The advancement of mixed reality marks a significant milestone in our technological journey. It's not merely about new gadgets; it's about a new way of experiencing and interacting with our world. The developments that we are seeing in MR today are setting the stage for a future where the digital and physical realms will seamlessly coexist, bringing people together like never before.
As well as bringing people together, AWE brings together the leaders in the fields of MR, VR and AR at its global events. With AWE USA 2024 set to be the largest XR expo in the world, ever, and with 2024 set to pop-off for the XR industry, there has never been a better time to get involved with the AWE community and be a part of this exciting technological revolution.