Microsoft's 2018 Mixed Reality plans show progress toward immersive digital future

Last year Kipman described how our descendants would look at the first century of computing as a peculiar time in history where humans were forced to live in 2D space. He said this brief 100-year span would be a single period in the vastness of time where we humans were entertained, communicated and managed our lives from behind a screen.

Kipman is a futurist. He envisions a connected world where the power of augmented and virtual reality eradicates distance as a barrier to being in one another's presence. He conceives of a world where holographic representations of individuals can be "teleported" anywhere, anytime, to communicate, to explore, to feel holographic environments via granular haptic feedback, and not only see and hear them.

Kipman sees a reality where ones physical self can be in one place and their digital self another. Executive Vice President, Microsoft Artificial Intelligence and Research Group Harry Shum, also a futurist, predicts a future where everyone will have an AI alter ego that represents them in the digital space. Microsoft is developing the technology to make this mixed reality and AI-driven world a reality. In fact, Kipman recently identified the following industry-defining trends for 2018:

  • MR and AI
  • AR and VR
  • Immersive communications.

Mixed Reality and AI

Surgeons collaborate using HoloLens

It's important to note the expectation-setting trends Kipman identified are tangible steps Microsoft has invested in toward the future Kipman envisions. Microsoft CEO Satya Nadella sees MR and AI as the next evolution of computing and is fully invested in their development. Additionally, Windows Mixed Reality as part of Microsoft's device-agnostic Windows 10 OS, is foundational to the company's platform approach to democratizing MR.

Kipman stressed MRs ability to help people transcend space and to enable interaction with digital counterparts. Surgeons from various locations demoed this capability last year as they donned HoloLens' and collaborated during a surgery (above video).

That was admirable but Kipman's vision for intelligent MR required advancements in HoloLens' onboard AI. HoloLens must be capable of comprehensively perceiving and understanding our actions and intent, our environment, our interactions with it and how the digital world it manifests should respond to us. Toward that end last year Microsoft announced it was adding a dedicated AI coprocessor to HoloLens' custom Holographic Processing Unit (HPU).

This AI coprocessor enables native Dynamic Neural Networks (DNN) which approximate how human brains work. This is important to the advancement of computer vision and other recognition tasks required for HoloLens to "understand" its environment. Marc Pollefeys director for science at HoloLens adds:

This is the thinking you need if you're going to develop mixed reality devices that are themselves intelligent. Mixed reality and artificial intelligence represent the future of computing, and we're excited to be advancing this frontier.

Augmented and virtual reality

Microsoft's AR efforts have focused on collaborations with the enterprise, private and education sectors toward developing custom applications for HoloLens. Various industries like healthcare, entertainment, NASA and more have partnered with Microsoft to bring AR to its customers. Furthermore, HoloLens' AR tech, like VR is meant to be emulated and democratized by OEM partners.

Microsoft's VR efforts are more consumer-focused as it continues expanding its VR platform as partners bring VR headsets to market. VR is still a struggling category but Microsoft and Kipman hope the masses can be won to what it is expected to be an AR and VR computing future.

An immersive experience

Immersion is critical to Kipman's vision of screen-less AR and VR computing. He believes a "killer app" for MR will be a social communication app that tears down the walls of isolation. Current communication methods provide 2D "screened" experiences at best. Kipman envisions a MR platform that replaces 2D communication with immersive "virtual presence" sharable experiences.

Kipman shared how in work or life, communication and presence, are the fabric that ties us together. He shared how he wants his daughter, in the U.S., to be able to "physically" play a board game with her cousins in Brazil. That technology is in its early stages today.

Kipman was challenged regarding the HoloLens' limited field of view (FOV) to which responded, "ultimately that's the wrong question." He said,

That's the equivalence of showing someone holograms for the first time, and you then saying, What's the size of your television? The field of view is almost irrelevant, what we should be talking about is the density of light or radiance that shows up.

Still, the "picture frame" FOV is a current technological limitation preventing the fully immersive AR experience Kipman describes. Of course, technology will evolve, and one day AR holograms may engulf us as entirely as VR does in many scenarios.

Collaboration, AR glasses and AltSpaceVR

Kipman's vision for collaboration requires a foundation that connects people in life and business. Microsoft Teams was demonstrated capable of connecting users across different devices and experiences, including AR via HoloLens as users worked on a car (above video). At the other end of the spectrum is Microsoft's investment in AltSpaceVR, a social platform for VR. Both of these tools are being evolved to provide the foundation for robust collaborative AR and VR experiences.

As we move forward, Kipmans vision of smartglasses that transition from opaque to clear to accommodate AR or VR scenarios may become the norm. Furthermore, as more devices become part of the cellular roadmap, and the power of Windows PCs makes it to increasingly non-traditional form factors, perhaps in time Kipmans and Windows Insider Chiefs, Dona Sarkar's, vision of smartglasses as smartphone replacements will become a reality.

Smartglasses as smartphones - your social and privacy concerns won't matter

If in the future smartglasses become the norm, humans will exist in that "screenless" world Kipman envisions where we are surrounded by and can interact with our data and where our digital selves are persistently active on a digital plane. It may sound like fantasy, but the trends Kipman suggest we watch in 2018 and that Microsoft is investing in, are moving in that direction.

If you want to jump on the MR bandwagon now's the time. We'll have to wait and see whether Microsoft can achieve its aims long term.

Save $100 on Samsung's HMD Odyssey

CATEGORIES
Jason Ward

Jason L Ward is a columnist at Windows Central. He provides unique big picture analysis of the complex world of Microsoft. Jason takes the small clues and gives you an insightful big picture perspective through storytelling that you won't find *anywhere* else. Seriously, this dude thinks outside the box. Follow him on Twitter at @JLTechWord. He's doing the "write" thing!

Latest in Virtual Reality
Call of Duty: Modern Warfare II
Call of Duty: Modern Warfare 2 becomes the first 3D holographic gaming experience
Apple Vision Pro, cracked
Apple Intelligence will reportedly ship to the Vision Pro to boost its $3,500 value — as HoloLens marinates in the Microsoft Graveyard before its prime-time
A hand holding the Lenovo Legion Glasses 2 with the Windows Central CES 2025 badge
Lenovo's latest answer to XREAL AR glasses arrives sooner than you think
New XREAL One AR glasses showcasing its augmented capabilities.
The new XREAL AR glasses are what Microsoft HoloLens should've been: on-board spatial computing takes the new 'XREAL AR One' and 'One Pro' to the next level
IVAS headset based on Microsoft HoloLens
$22 billion Microsoft deal with U.S. Army could be in jeopardy unless the tech giant can get the price of its militarized HoloLens to be "substantially less than" $80,000 per unit
Microsoft HoloLens
It's over. Microsoft discontinues HoloLens 2, ending its foray into VR.
Latest in News
Two hands holding a gaming handheld
This new gaming handheld could be perfect for Xbox Cloud and more reasonably priced than my current go-to
Cloud servers
Microsoft has killed "several" data center projects in the U.S. and Europe, according to reports — Microsoft responds (Updated)
Photo of Microsoft's new sign-in page for Xbox.com using the Microsoft Edge browser.
Over one billion users will get a new Microsoft user experience, and it has a dark mode
The Thing: Remastered key art
The Thing comes to Xbox Cloud Gaming's "Stream Your Own Game" library alongside other new arrivals
Promotional screenshot of heroes fighting a giant in Pillars of Eternity
Obsidian's classic Baldur's Gate successor 'Pillars of Eternity' is getting a surprise turn-based mode later this year, alongside other updates
Atomfall
Atomfall reviews and Metacritic scores are in: Here's a roundup of what everyone's saying about this new Game Pass survival game