How Microsoft's mixed reality strategy may augment its Home Hub and IoT visions

Now a visit to a local Microsoft store can give you access to a world where alien robots burst through walls. Well, that's as long as you've donned Microsoft's HoloLens. The presence of digital elements in our physical world is a reality.

Microsoft's goal, as seen in the company's vision of a scuba diver's augmented POV in the video below, is to provide a platform where holographic computing is applied in a variety of scenarios.

Redmond envisions a future where fluid interaction with digital elements that populate our world, augment our perceptions and provide additional information about our environment is the norm.

First things first: Holo-foundation

HoloLens is Microsoft's untethered wearable computer that showcases the augmented-reality extreme of Redmond's Windows Holographic mixed-reality vision.

Microsoft envisions a future where AR is the norm.

Head Mounted Displays (HMD's) like those coming from Acer, Asus HP, Dell and Lenovo will showcase the Virtual Reality extreme of that vision. VR opposed to AR's overlaying of digital artifacts onto the real world, immerses a user in a virtual environment.

Microsoft's Windows Holographic platform (as part of Windows 10) is being positioned to mainstream AR and VR. Redmond is working to democratize the technology so that everyone has access to it. Affordable AR glasses, I believe, are the long-term goal.

Ambient computing, interfaces in our faces

To understand Microsoft's augmented reality goal, we must consider their ambient computing goal. Ambient computing refers to computing that is part of our environment and not restricted to a localized device such as a desktop PC. Static sedentary computing is personal computing's past.

I expounded on Microsoft's vison of a device-less future in October 2015:

The future of mobile computing is "device-less… The company that has a pervasive presence across all platforms …and supports hardware and software that seamlessly transitions with a user's cloud-based computing needs, will be positioned ahead of the curve…Microsoft is forging a foundation for tomorrow's computing experience.

Microsoft's CEO Satya Nadella has been reinforcing his mobility of experiences vision since he took the helm in 2014. The mobility of a user's digital experiences, not the mobility of our devices is the focus. You are the hub. The Cloud, as an intelligent omnipresent backend, powers these experiences and conveys them across devices. Microsoft's cloud-based services, Universal Windows Platform and Windows 10 family of devices are foundational to this vision. They, however, are not the totality of the mobility of experiences vision.

The Cloud, UWP and Windows 10 devices are just the foundation of Microsoft's mobile strategy.

Microsoft's platform company, "do more" vision, includes every medium through which a user's digital experiences will travel. Thus, a countless array of Internet of Things devices, which Microsoft is determined to infuse with intelligence, are also part of Redmond's mobile strategy.

Connected devices are expected to reach 34 billion by 2020. Microsoft hopes its intelligent cloud, Cortana and bots will be the platform for the majority of those devices. In time, IoT devices may permeate our environment and "know" us at work, in public, and at home. This is Microsoft's (and its rival's) goal.

Seeing the screen-less interface

Microsoft's Home Hub strategy will utilize low-cost ARM-based devices with screens that give us a visual interface to communicate with the IoT:

There will, however, also be millions of devices in our general environment and many in our homes that will be screen-less. Voice will be one way to interact with these devices as Microsoft's investment's in far-field communication indicates. Vision and motion I believe will be another.

AR glasses may allow us to see digital interfaces for screen-less devices.

I believe augmented reality will enable us to "see" digital interfaces and the data associated with screen-less devices. It's conceivable that companies like Microsoft envision a world where the IoT will have both a physical and digital make-up. Not unlike a human being can have a physical body and an invisible soul. This digital component would be visible with AR glasses (that would connect to devices) and could be interacted with via gestures like those we currently see employed with HoloLens.

In the video below London designer, Keiichi Matsuda, explains a vision of AR that helps us visualize this concept.

I contend that Microsoft's IoT and augmented reality visions are not independent of one another. I believe Redmond's goal is that holographic computing will supplement the ambient computing platform IoT may make ubiquitous.

Microsoft shares their vision

Microsoft has produced several videos over the years that provide inspiring imagery of incredible technological vision. At the time of the videos releases most of their content seemed like science fiction. I recently noted, however, that the videos foreshadowed technology that ultimately became a reality.

Microsoft's AR/IoT vision will broaden the scope of managing digital experiences.

Continuum, OS-level digital inking, real-time language translation, 3D computing, computer vision and more were once mere concepts in Microsoft's future vision videos. They are all now real.

The following future vision video which shows a woman using AR glasses (0:01 mark) and a young girl interacting with holographic images at home (4:23 mark) may hint at Microsoft's IoT/AR computing vision.

The realization of this vision is years away, but its foundation is being laid by the industry's fervent embrace of AR and a battle for IoT dominance among the likes of Google, Microsoft, and Samsung.

An all-encompassing ambient computing environment that will be a composite of physical and digital structures is before us. AI that will recognize and "perceive" us via those intelligent devices may work in concert with AR glasses that will allow us to perceive and give us the option to interact with the "digital extensions" and "digital interfaces" of those devices.

Mobile is bigger than we think

Microsoft's IoT, Home Hub and AR visions are components of its mobility of experiences vision. Redmond's vision of facilitating a user's digital experience goes beyond the PC, phone, HoloLens and other members of the Windows 10 device family. The IoT computers that will permeate our environment and help us "do more" at work and play are part of Microsoft's mobile vision.

If Microsoft is successful, rather than a hodge-podge of disparate devices, its single cloud-based intelligent platform will be a singular intelligence linking a bulk of the IoT.

What might that content-saturated digital world look like through augmented reality glasses?

Jason L Ward is a columnist at Windows Central. He provides unique big picture analysis of the complex world of Microsoft. Jason takes the small clues and gives you an insightful big picture perspective through storytelling that you won't find *anywhere* else. Seriously, this dude thinks outside the box. Follow him on Twitter at @JLTechWord. He's doing the "write" thing!

  • Thanks for reading folks! One thing to keep in mind is that Microsoft wants their technologies to work together not independently. Thus the mobility off experiences vision much naturally include a users interactions with intelligence infused IoT devices. Moreover, Redmond's goal to democratize AR and herald the age of holographic computing must also be included in this ambient computing conversation. As computing becomes less centralized different UI modalities become increasingly important. In addition to voice believe that AR will affect how we interact with and perceive IoT devices. You know the drill...LET'S TALK!!!
  • I think Microsoft should be more aggressive in putting VR devices in consumer's hand and establish it as a standard. HoloLens have been around for a while now. Yet the device is available in selected markets only as a development kit costing silly money.
    I sense that Apple will step in and 're-invent' the concept/standard for VR. We all know what will happen next.
    MS, cannot afford to miss on another opportunity.
  • This is why Microsoft is making it part of Windows and working with partners (as well as licensing Windows Holographic SKU), their reach can be much broader. Though i kinda hope that Microsoft will be investing more into this platform than right now, they need far more people to work on Windows Holographic and just that, especially adding more default gestures and more intuitive ways to interact with objects.
  • Based on what I have read, I don't think the dev version of Hololens is what should be released to consumers. I think the strategy of using Windows as a platform for other vendors will strengthen the ecosystem and give Microsoft valuable input as to what works from a VR perspective. And opening up so that there really is support for both AR, VR and everything in between is smart. As for the first version of Hololens for consumers, my guess is that it will run on a Snapdragon 835. That makes so much more sense in terms of both raw power and power efficiency. The dev version doesn't have awesome battery life and more raw power is always good. Other than that, we know, from the great work done by Windows Central and others, that Microsoft is working on expanding the field of view, which it sounds like is necessary. Finally there are some other complementary stuff which I think should be addressed, one being the question of having a controller or not. And then there is the question of external sensors, much like the HTC Vive, which would be necessary for holoportation. And then there's the whole thing about the requirement for a server or something to be able to even run the simultaneous experiences we've seen on video. They're touting it as "untethered" but that doesn't mean fully independent for all scenarios. As for when we will see a consumer ready Hololens is anyone's guess. There are rumors about a big batch of hardware coming in the spring. But honestly, I'd be surprised if we see it out in the market until around RS3, expected in the fall, at the very earliest. We might see it demoed before then, though. And we might even "just" get an updated dev version before we see the "final" or first consumer version.
  • Good article, just a little detail: “Head Mounted Displays (HMD's) like those coming from Acer, Asus HP, Dell and Lenovo will showcase the Virtual Reality extreme of that vision.” Chinese leading VR HMD manufacturer 3Glasses officially joined as the 6th partner during WinHEC 2016 keynote (​ at 16:05).
    This is quite important as it means more OEMs are joining the Windows Holographic platform.
  • Looks like Neo Tokyo.  People should be driving rad motorcycles with fat tires through the streets.
  • So each device will have a physical body and an AR soul?
  • all this look nice. but as I search about ces all i see is alexa, alexa and alexa. I don't know if she's that good, but seems like amazon is taking the lead on IoT
  • Yes Amazon has been very agressive with building key partnerships.
  • I mean, the LG fridge we saw with windows 10 now it has Alexa. Wtf we need Cortana. What do you think is the right move to reverse things
  • For Microsoft to stop playing, moving about lackadaisical, and get aggressive about it's position, products, and services. A ceo of one of ms acquired companies spoke on the risk of ms becoming irrelevant due to it's slow moving nature and received hate mail from long term employees saying "respect the legacy". That ceo was 100% right and looking from a perspective that's not inline with Ms' internal one. Alexa is now synonymous with voice/digital assistant just as apple's ipad is with "tablet" because ms refuses to take themselves seriously. If they don't, why would anyone else?
  • It is quite amazing to me, that out of the 4 companies, Amazon/Alexa has emerged as the early front-runner (at least in mind share & partnerships).  Compared to Microsoft, Apple, & Google; Amazon started with the smallest installed user base.  By making the cost of their devices so low, near impulse buy territory, they have built themselves into a force in the personal assistant space. I was really expecting to see Cortana as the "assistant" being used by the vast majority of OEM's at CES.  I feel that this was Microsoft's only chance to compete & win in this space.  I know it's early still but I feel like Microsoft may have already missed the boat, again.    
  • I agree with you. Not even google assistant isn't around so much at ces
  • it's because amazon is a big store, alexa is yet another amazon vending machine.
  • I don't know dude, I haven't even tried Alexa. Is she that good?
  • You're def not wrong. Forget that, "it's still early" claim. You have to work to break that lead if you're behind. We, people like us who visits sites such as these, don't see it so I know the general consumer doesn't. Has the Xbox been working harder than the Ps team after the failed launch? Objectively, yes! Ps is still miles beyond the Xbox in terms of numbers.
  • It's a scary but exciting future! Hopefully humanity doesn't take it as far as the "Nosedive" episode of the sci-fi series "Black Mirror".
  • I would reply but you appear to be below a 3.5 and therefore do not deserve a response...Just kidding.  I'm sure IRL you're above a 4.2... Seriously though, I agree with the theme of that series (Technology could ruin Humanity) but I found myself wanting almost all of the tech showcased.  Those AR contacts!!  Great series! 
  • @Jason Thanks for all the articles in the past year! Welcome to 2017, may it be prosperous and joyfull.
    Good to see you are starting right of in the new year. But, I have yet to read it ;-) .  
  • Thanks Joscelin, Happy New Year to you too! 🙂 Hope you enjoy the read!😉
  • Nice recap connecting the pieces, as always.
    ​Looking forward to that cellular, ARM, pockatable device that includes 360 degrees image capture, still and video. Also being able to scan the environment like Google Tango, giving us 3D to share with others, for them to have a VR tour in. :-)
  • Asus zenfone AR just kill hololens
  • AR on phones are purely Gimmick. Looks like children seeing magic thru peepholes in toys. AR will be good only on wEYEables(wareables).
  • Hahaha good one! Oh, that wasn’t a joke to make fun of ZenFone? You seriously believe video-feed-composed AR through a tiny 2D screen kills volumetric AR in your direct field of view? Wow, talk about brand loyalty blinding your judgement!
  • Till now, did you see people around you using hololens, didn't you?
  • Didn't see anyone with a ZenFone either, what's your point? Saying a bike is more advanced than a spaceship, taking for proof that more people own a bike? Doesn't really make sense...
    Future will be light glasses-like see-through devices, not phones or VR HMD with video feedback. HoloLens isn't there yet, but at least is on the correct path. ​If you want an Adroid-based device that can compete with HoloLens, you should at least talk about the ODG R-9, not some phone that can do VR "and MR as well because it has a depth camera" with a plastic box to strap it on your face.
  • Till Now, did you see people around you using hololens, didn't you?
  • HoloLens in my opinion is the ultimate AR platform, however the only trouble is that it might take forever for it to reach market. I just hope MS know what they are doing
  • Great articles. The videos are amazing. One day!
  • Reading this, I perceive that AR has a distinct advantage. Regardless of manufacturer, as long as there is an API/SDK you can create a visual interface to any piece of technology. Thus Microsoft do not need to own every device or even form allegiances with manufacturers, just take advantage of the API/SDK via the cloud and deliver an integrated experience. Register the device with your network/hub, put on your AR glasses and it should become visible....
  • A big example of this concept of interfaces to screenless intelligent devices through AR is the robot demo back in one of the early Hololens presentations.