How (and why) one Windows Mixed Reality hater became a believer

For the people who follow my work and my social media posts, you might know me as someone who persistently hates on virtual reality (VR).

Part of my skepticism has been driven by Microsoft's handling of the mobile paradigm, which it famously failed at, burning what little good about it the company had down to the ground. I believe that includes developer relationships, faith in the Universal Windows Platform (UWP), and perhaps most importantly, the destruction of trust among its most hardcore fans, customers who might have forgiven the awkwardness of the first generation of consumer-facing Windows Mixed Reality devices.

Of course, whether or not Windows Mixed Reality becomes a success (and VR in general) remains to be seen. But after extensive time with HoloLens, HP's head mounted display (HMD) (opens in new tab), and Microsoft's new Motion Controllers, I'm starting to believe.

My experience with Windows Mixed Reality

Windows Mixed Reality refers to the entire continuum of VR and augmented reality (AR) experiences on the Windows platform, whether it's AR via a holographic headset like HoloLens, or VR in a fully immersive headset like those offered by Lenovo, HP, and Samsung.

In the near-term, head-mounted displays (HMDs) aren't going to replace laptops or TVs, but they enable compelling experiences that simply aren't possible in other paradigms.

Utilizing "Cliff's House," Windows Mixed Reality's staging area, I was able to view my Movies & TV library on a huge cinema-size screen in a virtual theatre. I was able to snap UWP games, such as GWENT: A Witcher Card Game, side-by-side with other programs, such as GroupMe, Skype, and Microsoft Edge, taking advantage of the spacious environment for a next-level multitasking experience.

I had experienced these things before using Microsoft's HoloLens developer kit, which allows you to position UWP apps as holograms in your existing world. HoloLens has a slightly debilitating letterboxing effect, though, meaning you're fairly restricted when it comes to true augmented immersion.

I had my WMR 'a ha moment' playing Arizona Sunshine.

Instead of simply looking to the left, HoloLens demands that you tilt your entire head to locate the missing holograms. With a Mixed Reality VR HMD, this problem is gone, as the lenses provide full viewing angles.

There are plenty of WMR apps such as HoloTour (opens in new tab) that are "cool," but I doubt I'll use them more than once. I had my WMR "a ha moment" playing Arizona Sunshine (opens in new tab), which is a zombie killin' shoot 'em up from Vertigo Games. Even within the game world, I was still receiving notifications from UWP Twitter and GroupMe, and I even received Xbox achievements for playing the game. The pervasive nature of Windows Mixed Reality really brings the "operating system" experience with you in Mixed Reality, reducing the isolated feeling I had while playing PlayStation VR, which not only blinds you to the outside world, but also stops you from being able to multi-task without removing the headset.

Just got my first ever Mixed Reality Xbox achievements ... I BELIEVE. #ArizonaSunshine— Jez☕ (@JezCorden) November 16, 2017

Multitasking in WMR was is simple as pressing the Start button, just like on a regular Windows PC. Arizona Sunshine UWP went into a suspended mode, allowing me to get back into Cliff's House. When I was finished replying to messages, I jumped straight back into the game, right from where I left off.

Windows Mixed Reality still has problems

Some of the issues I encountered aren't so much problems with Mixed Reality as a virtual OS, but more limitations from today's technology.

WMR is still a fledgling idea, and there are clear usability issues that prevent it from achieving mainstream adoption beyond niche use cases. While HoloLens might be great for specific tasks, such as remote learning, adding rich information to an existing scene, and viewing complex 3D models rather than expensive real-world prototypes, the letterbox effect I mentioned earlier limits its capabilities as a true PC replacement. Microsoft is undoubtedly working to rectify this issue with HoloLens, which hasn't yet seen a hardware update since its initial reveal.

VR-based Windows Mixed Reality headsets eliminate the letterbox effect, but there are other annoyances that put a damper on the experience. While it's awesome being able to have tons of screen space for different windows, not being able to see keyboards and mice is a little annoying for orientation. The motion controller keyboard "shooting" isn't anywhere near as fast or intuitive as touch typing, and I still find voice recognition typing to be clunky, at least with my West Midlands British accent. The next step for WMR is sure to bring your physical desk into the virtual space, too, to take advantage of all that multi-tasking real estate, without limiting your ability to actually interact and input information.

Beyond basic usability issues, there are other systemic problems preventing WMR from being a truly comprehensive or intuitive experience. Not only do you need a powerful computer to utilize it, costing $1,000 or more, you also need to pick up a headset that costs at least $350 with the motion controllers bundled. That's a pretty big ask for anyone outside the early adopter arena, particularly since it's arguably less productive for touch typists and Windows power users versus simply using a regular laptop with plain old regular reality.

Alex Kipman in VR

Windows and Devices Group Technical Fellow Alex Kipman leads the charge for Windows Mixed Reality at Microsoft. (Image credit: Windows Central)

It's also just clunky, with annoying additional cables, batteries for the motion controllers, giving up your hands for motion controllers, and having to wear something on your head. With HP's headset, I found myself struggling with condensation on the lenses, constantly. All of these things make Windows Mixed Reality feel less intuitive to me than just sitting on the sofa with a laptop. It's kind of cool having a full-size cinema screen for Microsoft Movies & TV, but I'm not sure I'm willing to put up with wearing a headset and having to use motion controllers to navigate it.

What about gaming?

Gaming is a different beast, because the Mixed Reality experience there isn't directly comparable to regular games. It's a completely different, immersive experience, and strangely, my time with Arizona Sunshine took me back to the '90s, in a good way.

When I was as kid, there were arcade machines in our local movie theatre called Time Cop and House of the Dead, which had advanced graphics, and a super accurate physical light guns. They were on-rails shooters, with PlayStation 2-level graphics at best, but they offered a different dimension of immersion that mouse or joystick-bound shooters couldn't. There's something visceral about pointing a physical gun, and VR takes that experience to the next level.

Arizona Sunshine on Windows Mixed Reality.

Arizona Sunshine on Windows Mixed Reality.

In Arizona Sunshine, you're not just pointing the gun. You're able to manipulate doors as you would in real life, pick up objects and throw them, and traverse the environment at will, as opposed to being stuck on rails. Arizona Sunshine might not be the prettiest game in the world, with dated-looking animations and environments, but the sheer potential is clear.

There are only a handful of decent UWP-based VR games as of writing, with dozens more as part of the SteamVR bridge, but it's exciting to think what the future could hold for this paradigm.

An uphill battle

Microsoft is faced with stiff competition from Apple's mobile-facing ARKit when it comes to building HoloLens-like AR apps, with competition from other gaming storefronts when it comes to support for Microsoft's own Windows Store.

The path to proliferation is going to be a difficult one for Microsoft.

With Windows 10 Mobile shut down, support for native Microsoft Store apps seems to have cratered along with it, with engagement on PC being far lower than that of a phone. Apps ported to the store via Win32 or Electron don't work properly in Windows Mixed Reality, opening instead in a dedicated "Desktop" view.

VR is a paradigm where native apps could function better than desktop PC websites, designed primarily for mice, but I'm not sure how willing developers will be to jump on this ship, considering the install base is going to be miniscule even when compared to that of Windows 10 Mobile. Progressive web apps could help fill the gap, but again, it could be too late for Microsoft, which already seems to be lagging behind iOS and Android when it comes to defining AR app support.

The path to proliferation is going to be a difficult one for Microsoft, but the company seems to understand what it needs to do to ensure it's not left out of yet another computing ecosystem.

I'm no longer a Windows Mixed Reality sceptic, but Microsoft may have a hard job convincing other consumers, and more importantly, developers that it is the right company to back and to usher in this future.

Jez Corden
Co-Managing Editor

Jez Corden a Managing Editor at Windows Central, focusing primarily on all things Xbox and gaming. Jez is known for breaking exclusive news and analysis as relates to the Microsoft ecosystem while being powered by caffeine. Follow on Twitter @JezCorden and listen to his Xbox Two podcast, all about, you guessed it, Xbox!

  • Considering Microsoft's track record at launching and cancelling products, I would not support their platform . 
  • I'd just switch to Apple and Android then and be done with it. No need to follow Windows or Xbox news anymore. Also, if you think Microsoft's track record is any higher than Apple or Google for cancelling services or products, you simply don't read or follow enough tech news.
  • That is so true and so frustrating. Many modern tech companies seem to alway be chasing "the next thing" and when that doesn't pan out in 5 minutes, they just give up. They seem to rarely have a long term stategy.
  • Giving up MS is not so easy though Dan. Whatever the sensible thing to do is, it is too easy to fall in love with the possibilities despite Nadella, only to get another beating when he gets fed up with us again. Remember, we are no longer fans, we are victims and will react as such. Applying the logic from the good old days won't work anymore Dan. We're convinced we can change MS if only we could be better customers for them. It's just we have to keep our phones covered up in public in case anyone sees...
  • While it's true that Apple and Google cancel products and services, I think you're hearing a reaction of extreme angst from the Microsoft apostles.  I was one of those.  I was all-in with Microsoft with Bands, Surfaces, Groove, Office 365, Lumias, Xboxes, you name it.  If it had a multi-colored Window, it meant quality and integration into the ecosystem I enjoyed.  The difference with Microsoft is they abandon key components which are essential to the ecosystem.
  • What "key components" have they abandoned? 
  • Notable mentions of the list: Google Reader, Google Buzz, Orkut. And let's not forget about the many reboots of their messaging service, going from Google Talk to Google Hangouts and now to Google Allo and Google Duo.
  • Those combined aren't nearly as bad as the mobile history within Microsoft. Those are pieces of software that you can find anywhere. Microsoft, on the other hand, drops the ball on complete platforms at once. It's one thing to lose out on messasing, a mostly free service offered by loads of companies. It's another entirely to miss out on mobile, where you then have the hooks to suck people into your various services. Mobile's why Cortana will have issues with growth, even though it's generally been the most feature-complete of the digital assistants, for example. Microsoft missed on markets, not just services (which they missed on as well). That includes media players, the beginning of the mobile app gap. It includes motion gaming. It includes mobile. They flopped the start of this console generation, which dug them a massive hole and is costing them in software in that market now. They are now joining VR late, and they're doing it with an app store that has been thrashed about by the competition's marketplaces. They at least didn't take AS long to get into VR, and they were smart enough to latch on to Steam quickly, but being late to this party just pairs with late to PC gaming too well. They've gotta push forward in a HURRY because they're not quite at the table with a reason to move from Rift or Vive, which will just match or exceed Microsoft's moves when they iterate next (basically what really hurt WP8).
  • Maybe WebVR is a path to proliferation rather than UWP apps. WebVR covers a lot of mainstream usages and you can deploy easily (the web), there is no user friction and best of all the content works everywhere (Android for example). With WebAssembly shipping on all browsers it's only a matter of time before we see advanced gaming in the web browser. That's also why Edge has a great WebVR support (inside the virtual house but on the desktop as well).
  • Yeah you could be right there, I totally missed the WebVR angle. Would be cool if the store had some WebVR website suggestions.
  • Take that second paragraph, the general attitude that Microsoft is a failure, add in the influx of trolls fueled by this attitude, and that is why it has been a long time since I last commented here, why I visit once or twice a day rather than every 1/2 hour, and why it has dropped on my list of favorite sites. And the new layout doesn’t help much either.
  • k
  • I do agree on that last point, the layout is now a cluttered mess. I dont' know what is new or old or care to jump left and right and up and down. I exclusively go to the /blog portoin of the site because it's such a straightforward, clean means of getting the whole of the reporting in a streamlined interface.
  • I mean, that's why we created /blog. Traffic has been up since we altered the layout, so we're serving the wider internet a lot better than we did previously.
  • "arguably less productive for touch typists" Um, if you are a touch typist, then grab a real keyboard, as you don't have to look at it to type.  I push fairly high wpm numbers, and using VR at a desk is 'more productive' while typing by having a large number of windows open in a big open space.     Also finding the mouse next the keyboard is not hard if you are a typist and have gotten used to never looking down. In contrast, I can see how it would affect non-touch typists that rely on looking at the keyboard.  (Ironically a lot of typists look at the keyboard based on a lack of confidence when they could be pushed to stop looking down.  I know one VR user that thought this would be a major issue for them and having them come back after realizing they really didn't need to look at the keyboard when forced to try.) PS Maybe it isn't such a bad idea for people to pick up touch typing, which has gotten to be less common.  It is a base skill that can be taught in a couple of hours. Even grab a typing book from the 70s and sit down for a couple of hours and you will be a touch typist.  Having familiarity with the keyboard will also make the speed of your typing increase really fast compared to people learning touch typing back before they had experience with a keyboard.
  • lol fumbling around blind across a desk as cluttered with paperwork and stuff like 99% of the offices I've worked in isn't productive, they just need to bring a 3D model of the keyboard into WMR like they did with the motion controllers, simple.
  • Then you have much bigger issues.  I keep a lean office, and I'm a touch typist, so since my keyboard is always in the same place, that won't be a problem.  I don't like the idea of using controllers, that's my issue.  I don't game on Xbox because of that, and the only value MR/VR would be to me would be purely for the visuals.  I'll stick to keyboard, mouse, joystick.  Now, having said that, if they come up with untethered GLOVES that will provide virtual handling with decent precision, THEN we're talking a different game.
  • Those gloves are something Iam looking forward to. I saw some prototypes for oculus, but I don't know if there has been any progress made or if it hit a dead end. Hopefully it will make it to market. This would create beautiful immersion eveb for typing :-)
  • You say "simple," but only Logitech has worked towards it, and it isn't cheap. You kind of have it with the Vivei n the future, but it's also not cheap. Microsoft would need to partner with someone or get back to the Sidewinder line, IMO, to make this happen. You'd probably be throwing $200 at a keyboard and interface, I'd guess.
  • What papers lol? I don't get why so many people still use so many stickers etc when you can have all that in digital form like one note that can sync across devices and os... My desk is perfectly clean so such people think I actually must be doing nothing :-D I understand it's not that easy to adapt this technology by everyone, but I feel I will love this VR WINDOWS a lot for work. I am planning to grab some mixed reality headset as I see a huge potential for work in it. Hopefully samsung will bring odyssey to eu soon.
  • Microsoft is undoubtedly working to rectify this issue with HoloLens, which hasn't yet seen a hardware update since its initial reveal
    I'm almost sure they said the limited FOV was a choise to keep down costs and have a better battery life.
  • In conversations with people working on HoloLens, it is their goal to remove the limited FoV.
  • I read somewhere recently the Microsoft received a patent for a method which willl double the FoV of Hololens. I have no idea if it will be a practical solution they can implement
  • The limited FOV has nothing to do with battery life, as it's a problem of the physical optics used. In a see-through HMD, you can't use the same kind of wide-angle distorting lenses that are used in regular VR headsets. A larger field of view would result in a much heavier headset, and more costly, but would not directly affect battery life.
  • Are you sure? I remember reading somewhere that MS had larger FOV, but that the battery life was drastically reduced. Like just 30 minutes or something like that. Still Hololens has a long way to go.
    It was interesting experience, but all felt like really a prototype which it still is. But the idea is great :-)
  • I'm not "sure" in the sense of having inside information on HoloLens, but I'm "sure" in the sense of having worked with VIrtual Reality for 15 years, and no see through headset has had a wide field of view for the reasons I said, even those tethered to PCs and thus not reliant on batteries. Every VR headset with a reasonably wide field of view (like Oculus, Vive, Windows MR, etc) uses a lens that heavily distorts the image in order to get such a view. But in a see through headset, the real world is also seen through whatever optics are in front of the eyes, so you cannot have any distortion, or the real world looks wrong. This severely limits how wide your field of view can be without some new kind of optical technology unused in any HMD so far. Indirectly, if you had such a tech, it would likely require more pixels on screen and thus more GPU power, which would affect battery life. However, the optics problem is still the main issue.
  • Another great article Jez, It highlights what I have been saying endlessly - by disparaging their own UWP platfrom Microsoft has hobbled the foundation which would have enabled Mixed reality to thrive. They really messed up the transition stage to a point it's become undescribable and not in a good way. By neglecting smartphones (windows phone 8 / windows mobile 10) they have cut off any transitional element of UWP to Mixed reality.  Microsoft could have used these UWP apps and branched using smartphones as the HMD like some OEMs - this would have provided the transitional frame work for UWAs geared and designed for WMR. But the current leadership appears to lack any form of transitional foresight and reeks of absolute risk aversion, I would not include xbox in that statement as everything they have done has been pretty much spot on. This also shows by solely focusing on analytics and telemetry, you lose the true focus of a product - addressing needs, solving issues, creating new markets for people - not your shareholders.
  • Yeah, they are probably just now in a position to feel the pain of their behavior with WMR. I really like WMR. The one thing I hate is that it's attached to a company that kicks me in the teeth every time I buy into its platform (Kinect, Zune, mobile). They've done such a poor job of proving they'll support a product that it's hard to commit $400-500 to WMR when I can already get a semi-mature, supported platform in the Vive for $350...with free games attached. The inside-out tracking is awesome, but slightly limiting, but I would gladly take the trade-off for the convenience. They've done everything right with this platform, but they did so much wrong with so many other platforms that it feels like accepting WMR is a sign of Stockholm Syndrome. Plus, the delayed ASUS headset is a big letdown. I wanted that one for sure. I'm not excited for any of the others (Odyssey doesn't like glasses, can't stand HP or Lenovo as companies, Acer's looks like cheap junk, and Dell's would be perfect if it came in black, but the aesthetic is a major turn-off to me).
  • It's that trust issue which will hurt sales amongst the a fair amount of pro-sumers and early adopters, it's usually the early adopters that enable products to proceed to a second generation of that product. In regards to WMR can provide a segway to other forms of technology but that would require quantum computing to be ubquitious like x86/x64/ARM computing is now. Along with profound leaps in battery and wearable technology but that is here nor there at the moment so the focus should be getting more UWA's and not ios or android if they are to regain back the trust of pro-sumers and early adopters. Microsoft really needs a dedicated mobile play if they are to provide a compelling foundation for WMR and windows on arm depending on the implementation can provide that - however that also depends how Microsoft sells and markets the product. They better not lock it down to the US and select markets only - if they do it will be D.O.A. Right WMR userbase is a fledgling compared to Apple's ARkit as that is using current tech to transition users to AR/VR.
  • Well jezz. You got one thing right money. I just purchased an Xbox one x. Until this tech works with it why should I invest? I have a pc it's in specs. But if I buy now what gurentee do I have that headset will work on xbox if and when M.R comes? I don't want to be burnt and need to purchase two Headsets.
  • You don't have any guarantes. if that is important to you, then wait. 
  • Wow, my favorite part of the whole review was paragraph #2. The damage MS has done to a once passionate user base is astounding. I'll always be an MS Fanboy, and will try to look forward to what is next, but man oh man brutal is the beating the fanbase has endured.
  • I won't believe Jez and Mr. Rubino have made up until I see them kiss. XD
  • you look at people sticking their head into these things to game and it reminds you of virtual boy all those years ago
  • Anecdotally, I can report that once people try the Samsung HMD or HoloLens, they come away impressed, even if they are not normally a pro-Microsoft person. I am cautiously optimistic versus other areas like Windows Phone, where people were predisposed to not like it.
  • It will be about money. Consumers won't buy it. Because they will see it as expensive toy. I can see a huge potential for work.
    Build new version of cliff house - office - let people to customise it. Then let them join each other in selected office. Each person could attach their screen and share it with others... Let them interact with shared screen. Huge potential for people working from home or if you need to consult with a colleague located eg in Japan? It would not be popular over night, but this is a super strong tool for work and content consumption for the future I think. But it will need marketing, demonstration of use cases etc... MS could fail in this regard.
    Also as was stated if they focus solely on US it will fail. You need all tech nerds around the world to love your tech so the word spreads and it will slowly start to interest common consumers...
  • I jumped into the WMR world mostly just to see what the VR fuss was about, and I have to say it's really fun and engaging. It's probably best suited for early adopter types and those willing to put up with a few minor snafus, but now that SteamVR is fully operational it's opened up a lot of additional content that doesn't rely solely on the Windows Store. Maybe the novelty will wear off, but for now it's the most fun I've had in years.
  • I have been bashing Microsoft for Windows 8, Windows Phone, etc. in the past and I will in the future for other things that I don't like. But honestly Windows Mixed Reality to me is a good idea and a good product. Virtual Reality seems to be a train that never leaves the station. To me Microsoft came out with something that is able to make that train finally leave the station and get some speed. Here: Super-easy to set up thanks to the inside-out cameras and the integration into Windows 10. No external boxes, just plug the two connectors, do an easy configuration and that's it.Which means it caters to the average non-tech savant people. Runs existing VR apps/games thanks to Steam VR support. Which means that all the Steam customers (gamers!) are potential customers) Price is going down towards a "commodity" level: I have seen the Lenovo Explorer + controllers offered at 299 USD, and I expect prices to go down further. Which means you can buy as a gift for Christmas or get one for yourself just because it's cool. I have tested a sample of the Lenovo Explorer for 2 weeks and I think I will get a VR headset some time soon. Which in itself is already a victory for MR.
  • Actually you don't need to spend anywhere close to 1000 USD for a VR capable PC anymore. All you need is a 1060 and an i5 and you are good to go. 400 USD should be enough for the PC. Adding the cost of the headset and controllers it means that for 700 USD you can get a decent VR experience.