Skip to main content

With Apple and other rivals in the wings, mixed reality is Microsoft's race to lose

Hololens 2015
Hololens 2015 (Image credit: AP (2015))

Like many in the tech industry, I have been following the evolution of virtual reality (VR), augmented reality (AR), and mixed reality (MR), all now under the rubric of extended reality (XR). Suffice it to say that the entire category has generated far more interest than actual usefulness beyond just gaming (looking at you, VR).

Microsoft has been pushing mixed reality with its HoloLens headset since it was first introduced in 2015. And six years later, there has been a lot of significant progress in the business space with little push into the consumer market.

But Microsoft does have one not-so-secret advantage over potential rivals in MR: it actually exists.

Microsoft is the only company making millions on MR, and now billions. HoloLens is used by companies like Airbus (opens in new tab), Lockheed Martin/NASA, Medivis, Case Western Reserve University, in surgical operating rooms, and now the US Military where Microsoft just landed a massive 22 billion dollar deal that could span a decade.

Hololens Us Army 2021 Microsoft Press

Source: Microsoft (Image credit: Source: Microsoft)

Last month, Microsoft also touted partnerships with OceanX, Niantic, and Lune Rouge for its new Microsoft Mesh technology, which includes holoportation.

Even if Microsoft doesn't get into consumer AR, it will make the tech and platform for other companies.

Why is this a big deal? While there is talk of eventual rivals from Apple, Facebook, Google, and lesser-known ones like Nreal Light, or the issue-prone Magic Leap One, Microsoft is the first to market. And the company is doing quite well, recouping its R&D investment before others have even launched. Microsoft is quite literally years ahead of anyone else.

It's not just about selling headsets, however. Microsoft is now doing the less exciting work of building the MR framework for the future. Its cross-platform dev tools (Azure spatial anchors, remote rendering, object anchors, Mesh) are building the groundwork so MR is more than fancy hardware. Inventing HoloLens is one thing, but making the dev tools and the entire platform is another.

Source: Microsoft (Image credit: Source: Microsoft)

There is also user response. Microsoft is open about how it works with companies on HoloLens and adjusts features, hardware, software, and more based on what companies request from actual usage in the real world. See the capability to add LTE and 5G as one recent example. That feedback loop is something that cannot be replicated or inferred in a lab behind closed doors. Right now, other companies are simply guessing.

The proverbial elephant in the room for Microsoft watchers is if it will push MR to the consumer space, presumably via its Surface hardware division. The question has been asked since HoloLens was revealed with Minecraft back in 2015, but I'm not sure even Microsoft (or anyone) knows the answer today.

Mixed Reality for consumers: Apple, Microsoft, Snapchat and more

Source: Antonio De Rosa (Image credit: Source: Antonio De Rosa)

Right now, MR (and AR) are mostly vague concepts for consumers — the entire category is undefined. It sounds neat but going beyond the cool factor to real-world usefulness is not without its challenges. Like 3D screens for phones, the tech needs to prove itself useful and not just "cool." Google learned the hard way with its Glass headset back in 2013 (talk about being first), but the project has floundered under the newer Google Glass Enterprise Edition 2 launched in 2019, which earns nary any attention from the press or businesses.

When it comes to mixed reality, it is Microsoft's race to lose.

Facebook is now on the radar with its published interest in AR, which is not surprising. But Facebook faces myriad challenges around privacy and, so far, has had little success with consumer hardware that's not Oculus. Snapchat is also reportedly getting into AR, which has thrilled precisely no one (yes, Snapchat is still toying with hardware).

But it is Apple that has tech watchers the most excited. Despite recent reports that any AR system, dubbed 'Apple Glass' by the press, won't be an actual product until 2025, the company is riding on reputation alone to nail the category. That's still at least four years off (it's reportedly not even prototyped yet). Instead, a more realistic "helmet-like" and traditional AR/VR accessory could launch in 2022 as the first phase (with AR contact lenses later on in 2030 as the third phase. No joke.).

But Apple is best positioned to offer a 'distributed' wearable MR system for the consumer market.

I think that Apple is the most poised for consumer success in AR/MR. The idea of a distributed wearable system, which it is supposedly pursuing, is a formidable one. Powered by the potent iPhone as a CPU, Apple Watch as an interface, and AirPods for audio, it is only missing a heads-up-display (HUD) in the form of wearable lenses. No other company has such a unified ecosystem and software stack, not to mention the robust developer network that will undoubtedly follow when commanded.

That said, I don't think Apple will touch the space where Microsoft is going right now. The idea of Apple providing somewhat controversial military gear is anathema to its ethos, and jumping right into surgery or working on NASA's Orion spacecraft headed to Mars years after Microsoft secured contracts seems improbable. This bifurcation could be a similar divvying of the market we see now with PCs where Apple rules 'boutique' prosumers and Microsoft owns enterprise, professional, and everything else.

There are two noteworthy side points I'd like to make, however. We know Microsoft is exploring AR/MR for consumers, or at least making the tech more ready for everyday usage. A patent from 2019 (and going back to 2011) reveals a more realistic version of what a smaller HoloLens could look like if adapted for either civilian or military use.

We also know Microsoft is working on "a boundless FOV (Infinite Field of View)," likely for the next generation of HoloLens, which would be a massive breakthrough in mixed reality technology. Alex Kipman, the head of HoloLens and holographic computing at Microsoft, believes wearable devices could replace physical screens in the future. And despite little interest, Windows Mixed Reality still exists as a virtual Windows environment right now. Windows Mixed Reality is arguably way better than it should be at this stage – it's really good. What other company has virtualized an entire operating system as vast as Windows? Microsoft did it in 2017.

Basically, Microsoft is in very deep with augmented and mixed reality systems both for software and hardware.

HoloLens 2 with phone

Source: Windows Central (Image credit: Source: Windows Central)

The second point is, even if Microsoft doesn't jump into consumer MR/AR quite as hard as Apple, it will, as it always does, play a significant role in powering its tech. See how Azure and Spatial Anchors already work on iOS (opens in new tab). Similar to how Microsoft approaches software on iOS and Android, it is looking to build the services and platforms for AR /MR that other companies will want to use. Microsoft knows it is easier to pay for services and dev tools than create your own, and that's big business for them.

Of course, being first is never a predictor of being the best or the most widely used. Microsoft's early achievements with HoloLens are notable but not necessarily a future indicator. Technological history is replete with examples of a company doing well only to suffer disruption from a rival. That could happen here, too. But while the market dynamics will undoubtedly change in the coming years, right now, it is Microsoft's race to lose.

Daniel Rubino
Daniel Rubino

Daniel Rubino is the Executive Editor of Windows Central, head reviewer, podcast co-host, and analyst. He has been covering Microsoft here since 2007, back when this site was called WMExperts (and later Windows Phone Central). His interests include Windows, Microsoft Surface, laptops, next-gen computing, and arguing with people on the internet.

  • Great article. Really appreciate your writing long pieces like this to chart different visions and the competitive landscape.
  • This scenario will play the same as the mobile one.. Windows Mobile against iOS.
  • Back then Microsoft was very late to the party.
  • Microsoft has released Windows Mobile in early 2000s, much before Apple has released iPhone. Obviously it is hard to say what happens next, but at the moment they have a product with limited mass appeal and are in the status of 'waiting' 'we are leaders' similar to what was before. It doesn't have to end the same, but at this point it is sad that they don't even try to reach some price points like 1000$ or lower that would open the door for the mass market. For the 5 years they failed to provide anything below 3000$ and that's kind a long.
  • "Microsoft has released Windows Mobile in early 2000s, much before Apple has released iPhone."
    True, but there's an important distinction: WM was for business/enterprise use (literally it's Windows CE), not a consumer product (though HTC, others tried to 'skin' it with moderate success) WP7 was the big consumer push with a focus on non-keyboard designs e.g. Samsung Focus. It was 3 years after the iPhone. I'd also argue that MS never took WM seriously. Even by Ballmer's admission, Microsoft was too focused on fixing Windows Vista (dumpster fire) to focus on the NBT, which was mobile. It's one of his biggest regrets at MS (Gates said something similar). I don't think this time we can say Microsoft isn't taking AR/MR seriously. To put it another way, if WM/WP7 ever made a billion dollars (let alone 22B), we'd be in a very different world right now. Microsoft goes where the money is. See Microsoft Teams and Office 365. Where's the no money, there's no more product (Zune, WP, Band, etc.) When your division at MS brings in $22B in one contract it's probably easy to convince the CEO that MS should invest more into R&D and that this category may have a future.
  • "True, but there's an important distinction: WM was for business/enterprise use (literally it's Windows CE), not a consumer product (though HTC, others tried to 'skin' it with moderate success)" How is this different from HoloLens? Or I missed something that you can buy it in retail?
  • My point is by the time Microsoft got serious about mobile/phones it was way too late. Moble CE was already years old/outdated when they began pushing it up against iPhone in 2007, which is why WP7 was a complete reboot - CE and that app model couldn't do things they needed it to up against a modern device like iPhone built from the ground up with a modern OS. Ballmer never took the iPhone threat seriously, laughed it off, didn't invest the company into this new form factor. It found itself reacting instead of leading. Even in its heyday against BB and Treo WM was effectively a side project compared to Windows desktop with little investment. Nor did Microsoft take the lead by building its own hardware (something Ballmer now laments they didn't do). This time, Microsoft is investing a lot into AR/MR, created/invented new technology, and has built this for the next 10/20 years. They're taking this seriously and landing a $22B contract is sure to whet its appetite to push harder. Even at Ignite just a few weeks ago 80% of the keynote was only focused on HoloLens/AR. Had MS had such early success with WM, things would be very different today. tl;dr Windows Mobile = Not a lot of investment; didn't build own HW, didn't take the market seriously. reacted to the changing market/moved slowly MR/AR = A lot of investment, own hardware, takes the category very seriously, creating the market/moving quickly
  • Clearly, Ballmer did not possess a strategic mind and a good understanding of technology. Was he a great business leader? Sure. Just look at the profits during his time as CEO. He was able to survive the Dot.Com bubble bust. But he did not understand the implications of the Dot.Com bubble. Plenty of tech companies did, and they were able to survive the shakeout. Did MSFT have a digital map business? Did MSFT miss Search? Did MSFT miss the mobile phone business? Sure they did. But they were focused on the enterprise and from that Azure was born (a year or so after AWS. But they did build Azure. MSFT has been a good long-term investment for many people. there was just a time between 1998 and 2000 that you should have avoided buying the stock.
  • You are right. If there was a different leader for the consumer space that reported directly to Gates - and could thus avoid being shut down by Ballmer - we could have had a different consumer story for Microsoft. Panos Panay is someone that understands both worlds (consumer and business) and can push his points with the boss (same with the Phil Spencer), thanks to the restructuring at Microsoft (as a result of their defining losses in mobile, search, and social).
  • I'd also argue that perhaps both or either the tech and/or cost isn't there yet for a consumer version to be a real product. There have been other tries not just by Microsoft (ie Google glasses among others) that also hasn't materialized as a more affordable consumer device.
  • Very true. Indeed, rumors around the Apple VR headset coming this year or next is it could be priced at $3,000 and is expected to only sell in low volume to early adopters/devs.
  • Ye, outside of gaming, there is currently little use for the consumer space other than novelty. Movie experiences may be next, but main use will be for actual work, not entertainment, at this point
  • How can they "play the same" when iOS and Android were well established and enjoyed an entrenched development duopoly while MS could never break in? MS is the only, non-vaporware, working and actually sold device. If Apple were to "win" the AR war it will be because they have tight integration with iPhone and the device used as the processing base unit (which actually might be the best solution).
  • Windows Mobile came before iOS and Android.
  • Awesome article and genuinely good research into details.
    Thanks for publishing what I have been thinking and telling myself. MR, AR, VR is a tricky category, I do believe MSFT handled it well going for enterprise and government first, I do believe they will address consumers in time, they have the infrastructures and services ready, plus having dip their toes into android devices making cannot hurt if and when they decide to visit MR consumers market.
  • I truly want a Microsoft ar head set with windows core os to replace my smart phone
  • It is tough to make guesses whether that will happen or won't, but what is sure is that just like smartphones haven't fully replaced laptops, this will even less replace smartphones. I could imagine some (not too many) people deciding not to buy laptop because they have a smartphone, but it is very tough to imagine anyone wearing AR glasses for whole day. It is hard to ever happen and if it does that technology is decades far from today.
  • The issue here is input methods. They don't have a natural, easy, and reliable way to interact with such a device. That will be the breakthrough needed.
  • I'd argue HoloLens 2 does a lot in this area with eye-tracking and the ability to detect finger movement (not just hands). You can now use a swipe keyboard for typing, play a virtual piano, and effectively "grab" objects. There's the 'hand ray' pointer, the ability to touch objects, grab things, gaze, etc. I do agree it's not quite there yet — and, more importantly, not small enough for regular use — but it's impressive stuff.
  • LOL thanks, you just jinxed them 🤦‍♂️ That said, the reason Microsoft will win with Hololens is because they're a longtime trusted defense contractor, while Apple aren't.
  • I'll buy whatever Apple releases - not because I'm an Apple fan, but because I'm a physicist and the fact that Microsoft keeps referring to the visuals in their Hololens as "holograms" really, really irks me (hint: they're not) ... that said, Microsoft *has* been doing some really interesting stuff in this space and I'd like to see them continue - just maybe with a different person in charge of marketing buzzwords.
  • "I'll buy whatever Apple releases - not because I'm an Apple fan, but because I'm a physicist and the fact that Microsoft keeps referring to the visuals in their Hololens as "holograms" really, really irks me "
    Petty, but I respect it lol.
  • Yeah, their marketing team has not always been on top of their game. They seem to be enamoured of the Halo games.
  • Great article. I like what MS is doing in this field.
  • Microsoft's AR/VR disadvantage is having Nadella as CEO. They should be spending way more on R & D in this. WMR was a disrupter with only shoestring budget. No telling where it could be if they had visionary controlling the purse strings. Hololens gets a bit more budget but judging by the donkey-slow output not nearly enough either. I agree about Facebook privacy issues but unfortunately not enough people seem to care about that. Both Apple and Facebook are putting much more money and effort into into it and both leave Microsoft far behind in this
  • "Microsoft's AR/VR disadvantage is having Nadella as CEO. They should be spending way more on R & D in this."
    How much are they spending and how much do you think they should spend? By what metric do you consider it "not enough"?
    "judging by the donkey-slow output not nearly enough either."
    I just wrote an article detailing how much HoloLens is being used in the real world, with real customers, and earning billions of dollars, while everyone else is drawing-board stage losing money on the category (straight investment, zero revenue). Are you suggesting the market for AR/MR is already here but no one is properly capitalizing on it? Because to my knowledge, the HW and SW are years away from being shippable by anyone. The category is new, undefined, with interaction models still being solved (not to mention use cases). Windows 10 itself is already virtual and that was done 4 years ago. Who else is this far ahead in 'holoportation'?
  • Nadella ran Azure from the early beginnings. I think we all understand that XR (I rather like that term) will rely on a huge investment in software and cloud support. I think Nadella understands the cloud and software pretty well. MSFT built its riches on providing the software so people could supply hardware and software applications to meet the needs of many different types of users. IF MSFT does build the operating system and tools to provide solutions for XR, then they will have a huge profit-generating business for several decades.
  • One point I think should be stressed is that we can only see Microsoft's tech, as Apple is following its usual playbook and developing in secret. We have no idea how far Apple is in developing its project. That said, I would be surprised if Apple has a breakthrough in the pocket, but I wouldn't rule the possibility out.
  • "We have no idea how far Apple is in developing its project. "
    Partially true. There's a lot of great reporting on it already with high expectations of a late 2021 announcement, a release for 2022, and a price tag of around $3,000 with low initial sales to developers and early adopters.
  • The breakthrough isn't here yet. Whoever figures out a reliable, natural input method for these devices will come out ahead. We are still in early days and no one has figured this tech out yet.
  • IF you look at the solution for the army, the inputs are already there. They have been testing this stuff at the infantry school at Ft Being for over a year. The HoloLens integrates the gun site, soldier location, maps with way pints, etc. This way a group of soldiers can maneuver and shoot with much greater accuracy and coordination. Will it take time to harden the device of combat conditions? Will it take time to improve the integration of info into the system? Will it take time for the Army to adapt its training manuals and doctrines to incorporate this tech into its operations? Yes to all this and it will take time and lots of money. Money that MSFT will earn. Will this effort translate to other industries? Sure. how and when? don't have a clue.
  • Those is very simple inputs, if it even requires user input at all and isn't just a heads up display. When you have the reliability and input precision of a touchscreen, let alone a keyboard and mouse, then this tech will take off. They need an iPhone moment. A breakthrough that makes using them fun and easy to use. They are frustrating and hard right now. Only useful in very specific cases.
  • Great article. This is MS going to their strengths - building the main "OS" or platform that business uses. Consumer will bring more people's attention, but it isn't what "builds"
  • Hololens is like Tesla, Tesla were the first to really invest in electric cars while the others barely tried and now they are the N1 choice for electric cars, now they are WAY ahead in technology when compared to every other car company and it's very likely that they are going to have a massive market share once electric cars become cheaper then ICE cars and the infrastructure is there, something that should take just 3/4 years.
  • the only problem is that the energy density of a tank of gas is far higher than an equivalent size/weight battery.
  • That's a solvable problem though and it is being solved. New Model 3 long-range gets up to 353 miles on a charge (non-LR = 260 miles). New Mercedes-Benz EQS reporters to be 478-mile range. Those are well below long-range sedans, but as new battery tech comes online it too will be fixed. Americans drive an average of just 30 miles a day (AAA, 2015). I have a home charger and get free charging from 11 PM to 6 AM from a 100% green energy provider. The range argument is moot for me. Having a "gas station" in my garage > having to hit a gas station any day of the week. If I go on a long trip, the computer programs in stops to recharge. Level 3 charging delivers a "full tank" in 20 minutes. While I wait I can go to the bathroom, grab some food/coffee, watch Netflix in my car (free Wi-Fi), YouTube, Hulu, or play one of the many video games with my Xbox controller via the huge display.
  • Microsoft is usually first and usually does the inventions. However, they don't spend as much on research as Apple. They spend on these projects like super-sized kickstarters. Most of their dev teams are pretty small.
    The Surface team. The HoloLens team. When Apple decides to get into a field, they are willing to just pour money and resources into it. They see an innovative concept and then have the resources to get into the field and catch up quickly. So while I love the concept of the HoloLens 2, Microsoft needs to put more time and money into it and make it a consumer item. I've wanted a HoloLens since its first conception!
  • "However, they don't spend as much on research as Apple. They spend on these projects like super-sized kickstarters. Most of their dev teams are pretty small ... When Apple decides to get into a field, they are willing to just pour money and resources into it."
    Curious if you have any data on that? Because any published numbers speak differently on the subject. Granted, we don't know where either company is putting R&D money, but I find your blanket statement a bit of shooting from the hips. From Aug 2020:
    "During the second quarter of this year, Apple spent $4.8 billion on research and development, according to its own financial statement (pdf). That’s 8.0% of its total revenue for the quarter." "Compare that to Microsoft, who spent $5.2 billion in the April-through-June quarter on R&D, which was 13.6% of total revenue for the period. And Google’s parent company Alphabet spent $6.2 billion, a full 15.9% of revenue on R&D."
    Microsoft spent $19.27 billion on R&D during 2020, an increase of 14% compared to 2019. Apple is reported at $18.75 billion for 2020. Unless you can tell me how much money Apple is spending on 'Project X' aka Apple Glass and AR/VR and how much Microsoft is spending on HoloLens, I don't find your statement especially convincing in lieu of overall R&D expenditures between the two companies.
  • Yeah, I think I would need to disagree on that as well. The very idea that Microsoft doesn't spend on research doesn't sit well because there's no way they would have remained in business at this level if they did not do the research. It's like saying Intel doesn't do research because AMD came up to disrupt and win sizeable market share from them. It's just the way the business is. If you are talking about the consumer space then it's more a case of bad bets and internal shifts in focus and management that leads to failures of some of their more ambitious projects which others may be better positioned to exploit to the max Apple had to scrap the HomePod even though they obviously did a lot of research into it, because Google and Amazon had gone so far ahead and Apple were not making the impact they hoped.
  • Look at Google glasses. Everyday people don't want to wear them.
  • That was due to it having a front camera that was "always recording." While sensors are needed I'm not sure you need a video camera to record things as part of AR. Regardless, such things are not stopping Apple, Google, Microsoft, Snapchat, Facebook, and the other half dozen companies from pursuing this tech.
  • Hardware becomes a commodity business. Yes, Apple makes a ton of money selling phones and other hardware. But they are a closed ecosystem. Whereas WinTel is an open system where suppliers make money along the entire supply chain. Will MSFT make HoloLens for a consumer? Maybe but it will be a premium device. MSFT will make the most of its money by providing the service that powers MR/AR/XR. Let the OEM provide the different quality devices for the consumer.
  • Nice article. I do agree that Microsoft will continue to own the military and space (government) business for AR/MR in to the foreseeable future, and also the backend services for much of the market. They will also maintain a healthy lead in the health and business/engineering spaces. It was good that they invested early and heavily in the technologies. I see Facebook using AR/MR to enhance the experience of its core social media and gaming platforms, including for untethered livestreaming, concert attendance, meetings, and so on. For Snapchat, I don't see them going anywhere in the long run. No traction to leverage on in my opinion. Apple will bring in the coolest consumer tech but may not have more than niche (but relatively strong) adoption only within its existing user ecosystem, mainly for media consumption experiences. Since, following your points in the article, their AR/MR device would probably need to be complemented by the iPhone, AirPods and Apple Watch (a costly four-in-one solution) rather than being a standalone device. I still see them making a play for the engineering/business market with a standalone head-mounted device akin to the Hololens. One challenge to my understanding, still remains perception of the lack of a sense of privacy from onlookers and strangers when they know you are wearing a possibly always-recording device in your everyday interactions. Like Google Glass, strangers may not be comfortable with that. It's different in business and military contexts because this kind of a device is put on for a purpose and timeframe, and is taken off or flipped up to engage directly with your audience. A new generation raised on a mature version of this technology would be those that make it normal. (edited, to add the word "costly" to Apple's four-device solution based on the article's direction)