Skip to main content

Microsoft HoloLens augments Xbox One @ E3 2015: Speculation, hopes, dreams

On April 29, Microsoft blew up the internet for a second time with elaborate live demos of HoloLens

Microsoft's Build conferences are typically focused on developers, companies and commercial applications of their products, but team Redmond often announce new consumer experiences there as well. The popular Unity 3D engine is a confirmed partner for HoloLens, which powers a vast amount of games with its cross-platform publishing capabilities. Those capabilities are coming to HoloLens.

The HoloLens developer website (opens in new tab) has been updated, proudly sporting the fact that the device, and more information on HoloLens games development, will be arriving at the world largest games expo - E3 2015 on June 16th.

It's been known for a while that the Xbox team is involved with HoloLens. A HoloLens studio head's LinkedIn (opens in new tab) found its way onto Reddit following the initial reveal, and within, it explicitly states that he's part of a team working on HoloLens Xbox experiences.

...I Lead 4 amazing experience/game studios within XBOX that create amazing consumer experiences for Microsoft Hololens Manage an organization of over 125 rockstars that want to change the world every single day...

This info rests on the back of various other patents and leaks dating back all the way to 2012 and beyond, detailing numerous AR experiences in lengthy descriptions as well as frequently hilarious patent doodles. The technology for gaming on HoloLens is available within Microsoft; Kinect gesture tracking; the Xbox Live platform; close relationships with hundreds, if not thousands of games developers. All that remains is to tie all these elements together into a cohesive, gaming whole.

Could we soon be doing battle with evil augmented reality bananas? Time will tell.

AR vs. VR

If powerful enough, HoloLens could supersede VR in important ways. Unlike VR devices like Oculus and Project Morpheus, AR doesn't blind the player to the surrounding environment, meaning you're persistently oriented to the real world, eliminating motion sickness. As mentioned earlier, and unlike some of the major VR devices in development today, HoloLens doesn't need to be tethered to a PC or console in order to work - which creates versatility.

I honestly feel that the teams at HTC Vive and Oculus Rift may be wondering if they got it wrong, the applications for VR are likely limited. I tried Oculus Rift at Gamescom 2014, and whilst the experience was vivid, I can't imagine I'd ever use it for long periods of time in any gaming scenario. The prototype headsets are bulky, and there's that old motion sickness problem - both of which I'm sure will be amended in future iterations but they're significant hurdles the augmented HoloLens has already solved.

VR is limited to the first person perspective, and requires the use of another peripheral to traverse flat plains. What inputs are available to VR headsets besides game controllers? There are some solutions, Leap Motion has partnered up with Oculus Rift to provide HoloLens-like air-touch technology to the headset. However, unless Oculus can leverage Leap Motion and similar technologies directly into the headset, the idea of needing separate devices to achieve that functionality already feels dated, thanks to HoloLens.

I believe Sony's Project Morpheus is in a far stronger position in a gaming context. Sony already have powerful technologies in-house to create experiences that are similar to HoloLens, albeit across several different devices working together. Sony already have strong relationships with games developers, in addition to running their own big budget internal studios. Most of the games that utilize the Playstation Camera did so using augmented reality, albeit via your TV, and the under-rated Playstation Move peripherals will allow Morpheus owners to interact in a virtualised 3D space more naturally than using a controller. Whilst Sony can likely create a more cohesive than a group of separate companies, the requirement to use external devices will create problems for adoption.

HoloLens likely won't put a 3D stereoscopic video game experience in-front of your eyes, but it doesn't have to. The VR headsets in development for gaming will likely be too expensive for what they will provide - the ability to move your head independently of a game's camera controls to view a first person game world. HoloLens could augment any game, whether its providing extra elements like HUD data or providing air gestures, its vastly more versatile than Oculus' first-person-only form factor. The limitations of VR put it at a real danger of being confined to a small subset of games and apps at home, like Kinect and Playstation Move.

HoloLens on the other hand needn't worry about these problems. The headset operates independently of any other device, inputs and all, making its versatility unmatched. Developers needn't build entire games into HoloLens (although there's no reason why they can't, thanks to Unity 3D, on-board spatial awareness tech and universal app support), but instead they could just add small augmentations to existing games you play via your TV. When it comes to value, gamers can trust that their HoloLens will receive functionality beyond gaming thanks to the Windows 10 universal app platform, already creating confidence any new platform needs to succeed, gaming or not.

Games in an augmented reality

The applications for HoloLens in gaming are fairly obvious. At Build, Microsoft showed off cartoon robots and holographic dogs roaming around the room utilizing HoloLens' spatial awareness capabilities. HoloLens can detect floors, walls and surfaces like tables, and you can anchor holograms directly onto these real-world platforms.

Kids' gaming is a blatant example. Sony themselves launched the 'Wonderbook' AR experience for their Playstation Camera, which feeds data from a physical book directly into the camera, which then outputs effects on your TV in kind. The hardcore audience mocked it, but Playstation Wonderbook enjoys a 4.5/5 rating (opens in new tab) on Amazon UK across 101 reviews, showing that there is certainly an audience for these targeted experiences.

On the third party end, the robot Microsoft displayed to show off the holographic anchoring could be used to power desirable (and expensive) kids toys. Sorry parents. Xbox could leverage games like Zoo Tycoon and Kinectimals to powerful effect on HoloLens, and of course, we've already seen concepts for Minecraft experiences using the tech.

When it comes to core gamers, integrating with Xbox One, HoloLens could deliver HUD data like ammo counts, or maybe even throw debris onto the visor to further a player's immersion. Most first person shooters display ammo counts, mini maps and other information on screen, these elements could easily be lifted onto HoloLens, freeing up screen real estate for the game running on your monitor. Displaying HUD data on the HoloLens could also impact tournament gaming, preventing pesky players around you from taking advantage.

Watch_Dogs features mini-games which augment reality within the game world via an advanced smartphone, similar to many Android and iOS titles today. Considering the fact the HoloLens operates completely independently of any other device and supports Unity 3D, it seems more than likely we'll see standalone casual titles in that sort of vein as well. The finger tracking tech Microsoft have been working on could act as a peripheral for all sorts of gestures, like shooting a virtual gun or landing virtual punches, and so on.

Potential applications for core gamers is endless. Could we see flagship titles like Halo 5 use HoloLens to display visor information typically shown on screen? It seems entirely likely.

If Microsoft are positioning HoloLens as a companion device for Xbox One gamers... or even a standalone AR gaming device, hopefully it'll be on the more affordable end of the spectrum. I can't imagine they would expect their core gaming audience, who might not use the other apps, to be interested in a £1000 companion device. Maybe. Hopefully. Please?

We'll have to wait until June 16th at E3 to find out more.

Keep your eyes on Windows Central for all the latest emerging from this year's Build conference.

Jez Corden is a Senior Editor for Windows Central, focusing primarily on all things Xbox and gaming. Jez is known for breaking exclusive news and analysis as relates to the Microsoft ecosystem while being powered by caffeine. Follow on Twitter @JezCorden and listen to his Xbox Two podcast, all about, you guessed it, Xbox!

92 Comments
  • Two words. Horror games. OMG!
  • Haha, I hadn't even thought of that. Imagine you're playing The Evil Within 2 with HoloLens, and some crazy monster crawls out of the TV. D:
  • Actual poo would come out of my body.
  • LOL!!!
  • Can't stop laughing!
  • LOL
  • Can you imagine rewatching The Ring and this time she crawls out of YOUR television.
  • OMG, no... don't even give them that idea!  :)
  • Nice article Jez. You write good ones.
  • Thanks man :)
  • Dude, or even better. You walk into your living room and a monstor jumps out from behind the couch or something. If this thing does half of what it promises, I want one.
  • Great article man. Just the mention of HUD enhancements and splash damage for core Xbox and PC games got me excited. I so badly want every Xbox game to be enhanced by these features, not to mention the spatial speakers in the HoloLens. Using the Kinect to show everyone else in the room what the HoloLens user is seeing would be very cool as well.
  • That would be awesome! Instant purchase for me right there.
  • Except that we don't know the price. With all hardware components that it includes and everything that it does, it would seem that the price would  need to be around $1000. That would severly limit sales and therefore limit development of games that utilize it.
  • One word....PORN !!!!  D:
     
  • One word: Porn. Imagine Megan Fox in bed with you. Or imagine those Japanese sex dolls (or even the cheap inflatable dolls) modded into the subject of whatever sex fantasy you have. Meanwhile at Apple they invented a device that you can wear at your wrist and tell you the time.
  • That would probably make for some bad PR for Microsoft.  I doubt they would allow any adult content into their Windows 10 store.  But that doens't mean companies wont find another way (like sideloading).
  • To make it awesome, simply include the ability to stream ANY game from your xbox or PC to the HoloLens. I could be on a flight, with my laptop tucked away, streaming/playing Halo off the HoloLens.
  • Fighting game + Xbox Fitness + Online multiplayer.
  • It'd be cool if Kinect could show you videos of the people you're exercising against :3
  • Strategy games would be awesome.
    Or asymettrical gaming like fable legends. The villain plans with the hololens.
  • So many ways it could be used, crazy exciting. Project Spark as well!
  • I'm sure the military is interested in this.  Could you imagine coordinating a battle using this technology?
  • Phil spencer did say they had strategy games in development... https://twitter.com/XboxP3/status/592783518553513984
  • This is no doubt cool and all but let me slip into cautious old geezer mode for a second.(I'm still under 35) How far is technology going to go and how much of this tech do we really need? Again cool but a part of me wonders.
  • Want is always what gets us buying this stuff. Never need. I mean...we don't really need the Xbox One in our lives at all, right?
  • I asked myself this question when I paid £300 to import a Microsoft Band from the USA haha. All this tech stuff is fun-to-haves, most of the time.
  • The Xbox One is a need.lol But more so I'm more fascinated by the technology than I am questioning it.
  • you really sound very old :D
  • Actually hololens is beyond gaming it has huge potential in designing automobiles and architecture it's kind of engineers thing not for core gaming
  • That's the same as saying an LCD screen is for engineers not core gaming or a brick is for buildings not a BBQ pit. The best technology has multiple uses.
  • I think you misunderstood my comment. I'm just amazed at how far things have come is all. Posted via the Windows Phone Central App for Android
  • I agree with some points mentioned. But I believe that VR has a superior immersion factor. And the fact that it is powered by an external PC allows it to have superior computational power (on paper).
    The main drawback for now is the player's interaction with the virtual world. But I believe this will be solved in the future.
  • I hope you're right. My experienced at Gamescom was soured cus of the motion sickness thing, I've actually spent the last 6 months using my Lumia in every vehicle ride to try and train myself for a VR future haha (it hasn't worked). I think the immersion is limited, personal preference though. I was still very aware that I was sitting down with a controller, the difference felt similar going from my TV speakers to surround sound headset - an upgrade but, not a necessity.
  • There is a place for vr and ar. But ar can adapt to more use cases. I think that's the message.
  • That is awesome.
  • Most games are created in a 3D environment. Just use HoloLens to extend that around the player. While playing Minecraft, you can actually see the word around your character that is normally limited to what is on screen.
  • Like this? http://research.microsoft.com/en-us/projects/illumiroom/
  • Exactly! I wouldn't be surprised if HoloLens came from a desire to better execute Illumiroom.
  • +1  or at least I hope it was (cant wait to play the COD black ops 4 with Hololens, would be freaking epic)
  • Just a thought, could it be possible to put VR in AR, hololens could project a VResque in front of your eyes and rest works as VR does.
  • During the original press demos, I think they used to hololens to simulate the landscape of Mars in a room.
  • Yes. I tried out the hololens yesterday and in one demo it did placed me in front of a building in middle of Denver. I turned my head and saw the city around me.
  • Jealous!
  • Sorcery
  • Until they release prototypes and sdk to developers it is all just vapourware. We were hoping these would be available at Build 2015. So it remains just hype.
  • They brought hundreds to Build and are actually hosting a class where developers will actually create their first app for the device.
  • Gee... Aren't you fun to hang around...
  • This is no vaporware.  It is the real deal, and if it comes out this year then they will beat a lot of companies to market. Not sure it will hit mainstream use any time soon, but if you want one you should be able to pick one up before the end of the year.
  • Also, where are our evil augmented reality bananas shirts?
  • As the innocent and playful individuals we are we're excited about gaming with these, but sadly as we all know I'm sure, the porn industry will try to get a piece of the pie some how. I wonder if MS can keep them at bay and for how long.
  • Porn is the largest seller on the internet, even more than the big retailers (amazon, ebay, etc), As Microsoft will not offically support porn, they offer the development tools for anyone to get to support it. So, in a nut shell, they just turn a blind eye to it and if it will sell millions more of these devices because of the possable Porn aspect of it, do you think they would actually say no to these sales ?
  • Wtf buddy like if they actually do it, it won't even affect you, you'd have to download the porn yourself, why should MS stop them?
  • We found the prude.  Sex is a part of life. It's integral to humanity and who we are it's not something to be shamed. A smart MS salesman and porn executive would be morons not to get in bed together on this. Besides, the more they make and sell, the cheaper they could become.  
  • This will be so hard to get hold of on release. Hoping as soon as pre orders are available I'll be quick enough to jump on it and order
  • Hololens has a future! Porn always sels :D
  • At this point, I just want to know the retail price on it for consumers. If it's $299.99, I'll be all over it but, if it's like $600+, I'll wait till I can find one used or till there is a TON of sofware.. Crossing fingers for the sub $300 price point... I really hope the design for games can bring this to life with out projectors and kinects all over the rooms http://research.microsoft.com/en-us/projects/illumiroom/ Just the though of the demsion that it could add to games and other aspects in life, is crazy, love the concept and how it's shown but, these are demos at this point, it's only going to be as good as the software is. They could release it for $25 and if there is NO software support besides some demos, you will get board of it in 2 weeks and it will just sit on a shelf... Devices like this are ONLY as good as their software support.
  • I think hoping for less than $300 is pretty unrealistic to be honest. Consider the Surface 3, an all in one tablet computer for $499 base price. Now of course HoloLens doesn't have the same raw computing power as a Surface 3, but it is still an all in one computer, filled with sensors and other custom and complex hardware in a relatively small package. I couldn't see it going for less than $499, but more realistically $599 - $699 if Microsoft position it as a consumer device.
  • Psh...I definitely see this at around $1599. Anything sub 1k is a steal.
  • I could be wrong but that sounds way to high.  Unless they don't think they can't keep up with demand or if they want to market it first as a niche device for proffessionals and businesses.  I'm sure they want to make money off it too but if they want developers to really embrace it, they need a wider audiance.  It seems like that is what they are trying to appeal to since its planned on being a gaming device as well.  Think of the Minecraft audiance.
  • Games!
  • I hope they pair it with Kinect and it leads to a resurgence in Kinect game development.
  • This is the only way Kinect can come back in my opinion. Pairing it with a Kinect can allow games to have more gestures being recognized and environments recognized to determine what a person is doing. The future.
  • Tried the hololens yesterday! Completely blew me away. This works for real as shown! Can't wait to get one for Christmas!
  • You're at //build/? Describe the demos.
  • I just hope its not too expensive.
  • Gotta be at least $999.
  • The day I can do this stuff without wearing a clunky head piece is the day to rejoice over this. Until then I see no practical use of using hololens in a home.
  • "Thank you" for your narrow minded point of view... Anyway, most of us are excited about the possibilities.
  • Yeah, they day when I can go to my office without buying a bulky car I have to keep in a special building called garage is the day to rejoice over working. Until then I see no use of transportation.
  • I hope it'll be compatible with Xbox Controllers. It would be amazing to play racing games with little virtual cars (like Micro Machines games!) where you could create your own track with furniture in your living room and then add virtual boosts and stuff à la Mario Kart. Another cool thing would be Lego games! It could be just like the Lego games on PC, but you would have the whole level, to scale, on your kitchen table. You could then move around your table to try to find shortcuts and secrets and solve puzzles, while controlling your little Lego character with an Xbox Controller!
  • Lego dimensions with hololens. You had an awesome idea mate.
  • Lego Dimensions is actually what gave me the idea! When they announced it the first trailer that came out was of someone putting together a Lego model. He then placed the model on his table and it came to life. My first reaction was "OMG, I want to play this with the HoloLens!!!". Here's the video: https://www.youtube.com/watch?v=1GrLE0f_z20
  • Windows 10 comes with Xbox One drivers. Hololens runs Windows 10. No reason why it couldn't. Now the wireless thing is something I thought of this morning that could present an issue.
  • Considering that it integrates everything into one, its cost would be too prohibitive to get widely adopted in games. Kinect is a mere $100 and most games don't utilize it at all other than voice commands. This thing can easily costs $1000+ even in 3 or 4 years when it finally comes out, and not that many gamers want to buy a $1000+ device for enhanced HUDs. And thus not many game developers will bother to include some decent hololens support in games. Nice concept but I don't think it will not get used that much in gaming.
  • Gaming will not be the only use. I even think that gaming almost killed Kinect, so I don't think they will make the same mistake again. On the other hand, plenty of people play Minecraft and you can bet this will be awesome on the hololenses
  • I think it would get used in gaming but it will be a tough sell. I too agree with the 1k+ price tag. I think that it's just a future PC rather than just a gaming peripheral...and that's how they sell it. If this was ONLY for games, then yea, price tag is ridiculous, but because it's a PC...it kinda changes things.
  • I am very excited about this.... but.... At the end of the day I do not want to wear a headset.  That thing is big, and bulkey, and no dobut has crap battery life (unless it is also super heavy).  As technology progresses maybe we will get something acceptable for long-term daily use, but I think that the reason this is aimed mostly as business applicaitons right now is because it is designed as a tool that you put on for a few minutes at a time, and then take right back off again. Maybe we get a wired version, or a device with wireless power capabilities and no battery that could be used for longer sessions, but this is just not something I see taking off for a few generations.   That said, if I can afford it, then I will likely buy it, because it is pretty damn cool... just not sold on how useful it will be until they get the weight/size down considerably.
  • Cortana, literally popping up during gameplay...game changer. I swear to gawd if MS doesn't release a Hololens in the form of a Master Chief helmet, they've lost. That shit will sell, I don't care what you say.
  • The next halo wont involve Cortana and master chief.
  • I need the GOW and Crackdown teams to do something cool with this!
  • So HoloLens could be used for SmartGlass capabilities.
  • I see hololens working together with Kinect for an even immersive gameplay. This is exactly what illumiroom is about. And they say couldn't be done
  • 3D or 4D Angry Birds in the living room would be fun. Question though... How would one share the experience.  MS should be thinking about 3rd person sharing, or 3D capture technology.  Just a thought...
  • Did you missed the Mars demo?
  • I meant after the fact.  Yes, real-time 3D sharing was shown.  I am talking about recording the experience and being able to share it to others at a later time.  A use case would be allowing yourself to re-experience that Mars demo from a third person perspective as an observer at a later time.  Essentially what they are doing with their camera rig that accompanies the on stage demo's, just at any time later.  It would be cool to be able to save those "sessions" and then go back and see them from a 3rd person perspective.  Kinda like travelling back in time (although the person who was having the first person experience would be saved as a characature (like in the Mars Demo).  Saving that session would allow for an alternate 3rd person perspective (or would that be 4th person?).  Some people do not realize the potential implications of such a system and device in design and build processes.  Sometimes showing a 2D video of a 3D session is not detailed enough for design work.  Sometimes a 3D walkable video would be more helpful.  The data is there, why not capture it? An alternative use-case would be showing the building and eventual explosion of my mincraft castle in 3D on my coffee table to my friends via Xbox Live so they could watch it in 3D from a 3rd person perspective. 
  • It needs to work with Windows 10 PC as well as Xbox One
  • The biggest issue I have read about from people that have had hands on is that the field of view for this thing isn't great and field of view would be pretty important for gaming.  Maybe they will have a more custom one to work with the xbox since this thing originally started as an xbox only peripheral. 
  • I still can't wait...  But I must :(  I was looking on Major Nelson's site at the discounted games, saw the Star Wars table for Pinball Fx2 was on sale then thought "Ooooooh!  Finally can have "real" 3D pinball machines again!"  I know that's not a big deal for the younger gen where they may have never actually seen a physical pinball machine but I AM SO hoping they do that!   I was wondering why MS was going to stress 1st party games this year at E3, I think (now) they probably are going to bring Hololens support to all first party games and stop relying on (begging) 3rd party games to support the XBox features only to have them drop them just before release.  If that's the case E3 will be SICK!
  • the real question is, will they make a separate hololens for the xbox one with no windows 10 capabilities?
  • Since Xbox One itself is a windows 10 device I doubt that.
  • It seems highly unlikely that a device with so much technology stuffed into it would be cheap. Although it would be nice to buy one for < $1,000 i doubt it will be a possibility. At first anyway.