This is how the new Windows Mixed Reality controllers work

The final Build keynote surprise was a love letter to everyone excited about Windows Mixed Reality, and it came in the form of motion controllers that allow you to reach out and "touch" the virtual world. These controllers follow a lot of the same rules as the Mixed Reality headsets that are expected later this year, which means they have new tech that will make them unique in the world of VR motion controllers.

If you were excited about Mixed Reality before, this addition will have you scrambling for your wallet. Here's what you need to know.

Inside-out tracking

Just like the associated headset, Windows Mixed Reality controllers will not require independent sensors to interact with the virtual world. The controllers rely on the same inside-out tracking that's needed for HoloLens to function, which means you get the same six degrees of freedom (6DOF) motion control seen with Oculus Rift, HTC Vive and PlayStation VR, without any of the extra sensors. The only thing connected to your PC is the single cable coming from the headset, so there's none of that extra mess.

Acer Mixed Reality

This works, in large part, because of the sensor array on the headset. As long as the cameras and sensors can "see" the controllers, they will be positioned exactly where your hands are in the real world. As we've seen with HoloLens, that field of view isn't exactly as wide as what your eyes can see. But it's close enough to have a lot of fun.

Microsoft says the sensors on the headset will allow the controllers to be tracked slightly beyond the camera field of view, but right now it's not clear how far beyond. You may not be able to reach behind your back to grab a virtual shield or another arrow from your virtual quiver, as you can with an HTC Vive controller. But everything else you can do with an existing motion controller you can do with these new controllers. At a minimum, that means existing VR games and apps could be ported to Windows Mixed Reality if their developers are interested.

Different controllers for different folks?

Like the Mixed Reality headsets, Microsoft's partners are the ones doing the design, retail, and shipping of these new controllers. It's not clear if there will be one unified design for these controllers yet, or if, like with the headsets, there will be different designs based on what the OEMs think is best. But the inside tech will be the same.

Currently, the only manufacturer that has announced controllers is Acer. These new controllers are expected to be available in a bundle with the new Mixed Reality headsets later this year for $399, but developers are now able to preorder the bundle with an expected ship time this summer.

Russell Holly

Russell is a tech nerd who chases the best of everything, from phones to game consoles to laptops and everything glowing or beeping. He's the Managing Editor of gaming content for Mobile Nations and can be found contributing to all of the Mobile Nations sites. Reach out on Twitter!

  • Well, I know what's on my Santa list for this year. 
  • Cool stuff!
  • VR/AR/MR tech is moving extremely fast, this MR devices from Acer/Asus/etc. make the other ones like the Vive look dated! The fact that it doesn't need a tracking device makes them already 10x better than the rest.
    I'm excited to see where this tech going in the near future!
  • It's mixed (pun intended) With external beacons you get more robust tracking over a larger area than with this, that only gives you precise tracking when the controllers are infront of you and seen by the cameras.
  • Mixed reality has just been confirmed for Project Scorpion.  Hell yes!!
  • It was confirmed (or at least heavily implied) a long time ago. 
  • If it was just heavily implied then it was confimed.  Now it is.
  • I'm sorry, but I'm waiting until there's much better tech for this.  I'm not wearing those big, bulky, ugly headsets, and the only sensible controller for VR is gloves.  I can't stand gaming controllers to begin with, and that's most what these kinds of things are.
  • Well this is the way to get the tech to progress more quickly. Become mainstream and create a demand for it so companies put more R&D into it since that means more money for the manufacturers
  • I guess you are waiting for Apple then...
  • These controllers look like if the Oculus Touch and Vive wands mated this would be their offspring.
  • More importantly, It looks like they took the best parts of both.
  • Mixed Reality Controller + Project Scorpio = ❤🙂
  • If the controllers have accelerometers and gyroscopes inside them as well, you can do some limited gesture recognition even if they are out of sight of the HMD. I am not sure if they _do_ have that hardware, but it would be beneficial if they did.
  • I hope so. For now technically those packs are almost perfect (except some shared limitations with all headsets), but this limitations to the controller might be really bad. If you look at VR games using motion controllers, you'll find that a lot of them have use for out of sight gestures (firing a bow for example). Let's wait for the first feedbacks...
  • Not being able to point/aim/gesture where I'm not looking sort of kills it for me.
    I was super hype for this due to the inside out tracking... But even if they get more game dev support, I don't think I can justify a sub-par experience like that, even if setup as vastly less complicated, and spec requirements are lower, and it's cheaper.
    If I'm going to invest in vr, everything but the best is just a waste of money. I was hoping that this would be the best, but it seems like it has a few too many sacrifices.
  • The main obstacle for mainstream adoption, is price. This console+acer combo, or PC+acer combo is around the price of an iPhone 8 or cheaper. And I know which one is cooler :P
    Until we get more adoption, there will be interest in AAA titles. AAA titles is where stuff gets interesting.
    Personally, this is just what I was waiting for to get into VR - there's no point on spending thousands of dollars on something that will be superceeded. The console helps provide a stable base that will make the experience endure with support. The fact that its intergrated into windows as an OS, is just even better, thanks to the HoloLens.
  • I can't wait till we have hmds capable of full fov AR & VR along with full 360 mounted sensor free tracking. Right now I have an Oculus and 3 sensors which gives me room scale tracking which is great except just like the Vive im stuck to my PC by a wire so I can't roam as far as I should be able to plus the wires or even just wire get in the way when gaming. The future for both these technologies is so exciting