Hands down the biggest problem all forms of VR have right now is showing someone who isn't in the headset what is going on. Putting a 2D image on a screen that shows me what you see out of one eye doesn't do much for me. It doesn't give me the emotional feel, no sense that you as the player are actually in the world you are viewing. It's a real problem, and a lot of very smart people have found clever ways to solve this in some cases with depth-sensing cameras and green screens. The results are a little mixed, and difficult to reproduce in every game, but still very cool.
Microsoft has its own solution. Originally, this solution was a way to show everyone what was happening inside of Hololens. It has been adapted for occlusive Mixed Reality headsets, and the results are ridiculously cool. Unfortunately, it's not something any of us are ever going to be able to do on our own.
This big, complicated box of wires here has three different very important parts. At the top, you have Hololens to act as a sort of virtual camera. Wherever the Hololens is in the real world, that's where the camera will be recording from in the virtual world. Under that you have a big DSLR camera, which is capturing a high quality video of the person in the real world. In this situation, Alex Kipman.
Under that camera you have a Kinect sensor. That sensor is grabbing a depth map around Alex and editing out everything but him. This means there's no need for a green screen, because the Kinect is doing all of the heavy lifting here. All of this video and real-time editing comes together to create what appears to be real-world Alex standing inside the virtual world.
The end result is just plain cool looking. Alex is represented as a silhouette in the virtual world so you can see as much of his "house" and the things he is doing in Mixed Reality. This capture method creates presence, it helps you feel closer to how Alex is feeling in the headset. You can see him physically reach out and interact with the virtual world. If he were playing a scary game, you'd know exactly why he jumped instead of just seeing a shaky camera on his face. It's a much more complete picture.
It's also incredibly challenging from a technical perspective. Not only is this rig prohibitively expensive and incredibly complicated, it requires a great deal of precision to hold the rig steady and perfect lighting to Kinect can do its job effectively. It is unlikely we'll see a way to recreate this effect in the home anytime soon, but it is one of several tools available to developers and Holographic Academy attendees at Microsoft's Reactor space in San Francisco.
We may earn a commission for purchases using our links. Learn more.
Review: EufyCam 2 Kit is the best home-security setup with no fees
If you're looking for a sound home security camera system, Eufy is your best bet. All your video data is kept local, with no cloud storage, and there are no subscriptions whatsoever. Combined with excellent features, low-light ability, and a battery that lasts a year, Eufy is way better than Ring. Here's why.
These are the best laptops HP has to offer
The Spectre x360 13t is currently the best HP laptop you can buy thanks to a potent blend of performance and design. However, there are plenty more options that come close to taking the top spot.
Microsoft's upcoming Surface Pro 8 and Surface Laptop 4 images leak
Microsoft is expected to launch new versions of its Surface Pro and Surface Laptop early next year, and today images of those products have appeared online as they pass through certification in Korea. These images confirm that the devices won't be featuring any new designs, instead opting to keep the external chassis' the same as the previous generation.
Best Xbox Series X, Series S games 2020
The Xbox Series X will bring a range of improvements to existing games, but what are the best games to break in a new system? Here are our picks.