Microsoft's Xbox Project xCloud will take a new approach to game streaming

Project xCloud
Project xCloud (Image credit: Microsoft)

Game-streaming has faced a troubled past, from the decade-old missteps of OnLive to the struggle to gain traction in 2018. Beaming high-quality gaming across the globe is no easy feat, and while networks are better than ever, gameplay is often compromised in the process.

Project xCloud is poised to be Microsoft's next major gaming endeavor, mobilizing its Xbox platform with a cloud-based streaming service. With strong foundations and internal trials underway, early reports indicate Microsoft aims to distance itself from rivals by overcoming these early hurdles.

Alleged details are now surfacing. Here's how the technology might be shaping up.

Xbox Project xCloud game streaming: Everything we know

Tackling game streaming latency

The logistics of game-streaming come with considerable complexity, with the need to ensure a high-quality image and timely delivery. Unlike music and video, the real-time nature of gaming emphasizes latency, the principal weakness of past failed services. Especially in fast-paced genres like shooters and racers, sub-second delays can make games unplayable.

Project xCloud's rollout pairs with dedicated data centers, comprised of server blades boasting custom Xbox One hardware. These servers will host games for clients region-wide, emulating full Xbox hardware without the reliance on a full-fledged console. Existing reports indicate a unique approach that merges local and cloud processing.

Existing game-streaming services like PlayStation Now and Google Project Stream feed full titles from servers via compressed low-latency video. Microsoft's rumored hybridized solution could set Project xCloud apart, leveraging the strengths of both local and cloud gaming. Such a system could see inputs, image processing, physics, and other latency-sensitive components processed locally while demanding elements are handled by the cloud.

This isn't Microsoft's first attempt at offloading processing to the cloud, as demonstrated by "Kahawai," a Microsoft Research and Duke University collaboration. The project aimed to split processing between mobile devices and cloud servers, primarily aiming to reduce bandwidth requirements. Dubbed "collaborative rendering," this allows devices to create a "rough sketch," while GPU-intensive textures and shaders pull from the cloud.

Kahawai in-action, with local rendering (left) and collaborative rendering (right).

Kahawai in-action, with local rendering (left) and collaborative rendering (right).

Watch: Microsoft Kahawai means high-end graphics for less bandwidth

Using Doom 3 as an example, the Kahawai tool managed to cut the bandwidth required to stream the game over a 1MB line by more than 80 percent (opens in new tab) while avoiding major cuts in visual quality. All this occurs while eliminating the delay usually associated with cloud gaming.

Project xCloud might predict your future

Breakdown of the Microsoft Research "Outatime" project architecture.

Breakdown of the Microsoft Research "Outatime" project architecture.

A recent report also detailed additional moves to clamp down on latency, "predicting" player actions in real time. Leveraging deep learning advancements, it appears Microsoft plans to anticipate inputs before they're made, while seamlessly correcting inaccuracies as they occur.

The technology seemingly derives from "Outatime," a 2015 Microsoft Research project (opens in new tab) that explored how speculation can eliminate latency issues. Time-traveling references aside, the publication demonstrated perceivable gains to counter delays. This also reportedly pairs with Microsoft's Project Brainwave (opens in new tab) platform, designed to accelerate real-time AI calculations in the cloud for further reduced latency.

Outatime explored analysis of gameplay, predicting what players will do next, and applying this to improve real-time streaming. The project could mask up to 120 milliseconds of latency at publication, achieved by preparing "speculative frames" of likely outputs before the action occurs. If correct, this image will be instantly reflected on-screen, while mispredictions can be locally warped before being displayed. The systems could also improve predictions over time by learning playstyles and reducing miscalculated frames. All this occurs in milliseconds, and in theory, retains smooth gameplay.

Microsoft also published side-by-side comparisons of Outatime technology and a standard baseline. Tested using DOOM 3 and Fable III, the demonstration showed improvements when aiming the camera, moving, and firing weapons. Although both are older titles, Microsoft claims these benefits transfer to modern titles due to similar foundations.

Reducing load times

Microsoft-registered patents also hint at an AI-enhanced backend, with instances of Xbox One titles cached in xCloud's active memory. These remote instances aren't linked to profiles, allowing players to seamlessly "boot" and request these preloaded titles without perceptible initial load time. Upon request, your unique player profile will be loaded into each instance, with save data, languages, and other preferences.

Comparing the concept to browsing TV channels, Microsoft hopes this will allow for a seamless boot-up across your library. This further enhances the on-demand nature of xCloud, reducing overall wait times. Paired with usage data, the service can anticipate demand, scaling resources on a regional basis and ensuring consistency. It's that extra feature that could showcase the parallels between xCloud and conventional media-streaming platforms.

Your thoughts on Project xCloud

Microsoft's Project xCloud is being pitched as the next major addition to the Xbox platform. Existing leaks look promising, and hopes are high for the game-streaming service. However, big challenges lay ahead. Let us know if you're looking forward to Project xCloud, and why, in the comments section.

Matt Brown

Matt Brown was formerly a Windows Central's Senior Editor, Xbox & PC, at Future. Following over seven years of professional consumer technology and gaming coverage, he’s focused on the world of Microsoft's gaming efforts. You can follow him on Twitter @mattjbrown.

14 Comments
  • I just hope that they also bring the LOCAL streaming option over from the Windows 10 Xbox App into the xCloud streaming apps. I've had great success with that. I've tried Google's Project Stream and it was unuseable for me (too much lag in the stream meant the image was choppy, the input lag was okay though), and PS Now was no good either so I don't really have high hopes for xCloud at my location. If I could just stream from my Xbox One locally to an app on Android TV and/or Fire TV I would be happy. Couple that with an Android based phone running Microsoft Launcher natively that also has an (optional?) 'xbox controller' case (pretty much exactly like the Moto Z GamePad - hhttps://www.motorola.com/us/products/moto-mods/moto-gamepad-console - except Xbox/Microsoft)
  • I agree with everything you said. My bandwidth does not support heavy internet based streaming, but I use Xbox to PC streaming quite frequently and it has been a great experience, even with both machines on wi-fi. Funny enough, my Surface Go is a better game streaming device than my Surface Book and Dell Gaming Laptop. I assume that is due to go's better wifi chip. Anyway, I hope in networking streaming stays as a big part of the future service, it is a very valuable feature to me.
  • I wonder what military and law enforcement applications this can lead to. For example, a drone controller or a smart tank.
  • We'll probably see something at some point.
    They did get $80M to militarize HoloLens after all.
    They're still in competition for the Pentagon's JEDI contract.
  • It'd be something if MS announced that they would become an internet service provider specifically for gaming. Comcast would have some real competition. I would jump ship instantly if the price was right. Bundle that with Xbox live and game pass= killer combo!! But I guess that would probably take years to implement and get properly put in place throughout the country.
  • Wow
    That is amazing.
  • It's a good idea.
    If my home Internet in my neck-of-the-literal-woods can support it then I'd sign up for it.
    Hopefully Microsoft is able to pull it off and stick with it (*cough*unlike their other projects).
  • This is going to be big, it's to be a hit. MS will do it right. I hope there's a public test soon.
  • Does anyone know the grip used in this article picture for the Xbox One controller?? I can't find one for this controller anywhere!! If anyone know where to buy one, please let me know! Thanks in advance!
  • A quick Amazon search gave me this
    https://www.amazon.com/dp/B07CL3GS9C?tag=hawk-future-20&ascsubtag=UUwpUd...
    https://www.amazon.com/dp/B07F1841QM?tag=hawk-future-20&ascsubtag=UUwpUd...
    https://www.amazon.com/dp/B07D9RPNZD?tag=hawk-future-20&ascsubtag=UUwpUd...
  • I have the same concern with this as I do with streamed video. This is essentially a lease hire service so games could be withdrawn or edited at any time, without users having a say. For example, if Microsoft fell out with a particular publisher they could loses dozens of AAA games in one go. Or if a publisher decided that it wanted to make a deal with a country like China that has strict censorship laws they could potentially push out the censored versions globally so that everyone in the world was using the same version. I've already seen shows that I want to watch vanishing from streaming services that I use due to publishers have made deals elsewhere, or have started up their own services. And I've seen shows losing entire episodes after they have aired for the first time or having episodes censored because of controversies in other countries, making it impossible for anybody who doesn't own a physical copy to actually watch this content. Can Microsoft guarantee me that I won't log in one day and discover that my M18 game is not going to be a PG-13 game? Or that it's not going to vanish because of some silly content provision agreement? I don't think that it can. I'm also concerned that a small online indiscretion could lead to you losing access to games that haven't even been released yet. If you say something that it common where you live but extremely offensive elsewhere, and you lose your account, you could be locked out of all Microsoft games forever. Or if you are accused of cheating in an online game you could essentially lose access to all offline games as well. Personally, I don't buy that many games, and I don't finish that many games as I have an annoyingly short attention span that means I lose interest in things quickly, so having an affordable service where I can play a new game for a few days before getting bored with it, and then moving on to the next game, without having to pay full retail sounds good. I'd subscribe if it were cheap enough, and if some of the above concerns could be addressed.
  • Why you are asking these question below this article? You can chat directly with Xbox support and ask them directly those questions. What is the problem?
  • If it were cheap, I'd settle for a graphically downgraded stream with no lag\stutter. Rather than paying for an expensive service that provided AAA quality graphics but with poor frame rate or bad lag.
  • A good place to start might be streaming older games with less intensive graphics at a lower price in order to build the foundations of a higher quality premium service later on. You could probably stream classic xbox titles down a 2meg internet connection at perfect quality with no lag using the technology that we have today.