Microsoft Research has prototyped Xbox controllers for phones and tablets (update)

By last updated

With Microsoft looking to bring "console quality" streaming to mobile devices, how will the company achieve that when touch controls are still pretty bad?

Update July 9, 2019: It seems Microsoft has now patented designs (via WindowsLatest) very similar to these. Perhaps this is one research project that may see the light of day as a real product. We'll have to wait and see!

Original article:

With Microsoft looking to bring "console quality" streaming to phones and tablets with Project xCloud, how will the company achieve that when touch controls are still pretty bad? It seems the company is looking to bring physical controllers to mobile devices to offset this problem, according to these Microsoft Research papers.

The research paper documents some of the popular solutions to gaming via a touch screen, while hailing the Nintendo Switch and other portable game consoles for circumventing touch-based control limitations with full joysticks and buttons.

As smartphones and tablets have become pervasive, so has mobile gaming. Not surprisingly, popular games for these platforms are focused on touchscreen-based interaction. However, many types of game are less well-suited to mobile devices. Despite systems like AdaptControl which can adapt to the 'drift' typically occurring when using virtual on-screen controls, touchbased emulations of traditional gaming controls like Dpads, buttons & joysticks are often unsatisfactory.Mobile gaming devices like the Sony PlayStation Portable and Nintendo's DS and Switch are dedicated mobile gaming platforms which overcome these limitations via physical controls. The success of the Switch is testament to the value of mobile gaming with physical controls. A number of cheaper products allow a smartphone to be clipped into or onto a modified handheld gaming controller; these include the ION iCade mobile, the GameCase, the GameVice and products from Moga. However, the fixed form of these accessories means they are bulky and inflexible.

Microsoft built the prototypes out of foam and then had them 3D printed, based on conceptual renders. The work was carried out quite a while ago, back in 2014, but it seems Microsoft Research has resurfaced their efforts recently, noting the recent success of the Nintendo Switch.

While this research may be far away from turning into an actual product, it's pretty imperative that Microsoft takes a serious role in exploring how it can improve the way Xbox games will handle on a mobile device to help take Project xCloud mainstream. Touch-based inputs have always felt like a half-way solution, and will feel even more like one when they come up against games designed from the ground-up for responsive, tactile inputs.

Would you like to see an Xbox controller like this hit store shelves for your phone or Surface Go? Let us know in the comments.

Jez Corden
Jez Corden

Jez Corden is a Senior Editor for Windows Central, focusing primarily on all things Xbox and gaming. Jez is known for breaking exclusive news and analysis as relates to the Microsoft ecosystem while being powered by caffeine. Follow on Twitter @JezCorden and listen to his Xbox Two podcast, all about, you guessed it, Xbox!
44 Comments
  • Just watch Asus ROG phone and the accessories they made to that to get an idea how thing should be done.
  • Looks like copied from Nintendo Switch?
  • How fortuitous of Microsoft Research, all the way back in 2014, to have based their prototype designs by copying a Switch that did not exist yet. /s
  • that may not be true....
    but what's true is ....
    other companies always implement Microsoft Research's ideas 10 times faster than Microsoft...sad!!!
  • That's the entire point though.... MS is a patent behemoth... The stream revenue that comes from other implementing it is non negligeable... And more than that they render MS impossible to avoid as a partner for most tech company out there.... MS doesn't need to go faster... That's just sad for MS fans or customers in general... But it isn't a problem for MS... Beside for fad brands like apple mass market adoption is not a necessity for sustainability... MS foundation are far stronger than Apple and still more than Google... The later being the most threatening but still decades away of reaching MS long term strength foothold or whatever you wanna call it...
  • The problem with wireless game controller pieces you clip to your device is that you cannot use it during flights, which is a pretty big use case for mobile gaming. A better approach would be to allow electrical connections like with the Surface keyboard cover, maybe allow bluetooth for compatibility but enable non-wireless connection to a set of devices designed to work with it (like Andromeda ?). I'm trying to push this for their pocketable form factor, so you can check the details and upvote if you agree in the Feedback Hub: https://aka.ms/AA17ikl
  • If this gamepad has a USB connector, any (latest) Android or Windows NB should work fine. iOS...
  • Yeah, but that's more of a workaround and would need either two USB ports, one for each piece, or another cable connecting the two pieces together... Being using mobile devices since decades, I can say from experience that extra cables and adapters are what makes smaller devices less portable than slightly bigger devices with less cables, if the sum of what you're carrying with your pocketable device is as big in a bag as a whole Surface Pro, you've just settled for a smaller screen for no extra portability benefit. I agree USB would be the way to go, but my point is magnetic USB connectors on the sides of the device is much better than two USB type-C cables to connect after snapping on the controller sides.
    This is already used in the ruggedized mobile industry for years (think industrial tablets and handhelds, not smartphones), a few contact points on every side would let you snap on gamepad halves, keyboard, digital camera flash or higher quality sensor, eye tracker bar, mixed reality tracking beacon to turn it into an input device, etc...
    All could simply be snapped to either of the 4 sides to make these devices more customized to your use scenario, and with no messy cable.
  • Can't remember last time I followed no device rule set by the FAA... It's just a load of crap and has been for decades.... I never shut down anything.... And I have been taking planes several times a year for decades... My father use to pilot never had issues either... You have more chance to have a fire in a gas station with your phone on than with any wireless device during a commercial flight....
  • Actually, at least in the US, short-range Bluetooth accessories are allowed (the FAA specifically calls out keyboards as an example and these controllers are just a different form of keyboard)
  • Uh, why can't you use them on a flight? I use wireless devices on planes all the time.
  • Why can't you use them in flights? Because of BT? Some allow for a OTG cable connection. WHICH would be ideal since BT introduces a small delay to the chain.
  • I think full xbox controls are a but much for phones, the Wasp controller is less bulky and quite unique, MS should buy that company call it the Xbox Edge controller and use that for streaming games but also getting mobile games to plug into Xbox with standardized controls. A really good review of the wasp controller.
    https://www.youtube.com/watch?v=bkACYOvQXFg
  • 1) need to work wz different sizes and shapes 2) need 2 analogs, 1 d-pad, abxy, L/R bumer&trigger, menu, view and xbox button.
  • Same, just makes it better
    And the Nintendo Switch Joy-Cons don't have a D-Pad so yeah
  • First, the Wasp device is designed to add to ease of input for phone games. It still expects you to interact with the game as you normally would (lots of screen touching). It just gives the extra bonus of using that a little (and very limited) physical input rather than being totally dependent on all touch input for the mobile device and game. Any device that MS brings out must replicate the controller itself. The games from XCloud (seem to be as of now) will be Xbox streamed games. Those games don't have screen touching interfacing action in mind. And although the small demo of what is planned for XCloud shows MS emulating the controllers on the screen using touch it did not show you using touch any other way to manipulate the game that are built-into mobile games that Wasp expects to be working with. Let us not forget Xbox games expect a certain layout of input to work with (2 analog sticks, A,B,X,Y buttons, digital, triggers, bumpers, home & menu buttons). I don't see that changing nor the developers changing the games. The Wasp design would work horribly with the current phone designs, those without any physical buttons (no Home on an iPhone for example) and phones with small bezzles (or even without bezzles). And would completely fail with a tablet device. The reviewer even noted with that even the older iPhone model (with larger bezzles) he was demoing the Wasp on still extended onto the screen and he said he would have to move the bottom game interface for it to be playable. That is a no-go for casual games to deal with. And I doubt MS wants to bring out dozens of different SKUs for all different shapes and sizes of devices. Plus, that phone device becomes a worthless controller if you want to detach it from the device and (like the Switch controllers) be able to play using controllers not attached. I suspect being able to do that will be (should be from the old prototype pictures) a high priority for any controller MS brings out for mobile XCloud support. There so many variables that must be taken into consideration and a device similarly designed to the Wasp, although a nice option for some simple mobile gaming, would fail miserably for what MS is planning.
  • If they make these and they're good it'll be a huge step forward for streaming Xbox games. That phone clipped on top of a controller might be workable but it sure looks like an inelegant solution.
  • Awesome, they have to release andromeda if they are to capitalise on mobile gaming and xcloud. As I said it time and time again closing Xbox Entertainment Studios was a massive mistake. Can you imagine the money they would have made with accessories and licensing out the designs for said accessories? Also they could have leverage franchises and sold even more accessories which would have enabled them to get Cortana into more households. Smh...
  • there's a design challenge in case of full screen phones...
  • I thought this exact same thing when I noticed the bezel on that phone in the picture.
  • Aren't most "bezel-less" phones wider than 16:9? I know my Galaxy S9 is. Also: If this is designed primarily for xCloud games, Microsoft controls the app and can not render to the parts covered by the physical controller.
  • OMG that's what the notch is for!!!
  • Dual notch phones incoming.
  • Game vice it's going to have a field day with this. Remember when they tried to go after the Nintendo switch?
  • Been able to use the ps3 controller on Android for quit some time. Played several final fantasy games on my phone using it.
  • The Xbox controller (all versions that came after the release of the One S) has bluetooth built in and works on Android as well. This is meant to be a less bulky solution that lets you convert your device into a handheld similar to accessories like the iPega 9023.
  • You go, MSFT.
  • Microsoft's track record of turning Microsoft Research projects into successful products is pretty bad except maybe for Cortana recently. I wouldn't hold my breath on any of this.
  • Oh look it is the new Nintendo switch
  • I'd prefer buying a switch, I have one, vs carrying around the extra baggage. why not buy a chunk of Nintendo stock or work a deal out and stream games through the switch. I imagine nintendos v2 of the switch will be killer if they figure in streaming games like google and MS. It's nostalgic to me to see Nintendo being one step ahead of both Sony and MS.
  • I would say that by you already carrying around your phone with you as well as a Switch would be considered "extra baggage" already. If the idea is that everyone has their phone on them, anything extra is just that...extra. Your Switch plus phone fits that scenario.
  • I have zero time to play video games during my daily grind. With that being said, When I walk the halls of my school. The halls are filled with hand me down phones, and the switch is starting to gain ground. Besides Kids are to interested in watching youtube videos of other kids playing the latest game. At home, I rarely plug the switch into the TV. I prefer the portability of going into whatever room I feel like and that includes the front or back yard. The best part about the Switches battery life is that I don't waste my life away on a game.
  • Create a mount that has an adjustable holder for your phone that attaches to the top of an Xbox controller. Then connect via Bluetooth/xbox wireless (via some kinda of dongle).
  • I don't think Microsoft should rely on any one thing here...they need to give consumers as many options as possible. Some might think that mount attachment is bulky, but in all honesty, that works best for me on a phone. Sure, it isn't elegant...but I don't need it to be...I need it to be functional. Nothing beats an Xbox controller for Xbox games...and I can't see anything being more comfortable to hold for long periods of time either. Keep the options open so that gamers can choose which works best for their needs.
  • One issue I see with this solution is that anyone who is a bit physical with their controller could easily pull one of the two parts off the phone, possibly damaging something in the process. By using a mount on an existing Xbox controller, you have the more robust single-piece controller in your hands and are less likely do any damage.
  • I can see the appeal of having this option for a lot of game types. One thing I would recommend other commenters check out is what Motorola/Lenovo has done with their Z series line up of mobile devices and their implementation of Moto Mods! No cables,cords or issues with connectivity on this product albeit it does require the Z Series phone to work! My point of emphasis is the modular abilities they have accomplished with the pinouts on the backplate and different options including the for thought of a GamePad attachment. More manufacturers should consider something to this liking and hopefully develop a standard so companies like MSFT could put out something like their XBOX controller or a PS4 Unit from Sony, both able to simply adapt to the device and go, depending on the users preferences. My thoughts on it at least! Check out the link if it's not something you've heard of before- Motorola Moto Gamepad https://www.google.com/search?stick=H4sIAAAAAAAAAOPgEhIxNDUxMwIiC3MDczNL...
  • I'd love one of these for the Surface Go, especially if it can be packed smaller or flatter than a regular Xbone controller with the grips removed. A (small) concern I'd have is battery life as it looks like all the internals are located in just the two square halves. It also looks like the triggers/bumpers are in the middle of the controller (why?!) unless that's just a mistake made on that photo.
  • You want the same thing as me. The Surface Go is actually quite capable for light gaming and screams for attachable controllers. Hopefully MS makes this happen.
  • Yes, Microsoft should already have this on the market. Android will work with controllers. Get millions to buy the Xbox controllers that will work on any Android game supporting controllers and you will reduce the barriers to streaming Xbox games to Android in the future. Hopefully the controllers would work on iPhone also, but I don't know if that is possible. However, for phones with automatic brightness adjustments, covering the front sensors could be a problem.
  • I just want something to play games like "World of Tanks blitz" with a normal controller... The touch screen method is just ridiculous causing games like WOT simply broken and wasted potential... I do so much better on the Xbox version of tanks using logical controls and the game is the same other than ones irrelevant to what were talking about. I'm just saying that mobile gaming has gotten a bit beyond the "touch controls" practicality, rendering some very good games useless and broken. This is especially the case with cross-platform titles.. PC users will murder folks using a mobile touch screen every time.
  • PC users will murder controllers too.
  • I think we're gonna be seeing a lot of advancements in this when Xcloud arrives. Personally I'm going to want to use my Xbox controllers with the phone. I need a good clip, but something better could come along. It's just design, it can be done. I'm optimistic.
  • I don't think we'll see many advancements with XCloud as there is only much you can push on competitor platforms without direct hardware integration at the SOC level. Case in point, the work that the Xbox team did at the hardware level with AMD is what enabled the ridiculously ability of true backwards compatibility on a low end Jaguar SOC in the OG xbox one. To truely emulate the PowerPC architecture on x86 is mind boggling insane. On the mobile front, you just need to look at Cortana's integration on Android vs ios - so if Microsoft had their own Windows based mobile platform with which that could completely fine tuned the XCloud experience as they did with backwards compatibility that would have enabled them to move that know how onto Hololens and WMR. Imagine streaming games in VR... that is a tantalising prospect.
  • I really like this design. I'm working an open-source phone gamepad so I've researched existing products but I haven't searched patents yet. This look like it'll actually fit in a pocket, unlike many phone controllers. Writers: you should've posted Microsoft's PDF of the research article instead of a paywalled site. The PDF has been available since Oct 2018.