Here's how Microsoft handles all that Windows 10 feedback from Insiders

Last week, Microsoft revealed that there are now over 4 million members of its Windows Insider program. That's a lot of folks offering feedback on the preview versions of Windows 10 and Windows 10 Mobile. Now the company is offering a behind-the-scenes look at how it handles all that data and feedback to make Windows 10 better before it launches on July 29 for the PC.

Windows Insider feedback

Microsoft's blog post on this subject is quite extensive but here are some of the highlights:

  • Microsoft has received over 3 million pieces of feedback through the Windows Insider app in Windows 10, along with over 2.5 million responses to the survey questions that show up in the app. All of the Windows team members can access all of that raw feedback data, using the interface you can see above, which is based on Visual Studio Online.
  • The team has developed an algorithm so that feedback with a lot of up-votes from the Windows Insider app automatically gets turned into work items.
  • Microsoft scans for which specific words and phrases from Windows Insider feedback are popping out. An internal tool called Gestalt generates a "word cloud" that the team can dive in to see specfic bits of feedback.
  • There are only four countries on the planet that don't currently have a Windows Insider testing Windows 10.
  • 80 engineers on the Windows team have engaged in conversations with members of the Windows Insider forums.

There's a ton of information that we didn't touch on here, and we would highly recommend you check out the full blog post for yourself if you are at all interested in seeing how Microsoft handles your Windows 10 suggestions.

Source: Microsoft

  • It's good to see them using their own tools, Team Foundation Server in this case.
  • This all bullshit. They go ini mini maini mo on the feedback or else we would anyway have that gap less thing by now.
  • Just because it has a lot of votes does not mean they have to do it. They can still decide what they want to add, fix or remove.
  • Welcome to Windows 8. If MS is intent of doing something, they will do it regardless of feedback. So why bother asking for feedback?
  • Stop whining, gapless audio is coming with windows 10. Sources :
  • I wonder why WindowsCentral didn't post on that?
  • I remember that they did, just couldn't find them right now.
  • Yesyesyesyes!!!!
  • Or maybe it's not a high priority compared to other things. I know people care about gapless playback (assuming that's what you mean) but I doubt it breaks the user experience the way you seem to think.
  • Wow. Of all things, that is your standard for deciding truth? LOL.
  • MS is """NOT"" CrApple. 
  • Give it up, pal, 99.99% of users wouldn't even know wht you're talking about and, even as a music lover (way moreso than you, guaranted), it is an insignificant issue for me.
  • So cool and weird/neat that an OS is being crowdsourced in many ways.
  • Yes, I always wondered how they managed so many feedbacks and also the mails.
  • They have a huge digital shredder, don't they? ; )
  • It's pretty awesome that a lot of what they do behind the scenes is shared with their users. Case in point: Technical previews.
  • Yeah, this is a pretty new way to get feedback as far as OS is concerned. The approach itself isn't so new if you consider the PC gaming world of alphas and betas, but the scale and reach is far larger. I wonder what this will ultimately for the OS democratizing development priorities certainly seems smart, but...
  • Democracy seems to get better but we already know who'll win.
  • "There are only 4 countries on the planet that don’t have a Insider running Windows10" This is crazy! Very nice to read and see all this Feedback treatment by MS.
  • Anyone want to make a guess at which four countries aren't represented?
  • You may haven't heard those country names yet. ;p
  • Antarctica is probably one of them... Oh wait... Its a full continent
  • Bet not...that place is full of scientists.
  • North Korea for one?
  • North Korea Atlantis Narnia Rivendell
  • I disagree
    1) Kings Landing
    2) Winterfell
    3) The Wall
    4) CastleRock.
    :D :D
  • and Gotham , Batman wont let them do it , cus he is BATMAN!!!
  • Do they mention which 4?
  • Apparently, not. I assume they mustn't be very popular.
  • No.
  • North Korea
    Saudi Arabia
  • Idiot!
    Iraq , Syria and Saudi Arabia do have windows insiders. Although I'm not sure about North korea
  • You obviously have no idea about Syria, Iraq and Saudi Arabia.
  • Four countries not trying Windows 10. Almost the number of countries who will have Cortana on launch.
  • Look how much feedback they got from Brazilian Mobile users. Now please prioritize making Cortana speaks Brazilian Portuguese.
  • Até o fim do ano deve chegar.
  • Do ano que vem. Se não me engano já vazaram informações sobre isso e será ano que vem.
  • Português do Brasil? De Portugal devia ser primeiro! Foi aqui que começou... Compreendo que o Brasil seja muito maior mas eu simplesmente não gosto da vossa pronúncia e basicamente forma de falar... (não é nenhum ataque pessoal)
  • Me gusta. Haha
  • That's really cool. My only request with the whole process would be for them to trim the feedback that's already been completed and doesn't make sense anymore. Otherwise I love the whole process.
  • Yep they would most likely already do that. The screenshot is from build 10080 so...
  • I love it too!  i wish they marked feedback as "completed" or just simply erased them from the lists when completed or as designed.  A lot of old feedback is still showing in the app when it's already been fixed.  Also, I hope they speed up the Feedback app, it's a bit slow sometimes.  Other than that, i Love this program.
  • You might want to check your graphics driver or your Wi Fi / ethernet connection or it might just be your internet speed.
  • Only for countries? Wow
  • *four
  • Ah, thank you
  • That "Machine Translated" stuff though! Hope there are better in-house ways to get it translated (Chinese engineers I suppose)!
  • I'm not trying the W10 stuff on my phone, but I assume that they are doing the same for that? Same with Xbox?
  • The article it's about PC and Phone.
  • Much of the feedback in the screenshot was for Phone.
  • It's certainly been expanded and better automated than I remember. Still, just over 90% of what you see will be tossed primarily due to duplication and "by design" designation. Another small percentage will go to a future update. Realistic percentage of usable feedback? I would conjecture approximately 3% which is still significant.
  • Feedback is one of the best features, glad they are opening up and listening.
  • I always wondered how they did it... :)
  • That is great but the drumbeat rolling through here is "they never respond"  When my clients report a problem, I fix it, and get back to them, or suggest a workaround.  The forums are full of people who are seeking a response and never get one.  And, we are the techies who participate. Try working the MSFT support services for a problem.  It takes hours, with inappropriate or inaccurate scripted responses.  HOURS.  ( BTDT recently ). Today's devlopers have missed the whole point of documentation and support.  Sadly,  when 7/29 rolls around, and the marketing has convinced millions to jump on board, and then the problems arise, the current state of affairs with the support system will lead to a major disaster.  People are no longer patient, and they will be enraged and utilize social media to let their concerns be known.  From there, the mainstream media will interview every person who is unhappy and push out the soundbytes, fueling a "not for me" response.  A billion PC's ?  I'd like that, but unless the support gets in place in a MUCH better way, my clients would prefer playing russian roulette with XP. Sorry, but opinion polls, trending themes, etc. don't do much for me.  The people I know depend on their computers, expect them to work, and are concerned about the future with MSFT.   I'm concerned ... I want success. 
  • I'm sure you do an amazing job returning feedback with your clients, but I'm also sure you don't have 4 million of them all giving feedback at once too. I'd rather they return feedback via blog post of all the new stuff and just publish the full release notes for the nerds like us who care about each individual bug/fix/change. Just because we don't see those notes doesn't mean the documentation doesn't exist. In fact leaked builds often include those notes.
  • It clearly shows in the disclosure that they can't respond to every request. ;) and to personally respond to 4 million+ users is, quite obviously, a task no one or company in their right mind will ever be able to accomplish without considerable manpower, resources, time, money paid, etc. Beta testing is simply that: Testing and feedback given willingly by its users, usually without any response back in return.
  • I did not say individual feedback. When 200 people mark "me too" a posted response could be useful.  Ive done testing for over 30 years. I get it. My point was on release day all these people encouraged to update expect a finished product with support for issues. Neither exists at this point. I liked the October timeline because i really want this to succeed. 
  • The Windows Insider program is for TESTING, not for CLIENT SUPPORT.  They have channels for that already, and they respond to customers there promptly.  In fact, in Windows 10 they even added a Contact Support app for furthering that customer support.  So, I think Microsoft is doing just fine with that.  Giving feedback and testing does not require responses, it would be nice, but it wouldn't make sense to reply to every single one.  But based on the article, it says they "80 engineers have engaged with Windows Insiders" so they are probably are contacting insiders on a 1-to-1 basis.
  • I and many others will disagree that normal support is doing fine. That is the issue. Set Win 10 aside and listen to the experiences people are having. Now in 6 weeks add millions of Win 10 updates to that que. as for input, we old timer developers still believe communication is two way and integral to building proper software. I know times have changed and thats what keeps me in business
  • I agree 100% Tim. For example, RSAT broke a few builds ago and a number of us IT Pros have submitted feedback, have TechNet and Reddit threads going and we've been tweeting Gabe as well. Nothing. These are basic tools that every IT Pro needs, how can we test this software properly if it doesn't work. MS simply pick and choose their battles and ignore the rest until it 'trends'or until it serves their own interests.
  • @Tim Stone: I agree... MS has never had very good support without expensive support packages and average consumer customers potentially migrating from a competitors ecosystem will have grown to expect live support. Imagine an iPad user buying a surface 3... The support experience would be very different. That said MS is adding retail stores with support all the time and i feel they recognize the issue which at it its heart is that competition has made better support an expected item... Since MS has not historically had to provide that they must now scramble to catch up
  • The support service for the Surface pro 3 is above and beyond what Apple ever provided for the iPad and iPhone. Personal experience has MS support answering within 1 minute, even if I request a call and promptly solving my problem as soon as I'd explained my problem. Even if the solution required a complete replacement of my device I instantly got my shipping label without any hassel.
  • Roger, I did try to discuss a display issue with Surface support and ended up with a contracted person who had no idea how to handle the question.  The issue is simple.  A 1920x1080 window does not properly display on the Surface.  The automatic resizing does not handle it properly.  Turn that off and it becomes too small.   I even took it to the store ( with an appointment ) and got no response.  Your comment suggests that support service has improved. On the other hand, my Band has had issues and the support is excellent, and puts any other fitness product CS to shame.  The difference here is these two items are Microsoft hardware, and they should be supported strongly.  In addition, they are "known platforms".  My comments concern the millions of upgraders who will be on non-Microsoft devices.  Most hardware companies now have very little support, and the line from Acer/Gateway, Samsung, and many others will be "We only support and service the computer in it's original configuration, and OS, with all programs removed.  This has become the standard policy.  Thus, when problems occur ( and with the multitude of configuration possiblities, add on hardware, and software conflicts ), people will turn to Microsoft for support.  When that is overwhelmed, and people get poor responses from the contractors, they will be angry, and lash out.  My lack of confidence comes from the lack of proper support on other products, though sold through the Microsoft store, the inaccurate responses posted by contractors on the forums, and the disaster with tech support.  The other day one of these people told me my new HP Stream 7 tablet need a full OS, and I needed to order the disk through him to get support.  The fact I bought it at the Microsoft Store meant nothing to him.  Finally he said he was only there to provide support on Windows and not devices ... and this is the support that came direct from the Store Support portion of the website. I have seen this too many times before.  Thus ... my concerns.
  • I want to emphasize I am NOT knocking Microsoft, or Windows 10.  I go to my local Microsoft Store when I can, but they are not everywhere.  My concern is that the online support services are far too dependent on script readers who give responses that are totally off the actual question, or send people into unecessary and costly ( time and money ) tasks.  And, I'm not concerned about those of us who are techies, but the vast majority of people who accept that prompt and tell the computer to install Win 10 ... and then there is some variant on the device that the update reacts to and problems result.  My concern is "where do they turn".  Yes, many of us eventually get to qualified technicians, but way too many people end up with a script reader who has no understanding how to really trouble shoot a problem.  They push the buttons and select the kind words, but that doesn't help people.  And if the help is not there, people will be VERY mad and Microsoft, and that is exactly what I do not want to see happen.
  • @Tim Stone, I shall add to that try looking for Office 365 live support on the web. It's not easy and if your anywhere but the USA.. good luck with that. I stumbled into it by mistake out of sheer fustration as registering to wait in the queue requires a US number. Even when I did, the guy I got was a script reader despite me telling him in depth what the issue was he proceeded to go about it in a round about manner. The issue in this case was that my aunt had got a HP stream 13 laptop and the presupplied office key was not validating as the option to select the region in the wizard was greyed out. In the end I got round that issue by using my phone to login into her account, validate online, log out, log in to her account on the laptop and download the validated installer. So, if someone like myself had to go through all that rigmarole what chance does the average joe have (especially since the UK has no Microsoft stores)?
  • Seeing this i would most definitely want to know what happened to the Feedbacks with the most upvotes which undoubtedly will be the "anti-hamburger menu"- and the new "circle than square design look" votes. Since they stick with this two "annoying" new standards i asked myself if there would be any chance that they rethink their choices.
  • "So while we do make use of upvotes as a key indicator for what needs to be addressed, we use many other methods to search out and find feedback that should be acted on."
  • And you'll probably find that the anti-hamburger and anti-circle feedback is much smaller than you expect.
  • Yup. When I spoke to Gabe about this what I learned is they have their own roadmap/features and they use this data. However, the process is not 100% driven by userfeedback.
  • Daniel, some will never get it that the O/S is not being designed by a committee of 4 million+ users. If it was, Microsoft would never get anywhere as they wouldn't be able make everyone happy and to expect them to accomplish such an impossible task would be just plain moronic.
  • They wrote detailed responses as to why they are sticking with those design choices. So they were not ignored or unheard.
  • They can't soley make changes based on feedback.  They have to make decisions based on business requirements and future innovation.  They clearly explained why those design choices needed to be made already, and they also mentioned they've seen those feedback already.  They didn't simply ignore them, but rather they just didn't take action on them because it would conflict with their own roadmap.  I bet the swipe to open the menu is coming though!  Based on that last leak, they added the swipe pivots to the phone/alarms/messaging apps, so they're listening :)
  • So this feedback is only of windows 10 not for mobile right? Or do they collect feedback for mobile too?
  • Look at the screenshot, it's almost all mobile.
  • It looks like in that screenshot he has it filtered to only show Mobile.
  • His question was 'do they collect feedback for mobile'.
  • both, same app, same program
  • Which are the four countries?
  • 4 million Insiders and only 3 million feedback posts? How? My person alone submitted 25 posts already in total for pc/mobile. You lazy ass Insiders...
  • Most of them just whine and complain in forums mate
  • You can do both--submit actual feedback items AND complain in the forums.
  • Some doesn't give feedbacks, some just upvote existing ones, etc
  • A lot of feedback is repeatitive, so they just upvote.  Others don't give feedback at all and just want to test the software.
  • One thing isn't clear, what about the rare problems such as device specific, you world probably have less feedback for them, how do they use that feedback.
  • 25 unique feedback posts?  That's pretty impressive.   Of course, if they WEREN'T unique (you checked, right?) then you're not helping as much as you think you are.
  • Sweeeetttttt !!
  • If i install the Windows 10 preview on my HP Stream 11, will i be upgraded to Windows 10 when released without going back to 8.1?
  • I'm positive they filter mine out completely.
  • If anyone want any apps from the other platform please vote for your favourite apps on this site it has its official app also.
  • Wow Thank you Microsoft for working hard, for make sure W10 is for made for us and by us. You guys rock! Bravo Microsoft
  • Thank you "for" your comment. MS "for" sure appreciate it! LOL.
  • Thanks;D