Skip to main content

Microsoft explains how it will improve quality assurance after Windows 10 October 2018 Update debacle

Windows Update Insider
Windows Update Insider (Image credit: Windows Central)

Microsoft has had a rough month when it comes to quality assurance and Windows 10.

The Windows 10 October 2018 Update was pulled just a few days after it started rolling out officially, after a file deletion bug caused a small percentage of users to lose their entire Documents folders.

Since then, Microsoft has been taking this issue very seriously and wants to inform the world on its approach to quality on Windows 10. In a new blog post from Microsoft's Corporate Vice President for Windows, Michael Fortin, the company goes into extensive detail on how it's using data to improve the overall quality of Windows 10 feature updates.

According to Microsoft, during the last three years, the company has seen the volume of support calls and chats drop, as the amount of Windows 10 users increased, suggesting that each new Windows 10 update is more stable than the previous one. The Windows 10 April 2018 Update has the highest Net Promoter Score of any Windows 10 feature update to date, which further suggests that there are fewer issues for users to deal with as each new Windows 10 update rolls out.

Microsoft also detailed how it tests Windows internally, using "self-hosting" as one major way of ensuring quality. Fortin explained:

Internally, Windows has what we call an aggressive 'self-host' culture. 'Self-host; means that employees working on Windows run the latest internal versions on their machines to ensure they are living with Windows. The 'aggressive' part refers to the tenacious push to make sure local teams run their own builds and pursue any issues found. A strong self-host culture is a source of pride for those of us working on Windows.

Windows Insiders are also a huge part of the testing phase of Windows, and the data provided from Insiders helps Microsoft shape the quality of Windows 10 feature updates. Of course, sometimes issue slip through, as one did with the October 2018 Update, but this is the first (and hopefully last) time this has happened with a major release, and Microsoft is upping its game to ensure it doesn't happen again.

Windows as a Service

Microsoft Logo at Ignite

Microsoft Logo at Ignite (Image credit: Windows Central)

The company says that it doesn't ship Windows 10 feature updates like it used to ship old versions of Windows. Windows as a Service allows Microsoft to be more agile when it comes to rolling out feature updates to the world, usually targeting a tiny percentage at first before increasing the number of users after there are no initial launch growing pains found.

The first principle of a feature update rollout is to only update devices that our data shows will have a good experience. One of our most recent improvements is to use a machine learning (ML) model to select the devices that are offered updates first. If we detect that your device might have an issue, we will not offer the update until that issue is resolved. Second among our principles is to start slowly – to prioritize the update experience over rollout velocity. When a new feature update release is available, we first make it available to a small percentage of 'seekers,' users who take action to get the updates early.

Finally, Microsoft promises that it's going to be more transparent when it comes to issues like this. Microsoft is notoriously bad at communicating, which is not good when there is a major issue ongoing with a new version of Windows 10. For example, Microsoft was almost dead silent about the October 2018 Update for almost a month, giving us no information as to what's happening, and when or if it would be made available again.

Our focus until now has been almost exclusively on detecting and fixing issues quickly, and we will increase our focus on transparency and communication. We believe in transparency as a principle and we will continue to invest in clear and regular communications with our customers when there are issues.

The October 2018 Update is fixed, and Microsoft is going to extra measures to ensure future Windows 10 feature updates don't roll out with any showstopping bugs. It's apparent that Microsoft is taking this whole thing very seriously, and rightly so. It's unforgivable to ship a version of Windows 10 to the public that has a showstopping bug in it, especially one that was spotted by Insiders beforehand.

The Windows 10 October 2018 Update is rolling out once again for seekers, so if you want the update now, make sure you check for updates in Windows Update.

What are your thoughts on the October 2018 Update debacle? Let us know in the comments.

Zac Bowden
Senior Editor

Zac Bowden is a Senior Editor at Windows Central. Bringing you exclusive coverage into the world of Windows 10 on PCs, tablets, phones, and more. Also an avid collector of rare Microsoft prototype devices! Keep in touch on Twitter: @zacbowden.

  • I put this update on like 6 machines (all without problems), including my main desktop at work. It is actually a *really* nice update. I didn't get it on my laptop, and I miss it every single time I sit down to use that laptop. Super happy to be able to get it on there, now, too. If it were up to me, the next update would only fix bugs and improve the interface. W10 is an ugly, discombobulated mess without a working touch interface--go back to W8.1 and see what worked there. I'd love to see an all-hands effort beautify (with dark mode being the new default) and "touchify" the interface. Right now, there really is no such thing as a Windows slate computer. You *need* a mouse and keyboard to be happy.
  • Agreed. You make some good points and I've also been enjoying the update.
  • Meant to cancel... Disregard.
  • maybe instead of making such huge updates they should stop calling apps part of the os.
    there's no need to wait for a whole release of windows to get things that aren't part of the core os.
    edge, sticky notes, etc.. should all be able to be updated on their own release cadence, not part of a windows update.
  • Sticky notes is, not many apps are really released with the OS updates, even if new features are sometimes announced at the same time.
    Edge however is really left untouched if the OS is not updated however...
  • Unfortunately the OS and apps are closely tied for the "big features". Since they're using UWP apps for most of the core apps, in order to get the new features in say, Edge, or Mail, those app teams work with the Windows teams to create new APIs. However, smaller updates that don't require OS API updates should be released on a more regular cadence
  • Quote: "... those app teams work with the Windows teams to create new APIs" This is a major problem. You cannot expect developers to target the platform (UWP) if you only add the APIs they need when you decide to implement a feature that requires it... and it needs a major OS update to propagate to users' machines (and these updates are staged slower than Carrier Android updates, easily). This has been the problem with UWP since Day One. It was the problem with Metro as well, and developers complained about this to Microsoft. There is a huge API gap between Android, iOS and Windows [Then] Metro and [Now] UWP. Many apps that are possible on those platforms simply cannot be implemented easily or at all on UWP. Microsoft doesn't seem to give a f*ck until they want to implement a feature that depends on a similar API in one of their apps. And with the way this is pushed to users... it's WAY TOO SLOW, especially when you consider how behind they are with this.
  • This update is actually really high quality, and I've really been enjoying it. However, Windows 10 could really benefit from a fully fledged update focusing entirely on optimizations, bug fixes, polishing and UI changes. Windows 10 looks better with every update but it's still drastically inconsistent and its backend is still a mess.
  • that's 19H1 builds so far
  • That's probably because they didn't want insiders to rush to 19H1 before 1809 was really delivered.
  • Not sure how "be more transparent about issues" will help to avoid them? Will wait for next posts where they may mention more QA personal that was reduced before Win10.
  • There are none left to fire...Nadella made sure to ditch them all, to save money.
  • While I like the update overall, I'm still missing the brightness toggle on my Surface Pro 4, which is annoying.
  • "but this is the first (and hopefully last) time this has happened with a major release" really? How about an update that had the deleted desktop bug that needed a fresh install in most cases?
  • I never had that problem.
  • My mom's laptop did and she was left without a computer for like 3 months, cause I am living in a different city and couldn't fix it for her...
  • There have been problems with updates, but not as bad as this one where Ms had to stop the update.
    The problem is these large updates are more or less re-installing the whole OS.
    I think they should stick to just updating the UI, bugs and security and forget about the new features, which the majority of people do not use.
  • "the company has seen the volume of support calls and chats drop, as the amount of Windows 10 users increased, suggesting that" the support is worthless practically.
  • Could be. People just gave up on the msft support and seek assistance elsewhere.
  • Their customer support is a joke, bunch of incompetents reading from a .doc file the same solution every time: reinstall your PC, or searching on the web for a solution.
  • "Is it okay if I log into your PC remotely." Lol. No... No, it's not okay. But thanks for asking, anyways.
  • I'm just happy that Microsoft not only fixed the issues found, they also took steps internally to ensure quality is up-to-par to prevent recurrence. This is the first step in effective Problem Management as it relates to ITIL. I give Microsoft credit for doing the best they could with the information they had.
  • I wonder what will you and they say next time there is a big fat f**k-up that inevitably will happen.
  • They will say the same things again and again...crap talking everytime with no actions. Nadella style.
  • and you sir? are you well versed in computer programming to prevent such bugs ? if so please do make a platform without any bugs at all and put it out there cause iOS has its own bugs, MacOS has its own bugs, Linux has its own bugs, Android has its own bugs.. it's man made, there will always be some flaw when innovation or change is coming, its not always preventable but it can be controlled or reduced, I've built certain programs for projects and i know that regardless of knowing what you do or even doing everything by the book like following a proper development system, it still doesn't mean that there wont be some kind of new issue after the whole application is built.
  • The level of mediocrity in windows 10's quality is way beyond the other systems you have mentioned...Windows 7 did not had such a plethora of issues, not even windows 8.
    With every update, MS is messing up the driver model, useless features, half baked functionality...
  • What's the point in calling for support, you get the well unfortunately you'll have to reinstall.
  • Scheduling updates based on timing (e.g. every 6 months) is no longer useful. MS should simply add new features / new programs when they are ready - e.g. include them in the monthly cumulative updates. I'm wondering how awkward this update will be at this time - entering the chaos of Black Friday and year end business.
  • They can't do cumulative updates due to the way they support Enterprise SKUs. We get to suffer because they are trying to stick to deadlines tied to Enterprise. We are just Beta Testers, anyways, so they're very happy "you guys" caught the bug for them. Better you get your data deleted than their friends at some Fortune 500 company! Cause you know.... their data is definitely "worth more" than yours. I'm surprised any company can trust Windows 10 or Microsoft after this massive fail, frankly.
  • They'd better put their money where their mouth is on the next update. They can't afford another debacle like this. The Windows 10 brand is getting trashed while these quality problems continue to plague the OS.
  • How about giving everyone the ability to not install an update? At the very least, let us delay the update until the next update comes out and then you can shove it down our throats like you currently do. That would mean noone would be more than 6 months behind and yet we could all delay updates until we know they are safe and are ready to apply them.
  • You can already delay updates, there is nonsensical troubleshooter tool that "hides" updates but it's practically useless if you're using Windows 10 home.
  • There are ways to disable it in windows home, then just enable updates when you want to, not recommended, but it can be done.
    I use Pro and all my updates are delayed for as long as i can.
  • Exactly, this is only useful on the Pro version and with MS removing the Pro version even from devices like the Surface Pro, this means less and less people can delay updates which is why they are doing this. So I'm suggesting them simply allow everyone to go in and easily choose to delay the update for up to 6 months or until the next update is released. That way we have some control and they only have to support the last 2 versions.
  • What on earth are you talking about, surface pro's on my ship with windows pro.
  • You can delay updates up to a year, IIRC, but you have to have Windows 10 Pro. Also, I think this may also stop you from installing Driver Updates and other things, so they've rigged the system such that you're basically forced to be force fed these updates. Game developers really need to start supporting Linux, FFS. That's the only reason I'm locked into this platform, ATM... Next Console generation may finally release me from this prison, however.
  • I'll believe it when it happens until then this is just lip service and PR damage control. I get the principle reasons why they are "self hosting" or whatever they want to call it. I still believe that it's too much too place on developers to implement, test, reiterate every single feature as there aren't enough coding hours to do so. Furthermore, this isn't the only time insider feedback wasn't even addressed. I gave up becoming a Windows insider when Microsoft took honest feedback and decided to literally axe all of phones that could supported consecutives updates of WM10. Instead of acting upon that feedback and optimizing the software. Some.people may have crappy short term memory, however I don't - still remember the fact they shipped outlook with zero ability to attach files in Windows phone 8.0. It only opened the photo hub, the only ways round it was to share a file but that meant losing/breaking the email chain. Or you had to attach a onedrive/cloud share link (which incidentally was their go to solution). They were adamant the phone o/s didn't need a file picker/file explorer smh... So I'll believe this blog post (by Microsoft) when it happens.
  • If Apple would release OS X free for compatible hardware, they'd take a large chunk out of the Windows market.
  • And then it would have the same problem as Win10.
  • macOS is actually amazing. You should try using it sometime, rather than being a retarded fanboy. It's amazing. The overall quality is on a different stratosphere compared to Windows 10, and it's almost thoroughly consistent (only very few exceptions). It's also an amazing development platform (as in, a platform to target), because Apple doesn't have 20 programming languages and 5 different SDKs/Application frameworks. They know how to deprecate and move on when it's necessary. The issue with Mac is nto the platform, it's the value of the hardware (outside of looks/design) at their respective price ranges. The platform itself is excellent. Better than Windows, IMHO. I use both: day after day, side by side, on the same desk. There is no comparison. I don't think I've ever had macOS BSOD. I've never seen Apple's software like iWork, GB, iMovie, Safari, etc. crash on my iMac - ever. Even across several updates (Yosemite -> Sierra -> High Sierra -> Mojave) I have had 0 issues with drivers (even third party, like Razer, Logitech, etc.) or with the update process, etc. Nor did I have to wait months or download a tool to force the update on my machine... it's there on Day One. It is the most reliable platform I have ever used... including in comparison to Microsoft's own Surface devices (1st party, so supposed to work yet always having bugs introduced with Microsoft Windows updates, etc.).
  • What compatible hardware are you talking about? The only compatible hardware for OS X are Apple computers. If they ever tried to make it compatible with other manufacturer's hardware they would see what sort of a nightmare Microsoft has been going through for over two decades.
    A monumental task, which Microsoft might not be handling perfectly, but Apple would not stand a chance - they are more than 20 years behind as they only ever developed for their own hardware.
  • So, is it a fair comment to say that the bulk of MS QA work is done by Insiders rather than MS itself? That would be extraordinary if true. Has anyone sought Nadella's position on quality assurance?
  • He does not care about quality...
  • Yeah right, more crap talking from MS. Their garbage as a service is a joke, like their OS. The quality became = ZERO when that obsessed imbecile fired the testers.
  • In my ideal world whenever an update occurs, there should be a tick box for following items: Do you want to update / replace installed
    1) Display drivers (yes / no)
    2) Sound card drivers (yes / no)
    3) Scanner drivers (yes / no)
    4) Printer drivers (yes / no)
    5) ... I know everything is intertwined but hey one can dream right :-) In particular on older systems (some of them being upgraded from XP-> Vista --> W7 --> W8.1 --> W10) this (yes / no) option would certainly ease matters for a lot of users of legacy upgraded systems. A friend of mine is heavily in to home music production. He has 2 "ancient" workstations that are solely used for sound matters. There is no need to upgrade them as for music production these workstations have plenty of horsepower. One of his systems had a SB card he managed to upgrade his 10 year old system to W10 without any issues. Everything worked perfectly (Mixcraft, Audacity, Cubase.. it all worked). And then in spring his system was "automatically" upgraded to 1803. His SB is no longer recognized, famous red X on his audio task bar icon etc. I tried to help him out but we gave up and he returned to the previous W10 build. So I was wondering how come that his system was automatically "qualified" for the 1803 upgrade whereas it broke his sound card drivers? So room for improvement in the quality checking of the automated upgrade process...
  • > "qualified" for the 1803
    Years ago, Alienware team blocked my old Alienware NB from Win10 update, even when I click on the notification manually.
    Everything is done with automatic system nowadays. I think HW makers will need to give a go or no-go sign on this.
    A custom made PC is more troublesome because, every user has different combination of HW parts, drivers, Windows versions, user installed background services, etc. There's a lotta more variable than a brand PC. Have you ever seem fans of different card maker bashing each other over which $500 card crashes more? Those cards do need to score a 100 in their QA test before they got release to the world no? Unless you have the exact same PC setup as those QA machines, issues might happen. iOS and Android is more simple don't you think? Since there's no custom phones and no driver updates.
    Apple deals only their HW, no OEM. Google has no control over OEM, OEM tamper the OS / API as they see fit. Yet, Apple has managed to screw up auto correction, camera cease working, phone cease to boot and data wipe happened after update. Backup with iTunes gets corrupt or incompatible. Android update too introduced mem-leak or connection issues. Pixel3 has mem leak, double notch, overheat during charging. Galaxy flagships give devs backscreens when trying to use the latest Android API and OpenGL. Some Xperia models have audio playback lag. Well... we are all just human.
  • Well, this PC was not custom made as I was told, apparently it was a workstation ordered at Dell, including SB sound card installed ex. factory.
  • Disabling doesn't function.
    I am wondering what the EU commision will think about this.
    This for sure is an intrusion on a machine I own. This is like a burglar coming into my house. I hate companies who think this is correct. This is NOT correct.
    When I say NO in my settings, I mean NO and not maybe or yes.
  • Who else believes that the reason MS has seen a drop in support calls is due to them outsourcing support to a not so robust third party?
  • I do not care about transparency and communication, it is enough if they roll out good software and then nothing else is needed. How exactly can they explain disasters like Skype? Recently i had to abandon the UWP version, as it reached version 8x and it would crash upon launch on a SP 2016. And whenever it would run i had to put up with all the missing functionality compared to the desktop version. A f***ing mess... I uninstalled it and went back to a 7x desktop version, which works and offers full functionality. How they managed to transform the pinnacle of video conferencing software into this crap it is today is beyond me. And Windows is heading that way too as apparently features > quality & performance. But who cares about those when we get ninja cat wallpapers and Dona Sarkar meeting photos?
  • The entire guinea pig insider program is nothing but a bad joke, the worst public beta system in history. But hey, MS is fine with it, that's what matters for them, oh and yes, ninja cats, hustling and useless features.
  • Assuming the self-hosting process is already in place and they're employing it, how are they planning on improving QA? All I got from the article is that they're not backtracking on WaaS and going full steam ahead as before.
  • Complaints are so overblown. We forget how much Windows 10 has improved. Microsoft has the guts to do two major releases a year. Kudos to them. Hope they stick with the pace. The supremely innovative features we have received make the tiny bit of instability palatable... Set Aside Tabs in Edge, Pick up Where you Left Off, Cloud Clipboard, and Cortana's many helpful suggestions in Edge just a few of the many revolutionary new features that have changed our lives forever for the better.
  • Senior management is continuing to stonewall on the file deletion issue. An unknown number of users were affected by Microsoft's negligence. I assume their legal team told them to keep quiet which explains Microsoft's silence. This is not acceptable. Files were permanently deleted. This was a serious failure and breach of trust which must be acknowledged and addressed by Nadella himself. The silence is deafening. Just because the number of users may be small does not justify Microsoft's guilt. Taking responsibility requires action. Hiding and pretending nothing happened should alarm everyone concerned about the serious problems within the organization. Supporters of this company should demand action not add to the pile of excuses.