Microsoft's dependence on data trends and telemetry lacks human intuition
Does Microsoft's decision making need a more human touch?
This article is unscientific. I don't have any data to back up my claims. And indeed, the opposite of my claims could very well be true. I'm also in a very easy position. My choices don't impact hundreds, perhaps thousands of employees, their families, and shareholder investments. It's with this in mind that I very much understand why Microsoft is so obsessed with data. Yet, their decisions are often baffling, and produce cascading unintended effects that lead to partial, and sometimes, total, retraction.
Case in point: Last week, Microsoft announced that new customers on Xbox Live Gold would need to pay twice to access online multiplayer what its closest competitor charges. A year of Xbox Live Gold was going up to $120 dollars, while Sony's PlayStation Plus costs only $60. The backlash was rapid and intense across social media, with Reddit communities incensed, YouTube rendering queues enflamed, and blogs in rapid-fire. And hey, I even wrote my own article on the matter.
While I was asleep (and yes, I do sleep sometimes!), Microsoft not only did a total 180, reverting the price increase, they also lifted Xbox Live Gold's subscription requirement on free-to-play online games, like Fortnite and Rocket League. So, in some crazy way, we're actually better off now than before the backlash, although it stands to reason the free-to-play restriction was probably already in the works.
Regardless, this isn't the first time Microsoft has had to revert an unpopular change. And, wholly unscientifically, I feel it in my gut that it has something to do with Microsoft's over-reliance on "data."
Obviously, data is extremely important
Microsoft is a massive company with massive audiences. Using the Xbox Live Gold issue as an example, Microsoft says there are roughly 65 million or more Xbox Live users on a monthly basis, and while all of those aren't necessarily Gold subscribers, a large chunk of them undoubtedly are.
Millions and millions of customers all have different motivations and values, and understanding these motivations and values is of critical importance when building platforms and products. It's user research that probably let to Microsoft splitting the next-gen Xbox offering two SKUs, with a more affordable Xbox Series S targeting users who have no intention to upgrade to a 4K UHD TV, and the Xbox Series X for the cutting-edge hardcore gamer.
Microsoft is on the record saying that they expect the Xbox Series S to sell more over the span of the generation, and historical trends and data would suggest that they're right. Some things are a little more measurable than others, though.
When it comes to things like chemistry and physics, we can measure things more accurately using tools and mathematics. The problem with user-driven data is that it assumes a human being is a reliable source of information. It assumes that a human being knows what they want, and what they value when asked to recall it for a survey. Social and psychological studies tell us that this is often not the case, and when you merge this information with an agenda, biases can emerge.
Purely hypothetically, someone at Microsoft might've thought that data suggesting most new users go straight to Xbox Game Pass Ultimate was evidence enough that Xbox Live Gold could be essentially "phased out" by pricing itself out of the market. Hypothetically again, if this was the case, did Microsoft account for the empathy its customers felt despite the fact most of them weren't going to be affected by this change?
What about the human element?
Microsoft made some effort in this whole debacle to avoid a backlash by grandfathering existing subscribers at the same price they pay now. If you are a current subscriber in the Xbox Live Gold ecosystem, you wouldn't have been affected by this change at all. In fact, Microsoft even offers long-term Xbox Live Gold members the opportunity to convert their subscription to Xbox Game Pass Ultimate at no additional cost, which is a huge saving.
I do wonder if Microsoft thought that most people wouldn't have cared about all of this, because the vast, vast majority of users weren't going to be affected in any way.
When I read the news, the first thing I thought wasn't "heh, this isn't going to affect me because I have Xbox Game Pass Ultimate until 2023 anyway," it was "well, that really fucking sucks for parents of small kids who may be struggling in the midst of this unprecedented global pandemic." I have memories of my parents fighting over money. We used to get all games second hand. There's a huge debate in the UK over making sure kids from poor families are still able to access free meals while schools are closed.
I've seen callous comments from people that run along the lines of "well if they can afford a console they can afford Xbox Live Gold at $120," but it's nowhere near that simple. Kids at my school would get picked on over the brand of their shoes. Parents bend over backward to make sure their kids can have the same tools and toys and "fit in" with the richer kids. When you factor in that Microsoft gloated about record engagement a couple of months ago, thanks to the pandemic, this whole business just came across as aggravatingly tone-deaf.
This is anecdotal evidence and just a single example. This isn't a research paper, it's some thoughts I wrote down in a blog post. But really, if you'd have asked me in a user research survey about how things affect me, and my financial situation, I would've answered very differently to if you'd asked me how I felt if prices were being increased for others.
Ironically, the backlash is also user data, and Microsoft reacted swiftly to that new data, reversing course. But seriously, how did they not see this coming? For a few hours, the curtain was peeled back, reminding us the "socially progressive bright cheerful fun Xbox brand" can still trip into a corporatist void if customers aren't paying attention.
Human intuition vs. trending data
I genuinely believe that the people at Xbox aren't trying to make decisions that screw people over, having met many of the people that work there. As an entity, and certainly among the other tech companies, Microsoft is the one that usually seems well-intentioned. But clearly, somewhere along the line, some form of user research and data trends were used to push through these changes. If the data had included figures on the YouTube videos, blog posts, and social media comments expressing anger for the change — despite the fact most people weren't going to be affected by it — they probably wouldn't have done it.
Hindsight is 20/20 of course, and Microsoft, like many corporations, demands perpetual growth. Many tech companies saw a boom as the work-from-home culture bit down last year, and despite that, shareholders will still want to see growth on top of that boom, even when it subsides. It falls on Xbox leadership to find where to extract growth from for Microsoft's insatiable shareholders, and falls on us to call them out on any bullshit as and when the need arises.
This whole Xbox Live debacle just reminds me of a long list of scenarios where Microsoft just ... did stuff, without thinking about how it would affect users, or the perception of the company as a whole. The way Windows Phone was just left to die without any explanation, despite having tens of millions of users. The OneDrive backlash from 2015. The slow, painful demise of the Skype brand. Microsoft often just doesn't seem to get humans.
And sure, I'm not naive. The only data trend that matters really is revenue, and in that, Microsoft is clearly doing something — many things — correctly. Microsoft doesn't have the same loyalty capital that Apple and some other companies enjoy, where their customers will simply shut up and endure stupid decisions, as evidenced by how rapidly people reacted to this whole thing.
Apologies for all the angst and emotion this caused today for our customers. As always, we appreciate the feedback. This is a good learning opportunity for us and we will learn from it.Apologies for all the angst and emotion this caused today for our customers. As always, we appreciate the feedback. This is a good learning opportunity for us and we will learn from it.— Phil Spencer (@XboxP3) January 23, 2021January 23, 2021
I'm sure there were people at Xbox telling Microsoft's finance department that this was a bad idea and that people wouldn't stand for it. It's also easy to imagine that there was someone at Microsoft armed with "data" to "prove" otherwise. How can you argue against data?
I'm certainly no data scientist, and I would hope that Microsoft does account for some of those human elements and data bias. I certainly don't expect Microsoft to take policy inspiration from a random blogger either. I also know that all of this is easy for me to say without responsibility. But, after following them for so long, and seeing the same patterns emerge time and time again, I feel it in my heart that Microsoft should at least consider factoring in some good old human intuition a bit more. Telemetry, trends, and user surveying never paint a complete picture.
Windows Central Newsletter
Get the best of Windows Central in your inbox, every day!
Jez Corden a Managing Editor at Windows Central, focusing primarily on all things Xbox and gaming. Jez is known for breaking exclusive news and analysis as relates to the Microsoft ecosystem while being powered by caffeine. Follow on Twitter @JezCorden and listen to his Xbox Two podcast, all about, you guessed it, Xbox!
Can’t argue with your thoughts Jez, there are often user surveys on the rewards app but again, that data is only targetted at the hardcore because a casual user doesn’t bother with searching the rewards app for points opportunities in the first place!
1. Were you so concerned that you "even wrote an article on that matter"? I thought that was your job.
2. As a data professional, I'd be surprised if any algorithm told Microsoft "IF user_engagement = low THEN double(membership_price)".
3. Actually the opposite could be at work:
Microsoft: New Xbox game releases, Press: Nah
Microsoft: EA games included in game pass, Press: Nah, look PS5 has shiny lights!
Microsoft: Live Gold Membership double the price, Press: Hey Microsoft is alive! And is sticking it to the users!
Microsoft: Just kidding actually I have decided to let you play free-to-play games for free, Press: Now everybody is talking about Microsoft. Jez, making the decision of making free-to-play games actually free definitely is not something that can be just mandated by Phil Spencer by email in one day as an emergency turn. Probably it was a move planned ahead of time. Jacking up the Live Gold membership prices temporarily was probably a strategy to get noticed. BTW data FTW.
I also wouldn't be surprised with being a "data driven" decision. Maybe they should stop crossing their eyes when they read that data? If they had half a brain, they would have done an incremental increase over time. Which I'm not proposing, lol. They've used data to kill off Cortana on most places, kill off Kinect all together. Cortana worked GREAT on my L950, I'd say even better than GA or Siri now, vs 4 yrs ago. My Invoke is losing it, and my son is literally distraught over it. Xbox lost it. They literally look at something and say "People aren't using it, might as well throw away that investment and kill it." instead of saying "People aren't using it. Why? Let's make it better, be consistent, and let people see how great it is but not force it on them." They constantly move on to the "next big thing" instead of making their existing product better. Xbox has been one of their few constant bright spots, despite a few missteps. This price hike was a big step to trip over.
A company clearly with talented people, but almost certainly tarnished with toxic management. Many stories of the “ill have you fired” culture. Look at Windows on ARM or UWP.
You can almost smell the internal warring factions. You could add to this article:
Aggressive/ underhand license tactics deliberately designed to ensure you “owe” Microsoft. SQL server needing a license for every VMware server it can move to under load balancing if you don’t by software assurance! Or terrible / non existent support.
Without buying a support plan, their third party outsourced support is terrible. You have to pay on top of the already high prices to get reasonable
Support. It’s outrageous. A company that does make some good products. But doesn’t win hearts or minds due to being overly focused on numbers not even data.
It's why Nadela is so hot on the Azure and Hosting side as it is seeing meteoric growth and profits and that makes the other Divisions, while still making good money, look WEAK in comparison, which in turn makes the investors clamor for him to de-fund those divisions.
They will pull-back, regroup, and figure out a way to get customers moved over to Game Pass from Xbox Live Gold another way.