Skip to main content

Microsoft researchers master Ms. Pac-Man with AI

A research team at Microsoft has used an artificial intelligence (AI) system to master Ms. Pac-Man for the Atari 2600. What makes the feat so impressive is that the conglomeration of AI agents working in unison managed to achieve the highest score possible, 999,990. That's something that even no human has ever achieved, the team says.

Video games, it turns out, are an excellent testing ground for teaching AI to exercise human-like intelligence. Ms. Pac-Man itself is particularly well suited to this task because it was designed to be less predictable than Pac-Man.

To take on the task, the researchers concocted a system of more than 150 AI agents working together to obtain the best outcome from any given move. From Microsoft (opens in new tab):

The method, which the Maluuba team calls Hybrid Reward Architecture, used more than 150 agents, each of which worked in parallel with the other agents to master Ms. Pac-Man. For example, some agents got rewarded for successfully finding one specific pellet, while others were tasked with staying out of the way of ghosts.Then, the researchers created a top agent – sort of like a senior manager at a company – who took suggestions from all the agents and used them to decide where to move Ms. Pac-Man.The top agent took into account how many agents advocated for going in a certain direction, but it also looked at the intensity with which they wanted to make that move. For example, if 100 agents wanted to go right because that was the best path to their pellet, but three wanted to go left because there was a deadly ghost to the right, it would give more weight to the ones who had noticed the ghost and go left.

The research comes from Candian company Maluuba, which Microsoft only just acquired earlier in 2017. According to the team, the results of this research could prove valuable for researchers across the AI field, but particularly in the area of reinforcement learning. Eventually, that could lead to AI that can make decisions on its own, eventually allowing it to take over complex tasks currently performed by humans.

Dan Thorp-Lancaster is the Editor in Chief for Windows Central. He began working with Windows Central as a news writer in 2014 and is obsessed with tech of all sorts. You can follow Dan on Twitter @DthorpL and Instagram @heyitsdtl. Got a hot tip? Send it to daniel.thorp-lancaster@futurenet.com.

10 Comments
  • We are so close to SkyNet...
  • That is good news! I think...
  • Engineers are able to solve so many difficult problems and come up with things that we never thought possible, but doctors on the other hand are still telling us that they have no cure for cancer. They're more focusing on hatching women breasts off their chests
  • Because understanding cancer is the same as playing pacman....  /s
  • your sarcasm is pointless. Obviously I wasn't just referring to pacman. The point of the story was to highlight advancement in Artificial Intelligence. PacMan was used as an illustration there. Think before you write, bro! Or Take a chill pill
  • How could you possibly blame doctors? If anyone, it's Big Pharm's fault for thinking/knowing there is more money in giving rich white men erections than there is in researching drugs that may not work in the 20 years it takes for development and testing to cure poor people of terminal diseases. I realize that its not their fault either, but when you have a company not just raising, but gouging its customer base while its CEO is taking record profits, how can one not think that?
  • even if we "could" cure ALL diseases we "wouldn't" cause the overpopulation issues we have now would be exacerbated exponentially.
  • They cured polio though, and the measles, and a bunch of other things.
  • You know those guys doing boobjobs are not the same as the one doing cancer treatments...
  • Impressive.