Skip to main content

Microsoft's Singularity aims to reduce the cost of artificial intelligence

Microsoft Logo
Microsoft Logo (Image credit: Daniel Rubino / Windows Central)

What you need to know

  • Microsoft's AI infrastructure, which is codenamed Singularity, aims to reduce the cost of artificial intelligence.
  • Singularity allows hundreds of thousands of GPUs and AI accelerators to work together, which reduces wasted effort.
  • Microsoft has invested heavily in AI, including a $1 billion investment in OpenAI in 2019.

Microsoft is working to reduce the cost of artificial intelligence (AI) and wasted efforts when computing at a global scale. A recently published paper by Microsoft's Azure and Research teams discusses the company's AI service, which is codenamed Singularity. The paper, titled Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads (PDF), breaks down Microsoft's work at a technical level.

"Singularity is a fully managed, globally distributed infrastructure service for AI workloads at Microsoft, with support for diverse hardware accelerators. Singularity is designed from the ground up to scale across a global fleet of hundreds of thousands of GPUs and other AI accelerators," explains Microsoft's Azure and Research teams in their paper. "Singularity is built with one key goal: driving down the cost of AI by maximizing the aggregate useful throughput on a given fixed pool of capacity of accelerators at planet scale, while providing stringent SLAs for multiple pricing tiers."

In layman's terms, Microsoft's Singularity lets hundreds of thousands of GPUs and AI accelerators to work together. Singularity is a global infrastructure service designed to reduce wasted efforts. It treats all devices within the infrastructure as a single cluster, which helps ensure that the devices are used to their full potential.

Singularity can also adapt to prioritize different workloads. "While opportunistically using spare capacity, Singularity simultaneously provides isolation by respecting job-level SLAs," says Microsoft. "For example, Singularity adapts to increasing load on an inference job, freeing up capacity by elastically scaling down or preempting training jobs."

In contrast to some other systems that require restarting from scratch following failure, Singularity can jump back in where a job was cut off. This greatly reduces wasted effort as DNN training jobs can take several weeks.

Microsoft has invested heavily in AI over the years, including a $1 billion investment in OpenAI in 2019. An Azure system is one of the ten most powerful supercomputers in the world, as of November 2021. Azure systems are used for large-scale computing and machine learning.

As noted by ZDNet, Microsoft used the Singularity codename for an unrelated project in the past. That Singularity was a microkernel operating system.

Sean Endicott
News Writer and apps editor

Sean Endicott brings nearly a decade of experience covering Microsoft and Windows news to Windows Central. He joined our team in 2017 as an app reviewer and now heads up our day-to-day news coverage. If you have a news tip or an app to review, hit him up at sean.endicott@futurenet.com (opens in new tab).

4 Comments
  • "Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads" The most futuristic thing I've read all week. But why not just call it what it really is: Skynet? 😁
  • cus every government and enterprise is going to start worrying 😁
  • Uh, um, OK.
  • Really, just really... Microsoft went with 'Singularity'? Either they are just tempting fate or have a very twisted sense of humor.