Here’s the secret behind Microsoft’s AI agents — meet Mu, the compact AI model transforming Settings on your PC

Microsoft just unveiled a new on-device small language model that powers AI agents in Windows 11's Settings app. The model is called "Mu," and it runs on NPUs (Neural Processing Units) to take full advantage of the AI-optimized hardware seen in Copilot+ PCs.

Those agents, which were announced earlier this year, will be able to automate and execute tasks on your PC if you've granted the appropriate permissions.

For example, you could type into Settings, "my mouse pointer is too small," and an AI agent within Settings will present the option to increase the size of your mouse pointer from 1 to 5. A large "apply" button will appear and let you enable the suggested setting.

AI agents in Settings are now in testing among Windows Insiders in the Dev Channel. We now have more information on how they work and how the model that powers the agents was developed.

Mu was designed by Microsoft to run efficiently on NPUs. Developers of the model considered the constraints of NPUs, such as memory limits, to best take advantage of the hardware. The model also exclusively uses NPU-optimized operations to avoid inefficiency.

Microsoft's blog post provides insight into how Mu was developed and how the tech giant optimized the model to work within the constraints of NPUs. Vivek Pradeep, Microsoft Corporate Vice President and Distinguished Engineer for Windows Applied Sciences, breaks down the topic and shares quite a bit of detail.

MU: Small Language Model Inference on NPU - YouTube

Watch On

I suspect most users are more interested in what Mu can accomplish than how the model was developed, but Pradeep's post is an intriguing read, and I recommend taking some time to study it.

"Our goal was to create an AI-powered agent within Settings that understands natural language and changes relevant undoable settings seamlessly," said Pradeep. "We aimed to integrate this agent into the existing search box for a smooth user experience, requiring ultra-low latency for numerous possible settings."

Microsoft developers tested several models but needed something smaller than what was previously available to meet latency targets. Mu was developed and fine-tuned to hit the company's targets.

"The Mu model [(fine-tuned)] achieved response times of under 500 milliseconds, aligning with our goals for a responsive and reliable agent in Settings that scaled to hundreds of settings," explained Pradeep.

AI agents in Settings and the Mu model that powers those agents are now in testing among Windows Insiders. Those enrolled in the Windows Insider program can also test a major update to the Recall app.