Cortana 'conversational AI' enables smarter voice assistance

Today at Build 2019, Microsoft announced a new conversational engine that will integrate with Cortana and enable new capabilities and more advanced interactions. Known as "conversational AI," Microsoft has been working with the technology it acquired when it purchased Semantic Machines late last year, crafting it into a "breakthrough new conversational AI technology that will power a new class of multi-turn, multi-domain, and multi-agent experiences," according to the announcement post.

In the past, Cortana, like most virtual assistants on the market, has been limited in how you talk to it. To interact with a virtual assistant, you must say things in the correct way for it to understand you, and you can only do one thing at a time. With conversational AI, this requirement is relaxed. Cortana should now better understand natural human conversation and be able to handle multiple queries and skills at once.

From the Microsoft announcement:

Our technology builds up memory from turn to turn to let you get more done. It crosses skill boundaries, connecting together back-end services, both within Microsoft and externally. This new conversational engine will be integrated into Cortana and made available to developers through the Bot Framework, as well as other Azure surfaces and beyond, powering conversational experiences across the company and our customer ecosystem.

The news also means Cortana can better handle "umms" and "ahhs" during conversation flow and can string together multiple different commands said at different times. This is one of the first significant updates to the Cortana experience since Cortana came to the desktop, iOS, and Android. This is also our first proper look at Cortana as a productivity assistant, as today's new conversational capabilities are all designed to help you stay productive.

The new conversational engine will also be made available to developers, allowing third parties to build their own virtual agents for specific use cases. It's unknown when we can expect to see this new engine show up in Cortana, but Microsoft says it should arrive relatively soon. In the meantime, what are your thoughts on these new Cortana experiences? Let us know in the comments.

Zac Bowden
Senior Editor

Zac Bowden is a Senior Editor at Windows Central. Bringing you exclusive coverage into the world of Windows on PCs, tablets, phones, and more. Also an avid collector of rare Microsoft prototype devices! Keep in touch on Twitter: @zacbowden.

  • Can I flow through commands like "Cortana, ask Alexa to turn on the living room lights?" or "Cortana, order a large pepperoni pizza from dominos."
  • That's been a capability of Cortana since it launched.
    Not many have implemented it though. The Build demo was back and forth dialog invoking many different "skills"
    There was a meeting reschedule
    A scheduling query which was followed up by a request for weather which was contextualized to the lunch date. Finally the user then said, send instructions to my car navigation and bam, another skill invoked, all within the same conversation. Much more like talking to a real person.
  • And once again, the "Cortana is dead" takes appear premature at best. I'm glad they're still working on this, perhaps this platform level update is what has been needed for the next stages of Cortana. I assume this work wouldn't be done if she was going to be killed off
  • Windows Phone continued receiving updates and "improvements" long after it started its decline into the abyss as well.
  • it received an update last month too
  • Give it a couple months, the takes will be back at the end of summer
  • OMG CORTANAZ DEA-- Oh, right, sorry.
  • Microsoft: we having amazing AI!
    The World: how’s that possible when your digital assistant is so awful?
    Microsoft: Umm...ahh… So now you know why Cortana got better with “umms” and “ahhs”.
  • MCGA!!! 😢😢😢😢... Come on MS!
  • Great news.. Keep the update coming...
  • Looking forward to seeing how this will improve the experiences on my phone or PC.
  • On your Android phone, or WP?
  • I'm cautiously optimistic. This is where I wanted this to go. But it will NOT be good if it will be buried inside Office. If the demo in the video is an indication, it seems like it can be up front. I just need it to be ABLE to be 100% hands-free, as it is now on native Windows (it's embarrassingly lame on all other platforms). My HOPE is that this new conversational AI will be able to finally integrate all instances of Cortana existing on any device.
  • I mainly use Cortana on my Invoke in the kitchen to play music on Spotify and give me the weather or tell me if a store is still open. The have promised Pandora since it was released but it never came. I'm not holding my breath but I'm wondering if I can start up Alexa from Cortana and the start Pandora that way? If they could just enable Pandora, I would be happy.
  • Can't wait to build conversational AI into my desktop apps! I can literally think of about 20 use cases for this right off the top of my head.
  • I have two Invoke's in my place and I'm always afraid of saying "Hey Cortana"--never a problem with my Echo Dots and "Alexa..." Amazon seems to have done very well in handling the interaction at the closest speaker while dismissing further activity on further-away speakers.
  • Is there any hint as to when this conversational AI mode will come to Cortana?
  • tbh, as long as enterprises, IOT and devs are still using MS's AI and deep learning (りんな and other variants are still active too)... the research and investment of AI and deep learning will continue.
    And MS can name those AI whatever it wants e.g. Cortana? We have those AI cashiers in my town, it's reading people's face (gender, age and all that) and monitoring people's behavior. (Once shoplifted, flagged forever I guess?) Who knows... maybe one day, MS will allow enterprise to use MS's AI (and call it a "Cortana skill") to interact with those data, to interact with your device, your computer, your account, cross machines (car talks to kiosk and if MS start calling it Cortana... suddenly we have a Cortana in our car, able to order our Starbucks coffee from miles away for us) and cross business entities.
    Office365 uses AI too and one day, MS might as well just call it a "Cortana skill".
    Enterprises are using Microsoft Cognitive Services too right? The investment / study of MS's (and devs and machines are using it too) AI & deep learning is dead? That's a weird logic.
    Name is not important when we are talking about programming. * AI/machines talk to AI/machines.
    Cashier, arcade cabinet, fridge, robot, surveillance, car, coffee machine, water meter, etc that runs Windows (WCOS) and connect to Azure.
  • it would be nice if MS allowed an easy way to disable cortana, then I would not have to delete the directory it is in.