What you need to know
- Microsoft's DeBERTa AI model outperformed humans in a test of natural language understanding.
- The AI earned higher marks than the human baseline in the SuperGLUE test.
- Google also has an AI that beats the human baseline, though Microsoft's AI model scores higher on the same test.
Microsoft invests heavily in artificial intelligence in a wide range of sectors. One of those sectors is natural language understanding, which aims to have AI models understand everyday speech. This is a particularly tricky challenge for machines, but Microsoft's DeBERTa AI model recently scored higher than the human baseline in the SuperGLUE test.
As explained by Microsoft (opens in new tab), SuperGLUE is one of the most challenging benchmarks for natural language understanding. Microsoft shares an example in its recent blog post:
This is a simple question for humans. We have background information and are used to placing things within context, but it's a challenging question for AI. To make an AI model answer this question correctly, it needs to understand cause and effect, and both options presented to it. The SuperGLUE test includes natural language inference, co-reference resolution, and word sense disambiguation, as explained by Microsoft.
The DeBERTa model was recently updated to include 48 Transformer layers and 1.5 billion parameters. As a result, the DeBERTa model earned a macro-average score of 90.3 in the SuperGLUE test. The human baseline for the same test is 89.8.
Microsoft states that it will release the DeBERTa model and its source code to the public.
Microsoft explains that the DeBERTA AI model beating out humans in the SuperGLUE test doesn't mean that it's as intelligent as humans.
Microsoft's DeBERTa model isn't the first to beat the human baseline on the SuperGLUE test. Google's T5 + Meena" model hit a score of 90.2 on January 5, 2021. Microsoft's DeBERTa model beat Google's with a score of 90.3 just a day later.
Sean Endicott is the news writer for Windows Central. If it runs Windows, is made by Microsoft, or has anything to do with either, he's on it. Sean's been with Windows Central since 2017 and is also our resident app expert. If you have a news tip or an app to review, hit him up at email@example.com.
Get the best of Windows Central in in your inbox, every day!
Thank you for signing up to Windows Central. You will receive a verification email shortly.
There was a problem. Please refresh the page and try again.