Microsoft's neural language AI model surpasses human performance in SuperGLUE test
Microsoft continues to improve AI's understanding of language.
What you need to know
- Microsoft's DeBERTa AI model outperformed humans in a test of natural language understanding.
- The AI earned higher marks than the human baseline in the SuperGLUE test.
- Google also has an AI that beats the human baseline, though Microsoft's AI model scores higher on the same test.
Microsoft invests heavily in artificial intelligence in a wide range of sectors. One of those sectors is natural language understanding, which aims to have AI models understand everyday speech. This is a particularly tricky challenge for machines, but Microsoft's DeBERTa AI model recently scored higher than the human baseline in the SuperGLUE test.
As explained by Microsoft, SuperGLUE is one of the most challenging benchmarks for natural language understanding. Microsoft shares an example in its recent blog post:
This is a simple question for humans. We have background information and are used to placing things within context, but it's a challenging question for AI. To make an AI model answer this question correctly, it needs to understand cause and effect, and both options presented to it. The SuperGLUE test includes natural language inference, co-reference resolution, and word sense disambiguation, as explained by Microsoft.
The DeBERTa model was recently updated to include 48 Transformer layers and 1.5 billion parameters. As a result, the DeBERTa model earned a macro-average score of 90.3 in the SuperGLUE test. The human baseline for the same test is 89.8.
Microsoft states that it will release the DeBERTa model and its source code to the public.
Microsoft explains that the DeBERTA AI model beating out humans in the SuperGLUE test doesn't mean that it's as intelligent as humans.
Microsoft's DeBERTa model isn't the first to beat the human baseline on the SuperGLUE test. Google's T5 + Meena" model hit a score of 90.2 on January 5, 2021. Microsoft's DeBERTa model beat Google's with a score of 90.3 just a day later.
Get the Windows Central Newsletter
All the latest news, reviews, and guides for Windows and Xbox diehards.
Sean Endicott brings nearly a decade of experience covering Microsoft and Windows news to Windows Central. He joined our team in 2017 as an app reviewer and now heads up our day-to-day news coverage. If you have a news tip or an app to review, hit him up at email@example.com.