Ubisoft and NVIDIA are working together on AI-generated NPCs for upcoming games

What you need to know

• A small R&D team at Ubisoft's Paris studio is working on using AI with NPC interactions in games. 
• Ubisoft is working with Nvidia's Audio2Face application and Inworld's Large Language Model (LLM) to test how effective AI can be in games. 
• The NEO NPC project "uses GenAI to prod at the limits of how a player can interact with an NPC without breaking the authenticity of the situation they are in, or the character of the NPC itself."

NVIDIA is practically on top of the tech world, if you haven't noticed. It's now one of the top-valued companies in the world and its stock is continuing its insane upward trajectory. In the midst of all of that success, NVIDIA is working on smaller and larger initiatives, but they all seem to have one thing in common: they are AI-centered. 

One of these smaller initiatives comes from Ubisoft Paris, where a small team is testing out how to use AI, specifically Nvidia’s Audio2Face application and Inworld’s Large Language Model (LLM), to try to make a new generation of NPCs. Just in case you haven't heard of an NPC, it stands for a non-player character and refers to characters in a game world that are driven by the game code and not by another player.

How can AI be used for NPCs in games?

NPC dialogue has always been one of the most extensive and labor-intensive workflows for video game writers. Some larger RPGs, like Starfield, are well-known for having huge scripts. Starfield's script had a nearly unimaginable 150,000 lines of dialogue in its script. AI's promise and anxiety to the video game industry and many other industries is that it could help alleviate workloads, which could help human developers get more done. Still, it could also lead to layoffs in a worst-case scenario. 

As we see many studios, especially under Microsoft, begin to form unions, like the recent announcement from Activision QA workers, it might be possible to alleviate some of the risks around introducing AI. As long as workers' careers are protected, introducing AI into game development could make for a better, more immersive product and possibly cut down development time. 

One of the things highlighted by Ubisoft is that instead of having to write thousands of lines of dialogue, a writer could focus on creating a personality in an AI LLM with a backstory, motivations, and personality traits. This could then allow the player to have a genuine conversation of discovery that provides a bespoke unique experience but would always still be true to the human writer's intention. I think this scenario is a win-win for everybody involved. 

What is the future of AI in gaming?

As an Xbox fan, I'm slowly realizing that Microsoft will likely start integrating Copilot into the Xbox division sooner rather than later. Copilot has led to Microsoft becoming the most valued company in the world, and it makes little sense for Microsoft to ignore AI in the gaming sector even if the die-hard fans aren't excited about it.  

There has been a mostly negative sentiment about AI in gaming over the last few years, possibly due to a partial correlation of AI with things like Blockchain and NFTs. However, with the improvement of ChatGPT over time and image and video generation, there seems to be a more open mind around the idea of having some games use integrated large language models (LLMs) for NPC interactions. 

There have even been mods for popular games like Grand Theft Auto 5, where you can talk to NPCs with ChatGPT running to answer queries. Unfortunately, there is a long delay while ChatGPT takes the input, calculates an answer, and then speaks it back to you.

While this is a fundamental implementation, as AI voices get better and can be tuned to use paid voice actors' voices, as we saw in the 8 hours of AI-generated voice acting in Galactic Civilizations IV Supernova DLC, this type of technology could be a new step forward for game development.