What you need to know
- A new post from Microsoft breaks down how Project Tokyo helps blind people recognize faces.
- The project uses artificial intelligence and specialized hardware to help send cues and information to blind people.
- The post focuses on a 12-year old boy, Theo, and the growth of Project Tokyo.
Microsoft continues to use technology to improve accessibility. A new blog post from Microsoft shares the growth and development of Project Tokyo, a combination of technologies that extend people's capabilities. Project Tokyo uses artificial intelligence, specialized hardware such as a custom HoloLens, and the results of several research efforts to help improve accessibility. One of the key areas that the post highlights is the ability to send spatial audio cues and information to blind people and people with low vision.
The post focuses heavily on the story of Theo, a 12-year old boy who uses a customized HoloLens to improve his communication. The headband detects people's faces, and if a face is on the system, sends an audio cue to Theo to tell who who he's facing. The post explains that some blind people can miss cues that people with full vision take for granted, such as knowing when someone is making eye contact with them. The system Theo uses helps communicate those cues and gives Theo and others an improved sense of connection.
Theo states that "It is so exciting to be able to find out where the people are in my environment," adding, "Not just who chooses to talk, but all of the people who are silent that you can see by their face, but I can't."
Project Tokyo isn't restricted to vision-related aspects of communication. Researcher Cecily Morrison views the future of AI as a set of resources that people can use in a variety of ways, "All of a sudden we don't have to say, 'Hey you are blind and I just made this accessible to you.' We say, 'Hey, you are you and I have just built a system that works for you.'"
Project Tokyo came from a challenge in 2016 to create AI systems that could do more than simple tasks. The blog post shares the story of how a team from Microsoft followed athletes on a trip to the 2016 Paralympic Games in Rio de Janeiro, Brazil. The trip helped Microsoft's team understand communication and group dynamics affected by blindness and vision impairments.
Ed Cutrell, a senior principal researcher at Microsoft, states, "We, as humans, have this very, very nuanced and elaborate sense of social understanding of how to interact with people – getting a sense of who is in the room, what are they doing, what is their relationship to me, how do I understand if they are relevant for me or not... And for blind people a lot of the cues that we take for granted just go away."
People working on Project Tokyo realized that the HoloLens already picks up a lot of the information that they needed in a system. It has a wide field of view, can recognize faces, and has speakers that support spatial audio. These features helped make a system that can identify people, contextually draw people's attention to a location with spatial audio, and to deliver cues to improve communication.
In addition to improving communication directly, Microsoft is looking into if Project Tokyo can help teach blind children or children with low vision to improve interactions.
Microsoft plans to expand Project Tokyo in the future. Its goal is to be able to adapt the system to different situations and preferences.
We may earn a commission for purchases using our links. Learn more.
Surface, Windows won't hit revenue guidance due to coronavirus
As the effects of the coronavirus are felt throughout the world, Microsoft warned today that it won't meet its expectations for its Surface and Windows revenues when it reports its next quarterly earnings.
Microsoft Edge's Surf game splashes into the Dev channel
Now you can get your Surf on in the Microsoft Edge Dev channel. The latest update brings the Easter egg to Dev channel testers, along with a tweak that lets you add the share button next to the address bar and more.
NBA 2K21: Everything you need to know
The NBA 2K series is a perennial favorite for good reason, and expected to elevate its game on next-gen consoles with NBA 2K21, we fill you in on everything you need to know.
Check out this killer compact AMD Ryzen 9 3900X gaming PC
The NZXT H1 is a compact case that allows you to pack it full of powerful components, including an AMD Ryzen 9 3900X and RTX 2060 Super. Here's a killer gaming build.