Microsoft has launched a new app that builds on its efforts in the artificial intelligence (AI) space to help the blind and visually impaired, in a way, see the world around them. Fittingly called Seeing AI (opens in new tab), Microsoft says the app "narrates the world around you," helping to describe people, objects, and even text wherever you point your camera.
What's particularly impressive about the app is the breadth of things it can describe. Seeing AI can recognize saved friends and describe their emotions. It can even describe people you don't know with things like their gender, what they're wearing and their estimated age. On top of that, it can quickly recognize and read short text snippets, as well as full documents and much more. An experimental feature in the app can even describe the scene around you.
Here's a look at everything the app can recognize and what it can do:
- Short Text - Speaks text as soon as it appears in front of the camera.
- Documents - Provides audio guidance to capture a printed page, and recognizes the text, along with its original formatting.
- Products - Scans barcodes, using audio beeps to guide you; hear the name, and package information when available.
- People - Saves people's faces so you can recognize them, and get an estimate of their age, gender, and emotions.
- Scenes (early preview) - Hear an overall description of the scene captured.
- Images in other apps - Just tap "Share" and "Recognize with Seeing AI" to describe images from Mail, Photos, Twitter, and more.
Seeing AI, which Microsoft describes as a research project, is currently only available on the App Store (opens in new tab) for iPhone users. However, hopefully, we'll see this research carry over onto other platforms, as it could be a very beneficial tool for the visually impaired.
