Microsoft has launched a new app that builds on its efforts in the artificial intelligence (AI) space to help the blind and visually impaired, in a way, see the world around them. Fittingly called Seeing AI, Microsoft says the app "narrates the world around you," helping to describe people, objects, and even text wherever you point your camera.
What's particularly impressive about the app is the breadth of things it can describe. Seeing AI can recognize saved friends and describe their emotions. It can even describe people you don't know with things like their gender, what they're wearing and their estimated age. On top of that, it can quickly recognize and read short text snippets, as well as full documents and much more. An experimental feature in the app can even describe the scene around you.
Here's a look at everything the app can recognize and what it can do:
- Short Text - Speaks text as soon as it appears in front of the camera.
- Documents - Provides audio guidance to capture a printed page, and recognizes the text, along with its original formatting.
- Products - Scans barcodes, using audio beeps to guide you; hear the name, and package information when available.
- People - Saves people's faces so you can recognize them, and get an estimate of their age, gender, and emotions.
- Scenes (early preview) - Hear an overall description of the scene captured.
- Images in other apps - Just tap "Share" and "Recognize with Seeing AI" to describe images from Mail, Photos, Twitter, and more.
Seeing AI, which Microsoft describes as a research project, is currently only available on the App Store for iPhone users. However, hopefully, we'll see this research carry over onto other platforms, as it could be a very beneficial tool for the visually impaired.
Reader comments
Microsoft's Seeing AI app helps describe the world to the visually impaired
That's awesome, I am so sorry, I mean, I was so sorry about windows phone, it was a nice and is a mature OS with the same issue as yesterday, the lack of mainstream apps and people adoption. It was not a worth business yet, let's see what's gone happen next with windows mobile, meanwhile enjoy and discover the superpowers of PC and discover your creativity capacity.
Microsoft clearly has a step ahead in AI, Mixed Reality, Office Suite Apps, Cloud Services, Gaming and PC Market Share, however it is a big issue and raises concerns about their inability to break into the mobile world with a decent market share. Meanwhile let's see what happens next.
I stopped feeling sorry about this windows phone matter some time ago and start to look at the whole thing as a business and whatever is more convenient for me. Don't get me wrong, I use almost all Microsoft services as a consumer and I really like them for the value they represent for me, but the mobile part. It's just not there yet.
Ahh. I see. So will the government. 😉 but, of course, this will also empower the visually impaired. I dont have a problem with that, though.🐸
It makes sense in this case for it to start out on iPhones. The blind and visually impaired community pretty much exclusively uses iPhones. Apple are the only ones making accessibility a priority with their mobile OS.
The problem isn't that Microsoft makes their software available on iOS/Android, it is that they don't make it available on Windows. And it isn't even about being first to Windows, either. That they announce some new app, service, whatever, which may happen to be coming to iPhones/Droids first, and don't even mention Windows, let alone provide a timeframe for the release of a Windows version. This is the travesty. This is Microsoft's disgrace. Its not even like they don't have the means to make this happen: its why they purchased Xamarin after all!
<rant>
For f***s sake Microsoft, practice what you preach to third-party developers, use the tools you already have, and make apps like this available on all three mobile platforms of note. Yes, you actually own one of them (in case you'd forgotten)! I get so sick of this crap from Microsoft, makes a total lie of Nadella's promise of "best on Windows" Microsoft experience. You can't have the best Microsoft experience on Windows if the Windows version of the software doesn't even exist.
</rant>
You got my point 👍🏻
I agree 100%. Well said.
I'm Windows Mobile user, i don't care about iPhone. All they need is to shape up their w10m store which has loosed many apps as compared to w8.1m.
MS knows their apps are running better on ios/android instead of abandoned w10m . 😡
Hum
Let me get back to the subject...Nice app Microsoft.
Who keeps on down-voting my 100% accurate comments?
"Empowering everyone NOT on Windows to do more"
I'm pretty sure that what the Microsoft motto is. Correct me if I'm wrong.
This makes me hate Microsoft. Not sure why im even on this platform, Nokia or Microsoft. Even if Microsoft is the reason, they're killing it. They can lose a user for some time, but they lose a fan for ever.
I agree. This "Windows is for Enterprise Users Only" mentality is going to bite them hard one day.
Maybe Microsoft will use Project Islandwood to bring this to Windows. Hahaha.
Funny heh? Or maybe they could have used Xamarin to simply bring this app to multiple platforms at the same time? No bridges needed.
I give up trying to figure out their thinking.
You mean there's thinking involved?
In a lot of cases they should use Xamarin, something like this COULD be a bit more tricky and need specific work doing due to way it accesses the camera directly in the app but sure they could have done it
The camera issue could be more tricky because they are using a competitors device; one in which they have no native code access to.
If they used their own phones,... (well, first they would have to care.)
It's a good proof of concept, but being honest a large percentage of us would not actually use it regularly if it was on Windows...
Seriously, I'm pretty sure my mate would use it. OK so she has an Android phone, but it would give her a better choice of device if she could grab a proper Windows tablet with this on it. Great for use at work.
Sure, there are definitely some people that would. But I'd best the majority of people moaning here wouldn't
Checking to see if comments are working. Having issues
Nope, not working. Can't understand what you wrote.
Who wrote what?
Robert Louis Stephenson wrote 'Arrrrr, Jim lad...'.
Here we go... On Apple Store.
Of coarse. It's so annoying.
True, its annoying. Where, the hell is version for Android?
If it makes you feel better, it'll be on Android before Windows gets it; because Windows won't ever get it.
Yes, it will be Android next. I've just recommended it to my Android owning very partially sighted pal on that basis. Particularly annoying there's no UWP app, as this is the best AR app I've seen yet and Apple is already about to get a head start on iOS devices. Looks like MS have given up the AR battle to Apple in the first round? NADELLAAAAaaaaa!!!!!