Skip to main content

A blind Microsoft developer has created an app to help him 'see' the world

Near the end of Microsoft's Build 2016 keynote, Microsoft promoted how its APIs have inspired one of its software engineers, Saqib Shaikh, to create an app that could help blind people navigate in the world.

Shaikh himself has been blind since he was seven years old. He has worked at Microsoft for the past 10 years in London. He made an app called Seeing AI as a research project, using intelligence APIs from Microsoft Cognitive Services. The app works on both smartphones as well as the Pivothead SMART glasses. The app can "see" objects and people and then translate what it sees to Shaikh via an audio message.

For example, in a meeting the app sees other people and can tell Shaikh their gender, age and even their emotional state, such as if they appear happy or sad or surprised. The smartphone version can also take a picture of a paper with text, like a menu, and then offer an audio version of that menu. Since this is a research project, there's no indication when, or even if, it will be released as a real commercial app.

45 Comments
  • My first and only reaction after seeing this stuff: 'Holy ****, we're in the future!'
  • And its real!
  • This was a great presentation about how this software engineer shared his experience and his daily life supprted with artificial intelligence .!!! Amazing I was clapping hard even while watching online :')
  • I had the same reaction, I couldn't help but think of the applications to the elderly, disabled, partially blind, etc. I think when Microsoft shows what their technology can do for humans, it really drives home how powerful their engineering might is.
  • great
  • Amazing
  • Impressive Posted via the Windows Central App for Android
  • This was the perfect example of what can be done with the newly announced API's!
  • This is the best thing from Build so far! Posted from my Moto X Pure Edition via the Windows Central App for Android
  • What a great first step!
  • This should be open sourced. :)
  • It is. Did you watch the presentation? This is a platform microsoft is developing and  the APIs will be available to developers. It will be available on android, IOS, Windows, etc. It will be up to developers to figure out how to make it work for them. This was just an example how one Microsoft developer used it to create an app that would help him since he is blind. 
  • wow. No words. Just wow. That's incredible
  • Bohot aala Saqib bhai!!! - Posted via W10 app
  • Great!!
  • It's like a privacy nightmare when photos of random people are getting transfered to Microsoft without their consent. I guess it'll be forbidden in many European countries, same like Google had to remove buildings and people in Street-View.
  • It will store in the internet in any form. So no problem
  • I guess anyone opposed to this don't really care about the blind.
  • There's always one pisser on the parade. This is vastly different than Street View which publicly displayed images. These images are for the individual to "see" what they would "see" on their own, were they not blind.  I can't imagine this would be banned.
  • That's great but don't u think he could have got eyes of any doner from eye hospital.
  • Because it is yeah......that simple....yeah...like swapping out light bulbs....yeah.... Sigh. (I hope Aashish13 wasn't serious or maybe where he or she lives, replacing eyes is cheaper and easier to successfully complete than where I live)
  • Glass eyes are even cheaper!
  • Really?!?!? Eyeglasses for the blind?  SMDH
  • except that says glass eyes, as in eyes made out of glass
  • Cortana...where's the 'Eye' Hospital...Ans:  Are you stupid?  This isn't a Cyberdyne Systems Series Model 800...you don't just plug them in you know...
  • This means a lot to me​. Loved it! Absolutely wonderful...
  • They showed him program code in the clip. I wonder how a blind person is able to be productive as a programmer, as one would be unable to quickly skim a piece of code in order to understand and build on it. Maybe he has incredible memory. These are fantastic developments though, wonder when it will be ready for everyday use.    
  • This is exactly what I was thinking...with all the explicit syntax that coding has it's rather incredible that he's programming in the first place. I did notice that he had a set of headphones on, so perhaps he's trained his brain to visualize the code as he's typing it - getting verbal confirmation with each keystroke
  • Working tech support I once received a call from a blind customer. I remoted into his machine and the speed and accuracy with which he worked was fantastic using nothing other than the narrator as feedback
  • What I am sure he can do faster than most of us with good vision is typing. He was forced to learn touch typing after all, which most of us didn't.
  • Amazing
  • ++ for this.... AI at it's best use..
  • I wish this app was commercial....
  • Awesome
  • As a legally blind individual with limited sight, this is incredible.
  • It is a bit weird that this app will be commercial. Does ms see big market to milk blind users?
  • That's is something I would think could be put to good use.
  • Wow, that is cool & helpful!
  • Just wow... Speechless..we r in future ..
  • Way to go MS
  • WC seriously you can try his bot now..https://www.captionbot.ai/
  • Not like google or apple were bothered to do something like this Posted via the Windows Central App for Android
  • Fantastic.
  • Fantastic app. Sahib Shaikh what's the name of this app? What's ur software specialization? Are there any other such commercially available apps?
  • It's really amazing!