Skip to main content

Microsoft's new CaptionBot site tries to describe what's in a photo

Microsoft's latest effort to promote its image recognition tools quietly launched during Build 2016. It's called Captionbot, and it's a website that lets people upload a photo to see if the bot can accurately describe what's being depicted in the image.

The website comes from Microsoft Cognitive Services, the company's division that created the APIs that are used for the previously launched HowOld.net or TwinorNot.com websites. In this case, the Captionbot site uses Microsoft's Computer Vision, Emotion and Bing Image APIs to come up with the captions for the uploaded photos.

How accurate is it? At the moment, it looks like some photos and images are getting solid captions, while others are highly innacurate, and unintentionally funny. Sometimes the site cannot come up with a caption at all. As with many of Microsoft's other cloud-based services, the CaptionBot site is designed to learn with more experience and as such the captions will become more accurate over time.

Check out the Captionbot.ai site

19 Comments
  • Lol this will be fun when drunk
  • You want me to do WHAT? To WHO? For HOW MANY chocolate-chip cookies?
  • They can create bot that captions a photo, they can create a bot to chat with on twitter, they can create a bot that tells your age, they can create a ground-breaking hardware but they can't fix a photos app in their mobile OS. ;P
  • Camera app.
  • I love the photos app actually. Works great now. 
  • And no OTG support Posted from my Lumia 640 Xl dual sim 10586.218
  • Simple objects are recognized well, even in unfortunate lighting, complicated single objects like the instruments in my car are captioned ok ("looks like a clock"), many objects aren't recognized at all
  • It wasnt quiet. I told mr Paully Acevedo aboit it 2weeks ago. Im sure I tiped... its fun. Posted via the Windows Central App for Android
  • Hehe, wait until caption bot gets more accurate and can recognize individual faces. Then wait until it can analyze streaming video real-time.  Hrm, maybe Facebook or Google will jump on that.  Sigh. Sad times ahead.  Nevermind the potential for conspiracy folk, imagine advertising that changes as you walk by bill boards.  Or think of Minority Report.  LOL  :) In all seriousness, I think this is fun stuff.
  • MS needs to stop tormenting us with the use of Fable Legends related images.
  • I'm pretty sure that is an image Windows Central used. I'm at work and when I played with it I used whatever images were available to me on my work computer. Windows Central just did a story yesterday on Fables using that exact image.
  • I tested it using a screenshot of Chuggaa's current Twitter profile picture and it said it "looks like a picture of a bus." Hilarious, unintended results indeed.
  • I tried the Murphy bot on skype and asked what would I look like old. It have me a picture of an old lady. Needs work
  • But you could end up looking like an old lady!
  • LOL https://s24.postimg.org/smsfenz6t/Cattura.png
  • i just tried it got pretty hillarious but totally wrong answers
  • Internet memes give totally hilarious answerers
  • https://www.facebook.com/photo.php?fbid=10209522476084416&set=a.11671635...