Microsoft, VideoSurf and the Windows Phone 8 connection to Bing

Is VideoSurf's technology used in Windows Phone 8?

With today’s leak of supposed Windows Phone 7.8 features (that list is reportedly not complete either) perhaps the most interesting aspect was the new Windows Phone 8 feature with TV support via Bing Audio.

We speculated that this was an IntoNow-like service whereby you can ID television shows by letting your phone sample audio from whatever you are watching. This is analogous to the Bing Audio music ID feature introduced in Windows Phone 7.5 Mango. The question is how would Microsoft do this? Did they just invent their own video-audio ID service?

The answer may be found with an acquisition by Microsoft late last year of a company called VideoSurf (; press release (opens in new tab)).

VideoSurf allows you to search for videos but across services like YouTube, Hulu, CNN, TMZ, Dailymotion, Comedy Central and more. It’s a tremendously powerful search service that is actually legitimate i.e. not one looking to infringe on copyrighted work.

Developed in 2006 by some Israeli entrepreneurs, the company raised $28 million dollars from numerous investors before being acquired by Microsoft on November 22nd, 2011 for a reported $100 million dollars.

VideoSurf's technology integrated into the Xbox 360 at E3 2012

So how will VideoSurf be used by Microsoft? Like Skype, VideoSurf’s technology will be incorporated into existing service layers at Microsoft. Some of this was demonstrated at E3 when during the presentation (opens in new tab) (~29 min marker) it was shown how you can search for a movie on the Xbox 360 and it would display results from Hulu, Netflix or Amazon Video, allowing the user to select which service they preferred (or had a membership for).

What really piqued our interest though was this bit of information from TechCrunch who wrote about the acquisition last year:

“Not only that, but VideoSurf also implements some cool audio recognition technology along with facial recognition functionality that allows the startup to pull from specific frames within videos, which, when combined, the company (and Microsoft clearly agrees) can provide better search results (and relevancy) coupled with an enhanced user experience. “

And that’s our 'gotcha' moment. Microsoft integrating video recognition via audio samples was already accomplished by VideoSurf, a company which they’ve owned since last year. So the idea that Windows Phone 8 will be using the technology (in addition to the Xbox 360 and Windows 8, the web, etc.) is extremely likely and corroborates the information from earlier today.

Assuming it's true, exactly how it will work on Windows Phone will be the interesting part. Since VideoSurf can evidently ID video frames via facial ID, perhaps we will use the phone’s camera to sample a clip from the TV (coupled with audio) which will bring near-instant search results.  That’s a tantalizing service that as far as we know even Google can’t match (though they do have Google Goggles which could be extended).

Surely we’ll find out more about this feature in the coming weeks but if you were wondering what the connection between Bing Audio with TV was you can now know it is Microsoft’s VideoSurfThanks, clubdirthill, for the heads up.

Update: If you are curious how the data could be used, we know that Windows Phone 8 will allow apps to "talk to each other" meaning the data could be passed off to any 3rd party application or service. In addition, things like buying the show on DVD, adding it to your Netflix queue, sharing with your social networks, recommending it others on Bing or working with Microsoft's SmartGlass are just a few potential uses. Can you think of others?

Daniel Rubino

Daniel Rubino is the Editor-in-chief of Windows Central, head reviewer, podcast co-host, and analyst. He has been covering Microsoft since 2007 when this site was called WMExperts (and later Windows Phone Central). His interests include Windows, laptops, next-gen computing, and for some reason, watches. Before all this tech stuff, he worked on a Ph.D. in linguistics, watched people sleep (for medical purposes!), and ran the projectors at movie theaters because it was fun.

  • Damn...
  • Now, I hope this wouldn't be a simple ID service. I mean, if you're watching TV in order to ID the show you simply hit info or guide on your remote. Would I be searching just to find where I can purchase the show? I guess if I were at a friends house and a show was on, I could do a search for it which would let me know if the show was on whatever service I may use (Netflix, Hulu, etc.) Hmm. I need to think about how this could be used, practically speaking. Anyone got a better use case for this tech? P.S. It does seem cool, regardless.
  • Integration with SmartGlass. Add it to your Netflix queue. Share with your friends/social networks what you're doing. Purchase the show (movie?) on DVD, Bing/Social suggestions and recommendations are a few I can think of... Also, WP8 will allow devs to "connect to" other apps so the resuts might be able to be fed into 3rd party apps.
  • Seriously mr. Rubino... You would be a fantastic asset to Microsoft in product development. Care to cooperate on an app-idea I have?
  • Ha, thanks. I can't guarantee I'll have the time but you can shoot me an email: daniel AT wpcentral DOT com
  • u could see the show in the mall or retail store while u were passing by and was curious what it was so u can watch it when u get home, then BOOM , bing search and the name is there....
    ok... i've had that issue before once upon a time :P
  • Tried to tip you on this when I read the other article, I guess clubdirthill beat me to it :)
  • Need a Suface NOWWE
  • This may be slightly off topic... But, does anybody know if Hulu is planning to make an appearance an Windows Phone? That would surely complete the "Bing TV" experience.
  • Wondering if all the audio is also transcribed. Could i search for a line from a movie and have it take me right to that clip?
  • It would also be cool if you were watching a movie at home and wanted to switch from your tv to on Surface... Just ID and go
  • I will enjoy not getting this, as well as many other features, simply because I'm in Australia. We still don't have Bing Vision stuff like barcode scanning and text translation :(
  • It would be nice if you could ID a show and then use the DirecTV app to set your DVR to record it.
  • This will probably be 100% useless to those of us living outside the US, but a nice feature to have, I guess.
  • What ever it is, it will not work outside US :-/
  • The ability for apps to "talk to each other" is something I have been anticipating for a long time. Imagine if you could get the latest camera app to take a picture, then send that picture to be processed by the latest image processing app (Adobe? Instagram?) then send those results to yet another app to be used some other way. Something like that would finally break apps free from the silos they now occupy.