Microsoft, VideoSurf and the Windows Phone 8 connection to Bing

WP Central

Is VideoSurf's technology used in Windows Phone 8?

With today’s leak of supposed Windows Phone 7.8 features (that list is reportedly not complete either) perhaps the most interesting aspect was the new Windows Phone 8 feature with TV support via Bing Audio.

We speculated that this was an IntoNow-like service whereby you can ID television shows by letting your phone sample audio from whatever you are watching. This is analogous to the Bing Audio music ID feature introduced in Windows Phone 7.5 Mango. The question is how would Microsoft do this? Did they just invent their own video-audio ID service?

The answer may be found with an acquisition by Microsoft late last year of a company called VideoSurf (www.videosurf.com; press release).

VideoSurf allows you to search for videos but across services like YouTube, Hulu, CNN, TMZ, Dailymotion, Comedy Central and more. It’s a tremendously powerful search service that is actually legitimate i.e. not one looking to infringe on copyrighted work.

Developed in 2006 by some Israeli entrepreneurs, the company raised $28 million dollars from numerous investors before being acquired by Microsoft on November 22nd, 2011 for a reported $100 million dollars.

WP Central

VideoSurf's technology integrated into the Xbox 360 at E3 2012

So how will VideoSurf be used by Microsoft? Like Skype, VideoSurf’s technology will be incorporated into existing service layers at Microsoft. Some of this was demonstrated at E3 when during the presentation (~29 min marker) it was shown how you can search for a movie on the Xbox 360 and it would display results from Hulu, Netflix or Amazon Video, allowing the user to select which service they preferred (or had a membership for).

What really piqued our interest though was this bit of information from TechCrunch who wrote about the acquisition last year:

“Not only that, but VideoSurf also implements some cool audio recognition technology along with facial recognition functionality that allows the startup to pull from specific frames within videos, which, when combined, the company (and Microsoft clearly agrees) can provide better search results (and relevancy) coupled with an enhanced user experience. “

And that’s our 'gotcha' moment. Microsoft integrating video recognition via audio samples was already accomplished by VideoSurf, a company which they’ve owned since last year. So the idea that Windows Phone 8 will be using the technology (in addition to the Xbox 360 and Windows 8, the web, etc.) is extremely likely and corroborates the WP7Forum.ru information from earlier today.

Assuming it's true, exactly how it will work on Windows Phone will be the interesting part. Since VideoSurf can evidently ID video frames via facial ID, perhaps we will use the phone’s camera to sample a clip from the TV (coupled with audio) which will bring near-instant search results.  That’s a tantalizing service that as far as we know even Google can’t match (though they do have Google Goggles which could be extended).

Surely we’ll find out more about this feature in the coming weeks but if you were wondering what the connection between Bing Audio with TV was you can now know it is Microsoft’s VideoSurfThanks, clubdirthill, for the heads up.

Update: If you are curious how the data could be used, we know that Windows Phone 8 will allow apps to "talk to each other" meaning the data could be passed off to any 3rd party application or service. In addition, things like buying the show on DVD, adding it to your Netflix queue, sharing with your social networks, recommending it others on Bing or working with Microsoft's SmartGlass are just a few potential uses. Can you think of others?


Reader comments

Microsoft, VideoSurf and the Windows Phone 8 connection to Bing


Now, I hope this wouldn't be a simple ID service. I mean, if you're watching TV in order to ID the show you simply hit info or guide on your remote. Would I be searching just to find where I can purchase the show? I guess if I were at a friends house and a show was on, I could do a search for it which would let me know if the show was on whatever service I may use (Netflix, Hulu, etc.) Hmm. I need to think about how this could be used, practically speaking. Anyone got a better use case for this tech? P.S. It does seem cool, regardless.

Integration with SmartGlass. Add it to your Netflix queue. Share with your friends/social networks what you're doing. Purchase the show (movie?) on DVD, Bing/Social suggestions and recommendations are a few I can think of...

Also, WP8 will allow devs to "connect to" other apps so the resuts might be able to be fed into 3rd party apps.

Seriously mr. Rubino... You would be a fantastic asset to Microsoft in product development. Care to cooperate on an app-idea I have?

u could see the show in the mall or retail store while u were passing by and was curious what it was so u can watch it when u get home, then BOOM , bing search and the name is there....
ok... i've had that issue before once upon a time :P

This may be slightly off topic... But, does anybody know if Hulu is planning to make an appearance an Windows Phone? That would surely complete the "Bing TV" experience.

Wondering if all the audio is also transcribed. Could i search for a line from a movie and have it take me right to that clip?

It would also be cool if you were watching a movie at home and wanted to switch from your tv to on Surface... Just ID and go

I will enjoy not getting this, as well as many other features, simply because I'm in Australia. We still don't have Bing Vision stuff like barcode scanning and text translation :(

This will probably be 100% useless to those of us living outside the US, but a nice feature to have, I guess.

The ability for apps to "talk to each other" is something I have been anticipating for a long time. Imagine if you could get the latest camera app to take a picture, then send that picture to be processed by the latest image processing app (Adobe? Instagram?) then send those results to yet another app to be used some other way. Something like that would finally break apps free from the silos they now occupy.