Microsoft has officially announced Video Breakdown as part of its experimental Garage division. The product lets people upload video, which is then analyzed so that it can be searched for specific words and elements.
In a blog post, Microsoft says the goal of Video Breakdown is to blur the line between video content creators and video consumers:
After you upload a video file to the platform, that file's content undergoes analysis through a series of Microsoft Cognitive Services and Azure Media Analytics as well as other Azure services (Azure Websites, Azure Blob storage, Azure Search, Azure Media Services). The process produces an audio transcript; face tracking, grouping and identification; differentiation of speakers; optical character recognition; and extracts sentiments and topics.
All those pieces combine in an index that enables searching a three-hour long video of a speech or a keynote for the exact moment where "healthcare" was mentioned on speech (using automated transcript) or on screen (using OCR), jump to the first moment when the face of a specific actor or speaker appears, focus on moments with positive sentiment and more.
In addition, Video Breakdown also allows users to edit several videos together, based on search results on a single topic or person:
Viewers can then play videos using the insights as jump points and controls. This makes finding the moment where a specific person appears or a keyword is mentioned for the first time easier than ever.