New Microsoft project turns first person videos into smooth timelapses

A new project being undertaken at Microsoft Research involves turning first person videos into smooth timelapses, which Microsoft calls hyperlapses. Normally, when a first person video is sped up into a timelapse, the footage is shaky due to the constant movement. Microsoft's project aims to fix that by running the video through its algorithms to create a smoother camera path.

Helmed by researchers Johannes Kopf, Michael Cohen and Richard Szeliski, the project involves reconstructing the video using a 3D camera path that finds the optimal viewpoint, which may differ from the original perspective. The goal is to create the smoothest movement possible so that the end result doesn't look jittery when played back at 10 times the speed of the original video.

Here is a detailed video on how the videos are created:

Interested in the technology? Microsoft is working on making the Hyperlapse algorithm available as a Windows app.

Source: Microsoft Research (opens in new tab)

  • I would love to have such app! I think it would be awesome :-)
  • I would love to have Lumia 1030 with 3D camera...
  • I am so excited! I always loved Time lapse, but never got the methods to make one. My dad loves it too, and now I can surprise him. Hope there also comes a Windows phone app after the release of the Hyper-lapse software... Can anyone tell me when the software might get public and downloadable???
  • the way your wrote this made me imagine a comerssials like happy family.
  • Oooo....that does sound enticing! Taking 3D pictures would be amazing!!! But how long before it becomes a 'fad'? :/
  • I imagined about this technology when I heard about the floating lens technology of Lumia 1020
  • This is awesome. At first I was like eh... But now I want to go out and film and film worth this app
  • get the iphone 6.. good chances they will release it there one year before any other system :)
  • go to iMore and post there this comment. THIS IS ABOUT WINDOWS PHONE.
  • No.. This is about a technology developed by Microsoft. And just like other Microsoft technology (Photosynth) they might release it on apple one year before the rest
  • No they don't. Who got the first OIS? Who got the first cleaner UI? And so on. So, yeah, such a remark belongs on iMore and not on WPCentral.
  • do you kids dont need to be at school already. What actually belongs on WPCentral. Just windows phone stuff? because this has nothing to do with Windows Phone. so is it just microsoft related stuff? than you have to understand the fact that microsoft is clearly not prioritizing their plattform enough.. And listing things now that Microsoft / Nokia implemented in their sofware / hardware is not really the smartest thing to do, is it?
  • Except that if you bothered to read the source it says that it will be released as a windows app. As much as I want this on my phone I think this program will need a lot of processing power. If done on a phone it would probably take way too long to do and kill your battery.
  • Reminds me of a walk in Photosynth to a point
  • Great. Is it similar to illumiroom?
  • Nope, I think they have shown such project before at TedEx.
  • Very cool.
  • It'd be nice to know a timeframe.
  • This is amazing. Hope they implement it in WP 8.1.
  • I've got feeling it's going to take a while and I would expect it maybe with WP9.... but I might be wrong?
  • Did anybody bother to read the page at Microsoft Research ? This will be released as a Windows app. Nowhere is there a mention about Windows Phone. I think the kind of processing power required to do this kind of 3D modelling and rendering is far beyond the current processors of smartphones. in ten years maybe :-)    
  • That would be useful for PureView Lumias. Hope they'll bring the feature to older devices in a software update.
  • The mountain climbing videos are best. You can see how the terrain image re-renders
  • Yes exactly and its so amazing!!!
  • Brilliant!
  • This is incredible! I think an app that could do this would be a great advertising point for Windows phone.
  • just like photosynth. wonder why Apple didnt advertise with this the year they had it before anyone else :P
  • Photosynth has a tech preview on the website.. Walk reminds me of this video option.
  • Microsoft is not childish as Google or Apple to release those apps only on their platform, I find their services everywhere is an amazing approach.
  • The point is not where they release there services but which plattforms they prioritize  to release a service. You know.. showing commitment to your own plattforms rather than competing ones first.. has nothing to do with being childish just with good management
  • Again that is marketing strategy because once you are familiar and addicted to the app you will not look for a change, so when it is released on WP with more better features slowly the users will switch to WP and meanwhile they will still make money on the other platforms.
  • I just hope you didnt learn marketing anywhere on your educational system
  • Okay. We get it. You're pissed Microsoft has released apps on other platforms before Windows Phone. One rant per article is sufficient. Unless your real goal is to render your opinions irrelevant by repeating it so many times people quit reading them.
  • In this case it was the fact somone mentioned it as an advertising point for windows phone . in the previous case I was the originater of the comment. maybe the problem is the a comment system works not on topic but on previous comments
  • Great improvements on the wayyy! :)))
  • Whoa. That's awesome.
    Thumbs up for Greenlake!
  • This is amazing, I recall seeing this in a MSR talk. Can't remember which one, brain still asleep 8.25 am now lol. Need to attend unpaid training doing brain dead scanninv bleugh... I really dislike it when people try to set you up with work. I'm more than capable of doing that lol, really started to enjoy training basics, since it is more engaging and rather do that for free as opposed to sitting zombie like in a cubicle no wider than a lavatory domicile for 8 to 12 hrs.
  • I took my time to read your comment, but, why would you tell us about your life though?
  • +920?...
  • Hope it has 512 support. :)
  • I'd rather to hope it will be preinstalled on every Lumia phones.
  • Would love such an app!
  • WOW, incredible result :)
    Really wants Hyperlapse on my 1020
  • Love it looks like a bit of Photosynth in the backend
  • Bet they'll release this for iOS first... jk.
  • This is really awesome. Too bad someone else will release it before them...
  • Prove it! Who else has a tech video like this???
  • He is probably referring to Google. The guy behind this project has moved to Google in balmer's days but I can't remember his name.
  • This is awesome!!
  • MS just doesn't get it! Why instead of releasing a technical paper detailing the technology so that anyone can adopt it, don't they release a Lumia Pure View model just centered around this, like a sports/outdoor/radical model with this technology? That would be a something new and actually pretty cool and a major selling point WP. Instead, they'll probably release a beta app no one knows when, and looking at the current state of affairs, iOS first
  • That's why there's something called " Patents "
  • Wow o.o when viewing the mountain, you will see how they do it... It's amazing!!! Microsoft research is really wonderful. :)
  • It would be awesome when we have this on WP, and after the partnership with Canon, I hope they will release one of the best phone with quality camera.
  • U stole my words. Anyways wp central ppl don't fight
  • Fuck WOW
  • That's more like it. More in goal with their "be productive" philosophy and if done rightly might propel the platform forward.
  • Wow that is awesome I can't wait to do my own hyperlapse video!
  • That's awesome ! Can't wait for app.Stop teasing us Microsoft.
  • Impressive indeed.  
  • This is what I love about Microsoft. Their research group actually makes technology evolve, unlike Apple that just waits for Google and Microsoft to get good ideas, and then recopies them explaining why "their" idea is a revolution.
  • Please explain who Apple copied for: - magnetic power connector - rubber-banding effect in a mobile scroll list - acceleration in a mobiile scroll list - AirPlay/AirDrop - TouchID - Continuity - reversible connector plug(Lightning) - Mac Pro design - iMac design - Macbook Air
  • Airplay like stuff was invented ages b4 apple.
  • Awesome project !!
  • I like the idea of this, I'm looking forward to utilising it with my gopro once its available.
  • Great. I love the research group
  • Can't wait to see this with POV porn.
  • Oh my
  • It'll be out for iOS first....
  • I want to see this tech in a Lumia someday! ^_^
  • I want that app!
  • The title of the WPC article should have started with 'Awesome', because the current title doesn't do this justice. Amazing tech. 
  • This thing is just phenomenal! Footages get a lot more dynamic!!
  • That is something you don't see every platform :D
  • This is really interesting stuff. I'm surprised at how well it handles moving objects in the final render, like people and cars zipping by.
  • No offence. Just tick video stabilization after you upload you video to youtube and all done.
  • Great! But needs to be improved, people look so artificial, I mean that moving objects go like to 2 frames per second. And the mountain looks like if it was being rendered in a 3D program.
  • Brilliant
    That looks genius!
  • Saw this the other day looking at some Photosynths. They dubbed it there as 3D Photosynths.
  • That was so cool!
  • MS please tell me you're going to use this in Bing Maps street view, otherwise it's a waste of a brilliant idea. Please don't fuck this up.
  • That's super cool !! (y) and seems faster :D
  • You know, these videos would be a heck of a lot easier to watch and listen to if video ads didn't clog the pipe (making the video jerky) and ruin the audio with audio of their own. I know WPC needs ad revenue to keep the content flowing, but ads hindering presentation of content is counterproductive. Static ads are much more pleasant, and sometimes I even look at them.
  • I hope Microsoft turns this into a product, like they did with Photosynth. This would make be buy a GoPro.
  • How freaking awesome is that! Go Microsoft! Proud of you as always
  • I think microsoft ICE and Hyperlapse are going to be microsofts two most brilliant recommendable apps ever! Feeling all happy inside now.
  • Remove that g flop glass
  • This is so cool and brilliant! Release it as BETA and let me try it now, microsoft! :)
  • This should be availabe to Microsoft camera app. Seriously will be boombastic!
  • Hyperlapse is available on iOS instagram. Hmm...
  • Hey for any participating - join my new group on Flickr!