00:00
534
More
See all Show me
6. Enhancing and Experiencing Spacetime Resolution wi…
3 years ago
Followup work to the "Using Photographs to Enhance Videos of a Static Scene" project:
vimeo.com/1513129.

In this new work we focus on using photographs to increase the spatial resolution and temporal resolution (i.e., frame rate) of videos containing moving objects (i.e., dynamic scenes).

Project website:
grail.cs.washington.edu/projects/enhancing-spacetime/

Credits

Likes

See all likes
  • Michael Crumpton 3 years ago
    Quite Amazing. One big advantage I can see is that shooting low rez video with high rez key frames would take a lot less disk space and the equipment needed is less expensive than for a HD or Imax camera. The heavy lifting required to do the interpolation can be done after the video is shot on a regular computer.
  •  
  • dalas verdugo staff 3 years ago
    Wow, I would actually want to reproduce some of those morphing artifacts. That's a very cool stylistic effect, even though to you it is undesired.
  •  
  • Louis du Mont plus 3 years ago
    Makes me wonder what sort of computing power would be able to calculate the footage and stills in real-time. It would amazing to make a hybrid camera with this sort of processing built in, allowing image clarity and frame rate way beyond the native capability of the cameras sensor system.
  •  
  • Jared Lyon plus 2 years ago
    This video as well as the early piece both completely amaze me. Keep up the great work!
  •  
  • Ian Johnson 2 years ago
    Are the high resolution images necessary for increasing the temporal resolution, if you already have high quality SD or HD video? As a video editor, I run into those artifacts all the time using optical flow plugins for slow motion. Your method would have application now, even without a hybrid camera if it can improve on current methods of interpolation between video frames.
  • pro 2 years ago
    No. High resolution images are not necessary for increasing the temporal resolution of the video.
    The cricket(@6:50) and soccer(@7:51) results in the video did not involve any photographs.

    Recently there was also a paper at Siggraph 2009 which specifically dealt with the problem of increasing video frame-rate:
    "Moving Gradients: A Path-Based Method for Plausible Image Interpolation"
    .cs.columbia.edu/~dhruv/
  •  
  • David Coiffier plus 2 years ago
    What makes me skeptical about all of this demonstration, is that lores material is just derived from hires material, providing perfect match between hi & lo res. But that's a case that won't ever exist in real life. What about taking stills apart from video shooting, that won't perfectly match in space or colors ? That would be more interesting, and more related to real cases...
    Or is there something I didn't get ?
  • pro 2 years ago
    The paper showed a couple of examples captured by putting a video camera next to a point and shoot. These examples were meant to be an evaluation of our system on real-world data. The one of these examples can be seen at the very end of the video - @ 8:04.

    Also, the experiments involving low-res material derived from high-res material is a realistic test case for hybrid cameras. These cameras can simultaneously capture low-res video with a few photographs every so often. Now you have the problem of converting the lowres-video into high-res using the intermittent photos that do not match perfectly in spacetime when the camera and the scene objects are moving. This is exactly the problem we address in this paper. To test our system we take a high-res video, down-sample and add noise to simulate low-res video and keep a few of the high-resolution frames from the original video to simulate the photographs. Then we compare our reconstruction to the original video and see what the error rate is.
  •  
  • Diamond Jef 2 years ago
    keep on going, i am waiting for commercial program able to use it in my home, what You are doing is amaizing !!!!!!!!!!!!!!!
  •  
  • Jim Smith 2 years ago
    Total BS. A decent video editor such as Sony Vegas or Final Cut Express can do all of this with far less trouble and effort.

    If you are that concerned about your video, why not get a decent camera and learn something about making videos to start with? None of that is terribly expensive and works far better than this nonsense.

    Finally, the narrator has an annoying voice and delivery. She also obviously is working from a script and doesn't know her subject or even the script very well.
  • Vladimir Tomin 1 year ago
    "A decent video editor such as Sony Vegas or Final Cut Express can do all of this with far less trouble and effort."

    So Final Cut can take data from photos and increase resolution of video using it? I'm not saying you're wrong, I'm just asking you to teach me how to do this in Final Cut Pro. Step-by-step would be awesome.
  • Gabriel Netto 1 year ago
    "A decent video editor such as Sony Vegas or Final Cut Express can do all of this with far less trouble and effort."

    I work with postproduction. No program does this yet. Not Final Cut (express!!!) not (decent?) vegas, not After effects, not Nuke, not Combustion, Flame, or any software at all...
    Try something simple on your vegas: Take 24p footage and convert it to 29,97 for example. Then make a slowmotion with the result footage. Even with Final Cut Pro 7 you'll get issues. Now enhance quality with some photos taken by another camera... take a look at Pro's "Using Photographs to Enhance Videos of a Static Scene" and tell me if there's ANY software capable of this today!
  •  
  • Gabriel Netto 1 year ago
    By the way, incredible, Pro! I'm following your research long time ago.
    I would like to know if your system is capable of consistent Interlaced to Progressive conversion.
    Keep it on!
  •  
  • Jim Smith 1 year ago
    I disagree. I no longer use Vegas but I have excellent results with Final Cut and there are many third-party addition that do all of these things very will.

    So you maybe shilling for a product but that doesn't make any of what you post true.
  •  
  • OutlandishMediaCorp 2 months ago
    Have there been any updates to this? Your research area is very exciting.
  •  
  • Orhan Nasufovski 2 months ago
    Who's still sitting on this technology? Get it out there!
  •  
This conversation is missing your voice. Take five seconds to join Vimeo or log in.

Advertisement

About this video

FLV
00:08:54
  • 504x380, 19.31MB
  • Uploaded Fri January 23, 2009
  • Please join or log in to download
1 Related collection

Statistics

Date Plays Comments
Totals 32.2K 78 16
Feb 14th 0 0 0
Feb 13th 1 0 0
Feb 12th 4 0 0
Feb 11th 0 0 0
Feb 10th 4 0 0
Feb 9th 3 0 0
Feb 8th 7 0 0