Enhancing and Experiencing Spacetime Resolution with Videos and Stills on Vimeo

See all Show me

7. Enhancing and Experiencing Spacetime Resolution wi…

by pro

11 months ago

6. Prasad and Deepti

by pro

1 year ago

5. Appearance stabilization in PhotoTourism

by pro

1 year ago

4. Interactive Video Cutout

by pro

1 year ago

3. Using Photographs to Enhance Videos of a Static Scene

by pro

1 year ago

2. GradientShop: A novel approach to image and video …

by pro

1 year ago

1. Painterly rendering using GradientShop

by pro

1 year ago

Followup work to the "Using Photographs to Enhance Videos of a Static Scene" project:
vimeo.com/1513129.

In this new work we focus on using photographs to increase the spatial resolution and temporal resolution (i.e., frame rate) of videos containing moving objects (i.e., dynamic scenes).

Project website:
grail.cs.washington.edu/projects/enhancing-spacetime/

Credits

59 Likes

LuxAudio
30 days ago
Zenzuke
1 month ago
Joshua Albers
1 month ago
Dan Will
2 months ago
Eric Bintner
2 months ago
Hannu Hoffrén
4 months ago
Daniel Kutz
5 months ago
Wes Scoggins
5 months ago
Kimberly Shannon
6 months ago
guponex
7 months ago
See all 59 likes

Tags

Michael Crumpton 11 months ago

Quite Amazing. One big advantage I can see is that shooting low rez video with high rez key frames would take a lot less disk space and the equipment needed is less expensive than for a HD or Imax camera. The heavy lifting required to do the interpolation can be done after the video is shot on a regular computer.
dalas verdugo 11 months ago

Wow, I would actually want to reproduce some of those morphing artifacts. That's a very cool stylistic effect, even though to you it is undesired.
Louis du Mont 10 months ago

Makes me wonder what sort of computing power would be able to calculate the footage and stills in real-time. It would amazing to make a hybrid camera with this sort of processing built in, allowing image clarity and frame rate way beyond the native capability of the cameras sensor system.
Jared Lyon 9 months ago

This video as well as the early piece both completely amaze me. Keep up the great work!
Ian Johnson 8 months ago

Are the high resolution images necessary for increasing the temporal resolution, if you already have high quality SD or HD video? As a video editor, I run into those artifacts all the time using optical flow plugins for slow motion. Your method would have application now, even without a hybrid camera if it can improve on current methods of interpolation between video frames.
pro 7 months ago

No. High resolution images are not necessary for increasing the temporal resolution of the video.
The cricket(@6:50) and soccer(@7:51) results in the video did not involve any photographs.

Recently there was also a paper at Siggraph 2009 which specifically dealt with the problem of increasing video frame-rate:
"Moving Gradients: A Path-Based Method for Plausible Image Interpolation"
www1.cs.columbia.edu/~dhruv/
David Coiffier 8 months ago

What makes me skeptical about all of this demonstration, is that lores material is just derived from hires material, providing perfect match between hi & lo res. But that's a case that won't ever exist in real life. What about taking stills apart from video shooting, that won't perfectly match in space or colors ? That would be more interesting, and more related to real cases...
Or is there something I didn't get ?
pro 7 months ago

The paper showed a couple of examples captured by putting a video camera next to a point and shoot. These examples were meant to be an evaluation of our system on real-world data. The one of these examples can be seen at the very end of the video - @ 8:04.

Also, the experiments involving low-res material derived from high-res material is a realistic test case for hybrid cameras. These cameras can simultaneously capture low-res video with a few photographs every so often. Now you have the problem of converting the lowres-video into high-res using the intermittent photos that do not match perfectly in spacetime when the camera and the scene objects are moving. This is exactly the problem we address in this paper. To test our system we take a high-res video, down-sample and add noise to simulate low-res video and keep a few of the high-resolution frames from the original video to simulate the photographs. Then we compare our reconstruction to the original video and see what the error rate is.
Diamond Jef 6 months ago

keep on going, i am waiting for commercial program able to use it in my home, what You are doing is amaizing !!!!!!!!!!!!!!!
Jim Smith 18 days ago

Total BS. A decent video editor such as Sony Vegas or Final Cut Express can do all of this with far less trouble and effort.

If you are that concerned about your video, why not get a decent camera and learn something about making videos to start with? None of that is terribly expensive and works far better than this nonsense.

Finally, the narrator has an annoying voice and delivery. She also obviously is working from a script and doesn't know her subject or even the script very well.