00:00
297
More
See all Show me
2. Rendering Synthetic Objects into Legacy Photographs
5 months ago
Supplementary material video for our 2011 SIGGRAPH Asia paper (see the project page here: kevinkarsch.com/publications/sa11.html). 3D objects are rendered using LuxRender (luxrender.net).

Authors: Kevin Karsch, Varsha Hedau, David Forsyth, Derek Hoiem

Abstract: We propose a method to realistically insert synthetic objects into existing photographs without requiring access to the scene or any additional scene measurements. With a single image and a small amount of annotation, our method creates a physical model of the scene that is suitable for realistically rendering synthetic objects with diffuse, specular, and even glowing materials while accounting for lighting interactions between the objects and the scene. We demonstrate in a user study that synthetic images produced by our method are confusable with real scenes, even for people who believe they are good at telling the difference. Further, our study shows that our method is competitive with other insertion methods while requiring less scene information. We also collected new illumination and reflectance datasets; renderings produced by our system compare well to ground truth. Our system has applications in the movie and gaming industry, as well as home decorating and user content creation, among others.
  • Devank Patel 4 months ago
    nice!
  •  
  • Skeel Lee 4 months ago
    Excellent work!
  •  
  • xernobyl 4 months ago
    Awesome. Any link to the paper?
  •  
  • Kevin Karsch 4 months ago
    Thanks for the feedback! Here's a link to the project/paper: kevinkarsch.com/publications/sa11.html.
  •  
  • Kyle McDonald plus 4 months ago
    at 1:11 my brain exploded when i saw that light shaft estimate. great work!
  • Kevin Karsch 4 months ago
    Hah, thanks! That was one of our favorite results.
  • Raudel Mesa 2 months ago
    I was thinking about it. Great job!
  •  
  • Alex Light 4 months ago
    Totally amazing. Tremendous work!
  •  
  • JEsús Diez Pérez 4 months ago
    Impressive results! Congratulations.
  •  
  • sukhjinder 4 months ago
    Really Good..:-)
  •  
  • Mikkel Johansen plus 4 months ago
    Absolutely mindblowing. Hats off to the light shafts :). Do you use your own rendering algorithm?
  • Kevin Karsch 4 months ago
    Thanks! We actually use LuxRender for rendering (coupled with the Blender plugin). We obtain rough estimates of geometry, including windows, and shafts are a byproduct of placing a distant source (e.g. the sun) outside the scene.
  •  
  • Da Vinci Fusion Inc. plus 4 months ago
    Stunning, game changing work. Bravo
  •  
  • Zola plus 4 months ago
    Mind = Blown
  •  
  • kaustubh.kaloti 4 months ago
    wow!! amazing!! i was wondering about camera moves... is it possible?
  •  
  • Lee Perry-Smith 4 months ago
    Just stunning.
  •  
  • Paeu 4 months ago
    Wow ! I'd love to play with this tool !
  •  
  • Rokotyan 4 months ago
    Mindblowing …
  •  
  • wow!!
  •  
  • Ricardo Uribe 4 months ago
    thumbs up!
  •  
  • Kert Gartner plus 4 months ago
    I'm assuming we'll see this on the iPhone next year? :)
  •  
  • Stephen Lewis plus 4 months ago
    Scary.
  •  
  • Benjamin Bailey 4 months ago
    Insane! Awesome work. Are you planning on expanding this technology to include camera tracking?
  • Kevin Karsch 4 months ago
    We haven't explored this yet, but it is something we're looking into. I think it should be a possible extension, but it will take some rethinking.
  •  
  • Dave Clarks 4 months ago
    Это очень круто!
  •  
  • Corniger 4 months ago
    Awesome. Great there are people with good brains on this planet :)
  •  
  • Gabriel Ferreira plus 4 months ago
    awesome! well done! when can we get our hands on this type of technology/software?
  • Kevin Karsch 4 months ago
    We're in the early stages of developing an online demo, but there's no time frame for completion yet. All updates, including the demo, will show up on our project page.
  •  
  • Hugo Arcier plus 4 months ago
    I can't believe this video
  •  
  • Ram 4 months ago
    that's incredible o_O
  •  
  • Christiant D'alberto 4 months ago
    Cant believe it !
  •  
  • 3DTotal.com plus 4 months ago
    Amazing! Posted on the 3dtotal.com frontpage.
  • Kevin Karsch 4 months ago
    Thanks for the share!
  •  
  • Robin Parmar 4 months ago
    ! a-ma-zing !
  •  
  • Philippe Da Silva 4 months ago
    Impressive! Add this algorithm to a camera movement recognition and you can get 360° real time scenes. That would be awesome for interior architects ;)
  •  
  • Enzo Pepe 4 months ago
    very very cool..
  •  
  • ilan katin plus 4 months ago
    The room with the flower shaped window was shot in the gymnasium room at Beelitz outside of Berlin.
  •  
  • Florian Juri 4 months ago
    insane! wow!!
  •  
  • Sylvain Magne 4 months ago
    This is tremendous programming ! You guys rock !
  •  
  • Luiz Amaral 4 months ago
    Amazing!!!
  •  
  • George Schnakenberg plus 4 months ago
    I love it! This is a great example of only using the boiling a concept down to the important elements to make it simple to use and to understand.
  •  
  • lilstarfish 4 months ago
    Wow!! Unbelievably cool!
    Working with regular old 3d renders will feel like going from Photoshop CS5 to MS Paint.
    I'll now have to cry myself to sleep till we'll be able to toy around with an actual app using this cool tech ='(
  •  
  • Solomon W. Jagwe 4 months ago
    this is nice :)
  •  
  • Warren Anthony 4 months ago
    This is epic. Looking forward to having a play when it's into Beta. Great work!
  •  
  • Eric Tosstorff plus 4 months ago
    when can we see this in a software package ore as a standalone app? would be nice to have it in nuke though.
  •  
  • Sebastian TR 4 months ago
    very awesome - I hope someone at adobe buys this !!
  •  
  • Nazafi 4 months ago
    this is cool. i hope this will be available in market ver soon
  •  
  • Kultnation plus 4 months ago
    Brilliant!
  •  
  • François Mathey plus 4 months ago
    Amazing !
  •  
  • Josh Johnson plus 4 months ago
    Really amazing.
  •  
  • Remo Gee plus 4 months ago
    mind blowing.
  •  
  • peter papelitzky 4 months ago
    this stuff is really amazing and hopefully you find the right partners to develop a (mac?-)app
    as an interior designer i cant wait to get something like this to my workflow.
    real great work!
  •  
  • JulPer 4 months ago
    Awesome!!!
  •  
  • Paolo De Bonis 4 months ago
    and now what else? what's next? now i feel myself old and dated
  •  
  • Michael DePaula 4 months ago
    This looks fantastic. Maybe I'm the only one, but the clips in the Additional Results section appear more noticeably fake than those during the exposé. It could be the 3D object itself, or its movement, but it seemed different.
  •  
  • ASIER ARDINES 4 months ago
    Great step 4 the photography future
  •  
  • Damien 4 months ago
    This is really impressive! Even though the commentary couldn't sound less excited the actual content shines through loud and clear. Great work!
  •  
  • Mohamed Almonajed 4 months ago
    God bless you Kevin
  •  
  • Raber Umphenour 4 months ago
    Great work - -
  •  
  • Gonzalo Hernandez 4 months ago
    You should get in touch with these guys!!! youtube.com/watch?v=quGhaggn3cQ&feature=player_embedded
  •  
  • Joe Kane plus 4 months ago
    The visualization industry just got spanked in the face. Awesome work.
  •  
  • JohnLong plus 4 months ago
    amazing
  •  
  • greg boyle 4 months ago
    Very cool
  •  
  • myrrh larsen 4 months ago
    Brilliant! First-rate work.
  •  
  • itai bachar 4 months ago
    Wow!!!!!!!!!
  •  
  • satishkrr 4 months ago
    mind blowing
  •  
  • Andy Sapp plus 4 months ago
    I feel like I just watched the future.
    Today.
  •  
  • Constantin Ehrenstein 4 months ago
    That. Is. Just. Amazing.
    Now I know that I can't trust any photo anymore.
    When will this become a commercial product?
  •  
  • FREMOX plus 4 months ago
    Wow, seems really really promising ! Can't wait to try this !
    Any idea of the price for this wonderfull software ? Will it work on a mac too or just on PC ??
    Thank you sooooo much for such an innovative technology !
  •  
  • augmented.org blog 4 months ago
    Great paper, awesome work! Thanks, guys! Can't wait to see it implemented. Do you happen to be at ISMAR2011 in Basel, too? Would love to see more.
  •  
  • Zblur 4 months ago
    Amazing! We put your video on our new group "Photorealistic CG Animations & VFX"
  •  
  • AGudmund 4 months ago
    Impressive! Looking over and over again;-)
  •  
  • Jude Jackson 4 months ago
    This strongly resembles the sort of imagery you see in Flash-based Adventure games, such as Trauma. I look forward to seeing it in the wild.

    I'll just say, I spotted the real photo instantly :P
  •  
  • Esteban bustos 4 months ago
    just awesome..
  •  
  • ian macdonald 4 months ago
    i have lots of ideas thanks
  •  
  • Ray Roman 4 months ago
    Huh.. I'd like to try this.
  •  
  • Paul Swendsen 4 months ago
    As a matte artist that started out with paint on glass for "Willow", your program is a mind blower. Bravo! I look forward to it's release.
  •  
  • Jim Smith 4 months ago
    That looks nice but I've been doing it with Photoshop for many years. With Final Cut or other video editors, it is easily done in videos, too. Sorry, but I am not at all impressed.
  • Daniel Trezub 4 months ago
    I´ve doing this with photoshop too, but seeing this as an automated tool is the amazing part.
  • Jim Smith 4 months ago
    I agree. Although without actually using it, I'm not sure how automated it is. Ig it truly gets the alignment, shadows, and proportions correct automatically, It really is an accomplished,
  • Richard O 4 months ago
    Did you watch the video? Pretty much answers all your questions right there within the first 2 mins. There are adjustments that an amateur end user can make and then the rest is taken care of. I highly doubt you were pulling off the ball bouncing down the stairs at the end of video without putting in days of work.
  • Jim Smith 4 months ago
    Of course I watched the video. No, the questions were not answered. I have no doubt that it would make things easier and faster but that doesn't mean it was all automated. As anyone that has seen what can now be done in even consumer programs like Final Cut Express, anything is possible now.

    I repeat, until one actually works with a program, you don't know what is actually possible, seeing the final result is not proof.
  • AboveTheLine 3 months ago
    Jim, you are a dick head. If you havent used it, dont criticise it.

    Take your cynical unwanted abuse to your grave, you sad, negative, bitter old man.
  • Jim Smith 3 months ago
    YOU obviously can't read. I said, "until one actually works with a program, you don't know what is actually possible, seeing the final result is not proof."

    That isn't being a dick head, it's being realistic. I've used lots of software that looked good in the Demo, but the reality turned out that it wasn't that easy to you.

    You should try not being an obnoxious, ignorant asshole.
  • Marc Rühl plus 3 months ago
    I totally agree with you on the fact that Demos nearly always look better than the final software and I'm really curious how it will do its job. But if you really say this could be all done in FinalCut, you have no idea what you're talking about. That's not even possible in After Effects, you would need at least a good 3D-software, to rebuild the whole scene.
  • Jim Smith 3 months ago
    I do know what I am talking about. It does take some add-ons and a lot of work to do, though. If this rogram automates the process and speeds up the task, it is truly a wonder.

    I think if they offered a free trial package (perhaps time-limited) I'd love to try it and then write a review.
  • Marc Rühl plus 3 months ago
    I would love to see someone rebuild the shot at 4:20 with final cut. However would be a nice to try it too
  • Jim Smith 3 months ago
    I think it could be done but it would require an add-on or two. I have seen them but didn't buy them as they were not something I expected to ever need. If you look at my videos on here, you'll see that my goals are bit more modest. Although in Brazilian Bust Gone Bad, I did a few little tricks at the early part. The most successful was giving myself super white teeth. Although I liked the Halo and horn effects, too. :)

    With one of the add-ons that keep objects in place automatically, these would have been far easier. As it was, I had to try to move them frame by frame. Because the camera was inexpensive and not steady at all, this was harder than it should have been, but all we had was an inexpensive Casio hand-help.

    I still think the system shown here, if it is even half automated would save many hours of tedious work. I'd have to be far more serious to justify it and a much better camera, too. :D
  • Johnny Farmfield 2 months ago
    With 20 years in Photoshop behind me, I can also fake whatever, hehe, it's kinda the thing I've been doing all these years. But going from that to illuminating 3D generated objects with a semi-simulated/estimated enviroment map projecting light with physically accurate radiosity, refractions and caustics, well, that's a whole other ballgame. You just can't fake what's basically simulated reality, there's no plugins for that, hehe... ;D
  •  
  • Francisco Olivares 4 months ago
    Great work, Kevin...looks awesome. I'll check up the docs about your system :)
  •  
  • Mars Oliva 4 months ago
    This is the coolest thing i've seen this month.
    this would help every project i'm working on.
    With 3D motion tracking, this would be the number 1 tool for 3D.
    For the love of everything, release that Demo.
  •  
  • Fizah Rahim plus 4 months ago
    Wow
  •  
  • Ilya Kushnir plus 4 months ago
    Epic
  •  
  • Reynante Martinez 4 months ago
    Very impressive! Given it's still in its early stages. Really promising.
  •  
  • Noah Serry 4 months ago
    very cool .. I'd like to try this.
  •  
  • Diego Daly 4 months ago
    This is great! keep it up, can't wait to use it myself :D
  •  
  • redz.StuDIOz 4 months ago
    wow...gempak siut....
  •  
  • Mrfoxtalbot 4 months ago
    Interesting.
  •  
  • Micuik Ventus 4 months ago
    Amazing!
  •  
  • Jesper Anhede 4 months ago
    Most impressive =)
  •  
Showing 100 of 143 comments. Want to see the rest?
This conversation is missing your voice. Take five seconds to join Vimeo or log in.

Advertisement

Statistics

Date Plays Comments
Totals 927K 3,512 144
Feb 23rd 133 1 0
Feb 22nd 326 1 0
Feb 21st 444 0 0
Feb 20th 289 2 0
Feb 19th 299 2 0
Feb 18th 280 3 0
Feb 17th 390 1 0