00:00
223
More
See all Show me
1. Sketch2Photo: Internet Image Montage
2 years ago
Due to a name conflict with a Google SketchUp Plugin, we now change title to Sketch2Photo.
We present a system that composes a realistic picture from a simple
freehand sketch annotated with text labels. The composed picture
is generated by seamlessly stitching several photographs in agreement
with the sketch and text labels; these are found by searching
the Internet. Although online image search generates many inappropriate
results, our system is able to automatically select suitable
photographs to generate a high quality composition, using a filtering
scheme to exclude undesirable images. We also provide a novel
image blending algorithm to allow seamless image composition.
Each blending result is given a numeric score, allowing us to find
an optimal combination of discovered images. Experimental results
show the method is very successful; we also evaluate our system using
the results from two user studies.

Credits

Likes

  • Andrew Mager 2 years ago
    What year are you guys from?
  • Andy Smith plus 2 years ago
    That's the best comment I've ever seen!
  • Bastian Allenstein 2 years ago
    lol
  • Alexander 2 years ago
    definitely past 2020
  • Oscar Tong 2 years ago
    Which planet are you guys from ....
  • richard cave 2 years ago
    wow, a lot of matte painters are going to be so interested in this.
  •  
  • Andrew Mager 2 years ago
    My blost post about it: blogs.zdnet.com/weblife/?p=965
  •  
  • dlew plus 2 years ago
    a lot of agency people just lost their jobs
  • Zoltan Csaki 2 years ago
    ummm, no. a lot of clients just got the ability to add more 'clip-art' to their crappy presentations.
  • dlew plus 2 years ago
    hahahaha yes!
  • XSportSeeker 2 years ago
    Fortunately, only the ones who deserves it...
  • MsTeeCee 2 years ago
    Maybe it'll be really expensive or too complicated and requires design management.
  •  
  • Mike B. 2 years ago
    I'm speechless... I should draw a stick figure bowing down to this project. Incredible.
  •  
  • Chris Mallgogny 2 years ago
    Fake.
  • Tim Mrozek 2 years ago
    Nah, not fake, I saw them showing this off at Siggraph
  •  
  • Nilesh Babu 2 years ago
    unbelievable!
  •  
  • simon scott 2 years ago
    moving images are still safe !
  • ReFreezed 2 years ago
    Not for much longer, I fear...
  •  
  • Ivan George 2 years ago
    And I thought Photoshop was amazing...
  •  
  • Sharath Patali 2 years ago
    Brilliant
  •  
  • roger 2 years ago
    Awesome. Their site must have been flooded, the high interest has taken their site down.
  •  
  • happyjimmy 2 years ago
    My Brother has been working in this field for a number of years.. This is highly dooable.
  •  
  • dan heaver 2 years ago
    this is brilliant.
    but you guys could use a writer.
    I'm available ;)
  • Craig Ellis 2 years ago
    Now if only they could make a version which made promos...

    Get off my internetz Heaver! ;-)
  •  
  • adam kitingan 2 years ago
    please be fake im graduating soon haha
  •  
  • Knut Sverre 2 years ago
    And the use for this is..?
  • Darko Stanimirović 2 years ago
    Agencies, art directors, designers, clients themselves... Though, I feel like some clients would make an end product out of this!

    Perfect for this "crisis". Do an ugly composite, steal a couple of images off the web, don't hire a professional!
    WooHoo!
  •  
  • Darko Stanimirović 2 years ago
    I like how this video got 72 plays at Oct 4th and then 26K the next day! :)
  •  
  • StudioThought, LLC 2 years ago
    Superpixel to the rescue!!
  •  
  • Jassu 2 years ago
    Great Job!!!
  •  
  • Quba Michalski plus 2 years ago
    So I am to believe that while we still don't have the technology to mask an object out of a photo automatically, this system will not only do that, but find the images for me?

    Good one.
  •  
  • Felipe Bachian 2 years ago
    Awesome!
  •  
  • Sysop 2 years ago
    Looks too good to be true?
  •  
  • ugur3d 2 years ago
    Superpixel ???? => A Pixel is pixel.
    Its a fake
  • Annette Price 2 years ago
    They just mean groups of similar pixels.
  •  
  • Marck Al 2 years ago
    I don't think it's real, looks like a bogus to me.
  •  
  • Jens Roland 2 years ago
    The examples lead me to believe that it's a fake. It's a good fake, but it's fake. The lights&shadows match too well, the hand-drawn outlines are nowhere close to the results (e.g. the sitting dog, which is drawn from the side but inserted from the front, which is a completely different figure), and the "superpixel masking/blending" looks basically flawless. The state of the art Photoshop masking plugins can't get close to those examples in any automated setting. With some user involvement in picking the source images and adjusting the masking, this is doable, but fully automated? I call fake.
  • Pak-Kei Mak 2 years ago
    Even if it's fake, it's a great concept.
  • Christina Lee 2 years ago
    Yea.
  •  
  • jonwadsworth 2 years ago
    I was ready to call fake... but they posted (alleged) executables:

    cg.cs.tsinghua.edu.cn/montage/files/Binary.zip

    And their main project site:

    cg.cs.tsinghua.edu.cn/montage/main.htm

    Anybody with technical skeelz wanna take a look and see if it's for real?
  •  
  • Juancito Caminador 2 years ago
    It's a Trap!
    lol
  •  
  • Erick Magnus 2 years ago
    Awesomeeee!
  •  
  • Tony xu 2 years ago
    it will have a good future!
  •  
  • Justin Phillips 2 years ago
    Inconceivable!
  •  
  • Michael Holroyd 2 years ago
    If you think it's fake then you don't have much faith in peer-reviewed journals, considering this was published at SIGGRAPH Asia this year...
    Go read the paper, the solution is pretty obvious.
  •  
  • Jamtron pro 2 years ago
    This was on SNL last Saturday.
  •  
  • Robb M. 2 years ago
    amazing. we live in the future.
  •  
  • Christina Lee 2 years ago
    Criticism is a human attempt for excusing our own lack of creativity...I say, if you don't think it's all that...think of something more original. Realism is not exactly what they are going for here, we're hopefully all having fun and harnessing technology in new ways that work for us and others. Some people just don't get it.
  •  
  • Matt White 2 years ago
    It kind of reminds me of the series "Look Around You."

    If you haven't seen that do yourself a favor and watch it!
  • Wesley Kandel plus 2 years ago
    Haha! Yes!
  • Stuart Paciej 2 years ago
    Haha I remember seeing that for the first time not knowing it wasn't a genuine science program though when they used a revolver to destroy the test tubes the penny dropped...
  •  
  • Vinod P 2 years ago
    Incredible!!. Imagine a PhotoSketch enabled Electronic Color Story book for Kids!
  •  
  • Nik Skavinsky plus 2 years ago
    So where's the link to this tool? or is it still a project.
  •  
  • Andreas Beer 2 years ago
    where can i download the freeware tool? :D ;)
  •  
  • Voetmann 2 years ago
    The end of photoshop and all AD´s ;o)
  •  
  • Silvio Vailante 2 years ago
    OK…! I love it!
    But, where i take this for me????
  •  
  • Youssef Sarhan 2 years ago
    this pwns... ha i'd hate to be a graphic designer in the future..
  • James McWilliams 2 years ago
    Yeh cos a graphic designer's job relies entirely on doing bad photo montages? (sigh, some people ;) )
  •  
  • David C. 2 years ago
    So, I did a Google Scholar search and found an identically named program that was presented at SIGGRAPH 2009 (Aug 2009), but by a completely different set of authors:

    doi.acm.org/10.1145/1597990.1598050

    This paper lists authors from Berlin and Paris. And there is no overlap in authorship between this paper and this video.

    This video has the accompanying conference paper hosted at the Nat. Uni. of Singapore:

    ece.nus.edu.sg.nyud.net/stfpage/eletp/Papers/sigasia09_photosketch.pdf

    UPDATE: The paper by Eitz et al. is cited by this Tsinghua Uni group.
  • Gabriel Shalom plus 2 years ago
    Whether its a fake or its legitimate its awesome!
    tr.im/B4xJ
  •  
  • judy tillinger 2 years ago
    but wait - these images are just grabbed from ANYWHERE online? Is there no protection for photographers?
  • Will Marshall 2 years ago
    This is how the internet works. There'd be no other way to do it: and there's no meaningful way to evaluate the copyright status of a bunch of pixels.
  •  
  • Perre 2 years ago
    Genius. So glad to see masking and assembly pushed so far forward... Makes me imagine that Adobe-Macromedia is just sitting on piles of money doing little innovative R&D.
  • Gabriel Shalom plus 2 years ago
    Well they're doing this at adobe... vimeo.com/2345579
  • Perre 2 years ago
    Hey Gabriel, thanks for reminding me of that demo. You're right. I likely understated Adobe's 'little' innovation. My point is more about what we see in their products, which hasn't made masking radically easier despite the regular updates and new masking features. If they're rethinking it in R&D, we designers aren't seeing it show up in their products.

    When I saw the video you pointed to (I think it was on digg several months ago), I thought of it like I think of car companies showing us fantastic cool 'concept cars' that'll never make it into consumer's hands. Instead I see another taurus or camry. That metaphor is apt, I think. When are we going to see these features in Premier or After Effects? Hard to justify spending so much bill on their updates when they don't deliver leaps in thinking like we just witnessed in any of these video demos...
  • Dan Goldman 2 years ago
    These things take time to get into commercial products. Have patience! And chew on this in the meantime: vimeo.com/5024379

    ;-)
  •  
  • Culture Now 2 years ago
    Hey Tao and Co, We are completely in awe of this project. We posted it over on our site (culturenow.com/site/item.cfm?item=39521) and would be thrilled with any updates in the future. Keep up the fantastic and inspiring work.
  •  
  • Adrian Diaconescu 2 years ago
    You had me at "We present a system..." :) Awesome!
  •  
  • Henk Dawson plus 2 years ago
    I've been doing this kind of work for 20 years "by hand" using the computer and it's still a challenge for me to get it right after all these years of practice. I'd be really surprised if this can be automated as well as this demo suggests. Really surprised.
  •  
  • Kevin Bjorke 2 years ago
    Alternative Title: "Final Nails in the Still-Creaking Coffin of Image Copyright"
  •  
  • David C. 2 years ago
    Ariel Shamir, one of the authors listed here, is also co-author on seam carving:


    youtube.com/watch?v=vIFCV2spKtg
  •  
  • idtdi 2 years ago
    不错哦
  •  
  • Ivan 2 years ago
    Excellent tool for Stalinist regimes, dictators etc.
  •  
  • Timothy Peter Robb 2 years ago
    I call shop! This is so fake, you can see the pixels!
  •  
  • paul redican 2 years ago
    could be a useful tool for creating basic storyboard type composited images , but light years away from replacing a skilled compositor or mac op. The image search does appear to function and fufill the basic task of finding and blending images but in a very crude way , there is no allowance for thing such as lighting direction colour cast , shadows reflections , atomoshere, depth haze image grain matching black levels, chromatic lens effects and the field of view of each element used. there is a nearly endless list of thing that a good artist needs to take into consideration to complete a believable composite image. I'm not knocking it as it is very interesting product just explaning why it wont replace an exerperienced artist, Yet ;)
  • James McWilliams 2 years ago
    First sensible comment on here :)... lets just go crazy and assume that you COULD automate masking/compositing entirely... why would that suddenly make artists redundant? These are technical things - do people really believe that automatically comping together a bunch of photos is the be all and end all of an compositor/artists job?? Are some people that stupid? I think most compositors would love to not sit for hours keying out elements of an image and would prefer to focus on the creative aspects that no automation can solve.
    It never ceases to amaze me how people get so caught up in techno-hype. Rant over ;)
  •  
  • Oli 2 years ago
    AWESOME!
    great interface,
    super thinking, great development, this can act as a plugin for google imagebank, tineye, flickr like online app. and any photobank services.
    you are millionaire:)
    i had to reblog it on abracadabran.posterous.com/photosketch-how-come-this-was-not-done-earlie

    hope you do not mind!
  •  
  • Julian Zoller 2 years ago
    Pretty shure its fake.

    Reminds me of this video:
    chilloutzone.to/video/bildbearbeitung-2-0.html
  •  
  • Thomas Mathai 2 years ago
    When you say fake, what do you mean? That it's not professional grade compositing?

    This is an academic presentation. They aren't showing you how to create a professional quality composition automatically.

    They are presenting new ways of gathering and isolating images based on user defined criteria.

    This technology still needs a person with skills to make it look good, it just speeds up the process.
  • James McWilliams 2 years ago
    Great post
  •  
  • Tobiasz Usewicz 2 years ago
    What will come up if I look for bacon + apollo13 + dancing reindeer + hungry teapot?
  •  
  • Great interface, where can one get it?
  •  
  • Janet F 2 years ago
    AWESOME fun and useful, too. Please inform us as to where to obtain this!
  • Alex Beston 2 years ago
    i think its just a scholars project
  •  
  • Tao Chen 2 years ago
    I am so excited that so many people like our work. I want to respond to some concerns about our work: It can automatically choose some images for composition, but it is not fully automatic, it still needs some simple, intuitive user interactions in composition step. You can check cg.cs.tsinghua.edu.cn/montage/ to see our paper, user studies and supplemental materials.

    As this is a research project, the online software/product
    is not ready for publish, we are still working on it.
  •  
  • Lorenzo Araneo plus 2 years ago
    eventually we will have a world where no talent is needed, just software, here is to the downfall of the creative world! blah
  • Thomas Mathai 2 years ago
    Wow, this sounds familiar. Guess you don't see this as a tool to help with the creative process instead of subverting it.
  • Lorenzo Araneo plus 2 years ago
    this not a tool, this is a substitute for a designer, there is no "process". you scribble a few lines and it does everything for you, the process is done by this concept not the designer then again if you use this you aren't really a designer anyways. Someone who uses a Ghost Writer for a book is not a writer, someone who would use this concept is not a designer and hardly a "creative".
  •  
  • Thorsten Miess 2 years ago
    dude? what the? seriously...speechless .
    Seems to be impressive.
  •  
  • Philip Riley 2 years ago
    Research is fun and all but then so is making art. While technically impressive you are not doing anything genuinely useful. Sure this will jolly up some powerpoint presentations for the aesthetically challenged but in real terms it's a novelty. People like making pictures they are not going to stop just because you automated the interesting bits.

    Please use this technology to improve as in make actually useful the myriad masking programs out there that cannot distinguish properly between the foreground and the background. Then you might make something people can use and yourselves some money.
  • Thomas Mathai 2 years ago
    What don't you find useful? If anything it would be like the Variation tool in Photoshop. If you have a lot of elements that you would like to composite together, but would like to see the possiblities quickly, I can see this tool to help with the process.

    I'm sure Photoshop was very novel at version 1.0. Is it still novel?
  • Philip Riley 2 years ago
    Photoshop took an arduous task and made it simple. This takes a simple task and makes it pointless
  • Starman 2 years ago
    Wrong. It took a difficult task and made it SEARCHABLE.
  • Philip Riley 2 years ago
    ?
  •  
  • Talitha Shipman 2 years ago
    Probably fake and illegal. Hardly anyone mentioned that second little issue with this system.
  • Thomas Mathai 2 years ago
    Illegal in what way and where?

    Most photoshop work is taking different elements and combining them together.

    If you mean using other people's images from the internet, that would be considered copyright infringement. Though I think they are only using those as examples of what it can do.
  •  
  • Jason Drakeford plus 2 years ago
    I can only see this used as a storyboard/comp tool. There is no way you can use these stitched images in a final product or ad since the random internet images it uses are copyrighted and opens you up to a rabbit hole of legal issues.
  • Thomas Mathai 2 years ago
    I don't think it's designed just to grab images from the internet.
  •  
  • occubus jive 2 years ago
    wow, can't believe all the super paranoid and judgemental comments! this is what it must have been like when scientists told people the earth was round! haha!

    this is awesome on so many levels! tao chen thank you and don't let small minded people stand in your way!

    for the trolls and naysayers:
    do not fear what you may not understand...it makes no sense...educate yourself instead
  •  
Showing 100 of 146 comments. Want to see the rest?
This conversation is missing your voice. Take five seconds to join Vimeo or log in.

Advertisement

Statistics

Date Plays Comments
Totals 1.1M 1,315 146
Feb 13th 13 0 0
Feb 12th 77 0 0
Feb 11th 67 0 0
Feb 10th 38 0 0
Feb 9th 58 0 0
Feb 8th 56 0 0
Feb 7th 48 0 0