More
See all Show me
1. Sketch2Photo: Internet Image Montage
2 months ago
Due to a name conflict with a Google SketchUp Plugin, we now change title to Sketch2Photo.
We present a system that composes a realistic picture from a simple
freehand sketch annotated with text labels. The composed picture
is generated by seamlessly stitching several photographs in agreement
with the sketch and text labels; these are found by searching
the Internet. Although online image search generates many inappropriate
results, our system is able to automatically select suitable
photographs to generate a high quality composition, using a filtering
scheme to exclude undesirable images. We also provide a novel
image blending algorithm to allow seamless image composition.
Each blending result is given a numeric score, allowing us to find
an optimal combination of discovered images. Experimental results
show the method is very successful; we also evaluate our system using
the results from two user studies.

Credits

1,156 Likes

  • Andrew Mager 1 month ago
    What year are you guys from?
  • Andy Smith 1 month ago
    That's the best comment I've ever seen!
  • Bastian Allenstein 1 month ago
    lol
  • definitely past 2020
  • Oscar Tong 1 month ago
    Which planet are you guys from ....
  • richard cave 1 month ago
    wow, a lot of matte painters are going to be so interested in this.
  •  
  • Andrew Mager 1 month ago
    My blost post about it: blogs.zdnet.com/weblife/?p=965
  •  
  • dlew 1 month ago
    a lot of agency people just lost their jobs
  • Zoltan Csaki 1 month ago
    ummm, no. a lot of clients just got the ability to add more 'clip-art' to their crappy presentations.
  • dlew 1 month ago
    hahahaha yes!
  • XSportSeeker 1 month ago
    Fortunately, only the ones who deserves it...
  • MsTeeCee 1 month ago
    Maybe it'll be really expensive or too complicated and requires design management.
  •  
  • M .Barron 1 month ago
    I'm speechless... I should draw a stick figure bowing down to this project. Incredible.
  •  
  • Chris Mallgogny 1 month ago
    Fake.
  • Tim Mrozek 1 month ago
    Nah, not fake, I saw them showing this off at Siggraph
  •  
  • Nilesh Babu 1 month ago
    unbelievable!
  •  
  • simon scott 1 month ago
    moving images are still safe !
  • ReFreezed 1 month ago
    Not for much longer, I fear...
  •  
  • Ivan George 1 month ago
    And I thought Photoshop was amazing...
  •  
  • Sharath Patali 1 month ago
    Brilliant
  •  
  • roger 1 month ago
    Awesome. Their site must have been flooded, the high interest has taken their site down.
  •  
  • happyjimmy 1 month ago
    My Brother has been working in this field for a number of years.. This is highly dooable.
  •  
  • dan heaver 1 month ago
    this is brilliant.
    but you guys could use a writer.
    I'm available ;)
  • Craig Ellis 1 month ago
    Now if only they could make a version which made promos...

    Get off my internetz Heaver! ;-)
  •  
  • adam 1 month ago
    please be fake im graduating soon haha
  •  
  • Knut Sverre 1 month ago
    And the use for this is..?
  • Darko Stanimirović 1 month ago
    Agencies, art directors, designers, clients themselves... Though, I feel like some clients would make an end product out of this!

    Perfect for this "crisis". Do an ugly composite, steal a couple of images off the web, don't hire a professional!
    WooHoo!
  •  
  • Darko Stanimirović 1 month ago
    I like how this video got 72 plays at Oct 4th and then 26K the next day! :)
  •  
  • StudioThought, LLC 1 month ago
    Superpixel to the rescue!!
  •  
  • Jassu 1 month ago
    Great Job!!!
  •  
  • Quba Michalski 1 month ago
    So I am to believe that while we still don't have the technology to mask an object out of a photo automatically, this system will not only do that, but find the images for me?

    Good one.
  •  
  • Felipe Bachian 1 month ago
    Awesome!
  •  
  • Sysop 1 month ago
    Looks too good to be true?
  •  
  • ugur3d 1 month ago
    Superpixel ???? => A Pixel is pixel.
    Its a fake
  • Rhythmic Circus 1 month ago
    They just mean groups of similar pixels.
  •  
  • Marck Al 1 month ago
    I don't think it's real, looks like a bogus to me.
  •  
  • Jens Roland 1 month ago
    The examples lead me to believe that it's a fake. It's a good fake, but it's fake. The lights&shadows match too well, the hand-drawn outlines are nowhere close to the results (e.g. the sitting dog, which is drawn from the side but inserted from the front, which is a completely different figure), and the "superpixel masking/blending" looks basically flawless. The state of the art Photoshop masking plugins can't get close to those examples in any automated setting. With some user involvement in picking the source images and adjusting the masking, this is doable, but fully automated? I call fake.
  • Pak-Kei Mak 1 month ago
    Even if it's fake, it's a great concept.
  • Christina Lee 1 month ago
    Yea.
  •  
  • jonwadsworth 1 month ago
    I was ready to call fake... but they posted (alleged) executables:

    cg.cs.tsinghua.edu.cn/montage/files/Binary.zip

    And their main project site:

    cg.cs.tsinghua.edu.cn/montage/main.htm

    Anybody with technical skeelz wanna take a look and see if it's for real?
  •  
  • Juancito Caminador 1 month ago
    It's a Trap!
    lol
  •  
  • Erick Magnus 1 month ago
    Awesomeeee!
  •  
  • Tony xu 1 month ago
    it will have a good future!
  •  
  • Justin Phillips 1 month ago
    Inconceivable!
  •  
  • Michael Holroyd 1 month ago
    If you think it's fake then you don't have much faith in peer-reviewed journals, considering this was published at SIGGRAPH Asia this year...
    Go read the paper, the solution is pretty obvious.
  •  
  • Jamtron plus 1 month ago
    This was on SNL last Saturday.
  •  
  • Robb M. 1 month ago
    amazing. we live in the future.
  •  
  • Christina Lee 1 month ago
    Criticism is a human attempt for excusing our own lack of creativity...I say, if you don't think it's all that...think of something more original. Realism is not exactly what they are going for here, we're hopefully all having fun and harnessing technology in new ways that work for us and others. Some people just don't get it.
  •  
  • Matt White 1 month ago
    It kind of reminds me of the series "Look Around You."

    If you haven't seen that do yourself a favor and watch it!
  • Wesley Kandel plus 1 month ago
    Haha! Yes!
  • Stuart Paciej 1 month ago
    Haha I remember seeing that for the first time not knowing it wasn't a genuine science program though when they used a revolver to destroy the test tubes the penny dropped...
  •  
  • Vinod P 1 month ago
    Incredible!!. Imagine a PhotoSketch enabled Electronic Color Story book for Kids!
  •  
  • Nik Skavinsky 1 month ago
    So where's the link to this tool? or is it still a project.
  •  
  • Andreas Beer 1 month ago
    where can i download the freeware tool? :D ;)
  •  
  • Voetmann 1 month ago
    The end of photoshop and all AD´s ;o)
  •  
  • Silvio Vailante 1 month ago
    OK…! I love it!
    But, where i take this for me????
  •  
  • Youssef Sarhan plus 1 month ago
    this pwns... ha i'd hate to be a graphic designer in the future..
  • James McWilliams 1 month ago
    Yeh cos a graphic designer's job relies entirely on doing bad photo montages? (sigh, some people ;) )
  •  
  • David C. 1 month ago
    So, I did a Google Scholar search and found an identically named program that was presented at SIGGRAPH 2009 (Aug 2009), but by a completely different set of authors:

    doi.acm.org/10.1145/1597990.1598050

    This paper lists authors from Berlin and Paris. And there is no overlap in authorship between this paper and this video.

    This video has the accompanying conference paper hosted at the Nat. Uni. of Singapore:

    ece.nus.edu.sg.nyud.net/stfpage/eletp/Papers/sigasia09_photosketch.pdf

    UPDATE: The paper by Eitz et al. is cited by this Tsinghua Uni group.
  • Gabriel Shalom plus 1 month ago
    Whether its a fake or its legitimate its awesome!
    tr.im/B4xJ
  •  
  • judy tillinger 1 month ago
    but wait - these images are just grabbed from ANYWHERE online? Is there no protection for photographers?
  • Will Marshall 1 month ago
    This is how the internet works. There'd be no other way to do it: and there's no meaningful way to evaluate the copyright status of a bunch of pixels.
  •  
  • Perre DiCarlo 1 month ago
    Genius. So glad to see masking and assembly pushed so far forward... Makes me imagine that Adobe-Macromedia is just sitting on piles of money doing little innovative R&D.
  • Gabriel Shalom plus 1 month ago
    Well they're doing this at adobe... vimeo.com/2345579
  • Perre DiCarlo 1 month ago
    Hey Gabriel, thanks for reminding me of that demo. You're right. I likely understated Adobe's 'little' innovation. My point is more about what we see in their products, which hasn't made masking radically easier despite the regular updates and new masking features. If they're rethinking it in R&D, we designers aren't seeing it show up in their products.

    When I saw the video you pointed to (I think it was on digg several months ago), I thought of it like I think of car companies showing us fantastic cool 'concept cars' that'll never make it into consumer's hands. Instead I see another taurus or camry. That metaphor is apt, I think. When are we going to see these features in Premier or After Effects? Hard to justify spending so much bill on their updates when they don't deliver leaps in thinking like we just witnessed in any of these video demos...
  • Dan Goldman 10 days ago
    These things take time to get into commercial products. Have patience! And chew on this in the meantime: vimeo.com/5024379

    ;-)
  •  
  • Culture Now 1 month ago
    Hey Tao and Co, We are completely in awe of this project. We posted it over on our site (culturenow.com/site/item.cfm?item=39521) and would be thrilled with any updates in the future. Keep up the fantastic and inspiring work.
  •  
  • Adrian Diaconescu 1 month ago
    You had me at "We present a system..." :) Awesome!
  •  
  • Henk Dawson 1 month ago
    I've been doing this kind of work for 20 years "by hand" using the computer and it's still a challenge for me to get it right after all these years of practice. I'd be really surprised if this can be automated as well as this demo suggests. Really surprised.
  •  
  • Kevin Bjorke 1 month ago
    Alternative Title: "Final Nails in the Still-Creaking Coffin of Image Copyright"
  •  
  • David C. 1 month ago
    Ariel Shamir, one of the authors listed here, is also co-author on seam carving:


    youtube.com/watch?v=vIFCV2spKtg
  •  
  • idtdi 1 month ago
    不错哦
  •  
  • Ivan 1 month ago
    Excellent tool for Stalinist regimes, dictators etc.
  •  
  • Timothy Peter Robb 1 month ago
    I call shop! This is so fake, you can see the pixels!
  •  
  • nacider 1 month ago
    could be a useful tool for creating basic storyboard type composited images , but light years away from replacing a skilled compositor or mac op. The image search does appear to function and fufill the basic task of finding and blending images but in a very crude way , there is no allowance for thing such as lighting direction colour cast , shadows reflections , atomoshere, depth haze image grain matching black levels, chromatic lens effects and the field of view of each element used. there is a nearly endless list of thing that a good artist needs to take into consideration to complete a believable composite image. I'm not knocking it as it is very interesting product just explaning why it wont replace an exerperienced artist, Yet ;)
  • James McWilliams 1 month ago
    First sensible comment on here :)... lets just go crazy and assume that you COULD automate masking/compositing entirely... why would that suddenly make artists redundant? These are technical things - do people really believe that automatically comping together a bunch of photos is the be all and end all of an compositor/artists job?? Are some people that stupid? I think most compositors would love to not sit for hours keying out elements of an image and would prefer to focus on the creative aspects that no automation can solve.
    It never ceases to amaze me how people get so caught up in techno-hype. Rant over ;)
  •  
  • Oli 1 month ago
    AWESOME!
    great interface,
    super thinking, great development, this can act as a plugin for google imagebank, tineye, flickr like online app. and any photobank services.
    you are millionaire:)
    i had to reblog it on abracadabran.posterous.com/photosketch-how-come-this-was-not-done-earlie

    hope you do not mind!
  •  
  • Julian Zoller 1 month ago
    Pretty shure its fake.

    Reminds me of this video:
    chilloutzone.to/video/bildbearbeitung-2-0.html
  •  
  • Thomas Mathai 1 month ago
    When you say fake, what do you mean? That it's not professional grade compositing?

    This is an academic presentation. They aren't showing you how to create a professional quality composition automatically.

    They are presenting new ways of gathering and isolating images based on user defined criteria.

    This technology still needs a person with skills to make it look good, it just speeds up the process.
  • James McWilliams 1 month ago
    Great post
  •  
  • Tobiasz Usewicz 1 month ago
    What will come up if I look for bacon + apollo13 + dancing reindeer + hungry teapot?
  •  
  • Great interface, where can one get it?
  •  
  • Janet F 1 month ago
    AWESOME fun and useful, too. Please inform us as to where to obtain this!
  • Alex Beston 1 month ago
    i think its just a scholars project
  •  
  • Tao Chen 1 month ago
    I am so excited that so many people like our work. I want to respond to some concerns about our work: It can automatically choose some images for composition, but it is not fully automatic, it still needs some simple, intuitive user interactions in composition step. You can check cg.cs.tsinghua.edu.cn/montage/ to see our paper, user studies and supplemental materials.

    As this is a research project, the online software/product
    is not ready for publish, we are still working on it.
  •  
  • Lorenzo Araneo 1 month ago
    eventually we will have a world where no talent is needed, just software, here is to the downfall of the creative world! blah
  • Thomas Mathai 1 month ago
    Wow, this sounds familiar. Guess you don't see this as a tool to help with the creative process instead of subverting it.
  • Lorenzo Araneo 1 month ago
    this not a tool, this is a substitute for a designer, there is no "process". you scribble a few lines and it does everything for you, the process is done by this concept not the designer then again if you use this you aren't really a designer anyways. Someone who uses a Ghost Writer for a book is not a writer, someone who would use this concept is not a designer and hardly a "creative".
  •  
  • Thorsten Miess 1 month ago
    dude? what the? seriously...speechless .
    Seems to be impressive.
  •  
  • Philip Riley 1 month ago
    Research is fun and all but then so is making art. While technically impressive you are not doing anything genuinely useful. Sure this will jolly up some powerpoint presentations for the aesthetically challenged but in real terms it's a novelty. People like making pictures they are not going to stop just because you automated the interesting bits.

    Please use this technology to improve as in make actually useful the myriad masking programs out there that cannot distinguish properly between the foreground and the background. Then you might make something people can use and yourselves some money.
  • Thomas Mathai 1 month ago
    What don't you find useful? If anything it would be like the Variation tool in Photoshop. If you have a lot of elements that you would like to composite together, but would like to see the possiblities quickly, I can see this tool to help with the process.

    I'm sure Photoshop was very novel at version 1.0. Is it still novel?
  • Philip Riley 1 month ago
    Photoshop took an arduous task and made it simple. This takes a simple task and makes it pointless
  • Starman 1 month ago
    Wrong. It took a difficult task and made it SEARCHABLE.
  • Philip Riley 1 month ago
    ?
  •  
  • Talitha Shipman 1 month ago
    Probably fake and illegal. Hardly anyone mentioned that second little issue with this system.
  • Thomas Mathai 1 month ago
    Illegal in what way and where?

    Most photoshop work is taking different elements and combining them together.

    If you mean using other people's images from the internet, that would be considered copyright infringement. Though I think they are only using those as examples of what it can do.
  •  
  • Jason Drakeford plus 1 month ago
    I can only see this used as a storyboard/comp tool. There is no way you can use these stitched images in a final product or ad since the random internet images it uses are copyrighted and opens you up to a rabbit hole of legal issues.
  • Thomas Mathai 1 month ago
    I don't think it's designed just to grab images from the internet.
  •  
  • occubus jive 1 month ago
    wow, can't believe all the super paranoid and judgemental comments! this is what it must have been like when scientists told people the earth was round! haha!

    this is awesome on so many levels! tao chen thank you and don't let small minded people stand in your way!

    for the trolls and naysayers:
    do not fear what you may not understand...it makes no sense...educate yourself instead
  •  
Showing 100 of 142 comments. Want to see the rest?
This conversation is missing your voice. Take five seconds to join Vimeo or log in.
Hey, there is 1 more comment in 1 group

Advertisement

Statistics

  •  
    plays
    likes
    comments
  • Total
    plays 937K
    likes 1,155
    comments 142
  • Nov 20th
    plays 1,865
    likes 3
    comments 0
  • Nov 19th
    plays 2,276
    likes 1
    comments 0
  • Nov 18th
    plays 1,698
    likes 0
    comments 0
  • Nov 17th
    plays 1,512
    likes 1
    comments 0
  • Nov 16th
    plays 1,463
    likes 2
    comments 2
  • Nov 15th
    plays 829
    likes 2
    comments 0
  • Nov 14th
    plays 844
    likes 1
    comments 0
  • Nov 13th
    plays 986
    likes 1
    comments 0
Previous Week

Downloads

Please join Vimeo or log in to download the original file. It only takes a few seconds.