00:00
154
Update: check the 2.0 version! vimeo.com/34824490

A quick installation prototype Emily and I hooked up with the libfreenect Kinect drivers and ofxKinect. The system is doing skeleton tracking on the arm and determining where the shoulder, elbow, and wrist is, using it to control the movement and posture of the giant funky bird!

Speed project - made in a day using openFrameworks and libFreenect.

Concept and Production by Design I/O
Emily Gobeille - Theo Watson
design-io.com

3D depth camera for arm tracking, courtesy of Microsoft and the open source / diy community :)

If you're interested in making projects with the Kinect and openFrameworks check out the addon as it progresses on the OF forums. openframeworks.cc/forum/viewtopic.php?p=24948#p24948

Also keep an eye on the libfreenect GitHub for latest updates to the driver - github.com/OpenKinect/libfreenect/

Likes

  • Jamie Dubs pro 1 year ago
    This is unreal! Wow!
  •  
  • joshua noble 1 year ago
    hells yeah! Nice work Theo+Emily!
  •  
  • Bones Magazine 1 year ago
    Wow!!!
  •  
  • crookedspoon 1 year ago
    that's so cool Emily & Theo! Wondering though: this could be done with any camera, right? doesn't need to be a Kinect?
  • Caedmon Mullin 1 year ago
    I was thinking the same thing...
  • Theo Watson plus 1 year ago
    it allows for really easy person detection and a cleaner silhouette. with a regular ir camera we would have to worry about the color of the clothes people are wearing, or do backlighting with IR lamps. The kinect also gives us the z distance of each joint - which means we could in theory have the puppet turn to face the viewer :)

    also we can do stuff when another person touches the projection, or is near - ie have the birds react, stare etc.

    for the simple silhouette isolation you are right, the kinect just makes it super easy - but for what we have in mind there will be a lot of fun stuff we can add using the device.
  • Caedmon Mullin 1 year ago
    I see... thanks. Great work by the way!
  •  
  • 1n0ut 1 year ago
    love it, we are currently working on a skeleton solution for kinect and other depth cameras, which will be open source soon.
    Our research lab will be focused in developing a mixed reality framework with components as such depth camera interaction modules.
    cadet.at
    will be online next week...
  •  
  • Dan Higham plus 1 year ago
    Thats very cool, nice work!
  •  
  • Beth DiFontaine 1 year ago
    Amazing work!
  •  
  • mpared plus 1 year ago
    amazing its up at the curious brain !! Well done
  •  
  • MRfrukta! plus 1 year ago
    Hello, this is cool!!
    posted on twitter.com/Vimeorussia
  •  
  • _ps 1 year ago
    Just wow. And beautiful!
  •  
  • coooool.
  •  
  • Robert Ropotan 1 year ago
    Pretty good work, congrats! Can you give a few hints on how do you do the hand tracking (joints distance, skeleton tracking)? Is it info the kinect is providing, done using some library or your own solution?
  • Theo Watson plus 1 year ago
    its all in software ( ie not on the kinect ). currently its looking for very specific features so its really tuned to this task at the moment. It uses some code I found here ( bit.ly/bWBfSh ) for the thinning and then is just doing analysis on an extracted contour.

    I would like to work on a more general approach to skeleton tracking, I think there is a lot of others working on this right now though :)
  •  
  • André Mattos 1 year ago
    PS3 Eye cam is dead...
  • Philnolan3d 1 year ago
    Hardly. Check out ipisoft.com using the PS3 Eyes to do full body motion capture.
  • André Mattos 1 year ago
    Pretty awsome this guys work! Like a friend says, software is 75% of thing... But imagine it with depth data coming from an affordable hardware...
  •  
  • Jose Saenz 1 year ago
    Wow, I'd love to be a part of your project! Take a look at my art and let me know if you would like to collaborate on something:

    josesaenz.com
  •  
  • Don Blaauw 1 year ago
    Nice Work Theo and and Emily.. Takes me back to shadow monsters in 2005 from Phillip Worthington...you should do something together.. Meet again sometime? Don
  •  
  • Chris Rojas 1 year ago
    haha, excellent!
  •  
  • Fabiano Waewell 1 year ago
    cool!
  •  
  • Very nice, looking forward to your upcoming videos :).
  •  
  • ArtSocket plus 1 year ago
    haha this is cute
  •  
  • Patrick Haugen 1 year ago
    You're my hero.
  •  
  • braitsch 1 year ago
    This is amazing guys, really nice work!
  •  
  • Wray Bowling 1 year ago
    Hey guys! I've been doing a fair amount of digital puppetry research. I had the same idea this week, and I'm really excited that you've already found a way to do such a thing :)

    I'd be more than interested in working with you to create something that could interface with Blender 2.5 so the armature can be used for a real production environment.
  •  
  • a nomad 1 year ago
    . great job!
  •  
  • IndiEvolution 1 year ago
    Excellent work. Wondering where you will go next.

    Am interested in its implementation for markerless performance capture - with six or eight kinects linked up.
  •  
  • Rio Kierkels 1 year ago
    When I heard about the kinect drivers I thought: Theo would love this!. I should have known nothing on the internet happens without you knowing.

    Good work and great stuff you did with nexus productions, congratulations on that.
  • Theo Watson plus 1 year ago
    hey thanks man!!
    nice to hear from you :)
  •  
  • Brian Stokes 1 year ago
    Hi Theo and Emily,
    Very cool! I have been waiting patiently for (and writing about) affordable digital puppets on console game systems ever since the Nintendo N64 was announced back in 1994. Your demo shows that we are getting closer and closer. Great work!

    One technical question: is the cause of the lag due to the Kinect itself, or the Vision libraries? Or something else?
  • Theo Watson plus 1 year ago
    thanks!

    actually the lag is due to the fact we're using the old libfreenect driver which results in 8-12 fps video from the kinnect.

    its actually been patched to run at 30fps now ( github.com/ofTheo/ofxKinect/pull/9 ) so with that patch the project runs with much much less noticeable lag.

    :)
  •  
  • VJzoo plus 1 year ago
    Great work, you've got it looking great and working well so quickly :)
  •  
  • Ray Roman 1 year ago
    Nice work, I'd love to see more!
  •  
  • Ben Malouf 1 year ago
    I'll be showing this to my Digital Tech in the Arts class next week! Awesome work.
  •  
  • Frederico Soares 1 year ago
    show de bola! fico imaginando a tecnica com holografia.Passaremos a acreditar em fantasmas.Valeu Theo.
  •  
  • Roger C. Rocha 1 year ago
    Congratulations! You could definitely say: "Hi! I'm a Kinect hack made with a Mac."
    -----
    "And I'm a Kinect hack made with a PC." youtu.be/M-wLOfjVfVc
  •  
  • Ted Tagami 1 year ago
    fantastic work! So many possibilities!
  •  
  • Ralph Hauwert 1 year ago
    Just wanted to say congrats, theo, emily! Well done, the best use of openkinect I've seen so far. Love that you made it so playful, typical to your style!
  •  
  • Gene Davison 1 year ago
    Great work! Awesome idea!
  •  
  • P. Dunand Filliol 1 year ago
    poetic and beautiful
  •  
  • dustin o'connor 1 year ago
    nice i got ofxkinect running
  •  
  • SHOOT THE BOSS plus 1 year ago
    Amazing good job
  •  
  • Sebastian Bauer 1 year ago
    Very cool stuff - thumbs up!
  •  
  • Kamen Dimitrov 1 year ago
    Good work guys! :)
  •  
  • tomgalle 1 year ago
    Amazing!
  •  
  • Jonathan Silén 1 year ago
    Hello Theo! My name is Jonathan Silén and i live in Gothenburg, Sweden. I run The Private Theatre. We play for one spectator at the time. We work alot with projections and Im interested in producing things together with you and Emily! I dont have any showcase of our work up yet, but if you want you can se a short feature at my vimeo.
  •  
  • guilherme trindade 1 year ago
    Gênial.! Esta de parabéns!
  •  
  • pATREUS 1 year ago
    Very clever, well done. Young children will love this to bits!
  •  
  • Lorelei Pepi 1 year ago
    Wonderful!! Performers will LOVE this! Puppeteers, dancers, animators, multi-media theater... excellent!! Great job, and thanks for posting!
  •  
  • Philnolan3d 1 year ago
    One step closer to a home mocap system, just need to get depth going and record the data
  •  
  • Kyle McDonald plus 1 year ago
    and now for the important question: which beer?
  • Theo Watson plus 1 year ago
    Harpoon IPA I believe!
    :)
  •  
  • Sermad 1 year ago
    I love this experiment so much - A few years back I saw the phil worthington piece and came up with a twist on his concept.

    When you create a puppet animal - it leaps from your hand and then becomes its own animal - so you can create an animal zoo - birds, rabbits, snakes etc - they all have the behaviour of that animal.

    Snakes chase birds, birds fly around etc. Anyways. Love your work as ever guys.
  •  
  • jim le fevre plus 1 year ago
    Ah man, so good.

    I keep meaning to try and find when you're going to be popping by work (nexus).

    Will persevere!

    Well done in the meantime!
  •  
  • Jeremy Le Fevre plus 1 year ago
    wonderful & magic
    so intriguing
    this is going to open up story telling in an amazing way
  •  
  • Pau_smiley 1 year ago
    great job. LOVE IT
  •  
  • arek 1 year ago
    so awesome! love it!
  •  
  • Adam Lewen 1 year ago
    This is really amazing work guys!
    O my... the possibilities... :-)
  •  
  • ebba erikzon 1 year ago
    This looks like something for the future loads of potential!!
  •  
  • Benjamin Friedberg 1 year ago
    super work emilie and theo. brilliant. my four year old would really dig this type of interactivity. pure fun.
  •  
  • givision 1 year ago
    Great work guys Can imagine to use this in commercial practice. Well done
  •  
  • Hiawatha 1 year ago
    lovely work, looks like very fun to use :)
  •  
  • EL BOGO 1 year ago
    que bueno!!!!!!!!!!!!!
  •  
  • Andrew Garrison plus 1 year ago
    Hi, Theo- I've written a couple of times. Trying to reach you two about designing something to work with the subject of a film I am doing. Please drop me a line. You can see some of the work at vimeo.com/14022223
  •  
  • mikekimike 11 months ago
    Cool Theo! I found your page through Robert Twomey. I would love to connect with you and mashup some ideas for a human music machine I'm working on.
  •  
  • Rune Spaans plus 9 months ago
    Without doubt the most interesting Kinect hack I've seen so far!
  •  
  • Iden 6 months ago
    This is terrific! I see mention of some open source code becoming available in the comment thread. Has this happened? If not will it? Does the availability of an official MS sdk for Kinect change things for this? Bottom line I want to do this too. Can you tell me of the availability of any open source code that I could contribute to helping develop? What is the state of Kinect puppeteering today?
  •  
  • Joe Attanasio plus 4 months ago
    Love the colors on it. :)
  •  
  • Barbara Castro 4 months ago
    I've posted this video on my blog about body, media and art. ctrlbarbara.wordpress.com/2011/06/09/design-i-o/
  •  
This conversation is missing your voice. Take five seconds to join Vimeo or log in.

Advertisement

Statistics

Date Plays Comments
Totals 675K 1,199 73
Feb 23rd 25 0 0
Feb 22nd 61 1 0
Feb 21st 103 1 0
Feb 20th 91 0 0
Feb 19th 82 0 0
Feb 18th 94 0 0
Feb 17th 125 0 0