00:00
111
More
See all Show me
I have been working on a real-time version of the Solar piece from a couple years ago. Since it is going to be responding to people's voices and ambient noise instead of music, I started listening to podcasts while I was developing it. I made this video to commemorate my new found love for WNYC's RadioLab podcast. Thanks to Branden Hall and Bill Lindmeier for introducing me to it.

Credits

Likes

  • Reza plus 2 years ago
    Sweet! How are you getting the psychedelic colorful background effect? I am interested in how you mapped your audio input into your system? Also, if this is running real-time are you using shaders for the super cool effects here?
  • flight404 plus 2 years ago
    Thanks!

    The background is all happening on the shader. I have a few moving points and using the distances from each frag to the points to determine the texture coordinates. The texture is a dynamic texture based on the audio.

    I don't understand your second question: how did I map the audio input into the system?

    This project runs real-time, but since I had to render out a video (which cant happen in real-time), I cranked up the particle count for the smoke and haze. Its very close to what I can get in real-time on the PC with Nvidia GTX 260. The only thing happening on the shader right now is the background.
  • Reza plus 2 years ago
    I guess my question was a bit vague, sorry, I meant to ask, since the piece is reacting to audio in realtime, what is the mapping of the audio to the visuals, for example are you taking a FFT of the audio and mapping the frequencies to the sphere sizes or taking the waveform of the incoming audio and assigning accelerations with those values? I love the concept of the whole piece...you are a huge source of inspiration for me.
  •  
  • robert chu 2 years ago
    OMG why are u such a genius?!?!
  •  
  • Bryce Stewart 2 years ago
    Great work! I really like the audio as well. It fit the visual. What software are you using to create this? If you don't mind me asking? Thanks for the inspiration!
  •  
  • Daniel Hayek staff 2 years ago
    We are the coolest machines!
  •  
  • LCDoering plus 2 years ago
    Very cool and scientifically accurate!
  •  
  • Matt S plus 2 years ago
    I like. Great visuals and very interesting voice over.
  •  
  • Sam Arias 2 years ago
    computers ain't got nothing on the Human Brain! Good stuff.
  •  
  • andy buchan 2 years ago
    Soooooo good!
  •  
  • Michael Fawke plus 2 years ago
    very very neat. and i learnt! woop
    good stuff
  •  
  • Ilya Nuform plus 2 years ago
    Cool! First of all i tought this is production, and it looks rather weird for 3D/compose soft. But for real time (maybe not real time render, but a real time work inside) it's great!!! I use v4, but i'm a novice still. Processing is cool of course. I think it is some kind of chemical reaction inside the soap bubble)
  •  
  • Andrew C. 2 years ago
    Very well done; impressive algorithm I assume coupled with a nice voice over. Keep it going!
  •  
  • James Alliban plus 2 years ago
    Great work Robert. Can't wait to see it. Did you use C++ for this or have you kept it in Processing?
  •  
  • Luke Perkin 2 years ago
    YES! I loooove radiolab. They do the most amazing podcasts.
  •  
  • ross kidder 2 years ago
    awesome. fascinating.
  •  
  • Daniel Péger 2 years ago
    yooo
  •  
  • InsightVR 2 years ago
    What are you using to connect it to ambient sound? Jack?
  •  
  • flight404 plus 2 years ago
    All but one comment has disappeared. Vimeo must be sorting some issues. I will try to answer the questions I have in my inbox.

    UPDATE ---- As soon as I hit Post Comment, all the comments reappeared. Odd!

    Firstly, thanks for the feedback and kind words!

    Bryce: Its made with Processing (processing.org) and uses Amit Pitaru's audio analysis library Sonia, and Andrew Bell's convenience classes.

    Syed: I use FFT on the incoming audio to get distinct frequencies. Each sphere 'listens' to a specific frequency and adjusts its mass and charge accordingly. There are also several thousand invisible particles who aid in coloring the sphere surface. I will likely write a blog post describing the process more thoroughly but it isn't that different from how I made the original piece. Read about it here: flight404.com/blog/?p=111

    planete: It always seems to happen this way. I will start working on a piece, I will get it working at realtime fps, I decide to post a video and the best way for me to do that is to save out the frames one by one as it runs, which slows it down. So of course I think, "well, since it isn't rendering out in realtime, I might as well crank up the particle count and get more bang". I will probably shoot a video off the monitor to better show how it looks running realtime. Its not too different from this video, but much less haze effect.

    James: Ended up staying with Processing. The next order of business, once this and Fuji are shipped, is to port both to Cinder/C++.

    InsightVR: For the live version, using a USB microphone. For this rendering, I pulled the audio bit I wanted into the project and analyzed it 1/30th of a second at a time. For each of those blips, I save out the resulting image, then piece the images back together and paste in the original audio.
  •  
  • flight404 plus 2 years ago
    And now, the comments are gone again. Strange.
  •  
  • InsightVR 2 years ago
    I'm seeing the same thing with the comments coming and going
  •  
  • wendy walker 2 years ago
    Lovely. What are you using to create the visuals in terms of software?
  •  
  • Paul Anthony Webb 2 years ago
    It is thought-provoking things such as this video that makes my day. =)

    I think I'll actually take a crack at Processing now.
  •  
  • Mac Rutan plus 2 years ago
    Nice work on the audio, graphics, and code! You're work is very interesting and inspirational.
  •  
  • jose nicolas 2 years ago
    Vert gréât idea. I like it
  •  
  • toby*spark 2 years ago
    great work, glad to see you pushing it in realtime.

    as a suggestion for creating videos of realtime/gpu applications, you can capture DVI output at 720P or 1080i using a DVI-HDMI lead and another pc with a BlackMagic intensity card. The card is cheap, your output runs unencumbered, you get a lossless recording.
  •  
  • Tony Santoloci 2 years ago
    great.... i love it... amazing sound clips as well
  •  
  • Daniel Maggi 2 years ago
    This was awesome. Nice work
  •  
  • Ravi Vora plus 2 years ago
    oooh, that is neat.
  •  
  • Jared Levy plus 2 years ago
    Congratulations man. Seriously. Just great.
  •  
  • Ozan Akinci 2 years ago
    halucinogenic!
  •  
  • subpixel 2 years ago
    Wicked. Thanks for the link to the discussion of the "original" Solar.
  •  
  • jessica fenlon plus 2 years ago
    fantastic, very complete work. stunner. thanks for sharing.
  •  
  • Nick Moore 2 years ago
    That was really awesome.
    Great scientific explanation.

    I wish the video would have ended at 1:43 though.
  •  
  • Marilyn Carren 2 years ago
    This is tasty food for thought! Great Job!
  •  
  • Max Siddha 2 years ago
    Very nice and sweet video, really!

    But does it somehow connected to a speech on background? Or is it just a fancy visualization to keep viewer's attention in?
  •  
  • nebulus design 2 years ago
    Really beautiful work as usual!
    Its a shame this isn't linux based, it's really easy to use pipes to link two applications and so send the pixels from GL to an encoder app like mencoder and do the recording in real time as well...
    On a decent PC I can record 720p at 30Hz :)
    That's how I've recorded a load of my vimeo clips.
  •  
  • peepeeland 2 years ago
    Sounds a bit like Max Cohen in π, haha.
  •  
  • al3000 2 years ago
    this is awesomeness
  •  
  • I saw this at the V&A in December. I had no idea that it was going to be there. It was bloody brilliant!
    Did you get a chance to see the other stuff there?
  •  
  • LVX 1313 2 years ago
    This one is very special.. Thank you for producing it.
  •  
  • Ashley James Brown 1 year ago
    seen at v&a - absolutely awesome
  •  
  • sarcasmosis 1 year ago
    Could you link up this specific episode of the podcast? I have to hear the rest of this.
  • sarcasmosis 1 year ago
    Oh god. Gold mine. Thanks so much and keep up that amazing work! Can't believe you've got it pumping out lyrics too. Crazy.
  •  
  • Pilar Narváez 11 months ago
    love it!
  •  
  • Oswaldo Glatt 8 months ago
    excelent! amazing!
  •  
  • alina.rea 3 months ago
    the voices are from a Radiolab podcast? they sound very similar :D
  • flight404 plus 3 months ago
    Yup. Krulwich talking to Jonah Lehrer.
  •  
This conversation is missing your voice. Take five seconds to join Vimeo or log in.

Advertisement

Statistics

Date Plays Comments
Totals 73.1K 1,187 49
Feb 13th 6 0 0
Feb 12th 12 0 0
Feb 11th 13 0 0
Feb 10th 21 0 0
Feb 9th 17 0 0
Feb 8th 20 0 0
Feb 7th 14 0 0