Made by Andreas Refsgaard and Gene Kogan at Nabi AI Hackathon 2016 in Seoul.
***What is it?***
Doodle Tunes lets you turn doodles (drawings) of musical instruments into actual music. A camera looks at your drawing, detects instruments that you have drawn, and begins playing electronic music with those instruments.
***How does it work?***
Doodle Tunes is a software application, built with openFrameworks, that uses computer vision (OpenCV) and convolutional neural networks (ofxCcv) to analyze a picture of a piece of paper where instruments have been hand drawn, including bass guitars, saxophones, keyboards, and drums. The classifications are made using a convolutional neural network trained on ImageNet, and sends OSC messages to Ableton Live, launching various clips playing the music for each instrument. The software is available at github.com/ml4a/ml4a-ofx