“AI DJ Project” is a live performance featuring an Artificial Intelligence (AI) DJ playing alongside a human DJ. Utilizing various deep neural networks, the software(AI DJ) selects vinyl records and mixes songs. Playing alternately, each DJ selects one song at a time, embodying a dialogue between the human and AI through music. DJ-ing “Back to Back” serves as a critical investigation into the unique relationship between humans and machines.
The system of AI DJ consists of the following three features:
1. Music selection
We trained three different neural networks, each of these models can infer genres, musical instruments and drum machine used in the track respectively from spectrogram images. When human DJ is playing, the system feeds the incoming audio into the model and extract auditory features in a high dimensional vector. The vector will be compared with those of all tracks in our pre-selected record box (with over 350 tracks for the present), so that the system can select the closest track, which presumably has similar musical tone/mood/texture, as the next one to play.
It is a task for AI DJ to control the pitch of the turntable to match the beat with music human DJ plays. We used “reinforcement learning”(RL) to teach the model how to speed up/down, nudge/pull the turntable to align downbeats through trials and errors. For this purpose, we built an OSC-compatible custom turntable and robot fingers to manipulate the rotation speed.
Also, A good DJ needs to pay attention to the energy of the audience. We utilize a deep leaning based motion tracking technique to quantify how much people in the audience dance to the music AI plays for future music selection.
We have performed several times in different locations in Japan and Europe. AI's slight unpredictability always brought amusing tension into the performance, which is only made possible by the interaction between the human and AI DJs. In the performance AI is not a replacement for the human DJ, instead it is a partner that can think and play alongside its human counterpart, bringing forth a wider perspective of our relationship to contemporary technologies.