MAGE/pHTS is a modified version of the HTS speech synthesis engine (hts.sp.nitech.ac.jp) which allows HMM-based speech synthesis to be used in reactive programming designs. In this video, the beginning of "Alice in Wonderland" is first synthesized with natural prosody. Then, pitch, speed and vocal tract length are altered in real time by moving the pen on the surface of the HandSketch, showing how voice in which way the synthesis properties can be updated as the synthesis is going. We conclude by adding a circular phoneme sequencer to this set-up, which enables us to build a sentence on-the-fly (here, "Alice was beginning to get"), showing how content information can be also sent in real time to MAGE.
This work was performed at the NUMEDIART Institute ( numediart.org ) and during the 7th eNTERFACE workshop, Plzen ( Czech Republic ), August 2011 (enterface11.zcu.cz ) under the supervision of Nicolas d'Alessandro and Thierry Dutoit.