We were recently engaged by Goodby Silverstein and Partners to build an Augmented Reality experience that educates consumers on 'tempering', the 2 minute period you should wait till your Haagen-Dazs reaches the ideal soft consistency. Entitled 'Concerto Timer', the app entertains by offering an elegant duet of a violinist and cellist playing a classical music piece while you wait.
Once the app is launched the user is directed to point their iPhone at a carton of Haagen Dazs. The phone’s camera which recognizes all flavours and sizes, immediately proceeds to the AR experience. An elegant violinist created out of particles appears out the carton lid, beginning the classical serenade. Adding a second carton of Haagen-Dazs to the experience reveals a cellist, accompanying the violinist in perfect sync.
There are 2 typical approaches to AR, either create a video which will live on a flat pane or create a low poly 3D model that looks video gamey. Never satisfied with taking the typical route, we decided to invent a new one. We combined video with a particle engine to give our musicans a truly unique 3 dimensional form to that hasn't been done before.
In order to achieve the ethereal effect of the musicians, footage of the violinist and cellist were shot with both a DSLR camera and a Kinect camera (Microsoft’s depth and motion sensing device for Xbox). The two cameras were synchronized on the shoot, and during our in house post production process the data was combined to create the holographic like effect.
As for the other nuts and botls, the Concerto Timer app was written natively in Objective-c and includes Qualcomm’s VUFORIA AR technology. After exhaustive research with various competitors, Vuforia was selected as it offered the most stable performance in tracking the Haagen Dazs branding on the wide array of ice cream products while still leaving room for our engine to work as it does.