MobileHCI 2012 Paper
(Best Paper Nominee)
In the future mobile devices will have depth sensors. In this paper we study the design space of free-space interactions afforded by mobile depth sensors. Such interactions will have a greater working volume, more fluid control and avoid screen occlusion effects intrinsic to touch screens. This work contributes the first study to show that mobile free-space interactions can be as good as touch. We are also the first to analyze interaction volume usage, resulting in strong implications for how sensors should be placed on mobile devices. We describe a user study evaluating mobile free-space navigation techniques and the impacts of sensor orientation on user experience. Finally, we discuss guidelines for future mobile free-space interaction techniques and sensor design.
A Kinect-projector visualization where distances from the Kinect sensor are visualized through colors (blue=close and red=far). The Kinect sensor and a standard projector are calibrated using the Projector-Camera Calibration Library (an open-source project to be released soon).
Information can be overlaid onto physical object through projection-based interfaces such as our work in: vimeo.com/12154930.
ISMAR 2010 - Best Student Paper Award (http://www.ismar10.org)
We present a novel way of interacting with everyday objects by representing content as interactive surface particles. Users can build their own physical world, map virtual content onto their physical construction and play directly with the surface using a stylus. A surface particle representation allows programmed content to be created independent of the display object and to be reused on many surfaces. We demonstrated this idea through a projector-camera system that acquires the object geometry and enables direct interaction through an IR tracked stylus. We present three motivating example applications, each displayed on three example surfaces. We discuss a set of interaction techniques that show possible avenues for structuring interaction on complicated everyday objects, such as Surface Adaptive GUIs for menu selection. Through an informal evaluation and interviews with end users, we demonstrate the potential of interacting with surface particles and identify improvements necessary to make this interaction practical on everyday surfaces.