masterhead masterhead  masterhead

High Speed Gesture UI: Ultra Low latency with Proprioception


In this research, we propose a high speed gesture user interface as an example of ultra low latency interface enhancing the usability.

In user interface of smart TV, computer, video game, etc., the high speed property of a response has a significant influence on the operation feeling, because proprioception which is the sense of linking one's own gesture to the operation on the display, improves by low latency of the datapath from user input to output to the user.

The figure below shows the outline of our system. In gesture interface, a delay occurred due to A) Capture delay (caused by scanning imager array = the reciprocal of frame rate), B) Input delay (caused by data transfer to an input buffer of a computer or an image processing unit), C) Processing delay (caused by image processing including gesture recognition), D) Output delay (caused by data transfer to an output buffer), E) Display delay (= the reciprocal of frame rate). In the case that each device does not achive synchronization, buffering time which prevents synchronization deviation called frame dropping is needed. In the system based on ordinary video rate, total latency takes about 150 - 200 ms even if gesture recognition is finished within 1 frame. We implemented each step A) - D) based on 1,000 fps, and used a display with 8 ms delay. As a result, this system realized high speed gesture interface which has only about 32 ms total latency (the time from the camera observing LED light to a photodetector observing the change of the display).

high speed ui system
high speed ui A high speed ui B high speed ui C


Ishikawa Senoo Laboratory, Department of Information Physics and Computing, Department of Creative Informatics,
Graduate School of Information Science and Technology, University of Tokyo
Ishikawa Senoo Laboratory WWW admin:
Copyright © 2008 Ishikawa Senoo Laboratory. All rights reserved.