Reconstructing Hand Poses Using Visible Light

Tianxing Li, Xi Xiong, Yifei Xie, George Hito, Xing-Dong Yang, and Xia Zhou
Department of Computer Science, Dartmouth College


Free-hand gestural input is essential for emerging user interactions. We present Aili, a table lamp reconstructing a 3D hand skeleton in real time, requiring neither cameras nor on-body sensing devices. Aili consists of an LED panel in a lampshade and a few low-cost photodiodes embedded in the lamp base. To reconstruct a hand skeleton, Aili combines 2D binary blockage maps from vantage points of different photodiodes, which describe whether a hand blocks light rays from individual LEDs to all photodiodes. Empowering a table lamp with sensing capability, Aili can be seamlessly integrated into the existing environment. Relying on such low-level cues, Aili entails lightweight computation and is inherently privacy-preserving. We build and evaluate an Aili prototype. Results show that Aili’s algorithm reconstructs a hand pose within 7.2 ms on average, with 10.2° mean angular deviation and 2.5-mm mean translation deviation in comparison to Leap Motion. We also conduct user studies to examine the privacy issues of Leap Motion and solicit feedback on Aili’s privacy protection. We conclude by demonstrating various interaction applications Aili enables.


  • Paper
  • Presentation: slides



We sincerely thank our reviewers for their insightful comments. Special thanks to Yifei Xie and George Hito for the help on the user study and development of demo video. This work is supported in part by the National Science Foundation under grant CNS-1552924 and CNS-1421528. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the funding agencies or others.


DartNets Lab

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Dartmouth College