Real-Time Screen-Camera Communication Behind Any Scene

Tianxing Li, Chuankai An, Xinran Xiao, Andrew T. Campbell, and Xia Zhou
Department of Computer Science, Dartmouth College


We present HiLight, a new form of real-time screen-camera communication without showing any coded images (e.g., barcodes) for off-the-shelf smart devices. HiLight encodes data into pixel translucency change atop any screen content, so that camera-equipped devices can fetch the data by turning their cameras to the screen. HiLight leverages the alpha channel, a well-known concept in computer graphics, to encode bits into the pixel translucency change. By removing the need to directly modify pixel RGB values, HiLight overcomes the key bottleneck of existing designs and enables real-time unobtrusive communication while supporting any screen content. We build a HiLight prototype using off-the-shelf smart devices and demonstrate its efficacy and robustness in practical settings. By offering an unobtrusive, flexible, and lightweight communication channel between screens and cameras, HiLight opens up opportunities for new HCI and context-aware applications, e.g., smart glasses communicating with screens to realize augmented reality.




bibtex Tianxing Li, Chuankai An, Xinran Xiao, Andrew T. Campbell, and Xia Zhou. Real-Time Screen-Camera Communication Behind Any Scene, The International Conference on Mobile Systems, Applications, and Services (MobiSys), May, 2015, Florence, Italy. Best Demo Award. [PDF]
bibtex Tianxing Li, Chuankai An, Andrew T. Campbell, and Xia Zhou. Hiding Bits in Pixel Translucency Changes, The 1st ACM Workshop on Visible Light Communication Systems (VLCS), September, 2014, Maui, Hawaii. Best Paper Award. [PDF]


We sincerely thank the reviewers for their valuable feedback and Jiawen Chen at Google for his insights on GPU programming. We also thank DartNets Lab members Zhao Tian, Rui Wang, Fanglin Chen, and Xiaole An for their support on our study. Special thanks to Jacob A. Gaba for helping prepare the demo video. This work is supported in part by the Dartmouth Burke Research Initiation Award and the National Science Foundation under grant CNS-1421528. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the funding agencies or others.



DartNets Lab

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Dartmouth College