Towards On-the-wall Tangible Interaction: Using Walls as Interactive, Dynamic, and Responsive User Interface

Team: Zeyu Yan, Anup S, Pedro Carvalho, Yongqao Hu, Annan Li

Advisor: Huaishu Peng

Contributions: Electronic design, firmware design

Skills picked up/strengthened: Firmware design, PCB design, power management, Flask apps using Python.

Publications: Towards On-the-wall Tangible Interaction: Using Walls as Interactive, Dynamic, and Responsive User Interface to appear in CHI Late Breaking Work, 2021.

This paper presents our vision of on-the-wall tangible interaction. We envision a future where tangible interaction can be extended from conventional horizontal surfaces to vertical surfaces; indoor vertical areas such as walls, windows, and ceilings can be used for dynamic and direct physical manipulation. We first discuss the unique properties that vertical surfaces may offer for tangible interaction and the interaction scenarios they imbue. We then propose two potential paths for realizing on-the-wall interaction and the technical challenges we face. We follow with one prototype called Climbot. We showcase how Climbot can be used as an on-the-wall tangible user interface for dynamic lighting and as a wall switch controller. We conclude with a discussion of future work.