Multimodal Sensing and Physical-Digital Interaction

Connect cameras, gestures, rotary controls, physical props, books and lighting to develop intuitive and maintainable interaction.

Explore motion and object-contour recognition, encoder inputs, page recognition, near-infrared tracking and lighting integration. Use gesture browsing, projection tables and tracking prototypes to test physical actions against digital responses.

Recommended testing covers ambient light, occlusion, interference, latency and false triggers. Infrared sensing needs wavelength, material and camera validation; lighting needs protocol compatibility checks; tracking precision requires measurement.

• [Room-Scale VR](/en/research/projects/room-scale-vr-rd)
• [Interactive Crossbow Game](/en/research/projects/crossbow-interaction-rd)
• [Interactive Projection Table](/en/research/projects/projection-dining-table)
• [Gesture-Based Silk Manuscript Browsing](/en/research/projects/gesture-manuscript-browsing)
• [Changde Rice Noodle Making Interaction](/en/research/projects/changde-rice-noodle-interaction)
• [Huaihua Brocade Music Interaction](/en/research/projects/huaihua-brocade-music)
• [Programmable Light Tree](/en/research/projects/programmable-light-tree)
• [AI Beauty and Skin Visualisation](/en/research/projects/ai-beauty-visualisation)
• [Nuo Theatre Transformation Experience](/en/research/projects/nuo-theatre-face-effects)
• [Anhua Dark Tea Scent Experience](/en/research/projects/anhua-tea-scent)
• [Lacquer-Inspired Souvenir Generator](/en/research/projects/lacquer-souvenir-generator)
• [Historical Astronomy Dial](/en/research/projects/historical-astronomy-dial)
• [Cloud-Pattern Lacquerware Puzzle](/en/research/projects/cloud-pattern-lacquer-puzzle)
• [Gesture-Based Floral Tribute](/en/research/projects/flower-tribute-gesture)
• [Interactive Telegraph Exhibit](/en/research/projects/telegraph-interaction)
• [Integrated Interactive Book Device](/en/research/projects/integrated-interactive-book)
• [Three-Point Infrared Tracking Interaction](/en/research/projects/three-point-infrared-tracking)

Synthesised from slides 8、32–33、38–43 of the 2025 R&D Planning Framework. This thematic grouping interprets the source's cultural-IP and modular-technology approach. Validation steps are recommendations, not completed work or scheduled commitments.