A Dual-Modal Silent Speech Interface via Surface Electromyography (sEMG) and Vibration Sensing

Human voices serve as a primary channel for rich information exchange; however, conventional acoustic speech recognition is susceptible to degradation under adverse environmental conditions or in cases of vocal impairment. Herin, we present a dual-modal silent speech recognition system that integrates hydrogel-based surface electromyography (sEMG) electrodes with a polyvinylidene fluoride (PVDF) vibration sensor to enable robust acquisition of nonacoustic throat signals. The sEMG electrodes, fabricated via a facile one-pot synthesis of poly(vinyl alcohol) (PVA)/poly(acrylic acid) (PAA) hydrogels, exhibit low interfacial impedance (270 Ω at 1 kHz), ensuring high-fidelity biopotential recording. In parallel, a 28-μm-thick PVDF piezoelectric unit demonstrates a sensitivity of 365 mV/Pa at 300 Hz and maintains a signal-to-noise ratio exceeding 48 dB across a wide frequency range (50 Hz-1 kHz), facilitating precise mechanical vibration detection. Both modalities are co-integrated onto a conformal polyimide (PI) substrate, enabling simultaneous and colocalized sensing of neuromuscular and vibrational signatures. To decode these complex spatiotemporal signals, we developed a dual-branch silent interaction network (DSI-Net), which fuses multimodal inputs to significantly improve the recognition accuracy. This system provides a scalable framework for silent, privacy-preserving human-machine communication, with potential applications in assistive technology and resilient voice interfaces.