Chinese Lip-Reading Based on Residual Network and Attention Mechanism
Authors : Chuan Qin; ZiLong Liu
Volume/Issue : Volume 11 - 2026, Issue 11 - November
Google Scholar : https://tinyurl.com/mr46c6tb
DOI : https://doi.org/10.38124/ijisrt/26nov599
Note : A published paper may take 4-5 working days from the publication date to appear in PlumX Metrics, Semantic Scholar, and ResearchGate.
Abstract : To address the common issues in existing lip-reading recognition systems, such as insufficient feature extraction and visual ambiguities, a Chinese sentence-level lip-reading recognition system based on residual networks and attention mechanisms (Spatio-Temporal Cascade Network, STCNet) is proposed. This system can first translate continuous videos containing speakers into pinyin sequences sentence by sentence, and then convert them into continuous Chinese sentences. Firstly, the residual network in the front-end feature extraction module can achieve bidirectional transmission and fusion of multi-level features, enhancing the extraction capability of long-term temporal features. In the back-end pinyin classification module, a Temporal Convolutional Network (TCN) is used to extract long-term dependency information, and an attention mechanism is introduced to weight discriminative features at different time intervals to reduce visual ambiguities
Keywords : Lip-Reading; Feature Extraction; Residual Network; Attention Mechanism; Deep Neural Network.
References :
Keywords : Lip-Reading; Feature Extraction; Residual Network; Attention Mechanism; Deep Neural Network.
