Audio-Visual Speech Recognition Using Red Exclusion and Neural Networks

Lewis, T.W. and Powers, D.M.W.

Automatic speech recognition (ASR) performs well under restricted conditions, but performance degrades in noisy environment. Audio-Visual Speech Recognition (AVSR) combats this by incorporating a visual signal into the recognition. This paper briefly reviews the contribution of psycholinguistics to this endeavour and the recent advances in machine AVSR. An important first step in AVSR is that of feature extraction from the mouth region and a technique developed by the authors is briefly presented. This paper examines how useful this extraction technique in combination with several integration architectures is at the given task, demonstrates that vision does infact assist speech recognition when used in a linguistically guided fashion, and gives insight remaining issues.

Cite as: Lewis, T.W. and Powers, D.M.W. (2002). Audio-Visual Speech Recognition Using Red Exclusion and Neural Networks. In Proc. Twenty-Fifth Australasian Computer Science Conference (ACSC2002), Melbourne, Australia. CRPIT, 4. Oudshoorn, M. J., Ed. ACS. 149-156.

(from crpit.com) (local if available)