When combined with acoustical speech information, visual speech information (lip movement) significantly improves
Automatic Speech Recognition (ASR) in acoustically noisy environments. Previous research has demonstrated that
visual modality is a viable tool for identifying speech. However, the visual information has yet to become utilized in
mainstream ASR systems due to the difficulty in accurately tracking lips in real-world conditions. This paper presents
our current progress in tracking face and lips in visually challenging environments. Findings suggest the mean shift
algorithm performs poorly for small regions, in this case the lips, but it achieves near 80% accuracy for facial tracking.