Since long time ago, speech recognition has been researched, though it does not work well in noisy places such as in the car or in the train. In addition, people with hearing-impaired or difficulties in hearing cannot receive benefits from speech recognition. To recognize the speech automatically, visual information is also important. People understand speeches from not only audio information, but also visual information such as temporal changes in the lip shape. A vision based speech recognition method could work well in noisy places, and could be useful also for people with hearing disabilities. In this paper, we propose an automatic lip-reading method for recognizing the speech by using multimodal visual information without using any audio information such as speech recognition. First, the ASM (Active Shape Model) is used to track and detect the face and lip in a video sequence. Second, the shape, optical flow and spatial frequencies of the lip features are extracted from the lip detected by ASM. Next, the extracted multimodal features are ordered chronologically so that Support Vector Machine is performed in order to learn and classify the spoken words. Experiments for classifying several words show promising results of this proposed method.
This paper proposes a method for tracking and recognizing the white line marked in the surface of the road from the
video sequence acquired by the camera attached to a walking human, towards the actualization of an automatic
navigation system for the visually handicapped. Our proposed method consists of two main modules: (1) Particle Filter
based module for tracking the white line, and (2) CLAFIC Method based module for classifying whether the tracked
object is the white line. In (1), each particle is a rectangle, and is described by its centroid's coordinates and its
orientation. The likelihood of a particle is computed based on the number of white pixels in the rectangle. In (2), in order
to obtain the ranges (to be used for the recognition) for the white line's length and width, Principal Component Analysis
is applied to the covariance matrix obtained from valid sample particles. At each frame, PCA is applied to the covariance
matrix constructed from particles with high likelihood, and if the obtained length and width are within the abovementioned
ranges, it is recognized as the white line. Experimental results using real video sequences show the validity of the proposed method.