With the growing number of population in the world nowadays, novel human-computer interaction systems and techniques can be used to help improve their quality of life. A gesture based technology can help to maintain the safety and needs of the disable as well as the general people. Gesture recognition from video streams is a challenging task due to the high changeability in the features of each gesture with respect to different person. In this work, we propose a vision-based hand gesture recognition from RGB video data using SVM. Gesture-based interfaces are more natural, spontaneous, and straightforward. Previous works attempted to recognize hand gesture for different scenarios. Throughout our studies, gesture recognition system can be based on wearable sensor or it can be vision based. Our proposed method is applied on a vision based gesture recognition system. In our proposed system image acquisition starts from RGB videos capture using Kinect sensor. We convert the image frames from videos to blur for background noise removal. Then, we convert the images into hsv color mode. After that, we do the dilation, erosion, filtering, and thresholding the image for converting to black and white format. Finally, using the prominent classification algorithm SVM, hand gestures has been recognized. In conclusion, the framework aims to create a better vision-based hand gesture recognition system with novel techniques.