In this paper, we propose an economical system for remote video player control. Through this system, we can use several simple gestures to control the video player, and these gestures can be alternated based on the user’s requirement and habits. The datasets used to train the gesture recognition model are recorded by a simple web camera in the laboratory. We utilize the CNN (convolutional neural network) to train the datasets and the user interface is designed by PyQt5. The gesture recognition system can be applied to switching television programs, controlling video games and household appliances etc.