Research in this visual servoing field in the past few decades has produced remarkable results, leading to many exciting expectations as well as new challenges. However, because of the complicated calculation of the inverse Jacobian, it is difficult to implement in real time. Therefore, instead of using the inverse Jacobian, this paper employs the ANFIS (Adaptive Neuro Fuzzy Inference System) approach for visual servo control of a robot manipulator. It is based on visual feedback and no prior information about the kinematics of robot and the camera calibration are unnecessary. Firstly, to efficiently control a manipulator, 3D space is divided into two 2D spaces. And then, we acquire training data from each 2D space and ANFIS is learned by the training data. We categorize the robot movement into two kinds of actions. That is, TOWARD action is performed, in the xy plane, by joint 1 and APPROACH action is performed, in the plane orthogonal to the xy plane, by joint 2 and joint 3. The time varying object can be tracked by controlling both actions in each plane and the simulation results show the validation of our approach.