To solve problems including the surface distortion and profile error cannot be considered in robotic off-line programming, and the object parts are often texture-less or even without contour, we propose a new visual servo control method directly using the phase map as the visual features. The high accuracy of phase map ensures the high accuracy of positioning or target tracking tasks. Every entry of the phase map is used as visual features so no complex image processing and feature extracting and matching are needed. The corresponding interaction matrix is deduced and control law is constructed. The validity of the proposed method is proved by simulation.