Auto-focus is a common function in consumer digital cameras and smart phones, but less applied into stereo microscopes, which often require repeated manual focusing to obtain a clear image. To be less time-consuming and labor-consuming during focusing operation with stereo microscopes, this paper presents a real-time and feasible continuous auto-focus algorithm. Firstly, a direction decision method is proposed to determine the beginning direction of motor for sensor moving. Secondly, an auto-focus search algorithm is improved to find the best focus position. Thirdly, in order to guarantee higher efficiency and accuracy of motor movement, an adaptive step size method is introduced to optimize the value of focus objective function. Lastly, to meet the need of continuous auto-focus, an auto-focus monitoring algorithm utilizing a finite-state-machine model is proposed to determine when to refocus and enhance the stability and responsiveness of auto-focus system. This approach has been implemented on the prototype of our stereo microscope camera to verify the auto-focus performance. Experimental results show that the proposed auto-focus algorithm can almost resist the interference in the focusing process, and achieve a good balance between accuracy and efficiency.