Tracking of ball in sports videos is one of the most challenging tasks in computer vision and video processing domain. Recent ball tracking approaches fail to handle tracking of a small size and fast moving ball. Inaccurate 2D ball detection leads to further deterioration of 3D ball tracking results. This paper presents a soccer ball tracking by detection approach using a pre-trained Convolutional Neural Network (CNN). The proposed algorithm used CNN for identifying ball from background and other moving objects like players and referees. The 2D ball detection results are fine-tuned for identifying true ball positions. True ball positions, from cameras shooting the scene from different angle, are further mapped on ground plane. The actual ball movement is tracked in 3D from top-view. Experiments show that the proposed algorithm can tackle challenges like small ball size, shape changes, occlusion and tracking high-speed balls.