Coral reef ecosystems is essential in healthy ocean and marine fishery. In the past decades, substantial of images and videos haven been collected from these cruises. These images are analyzed to quantify coral abundance in certain specific areas. However, the current manual analysis are time-consuming and labor intensive. In this paper, we proposes a fast automated tool for coral identification only based on sparse annotated labels by using deep learning method. There are two challenges to identify coral from such sparse labels and large images: one is to obtain denser labeled training data and the other is to improve the speed of testing on large images. In order to solves these problems, we propose a label augmentation algorithm to generate more labels and coarse-to-fine approach to find the location of corals quickly. Our methods were validated using the coral image dataset collected in Pulley Ridge region in the Gulf of Mexico, which substantial speed up the process of quantifying the corals while preserving accuracy.