This paper presents a deep learning-based concrete crack detection technique using hybrid images. The hybrid images combining vision and infrared (IR) thermography images are able to improve crack detectability while minimizing false alarms. Large scale concrete-made infrastructures such as bridge, dam, and etc. can be effectively inspected by spatially scanning the hybrid imaging system including vision camera, IR camera and continuous-wave line laser. However, the decision-making for the crack identification often requires experts’ intervention. As a target concrete structure gets larger, automated decision-making becomes more necessary in the practical point of view. The proposed technique is able to achieve automated crack identification by modifying a well-trained convolutional neural network using a set of crack images as a training image set, while retaining the advantages of hybrid images. The proposed technique is experimentally validated using a lab-scale concrete specimen developed with various-size cracks. The test results reveal that macro- and micro-cracks are automatically detected with minimizing false-alarms.