In order to ensure the safety of construction, all kinds of construction machinery are widely applied to the construction site. Tower crane, as a material handling equipment, has the characteristics of wide operating range and large potential energy, and has become the core machinery in the construction site. The tower crane driver’s field of vision is often blocked, which seriously affects the safety of hoisting. To increase the view of tower crane drivers, most of the current monitoring systems will install a camera on the boom above the hook. But this camera can only view the situation around the hook, and it cannot be quantified. Based on this, this paper proposes a hoisting security detection technology based on deep learning. Firstly, the camera in the monitoring system is used to collect data sets. Secondly, the hook and workers are marked in the image. Then, Faster R-CNN is used to train and evaluate the data sets. The results show that the method has high recognition accuracy. However, the worker and the hook are not on a horizontal plane, so a verification test of the relationship between the height and the ratio of pixel length to true length was completed. The results show that the method can convert the ratio of the hook to the ratio of the worker, and then the real distance between the worker and the hook can be calculated.