As an effective fall accident preventive method, insight into near-miss falls provides an efficient solution to find out the causes of fall accidents, classify the type of near-miss falls and control the potential hazards. In this context, the paper proposes a method to detect and identify near-miss falls that occur when a worker walks in a workplace based on artificial neural network (ANN). The energy variation generated by workers who meet with near-miss falls is measured by sensors embedded in smart phone. Two experiments were designed to train the algorithm to identify various types of near-miss falls and test the recognition accuracy, respectively. At last, a test was conducted by workers wearing smart phones as they walked around a simulated construction workplace. The motion data was collected, processed and inputted to the trained ANN to detect and identify near-miss falls. Thresholds were obtained to measure the relationship between near-miss falls and fall accidents in a quantitate way. This approach, which integrates smart phone and ANN, will help detect near-miss fall events, identify hazardous elements and vulnerable workers, providing opportunities to eliminate dangerous conditions in a construction site or to alert possible victims that need to change their behavior before the occurrence of a fall accident.