|
|
1.INTRODUCTIONIn the field of engineering, crane as an important handling equipment, its safety and efficiency directly affect the smooth progress of engineering construction [1]. The load swing angle, as one of the important parameters in crane operation, has a very important influence on the operator’s accurate control of the crane’s movement, as well as on the experimental control accuracy of the closed-loop crane positioning anti-shaking control system designed based on the swing angle feedback. The methods of crane swing angle detection can be categorized into conventional sensor methods and visual detection methods[2].The unified sensor method is mostly contact-based, through the installation of rotary encoders[3][4], gyroscopes[5], accelerometers[6] and other sensor devices to obtain the crane’s angular velocity and acceleration and other attitude informatio, there are also methods to predict the crane swing angle by using mathematical models without sensors [7];Visual detection methods, on the other hand, use monocular vision[8][11], binocular vision[12] camera measurement method or laser scanning method[13], etc.. The principle of this method is based on image processing and target detection technology, and the state of the load is inferred by analysing the position and morphology of the object in the image, but is more sensitive to the influence of ambient light and background. When cameras are used as visual measurement devices, they are mostly based on monocular vision, Huang et al. developed a robust real-time detection method for bridge crane load swing angles, using spherical markers and combining a mean drift algorithm and Kalman filtering for accurate tracking.[14]. In addition, some research teams have also used binocular vision methods, Inukai T et al. Visual feedback control using stereo vision effectively tracks, localises and suppresses the swing of a boom crane by measuring the threedimensional position and swing angle of the load[15]. The objective of this research is to utilize the YOLOV5s object detection algorithm to determine the pixel displacement of the load relative to its resting position. By employing the camera imaging model and data from the rope-length encoder, the actual displacement of the load is calculated. Subsequently, the load swing angle is determined based on the spatial positions of the trolley and the load. To validate the effectiveness of the proposed measurement method, a visual measurement system is constructed. 2.MODEL BUILDING AND CAMERA CALIBRATIONThe accurate measurement of load displacement is very much related to the accuracy of the establishment of the mapping model of the load coordinate information in the 2D image and the load offset in the 3D space. In order to ensure a higher accuracy of the obtained mapping model, it is necessary to make the internal and external parameters of the camera more accurate and the image distortion smaller. According to the monocular camera imaging model, this paper adopts the camera calibration based on the 2D planar checkerboard grid[16], and only considers the tangential aberration which has the greatest influence, and adopts the 8 × 9 checkerboard grid with the size of 25mm × 25mm for each grid, and collects 13 checkerboard grid images with different poses and with the size of 2592 × 1944 pixels, and through the computation, the transformation relationship between the target points in the camera coordinate system and the projected points in the pixel coordinate system is obtained, i.e., the inner parameter matrix is as follows The transformation relationship between the target points in the camera coordinate system and the projected points in the pixel coordinate system, i.e., the inner parameter matrix Kin is as follows: Inner parameter matrix: Aberration parameters: The standard variance of the calibration error is: The external parameter matrix of the camera varies for different orientations of the checkerboard. In calibration experiments, the obtained external parameter matrix can represent the pose of the checkerboard relative to the camera coordinate system. 3.MODEL TRAINING3.1Data set productionConsidering the influence of factors such as light and background on the detection effect, this paper collects a large number of images with different light intensities, different distances, and different backgrounds, and adds a large number of background objects to improve the quality of the dataset. The dataset consists of 1650 pictures with a resolution of 2592 pixels. In this paper, we first pre-processed the images, and used Gaussian filtering to denoise the images and histogram equalisation to enhance the filtered images[17]. After Gaussian filtering, the noise signal is no longer obvious, and the edge information of the picture is also retained; after the histogram equalisation enhancement, the pixel value of the whole picture tends to be uniformly distributed in each grey level, and the enhancement effect is good. The processing effect is shown below. 3.2Model training based on YOLO v5s target detection algorithmIn this paper, the single-stage target detection algorithm YOLOv5s is used, and the constructed dataset is chosen to be labelled with LabelImg tool, and then divided into validation set and training set according to the ratio of 2:8 to train the model. The training results and related indicators are shown below. During the 100 epochs, with each iteration, the first two types of losses tend towards 0, indicating effective parameter optimization. Since this study employs single-object detection with no misclassified images, the classification loss is 0. The precision of the network reaches 99.67%, with a recall rate of 99.57%. The mAP@0.5 approaches 1, indicating excellent model performance. Additionally, the mAP@0.5: 0.95 gradually increases in the graph, demonstrating that with an increase in training iterations, the model predictions become more accurate. 4.PENDULUM ANGLE MEASUREMENT EXPERIMENTS AND ANALYSIS OF RESULTS4.1Experimental platform constructionThe vision measurement system for the load swing angle of the bridge crane mainly includes several components: (1) rope length measurement device (2) monocular industrial camera (3) industrial control machine (4) PLC motion controller, which directly uses Open VINO in the industrial control machine to accelerate the reasoning of the YOLO v5s detection algorithm, the reasoning speed is more than twice as fast as compared to the direct deployment and the detection effect is comparable to it. 4.2Load swing Angle solutionFrom the results of the above detection algorithm, the pixel displacement of the load with respect to rest can be obtained. The errors in the geometric centres of the actual hooks and preselection frames are 47 pixels and 17 pixels in the horizontal and vertical directions, respectively, and the pixel displacement error is about 50 pixels. In the image with a resolution of 2592 × 1944, the errors are 1.93% and 2.57% of the image length and width. The error in the right image is 6 pixels, which is 0.23% and 0.31% of the image length and width. Therefore, it is feasible to use the centre of the preselection frame instead of the geometric centre of the hook to calculate the load displacement. The camera was mounted vertically under the crane, ensuring that the load was parallel to the optical axis Z of the camera. The load must not oscillate when mounted and the hook geometric centre must not be more than 3 pixels away from the centre of the hook at different lifting heights, ensuring that the optical axis is parallel to ZW. In this paper, we use the fitting method to solve the vertical distance from the load to the camera, by calculating the pixel distance Δh for the width of the hook and reading the rope length D from the encoder, we get to obtain an approximate relationship between the hook distance and the vertical distance between the camera and the load as shown below: Therefore, the pixel displacement of the hook in relation to the hook displacement can be expressed as: From the vertical distance between the camera and the load, through the camera, the load, the lifting centre of the relationship between the three to solve the load swing angle. Load swing angle calculation schematic shown in the figure, in the main view, the load and lifting centre horizontal distance S, rope length l and the camera to the vertical distance of the lifting centre D constitutes a right-angled triangle; in the top view, S and the load with respect to the world coordinate system of XW and YW, the direction of the offset ΔXW and ΔYW also constitutes a right-angled triangle, according to the triangular relationship between the conversion, you can get the formula for the load swing angle: 4.3Static load pendulum angle measurement experimentThe experiment uses an angle ruler to measure the load swing angle at a specific location with a resolution of 0.01°. Experiment in charge of the same height (load from the beam 1.6m) as well as different heights of different positions will be placed in eight different places, the experiments measured pendulum angle data and the measurement error as shown in Table 1 and Table 2. Table 1.Pendulum Angle Measurements at Different Positions at the Same Height with Errors.
Table 2.Pendulum Angle Measurement Results and Errors for Different Positions at Different Heights.
Regardless of the position of the load, the error between the results measured by the method of measuring the load pendulum angle in this paper and the results measured by the angle ruler will not be more than 0.21°, but the error measured at different heights is obviously larger than that measured at the same height, and this paper statistically measures the error rate of the two experimental measurements, and plots the error rate of the two measurements as shown in Figure 7. In the figure, except for the 4th experiment and the 6th experiment in which the error rate is slightly higher, the results of the rest of the experiments are the error rate of the measurement of the pendulum angle at different heights is high, and the average error rate at different heights is higher than that of the measurement results at the same height. According to the principle of the calculation of the pendulum angle, each time before the calculation of the pendulum angle needs to use the fitted formula to estimate the perpendicular distance between the load and the camera, and at different heights, it is necessary to estimate the distance for many times, and the error arises from this, therefore, in order to reduce the error, the method proposed in this paper as far as possible in the same length of the rope to carry out measurements. 4.4Dynamic pendulum angle measurement experimentsThis experiment compared the load pendulum angles measured by the visual measurement system and the background modeling method[18]. Figure 8(a) illustrates the simulation and experimental results for a 5° initial pendulum angle applied to the load while the cart remains stationary. The simulation curve shows the load exhibiting simple pendulum motion, with angle variations similar to a cosine function. The angles measured by the visual system are closer to the simulation results, with errors within 0.5° and smaller deviations at the maximum and minimum angles. Although the background modeling method has a shorter sampling period, it introduces errors. Figure 8(b) presents the simulation and experimental results for the load angles when a 6-second, 80-newton step input is applied to the cart. This experiment demonstrated more pronounced angle changes and peak oscillations. The results of both measurement methods are similar, with noise interference present in the signals. Mechanical vibrations from the cart’s movement caused pixel value shifts in the camera imaging, resulting in errors. Despite measurement errors remaining within 0.5°, the tracking performance is good, and future research can focus on improving angle measurement accuracy. 5.CONCLUSION
ACKNOWLEDGMENTSThis work was supported by the Hubei Province key research and development project (number: 2023BEB046). REFERENCESYang, B. and MA, L.,
“An Anti-Sway Controller’s Design of the Overhead Crane,,”
Advanced Materials Research, 328 1868
–1871
(2011). Google Scholar
Xiong, X. L., Zhang, Y., Zhou, Q., et al.,
“Bridge Crane Pendulum Angle Detection System Based on YOLOv3,,”
Hoisting and Conveying Machinery(04), 30
–33
(2021). Google Scholar
Tuan, L. A., Kim, J. J., Lee, S. G., et al.,
“Second-order sliding mode control of a 3D overhead crane with uncertain system parameters,,”
International journal of precision engineering manufacturing, 15 811
–819
(2014). Google Scholar
Aksjonov, A., Vodovozov, V., and Petlenkov, E.,
“Three-dimensional crane modelling and control using Euler- Lagrange state-space approach and anti-swing fuzzy logic,,”
Electrical, Control Communication Engineering, 9
(1), 5
–13
(2015). Google Scholar
Lu, K., and Wang, T.,
“Experimental research on anti-sway control of hoisting manipulator end spreader,,”
Journal of Mechanical & Electrical Engineering, 39
(12), 1776
–1783
(2022). Google Scholar
Xiao, P., and Wang, B.,
“Research of Vision-Sensorless Anti-Sway Control System based on MEMS Microaccelerometer,,”
Journal of Mechanical & Electrical Engineering(01), 1
–5
(2005). Google Scholar
Ohtomo, S., and Murakami, T.,
“Estimation method for sway angle of payload with reaction force observer.,”
in 13th International Workshop on Advanced Motion Control (AMC). IEEE.,
581
–585
(2014). Google Scholar
Wu, Q. X., Wang, X. K., Hua, L., et al.,
“The real-time vision measurement of multi-information of the bridge crane’s workspace and its application,,”
Measurement, 151 107207
(2020). Google Scholar
Huang, L. S., Wang, H. X., Yan, F., et al.,
“Machine Vision Based Pendulum Angle Measurement of Large Amusement Rides,,”
Safety Technology of Special Equipment,
(01), 42
–44
(2024). Google Scholar
Zhang, W. P., Xu, W. M., Gu, X. T., et al.,
“Load spatial localization of bridge crane based on improved circle detection algorithm,,”
Transducer and Microsystem Technologies, 9
(12), 46
–49
(2020). Google Scholar
Hyla, P., and Szpytko, J.,
“Crane payload position measurement vision-based system dedicated for anti-sway solutions.,”
Telematics - Support for Transport. TST 2014, 404
–413
(2014). Google Scholar
Rahman, M. S., and Vaughan, J.,
“Simple near-realtime crane workspace mapping using machine vision.,”
in Dynamic Systems and Control Conference. American Society of Mechanical Engineers.,
V003T28A005
(2014). Google Scholar
Cheng, J. W., Gong, M. L., Liu, X. Z., et al.,
“Comparison and Design of Two Different Deflection Angle Measurement System for a2DOF Laser Scanner,,”
Laser & Infrared, 32
(4), 237
–239
(2002). Google Scholar
Huang, J. L., Xu, W. M., et al.,
“An improved method for swing measurement based on monocular vision to the payload of overhead crane,,”
Transactions of the Institute of Measurement Control, 44
(1), 50
–59
(2022). Google Scholar
Inukai, T. and Yoshida, Y.,
“Control of a boom crane using installed stereo vision.,”
in Sixth International Conference on Sensing Technology (ICST), IEEE,
189
–194
(2012). Google Scholar
Zhang, Z. Y.,
“A flexible new technique for camera calibration,”
in IEEE Transactions on Pattern Analysis and Machine Intelligence,, 22
(11), 1330
–1334
(2000). Google Scholar
Tu, G., Liu, H. Q. and Zhu, C. P.,
“An improved adaptive non-local mean image denoising method,,”
Control Eng, 23
(6), 839
–843
(2016). Google Scholar
Xu, P., Fang, Y. C. and Chen, H.,
“Background Modeling Based Payload Swing Angle Measuring Method of Bridge Crane System,,”
Control Engineering of China, 26
(9), 1613
(2019). Google Scholar
|
