CN113947116B - A camera-based non-contact real-time detection method for train track looseness - Google Patents

Abstract

In view of the current problem of low efficiency of manual inspection for looseness of train tracks, the present invention discloses a camera-based non-contact real-time detection method for looseness of train tracks. This method uses a high-speed camera to obtain a video of the vibration of the track when a train passes or is artificially stimulated. Using the characteristics of the staggered arrangement between the train track and sleepers, a virtual feature point detection method based on image pixel gray gradient and feature clustering algorithm is proposed. The optical flow algorithm calculates the optical flow of virtual feature points, and then measures the time-domain vibration of the track. The natural vibration frequency of the track is obtained through FFT decomposition, and then whether the track is loosened is judged in real time through changes in the natural vibration frequency. The proposed method has a simple measuring device, high precision, low cost and easy operation.

Description

A non-contact real-time detection method of train track looseness based on camera

Technical field

The invention belongs to the technical field of structural motion measurement, and in particular relates to a camera-based non-contact real-time detection method of train track looseness.

Background technique

The train track system is an important part of the transportation system, mainly composed of tracks, sleepers, fasteners and roadbed. The track is fixed to the sleepers by fasteners. When the train is running, the periodic impact dynamic load generated by it can easily cause the fasteners to vibrate. If things go on like this, the track will loosen. As track loosening intensifies, the dynamic response amplitude of the track structure becomes significantly larger, which may cause train derailment accidents in severe cases. Therefore, train track looseness detection is of great significance to ensure the safety of train driving.

At present, the commonly used method for detecting train track looseness is mainly manual inspection, and experienced inspection personnel usually use the naked eye to judge the status of the fasteners. This method is simple to operate, has low efficiency, high cost, high missed detection rate and great safety hazards. In recent years, with the development of image processing technology, computer vision technology has gradually been applied to structural health monitoring of train tracks. Currently, the computer vision detection method used requires marking the structure and measuring the orbital vibration by tracking the marked points. However, due to the large number of train tracks and their long-term outdoor environment, the marking points are easy to fall off, which brings great challenges to measurement.

Contents of the invention

In order to overcome the shortcomings of the above-mentioned existing technologies and solve the problems of low efficiency and low accuracy of the train track looseness detection method based on manual inspection, the purpose of the present invention is to provide a camera-based non-contact real-time detection method of train track looseness, using the train track and the characteristics of the staggered arrangement between sleepers, a virtual feature point detection method based on image pixel gray gradient and feature clustering algorithm is proposed. The optical flow algorithm is used to calculate the optical flow of the virtual feature points, and then the time domain vibration of the track is calculated. Measurements are made, and the natural vibration frequency of the track is obtained through FFT decomposition, and then whether the track is loosened is judged in real time through changes in the natural vibration frequency.

In order to achieve the above objects, the technical solution adopted by the present invention is:

A camera-based non-contact real-time detection method for train track looseness, including the following steps:

Step 1), use a high-speed camera to video record the vibration of the train track;

Step 2), perform grayscale processing on the recorded video frame by frame;

Step 3), calculate the gray gradient of the pixels in the first frame of the image, use the K-means clustering algorithm to cluster the gray gradient, determine the track and fastener areas, and use the pixel gray extreme values in the area as the virtual value of the area. Feature points;

Step 4), build an image multi-scale pyramid through image multi-scale decomposition technology, establish optical flow equations for images at different scales based on the short-term brightness constant theory, and use the least squares algorithm to calculate the optical flow of virtual feature points on images at different scales;

Step 5), use the scale relationship between pyramid images to fuse the optical flow of virtual feature points on images of different scales to obtain the calculation results of the optical flow of virtual feature points. Based on the image calibration technology, obtain the time domain vibration signals of the track and fasteners. ;

Step 6): Conduct frequency domain analysis on the time domain vibration signals of the track and fasteners to obtain the natural frequency of the track, and detect track looseness through the natural frequency change characteristics.

Further, step 2 is specifically, in order to improve the screening efficiency of virtual feature points, perform grayscale processing on the color images obtained by the camera frame by frame to obtain grayscale images:

I9x,y)＝0.299*R9x,y)+0.579*G9x,y)+0.114*B(x,y)

In the formula: I(x,y) is the gray value of pixel (x,y); R(x,y), G(x,y), B(x,y) are the gray value of pixel (x,y) respectively. Pixel values of three channels.

Further, step 3 specifically includes screening virtual feature points based on image pixel grayscale gradient and feature clustering algorithm.

In the present invention, it is assumed that the gradient set in the horizontal direction calculated by the pixels of the first frame of image is The K-means clustering algorithm is used to perform cluster analysis on the gradient in the horizontal direction:

In the formula: _E =2 represents other areas; represents the mean horizontal gradient; l is the number of image pixels.

Let the gradient set in the vertical direction calculated by the pixels of the first frame of image be The K-means clustering algorithm is used to perform cluster analysis on the gradient in the vertical direction:

In the formula: E _y is the calculation result of the sum of squares of classification in the vertical direction. When its value is the smallest, it is the best clustering; h is the type of clustering. In the present invention, h is 2, and h=1 represents the sum of fasteners. Sleeper area, h=2 represents other areas; Represents the vertical gradient mean; l is the number of image pixels.

In the present invention, it is assumed that the detected track and fastener areas are recorded as Ω = (Ω ₁ , Ω ₂ ,..., Ω _q ), and q is the number of determined track and fastener areas. The pixel grayscale extreme points in different areas are used as virtual feature points of the area. For the area Ω ₁ , the extreme value of pixel gray level in this area is expressed as:

In the formula: is the pixel coordinate of the point with the largest grayscale value in the Ω ₁ area. In the present invention, this point is used as the virtual feature point in the Ω ₁ area,/> The gray value of the virtual feature point, I (x, y) is the gray value at the pixel (x, y) position, and n ² is the number of pixels in the Ω ₁ area.

The grayscale extreme points in the track and fastener areas at different positions are detected to obtain virtual feature points at different positions.

Further, the step 4 is specifically, for the virtual feature point (k _x , _ky ), obtain the optical flow equation of the image at different scales based on the short-term brightness constant theory and the spatial consistency assumption. Select the virtual feature point neighborhood window size m×m. According to the principle of pixel motion consistency in the neighborhood, m×m pixels in the virtual feature point neighborhood satisfy:

In the formula: u, v are the optical flows of virtual feature points (k _x , k _y ) in the horizontal and vertical directions, (k _x -m, k _y -m),..., (k _x +m, k _y +m) are respectively the pixel coordinates in the neighborhood of the virtual feature point (k _x , k _y ), and I _y (k _x +m, k _y +m) and I _y (k _x +m, k _y +m) respectively represent The gradient of the grayscale of the pixels in the virtual feature point neighborhood in the x and y directions, I _t (k _x +m, k _y +m) represents the derivative of the grayscale of the pixels in the virtual feature point neighborhood with respect to time t.

Considering that the above equation is a statically determinate equation, in the present invention, the least squares method is used to solve the equation, and the optical flow information of the virtual feature point at any scale can be obtained.

Further, the step 5 is specifically to fuse the optical flow information of the virtual feature points at different scales according to the scale relationship between the images to obtain the virtual feature point optical flow. The optical flow of virtual feature points is used to calculate the vibration of the track and fasteners in the pixel coordinate system.

Assume that the optical flow of virtual feature points in different frames is {u _k ,v _k |k=1,2,3,...,K} in the horizontal and vertical directions respectively, the unit is pixel/frame, and K is the video The total number of frames, the movement of the structure can be obtained through the optical flow of virtual feature points:

In the formula: m _x and m _y are the movements of virtual feature points in the horizontal and vertical directions respectively, and f is the shooting frame rate.

In the present invention, a grid calibration plate is used to calibrate the camera, the scale factor is obtained through the relationship between the grid size and its size in the pixel coordinate system, and the time domain vibration of the track and fasteners in the physical coordinate system is calculated.

Let the calibrated grayscale image be I(x,y), and the grayscale gradients of the image in the horizontal and vertical directions are:

In the formula: is the convolution operation, H _x and H _y are the gradient operators in the x direction and y direction respectively.

The gradient amplitude is:

By comparing the gradient amplitude, the size of the grid in the pixel coordinate system can be determined, which is recorded as J and the unit is pixel (pixel). The actual size of the grid is R and the unit is millimeter (mm). Then the scale factor SF is:

The time domain vibration of the orbit is:

In the formula: S _x and S _y are the time domain vibrations of the track and fastener in the horizontal and vertical directions respectively under the physical coordinate system.

Further, the step 6 is specifically to perform FFT decomposition on the track time domain vibration signal to obtain the track natural vibration frequency:

It can be judged in real time whether the track is loosened by the change of natural vibration frequency.

Compared with existing track looseness detection methods, the beneficial effects of the present invention are:

1) The measurement efficiency is high and all tracks within the field of view can be monitored simultaneously;

2) No marking is required. Virtual feature points are used to replace the original manual marking, which has a wider scope of application.

Description of the drawings

Figure 1 is a schematic diagram of the detection device of the rail fastener monitoring method in the present invention.

Figure 2 is a flow chart of monitoring rail fasteners in the present invention.

Figure 3 is a schematic diagram of the principle of track and fastener area identification based on pixel gradient and clustering algorithms.

Figure 4 is a schematic diagram of the detection of virtual feature points.

Figure 5 is a schematic diagram of changes in pixel position and grayscale matrix over a period of time.

Figure 6 is a schematic diagram of the calculation process of the scale factor.

Figure 7 is a schematic diagram of the calculation process of fastener time domain vibration.

Detailed ways

The embodiments of the present invention will be described in detail below with reference to the drawings and examples.

As shown in Figure 1, the present invention uses a camera-based non-contact real-time detection method for train track looseness. It uses a high-speed camera to obtain a video of the vibration of the track when a train passes or is artificially stimulated. Taking advantage of the staggered arrangement characteristics of the train track and sleepers, it is proposed that A virtual feature point detection method based on image pixel gray gradient and feature clustering algorithm is proposed. The optical flow algorithm is used to calculate the optical flow of the virtual feature points, and then the time domain vibration of the track is measured. The natural vibration of the track is obtained through FFT decomposition. frequency, and then determine whether the track is loosened in real time through changes in the natural vibration frequency. The present invention will be further described below in conjunction with the accompanying drawings.

Step 1: As shown in Figure 2, use a high-speed camera to video record the vibration of the track fastener.

Step 2: In order to improve the screening efficiency of virtual feature points, perform grayscale processing on the color image obtained by the camera frame by frame to obtain a grayscale image:

I(x,y)＝0.299*R(x,y)+0.579*G(x,y)+0.114*B(x,y) (1)

In the formula: I(x,y) is the gray value of pixel (x,y); R(x,y), G(x,y), B(x,y) are the gray value of pixel (x,y) respectively. Pixel values of three channels.

Step 3: As shown in Figure 3, in the present invention, the gradient set in the horizontal direction calculated by the pixels of the first frame of the image is The K-means clustering algorithm is used to perform cluster analysis on the gradient in the horizontal direction:

In the formula: _E represents the mean horizontal gradient, and l is the number of image pixels.

Let the gradient set in the vertical direction calculated by the pixels of the first frame of the image be The K-means clustering algorithm is used to perform cluster analysis on the gradient in the vertical direction:

In the formula: E _y is the calculation result of the sum of squares of classification in the vertical direction, h is the type of clustering, in the present invention, h is 2, h=1 represents the fastener and sleeper area, h=2 represents other areas, represents the vertical gradient mean, and l is the number of image pixels.

In the present invention, as shown in Figure 4, it is assumed that the detected track and fastener areas are recorded as Ω = (Ω ₁ , Ω ₂ ,..., Ω _q ), and q is the number of determined track and fastener areas. , in the present invention, the extreme value points of pixel grayscale in different areas are used as the virtual feature points of the area. For the area Ω ₁ , the extreme value of pixel gray level in this area is expressed as:

In the formula: is the pixel coordinate of the point with the largest grayscale value in the Ω ₁ area. In the present invention, this point is used as the virtual feature point in the Ω ₁ area,/> The gray value of the virtual feature point, I (x, y) is the gray value at the pixel (x, y) position, and n ² is the number of pixels in the Ω ₁ area.

The grayscale extreme points in the track and fastener areas at different positions are detected to obtain virtual feature points at different positions.

Step 4: As shown in Figure 5, for the virtual feature points (k _x , k _y ), the optical flow equations of images at different scales are obtained based on the short-term brightness constant theory and the spatial consistency assumption. Select the virtual feature point neighborhood window size m×m. According to the principle of pixel motion consistency in the neighborhood, m×m pixels in the virtual feature point neighborhood satisfy:

In the formula: u, v are the optical flows of virtual feature points (k _x , k _y ) in the horizontal and vertical directions, (k _x -m, k _y -m),..., (k _x +m, k _y +m) are respectively the pixel coordinates in the neighborhood of the virtual feature point (k _x , k _y ), and I _y (k _x +m, k _y +m) and I _y (k _x +m, k _y +m) respectively represent The gradient of the grayscale of the pixels in the virtual feature point neighborhood in the x and y directions, I _t (k _x +m, k _y +m) represents the derivative of the grayscale of the pixels in the virtual feature point neighborhood with respect to time t.

Considering that the above equation is a statically determinate equation, in the present invention, the least squares method is used to solve the equation, and the optical flow information of the virtual feature point at any scale can be obtained.

Step 5: According to the scale relationship between images, fuse the optical flow information of virtual feature points at different scales to obtain the virtual feature point optical flow. The optical flow of virtual feature points is used to calculate the vibration of the track and fasteners in the pixel coordinate system.

Assume that the optical flow of virtual feature points in different frames is {u _k ,v _k |k=1,2,3,...,K} in the horizontal and vertical directions respectively, the unit is pixel/frame, and K is the video The total number of frames, the movement of the structure can be obtained through the optical flow of virtual feature points:

In the formula: m _x and m _y are the movements of virtual feature points in the horizontal and vertical directions respectively, and f is the shooting frame rate.

In the present invention, a grid calibration plate is used to calibrate the camera, the scale factor is obtained through the relationship between the grid size and its size in the pixel coordinate system, and the time domain vibration of the track and fasteners in the physical coordinate system is calculated.

Referring to Figure 6 and Figure 7, let the calibrated grayscale image be I(x,y), and the grayscale gradient of the image in the horizontal and vertical directions:

In the formula: is the convolution operation, H _x and H _y are the gradient operators in the x direction and y direction respectively.

The gradient amplitude is:

By comparing the gradient amplitude, the size of the grid in the pixel coordinate system can be determined, which is recorded as J and the unit is pixel (pixel). The actual size of the grid is R and the unit is millimeter (mm). Then the scale factor SF is:

The time domain vibration of the orbit is:

In the formula: S _x and S _y are the time domain vibrations of the track and fastener in the horizontal and vertical directions respectively under the physical coordinate system.

Step 6: Perform FFT decomposition of the orbit time domain vibration signal to obtain the orbit’s natural vibration frequency:

It can be judged in real time whether the track is loosened by the change of natural vibration frequency.

Claims (5)

1. A camera-based non-contact real-time detection method of train track looseness, which is characterized by including the following steps: Step 1), use a high-speed camera to video record the vibration of the train track; Step 2), perform grayscale processing on the recorded video frame by frame; Step 3), calculate the gray gradient of the pixels in the first frame of the image, use the K-means clustering algorithm to cluster the gray gradient, determine the track and fastener areas, and use the pixel gray extreme values in the area as the virtual value of the area. Feature points; Step 4), build an image multi-scale pyramid through image multi-scale decomposition technology, establish optical flow equations for images at different scales based on the short-term brightness constant theory, and use the least squares algorithm to calculate the optical flow of virtual feature points on images at different scales; Step 5), use the scale relationship between pyramid images to fuse the optical flow of virtual feature points on images of different scales to obtain the calculation results of the optical flow of virtual feature points. Combined with image calibration technology, obtain the time domain vibration signals of the track and fasteners. ; Step 6): Conduct frequency domain analysis on the time domain vibration signals of the track and fasteners to obtain the natural frequency of the track, and detect track looseness through the natural frequency change characteristics; Among them, in step 3), it is assumed that the set of gray gradients in the horizontal direction calculated by the pixels of the first frame of the image is The K-means clustering algorithm is used to analyze the gray gradient clustering in the horizontal direction: In the formula: _E represents the mean horizontal gradient; l is the number of image pixels; Let the gray gradient set in the vertical direction calculated by the pixels of the first frame of image be The K-means clustering algorithm is used to perform cluster analysis on the gray gradient in the vertical direction: In the formula: E _y is the calculation result of the sum of squares of classification in the vertical direction. When its value is the smallest, it is the best cluster; h is the type of cluster, h=1 represents the fastener and sleeper area, h=2 represents other area; Represents the vertical gradient mean; l is the number of image pixels; The detected track and fastener areas are recorded as Ω = (Ω ₁ , Ω ₂ ,..., Ω _q ), q is the number of determined track and fastener areas, and the pixel grayscale extreme points in each area are used. As the virtual feature point of this area, for the area Ω ₁ , the extreme value of the pixel gray level in this area is expressed as: In the formula: is the pixel coordinate of the point with the largest grayscale value in the Ω ₁ area, and this point is regarded as the virtual feature point in the Ω ₁ area;/> The gray value of the virtual feature point; I (x, y) is the gray value at the pixel (x, y) position; n ² is the number of pixels in the Ω ₁ area; The grayscale extreme points in the track and fastener areas at different positions are detected to obtain virtual feature points at different positions. 2. The camera-based non-contact real-time detection method of train track looseness according to claim 1, characterized in that in step 4), for the virtual feature point (k _x , _ky ), based on the short-term brightness constant theory and The spatial consistency assumption is used to obtain the optical flow equation of images of different scales. The size of the virtual feature point neighborhood window is selected to be m×m. According to the consistency principle of pixel motion in the neighborhood, m×m pixels in the virtual feature point neighborhood satisfy the superstatically definite equation. : In the formula: u, v are the optical flow of the virtual feature point (k _x , k _y ) in the horizontal and vertical directions; (k _x -m, k _y -m),..., (k _x +m, k _y +m) are respectively the pixel coordinates in the neighborhood of the virtual feature point (k _x , k _y ); I _y (k _x +m, k _y +m) and I _y (k _x +m, k _y +m) respectively represent The gradient of the grayscale of pixels in the neighborhood of virtual feature points in the x and y directions; I _t (k _x +m, k _y +m) represents the derivative of the grayscale of pixels in the neighborhood of virtual feature points with respect to time t The least squares method is used to solve the equation and obtain the optical flow of virtual feature points on different scales. 3. The camera-based non-contact real-time detection method of train track looseness according to claim 1, characterized in that in step 5), the optical flow of virtual feature points in different frames is assumed to be { in the horizontal and vertical directions respectively. u _k ,v _k |k＝1,2,3,...,K}, the unit is pixel/frame, K is the total number of frames of the video, and the movement of the structure is obtained through the optical flow of virtual feature points: In the formula: m _x and m _y are the movements of virtual feature points in the horizontal and vertical directions respectively, and f is the shooting frame rate; A grid calibration plate is used to calibrate the camera. The scale factor is obtained through the relationship between the grid size and its size in the pixel coordinate system, and the time domain vibration of the track and fasteners in the physical coordinate system is calculated. 4. The camera-based non-contact real-time detection method of train track looseness according to claim 3, characterized in that when the camera is calibrated, the calibrated grayscale image is I(x, y), and the image is horizontally and vertically. The gray gradients in the straight direction are: In the formula: is the convolution operation, H _x and H _y are the gradient operators in the x direction and y direction respectively; The gradient amplitude is: By comparing the gradient amplitude, the size of the grid in the pixel coordinate system can be determined, which is recorded as J and the unit is pixel (pixel). The actual size of the grid is R and the unit is millimeter (mm). Then the scale factor SF is: The time domain vibration of the orbit is: In the formula: S _x and S _y are the time domain vibrations of the track and fastener in the horizontal and vertical directions respectively under the physical coordinate system. 5. The imaging-based non-contact real-time detection method of train track looseness according to claim 1, characterized in that in step 6), the track time domain vibration signal is subjected to FFT decomposition to obtain the track natural vibration frequency: It can be judged in real time whether the track is loosened by the change of natural vibration frequency.