CN112991347B - Three-dimensional-based train bolt looseness detection method - Google Patents

Abstract

The invention discloses a three-dimensional-based train bolt looseness detection method, which specifically comprises the following steps: acquiring a train bottom two-dimensional image and a depth image to be overhauled by adopting an industrial three-dimensional line scanning camera; for the two-dimensional graph, a two-step ACF target detection algorithm is adopted to position the bolt position; mapping a bolt target region in the two-dimensional map to an acquired depth map, and calculating the thickness of a nut based on a RANSAC + SRG region segmentation algorithm; and (5) making a difference with the historical data and the standard data, comparing a fault threshold value, and judging the bolt state. The method combines the characteristics of three-dimensional data and two-dimensional data, utilizes the characteristics of the three-dimensional data and the two-dimensional data to play different roles, realizes the functions of stably realizing bolt positioning, segmentation, calculation and looseness identification under the complex environment of the whole subway train bottom, and judges looseness by calculating related objective real data; not only can refine the bolt looseness degree, can also solve the not hard up degree of mark line unable detection bolt rotation one round after.

Description

Three-dimensional-based train bolt looseness detection method

Technical Field

The invention belongs to the field of train bolt fault detection, and particularly relates to a three-dimensional-based train bolt looseness detection method.

Background

The rail transit has extremely important significance in the development of China, and along with the rapid development of economy of China, the technical level and the construction scale of the railway track of China make great progress. A large number of trains and subways bring traffic safety problems, so that the establishment of a train safety guarantee system becomes an essential link in the construction of a railway system. The traditional manual detection is still the most important mode for train maintenance. The manual detection is not only low in efficiency, but also easy to detect some hidden details. With the increasing maturity of technologies such as machine learning, deep learning, computer vision, etc., artificial intelligence appears in more fields to meet the needs of human beings. The intelligent detection mode combining the train safety and the artificial intelligence can replace the artificial detection to a certain extent.

Computer vision is mainly applied to the fields of video monitoring, face recognition, intelligent driving and the like at present. There are three main directions in which research can be applied in fault detection: (1) the template matching method judges whether the abnormity exists or not according to the similarity between the picture to be detected and the standard template, has high requirements on all aspects of the image, and causes great influence on the result by a plurality of factors. (2) The machine learning method based on statistics is to extract the characteristics of the samples and then classify the samples according to the characteristic distribution. The key point is the algorithm of feature extraction, and the generalization is poor because the reliability can be ensured only by developing different algorithms for different target tasks. (3) Based on the defect and anomaly detection of deep learning, in order to obtain a high-precision model, the method needs to collect a large amount of data for each target to train the model. The workload is large and has a certain randomness.

At present, bolt abnormity detection schemes mainly depend on two-dimensional pictures. For the fault detection of bolt falling, damage and the like, a detection model is mainly trained by an algorithm based on deep learning, and the method can achieve good effect under the condition that positive and negative samples are sufficient, but the defects are already explained above. The existing main method for detecting the bolt looseness manually marks a reference line on the surface of the bolt, and judges whether the bolt looseness occurs or not through two-dimensional visual detection of the state of the reference line. One such method requires manual drawing of lines, which is time consuming and labor intensive. Secondly, the detection efficiency is low due to uneven quality of the manual drawing lines. The marking line may be affected by factors such as dirt, light, shooting angle and the like in actual operation, so that detection errors are caused. There is also a serious problem in that a failure may not be detected from the surface after loosening for more than one week, and the degree of loosening cannot be refined. The distance from an object to the shooting equipment is not recorded in the ordinary image, and only the object which is far away from the object and the object which is close to the object can be obtained through semantics, so that accurate distance data is not obtained. While three-dimensional data is more reliable than two dimensions, it is currently less used in computer vision research applications. Firstly, because the three-dimensional algorithm is not mature enough, secondly, the price of high-precision three-dimensional equipment is expensive, thirdly, the engineering application is difficult only by depending on a depth map or point cloud, for example, the target detection can cause the calculation amount to be increased sharply, and the effect is not good.

From the background of the above complaints, four key points that must be solved for bolt loosening detection using computer vision can be clearly obtained: (1) the algorithm model can effectively inhibit the interference of environmental factors such as illumination, stains and the like, has strong robustness and overcomes the limitation of common pictures. (2) The algorithm model can still train a satisfactory model under the condition of few samples, and is favorable for convenience in practical application. (3) The algorithm model has to have the characteristics of high precision, high stability and generalization, so that the mode of manual overhaul can be replaced, and the train running safety is also ensured, otherwise, the intelligent detection has no practical significance. (4) The method has the advantages that the efficiency is high, the train maintenance time can only be the operation idle period, the whole train needs to be checked and maintained in the idle period, the detection efficiency is high, and the algorithm model is required to accurately complete the detection of the item point in a short time.

Disclosure of Invention

In order to realize the functions of stably realizing bolt positioning, segmentation, calculation and looseness identification in the complex environment of the whole subway train bottom, the looseness is judged by calculating relevant objective real data. Not only can refine the bolt looseness degree, but also can solve the difficult problem that the mark line can not detect the looseness degree of the bolt after rotating for one circle. The invention provides a three-dimensional-based train bolt looseness detection method.

The invention discloses a three-dimensional-based train bolt looseness detection method, which comprises the following steps of:

step 1: and acquiring a two-dimensional image and a depth image of the train bottom to be overhauled by adopting an industrial three-dimensional line scanning camera.

Step 2: for bolt positioning, a two-step ACF (aggregate channel feature) target detection algorithm is used to position the bolt position in a two-dimensional map.

S21: and loading a pre-trained first target detection model in the mat file, and positioning a preliminary region of interest containing a target bolt in the whole graph.

S22: and loading a second target detection model, cutting the primary region of interest from the full image, and positioning each bolt to be detected in the primary region of interest.

And step 3: nut thickness was calculated based on the area segmentation algorithm of RANSAC (random sample consensus) + SRG (area seed growth).

S31: and mapping the bolt target area in the two-dimensional map to a corresponding area in the acquired depth map.

S32: the depth map is converted into point cloud data, a RANSAC algorithm is used for detecting a global optimal plane in the area, the influence of noise can be eliminated, and all local interior points which accord with the plane parameters are output.

S33: and selecting one inner point in the local inner point set to be detected as an initial seed point of the region growing point, obtaining a binary image by using a region growing algorithm depending on a threshold value, and obtaining a white target region as a lower surface region of the bolt.

S34: a small target region in the binary image is deleted, then the binary image is inverted so that the upper surface region is the target region and the centroid of this region is calculated.

S35: and detecting the plane again in the point cloud of the upper surface obtained by segmentation, and outputting the local point of the upper surface plane.

S36: calculating the average value of the depth values of the local inner points of the upper surface, indexing the distance between the local inner points of the lower surface and the mass center of the upper surface, selecting a part of pixels with the closest distance to calculate the average value of the depth values, wherein the difference value between the upper surface and the lower surface is the detected thickness of the nuth：

Wherein the content of the first and second substances,p _{on the upper part} 、p _{Lower part}The local point sets of the upper and lower optimal planes are detected for RANSAC respectively,i=1,2,…,nfor the selected upper surface pixel points,j=1,2,…,mis the selected lower surface pixel point.n、mThe total number of pixels on the upper surface and the lower surface respectively.

And 4, step 4: and detecting the fault of the bolt.

S41: calculating the thickness of the nut, historical data and standard datahComparing to obtain a difference value, comparing a fault threshold value, and judging the bolt state;

wherein, T_{Loosening of the screw}Denotes the upper threshold, T_{Damage of}Representing a lower threshold.

S42: calculating the loosening degree of the bolt;

wherein the content of the first and second substances,εstandard values for thickness variation for one revolution of the bolt are shown.

The beneficial technical effects of the invention are as follows:

1. the invention utilizes an ACF target detection algorithm to realize accurate positioning to the position of a small bolt in the train bottom, and the main scheme is to design a combined model of two ACFs to carry out preliminary positioning and accurate positioning respectively. The method improves the precision and the detection speed relative to one-time positioning. The difficult problem of positioning small parts at the bottom of a complex train is solved.

2. According to the characteristics of the depth map, the invention designs a region segmentation algorithm combining RANSAC + SRG in the scene, and solves the problem of selecting initial seeds by a region growing algorithm. And a very good segmentation effect is obtained. The method for detecting the plane in the point cloud by RANSAC effectively eliminates noise and local points, and can also eliminate the influence caused by the acquisition angle, so that the algorithm has high robustness.

3. The invention combines the three-dimensional data and the two-dimensional data to realize the corresponding function by combining the characteristics of the three-dimensional data and the two-dimensional data and the mutual mapping relation, thereby effectively eliminating the problems of the self limitation of the two-dimensional data and the external interference such as: the influence brought by illumination, rainwater, stains and the like solves the defect of the template matching method based on the marking line. The designed thickness calculation mode ensures the accuracy of detection. The intermediate process and the result of the detection are clear and visual, the loosening degree of each bolt can be refined, and the loosening angle of each bolt can be returned. The database of historical data and standard data established for each bolt enables the detection result to be more reliable and avoids the situations of misinformation and the like. This scheme provides powerful guarantee for the train safety operation.

Drawings

Fig. 1 is a flow chart of a three-dimensional-based train bolt looseness detection method of the invention.

Detailed Description

The invention is described in further detail below with reference to the figures and the detailed description.

The invention discloses a three-dimensional-based train bolt looseness detection method which is shown in figure 1 and comprises the following steps:

step 1: and acquiring a train bottom two-dimensional image and a depth image to be overhauled by adopting an industrial three-dimensional line scanning camera.

The principle of three-dimensional data acquisition is laser triangulation, and the acquired data is a depth map and a two-dimensional map. The two-dimensional image and the depth image are two types of data generated simultaneously, the two types of data have the same size, and pixel points are mapped one by one.

Step 2: and for the two-dimensional graph, positioning the bolt position by adopting a two-step ACF target detection algorithm.

In order to realize quick and effective detection, the input of target detection is a two-dimensional graph with rich texture and semantic information, and the two requirements can be considered when the two-dimensional graph is used for positioning a target bolt. The target detection uses an ACF algorithm based on traditional machine learning, and has high training speed and simple structure. The target detection algorithm based on multi-channel feature extraction constructs a feature channel pyramid by using the image features of the ACF, and extracts the ACF features of the image through a sliding window. It is difficult to locate a 20 x 20 pixel bolt in a 1536 x 7500 pixel underbody map. The direct one-step localization effect is certainly not sufficient enough, and the detection success rate becomes low due to the fact that the target cannot be detected and the target is detected in error. In order to solve the problem, the target detection is divided into two steps by adopting an ACF algorithm twice to complete, a primary region of interest containing a target bolt is positioned in the first step, and then the accurate position of the bolt to be detected is positioned by performing the second target detection in the region. The scheme has a much higher success rate than one-time positioning, and the detection speed is faster. This is because the size difference between the two positioning steps of the original image and the target is reduced, and the number of sliding windows for extracting features each time is reduced.

And step 3: and calculating the thickness of the nut based on a RANSAC + SRG region segmentation algorithm.

The three-dimensional data can reflect the objective shape attribute and the position relation of the object, and the target area based on the two-dimensional target detection is mapped into the depth map, so that the effect of area segmentation is facilitated. And detecting the global optimal plane of the region by using RANSAC (random sample consensus) in the point cloud data. RANSAC is a random sample parameter estimation method, can calculate the global optimal mathematical model parameters of data in a group of sample data sets containing noise to obtain an algorithm of effective sample data (local interior points), and is commonly used for detecting straight lines and planes in three-dimensional point cloud. Selecting a point from a detected plane local point set as a seed of a region growing algorithm, wherein the region growing algorithm can usually divide a communicated region with the same characteristics, similar adjacent pixels or regions are combined with the seed point from the seed point, and the lower surface of the bolt and other pixel points can be divided to generate a binary image. And then, obtaining the area of the upper surface of the bolt by processing methods such as deleting small objects, carrying out binary inversion, intercepting the smallest surrounding frame and the like. The coordinates of the centroid of the upper surface are calculated as the seed for the second region growing. The average value of the depth values of the pixel points of the second growth result is calculated, and the distance between the lower surface pixel point and the upper surface centroid is searched for the nearest part, so that the pixel points calculated each time are guaranteed to be consistent as much as possible.

Wherein the content of the first and second substances,p _{on the upper part} 、p _{Lower part}The local point sets of the upper and lower optimal planes are detected for RANSAC respectively,i=1,2,…,nfor the selected upper surface pixel points,j=1,2,…,mis the selected lower surface pixel point.n、mThe total number of pixels on the upper surface and the lower surface respectively.

And 4, step 4: and detecting the fault of the bolt.

Calculating the thickness of the nut, historical data and standard datah' make a comparison to a difference. Setting a suitable threshold value, exceeding the upper threshold value T_{Loosening of the screw}That is, the bolt is loosened and is lower than the lower threshold value T_{Damage of}Representing possible damage.

Wherein, T_{Loosening of the screw}Denotes the upper threshold, T_{Damage of}Representing a lower threshold.

The loosening degree delta of the bolt can be refined through the standard value of the thickness change of each kind of bolt rotating for one circleθ。

Wherein the content of the first and second substances,εstandard values for thickness variation for one revolution of the bolt are shown.

If the data is normal, the detection data is uploaded to a database as historical data.

In the field maintenance of the train, the mode of detecting looseness by using the marking line does not need to waste time and labor, the template matching method is very easy to be illuminated, and the detection error is caused by the change of a shooting angle and the like. And if the bolt loosens and exceeds the problem that the detection can not refine the loosening degree after 360 degrees, certain potential safety hazard can be caused. The scheme that utilizes two-dimentional target detection to combine to cut apart based on the depth map can solve these two problems well, can be applicable to the most bolt detection in vehicle bottom, has good generalization nature.

In a relatively stable similar environment, it takes only a few minutes to train the ACF target detection model for the 7500 × 1536 pixel picture input. The success rate of target detection can be more than 95% only by 100 training sets. The two-step ACF detection model can reach the speed of 0.5 s under the running of the CPU, the whole algorithm finishes about 1.5s required for detecting the looseness of bolts (a plurality of bolts can be arranged) in one region, and the real-time detection of the bolt looseness can be realized.

The RANSAC + SRG segmentation algorithm is designed, and a RANSAC detection plane is utilized to obtain consistent pixel points from random samples. The problem of automatic seed selection in the region growth is solved, and the method has good performance in the application under the scene. RANSAC can effectively eliminate noise and local points, detect out the model parameters which best conform to the plane, avoid the influence of noise data and burrs on the surface of the bolt, inhibit the influence caused by the inclination of the acquisition angle of a camera, and greatly improve the robustness of loosening detection.

Compared with the unexplainable property of the deep learning intermediate process, the fault detection based on the three-dimensional data can obtain the detection data reflecting the objective state of the object, the loosening degree of bolts of various types is refined, and the fault judgment can be more accurate and reliable. In the bolt test of three part positions, the average error reaches within 0.2mm by means of a continuous train passing data test set of the same train at different moments, the bolt detection error of M10 can be controlled within 15 degrees of looseness, the bolt detection error of M24 can be controlled within 30 degrees, and the requirement of actual detection standards can be met.

Claims (1)

1. A three-dimensional-based train bolt looseness detection method is characterized by comprising the following steps:

step 1: acquiring a train bottom two-dimensional image and a depth image to be overhauled by adopting an industrial three-dimensional line scanning camera;

step 2: for the two-dimensional graph, positioning the position of a bolt by adopting a two-step polymerization channel characteristic ACF target detection algorithm;

s21: firstly, positioning a primary region of interest containing a target bolt, and marking a label of a primary positioning frame on a two-dimensional map by using an MATLAB software label tool to obtain a primary target detection training set;

s22: cutting the target area marked for the first time from the whole image, and marking the bolt to be detected in the primary area of interest to obtain a second target detection training set;

s23: based on a prepared data training set, positioning the accurate position of the bolt to be detected by adopting a two-step polymerization channel characteristic ACF target detection algorithm;

and step 3: calculating the difference value between the upper surface and the lower surface based on a region segmentation algorithm of random sample consensus RANSAC + region seed growth SRG;

s31: mapping a bolt target area in the two-dimensional map to the acquired depth map;

s32: detecting an optimal plane in the region by using an RANSAC algorithm, and outputting all pixel points conforming to the plane;

s33: selecting an inner point belonging to a fitting plane as an initial seed point of a region growing point, obtaining a binary image by using a region growing algorithm depending on a threshold value, wherein a white target region is a lower surface region of the bolt;

s34: deleting a small target area in the binary image, and then reversing the binary image to enable the upper surface area to be the target area;

s35: calculating the mass center of the upper surface area at the moment as a seed for the second time of area growth, and setting a threshold value to obtain pixel points of the upper surface;

s36: calculating the average value of depth values of pixels on the upper surface, indexing the distance between the pixels on the lower surface and the centroid of the upper surface, selecting a part of pixels with the closest distance to calculate the average value of depth values, and calculating the difference value between the upper surface and the lower surfaceh；

Wherein the content of the first and second substances,p _{on the upper part} 、p _{Lower part}The local point sets of the upper and lower optimal planes are detected for RANSAC respectively,i=1,2,…,nfor the selected upper surface pixel points,j=1,2,…,mthe selected lower surface pixel points;n、mrespectively the total number of the upper and lower surface pixel points;

and 4, step 4: detecting a bolt fault;

s41: calculating the difference between the upper surface and the lower surfacehWith historical and standard datahComparing to obtain a difference value, comparing a fault threshold value, and judging the bolt state;

wherein, T_{Loosening of the screw}Denotes the upper threshold, T_{Damage of}Represents a lower threshold;

s42: calculating the loosening degree of the bolt

；

Wherein the content of the first and second substances,εstandard values for thickness variation for one revolution of the bolt are shown.

Images