CN114067296A - Method and device for identifying surface defects of steel rail - Google Patents
Method and device for identifying surface defects of steel rail Download PDFInfo
- Publication number
- CN114067296A CN114067296A CN202111334140.9A CN202111334140A CN114067296A CN 114067296 A CN114067296 A CN 114067296A CN 202111334140 A CN202111334140 A CN 202111334140A CN 114067296 A CN114067296 A CN 114067296A
- Authority
- CN
- China
- Prior art keywords
- area
- steel rail
- light band
- image
- defect
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
- G06F18/24155—Bayesian classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/24323—Tree-organised classifiers
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention discloses a method and a device for identifying surface defects of steel rails, wherein the method comprises the following steps: carrying out continuous defect identification on the surface image of the steel rail by using a pre-trained continuous defect detection model, and identifying the position of the steel rail and a continuous defect image; aiming at each steel rail surface image without continuous defects, carrying out light band extraction on a steel rail area according to the position of the steel rail to generate a light band binary image; calculating the area light band index of the steel rail area according to the light band binary image; inputting the zone light band index into a classifier, and judging whether the surface of the steel rail has suspected defects; inputting the surface image of the steel rail with the surface suspected defect into a pre-trained local defect detection model for local defect identification to obtain a local defect image, a local defect type and a local defect position; and outputting the continuous defect image, the local defect type and the corresponding local defect position. The method can comprehensively and accurately identify the surface defects of the steel rail in the steel rail image.
Description
Technical Field
The invention relates to the technical field of rail detection and image recognition, in particular to a method and a device for recognizing surface defects of steel rails.
Background
This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
The steel rail is a main component of the whole railway track, is one of railway transportation infrastructures, and is used for guiding the advancing direction of wheels of a train, directly bearing the acting force of the wheels of the train and transmitting the acting force to a sleeper. The state of the steel rail directly influences the running safety of the train. Because the steel rail is exposed in the field for a long time, various damages are inevitably generated on the steel rail under the influence of various natural conditions such as wind, frost, rain, snow, high temperature and low temperature and the pressure caused by the running of a train, and various potential safety hazards are further caused.
The major defects of the rail surface include: scratch of rail surface, stripping and block falling, crack, crushing of rail head, wave abrasion of tread, and the like. The influence of different defects on the train operation safety is different, and the detection of the surface defects of the steel rail is one of the most important links in the detection and maintenance of the track safety facilities.
When detecting a rail image, a camera is usually mounted on the bottom of a detection vehicle, and the camera continuously photographs and images the surface of the rail when the detection vehicle is running, so as to obtain a rail surface image. And then, identifying the surface defects of the steel rail by a steel rail surface defect image identification technology. Rail surface defect detection identification has its particular difficulties and challenges compared to general identification tasks. The railway is subjected to complex environment and various natural conditions such as illumination, wind, frost, rain and snow, so that the quality of the acquired on-site pictures is different, and the difficulty of defect identification is increased to a great extent. In addition, the mode of the surface defects of the steel rail is complex, for example, some defects are fine and occupy a small part compared with the steel rail, and the detection and identification difficulty is increased to a certain extent.
In the prior art, a method for identifying the surface defects of the steel rail through deep learning, a deep neural network, a deep forest model and the like is provided.
The method for identifying the surface defects of the steel rails through deep learning comprises the steps of firstly manually collecting a large number of images of the surface defects of the steel rails and carrying out data annotation, training a steel rail surface defect detection model constructed based on a deep learning target positioning algorithm based on annotated data, and then identifying the surface defects of the steel rails in the steel rail images by using the trained model. However, this technique requires a large number of rail defect image samples, the detectable defects must be defects of the type that have been detected in the samples, and the rail surface defects in the case of small samples and in the case of no samples are prone to missing.
The method comprises the steps of adopting a deep neural network and a deep forest model, respectively adopting the deep neural network and the deep forest model to model the surface scratch of the steel rail, preprocessing an image, and detecting the surface defect of the steel rail in the image through the trained model, wherein the method comprises the special treatment of a small sample problem. However, the surface defects of the rail have continuous defects and local defects. The technology can detect local defects through target detection, but as the forms and the characteristics of continuous defects are different from those of the local defects, the technology is easy to miss detection due to the fact that the length-width ratio of the target of the defects is too extreme when the continuous defects are detected. In addition, the defect types which do not appear in the sample are easy to miss detection.
Disclosure of Invention
The embodiment of the invention provides a steel rail surface defect identification method, which is used for comprehensively and accurately identifying steel rail surface defects in a steel rail image by using an image identification algorithm and comprises the following steps:
acquiring a shot rail surface image;
carrying out continuous defect identification on the surface image of the steel rail by using a pre-trained continuous defect detection model, and identifying the position of the steel rail and a continuous defect image, wherein the position of the steel rail is the pixel row position of the steel rail in the image;
aiming at each steel rail surface image without continuous defects, carrying out light band extraction on a steel rail area according to the position of the steel rail to generate a light band binary image;
calculating the area light band index of the steel rail area according to the light band binary image;
inputting the zone light band index into a classifier, and judging whether the surface of the steel rail has suspected defects;
inputting the surface image of the steel rail with the surface suspected defect into a pre-trained local defect detection model for local defect identification to obtain a local defect image, a local defect type and a local defect position;
and outputting the continuous defect image, the local defect type and the corresponding local defect position.
The embodiment of the invention also provides a steel rail surface defect identification device, which is used for comprehensively and accurately identifying the steel rail surface defects in the steel rail image by using an image identification algorithm, and comprises the following components:
the acquisition module is used for acquiring a shot rail surface image;
the continuous defect identification module is used for carrying out continuous defect identification on the surface image of the steel rail by utilizing a pre-trained continuous defect detection model, and identifying the position of the steel rail and a continuous defect image, wherein the position of the steel rail is the pixel row position of the steel rail in the image;
the light band extraction module is used for carrying out light band extraction on a steel rail area according to the position of the steel rail aiming at each steel rail surface image without continuous defects identified to generate a light band binary image;
the calculation module is used for calculating the area light band index of the steel rail area according to the light band binary image;
the suspected defect judging module is used for inputting the zone light band indexes into the classifier and judging whether the surface suspected defect exists on the steel rail or not;
the local defect identification module is used for inputting the surface image of the steel rail with the surface suspected defect into a local defect detection model trained in advance to carry out local defect identification so as to obtain a local defect image, a local defect type and a local defect position;
and the identification result output module is used for outputting the continuous defect image, the local defect type and the corresponding local defect position.
The embodiment of the invention also provides computer equipment which comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor executes the computer program to realize the steel rail surface defect identification method.
The embodiment of the invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program for executing the rail surface defect identification method.
In the embodiment of the invention, a method for identifying the surface defects of the continuous to local multistage steel rails is provided, the continuous defect identification is firstly carried out on the surface images of the steel rails, then the surface state of the steel rails is automatically evaluated through a classifier, the surface images of the steel rails with the surface defects possibly exist are identified, then the local defects are detected by utilizing a local defect detection model, the combination of deep learning and the traditional classification method is realized, further the comprehensive detection and the automatic detection of the global continuous defects and the local defects of the surface of the steel rails are realized, the detection rate and the accuracy of the defects are improved, and the working efficiency of a first-line detector is greatly improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts. In the drawings:
FIG. 1 is a flow chart of a method for identifying surface defects of a steel rail according to an embodiment of the present invention;
FIG. 2(a) is a schematic view of a steel rail with continuous wave grinding defects according to an embodiment of the present invention;
FIG. 2(b) is a schematic view of a rail having a localized gouging defect in accordance with an embodiment of the present invention;
FIG. 3(a) is a schematic illustration of an image annotation of a rail without continuous defects according to an embodiment of the present invention;
FIG. 3(b) is a schematic diagram illustrating image annotation of a steel rail with continuous defects according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a process for generating a binary image of an optical band according to an embodiment of the present invention;
FIG. 5 is a schematic diagram illustrating a sub-division of an optical band according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a prior art fast R-CNN framework;
fig. 7 is a block diagram of a regional candidate network RPN in the prior art;
FIG. 8 is a schematic diagram of a process for identifying global rail conditions using the Faster RCNN framework according to an embodiment of the present invention;
FIG. 9 is a schematic illustration of an image annotation of a locally defective rail according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of a process for identifying local defects in a rail using the Faster RCNN framework according to an embodiment of the present invention;
FIG. 11 is a schematic structural diagram of a rail surface defect identification device according to an embodiment of the present invention;
fig. 12 is a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.
The embodiment of the invention provides a method for identifying surface defects of a steel rail, which comprises the following steps of 101 to 107:
The acquisition source of the surface image of the steel rail is track inspection acquisition equipment. The track inspection acquisition equipment adopts a linear array camera to continuously shoot the track. The rail surface image collected by the rail inspection collecting equipment is a gray image shot perpendicular to the rail surface.
Defects on the rail surface can be divided into continuous defects and local defects according to different defect forms, wherein the continuous defects generally comprise corrugation, continuous punch damage, continuous abrasion and the like; local defects typically include local scratches, local chipping, and the like. Illustratively, fig. 2(a) shows an image of a rail defect with a continuous corrugation defect, and fig. 2(b) shows an image of a rail defect with a localized scratch.
And 102, carrying out continuous defect identification on the surface image of the steel rail by using a pre-trained continuous defect detection model, and identifying the position of the steel rail and the continuous defect image.
Wherein, the position of the steel rail is the pixel row-column position of the steel rail in the image. The image is composed of pixels, and the rail position is that the rail is located at the pixels of the rows and columns, for example, the pixels of the 1 st row to the 200 th row in the longitudinal direction and the pixels of the 80 th column to the 120 th column in the transverse direction.
Continuous defects refer to continuous defects in a single image having a defect length that exceeds 50% of the length of the rail in the image. The occupied area of the continuous defects is large, and the large part of the continuous defects can be identified by comprehensively identifying the rail surface images of the steel rails. And the comprehensive rail surface state identification is carried out by taking the whole section of steel rail in the image to be identified as a whole so as to carry out positioning and continuous defect identification on the steel rail. And the global rail surface state identification is realized by a deep learning target detection model.
Firstly, acquiring a first sample image comprising a steel rail image with continuous defects and a steel rail image without continuous defects; marking the positions of the complete steel rails in all the first sample images, and marking continuous defect marks in the steel rail images with continuous defects; and then, training the deep learning target detection model by using all the labeled first samples to obtain a continuous defect detection model.
For example, the marked rail images are shown in fig. 3(a) and 3(b), fig. 3(a) is a schematic diagram of marking the rail image without continuous defects, and fig. 3(b) is a schematic diagram of marking the rail image with continuous defects. In the schematic diagram, the position of a complete steel rail is marked by a black thick square frame, the character 'continuous defect steel rail' is used as a continuous defect identifier, and the character 'continuous defect-free steel rail' is also identified in a steel rail image without continuous defects, so that a deep learning target detection model in the training process can more clearly distinguish the two steel rail images.
The global rail surface state identification result comprises the position of the steel rail in the image and the category of the steel rail, the category of the steel rail is divided into a continuous defect-free steel rail and a continuous defect steel rail, and the shot steel rail surface image of the continuous defect is a continuous defect image.
And 103, aiming at each steel rail surface image without continuous defects, carrying out light band extraction on a steel rail area according to the position of the steel rail to generate a light band binary image.
The rail locations in the continuous defect images and all rail surface images can be identified by global rail surface state identification in step 102. And for the steel rail surface image without continuous defects identified, extracting light bands of the steel rail area in the image according to the position of the steel rail, specifically extracting the light bands in the image through an image segmentation algorithm, and generating a light band binary image. In the image segmentation algorithm binary image, the result of the rail region is 255, and the results of the other regions are 0. The image segmentation algorithm may employ a deep learning based semantic segmentation algorithm or directly binarize the image using static or dynamic thresholds.
For example, FIG. 4 shows a schematic diagram of a process for generating a binary map of an optical band. In fig. 4, the area 1 and the area 3 are identified by straight line boxes, and the area 2 and the area N are identified by dotted line boxes.
And 104, calculating the area light band index of the steel rail area according to the light band binary image.
In the embodiment of the invention, before the regional light band index of the steel rail region is calculated according to the light band binary image, the light band in the light band binary image is divided into a plurality of equal-length sub-regions according to a first preset length. Meanwhile, considering that the local defect is small, if the local defect is just divided into two small parts by the boundary line of the two sub-areas, each small part may be too small to be identified, so that the two adjacent sub-areas are overlapped, and thus the two sub-areas have an overlapped area with a second preset length, thereby avoiding the situation that the local defect is divided into two parts as much as possible. Illustratively, fig. 5 shows a schematic diagram of zone division of an optical band. Referring to fig. 5, there is a certain overlap area for each of the areas 1 and 2, the areas 2 and 3, and so on.
The first preset length and the second preset length may be set by a worker, and specific length values thereof are not limited herein.
After the sub-regions are divided, the region band indicators of the rail region are extracted based on the band region pixel statistic combination. Specifically, light band characteristics are extracted from the light band binary image, and the light band characteristics comprise the width of each line of light band in an area, the total line number of the area light band, the maximum value of the width of the area light band, the minimum value of the width of the area light band, the pixel value of each pixel in the area and the total column number of the area light band; and calculating the zone light band index of the steel rail zone according to the light band characteristics.
The zone band indicators include a zone mean width, a zone width standard deviation, a zone width absolute value extreme difference, a zone width relative extreme difference, a zone mean gradient, and a zone band area.
The light band index for each zone is calculated as follows:
first, average width of region:
in the above formula, μ represents the area average width; x is the number ofiRepresents the width of the ith row of light bands; n represents the total number of rows of zone bands.
II, standard deviation of area width:
in the above equation, σ represents a standard deviation of the region width.
And thirdly, absolute value difference of area width:
Δabs＝xmax-xmin
in the above formula,. DELTA.absRepresents absolute extreme difference; x is the number ofmaxRepresenting the maximum value of the zone light band width; x is the number ofminIndicating an area band width minimum.
Fourthly, relative extreme difference of area width:
in the above formula,. DELTA.relIndicating the relative extreme difference in zone width.
Fifthly, average gradient of area:
in the above formula, the first and second carbon atoms are,
represents the regional average gradient; p is a radical ofi,jThe pixel values of the ith row and the jth column of pixels are represented, and the values are 0 or 1; m represents the total number of columns of zone bands.
Sixth, area light band area:
in the above formula, S represents an area band area.
In each row of pixels of the band binary image, a pixel portion belonging to a band is defined as a band of one row, and in each column of pixels of the band binary image, a pixel portion belonging to a band is defined as a column of a band.
And 105, inputting the zone light band index into a classifier, and judging whether the surface of the steel rail has suspected defects.
The classifier is obtained by training a plurality of groups of regional light band indexes without local defects and a plurality of groups of regional light band indexes with local defects in advance.
And (3) forming the area light band indexes into a one-dimensional characteristic vector, then respectively inputting the area light band indexes of each sub-area and the area light band indexes of the complete steel rail area into a classifier, referring to fig. 5, namely respectively calculating N groups of area light bands from an area 1, an area 2, an area 3 to an area N, and 1 group of area light band indexes calculated from the complete steel rail area from the area 1 to the area N, and respectively inputting the total of N +1 groups of area light band indexes into the classifier to judge whether suspected defects exist on the surface of the steel rail.
In the embodiment of the invention, the area light band index obtained by calculating the complete steel rail area is input into the classifier to judge the surface suspected defect, and the main purpose is to further screen whether the continuous defect exists on the steel rail, so that the continuous defect which is not identified by the continuous defect detection model can be checked for leakage, and a more accurate continuous defect identification result can be obtained.
The classifier can be selected from a naive Bayes classifier, a Support Vector Machine (SVM), a random forest and other classifiers.
And 106, inputting the surface image of the steel rail with the surface suspected defect into a pre-trained local defect detection model for local defect identification to obtain a local defect image, a local defect type and a local defect position.
Local rail surface defect identification needs to be constructed. Firstly, acquiring a large number of second sample images, wherein the second sample images comprise steel rail images with local defects; then, marking a local defect position and a local defect type identifier in the second sample image, wherein the local defect type identifier indicates the specific type of the local defect, such as local scratch, local block dropping and the like; and then, training the deep learning target detection model by using the labeled second sample image to obtain a local defect detection model.
And inputting the surface image of the steel rail judged to have the surface suspected defect into a local defect detection model for local defect identification, and identifying to obtain the surface image of the steel rail with the local defect, the type of the local defect and the corresponding local defect position.
And step 107, outputting the continuous defect image, the local defect type and the corresponding local defect position.
In the embodiment of the invention, a method for identifying the surface defects of the continuous to local multistage steel rails is provided, the continuous defect identification is firstly carried out on the surface images of the steel rails, then the surface state of the steel rails is automatically evaluated through a classifier, the surface images of the steel rails with the surface defects possibly exist are identified, then the local defects are detected by utilizing a local defect detection model, the combination of deep learning and the traditional classification method is realized, further the comprehensive detection and the automatic detection of the global continuous defects and the local defects of the surface of the steel rails are realized, the detection rate and the accuracy of the defects are improved, and the working efficiency of a first-line detector is greatly improved.
In order to further understand the method for identifying the surface defect of the steel rail, the following will specifically describe the embodiment of the invention with reference to a specific example. In the following example, the rail surface defect identification method is divided into 4 stages, namely a global rail surface state identification stage (i.e. a continuous defect identification stage), a rail optical band local feature discrimination stage (i.e. a surface suspected defect discrimination stage), a local rail surface defect identification stage and a result output stage.
1. Global rail surface state identification
In the global rail surface state identification stage, 2000 rail surface pictures are prepared, wherein 1000 are discontinuous defect rail surfaces and 1000 are continuous defect rail surfaces, and the two steel rails are respectively subjected to frame drawing and marking according to types. The annotated images can be seen in fig. 3(a) and 3 (b).
The fast R-CNN framework was then used to train the serial defect detection model. The fast R-CNN is the first true end-to-end deep learning detection algorithm, and the biggest innovation is that a candidate frame is generated by adding a Region candidate Network (RPN) and based on an anchor point mechanism, and finally, feature extraction, candidate frame selection, frame regression and classification are integrated into one Network, so that the detection precision and the detection efficiency are effectively improved.
The specific process is that the input image is zoomed and enters the convolution layer to extract the characteristic to obtain the characteristic graph, then the characteristic graph is sent to the area candidate network to generate a series of possible candidate frames of the target, then the original characteristic graph and all the candidate frames output by the area candidate network are input to the ROI pooling layer, the candidate area is extracted and collected, the characteristic graph of the candidate area with the fixed size of 7 multiplied by 7 is calculated, and the characteristic graph is sent to the full connection layer to carry out the target classification and the coordinate regression. The framework of the Faster R-CNN is shown in FIG. 6, and the framework of the regional candidate network RPN is shown in FIG. 7.
The core idea of the candidate regional network RPN is to use a sliding window and anchor mechanism to generate the candidate boxes. The specific method is that on a 40 x 60 characteristic diagram obtained by convolution of a previous convolutional layer, 9 candidate frames with different length-width ratios and different scales are constructed at the center point of each sliding window in a sliding window mode, the candidate frames are mapped to an image middle frame, then sorting is carried out from large to small according to the score of each region, the first 2000 regions are extracted, the 2000 regions are subjected to non-maximum value inhibition (the obtained non-maximum values are inhibited, finally, the obtained regions are sorted again, 300 regions are output to a master RCNN for prediction, in an ROI pooling stage, the master RCNN maps about 300 candidate frames obtained by a region candidate network to a convolution characteristic diagram, and the characteristic diagrams with fixed sizes can be obtained by virtue of different-size boxes through ROI pooling.
The specific operation is as follows:
1) mapping the ROI to a position corresponding to the feature map according to the input image;
2) dividing the mapped area into areas with the size of 7 multiplied by 7;
3) and performing maximum pooling operation on each area.
This allows a fixed size of the corresponding map to be obtained from different size blocks. And the size of the output signature does not depend on the ROI and the convolution signature size. The greatest benefit of ROI pooling is the greatly increased training and testing speed. The Faster R-CNN designs a network RPN for extracting a candidate area, and replaces time-consuming selective search, so that the detection speed is greatly improved.
When a Faster RCNN frame is used for recognizing the global state of the steel rail, a recognition model is formed by training marked data, and after an image to be recognized is input into the model, recognition is realized through model reasoning. The specific identification process is shown in fig. 8.
The global rail surface state identification can be firstly positioned at the position of the steel rail in the image, and then whether the steel rail has continuous defects or not can be judged.
2. Light band local characteristic discrimination for rail
And (4) identifying the images which are obtained in the step (1) and have no continuous defects by using a steel rail light band local characteristic distinguishing module. Firstly, a steel rail subgraph is taken out from the picture according to the position of the steel rail obtained by the step 1. And then, binarizing the steel rail area in the image by adopting a segmentation algorithm. The rail surface is divided by means of a hard threshold. The pixel values of the gradation value 230 or more are binarized to 255, and the pixel values of 230 or less are 0. And after generating the binary image, longitudinally dividing the binary image by 8 equal parts, extracting the average width of the region, the standard deviation of the width of the region, the absolute extreme difference of the width of the region, the relative extreme difference of the width of the region, the average gradient of the region and the area of the light band of the region in the whole and each divided region, and forming a feature vector.
After the characteristics are extracted, the characteristics of a naive Bayes classifier are adopted for classification, and whether the characteristics meet the defect standard or not is judged. Naive Bayes classification is a method based on Bayes' theorem and assuming mutual independence between feature conditions, learning a joint probability distribution from input to output by a given training set and assuming independence between feature words as a precondition, and then inputting X to solve an output Y which maximizes the posterior probability based on the learned model.
Firstly, extracting 500 feature vectors of local defects, then extracting 500 feature vectors without local defects, and training and using a naive Bayes classifier. And then when the surface state of the steel rail is identified, judging whether the corresponding area has suspected defects or not by using a trained classifier.
3. Local rail surface defect identification
In the stage of identifying the local rail surface defects, 1000 pictures with the rail surface local defects are prepared, and the positions of the local defects on the steel rail are marked by drawing frames. And then training a local rail surface defect identification model by using a Faster R-CNN framework. The label is shown in figure 9.
When a Faster R-CNN frame is used for identifying the local defects of the steel rail, a local defect detection model is formed by training marked data, and after the images judged to be the suspected surface defects in the step 2 are input into the model, the identification is realized through model reasoning. The specific identification process is shown in fig. 10.
Local defect identification can be located first where the local defect is located in the image and second the type of defect (rail scratch, rail chipping, etc.) can be identified.
4. Result output
And finally, simultaneously outputting the global defects identified in the step 1 and the local defects identified in the step 3, namely completing the process of identifying the surface defects of the steel rail.
The embodiment of the invention also provides a steel rail surface defect identification device, which is described in the following embodiment. Because the principle of solving the problems of the device is similar to the method for identifying the surface defects of the steel rails, the implementation of the device can refer to the implementation of the method for identifying the surface defects of the steel rails, and repeated parts are not repeated.
As shown in fig. 11, the apparatus 1100 includes an acquisition module 1101, a consecutive defect identification module 1102, a light band extraction module 1103, a calculation module 1104, a suspected defect discrimination module 1105, a local defect identification module 1106, and an identification result output module 1107.
The acquiring module 1101 is used for acquiring a shot rail surface image;
the continuous defect identification module 1102 is configured to perform continuous defect identification on the surface image of the steel rail by using a pre-trained continuous defect detection model, and identify a continuous defect image and a steel rail position in the image, where the steel rail position is a pixel row position of the steel rail in the image;
the light band extraction module 1103 is used for extracting light bands of the steel rail area aiming at each steel rail surface image without continuous defects identified to generate a light band binary image;
the calculation module 1104 is used for calculating the area light band index of the steel rail area according to the light band binary image;
a suspected defect determining module 1105, configured to input the zone light band index into the classifier, and determine whether there is a suspected defect on the surface of the steel rail;
a local defect identification module 1106, configured to input the surface image of the steel rail with the surface suspected defect into a local defect detection model trained in advance to perform local defect identification, so as to obtain a local defect type and a local defect position;
and the recognition result output module 1107 is configured to output the continuous defect image and the position of the steel rail in the image, as well as the local defect type and the corresponding local defect position.
In an implementation manner of the embodiment of the present invention, the apparatus further includes an area dividing module, configured to:
dividing the light band in the light band binary image into a plurality of sub-areas with equal length according to a first preset length, wherein an overlapping area with a second preset length exists between every two adjacent sub-areas;
a calculation module to:
calculating the area light band indexes of each sub-area and the complete steel rail area according to the light band binary image;
a suspected defect discrimination module for:
and respectively classifying the area light band index of each sub-area and the area light band index of the complete steel rail area into a classifier, and judging whether the steel rail has surface suspected defects.
In an implementation manner of the embodiment of the present invention, the calculation module is configured to:
extracting light band characteristics from the light band binary image, wherein the light band characteristics comprise the width of each line of light band in an area, the total line number of the area light bands, the maximum value of the width of the area light bands, the minimum value of the width of the area light bands, the pixel value of each pixel in the area and the total column number of the area light bands;
and calculating the zone light band index of the steel rail zone according to the light band characteristics.
In an implementation manner of the embodiment of the present invention, the calculation module is configured to:
calculating the average width of the area according to the width of each line of the light band and the total line number of the area light bands;
calculating the area width standard deviation according to the area average width, the width of each line of light band and the total line number of the area light bands;
calculating the absolute extreme difference of the area width according to the maximum value and the minimum value of the area light band width;
calculating the relative extreme difference of the area width according to the maximum value of the area light band width, the minimum value of the area light band width and the average width of the area;
calculating the average gradient of the area according to the pixel value of each pixel in the area, the total row number of the area light bands and the total column number of the area light bands;
calculating the area of the area light band according to the pixel value of each pixel in the area, the total row number and the total column number of the area light band;
and determining the area average width, the area width standard deviation, the area width absolute extreme difference, the area width relative extreme difference, the area average gradient and the area light band area as the area light band index of each area.
In an implementation manner of the embodiment of the present invention, the apparatus further includes a module training module, configured to:
acquiring a first sample image, wherein the first sample image comprises a steel rail image with continuous defects and a steel rail image without continuous defects;
marking the position of a complete steel rail and a continuous defect mark in a steel rail image with continuous defects;
and training the deep learning target detection model by using the marked steel rail images with continuous defects and the marked steel rail images without continuous defects to obtain a continuous defect detection model.
In an implementation manner of the embodiment of the present invention, the model training module is configured to:
acquiring a second sample image, wherein the second sample image comprises a steel rail image with local defects;
marking the local defect position and the local defect type identification in the second sample image;
and training the deep learning target detection model by using the labeled second sample image to obtain a local defect detection model.
In the embodiment of the invention, a method for identifying the surface defects of the continuous to local multistage steel rails is provided, the continuous defect identification is firstly carried out on the surface images of the steel rails, then the surface state of the steel rails is automatically evaluated through a classifier, the surface images of the steel rails with the surface defects possibly exist are identified, then the local defects are detected by utilizing a local defect detection model, the combination of deep learning and the traditional classification method is realized, further the comprehensive detection and the automatic detection of the global continuous defects and the local defects of the surface of the steel rails are realized, the detection rate and the accuracy of the defects are improved, and the working efficiency of a first-line detector is greatly improved.
An embodiment of the present invention further provides a computer device, and fig. 12 is a schematic diagram of the computer device in the embodiment of the present invention, where the computer device is capable of implementing all steps in the method for identifying a surface defect of a steel rail in the embodiment, and the computer device specifically includes the following contents:
a processor (processor)1201, a memory (memory)1202, a communication Interface 1203, and a communication bus 1204;
the processor 1201, the memory 1202 and the communication interface 1203 complete mutual communication through the communication bus 1204; the communication interface 1203 is used for implementing information transmission between related devices;
the processor 1201 is configured to call a computer program in the memory 1202, and when the processor executes the computer program, the processor implements the rail surface defect identification method in the above embodiment.
The embodiment of the invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program for executing the rail surface defect identification method.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (14)
1. A method of identifying surface defects of a rail, the method comprising:
acquiring a shot rail surface image;
carrying out continuous defect identification on the surface image of the steel rail by using a pre-trained continuous defect detection model, and identifying the position of the steel rail and a continuous defect image, wherein the position of the steel rail is the pixel row position of the steel rail in the image;
aiming at each steel rail surface image without continuous defects, carrying out light band extraction on a steel rail area according to the position of the steel rail to generate a light band binary image;
calculating the area light band index of the steel rail area according to the light band binary image;
inputting the zone light band index into a classifier, and judging whether the surface of the steel rail has suspected defects;
inputting the surface image of the steel rail with the surface suspected defect into a pre-trained local defect detection model for local defect identification to obtain a local defect image, a local defect type and a local defect position;
and outputting the continuous defect image, the local defect type and the corresponding local defect position.
2. The method of claim 1, wherein prior to calculating the zone band index for the rail zone from the band binary map, the method further comprises:
dividing the light band in the light band binary image into a plurality of sub-areas with equal length according to a first preset length, wherein an overlapping area with a second preset length exists between every two adjacent sub-areas;
calculating the area light band index of the steel rail area according to the light band binary image, comprising the following steps:
calculating the area light band indexes of each sub-area and the complete steel rail area according to the light band binary image;
inputting the zone light band index into a classifier, and judging whether the steel rail has surface suspected defects or not, wherein the method comprises the following steps:
and respectively classifying the area light band index of each sub-area and the area light band index of the complete steel rail area into a classifier, and judging whether the steel rail has surface suspected defects.
3. The method of claim 1 or 2, wherein calculating the zone band indicator for the rail zone from the band binary map comprises:
extracting light band characteristics from the light band binary image, wherein the light band characteristics comprise the width of each line of light band in an area, the total line number of the area light bands, the maximum value of the width of the area light bands, the minimum value of the width of the area light bands, the pixel value of each pixel in the area and the total column number of the area light bands;
and calculating the zone light band index of the steel rail zone according to the light band characteristics.
4. The method of claim 3, wherein calculating an area band indicator for a rail area from band characteristics comprises:
calculating the average width of the area according to the width of each line of the light band and the total line number of the area light bands;
calculating the area width standard deviation according to the area average width, the width of each line of light band and the total line number of the area light bands;
calculating the absolute extreme difference of the area width according to the maximum value and the minimum value of the area light band width;
calculating the relative extreme difference of the area width according to the maximum value of the area light band width, the minimum value of the area light band width and the average width of the area;
calculating the average gradient of the area according to the pixel value of each pixel in the area, the total row number of the area light bands and the total column number of the area light bands;
calculating the area of the area light band according to the pixel value of each pixel in the area, the total row number and the total column number of the area light band;
and determining the area average width, the area width standard deviation, the area width absolute extreme difference, the area width relative extreme difference, the area average gradient and the area light band area as the area light band index of each area.
5. The method of claim 1, wherein prior to the continuous defect identification of the rail surface images using the pre-trained continuous defect detection model, the method further comprises:
acquiring a first sample image, wherein the first sample image comprises a steel rail image with continuous defects and a steel rail image without continuous defects;
marking the position of a complete steel rail in the first sample image, and marking a continuous defect mark in the steel rail image with continuous defects;
and training the deep learning target detection model by using the labeled first sample image to obtain a continuous defect detection model.
6. The method of claim 1, wherein before inputting the rail surface image with surface suspected defects into a pre-trained local defect detection model for local defect identification, the method further comprises:
acquiring a second sample image, wherein the second sample image comprises a steel rail image with local defects;
marking the local defect position and the local defect type identification in the second sample image;
and training the deep learning target detection model by using the labeled second sample image to obtain a local defect detection model.
7. A rail surface defect identification apparatus, the apparatus comprising:
the acquisition module is used for acquiring a shot rail surface image;
the continuous defect identification module is used for carrying out continuous defect identification on the surface image of the steel rail by utilizing a pre-trained continuous defect detection model, and identifying the position of the steel rail and a continuous defect image, wherein the position of the steel rail is the pixel row position of the steel rail in the image;
the light band extraction module is used for carrying out light band extraction on a steel rail area according to the position of the steel rail aiming at each steel rail surface image without continuous defects identified to generate a light band binary image;
the calculation module is used for calculating the area light band index of the steel rail area according to the light band binary image;
the suspected defect judging module is used for inputting the zone light band indexes into the classifier and judging whether the surface suspected defect exists on the steel rail or not;
the local defect identification module is used for inputting the surface image of the steel rail with the surface suspected defect into a local defect detection model trained in advance to carry out local defect identification so as to obtain a local defect image, a local defect type and a local defect position;
and the identification result output module is used for outputting the continuous defect image, the local defect type and the corresponding local defect position.
8. The apparatus of claim 7, further comprising a region partitioning module configured to:
dividing the light band in the light band binary image into a plurality of sub-areas with equal length according to a first preset length, wherein an overlapping area with a second preset length exists between every two adjacent sub-areas;
a calculation module to:
calculating the area light band indexes of each sub-area and the complete steel rail area according to the light band binary image;
a suspected defect discrimination module for:
and respectively classifying the area light band index of each sub-area and the area light band index of the complete steel rail area into a classifier, and judging whether the steel rail has surface suspected defects.
9. The apparatus of claim 7 or 8, wherein the computing module is configured to:
extracting light band characteristics from the light band binary image, wherein the light band characteristics comprise the width of each line of light band in an area, the total line number of the area light bands, the maximum value of the width of the area light bands, the minimum value of the width of the area light bands, the pixel value of each pixel in the area and the total column number of the area light bands;
and calculating the zone light band index of the steel rail zone according to the light band characteristics.
10. The apparatus of claim 9, wherein the computing module is configured to:
calculating the average width of the area according to the width of each line of the light band and the total line number of the area light bands;
calculating the area width standard deviation according to the area average width, the width of each line of light band and the total line number of the area light bands;
calculating the absolute extreme difference of the area width according to the maximum value and the minimum value of the area light band width;
calculating the relative extreme difference of the area width according to the maximum value of the area light band width, the minimum value of the area light band width and the average width of the area;
calculating the average gradient of the area according to the pixel value of each pixel in the area, the total row number of the area light bands and the total column number of the area light bands;
calculating the area of the area light band according to the pixel value of each pixel in the area, the total row number and the total column number of the area light band;
and determining the area average width, the area width standard deviation, the area width absolute extreme difference, the area width relative extreme difference, the area average gradient and the area light band area as the area light band index of each area.
11. The apparatus of claim 7, further comprising a module training module to:
acquiring a first sample image, wherein the first sample image comprises a steel rail image with continuous defects and a steel rail image without continuous defects;
marking the position of a complete steel rail in the first sample image, and marking a continuous defect mark in the steel rail image with continuous defects;
and training the deep learning target detection model by using the labeled first sample image to obtain a continuous defect detection model.
12. The apparatus of claim 7, wherein the model training module is configured to:
acquiring a second sample image, wherein the second sample image comprises a steel rail image with local defects;
marking the local defect position and the local defect type identification in the second sample image;
and training the deep learning target detection model by using the labeled second sample image to obtain a local defect detection model.
13. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 6 when executing the computer program.
14. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for executing the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111334140.9A CN114067296A (en) | 2021-11-11 | 2021-11-11 | Method and device for identifying surface defects of steel rail |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111334140.9A CN114067296A (en) | 2021-11-11 | 2021-11-11 | Method and device for identifying surface defects of steel rail |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114067296A true CN114067296A (en) | 2022-02-18 |
Family
ID=80275173
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111334140.9A Pending CN114067296A (en) | 2021-11-11 | 2021-11-11 | Method and device for identifying surface defects of steel rail |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114067296A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115272334A (en) * | 2022-09-29 | 2022-11-01 | 江苏美克美斯自动化科技有限责任公司 | Method for detecting micro defects on surface of steel rail under complex background |
| CN117253066A (en) * | 2023-11-20 | 2023-12-19 | 西南交通大学 | Rail surface state identification method, device, equipment and readable storage medium |
-
2021
- 2021-11-11 CN CN202111334140.9A patent/CN114067296A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115272334A (en) * | 2022-09-29 | 2022-11-01 | 江苏美克美斯自动化科技有限责任公司 | Method for detecting micro defects on surface of steel rail under complex background |
| CN115272334B (en) * | 2022-09-29 | 2023-08-25 | 研索仪器科技(上海)有限公司 | Method for detecting tiny defects on surface of steel rail under complex background |
| CN117253066A (en) * | 2023-11-20 | 2023-12-19 | 西南交通大学 | Rail surface state identification method, device, equipment and readable storage medium |
| CN117253066B (en) * | 2023-11-20 | 2024-02-27 | 西南交通大学 | Rail surface state identification method, device, equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112330591B (en) | Steel rail surface defect detection method and device capable of achieving sample-less learning | |
| CN111899288B (en) | Tunnel leakage water area detection and identification method based on infrared and visible light image fusion | |
| CN109101924B (en) | Machine learning-based road traffic sign identification method | |
| CN114067296A (en) | Method and device for identifying surface defects of steel rail | |
| CN110796643A (en) | Rail fastener defect detection method and system | |
| CN113436157B (en) | Vehicle-mounted image identification method for pantograph fault | |
| CN111311567A (en) | Method for identifying fastener and steel rail diseases of track line image | |
| CN105225226A (en) | A kind of cascade deformable part model object detection method based on Iamge Segmentation | |
| CN110782443B (en) | Railway track defect detection method and system | |
| US9547805B1 (en) | Systems and methods for identifying roads in images | |
| Zhao et al. | Image-based comprehensive maintenance and inspection method for bridges using deep learning | |
| Kanaeva et al. | Road pavement crack detection using deep learning with synthetic data | |
| Abou Chacra et al. | Fully automated road defect detection using street view images | |
| CN115131330A (en) | Method and system for detecting and measuring surface defects of steel rail by fusing multi-source sensors | |
| CN113781391A (en) | Image defect detection method and related equipment | |
| Ahmed et al. | Traffic sign detection and recognition model using support vector machine and histogram of oriented gradient | |
| CN111242899A (en) | Image-based flaw detection method and computer-readable storage medium | |
| CN113689392A (en) | Railway fastener defect detection method and device | |
| CN112419289A (en) | Intelligent detection method for urban subway rail fastener defects | |
| CN111179278B (en) | Image detection method, device, equipment and storage medium | |
| Peng et al. | Real-time traffic sign text detection based on deep learning | |
| CN116258908A (en) | Ground disaster prediction evaluation classification method based on unmanned aerial vehicle remote sensing image data | |
| Kembhavi et al. | Low-cost image processing system for evaluating pavement surface distress | |
| CN112256906A (en) | Method, device and storage medium for marking annotation on display screen | |
| Elanangai et al. | Automated system for defect identification and character recognition using IR images of SS-plates |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |