CN111652227B - Method for detecting damage fault of bottom floor of railway wagon - Google Patents
Method for detecting damage fault of bottom floor of railway wagon Download PDFInfo
- Publication number
- CN111652227B CN111652227B CN202010437719.7A CN202010437719A CN111652227B CN 111652227 B CN111652227 B CN 111652227B CN 202010437719 A CN202010437719 A CN 202010437719A CN 111652227 B CN111652227 B CN 111652227B
- Authority
- CN
- China
- Prior art keywords
- image
- floor
- binary
- detected
- target area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B61—RAILWAYS
- B61F—RAIL VEHICLE SUSPENSIONS, e.g. UNDERFRAMES, BOGIES OR ARRANGEMENTS OF WHEEL AXLES; RAIL VEHICLES FOR USE ON TRACKS OF DIFFERENT WIDTH; PREVENTING DERAILING OF RAIL VEHICLES; WHEEL GUARDS, OBSTRUCTION REMOVERS OR THE LIKE FOR RAIL VEHICLES
- B61F1/00—Underframes
- B61F1/08—Details
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
- G01M17/00—Testing of vehicles
- G01M17/08—Railway vehicles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
Abstract
A fault detection method for detecting damage of a bottom floor of a railway wagon belongs to the technical field of safety detection of railway wagons. The invention aims at the problems of poor reliability and low efficiency of the safety detection of the bottom floor of the existing railway wagon by adopting manual image inspection. The method comprises the following steps: acquiring full-train linear array images at the bottom of a running railway wagon under different conditions, and processing to obtain a sample data set; constructing a semantic segmentation neural network model based on an encoder and a decoder, training by adopting a sample data set, and updating the semantic segmentation neural network model after finding an optimal weight coefficient; then, acquiring a current whole-train linear array image at the bottom of the running railway wagon, obtaining a current target area to be detected by adopting an updated semantic segmentation neural network model, performing pre-judgment, and performing fault detection on the current target area to be detected which is pre-judged to have a fault by adopting a single-stage ssd model. The invention realizes the automatic identification of the damage of the bottom floor.
Description
Technical Field
The invention relates to a method for detecting damage faults of a bottom floor of a railway wagon, and belongs to the technical field of safety detection of railway wagons.
Background
The floor area of the bottom of the railway wagon is large, and a plurality of key components for maintaining the normal running of the train are distributed on the floor area. Because the goods transported by the rail wagon are various in types, can be hard and slender steel bars, and can also be packaged heavy objects, the damage to the bottom floor can be caused after long-term use or accidental collision, and the goods can fall out of the formed broken holes; the goods can be pressed to steel pipes distributed on the bottom floor in the falling process, and the failure of the truck can be further caused. Therefore, no matter the accident such as breakage and loss of the goods or friction between the hard goods and the rail at the bottom of the truck occurs, the safe operation of the railway is threatened, and further the loss is caused.
At present, the safety detection of the bottom floor of the railway wagon is carried out in a mode of manually checking images. Due to the fact that the area of the bottom floor is large, a vehicle inspection worker needs to inspect the use conditions of multiple parts of the truck at the same time, the conditions of missing inspection and wrong inspection are prone to occurring in the working process, and the accuracy of the inspection result is difficult to guarantee; meanwhile, manual detection has the problem of low efficiency.
Therefore, aiming at the defects, the failure detection can be carried out on the truck in an automatic identification mode to improve the detection efficiency and accuracy. In recent years, deep learning and artificial intelligence are continuously developed, and the technology is continuously mature. The method for realizing automatic detection of the bottom floor fault by applying deep learning in image processing can be provided, and the conversion from manual detection operation to mechanical detection operation is realized, so that the labor cost is effectively saved, and the operation quality and the operation efficiency are improved.
Disclosure of Invention
The invention provides a method for detecting damage faults of a bottom floor of a railway wagon, aiming at the problems of poor reliability and low efficiency of the conventional safety detection of the bottom floor of the railway wagon by adopting manual image inspection.
The invention discloses a method for detecting damage faults of a bottom floor of a railway wagon, which comprises the following steps:
acquiring full-train linear array images at the bottom of a running railway wagon under different conditions;
uniformly splitting each carriage part in the whole linear array image into n gray level images along the length direction of the railway wagon; n is a preset splitting amplitude number;
carrying out image preprocessing on the gray level image, and carrying out data amplification to obtain a sample data set;
constructing a semantic segmentation neural network model based on an encoder and a decoder, and enabling each convolution layer to have a convolution function, a batch standardization function and a Relu activation function;
dividing a sample data set into a training data set and a test data set; predicting a gray level image in a training data set through the semantic segmentation neural network model to obtain an image target region, calculating the image target region and a mark truth value region of the gray level image to obtain a cross entropy loss function loss value, and optimizing the weight of the semantic segmentation neural network model through an optimizer to reduce the cross entropy loss function loss value to a preset threshold value;
predicting the gray level image of the test data set through the trained semantic segmentation neural network model to obtain a target region to be detected, calculating the target region to be detected and a mark truth value region of the corresponding gray level image, further obtaining a cross entropy loss function loss value until the cross entropy loss function loss value reaches a standard loss value, finding an optimal weight coefficient, and updating the semantic segmentation neural network model;
acquiring a current whole-train linear array image at the bottom of a running railway wagon, uniformly splitting each carriage part in the current whole-train linear array image into n current gray level images along the length direction of the railway wagon, and predicting the current gray level images by adopting an updated semantic segmentation neural network model to obtain a current target area to be detected; and carrying out fault pre-judgment on the current target area to be detected, judging the current target area to be detected with a fault, and carrying out fault detection by adopting a single-stage ssd model.
According to the method for detecting the breakage fault of the bottom floor of the railway wagon,
the full linear array image is acquired by a camera or a video camera arranged at the position of the freight car track.
According to the method for detecting the breakage fault of the bottom floor of the railway wagon,
the n is chosen to be 14.
According to the method for detecting the breakage fault of the bottom floor of the railway wagon,
the image preprocessing of the grayscale image comprises:
obtaining a bottom floor binary segmentation image from the gray level image in a manual marking mode, wherein the bottom floor area value in the binary segmentation image is 0, and the rest area values are 255;
obtaining a bottom floor binary fault image from the bottom floor binary segmentation image in a manual marking mode, wherein the value of a damaged area of the bottom floor in the binary fault image is 255, and the values of the rest areas are 0;
determining a damaged rectangular frame of the floor in the binary fault image in a manual marking mode to obtain a damaged XML file of the floor and form an XML file set; and performing at least one of horizontal and vertical turning, zooming, translation, contrast enhancement and brightness change on the XML file with the damaged floor to obtain a sample data set.
According to the method for detecting the breakage fault of the bottom floor of the railway wagon,
the building of the semantic segmentation neural network model comprises the following steps:
constructing an encoder network model, comprising:
the coding unit I: comprises 3 × 3 convolution layers → 3 × 3 convolution layers;
and a second coding unit: comprises 3 × 3 convolution layers → 1 × 1 convolution layers;
and a coding unit III: comprises 3 × 3 convolution layers → 1 × 1 convolution layers;
and a coding unit IV: comprises 3 × 3 convolution layers → 1 × 1 convolution layers;
pyramid pooling unit comprising three parallel layer structures:
one) 1x1 pooling layer → 3 x 3 convolutional layer → 1x1 upsampling layer;
two) 2 x 2 pooling layers → 3 x 3 convolutional layers → 2 x 2 upsampling layers;
three) 4 × 4 pooling layers → 3 × 3 convolutional layers → 4 × 4 upsampling layers;
and the gray level image is processed by a first coding unit, a second coding unit, a third coding unit and a fourth coding unit in sequence, and then is output after being processed by three parallel layer structures of the pyramid pooling unit.
According to the method for detecting the breakage fault of the bottom floor of the railway wagon,
the building of the semantic segmentation neural network model further comprises the following steps:
constructing a decoder network model, comprising:
the decoding unit I: 1x1 convolution layer → 3 x 3 convolution layer → 2 x 2 convolution layer;
and a second decoding unit: 3 x 3 convolutional layer → 2 x 2 upsampling layer;
a third decoding unit: 3 x 3 convolutional layer → 2 x 2 upsampling layer;
and a decoding unit IV: 1 × 1 convolutional layer → 1 × 1 convolutional layer;
and images output by the three parallel layer structures of the pyramid pooling unit are sequentially processed by the first decoding unit, the second decoding unit, the third decoding unit and the fourth decoding unit to obtain an image target area of the gray level image.
According to the method for detecting the breakage fault of the bottom floor of the railway wagon,
the method for predicting the current gray level image by adopting the updated semantic segmentation neural network model to obtain the current target area to be detected comprises the following steps:
the updated semantic segmentation neural network model carries out positioning prediction on the current gray level image to obtain a current target area to be detected; and after the binarization of the current target area image to be detected, performing negation operation to obtain the binarization target area image to be detected.
According to the method for detecting the breakage fault of the bottom floor of the railway wagon,
the method for pre-judging the fault of the current target area to be detected comprises the following steps:
adding the result obtained by performing point multiplication on the binary target area image to be detected and the binary image of the standard positioning algorithm to obtain an image value, if the image value is not greater than a set threshold value, determining that no damage exists in the binary target area image to be detected, and otherwise, performing fault detection by adopting a single-stage ssd model;
and a non-fault functional area is marked on the standard positioning algorithm binary image.
According to the method for detecting the breakage fault of the bottom floor of the railway wagon,
carrying out dot multiplication on the to-be-detected binary target area image and a standard positioning algorithm binary image, and then adding to obtain an image fault-free score;
the method for fault detection by adopting the single-stage ssd model comprises the following steps:
comparing the image failure-free score of the to-be-detected binary target area image with a preset score, wherein the image failure-free score is judged to be broken; otherwise, carrying out the next judgment;
obtaining coordinate data according to a to-be-detected binary target area image, obtaining a corresponding binary image of the to-be-detected binary target area image according to the coordinate data, performing point multiplication on the corresponding binary image and the image value, and adding the corresponding binary image and the image value, wherein if the obtained value is not greater than a judgment threshold value, the detection result is that the bottom floor is not damaged; otherwise, the detection result is that the bottom floor is damaged, and the detection result is reported.
The invention has the beneficial effects that: the method of the invention replaces the manual detection of the existing bottom floor with the mode of automatic image identification, so that the operation standard is unified, the influence of personnel quality and responsibility is avoided, the operation quality can be effectively improved, and the stability and the precision of the detection are improved.
The method applies the deep learning algorithm to the automatic detection of the damage of the bottom floor of the railway wagon, improves the stability and the precision of the whole algorithm, and has high flexibility, accuracy and robustness compared with the traditional machine vision detection method of manual standard feature extraction.
The current target area to be detected is obtained through the designed semantic segmentation neural network model prediction, so that the prediction time is shortened, and the segmentation speed is improved.
The method effectively reduces false alarm caused by image background interference through the prediction of the semantic segmentation neural network model and the judgment of the single-stage ssd model.
Drawings
FIG. 1 is an exemplary flow chart of a method of detecting a breakage failure in a bottom floor of a railway wagon according to the present disclosure;
FIG. 2 is a schematic disassembled view of each carriage part in a full linear array image of the bottom of a railway wagon;
FIG. 3 is a schematic diagram of a semantically segmented neural network model;
fig. 4 is a schematic diagram of addition of results obtained by dot-multiplying a binarized image.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.
The invention is further described with reference to the following drawings and specific examples, which are not intended to be limiting.
In a first embodiment, referring to fig. 1 to 3, the present invention provides a method for detecting a breakage fault of a bottom floor of a railway wagon, including:
acquiring full-train linear array images at the bottom of a running railway wagon under different conditions;
uniformly splitting each carriage part in the whole linear array image into n gray level images along the length direction of the railway wagon; n is a preset splitting amplitude number;
carrying out image preprocessing on the gray level image, and carrying out data amplification to obtain a sample data set;
constructing a semantic segmentation neural network model based on an encoder and a decoder, and enabling each convolution layer to have a convolution function, a batch standardization function and a Relu activation function;
dividing a sample data set into a training data set and a test data set; predicting a gray level image in a training data set through the semantic segmentation neural network model to obtain an image target region, calculating the image target region and a mark truth value region of the gray level image to obtain a cross entropy loss function loss value, and optimizing the weight of the semantic segmentation neural network model through an optimizer to reduce the cross entropy loss function loss value to a preset threshold value;
predicting the gray level image of the test data set through the trained semantic segmentation neural network model to obtain a target region to be detected, calculating the target region to be detected and a mark truth value region of the corresponding gray level image, further obtaining a cross entropy loss function loss value until the cross entropy loss function loss value reaches a standard loss value, finding an optimal weight coefficient, and updating the semantic segmentation neural network model;
acquiring a current whole-train linear array image at the bottom of a running railway wagon, uniformly splitting each carriage part in the current whole-train linear array image into n current gray level images along the length direction of the railway wagon, and predicting the current gray level images by adopting an updated semantic segmentation neural network model to obtain a current target area to be detected; and carrying out fault pre-judgment on the current target area to be detected, judging the current target area to be detected with a fault, and carrying out fault detection by adopting a single-stage ssd model.
In the embodiment, the imaging device can be carried around the track of the truck by using the fixing device, and the running truck is shot so as to obtain the linear array images of the whole truck at the two sides and the bottom of the truck. From the wheelbase and a priori information of the component position, a coarse positioning region containing the bottom floor can be obtained from the image macro. And carrying out preprocessing modes such as data amplification and the like on the rough positioning image to construct a neural network training set and a neural network structure, and identifying the outline of the bottom floor area in the image. And analyzing by using logic judgment to detect whether the bottom floor part area is damaged or not. And if the train is damaged, alarming is carried out, and the staff carries out corresponding processing according to the identification result, so that the safe operation of the train is ensured.
Optimizer and penalty function selection in this embodiment
The image data is subjected to prediction (predict) image output by a segmentation network and true (GT) image of an original mark to calculate a cross entropy loss function loss value (loss value of a cross entropy loss function), and an optimizer Adam is used for optimizing weight to gradually reduce the loss value, so that a segmentation network model continuously learns real data characteristics. The Adam optimizer has the advantages of high efficiency, small occupied memory, suitability for large-scale data and the like. And continuously training through a loss function and an optimizer, and updating and iterating the weight coefficients until the optimal weight coefficients are found by taking the loss value of the cross entropy loss function calculated by the prediction image output by the segmentation network of the image not participating in training and the true value image of the original mark as a standard.
As an example, the full line image is acquired by a camera or video camera provided at the truck track.
And collecting the whole linear array image, and carrying a camera or a video camera on fixed equipment around the rail of the wagon to shoot the running wagon. And after the truck passes through the equipment, acquiring a high-definition gray full-truck linear array image.
Image quality is mainly affected by two aspects, one is the influence of natural conditions: rain, snow, mud, etc.; one is the influence of human conditions: oil stain, black paint, installation differences of equipment, and the like. Further, since many parts whose positions vary are attached to the bottom floor, the image of the bottom floor portion varies greatly because the area of the bottom floor is large. In order to enhance the robustness of the recognition algorithm, it is endeavored to overlay images under various conditions during the process of collecting image data.
As an example, as shown in connection with fig. 2, n is selected to be 14.
The railway wagon is formed by connecting a certain number of wagon carriages, and the floor at the bottom of each carriage is very large. In order to obtain clear pictures, the length of the whole vehicle linear array image can reach 17000 pixels, and the width is generally 2048 pixels. The large picture cannot be directly input into a semantic segmentation neural network model for detection, so that the image needs to be uniformly split. According to the characteristics that parts at the wheel positions of the bottom floor are complicated and concentrated and a plurality of pipelines connected among the parts are arranged, if the parts are split along the length direction and the width direction of an image, split bottom subgraphs are disordered, and the recognition effect is influenced. In the present embodiment, the image of one car is uniformly divided into 14 parts in the longitudinal direction, and is not divided in the width direction, as shown in fig. 2.
Further, as shown in fig. 1, the image preprocessing on the grayscale image includes:
obtaining a bottom floor binary segmentation image from the gray level image in a manual marking mode, wherein the bottom floor area value in the binary segmentation image is 0, and the rest area values are 255;
obtaining a bottom floor binary fault image from the bottom floor binary segmentation image in a manual marking mode, wherein the value of a damaged area of the bottom floor in the binary fault image is 255, and the values of the rest areas are 0;
determining a damaged rectangular frame of the floor in the binary fault image in a manual marking mode to obtain a damaged XML file of the floor and form an XML file set; and performing at least one of horizontal and vertical turning, zooming, translation, contrast enhancement and brightness change on the XML file with the damaged floor to obtain a sample data set.
In the embodiment, a single object segmentation model or a single target detection model cannot effectively suppress a large number of false alarms caused by complexity and changeability of the bottom floor, so that three network models can be matched to realize fault identification. The three network models include the ssd detection model (single-stage ssd model) corresponding to dataset 1, the segmented network (fault) corresponding to dataset 2 (semantic segmented neural network model), and the segmented network (localization) corresponding to dataset 3 in fig. 1.
The sample data set comprises the following four parts:
1) a set of grayscale images consisting of all grayscale images. The gray level image set is a size-changed sub-image set which is split from a large image;
2) the bottom floor binary segmentation image (Ground Truth image). The image set is a binary segmentation image (the value of a bottom floor area is 0, and the values of other parts and background areas are 255) set of a bottom floor, which is obtained by manually marking sub-images;
3) binary failure picture set (group Truth picture). The binary fault image set is a binary segmentation image set of the bottom floor damage fault, which is obtained by manually marking the binary segmentation image of the bottom floor (the value of the damage position of the bottom floor is 255, and the values of other components and a background area are 0);
4) an XML document collection. The XML file set is a rectangular frame XML file set of the position of the damaged floor obtained through marking.
Data amplification on the sample dataset: to further improve the robustness of the algorithm, data amplification of the sample data set is still required. The amplification form is mainly random combination of operations such as horizontal and vertical turning, scaling and translation of the image, contrast enhancement, brightness change and the like.
Constructing a semantic segmentation neural network model algorithm:
on the basis of ensuring the accuracy of the algorithm, the implementation method also needs to ensure the following two requirements:
1) the requirement on real-time performance is high in the automatic truck fault identification process. For the identification of the bottom floor damage, the target is overlarge (most trains contain about 800 subgraphs to be detected), and at least 50 pictures per second need to be detected according to actual requirements;
2) in the automatic truck fault identification process, the smaller the memory occupation ratio required under static memory capacity and dynamic operation after the algorithm is changed into an executable file is better.
Still further, as shown in fig. 3, the constructing the semantic segmentation neural network model includes:
constructing an encoder network model, comprising:
the coding unit I: comprises 3 × 3 convolution layers → 3 × 3 convolution layers;
and a second coding unit: comprises 3 × 3 convolution layers → 1 × 1 convolution layers;
and a coding unit III: comprises 3 × 3 convolution layers → 1 × 1 convolution layers;
and a coding unit IV: comprises 3 × 3 convolution layers → 1 × 1 convolution layers;
pyramid pooling unit (concatenate) comprising a three parallel layer structure:
one) 1x1 pooling layer → 3 x 3 convolutional layer → 1x1 upsampling layer;
two) 2 x 2 pooling layers → 3 x 3 convolutional layers → 2 x 2 upsampling layers;
three) 4 × 4 pooling layers → 3 × 3 convolutional layers → 4 × 4 upsampling layers;
and the gray level image is processed by a first coding unit, a second coding unit, a third coding unit and a fourth coding unit in sequence, and then is output after being processed by three parallel layer structures of the pyramid pooling unit.
Still further, as shown in fig. 3, the constructing the semantic segmentation neural network model further includes:
constructing a decoder network model, comprising:
the decoding unit I: 1x1 convolution layer → 3 x 3 convolution layer → 2 x 2 convolution layer;
and a second decoding unit: 3 x 3 convolutional layer → 2 x 2 upsampling layer;
a third decoding unit: 3 x 3 convolutional layer → 2 x 2 upsampling layer;
and a decoding unit IV: 1 × 1 convolutional layer → 1 × 1 convolutional layer (softmax activation function);
and images output by the three parallel layer structures of the pyramid pooling unit are sequentially processed by the first decoding unit, the second decoding unit, the third decoding unit and the fourth decoding unit to obtain an image target area of the gray level image.
The semantic segmentation neural network model of the embodiment has the following characteristics:
1) each convolution layer comprises an added convolution function, a batch normalization function (batch normalization) and a Relu activation function so as to accelerate the learning convergence speed during training;
2) many convolution kernels employ 1x1 convolution layers to reduce the model parameters.
3) And a concatenate function is adopted in the model to fuse high-level pyramid pooling feature information of the image.
In fig. 3, k × k conv (deconv, maxporoling, upsampling), n,/() s represent the convolution function (deconvolution function, pooling function, upsampling function) used, where the convolution kernel size is set to k × k, the number of convolution kernels is set to n, and the convolution kernel shift step size is s;
concat represents the concatenate function used;
the first convolutional layer was followed by a batch normalization layer (Batchnormalization) and Relu activation function.
Further, the step of predicting the current gray image by using the updated semantic segmentation neural network model to obtain the current target region to be detected comprises the following steps:
the updated semantic segmentation neural network model carries out positioning prediction on the current gray level image to obtain a current target area to be detected; and after the binarization of the current target area image to be detected, performing negation operation to obtain the binarization target area image to be detected.
Further, the fault pre-judging of the current target area to be detected comprises:
performing point multiplication on the binary target area image b to be detected and the standard positioning algorithm binary image a, and adding the obtained results to obtain an image value c, wherein the process of the point multiplication is shown in fig. 4; if the image value is not greater than the set threshold value, the image of the binary target area to be detected is not damaged, otherwise, a single-stage ssd model is adopted to detect the fault;
and a non-fault functional area is marked on the standard positioning algorithm binary image.
The standard positioning algorithm binary image is obtained after training through a segmentation network (fault) corresponding to the data set 2 in fig. 1, and useful parts fixed on the bottom floor are marked in the standard positioning algorithm binary image, so that the functional parts are prevented from being mistakenly identified as damaged areas.
Further, with reference to fig. 1, the binary target area image to be detected and the standard positioning algorithm binary image are subjected to point multiplication and then added to obtain an image fault-free score;
the method for fault detection by adopting the single-stage ssd model comprises the following steps:
comparing the image failure-free score of the to-be-detected binary target area image with a preset score, wherein the image failure-free score is judged to be broken; otherwise, carrying out the next judgment;
obtaining coordinate data according to a to-be-detected binary target area image, obtaining a corresponding binary image of the to-be-detected binary target area image according to the coordinate data, performing point multiplication on the corresponding binary image d and the image value c, and adding the dot multiplication result, wherein if the obtained value e is not greater than a judgment threshold value, the detection result is that the bottom floor is not damaged; otherwise, the detection result is that the bottom floor is damaged, and the detection result is reported.
The single-stage ssd model not only meets the real-time requirement in speed, but also is very small, and very meets the detection requirement on multi-floor damage.
The specific process for judging the floor damage fault is as follows:
the real vehicle-passing image is split into sub-images, then as shown in a flow chart of a prediction part of a figure 1, the following flow is distinguished, because the components on the floor are complex in arrangement and the positions can change, a large amount of false alarms are caused by the conditions of a large amount of interference and the like on the floor, the invention adopts the combination of 3 neural networks, and the determined fault area can be reported, thereby improving the fault recognition effect; the method mainly comprises the following steps:
1) acquiring a linear array subgraph (gray level image);
2) positioning and predicting the gray level image by using a trained semantic segmentation neural network model, and carrying out binarization and negation operation on the output image (0 is changed into 1, and 1 is changed into 0);
using a trained fault segmentation neural network to carry out fault prediction on the image, and binarizing the output image;
3) performing fault pre-judgment on a current target area to be detected, and if a value c obtained by adding point multiplication of two binary images (a binary target area image to be detected and a binary image of a standard positioning algorithm) is not more than a set threshold value (sum (c) <th 1), verifying that the image is not damaged, and processing the next image; otherwise, carrying out the next step;
4) predicting the image by using a trained single-stage ssd model, judging, and if the score with no fault is greater than a set value th2(score > th2), proving that the image is not damaged, and processing the next image; otherwise, carrying out the next step;
5) generating a binary image from the coordinates obtained by ssd prediction, performing dot multiplication and addition on the binary image value c obtained in step 3, and if the obtained value e is not greater than a set threshold th3(sum (e) <th 3), then processing the next picture after proving that the image is not damaged; otherwise, reporting the identification result.
Although the invention herein has been described with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present invention. It is therefore to be understood that numerous modifications may be made to the illustrative embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that features described in different dependent claims and herein may be combined in ways different from those described in the original claims. It is also to be understood that features described in connection with individual embodiments may be used in other described embodiments.
Claims (7)
1. A method for detecting damage faults of a bottom floor of a railway wagon comprises the following steps:
acquiring full-train linear array images at the bottom of a running railway wagon under different conditions;
uniformly splitting each carriage part in the whole linear array image into n gray level images along the length direction of the railway wagon; n is a preset splitting amplitude number;
carrying out image preprocessing on the gray level image, and carrying out data amplification to obtain a sample data set;
constructing a semantic segmentation neural network model based on an encoder and a decoder, and enabling each convolution layer to have a convolution function, a batch standardization function and a Relu activation function;
dividing a sample data set into a training data set and a test data set; predicting a gray level image in a training data set through the semantic segmentation neural network model to obtain an image target region, calculating the image target region and a mark truth value region of the gray level image to obtain a cross entropy loss function loss value, and optimizing the weight of the semantic segmentation neural network model through an optimizer to reduce the cross entropy loss function loss value to a preset threshold value;
predicting the gray level image of the test data set through the trained semantic segmentation neural network model to obtain a target region to be detected, calculating the target region to be detected and a mark truth value region of the corresponding gray level image, further obtaining a cross entropy loss function loss value until the cross entropy loss function loss value reaches a standard loss value, finding an optimal weight coefficient, and updating the semantic segmentation neural network model;
acquiring a current whole-train linear array image at the bottom of a running railway wagon, uniformly splitting each carriage part in the current whole-train linear array image into n current gray level images along the length direction of the railway wagon, and predicting the current gray level images by adopting an updated semantic segmentation neural network model to obtain a current target area to be detected; performing fault pre-judgment on the current target area to be detected, judging the current target area to be detected with a fault, and performing fault detection by adopting a single-stage ssd model;
the method for predicting the current gray level image by adopting the updated semantic segmentation neural network model to obtain the current target area to be detected comprises the following steps:
the updated semantic segmentation neural network model carries out positioning prediction on the current gray level image to obtain a current target area to be detected; after the binarization of the current target area image to be detected, performing negation operation to obtain a binarization target area image to be detected;
it is characterized in that the preparation method is characterized in that,
the method for pre-judging the fault of the current target area to be detected comprises the following steps:
adding the result obtained by performing point multiplication on the binary target area image to be detected and the binary image of the standard positioning algorithm to obtain an image value, if the image value is not greater than a set threshold value, determining that no damage exists in the binary target area image to be detected, and otherwise, performing fault detection by adopting a single-stage ssd model;
a non-fault functional area is marked on the standard positioning algorithm binary image;
and the standard positioning algorithm binary image is obtained after fault training of a segmentation network.
2. The method of detecting breakage of a floor of a railway wagon according to claim 1, wherein the detecting step includes the step of detecting breakage of the floor,
the full linear array image is acquired by a camera or a video camera arranged at the position of the freight car track.
3. The method of detecting breakage of a floor of a railway wagon according to claim 1, wherein the detecting step includes the step of detecting breakage of the floor,
the n is chosen to be 14.
4. The method of detecting breakage of a floor of a railway wagon according to claim 1, wherein the detecting step includes the step of detecting breakage of the floor,
the image preprocessing of the grayscale image comprises:
obtaining a bottom floor binary segmentation image from the gray level image in a manual marking mode, wherein the bottom floor area value in the binary segmentation image is 0, and the rest area values are 255;
obtaining a bottom floor binary fault image from the bottom floor binary segmentation image in a manual marking mode, wherein the value of a damaged area of the bottom floor in the binary fault image is 255, and the values of the rest areas are 0;
determining a damaged rectangular frame of the floor in the binary fault image in a manual marking mode to obtain a damaged XML file of the floor and form an XML file set; and performing at least one of horizontal and vertical turning, zooming, translation, contrast enhancement and brightness change on the XML file with the damaged floor to obtain a sample data set.
5. The method of detecting breakage of the floor of a railway wagon according to claim 4, wherein the detecting step includes the step of detecting breakage of the floor,
the building of the semantic segmentation neural network model comprises the following steps:
constructing an encoder network model, comprising:
the coding unit I: comprises 3 × 3 convolution layers → 3 × 3 convolution layers;
and a second coding unit: comprises 3 × 3 convolution layers → 1 × 1 convolution layers;
and a coding unit III: comprises 3 × 3 convolution layers → 1 × 1 convolution layers;
and a coding unit IV: comprises 3 × 3 convolution layers → 1 × 1 convolution layers;
pyramid pooling unit comprising three parallel layer structures:
one) 1x1 pooling layer → 3 x 3 convolutional layer → 1x1 upsampling layer;
two) 2 x 2 pooling layers → 3 x 3 convolutional layers → 2 x 2 upsampling layers;
three) 4 × 4 pooling layers → 3 × 3 convolutional layers → 4 × 4 upsampling layers;
and the gray level image is processed by a first coding unit, a second coding unit, a third coding unit and a fourth coding unit in sequence, and then is output after being processed by three parallel layer structures of the pyramid pooling unit.
6. The method of detecting breakage of a floor of a railway wagon according to claim 5, wherein the detecting step includes the step of detecting breakage of the floor,
the building of the semantic segmentation neural network model further comprises the following steps:
constructing a decoder network model, comprising:
the decoding unit I: 1x1 convolution layer → 3 x 3 convolution layer → 2 x 2 convolution layer;
and a second decoding unit: 3 x 3 convolutional layer → 2 x 2 upsampling layer;
a third decoding unit: 3 x 3 convolutional layer → 2 x 2 upsampling layer;
and a decoding unit IV: 1 × 1 convolutional layer → 1 × 1 convolutional layer;
and images output by the three parallel layer structures of the pyramid pooling unit are sequentially processed by the first decoding unit, the second decoding unit, the third decoding unit and the fourth decoding unit to obtain an image target area of the gray level image.
7. The method of detecting breakage of a floor of a railway wagon according to claim 6, wherein the detecting step includes the step of detecting breakage of the floor,
carrying out dot multiplication on the to-be-detected binary target area image and a standard positioning algorithm binary image, and then adding to obtain an image fault-free score;
the method for fault detection by adopting the single-stage ssd model comprises the following steps:
comparing the image failure-free score of the to-be-detected binary target area image with a preset score, wherein the image failure-free score is judged to be broken; otherwise, carrying out the next judgment;
obtaining coordinate data according to a to-be-detected binary target area image, obtaining a corresponding binary image of the to-be-detected binary target area image according to the coordinate data, performing point multiplication on the corresponding binary image and the image value, and adding the corresponding binary image and the image value, wherein if the obtained value is not greater than a judgment threshold value, the detection result is that the bottom floor is not damaged; otherwise, the detection result is that the bottom floor is damaged, and the detection result is reported.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010437719.7A CN111652227B (en) | 2020-05-21 | 2020-05-21 | Method for detecting damage fault of bottom floor of railway wagon |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010437719.7A CN111652227B (en) | 2020-05-21 | 2020-05-21 | Method for detecting damage fault of bottom floor of railway wagon |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111652227A CN111652227A (en) | 2020-09-11 |
| CN111652227B true CN111652227B (en) | 2021-01-22 |
Family
ID=72346737
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010437719.7A Active CN111652227B (en) | 2020-05-21 | 2020-05-21 | Method for detecting damage fault of bottom floor of railway wagon |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111652227B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP4047550A1 (en) * | 2021-02-23 | 2022-08-24 | Hitachi, Ltd. | Train inspection system |
| JP7470864B2 (en) | 2021-02-23 | 2024-04-18 | 株式会社日立製作所 | Train inspection system and method |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112257711B (en) * | 2020-10-26 | 2021-04-09 | 哈尔滨市科佳通用机电股份有限公司 | Method for detecting damage fault of railway wagon floor |
| CN112330636B (en) * | 2020-11-06 | 2021-07-06 | 哈尔滨市科佳通用机电股份有限公司 | Method for detecting damage of railway wagon wheels based on image processing |
| CN112488049B (en) * | 2020-12-16 | 2021-08-24 | 哈尔滨市科佳通用机电股份有限公司 | Fault identification method for foreign matter clamped between traction motor and shaft of motor train unit |
| CN112669301B (en) * | 2020-12-31 | 2022-06-03 | 哈尔滨市科佳通用机电股份有限公司 | High-speed rail bottom plate paint removal fault detection method |
| CN112712516B (en) * | 2021-01-08 | 2021-10-15 | 哈尔滨市科佳通用机电股份有限公司 | High-speed rail bottom rubber strip fault detection method and system based on YOLOv5 |
| CN114674838B (en) * | 2022-03-18 | 2022-11-25 | 哈尔滨市科佳通用机电股份有限公司 | Automatic detection system and detection method for motor train unit vehicle bottom |
| CN115424129B (en) * | 2022-10-13 | 2023-08-11 | 哈尔滨市科佳通用机电股份有限公司 | Abnormality detection method and abnormality detection system for wallboard damage |
| CN115719475B (en) * | 2022-10-24 | 2023-09-19 | 北京交通大学 | Three-stage trackside equipment fault automatic detection method based on deep learning |
| CN116051682A (en) * | 2022-11-30 | 2023-05-02 | 四川省中车铁投轨道交通有限公司 | Intelligent tramcar chassis fault detection method and system |
| CN116310734B (en) * | 2023-04-25 | 2023-12-15 | 慧铁科技股份有限公司 | Fault detection method and system for railway wagon running part based on deep learning |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102654464A (en) * | 2012-03-13 | 2012-09-05 | 浙江工业大学 | Copper strip surface defect detection system based on multi-feature fuzzy recognition |
| CN106338520A (en) * | 2016-09-18 | 2017-01-18 | 南京林业大学 | Recognition method of surface defects of multilayer solid wood composite floor with surface board being jointed board |
| CN108126987A (en) * | 2017-05-04 | 2018-06-08 | 汉威广园(广州)机械设备有限公司 | A kind of high-speed rod-rolling mill remote online monitoring and intelligent diagnosis system and method |
| CN108876781A (en) * | 2018-06-26 | 2018-11-23 | 广东工业大学 | Surface defect recognition method based on SSD algorithm |
| CN109544522A (en) * | 2018-11-12 | 2019-03-29 | 北京科技大学 | A kind of Surface Defects in Steel Plate detection method and system |
| CN110738663A (en) * | 2019-09-06 | 2020-01-31 | 上海衡道医学病理诊断中心有限公司 | Double-domain adaptive module pyramid network and unsupervised domain adaptive image segmentation method |
| CN111079627A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Railway wagon brake beam body breaking fault image identification method |
| CN111079628A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Floor damage fault image identification method |
| CN111079734A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Method for detecting foreign matters in triangular holes of railway wagon |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10648925B2 (en) * | 2017-06-05 | 2020-05-12 | Kla-Tencor Corporation | Repeater defect detection |
-
2020
- 2020-05-21 CN CN202010437719.7A patent/CN111652227B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102654464A (en) * | 2012-03-13 | 2012-09-05 | 浙江工业大学 | Copper strip surface defect detection system based on multi-feature fuzzy recognition |
| CN106338520A (en) * | 2016-09-18 | 2017-01-18 | 南京林业大学 | Recognition method of surface defects of multilayer solid wood composite floor with surface board being jointed board |
| CN108126987A (en) * | 2017-05-04 | 2018-06-08 | 汉威广园(广州)机械设备有限公司 | A kind of high-speed rod-rolling mill remote online monitoring and intelligent diagnosis system and method |
| CN108876781A (en) * | 2018-06-26 | 2018-11-23 | 广东工业大学 | Surface defect recognition method based on SSD algorithm |
| CN109544522A (en) * | 2018-11-12 | 2019-03-29 | 北京科技大学 | A kind of Surface Defects in Steel Plate detection method and system |
| CN110738663A (en) * | 2019-09-06 | 2020-01-31 | 上海衡道医学病理诊断中心有限公司 | Double-domain adaptive module pyramid network and unsupervised domain adaptive image segmentation method |
| CN111079627A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Railway wagon brake beam body breaking fault image identification method |
| CN111079628A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Floor damage fault image identification method |
| CN111079734A (en) * | 2019-12-12 | 2020-04-28 | 哈尔滨市科佳通用机电股份有限公司 | Method for detecting foreign matters in triangular holes of railway wagon |
Non-Patent Citations (2)
| Title |
|---|
| A two-stage attention aware method for train bearing shed oil inspection based on convolutional neural networks;Xiao Fu等;《Neurocomputing》;20200307;第212–224页 * |
| 基于卷积神经网络的晶圆缺陷检测与分类算法;邡鑫等;《计算机工程》;20180830;第218-223页 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP4047550A1 (en) * | 2021-02-23 | 2022-08-24 | Hitachi, Ltd. | Train inspection system |
| WO2022179786A1 (en) * | 2021-02-23 | 2022-09-01 | Hitachi, Ltd. | Train inspection system |
| JP7470864B2 (en) | 2021-02-23 | 2024-04-18 | 株式会社日立製作所 | Train inspection system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111652227A (en) | 2020-09-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111652227B (en) | Method for detecting damage fault of bottom floor of railway wagon | |
| Wu et al. | Hybrid deep learning architecture for rail surface segmentation and surface defect detection | |
| CN111079747B (en) | Railway wagon bogie side frame fracture fault image identification method | |
| CN111091555B (en) | Brake shoe breaking target detection method | |
| CN111080650B (en) | Method for detecting looseness and loss faults of small part bearing blocking key nut of railway wagon | |
| CN109766746B (en) | Track foreign matter detection method for aerial video of unmanned aerial vehicle | |
| CN111079734B (en) | Method for detecting foreign matters in triangular holes of railway wagon | |
| CN115272334A (en) | Method for detecting micro defects on surface of steel rail under complex background | |
| CN111079746B (en) | Railway wagon axle box spring fault image identification method | |
| CN111652295B (en) | Railway wagon coupler yoke key joist falling fault identification method | |
| CN111080609A (en) | Brake shoe bolt loss detection method based on deep learning | |
| CN111080621B (en) | Method for identifying railway wagon floor damage fault image | |
| CN111080600A (en) | Fault identification method for split pin on spring supporting plate of railway wagon | |
| CN111080602A (en) | Method for detecting foreign matters in water leakage hole of railway wagon | |
| CN111079748A (en) | Method for detecting oil throwing fault of rolling bearing of railway wagon | |
| CN112967252A (en) | Rail vehicle machine sense hanger assembly bolt loss detection method | |
| CN115527170A (en) | Method and system for identifying closing fault of door stopper handle of automatic freight car derailing brake device | |
| CN111652228B (en) | Railway wagon sleeper beam hole foreign matter detection method | |
| CN111080599A (en) | Fault identification method for hook lifting rod of railway wagon | |
| CN113657305B (en) | Video-based intelligent detection method for black smoke vehicle and ringeman blackness level | |
| CN108648210B (en) | Rapid multi-target detection method and device under static complex scene | |
| CN113221839A (en) | Automatic truck image identification method and system | |
| CN115424128A (en) | Fault image detection method and system for lower link of freight car bogie | |
| CN112614094B (en) | Insulator string abnormity positioning and identifying method based on sequence state coding | |
| Ashraf et al. | Efficient Pavement Crack Detection and Classification Using Custom YOLOv7 Model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |