TW202209256A - Method and device for defect detection, electronic device, and computer storage medium - Google Patents

Abstract

The invention provides a defect detection method and device, electronic equipment and a computer storage medium. The method comprises the following steps: obtaining a to-be-detected image of a high-speed rail overhead line system; segmenting an image of a first part of the high-speed rail overhead line system from the to-be-detected image; segmenting an image of a second part of the high-speed rail overhead line system from the image of the first part, wherein the second part is a sub-part of the first part; and performing defect classification on the first part and the second part based on the image of the first part and the image of the second part to obtain a defect classification result. According to the embodiment of the invention, defect detection is carried out on the high-speed rail overhead line system in a cascade mode, the omission ratio of defect detection of the high-speed rail overhead line system is reduced, and the detection accuracy is improved.

Description

Defect detection method, device, electronic device and computer storage medium

The present application relates to the field of computer vision technology, in particular to a defect detection method, device, electronic device and computer storage medium.

With the continuous development of high-speed railway construction, the requirements for the safety and reliability of the high-speed railway power supply system are gradually increasing, and the safety inspection and maintenance of the high-speed railway catenary network is particularly important. The popularization of catenary suspension monitoring devices has greatly improved the efficiency of image acquisition in the operation and maintenance of catenary. However, in the face of a large amount of image data, the traditional method of detecting defects in catenary by manual inspection is not only time-consuming It is long, low in efficiency, and has the problem of high missed detection rate, resulting in low accuracy of catenary defect detection.

In view of the above problems, the present application provides a defect detection method, device, electronic equipment and storage medium, which are beneficial to reduce the missed detection rate of high-speed rail catenary defect detection and improve the accuracy of catenary defect detection.

In order to achieve the above purpose, the first aspect of the embodiment of the present application provides a defect detection method, the method includes:

Obtain the image to be inspected of the high-speed rail catenary;

Segment the image of the first component of the high-speed rail catenary from the to-be-detected image;

Segment the image of the second part of the high-speed rail catenary from the image of the first part; the second part is a sub-part of the first part;

Defect classification is performed on the first part and the second part based on the image of the first part and the image of the second part, and a defect classification result is obtained.

In combination with the first aspect, in a possible implementation manner, the image of the first component of the high-speed rail contact net is segmented from the to-be-detected image, including:

Perform feature extraction on the to-be-detected image to obtain a first feature map;

Locating and classifying the first component based on the first feature map, to obtain a first rectangular detection frame of the first component;

The image of the first component is segmented from the to-be-detected image according to the first rectangular detection frame.

In combination with the first aspect, in a possible implementation, the first component is located and classified based on the first feature map to obtain a first rectangular detection frame of the first component, including:

Predicting the coordinates of the candidate region of the first component and classifying the front and the background on the first feature map, to determine the foreground target of the first component;

performing pooling processing on the corresponding feature of the foreground target of the first component in the first feature map to obtain a first pooling feature;

Classify the first part based on the first pooling feature to obtain the category of the first part and the first rectangular detection frame.

In combination with the first aspect, in a possible implementation manner, the image of the second component of the high-speed rail contact net is segmented from the image of the first component, including:

performing gamma correction on the image of the first part to obtain the image to be segmented of the first part;

The image of the second part is segmented from the image to be segmented of the first part.

With reference to the first aspect, in a possible implementation manner, segmenting the image of the second part from the image to be segmented of the first part includes:

performing feature extraction on the to-be-segmented image of the first component to obtain a second feature map;

Locating and classifying the second component based on the second feature map, to obtain a second rectangular detection frame of the second component;

The image of the second part is segmented from the image to be segmented of the first part according to the second rectangular detection frame.

In combination with the first aspect, in a possible implementation, the second component is located and classified based on the second feature map to obtain a second rectangular detection frame of the second component, including:

On the second feature map, the candidate region coordinate prediction and front and background classification of the second component are performed, and the foreground target of the second component is determined;

Perform pooling processing on the features corresponding to the foreground target of the second component in the second feature map to obtain second pooling features;

Classify the second part based on the second pooling feature to obtain the category of the second part and the second rectangular detection frame.

In combination with the first aspect, in a possible implementation manner, the first part and the second part are subjected to defect classification based on the image of the first part and the image of the second part to obtain a defect classification result, include:

respectively perform feature extraction on the image of the first component and the image of the second component;

Predict the defect type of the first part based on the features extracted from the image of the first part, and obtain the defect classification result of the first part; the defect classification result of the first part includes the defect type of the first part and the defect classification result of the first part. the class of the first part; and

Predict the defect type of the second part based on the features extracted from the image of the second part, and obtain the defect classification result of the second part; the defect classification result of the second part includes the defect type of the second part and the defect classification result of the second part. The category of the second part.

In combination with the first aspect, in a possible implementation manner, after performing defect classification on the first part and the second part based on the image of the first part and the image of the second part, and obtaining a defect classification result , the method includes:

According to the defect classification result, defect early warning is carried out.

With reference to the first aspect, in a possible implementation, the defect early warning is performed according to the defect classification result, including:

For the first part, output the defect type of the first part, the position of the first part in the image to be inspected, the type of the first part, and the high-speed rail line where the first part is located, so as to perform defect warning;

For the second part, determine the target first part to which the second part belongs, and output the defect type of the second part, the position of the second part in the image to be inspected, and the class of the second part identification, the position of the target first part in the to-be-detected image, the type of the target first part and the high-speed rail line where the second part is located, for defect warning.

With reference to the first aspect, in a possible implementation manner, the acquired image of the high-speed rail catenary to be detected includes:

Obtain the original image of the high-speed rail catenary collected by the imaging equipment;

The original image of the high-speed rail catenary is filtered to obtain the image to be detected.

A second aspect of the embodiment of the present application provides a defect detection device, the device comprising:

The image acquisition module is used to acquire the image to be detected of the high-speed rail catenary;

a first detection module for segmenting the image of the first component of the high-speed rail contact net from the to-be-detected image;

The second detection module is used to segment the image of the second part of the high-speed rail contact net from the image of the first part; the second part is a sub-component of the first part;

The defect classification module is used for classifying defects of the first part and the second part based on the image of the first part and the image of the second part, and obtaining a defect classification result.

A third aspect of the embodiment of the present application provides an electronic device, the electronic device includes an input device and an output device, further includes a processor, and is suitable for implementing one or more instructions; and a computer storage medium, the computer storage medium stores a One or more instructions adapted to be loaded by the processor to perform the steps of any of the implementations of the first aspect above.

A fourth aspect of the embodiment of the present application provides a computer storage medium, the computer storage medium stores one or more instructions, and the one or more instructions are suitable for being loaded by a processor and executing any one of the implementations of the first aspect above steps in the method.

It can be seen that in the embodiment of the present application, the image to be detected of the high-speed rail contact net is obtained; the image of the first part of the high-speed rail contact net is segmented from the to-be-detected image; and the image of the first part is segmented. The image of the second part of the high-speed rail contact net; the second part is a sub-part of the first part; based on the image of the first part, the image of the second part, the first part, the second part Parts are classified into defects, and the result of defect classification is obtained. In this way, the first-level component (ie, the first component) is detected on the image to be detected of the high-speed rail contact net, the image of the first-level component is segmented from the to-be-detected image of the high-speed railway contact line, and then the image of the first-level component is subjected to the second-level detection. Parts (ie, the second part) are inspected to identify the second-level parts on the first-level parts, segment the images of the second-level parts, and use the images of the first-level parts and the images of the second-level parts for defect classification, realizing the first-class part. Linked high-speed rail catenary defect detection is beneficial to reduce the missed detection rate of high-speed rail catenary defect detection, thereby improving the accuracy of catenary defect detection. At the same time, it is also beneficial to reduce the cost of manual inspection, shorten the detection time, and improve the detection efficiency.

S11, S12, S13, S14, S61, S62, S63, S64, S65, S66, S67: Steps

81: Image acquisition module

82: The first detection module

83: The second detection module

84: Defect classification module

85: Defect warning module

1001: Processor

1002: Input device

1003: Output device

1004: Computer Storage Media

In order to more clearly illustrate the technical solutions in the embodiments of the present application or in the prior art, the following briefly introduces the diagrams that need to be used in the description of the embodiments or the prior art. Obviously, the diagrams in the following description are only the For some embodiments of the application, for those with ordinary knowledge in the technical field, other diagrams can also be obtained according to these diagrams without any creative effort.

1 is a schematic flowchart of a defect detection method provided by an embodiment of the present application;

2 is a schematic diagram of an application environment for defect detection of a high-speed rail catenary provided by an embodiment of the present application;

3 is a schematic diagram of filtering an original image of a high-speed rail catenary according to an embodiment of the present application;

4 is a schematic structural diagram of a defect detection model for a high-speed rail catenary provided by an embodiment of the present application;

5A is a schematic diagram of dividing a first component according to an embodiment of the present application;

5B is a schematic diagram of dividing a second component according to an embodiment of the present application;

6 is a schematic flowchart of another defect detection method provided by an embodiment of the present application;

7 is a schematic diagram of generating a candidate region based on a feature map according to an embodiment of the present application;

FIG. 8 is a schematic structural diagram of a defect detection device provided by an embodiment of the present application;

FIG. 9 is a schematic structural diagram of another defect detection device provided by an embodiment of the present application;

FIG. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the illustrations in the embodiments of the present application. Obviously, the described embodiments are only The embodiments are part of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by persons with ordinary knowledge in the technical field without creative work shall fall within the protection scope of this application.

The terms "comprising" and "having" and any variations thereof appearing in the specification, scope and drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally includes unlisted steps or units, or optionally includes For other steps or units inherent to these processes, methods, products or devices. Furthermore, the terms "first," "second," and "third," etc. are used to distinguish between different items, and are not used to describe a particular order.

The embodiment of the present application proposes a defect detection scheme for a high-speed rail catenary, so as to reduce the missed detection rate of defect detection of the high-speed rail catenary and improve the accuracy of defect detection. In the specific implementation, based on The deep learning high-speed rail catenary defect detection model first locates the first-level components from the images of the high-speed rail catenary to be inspected, and then locates the second-level components that have a cascading relationship with the first-level components from the images of the first-level components, which is conducive to reducing The missed detection rate of parts, using the images of the first-level parts and the images of the second-level parts to predict the defect types of the first-level parts and the second-level parts, and finally output the specific location of the defect, the defect type, the part to which the defect belongs, The entire inspection process is in a tree-like structure for the superior components of the component to which the defect belongs and the specific line section where the defect is located. The structured information between components is helpful for the operation and maintenance personnel to quickly determine the defect location and defect route, so as to carry out the maintenance work of the catenary , in order to ensure the safe operation of high-speed rail, and at the same time, it is conducive to the expansion of new components and new defects.

The defect detection method provided by the embodiments of the present application will be described in detail below with reference to the relevant figures.

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of a defect detection method provided by an embodiment of the application. The defect detection method is applied to a server, for example, a server, a computer, and a computer equipped with a deep learning-based high-speed rail catenary defect detection model. The host, cloud server, etc., as shown in Figure 1, includes steps S11-S14:

S11, acquiring an image to be detected of the high-speed rail catenary.

In the specific embodiment of the present application, as shown in FIG. 2 , the high-speed rail inspection vehicle usually operates at night. The inspection vehicle is equipped with high-definition imaging equipment and on-board sensors. The inspection vehicle travels on the high-speed rail line. When the detector detects the pillars on both sides of the route, it triggers the imaging device to collect images of the catenary to obtain the original image of the high-speed rail catenary. When the sensor detects that there are pillars within the preset range, it triggers two sets of imaging equipment to image the front and back and overall layout of the supporting parts and suspension parts of the catenary, thus obtaining a large number of high-speed rail catenary from different angles. The original image. The resolution of the original image of the high-speed rail catenary usually has a better value, for example For example: 6576*4384 pixels, but due to environmental factors such as night work and foggy, there will still be images with lower resolution in the original images of the high-speed rail catenary collected, for example: the resolution length and width are lower than 2000 pixels, Therefore, as shown in Figure 3, it is necessary to filter the collected original images of the high-speed rail catenary, and select the original images of the high-speed rail catenary whose resolution length and width reach the preset pixel value as the image to be detected for subsequent defect detection. Filter out the original image of the high-speed rail catenary whose resolution length and width are lower than the preset pixel value.

S12, segment the image of the first component of the high-speed rail catenary from the to-be-detected image.

In the specific embodiment of the present application, a pre-trained deep learning-based high-speed rail catenary defect detection model is used to perform defect detection on each component in the to-be-detected image obtained in step S11, and the high-speed rail catenary defect detection model includes a first component The detector, the second part detector, the defect classifier and the defect early warning module, as shown in Figure 4, the input of the first part detector is the image to be inspected, which is used to detect the high-speed rail catenary from the image to be inspected The first parts of the equipment, such as: column top cover, insulator, ring rod-right-angle hanging plate joint, arm wrist base, AF wire shoulder frame base, contact wire center anchor clamp, drop weight limit frame, etc., the second part detects The detector is used to detect the second part on the first part from the image of the first part output by the first part detector, such as: bolts, nuts, split pins, etc. at both ends of the insulator, and the output is the image of the second part , the defect classifier is used to classify the defects of the first part according to the image of the first part, and classify the defects of the second part according to the image of the second part, and the defect early warning module is used to classify the defects according to the output of the defect classifier. Carry out defect early warning, and the output includes the location of the defect, the type of defect (such as the angle of the cotter pin on the arm and wrist base is not in place), the superior component of the component to which the defect belongs, and the high-speed rail line where the defect is located. Optionally, the first component detector may be a two-stage detector or a one-stage detector. The two-stage detector generates candidate regions based on the feature map extracted from the image to be detected, and then evaluates the candidate regions. The region is classified and predicted, and the category of the first part and the coordinates of the rectangular detection frame are obtained. The coordinates of the rectangular detection frame can be the coordinates of the upper left corner and the lower right corner, or the coordinates of the center point, length and width, etc., which are not limited. As shown in FIG. 5A , an image of the first component, such as an insulator, an arm-wrist base, etc., is segmented from the image to be detected according to a rectangular detection frame. The one-stage detector does not need to generate candidate regions. It directly classifies and predicts the input image to be detected, obtains the category of the first part and the coordinates of the rectangular detection frame, and then divides the image of the first part according to the rectangular detection frame. picture. Optionally, the first component detector is trained using a sample image of the high-speed rail catenary, the first component in the sample image has a class label, and the first component is detected by a preset loss function during the training process. to tune the device.

S13, segment the image of the second part of the high-speed rail contact network from the image of the first part; the second part is a sub-part of the first part.

In the specific embodiment of this application, the second component refers to a sub-component on the first component, and the two have a cascade relationship. The image of the first part segmented in step S12 is detected, and the possibility of missed detection is high. Therefore, it is necessary to perform gamma correction on the image of the first part to improve the image quality and obtain the image of the first part. The to-be-segmented image (that is, the image obtained after gamma correction) is then segmented from the to-be-segmented image by the second part detector to obtain an image of the second part of the high-speed rail contact network. Optionally, the second part detector can be the same as the first part detector, or it can be different, it can be trained together with the first part detector, or can be trained separately, in the same way, the category and rectangle of the second part can be obtained. After the frame is detected, as shown in FIG. 5B , the image of the second part is segmented from the image of the first part according to the rectangular detection frame.

S14: Perform defect classification on the first part and the second part based on the image of the first part and the image of the second part, and obtain a defect classification result.

In a specific embodiment of the present application, optionally, the above-mentioned defect classification is performed on the first part and the second part based on the image of the first part and the image of the second part to obtain a defect classification result, including:

respectively perform feature extraction on the image of the first component and the image of the second component;

Predict the defect type of the first part based on the features extracted from the image of the first part, and obtain the defect classification result of the first part; the defect classification result of the first part includes the defect type of the first part and the defect classification result of the first part. the class of the first part; and

Predict the defect type of the second part based on the features extracted from the image of the second part, and obtain the defect classification result of the second part; the defect classification result of the second part includes the defect type of the second part and the defect classification result of the second part. The category of the second part.

Please continue to refer to Figure 4. After obtaining the image of the first part and the image of the second part, they are input into the defect classifier for probability prediction of defect types. Specifically, the feature of the image of the first part and the feature of the image of the second part are extracted by the backbone network of the defect classifier. The backbone network mainly performs convolution processing, and then based on the extracted features, the The feature is input to the fully connected layer for probability prediction of defect types, and the defect type with the highest probability is taken as the defect type of the part. If the probability of cracks in the pendulum restraining frame reaches 95% (the highest), the defect type of the pendant restraining frame is the existence of cracks. Of course, in addition to the defect type of the part, the final output of the defect classifier also has the class index of the part and the coordinates of the rectangular detection frame, for example: c05, if there is a crack, c05 means the class index of the part, among which, the class index of the first part The class index of the second part can be determined when the class of the first part is obtained by the first part detector, and the class index of the second part can be determined when the class of the second part is obtained by the second part detector. Wherein, the defect classifier can be trained together with the first part detector, can also be trained together with the second part detector, or can be trained separately.

Optionally, after performing defect classification on the first part and the second part based on the image of the first part and the image of the second part, and obtaining a defect classification result, the method further includes: classifying according to the defect. The result is a defect warning.

In the specific embodiment of the present application, the defect early warning according to the defect classification result includes:

For the first part, output the defect type of the first part, the position of the first part in the image to be inspected, the type of the first part, and the high-speed rail line where the first part is located, so as to perform defect warning;

For the second part, determine the target first part to which the second part belongs, and output the defect type of the second part, the position of the second part in the image to be inspected, the type of the second part, and the target The position of the first part in the image to be inspected, the type of the target first part, and the high-speed rail line where the second part is located, for defect warning.

Specifically, the target first part refers to the upper-level part of the second part. The input of the defect warning module includes the defect classification results of the first part and the second part, the coordinates of the rectangular detection frame of the first part and the second part, and the output can be The defect type of the part, the position of the part in the image to be inspected, the category of the part, the position of the superior part to which the part belongs in the image to be inspected, the class of the superior part of the part, and the high-speed rail line where the part is located. Optionally, the position of the component in the image to be detected and the position of the upper-level component of the component in the image to be detected are presented in the rectangular detection frame of the component, and the category of the component and the category of the upper-level component of the component are presented with the category index. , the high-speed rail line where the component is located can be output according to the logo carried when the imaging device uploads the original image of the high-speed rail catenary. The identification will exist in the entire defect detection process. Of course, this is only an example, and does not limit the embodiments of the present application.

In addition, since the first part does not have a parent part to which it belongs, the early warning information output by the first part includes the defect type of the first part (such as the nut on the AF wire shoulder frame falling off), the first part in the image to be detected position (such as the rectangular detection frame of the base of the AF cable shoulder), the class of the first part Different from the high-speed rail line where the first component is located. The second part usually has an upper-level part. Therefore, it is necessary to determine the target first part to which the second part belongs, and output the defect type of the second part itself, the position of the second part in the image to be inspected and other information, and also need to output The category of the first part of the target, the position in the image to be detected, etc.

Further, determining the target first component to which the second component belongs includes:

Obtain the intersection of the second rectangular detection frame of the second component and all the first rectangular detection frames of the first component;

obtaining the ratio between the intersection and the second rectangular detection frame;

The target first component to which the second component belongs is determined according to the ratio between the second rectangular detection frames.

Specifically, the first rectangular detection frame refers to the bounding box regression of the first component, and the second rectangular detection frame refers to the bounding box regression of the second component. For the currently detected positioning hook (belonging to the second component), it is assumed that its second The rectangular detection frame is A, and the first rectangular detection frames of all the first components are B1, B2, B3...Bn. First obtain the second rectangular detection frame A and all the first rectangular detection frames B1, B2, B3.. .Bn intersection C1, C2, C3...Cn, and then calculate the ratios C1/A, C2/A, C3/A... Cn/A, if C1/A is the maximum value, the first part corresponding to the first rectangular detection frame B1 is taken as the target first part to which the positioning hook belongs. In this embodiment, since the defect early warning module inputs the defect classification results of the first part and the second part, and the coordinates of the rectangular detection frame of the first part and the second part, it can be determined according to the difference between the second part and the first part. The ratio of the intersection of the rectangular detection frame and the rectangular detection frame of the second component to determine the upper-level component of the second component does not affect the overall detection time, and the accuracy meets the requirements.

In the embodiment of the present application, the image to be detected of the high-speed rail catenary is obtained; the image of the first part of the high-speed rail catenary is segmented from the to-be-detected image; the high-speed rail catenary is segmented from the image of the first part image of the second part of the , get the defect classification result. In this way, the first-level components are detected on the images of the high-speed rail catenary to be detected, the images of the first-level components are segmented from the to-be-detected images of the high-speed rail contact network, and then the images of the first-level components are detected for the second-level components to identify The secondary part on the primary part is extracted, the image of the secondary part is segmented, and the image of the primary part and the image of the secondary part are used for defect classification, which realizes the cascading high-speed rail catenary defect detection, which is beneficial to Reduce the missed detection rate of high-speed rail catenary defect detection, thereby improving the accuracy of catenary defect detection. At the same time, it is also beneficial to reduce the cost of manual inspection, shorten the detection time, and improve the detection efficiency.

Please refer to FIG. 6, which is a schematic flowchart of another defect detection method provided by an embodiment of the present application, as shown in FIG. 6, including steps S61-S67:

S61, acquiring the image to be detected of the high-speed rail catenary;

S62, perform feature extraction on the to-be-detected image to obtain a first feature map;

S63, locating and classifying the first component of the high-speed rail catenary based on the first feature map, to obtain a first rectangular detection frame of the first component;

S64, segment the image of the first component from the to-be-detected image according to the first rectangular detection frame;

S65, segment the image of the second part of the high-speed rail contact net from the image of the first part; the second part is a sub-component of the first part;

S66, perform defect classification on the first part and the second part based on the image of the first part and the image of the second part, to obtain a defect classification result;

S67, perform a defect early warning according to the defect classification result.

In the specific embodiment of the present application, for the image to be detected input to the first component detector, feature extraction is performed on the image by the backbone network of the first component detector. The backbone network mainly performs convolution processing, and the output features The figure is the above-mentioned first feature map. On the first feature map, the coordinates of the candidate region of the first component are predicted to generate a candidate region as shown in Figure 7. The front and background classifications are performed in the candidate region to obtain the first The foreground target of the component, and then the corresponding feature of the foreground target in the first feature map is pooled, and the output feature is the above-mentioned first pooling feature, and the first pooling feature is input to the fully connected layer for finalization. The classification of the first part in the image to be detected and the rectangular detection frame (that is, the first rectangular detection frame) are output, and the image of the first part is segmented from the image to be detected according to the first rectangular detection frame as a defect. The input to the classifier. In this embodiment, the first part detector based on the candidate region is used to classify the first part in the image to be detected, and the accuracy is higher.

Optionally, the image of the second component of the high-speed rail contact net is segmented from the image of the first component, including:

performing gamma correction on the image of the first part to obtain the image to be segmented of the first part;

The image of the second part is segmented from the image to be segmented of the first part.

In this embodiment, performing gamma correction on the image of the first component can obtain an image to be segmented with better quality, which is beneficial to overcome the influence of poor light on detection, so as to accurately segment the image of the second component , reducing the missed detection rate of the second component.

Optionally, segmenting the image of the second component from the to-be-segmented image of the first component includes:

performing feature extraction on the to-be-segmented image of the first component to obtain a second feature map;

Locating and classifying the second component based on the second feature map, to obtain a second rectangular detection frame of the second component;

The image of the second part is segmented from the image to be segmented of the first part according to the second rectangular detection frame.

Optionally, the second component is located and classified based on the second feature map to obtain a second rectangular detection frame of the second component, including:

On the second feature map, the candidate region coordinate prediction and front and background classification of the second component are performed, and the foreground target of the second component is determined;

Perform pooling processing on the features corresponding to the foreground target of the second component in the second feature map to obtain second pooling features;

Classify the second part based on the second pooling feature to obtain the category of the second part and the second rectangular detection frame.

In the specific embodiment of this application, the second feature map refers to the feature map extracted by the second component detector from the sub-graph of the first component through the backbone network, and the second pooled feature refers to the second component detector's detection of the first component. The features obtained by pooling the corresponding features of the two-component foreground target in the second feature map. It should be noted that the processing procedure of the second component detector is the same as that of the first component detector, and the category prediction of the second component is also performed based on the generated candidate area, and the category of the second component and the second rectangular detection frame are output.

The specific implementations of the above steps S61 to S67 have been described in the embodiment shown in FIG. 1 , and can achieve the same or similar beneficial effects, and will not be repeated here.

Based on the description of the method embodiment shown in FIG. 1 or FIG. 6 , an embodiment of the present application further provides a defect detection apparatus, please refer to FIG. 8 , and FIG. 8 is a schematic structural diagram of a defect detection apparatus provided by an embodiment of the present application, as shown in FIG. 8 As shown, the device includes:

The image acquisition module 81 is used to acquire the image to be detected of the high-speed rail catenary;

The first detection module 82 is used to segment the image of the first component of the high-speed rail contact net from the to-be-detected image;

The second detection module 83 is used to segment the image of the second part of the high-speed rail contact net from the image of the first part; the second part is a sub-component of the first part;

The defect classification module 84 is configured to perform defect classification on the first part and the second part based on the image of the first part and the image of the second part, and obtain a defect classification result.

In a possible implementation manner, the first detection module 82 is specifically used to:

Perform feature extraction on the to-be-detected image to obtain a first feature map;

Locating and classifying the first component based on the first feature map, to obtain a first rectangular detection frame of the first component;

The image of the first component is segmented from the to-be-detected image according to the first rectangular detection frame.

In a possible implementation manner, after locating and classifying the first component based on the first feature map to obtain a first rectangular detection frame of the first component, the first detection module 82 is specifically used for:

Predicting the coordinates of the candidate region of the first component and classifying the front and the background on the first feature map, to determine the foreground target of the first component;

performing pooling processing on the corresponding feature of the foreground target of the first component in the first feature map to obtain a first pooling feature;

Classify the first part based on the first pooling feature to obtain the category of the first part and the first rectangular detection frame.

In a possible implementation manner, the second detection module 83 is specifically used to:

performing gamma correction on the image of the first part to obtain the image to be segmented of the first part;

The image of the second part is segmented from the image to be segmented of the first part.

In a possible implementation manner, in segmenting the image aspect of the second part from the to-be-segmented image of the first part, the second detection module 83 is specifically used for:

performing feature extraction on the to-be-segmented image of the first component to obtain a second feature map;

Locating and classifying the second component based on the second feature map, to obtain a second rectangular detection frame of the second component;

The image of the second part is segmented from the image to be segmented of the first part according to the second rectangular detection frame.

In a possible implementation manner, the second component is located and classified based on the second feature map to obtain a second rectangular detection frame of the second component, and the second detection module 83 is specifically used for:

On the second feature map, the candidate region coordinate prediction and front and background classification of the second component are performed, and the foreground target of the second component is determined;

Perform pooling processing on the features corresponding to the foreground target of the second component in the second feature map to obtain second pooling features;

Classify the second part based on the second pooling feature to obtain the category of the second part and the second rectangular detection frame.

In a possible implementation manner, performing defect classification on the first part and the second part based on the image of the first part and the image of the second part to obtain a form of the defect classification result, the defect classification model Group 84 is specifically used for:

respectively perform feature extraction on the image of the first component and the image of the second component;

Predict the defect type of the first part based on the features extracted from the image of the first part, and obtain the defect classification result of the first part; the defect classification result of the first part includes the defect type of the first part and the defect classification result of the first part. the class of the first part; and

Predict the defect type of the second part based on the features extracted from the image of the second part, and obtain the defect classification result of the second part; the defect classification result of the second part includes the defect type of the second part and the defect classification result of the second part. The category of the second part.

In a possible implementation, as shown in Figure 9, the device further includes a defect early warning module 85; the defect early warning module 85 is specifically used for:

According to the defect classification result, defect early warning is carried out.

In a possible implementation, when performing a defect early warning mode according to the defect classification result, the defect early warning module 85 is specifically used for:

For the first part, output the defect type of the first part, the position of the first part in the image to be inspected, the type of the first part, and the high-speed rail line where the first part is located, so as to perform defect warning;

For the second part, determine the target first part to which the second part belongs, and output the defect type of the second part, the position of the second part in the image to be inspected, the type of the second part, and the target The position of the first part in the image to be inspected, the type of the target first part, and the high-speed rail line where the second part is located, for defect warning.

In a possible implementation manner, in acquiring the to-be-detected image aspect of the high-speed rail catenary, the image acquisition module 81 is specifically used for:

Obtain the original image of the high-speed rail catenary collected by the imaging equipment;

The original image of the high-speed rail catenary is filtered to obtain the image to be detected.

According to an embodiment of the present application, each unit in the defect detection apparatus shown in FIG. 8 or FIG. 9 may be respectively or all merged into one or several other units to form, or some unit(s) may also be It is further divided into multiple units with smaller functions, which can realize the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit may also be implemented by multiple units, or the functions of multiple units may be implemented by one unit. In other embodiments of the present application, the defect-based detection device may also include other units, and in practical applications, these functions may also be implemented with the assistance of other units, and may be implemented by cooperation of multiple units.

According to another embodiment of the present application, by including a central processing unit (CPU), random access memory (RAM), read-only memory (ROM) A computer program (including program code) capable of executing the steps involved in the corresponding method as shown in FIG. 1 or FIG. 6 is executed on a general-purpose computing device such as a computer, such as a processing element and a memory element, to construct FIG. 8 or FIG. Defect detection device equipment shown in 9 to And to realize the defect detection method of the embodiment of the present application. The computer program can be recorded on, for example, a computer-readable recording medium, and loaded into the above-mentioned computing device via the computer-readable recording medium, and executed therein.

Based on the descriptions of the foregoing method embodiments and apparatus embodiments, the embodiments of the present application further provide an electronic device. Referring to FIG. 10 , the electronic device at least includes a processor 1001 , an input device 1002 , an output device 1003 and a computer storage medium 1004 . The processor 1001 , the input device 1002 , the output device 1003 and the computer storage medium 1004 in the electronic device can be connected by bus bars or other means.

The computer storage medium 1004 can be stored in the memory of the electronic device. The computer storage medium 1004 is used for storing a computer program, and the computer program includes program instructions. The processor 1001 is used for executing the program instructions stored in the computer storage medium 1004 . The processor 1001 (or CPU) is a computing core and a control core of an electronic device, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement corresponding method processes or corresponding functions.

In one embodiment, the processor 1001 of the electronic device provided in this embodiment of the present application may be configured to perform a series of defect detection processes:

Obtain the image to be inspected of the high-speed rail catenary;

Segment the image of the first component of the high-speed rail catenary from the to-be-detected image;

Segment the image of the second part of the high-speed rail catenary from the image of the first part; the second part is a sub-part of the first part;

Defect classification is performed on the first part and the second part based on the image of the first part and the image of the second part, and a defect classification result is obtained.

In yet another embodiment, the processor 1001 executes the segmentation of the image of the first component of the high-speed rail contact net from the to-be-detected image, including:

Perform feature extraction on the to-be-detected image to obtain a first feature map;

Locating and classifying the first component based on the first feature map, to obtain a first rectangular detection frame of the first component;

The image of the first component is segmented from the to-be-detected image according to the first rectangular detection frame.

In yet another embodiment, the processor 1001 executes the positioning and classification of the first component based on the first feature map to obtain a first rectangular detection frame of the first component, including:

Predicting the coordinates of the candidate region of the first component and classifying the front and the background on the first feature map, to determine the foreground target of the first component;

performing pooling processing on the corresponding feature of the foreground target of the first component in the first feature map to obtain a first pooling feature;

Classify the first part based on the first pooling feature to obtain the category of the first part and the first rectangular detection frame.

In yet another embodiment, the processor 1001 executes the segmentation of the image of the second component of the high-speed rail catenary from the image of the first component, including:

performing gamma correction on the image of the first part to obtain the image to be segmented of the first part;

The image of the second part is segmented from the image to be segmented of the first part.

In yet another embodiment, the processor 1001 executes the segmentation of the image of the second component from the to-be-segmented image of the first component, including:

performing feature extraction on the to-be-segmented image of the first component to obtain a second feature map;

Locating and classifying the second component based on the second feature map, to obtain a second rectangular detection frame of the second component;

According to the second rectangular detection frame, the segment is segmented from the image to be segmented of the first part. Image of the second part.

In yet another embodiment, the processor 1001 executes the positioning and classification of the second component based on the second feature map to obtain a second rectangular detection frame of the second component, including:

On the second feature map, the candidate region coordinate prediction and front and background classification of the second component are performed, and the foreground target of the second component is determined;

Perform pooling processing on the features corresponding to the foreground target of the second component in the second feature map to obtain second pooling features;

Classify the second part based on the second pooling feature to obtain the category of the second part and the second rectangular detection frame.

In yet another embodiment, the processor 1001 performs the defect classification of the first part and the second part based on the image of the first part and the image of the second part to obtain a defect classification result, including:

respectively perform feature extraction on the image of the first component and the image of the second component;

Predict the defect type of the first part based on the features extracted from the image of the first part, and obtain the defect classification result of the first part; the defect classification result of the first part includes the defect type of the first part and the defect classification result of the first part. the class of the first part; and

Predict the defect type of the second part based on the features extracted from the image of the second part, and obtain the defect classification result of the second part; the defect classification result of the second part includes the defect type of the second part and the defect classification result of the second part. The category of the second part.

In yet another embodiment, after the defect classification is performed on the first part and the second part based on the image of the first part and the image of the second part, and a defect classification result is obtained, the processor 1001 is further configured to execute: :

According to the defect classification result, defect early warning is carried out.

In yet another embodiment, the processor 1001 performs the defect early warning according to the defect classification result, including:

For the first part, output the defect type of the first part, the position of the first part in the image to be inspected, the type of the first part, and the high-speed rail line where the first part is located, so as to perform defect warning;

For the second part, determine the target first part to which the second part belongs, and output the defect type of the second part, the position of the second part in the image to be inspected, the type of the second part, and the target The position of the first part in the image to be inspected, the type of the target first part, and the high-speed rail line where the second part is located, for defect warning.

In yet another embodiment, the processor 1001 executes the acquisition of the to-be-detected image of the high-speed rail catenary, including:

Obtain the original image of the high-speed rail catenary collected by the imaging equipment;

The original image of the high-speed rail catenary is filtered to obtain the image to be detected.

Exemplarily, the above-mentioned electronic device may be a computer, a computer host, a server, a cloud server, a server cluster, etc. The electronic device may include but is not limited to a processor 1001, an input device 1002, an output device 1003, and a computer storage medium 1004, The input device 1002 can be a keyboard, a touch screen, etc., and the output device 1003 can be a speaker, a display, a radio frequency transmitter, and the like. in the technical field Those skilled in the art can understand that the schematic diagram is only an example of an electronic device, and does not constitute a limitation on the electronic device, and may include more or less components than the one shown, or combine some components, or different components.

It should be noted that, since the processor 1001 of the electronic device implements the steps in the above-mentioned defect detection method when executing the computer program, the embodiments of the above-mentioned defect detection method are all applicable to the electronic device, and can achieve the same or similar benefits. Effect.

Embodiments of the present application also provide a computer storage medium (Memory), where the computer storage medium is a memory device in an electronic device and is used to store programs and data. It can be understood that, the computer storage medium here may include both the built-in storage medium in the terminal, and certainly also the extended storage medium supported by the terminal. The computer storage medium provides storage space, and the storage space stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor 1001 are also stored in the storage space, and these instructions may be one or more computer programs (including program codes). It should be noted that the computer storage medium here can be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory; optionally, it can also be at least one A computer storage medium located remote from the processor 1001 . In one embodiment, one or more instructions stored in the computer storage medium can be loaded and executed by the processor 1001 to implement the corresponding steps of the above-mentioned defect detection method.

Exemplarily, the computer program of the computer storage medium includes computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form, and the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB hard disk, removable hard disk, magnetic disk, optical disk, computer memory, ROM, RAM, electrical carrier signal, telecommunication signal and software distribution media.

It should be noted that, since the computer program of the computer storage medium is executed by the processor to realize the steps in the above-mentioned defect detection method, all the embodiments of the above-mentioned defect detection method are applicable to the computer storage medium, and can achieve the same or similar beneficial effects.

The embodiments of the present application are described in detail above, and specific examples are used in this paper to illustrate the principles and implementations of the present application. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present application; at the same time, for Persons of ordinary skill in the art, based on the idea of the present application, will have changes in the specific implementation manner and application scope. In summary, the contents of this specification should not be construed as limitations on the present application.

S11, S12, S13, S14: Steps

Claims (12)

A defect detection method, wherein the method comprises: Obtain the image to be inspected of the high-speed rail catenary; Segment the image of the first component of the high-speed rail catenary from the to-be-detected image; Segment an image of a second part of the high-speed rail catenary from the image of the first part; the second part is a sub-part of the first part; and Defect classification is performed on the first part and the second part based on the image of the first part and the image of the second part, and a defect classification result is obtained. The method according to claim 1, wherein the segmentation of the image of the first component of the high-speed rail catenary from the to-be-detected image includes: Perform feature extraction on the to-be-detected image to obtain a first feature map; Locating and classifying the first part based on the first feature map to obtain a first rectangular detection frame of the first part; and The image of the first component is segmented from the to-be-detected image according to the first rectangular detection frame. The method according to claim 2, wherein the first component is located and classified based on the first feature map to obtain a first rectangular detection frame of the first component, comprising: Predicting the coordinates of the candidate region of the first component and classifying the front and the background on the first feature map, to determine the foreground target of the first component; performing a pooling process on the corresponding features of the foreground target of the first component in the first feature map to obtain a first pooling feature; and Classify the first part based on the first pooling feature to obtain the category of the first part and the first rectangular detection frame. The method according to claim 3, wherein segmenting the image of the second component of the high-speed rail contact net from the image of the first component includes: Perform gamma correction on the image of the first part to obtain an image to be segmented of the first part; and The image of the second part is segmented from the image to be segmented of the first part. The method according to claim 4, wherein segmenting the image of the second part from the image to be segmented of the first part comprises: performing feature extraction on the to-be-segmented image of the first component to obtain a second feature map; Locating and classifying the second component based on the second feature map to obtain a second rectangular detection frame of the second component; and The image of the second part is segmented from the image to be segmented of the first part according to the second rectangular detection frame. The method according to claim 5, wherein the second component is located and classified based on the second feature map to obtain a second rectangular detection frame of the second component, comprising: On the second feature map, the candidate region coordinate prediction and front and background classification of the second component are performed, and the foreground target of the second component is determined; performing a pooling process on the corresponding feature of the foreground target of the second component in the second feature map to obtain a second pooling feature; and Classify the second part based on the second pooling feature to obtain the category of the second part and the second rectangular detection frame. The method according to claim 6, wherein performing defect classification on the first part and the second part based on the image of the first part and the image of the second part to obtain a defect classification result, comprising: respectively perform feature extraction on the image of the first component and the image of the second component; Predict the defect type of the first part based on the features extracted from the image of the first part, and obtain the defect classification result of the first part; the defect classification result of the first part includes the defect type of the first part and the defect classification result of the first part. the class of the first part; and Predict the defect type of the second part based on the features extracted from the image of the second part, and obtain the defect classification result of the second part; the defect classification result of the second part includes the defect type of the second part and the defect classification result of the second part. The category of the second part. The method according to claim 7, wherein after the defect classification is performed on the first part and the second part based on the image of the first part and the image of the second part, and a defect classification result is obtained, the method Complex includes: According to the defect classification result, a defect early warning is performed. The method according to claim 8, wherein the defect early warning according to the defect classification result includes: For the first part, output the defect type of the first part, the position of the first part in the to-be-detected image, the type of the first part, and the high-speed rail line where the first part is located, so as to perform defect warning; as well as For the second part, determine the target first part to which the second part belongs, and output the defect type of the second part, the position of the second part in the image to be inspected, the type of the second part, and the target The position of the first part in the image to be inspected, the type of the target first part, and the high-speed rail line where the second part is located, for defect warning. According to the method described in claim 1, the acquisition of the image to be detected of the high-speed rail catenary includes: Obtain raw images of the high-speed rail catenary captured by imaging equipment; and The original image of the high-speed rail catenary is filtered to obtain the image to be detected. An electronic device, comprising an input device and an output device, wherein the complex comprises: a processor adapted to implement one or more instructions; and A computer storage medium storing one or more instructions adapted to be loaded by the processor and perform the method of any of claims 1-10. A computer storage medium, wherein the computer storage medium stores one or more instructions adapted to be loaded by a processor and perform the method of any one of claims 1-10.

Images