WO2022036919A1

WO2022036919A1 - Defect detection method and apparatus, and electronic device and computer storage medium

Info

Publication number: WO2022036919A1
Application number: PCT/CN2020/132532
Authority: WO
Inventors: 孙明珊; 暴天鹏; 吴立威
Original assignee: 深圳市商汤科技有限公司
Priority date: 2020-08-17
Filing date: 2020-11-28
Publication date: 2022-02-24
Also published as: KR20220023327A; JP2022549541A; CN111986174A; TW202209256A

Abstract

A defect detection method and apparatus, and an electronic device and a computer storage medium. The method comprises: acquiring an image, to be subjected to detection, of a high-speed-railway catenary (S11); performing segmentation on said image to obtain an image of a first component of the high-speed-railway catenary (S12); performing segmentation on the image of the first component to obtain an image of a second component of the high-speed-railway catenary, the second component being a sub-component of the first component (S13); and performing defect classification on the first component and the second component on the basis of the image of the first component and the image of the second component, so as to obtain a defect classification result (S14). Defect detection is performed on a high-speed-railway catenary in a cascade manner, thereby facilitating the reduction of the missed detection rate of the defect detection of the high-speed-railway catenary and the improvement of the detection accuracy.

Description

Defect detection method, device, electronic device and computer storage medium

This application claims the priority of the Chinese patent application filed on August 17, 2020 with the application number 202010835350.5 and the title of the invention is "defect detection method, device, electronic device and computer storage medium", the entire content of which is by reference Incorporated in this application.

technical field

The present application relates to the field of computer vision technology, and in particular, to a defect detection method, apparatus, electronic device, and computer storage medium.

Background technique

With the continuous development of high-speed railway construction, the requirements for the safety and reliability of the high-speed railway power supply system are gradually increasing, and the safety inspection and maintenance of the high-speed railway catenary network is particularly important. The popularization of catenary suspension monitoring devices has greatly improved the efficiency of image acquisition in the operation and maintenance of catenary. However, in the face of massive image data, the traditional method of detecting defects in catenary by manual inspection is not only time-consuming and efficient. It also has the problem of high missed detection rate, resulting in low accuracy of catenary defect detection.

SUMMARY OF THE INVENTION

In view of the above problems, the present application provides a defect detection method, device, electronic device and storage medium, which are beneficial to reduce the missed detection rate of high-speed rail catenary defect detection and improve the accuracy of catenary defect detection.

In order to achieve the above purpose, a first aspect of the embodiments of the present application provides a defect detection method, the method includes:

Obtain the image to be inspected of the high-speed rail catenary;

Segment the image of the first component of the high-speed rail catenary from the image to be detected;

Segment the image of the second part of the high-speed rail catenary from the image of the first part; the second part is a sub-part of the first part;

Defect classification is performed on the first part and the second part based on the image of the first part and the image of the second part, and a defect classification result is obtained.

With reference to the first aspect, in a possible implementation manner, the segmentation of the image of the first component of the high-speed rail contact net from the to-be-detected image includes:

Perform feature extraction on the to-be-detected image to obtain a first feature map;

Locating and classifying the first component based on the first feature map to obtain a first rectangular detection frame of the first component;

The image of the first component is segmented from the to-be-detected image according to the first rectangular detection frame.

With reference to the first aspect, in a possible implementation manner, the positioning and classification of the first component based on the first feature map to obtain a first rectangular detection frame of the first component includes:

Performing coordinate prediction and front and background classification of the candidate region of the first component on the first feature map, and determining the foreground target of the first component;

performing pooling processing on the corresponding features of the foreground target of the first component in the first feature map to obtain a first pooling feature;

The first part is classified based on the first pooling feature to obtain the category of the first part and the first rectangular detection frame.

With reference to the first aspect, in a possible implementation manner, the segmentation of the image of the second component of the high-speed rail contact net from the image of the first component includes:

Perform gamma check on the image of the first part to obtain the image to be divided of the first part;

The image of the second part is segmented from the image to be segmented of the first part.

In conjunction with the first aspect, in a possible implementation manner, segmenting the image of the second component from the to-be-segmented image of the first component includes:

performing feature extraction on the to-be-segmented image of the first component to obtain a second feature map;

Locating and classifying the second component based on the second feature map, to obtain a second rectangular detection frame of the second component;

The image of the second part is segmented from the image to be segmented of the first part according to the second rectangular detection frame.

With reference to the first aspect, in a possible implementation manner, the positioning and classification of the second component based on the second feature map to obtain a second rectangular detection frame of the second component includes:

Performing coordinate prediction and front and background classification of the candidate region of the second component on the second feature map to determine the foreground target of the second component;

performing pooling processing on the corresponding features of the foreground target of the second component in the second feature map to obtain a second pooling feature;

Classify the second part based on the second pooling feature to obtain the category of the second part and the second rectangular detection frame.

With reference to the first aspect, in a possible implementation manner, the defect classification is performed on the first part and the second part based on the image of the first part and the image of the second part to obtain defects Classification results, including:

Perform feature extraction on the image of the first component and the image of the second component respectively;

Predict the defect type of the first part based on the features extracted from the image of the first part, and obtain the defect classification result of the first part; the defect classification result of the first part includes the defect classification result of the first part. the type of defect and the category of the first part; and,

Predict the defect type of the second part based on the features extracted from the image of the second part, and obtain the defect classification result of the second part; the defect classification result of the second part includes the defect classification result of the second part. Defect type and category of the second part.

In combination with the first aspect, in a possible implementation manner, defect classification is performed on the first part and the second part based on the image of the first part and the image of the second part to obtain a defect classification After the result, the method further includes:

Defect early warning is performed according to the defect classification result.

With reference to the first aspect, in a possible implementation, the performing a defect early warning according to the defect classification result includes:

For the first part, output the defect type of the first part, the position of the first part in the image to be inspected, the type of the first part, and the high-speed rail line where the first part is located, for defect warning;

For the second part, determine the target first part to which the second part belongs, and output the defect type of the second part, the position of the second part in the image to be inspected, the second part The category of the target first part, the position of the target first part in the to-be-detected image, the category of the target first part and the high-speed rail line where the second part is located, so as to carry out defect early warning.

With reference to the first aspect, in a possible implementation manner, the obtaining of the image to be detected of the high-speed rail catenary includes:

Obtain the original image of the high-speed rail catenary collected by the imaging equipment;

The original image of the high-speed rail catenary is filtered to obtain the to-be-detected image.

A second aspect of the embodiments of the present application provides a defect detection device, the device comprising:

The image acquisition module is used to acquire the image to be detected of the high-speed rail catenary;

a first detection module, used for segmenting the image of the first component of the high-speed rail catenary from the to-be-detected image;

a second detection module, configured to segment the image of the second part of the high-speed rail contact net from the image of the first part; the second part is a sub-component of the first part;

The defect classification module is configured to perform defect classification on the first part and the second part based on the image of the first part and the image of the second part, and obtain a defect classification result.

A third aspect of the embodiments of the present application provides an electronic device, the electronic device includes an input device and an output device, and further includes a processor, adapted to implement one or more instructions; and, a computer storage medium, the computer storage medium storing There is one or more instructions adapted to be loaded by the processor and to perform the steps in any of the embodiments of the first aspect above.

A fourth aspect of the embodiments of the present application provides a computer storage medium, where the computer storage medium stores one or more instructions, and the one or more instructions are suitable for being loaded by a processor and executing any one of the foregoing first aspects steps in the implementation.

It can be seen that the embodiment of the present application obtains the image to be detected of the high-speed rail catenary; the image of the first part of the high-speed rail catenary is segmented from the to-be-detected image; the high-speed rail contact is segmented from the image of the first part an image of a second part of the web; the second part is a sub-part of the first part; based on the image of the first part, the image of the second part Parts are classified into defects, and the result of defect classification is obtained. In this way, the first-level component (ie, the first component) is detected on the image to be detected of the high-speed rail contact net, the image of the first-level component is segmented from the to-be-detected image of the high-speed railway contact line, and then the image of the first-level component is processed for the second-level component. (that is, the second part), to identify the secondary part on the primary part, segment the image of the secondary part, use the image of the primary part and the image of the secondary part to classify the defects, and realize the cascade type The high-speed rail catenary defect detection is beneficial to reduce the missed detection rate of high-speed rail catenary defect detection, thereby improving the accuracy of catenary defect detection. At the same time, it is also beneficial to reduce the cost of manual inspection, shorten the detection time, and improve the detection efficiency.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

1 is a schematic flowchart of a defect detection method provided by an embodiment of the present application;

2 is a schematic diagram of an application environment for defect detection of a high-speed rail catenary provided by an embodiment of the present application;

3 is a schematic diagram of filtering an original image of a high-speed rail catenary according to an embodiment of the present application;

4 is a schematic structural diagram of a defect detection model for a high-speed rail catenary provided by an embodiment of the present application;

5A is a schematic diagram of dividing a first component according to an embodiment of the present application;

5B is a schematic diagram of dividing a second component according to an embodiment of the present application;

6 is a schematic flowchart of another defect detection method provided by an embodiment of the present application;

7 is a schematic diagram of generating a candidate region based on a feature map according to an embodiment of the application;

FIG. 8 is a schematic structural diagram of a defect detection device provided by an embodiment of the application;

FIG. 9 is a schematic structural diagram of another defect detection device provided by an embodiment of the application;

FIG. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

detailed description

In order to make those skilled in the art better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only The embodiments are part of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the scope of protection of the present application.

The appearances of the terms "comprising" and "having" and any variations thereof in the specification, claims and drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally also includes For other steps or units inherent to these processes, methods, products or devices. In addition, the terms "first", "second", "third", etc. are used to distinguish different objects, and not to describe a specific order.

The embodiment of the present application proposes a defect detection scheme for a high-speed rail catenary, so as to reduce the missed detection rate of defect detection of the high-speed rail catenary and improve the accuracy of defect detection. In the specific implementation, a high-speed rail catenary defect detection model based on deep learning is adopted. First, the first-level components are located from the images of the high-speed rail catenary to be inspected, and then the first-level components are located from the images of the first-level components. The second-level components of the relationship are beneficial to reduce the missed detection rate of the components. The images of the first-level components and the images of the second-level components are used to predict the defect types of the first-level components and the second-level components, and the specific location of the defect can be output in the final defect warning. , defect type, the component to which the defect belongs, the superior component of the component to which the defect belongs, and the specific line segment where the defect is located. , so as to carry out the maintenance of the catenary to ensure the safe operation of the high-speed rail, and at the same time, it is conducive to the expansion of new components and new defects.

The defect detection method provided by the embodiments of the present application will be described in detail below with reference to the relevant drawings.

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of a defect detection method provided by an embodiment of the application. The defect detection method is applied to a server, such as a server and a computer host where a deep learning-based high-speed rail catenary defect detection model is deployed. , cloud server, etc., as shown in Figure 1, including steps S11-S14:

S11, acquiring an image to be detected of the high-speed rail catenary.

In the specific embodiment of the present application, as shown in FIG. 2 , the high-speed rail inspection vehicle usually operates at night. The inspection vehicle is equipped with high-definition imaging equipment and on-board sensors. The inspection vehicle travels on the high-speed rail line. Whenever the on-board sensor detects When there are pillars on both sides of the route, the imaging device is triggered to collect images of the catenary, and the original image of the high-speed rail catenary is obtained. When there are pillars inside, two sets of imaging equipment are triggered to image the front and back and overall layout of the support parts and suspension parts of the catenary, thus obtaining a large number of original images of the high-speed rail catenary from different angles. The resolution of the original image of the high-speed rail catenary usually has a better value, for example: 6576*4384 pixels, but due to environmental factors such as night operations and foggy, the collected original images of the high-speed rail catenary still have low resolution. For example, the resolution length and width are less than 2000 pixels. Therefore, as shown in Figure 3, it is necessary to filter the collected original images of the high-speed rail catenary, and filter out the high-speed rail catenary whose resolution length and width reach the preset pixel value. The original image is used as the image to be detected for subsequent defect detection, and the original image of the high-speed rail catenary whose resolution length and width are lower than the preset pixel value is filtered out.

S12, segment the image of the first component of the high-speed rail catenary from the to-be-detected image.

In the specific embodiment of the present application, a pre-trained deep learning-based high-speed rail catenary defect detection model is used to perform defect detection on each component in the to-be-detected image obtained in step S11, and the high-speed rail catenary defect detection model includes a first component detection model. As shown in Figure 4, the input of the first part detector is the image to be detected, which is used to detect the first part of the high-speed rail catenary from the image to be detected , such as: column top cover plate, insulator, ring rod-right angle hanging plate joint, arm wrist base, AF wire shoulder frame base, contact wire center anchor clamp, drop weight limit frame, etc., the second part detector is used to The second part on the first part is detected in the image of the first part output by the first part detector, such as bolts, nuts, cotter pins, etc. at both ends of the insulator, and the output is the image of the second part, and the defect classifier is used for The first part is classified according to the image of the first part, and the second part is classified according to the image of the second part; The location, the type of defect (such as the angle of the cotter pin on the arm base is not in place), the superior component of the component to which the defect belongs, the high-speed rail line where the defect is located, etc. Optionally, the first component detector may be a two-stage detector or a one-stage detector. The two-stage detector generates candidate regions based on the feature maps extracted from the images to be detected, and then analyzes the candidate regions. Perform classification prediction to obtain the category of the first component and the coordinates of the rectangular detection frame. The coordinates of the rectangular detection frame can be the coordinates of the upper left corner and the lower right corner, or the coordinates of the center point and length and width, etc., which are not limited in detail, as shown in Figure 5A As shown, the image of the first component, such as the insulator, the arm-wrist base, etc., is segmented from the image to be detected according to the rectangular detection frame. The one-stage detector does not need to generate a candidate area, it directly performs classification prediction for the input image to be detected, obtains the category of the first part and the coordinates of the rectangular detection frame, and then divides the image of the first part according to the rectangular detection frame. Optionally, the first component detector is trained using a sample image of the high-speed rail catenary, the first component in the sample image has a class label, and the first component detector is adjusted by a preset loss function during the training process. excellent.

S13, segment the image of the second part of the high-speed rail contact net from the image of the first part; the second part is a sub-part of the first part.

In the specific embodiment of this application, the second component refers to the sub-component on the first component, and the two are in a cascade relationship. The image of the first part segmented in step S12 is detected, and the possibility of missed detection is high. Therefore, it is necessary to perform gamma check on the image of the first part to improve the image quality and obtain the to-be-segmented image of the first part. image (that is, the image obtained after gamma verification), and then segment the image of the second component of the high-speed rail catenary from the image to be segmented by the second component detector. Optionally, the second part detector can be the same as the first part detector, or it can be different, it can be trained together with the first part detector, or can be trained separately, in the same way, the category and rectangle of the second part can be obtained. After the frame is detected, as shown in FIG. 5B , the image of the second part is segmented from the image of the first part according to the rectangular detection frame.

S14: Perform defect classification on the first part and the second part based on the image of the first part and the image of the second part, to obtain a defect classification result.

In a specific embodiment of the present application, optionally, the above-mentioned performing defect classification on the first part and the second part based on the image of the first part and the image of the second part, to obtain a defect classification result, including :

Please continue to refer to Figure 4. After obtaining the image of the first part and the image of the second part, they are input into the defect classifier for probability prediction of defect types. Specifically, the feature of the image of the first part and the feature of the image of the second part are extracted through the backbone network of the defect classifier. The backbone network mainly performs convolution processing, and then based on the extracted features, the features are input into the fully connected layer Predict the probability of defect types, and take the defect type with the highest probability as the defect type of the part. For example, the characteristics of the first part of the pendulum limit frame are currently input, and the fully connected layer is classified to predict the existence of cracks in the pendulum head restraint frame. If the probability reaches 95% (the highest), the defect type of the pendulum limit frame is the existence of cracks. Of course, in addition to the defect type of the part, the final output of the defect classifier also has the class index of the part and the coordinates of the rectangular detection frame, for example: c05, if there is a crack, c05 means the class index of the part, among which, the class index of the first part The class index of the second part can be determined when the class of the first part is obtained by the first part detector, and the class index of the second part can be determined when the class of the second part is obtained by the second part detector. Wherein, the defect classifier can be trained together with the first part detector, can also be trained together with the second part detector, or can be trained separately.

Optionally, after the defect classification is performed on the first part and the second part based on the image of the first part and the image of the second part, and a defect classification result is obtained, the method further includes: according to The defect classification result is used for defect early warning.

In a specific embodiment of the present application, the performing defect early warning according to the defect classification result includes:

Specifically, the target first part refers to the upper-level part of the second part. The input of the defect warning module includes the defect classification results of the first part and the second part, the coordinates of the rectangular detection frame of the first part and the second part, and the output part The type of defect, the location of the component in the image to be inspected, the category of the component, the location of the superior component to which the component belongs in the image to be inspected, the category of the superior component of the component, and the high-speed rail line where the component is located. Optionally, the position of the component in the image to be detected and the position of the upper-level component of the component in the image to be detected are presented in the rectangular detection frame of the component, the category of the component and the category of the parent component of the component are presented with the category index, the component The high-speed rail line you are in can output according to the logo carried when the imaging device uploads the original image of the high-speed railway catenary. It will exist in the entire defect detection process. Of course, this is only an example, and does not limit the embodiments of the present application.

In addition, since the first part does not have a parent part to which it belongs, the early warning information output by the first part includes the defect type of the first part (for example, the nut on the AF wire shoulder frame has fallen off), the defect of the first part in the image to be detected. Position (such as the rectangular detection frame of the AF line shoulder frame base), the type of the first part, and the high-speed rail line where the first part is located. The second part usually stores the upper-level part. Therefore, it is necessary to determine the target first part to which the second part belongs. When outputting information such as the defect type of the second part itself, the position of the second part in the image to be inspected, etc., it is also necessary to output the The category of the first part of the target, the position in the image to be detected, etc.

Further, the determining of the target first component to which the second component belongs includes:

obtaining the intersection of the second rectangular detection frame of the second component and the first rectangular detection frame of all the first components;

obtaining the ratio between the intersection and the second rectangular detection frame;

The target first part to which the second part belongs is determined according to the ratio between the second rectangular detection frames.

Specifically, the first rectangular detection frame refers to the bounding box regression of the first component, and the second rectangular detection frame refers to the bounding box regression of the second component. For the currently detected positioning hook (belonging to the second component), it is assumed that its second The rectangular detection frame is A, and the first rectangular detection frames of all the first components are B1, B2, B3...Bn, and the intersection of the second rectangular detection frame A and all the first rectangular detection frames B1, B2, B3...Bn is obtained first. C1, C2, C3...Cn, and then calculate the ratios C1/A, C2/A, C3/A...Cn/A of the intersection C1, C2, C3...Cn and the second rectangular detection frame A respectively, if C1/A is the largest value, the first component corresponding to the first rectangular detection frame B1 is used as the target first component to which the positioning hook belongs. In this embodiment, since the defect early warning module inputs the defect classification results of the first part and the second part, and the coordinates of the rectangular detection frame of the first part and the second part, the rectangular detection frame coordinates of the first part and the second part can be The ratio of the intersection of the detection frame and the rectangular detection frame of the second component is used to determine the upper-level component of the second component, which does not affect the overall detection time, and the accuracy meets the requirements.

In the embodiment of the present application, the image to be detected of the high-speed rail catenary is obtained; the image of the first part of the high-speed rail catenary is segmented from the to-be-detected image; the second part of the high-speed rail catenary is segmented from the image of the first part an image of a part; the second part is a sub-part of the first part; the first part and the second part are classified as defective based on the image of the first part and the image of the second part , get the defect classification result. In this way, the first-level component is detected on the image to be detected of the high-speed rail contact line, the image of the first-level component is segmented from the to-be-detected image of the high-speed rail contact line, and then the second level component is detected on the image of the first-level component to identify The secondary part on the primary part, the image of the secondary part is segmented, and the image of the primary part and the image of the secondary part are used for defect classification, which realizes the cascading high-speed rail catenary defect detection, which is beneficial to reduce the high-speed rail. The missed detection rate of catenary defect detection, thereby improving the accuracy of catenary defect detection. At the same time, it is also beneficial to reduce the cost of manual inspection, shorten the detection time, and improve the detection efficiency.

Please refer to FIG. 6, which is a schematic flowchart of another defect detection method provided by an embodiment of the present application, as shown in FIG. 6, including steps S61-S67:

S61, acquiring an image to be detected of the high-speed rail catenary;

S62, perform feature extraction on the to-be-detected image to obtain a first feature map;

S63, locating and classifying the first component of the high-speed rail catenary based on the first feature map, to obtain a first rectangular detection frame of the first component;

S64, segment the image of the first component from the to-be-detected image according to the first rectangular detection frame;

S65, segment the image of the second component of the high-speed rail contact network from the image of the first component; the second component is a subcomponent of the first component;

S66, perform defect classification on the first part and the second part based on the image of the first part and the image of the second part, to obtain a defect classification result;

S67, perform a defect early warning according to the defect classification result.

In the specific embodiment of the present application, for the image to be detected that is input to the first component detector, feature extraction is performed on it through the backbone network of the first component detector. The backbone network mainly performs convolution processing, and the output feature map is the above-mentioned feature map. The first feature map, the coordinates of the candidate region of the first component are predicted on the first feature map, and the candidate region shown in Figure 7 is generated, and the front and background classification is performed in the candidate region to obtain the foreground target of the first component. , and then the corresponding features of the foreground target in the first feature map are pooled, and the output features are the above-mentioned first pooling features, the first pooling features are input into the fully connected layer for final classification, and the output The category of the first part in the image to be inspected and the rectangular inspection frame (ie, the first rectangular inspection frame), and the image of the first part is segmented from the image to be inspected according to the first rectangular inspection frame, as the input of the defect classifier. In this embodiment, the first part detector based on the candidate region is used to classify the first part in the image to be detected, and the accuracy is higher.

Optionally, the segmentation of the image of the second component of the high-speed rail catenary from the image of the first component includes:

In this embodiment, performing gamma verification on the image of the first component can obtain an image to be segmented with better quality, which is beneficial to overcome the influence of poor light on detection, so as to accurately segment the image of the second component, reduce The miss rate of the second part.

Optionally, segmenting the image of the second part from the image to be segmented of the first part includes:

Optionally, the positioning and classification of the second component based on the second feature map to obtain a second rectangular detection frame of the second component includes:

In the specific embodiment of the present application, the second feature map refers to the feature map extracted by the second component detector from the sub-graph of the first component through the backbone network, and the second pooling feature refers to the second component detector's effect on the second component. The features obtained by pooling the corresponding features of the foreground target in the second feature map. It should be noted that the processing procedure of the second component detector is the same as that of the first component detector. It also predicts the category of the second component based on the generated candidate area, and outputs the category of the second component and the second rectangular detection frame.

The specific implementations of the above steps S61 to S67 have been described in the embodiment shown in FIG. 1 , and can achieve the same or similar beneficial effects, and will not be repeated here.

Based on the description of the method embodiment shown in FIG. 1 or FIG. 6 , an embodiment of the present application further provides a defect detection apparatus. Please refer to FIG. 8 . FIG. 8 is a schematic structural diagram of a defect detection apparatus provided by an embodiment of the present application. As shown in Figure 8, the device includes:

The image acquisition module 81 is used to acquire the to-be-detected image of the high-speed rail catenary;

The first detection module 82 is used for segmenting the image of the first component of the high-speed rail catenary from the to-be-detected image;

The second detection module 83 is configured to segment the image of the second part of the high-speed rail contact net from the image of the first part; the second part is a sub-component of the first part;

The defect classification module 84 is configured to perform defect classification on the first part and the second part based on the image of the first part and the image of the second part to obtain a defect classification result.

In a possible implementation manner, in terms of segmenting the image of the first component of the high-speed rail catenary from the to-be-detected image, the first detection module 82 is specifically configured to:

performing feature extraction on the to-be-detected image to obtain a first feature map;

In a possible implementation manner, in terms of locating and classifying the first component based on the first feature map to obtain a first rectangular detection frame of the first component, the first detection module 82 is specifically configured to :

In a possible implementation manner, in terms of segmenting the image of the second component of the high-speed rail catenary from the image of the first component, the second detection module 83 is specifically configured to:

In a possible implementation manner, in terms of segmenting the image of the second part from the image to be segmented of the first part, the second detection module 83 is specifically configured to:

In a possible implementation manner, the second detection module 83 is specifically configured to:

In a possible implementation manner, in terms of performing defect classification on the first part and the second part based on the image of the first part and the image of the second part, and obtaining a defect classification result, the defect classification Module 84 is specifically used to:

In a possible implementation manner, as shown in FIG. 9 , the device further includes a defect early warning module 85; the defect early warning module 85 is specifically used for:

In a possible implementation manner, in terms of performing a defect early warning according to the defect classification result, the defect early warning module 85 is specifically used for:

In a possible implementation manner, in terms of acquiring the to-be-detected image of the high-speed rail catenary, the image acquisition module 81 is specifically used for:

According to an embodiment of the present application, each unit in the defect detection apparatus shown in FIG. 8 or FIG. 9 may be respectively or all merged into one or several other units to form, or some unit(s) may also be It is further divided into multiple units with smaller functions, which can realize the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit may also be implemented by multiple units, or the functions of multiple units may be implemented by one unit. In other embodiments of the present application, the defect-based detection device may also include other units, and in practical applications, these functions may also be implemented with the assistance of other units, and may be implemented by cooperation of multiple units.

According to another embodiment of the present application, a general-purpose computing device, such as a computer, may be implemented on a general-purpose computing device including a central processing unit (CPU), a random access storage medium (RAM), a read-only storage medium (ROM), and other processing elements and storage elements. Running a computer program (including program code) capable of executing the steps involved in the corresponding method as shown in FIG. 1 or FIG. 6, to construct the defect detection apparatus as shown in FIG. 8 or FIG. 9, and to realize the present invention. The defect detection method of the application embodiment is provided. The computer program can be recorded on, for example, a computer-readable recording medium, and loaded in the above-mentioned computing device through the computer-readable recording medium, and executed therein.

Based on the descriptions of the foregoing method embodiments and apparatus embodiments, the embodiments of the present application further provide an electronic device. Referring to FIG. 10 , the electronic device includes at least a processor 1001 , an input device 1002 , an output device 1003 and a computer storage medium 1004 . The processor 1001 , the input device 1002 , the output device 1003 and the computer storage medium 1004 in the electronic device may be connected through a bus or other means.

The computer storage medium 1004 can be stored in the memory of the electronic device, the computer storage medium 1004 is used for storing a computer program, the computer program includes program instructions, and the processor 1001 is used for executing the program stored in the computer storage medium 1004 instruction. The processor 1001 (or called CPU (Central Processing Unit, central processing unit)) is the computing core and the control core of the electronic device, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve Corresponding method flow or corresponding function.

In one embodiment, the processor 1001 of the electronic device provided in this embodiment of the present application may be configured to perform a series of defect detection processes:

Obtain the image to be inspected of the high-speed rail catenary;

In yet another embodiment, the processor 1001 executes the segmentation of the image of the first component of the high-speed rail catenary from the to-be-detected image, including:

In yet another embodiment, the processor 1001 executes the positioning and classification of the first component based on the first feature map to obtain a first rectangular detection frame of the first component, including:

In yet another embodiment, the processor 1001 executes the segmentation of the image of the second component of the high-speed rail catenary from the image of the first component, including:

In yet another embodiment, the processor 1001 performing the segmenting of the image of the second part from the image to be segmented of the first part includes:

In yet another embodiment, the processor 1001 executes the positioning and classification of the second component based on the second feature map to obtain a second rectangular detection frame of the second component, including:

In yet another embodiment, the processor 1001 performs the defect classification on the first part and the second part based on the image of the first part and the image of the second part, to obtain a defect classification result, including: :

In yet another embodiment, after the defect classification is performed on the first part and the second part based on the image of the first part and the image of the second part, and a defect classification result is obtained, the processor 1001 further uses To execute:

In yet another embodiment, the processor 1001 executes the defect early warning according to the defect classification result, including:

In yet another embodiment, the processor 1001 executes the obtaining of the image to be detected of the high-speed rail catenary, including:

Exemplarily, the above-mentioned electronic device may be a computer, a computer host, a server, a cloud server, a server cluster, etc. The electronic device may include but is not limited to a processor 1001, an input device 1002, an output device 1003, and a computer storage medium 1004. The input device 1002 It can be a keyboard, a touch screen, etc., and the output device 1003 can be a speaker, a display, a radio frequency transmitter, and the like. Those skilled in the art can understand that the schematic diagram is only an example of an electronic device, and does not constitute a limitation to the electronic device, and may include more or less components than the one shown, or combine some components, or different components.

It should be noted that, since the processor 1001 of the electronic device implements the steps in the above-mentioned defect detection method when executing the computer program, the embodiments of the above-mentioned defect detection method are all applicable to the electronic device, and can achieve the same or similar benefits. Effect.

Embodiments of the present application further provide a computer storage medium (Memory), where the computer storage medium is a memory device in an electronic device and is used to store programs and data. It can be understood that, the computer storage medium here may include both a built-in storage medium in the terminal, and certainly also an extended storage medium supported by the terminal. The computer storage medium provides storage space, and the storage space stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor 1001 are also stored in the storage space, and these instructions may be one or more computer programs (including program codes). It should be noted that the computer storage medium here can be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory; optionally, it can also be at least one memory located far away from the aforementioned processing The computer storage medium of the device 1001 . In one embodiment, one or more instructions stored in a computer storage medium can be loaded and executed by the processor 1001 to implement the corresponding steps of the above-mentioned defect detection method.

Exemplarily, the computer program of the computer storage medium includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, and the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

It should be noted that, since the computer program of the computer storage medium is executed by the processor to realize the steps in the above-mentioned defect detection method, all the embodiments of the above-mentioned defect detection method are applicable to the computer storage medium, and can achieve the same or similar beneficial effects.

The embodiments of the present application are described in detail above, and specific examples are used in this paper to illustrate the principles and implementations of the present application. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present application; at the same time, for Persons of ordinary skill in the art, based on the idea of the present application, will have changes in the specific implementation manner and application scope. In summary, the contents of this specification should not be construed as limitations on the present application.

Claims

A defect detection method, characterized in that the method comprises:

Obtain the image to be inspected of the high-speed rail catenary;

Segment the image of the first component of the high-speed rail catenary from the image to be detected;

Segment the image of the second part of the high-speed rail catenary from the image of the first part; the second part is a sub-part of the first part;

Defect classification is performed on the first part and the second part based on the image of the first part and the image of the second part, and a defect classification result is obtained.
The method according to claim 1, wherein the step of segmenting the image of the first component of the high-speed rail catenary from the to-be-detected image comprises:

Perform feature extraction on the to-be-detected image to obtain a first feature map;

Locating and classifying the first component based on the first feature map to obtain a first rectangular detection frame of the first component;

The image of the first component is segmented from the to-be-detected image according to the first rectangular detection frame.
The method according to claim 2, wherein the locating and classifying the first part based on the first feature map to obtain a first rectangular detection frame of the first part, comprising:

Performing coordinate prediction and front and background classification of the candidate region of the first component on the first feature map, and determining the foreground target of the first component;

performing pooling processing on the corresponding features of the foreground target of the first component in the first feature map to obtain a first pooling feature;

The first part is classified based on the first pooling feature to obtain the category of the first part and the first rectangular detection frame.
The method according to claim 3, wherein the step of segmenting the image of the second part of the high-speed rail catenary from the image of the first part comprises:

Perform gamma check on the image of the first part to obtain the image to be divided of the first part;

The image of the second part is segmented from the image to be segmented of the first part.
The method according to claim 4, wherein the segmenting the image of the second part from the image to be segmented of the first part comprises:

performing feature extraction on the to-be-segmented image of the first component to obtain a second feature map;

Locating and classifying the second component based on the second feature map, to obtain a second rectangular detection frame of the second component;

The image of the second part is segmented from the image to be segmented of the first part according to the second rectangular detection frame.
The method according to claim 5, wherein the locating and classifying the second component based on the second feature map to obtain a second rectangular detection frame of the second component, comprising:

Performing coordinate prediction and front and background classification of the candidate region of the second component on the second feature map to determine the foreground target of the second component;

performing pooling processing on the corresponding features of the foreground target of the second component in the second feature map to obtain a second pooling feature;

Classify the second part based on the second pooling feature to obtain the category of the second part and the second rectangular detection frame.
The method according to claim 6, wherein the defect classification is performed on the first part and the second part based on the image of the first part and the image of the second part to obtain a defect classification Results, including:

Perform feature extraction on the image of the first component and the image of the second component respectively;

Predict the defect type of the first part based on the features extracted from the image of the first part, and obtain the defect classification result of the first part; the defect classification result of the first part includes the defect classification result of the first part. the type of defect and the category of the first part; and,

Predict the defect type of the second part based on the features extracted from the image of the second part, and obtain the defect classification result of the second part; the defect classification result of the second part includes the defect classification result of the second part. Defect type and category of the second part.
The method according to claim 7, wherein the defect classification is performed on the first part and the second part based on the image of the first part and the image of the second part, and a defect classification result is obtained. Afterwards, the method further includes:

Defect early warning is performed according to the defect classification result.
The method according to claim 8, wherein the performing a defect early warning according to the defect classification result comprises:

For the first part, output the defect type of the first part, the position of the first part in the image to be inspected, the type of the first part, and the high-speed rail line where the first part is located, for defect warning;

For the second part, determine the target first part to which the second part belongs, and output the defect type of the second part, the position of the second part in the image to be inspected, the second part The category of the target first part, the position of the target first part in the to-be-detected image, the category of the target first part and the high-speed rail line where the second part is located, so as to carry out defect early warning.
The method according to claim 1, said acquiring the image to be detected of the high-speed rail catenary, comprising:

Obtain the original image of the high-speed rail catenary collected by the imaging equipment;

The original image of the high-speed rail catenary is filtered to obtain the to-be-detected image.
A defect detection device, characterized in that the device comprises:

The image acquisition module is used to acquire the image to be detected of the high-speed rail catenary;

a first detection module, used for segmenting the image of the first component of the high-speed rail catenary from the to-be-detected image;

a second detection module, configured to segment the image of the second part of the high-speed rail contact net from the image of the first part; the second part is a sub-component of the first part;

The defect classification module is configured to perform defect classification on the first part and the second part based on the image of the first part and the image of the second part, and obtain a defect classification result.
The device according to claim 11, wherein in terms of segmenting the image of the first component of the high-speed rail contact net from the to-be-detected image, the first detection module is specifically configured to:

Perform feature extraction on the to-be-detected image to obtain a first feature map;

Locating and classifying the first component based on the first feature map to obtain a first rectangular detection frame of the first component;

The image of the first component is segmented from the to-be-detected image according to the first rectangular detection frame.
The device according to claim 12, wherein, in terms of locating and classifying the first part based on the first feature map to obtain a first rectangular detection frame of the first part, the first The detection module is specifically used for:

Performing coordinate prediction and front and background classification of the candidate region of the first component on the first feature map, and determining the foreground target of the first component;

performing pooling processing on the corresponding features of the foreground target of the first component in the first feature map to obtain a first pooling feature;

The first part is classified based on the first pooling feature to obtain the category of the first part and the first rectangular detection frame.
The device according to claim 13, wherein, in terms of segmenting the image of the second part of the high-speed rail catenary from the image of the first part, the second detection module is specifically used for:

Perform gamma check on the image of the first part to obtain the image to be divided of the first part;

The image of the second part is segmented from the image to be segmented of the first part.
The device according to claim 13, wherein, in terms of segmenting the image of the second part from the image to be segmented of the first part, the second detection module is specifically configured to:

performing feature extraction on the to-be-segmented image of the first component to obtain a second feature map;

Locating and classifying the second component based on the second feature map to obtain a second rectangular detection frame of the second component;

The image of the second part is segmented from the image to be segmented of the first part according to the second rectangular detection frame.
The device according to claim 15, wherein, in terms of locating and classifying the second part based on the second feature map to obtain a second rectangular detection frame of the second part, the second The detection module is specifically used for:

Performing coordinate prediction and front and background classification of the candidate region of the second component on the second feature map to determine the foreground target of the second component;

performing pooling processing on the corresponding features of the foreground target of the second component in the second feature map to obtain a second pooling feature;

Classify the second part based on the second pooling feature to obtain the category of the second part and the second rectangular detection frame.
The apparatus according to claim 16, wherein the defect classification result is obtained by classifying the first part and the second part based on the image of the first part and the image of the second part. In one aspect, the defect classification module is specifically used for:

Perform feature extraction on the image of the first component and the image of the second component respectively;

Predict the defect type of the first part based on the features extracted from the image of the first part, and obtain the defect classification result of the first part; the defect classification result of the first part includes the defect classification result of the first part. the type of defect and the category of the first part; and,

Predict the defect type of the second part based on the features extracted from the image of the second part, and obtain the defect classification result of the second part; the defect classification result of the second part includes the defect classification result of the second part. Defect type and category of the second part.
The device according to claim 17, wherein the device further comprises a defect early warning module, and the defect early warning module is specifically used for:

Defect early warning is performed according to the defect classification result.
The device according to claim 18, wherein, in terms of performing defect early warning according to the defect classification result, the defect early warning module is specifically used for:

For the first part, output the defect type of the first part, the position of the first part in the image to be inspected, the type of the first part, and the high-speed rail line where the first part is located, for defect warning;

For the second part, determine the target first part to which the second part belongs, and output the defect type of the second part, the position of the second part in the image to be inspected, the second part The category of the target first component, the position of the target first component in the image to be inspected, the category of the target first component, and the high-speed rail line where the second component is located, so as to perform defect early warning.
The device according to claim 11, wherein, in terms of acquiring the image to be detected of the high-speed rail catenary, the image acquisition module is specifically used for:

Obtain the original image of the high-speed rail catenary collected by the imaging equipment;

Filtering the original image of the high-speed rail catenary to obtain the to-be-detected image.
An electronic device, comprising an input device and an output device, is characterized in that it also includes:

a processor adapted to implement one or more instructions; and,

A computer storage medium having stored thereon one or more instructions adapted to be loaded by the processor and perform the method of any one of claims 1-10.
A computer storage medium, characterized in that, the computer storage medium stores one or more instructions, and the one or more instructions are suitable for being loaded and executed by a processor according to any one of claims 1-10. method.