CN116452549A - Simplified and unified contact net defect detection method and system - Google Patents

Abstract

The invention discloses a simplified and unified contact network defect detection method and system, which adopts a mixed cutting mode to amplify a simplified and unified contact network image, performs multi-scale feature extraction on the simplified and unified contact network image, trains an upstream self-supervision model by utilizing multi-scale feature contrast loss and dense contrast loss, converts the trained self-supervision model into a downstream defect detection model to be trained, detects the simplified and unified contact network image to be detected by adopting the trained defect detection model, can obtain a high-precision detection result, and is suitable for the conditions of lack of target samples and more defect categories.

Description

Simplified and unified contact net defect detection method and system

Technical Field

The invention relates to a simplified and unified contact net defect detection method and system, and belongs to the technical field of image processing.

Background

Along with the continuous high-speed rail of modern technology, the construction of high-speed rail is very different, and the traditional railway overhead contact system has finished advancing to the simplified and unified overhead contact system at present. However, as the erection of the high-speed rail line expands, the high-speed rail line becomes more complex, the maintenance work for the high-speed rail line becomes very difficult, defects and faults generated in the high-speed rail line are more and more, and the defects such as bird nests, foreign matters, missing parts and the like are detected, the position where the contact net breaks down is timely detected and positioned, so that the high-speed rail line is very necessary to ensure safe operation.

For defect detection of a high-speed rail simplified contact net, a camera device is erected on the high-speed rail contact net to collect 2C or 4C images, an analysis room analyzes the images, defects or faults possibly existing in the images are manually selected, then the images are collected to a maintenance department to record, maintenance personnel are dispatched to conduct one-by-one investigation, and the method is very dependent on experienced staff, is long in response time and is easy to occur in the situations of false detection, omission detection and the like.

In recent years, although many researchers are researching on defect detection problems of a high-speed rail simplified contact net, due to various railway running environments, the types and types of contained defects are very complex, the labeling of defect samples is time-consuming and labor-consuming, and an effective target detection defect library and defect detection means are difficult to form. Therefore, for simplifying defect detection of the overhead line system, in order to adapt to various types of defect detection, it is highly desirable to design a defect detection method which can still maintain a good detection effect under the conditions that target samples are lack and defect types are more.

Disclosure of Invention

The invention provides a simplified and unified contact net defect detection method and system, which solve the problems disclosed in the background technology.

In order to solve the technical problems, the invention adopts the following technical scheme:

a simplified contact net defect detection method comprises the following steps:

acquiring a simplified and unified contact net image;

inputting the simplified and unified contact net image into a pre-trained defect detection model to obtain a simplified and unified contact net defect detection result; the defect detection model is formed by converting a trained self-supervision model, a mixed cutting mode is adopted to amplify a simplified and unified contact network image in a training set when the self-supervision model is trained, multi-scale feature extraction is carried out on the simplified and unified contact network image, and the training is carried out by utilizing multi-scale feature contrast loss and dense contrast loss according to the extracted multi-scale features.

The method for amplifying the simplified and unified contact net image in the training set by adopting the mixed cutting mode comprises the following steps:

in the preset iteration times, cutting out simplified and unified contact network images in the training set by adopting a random cutting method, and amplifying the simplified and unified contact network images in the training set by using the cut-out images;

and updating parameters of the self-supervision model according to the weight of the self-supervision model outside the preset iteration times, adopting a comparison cutting method to correct the position judgment of the cutting frame, cutting the simplified and unified contact network image in the training set according to the cutting frame after the position correction, and using the cut image to augment the simplified and unified contact network image in the training set.

The loss function of the self-supervision model is:

wherein L is a self-supervision model loss function calculated value, N _M For the number of multi-scale feature contrast losses, N _L L is the total loss number _mfc Calculating a value for a multi-scale feature contrast loss function, L _dc For global image I _q 、I _k Intensive contrast loss function calculation value between, global image I _q 、I _k Different views generated by the same simplified contact net image in a mixed cutting mode are obtained;

W _i weight for i-th scale feature contrast loss, W _j Weight for j-th scale feature contrast loss, W _k Weighting the loss for the j-th scale feature contrast, N _m The number of the multi-scale features;

is I _q 、I _k A comparison loss function calculation value between +.>Is I _q Through the ith dimension feature of the encoder, < >>Representation and I _q Is opposite to I _k Through the ith dimension feature of the encoder, T is the temperature super-parameter, +.>Is equal to I _q N-th I of negative pair _k Through the ith dimension feature of the encoder, K is the negative sample queue length;

is P _q And I _k A comparison loss function calculation value between +.>Is P _q Through the j-th dimension characteristic of the encoder, < >>Representation and P _q Is opposite to I _k Through the j-th dimension characteristic of the encoder, < >>Representation and P _q N-th I of negative pair _k Through the ith dimension feature, P, of the encoder _q Is I _q I is a local image of (1) _q Splitting into a plurality of image blocks according to a jigsaw mode, and adjusting each image block into I after random cutting _q Is of the size of (1) to obtain I _q Is a local image of (a);

is P _q 、P _k A comparison loss function calculation value between +.>Is P _q Through the kth dimension characteristic of the encoder, < >>Representation and P _q P is opposite to _k Through the kth dimension characteristic of the encoder, < >>Representation and P _q N-th P of negative pair _k Through the kth dimension characteristic, P, of the encoder _k Is I _k I is a local image of (1) _k Splitting into a plurality of image blocks according to a jigsaw mode, and adjusting each image block into I after random cutting _k Is of the size of (1) to obtain I _k Is a local image of (a);

is I _q Via the minimum scale feature of the encoder, +.>Representation and I _q Is opposite to I _k Via the minimum scale feature of the encoder, +.>Representation and I _q N-th I of negative pair _k Through the smallest scale feature of the encoder.

The rules for converting the trained self-supervision model into the defect detection model are as follows:

the key set and the value set of the self-supervision model and the defect detection model after training are converted according to the fuzzy matching principle;

the fuzzy matching principle is as follows: key set C of defect detection model _dk As a standard key set, a trained self-supervision model key set C _uk As a set of matching keys, remove C _uk The initial structure name of (C) _uk The remaining key names and C _dk Performing full character matching on key names in the key name matching pair, obtaining a key name matching pair, and matching k in the key name matching pair _d And k _u Corresponding v _u Storing the defect detection model in a key value pair mode;

k _d and k _u Form key name matching pairs, k _d Is C _dk Key name, k of (a) _u Is C _uk Key name, v of (v) _u Is a value in the trained self-supervising model value set.

The loss function of the defect detection model is:

L _s ＝BCE(obj _y ,obj _s )+BCE(cls _y ,cls _s )+IOU(box _y ,box _s )

wherein L is _s For the defect detection model loss function calculation value, BCE represents a BCEWITHLogitsLoss binary cross entropy loss function, obj _y Indicating whether the defect detection model predicts the input image to be defective or not, obj _s Indicating whether the input image of the defect detection model has defects actually, cls _y Representing the defect class predicted by the defect detection model, cls _s For the actually existing defect category, IOU represents IOULSO loss function for detecting prediction box _y And a real box _s Overlapping area between.

A simplified and unified catenary defect detection system, comprising:

the acquisition module is used for acquiring a simplified and unified contact net image;

the detection module inputs the simplified and unified contact net image into a pre-trained defect detection model to obtain a simplified and unified contact net defect detection result; the defect detection model is formed by converting a trained self-supervision model, a mixed cutting mode is adopted to amplify a simplified and unified contact network image in a training set when the self-supervision model is trained, multi-scale feature extraction is carried out on the simplified and unified contact network image, and the training is carried out by utilizing multi-scale feature contrast loss and dense contrast loss according to the extracted multi-scale features.

In the detection module, the simplified and unified contact network image in the training set is enhanced by adopting a mixed cutting mode, and the method comprises the following steps:

in the preset iteration times, cutting out simplified and unified contact network images in the training set by adopting a random cutting method, and amplifying the simplified and unified contact network images in the training set by using the cut-out images;

and updating parameters of the self-supervision model according to the weight of the self-supervision model outside the preset iteration times, adopting a comparison cutting method to correct the position judgment of the cutting frame, cutting the simplified and unified contact network image in the training set according to the cutting frame after the position correction, and using the cut image to augment the simplified and unified contact network image in the training set.

In the detection module, the loss function of the self-supervision model is as follows:

wherein L is a self-supervision model loss function calculated value, N _M For the number of multi-scale feature contrast losses, N _L L is the total loss number _mfc Calculating a value for a multi-scale feature contrast loss function, L _dc For global image I _q 、I _k Intensive contrast loss function calculation value between, global image I _q 、I _k Different views generated by the same simplified contact net image in a mixed cutting mode are obtained;

W _i weight for i-th scale feature contrast loss, W _j Weight for j-th scale feature contrast loss, W _k Weighting the loss for the j-th scale feature contrast, N _m The number of the multi-scale features;

is I _q 、I _k A comparison loss function calculation value between +.>Is I _q Through the ith dimension feature of the encoder, < >>Representation and I _q Is opposite to I _k Through the ith dimension feature of the encoder, T is the temperature super-parameter, +.>Is equal to I _q N-th I of negative pair _k Through the ith dimension feature of the encoder, K is the negative sample queue length;

is P _q And I _k A comparison loss function calculation value between +.>Is P _q Through the j-th dimension characteristic of the encoder, < >>Representation and P _q Is opposite to I _k Through the j-th dimension characteristic of the encoder, < >>Representation and P _q N-th I of negative pair _k Through the ith dimension feature, P, of the encoder _q Is I _q I is a local image of (1) _q Splitting into a plurality of image blocks according to a jigsaw mode, and adjusting each image block into I after random cutting _q Is of the size of (1) to obtain I _q Is a local image of (a);

is P _q 、P _k A comparison loss function calculation value between +.>Is P _q Through the kth dimension characteristic of the encoder, < >>Representation and P _q P is opposite to _k Through the kth dimension characteristic of the encoder, < >>Representation and P _q N-th P of negative pair _k Through the kth dimension characteristic, P, of the encoder _k Is I _k I is a local image of (1) _k Splitting into a plurality of image blocks according to a jigsaw mode, and adjusting each image block into I after random cutting _k Is of the size of (1) to obtain I _k Is a local image of (a);

is I _q Via the minimum scale feature of the encoder, +.>Representation and I _q Is opposite to I _k Via the minimum scale feature of the encoder, +.>Representation and I _q N-th I of negative pair _k Through the smallest scale feature of the encoder.

In the detection module, the rule for converting the trained self-supervision model into the defect detection model is as follows:

the key set and the value set of the self-supervision model and the defect detection model after training are converted according to the fuzzy matching principle;

the fuzzy matching principle is as follows: key set C of defect detection model _dk As a standard key set, a trained self-supervision model key set C _uk As a set of matching keys, remove C _uk The initial structure name of (C) _uk The remaining key names and C _dk Performing full character matching on key names in the key numbers to obtain key name matching pairsK in key name matching pair _d And k _u Corresponding v _u Storing the defect detection model in a key value pair mode;

k _d and k _u Form key name matching pairs, k _d Is C _dk Key name, k of (a) _u Is C _uk Key name, v of (v) _u Is a value in the trained self-supervising model value set.

A computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to perform a simplified and generalized catenary defect detection method.

The invention has the beneficial effects that: the invention adopts a mixed cutting mode to amplify the simplified and unified contact network image, carries out multi-scale feature extraction on the simplified and unified contact network image, trains an upstream self-supervision model by utilizing multi-scale feature contrast loss and dense contrast loss, converts the trained self-supervision model into a defect detection model to be trained at the downstream, detects the simplified and unified contact network image to be detected by adopting the trained defect detection model, can obtain a high-precision detection result, and is suitable for the conditions of lack of target samples and more defect categories.

Drawings

Fig. 1 is a schematic diagram of a simplified catenary defect detection method.

Detailed Description

The invention is further described below with reference to the accompanying drawings. The following examples are only for more clearly illustrating the technical aspects of the present invention, and are not intended to limit the scope of the present invention.

A simplified contact net defect detection method comprises the following steps:

step 1, acquiring a simplified and unified contact net image;

step 2, inputting the simplified and unified contact net image into a pre-trained defect detection model to obtain a simplified and unified contact net defect detection result; the defect detection model is formed by converting a trained self-supervision model, a mixed cutting mode is adopted to amplify a simplified and unified contact network image in a training set when the self-supervision model is trained, multi-scale feature extraction is carried out on the simplified and unified contact network image, and the training is carried out by utilizing multi-scale feature contrast loss and dense contrast loss according to the extracted multi-scale features.

The method comprises the steps of adopting a mixed cutting mode to amplify a simplified and unified contact network image, carrying out multi-scale feature extraction on the simplified and unified contact network image, training an upstream self-supervision model by utilizing multi-scale feature contrast loss and dense contrast loss, converting the trained self-supervision model into a downstream defect detection model to be trained, and detecting the simplified and unified contact network image to be detected by adopting the trained defect detection model, so that a high-precision detection result can be obtained; and because the image augmentation is carried out by the method, the method is suitable for the condition that the target sample is lacking, the multi-scale feature extraction module has a large framework, the feature information contained in the image is extracted more completely, the more complex detection problem can be solved, and the method is suitable for the condition that the defect types are more.

In the method, the self-supervision model is a multi-scale dense contrast enhancement self-supervision network, the defect detection model is a target detection network compatible with an upstream self-supervision network, the self-supervision model is required to be trained firstly, then the trained self-supervision model is subjected to transfer learning to obtain a defect detection model to be trained at the downstream, the defect detection model is trained, and the trained defect detection model is adopted to carry out simplified contact net defect detection.

As shown in fig. 1, the self-supervision model is trained by adopting a simplified and unified overhead line system conventional image, the simplified and unified overhead line system conventional image is a label-free image, features are extracted by adopting PCA principal component analysis according to feature similarity of the images, aggregation is carried out by a K-means method, the images are classified according to the aggregation degree, and the images are classified into a training set and a verification set according to a preset proportion.

The simplified and unified contact net image in the training set is enhanced by utilizing a mixed cutting mode in the training, and the method is specifically as follows:

1) In the preset iteration times, cutting out simplified and unified contact network images in the training set by adopting a random cutting method, and amplifying the simplified and unified contact network images in the training set by using the cut-out images;

the original image is cut according to random size and random length-width ratio, then the picture is adjusted to the set pixel size (800 ), and the brightness, contrast, gray level and turnover degree of the image are adjusted according to a certain probability to be used as the augmented image of the original image.

2) Updating parameters of the self-supervision model according to the weight of the self-supervision model outside the preset iteration times, adopting a comparison cutting method to correct position judgment of the cutting frame, cutting the simplified and unified contact network image in the training set according to the cutting frame after the position correction, and using the cut image to augment the simplified and unified contact network image in the training set;

in the preset iteration times, the position of an object in an original image is initially learned through a random clipping method, the position is taken as the center outside the preset iteration times, the sensitivity is added to a central neighborhood through histogram change, the higher sensitivity is added to the neighborhood which tends to be consistent with a central pixel, otherwise, the lower sensitivity is given, a clipping frame is biased to the neighborhood side with the center and the higher sensitivity for clipping, then the image is adjusted to the set pixel size (800 ), and the brightness, the contrast, the gray level and the turnover degree of the image are adjusted according to a certain probability to serve as an enlarged image of the original image.

The same image can generate two global images I after being amplified by a mixed clipping mode _q 、I _k ，I _q 、I _k For two different views, I will be _q 、I _k As input, performing multiple scale feature extraction to obtain I _q 、I _k The contrast loss function between the two can be processed as follows:

a1 I) will be I _q 、I _k Fed into encoder f _q Respectively obtain N _m The method comprises the steps of (1) sending the feature images with the different scales into multi-layer perceptron (MLP) which are not shared by parameters, and splicing the output of each MLP according to the dimension.

A2 I) to be different images _q 、I _k Is a multi-scale feature of (2)Into a negative sample queue, key encoder f _k Parameter initialization time and f _q The parameters of (2) remain consistent and the parameters do not participate in back propagation; wherein the key encoder is operated for consistency, i.e. f is desired _q Can be from similar or identical f _k Sample characteristics are obtained, and the effectiveness of comparison is ensured.

A3)As query representation +.>As->Similar sample representation, ++>As->Different sample representations, if I _q 、I _k From the same image, then it is indicated as positive, otherwise it is indicated as negative.

To ensure thatAnd->The similarity between them is as large as possible, but with other +.>The similarity between the two images is as small as possible, and the global image I of the same image _q 、I _k The contrast loss function between is:

wherein,,is I _q 、I _k A comparison loss function calculation value between +.>Is I _q Through encoder f _q Is the ith dimension characteristic of>Representation and I _q Is opposite to I _k Through encoder f _q Is the ith dimension characteristic of>Is equal to I _q N-th I of negative pair _k Through encoder f _q Is the i-th dimension characteristic of K is the negative sample queue length, N _m For the number of multi-scale features, T is a temperature super parameter, so that negative samples with higher similarity are far away, and the obtained representation space is more uniform.

Will I _q 、I _k As input, partial images P are obtained by means of tile enhancement, respectively _q 、P _k The method specifically comprises the following steps: i _k Splitting into a plurality of image blocks according to a jigsaw mode, and adjusting each image block into I after random cutting _k Is of the size of (1) to obtain I _k Is a local image P of (2) _k ；I _q Splitting into a plurality of image blocks according to a jigsaw mode, and adjusting each image block into I after random cutting _q Is of the size of (1) to obtain I _q Is a local image P of (2) _q . Further extracting multiple scale features to obtain P _q 、P _k The contrast loss function between the two can be processed as follows:

b1 I) will be I _q 、I _k Splitting an image into a plurality of partial images in a jigsaw manner, and feeding the partial images into an encoder f _q Respectively obtain N _m Feature map of=3 different scales, different scalesThe feature map of the degree is sent into multi-layer perceptron MLP which are not shared with each other, and the output of each MLP is spliced according to the dimension.

B2 P) to be different pictures _q 、I _k Is a multi-scale feature of (2)Into a negative sample queue, key encoder f _k Parameter initialization time and f _q Is kept consistent and its parameters do not participate in the back propagation.

B3)As query representation +.>As->Similar sample representation, ++>As->Different sample representations, if P _q 、I _k From the same image, then it is indicated as positive, otherwise it is indicated as negative.

To ensure thatAnd->The similarity between them is as large as possible, but with other +.>The similarity between the two images is as small as possible, and the partial image P of the same image _q With global image I _k The contrast loss function between is:

wherein,,is P _q And I _k A comparison loss function calculation value between +.>Is P _q Through encoder f _q Is the j-th dimensional feature of>Representation and P _q Is opposite to I _k Through encoder f _q Is the j-th dimensional feature of>Representation and P _q N-th I of negative pair _k Through encoder f _q Is the i-th dimensional feature of (a).

B4 P) of the same image _q 、P _k Is a multi-scale feature of (2) Put into the encoder f _q In which P of different pictures _q 、P _k Multi-scale features of->Into a negative sample queue, key encoder f _k Parameter initialization time and f _q Is kept consistent and its parameters do not participate in the back propagation.

B5)f _q In (a) and (b)As query representation +.>As->Similar sample representation, ++>As->Different sample representations, if P _q 、P _k From the same image, then it is indicated as positive, otherwise it is indicated as negative.

To ensure thatAnd->The similarity between them is as large as possible, but with other +.>The similarity between the two images is as small as possible, and the partial image P of the same image _q 、P _k The contrast loss function between is:

wherein,,is P _q 、P _k A comparison loss function calculation value between +.>Is P _q Through encoder f _q Is the kth dimensional feature of>Representation and P _q P is opposite to _k Through encoder f _q Is the kth dimensional feature of>Representation and P _q N-th P of negative pair _k Through encoder f _q Is the kth dimensional feature of (c).

Will I _q 、I _k As input, extracting multi-scale features, and sending the feature with minimum scale, namely the general feature map, into Dense perceptron Dense MLP to obtain global image I _q 、I _k The dense contrast loss function between the model and the model is combined with the multi-scale characteristic contrast loss function to obtain the loss function of the contrast loss module, and the loss function is used as the total loss function of the model, and the self-supervision model for transfer learning is obtained after iteration, and the process can be as follows:

c1 I) will be I _q 、I _k Respectively sent to the encoder f _q Respectively obtain N _m The feature map with the minimum scale is sent to the dense sensor, and the dense sensor adopts a convolution layer to replace a linear layer to output dense feature information.

C2 Will) bePut-in key encoder f _k Key encoder f _k Parameter initialization direct replication f _q And its parameters do not participate in the back propagation.

C3)As query representation +.>As->Similar sample tableShow (I)>As->Different sample representations, if +.>Features from the same image are represented as positive pairs, otherwise as negative pairs.

To ensure thatAnd->The similarity between them is as large as possible, but with other +.>The similarity between the two images is as small as possible, I of the same image _q 、I _k The dense contrast loss function between is:

wherein L is _dc For global image I _q 、I _k The dense contrast loss function between them calculates a value,is I _q Through encoder f _q Minimum dimensional features of->Representation and I _q Is opposite to I _k Through encoder f _q Minimum dimensional features of->Representation and I _q N-th I of negative pair _k Through encoder f _q Is a minimum scale feature of (2).

When the image in the batch enters f _q Then, the negative-sample multi-scale characteristics of the image are sent to the tail of a negative-sample queue, when the length of the negative-sample queue reaches the upper limit, the head of the queue is removed, and more negative samples are read in a queue updating mode; f (f) _k The parameter is f _q Momentum update, update formula is:

m(1-θ _q )+mθ _k

wherein m is a dynamic value, θ _q Is f _q Network structure parameters, θ _k Is f _k Network structure parameters of (a).

The encoder is a specific network architecture of the multi-scale feature extraction module, and consists of a 53-layer backbone network, 3 multi-layer perceptron MLPs and a Dense perceptron Dense MLP, wherein the encoder f _q Key encoder f _k Is consistent with the architecture of f _k The parameter is f _q According to m (1-theta) _q )+mθ _k The momentum of the data is updated and,and->Is different from the encoder of the encoder in that only the passing sensor is different.

C4 Combining the multi-scale characteristic contrast loss function and the dense contrast loss function to obtain a total loss function, and obtaining a self-supervision model for transfer learning after iteration.

The loss function of the self-supervision model is:

wherein L is a self-supervision model loss function calculated value, N _M =9 is the number of multiscale feature contrast losses, N _L =10 is the total loss number, L _mfc Calculating a value, W, for a multi-scale feature contrast loss function _i Weight for i-th scale feature contrast loss, W _j Weight for j-th scale feature contrast loss, W _k The lost weighting weights are compared for the j-th scale feature.

Performing migration learning by using a trained upstream self-supervision model, converting the model into a downstream defect detection model to be trained, wherein the conversion rule between the model and the defect detection model is as follows:

the key set and the value set of the self-supervision model and the defect detection model after training are converted according to the fuzzy matching principle; the fuzzy matching principle is as follows: key set C of defect detection model _dk As a standard key set, a trained self-supervision model key set C _uk As a set of matching keys, remove C _uk The initial structure name of (C) _uk The remaining key names and C _dk Performing full character matching on key names in the key name matching pair, obtaining a key name matching pair, and matching k in the key name matching pair _d And k _u Corresponding v _u Storing the defect detection model in a key value pair mode; k (k) _d And k _u Form key name matching pairs, k _d Is C _dk Key name, k of (a) _u Is C _uk Key name, v of (v) _u Is a value in the trained self-supervising model value set.

Specifically, k is _u ∈C _uk In the layer structure name starting with 'encoder_k', k is removed _u ∈C _uk The layer structure name containing 'mlp' is removed and k is removed _u ∈C _uk Layer structure name and k starting from' encoder_q _d ∈C _dk The layer structure names in (a) are subjected to full character matching, and the matched k is obtained _u Change to k _d And let k _u Corresponding v _u And storing the key value pairs into a pre-training model, and skipping if the key value pairs are not matched.

The defect detection model is trained by adopting a simplified and unified contact net defect image, and the defect detection model is specifically as follows:

d1 A training set and a verification set for simplifying and unifying defects of the tube caps, insulator dirt, backup and parent defects of the overhead contact system are manufactured.

D2 After the defect image is adjusted, the defect image is sent to a defect detection module, and a defect detection model is trained through iteration.

The loss function of the defect detection model is:

L _s ＝BCE(obj _y ,obj _s )+BCE(cls _y ,cls _s )+IOU(box _y ,box _s )

wherein L is _s For the defect detection model loss function calculation value, BCE represents a BCEWITHLogitsLoss binary cross entropy loss function, obj _y Indicating whether the defect detection model predicts the input image to be defective or not, obj _s Indicating whether the input image of the defect detection model has defects actually, cls _y Representing the defect class predicted by the defect detection model, cls _s For the actually existing defect category, IOU represents IOULSO loss function for detecting prediction box _y And a real box _s Overlapping area between.

And after training is finished, defect detection can be directly carried out, a simplified and unified contact network image is obtained, the obtained simplified and unified contact network image is input into a trained defect detection model, a simplified and unified contact network defect detection result is obtained, and a defect type, a confidence coefficient and a prediction frame are specifically obtained.

The method can obtain a high-precision detection result and is suitable for the conditions of lack of target samples and more defect types.

Based on the same technical scheme, the invention also discloses a software system of the method, a simplified and unified contact net defect detection system, which comprises:

and the acquisition module acquires a simplified and unified contact network image.

The detection module inputs the simplified and unified contact net image into a pre-trained defect detection model to obtain a simplified and unified contact net defect detection result; the defect detection model is formed by converting a trained self-supervision model, a mixed cutting mode is adopted to amplify a simplified and unified contact network image in a training set when the self-supervision model is trained, multi-scale feature extraction is carried out on the simplified and unified contact network image, and the training is carried out by utilizing multi-scale feature contrast loss and dense contrast loss according to the extracted multi-scale features.

In the detection module, the simplified and unified contact network image in the training set is enhanced by adopting a mixed cutting mode, and the method comprises the following steps:

in the preset iteration times, cutting out simplified and unified contact network images in the training set by adopting a random cutting method, and amplifying the simplified and unified contact network images in the training set by using the cut-out images;

and updating parameters of the self-supervision model according to the weight of the self-supervision model outside the preset iteration times, adopting a comparison cutting method to correct the position judgment of the cutting frame, cutting the simplified and unified contact network image in the training set according to the cutting frame after the position correction, and using the cut image to augment the simplified and unified contact network image in the training set.

In the detection module, the loss function of the self-supervision model is as follows:

wherein L is a self-supervision model loss function calculated value, N _M For the number of multi-scale feature contrast losses, N _L L is the total loss number _mfc Calculating a value for a multi-scale feature contrast loss function, L _dc For global image I _q 、I _k Intensive contrast loss function calculation value between, global image I _q 、I _k Different views generated by the same simplified contact net image in a mixed cutting mode are obtained;

W _i weight for i-th scale feature contrast loss, W _j Weight for j-th scale feature contrast loss, W _k Weighting the loss for the j-th scale feature contrast, N _m The number of the multi-scale features;

is I _q 、I _k A comparison loss function calculation value between +.>Is I _q Through the ith dimension feature of the encoder, < >>Representation and I _q Is opposite to I _k Through the ith dimension feature of the encoder, T is the temperature super-parameter, +.>Is equal to I _q N-th I of negative pair _k Through the ith dimension feature of the encoder, K is the negative sample queue length;

is P _q And I _k A comparison loss function calculation value between +.>Is P _q Through the j-th dimension characteristic of the encoder, < >>Representation and P _q Is opposite to I _k Through the j-th dimension characteristic of the encoder, < >>Representation and P _q N-th I of negative pair _k Through the ith dimension feature, P, of the encoder _q Is I _q I is a local image of (1) _q Splitting into a plurality of image blocks according to a jigsaw mode, and adjusting each image block into I after random cutting _q Is of the size of (1) to obtain I _q Is a local image of (a);

is P _q 、P _k A comparison loss function calculation value between +.>Is P _q Through the kth dimension characteristic of the encoder, < >>Representation and P _q P is opposite to _k Through the kth dimension characteristic of the encoder, < >>Representation and P _q N-th P of negative pair _k Through the kth dimension characteristic, P, of the encoder _k Is I _k I is a local image of (1) _k Splitting into a plurality of image blocks according to a jigsaw mode, and adjusting each image block into I after random cutting _k Is of the size of (1) to obtain I _k Is a local image of (a);

is I _q Via the minimum scale feature of the encoder, +.>Representation and I _q Is opposite to I _k Via the minimum scale feature of the encoder, +.>Representation and I _q N-th I of negative pair _k Through the smallest scale feature of the encoder. Inspection and detectionIn the testing module, the rule for converting the trained self-supervision model into the defect detection model is as follows:

the key set and the value set of the self-supervision model and the defect detection model after training are converted according to the fuzzy matching principle;

the fuzzy matching principle is as follows: key set C of defect detection model _dk As a standard key set, a trained self-supervision model key set C _uk As a set of matching keys, remove C _uk The initial structure name of (C) _uk The remaining key names and C _dk Performing full character matching on key names in the key name matching pair, obtaining a key name matching pair, and matching k in the key name matching pair _d And k _u Corresponding v _u Storing the defect detection model in a key value pair mode;

k _d and k _u Form key name matching pairs, k _d Is C _dk Key name, k of (a) _u Is C _uk Key name, v of (v) _u Is a value in the trained self-supervising model value set.

The data processing flow of each module of the system is consistent with the corresponding steps of the method, and the description is not repeated here.

Based on the same technical solution, the present invention also discloses a computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to perform a simplified and unified catenary defect detection method.

Based on the same technical scheme, the invention also discloses a computing device, which comprises one or more processors, one or more memories and one or more programs, wherein the one or more programs are stored in the one or more memories and are configured to be executed by the one or more processors, and the one or more programs comprise instructions for executing the simplified catenary defect detection method.

It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The foregoing is illustrative of the present invention and is not to be construed as limiting thereof, but rather as providing for the use of additional embodiments and advantages of all such modifications, equivalents, improvements and similar to the present invention are intended to be included within the scope of the present invention as defined by the appended claims.

Claims (10)

1. The simplified contact net defect detection method is characterized by comprising the following steps of:

acquiring a simplified and unified contact net image;

inputting the simplified and unified contact net image into a pre-trained defect detection model to obtain a simplified and unified contact net defect detection result; the defect detection model is formed by converting a trained self-supervision model, and when the self-supervision model is trained, a mixed cutting mode is adopted to amplify a simplified and unified contact network image in a training set, multi-scale feature extraction is carried out on the simplified and unified contact network image, and multi-scale feature contrast loss and dense contrast loss are utilized for training.

2. The simplified and unified catenary defect detection method of claim 1, wherein the simplified and unified catenary image in the training set is augmented by adopting a hybrid clipping mode, comprising:

in the preset iteration times, cutting out simplified and unified contact network images in the training set by adopting a random cutting method, and amplifying the simplified and unified contact network images in the training set by using the cut-out images;

and updating parameters of the self-supervision model according to the weight of the self-supervision model outside the preset iteration times, adopting a comparison cutting method to correct the position judgment of the cutting frame, cutting the simplified and unified contact network image in the training set according to the cutting frame after the position correction, and using the cut image to augment the simplified and unified contact network image in the training set.

3. The simplified and unified catenary defect detection method of claim 1, wherein the loss function of the self-supervision model is:

wherein L is a self-supervision model loss function calculated value, N _M For the number of multi-scale feature contrast losses, N _L L is the total loss number _mfc Calculating a value for a multi-scale feature contrast loss function, L _dc For global image I _q 、I _k Intensive contrast loss function calculation value between, global image I _q 、I _k Different views generated by the same simplified contact net image in a mixed cutting mode are obtained;

W _i weight for i-th scale feature contrast loss, W _j Weight for j-th scale feature contrast loss, W _k Weighting the loss for the j-th scale feature contrast, N _m The number of the multi-scale features;

is I _q 、I _k A comparison loss function calculation value between +.>Is I _q Through the ith dimension feature of the encoder, < >>Representation and I _q Is opposite to I _k Through the ith dimension feature of the encoder, T is the temperature super-parameter, +.>Is equal to I _q N-th I of negative pair _k Through the ith dimension feature of the encoder, K is the negative sampleA queue length;

is P _q And I _k A comparison loss function calculation value between +.>Is P _q Through the j-th dimension characteristic of the encoder, < >>Representation and P _q Is opposite to I _k Through the j-th dimension characteristic of the encoder, < >>Representation and P _q N-th I of negative pair _k Through the ith dimension feature, P, of the encoder _q Is I _q I is a local image of (1) _q Splitting into a plurality of image blocks according to a jigsaw mode, and adjusting each image block into I after random cutting _q Is of the size of (1) to obtain I _q Is a local image of (a);

is P _q 、P _k A comparison loss function calculation value between +.>Is P _q Through the kth dimension characteristic of the encoder, < >>Representation and P _q P is opposite to _k Through the kth dimension characteristic of the encoder, < >>Representation and P _q N-th P of negative pair _k Through the kth dimension characteristic, P, of the encoder _k Is I _k I is a local image of (1) _k Splitting into a plurality of image blocks according to a jigsaw mode, and adjusting each image block into I after random cutting _k Is of the size of (1) to obtain I _k Is a local image of (a);

is I _q Via the minimum scale feature of the encoder, +.>Representation and I _q Is opposite to I _k Via the minimum scale feature of the encoder, +.>Representation and I _q N-th I of negative pair _k Through the smallest scale feature of the encoder.

4. The simplified and unified catenary defect detection method of claim 1, wherein the rules for converting the trained self-supervision model into the defect detection model are as follows:

the key set and the value set of the self-supervision model and the defect detection model after training are converted according to the fuzzy matching principle;

the fuzzy matching principle is as follows: key set C of defect detection model _dk As a standard key set, a trained self-supervision model key set C _uk As a set of matching keys, remove C _uk The initial structure name of (C) _uk The remaining key names and C _dk Performing full character matching on key names in the key name matching pair, obtaining a key name matching pair, and matching k in the key name matching pair _d And k _u Corresponding v _u Storing the defect detection model in a key value pair mode;

k _d and k _u Form key name matching pairs, k _d Is C _dk Key name, k of (a) _u Is C _uk Key name, v of (v) _u Is a value in the trained self-supervising model value set.

5. The simplified and unified catenary defect detection method of claim 1, wherein the defect detection model has a loss function of:

L _s ＝BCE(obj _y ,obj _s )+BCE(cls _y ,cls _s )+IOU(box _y ,box _s )

wherein L is _s For the defect detection model loss function calculation value, BCE represents a BCEWITHLogitsLoss binary cross entropy loss function, obj _y Indicating whether the defect detection model predicts the input image to be defective or not, obj _s Indicating whether the input image of the defect detection model has defects actually, cls _y Representing the defect class predicted by the defect detection model, cls _s For the actually existing defect category, IOU represents IOULSO loss function for detecting prediction box _y And a real box _s Overlapping area between.

6. A simplified and unified catenary defect detection system, comprising:

the acquisition module is used for acquiring a simplified and unified contact net image;

the detection module inputs the simplified and unified contact net image into a pre-trained defect detection model to obtain a simplified and unified contact net defect detection result; the defect detection model is formed by converting a trained self-supervision model, and when the self-supervision model is trained, a mixed cutting mode is adopted to amplify a simplified and unified contact network image in a training set, multi-scale feature extraction is carried out on the simplified and unified contact network image, and multi-scale feature contrast loss and dense contrast loss are utilized for training.

7. The simplified and unified catenary defect detection system of claim 6, wherein the simplified and unified catenary image in the training set is augmented by hybrid clipping in the detection module, comprising:

in the preset iteration times, cutting out simplified and unified contact network images in the training set by adopting a random cutting method, and amplifying the simplified and unified contact network images in the training set by using the cut-out images;

and updating parameters of the self-supervision model according to the weight of the self-supervision model outside the preset iteration times, adopting a comparison cutting method to correct the position judgment of the cutting frame, cutting the simplified and unified contact network image in the training set according to the cutting frame after the position correction, and using the cut image to augment the simplified and unified contact network image in the training set.

8. The simplified and unified catenary defect detection system of claim 6, wherein in the detection module, the loss function of the self-supervision model is:

wherein L is a self-supervision model loss function calculated value, N _M For the number of multi-scale feature contrast losses, N _L L is the total loss number _mfc Calculating a value for a multi-scale feature contrast loss function, L _dc For global image I _q 、I _k Intensive contrast loss function calculation value between, global image I _q 、I _k Different views generated by the same simplified contact net image in a mixed cutting mode are obtained;

W _i weight for i-th scale feature contrast loss, W _j Weight for j-th scale feature contrast loss, W _k Weighting the loss for the j-th scale feature contrast, N _m The number of the multi-scale features;

is I _q 、I _k A comparison loss function calculation value between +.>Is I _q Through the ith dimension feature of the encoder, < >>Representation and I _q Is opposite to I _k Through the ith dimension feature of the encoder, T is the temperature super-parameter, +.>Is equal to I _q N-th I of negative pair _k Through the ith dimension feature of the encoder, K is the negative sample queue length;

is P _q And I _k A comparison loss function calculation value between +.>Is P _q Through the j-th dimension characteristic of the encoder, < >>Representation and P _q Is opposite to I _k Through the j-th dimension characteristic of the encoder, < >>Representation and P _q N-th I of negative pair _k Through the ith dimension feature, P, of the encoder _q Is I _q I is a local image of (1) _q Disassembling according to jigsaw modeDividing into a plurality of image blocks, randomly cutting each image block, and adjusting the image block into I _q Is of the size of (1) to obtain I _q Is a local image of (a);

is P _q 、P _k A comparison loss function calculation value between +.>Is P _q Through the kth dimension characteristic of the encoder, < >>Representation and P _q P is opposite to _k Through the kth dimension characteristic of the encoder, < >>Representation and P _q N-th P of negative pair _k Through the kth dimension characteristic, P, of the encoder _k Is I _k I is a local image of (1) _k Splitting into a plurality of image blocks according to a jigsaw mode, and adjusting each image block into I after random cutting _k Is of the size of (1) to obtain I _k Is a local image of (a);

is I _q Via the minimum scale feature of the encoder, +.>Representation and I _q Is opposite to I _k Via the minimum scale feature of the encoder, +.>Representation and I _q N-th I of negative pair _k Through the smallest scale feature of the encoder.

9. The simplified and unified catenary defect detection system of claim 6, wherein in the detection module, the rule for converting the trained self-supervision model into the defect detection model is as follows:

the key set and the value set of the self-supervision model and the defect detection model after training are converted according to the fuzzy matching principle;

the fuzzy matching principle is as follows: key set C of defect detection model _dk As a standard key set, a trained self-supervision model key set C _uk As a set of matching keys, remove C _uk The initial structure name of (C) _uk The remaining key names and C _dk Performing full character matching on key names in the key name matching pair, obtaining a key name matching pair, and matching k in the key name matching pair _d And k _u Corresponding v _u Storing the defect detection model in a key value pair mode;

k _d and k _u Form key name matching pairs, k _d Is C _dk Key name, k of (a) _u Is C _uk Key name, v of (v) _u Is a value in the trained self-supervising model value set.

10. A computer readable storage medium storing one or more programs, wherein the one or more programs comprise instructions, which when executed by a computing device, cause the computing device to perform any of the methods of claims 1-5.