CN115880224A - Crack detection method based on automatic filling of pseudo labels - Google Patents
Crack detection method based on automatic filling of pseudo labels Download PDFInfo
- Publication number
- CN115880224A CN115880224A CN202211405696.7A CN202211405696A CN115880224A CN 115880224 A CN115880224 A CN 115880224A CN 202211405696 A CN202211405696 A CN 202211405696A CN 115880224 A CN115880224 A CN 115880224A
- Authority
- CN
- China
- Prior art keywords
- sample
- stage
- image
- crack
- prediction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides a crack detection method based on automatic filling of pseudo labels, which comprises the steps of training a lightweight target detector by using a data sample, obtaining network model weight, carrying out one-stage crack detection on a newly acquired image by using the trained network model weight, screening a suboptimal prediction sample as a candidate region for two-stage detection, aiming at the suboptimal prediction sample, improving the image identification degree by processing methods such as wiener filtering and morphology, carrying out two-stage crack detection on cracks by using a Canny edge detection operator, and integrating prediction results of the two stages to realize final crack detection; in addition, high-confidence samples of the prediction results in the first stage are screened to serve as pseudo labels, the confidence of suboptimal prediction samples is improved according to the prediction results in the second stage, the pseudo label samples are filled, and the network model is retrained by using label data and pseudo label data. The method has high robustness and strong generalization and is more suitable for target detection tasks.
Description
Technical Field
The invention relates to the technical field of crack detection, in particular to a crack detection method based on automatic filling of a pseudo label.
Background
The method for intelligently detecting the cracks is extremely important for avoiding unnecessary harm because the cracks appear due to the reasons of overlong service life, ground subsidence, substandard quality and the like, and the cracks can cause instability and even collapse of a building structure once exceeding a safety threshold value.
The early intelligent crack detection method mainly depends on background frame difference, related filtering, incremental learning, sparse representation and other methods to detect cracks of the to-be-detected area, and the detection method is high in speed but low in precision. With the improvement of the computing power of a computer, particularly the development of a Graphics Processing Unit (GPU), the wide application of deep learning in target detection is promoted, and the target characterization mode is also changed. However, the detection method based on deep learning tends to be large-scale modeling, requires a large number of label data samples for training, is not friendly to the detection method based on small data samples, and is easy to cause false detection and missed detection.
Disclosure of Invention
Therefore, the invention aims to provide a crack detection method based on automatic filling of a pseudo label, which can reliably judge a target to be detected, automatically expand a label data sample and improve the target detection precision.
The technical scheme adopted by the invention is as follows:
a crack detection method based on automatic filling of pseudo labels comprises the following steps:
step 1, acquiring a plurality of crack images in different scenes, marking the cracks in the images by a labellimg marking tool, and constructing a data set sample library;
step 2, training data samples based on the light-weight target detector to obtain network model weight;
step 3, performing one-stage crack detection on the newly acquired image database based on the trained network model weight, and screening out high-confidence samples and suboptimal samples;
step 4, setting a mask to obtain an image in the suboptimal sample area, and enhancing the image identification degree by using a morphological processing and related filtering processing method;
step 5, performing two-stage crack detection on the suboptimal sample by using a Canny edge detection operator, and optimizing a detection result;
and 6, acquiring a final detection result, constructing a new data sample library by using the data samples with the same labels by using the first-stage high-confidence-degree sample and the two-stage optimized sample as pseudo labels, and retraining a network model to be used as a new weight file for the first-stage detection.
Further, in step 2, the light-weight target detector is constructed by a light-weight convolution network, a feature fusion model and a classification detector;
the lightweight convolution network is used for learning image features, the convolution with convolution kernel of 3 x 3 and step length of 1 is adopted, the maximum pooling of convolution kernel of 3 x 3 and step length of 2 is adopted, 12 levels are stacked for realization, and nonlinear activation and normalization processing are carried out on the output of each layer of convolution;
the feature fusion model is used for fusing the output of a target apparent feature layer and a depth semantic feature layer, the target apparent feature layer is a shallow convolution layer of a light-weight convolution network and can acquire position information and contour edge information of a target, the depth semantic feature layer is a deep convolution layer of the light-weight convolution network and can acquire texture, color and category attribute information of the target, the fusion process is realized by bicubic interpolation and matrix operation, the bicubic interpolation is used for unifying the resolution of the apparent feature layer and the depth semantic feature layer, and the realization process is as follows:
wherein, (i, j) is the coordinate of a pixel point in the original image, v, u are the row offset and the column offset respectively, x, y are used to obtain the coordinates of (i, j) adjacent 16 pixels, the range is { -1,0,1,2}, K (·) is an interpolation formula, and the expression is as follows:
wherein a is a weighting coefficient with a value of-0.75,
the feature normalization processing of the apparent feature layer output after the uniform resolution is used as a correction unit to supplement target position information and edge contour information missing from the depth semantic feature layer, and the implementation process is as follows:
F(u ij )＝F 1 (u ij )·[σ(F 2 (u ij ))+1]
wherein u is ij As image characteristic response values, F 1 、F 2 Depths of uniform resolution are respectively set;
the sigma is a sigmoid activation function;
the classification detector is used for final result prediction, and through setting large, medium and small anchor frames with different sizes and performing class prediction through a Logistic classifier and non-maximum value inhibition, the optimal prediction frame and class confidence are obtained.
Further, in step 3, the high confidence sample and the sub-optimal sample are screened, and the processing of the prediction result of the light-weight target detector comprises:
high confidence samples: the confidence coefficient range of the object class in the predicted frame is 0.95-1.00;
sub-optimal samples: the confidence degree range of the object class in the predicted frame is 0.85-0.95.
Further, in step 5, the processing step of enhancing the image recognition of the suboptimal sample and improving the confidence of the predicted sample includes:
a) Setting a mask, and acquiring an image in the suboptimal sample area according to the suboptimal sample frame position;
b) Graying the image by using the weighted average value;
c) Enhancing the image, namely enhancing the image by utilizing gray histogram equalization;
d) Denoising the image, namely denoising the image by adopting wiener filtering;
e) Detecting a crack, namely performing edge detection on the denoised image by using a Canny operator to obtain a crack target;
f) And (4) result regression, identifying a crack area through expansion and corrosion morphological characteristics according to the crack target, and finally predicting a frame through calculating a rectangle regression externally connected with the crack target.
Further, in step 6, the method for obtaining the final judgment crack prediction result includes:
and correcting the frame position predicted by the suboptimal sample in the first-stage detection result according to the second-stage detection result, and calculating as follows:
x 1 ＝x 1 +x′ 1
y 1 ＝y 1 +y′ 1
x 2 ＝x 2 -x′ 2
y 2 ＝y 2 -y′ 2
wherein (x) 1 ,y 1 )、(x 2 ,y 2 ) Respectively obtaining the top left corner point and the bottom right corner point of the suboptimal sample region in the mask, (x' 1 ,y' 1 )、(x' 2 ,y' 2 ) Predicting the upper left corner point and the lower right corner point of the frame in two stages;
and outputting the frame of the prediction sample with high confidence coefficient in the first-stage detection and the frame of the suboptimal prediction sample with two-stage correction as a final prediction result.
Further, in step 6, the method for training the new network model weight file is as follows:
a) And (4) taking the prediction sample with high confidence degree in the first-stage prediction result and the suboptimal sample optimized in the second stage as crack labels, and expanding the original data sample library.
b) And (4) carrying out disorder processing on the expanded data sample library, and repeating the step (2) to randomly extract images in the sample library for training until all the images in the sample library are trained.
Compared with the prior art, the invention has the beneficial effects that:
1. robustness, the depth characteristics and superficial apparent characteristics of the data sample are comprehensively considered by the network model, and the characteristics of the target to be detected can be more completely represented; a multi-scale classification detector is constructed, and the deformation change of an object to be detected can be adaptively responded; and a two-stage detection optimizer is built, and the result of the one-stage prediction is optimized by the traditional image enhancement, edge detection and other methods, so that the class confidence and the detection precision of the prediction result can be improved. Finally, through the detection scheme, the method has a better detection effect.
2. Generalization, the invention automatically fills the pseudo label as a training data sample, is suitable for crack detection in different scenes, can attach the trained network model weight to other data samples for continuous training, and can continuously iteratively optimize the network model.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is apparent that the drawings in the following description are only preferred embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without inventive efforts.
Fig. 1 is a schematic overall flow chart of a crack detection method based on automatic filling of pseudo labels according to an embodiment of the present invention.
Fig. 2 is a schematic diagram of a phase processing result of a crack detection method based on automatic filling of pseudo labels according to an embodiment of the present invention.
Fig. 3 is an exemplary diagram of two-stage processing results of a crack detection method based on automatic filling of pseudo labels according to an embodiment of the present invention.
Detailed Description
The principles and features of this invention are described below in conjunction with the following drawings, the illustrated embodiments are provided to illustrate the invention and not to limit the scope of the invention.
Referring to fig. 1,2 and 3, the invention provides a crack detection method based on automatic filling of a pseudo label, which comprises the following steps:
step 1, acquiring a plurality of crack images in different scenes, marking the cracks in the images by a labellimg marking tool, and constructing a data set sample library;
step 2, training data samples based on the light-weight target detector to obtain network model weight;
step 3, performing one-stage crack detection on the newly acquired image database based on the trained network model weight, and screening out a high-confidence sample and a suboptimal sample;
step 4, setting a mask to obtain an image in the suboptimal sample area, and enhancing the image identification degree by using a morphological processing and related filtering processing method;
step 5, performing two-stage crack detection on the suboptimal sample by using a Canny edge detection operator to optimize a detection result;
and 6, acquiring a final detection result, constructing a new data sample library by using the data samples with the same labels by using the first-stage high-confidence-degree sample and the two-stage optimized sample as pseudo labels, and retraining a network model to be used as a new weight file for the first-stage detection.
In step 1, the source of the image is obtained, which is mainly based on a camera shooting or web crawler, and the obtained image is cut, rotated, and noise-added to expand the data sample, and the crack in the image is labeled using a labellimg labeling tool.
In step 2, the light-weight target detector is constructed by a light-weight convolution network, a feature fusion model and a classification detector;
the lightweight convolution network is used for learning image features, the convolution with convolution kernel of 3 x 3 and step length of 1 is adopted, the maximum pooling of convolution kernel of 3 x 3 and step length of 2 is adopted, 12 levels are stacked for realization, and nonlinear activation and normalization processing are carried out on the output of each layer of convolution;
the feature fusion model is used for fusing the output of a target apparent feature layer and a depth semantic feature layer, the target apparent feature layer is a shallow convolution layer of a light-weight convolution network and can acquire position information and contour edge information of a target, the depth semantic feature layer is a deep convolution layer of the light-weight convolution network and can acquire texture, color and category attribute information of the target, the fusion process is realized by bicubic interpolation and matrix operation, the bicubic interpolation is used for unifying the resolution of the apparent feature layer and the depth semantic feature layer, and the realization process is as follows:
wherein, (i, j) is the coordinate of a pixel point in the original image, v, u are the row offset and the column offset respectively, x, y are used to obtain the coordinates of (i, j) adjacent 16 pixels, the range is { -1,0,1,2}, K (·) is an interpolation formula, and the expression is as follows:
wherein a is a weighting coefficient with a value of-0.75,
the feature normalization processing of the apparent feature layer output after the uniform resolution is used as a correction unit to supplement the missing target position information and edge contour information of the depth semantic feature layer, and the implementation process is as follows:
F(u ij )＝F 1 (u ij )·[σ(F 2 (u ij ))+1]
wherein u is ij As image characteristic response values, F 1 、F 2 Depths of uniform resolution are provided;
the sigma is a sigmoid activation function;
the classification detector is used for final result prediction, and through setting large, medium and small anchor frames with different sizes and performing class prediction through a Logistic classifier and non-maximum value inhibition, the optimal prediction frame and class confidence are obtained.
In step 3, high confidence samples and suboptimal samples are screened, and the processing of the prediction result of the light-weight target detector comprises the following steps:
high confidence samples: the confidence coefficient range of the object class in the predicted frame is 0.95-1.00;
sub-optimal samples: the confidence degree range of the object class in the predicted frame is 0.85-0.95.
In step 5, the processing step of enhancing the image recognition of the suboptimal sample and improving the confidence of the predicted sample comprises:
g) Setting a mask, and acquiring an image in the suboptimal sample area according to the position of the suboptimal sample frame;
h) Graying the image, namely graying the image by using a weighted average value;
i) Enhancing the image, namely enhancing the image by utilizing gray histogram equalization;
j) Denoising the image, namely denoising the image by adopting wiener filtering;
k) Detecting a crack, namely performing edge detection on the denoised image by using a Canny operator to obtain a crack target;
l) regression of results, identifying crack regions by expansion and corrosion morphological characteristics according to crack targets, and finally predicting frames by computing the regression of rectangles externally connected to the crack targets.
In step 6, the method for obtaining the final judgment crack prediction result comprises the following steps:
and correcting the frame position predicted by the suboptimal sample in the first-stage detection result according to the second-stage detection result, and calculating as follows:
x 1 ＝x 1 +x′ 1
y 1 ＝y 1 +y′ 1
x 2 ＝x 2 -x′ 2
y 2 ＝y 2 -y′ 2
wherein (x) 1 ,y 1 )、(x 2 ,y 2 ) Respectively obtaining the top left corner point and the bottom right corner point of the suboptimal sample region in the mask, (x' 1 ,y' 1 )、(x' 2 ,y' 2 ) Predicting the upper left corner point and the lower right corner point of the frame in two stages;
and outputting the frame of the prediction sample with high confidence coefficient in the first-stage detection and the frame of the suboptimal prediction sample with two-stage correction as a final prediction result.
In step 6, the method for training the new network model weight file is as follows:
c) And (4) taking the prediction sample with high confidence degree in the first-stage prediction result and the suboptimal sample optimized in the second stage as crack labels, and expanding the original data sample library.
d) And (4) carrying out disorder processing on the expanded data sample library, and repeating the step (2) to randomly extract images in the sample library for training until all the images in the sample library are trained.
The invention provides a crack detection method based on automatic filling of a pseudo label, which can quickly and accurately detect the crack in an image by integrating detection results of two stages and has stronger robustness. In addition, the invention can warn in advance before the crack reaches the safety threshold value, thereby avoiding unnecessary loss. In addition, the invention takes the accurate detection results in different scenes as new crack labels to be filled in the training data, is suitable for the crack detection in a plurality of scenes, does not need to consume manpower to mark cracks to expand data samples, and has stronger generalization performance.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and should not be taken as limiting the scope of the present invention, which is intended to cover any modifications, equivalents, improvements, etc. within the spirit and scope of the present invention.
Claims (6)
1. A crack detection method based on automatic filling of pseudo labels comprises the following steps:
step 1, acquiring a plurality of crack images in different scenes, marking the cracks in the images by a labellimg marking tool, and constructing a data set sample library;
2, training data samples in a data set sample library based on a light-weight target detector to obtain network model weight;
step 3, performing one-stage crack detection on the newly acquired image database based on the trained network model weight, and screening out a high-confidence sample and a suboptimal sample;
step 4, setting a mask to obtain an image in the suboptimal sample area, and enhancing the image identification degree by using a morphological processing and related filtering processing method;
step 5, performing two-stage crack detection on the suboptimal sample by using a Canny edge detection operator to optimize a detection result;
and 6, acquiring a final detection result, constructing a new data sample library by using the data samples with the same labels by using the first-stage high-confidence-degree sample and the two-stage optimized sample as pseudo labels, and retraining a network model to be used as a new weight file for the first-stage detection.
2. The crack detection method based on the automatic filling of the pseudo label is characterized in that in the step 2, a light-weight target detector is constructed by three parts, namely a light-weight convolution network, a feature fusion model and a classification detector;
the lightweight convolution network is used for learning image features, the convolution with convolution kernel of 3 x 3 and step length of 1 is adopted, the maximum pooling of convolution kernel of 3 x 3 and step length of 2 is adopted, 12 levels are stacked for realization, and nonlinear activation and normalization processing are carried out on the output of each layer of convolution;
the feature fusion model is used for fusing the output of a target apparent feature layer and a depth semantic feature layer, the target apparent feature layer is a shallow convolution layer of a light-weight convolution network and can acquire position information and contour edge information of a target, the depth semantic feature layer is a deep convolution layer of the light-weight convolution network and can acquire texture, color and category attribute information of the target, the fusion process is realized by bicubic interpolation and matrix operation, the bicubic interpolation is used for unifying the resolution of the apparent feature layer and the depth semantic feature layer, and the realization process is as follows:
wherein, (i, j) is the coordinate of a pixel point in the original image, v, u are the row offset and the column offset respectively, x, y are used to obtain the coordinates of (i, j) adjacent 16 pixels, the range is { -1,0,1,2}, K (·) is an interpolation formula, and the expression is as follows:
wherein a is a weighting coefficient with a value of-0.75,
the feature normalization processing of the apparent feature layer output after the uniform resolution is used as a correction unit to supplement the missing target position information and edge contour information of the depth semantic feature layer, and the implementation process is as follows:
F(u ij )＝F 1 (u ij )·[σ(F 2 (u ij ))+1]
wherein u is ij As image characteristic response values, F 1 、F 2 Depths of uniform resolution are provided;
the sigma is a sigmoid activation function;
the classification detector is used for final result prediction, and through setting large, medium and small anchor frames with different sizes and performing class prediction through a Logistic classifier and non-maximum value inhibition, the optimal prediction frame and class confidence are obtained.
3. The crack detection method based on automatic filling of pseudo labels as claimed in claim 2, wherein in step 3, high confidence samples and suboptimal samples are screened, and the processing of the prediction results of the light-weight target detector comprises:
high confidence samples: the confidence coefficient range of the object class in the predicted frame is 0.95-1.00;
sub-optimal samples: the confidence degree range of the object class in the predicted frame is 0.85-0.95.
4. The crack detection method based on automatic filling of pseudo labels as claimed in claim 3, wherein in step 5, the image recognition of the suboptimal sample is enhanced, and the processing step of improving the confidence of the predicted sample comprises:
a) Setting a mask, and acquiring an image in the suboptimal sample area according to the suboptimal sample frame position;
b) Graying the image, namely graying the image by using a weighted average value;
c) Enhancing the image, namely enhancing the image by utilizing gray histogram equalization;
d) Denoising the image, namely denoising the image by adopting wiener filtering;
e) Detecting a crack, namely performing edge detection on the denoised image by using a Canny operator to obtain a crack target;
f) And (4) result regression, identifying a crack area through expansion and corrosion morphological characteristics according to the crack target, and finally predicting a frame through calculating the external rectangular regression of the crack target.
5. The crack detection method based on the automatic filling of the pseudo label as claimed in claim 4, wherein in the step 6, the method for obtaining the final judgment crack prediction result is:
and correcting the frame position of the suboptimal sample prediction in the detection result of the first stage according to the detection result of the second stage, and calculating as follows:
x 1 ＝x 1 +x 1 ′
y 1 ＝y 1 +y 1 ′
x 2 ＝x 2 -x 2 ′
y 2 ＝y 2 -y 2 ′
wherein (x) 1 ,y 1 )、(x 2 ,y 2 ) Respectively obtaining the upper left corner point and the lower right corner point of the suboptimum sample region in the mask, (x' 1 ,y′ 1 )、(x' 2 ,y' 2 ) Predicting the upper left corner point and the lower right corner point of the frame in two stages;
and outputting the frame of the prediction sample with high confidence coefficient in the first-stage detection and the frame of the suboptimal prediction sample with two-stage correction as a final prediction result.
6. The crack detection method based on the automatic filling of the pseudo labels as claimed in claim 5, wherein in step 6, the method for training the new network model weight file is as follows:
a) Using a prediction sample with high confidence level in a first-stage prediction result and a second-stage optimized suboptimal sample as crack labels to expand an original data sample library;
b) And (4) carrying out disorder processing on the expanded data sample library, and repeating the step (2) to randomly extract images in the sample library for training until all the images in the sample library are trained.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211405696.7A CN115880224A (en) | 2022-11-10 | 2022-11-10 | Crack detection method based on automatic filling of pseudo labels |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211405696.7A CN115880224A (en) | 2022-11-10 | 2022-11-10 | Crack detection method based on automatic filling of pseudo labels |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115880224A true CN115880224A (en) | 2023-03-31 |
Family
ID=85759624
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211405696.7A Pending CN115880224A (en) | 2022-11-10 | 2022-11-10 | Crack detection method based on automatic filling of pseudo labels |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115880224A (en) |
-
2022
- 2022-11-10 CN CN202211405696.7A patent/CN115880224A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108256562B (en) | Salient target detection method and system based on weak supervision time-space cascade neural network | |
| CN111914698B (en) | Human body segmentation method, segmentation system, electronic equipment and storage medium in image | |
| CN109840483B (en) | Landslide crack detection and identification method and device | |
| CN107038416B (en) | Pedestrian detection method based on binary image improved HOG characteristics | |
| WO2019136591A1 (en) | Salient object detection method and system for weak supervision-based spatio-temporal cascade neural network | |
| CN111209858B (en) | Real-time license plate detection method based on deep convolutional neural network | |
| CN112287941B (en) | License plate recognition method based on automatic character region perception | |
| CN110910420A (en) | Moving target detection tracking method based on image stream | |
| CN113111878B (en) | Infrared weak and small target detection method under complex background | |
| CN107844772A (en) | A kind of motor vehicle automatic testing method based on movable object tracking | |
| CN112749673A (en) | Method and device for intelligently extracting stock of oil storage tank based on remote sensing image | |
| CN111985314B (en) | Smoke detection method based on ViBe and improved LBP | |
| CN107871315B (en) | Video image motion detection method and device | |
| CN113409355A (en) | Moving target identification system and method based on FPGA | |
| CN114219773A (en) | Pre-screening and calibration method for bridge crack detection data set | |
| CN114429577B (en) | Flag detection method, system and equipment based on high confidence labeling strategy | |
| CN111626145A (en) | Simple and effective incomplete form identification and page-crossing splicing method | |
| CN116740758A (en) | Bird image recognition method and system for preventing misjudgment | |
| CN113963333B (en) | Traffic sign board detection method based on improved YOLOF model | |
| CN111832497B (en) | Text detection post-processing method based on geometric features | |
| CN109409224B (en) | Method for detecting flame in natural scene | |
| CN116758421A (en) | Remote sensing image directed target detection method based on weak supervised learning | |
| CN115880224A (en) | Crack detection method based on automatic filling of pseudo labels | |
| CN115937095A (en) | Printing defect detection method and system integrating image processing algorithm and deep learning | |
| CN110910332B (en) | Visual SLAM system dynamic fuzzy processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |