CN116883362A - Crack detection method and system based on image recognition and image processing equipment - Google Patents
Crack detection method and system based on image recognition and image processing equipment Download PDFInfo
- Publication number
- CN116883362A CN116883362A CN202310857782.XA CN202310857782A CN116883362A CN 116883362 A CN116883362 A CN 116883362A CN 202310857782 A CN202310857782 A CN 202310857782A CN 116883362 A CN116883362 A CN 116883362A
- Authority
- CN
- China
- Prior art keywords
- image
- resolution
- low
- feature
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 66
- 238000012545 processing Methods 0.000 title claims abstract description 36
- 238000000605 extraction Methods 0.000 claims description 100
- 238000012549 training Methods 0.000 claims description 38
- 238000000034 method Methods 0.000 claims description 36
- 238000001914 filtration Methods 0.000 claims description 19
- 238000003062 neural network model Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 9
- 230000004927 fusion Effects 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 9
- 238000010586 diagram Methods 0.000 claims description 8
- 238000006731 degradation reaction Methods 0.000 claims description 6
- 230000000694 effects Effects 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 7
- 230000000007 visual effect Effects 0.000 description 5
- 230000015556 catabolic process Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000003709 image segmentation Methods 0.000 description 2
- 230000033001 locomotion Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000009194 climbing Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000010076 replication Effects 0.000 description 1
- 238000009958 sewing Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/97—Determining parameters from multiple pictures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Abstract
The application relates to a crack detection method, a crack detection system and image processing equipment based on image recognition, which belong to the technical field of image processing, in detail, in the application, an obtained original image to be detected is input into an image reconstruction network, multi-level refined features of the original image are extracted through the image reconstruction network to obtain a first feature image, low-frequency information of the original image is extracted to obtain a second feature image, super-resolution reconstruction is carried out on the first feature image and the second feature image, and a resolution reconstruction image is output; and carrying out crack detection on the resolution reconstructed image, and outputting a crack detection result. Therefore, on one hand, more refined features can be obtained by using the first feature map, on the other hand, smoother and real features can be obtained by using the second feature map, the resolution of an original image is improved, and the effect of crack detection by using a resolution reconstructed image with higher quality is better, so that the accuracy of crack detection is improved.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a crack detection method and system based on image recognition, and an image processing device.
Background
Crack detection is an important item in periodic inspection of concrete structures such as bridges and highways, and along with the development of visual detection, various image processing technologies have been used for detecting cracks on the surface of the concrete structures, for example, methods such as Sobel edge detector, canny edge detector, fourier transform, median filtering, mathematical morphology, histogram analysis and the like are applied.
However, the method of visual detection is used for detecting the crack, the quality of the image collected by the image collecting equipment such as a camera is dependent, if the collected image is blurred or has lower definition due to the problems of camera motion blur or camera gesture and the like, the method of visual detection is used for detecting the crack, so that the situation that the crack is misjudged or the crack cannot be identified easily occurs, and the accuracy of crack detection is lower.
Disclosure of Invention
In order to improve the accuracy of crack detection, the application provides a crack detection method, a crack detection system and image processing equipment based on image recognition.
In a first aspect, the present application provides a crack detection method based on image recognition, which adopts the following technical scheme:
a crack detection method based on image recognition comprises the following steps:
acquiring an original image to be detected;
inputting an original image into an image reconstruction network, extracting multi-level refined features of the original image through the image reconstruction network to obtain a first feature image, extracting low-frequency information of the original image to obtain a second feature image, and performing super-resolution reconstruction on the first feature image and the second feature image to output a resolution reconstruction image;
and carrying out crack detection on the resolution reconstructed image, and outputting a crack detection result.
Through adopting above-mentioned technical scheme, utilize the image reconstruction network, utilize first feature map can obtain more refinement characteristics on the one hand, on the other hand utilize the second feature map can obtain smoother and true characteristic for the resolution ratio reconstructed image that rebuilds according to first feature map and second feature map and obtain, improved the resolution ratio of original image again and be difficult for making original image distortion, utilize the higher resolution ratio reconstructed image of quality, carry out the effect of crack detection better, thereby improved crack detection's accuracy.
Optionally, the method further includes a training step of training a preset neural network model to obtain an image reconstruction network, where the image reconstruction network includes a refinement feature extraction module, a low-frequency information extraction module, and a reconstruction module; the training step comprises the following steps:
acquiring a training data set, and respectively carrying out blurring treatment on a plurality of sample crack images included in the training data set to obtain a low-resolution image;
calibrating a resolution loss label for the sample crack image according to a preset resolution loss index;
inputting the low-resolution image into a refined feature extraction module to generate a first feature map;
inputting the low-resolution image into a low-frequency information extraction module to generate a second feature map;
reconstructing the first feature map and the second feature map to generate a reconstructed image;
and calculating a resolution reconstruction result according to the sample crack image and the reconstruction image, and carrying out iterative updating on model parameters of the preset neural network model according to the resolution reconstruction result and the resolution loss label to obtain the image reconstruction network.
By adopting the technical scheme, the sample crack image in the training data set is subjected to blurring processing to simulate the actually obtained low-resolution image, the resolution loss index is calculated through the low-resolution image and the sample crack image, the resolution loss label is calibrated, the first feature image and the second feature image extracted by the frequency information extraction module and the refinement feature extraction module are reconstructed to obtain a reconstructed image, and the calculated resolution reconstruction results of the low-resolution image and the reconstructed image are used for obtaining a loss function based on the resolution loss label and the resolution reconstruction result, so that the model parameters of the preset neural network model are subjected to iterative updating, the reconstructed image output by the image reconstruction network is enabled to be closer to the sample crack image, and the image reconstruction network obtained through training can have the capability of optimizing the resolution of the original image.
Optionally, the refinement feature extraction module includes a shallow feature extraction sub-module, a deep feature extraction sub-module, and a multi-layer feature fusion sub-module; the step of inputting the low-resolution image into a refinement feature extraction module to generate a first feature map specifically includes:
copying the low-resolution images to obtain a plurality of low-resolution images, and connecting the low-resolution images in the channel dimension to obtain a spliced image;
inputting the spliced image to a shallow feature extraction sub-module, expanding the channel dimension of the spliced image through the shallow feature extraction sub-module to obtain an expanded image, and extracting the shallow features of the spliced image;
inputting the expanded image to a deep feature extraction submodule, and extracting depth features through the deep feature extraction submodule;
and fusing the shallow layer features and the depth features through a multi-layer feature fusion sub-module to obtain a first feature map.
By adopting the technical scheme, the copied low-resolution images are spliced in the channel dimension to obtain the spliced image, so that the refined feature extraction module can process a plurality of identical images at the same time to extract richer features, the receptive field of feature extraction is increased, the capturing capacity of the shallow feature extraction submodule and the deep feature extraction submodule in the refined feature extraction module on the details of the low-resolution images is improved, and the extracted shallow features and depth features are fused, so that a first feature map with more refined features is obtained.
Optionally, the low-frequency information extraction module comprises a low-pass filtering sub-module and a low-frequency feature extraction sub-module; the low-resolution image is input to a low-frequency information extraction module to generate a second feature map, which specifically comprises:
inputting the low-resolution image into a low-pass filtering submodule to obtain a filtered image;
and inputting the filtered image into a low-frequency characteristic extraction sub-module, and extracting the characteristics of the filtered image through the low-frequency characteristic extraction sub-module to obtain a second characteristic diagram.
By adopting the technical scheme, the low-pass filtering sub-module can filter out high-frequency information in the low-resolution image so as to enable the low-resolution image to be smooth, and the low-frequency characteristic extracting sub-module is utilized to extract low-frequency characteristics so as to enhance the low-frequency information and improve the overall quality of the low-resolution image to enable the low-resolution image to be smoother and more natural.
Optionally, the resolution loss indicator includes peak signal-to-noise ratio and/or structural similarity.
By adopting the technical scheme, the peak signal-to-noise ratio is an index for measuring the distortion degree of the image, the structural similarity is an index for measuring the similarity degree between two images, and the restoration capability of the image reconstruction network to the image resolution can be represented by utilizing the peak signal-to-noise ratio or the structural similarity.
Optionally, the blurring process is a bicubic degradation process.
By adopting the technical scheme, the edge in the sample crack image can be smoother through bicubic degradation treatment so as to simulate and shoot a blurred crack image.
In a second aspect, the present application provides a crack detection system based on image recognition, which adopts the following technical scheme:
a crack detection system based on image recognition, comprising:
an image acquisition unit for acquiring an original image to be detected;
the image reconstruction unit is used for inputting the original image into the image reconstruction network, extracting multi-level refined features of the original image through the image reconstruction network to obtain a first feature image, extracting low-frequency information of the original image to obtain a second feature image, and performing super-resolution reconstruction on the first feature image and the second feature image to output a resolution reconstruction image;
and the result output unit is used for carrying out crack detection on the resolution reconstructed image and outputting a crack detection result.
Optionally, the method further comprises a training module for the image reconstruction network, wherein the training module comprises:
the data set processing unit is used for acquiring a training data set, and respectively carrying out blurring processing on a plurality of sample crack images included in the training data set to obtain a low-resolution image;
the label generating unit is used for calibrating a resolution loss label for the sample crack image according to a preset resolution loss index;
the first feature extraction unit is used for inputting the low-resolution image into the refined feature extraction module to generate a first feature map;
the second feature extraction unit is used for inputting the low-resolution image into the low-frequency information extraction module to generate a second feature map;
the image reconstruction unit is used for reconstructing the first feature image and the second feature image to generate a reconstructed image;
the model training unit is used for calculating a resolution reconstruction result according to the sample crack image and the reconstruction image, and carrying out iterative updating on model parameters of the preset neural network model according to the resolution reconstruction result and the resolution loss label to obtain the image reconstruction network.
Optionally, the second feature extraction unit specifically includes:
the filtering sub-unit is used for inputting the low-resolution image into the low-pass filtering sub-module to obtain a filtered image;
the low-frequency extraction subunit is used for inputting the filtered image into the low-frequency feature extraction subunit through the feature map, and extracting the features of the filtered image through the low-frequency feature extraction subunit to obtain a second feature map.
In a third aspect, the present application provides a computer device, which adopts the following technical scheme:
a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing a computer program according to any one of the methods described above.
In a fourth aspect, the present application provides a computer readable storage medium, which adopts the following technical solutions:
a computer readable storage medium comprising a computer program stored thereon that can be loaded by a processor and executed in any of the methods described above.
Drawings
FIG. 1 is a flow chart of a crack detection method according to an embodiment of the present application.
FIG. 2 is a flow chart of a method for training a reconstruction network according to an embodiment of the present application.
Fig. 3 is a schematic structural diagram of an image reconstruction network according to an embodiment of the present application.
Fig. 4 is a block diagram of an image processing apparatus according to an embodiment of the present application.
FIG. 5 is a block diagram of a crack detection system according to one embodiment of the present application.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
The embodiment of the application discloses a crack detection method based on image recognition. Referring to fig. 1, a crack detection method based on image recognition includes:
step S101: and acquiring an original image to be detected.
The original image to be detected can be obtained by an unmanned aerial vehicle carrying a camera, a patrol trolley or a wall climbing robot, and can also be manually obtained by a manual handheld camera.
Step S102: inputting the original image into an image reconstruction network, extracting multi-level refined features of the original image through the image reconstruction network to obtain a first feature map, extracting low-frequency information of the original image to obtain a second feature map, and performing super-resolution reconstruction on the first feature map and the second feature map to output a resolution reconstruction image.
Step S103: and carrying out crack detection on the resolution reconstructed image, and outputting a crack detection result.
Specifically, the method can be used for detecting the cracks of the resolved reconstructed image through visual detection methods such as an image segmentation network and the like, and outputting a crack detection result through the image segmentation network, wherein the crack detection result can comprise parameters such as a crack region mark, a crack length, a crack duty ratio and the like.
In the embodiment, more refined features can be obtained by using the first feature map on one hand, and smoother and more real features can be obtained by using the second feature map on the other hand by using the image reconstruction network, so that the resolution reconstructed image obtained by reconstructing according to the first feature map and the second feature map is improved, the resolution of the original image is not easy to distort, the effect of crack detection is better by using the resolution reconstructed image with higher quality, and the accuracy of crack detection is improved.
Referring to fig. 2, as a further embodiment of the image recognition-based crack detection method, the image recognition-based crack detection method further includes a training step of training a preset neural network model to obtain an image reconstruction network, the image reconstruction network including a refinement feature extraction module, a low-frequency information extraction module, and a reconstruction module; the training step includes the following steps S201 to S206, which are specifically described below.
Step S201: acquiring a training data set, and respectively carrying out blurring treatment on a plurality of sample crack images included in the training data set to obtain a low-resolution image;
in this embodiment, the blurring process is a bicubic degradation process. The bicubic degradation is an image processing method for obtaining a low-resolution image from a high-resolution image through bicubic downsampling, and the bicubic degradation processing can enable edges in a sample crack image to be smoother so as to simulate and shoot a blurred crack image. In other embodiments, the blurring process may also be implemented by mean filtering, gaussian filtering, smoothing blurring, and the like.
It should be appreciated that with different blurring processing methods, blurring caused by different reasons, such as raw image blurring caused by camera shake when capturing an image, raw image blurring caused by improper focal length, or raw image blurring caused by too fast movement of the camera, can be simulated.
Step S202: calibrating a resolution loss label for the sample crack image according to a preset resolution loss index;
as one possible implementation of the resolution loss indicator, the resolution loss indicator may include a peak signal-to-noise ratio and/or a structural similarity, among others. The peak signal-to-noise ratio (PSNR) is an indicator for measuring the degree of distortion of an image. Structural Similarity (SSIM) is an indicator for measuring the degree of similarity between two images, and the restoration capability of an image reconstruction network to image resolution is characterized by using the peak signal-to-noise ratio or structural similarity.
Specifically, the larger the peak signal-to-noise ratio represents the smaller the distortion degree of the two images; the peak signal-to-noise ratio is usually higher than 40dB, which indicates that the image quality is very good, the peak signal-to-noise ratio is usually between 30 and 40dB, which indicates that the image quality is poor, the peak signal-to-noise ratio is between 20 and 30dB, and the image cannot be used when the peak signal-to-noise ratio is lower than 20dB, for example, when the sewing signal-to-noise ratio between the generated low-resolution image and the sample crack image is smaller than a preset value, which indicates that the distortion of the low-resolution image is serious, the low-resolution image is difficult to be used as a training sample, and the blurring processing mode can be replaced to regenerate the low-resolution image. The structural similarity is a number between 0 and 1 and a larger structural similarity indicates a smaller degree of similarity of the two images, and the structural similarity has a value of 1 when the two images are identical.
Wherein the resolution loss index can be set by: the method comprises the steps of firstly calculating the peak signal-to-noise ratio and/or the value of structural similarity between a sample crack image and a low-resolution image, and then setting a proper resolution loss index according to the peak signal-to-noise ratio and/or the value of structural similarity. Because the distortion degree of the low-resolution images after the blurring processing is different, if a uniform resolution loss index is set, the set resolution loss index can be easily achieved for the low-resolution images with light distortion degree, and the low-resolution images with serious distortion degree can not easily meet the requirement of the resolution loss index after multiple rounds of training, so that the model training is not ideal, the proper resolution loss index is set based on the peak signal-to-noise ratio and/or the value of the structural similarity between the sample crack images and the low-resolution images, and the setting of the resolution loss index can be more reasonable. In addition to the above method for setting the resolution loss index, the preset resolution loss index may be set according to the actual situation.
Step S203: inputting the low-resolution image into a refined feature extraction module to generate a first feature map;
step S204: inputting the low-resolution image into a low-frequency information extraction module to generate a second feature map;
step S205: reconstructing the first feature map and the second feature map to generate a reconstructed image;
step S206: and calculating a resolution reconstruction result according to the sample crack image and the reconstruction image, and carrying out iterative updating on model parameters of a preset neural network model according to the resolution reconstruction result and the resolution loss label to obtain an image reconstruction network.
In particular, an L1 penalty function may be employed for iterative updating of model parameters.
In the above embodiment, the sample fracture image in the training data set is subjected to blurring processing to simulate the actually obtained low-resolution image, the resolution loss index is calculated through the low-resolution image and the sample fracture image, the resolution loss label is calibrated, the first feature image and the second feature image extracted by the frequency information extraction module and the refined feature extraction module are reconstructed to obtain a reconstructed image, and the calculated resolution reconstruction results of the low-resolution image and the reconstructed image are used to obtain a loss function based on the resolution loss label and the resolution reconstruction result, so that the model parameters of the preset neural network model are subjected to iterative update, the reconstructed image output by the image reconstruction network is enabled to be closer to the sample fracture image, and the image reconstruction network obtained by training can have the capability of optimizing the resolution of the original image.
As an implementation manner of step S203, the refined feature extraction module includes a shallow feature extraction sub-module, a deep feature extraction sub-module, and a multi-layer feature fusion sub-module; the step S203 specifically includes:
step S2031: and copying the low-resolution images to obtain a plurality of low-resolution images, and connecting the plurality of low-resolution images in the channel dimension to obtain a spliced image.
It should be appreciated that connecting multiple low resolution images in the channel dimension can map the low resolution images to a feature space of higher dimension, thereby extracting richer features. For example, the low resolution image is copied 8 sheets, assuming that the size of the low resolution image is [ H, W, C ], where H represents the height of the low resolution image, W represents the width of the low resolution image, and C represents the number of channels of the low resolution image. And 8 low-resolution images are obtained through copying and spliced to obtain spliced images, and the obtained spliced images have the dimensions of H, W and 8C.
Step S2032: inputting the spliced image to a shallow feature extraction sub-module, expanding the channel dimension of the spliced image through the shallow feature extraction sub-module to obtain an expanded image, and extracting the shallow features of the spliced image;
in particular, the shallow feature extraction submodule may employ a blueprint convolution layer. The blueprint convolution layer is an improved variant of original depth separable convolution (DSConv), and can effectively separate by utilizing the intra-core correlation better, so that the feature extraction efficiency is maintained while the number of model parameters is reduced.
Step S2033: and inputting the expanded image into a deep feature extraction submodule, and extracting depth features through the deep feature extraction submodule.
The deep feature extraction sub-module may enhance model capabilities by, among other things, a Conv-1 convolution layer and a feature refinement layer and introducing Enhanced Spatial Attention (ESA) and contrast-aware channel attention (CCA).
Step S2034: and fusing the shallow layer features and the depth features through a multi-layer feature fusion sub-module to obtain a first feature map.
Specifically, the multi-layer feature fusion submodule can use a 1×1 convolution layer and a GELU activation function to fuse and map features in different dimensions, then the multi-layer feature fusion submodule can use a blueprint convolution layer to further refine the features after fusing the features in different dimensions, and finally upsampling is realized by a pixelshuffle algorithm to obtain a first feature map.
In the above embodiment, the multiple low-resolution images obtained by replication are spliced in the channel dimension to obtain the spliced image, so that the refinement feature extraction module can process multiple identical images at the same time to extract richer features, the receptive field of feature extraction is increased, the capturing capability of the shallow feature extraction sub-module and the deep feature extraction sub-module in the refinement feature extraction module on the details of the low-resolution images is improved, and then the extracted shallow features and depth features are fused, so that the first feature map with more refinement features is obtained.
As an embodiment of step S204, the low-frequency information extraction module in step S204 includes a low-pass filtering sub-module and a low-frequency feature extraction sub-module; the step S204 specifically includes:
step S2041: and inputting the low-resolution image into a low-pass filtering sub-module to obtain a filtered image.
In particular, the low pass filtering sub-module may employ a gaussian low pass filter.
Step S2042: and inputting the filtered image into a low-frequency characteristic extraction sub-module, and extracting the characteristics of the filtered image through the low-frequency characteristic extraction sub-module to obtain a second characteristic diagram.
Specifically, the low frequency feature extraction submodule may employ 3*3 convolution layers for feature extraction.
In the above embodiment, the low-pass filtering sub-module can filter out the high-frequency information in the low-resolution image, so as to smooth the low-resolution image, and then the low-frequency feature extracting sub-module is used to extract the low-frequency features, so as to enhance the low-frequency information, and improve the overall quality of the low-resolution image to make the low-resolution image smoother and more natural.
Referring to fig. 3, fig. 3 is a schematic diagram of an image reconstruction network trained by steps S201 to S206. The specific processing flow of the image reconstruction network is as follows: dividing an original image into two branches, inputting the original image into a refined feature extraction module by a first branch, sequentially passing through a shallow feature extraction sub-module, a deep feature extraction sub-module and a multi-layer feature fusion module in the refined feature extraction module, and finally outputting a first feature image with reserved details and textures, namely, the first feature image reserves high-frequency information of the image. The second branch inputs the original image to the low-frequency information extraction module, sequentially passes through the low-pass filtering sub-module and the low-frequency feature extraction sub-module of the low-frequency information extraction module, and finally outputs a more natural and smooth second feature map. And splicing and reconstructing the first characteristic image and the second characteristic image obtained by the two paths by using a reconstruction module to obtain a resolution reconstructed image. The resolution reconstructed image comprehensively utilizes all the characteristic information extracted by the two paths to obtain a more comprehensive and accurate reconstruction result, so that the resolution reconstructed image can simultaneously contain high-frequency information and low-frequency information, and a better reconstruction effect on the original image is realized.
Especially, when the image reconstruction is applied to crack detection, cracks of structures such as concrete are often fine and slender linear defects, fine crack details can not be accurately captured by traditional image resolution, and as human eyes are more sensitive to low-frequency information, objects can be identified through rough images, and the image reconstruction is carried out by introducing the low-frequency information, the obtained resolution reconstruction image is more convenient for the human eyes to watch the resolution reconstruction image while meeting the requirements of resolution, definition, texture and the like of visual detection, and is more beneficial to the crack detection by manually carrying out the resolution reconstruction image.
In addition, the embodiment of the application discloses a crack detection system based on image recognition. The crack detection system based on image recognition can be applied to an image processing device, for example, as shown in fig. 4, and is a schematic structural diagram of an image processing device 11 for implementing the method according to an embodiment of the present application. In this embodiment, the image processing device 11 may include a crack detection system 110, a machine-readable storage medium 120, and a processor 130.
In this embodiment, the machine-readable storage medium 120 and the processor 130 may be located in the image processing apparatus 11 and separately provided. The machine-readable storage medium 120 may also be independent of the image processing device 11 and accessed by the processor 130. The crack detection system 110 may include a plurality of functional modules stored on a machine-readable storage medium 120, such as the software functional modules included in the crack detection system 110. When the processor 130 executes the computer program corresponding to the software function module in the crack detection system 110, the crack detection system based on image recognition provided in the foregoing method embodiment is implemented.
In this embodiment, the image processing apparatus 11 may include one or more processors 130. Processor 130 may process information and/or data related to the service request to perform one or more functions described herein. In some embodiments, processor 130 may include one or more processing engines (e.g., a single-core processor or a multi-core processor). By way of example only, the processor 130 may include one or more hardware processors such as one of a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), a special purpose instruction set processor (ASIP), a Graphics Processor (GPU), a physical operation processing unit (PPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), a controller, a microcontroller unit, a Reduced Instruction Set Computer (RISC), a microprocessor, etc., or the like, or any combination thereof.
The machine-readable storage medium 120 may store data and/or instructions. In some embodiments, the machine-readable storage medium 120 may store the obtained data or material. In some embodiments, the machine-readable storage medium 120 may store data and/or instructions for execution by or use by the image processing device 11, which may be executed or used by the image processing device 11 to implement the exemplary methods described herein. In some embodiments, the machine-readable storage medium 120 may include mass storage, removable storage, volatile read-write memory, read-only memory ROM, and the like, or any combination of the above. Exemplary mass storage devices may include magnetic disks, optical disks, solid state disks, and the like. Exemplary removable memory may include flash drives, floppy disks, optical disks, memory cards, compact disks, tape, and the like. Exemplary volatile read-write memory can include random access memory RAM. Exemplary random access memory may include dynamic RAM, double rate synchronous dynamic RAM, static RAM, thyristor RAM, zero capacitance RAM, and the like. Exemplary ROM may include masked ROM, programmable ROM, erasable programmable ROM, electronically erasable programmable ROM, compact disc ROM, digital versatile disc ROM, and the like.
Wherein the image processing device 11 comprises an image recognition based crack detection system 110 may comprise one or more software functional modules. The software functional modules may be stored as programs, instructions in the machine readable storage medium 120 for implementing the methods described above when executed by the corresponding processor 130, for example for implementing the method steps performed by the drone when executed by the processor of the drone, or for implementing the method steps performed by the image processing device 11 when executed by the image processing device 11.
The embodiment of the application discloses a crack detection system based on image recognition. Referring to fig. 5, a crack detection system based on image recognition includes:
an image acquisition unit for acquiring an original image to be detected;
the image reconstruction unit is used for inputting the original image into the image reconstruction network, extracting multi-level refined features of the original image through the image reconstruction network to obtain a first feature image, extracting low-frequency information of the original image to obtain a second feature image, and performing super-resolution reconstruction on the first feature image and the second feature image to output a resolution reconstruction image;
and the result output unit is used for carrying out crack detection on the resolution reconstructed image and outputting a crack detection result.
As a further embodiment of the image recognition based crack detection system, the image recognition based crack detection system further comprises a training module for the image reconstruction network, the training module comprising:
the data set processing unit is used for acquiring a training data set, and respectively carrying out blurring processing on a plurality of sample crack images included in the training data set to obtain a low-resolution image;
the label generating unit is used for calibrating a resolution loss label for the sample crack image according to a preset resolution loss index;
the first feature extraction unit is used for inputting the low-resolution image into the refined feature extraction module to generate a first feature map;
the second feature extraction unit is used for inputting the low-resolution image into the low-frequency information extraction module to generate a second feature map;
the image reconstruction unit is used for reconstructing the first feature image and the second feature image to generate a reconstructed image;
the model training unit is used for calculating a resolution reconstruction result according to the sample crack image and the reconstruction image, and carrying out iterative updating on model parameters of a preset neural network model according to the resolution reconstruction result and the resolution loss label to obtain an image reconstruction network.
As one embodiment of the second feature extraction unit, the second feature extraction unit specifically includes:
the filtering sub-unit is used for inputting the low-resolution image into the low-pass filtering sub-module to obtain a filtered image;
the low-frequency extraction subunit is used for inputting the filtered image into the low-frequency feature extraction subunit through the feature map, and extracting the features of the filtered image through the low-frequency feature extraction subunit to obtain a second feature map.
The crack detection system based on image recognition provided by the application can realize the crack detection method based on image recognition, and the specific working process of the crack detection system based on image recognition can refer to the corresponding process in the embodiment of the method.
In the foregoing embodiments, the descriptions of the embodiments are focused on, and for those portions of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.
Based on the same technical concept, the application also discloses an image recognition device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the computer program of any one of the methods.
The application also discloses a computer readable storage medium comprising a computer program stored with instructions executable by a processor to load and execute any of the methods described above.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.
In addition, functional modules in the embodiments of the present application may be integrated together to form a single part, or each module may exist alone, or two or more modules may be integrated to form a single part.
The foregoing description of the preferred embodiments of the application is not intended to limit the scope of the application in any way, including the abstract and drawings, in which case any feature disclosed in this specification (including abstract and drawings) may be replaced by alternative features serving the same, equivalent purpose, unless expressly stated otherwise. That is, each feature is one example only of a generic series of equivalent or similar features, unless expressly stated otherwise.
Claims (10)
1. The crack detection method based on image recognition is characterized by comprising the following steps of:
acquiring an original image to be detected;
inputting an original image into an image reconstruction network, extracting multi-level refined features of the original image through the image reconstruction network to obtain a first feature image, extracting low-frequency information of the original image to obtain a second feature image, and performing super-resolution reconstruction on the first feature image and the second feature image to output a resolution reconstruction image;
and carrying out crack detection on the resolution reconstructed image, and outputting a crack detection result.
2. The method of claim 1, further comprising a training step of training a predetermined neural network model to obtain an image reconstruction network, the image reconstruction network comprising a refinement feature extraction module, a low frequency information extraction module, and a reconstruction module; the training step comprises the following steps:
acquiring a training data set, and respectively carrying out blurring treatment on a plurality of sample crack images included in the training data set to obtain a low-resolution image;
calibrating a resolution loss label for the sample crack image according to a preset resolution loss index;
inputting the low-resolution image into a refined feature extraction module to generate a first feature map;
inputting the low-resolution image into a low-frequency information extraction module to generate a second feature map;
reconstructing the first feature map and the second feature map to generate a reconstructed image;
and calculating a resolution reconstruction result according to the sample crack image and the reconstruction image, and carrying out iterative updating on model parameters of the preset neural network model according to the resolution reconstruction result and the resolution loss label to obtain the image reconstruction network.
3. The method of claim 2, wherein the refined feature extraction module comprises a shallow feature extraction sub-module, a deep feature extraction sub-module, and a multi-layer feature fusion sub-module; the step of inputting the low-resolution image into a refinement feature extraction module to generate a first feature map specifically includes:
copying the low-resolution images to obtain a plurality of low-resolution images, and connecting the low-resolution images in the channel dimension to obtain a spliced image;
inputting the spliced image to a shallow feature extraction sub-module, expanding the channel dimension of the spliced image through the shallow feature extraction sub-module to obtain an expanded image, and extracting the shallow features of the spliced image;
inputting the expanded image to a deep feature extraction submodule, and extracting depth features through the deep feature extraction submodule;
and fusing the shallow layer features and the depth features through a multi-layer feature fusion sub-module to obtain a first feature map.
4. The method of claim 2, wherein the low frequency information extraction module comprises a low pass filtering sub-module and a low frequency feature extraction sub-module; the low-resolution image is input to a low-frequency information extraction module to generate a second feature map, which specifically comprises:
inputting the low-resolution image into a low-pass filtering submodule to obtain a filtered image;
and inputting the filtered image into a low-frequency characteristic extraction sub-module, and extracting the characteristics of the filtered image through the low-frequency characteristic extraction sub-module to obtain a second characteristic diagram.
5. The method according to claim 2, wherein the resolution loss indicator comprises peak signal-to-noise ratio and/or structural similarity.
6. The method of claim 2, wherein the blurring process is a bicubic degradation process.
7. A crack detection system based on image recognition, comprising:
an image acquisition unit for acquiring an original image to be detected;
the image reconstruction unit is used for inputting the original image into the image reconstruction network, extracting multi-level refined features of the original image through the image reconstruction network to obtain a first feature image, extracting low-frequency information of the original image to obtain a second feature image, and performing super-resolution reconstruction on the first feature image and the second feature image to output a resolution reconstruction image;
and the result output unit is used for carrying out crack detection on the resolution reconstructed image and outputting a crack detection result.
8. The system of claim 7, further comprising a training module for the image reconstruction network, the training module comprising:
the data set processing unit is used for acquiring a training data set, and respectively carrying out blurring processing on a plurality of sample crack images included in the training data set to obtain a low-resolution image;
the label generating unit is used for calibrating a resolution loss label for the sample crack image according to a preset resolution loss index;
the first feature extraction unit is used for inputting the low-resolution image into the refined feature extraction module to generate a first feature map;
the second feature extraction unit is used for inputting the low-resolution image into the low-frequency information extraction module to generate a second feature map;
the image reconstruction unit is used for reconstructing the first feature image and the second feature image to generate a reconstructed image;
the model training unit is used for calculating a resolution reconstruction result according to the sample crack image and the reconstruction image, and carrying out iterative updating on model parameters of the preset neural network model according to the resolution reconstruction result and the resolution loss label to obtain the image reconstruction network.
9. The system according to claim 8, wherein the second feature extraction unit specifically comprises:
the filtering sub-unit is used for inputting the low-resolution image into the low-pass filtering sub-module to obtain a filtered image;
the low-frequency extraction subunit is used for inputting the filtered image into the low-frequency feature extraction subunit through the feature map, and extracting the features of the filtered image through the low-frequency feature extraction subunit to obtain a second feature map.
10. An image processing apparatus, characterized by: comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the computer program to implement the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310857782.XA CN116883362A (en) | 2023-07-12 | 2023-07-12 | Crack detection method and system based on image recognition and image processing equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310857782.XA CN116883362A (en) | 2023-07-12 | 2023-07-12 | Crack detection method and system based on image recognition and image processing equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116883362A true CN116883362A (en) | 2023-10-13 |
Family
ID=88265706
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310857782.XA Pending CN116883362A (en) | 2023-07-12 | 2023-07-12 | Crack detection method and system based on image recognition and image processing equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116883362A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111461983A (en) * | 2020-03-31 | 2020-07-28 | 华中科技大学鄂州工业技术研究院 | Image super-resolution reconstruction model and method based on different frequency information |
| CN111986079A (en) * | 2020-06-16 | 2020-11-24 | 长安大学 | Pavement crack image super-resolution reconstruction method and device based on generation countermeasure network |
| CN114187181A (en) * | 2021-12-17 | 2022-03-15 | 福州大学 | Double-path lung CT image super-resolution method based on residual information refining |
| CN115760573A (en) * | 2022-11-19 | 2023-03-07 | 天翼电子商务有限公司 | Image super-resolution reconstruction model construction method based on multi-frequency level residual error network |
| CN116091313A (en) * | 2022-12-20 | 2023-05-09 | 无锡学院 | Image super-resolution network model and reconstruction method |
-
2023
- 2023-07-12 CN CN202310857782.XA patent/CN116883362A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111461983A (en) * | 2020-03-31 | 2020-07-28 | 华中科技大学鄂州工业技术研究院 | Image super-resolution reconstruction model and method based on different frequency information |
| CN111986079A (en) * | 2020-06-16 | 2020-11-24 | 长安大学 | Pavement crack image super-resolution reconstruction method and device based on generation countermeasure network |
| CN114187181A (en) * | 2021-12-17 | 2022-03-15 | 福州大学 | Double-path lung CT image super-resolution method based on residual information refining |
| CN115760573A (en) * | 2022-11-19 | 2023-03-07 | 天翼电子商务有限公司 | Image super-resolution reconstruction model construction method based on multi-frequency level residual error network |
| CN116091313A (en) * | 2022-12-20 | 2023-05-09 | 无锡学院 | Image super-resolution network model and reconstruction method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Alhashim et al. | High quality monocular depth estimation via transfer learning | |
| Fu et al. | Removing rain from single images via a deep detail network | |
| Pei et al. | Does haze removal help cnn-based image classification? | |
| Zhang et al. | A dense u-net with cross-layer intersection for detection and localization of image forgery | |
| CN111488865A (en) | Image optimization method and device, computer storage medium and electronic equipment | |
| Lu et al. | Deep texture and structure aware filtering network for image smoothing | |
| CN112329702B (en) | Method and device for rapid face density prediction and face detection, electronic equipment and storage medium | |
| CN107767358B (en) | Method and device for determining ambiguity of object in image | |
| CN107221005B (en) | Object detection method and device | |
| CN111192241B (en) | Quality evaluation method and device for face image and computer storage medium | |
| KR102311796B1 (en) | Method and Apparatus for Deblurring of Human Motion using Localized Body Prior | |
| CN111079764A (en) | Low-illumination license plate image recognition method and device based on deep learning | |
| Liu et al. | A motion deblur method based on multi-scale high frequency residual image learning | |
| CN109064504A (en) | Image processing method, device and computer storage medium | |
| CN112419295A (en) | Medical image processing method, apparatus, computer device and storage medium | |
| He et al. | A novel scheme based on the diffusion to edge detection | |
| Wu et al. | Restoration of low-informative image for robust debris shape measurement in on-line wear debris monitoring | |
| CN106702870A (en) | Road crack detecting system | |
| Chen et al. | Three dimensional synthetic non-ellipsoidal nuclei volume generation using bezier curves | |
| Chen et al. | Root gap correction with a deep inpainting model | |
| Zhou et al. | A perceptual framework for infrared–visible image fusion based on multiscale structure decomposition and biological vision | |
| CN113688839B (en) | Video processing method and device, electronic equipment and computer readable storage medium | |
| CN116309062A (en) | Remote sensing image super-resolution reconstruction method | |
| Hou et al. | Image enhancement and restoration: state of the art of variational retinex models | |
| CN114140481A (en) | Edge detection method and device based on infrared image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |