CN115689924A - Data enhancement method and device for concrete structure ultrasonic tomography image - Google Patents
Data enhancement method and device for concrete structure ultrasonic tomography image Download PDFInfo
- Publication number
- CN115689924A CN115689924A CN202211335574.5A CN202211335574A CN115689924A CN 115689924 A CN115689924 A CN 115689924A CN 202211335574 A CN202211335574 A CN 202211335574A CN 115689924 A CN115689924 A CN 115689924A
- Authority
- CN
- China
- Prior art keywords
- image
- ultrasonic
- scanning
- self
- concrete structure
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention discloses a data enhancement method for an ultrasonic tomography image of a concrete structure, which comprises the following steps: step 1, obtaining an ultrasonic B scanning image of a self-made reinforced concrete member, labeling the cross-section crack condition of the self-made reinforced concrete member to generate a cross-section crack image corresponding to the ultrasonic B scanning image, and forming a data set by the ultrasonic B scanning image and the cross-section crack image; step 2, a generation countermeasure network constructed based on a pixel-to-pixel structure, wherein the generation countermeasure network comprises a generator and a discriminator; step 3, training the generation countermeasure network by adopting a game means to obtain an image generation model for generating a virtual ultrasonic B scanning image; and 4, inputting the self-made section crack diagram into an image generation model to obtain a corresponding virtual ultrasonic B scanning diagram. The invention also provides a data enhancement device. The method provided by the invention can solve the problem that the automatic nondestructive detection of the concrete structure is inconvenient for subsequent application of deep learning due to small experimental data volume.
Description
Technical Field
The invention relates to the field of nondestructive testing of concrete structures, in particular to a concrete structure image data enhancing method and device based on deep learning.
Background
The concrete structure is inevitably subjected to coupling action of various factors such as load, environmental erosion and the like in the use process, and various damages are generated on the surface and inside, so that the use safety and practicability of the concrete structure are seriously influenced. The existing ultrasonic tomography method can acquire the related information of the internal damage of the concrete structure, but the original image generated by scanning is inconvenient for detection personnel to judge the damage condition. By applying the deep learning method, the real form of the concrete structure section can be reconstructed from the original B scanning image, so that the defects such as internal cracks and the like can be detected. However, deep learning is data driven, the implementation of which requires a large amount of data as a basis.
Too little data volume easily leads to overfitting of the model and thus affects the generalization capability of the model in actual engineering. The B scanning image obtained by the experimental method consumes a large amount of manpower and material resources, namely the data amount required by deep learning is difficult to achieve only by data obtained in experiments and engineering practice. Therefore, expansion of the data set by other methods is required.
Patent document CN114778693A discloses a concrete structure stress loading detection system based on ultrasonic waves, which comprises an ultrasonic propagation time testing module, a stress loading detection module and a stress loading detection module, wherein the ultrasonic propagation time testing module is used for enabling ultrasonic waves to transmit through concrete and testing the ultrasonic propagation time in the concrete to obtain the ultrasonic propagation time of the ultrasonic waves in the concrete; and the structural stress calculation module is used for calculating the stress of the concrete structure according to the travel time of the ultrasonic propagation. According to the method, calculation is only carried out through sound wave data, but subsequent treatment is still needed for further damage state judgment, and the final result is inaccurate due to the fact that the sound wave propagation medium in the concrete is not uniform.
Patent document CN115165910a discloses a system and a method for detecting concrete apparent quality defects based on image recognition, wherein the system comprises a prefabrication production line, a state conversion platform, a hoisting system and a detection production line, and the production and the transportation of a concrete prefabricated part are completed through the prefabrication production line; the posture of the concrete prefabricated part before demoulding is adjusted through the state conversion platform; the demolding and hoisting of the concrete prefabricated part before demolding are completed through the hoisting system, and the concrete prefabricated part is conveyed to a detection production line; the apparent quality defect type is judged by a high-definition camera of a detection assembly line and an image processing system based on a deep learning algorithm, so that quality detection is realized. The method needs a large number of sample images to train the image recognition model, and is high in cost and long in time consumption.
Disclosure of Invention
In order to solve the problems, the invention provides a data enhancement method which can effectively solve the problem that a large amount of manpower and material resources are consumed in the preparation of reinforced concrete, so that the data set quantity for training an algorithm model is increased, and the generalization capability of the algorithm model is further improved.
A data enhancement method for an ultrasonic tomography image of a concrete structure, comprising:
step 1, obtaining an ultrasonic B scanning image of a self-made reinforced concrete member, labeling the cross-section crack condition of the self-made reinforced concrete member to generate a cross-section crack image corresponding to the ultrasonic B scanning image, and forming a data set by the ultrasonic B scanning image and the cross-section crack image;
step 2, a generated countermeasure network constructed based on a pixel-to-pixel structure, wherein the generated countermeasure network comprises a generator and a discriminator, the generator comprises an encoder and a decoder, the encoder is based on a U-Net structure, the encoder is used for performing down-sampling operation on an input section crack diagram to obtain a corresponding characteristic image, the decoder is used for performing deconvolution up-sampling on the characteristic image to obtain a corresponding virtual ultrasonic B scanning image, and the discriminator is used for judging the authenticity of the input image;
3, training the countermeasure network by using the data set obtained in the step 1 and adopting a game method until the discriminator cannot judge whether the image is generated by the generator or not, and obtaining an image generation model for generating a virtual ultrasonic B scanning image;
and 4, inputting the self-made section crack diagram into the image generation model to obtain a corresponding virtual ultrasonic B scanning diagram, and taking the self-made section crack diagram and the corresponding virtual ultrasonic B scanning diagram as an enhanced data set for expanding the data set volume.
The invention is based on a pixel-to-pixel deep neural network, and trains the neural network by means of a countermeasure game to learn the implicit relation between an annotation image and a B scanning image, thereby generating a high-quality virtual ultrasonic B scanning image which is falsified and truthful.
Specifically, in step 1, the ultrasonic B-scan is obtained by scanning the self-made reinforced concrete member by using an array ultrasonic tomography device.
Specifically, the self-made reinforced concrete member is a reinforced concrete member with crack defects prepared in advance, and a measuring point lateral line is arranged on one side of the surface of the self-made reinforced concrete member after the self-made reinforced concrete member is formed, so that scanning is facilitated.
Specifically, in step 2, the generator is a deep neural network including 16 modules, wherein the first 8 modules include an encoder for feature extraction, the last 8 modules include a decoder for feature synthesis, the convolution kernel size of the encoder is 4 × 4 for three times of downsampling operation, the number of channels of the feature image after each downsampling operation is multiplied by 2, the decoder corresponds to each layer of the network structure of the encoder, deconvolution upsampling is performed on the feature image, the number of channels of the feature image after each upsampling is divided by 2, the decoder corresponds to each layer of the network structure of the encoder, and the feature image is normalized by splicing operation between corresponding layers.
Specifically, the input image size of the generator is 2304 × 1536 in pixels.
Specifically, in step 2, the discriminator adopts a PatchGAN strategy to judge the authenticity of the input image.
Specifically, in the training process of step 3, the game means includes training the discriminator for multiple times by separately adopting a gradient ascending strategy cycle, and then optimizing the generator by adopting a gradient descending strategy.
Preferably, in step 4, the self-made cross-sectional crack diagram is generated by specifying the horizontal distance, the vertical distance and the initial position of the crack by random numbers, and the angle between the oblique line of the crack generation and the horizontal line is 0 to 45 degrees.
The present invention provides a data enhancement apparatus comprising a computer memory, a computer processor and a computer program stored in said computer memory and executable on said computer processor, said computer program employing the image generation model of claim 1;
the computer processor executing the computer program implements the steps of:
and inputting the self-made section crack diagram into an image generation model, and outputting a virtual ultrasonic B scanning diagram for expanding the data set.
Compared with the prior art, the invention has the beneficial effects that:
(1) According to the method, after the ultrasonic tomography image data are acquired by using an experimental method, in order to ensure that enough data volume is available for training of the reinforced concrete structure section reconstruction neural network model, the existing experimental data are amplified by using a deep learning method, and excessive experimental investment is avoided on the premise of meeting the use requirements.
(2) According to the invention, a pixel-to-pixel-based deep neural network is built, the neural network is trained by means of a countermeasure game to learn the implicit relation between the marked image and the B scanning image, and the ultrasonic tomography image generated by the trained network model has the characteristics of same feature and same distribution as the experimental actual scanning image and certain randomness, so that the data set obtained by amplification has high fitting degree with the real situation, and the requirement of subsequent application can be met.
Drawings
Fig. 1 is a flow chart of a data enhancement method for an ultrasonic tomography image of a concrete structure according to the invention;
fig. 2 is an ultrasound B-scan image and a corresponding labeled image provided in the present embodiment;
FIG. 3 is a diagram of a generator architecture for a deep neural network based on a pixel-to-pixel structure;
FIG. 4 is a diagram of an arbiter architecture for a deep neural network based on pixel-to-pixel architecture;
fig. 5 is a self-made cross-sectional crack diagram and a corresponding virtual ultrasound B-scan.
Detailed Description
The invention will be described in detail with reference to the drawings and specific embodiments, which are provided herein for the purpose of illustrating the invention and are not to be construed as limiting the invention.
The embodiment discloses a data enhancement method for an ultrasonic tomography image of a concrete structure, which comprises the following steps as shown in figure 1.
Step 1, firstly, a concrete member with preset defects is manufactured according to a crack design drawing, and measuring point measuring lines are arranged on the surface of the member after the member is formed so as to facilitate scanning. Scanning the reinforced concrete member by using array ultrasonic tomography equipment to obtain an ultrasonic tomography image at the defect part;
as shown in fig. 2, (a) is an ultrasound B-scan image, and (B) is an annotation image, for each B-scan image, a corresponding annotation image is generated from the crack plan, and finally, a data set in a pairing form is constructed.
And 2, building a generation countermeasure network based on a pixel-to-pixel structure, wherein the network is roughly composed of two parts, namely a generator and a discriminator.
(1) The generator mainly functions to perform image translation, namely, to generate a corresponding B-scan image according to the marked image.
As shown in fig. 3, the generator uses a U-Net based architecture, with 16 layers of neural networks, the first 8 layers being the encoder and the last 8 layers being the decoder. The input image size of the original deep neural network generator is 256 × 256, and the number of channels is 64. In order to ensure the image precision, the original network needs to be improved, and the size of the input image after the improvement is 2304 × 1536. The size of the convolution kernel in the encoder is 4 × 4, three times of downsampling operation is performed, the number of channels of the feature map after each downsampling is multiplied by 2, and finally the number of channels of the feature map is changed to 512. The decoder uses deconvolution up-sampling, and the number of channels of the feature map after each up-sampling is divided by 2, so that the number of channels of the feature map is finally changed from 512 back to 64. Each layer of the network structure of the decoder and the encoder corresponds to each other, and the characteristic diagram is normalized by splicing operation between corresponding layers.
In order to ensure that the B-scan image generated by the generator has certain randomness, a random noise needs to be input into the generation countermeasure network in addition to the label image.
(2) The main function of the discriminator is to discriminate whether the input data is machine-generated.
As shown in fig. 4, the discriminator is composed of a 5-layer neural network. The input of the discriminator is the generated B-scan image (or the experimentally obtained B-scan image) and its corresponding label graph. The discriminator performs processing such as convolution on the input image, and outputs the discrimination result, that is, whether the input B-scan image is actually obtained by an experiment or generated by the discriminator. Unlike the conventional method of using the whole image as the target for the discrimination by the discriminator, the network architecture in this patent divides the input image into N × N image blocks, and then sequentially submits the image blocks to the discriminator, and this strategy is called PatchGAN and can be regarded as the loss of image texture.
Step 3, training the generated countermeasure network built in the step two by using the data set built in the step one, and performing iterative optimization on the generator and the discriminator by means of game;
initializing parameters of a discriminator and a generator, firstly enabling the generator to generate a batch of B scanning images by using labeled image samples in a data set and random noise, and then circularly training the discriminator for multiple times by using a gradient ascending strategy by using the generated images and original images; and after the discriminator is optimized, optimizing the generator by using a gradient descent strategy. Until the generator has the ability to generate samples that the arbiter cannot resolve; and after the training is finished, using the expert evaluation table as an evaluation index of the network model training effect.
And 4, step 4: inputting the self-made section crack diagram into the image generation model to obtain a corresponding virtual ultrasonic B scanning diagram;
when the self-made section crack diagram is generated, the horizontal distance, the vertical distance and the initial position of the crack are specified by random numbers, and the angle of the generated oblique line is between 0 and 45 degrees.
500 marked images are generated by the method and input into the deep neural network trained in the step 3 to be used as a test set. The corresponding virtual B-scan image is generated by a generator of the neural network and stored, and the enhancement of the ultrasonic tomography image data obtained by experiments is realized.
As shown in fig. 5, the left of the figure is the homemade cross-sectional crack diagram, and the right of the figure is the corresponding virtual ultrasound B-scan diagram.
Claims (8)
1. A data enhancement method for an ultrasonic tomography image of a concrete structure is characterized by comprising the following steps:
step 1, obtaining an ultrasonic B scanning image of a self-made reinforced concrete member, labeling the cross-section crack condition of the self-made reinforced concrete member to generate a cross-section crack image corresponding to the ultrasonic B scanning image, and forming a data set by the ultrasonic B scanning image and the cross-section crack image;
step 2, a generated countermeasure network constructed based on a pixel-to-pixel structure, wherein the generated countermeasure network comprises a generator and a discriminator, the generator comprises an encoder and a decoder which are based on a U-Net structure, the encoder is used for performing down-sampling operation on an input section crack diagram to obtain a corresponding characteristic image, the decoder is used for performing deconvolution up-sampling on the characteristic image to obtain a corresponding virtual ultrasonic B scanning diagram, and the discriminator is used for judging the authenticity of the input image;
3, training the countermeasure network by using the data set obtained in the step 1 and adopting a game method until the discriminator cannot judge whether the image is generated by the generator or not, and obtaining an image generation model for generating a virtual ultrasonic B scanning image;
and 4, inputting the self-made section crack diagram into the image generation model to obtain a corresponding virtual ultrasonic B scanning diagram, and taking the self-made section crack diagram and the corresponding virtual ultrasonic B scanning diagram as an enhanced data set for expanding the data set volume.
2. The data enhancement method for the ultrasonic tomography image of the concrete structure as claimed in claim 1, wherein in step 1, the ultrasonic B-scan is obtained by scanning a self-made reinforced concrete member by using an array ultrasonic tomography device.
3. The data enhancement method for the ultrasonic tomography image of the concrete structure as claimed in claim 1, wherein in step 2, the generator is a deep neural network comprising 16 modules, wherein the first 8 modules comprise an encoder for feature extraction, the last 8 modules comprise a decoder for feature synthesis, the convolution kernel size of the encoder is 4 x 4, three down-sampling operations are performed, the number of channels of the feature image after each down-sampling operation is multiplied by 2, the decoder corresponds to each layer of network structure of the encoder, the feature image is deconvoluted and up-sampled, and the number of channels of the feature image after each up-sampling operation is divided by 2.
4. The data enhancement method for the ultrasonic tomographic image of a concrete structure according to claim 1 or 3, wherein the input image size of the generator is 2304 x 1536 in units of pixels.
5. The data enhancement method for the ultrasonic tomography image of the concrete structure according to claim 1, wherein in step 2, the discriminator adopts a PatchGAN strategy to judge the authenticity of the input image.
6. The data enhancement method for the ultrasonic tomography image of the concrete structure as claimed in claim 1, wherein in the training process of the step 3, the game means comprises the steps of training the discriminator for a plurality of times by adopting a gradient ascending strategy cycle alone and then optimizing the generator by adopting a gradient descending strategy.
7. The data enhancement method for the ultrasonic tomography image of the concrete structure as claimed in claim 1, wherein in step 4, the homemade sectional crack map is generated by specifying the horizontal distance, the vertical distance and the initial position of the crack by random numbers, and the oblique line of the crack generation has an angle of 0-45 degrees with the horizontal line.
8. A data enhancement apparatus comprising a computer memory, a computer processor, and a computer program stored in the computer memory and executable on the computer processor, wherein the computer program employs the image generation model of claim 1; the computer processor executing the computer program implements the steps of: and inputting the self-made section crack diagram into an image generation model, and outputting a virtual ultrasonic B scanning diagram for expanding the data set.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211335574.5A CN115689924A (en) | 2022-10-28 | 2022-10-28 | Data enhancement method and device for concrete structure ultrasonic tomography image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211335574.5A CN115689924A (en) | 2022-10-28 | 2022-10-28 | Data enhancement method and device for concrete structure ultrasonic tomography image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115689924A true CN115689924A (en) | 2023-02-03 |
Family
ID=85046529
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211335574.5A Pending CN115689924A (en) | 2022-10-28 | 2022-10-28 | Data enhancement method and device for concrete structure ultrasonic tomography image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115689924A (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112837318A (en) * | 2021-03-29 | 2021-05-25 | 深圳大学 | Method for generating ultrasound image generation model, method for synthesizing ultrasound image generation model, medium, and terminal |
| CN114118362A (en) * | 2021-11-04 | 2022-03-01 | 同济大学 | Method for detecting structural surface crack under small sample based on generating type countermeasure network |
-
2022
- 2022-10-28 CN CN202211335574.5A patent/CN115689924A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112837318A (en) * | 2021-03-29 | 2021-05-25 | 深圳大学 | Method for generating ultrasound image generation model, method for synthesizing ultrasound image generation model, medium, and terminal |
| CN114118362A (en) * | 2021-11-04 | 2022-03-01 | 同济大学 | Method for detecting structural surface crack under small sample based on generating type countermeasure network |
Non-Patent Citations (1)
| Title |
|---|
| GEMOUMOU: "GAN生成对抗网络-PIX2PIXGAN原理与基本实现-图像翻译09", pages 1 - 9, Retrieved from the Internet <URL:https://blog.csdn.net/qq_37978800/article/details/111463055> * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6874846B2 (en) | Defect detection using ultrasonic scan data | |
| CN113850825A (en) | Remote sensing image road segmentation method based on context information and multi-scale feature fusion | |
| CN109345507A (en) | A kind of dam image crack detection method based on transfer learning | |
| Ijjeh et al. | Full wavefield processing by using FCN for delamination detection | |
| Wan et al. | Ceramic tile surface defect detection based on deep learning | |
| CN113643275B (en) | Ultrasonic defect detection method based on unsupervised manifold segmentation | |
| CN111915691B (en) | Image processing system, method, terminal and medium based on neural network | |
| CN113822880A (en) | Crack identification method based on deep learning | |
| CN110297041A (en) | A kind of 3D woven composite defect inspection method based on FCN and GRU | |
| CN113012123A (en) | Classification recognition and quantitative analysis method and system for defect and damage of carbon fiber composite material | |
| Cheng et al. | IRT-GAN: A generative adversarial network with a multi-headed fusion strategy for automated defect detection in composites using infrared thermography | |
| Aldrin et al. | Demonstration of using signal feature extraction and deep learning neural networks with ultrasonic data for detecting challenging discontinuities in composite panels | |
| WO2023109060A1 (en) | Device and method for identifying internal force chain of three-dimensional particle system | |
| Li et al. | A method of underwater bridge structure damage detection method based on a lightweight deep convolutional network | |
| CN115689924A (en) | Data enhancement method and device for concrete structure ultrasonic tomography image | |
| CN110779477B (en) | Acoustic method for identifying shape of object in real time | |
| Ni et al. | Toward high-precision crack detection in concrete bridges using deep learning | |
| Ziabari et al. | Simurgh: A Framework for CAD-Driven Deep Learning Based X-Ray CT Reconstruction | |
| Sutcliffe et al. | Automatic defect recognition of single-v welds using full matrix capture data, computer vision and multi-layer perceptron artificial neural networks | |
| CN110705615A (en) | Thyroid nodule detection and classification method based on TV model and GoogLeNet model | |
| CN115389514A (en) | Material defect detection method and device | |
| CN114972332A (en) | Bamboo laminated wood crack detection method based on image super-resolution reconstruction network | |
| Dahai et al. | A lightweight convolutional neural network for recognition and classification for Si3N4 chip substrate surface defects | |
| McKnight et al. | A comparison of methods for generating synthetic training data for domain adaption of deep learning models in ultrasonic non-destructive evaluation | |
| Yang et al. | Ultrasonic array tomography-oriented subsurface crack recognition and cross-section image reconstruction of reinforced concrete structure using deep neural networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |