CN115601572A

CN115601572A - Ultrasonic phased array image optimization reconstruction method and system based on semi-supervised CycleGan network

Info

Publication number: CN115601572A
Application number: CN202211337748.1A
Authority: CN
Inventors: 李兵; 高飞; 陈磊; 尚中昱; 刘春满
Original assignee: Xian Jiaotong University
Current assignee: Xian Jiaotong University
Priority date: 2022-10-28
Filing date: 2022-10-28
Publication date: 2023-01-13

Abstract

The invention discloses an ultrasonic phased array image optimization reconstruction method and system based on a semi-supervised CycleGan network, which generate training samples corresponding to each other in two domains of an ultrasonic image and a reconstructed image; overlapping the antagonism loss function, the cycle consistency loss function, the individual loss function and the real difference loss function to obtain a loss function; and constructing a semi-supervised CycleGan network structure based on the loss function, training the semi-supervised CycleGan network structure by using a training sample, and inputting an ultrasonic image actually detected by the ultrasonic phased array into the trained semi-supervised CycleGan network structure after network training to obtain a reconstructed image of the detected defect. The method can realize the heterogeneous reconstruction of the small target image on the premise of not cutting the original image; for the ultrasonic image, the position information of the defect in the image is reserved, the defect artifact in the image is eliminated, and the appearance of the defect is more accurate.

Description

Ultrasonic phased array image optimization reconstruction method and system based on semi-supervised CycleGan network

Technical Field

The invention belongs to the technical field of industrial ultrasonic nondestructive testing, and particularly relates to an ultrasonic phased array image optimization reconstruction method and system based on a semi-supervised CycleGan network.

Background

Ultrasonic and ultrasonic phased array detection are one of the important means for realizing the industrial nondestructive detection technology at present. The detection result shows the interior of the detected object visually in the form of a two-dimensional image, and mainly relates to the position, the geometric morphology and other characteristics of the defect. At present, in order to accurately and quantitatively characterize the internal defects of the object to be measured, the hardware performance of the equipment needs to be improved, so that a huge hardware development cost is spent. And due to the scattering influence of echo signals, the imaged defect appearance has artifacts, which are specifically represented by defect edge blurring, defect characterization distortion and the like. With the development of the field of deep learning, related network models are also gradually transplanted to the ultrasonic phased array, and researchers hope to perform artifact-free optimized reconstruction on an ultrasonic phased array image by using an image processing method in the deep learning.

At present, in the field of medical ultrasonic phased array imaging, a method for realizing ultrasonic image reconstruction by means of deep learning has been realized. But compared with an industrial ultrasonic phased array, the ultrasonic phased array has rich image content and strong correlation, and in industrial detection, the defects are less in the internal part of a detected object. This results in the content of the resulting image being background rich (in black) and the characterized defect being less correlated with the image background.

The existing method optimizes the performance of a discriminator in a CycleGan network by using the characteristic comparison of an input image and a reconstructed image. However, for an ultrasonic defect image, defect information is lost to a certain extent after multilayer convolution, so that the performance of the method in ultrasonic image reconstruction is not improved greatly.

How to accurately reconstruct a two-dimensional image (hereinafter referred to as an ultrasound image) formed by an ultrasound phased array into a two-dimensional image (hereinafter referred to as a reconstructed image) with defects in industrial detection is a problem to be solved at present. Research shows that the related method cannot be directly transplanted to the industrial ultrasonic phased array image reconstruction. Firstly, in industrial detection, the distribution and the appearance of defects inside an object are often random, and a supervised deep learning network cannot meet the requirements of open set practical application of industrial detection. Secondly, increasing the proportion of defects in the image by cropping enables the network to learn the image features, but this not only requires a huge cost for preprocessing the image, but also loses important characterization attributes of the ultrasound image for the defect location. The above methods cannot guarantee the accuracy of the reconstructed image information.

Disclosure of Invention

The technical problem to be solved by the invention is to provide an ultrasonic phased array image optimization reconstruction method and system based on a semi-supervised CycleGan network aiming at the defects in the prior art, and the method and system are used for solving the technical problem of high-precision defect reconstruction of ultrasonic phased array detection images.

The invention adopts the following technical scheme:

the ultrasonic phased array image optimization reconstruction method based on the semi-supervised CycleGan network comprises the following steps:

s1, generating training samples which correspond to each other in two domains of an ultrasonic image and a reconstructed image;

s2, overlapping the antagonism loss function, the cycle consistency loss function, the individual loss function and the real difference loss function to obtain a loss function;

and S3, building a semi-supervised CycleGan network structure based on the loss function obtained in the step S2, training the semi-supervised CycleGan network structure by using the training sample generated in the step S1, and inputting an ultrasonic image actually detected by the ultrasonic phased array into the trained semi-supervised CycleGan network structure after network training to obtain a reconstructed image of the detected defect.

Specifically, in step S1, performing ultrasonic imaging on the artificially designed defect in a two-dimensional plane, and forming a training sample by using an obtained ultrasonic image of the defect and an original defect image; or the actual measurement and the corresponding reconstruction calculation are carried out on the test block which is processed and contains accurate defect information, and a test block defect image and a corresponding ultrasonic image are respectively obtained from the processing information to form a training sample.

Specifically, in step S2, a loss function

The method specifically comprises the following steps:

wherein λ is _cyc ，λ _idt And λ _aut Respectively, the adjustable hyper-parameters are,

and

in order to combat the loss function, it is,

in order to be a function of the cyclic consistency loss,

as a function of the individual losses,

is a true difference loss function.

Further, a penalty function is resisted

And

respectively as follows:

wherein GA (a) represents samples a (a to P) of domain A _A ) Domain B image post-output via generator GA

GB (B) denotes the B samples of the domain B (B to P) _B ) Domain A image output after passing through generator GB

DA (b) represents the class score of the image b by the discriminator DA, DB (a) represents the class score of the image a by the discriminator DB,

representing the probability distribution of the image a obeying the domain a,

representing the probability distribution of image B obeying domain B.

Further, a circular consistency loss function

Comprises the following steps:

GB (B) denotes the B samples of the domain B (B to P) _B ) Domain A image output after generator GB

Representing the probability distribution of the image a obeying the domain a,

representing the probability distribution of image B obeying domain B.

Further, individual loss function

Comprises the following steps:

Representing the probability distribution that image a obeys domain a,

representing the probability distribution of image B obeying domain B.

Further, the true difference loss function

Comprises the following steps:

GB (B) denotes the B samples of the domain B (B to P) _B ) Domain A map output after generator GBImage (A)

Representing the probability distribution that image a obeys domain a,

representing the probability distribution of image B obeying domain B, MSE is a mean square error loss function.

Specifically, in step S3, the semi-supervised CycleGan network training process is as follows:

the ultrasonic image and the non-paired image of the defect image are simultaneously input into two different generators from two sides, and simultaneously output corresponding images conforming to the opposite domain, and the discriminator identifies the images output by the generators;

ultrasound image a as input side at

Stage, the image a distributed in domain A passes through generator GA and then outputs its defect image distributed in domain B

On the one hand, defective images

Similarity to domain B is resisted by discriminator DA against loss function

Evaluation of (2); on the other hand, in a loss function by true difference

Quantifying GA-generated defect images

The difference degree between the real defect image b corresponding to the ultrasonic image a;

then is on

Phase, domain B defect image of input generator GB

Outputting the corresponding domain A ultrasonic image

And with the input image a of the network by a cyclic consistency loss function

Performing domain similarity comparison;

defective image b as input side, successively

And with

Two stages, each of which is subjected to a discriminator DB penalty function

True difference loss function

And a cyclic consistency loss function

Of the system.

After finishing the bidirectional output of a batch of images, the semi-supervised CycleGan optimizes and adjusts the network parameters according to the calculated loss function items. When training is completed, the semi-supervised CycleGan network realizes the conversion from the ultrasonic image to the defect image and the conversion from the defect image to the ultrasonic image.

Further, the semi-supervised CycleGan network comprises a generator and a discriminator, wherein the generator comprises a Conv2d layer, a LeakyRelu layer, an InstanceNorm layer, a Relu layer, a TransConv2d layer and a Tanh layer; the discriminator adopts a pixel-by-pixel scoring structure built by multilayer convolution, and the pixel-by-pixel scoring structure comprises a Conv2d layer, a LeakyRelu layer and an InstanceNorm layer; the Conv2d layer is a two-dimensional convolution layer, and the TransConv2d layer is a two-dimensional transposed convolution layer.

In a second aspect, an embodiment of the present invention provides an ultrasonic phased array image optimization and reconstruction system based on a semi-supervised CycleGan network, including:

the sample module generates training samples which correspond to the two domains of the ultrasonic image and the reconstructed image one by one;

the function module is used for superposing the antagonism loss function, the cycle consistency loss function, the individual loss function and the real difference loss function to obtain a loss function;

and the reconstruction module is used for constructing a semi-supervised CycleGan network structure based on the loss function obtained by the function module, training the semi-supervised CycleGan network structure by using the training sample generated by the sample module, and utilizing the trained semi-supervised CycleGan network structure.

Compared with the prior art, the invention at least has the following beneficial effects:

the ultrasonic phased array image optimization reconstruction method based on the semi-supervised CycleGan network firstly generates a one-to-one corresponding training set required by a training model, secondly introduces a real difference loss function on the basis of the original unsupervised CycleGan network model, and carries out difference comparison on the different domain images respectively output by the generators GA and GB and the real different domain images in the training process, thereby enabling the network to carry out more targeted optimization on the defect appearance in the images. Finally, an image of ultrasonic phased array detection is input into the trained network, and the network can perform different-domain reconstruction aiming at the appearance of the defect on the basis of not losing defect positioning information, so that the influence of the artifact in the ultrasonic image on the appearance of the defect is eliminated.

Furthermore, a relatively objective detection environment can be set up in simulation software according to parameters of an actual detection instrument and a detected material, and defect characteristics of any position and morphology can be set. And the ultrasonic phased array is used for detecting the test block with the actual design defect, so that the defect image and the corresponding ultrasonic image can be more accurately acquired. No matter which method is adopted, the training set does not need to further cut the image or specifically mark the defect, so that the workload for manufacturing the training set is reduced. In addition, compared with the two methods, the cost of generating the data set by the simulation software is lower, and the flexibility of the sample is better.

Further, a loss function

By adding the sub-loss functions acting on all places in the network model, the accuracy of the image heterogeneous reconstruction is shown to be the result of the comprehensive action of all the sub-loss functions. Especially when the network model is reversely propagated through the pair

And the gradient of each sub-loss function can be solved by calculating the gradient, so that the code is convenient to realize.

Further, the countering loss function is an estimate by the discriminator of the authenticity of the distribution of the image output by the generator, wherein the authenticity of the distribution refers to whether the output image is proximate to the distribution of the expected domain. When the estimation is correct, the network model will improve the performance of the generator, and when the estimation is wrong, the network model will optimize the arbiter.

Further, the circular consistency loss function is a similarity comparison of a single-sided input with a corresponding output image. Because the domain of the image is converted for many times in the network model, the loss of the cycle consistency can ensure that only the style domain of the two images is changed on the basis of consistent content, thereby realizing the accurate reconstruction of the artifact defect in the ultrasonic image.

Further, the individual loss function is another parameter that evaluates the performance of the generator. Which calculates an estimate of the distribution of the image output by the generator in the same domain when the same domain image is input. The purpose is to ensure that the generator also outputs the in-field image when the in-field image is input, and to improve the stability of the generator from the other side.

Further, the real difference loss directly performs similarity evaluation on the different domain images output by the generators and the corresponding real different domain images in the training set. Whether the target contents in the images are consistent or not is emphasized, and the attention degree of the contents in the heterogeneous conversion task of the small target images is improved in a loss function mode.

Furthermore, a synchronous different-side sample input strategy is adopted to train the network model, firstly, only two different-domain images are randomly extracted from the training set in each batch, the training set amount depended on by network training is reduced, and the training efficiency of the network model is improved. Secondly, by configuring loss functions on each generator and each discriminator respectively and combining the loss functions evaluated by the whole network, the network can be optimized more pertinently, and the performance of the whole network can be ensured.

Furthermore, a generator and a discriminator are basic units for realizing image heterogeneous transformation by the network model, the generator can carry out heterogeneous reconstruction on the input image, and the discriminator carries out heterogeneous similarity estimation on the output image of the generator. The generator adopts a Unet network framework, which can not only extract high-dimensional features of the image, but also retain low-semantic information of the image. The discriminator adopts a multilayer convolution module structure which only performs dimensionality raising on a channel without changing the image size. The method can be used for scoring the reconstructed image pixel by pixel, so that the accuracy of image reconstruction is guaranteed at the pixel level. The network model optimizes the network parameters of the generator or the discriminator according to the scoring correctness of the discriminator, and improves the corresponding performance of the generator or the discriminator, thereby improving the performance of the generator and the discriminator after multiple iterations.

It is understood that the beneficial effects of the second aspect can be referred to the related description of the first aspect, and are not described herein again.

In conclusion, the method can realize the different-domain reconstruction of the small target image on the premise of not cutting the original image; for the ultrasonic image, the position information of the defect in the image is reserved, and the defect artifact in the image is eliminated, so that the appearance of the defect is more accurate; the simulation generated different domain images are used as a training set, and the defects in each image do not need to be specifically marked, so that the cost for generating the training set is greatly reduced.

The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.

Drawings

Fig. 1 is a schematic diagram of a training sample, in which (a) is a schematic diagram of an artificial defect design containing specific information, (b) is a defect image of a square hole, (c) is an ultrasound image generated by a simulation corresponding to the defect image of the square hole, (d) is a defect image of a circular hole, and (e) is an ultrasound image generated by a simulation corresponding to the defect image of the circular hole;

FIG. 2 is a schematic diagram of a semi-supervised CycleGan network training strategy, wherein (a) is a schematic diagram of a training process of the CycleGan network of the present invention, (B) is a flowchart of a training process from domain A to domain B, and (c) is a flowchart of a training process from domain B to domain A;

FIG. 3 is a schematic diagram of a network architecture of a generator;

FIG. 4 is a schematic diagram of a network structure of the arbiter;

FIG. 5 is a front-to-back comparison of a two-way reconstruction of an ultrasound phased array simulation image using the method of the present invention, wherein (a) is an input ultrasound image, (b) is a reconstructed defect image, (c) is a real defect image, (d) is an input defect image, (e) is a reconstructed ultrasound image, and (f) is a real ultrasound image;

fig. 6 is a comparison diagram before and after actual ultrasonic phased array image defect reconstruction by using the method, wherein, (a) is a defect image acquired by an ultrasonic phased array, and (b) is a defect image reconstructed by the network provided by the invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.

In the description of the present invention, it should be understood that the terms "comprises" and/or "comprising" indicate the presence of the stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It should be further understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and including such combinations, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.

It should be understood that although the terms first, second, third, etc. may be used to describe preset ranges, etc. in embodiments of the present invention, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from one another. For example, the first preset range may also be referred to as a second preset range, and similarly, the second preset range may also be referred to as the first preset range, without departing from the scope of the embodiments of the present invention.

The word "if" as used herein may be interpreted as "at 8230; \8230;" or "when 8230; \8230;" or "in response to a determination" or "in response to a detection", depending on the context. Similarly, the phrase "if determined" or "if detected (a stated condition or event)" may be interpreted as "upon determining" or "in response to determining" or "upon detecting (a stated condition or event)" or "in response to detecting (a stated condition or event)", depending on the context.

Various structural schematics according to the disclosed embodiments of the invention are shown in the drawings. The figures are not drawn to scale, wherein certain details are exaggerated and possibly omitted for clarity of presentation. The shapes of various regions, layers and their relative sizes and positional relationships shown in the drawings are merely exemplary, and deviations may occur in practice due to manufacturing tolerances or technical limitations, and a person skilled in the art may additionally design regions/layers having different shapes, sizes, relative positions, according to actual needs.

The invention provides an ultrasonic phased array image optimization reconstruction method based on a semi-supervised CycleGan network, which takes two image domains formed by a defect schematic diagram (hereinafter referred to as a defect image) designed manually and a corresponding ultrasonic image as a training sample set, realizes the mutual reconstruction of the images of the two domains through the proposed CycleGan semi-supervised neural network during training, and can output the reconstructed image only by inputting the ultrasonic image of the defect during testing; the maximum advantage is that the training sample size is extremely low, compared with a training set of thousands of images, the network model provided by the invention can meet the training requirement only by dozens of matched ultrasonic images and reconstruction images, and the corresponding feature learning of the ultrasonic images and the reconstruction images by the neural network is realized without any pretreatment on ultrasonic phased array imaging.

The invention relates to an ultrasonic phased array image optimization reconstruction method based on a semi-supervised CycleGan network, which comprises the following steps of:

s1, generating a training sample

The samples used for the network structure training proposed by the invention need to be respectively in one-to-one correspondence in two domains of an ultrasonic image and a reconstructed image, and are realized by two methods:

the first method is to perform ultrasonic imaging of a defect designed manually in a two-dimensional plane using simulation software, thereby acquiring an ultrasonic image of the defect and an original defect image.

The second method is to utilize ultrasonic phased array equipment to perform actual measurement and corresponding reconstruction calculation on the test block which is processed and contains accurate defect information, so as to respectively obtain a test block defect image and a corresponding ultrasonic image from the processing information.

Referring to fig. 1, a square hole with a side length of 1mm and a circular hole with a diameter of 1mm are respectively designed at two designated positions. And calculating by simulation software to obtain an ultrasonic image corresponding to the defect image. By the method, the defect image and the ultrasonic image in pairs can be designed and acquired at any position, so that a training sample is generated.

S2, designing a loss function

In the network model provided by the invention, four loss functions such as antagonism loss, cycle consistency loss, individual loss and real difference loss are respectively defined. The first three items are the same as the loss function calculation method defined in the standard CycleGan network. And superposing the four loss functions to obtain a final model loss function. The expectation of the four loss functions is different, and the network model is limited from different angles, but the four loss functions are not limited.

Specifically, the resistance loss is to identify the authenticity of an image output by a generator in a set of generators and discriminators in a CycleGan network by using the discriminators. In FIG. 2 (B), the domain A image a passes through the generator GA and the domain B image is output

For example, then the loss is resisted

Is represented as follows:

Discriminator DA respectively pairTrue samples B (B-P) in Domain B _B ) And false true samples

And (5) performing identification. In this process, it is desirable that the discriminator DA can discriminate accurately true and false samples of the same domain, so a maximization scheme is taken for the discriminator

At the same time, it is desirable that the false true samples of the generator are as similar in domain as the true samples as possible, so a minimization scheme is taken for the generator

Similarly, in FIG. 2 (c), the domain B image B passes through the generator GB and outputs the domain A image

Namely:

the cycle consistency loss comprises two sub items when the domain A is converted to the domain B and the domain B is converted to the domain A, and the two sub item structures simultaneously exert constraint on the network in the two-way training process. Taking the example of turning domain A to domain B and then to domain A in FIG. 2B, the cyclic consistency loss is shown in equation (3):

in equation (3), the domain A samples a first pass through the generator GA, and then pass through the generator GB, and the output after two conversions belongs to the domain A samples

Namely GB (GA (a)). Then by calculating the sample a and

the similarity of the image content is used for measuring the accuracy of the generator for reconstructing the image content.

Similarly, the cyclic consistency loss in the direction from domain B to domain A and then to domain B in FIG. 2 (c) is shown in equation (4):

and (3) accumulating the formula (3) and the formula (4) to obtain the cycle consistency loss of the CycleGan network, as shown in the formula (5):

the above loss functions are all for ensuring that the generator can accurately convert when the different domain image is input. When the same domain image is input, in order to ensure that the generator does not perform domain conversion on it, an individual loss is set, which is shown in equation (6):

in formula (6), b and a are input to the generators GA and GB, respectively, and similarity evaluation is performed with their own images, thereby ensuring that the images do not change.

Because the defect content in the ultrasonic image is less, on the basis that the discriminator judges whether the generated image is true or false, the different-domain same-content image of the input image is used as a truth label, and the real difference loss is directly compared with the generated image in a similarity way, so that the generator is guided to enhance the performance of the image, and the real difference loss is shown as a formula (7):

finally, the integral loss function of the network model is obtained by accumulating the above formulas (1) to (7), as shown in formula (8):

wherein λ is _cyc ，λ _idt And λ _aut Respectively, adjustable hyper-parameters, for balancing the weights of the corresponding three loss function components in the overall function.

S3, building a semi-supervised CycleGan network and training;

the standard CycleGan network is an unsupervised neural network. Which implements a style conversion of an image by inputting non-paired images of two different domains. However, due to the particularity of the industrial ultrasound image, the industrial ultrasound image cannot be effectively reconstructed into a defect image. Therefore, the invention provides a semi-supervised CycleGan network structure and a corresponding training strategy thereof, wherein the training strategy is shown in figure 2.

As seen in fig. 2 (a), the training of the CycleGan network is bi-directional, i.e., the unpaired images of domain a and domain B are input to two different generators from both sides simultaneously, and the corresponding images conforming to the counterpart domains are output simultaneously. In this process, the discriminator discriminates the image output by the generator to improve the imaging performance of the generator. When the training is completed, the network can realize the conversion from the ultrasonic image (domain A) to the defect image (domain B) (the red in the figure 2 (a) is in the direction of the arrow from left to right), and can also realize the conversion from the defect image (domain B) to the ultrasonic image (domain A) (the blue in the figure 2 (a) is in the direction of the arrow from right to left). During testing and application, the reconstruction of the defect image can be realized only by utilizing the model data of the generator A and inputting the ultrasonic image. Fig. 2 (B) and 2 (c) are training flow diagrams of two specific domains, where a and B belong to domain a and domain B, respectively. Taking fig. 2 (b) as an example, after the input image a of the domain a passes through the first generator GA, on the one hand, it is necessary to use the discriminator DA to compare the image generated by the GA

Whether or not the domain B distribution is indeed met is determined, and on the other hand, it is necessaryPut it in the output image of the domain B

And evaluating the similarity of the image B corresponding to the image a in the original domain B to ensure the accuracy of the reconstruction of the image a in the domain B. Then will be

Input to a second generator GB, the output of which

Is from domain B

Results of reconstruction to Domain A

FIG. 2 (c) shows the same principle.

In the semi-supervised CycleGan network, a generator adopts a Unet network structure to realize coding and decoding tasks of image characteristics, and a discriminator adopts a pixel-by-pixel scoring structure built by multilayer convolution to distinguish the authenticity of an image. It should be noted that the embodiments of the generator proposed by the present invention include not only the network using the Unet network, but also other network structures for image feature extraction, and the adjustment of the specific layer configuration in the Unet according to the specific image characteristics should also fall within the scope covered by the claims of the present invention. Taking the training process from domain a to domain B as an example, the generator is shown in fig. 3.

In fig. 3, the Conv2d layer is a two-dimensional convolution layer, the TransConv2d layer is a two-dimensional transposed convolution layer, and the other layers are normally expressed. In the training process from the domain A to the domain B, the input ultrasonic image not only realizes the downward extraction of features through the convolution module layer by layer on the left side so as to obtain high semantic information, but also transversely and directly transmits low semantic information to the up-sampling module on the right side in the channel dimension so as to ensure the completeness of the low semantic information.

The image output by the generator needs to be input into a discriminator to verify the reconstruction performance of the generator, and the invention uses a multi-layer convolution module which only carries out dimensionality lifting on channels without changing the image size. Compared with other Markov classifiers (PatchGan) based on the increased receptive field, the MarkchGan based reconstruction method can score the reconstructed image pixel by pixel, so that the accuracy of image reconstruction is guaranteed on a pixel level. The network structure of the arbiter is shown in fig. 4.

In fig. 4, the size of the pixel-by-pixel scoring matrix is consistent with the input ultrasound image and only expands in the channel dimension during convolution, without compressing the feature size, thereby ensuring the accuracy of the scoring at the pixel level.

In addition, the network model provided by the invention adopts an Adam optimizer to optimize the network loss function.

For convenience of discussion, L1 loss and MSE loss are used in equations (1) to (7) to evaluate the similarity between features and images, but in practice, other loss functions for evaluating the similarity between images, including, but not limited to, L2 loss and SSIM loss, should also be covered by this patent.

In another embodiment of the present invention, an ultrasonic phased array image optimization and reconstruction system based on a semi-supervised CycleGan network is provided, and the system can be used for implementing the ultrasonic phased array image optimization and reconstruction method based on the semi-supervised CycleGan network.

Wherein,.

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The method provided by the invention is adopted to test and verify the simulated ultrasonic image and the defect-containing image obtained by actual detection.

The resolution of the simulated image and the resolution of the actually detected defect image are both 256 × 256, wherein the reconstruction result by using the simulated ultrasound image is shown in fig. 5 and table 1:

TABLE 1 comparison of ultrasound simulation image reconstruction results

Numbering	Image name	X-axis/pixel	Y-axis/pixel	Area/pixel
					1	Input ultrasound image	51	64	318
2	Reconstructed defect image	52	64	44
					3	Real defect image	51	63	32
4	Input defect image	102	115	39
					5	Reconstructed ultrasound image	102	114	484
6	True ultrasound image	102	115	308

As can be seen from fig. 5 and table 1, fig. 5 (a), (b), (c) and numbers 1 to 3 show the performance of reconstruction from an input ultrasound image into a defect image. Compared with the real defect image, the reconstructed defect image has a 1-pixel shift on the X-axis/Y-axis, whereas the defect occupies 318 pixels in the ultrasound image, whereas the reconstructed defect occupies only 44 pixels in the image, and the corresponding real defect occupies 32 pixels in terms of defect area. Meanwhile, fig. 5 (d), (e), (f) and numbers 4 to 6 show the performance of reconstructing an ultrasound image from an input defect image. The defects of the reconstructed ultrasound image are only shifted by 1 pixel on the Y-axis. The above contrast illustrates that the present invention can ensure high accuracy reconstruction of defects from ultrasound images.

Finally, the method provided by the invention is adopted to verify the defect-containing image obtained by actual detection, and the result is shown in fig. 6 and table 2

TABLE 2 ultrasonic testing image reconstruction results

Defect numbering	X-axis/pixel	Y-axis/pixel	Area/pixel
				1-ultrasonic image	229	182	342
1-reconstruction of images	228	182	217
				2-ultrasonic image	177	130	212
2-reconstruction of images	177	130	130
				3-ultrasonic image	124	79	232
3-reconstruction of images	125	79	144
				4-ultrasonic image	73	27	244
4-reconstruction of images	73	27	155
				5-ultrasonic image	46	27	244
5-reconstructed image	46	27	154

As shown in fig. 6 and table 2, the defects numbered 1 to 5 can maintain reconstruction accuracy within 1 pixel in positioning after reconstruction, and 5 defects can be effectively reduced as seen from the defect area in the image, thereby further improving the appearance characterization of the defects in the image.

In conclusion, the ultrasonic phased array image optimization reconstruction method and system based on the semi-supervised CycleGan network can realize the different-domain reconstruction of the small target image on the premise of not cutting the original image. For the ultrasonic image, the position information of the defect in the image is reserved, the defect artifact in the image is eliminated, and the appearance of the defect is more accurate. In the method, the different domain images generated by simulation are used as the training set, and the defects in each image do not need to be specifically marked, so that the cost for generating the training set is greatly reduced.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The above-mentioned contents are only for illustrating the technical idea of the present invention, and the protection scope of the present invention is not limited thereby, and any modification made on the basis of the technical idea of the present invention falls within the protection scope of the claims of the present invention.

Claims

1. The ultrasonic phased array image optimization reconstruction method based on the semi-supervised CycleGan network is characterized by comprising the following steps of:

and S3, building a semi-supervised CycloGan network structure based on the loss function obtained in the step S2, training the semi-supervised CycloGan network structure by using the training sample generated in the step S1, and inputting an ultrasonic image actually detected by the ultrasonic phased array into the trained semi-supervised CycloGan network structure after network training to obtain a reconstructed image of the detected defect.

2. The ultrasonic phased array image optimization reconstruction method based on the semi-supervised CycleGan network as recited in claim 1, wherein in step S1, the artificially designed defect is ultrasonically imaged in a two-dimensional plane, and an obtained ultrasonic image of the defect and an original defect image form a training sample; or the processed test block containing the accurate defect information is actually measured and correspondingly reconstructed and calculated, and a test block defect image and a corresponding ultrasonic image are respectively obtained from the processing information to form a training sample.

3. The ultrasonic phased array image optimization reconstruction method based on semi-supervised CycleGan network as claimed in claim 1, wherein in step S2, the loss function

The method specifically comprises the following steps:

wherein λ is _cyc ，λ _idt And λ _aut Respectively, the adjustable hyper-parameters are the parameters,

and

in order to combat the loss-function,

in order to be a function of the cyclic consistency loss,

as a function of the individual losses,

as a function of the true variance loss.

4. The ultrasonic phased array image optimization reconstruction method based on semi-supervised CycleGan network as claimed in claim 3, wherein the penalty function is resisted

And

respectively as follows:

representing the probability distribution of the image a obeying the domain a,

representing the probability distribution of image B obeying domain B.

5. The ultrasonic phased array image optimization reconstruction method based on semi-supervised CycleGan network as claimed in claim 3, wherein the cyclic consistency loss function

Comprises the following steps:

Representing the probability distribution of the image a obeying the domain a,

representing the probability distribution of image B obeying domain B.

6. The ultrasonic phased array image optimization reconstruction method based on semi-supervised CycleGan network as claimed in claim 3, wherein individual loss function

Comprises the following steps:

Representing the probability distribution of the image a obeying the domain a,

representing the probability distribution of image B obeying domain B.

7. The ultrasonic phased array image optimization reconstruction method based on semi-supervised CycleGan network as claimed in claim 3, wherein the true difference loss function

Comprises the following steps:

Representing the probability distribution that image a obeys domain a,

8. The ultrasonic phased array image optimization and reconstruction method based on the semi-supervised CycleGan network as claimed in claim 1, wherein in the step S3, the training process of the semi-supervised CycleGan network is as follows:

the ultrasound image a is used as an input side

On the one hand, defective images

Similarity to domain B is resisted by discriminator DA against the loss function

Evaluation of (2); on the other hand, in a loss function by true difference

Quantifying GA-generated defect images

The difference degree of the real defect image b corresponding to the ultrasonic image a;

then is on

Phase, domain B Defect image input to Generator GB

Outputting the corresponding domain A ultrasonic image

And with the input image a of the network by a cyclic consistency loss function

Performing domain similarity comparison;

defective image b as input side, successively

And

two stages, each of which is subjected to a discriminator DB penalty function

True difference loss function

And a cyclic consistency loss function

The constraint of (2);

after finishing the bidirectional output of a batch of images, carrying out optimization adjustment on network parameters by the semi-supervised CycleGan according to the calculated loss function items; when training is completed, the semi-supervised CycleGan network realizes the conversion from the ultrasonic image to the defect image and the conversion from the defect image to the ultrasonic image.

9. The ultrasonic phased array image optimization reconstruction method based on the semi-supervised CycleGan network is characterized in that the semi-supervised CycleGan network comprises a generator and a discriminator, wherein the generator comprises a Conv2d layer, a LeakyRelu layer, an instanceNorm layer, a Relu layer, a TransConv2d layer and a Tanh layer; the discriminator adopts a pixel-by-pixel scoring structure built by multilayer convolution, and the pixel-by-pixel scoring structure comprises a Conv2d layer, a LeakyRelu layer and an InstanceNorm layer; the Conv2d layer is a two-dimensional convolutional layer, and the TransConv2d layer is a two-dimensional transpose convolutional layer.

10. An ultrasonic phased array image optimization and reconstruction system based on a semi-supervised CycleGan network is characterized by comprising:

the function module is used for superposing the antagonism loss function, the cyclic consistency loss function, the individual loss function and the real difference loss function to obtain a loss function;