WO2022126480A1

WO2022126480A1 - High-energy image synthesis method and device based on wasserstein generative adversarial network model

Info

Publication number: WO2022126480A1
Application number: PCT/CN2020/137188
Authority: WO
Inventors: 郑海荣; 胡战利; 梁栋; 刘新; 周豪杰
Original assignee: 深圳先进技术研究院
Priority date: 2020-12-17
Filing date: 2020-12-17
Publication date: 2022-06-23

Abstract

A high-energy image synthesis method based on a Wasserstein generative adversarial network model. The method comprises: obtaining a low-energy image to be synthesized (S11); and inputting the low-energy image to be synthesized into a pre-trained Wasserstein generative adversarial network model to obtain a synthesized target high-energy image (S12), wherein the Wasserstein generative adversarial network model is obtained by training a preset generative adversarial network model on the basis of a low-energy image sample, a standard high-energy image, and a preset loss function, and the preset loss function is established at least according to a loss function used for reducing image noise and removing image artifacts.

Description

High-energy image synthesis method and device based on Wasserstein generative adversarial network model

technical field

The present application relates to the technical field of image processing, and in particular, to a high-energy image synthesis method and device based on the Wasserstein generative confrontation network model.

Background technique

Dual-energy Computed Tomography (Dual-energy Computed Tomography, dual-energy CT) has gradually become a more effective non-invasive diagnostic method, which can be applied to traditional computed tomography, which uses two different energies of x-rays. By scanning, the obtained dataset has richer scanning information, which can be applied to more clinical applications, such as urinary tract stone detection, tophi detection, and bone and metal artifact removal. Moreover, compared with traditional computed tomography, since the scanning mode of dual-energy computed tomography can use half of the low-energy scan to replace the original high-energy scan, the radiation dose can also be reduced.

However, since dual-energy CT needs to use both high- and low-energy scanning during the scanning process, signal cross-interference is prone to occur, and there is a short time interval. Moreover, with the energy accumulation of high-energy scanning, it will cause the possibility of various diseases, which will affect human health.

Therefore, how to research and develop a method for generating high-quality high-energy images with less interference and less deviation is a technical problem that those skilled in the art need to solve at present.

SUMMARY OF THE INVENTION

The embodiment of the present application provides a high-energy image synthesis method and device based on the Wasserstein generative adversarial network model, so as to solve the problems of large interference deviation and poor image quality in the prior art when the dual-energy CT method is used to obtain CT images. .

The embodiments of the present application adopt the following technical solutions.

In a first aspect, this embodiment provides a high-energy image synthesis method based on a Wasserstein generative adversarial network model, including: acquiring a low-energy image to be synthesized; inputting the low-energy image to be synthesized into a pre-trained Wasserstein generative adversarial network model to obtain The synthesized target high-energy image; the Wasserstein generative adversarial network model is based on low-energy image samples, standard high-energy images and a preset loss function, and is obtained by training a preset generative adversarial network model. The Wasserstein generative adversarial network model includes a generator network and a discriminator network. The generator network is used to extract the image features of the low-energy images to be synthesized, and synthesize high-energy images based on the image features; the discriminator network is used to judge the high-energy images synthesized by the generator network and perform reverse adjustment training; preset loss function , at least established from the loss function used to reduce image noise and remove image artifacts.

Optionally, the loss function used to reduce image noise and remove image artifacts, based on the gradient of the standard high-energy image in the x direction, the gradient of the standard high-energy image in the y direction, the gradient of the synthetic high-energy image in the x direction, and the synthetic high-energy image. The gradient in the y direction is established.

Optionally, the preset loss function is further established according to at least one of the following loss functions: a preset pixel difference calibration function for calibrating the pixel difference between the synthesized high-energy image and the standard high-energy image; a preset pixel difference calibration function for calibrating the synthesized high-energy image A preset structural loss function for the structural information difference between the high-energy image and the standard high-energy image; a preset multi-scale feature loss function for calibrating the texture information difference between the synthesized high-energy image and the standard high-energy image.

Optionally, the preset loss function is established according to a preset gradient loss function, a preset pixel difference calibration function, a preset structural loss function, a preset multi-scale feature loss function, and a preset generative adversarial network model.

Optionally, before inputting the low-energy image to be synthesized into the Wasserstein generative adversarial network model obtained by pre-training to obtain the synthesized target high-energy image, the method further includes: based on the low-energy image sample, the standard high-energy image and the preset loss function, through The Wasserstein generative adversarial network model is obtained by training the preset generative adversarial network model; wherein, based on low-energy image samples, standard high-energy images and a preset loss function, the Wasserstein generative adversarial network model is obtained by training the preset generative adversarial network model, including: The sample is input to the generator network of the preset generative adversarial network model to obtain a synthesized first high-energy image; the first high-energy image is input to the discriminator network of the preset generative adversarial network model to obtain a first discrimination result; based on the first high-energy image For the image and the standard high-energy image, the first loss value is calculated according to the preset loss function, and the first loss value is used to update the parameters of the preset generative adversarial network model until the preset generative adversarial network converges; based on the first loss value and the first loss value The discrimination result updates the preset generative adversarial network model until the preset generative adversarial network model converges, and the converged preset generative adversarial network model is determined as the Wasserstein generative adversarial network model.

Optionally, if the preset loss function includes a preset pixel difference calibration function, then based on the first high-energy image and the standard high-energy image, calculating and obtaining the first loss value according to the preset loss function, including: using the preset pixel difference calibration function, Calculate the pixel difference value between the first high-energy image and the standard high-energy image; determine the pixel difference value as the first loss value.

Optionally, if the preset loss function includes a preset structural loss function, then based on the first high-energy image and the standard high-energy image, the first loss value is calculated and obtained according to the preset loss function, including: by using the preset structural loss function, A structural difference value between the first high-energy image and the standard high-energy image is determined; the structural difference value is determined as a first loss value.

Optionally, if the preset loss function includes a preset multi-scale feature loss function, then based on the first high-energy image and the standard high-energy image, the first loss value is calculated and obtained according to the preset loss function, including: using the preset multi-scale feature loss function to determine the texture information difference value between the first high-energy image and the standard high-energy image; and determine the texture information difference value as the first loss value.

Optionally, the generator network of the Wasserstein generative adversarial network model includes a semantic segmentation network with 4 layers of encoders and decoders. Each layer of encoders and decoders is connected by a skip link, and the encoding layer and the decoding layer of the semantic segmentation network include 9 layers of Residual network; the discriminator network of the Wasserstein generative adversarial network model includes 8 groups of 3*3 convolutional layers and activation function LReLU; among them, the convolutional layers and activation function LReLU convolution steps located in the singular position from left to right The length is 1, and the convolution stride of the convolutional layer at the even position and the activation function LReLU is 2.

In a second aspect, this embodiment also provides a high-energy image synthesis device based on the Wasserstein generative adversarial network model, including an acquisition module and an input module, wherein: the acquisition module is used to acquire the low-energy image to be synthesized; the input module is used to combine The low-energy image to be synthesized is input into the pre-trained Wasserstein generative adversarial network model to obtain the synthesized target high-energy image; the Wasserstein generative adversarial network model is trained by the preset generative adversarial network model learning method; Wasserstein generative adversarial network model Based on low-energy image samples, standard high-energy images and preset loss functions, it is obtained by training a preset generative adversarial network model. The Wasserstein generative adversarial network model includes a generator network and a discriminator network. The generator network is used to extract the low-energy images to be synthesized. image features, and synthesize high-energy images based on image features; the discriminator network is used to judge the high-energy images synthesized by the generator network, and perform reverse adjustment training; preset loss functions, at least according to the image noise reduction and image removal A loss function for the artifact is built.

Optionally, the loss function used to reduce image noise and remove image artifacts can be based on the gradient of the standard high-energy image in the x-direction, the gradient of the standard high-energy image in the y-direction, the gradient of the synthetic high-energy image in the x-direction, and the synthetic high-energy image. The gradient of the image in the y direction is built.

Optionally, the device further includes: a training module for obtaining a Wasserstein generative adversarial network model by training a preset generative adversarial network model based on low-energy image samples, standard high-energy images and a preset loss function; wherein, the training module includes: The first input unit is used to input the low-energy image samples into the generator network of the preset generative adversarial network model to obtain the synthesized first high-energy image; the second input unit is used to input the first high-energy image to the preset generative adversarial network. The discriminator network of the network model obtains the first discrimination result; the computing unit is used for calculating the first loss value according to the preset loss function based on the first high-energy image and the standard high-energy image, and the first loss value is used to update the preset generation parameters of the adversarial network model until the preset generative adversarial network converges; the updating unit is used to update the preset generative adversarial network model based on the first loss value and the first discrimination result until the preset generative adversarial network model converges, and after the convergence The preset generative adversarial network model is determined to be the Wasserstein generative adversarial network model.

Optionally, if the preset loss function includes a preset pixel difference calibration function, the calculation unit is configured to: calculate the pixel difference value between the first high-energy image and the standard high-energy image by using the preset pixel difference calibration function; The difference value is determined as the first loss value.

Optionally, if the preset loss function includes a preset structural loss function, the computing unit is configured to: determine the structural difference value between the first high-energy image and the standard high-energy image by using the preset structural loss function; The difference value is determined as the first loss value.

Optionally, if the preset loss function includes a preset multi-scale feature loss function, the computing unit is configured to: determine the texture information difference value between the first high-energy image and the standard high-energy image by using the preset multi-scale feature loss function ; Determine the texture information difference value as the first loss value.

An electronic device, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being executed by the processor to implement the first aspect Steps of a high-energy image synthesis method based on the Wasserstein generative adversarial network model.

A computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the high-energy image synthesis method based on the Wasserstein generative adversarial network model as described in the first aspect is implemented. step.

The above-mentioned at least one technical solution adopted in the embodiments of the present application can achieve the following beneficial effects:

In the embodiment of the present application, because the Wasserstein generative adversarial network model is based on low-energy image samples, standard high-energy images, and a preset loss function, and is obtained by training a preset generative adversarial network model, and the preset loss function, at least according to the The loss function for image noise and image artifact removal is established. Therefore, by inputting the low-energy image to be synthesized into the Wasserstein generative adversarial network model obtained by pre-training, the method of synthesizing the target high-energy image can reduce the pair of image noise and image artifact. The effect of image edges, thereby improving the quality of the synthetic target high-energy image.

Description of drawings

The drawings described herein are used to provide further understanding of the present application and constitute a part of the present application. The schematic embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation of the present application. in the attached image.

FIG. 1 is a schematic diagram of the implementation flow of a high-energy image synthesis method based on a Wasserstein generative adversarial network model provided by an embodiment of the present application.

FIG. 2 is a schematic structural diagram of a generator network of a Wasserstein generative adversarial network model provided by an embodiment of the present application.

FIG. 3 is a schematic diagram of a discriminator network structure of a Wasserstein generative adversarial network model provided by an embodiment of the present application.

FIG. 4 is a schematic diagram of a model training implementation flow of a Wasserstein generative adversarial network model provided by an embodiment of the present application.

FIG. 5 is a schematic diagram of an application process of the method provided by the embodiment of the present application in practice.

FIG. 6 is a schematic structural diagram of a high-energy image synthesis device based on a Wasserstein generative adversarial network model according to an embodiment of the present application.

FIG. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below with reference to the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

The technical solutions provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

Example 1

In order to solve the problems of large interference deviation and poor image quality in the related art when a CT image is obtained by using the dual-energy CT method, the embodiment of the present application provides a high-energy image synthesis method based on the Wasserstein generative adversarial network model.

The execution body of the method may be various types of computing devices, or may be an application or an application (Application, APP) installed on the computing device. The computing device, for example, may be a user terminal such as a mobile phone, a tablet computer, a smart wearable device, or the like, or a server.

For the convenience of description, the embodiment of the present application takes the execution body of the method as a server as an example to introduce the method. Those skilled in the art can understand that the server is used as an example to introduce the method in this embodiment of the present application, which is only an exemplary illustration, and does not limit the protection scope of the claims corresponding to the solution.

The implementation process of the method provided by the embodiment of the present application is shown in FIG. 1 , and includes the following steps.

Step 11, acquiring the low-energy image to be synthesized.

The low-energy image can be understood as the energy spectral CT image of the imaging object under the low-dose radiation/low-energy radiation. For example, taking the lung as the imaging object as an example, the low-energy image may include a lung energy spectrum CT image under a low-dose X-ray.

Generally, spectral CT images obtained under low-dose radiation/low-energy radiation may contain a lot of noise and artifacts, which affect the image quality. In order to reduce the impact of noise and artifacts on image quality, a preset method can be used based on the low-energy image to synthesize the low-energy image into a high-energy CT image with high density, high resolution, and low noise. Correspondingly, the low-energy image to be synthesized in the embodiment of the present application may include the low-energy CT image to be synthesized of the high-energy image.

In the embodiment of the present application, the low-energy image to be synthesized can be acquired through the X-ray tube under the condition of lower tube current and lower tube voltage. Alternatively, in the case of irregular sampling and missing data, the low-energy image to be synthesized can also be obtained by using the statistical reconstruction method, taking advantage of the advantages of its accurate physical model and insensitivity to noise.

It should be noted that the above-mentioned method for obtaining a low-energy image to be synthesized is only an exemplary description of the embodiment of the present application, and does not impose any limitation on the embodiment of the present application.

Step 12: Input the low-energy image to be synthesized into the Wasserstein generative adversarial network model obtained by pre-training to obtain the synthesized target high-energy image.

Among them, the Wasserstein generative adversarial network model is based on low-energy image samples, standard high-energy images and preset loss functions, and is obtained by training a preset generative adversarial network model. The Wasserstein generative adversarial network model includes a generator network and a discriminator network. The generator network is used for The image features of the low-energy images to be synthesized are extracted, and high-energy images are synthesized based on the image features; the discriminator network is used to judge the high-energy images synthesized by the generator network, and perform reverse adjustment training.

The target high-energy image can be understood as a high-energy CT image with high density, high resolution and low noise synthesized based on the low-energy image. A standard high-energy image can be understood as a high-energy CT image with high density, high resolution, high texture detail, and low noise.

In the embodiment of the present application, before the low-energy image to be synthesized is input into the Wasserstein generative adversarial network model obtained by pre-training, and the synthesized target high-energy image is obtained, the low-energy image sample, the standard high-energy image and the preset loss function can be pre-based, The Wasserstein generative adversarial network model is obtained by training the preset generative adversarial network model.

In one embodiment, the generator network, the discriminator network, and the parameters of the preset generative adversarial network model may be predetermined; then, the preset generative adversarial network model is determined according to the determined generator network, the discriminator network and the parameters; Finally, the preset generative adversarial network model is trained based on low-energy image samples, standard high-energy images and a preset loss function to obtain the Wasserstein generative adversarial network model.

For example, as shown in Fig. 2, the generator network in this embodiment of the present application may include a 4-layer encoding and decoding semantic segmentation network U-Net and a feature extraction network. Among them, a 9-layer residual network (Residual Block in the figure) can be included between the encoding layer and the decoding layer of the semantic segmentation network, and the residual network can be composed of nine 3x3 convolutions and ReLU activation functions. In addition, in order to avoid problems such as gradient disappearance and gradient explosion that may occur during the model training process, in the embodiment of the present application, a skip link mode may be selected to connect between each layer of codecs. The feature extraction network can include two 3x3 convolution and ReLU activation functions (Conv+LReLU in the figure); usually, when entering the next network layer through the feature extraction network, the feature information extracted by the feature extraction network can be processed first. In one pooling operation (Pooling in the figure), the number of channels can be gradually doubled three times from 64 (n64 in the figure) in the first layer to 512 (n512 in the figure), and then reach the residual network. It should be noted that the encoding process is symmetrical with the decoding process, and the final reconstruction network (Conv in the figure) is compressed to 1 channel (n1 in the figure) by 3*3 convolution.

As shown in Figure 3, the discriminator network can include 8 groups of 3*3 convolutional layers and activation functions LReLU (Conv+LReLU in the figure); among them, the convolutional layers and activation functions located in singular positions from left to right The convolution stride s of LReLU is 1 (s1 in the figure), and the convolution stride s of the convolutional layer and the activation function LReLU at the even position is 2 (s2 in the figure). In other words, the convolution stride s alternates between 1 and 2, respectively. Optionally, the number of channels n can be gradually doubled from 32 to 256. The last two layers (FC(1024) LReLU and FC(1) in the figure) include two convolutional layers to determine whether the output image is Standard high energy image.

After the above steps are performed and the generator network and the discriminator network of the preset generative adversarial network model are determined, the objective function and corresponding parameters of the preset generative adversarial network model can be further determined.

In an optional implementation, for example, a generative adversarial network model centered on the Wasserstein distance measure can be used as a preset generative adversarial network model, and the target parameters of the generative adversarial network model are shown in the following formula (1).

Among them, L _WGAN (G, D) represents the Wasserstein adversarial network model, G represents the generator network of the Wasserstein adversarial network model, D represents the discriminator network of the Wasserstein adversarial network model,

Represents a fixed discriminator network D, so that the discriminator can maximally determine whether the sample belongs to a synthetic high-energy image or a standard high-energy image,

represents the generator network G under the condition of a fixed discriminator network D,

Represents the expected value of the discriminator network D, P _r represents the probability distribution of high-energy images; P _z represents the probability distribution of synthesized high-energy images;

represents the probability distribution randomly collected in the distribution of standard high-energy images and synthetic high-energy images; λ represents the penalty coefficient, which is used to avoid the mode collapse and gradient disappearance problems that occur during the training of the preset generative adversarial network model.

After the preset generative adversarial network model is determined, model training can be performed on the preset generative adversarial network model based on low-energy image samples, standard high-energy images and a preset loss function to obtain a Wasserstein generative adversarial network model.

Wherein, when training the preset generative adversarial network model, as shown in FIG. 4 , the following steps 41 to 44 may be adopted.

Step 41 , inputting the low-energy image samples into the generator network of the preset generative adversarial network model to obtain a synthesized first high-energy image.

For example, a low-energy image sample with an image size of 256x256 can be input into the generator network of the preset generative adversarial network model, so that the feature extraction network in the generator network can extract the high-frequency information in the low-energy image based on the low-energy image. low-frequency information, and then perform image reconstruction on the extracted feature information to obtain a synthesized first high-energy image.

In one embodiment, the high-frequency information and low-frequency information of the low-energy image can be extracted first through the feature extraction network in the generator network; The information is encoded. In the encoding process, before entering the next layer, a pooling operation needs to be performed for the high-frequency information and low-frequency information of the low-energy image. The channel is gradually doubled from 64 in the first layer to three times. is 512, reaching the residual network in the generator network; finally, decoding is performed based on the decoding layer of the semantic segmentation network. In the decoding process, each time before entering the next layer, an Upsampling operation needs to be performed first. The channel is gradually compressed from 512 in the first layer to 64, and reaches the reconstruction network to obtain the first synthesized high-energy image.

Step 42: Input the first high-energy image into the discriminator network of the preset generative adversarial network model to obtain a first discrimination result.

In the embodiment of the present application, in order to determine whether the synthesized first high-energy image is similar to the standard high-energy image, after the first high-energy image is obtained, the first high-energy image may be input into the discriminator network of the preset generative adversarial network model to obtain The first judgment result.

Wherein, if the first discrimination result indicates that the first high-energy image is similar to the standard high-energy image, then it can be considered that the generator network of the preset generative adversarial network model has converged, that is, the first high-energy image synthesized based on the generator network has reached the standard Criteria for high-energy images that stop the training of the generator network.

Or, if the first discrimination result indicates that the first high-energy image is not similar to the standard high-energy image, then it can be considered that the generator network of the preset generative adversarial network model does not converge, that is, the first high-energy image synthesized based on the generator network The standard high-energy images cannot be reached for the time being, and further training of the generator network is still required.

The above two situations are only an exemplary description of the embodiments of the present application, and do not impose any limitations on the embodiments of the present application. For example, in an optional embodiment, even if the first discrimination result can represent that the first high-energy image is similar to the standard high-energy image, in order to avoid the situation that the discrimination result is inaccurate due to the low precision of the discriminator network, this The embodiment of the application may further train the generator network and the discriminator network of the preset generative adversarial network model based on the first discrimination result. For detailed steps, please refer to the following steps 43 to 44 .

Step 43: Based on the first high-energy image and the standard high-energy image, a first loss value is calculated according to a preset loss function, and the first loss value is used to update the parameters of the preset generative adversarial network model until the preset generative adversarial network converges.

The preset loss function is established at least according to the loss function for reducing image noise and removing image artifacts. For example, considering that the embodiments of the present application are intended to solve the problems of noise, artifact interference, and poor image quality when a CT image is obtained by scanning a dual-energy CT method in the prior art, in practical applications, it is possible to to enhance the gradient information of the image, especially the edge contour of the image, thereby reducing the influence of noise and artifacts on the image edge. The loss function can be a gradient loss function.

Among them, the gradient loss function used to reduce image noise and remove image artifacts can be based on the gradient of the standard high-energy image in the x direction, the gradient of the standard high-energy image in the y direction, the gradient of the synthetic high-energy image in the x direction, and the synthetic high-energy image. The gradient in the y direction is established. For example, as shown in the following formula (2):

Among them, L _gdl (G(x), Y) represents the gradient loss function; G(x) represents the synthetic high-energy image; Y represents the standard high-energy image;

Represents the gradient of the standard high-energy image in the x direction;

Represents the gradient of the standard high-energy image in the y direction;

Represents the gradient of the synthesized high-energy image in the x direction;

Represents the gradient of the synthesized high-energy image in the y-direction.

In the embodiment of the present application, it is assumed that the preset loss function is the gradient loss function shown in the above formula (2) for reducing image noise and removing image artifacts as an example, then based on the first high-energy image and the standard high-energy image, The first loss value is calculated according to the preset loss function, and the first loss value is used to update the parameters of the preset generative adversarial network model until the preset generative adversarial network converges, which may be as follows.

According to the first high-energy image and the standard high-energy image, the gradient loss function shown in formula (2) is used to calculate the gradient difference between the first high-energy image and the standard high-energy image, and the calculated gradient difference is determined as the first loss value.

Step 44, update the preset generative adversarial network model based on the first loss value and the first discrimination result until the preset generative adversarial network model converges, and determine the converged preset generative adversarial network model as the Wasserstein generative adversarial network model.

After obtaining the first loss value of the first high-energy image and the standard high-energy image according to step 43, the Adam optimizer can be used to optimize the preset generative adversarial network model based on the first loss value and the first discrimination result, and when the preset loss When the curve of the function converges to the preset range, the converged preset generative adversarial network model is determined as the Wasserstein generative adversarial network model.

After the Wasserstein generative adversarial network model is obtained through the above steps, the low-energy image to be synthesized can be input into the pre-trained Wasserstein generative adversarial network model to obtain the synthesized target high-energy image.

With the method provided in this embodiment of the present application, since the Wasserstein generative adversarial network model is based on low-energy image samples, standard high-energy images and a preset loss function, it is obtained by training a preset generative adversarial network model, and the preset loss function is at least based on the use of The loss function is established to reduce image noise and remove image artifacts. Therefore, by inputting the low-energy image to be synthesized into the Wasserstein generative adversarial network model obtained by pre-training, and synthesizing the target high-energy image, image noise and image noise can be reduced. The effect of artifacts on the edges of the image, thereby improving the quality of the synthetic target high-energy image.

Example 2

Due to the same application concept as the above-mentioned method, the embodiment of the present application is also based on the high-energy image synthesis method of the Wasserstein generative adversarial network model, to solve the problem of large interference deviation when the CT image is obtained by scanning the dual-energy CT method in the prior art, The problem of poor image quality.

The method is described in detail below.

In practical applications, considering that when training a pixel-level generator network, the offset between paired pixels usually occurs, causing errors in details and reducing the quality of synthesized high-energy images. Therefore, in the synthesis based on low-energy images to be synthesized When using high-energy images, in order to ensure the quality of the synthesized high-energy images, in addition to reducing the noise and artifacts of the synthesized high-energy images, it is also necessary to calibrate the pixel error between the synthesized high-energy image and the standard high-energy image. Based on this, the preset loss function in step 43 in Embodiment 1 of the present application may further include a preset pixel difference calibration function for calibrating the pixel difference between the synthesized high-energy image and the standard high-energy image.

The preset pixel difference calibration function for calibrating the pixel difference between the synthesized high-energy image and the standard high-energy image, for example, may be shown in the following formula (3).

Among them, L _MSE (G(x), Y) represents the preset pixel difference calibration function, G(x) represents the synthesized first high-energy image, Y represents the standard high-energy image; w and h represent the sampling width and height, respectively, ( i, j) represent the pixel points of the image.

Optionally, while calibrating the pixel error, it is also necessary to ensure the image brightness, contrast and structural information of the synthesized high-energy image. Therefore, the preset loss function may also include a structure used to calibrate the synthesized high-energy image and the standard high-energy image. The preset structural loss function of the sexual information difference, for example, is shown in the following formula (4).

L _SSIM (G(x),Y)=-log(max(0,SSIM(G(x),Y))) (4)

Among them, L _SSIM (G(x),Y) represents the preset structural loss function, G(x) represents the synthesized first high-energy image, Y represents the standard high-energy image; SSIM(G(x),Y) represents the structural The similarity function is calculated as shown in the following formula (5).

where μ and σ represent the mean and standard deviation of the image, respectively, and C ₁ =(k ₁ L) ² and C ₂ =(k ₂ L) ² are two smaller constant terms to avoid a denominator of 0.

Optionally, when generating the edge information of the first high-energy image, in order to ensure that the local pattern and texture information of the image can be effectively extracted without being constrained by specific pixels, in this embodiment of the present application, the preset loss function is also A preset multi-scale feature loss function for calibrating the difference in texture information between the synthesized high-energy image and the standard high-energy image may be included. After adding the preset loss function, high-frequency information of the image can be effectively extracted.

Wherein, the preset multi-scale feature loss function used for calibrating the difference in texture information between the synthesized high-energy image and the standard high-energy image may be, for example, as shown in the following formula (6).

Among them, L _content (G(x), Y) represents the preset multi-scale feature loss function; G(x) represents the first synthesized high-energy image, Y represents the standard high-energy image; conv represents the multi-scale convolution kernel, m represents The number of multi-scale convolution kernels, size is the size of the sampled image, β _m is the weight of each scale, for example, it can be set to 0.3, 0.2 and 0.3.

To sum up, the preset loss function in this embodiment of the present application may also be established according to at least one of the following loss functions.

a preset pixel disparity calibration function for calibrating the pixel disparity between the synthesized high-energy image and the standard high-energy image;

A preset structural loss function for calibrating the structural information difference between the synthesized high-energy image and the standard high-energy image;

Preset multi-scale feature loss functions for calibrating the difference in texture information between synthetic high-energy images and standard high-energy images.

In the following, the preset loss function is established based on the preset gradient loss function, the preset pixel difference calibration function, the preset structural loss function, the preset multi-scale feature loss function, and the preset generative adversarial network model as an example. Steps 43 and 44 in Example 1 are described:

The preset function may be shown in the following formula (7).

in,

represents the preset generative adversarial network model, G represents the generator network of the preset generative adversarial network model, D represents the discriminator network of the preset generative adversarial network model; L _MSE (G(x), Y) represents the preset pixel difference calibration function; L _SSIM (G(x), Y) represents the preset structural loss function; L _content (G(x), Y) represents the preset multi-scale feature loss function; L _gdl (G(x), Y) represents Gradient loss function; λ _adv , λ _mse , λ _ssim , λ _content , and λ _gdl respectively represent the weight of each loss function. For example, in an optional implementation, the weight of each loss function can be set as a hyperparameter.

It should be noted that, for the specific calculation method of each loss function, reference may be made to the above-mentioned related content, which is not described here in order to avoid repetition.

According to the above content, after the preset loss function is determined, the first loss value can be calculated according to the preset loss function based on the first high-energy image and the standard high-energy image, and the first loss value is used to update the parameters of the preset generative adversarial network model. , until the preset generative adversarial network converges.

In one embodiment, first, a pixel difference value between the first high-energy image and the standard high-energy image can be calculated by using a preset pixel difference calibration function, and the pixel difference value is determined as the first first loss value.

Secondly, by presetting the structural loss function, the structural difference value between the first high-energy image and the standard high-energy image is determined, and the structural difference value is determined as the second first loss value.

Then, by presetting a multi-scale feature loss function, the difference value of texture information between the first high-energy image and the standard high-energy image is determined, and the difference value of texture information is determined as the third first loss value.

Finally, the gradient difference between the first high-energy image and the standard high-energy image is determined through the gradient loss function for reducing image noise and removing image artifacts, and the gradient difference is determined as the fourth first loss value.

After the above four first loss values are obtained, the weighted summation can be performed based on the above four loss values to determine the final first loss value, and then the preset generation confrontation can be updated based on the final first loss value and the first discrimination result network model until the preset generative adversarial network model converges, and the converged preset generative adversarial network model is determined as the Wasserstein generative adversarial network model.

With the method provided in this embodiment of the present application, since the Wasserstein generative adversarial network model is based on low-energy image samples, standard high-energy images and a preset loss function, it is obtained by training a preset generative adversarial network model, and the preset loss function is at least based on the use of A loss function for reducing image noise and removing image artifacts, a preset pixel disparity calibration function for calibrating the pixel disparity between the synthesized high-energy image and the standard high-energy image, and a calibration function for calibrating the difference between the synthesized high-energy image and the standard high-energy image. A preset structural loss function for the structural information difference between the The input to the pre-trained Wasserstein generative adversarial network model, the method of synthesizing the target high-energy image, on the one hand, can reduce the influence of image noise and image artifacts on the edge of the image, thereby improving the quality of the synthesized target high-energy image. On the other hand, the pixel difference between the synthesized high-energy image and the standard high-energy image can also be calibrated to avoid differences in the details of the synthesized high-energy image; the structural information difference between the synthesized high-energy image and the standard high-energy image can be calibrated to ensure the synthesis The structural information, image brightness and contrast, etc. of the high-energy image; and, the difference in texture information between the synthesized high-energy image and the standard high-energy image can also be calibrated to ensure that the local pattern and texture information of the image can be effectively extracted.

Example 3

The following describes how the methods provided by the embodiments of the present application are applied in practice in combination with actual scenarios.

Please refer to FIG. 5 , which is a schematic diagram of an actual application process of the method provided by the embodiment of the present application. The process includes the following steps.

First, you can slice the low-energy image (LECT in the figure) to get a slice of size 256*256 (Patch in the figure), and then input the obtained slice into the generator network of the Wasserstein generative adversarial network model (Generator in the figure). ) to obtain a synthesized high-energy image (sHECT in the figure).

On the other hand, a standard high-energy image (HECT in the figure) is obtained, the standard high-energy image is sliced, and then the discriminator network (Discriminator in the figure) of the Wasserstein generative adversarial network model is trained based on the standard high-energy image, and based on the training The latter discriminator network discriminates the synthesized high-energy images.

After the synthesized high-energy image is obtained, the gradient difference between the synthesized high-energy image and the standard high-energy image can be calculated based on the gradient flow (Gradient Flow in the figure), and then reversed based on the preset gradient loss function (Gradient Difference in the figure) Update the parameters of the generator network.

In the embodiment of the present application, considering that when training a pixel-level generator network, the offset between paired pixels usually occurs, causing errors in details, thereby reducing the quality of synthesized high-energy images. Therefore, based on the preset gradient loss When the function reversely updates the parameters of the generator network, the preset gradient loss function can also include (MES in the figure), and in order to ensure the image brightness, contrast and structural information of the synthesized high-energy image, the preset gradient loss function can also be Including (SSIM in the figure), and ensuring that the local pattern and texture information of the image can be effectively extracted, the preset gradient loss function can also include (Content in the figure).

Example 4

The above provides a high-energy image synthesis method based on the Wasserstein generative adversarial network model provided by the embodiment of the present application. Based on the same idea, the embodiment of the present application also provides a high-energy image synthesis device based on the Wasserstein generative confrontation network model, as shown in FIG. 6 . shown.

The device 60 includes: an acquisition module 61 and an input module 62, wherein: the acquisition module 61 is used to acquire the low-energy image to be synthesized; the input module 62 is used to input the low-energy image to be synthesized into the Wasserstein generative confrontation obtained by pre-training network model to obtain the synthesized target high-energy image; among them, the Wasserstein generative adversarial network model is trained by the preset generative adversarial network model learning method; the Wasserstein generative adversarial network model is based on low-energy image samples, standard high-energy images and preset loss functions, Obtained through the training of a preset generative adversarial network model, the Wasserstein generative adversarial network model includes a generator network and a discriminator network. The generator network is used to extract the image features of the low-energy images to be synthesized, and synthesize high-energy images based on the image features; the discriminator network. It is used to judge the high-energy images synthesized by the generator network, and perform reverse adjustment training; the preset loss function is established at least according to the loss function used to reduce image noise and remove image artifacts.

Optionally, the preset loss function may also be established according to at least one of the following loss functions: a preset pixel difference calibration function for calibrating the pixel difference between the synthesized high-energy image and the standard high-energy image; a preset pixel difference calibration function for calibrating the synthesized high-energy image A preset structural loss function for the structural information difference between the high-energy image and the standard high-energy image; a preset multi-scale feature loss function for calibrating the texture information difference between the synthesized high-energy image and the standard high-energy image.

With the device provided in this embodiment of the present application, since the Wasserstein generative adversarial network model is based on low-energy image samples, standard high-energy images, and a preset loss function, it is obtained by training a preset generative adversarial network model, and the preset loss function is at least based on the use of The loss function is established to reduce image noise and remove image artifacts. Therefore, the low-energy image to be synthesized is input into the pre-trained Wasserstein generative adversarial network model through the input module to obtain the synthesized target high-energy image, which can reduce image noise. and image artifacts on the edge of the image, thereby improving the quality of the synthetic target high-energy image.

Example 5

7 is a schematic diagram of the hardware structure of an electronic device implementing various embodiments of the present application. The electronic device 700 includes but is not limited to: a radio frequency unit 701, a network module 702, an audio output unit 703, an input unit 704, a sensor 705, and a display unit 706 , the user input unit 707 , the interface unit 708 , the memory 709 , the processor 710 , and the power supply 711 and other components. Those skilled in the art can understand that the structure of the electronic device shown in FIG. 7 does not constitute a limitation on the electronic device, and the electronic device may include more or less components than the one shown, or combine some components, or different components layout. In the embodiments of the present application, electronic devices include, but are not limited to, mobile phones, tablet computers, notebook computers, handheld computers, vehicle-mounted terminals, wearable devices, and pedometers.

The processor 710 is used to obtain the low-energy image to be synthesized; input the low-energy image to be synthesized into the Wasserstein generative adversarial network model obtained by pre-training to obtain the synthesized target high-energy image; the Wasserstein generative adversarial network model is based on low-energy image samples , a standard high-energy image and a preset loss function, which are obtained by training a preset generative adversarial network model. The Wasserstein generative adversarial network model includes a generator network and a discriminator network. The generator network is used to extract the image features of the low-energy images to be synthesized, and High-energy images are synthesized based on image features; the discriminator network is used to judge the high-energy images synthesized by the generator network and perform reverse adjustment training; preset loss functions, at least according to the loss used to reduce image noise and remove image artifacts function creation.

Optionally, the preset loss function can also be established according to at least one of the following loss functions: a preset pixel difference calibration function for calibrating the pixel difference between the synthesized high-energy image and the standard high-energy image; a preset pixel difference calibration function for calibrating the synthesized high-energy image A preset structural loss function for the structural information difference between the high-energy image and the standard high-energy image; a preset multi-scale feature loss function for calibrating the texture information difference between the synthesized high-energy image and the standard high-energy image.

Optionally, if the preset loss function includes a preset pixel difference calibration function, then based on the first high-energy image and the standard high-energy image, the first loss value is calculated and obtained according to the preset loss function, including: using the preset pixel difference calibration function, Calculate the pixel difference value between the first high-energy image and the standard high-energy image; determine the pixel difference value as the first loss value.

The memory 709 is used to store a computer program that can be executed on the processor 710. When the computer program is executed by the processor 710, the above-mentioned functions implemented by the processor 710 are implemented.

It should be understood that, in this embodiment of the present application, the radio frequency unit 701 can be used for receiving and sending signals during sending and receiving of information or during a call. Specifically, after receiving the downlink data from the base station, it is processed by the processor 710; The uplink data is sent to the base station. Generally, the radio frequency unit 701 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 701 can also communicate with the network and other devices through a wireless communication system.

The electronic device provides the user with wireless broadband Internet access through the network module 702, such as helping the user to send and receive emails, browse web pages, access streaming media, and the like.

The audio output unit 703 may convert audio data received by the radio frequency unit 701 or the network module 702 or stored in the memory 709 into audio signals and output as sound. Also, the audio output unit 703 may also provide audio output related to a specific function performed by the electronic device 700 (eg, call signal reception sound, message reception sound, etc.). The audio output unit 703 includes a speaker, a buzzer, a receiver, and the like.

The input unit 704 is used to receive audio or video signals. The input unit 704 may include a graphics processor (Graphics Processing Unit, GPU) 7041 and a microphone 7042, and the graphics processor 7041 is used for still pictures or video images obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode data is processed. The processed image frames may be displayed on the display unit 706 . The image frames processed by the graphics processor 7041 may be stored in the memory 709 (or other storage medium) or transmitted via the radio frequency unit 701 or the network module 702 . The microphone 7042 can receive sound and can process such sound into audio data. The processed audio data can be converted into a format that can be transmitted to a mobile communication base station via the radio frequency unit 701 for output in the case of a telephone call mode.

The electronic device 700 also includes at least one sensor 705, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 7061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 7061 and the display panel 7061 when the electronic device 700 is moved to the ear. / or backlight. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (usually three axes), and can detect the magnitude and direction of gravity when stationary, and can be used to identify the posture of electronic devices (such as horizontal and vertical screen switching, related games , magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tapping), etc.; the sensor 705 may also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, Infrared sensors, etc., are not repeated here.

The display unit 706 is used to display information input by the user or information provided to the user. The display unit 706 may include a display panel 7061, and the display panel 7061 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like.

The user input unit 707 may be used to receive input numerical or character information, and generate key signal input related to user settings and function control of the electronic device. Specifically, the user input unit 707 includes a touch panel 7071 and other input devices 7072 . The touch panel 7071, also referred to as a touch screen, can collect touch operations by the user on or near it (such as the user's finger, stylus, etc., any suitable object or attachment on or near the touch panel 7071). operate). The touch panel 7071 may include two parts, a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the touch controller. To the processor 710, the command sent by the processor 710 is received and executed. In addition, the touch panel 7071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 7071 , the user input unit 707 may also include other input devices 7072 . Specifically, other input devices 7072 may include, but are not limited to, physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which will not be repeated here.

Further, the touch panel 7071 can be covered on the display panel 7071. When the touch panel 7071 detects a touch operation on or near it, it transmits it to the processor 710 to determine the type of the touch event, and then the processor 710 determines the type of the touch event according to the touch The type of event provides a corresponding visual output on display panel 7061. Although in FIG. 7 , the touch panel 7071 and the display panel 7061 are used as two independent components to realize the input and output functions of the electronic device, but in some embodiments, the touch panel 7071 and the display panel 7061 may be integrated The implementation of the input and output functions of the electronic device is not specifically limited here.

The interface unit 708 is an interface for connecting an external device to the electronic device 700 . For example, external devices may include wired or wireless headset ports, external power (or battery charger) ports, wired or wireless data ports, memory card ports, ports for connecting devices with identification modules, audio input/output (I/O) ports, video I/O ports, headphone ports, and more. The interface unit 708 may be used to receive input from external devices (eg, data information, power, etc.) and transmit the received input to one or more elements within the electronic device 700 or may be used between the electronic device 700 and the external Transfer data between devices.

The memory 709 may be used to store software programs as well as various data. The memory 709 may mainly include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; Data created by the use of the mobile phone (such as audio data, phone book, etc.), etc. Additionally, memory 709 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The processor 710 is the control center of the electronic device, using various interfaces and lines to connect various parts of the entire electronic device, by running or executing the software programs and/or modules stored in the memory 709, and calling the data stored in the memory 709. , perform various functions of electronic equipment and process data, so as to monitor electronic equipment as a whole. The processor 710 may include one or more processing units; optionally, the processor 710 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem The modulation processor mainly handles wireless communication. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into the processor 710.

The electronic device 700 may also include a power supply 711 (such as a battery) for supplying power to various components. Optionally, the power supply 711 may be logically connected to the processor 710 through a power management system, so as to manage charging, discharging, and power consumption through the power management system management and other functions.

In addition, the electronic device 700 includes some functional modules not shown, which will not be repeated here.

Embodiments of the present application further provide an electronic device, including a processor 710, a memory 709, and a computer program stored in the memory 709 and running on the processor 710. When the computer program is executed by the processor 710, the above-mentioned based Each process of the embodiment of the high-energy image synthesis method of the Wasserstein generative adversarial network model can achieve the same technical effect. To avoid repetition, it will not be repeated here.

Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, implements each of the above-mentioned embodiments of the high-energy image synthesis method based on the Wasserstein generative adversarial network model process, and can achieve the same technical effect, in order to avoid repetition, it will not be repeated here. Wherein, the computer-readable storage medium, such as read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), magnetic disk or optical disk and so on.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flows of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Memory may include forms of non-persistent memory, random access memory (RAM) and/or non-volatile memory in computer readable media, such as read only memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media includes both persistent and non-permanent, removable and non-removable media, and storage of information may be implemented by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cartridges, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

It should also be noted that the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device comprising a series of elements includes not only those elements, but also Other elements not expressly listed, or which are inherent to such a process, method, article of manufacture, or apparatus are also included. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article of manufacture, or device that includes the element.

The above descriptions are merely examples of the present application, and are not intended to limit the present application. Various modifications and variations of this application are possible for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included within the scope of the claims of this application.

Claims

A high-energy image synthesis method based on the Wasserstein generative adversarial network model, characterized in that it includes:

Obtain the low-energy image to be synthesized;

Inputting the low-energy image to be synthesized into the Wasserstein generative adversarial network model obtained by pre-training to obtain a synthesized target high-energy image;

The Wasserstein generative adversarial network model is obtained by training a preset generative adversarial network model based on low-energy image samples, standard high-energy images and a preset loss function. The Wasserstein generative adversarial network model includes a generator network and a discriminator network. The generator network is used to extract the image features of the low-energy images to be synthesized, and synthesize high-energy images based on the image features; the discriminator network is used to judge the high-energy images synthesized by the generator network and perform reverse adjustment training. ;

The preset loss function is established at least according to a loss function for reducing image noise and removing image artifacts.
The method according to claim 1, wherein the loss function for reducing image noise and removing image artifacts is based on the gradient of the standard high-energy image in the x direction, the gradient of the standard high-energy image in the y direction, the composite The gradient of the high-energy image in the x-direction and the gradient of the synthesized high-energy image in the y-direction are established.
The method of claim 1, wherein the preset loss function is further established according to at least one of the following loss functions:

a preset pixel disparity calibration function for calibrating the pixel disparity between the synthesized high-energy image and the standard high-energy image;

a preset structural loss function for calibrating the structural information difference between the synthesized high-energy image and the standard high-energy image;

A preset multi-scale feature loss function for calibrating the difference in texture information between the synthesized high-energy image and the standard high-energy image.
The method of claim 3, wherein the preset loss function is based on the preset gradient loss function, the preset pixel difference calibration function, the preset structural loss function, the preset A multi-scale feature loss function and the preset generative adversarial network model are established.
The method according to claim 1, characterized in that, before inputting the low-energy image to be synthesized into the Wasserstein generative adversarial network model obtained by pre-training to obtain the synthesized target high-energy image, the method further comprises: based on low-energy image samples With standard high-energy images and preset loss functions, the Wasserstein generative adversarial network model is obtained through preset generative adversarial network model training;

Wherein, based on low-energy image samples, standard high-energy images and a preset loss function, the Wasserstein generative adversarial network model is obtained by training a preset generative adversarial network model, including:

Inputting the low-energy image sample into a generator network of a preset generative adversarial network model to obtain a synthesized first high-energy image;

Inputting the first high-energy image to the discriminator network of the preset generative adversarial network model to obtain a first discrimination result;

Based on the first high-energy image and the standard high-energy image, a first loss value is calculated according to the preset loss function, and the first loss value is used to update the parameters of the preset generative adversarial network model until all The preset generative adversarial network converges;

The preset generative adversarial network model is updated based on the first loss value and the first discrimination result until the preset generative adversarial network model converges, and the converged preset generative adversarial network model is determined as the Wasserstein generative adversarial network model.
The method according to claim 5, wherein if the preset loss function includes a preset pixel difference calibration function, based on the first high-energy image and the standard high-energy image, according to the preset loss function Calculate the first loss value, including:

Calculate the pixel difference value between the first high-energy image and the standard high-energy image by using the preset pixel difference calibration function;

The pixel difference value is determined as the first loss value.
The method according to claim 5, wherein, if the preset loss function includes a preset structural loss function, based on the first high-energy image and the standard high-energy image, according to the preset loss function Calculate the first loss value, including:

determining a structural difference value between the first high-energy image and the standard high-energy image by using the preset structural loss function;

A structural difference value is determined as the first loss value.
The method according to claim 5, wherein if the preset loss function includes a preset multi-scale feature loss function, based on the first high-energy image and the standard high-energy image, according to the preset loss The function calculates the first loss value, including:

determining the texture information difference value between the first high-energy image and the standard high-energy image by using the preset multi-scale feature loss function;

The texture information difference value is determined as the first loss value.
The method of claim 1, wherein the generator network of the Wasserstein generative adversarial network model comprises a semantic segmentation network with four layers of encoding and decoding, and each layer of encoding and decoding is connected by a skip link, and the semantic segmentation A residual network of 9 layers is included between the encoding layer and the decoding layer of the network;

The discriminator network of the Wasserstein generative adversarial network model includes 8 groups of 3*3 convolutional layers and activation function LReLU; wherein, the convolutional layer and activation function LReLU of the convolutional layer located in the singular position from left to right have a convolution step size of 1. The convolutional layer at the even position and the convolutional stride of the activation function LReLU are 2.
A high-energy image synthesis device based on the Wasserstein generative adversarial network model, characterized in that it includes an acquisition module and an input module, wherein:

an acquisition module for acquiring the low-energy image to be synthesized;

The input module is used to input the low-energy image to be synthesized into the Wasserstein generative adversarial network model obtained by pre-training, and obtain the synthesized target high-energy image; wherein, the Wasserstein generative adversarial network model passes through the preset generative adversarial network model. Learning methods are trained;

The Wasserstein generative adversarial network model is obtained by training a preset generative adversarial network model based on low-energy image samples, standard high-energy images and a preset loss function. The Wasserstein generative adversarial network model includes a generator network and a discriminator network. The generator network is used to extract the image features of the low-energy images to be synthesized, and synthesize high-energy images based on the image features; the discriminator network is used to judge the high-energy images synthesized by the generator network and perform reverse adjustment training. ;

The preset loss function is established at least according to a loss function for reducing image noise and removing image artifacts.
An electronic device, characterized in that it comprises: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being executed by the processor to achieve the invention as claimed in the claims The steps of the high-energy image synthesis method based on the Wasserstein generative adversarial network model described in any one of 1 to 9.
A computer-readable storage medium, characterized in that, a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the Wasserstein-based method according to any one of claims 1 to 9 is realized. Steps of a high-energy image synthesis method for generative adversarial network models.