Disclosure of Invention
In order to solve the technical problems, the invention discloses a CBCT image enhancement method, a device and a storage medium based on artificial intelligence, the CBCT image enhanced by the method has the advantages of smaller noise, high contrast and more accurate CT value, the positioning precision and the positioning speed of a clinician can be effectively improved, the CBCT image is limited by the poorer accuracy of the CT value, the self-adaptive plan cannot be smoothly implemented, the CBCT image processed by the method is more beneficial to the manufacture of the self-adaptive plan, and the radiotherapy effect is further improved.
Specifically, the invention provides a CBCT image enhancement method based on artificial intelligence, which comprises the following steps: s1: preparing paired CBCT and CT training image data; s2: training a learning network by using the prepared pair of CBCT and CT training image data to obtain a network model for converting the CBCT image into the CT image; s3: and inputting the CBCT image into the network model to obtain an enhanced CBCT image.
In a further aspect, step S1 includes: s11: registering the CBCT image and the CT image by using a registration frame; s12: and carrying out pixel normalization and gray value normalization processing on the matched CBCT image and CT image.
In a further aspect, step S11 includes: s111: carrying out rigid registration on the CBCT image and the CT image so as to correct the global large-scale offset; s112: after rigid registration, elastic registration is further performed on the CBCT image and the CT image so as to ensure that the tissues in the body in the two groups of images correspond one to one.
In a further technical solution, in step S2, the depth learning is performed based on a pix2pix-gans model, where the pix2pix-gans model includes a generator and a discriminator, the generator generates a pseudo CT image based on an input CBCT image, and the discriminator is used for distinguishing the CBCT image from the pseudo CT image.
In a further technical scheme, the generator adopts a U-Net structure, and the arbiter adopts a PatchGAN structure.
In a further technical scheme, the U-Net structure adopts a structure that an input image is firstly down-sampled to a low dimension and then up-sampled to an original resolution, a skip-connection is added, and corresponding feature maps and feature maps with the same size after decode are spliced together according to a channel and used for reserving detail information of pixel levels under different resolutions.
In a further technical scheme, the PatchGAN structure uses GAN to construct high-frequency information, and performs true and false judgment on each patch with the size of N x N of an image, and averages the results of all the patches of one image to output as a final discriminator.
In a further technical solution, in step S12, the pixel size is unified to 1mm, the gray scale value is limited to (-1024,2048), if the pixel gray scale value is less than-1024, it is set to-1024, if the pixel gray scale value is greater than 2048, it is set to 2048, and then the gray scale value is normalized to (-1, 1).
The invention also provides a CBCT image enhancement device based on artificial intelligence, which comprises: the CBCT and CT training image data preparation module is used for preparing paired CBCT and CT training image data; the network model training module is used for training a learning network by using prepared paired CBCT and CT training image data to obtain a network model for converting the CBCT image into a CT image; and the enhanced CBCT image output module is used for inputting the CBCT image into the network model so as to obtain an enhanced CBCT image.
The invention additionally provides a computer storage medium storing a computer program which, when executed, performs the method as described above.
The algorithm in the invention can remove the noise in the image under the condition of completely keeping the original edge, and the denoising effect is better than that of the prior art.
Detailed Description
The technical solution of the present invention will be further described with reference to the following specific examples, but the present invention is not limited to these examples.
In the invention, a deep learning model is designed, the CBCT image can be converted into the CT image by using the deep learning model, after the conversion, the quality of the CBCT image is close to that of the CT image, the noise and the artifact in the image obviously disappear, and the image contrast is also enhanced.
As shown in FIG. 1, the CBCT image enhancement method based on artificial intelligence of the invention can comprise the following steps:
1. preparing paired data of CBCT and CT, acquiring CT images through a CT analog positioning machine, wherein the acquisition time of the CBCT is close to that of the CT, matching the overall contour of the CT and the CBCT by using rigid registration and then ensuring that the in vivo tissues in the two groups of images correspond one to one by using elastic registration in order to ensure that the spatial information of the CBCT and the CT images is consistent.
2. And carrying out pixel size normalization and gray value normalization on the well-matched CBCT and CT images.
3. Training a deep learning network by using the prepared CBCT and CT images to obtain a network model for converting the CBCT into the CT images;
4. and when the CBCT is reconstructed again subsequently, inputting the reconstructed CBCT image into the network model to obtain an enhanced image.
Because CBCT and CT data of a patient are acquired at different times, and therefore, certain offset and deformation exist, before a model is trained, a registration frame is required to be used for registering CBCT and CT images, and it is ensured that spatial information of the CBCT and CT images is consistent, an elastic registration frame mainly comprises two parts, namely rigid registration and elastic registration, wherein the rigid registration is mainly used for correcting global large-scale offset, and the elastic registration is mainly used for restoring local deformation such as soft tissue. The overall registration flow chart is shown in fig. 2.
The specific registration effect is shown in fig. 3A-3B, where fig. 3A is a CBCT and CT image before registration, a human body part in the image has obvious dislocation, fig. 3B is a result after rigid registration, the main body contour of the CBCT and CT is corrected, but the skin edge and the tissue in the human body have certain deformation, fig. 3C is a result after elastic registration, the local deformation of the human body has disappeared, and the CBCT and CT image after registration have one-to-one correspondence in pixels, so that they can be used for the subsequent model training.
In addition, for training the model, 200 sets of corresponding data of the CBCT and the CT may be prepared, the pixel size is unified to 1mm, the image layer thickness is also 1mm, the gray values of the CBCT and the CT are limited to (-1024,2048), the gray value of the pixel is set to-1024 when the gray value of the pixel is smaller than-1024, the gray value of the pixel is set to 2048 when the gray value of the pixel is larger than 2048, and then the gray value is normalized to (-1, 1).
The deep learning model designed by the invention is based on a pix2pix-gans model, the model consists of a generator G and a discriminator D, a CBCT picture needs to be input into the generator, and the generator generates a false CT image (fake CT). The input of the discriminator is CBCT and fake CT images, and the discriminator needs to distinguish the CBCT from the fake CT.
As shown in fig. 4, pix2pix-GANs uses a generator to solve the low frequency component (texture information) of an image and GAN (generative confrontation network) to solve the high frequency component (image edge information). The generator uses the traditional L1 loss or MSE loss to make the generated picture as similar as possible to the training picture, and GAN is used to construct the details of the high frequency part.
As shown in fig. 5, a U-Net network structure is used as a generator, the U-Net adopts an (Encoder-Decoder) structure that down-samples an input image to a low dimension, then up-samples the input image to an original resolution, and adds a skip-connection, and the corresponding feature maps and the feature maps with the same size after the decode are spliced together according to a channel (context) to retain the detail information of the pixel level at different resolutions. The effect of U-Net on detail improvement is very obvious.
On the other hand, whether or not the picture is generated is determined using PatchGAN as a discriminator. The idea of PatchGAN is that using GAN to construct high frequency information does not require the entire picture to be input into a discriminator, which can make a true or false discrimination for each patch of size N x N of the image. As different patches may be considered independent of each other. The pix2pix cuts a picture into different patch sizes of N x N, the discriminator judges whether each patch is true or false, and the results of all the patches of the picture are averaged to be output as a final discriminator.
PatchGAN uses a full convolution small network with NxN input, the probability that each pixel in the last layer outputs true after sigmoid is output, and then BCEloss is used for calculating final loss. This has the advantage that because the input dimension is greatly reduced, the parameters are reduced, the operation speed is faster than that of directly inputting, and the graph with any size can be calculated.
In an embodiment of the present invention, there is provided an artificial intelligence based CBCT image enhancement method, including the steps of: s1: preparing paired CBCT and CT training image data; s2: training a learning network by using the prepared pair of CBCT and CT training image data to obtain a network model for converting the CBCT image into the CT image; s3: and inputting the CBCT image into the network model to obtain an enhanced CBCT image.
In a further aspect, step S1 includes: s11: registering the CBCT image and the CT image by using a registration frame; s12: and carrying out pixel normalization and gray value normalization processing on the matched CBCT image and CT image.
In a further aspect, step S11 includes: s111: carrying out rigid registration on the CBCT image and the CT image so as to correct the global large-scale offset; s112: after rigid registration, elastic registration is further performed on the CBCT image and the CT image so as to ensure that the tissues in the body in the two groups of images correspond one to one.
In a further technical solution, in step S2, the depth learning is performed based on a pix2pix-gans model, where the pix2pix-gans model includes a generator and a discriminator, the generator generates a pseudo CT image based on an input CBCT image, and the discriminator is used for distinguishing the CBCT image from the pseudo CT image.
In a further technical scheme, the generator adopts a U-Net structure, and the arbiter adopts a PatchGAN structure.
In a further technical scheme, the U-Net structure adopts a structure that an input image is firstly down-sampled to a low dimension and then up-sampled to an original resolution, a skip-connection is added, and corresponding feature maps and feature maps with the same size after decode are spliced together according to a channel and used for reserving detail information of pixel levels under different resolutions.
In a further technical scheme, the PatchGAN structure uses GAN to construct high-frequency information, and performs true and false judgment on each patch with the size of N x N of an image, and averages the results of all the patches of one image to output as a final discriminator.
In a further technical solution, in step S12, the pixel size is unified to 1mm, the gray scale value is limited to (-1024,2048), if the pixel gray scale value is less than-1024, it is set to-1024, if the pixel gray scale value is greater than 2048, it is set to 2048, and then the gray scale value is normalized to (-1, 1).
The method of the invention can not lose the original information in the image, not only removes the noise and the artifact in the image, but also retains the edge information of the image, enhances the contrast of the image, moreover, in the process from the CBCT to the CT image training model, the model retains part of the spatial information in the CT image, and when the CBCT image is enhanced by using the model, the prior knowledge in the CT can be retained in the CBCT image.
In another embodiment of the present invention, there is also provided an artificial intelligence based CBCT image enhancement apparatus, including: the CBCT and CT training image data preparation module is used for preparing paired CBCT and CT training image data; the network model training module is used for training a learning network by using prepared paired CBCT and CT training image data to obtain a network model for converting the CBCT image into a CT image; and the enhanced CBCT image output module is used for inputting the CBCT image into the network model so as to obtain an enhanced CBCT image.
In other embodiments of the present invention, there is also provided a computer storage medium storing a computer program which, when executed, performs the method as described above.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various changes and modifications can be made without departing from the inventive concept of the present invention, and these changes and modifications are all within the scope of the present invention.