Disclosure of Invention
Aiming at the defects and the improvement requirements of the prior art, the invention provides a low-dose cone-beam CT image reconstruction method based on deep learning, and aims to improve the quality of a reconstructed cone-beam CT image while reducing the X-ray dose of cone-beam CT.
To achieve the above object, according to one aspect of the present invention, there is provided a low dose cone-beam CT image reconstruction method based on deep learning, including:
a projection transformation step: transforming the low-dose cone-beam CT original projection data into a plurality of projection images;
a projected image denoising step: inputting a projection image to be processed to a trained projection domain depth convolution neural network, predicting noise distribution in the projection image to be processed by the projection domain depth convolution neural network, subtracting the noise distribution from the prediction, and outputting a high-quality projection image;
three-dimensional reconstruction: and respectively executing a projection image denoising step on the plurality of projection images obtained in the projection transformation step to obtain high-quality projection images corresponding to the projection images, and then performing three-dimensional reconstruction on the obtained high-quality projection images to obtain cone beam CT images.
After the low-dose cone beam CT original projection data are converted into projection images, the noise distribution in the projection images is predicted by using a projection domain depth convolution neural network, and the noise in the projection images is subtracted, so that the noise and artifacts caused by low dose in the projection images can be removed while the original structure of the projection images is kept, and the quality of the projection images is effectively improved; the method carries out three-dimensional reconstruction based on the projected image after denoising, and can effectively improve the quality of the cone beam CT image obtained by reconstruction.
In the process of image reconstruction, the time-consuming iterative process is not required to be executed, so that the reconstruction speed is high.
Further, the projection domain depth convolution neural network comprises one or more first residual blocks which are connected in sequence, and the structures of the first residual blocks are the same;
the first residual block comprises one or more first units which are connected in sequence, the first units comprise a convolution layer, a normalization layer and an activation function layer which are connected in sequence, and the input of the first residual block is summed through a jump link with the output of the activation function layer through the jump link to be used as the output of the first residual block, so that a residual network is formed.
Further, the projection domain depth convolution neural network is used as a generator for generating a countermeasure network in the training process;
a generator in the countermeasure network is generated for predicting the noise distribution of the input image and subtracting the noise distribution from the input image to obtain a high-quality image; and the discriminator in the generation countermeasure network is used for discriminating the difference between the high-quality image output by the generator and the corresponding gold standard and feeding back the difference to the generator so as to update the generator.
The invention completes the training of the projection domain depth convolution neural network by means of the generation countermeasure network, and takes the projection domain depth convolution neural network as the generator for generating the countermeasure network, thereby ensuring the training effect of the projection domain depth convolution neural network by utilizing the discrimination function of the discriminator in the generation countermeasure network, effectively removing the noise in the projection image and improving the quality of the intermediate image when the cone beam CT image is reconstructed.
Further, the training method of the projection domain depth convolution neural network comprises the following steps:
randomly selecting a part of projection images from an original projection image data set according to a preset proportion; adding noise to the selected projection image, taking the projection image after noise addition as a low-dose projection image, taking the projection image before noise addition as a corresponding gold standard, forming a training sample by the low-dose projection image and the corresponding gold standard, and forming a projection domain training data set by all the training samples;
establishing a generated confrontation network, and training the generated confrontation network by utilizing a projection domain training data set;
and after training is finished, extracting generators in the generation countermeasure network as a projection domain deep convolution neural network.
The method randomly selects part of the projection images from the original projection image data set, simulates the low-dose projection images in a noise adding mode, and uses the original projection images without noise as a gold standard, so as to construct a training data set of the projection domain depth convolution neural network.
Further, the low dose cone-beam CT image reconstruction method based on deep learning provided by the present invention further includes:
and (3) denoising a reconstructed image: and inputting the cone beam CT image obtained in the three-dimensional reconstruction step into a trained image domain depth convolution neural network, eliminating noise and artifacts in the cone beam CT image by the image domain depth convolution neural network, outputting a high-quality cone beam CT image, and taking the high-quality cone beam CT image as a final reconstruction result.
On the basis of obtaining the cone beam CT image through three-dimensional reconstruction, the invention eliminates noise and artifacts in the cone beam CT image by using the image domain depth convolution neural network, and can further improve the quality of the cone beam CT image.
Further, the image domain deep convolutional neural network comprises: an encoder, one or more second residual blocks, and a decoder connected in sequence;
the encoder comprises one or more second units which are connected in sequence, and the second residual block comprises one or more second units which are connected in sequence; the second unit comprises a convolution layer, a normalization layer and an activation function layer which are connected in sequence;
the decoder comprises one or more third units which are connected in sequence; the third unit comprises an deconvolution layer, a normalization layer and an activation function layer which are connected in sequence;
the encoder and the decoder adopt a symmetrical structure, and the input of the first convolution layer, the third convolution layer and the fifth convolution layer of the encoder are sequentially added with the output of the corresponding deconvolution layer in the decoder through jump linkage to form a residual error structure; the input of the first convolutional layer of the second residual block is added to the output of the normalized second convolutional layer by a skip link to form a residual structure.
In the invention, the image domain deep convolutional neural network is a residual error structure, which can deepen the depth of the convolutional neural network, thereby improving the quality of an output image and accelerating the training of the convolutional neural network.
Further, the training method of the image domain deep convolutional neural network comprises the following steps:
constructing an image domain training data set; in the image domain training data set, each sample consists of two images, wherein one image is a cone beam CT image obtained by utilizing a three-dimensional reconstruction step and is used as the input of an image domain depth convolution neural network, and the other image is a cone beam CT image with normal dosage and is used as a reference image of the image domain depth convolution neural output;
and establishing an image domain deep convolutional neural network, and training the image domain deep convolutional neural network by using an image domain training data set, so that the trained image domain deep convolutional neural network is obtained after training is finished.
When the image domain depth convolution neural network is trained, the cone beam CT image reconstructed by the projection image denoised by the projection domain depth convolution neural network and the cone beam CT image with normal dose are used for constructing a training data set, so that the image domain depth convolution neural network obtained by training can effectively remove the noise and the artifact still existing in the projection image.
Further, in the three-dimensional reconstruction step, the obtained high-quality projection image is subjected to three-dimensional reconstruction by using an FDK algorithm.
The FDK algorithm (Feldkamp-Davis-Kress algorithm) has the advantage of high reconstruction speed as an analytical algorithm, and usually has good image quality under the condition of normal dose, but the reconstructed image has more serious noise and artifacts under the condition of low dose; because the invention utilizes the projection domain depth convolution neural network to remove the noise and the artifacts in the projection image obtained by the transformation of the low-dose cone beam CT projection data and obtain the high-quality projection image, the FDK algorithm is utilized to reconstruct the projection image, and the reconstruction quality can be ensured and the reconstruction speed can be effectively improved.
According to another aspect of the present invention, there is provided a computer readable storage medium comprising a stored computer program;
when the computer program is executed by the processor, the computer readable storage medium is controlled to execute the method for reconstructing a low-dose cone-beam CT image based on deep learning according to the present invention.
Generally, by the above technical solution conceived by the present invention, the following beneficial effects can be obtained:
(1) after the low-dose cone beam CT original projection data are converted into projection images, the noise distribution in the projection images is predicted by using a projection domain depth convolution neural network, and the noise in the projection images is subtracted, so that the noise and artifacts caused by low dose in the projection images can be removed while the original structure of the projection images is kept, and the quality of the projection images is effectively improved; the method carries out three-dimensional reconstruction based on the projected image after denoising, and can effectively improve the quality of the cone beam CT image obtained by reconstruction.
(2) In the process of image reconstruction, the time-consuming iterative process is not required to be executed, so that the reconstruction speed is high.
(3) On the basis of obtaining the cone beam CT image through three-dimensional reconstruction, the invention eliminates noise and artifacts in the cone beam CT image by using the image domain depth convolution neural network, and can further improve the quality of the cone beam CT image.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
In the present application, the terms "first," "second," and the like (if any) in the description and the drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
Aiming at the technical problem that the quality of a reconstructed image is not high because the conventional low-dose cone-beam CT image reconstruction method cannot eliminate noise and simultaneously reserve an image structure, the invention provides a low-dose cone-beam CT image reconstruction method based on deep learning, which has the overall thought that: after low-dose cone beam CT original projection data are converted into projection images, noise distribution in each projection image is predicted by using a projection domain depth convolution neural network, the noise distribution is subtracted from the projection images so as to improve the quality of the projection images, then three-dimensional reconstruction is carried out on the basis of the high-quality projection images so as to obtain cone beam CT images, therefore, the original structure of the projection images is kept while the noise and artifacts in the projection images are eliminated, and the quality of the reconstructed cone beam CT images is improved; on the basis, the noise and the artifact in the reconstructed cone beam CT image are further eliminated by utilizing the image domain depth convolution neural network, and the quality of the reconstructed image is further improved. The following are examples:
example 1:
a low dose cone-beam CT image reconstruction method based on deep learning, as shown in fig. 1 and 2, comprising:
a projection transformation step:
transforming the low-dose cone-beam CT original projection data into a plurality of projection images;
in this embodiment, the processed low-dose cone-beam CT original projection data may be obtained by sparse acquisition, or may be obtained by reducing the intensity of X-rays; when the low-dose cone-beam CT original projection data are converted into projection images, the beer-Lambert law is specifically adopted for conversion;
a projected image denoising step:
inputting a projection image to be processed to a trained projection domain depth convolution neural network, predicting noise distribution in the projection image to be processed by the projection domain depth convolution neural network, subtracting the noise distribution from the prediction, and outputting a high-quality projection image;
three-dimensional reconstruction:
and respectively executing a projection image denoising step on the plurality of projection images obtained in the projection transformation step to obtain high-quality projection images corresponding to the projection images, and then performing three-dimensional reconstruction on the obtained high-quality projection images to obtain cone beam CT images.
As an optional implementation manner, in this embodiment, the projection domain deep convolutional neural network is used as a generator for generating a countermeasure network (GAN) in the training process;
a generator in the countermeasure network is generated for predicting the noise distribution of the input image and subtracting the noise distribution from the input image to obtain a high-quality image; a discriminator in the generation countermeasure network is used for discriminating the difference between the high-quality image output by the generator and the corresponding gold standard and feeding back the difference to the generator so as to update the generator;
the structure of the generated countermeasure network is specifically shown in fig. 3, in which the Generator is a Generator, the Discriminator is a Discriminator, and the Generator specifically includes: the structure of each first residual block is the same;
the first residual block comprises one or more first units which are connected in sequence, the first units comprise a convolutional layer (Conv), a normalization layer (BN) and an activation function layer (ReLU) which are connected in sequence, and the input of the first residual block is summed with the output of the activation function layer through a jump link to form the output of the first residual block, so that a residual network is formed;
as shown in fig. 3, in the generated countermeasure network, the arbiter specifically includes 6 units, each unit includes a convolution layer, a normalization layer, and an activation function layer connected in sequence, where, except that the activation function used by the activation function layer of the last unit is a sigmoid activation function, the activation functions used by the activation function layers of the other units are all lreul;
based on the generation countermeasure network structure shown in fig. 3, when the deep convolutional neural network in the projection domain is trained, the real X-Ray projection images of 10 walnuts are specifically selected to construct a training data set; firstly, reading obtained walnut X-ray original projection data in a TIFF (Tag Image File Format) Format into an internal memory by using a python open source kit (Numpy), wherein a pixel value in the TIFF Image File represents the photon number obtained by detecting a corresponding detecting element; converting the raw projection data into a beam intensity loss image I, also referred to as a projection image, using Beer-Lambert's law; for each walnut, randomly selecting 200 projected images, obtaining 2000 projected images in total, and forming an original projected image data set by the 2000 projected images;
based on the original projection image data set, in this embodiment, the training method of the projection domain depth convolution neural network includes:
randomly selecting a part of projection images from an original projection image data set according to a preset proportion; adding noise to the selected projection image, taking the projection image after noise addition as a low-dose projection image, taking the projection image before noise addition as a corresponding gold standard, forming a training sample by the low-dose projection image and the corresponding gold standard, and forming a projection domain training data set by all the training samples;
specifically, the low-dose projection image is simulated by adding poisson noise to the projection image, and the relevant calculation expression is as follows:
Pld,sim=log(Ild,sim)
wherein P represents a cone beam CT projection image of normal dose, f is a flat field image of low dose, α is a tube current of an X-ray tube corresponding to the normal dose, β is a tube current of an X-ray tube corresponding to the low dose, Ild,simFor simulated low dose incident flux, Pld,simA simulated low dose projection image; optionally, in this embodiment, β/α is 1/8, i.e., projection images simulating one-eighth of the dose;
after the projection domain training data set is established, establishing a generation countermeasure network shown in fig. 3, and training the generation countermeasure network by using the projection domain training data set; in the training process, inputting the low-dose projection image into a generator, predicting the noise distribution of the low-dose projection image by the generator, and then subtracting the corresponding noise distribution from the low-dose projection image to obtain a projection image without noise; meanwhile, the discriminator judges the difference between the generated image and the normal dose projection image, and feeds the difference back to the generator to realize the updating of the generator, so that the generator generates a projection image with higher quality; meanwhile, the discriminator is correspondingly updated; thus, the generator and the discriminator are updated alternately, thereby generating a high-quality projection image; in consideration of the limitation of the computing power of the device, optionally, the present embodiment sets the batch _ size in the training process to 1;
in this embodiment, a difference between a generated image and a real projection image of a generated countermeasure network is measured by using a supervisory loss function, and an expression of the supervisory loss function is as follows:
representing the resistance loss function, specifically:
represents the peak signal-to-noise ratio loss function, specifically:
MSE represents a mean square error function, specifically:
the structural similarity loss function is specifically as follows:
where Θ represents the weight for generating the countermeasure network, λ1,λ2And λ3Is an equilibrium constant; g and D represent the generator and the arbiter, respectively, for generating an antagonistic network, PldAnd PndRespectively representing a low-dose projection image and a corresponding normal-dose projection image, and E represents the Wassertein distance; maxyA maximum of pixels representing a normal dose image; pgRepresenting an image generated by a generator generating the countermeasure network, M and N representing the height and width, respectively, of the image, (i, j) representing the pixel coordinates in the image; mu.s1And mu2Respectively representing the average values of the pixels of the generated countermeasure network generated image and the reference image. Sigma1,σ2And σ1,2Respectively representing the standard deviation of the pixel values of the generated countermeasure network generation image and the reference image and the covariance of the pixel values of the generated countermeasure network generation image and the reference image; c1And C2Respectively, are hyper-parameters;
in order to effectively generate the countermeasure network, the example adopts RMSprop (root mean square prop) as an optimizer, the learning rate is 0.0002, and the number of training rounds is 30; the whole training process is run on a tensorflow framework;
after training is finished, storing the model of the last iteration, and extracting a generator in the generated confrontation network as a projection domain depth convolution neural network;
it should be noted that the structure of the generator and the arbiter in the generation countermeasure network is only an optional embodiment, and should not be construed as the only limitation of the present invention, and in other embodiments of the present invention, the corresponding structure may be adjusted according to the actual situation; similarly, in this embodiment, the training of the projection domain deep convolutional neural network is completed by means of the generation of the antagonistic network, which is also only an optional embodiment of the present invention, and in some other embodiments of the present invention, the training of the projection domain deep convolutional neural network may also be completed without the generation of the antagonistic network, as long as the effect of the final projection domain deep convolutional neural network on removing noise in the projection image can meet the precision requirement of practical application.
In order to further improve the quality of the reconstructed cone-beam CT image, the present embodiment further includes:
and (3) denoising a reconstructed image: inputting the cone beam CT image obtained in the three-dimensional reconstruction step into a trained image domain depth convolution neural network, eliminating noise and artifacts in the cone beam CT image by the image domain depth convolution neural network, outputting a high-quality cone beam CT image, and taking the high-quality cone beam CT image as a final reconstruction result;
optionally, in this embodiment, a structure of the image domain deep convolutional neural network is shown in fig. 4, and includes: the encoder, the six second residual blocks and the decoder are connected in sequence; the encoder is used for extracting the characteristic information of the input image and transmitting the characteristic information to the six second residual blocks to further eliminate the noise and the artifacts of the input image, and the decoder recovers the high-quality cone beam CT image by utilizing the characteristic information provided by the second residual blocks;
as shown in fig. 4, the encoder includes 5 second units connected in sequence, and the second residual block includes 2 units connected in sequence; the second unit comprises a convolution layer (Conv), a normalization layer (BN) and an activation function layer (ReLU) which are connected in sequence;
the decoder comprises one or more third units which are connected in sequence; the third unit comprises a deconvolution layer (Deconv), a normalization layer (BN) and an activation function layer (ReLU) which are connected in sequence;
the encoder and the decoder adopt a symmetrical structure, and the input of the first convolution layer, the third convolution layer and the fifth convolution layer of the encoder are sequentially added with the output of the corresponding deconvolution layer in the decoder through jump linkage to form a residual error structure; the input of the first convolutional layer of the second residual block is added with the output of the second convolutional layer which is normalized through a jump link to form a residual structure;
in the embodiment, the image domain depth convolution neural network is a residual error structure, which can deepen the depth of the convolution neural network, thereby improving the quality of an output image and accelerating the training of the convolution neural network;
in this embodiment, the training method for the image domain deep convolutional neural network includes:
constructing an image domain training data set; in the image domain training data set, each sample consists of two images, wherein one image is a cone beam CT image obtained by a three-dimensional reconstruction step, namely the cone beam CT image obtained by three-dimensional reconstruction after the quality of a projection image is improved by a projection domain depth convolution neural network is used as the input of the image domain depth convolution neural network, and the other image is a cone beam CT image with normal dosage and is used as a reference image of the image domain depth convolution neural output;
establishing an image domain deep convolutional neural network, and training the image domain deep convolutional neural network by using an image domain training data set, so that a trained image domain deep convolutional neural network is obtained after training is finished;
optionally, in this embodiment, in the process of training the image domain depth convolutional neural network, the batch _ size is set to 2, and a traditional Mean Square Error (MSE) loss function is used to describe a difference between the output image and the cone beam image with a normal dose, where the expression is as follows:
wherein, omega represents the weight of the image domain depth convolution neural network, eta is the learning rate, F is the mapping function of the image domain depth convolution neural network, Im,1Representing cone-beam CT images generated by generating a countermeasure network, Im,2Is a normal dose cone-beam CT image, | | | | | non-woven phosphor2Represents a 2-norm, and M is the size of batch.
In order to ensure the quality of the reconstructed image and improve the reconstruction speed, as a preferred embodiment, in the three-dimensional reconstruction step of this embodiment, the three-dimensional reconstruction is performed on the obtained high-quality projection image by using the FDK algorithm;
the FDK algorithm (Feldkamp-Davis-Kress algorithm) has the advantage of high reconstruction speed as an analytical algorithm, and usually has good image quality under the condition of normal dose, but the reconstructed image has more serious noise and artifacts under the condition of low dose; in the embodiment, the projection domain depth convolution neural network is used for removing noise and artifacts in the projection image obtained by transforming the low-dose cone-beam CT projection data to obtain the high-quality projection image, so that the FDK algorithm is used for reconstructing the projection image, and the reconstruction speed can be effectively improved while the reconstruction quality is ensured; it should be noted that the FDK algorithm is only a preferred embodiment of the present invention, and other algorithms that can perform cone-beam CT image reconstruction using projection images can be applied to the present invention.
Example 2:
a low-dose cone beam CT image reconstruction method based on deep learning is similar to the embodiment 1, and is different in that the embodiment only does not include a reconstruction image denoising step, and a cone beam CT image obtained in a three-dimensional reconstruction step is directly used as a reconstruction result;
the specific implementation of this embodiment can refer to the description of embodiment 1, and will not be repeated here.
Example 3:
a computer readable storage medium comprising a stored computer program;
when the computer program is executed by the processor, the apparatus in which the computer readable storage medium is located is controlled to execute the method for reconstructing a low-dose cone-beam CT image based on deep learning provided in embodiment 1 or embodiment 2.
To further illustrate the effectiveness and reliability of the present invention, examples 1 and 2 above are compared with existing methods of boosting low dose cone-beam CT images, including: CGLS (continuous gradient least square) algorithm, SIRT (Simultaneous iterative detection technique), RED-CNN (Residual Encode DecoderConvolitional Neural network), and MAP-NN (modulated Adaptive Processing Neural network). CGLS is a traditional analytical algorithm, and the least square problem of an underdetermined system is solved by a conjugate gradient method; SIRT is a classical iterative algorithm, and an optimal solution is obtained by minimizing an objective function in an iterative process; RED-CNN and MAP-NN belong to the methods of applying the deep learning technology to the CT image domain process, and currently, a better effect is obtained in a quarter-dose fan-beam CT image.
The reconstruction results of the cone beam CT projection data of the same object by the various methods are shown in (a) to (h) in fig. 5, and it can be seen from the reconstruction results shown in fig. 5 that the reconstructed image under the FDK algorithm contains serious noise and artifacts, and the noise and artifacts cause difficulty in distinguishing image structure information; SIRT can eliminate noise and artifacts to some extent, but causes image distortion; the CGLS algorithm inhibits partial noise and artifacts, but the residual noise and artifacts still attach to an image structure, so that partial structure information is lost; RED-CNN and MAP-NN can effectively eliminate the noise and artifacts of the reconstructed image as a deep learning post-processing technology, but the image structure is also sacrificed to a great extent; in this embodiment 2, the quality of the projection image is improved by using the generated countermeasure network in the projection domain, and the corresponding reconstructed image can largely retain the structural information of the image; in this embodiment, an image domain deep learning post-processing technique is further adopted, so that noise and artifacts of the image in embodiment 2 can be further eliminated, and the contrast of the reconstructed image is improved. In conclusion, the method of the embodiment achieves better results than other algorithms, both in terms of noise suppression and structure retention.
In order to quantitatively compare the performance of improving the quality of the low-dose cone-beam CT images by various methods, three standards of PSNR (Peak Signal-to-Noise Ratio), SSIM (structural similarity) and RMSE (root Mean Square error) are adopted, the SSIM value is used for measuring the structure retention degree of the reconstructed images compared with the normal dose reconstructed images, the better the SSIM is, the PSNR and RMSE are used for measuring the visual difference between the reconstructed images and the normal dose images, and the larger the PSNR and RMSE are, the smaller the visual difference between the two images is. Specifically, the quantization values of 200 projection images are counted, and the quantization performance comparison of different methods on the task of implementing the sparse angle cone-beam CT image is shown in table one.
Table one: on-task performance comparison of different methods on low-dose cone-beam CT image realization
As can be analyzed from the table i, the reconstructed image under the method of this embodiment obtains the highest peak signal-to-noise ratio (PSNR) and the highest Structural Similarity (SSIM), which indicates that the low-dose reconstructed image under this embodiment is visually closest to the normal-dose reconstructed image and retains the structural information to the greatest extent. Meanwhile, the RMSE of the low-dose reconstructed image in this embodiment is the minimum compared with the normal-dose reconstructed image, that is, the difference of the average pixel value is the minimum, which indicates that the reconstruction accuracy of the method of this embodiment is the highest.
Generally speaking, the method transforms low-dose cone beam CT original projection data into projection images, predicts noise distribution in the projection images by using a projection domain depth convolution neural network, subtracts the noise in the projection images to obtain high-quality projection images, carries out three-dimensional reconstruction based on the high-quality projection images to obtain cone beam CT images, can remove noise and artifacts caused by low dose in the projection images while keeping the original structure of the projection images, and effectively improves the quality of the cone beam CT images. On the basis, the noise and the artifact in the cone beam CT image are eliminated by utilizing the image domain depth convolution neural network, so that the quality of the cone beam CT image can be further improved.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.