CN112396579A - Human tissue background estimation method and device based on deep neural network - Google Patents
Human tissue background estimation method and device based on deep neural network Download PDFInfo
- Publication number
- CN112396579A CN112396579A CN202011211249.9A CN202011211249A CN112396579A CN 112396579 A CN112396579 A CN 112396579A CN 202011211249 A CN202011211249 A CN 202011211249A CN 112396579 A CN112396579 A CN 112396579A
- Authority
- CN
- China
- Prior art keywords
- human tissue
- attenuation coefficient
- coefficient distribution
- neural network
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 34
- 238000009826 distribution Methods 0.000 claims abstract description 99
- 238000012549 training Methods 0.000 claims abstract description 60
- 238000013135 deep learning Methods 0.000 claims abstract description 49
- 239000002872 contrast media Substances 0.000 claims description 42
- 238000001228 spectrum Methods 0.000 claims description 16
- 238000013527 convolutional neural network Methods 0.000 claims description 14
- 230000006870 function Effects 0.000 claims description 14
- 230000008447 perception Effects 0.000 claims description 10
- 239000003795 chemical substances by application Substances 0.000 claims description 9
- 230000004913 activation Effects 0.000 claims description 6
- 238000013459 approach Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 5
- 229910052704 radon Inorganic materials 0.000 claims description 3
- SYUHGPGVQRZVTB-UHFFFAOYSA-N radon atom Chemical group [Rn] SYUHGPGVQRZVTB-UHFFFAOYSA-N 0.000 claims description 3
- 210000001519 tissue Anatomy 0.000 description 106
- 238000003384 imaging method Methods 0.000 description 14
- 238000001514 detection method Methods 0.000 description 10
- 230000000694 effects Effects 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 238000011160 research Methods 0.000 description 8
- 230000005855 radiation Effects 0.000 description 5
- 239000000126 substance Substances 0.000 description 4
- 206010006187 Breast cancer Diseases 0.000 description 3
- 208000026310 Breast neoplasm Diseases 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000010191 image analysis Methods 0.000 description 3
- 238000011176 pooling Methods 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 230000035945 sensitivity Effects 0.000 description 3
- 206010028980 Neoplasm Diseases 0.000 description 2
- 206010056342 Pulmonary mass Diseases 0.000 description 2
- 238000013170 computed tomography imaging Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000003709 image segmentation Methods 0.000 description 2
- 230000036039 immunity Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 210000000056 organ Anatomy 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 230000003595 spectral effect Effects 0.000 description 2
- 208000024172 Cardiovascular disease Diseases 0.000 description 1
- 206010007759 Cataract nuclear Diseases 0.000 description 1
- 208000032843 Hemorrhage Diseases 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 210000000988 bone and bone Anatomy 0.000 description 1
- 230000003925 brain function Effects 0.000 description 1
- 208000026106 cerebrovascular disease Diseases 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000002526 effect on cardiovascular system Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 230000003211 malignant effect Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 201000001441 melanoma Diseases 0.000 description 1
- 230000011278 mitosis Effects 0.000 description 1
- 208000029552 nuclear cataract Diseases 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 230000001575 pathological effect Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 230000002685 pulmonary effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
A human tissue background estimation method and a device based on a deep neural network are provided, the method comprises the following steps: (1) training a human tissue attenuation coefficient distribution estimation model based on a deep learning network WGAN-VGG, constructing deep learning training data, using a first iteration to reconstruct a human tissue attenuation coefficient distribution image as input data, and using an original human tissue CT slice image as sample data to input into the deep learning network for training; (2) and after training is finished to obtain the human tissue attenuation coefficient distribution estimation model, replacing subsequent iteration steps with the model, and inputting the first iteration reconstructed human tissue attenuation coefficient distribution image into the model so as to obtain an output human tissue attenuation coefficient distribution image with higher accuracy.
Description
Technical Field
The invention relates to the technical field of medical image processing, in particular to a human tissue background estimation method based on a deep neural network and a human tissue background estimation device based on the deep neural network, which are mainly used for estimating the human tissue attenuation coefficient distribution before and after K-edge.
Background
In recent years, a Photon Counting Detector (PCD) is rapidly developed, and a new possibility is brought to utilize the K-edge characteristic of a contrast agent substance in contrast enhanced imaging (in a theoretical K-edge characteristic, the ray attenuation coefficient reaches a maximum peak value under a certain specific energy). The X-ray photon counting detector has the energy resolution capability, can discriminate each incident photon as an independent event, judges an energy interval by setting the energy threshold value and the photon pulse amplitude of the detector and carries out accumulation counting in a subarea mode. By setting an electronic threshold, the X-ray photon counting detector can filter out pulses with lower energy, thereby eliminating the influence of low-energy noise on imaging and improving the signal-to-noise ratio. By setting more thresholds, the X-ray photon counting detector can discriminate the pulse signal after each incident photon and a substance interact, identify the energy of the incident photon and respectively accumulate the energy of the incident photon corresponding to different energy spectrum intervals, so that the X-ray sub-energy spectrum intervals with wider energy spectrum distribution are detected, and the image reconstruction is directly carried out in the different energy spectrum intervals. Thus, PCD-based spectral CT will be one of the research focuses in the field of future X-ray imaging. Bo Meng and the like combine PCD signal acquisition characteristics and a CT image reconstruction algorithm to provide a Projection data space K-edge Imaging Method (PDKI), the Method makes full use of the K-edge characteristics of a contrast medium substance, can reconstruct contrast medium concentration distribution from energy spectrum information on two sides of the K-edge, and effectively improves the precision and sensitivity of contrast medium distribution detection on the basis of reducing the irradiated radiation dose of a patient and the contrast medium concentration.
The PDKI algorithm is implemented by using human tissue background information before the contrast agent is injected. At present, there are two ways to acquire background information of human tissue, the first way is to perform CT pre-scan on a patient before injecting a contrast agent to acquire background information of human tissue, but this way will increase the radiation dose of the patient; the second method is to make the distribution of attenuation coefficient of human tissue gradually approach the true value by iterative computation, but the effect and precision of this method are greatly affected by image noise. Based on the method, the invention provides a human tissue background estimation method based on a deep neural network WGAN-VGG by combining with the latest progress in the field of deep learning.
In recent years, more and more deep learning methods are applied to the study of CT image reconstruction. The application of deep learning in CT imaging mainly focuses on several aspects including training data generation, image domain deep learning image reconstruction, projection domain deep learning image reconstruction, joint reconstruction strategy, end-to-end workflow, and the like. CT image reconstruction based on deep learning methods requires a large amount of high quality, standardized and labeled data to conduct training, correction, adjustment and testing. Referring to the successful application of deep learning in computer vision or image processing, such datasets like ImageNet contain millions of pictures. Due to the complexity of the mapping of CT image reconstruction from projection data space to image data space, it is unlikely that a CT reconstructed image dataset will possess a dataset of as large a number of pictures as ImageNet. As much CT image data as possible has been accumulated, but only a small percentage of it is available for scientific research due to privacy, law, and various commercial factors. For example, CT projection data is tightly protected by individual CT equipment companies and is not open to researchers. Also, research projects are often resource-limited and focused on a particular category of disease study, it is difficult to collect sufficient training data as ImageNet data sets. Insufficient training data can cause problems of over-training fitting, reduced precision and reconstructed image artifacts, and the development of an image reconstruction algorithm based on deep learning is hindered. Thus, the CT image reconstruction algorithm training data set can be augmented and enhanced by deep learning methods, such as GAN generation or transfer learning.
The image domain deep learning image reconstruction is to utilize deep learning to process a reconstructed image after the image is reconstructed based on a traditional algorithm so as to improve the image quality. Projection domain depth learning image reconstruction is a method of image imaging based on depth learning to leverage measured projection data. The joint reconstruction framework refers to image reconstruction by combining the advantages of the traditional CT image reconstruction algorithm, particularly the iterative reconstruction algorithm and the deep learning method. The iterative reconstruction algorithm essentially uses a priori formula designed by people to obtain an image with a complete low-noise structure, so a neural network module can be used for replacing a regularization item designed by people in the iterative reconstruction algorithm.
On the other hand, as deep learning has been continuously advanced in the field of computer vision, deep learning applications in medical image analysis have also received more and more attention. Medical image analysis has played an important role in clinical auxiliary screening, diagnosis, grading, treatment decision and guidance, curative effect evaluation and the like of major diseases such as tumors, brain functions, cardiovascular and cerebrovascular diseases and the like. The deep learning research of medical image analysis mainly focuses on the fields of medical image classification and identification, positioning and detection, tissue organ and focus segmentation and the like. In medical image classification and identification applications, the convolutional neural network can extract depth features of different levels of lesions, so that the classification accuracy of cancers is improved, the convolutional neural network is gradually becoming a mainstream technology in image screening and classification, and is widely applied to breast cancer diagnosis, automatic bone age assessment and nuclear cataract classification. The large-scale training artificial neural network and the convolution neural network are applied to the task of lung nodule detection and benign and malignant lung nodule classification in the CT image, and a better classification result is obtained. In the application of medical image target positioning and detection, the convolutional neural network is used for executing a pixel classification task, and then an interested target is obtained by adopting a post-processing mode, such as mitosis detection, pulmonary nodule detection and fundus hemorrhage detection in a breast cancer pathological image. In addition, other deep learning methods such as stacked self-encoders, deep belief networks, etc. are also used for the detection and identification of breast cancer stages and melanoma in medical images. In medical image segmentation applications, most organ and tissue segmentation methods are based on convolutional neural networks. Researchers have obtained good segmentation results using the image block-based approach to train the network. The U-net, DSN and V-net derived from FCN work well in image segmentation task.
In the aspect of research on the aspect of noise reduction of low-dose CT reconstructed images, a deep learning network is utilized to train a low-dose CT image and a conventional-dose CT image pair at the same position so as to obtain a mapping relation between the low-dose CT image and the conventional-dose CT image. Chen et al propose a method for estimating a corresponding conventional dose CT image based on a low dose CT image by using a convolutional neural network, and a good noise reduction effect is obtained. There are two problems with the use of the mean square error of the voxels as a loss function in the above studies. Firstly, due to the influence of noise, the consistency of the voxel values of a training target image is poor; second, the mean square error function can cause image blurring and lack of texture detail information.
In response to the two problems, wolternk et al propose a method for further optimizing the quality of low-dose CT imaging images using generation of a countermeasure network. In the research, a convolutional neural network is used as a generator and a convolutional neural network is used as a discriminator to construct a generation countermeasure network. The goal of the discriminator convolutional neural network is to distinguish between conventional dose CT images and generator-processed low dose CT images and to back-propagate the resulting countermeasures to optimize the generation network parameters. The introduction of the resistance loss enables the quality of the low-dose CT image to be closer to that of a conventional-dose CT image, and simultaneously partially solves the problem of image blurring caused by only using the mean square error. Further, in order to improve the stability of the training process, Yang et al propose that Wasserstein GAN is used for training, perception Loss (perceptual Loss) is introduced based on relevant research of SRGAN, and a VGG network is used as a high-level feature extractor to retain high-frequency information of an image and improve the sense of realism of the image.
Inspired by the low-dose CT image noise reduction research, the invention estimates the human tissue attenuation coefficient distribution before and after K-edge by using Wassertein GAN and combining with the perception loss based on the real human tissue attenuation coefficient distribution.
The prior technical scheme is as follows: the iterative calculation method for human tissue background estimation is based on the fact that human tissue attenuation coefficients are not greatly different in energy spectrum windows before and after a contrast agent substance K-edge, as shown in figure 1, it is assumed that the human tissue attenuation coefficients in the energy spectrum windows before and after the K-edge are equal, and then a human tissue attenuation coefficient subtraction term in a PDKI imaging formula is zero. Secondly, in the post-change PDKI imaging formula, only an unknown amount of contrast agent concentration distribution can be obtained through calculation. And then, calculating to obtain a first-order approximate value of the attenuation coefficient distribution of the human tissue by combining the concentration distribution of the contrast agent and the number of photons detected by an X-ray photon counting detector in energy spectrum windows before and after K-edge. And finally, substituting the human tissue attenuation coefficient distribution approximate value into the imaging formula, and gradually approaching the human tissue attenuation coefficient distribution approximate value to a true value after a plurality of iterations, so that the contrast agent concentration distribution graph is obtained while the human tissue attenuation coefficient distribution is estimated.
The defects of the prior art are as follows:
1. the main disadvantage of the method for acquiring the background information of the human tissue by CT pre-scanning is that the patient needs to be pre-scanned before the contrast agent is injected, and the radiation dose of the patient is increased.
2. The main disadvantages of the human tissue attenuation coefficient distribution iterative estimation method are that the iterative calculation precision is greatly influenced by image noise, and the robustness of the method is poor.
Disclosure of Invention
In order to overcome the defects of the prior art, the technical problem to be solved by the invention is to provide a human tissue background estimation method based on a deep neural network, which does not need to carry out pre-scanning on a patient before injecting a contrast medium, can greatly reduce the radiation exposure dose of the patient, and reduce the potential influence of radiation exposure on a human body; the robustness to imaging noise is better, and the detection precision and the sensitivity are both greatly improved.
The technical scheme of the invention is as follows: the human tissue background estimation method based on the deep neural network comprises the following steps:
(1) training a human tissue attenuation coefficient distribution estimation model based on a deep learning network WGAN-VGG, constructing deep learning training data, using a first iteration to reconstruct a human tissue attenuation coefficient distribution image as input data, and using an original human tissue CT slice image as sample data to input into the deep learning network for training;
(2) and after training is finished to obtain the human tissue attenuation coefficient distribution estimation model, replacing subsequent iteration steps with the model, and inputting the first iteration reconstructed human tissue attenuation coefficient distribution image into the model so as to obtain an output human tissue attenuation coefficient distribution image with higher accuracy.
The method utilizes the characteristic of high fidelity of the details of the perception loss function image, processes the first-order approximate distribution of the human tissue attenuation coefficient by combining the Wassertein GAN network and the perception loss function VGG, and adopts a feature extraction method that the VGG network is used as the perception loss function to inhibit the fuzzy effect of the human tissue attenuation coefficient distribution on the image. Training human tissue attenuation coefficient distribution estimation through a deep learning network WGAN-VGG, and further optimizing three dimensions of image structure similarity, peak signal-to-noise ratio and signal-to-noise ratio to obtain a human tissue attenuation coefficient distribution image. The method can obtain more accurate human tissue attenuation coefficient distribution from a first-order approximate value, has good noise immunity, and can accurately detect focus information under the conditions of low contrast agent concentration and small focus size.
There is also provided a deep neural network-based human tissue background estimation apparatus, including:
the training module is configured for training a human tissue attenuation coefficient distribution estimation model based on a deep learning network WGAN-VGG, constructing deep learning training data, reconstructing a human tissue attenuation coefficient distribution image as input data by utilizing first iteration, and inputting an original human tissue CT slice image as sample data into the deep learning network for training;
and the iteration module is configured to replace subsequent iteration steps by using the model after the human tissue attenuation coefficient distribution estimation model is obtained after training is finished, and input the first iteration reconstructed human tissue attenuation coefficient distribution image into the model so as to obtain an output human tissue attenuation coefficient distribution image with higher accuracy.
Drawings
Fig. 1 is a flowchart of a conventional iterative estimation method of human tissue attenuation coefficient distribution.
FIG. 2 is a schematic diagram of a human tissue background estimation method flow based on a deep learning network WGAN-VGG.
FIG. 3 is a flow chart of X-ray K-edge imaging based on the deep neural network WGAN-VGG.
Fig. 4 is a flowchart of a deep neural network-based human tissue background estimation method according to the present invention.
Detailed Description
As shown in fig. 4, the method for estimating the background of human tissue based on the deep neural network includes the following steps:
(1) training a human tissue attenuation coefficient distribution estimation model based on a deep learning network WGAN-VGG, constructing deep learning training data, using a first iteration to reconstruct a human tissue attenuation coefficient distribution image as input data, and using an original human tissue CT slice image as sample data to input into the deep learning network for training;
(2) and after training is finished to obtain the human tissue attenuation coefficient distribution estimation model, replacing subsequent iteration steps with the model, and inputting the first iteration reconstructed human tissue attenuation coefficient distribution image into the model so as to obtain an output human tissue attenuation coefficient distribution image with higher accuracy.
The method utilizes the characteristic of high fidelity of the details of the perception loss function image, processes the first-order approximate distribution of the human tissue attenuation coefficient by combining the Wassertein GAN network and the perception loss function VGG, and adopts a feature extraction method that the VGG network is used as the perception loss function to inhibit the fuzzy effect of the human tissue attenuation coefficient distribution on the image. Training human tissue attenuation coefficient distribution estimation through a deep learning network WGAN-VGG, and further optimizing three dimensions of image structure similarity, peak signal-to-noise ratio and signal-to-noise ratio to obtain a human tissue attenuation coefficient distribution image. The method can obtain more accurate human tissue attenuation coefficient distribution from a first-order approximate value, has good noise immunity, and can accurately detect focus information under the conditions of low contrast agent concentration and small focus size.
Preferably, in step (1), the deep learning network includes: the device comprises a generator, a VGG network and a discriminator.
Preferably, in step (1), the generator is an 8-layer convolutional neural network, wherein the first 7 hidden layers each include 32 filters with a size of 3 × 3, and the last layer is a 3 × 3 filter as the output of the generator G; after each convolutional layer, a modified linear unit ReLU is used as the activation function.
Preferably, in the step (1), a pre-trained VGG network is used as a perceptual loss calculator, the feature map output by the generator network G and the human tissue background without contrast agent are used as input to be fed into the VGG network to extract high-dimensional features, and the calculated perceptual loss is fed back to the neural network to update parameters of the generator network G.
Preferably, in the step (1), the discriminator network D is a 6-layer convolutional neural network, each layer comprising 64, 128, 256 and 256 3 × 3 filters; after the two full-connection layers are used for 6 convolutional layers, respectively outputting 1024 characteristic maps and 1 characteristic map; using Leaky ReLU as an activation function after each convolutional layer; the filter sliding step for the odd convolutional layers is 1 pixel, and the filter sliding step for the even convolutional layers is 2 pixels.
Preferably, the step (1) comprises the following substeps:
(1.1) optimizing the estimation of the attenuation coefficient distribution of human tissues;
(1.2) building a WGAN-VGG deep neural network training model;
and (1.3) training a human tissue attenuation coefficient distribution estimation model.
Preferably, in the step (1.1), the PDKI algorithm rebuild core formula is formula (1)
Based on equation (1), an iterative algorithm is used to estimate the distribution of attenuation coefficients in human tissue
And
preferably, in the step (1.2), in the formula (1), within the limited energy spectrum window width of the X-ray photon counting detector, assuming that the attenuation coefficient distribution of the human tissue without the contrast agent on both sides of the K-edge is equal,
then
Formula (1) becomes formula (2):
in formula (2), ^ W (r) alphaagent(r) dr is a Radon transform of the weighted contrast agent concentration profile,
and
the normalized X-ray photon numbers measured in the energy spectrum window widths of the left side and the right side of the K-edge are respectively; thus, the contrast agent concentration distribution αagent(r) is calculated by the formula (2); according to the Beer-Lambert theorem, the number of X-ray photons respectively detected by the energy spectrum window widths at two sides of the contrast agent K-edge is expressed by the following formulas (3) and (4):
in the formula (3) and the formula (4), the contrast agent concentration distribution αagent(r) has been solved; the linear attenuation coefficient of a contrast agent may be queried by the national committee for standardization (NIST), and thus
Preferably, in the step (2), the first-order approximation of the human tissue background estimation is brought back to the formula (3) and the formula (4), and a new iteration loop is started to continuously make the human tissue background estimation approach the true value.
It will be understood by those skilled in the art that all or part of the steps in the method of the above embodiments may be implemented by hardware instructions related to a program, the program may be stored in a computer-readable storage medium, and when executed, the program includes the steps of the method of the above embodiments, and the storage medium may be: ROM/RAM, magnetic disks, optical disks, memory cards, and the like. Therefore, corresponding to the method of the present invention, the present invention also includes a deep neural network-based human tissue background estimation apparatus, which is generally expressed in the form of functional modules corresponding to the steps of the method. The device includes:
the training module is configured for training a human tissue attenuation coefficient distribution estimation model based on a deep learning network WGAN-VGG, constructing deep learning training data, reconstructing a human tissue attenuation coefficient distribution image as input data by utilizing first iteration, and inputting an original human tissue CT slice image as sample data into the deep learning network for training;
and the iteration module is configured to replace subsequent iteration steps by using the model after the human tissue attenuation coefficient distribution estimation model is obtained after training is finished, and input the first iteration reconstructed human tissue attenuation coefficient distribution image into the model so as to obtain an output human tissue attenuation coefficient distribution image with higher accuracy.
Examples of the present invention are described in more detail below.
As shown in fig. 2, firstly, training a human tissue attenuation coefficient distribution estimation model based on a deep learning network WGAN-VGG, constructing deep learning training data, reconstructing a human tissue attenuation coefficient distribution image as input data by using first iteration, and inputting an original human tissue CT slice image as sample data into the deep learning network for training; secondly, after the training is finished to obtain the human tissue attenuation coefficient distribution estimation model, the model is used for replacing the subsequent iteration steps, and the human tissue attenuation coefficient distribution image which is reconstructed by the first iteration is input into the model, so that the output human tissue attenuation coefficient distribution image with higher accuracy is obtained.
The attenuation coefficient distribution of human tissues on two sides of the K-edge, which is calculated by the human tissue background estimation method, is only a first-order approximate value, and the human tissue background attenuation coefficient obtained by estimation has poor precision by adding the influence of noise in the reconstruction process. The invention obtains a model from a first-order approximation value to a true value of the human tissue background through the data training of the WGAN-VGG network, and can obtain a more accurate human tissue attenuation coefficient distribution image by replacing iterative computation with the model.
As shown in fig. 2, the whole training network is composed of three parts, wherein the red area in the figure is the first part and is a generator G; the yellow area is a second part and is a VGG network; the blue region is the third part, which is a discriminator D. The generator G is an 8-layer convolutional neural network, in which the first 7 hidden layers each include 32 filters of 3 × 3 size, and the last layer is a 3 × 3 filter as the output of the generator G. Each convolutional layer is followed by a modified Linear unit relu (rectified Linear unit) as an activation function. In the figure, the variable n is the number of filters, and s is the step size when the filters slide on the image. Thus, n32s1 is a 32 image feature map with a sliding step size of 1. The unused Pooling Layer (Pooling Layer) in WGAN-VGG networks is due to the loss of texture and structural information caused by each convolutional Layer followed by another Pooling Layer.
In WGAN-VGG networks, the present study used a pre-trained VGG network as a Loss-aware (Perceptual Loss) calculator. Perceptual loss was proposed by Johnson et al in 2016 to be used for comparison of two similar but different pictures, e.g., a picture shifted by one pixel to the original picture. The perceptual loss is compared with the difference of high-level information among pictures, and the high-level information of the pictures comprises content and global structure. The feature map output by the generator network G and the human tissue background without contrast agent are fed as inputs into the VGG network to extract high-dimensional features. The calculated perceptual loss is in turn fed back to the neural network to update the parameters of the generator network G.
The last part in fig. 2 is the arbiter network D. It is a 6-layer convolutional neural network, each layer containing 64, 128, 256, and 256 3 x 3 filters, respectively. After the two fully-connected layers are used for 6 convolutional layers, 1024 feature maps and 1 feature map are respectively output. Leaky ReLU was used after each convolutional layer as the activation function. The filter sliding step for the odd convolutional layers is 1 pixel, and the filter sliding step for the even convolutional layers is 2 pixels.
The generation of countermeasure networks (GANs) has performed well in many areas of generation tasks, such as music, painting, and human language. However, the generation of the countermeasure networks (GANs) also has the problems of unstable training, difficult convergence and the like, and the problems all bring challenges to the model training of the generation of the countermeasure networks (GANs).
The Wasserstein GAN network is a variant of the generation of countermeasure networks (GANs) to solve the problem of training difficulties. The Wasserstein GAN network replaces JS divergence in the generation countermeasure networks (GANs) by Wasserstein distance to serve as a similarity measurement index of two data distributions, so that the stability of a learning process when gradient descent is used is guaranteed.
By utilizing the WGAN-VGG human tissue background estimation method based on the deep neural network, provided by the invention, the human tissue attenuation coefficient distribution estimation is optimized, a WGAN-VGG deep neural network training model and a human tissue attenuation coefficient distribution estimation training model are built, and the improvement of the human tissue attenuation coefficient distribution estimation effect can be realized. The method is the first application of the deep learning technology in K-edge imaging, improves the accuracy of human tissue attenuation coefficient estimation, and further improves the effect of reconstructing an image by contrast agent concentration.
The core formula of PDKI algorithm reconstruction is shown as formula (1):
based on equation (1), an iterative algorithm is used to estimate the distribution of attenuation coefficients in human tissue
And
the iterative algorithm can simultaneously calculate the concentration of the contrast agent and the attenuation coefficient distribution of human tissues, avoids that a patient receives one more CT flat scan, and effectively reduces the irradiated dose of the patient in the treatment process. The experimental result proves the accuracy and the effectiveness of the iterative algorithm. However, the iterative algorithm also has the problems that the iterative process is time-consuming and is susceptible to imaging noise. In order to solve the problems, the invention provides a method for modeling and estimating the distribution of the attenuation coefficient of the human tissue by using a deep neural network, a model is obtained by carrying out data training by using a WGAN-VGG network based on a deep learning framework TensorFlow, and the model is used for reconstructing the distribution of the attenuation coefficient of the human tissue so as to reconstruct the concentration distribution of the contrast agent.
According to the description of the iterative estimation method of the human tissue background, in the formula (1), within the limited energy spectrum window width of the X-ray photon counting detector, the distribution of the attenuation coefficients of the human tissue without the contrast agent on the two sides of the K-edge can be assumed to be equal, namely, the attenuation coefficients of the human tissue without the contrast agent on the two sides are distributed equally, namely
Then
The formula (1) becomes:
in formula (2), ^ W (r) alphaagent(r) dr is a Radon transform of the weighted contrast agent concentration profile,
and
the normalized number of X-ray photons measured within the spectral window widths of the left and right sides of the K-edge, respectively. Thus, the contrast agent concentration distribution αagent(r) can be obtained from the formula (2). According to the Beer-Lambert theorem, the number of X-ray photons respectively detected by the energy spectrum window widths at two sides of the contrast agent K-edge is as follows:
in the formula (3) and the formula (4), the contrast agent concentration distribution αagent(r) has been solved; the linear attenuation coefficient of a contrast agent may be queried by the national committee for standardization (NIST), and thus
And
can be calculated; i isLAnd IRRespectively detected by X-ray photon counting detectors at the left and right sides of the K-edgeThe number of photons measured. In summary, in the formula (3) and the formula (4), the attenuation coefficient distribution of the human tissue on both sides of the K-edge
And
are each unique unknowns and can thus be calculated.
The attenuation coefficient distribution of the human tissue on two sides of the K-edge calculated by the method is only a first-order approximate value, and the accuracy of the estimated human tissue background attenuation coefficient is poor due to the influence of noise in the reconstruction process. In the analysis of chapter three, the first order approximation of the human tissue background estimation is brought back to formula (3) and formula (4), and a new iteration loop is started to continuously make the human tissue background estimation approach the true value.
The invention obtains a model from a first-order approximation value to a true value of the human tissue background through the data training of the WGAN-VGG network, and can obtain a more accurate human tissue attenuation coefficient distribution image by replacing iterative computation with the model.
The method is the first application of the deep learning technology in K-edge imaging, improves the accuracy of human tissue attenuation coefficient estimation, and further improves the effect of reconstructing an image by contrast agent concentration. Compared with the method for acquiring the attenuation coefficient distribution of the human tissue by pre-scanning, the method for acquiring the attenuation coefficient distribution of the human tissue does not need pre-scanning before injecting the contrast medium to the patient, so that the radiation irradiated dose of the patient can be greatly reduced, and the potential influence of radiation irradiation on the human body is reduced; compared with the iterative calculation estimation of the human tissue attenuation coefficient, the method has better robustness to imaging noise and greatly improves the detection precision and sensitivity.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention in any way, and all simple modifications, equivalent variations and modifications made to the above embodiment according to the technical spirit of the present invention still belong to the protection scope of the technical solution of the present invention.
Claims (10)
1. The human tissue background estimation method based on the deep neural network is characterized by comprising the following steps: which comprises the following steps:
(1) training a human tissue attenuation coefficient distribution estimation model based on a deep learning network WGAN-VGG, constructing deep learning training data, using a first iteration to reconstruct a human tissue attenuation coefficient distribution image as input data, and using an original human tissue CT slice image as sample data to input into the deep learning network for training;
(2) and after training is finished to obtain the human tissue attenuation coefficient distribution estimation model, replacing subsequent iteration steps with the model, and inputting the first iteration reconstructed human tissue attenuation coefficient distribution image into the model so as to obtain an output human tissue attenuation coefficient distribution image with higher accuracy.
2. The deep neural network-based human tissue background estimation method according to claim 1, wherein: in the step (1), the deep learning network includes: the device comprises a generator, a VGG network and a discriminator.
3. The deep neural network-based human tissue background estimation method according to claim 2, wherein: in the step (1), the generator is an 8-layer convolutional neural network, wherein each layer of the first 7 hidden layers comprises 32 filters with the size of 3 × 3, and the last layer is a 3 × 3 filter as the output of the generator G; after each convolutional layer, a modified linear unit ReLU is used as the activation function.
4. The deep neural network-based human tissue background estimation method according to claim 3, wherein: in the step (1), a pre-trained VGG network is used as a calculator of the perception loss, the feature map output by the generator network G and the human tissue background without contrast agent are used as input and fed into the VGG network to extract high-dimensional features, and the perception loss obtained by calculation is fed back to the neural network to update the parameters of the generator network G.
5. The deep neural network-based human tissue background estimation method according to claim 4, wherein: in the step (1), the discriminator network D is a 6-layer convolutional neural network, each layer including 64, 128, 256 and 256 filters of 3 × 3, respectively; after the two full-connection layers are used for 6 convolutional layers, respectively outputting 1024 characteristic maps and 1 characteristic map; using Leaky ReLU as an activation function after each convolutional layer; the filter sliding step for the odd convolutional layers is 1 pixel, and the filter sliding step for the even convolutional layers is 2 pixels.
6. The deep neural network-based human tissue background estimation method according to claim 5, wherein: the step (1) comprises the following sub-steps:
(1.1) optimizing the estimation of the attenuation coefficient distribution of human tissues;
(1.2) building a WGAN-VGG deep neural network training model;
and (1.3) training a human tissue attenuation coefficient distribution estimation model.
7. The deep neural network-based human tissue background estimation method according to claim 6, wherein: in the step (1.1), the core formula of the PDKI algorithm reconstruction is formula (1)
Based on equation (1), an iterative algorithm is used to estimate the distribution of attenuation coefficients in human tissue
And
8. the method according to claim 7The human tissue background estimation method of the deep neural network is characterized by comprising the following steps: in the step (1.2), in the formula (1), in the limited energy spectrum window width of the X-ray photon counting detector, if the attenuation coefficient distribution of human tissues without contrast agents on the two sides of the K-edge is equal,
then
Formula (1) becomes formula (2):
in formula (2), ^ W (r) alphaagent(r) dr is a Radon transform of the weighted contrast agent concentration profile,
and
the normalized X-ray photon numbers measured in the energy spectrum window widths of the left side and the right side of the K-edge are respectively; thus, the contrast agent concentration distribution αagent(r) is calculated by the formula (2); according to the Beer-Lambert theorem, the number of X-ray photons respectively detected by the energy spectrum window widths at two sides of the contrast agent K-edge is expressed by the following formulas (3) and (4):
in the formula (3) and the formula (4), the contrast agent concentration distribution αagent(r) has been solved; the linear attenuation coefficient of contrast agents may be characterized by the United states standardThe committee for standardization (NIST) query, and thus
And
calculating to obtain; i isLAnd IRThe number of photons detected by X-ray photon counting detectors on the left side and the right side of the K-edge respectively; in summary, in the formula (3) and the formula (4), the attenuation coefficient distribution of the human tissue on both sides of the K-edge
And
are each unique unknowns and can thus be calculated.
9. The deep neural network-based human tissue background estimation method according to claim 8, wherein: in the step (2), the first-order approximation of the human tissue background estimation is brought back to the formula (3) and the formula (4), and a new iteration loop is started to make the human tissue background estimation approach the true value continuously.
10. Human tissue background estimation device based on deep neural network, its characterized in that: it includes:
the training module is configured for training a human tissue attenuation coefficient distribution estimation model based on a deep learning network WGAN-VGG, constructing deep learning training data, reconstructing a human tissue attenuation coefficient distribution image as input data by utilizing first iteration, and inputting an original human tissue CT slice image as sample data into the deep learning network for training;
and the iteration module is configured to replace subsequent iteration steps by using the model after the human tissue attenuation coefficient distribution estimation model is obtained after training is finished, and input the first iteration reconstructed human tissue attenuation coefficient distribution image into the model so as to obtain an output human tissue attenuation coefficient distribution image with higher accuracy.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011211249.9A CN112396579A (en) | 2020-11-03 | 2020-11-03 | Human tissue background estimation method and device based on deep neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011211249.9A CN112396579A (en) | 2020-11-03 | 2020-11-03 | Human tissue background estimation method and device based on deep neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112396579A true CN112396579A (en) | 2021-02-23 |
Family
ID=74597847
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011211249.9A Pending CN112396579A (en) | 2020-11-03 | 2020-11-03 | Human tissue background estimation method and device based on deep neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112396579A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113962984A (en) * | 2021-11-15 | 2022-01-21 | 北京航空航天大学 | Proton CT imaging method and device based on deep learning algorithm and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110097512A (en) * | 2019-04-16 | 2019-08-06 | 四川大学 | Construction method and the application of the three-dimensional MRI image denoising model of confrontation network are generated based on Wasserstein |
| US20200034948A1 (en) * | 2018-07-27 | 2020-01-30 | Washington University | Ml-based methods for pseudo-ct and hr mr image estimation |
| WO2020214911A1 (en) * | 2019-04-19 | 2020-10-22 | Yale University | Method and system for generating attenuation map from spect emission data based upon deep learning |
-
2020
- 2020-11-03 CN CN202011211249.9A patent/CN112396579A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200034948A1 (en) * | 2018-07-27 | 2020-01-30 | Washington University | Ml-based methods for pseudo-ct and hr mr image estimation |
| CN110097512A (en) * | 2019-04-16 | 2019-08-06 | 四川大学 | Construction method and the application of the three-dimensional MRI image denoising model of confrontation network are generated based on Wasserstein |
| WO2020214911A1 (en) * | 2019-04-19 | 2020-10-22 | Yale University | Method and system for generating attenuation map from spect emission data based upon deep learning |
Non-Patent Citations (3)
| Title |
|---|
| BO MENG, ETC.: "Energy Window Optimization for X-Ray K-Edge Tomographic Imaging", 《IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING》 * |
| 冯永等: "GP-WIRGAN：梯度惩罚优化的Wasserstein图像循环生成对抗网络模型", 《计算机学报》 * |
| 王丽新等: "双能CT成像的数值仿真", 《航天医学与医学工程》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113962984A (en) * | 2021-11-15 | 2022-01-21 | 北京航空航天大学 | Proton CT imaging method and device based on deep learning algorithm and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Harms et al. | Paired cycle‐GAN‐based image correction for quantitative cone‐beam computed tomography | |
| Bahrami et al. | Comparison of different deep learning architectures for synthetic CT generation from MR images | |
| US10217216B2 (en) | Kind of x-ray chest image rib suppression method based on poisson model | |
| JP4170767B2 (en) | Image processing device | |
| CN110753935A (en) | Dose reduction using deep convolutional neural networks for medical imaging | |
| CN111899188B (en) | Neural network learning cone beam CT noise estimation and suppression method | |
| Han et al. | A dual-encoder-single-decoder based low-dose CT denoising network | |
| Zhang et al. | Low-dose lung CT image restoration using adaptive prior features from full-dose training database | |
| Ote et al. | Kinetics-induced block matching and 5-D transform domain filtering for dynamic PET image denoising | |
| Qi et al. | A quantitative study of motion estimation methods on 4D cardiac gated SPECT reconstruction | |
| Izadi et al. | Enhanced direct joint attenuation and scatter correction of whole-body PET images via context-aware deep networks | |
| CN112529977B (en) | PET image reconstruction method and system | |
| Li et al. | Unpaired low‐dose computed tomography image denoising using a progressive cyclical convolutional neural network | |
| Guo et al. | Graph filtering approach to PET image denoising | |
| CN112396579A (en) | Human tissue background estimation method and device based on deep neural network | |
| CN111477304A (en) | Tumor irradiation imaging combination method for fusing PET (positron emission tomography) image and MRI (magnetic resonance imaging) image | |
| CN111968192A (en) | Construction method of CT image, CT device and storage medium | |
| Guo et al. | Kernel graph filtering—A new method for dynamic sinogram denoising | |
| CN116342444A (en) | Dual-channel multi-mode image fusion method and fusion imaging terminal | |
| Roychowdhury et al. | Blind analysis of CT image noise using residual denoised images | |
| Xia et al. | Dynamic controllable residual generative adversarial network for low-dose computed tomography imaging | |
| Liu et al. | Segmentation for multimodal brain tumor images using dual-tree complex wavelet transform and deep reinforcement learning | |
| Gu et al. | Cross-modality image translation: CT image synthesis of MR brain images using multi generative network with perceptual supervision | |
| Huang et al. | The Effective 3D MRI Reconstruction Method Driven by the Fusion Strategy in NSST Domain | |
| Mahmoud et al. | Variant Wasserstein Generative Adversarial Network Applied on Low Dose CT Image Denoising. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210223 |