EP4623405A1 - Prior knowledge aware iterative denoising neural network for photon counting detector computed tomography - Google Patents

Abstract

Images acquired using computed tomography ("CT") imaging, such as multi-energy CT or spectral CT acquisitions, are denoised using a neural network that has been trained using prior knowledge iterative denoising, or other prior image information, in the training process. Noisy images and a lower-noise prior image are then input to the trained neural network to generate the denoised image(s). The neural network can be trained on noise-only images, signal-only images, and a prior image all generated from the input CT image data. The neural network can be iteratively trained by updating the training data in each iteration using the output of the neural network as an updated signal-only image.

Description

PRIOR KNOWLEDGE AWARE ITERATIVE DENOISING NEURAL NETWORK FOR PHOTON COUNTING DETECTOR COMPUTED TOMOGRAPHY

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Patent Application Serial No. 63/385,032, filed on November 27, 2022, and entitled “PRIOR KNOWLEDGE AWARE ITERATIVE DENOISING NEURAL NETWORK FOR PHOTON COUNTING DETECTOR COMPUTED TOMOGRAPHY,” which is herein incorporated by reference in its entirety.

STATEMENT OF FEDERALLY SPONSORED RESEARCH

[0002] This invention was made with government support under EB028590 awarded by the National Institutes of Health. The government has certain rights in the invention.

BACKGROUND

[0003] An advantage of photon-counting detector (“PCD”) computed tomography (“CT”) is the simultaneous acquisition of multi-energy data, from which virtual monoenergetic images (“VMIs”) can be produced at a high spatial resolution. However, noise levels observed in the ultra-high-resolution (“UHR”) VMIs are markedly increased.

[0004] Recent work involving deep learning methods has shown great potential in CT image denoising. Many convolutional neural network (“CNN”) applications involve training using spatially co-registered low-dose and high-dose CT images featuring high and low image noise, respectively. However, this training scheme is impractical in routine clinical practice. Further, typical denoising methods treat each VMI energy level independently, without consideration of the valuable information for denoising in the spectral domain.

SUMMARY OF THE DISCLOSURE

[0005] The present disclosure addresses the aforementioned drawbacks by providing a method for generating a denoised image of a subject from computed tomography (“CT”) image data acquired from the subject using a CT system. The method includes accessing CT image data with a computer system, where the CT image data have been acquired from the subject using the CT system. A prior image of the subject is generated from the CT image data using the computer system, where the prior image includes lower noise than other images in the CT image data. A neural network is accessed with the computer system, where the neural network has been trained on training data to generate a denoised image from CT image data. The CT image data and the prior image are input to the neural network using the computer system, generating output data as a denoised image having reduced noise as compared to the CT image data. The denoised image can be displayed to a user, or stored for later use, with the computer system.

[0006] It is another aspect of the present disclosure to provide a method for training a neural network to denoise a CT acquired from a subject using a CT system. The method includes accessing multi-energy CT image data with a computer system, where the multienergy CT image data have been acquired from the subject using the CT system. A prior image of the subject is generated from the multi-energy CT image data using the computer system, where the prior image comprises lower noise than other images in the multi-energy CT image data. A signal-only image is generated from the CT image data using the computer system, and a noise-only image is generated from the CT image data and the signal-only image using the computer system. A training data set is assembled from the noise-only image, the signal-only image, and the prior image using the computer system. A neural network is then trained on the training data set using the computer system, generating a trained neural network as an output. The trained network is stored with the computer system for later use.

[0007] The foregoing and other aspects and advantages of the present disclosure will appear from the following description. In the description, reference is made to the accompanying drawings that form a part hereof, and in which there is shown by way of illustration one or more embodiments. These embodiments do not necessarily represent the full scope of the invention, however, and reference is therefore made to the claims and herein for interpreting the scope of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

[0008] FIG. 1 is a flowchart setting for the steps of an example method for generating a denoised CT image by inputting CT image data (e.g., noisy images) and a prior image (e.g., a lower noise image generated from the CT image data) to a trained neural network.

[0009] FIG. 2 is an illustration of the high-resolution photon-counting detector CT virtual monoenergetic imaging ("VMI") characteristics at different energies.

[0010] FIG. 3 is an example of a U-Net architecture that can be implemented for a neural network in accordance with some embodiments described in the present disclosure. [0011] FIG. 4 is a flowchart setting forth the steps of an example method for training a neural network to denoise a CT image based on a prior knowledge aware iterative denoising neural network (“PKAID-Net”) framework.

[0012] FIGS 5A-5C illustrate an overview of an example workflow implementing a PKAID-Net framework. FIG. 5 A shows an overall pipeline of PKAID-Net. FIG. 5B shows an overview of training a PKAID-Net Module. FIG. 5C shows a simplified inference process via a well-trained CNN-M network from the PAKID-Net Module.

[0013] FIGS. 6A and 6B illustrate an overview of a non-iteratively trained PKAID- Net. FIG. 6A shows an overall pipeline of the PKAID-Net. FIG. 6B shows a simplified inference process via a well-trained CNN from the framework.

[0014] FIG. 7 illustrates a denoising performance comparison for a representative slice of 50keV VMI data from one patient in an example study. Images from columns left to right are FBP, IR, PKAID-Net (M=l), and PKAID-Net (M=3), where M represents the total number of iterations. The regions of interest marked by the yellow rectangle are zoomed below, respectively. Image display window (WW/WL): 400/1500 HU.

[0015] FIG. 8 is a block diagram of an example system for generating a denoised CT image using a PKAID-Net framework as described in some embodiments of the present disclosure.

[0016] FIG. 9 is a block diagram of example components that can implement the system of FIG. 8.

DETAILED DESCRIPTION

[0017] Described here systems and method for denoising images acquired using computed tomography (“CT”) imaging, such as multi-energy CT or spectral CT acquisitions, or the like. Images are denoised using a neural network that is trained using prior knowledge iterative denoising, or other prior image information, in the training process. Noisy images and a lower-noise prior image are then input to the trained neural network to generate the denoised image(s) of interest.

[0018] In one aspect, high-resolution, but noisy, virtual monoenergetic images (“VMIs”) are generated from CT image data. These high-resolution VMIs can be denoised using the systems and methods described in the present disclosure. Similarly, the systems and methods described in the present disclosure can be adapted for noise reduction of other spectral CT applications, including energy-thresholded images, basis material decomposition images, virtual non-contrast images, iodine overlay images, bone/calcium subtraction images, and so on.

[0019] As mentioned above, as one example, an image-based prior knowledge-aware iterative denoising neural network ("PK AID-Net") framework is provided to iteratively create refined datasets for training a better denoising convolutional neural network (“CNN”). In this framework, a lower-noise VMI can be treated as a prior input channel and the training targets are updated to improve the trained CNN denoising performance. The data preparation used for the PKAID-Net module training can include creating and spatially decoupling a “noise-only” image, which mitigates overfitting and improves randomization; slice averaging to produce a thicker (i.e., lower noise) slice for the signal-only and prior images; using thick reference images as training inputs, with reinsertion of spatially decoupled noise-only images and the refined prior images; and using training targets that include the corresponding thick reference images without noise insertion.

[0020] In this framework, the denoised image from the previous training iteration is used as an updated signal-only target image, which is included in the dataset for the next training iteration. This process can be implemented iteratively and can be used to gradually train high-performing denoising models.

[0021] Referring now to FIG. 1, a flowchart is illustrated as setting forth the steps of an example method for generating denoised CT image data using a suitably trained neural network or other machine learning algorithm. As will be described, the neural network or other machine learning algorithm takes CT image data and a prior image generated from the CT image data as input data and generates denoised CT image data as output data.

[0022] The method includes accessing CT image data with a computer system, as indicated at step 102. Accessing the CT image data may include retrieving such data from a memory or other suitable data storage device or medium. Additionally or alternatively, accessing the CT image data may include acquiring such data with a CT system and transferring or otherwise communicating the data to the computer system, which may be a part of the CT system.

[0023] The CT image data may include images reconstructed from data acquired from a subject using a CT system. Additionally or alternatively, the CT image data may include raw data acquired from a subject using a CT system. Images can then be reconstructed from the raw data, resulting in high-resolution, but noisy, images. The images can be reconstructed using any suitable reconstruction algorithm, including traditional analytical reconstruction algorithms (e.g., filtered backproj ection), iterative reconstruction algorithms, and the like.

[0024] In general, the CT image data can be multi-energy CT image data. As one example, the CT image data can be acquired using a photon counting detector (“PCD”) CT system. Advantageously, PCD-CT enables the simultaneous acquisition of multi-energy data, which can produce virtual monoenergetic images (“VMIs”) at a high spatial resolution. As also demonstrated in dual-energy CT, VMIs at different energies (e.g., different keVs) have distinct contrast and noise properties. Lower keV VMIs have been used to improve image contrast, while higher keV VMIs have been used to reduce beam hardening and metal artifacts. In addition, image noise varies among different keV VMIs.

[0025] In some embodiments, accessing the CT image data includes accessing multienergy CT image data and generating VMIs from the multi-energy CT image data. The VMIs can then be stored with the CT image data for subsequent processing. In some implementations, VMIs are generated from the CT image data and are stored as the CT image data that will be processed for denoising.

[0026] A low-noise prior image is generated from the CT image data, as indicated at step 104. In some embodiments, the low-noise prior image is generated by generating VMIs from the CT image data and selecting the VMI with the lowest noise as the prior image. For instance, a 70 keV (approximately the effective energy’ of 120 kV beam) VMI was observed to have the lowest noise among all VMI energies as shown in FIG. 2. As also shown in FIG. 2, the 50 keV and 100 keV VMIs showed the characteristics of increased image contrast and decreased calcium blooming, but higher noise, which can thus be chosen as the noisy VMI inputs to be denoised.

[0027] Additionally or alternatively, when the CT image data include multi-energy CT image data, the low-noise prior image can be generated from the full energy spectrum. As one example, the low-noise prior image can be generated as a low-energy' threshold image (e.g., a T3D image based on photon counts above 20 keV or another low energy threshold level).

[0028] A trained neural network (or other suitable machine learning algorithm) is then accessed with the computer system, as indicated at step 106. In general, the neural network is trained, or has been trained, on training data in order to denoise CT images based on an input of CT image data and a low-noise prior image. Accessing the trained neural network may include accessing network parameters (e.g., weights, biases, or both) that have been optimized or otherwise estimated by training the neural network on training data. In some instances. retrieving the neural network can also include retrieving, constructing, or otherwise accessing the particular neural network architecture to be implemented. For instance, data pertaining to the layers in the neural network architecture (e.g., number of layers, type of layers, ordering of layers, connections between layers, hyperparameters for layers) may be retrieved, selected, constructed, or otherwise accessed. Additionally or alternatively, accessing the trained neural network may include training the neural network based on the CT image data accessed in step 102. A method for training a neural network in accordance with some embodiments of the present disclosure is described below in more detail with respect to FIG. 4.

[0029] An artificial neural network generally includes an input layer, one or more hidden layers (or nodes), and an output layer. Typically, the input layer includes as many nodes as inputs provided to the artificial neural network. The number (and the type) of inputs provided to the artificial neural network may vary based on the particular task for the artificial neural network.

[0030] The input layer connects to one or more hidden layers. The number of hidden layers varies and may depend on the particular task for the artificial neural network. Additionally, each hidden layer may have a different number of nodes and may be connected to the next layer differently. For example, each node of the input layer may be connected to each node of the first hidden layer. The connection between each node of the input layer and each node of the first hidden layer may be assigned a weight parameter. Additionally, each node of the neural network may also be assigned a bias value. In some configurations, each node of the first hidden layer may not be connected to each node of the second hidden layer. That is, there may be some nodes of the first hidden layer that are not connected to all of the nodes of the second hidden layer. The connections between the nodes of the first hidden layers and the second hidden layers are each assigned different weight parameters. Each node of the hidden layer is generally associated with an activation function. The activation function defines how the hidden layer is to process the input received from the input layer or from a previous input or hidden layer. These activation functions may vary and be based on the type of task associated with the artificial neural network and also on the specific type of hidden layer implemented.

[0031] Each hidden layer may perform a different function. For example, some hidden layers can be convolutional hidden layers which can, in some instances, reduce the dimensionality of the inputs. Other hidden layers can perform statistical functions such as max pooling, which may reduce a group of inputs to the maximum value; an averaging layer; batch normalization; and other such functions. In some of the hidden layers each node is connected to each node of the next hidden layer, which may be referred to then as dense layers. Some neural networks including more than, for example, three hidden layers may be considered deep neural networks.

[0032] The last hidden layer in the artificial neural network is connected to the output layer. Similar to the input layer, the output layer typically has the same number of nodes as the possible outputs.

[0033] In general, the trained neural network(s) can implement any number of different neural network architectures. For instance, the neural network(s) could implement a convolutional neural network, a residual neural network, or the like. Alternatively, the neural network(s) could be replaced with other suitable machine learning or artificial intelligence algorithms, such as those based on supervised learning, unsupervised learning, deep learning, ensemble learning, dimensionality' reduction, and so on.

[0034] As one non-limiting example, the trained neural network may be a convolutional neural network based on a simplified U-Net architecture with nine blocks, such as the neural network architecture illustrated in FIG. 3. Each block includes convolution, batch normalization (“BN”), and exponential linear unit (“eLU”) activation operations sequentially. The max pooling layer and convolution transpose operator are applied in the network. The concatenation is added to the network to preserve the similarity between the input and output.

[0035] The CT image data and prior image are then input to the one or more trained neural networks, generating output data as denoised CT image data, as indicated at step 108. The denoised CT image data generated by inputting the CT image data to the trained neural network(s) can then be displayed to a user, stored for later use or further processing, or both, as indicated at step 110.

[0036] Referring now to FIG. 4, a flowchart is illustrated as setting forth the steps of an example method for training one or more neural networks (or other suitable machine learning algorithms) on training data, such that the one or more neural networks are trained to receive CT image data and a low-noise prior image as input data in order to generate denoised CT image as output data.

[0037] The method includes accessing CT image data with a computer system, as indicated at step 402. Accessing the CT image data may include retrieving such data from a memory or other suitable data storage device or medium. Alternatively, accessing the training data may include acquiring such data with a CT system and transferring or otherwise communicating the data to the computer system, which may be a part of the CT system.

[0038] Similar to the inference method described above, the CT image data may include images reconstructed from data acquired from a subject using a CT system. Additionally or alternatively, the CT image data may include raw data acquired from a subject using a CT system. Images can then be reconstructed from the raw data, resulting in high- resolution, but noisy, images. The images can be reconstructed using any suitable reconstruction algorithm, including traditional analytical reconstruction algorithms (e.g., filtered backproj ection), iterative reconstruction algorithms, and the like.

[0039] In general, the CT image data can be multi-energy CT image data. In some embodiments, accessing the CT image data includes accessing multi-energy CT image data and generating VMIs from the multi -energy CT image data. The VMIs can then be stored with the CT image data for subsequent processing.

[0040] A low-noise prior image is generated from the CT image data, as indicated at step 404. In some embodiments, the low-noise prior image is generated by generating VMIs from the CT image data and selecting the VMI with the lowest noise as the prior image. In general, a low noise VMI may be selected based on the effective energy of the x-ray beam used when acquiring the CT image data. For instance, for a 120 kV x-ray beam, a 70 keV VMI can be selected as the prior image since it is close to the effective energy of the x-ray beam. When an x-ray beam with a kV different from 120 kV is used, a VMI other than the 70 keV VMI can be selected as the prior image by selecting the VMI that is close to the effective energy of the x-ray beam. Additionally or alternatively, the low-noise prior image can be generated from the full energy spectrum. As one example, the low-noise prior image can be generated as a low- energy threshold image (e.g., a T3D image based on photon counts above 20 keV or another low energy threshold level).

[0041] A signal-only image is also generated from the CT image data, as indicated at step 406. As one example, the signal-only image can be initialized by performing an iterative reconstruction on the CT image data.

[0042] A noise-only image is then generated using the CT image data and the signal- only image, as indicated at step 408. For example, the noise-only image can be generated by subtracting a noisy image from the CT image data and the signal-only image. In some embodiments, spatial decoupling can be performed on the noise-only image to reduce the risk of overfitting and improve randomization. As an example, spatial decoupling can be performed based on a random translation of the noise-only image within the image plane (e.g., the axial plane) by a selected or random number of pixels (e.g., 1 to 16 pixels). Additionally or alternatively, a random inversion can be applied (e.g., multiplying by +1 or -1) to the noise- only image, which in some instances may include applying a random inversion on a pixel-by- pixel basis. A random noise image can be generated and stored as the noise-only image using this process.

[0043] Training data are then assembled from the CT noise-only image, the signal-only image, and the prior image, as indicated at step 410. In some embodiments, the signal-only image and the prior image can first be refined. A refined signal-only image and a refined prior image can be generated, for example, by averaging a number of adjacent signal-only images and prior images, respectively, to simulate thicker slice reference images with relatively low noise. For instance, five adjacent images can be averaged. Assembling the training data can then include extracting image patches from the noise-only image and the signal-only and prior images (or the refined signal only and refined prior images) using a patch extraction. The training target can be set as the signal-only (or refined signal-only) image patch without added noise.

[0044] One or more neural networks (or other suitable machine learning algorithms) are trained on the training data, as indicated at step 412. In general, the neural network can be trained by optimizing network parameters (e.g., weights, biases, or both) based on minimizing a loss function. As one non-limiting example, the loss function may be a mean squared error loss function. The inputs to training the neural network include the prior (or refined prior) image patches, and synthetic noisy image patches generated by multiplying the noise-only image patches by a weight factor and then combining the weighted noise-only image patches with the signal-only (or refined signal-only) image patches.

[0045] Training a neural network may include initializing the neural network, such as by computing, estimating, or otherwise selecting initial network parameters (e.g., weights, biases, or both). During training, an artificial neural network receives the inputs for a training example and generates an output using the bias for each node, and the connections between each node and the corresponding weights. For instance, training data can be input to the initialized neural network, generating output as a denoised CT image. The artificial neural network then compares the generated output w ith the actual output of the training example in order to evaluate the quality of the denoised CT image. For instance, the denoised CT image can be passed to a loss function to compute an error. The current neural network can then be updated based on the calculated error (e.g., using backpropagation methods based on the calculated error). For instance, the current neural network can be updated by updating the network parameters (e g., weights, biases, or both) in order to minimize the loss according to the loss function. The training continues until a training condition is met. The training condition may correspond to, for example, a predetermined number of training examples being used, a minimum accuracy threshold being reached during training and validation, a predetermined number of validation iterations being completed, and the like. When the training condition has been met (e.g., by determining whether an error threshold or other stopping criterion has been satisfied), the current neural network and its associated network parameters represent the trained neural network. Different types of training processes can be used to adjust the bias values and the weights of the node connections based on the training examples. The training processes may include, for example, gradient descent, Newton's method, conjugate gradient, quasi -Newton. Levenberg-Marquardt, among others.

[0046] In some embodiments, the training data set can be iteratively updated to refine the neural network. In these instances, a determination is made at decision block 414 whether training of the neural netw ork should be iterated based on an updated training data set. If so, then the current output of the neural network is stored as an updated signal-only image, as indicated at step 416, and steps 408-412 are repeated using the updated signal-only image to generate an updated noise-only image, assemble updated training data, and train the neural network on the updated training data. The number of training iterations can be set by a user, or based on satisfying a stopping criterion. As one example, the number of training iterations can be a preselected number of iterations, such as M = 3 iterations.

[0047] When the training condition is met. the one or more trained neural networks are then stored for later use. as indicated at step 418. Storing the neural network(s) may include storing network parameters (e g., weights, biases, or both), which have been computed or otherwise estimated by training the neural network(s) on the training data. Storing the trained neural network(s) may also include storing the particular neural netw ork architecture to be implemented. For instance, data pertaining to the layers in the neural network architecture (e.g., number of layers, type of layers, ordering of layers, connections between layers, hyperparameters for layers) may be stored.

[0048] Multiple variations can be made to the denoising framew ork described in the present disclosure. For example, loss functions focused on preserving noise properties of the prior image (e.g., 70 keV VMI, full spectrum image) could improve performance. Further noise reduction could also be achieved by adding noise to the prior image (e.g.. 70 keV VIM, full spectrum image) during training. Further noise reduction could also be achieved by denoising the prior image (e.g., 70 keV VMI, full spectrum image) prior to using it as an input to the trained neural network.

[0049] In an example study, a prior knowledge-aware iterative denoising neural network (“PKAID-Nef ’) according to some embodiments described in the present disclosure was used to denoise images acquired using a PCD-CT, focusing on ultra-high-resolution (‘ UHR”), multi-energy cardiac imaging. This approach fully utilized the data acquired in the spectral domain and iteratively constructed refined datasets for neural network training to improve the denoising performance. The method only required noisy and prior VMI data in the image domain, which are easily accessible in clinical scanners.

[0050] Advantageously, this denoising method described in the present disclosure was able to fully utilize PCD-CT image data in the spectral domain, allowing iterative refinement of training datasets, and which enabled image domain denoising deployment upon clinical scans. The experimental results testing on cardiac CTA patient data demonstrated that the PKAID-Net framework enables significant noise reduction while maintaining spatial and spectral fidelity of the UHR PCD-CT in clinical exams.

[0051] An overall illustration of the PKAID-Net framework implemented in this example study is shown in FIGS. 5A-5C. A non-iterative implementation is illustrated in FIGS. 6A (data acquisition, data preprocessing, and neural network training) and FIG. 6B (neural network inference). Given a series of PCD-CT VMI data, PKAID-Net first divided them into two inputs: the high-noise UHR VMI data input and the prior VMI data input. The 70 keV VMI was observed to have the lowest noise among all VMI energies, and was therefore selected for use as the prior VMI data input in this study. The characteristics of increased image contrast and decreased calcium blooming can be found in 50 keV and 100 keV images, while the noise is higher in both images. The 50 keV or 100 keV VMI reconstructed by the FBP method was chosen as the noisy VMI input.

[0052] As shown in FIG. 5A, for the initialization of creating the training dataset, an iterative reconstruction (HR”) method was first applied to the noisy image to generate the initialized signal-only image. The noisy, signal-only, and prior images were then fed into the PKAID-Net Module to train a CNN. At each iteration m (m = 1, ..., M), where M represents the total number of iterations, the denoised image from the previous trained CNN-///-/ was used as a new signal-only target of the dataset for the next round of PKAID-Net Module-/// training. A well-trained CNN will be obtained until a stable denoising performance is achieved during the iterations. For the inference of each trained CNN, the PCD-CT data with the noisy FBP and prior images were used as the inputs, as shown in FIG. 5C and/or FIG. 6B.

[0053] As show n in FIG. 5B, in the PKAID-Net Module for training a CNN, the noisy and signal-only images were first subtracted to generate images composed predominately of noise, called noise-only images. Spatial decoupling on the noise-only image was used to avoid overfitting and improve the randomization, which was defined as a random translation in the axial plane (1 to 16 pixels) and a random inversion (1 or -1 multiplier) of the noise-only image. A random noise image can be obtained with the above process. Meanwhile, five adjacent signal-only images and prior images were averaged to simulate thicker slice reference images as the refined signal-only and prior images with relatively low noise. Patch extractions were applied to these available images (noise-only, refined signal-only, and prior images) to create the training dataset.

[0054] The training target included the refined signal-only patch without added noise. The inputs to training a CNN included two channels: (1) noise-only images multiplied by the weight factor superimposed onto the refined signal-only patches, and (2) prior patches. Denoising strength was controlled by the introduced w eight factor of the noise patch. To this end, a CNN can be trained with the created dataset as an output of the PKAID-Net Module.

[0055] A simplified U-Net architecture with nine blocks was used in this study, such as the architecture shown in FIG. 3. As descried above, each block involves convolution, batch normalization (“BN”), and exponential linear unit (“eLU”) activation operations sequentially. The max pooling layer and convolution transpose operator were applied in the network. The concatenation was added to the network to preserve the similarity' betw een the input and output. [0056] The mean-squared-error (“MSE”) loss function was optimized during the training. To this end, a CNN can be trained as an output of the proposed framework.

[0057] In an example study to validate the PKAID-Net framework, coronary CTA exams from ten patients were acquired on a PCD-CT system and were included in this study. Scans were performed using the following parameters: prospective ECG-gated adaptive sequential mode, 120 kV, two energy thresholds (25 and 65 keV), 144 x 0.4 mm collimation, and 0.25 second rotation time. Automatic exposure control (“AEC”) was turned on, with CAREkeV optimized for vascular exams and an Image Quality⁷ (“IQ”) level of 32. VMIs at 50, 70, and 100 keV were reconstructed with both FBP and IR (QIR. strength level 4), with 1024 x 1024 matrix in the 200 mm field of view (FOV), a sharp vascular kernel (Bv68), 0.6 mm slice thickness, and 0.3 mm increment.

[0058] These patient images were divided into training (6 patients), validation (1 patient), and testing (3 patients) datasets. In this study, 50 or 100 keV images were used as the noisy VMI input and 70 keV images were used as the prior. For the network training, 144,000 patches were randomly extracted with a size of 128 x 128 pixels from the training data for each image type (noisy, signal-only, and prior) and 16,000 patches were extracted from the validation data by the ratio of 9: 1. The CNN model was built with TensorFlow 1. 15 on Python 3.6 and trained with an NVIDIA TITAN RTX (24-GB GPU memory ). The initial learning rate was set as 0.001 with a scheduled descent to 0.00001, and the Adam optimizer was chosen to minimize the loss function.

[0059] FIG. 7 shows the denoised results for a representative slice from the 50 keV VMI as observed with different algorithms. The testing data used was extracted from three patient datasets. Considering the UHR PCD-CT images using a Bv68 sharp kernel and 0.6 mm slice thickness, the PKAID-Net framework described in the present disclosure exhibited better visual quality' in terms of detail preservation and noise removal, compared to the standard IR. Images from PKAID-Net (M=l) represent the output of the 1st iteration of the model, which is equivalent to a non-iterative version of the CNN denoising. Substantial noise reduction has been achieved in these images, more than that of IR. Further noise reduction and detail enhancement was observed at higher iterations, e.g., PKAID-Net (M=3). The red arrow in FIG. 7 indicates that the PKAID-Net (M=3) better preserves structural edges while maintaining spatial resolution by comparison to the other variants.

[0060] The proposed method was evaluated quantitatively by measuring the mean and standard deviation (“STD”) values within a region-of-interest (“ROI”) placed on the aorta (red circle in FIG. 7). Values are summarized in Table 1.

Table 1. DENOISING PERFORMANCE comparison of between FBP, IR, and PKAID-Net, Unit: HU.

[0061] It is indicated that PKAID-Net (M=3) can remove 96% (41/1292) and 70% (41/154) noise relative to the FBP and IR method. The performance of different iterations for the PKAID-Net were compared and the results demonstrated that optimal performance can be achieved with a small number of iterations (M=3). after which the difference became negligible. Additionally, the mean attenuation value was similar to that of the FBP and IR methods, which verified the robustness of the PKAID-Net at maintaining the CT numbers and spectral information (as CT numbers of VMIs represent the spectral property of the images).

[0062] Referring now to FIG. 8. an example of a system 800 for generating denoised CT images in accordance with some embodiments of the systems and methods described in the present disclosure is shown. As shown in FIG. 8, a computing device 850 can receive one or more types of data (e.g., CT image data, training data, trained neural network parameters) from data source 802. In some embodiments, computing device 850 can execute at least a portion of a prior knowledge aware iterative denoising neural network (“PKAID-Net”) image denoising system 804 to generate denoised CT images from data received from the data source 802.

[0063] Additionally or alternatively, in some embodiments, the computing device 850 can communicate information about data received from the data source 802 to a server 852 over a communication network 854, which can execute at least a portion of the PKAID-Net image denoising system 804. In such embodiments, the server 852 can return information to the computing device 850 (and/or any other suitable computing device) indicative of an output of the PKAID-Net image denoising system 804.

[0064] In some embodiments, computing device 850 and/or server 852 can be any suitable computing device or combination of devices, such as a desktop computer, a laptop computer, a smartphone, a tablet computer, a wearable computer, a server computer, a virtual machine being executed by a physical computing device, and so on. The computing device 850 and/or server 852 can also reconstruct images from the data.

[0065] In some embodiments, data source 802 can be any suitable source of data (e.g., measurement data, images reconstructed from measurement data, processed image data), such as a CT system, another computing device (e.g., a server storing measurement data, images reconstructed from measurement data, processed image data), and so on. In some embodiments, data source 802 can be local to computing device 850. For example, data source 802 can be incorporated with computing device 850 (e.g., computing device 850 can be configured as part of a device for measuring, recording, estimating, acquiring, or otherwise collecting or storing data). As another example, data source 802 can be connected to computing device 850 by a cable, a direct wireless link, and so on. Additionally or alternatively, in some embodiments, data source 802 can be located locally and/or remotely from computing device 850, and can communicate data to computing device 850 (and/or server 852) via a communication network (e.g., communication network 854).

[0066] In some embodiments, communication network 854 can be any suitable communication network or combination of communication networks. For example, communication network 854 can include a Wi-Fi network (which can include one or more wireless routers, one or more switches, etc.), a peer-to-peer network (e.g., a Bluetooth network), a cellular network (e.g., a 3G network, a 4G network, etc., complying with any suitable standard, such as CDMA, GSM, LTE, LTE Advanced, WiMAX, etc.), other types of wireless netw ork, a wired netw ork, and so on. In some embodiments, communication network 854 can be a local area network, a wide area network, a public network (e.g., the Internet), a private or semi-pnvate network (e.g., a corporate or university intranet), any other suitable type of netw ork, or any suitable combination of networks. Communications links shown in FIG. 8 can each be any suitable communications link or combination of communications links, such as wired links, fiber optic links, Wi-Fi links. Bluetooth links, cellular links, and so on.

[0067] Referring now to FIG. 9, an example of hardware 900 that can be used to implement data source 802, computing device 850, and server 852 in accordance with some embodiments of the systems and methods described in the present disclosure is shown.

[0068] As shown in FIG. 9, in some embodiments, computing device 850 can include a processor 902, a display 904, one or more inputs 906, one or more communication systems 908, and/or memory 910. In some embodiments, processor 902 can be any suitable hardware processor or combination of processors, such as a central processing unit (‘'CPU”), a graphics processing unit (“GPU’’), and so on. In some embodiments, display 904 can include any suitable display devices, such as a liquid crystal display (“LCD”) screen, a light-emitting diode (“LED”) display, an organic LED (“OLED”) display, an electrophoretic display (e.g., an “e- ink” display), a computer monitor, a touchscreen, a television, and so on. In some embodiments, inputs 906 can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.

[0069] In some embodiments, communications systems 908 can include any suitable hardware, firmware, and/or software for communicating information over communication network 854 and/or any other suitable communication networks. For example, communications systems 908 can include one or more transceivers, one or more communication chips and/or chip sets, and so on. In a more particular example, communications systems 908 can include hardware, firmware, and/or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.

[0070] In some embodiments, memory 910 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 902 to present content using display 904, to communicate with server 852 via communications system(s) 908, and so on. Memory 910 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 910 can include random-access memory (“RAM”), read-only memory (“ROM”), electrically programmable ROM (“EPROM”), electrically erasable ROM (“EEPROM"), other forms of volatile memory, other forms of non-volatile memory, one or more forms of semi-volatile memory, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 910 can have encoded thereon, or otherwise stored therein, a computer program for controlling operation of computing device 850. In such embodiments, processor 902 can execute at least a portion of the computer program to present content (e.g., images, user interfaces, graphics, tables), receive content from server 852, transmit information to server 852, and so on. For example, the processor 902 and the memory 910 can be configured to perform the methods described herein (e.g., the method of FIG. 1, the method of FIG. 4, the workflow of FIGS. 5A- 5C. the workflow of FIGS. 6A and 6B).

[0071] In some embodiments, server 852 can include a processor 912, a display 914, one or more inputs 916, one or more communications systems 918, and/or memory 920. In some embodiments, processor 912 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on. In some embodiments, display 914 can include any suitable display devices, such as an LCD screen, LED display, OLED display, electrophoretic display, a computer monitor, a touchscreen, a television, and so on. In some embodiments, inputs 916 can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.

[0072] In some embodiments, communications systems 918 can include any suitable hardware, firmware, and/or software for communicating information over communication network 854 and/or any other suitable communication networks. For example, communications systems 918 can include one or more transceivers, one or more communication chips and/or chip sets, and so on. In a more particular example, communications systems 918 can include hardware, firmware, and/or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.

[0073] In some embodiments, memory 920 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 912 to present content using display 914, to communicate with one or more computing devices 850, and so on. Memory 920 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 920 can include RAM, ROM, EPROM, EEPROM, other types of volatile memory, other ty pes of non-volatile memory⁷, one or more types of semi-volatile memory, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 920 can have encoded thereon a server program for controlling operation of server 852. In such embodiments, processor 912 can execute at least a portion of the server program to transmit information and/or content (e.g., data, images, a user interface) to one or more computing devices 850, receive information and/or content from one or more computing devices 850, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone), and so on.

[0074] In some embodiments, the server 852 is configured to perform the methods described in the present disclosure. For example, the processor 912 and memory⁷ 920 can be configured to perform the methods described herein (e.g., the method of FIG. 1, the method of FIG. 4, the workflow of FIGS. 5A-5C, the workflow of FIGS. 6A and 6B).

[0075] In some embodiments, data source 802 can include a processor 922, one or more data acquisition systems 924, one or more communications systems 926, and/or memory⁷ 928. In some embodiments, processor 922 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU. and so on. In some embodiments, the one or more data acquisition systems 924 are generally configured to acquire data, images, or both, and can include a CT system. Additionally or alternatively, in some embodiments, the one or more data acquisition systems 924 can include any suitable hardware, firmware, and/or software for coupling to and/or controlling operations of a CT system. In some embodiments, one or more portions of the data acquisition system(s) 924 can be removable and/or replaceable.

[0076] Note that, although not shown, data source 802 can include any' suitable inputs and/or outputs. For example, data source 802 can include input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, a trackpad, a trackball, and so on. As another example, data source 802 can include any suitable display devices, such as an LCD screen, an LED display, an OLED display, an electrophoretic display, a computer monitor, a touchscreen, a television, etc., one or more speakers, and so on. [0077] In some embodiments, communications systems 926 can include any suitable hardware, firmware, and/or software for communicating information to computing device 850 (and, in some embodiments, over communication network 854 and/or any other suitable communication networks). For example, communications systems 926 can include one or more transceivers, one or more communication chips and/or chip sets, and so on. In a more particular example, communications systems 926 can include hardware, firmware, and/or software that can be used to establish a wired connection using any suitable port and/or communication standard (e.g., VGA, DVI video, USB, RS-232, etc ), Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.

[0078] In some embodiments, memory 928 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 922 to control the one or more data acquisition systems 924, and/or receive data from the one or more data acquisition systems 924; to generate images from data; present content (e.g., data, images, a user interface) using a display; communicate with one or more computing devices 850; and so on. Memory 928 can include any suitable volatile memory⁷, non-volatile memory⁷, storage, or any suitable combination thereof. For example, memory 928 can include RAM, ROM, EPROM, EEPROM, other ty pes of volatile memory, other ty pes of non-volatile memory, one or more types of semi-volatile memory, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 928 can have encoded thereon, or otherwise stored therein, a program for controlling operation of data source 802. In such embodiments, processor 922 can execute at least a portion of the program to generate images, transmit information and/or content (e.g., data, images, a user interface) to one or more computing devices 850, receive information and/or content from one or more computing devices 850, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone, etc.), and so on.

[0079] In some embodiments, any suitable computer-readable media can be used for storing instructions for performing the functions and/or processes described herein. For example, in some embodiments, computer-readable media can be transitory⁷ or non-transitory. For example, non-transitory computer-readable media can include media such as magnetic media (e.g., hard disks, floppy disks), optical media (e.g., compact discs, digital video discs. Blu-ray discs), semiconductor media (e.g., RAM, flash memory', EPROM, EEPROM), any suitable media that is not fleeting or devoid of any semblance of permanence during transmission, and/or any suitable tangible media. As another example, transitory computer- readable media can include signals on networks, in wires, conductors, optical fibers, circuits, or any suitable media that is fleeting and devoid of any semblance of permanence during transmission, and/or any suitable intangible media.

[0080] As used herein in the context of computer implementation, unless otherwise specified or limited, the terms “component,” “system,” “module,” “framework,” and the like are intended to encompass part or all of computer-related systems that include hardware, software, a combination of hardware and software, or software in execution. For example, a component may be, but is not limited to being, a processor device, a process being executed (or executable) by a processor device, an object, an executable, a thread of execution, a computer program, or a computer. By way of illustration, both an application running on a computer and the computer can be a component. One or more components (or system, module, and so on) may reside within a process or thread of execution, may be localized on one computer, may be distributed between two or more computers or other processor devices, or may be included within another component (or system, module, and so on).

[0081] In some implementations, devices or sy stems disclosed herein can be utilized or installed using methods embodying aspects of the disclosure. Correspondingly, description herein of particular features, capabilities, or intended purposes of a device or system is generally intended to inherently include disclosure of a method of using such features for the intended purposes, a method of implementing such capabilities, and a method of installing disclosed (or otherwise known) components to support these purposes or capabilities. Similarly, unless otherwise indicated or limited, discussion herein of any method of manufacturing or using a particular device or system, including installing the device or system, is intended to inherently include disclosure, as embodiments of the disclosure, of the utilized features and implemented capabilities of such device or system.

[0082] The present disclosure has described one or more preferred embodiments, and it should be appreciated that many equivalents, alternatives, variations, and modifications, aside from those expressly stated, are possible and within the scope of the invention.

Claims

1. A method for generating a denoised image of a subject from computed tomography (CT) image data acquired from the subject using a CT system, the method comprising:

(a) accessing CT image data with a computer system, wherein the CT image data have been acquired from the subject using the CT system;

(b) generating a prior image of the subject from the CT image data using the computer system, wherein the prior image comprises lower noise than other images in the CT image data;

(c) accessing a neural network with the computer system, wherein the neural network has been trained on training data to generate a denoised image from CT image data;

(d) inputting the CT image data and the prior image to the neural network using the computer system, generating output data as a denoised image having reduced noise as compared to the CT image data; and

(e) displaying the denoised image to a user, or stonng the denoised image with the computer system.

2. The method of claim 1, w herein generating the prior image comprises: generating virtual monoenergetic images (VMIs) from the CT image data; and selecting one of the VMIs as the prior image.

3. The method of claim 2, w herein the prior image is selected as the one of the VMIs based on an effective energy’ of an x-ray beam used when acquiring the CT image data.

4. The method of claim 3, wherein the prior image is selected as a 70 keV VMI.

5. The method of claim 1. wherein generating the prior image comprises generating a low energy threshold image from the CT image data and storing the low energy threshold image as the prior image.

6. The method of claim 1. wherein the neural network has been iteratively trained by updating the training data based on an output of the trained neural network in a previous training iteration and retraining the neural network using the updated training data.

7. A method for training a neural network to denoise a computed tomography (CT) image acquired from a subject using a CT system, the method comprising:

(a) accessing multi-energy CT image data with a computer system, wherein the multi-energy CT image data have been acquired from the subject using the CT system;

(b) generating a prior image of the subject from the multi-energy CT image data using the computer system, wherein the prior image comprises lower noise than other images in the multi-energy CT image data;

(c) generating a signal -only image from the CT image data using the computer system;

(d) generating a noise-only image from the CT image data and the signal-only image using the computer system;

(e) assembling a training data set from the noise-only image, the signal-only image, and the prior image using the computer system;

(1) training a neural network on the training data set using the computer system, generating a trained neural network as an output; and

(g) storing the trained network with the computer system.

8. The method of claim 7, wherein generating the noise-only image comprises subtracting the signal-only image from an image in the CT image data.

9. The method of claim 8, wherein generating the noise-only image further comprises performing spatial decoupling on the noise-only image.

10. The method of claim 9, wherein performing spatial decoupling on the noise- only image comprises applying random translation and random inversions to the noise-only image.

11. The method of claim 7. wherein generating the signal-only image comprises performing an iterative reconstruction on the CT image data to generate an estimate of the signal-only image.

12. The method of claim 7, wherein assembling the training data set comprises generating a weighted noise-only image by weighting the noise-only image with a weighting factor, generating a synthetic noisy image by combining the weighted noise-only image with the signal-only image, and storing the synthetic noisy image and the prior image as the training data set.

13. The method of claim 12, wherein assembling the training data set comprises refining the signal-only image by averaging a number of adjacent signal-only images, refining the prior image by averaging a number of adjacent prior images, and storing the refined signal-only image as the signal-only image and the refined prior image as the prior image.

14. The method of claim 12, wherein assembling the training data set comprises extracting random patches from the noise-only image, the signal-only image, and the prior image.

15. The method of claim 7, further comprising iteratively updating the training data set and retraining the neural network using the updated training data set.

16. The method of claim 15, wherein iteratively updating the training data set comprises updating the signal-only image as an output of the trained neural network generated based on the training data set and repeating steps (d)-(f) using the updated signal- only image.

17. The method of claim 7, wherein generating the prior image comprises: generating virtual monoenergetic images (VMIs) from the multi-energy CT image data; and selecting one of the VMIs as the prior image.

18. The method of claim 17, wherein the prior image is selected as the one of the VMIs based on an effective energy of an x-ray beam used when acquiring the CT image data.

19. The method of claim 18, wherein the prior image is selected as a 70 keV VMI.

20. The method of claim 7. wherein generating the prior image comprises generating a low energy threshold image from the multi-energy CT image data and storing the low energy threshold image as the prior image.

21. The method of claim 7. further comprising generating a denoised image of the subject by inputting the multi-spectral CT image data and the prior image to the trained neural network, generating the denoised image of the subject as an output.