CN114494498B - Metal artifact removing method based on double-domain Fourier neural network - Google Patents

Metal artifact removing method based on double-domain Fourier neural network Download PDF

Info

Publication number
CN114494498B
CN114494498B CN202210108825.XA CN202210108825A CN114494498B CN 114494498 B CN114494498 B CN 114494498B CN 202210108825 A CN202210108825 A CN 202210108825A CN 114494498 B CN114494498 B CN 114494498B
Authority
CN
China
Prior art keywords
metal
net
image
network
global
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210108825.XA
Other languages
Chinese (zh)
Other versions
CN114494498A (en
Inventor
单洪明
李子龙
张军平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fudan University
Original Assignee
Fudan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fudan University filed Critical Fudan University
Priority to CN202210108825.XA priority Critical patent/CN114494498B/en
Publication of CN114494498A publication Critical patent/CN114494498A/en
Application granted granted Critical
Publication of CN114494498B publication Critical patent/CN114494498B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/003—Reconstruction from projections, e.g. tomography
    • G06T11/008—Specific post-processing after tomographic reconstruction, e.g. voxelisation, metal artifact correction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Abstract

The invention belongs to the technical field of medical image analysis, and particularly relates to a metal artifact removing method based on a double-domain Fourier neural network. The invention removes metal artifacts from the two aspects of local and global by fast Fourier convolution in the chord image domain and the CT image domain respectively. By fully utilizing the global receptive field provided by Fourier convolution, the method can repair the chord graph damaged by metal on the chord graph by utilizing remote information, thereby removing metal artifacts on the original data. Meanwhile, the method eliminates the global inconsistency caused by secondary artifacts and artifact removal through a U-Net comprising fast Fourier convolution jump connection in the image domain. A multi-window loss function is used in training the network to optimize model performance under clinically significant window widths. The method has good effect of removing artifacts caused by metals with different sizes and shapes. The method improves the efficiency of removing the metal artifacts in the CT image, thereby reducing the interference of the metal artifacts on clinical diagnosis.

Description

Metal artifact removing method based on double-domain Fourier neural network
Technical Field
The invention belongs to the technical field of medical image analysis, and particularly relates to a CT metal artifact removing method.
Background
Computed Tomography (CT) is one of the primary modalities widely used for clinical diagnosis and screening. When there are high density objects such as metal implants and dental fillings, the raw data of the CT scan, i.e. the chord graph, is corrupted by physical factors such as beam hardening and scattering. The resulting reconstructed images exhibit severe radial artifacts, greatly limiting subsequent diagnosis. How to effectively reduce metal artifacts remains challenging. With the development of deep neural networks, the problem is also more and more focused.
To address this problem, there has been a related effort to remove Metal Artifacts (MAR) in the chord and image domains, respectively. However, in a single domain the method cannot efficiently recover the real organization from the corrupted data. This is because, on the one hand, metal artifacts on the reconstructed image are present globally, and artifacts are difficult to remove in the image domain when the metal becomes large. On the other hand, although only a part of the area in the chord graph is damaged, the restored original data cannot ensure the ideal image quality after reconstruction, because the small error discontinuity in the chord graph easily causes secondary artifacts in the image domain.
The currently popular MAR approach is to combine the advantages of two domains through a backward propagating Radon layer [1,2], significantly improving MAR performance. For example, [1] use two U-nets to enhance the chord graph and reconstruct the image to achieve promising results. However, convolutional layers lack a sufficiently large reception field, resulting in limited information to be captured, which hinders further development of the network. Therefore, at present, many efforts are made to improve the performance of the network by considering the joining and utilizing other information, such as image priority [3], adaptive scale [4], metal mask project [5], and so on.
The performance of the existing MAR method is unsatisfactory due to the following aspects. Firstly, in the chord graph, the existing method does not fully utilize the global context to perform interpolation, but only can continuously utilize surrounding pixels to perform recovery, and the recovery precision of the chord graph is greatly limited. Secondly, for those CT images that are heavily contaminated by artifacts, the locally repaired deep neural network may further introduce secondary artifacts; while some work [4] explored a non-local approach to eliminate global artifacts in the image domain, it may not be useful when the CT image is severely damaged. Third, most MAR methods measure image quality using a full window, ignoring contrast information for some clinically important CT windows. This leads to a situation where the network is overly smooth, making the restored image difficult to apply clinically.
Unlike prior methods, the present invention incorporates a fourier convolution into a neural network. The global receptive field provided by the Fourier convolution [6] is utilized to better remove artifacts in both the chord map and image domains. More specifically, in order to accurately recover a damaged area from information of an undamaged area in a chord graph domain, the invention provides a novel chord graph recovery network based on fast Fourier convolution, which can perform global interpolation by using a receptive field in a chord graph range. Meanwhile, the invention uses a local network to directly recover in the image domain. The above method removes metal artifacts locally and globally, respectively. In order to further remove existing secondary artifacts and inconsistency between different methods and simultaneously utilize the advantages of the methods, the invention further provides a U-Net based on the fast Fourier jump connection to further remove the artifacts in both local and global aspects and improve the image quality. In addition, because of the large numerical range of CT images, to further improve their clinical value, the present invention introduces multi-window losses to constrain the edges, pixel losses, and perceptual losses in the various CT windows.
Disclosure of Invention
The invention aims to provide a metal artifact removing method based on a dual-domain Fourier neural network aiming at the characteristics of metal artifacts on the whole situation and the local situation, so that the metal artifacts can be respectively processed on the local situation and the whole situation, and the image quality is improved.
The metal artifact removing method based on the double-domain Fourier neural network provided by the invention explores the receptive fields of a chord graph range and an image range of an MAR by utilizing fast Fourier convolution; in the chord map domain, the invention provides a chord map Fourier repair network, and a damaged area is repaired by using an undamaged area of a full chord map in a receptive field of the full chord map by means of fast Fourier convolution. Meanwhile, in the image domain, a Fourier U-Net is used for restoring the image from local to global through the context information of the image domain; in addition, the present invention introduces multi-window penalties to help the network perceive accurate feedback in different dynamic ranges to get better results in clinical applications. Experimental results demonstrate that the proposed method is superior to the most advanced metal artifact removal methods.
The invention provides a metal artifact removing method based on a double-domain Fourier neural network, which comprises the following specific steps:
(1) In the chord graph domain, a neural network based on fast Fourier convolution is utilized to realize global information interpolation so as to repair the chord graph damaged by metal.
Setting human body two-dimensional CT sliceThe attenuation coefficient distribution of (a) is: x = μ (i, j), where (i, j) represents a two-dimensional coordinate. The chord graph S in the ideal case can be used by projection
Figure BDA0003494683260000021
Determine, i.e.>
Figure BDA0003494683260000022
Analogously, by defining the back-projection process >>
Figure BDA0003494683260000023
An image X can be reconstructed from the chord graph S, i.e. < >>
Figure BDA0003494683260000024
Here->
Figure BDA0003494683260000025
Is/>
Figure BDA0003494683260000026
The inverse of (c). When the metal implant X m When present, the chords are damaged along the metal traces. Suppose the size of the chord graph is H s ×W s In which H is s And W s The number of detectors and the projection angle, respectively. Then the invention uses->
Figure BDA0003494683260000027
To represent a binary metal trace, where region M =1 corresponds to a damaged region of the metal in the chord graph. Thus, the chord graph S is influenced by the metal mc And the metal-free chord graph S can be expressed as:
S mc ⊙(1-M)＝S⊙(1-M) (1)
wherein the symbol |, indicates element-by-element multiplication. The problem of metal artifact removal in the chord graph is to recover the damaged area marked with metal tracks using the information of the area not affected by the metal. For the metal artifact problem of the chord graph, a fast Fourier convolution-based chord graph recovery network (FS-Net) is used, and the FS-Net can utilize a global receptive field on the chord graph to accurately recover the chord graph damaged by metal. The network firstly uses two down-sampling dimensionalities reduction, then uses six chord graph repair modules containing fast Fourier convolution to remove metal artifacts, and finally obtains a recovered chord graph through two up-sampling layers. In each of the chord graph repair modules, the input tensor is first split into two branches, called local branch and global branch, respectively. For local and global branches, three convolution modules are first used to capture the multiscale information. Meanwhile, for global branches, the chord graph is recovered in the fourier domain using a fast fourier convolution block. The output of FS-Net is written as:
S r ＝FS-Net(S mc ,M) (2)
wherein S is r Is the chord graph after repair. Finally, the restored chord graph is reconstructed through a Radon layer capable of back propagation, and the CT image X recovered from the angle of the chord graph s ：
X s ＝RL(S r ⊙M+S mc ⊙(1-M)) (3)
The invention uses the L1 loss function to measure the difference of the chord map domain, and uses the smooth-L1 loss function to measure the difference of the image domain, so as to avoid overfitting and gradient explosion. Thus, the penalty for optimizing FS-Net is defined as follows:
Figure BDA0003494683260000031
wherein S is gt And X gt Respectively, metal-free, true chord and image.
(2) In the image domain, a local artifact removal network IU-Net is used for removing metal artifacts in the image domain. FR-Net is then replaced by U-Net containing Fourier jump connection to better remove secondary artifacts and full image inconsistency, which is as follows:
firstly, a U-Net is used to primarily remove metal artifacts in an image domain, and the network is written as:
X u ＝LU-Net(RL(S mc )) (5)
for this network, L1 loss is used as its loss function
Figure BDA0003494683260000032
Setting CT image-X with artifact removed from local and global angle by different method s And X u . A fourier recovery network (FR-Net) uses information from the two coarsely recovered images to obtain accurate CT images after artifact removal. The proposed FR-Net uses the U-Net architecture, but uses local and global information for reconstruction to further remove artifacts, the module key being the fourier-based jump connection. The input tensor from the encoder is first split into two parts, one for the local branch and one for the global branch. For local branching, the image is processed locally using convolution layers, batch Normalization (BN) and ReLU activation functions. For global branching, the invention applies fast fourier convolution to globally eliminate secondary artifacts and global level inconsistencies in the fourier domain using the same architecture. Where the input data is fed into a common convolution layer and a fast fourier convolution, respectively, to remove artifacts locally and globally. The network can be written as:
X r ＝FR-Net(X s ,X u )+X u (7)
(3) Training by using a loss function with multiple window widths so as to improve the accuracy of the key window width in clinic;
training is usually performed using a large window exceeding [ -1000,2000] HU, where HU stands for Hounsfield units. Such large CT window widths result in networks that are not focused on some clinically important windows. In view of the clinical application, radiologists often use different CT windows for different clinical tasks. The effect of metal artifact removal at different window widths is therefore of paramount importance.
To address this problem, the present invention proposes a multi-window penalty to force the network to focus on those clinically important windows. Specifically, the invention uses three common windows, and the value ranges of the three windows are respectively as follows: the full range window [ -1000,2000] HU; the lung window [ -1000,200] HU and the soft tissue window [ -200,300] HU.
Order to
Figure BDA0003494683260000041
Three CT windows are represented, with w =1,2,3. The invention uses three important image quality indexes to ensure that the structure and the details of the structure artifact-removed picture are correct, and the structure artifact-removed picture respectively comprises pixel level loss, edge loss and perception loss.
(3.1) pixel level loss: the invention uses an L1 loss function to measure the pixel level difference under three CT windows:
Figure BDA0003494683260000042
(3.2) edge loss: edge information is crucial for clinical diagnostic measurement boundary information. The invention therefore uses a Sobel filter to extract gradient information for comparison. This process is performed under the CT window selected by the present invention. The process is defined as
Figure BDA0003494683260000043
(3.3) loss of perception
To ensure that the output of the network has a texture similar to the metal-free image, the present invention utilizes perceptual loss to extract high-level features for comparison. The present invention uses a pre-trained VGG-16 network to form the feature extractor phi. More specifically, the present invention concatenates images from three windows to form an RGB-like three channel image, since the VGG is pre-trained using natural images. The perceptual loss is defined as:
Figure BDA0003494683260000044
wherein
Figure BDA0003494683260000045
Representing the Frobenius norm. The multi-window loss function can be written as：
Figure BDA0003494683260000046
/>
In the invention, the model comprises three neural networks in total, wherein the neural networks comprise a Fourier chord graph network used for recovering the area polluted by the metal artifact in a chord graph domain through Fourier convolution, and a local network used for removing the local artifact in an image domain. And a network using fourier-hop connected U-Net in conjunction with two coarse recoveries further removes artifacts. Where the perceived loss is calculated using layers 2, 4 and 7 of VGG-16.
Wherein, regarding the chord graph repairing network FS-Net: firstly, two down-sampling layers are used, and then six Fourier residual modules are used for removing artifacts; in each Fourier residual block, firstly, an input vector X epsilon R is input according to channels B×C×H×W Split into two local and global tensors X = { X l ,X g To extract local and global features, respectively. Wherein
Figure BDA0003494683260000047
Is a local tensor, is based on>
Figure BDA0003494683260000051
Is the global tensor. B represents mini-batch size, C represents input channel number, H and W represent length and width of input vector, alpha 1 And alpha 2 To adjust the hyper-parameter, C = alpha is satisfied 1 +α 2 . Likewise, define the output vector as Y = { Y l ,Y g Denotes extracted local and global features, respectively. Then three convolution modules f are used ll ，f gl ，f lg And a fast Fourier convolution module f g To remove artifacts on different local scales and globally.
Y l ＝f ll (X l )+f gl (X g ) (12)
Y g ＝f lg (X l )+f g (X g ) (13)
Finally, two up-sampling layers corresponding to the down-sampling layers are used to make the input and output sizes consistent.
Regarding the local artifact removal network LU-Net in the image domain, a standard U-Net is used, which comprises four down-sampling layers and four up-sampling layers.
Regarding the Fourier repair network FR-Net of the image domain, three down-sampling layers and three up-sampling layers are used, i.e., the same structure as LU-Net. Except that one fourier block is used for the hop connection. Specifically, the method comprises a common convolution module comprising a convolution layer and a Fourier convolution module comprising fast Fourier transform, convolution, batch normalization, RELU activation function and fast inverse Fourier transform respectively.
According to the method, the Fourier convolution is applied to the metal artifact removal model, namely the metal is removed from the chord graph and the image domain to form the artifact through the global receptive field provided by the Fourier convolution, and the defect that the damaged part cannot be repaired on the chord graph by using remote information and the full-graph artifact cannot be modeled due to the fact that the traditional network lacks the receptive field is overcome. The model repairs the damaged part on the chord graph mainly by combining local information and global information on the chord graph. This solves the problem that the conventional network cannot utilize remote information. In the image domain, the invention first uses a neural network to locally remove artifacts. And then, the image with local artifact removal on the image domain and the reconstructed image after global repair on the chord graph are used, and the Fourier U-Net provided by the invention is combined with the advantages of two rough artifact removal graphs to repair in the aspects of local and global. According to the CT metal artifact removal method based on Fourier convolution, the image quality and the accuracy of metal artifact removal are greatly improved, and the problem that a CT image is difficult to diagnose due to metal artifacts in clinic can be solved.
Drawings
FIG. 1 is a model overall framework of the present invention.
Fig. 2 is a difference between the conventional deep learning-based method and the method on the chord graph.
Fig. 3 is an effect diagram of different metal artifact methods, wherein the sequence numbers respectively represent: (a) images of metal damage (input); (b) LI 7; (c) NMAR [8]; (d) CNNMAR [2]; (e) DuDoNet [1]; (f) DANnet [4]; (g) FD-MARs (method of the invention); (h) a view without damage by metal. The display window range is [ -200,300] HU. The yellow shade is the shape and position of the metal.
Detailed Description
The present invention is further described below by using a specific simulation example, and shows the effect of removing metal artifacts on metal artifact data, and the comparison with other methods, including image quality and quantization indexes.
The present invention uses the widely used DeepLeision dataset [9] in the metal artifact task to model metal data. 360,000 cases and 2,000 cases were used for training and testing, respectively. All CT images were resized to 208 x 208. A total of one hundred different metal implant data were used, with 90 metal shapes for training and 10 additional metal shapes for testing. The sizes of the 10 metal implants tested were: 9,13,27,28,30,59,108,220,223,515.
The invention uses the procedure of [5] to generate chord maps and CT images damaged by metals. The present invention uses fan beam CT to simulate forward and back projection. The distance from the X-ray source to the center of rotation was set to 59.5 cm, and a total of 320 projection angles were evenly distributed between 360 degrees. The number of detectors is set to 321. Thus, the size of the chord graph is 321 × 320, and each pixel of the chord graph represents the intensity received by the detector at its corresponding angle.
To speed up training, each subnetwork in all experiments was pre-trained for 20 rounds using training data. The hyper-parameter settings for all experiments were as follows: the learning rate is 0.0005, each 15 epochs are attenuated by 0.5, and the total epoch number is 100; where the mini-batch value is set to 4. The optimizer is Adam with a parameter of (. Beta.) 1 ,β 2 ) = (0.5,0.999). All experiments were carried out in a PyTorch framework and were performed using an NVIDIA GTX 2080Ti GPU.
In the experiment, three indexes of peak signal-to-noise ratio (PSNR) root-mean-square error (RMSE) and Structural Similarity (SSIM) are adopted to measure the experimental effect, and the PSNR is defined in the following way:
Figure BDA0003494683260000061
RMSE is defined as follows:
Figure BDA0003494683260000062
SSIM is defined as follows:
Figure BDA0003494683260000063
PSNR and RMSE represent the pixel-level match of the algorithm between the artifact removal result and the normal dose CT, while SSIM represents the structural similarity between the two.
Experimental example 1: comparison of different metal artifact removal methods
TABLE 1 comparison of different metal artifact removal methods
Figure BDA0003494683260000064
/>
Figure BDA0003494683260000071
The black bold in table 1 indicates the optimal results, and it can be seen from table 1 that the present invention has significant advantages over other methods, especially in the case of relatively large metal artifacts. The invention further compares the model of the invention with the following most advanced methods: LI [7], NMAR [8], CNNMAR [2], duDoNet [1] and methods of use in the present invention. LI and NMAR are widely used benchmark methods in MAR. CNNMAR is an image domain based deep learning method. DuDoNet is a two-domain method using two U-nets. DANNET is a two-domain network with adaptive scaling for obtaining additional information from damaged areas.
Table 1 shows the quantitative comparison. The values are the average values under three window widths, which are: [ -1000,2000] HU, [ -200,300] HU, [ -1000, -200] HU. Wherein the left side of the table is smaller metal and the right side of the table is larger metal, and the sizes of the smaller metal and the larger metal are increased from left to right. First, the two-domain based approach performs significantly better than the single-domain based approach. Although previous methods have worked well on small metals, the present invention further frees the potential for metal artifact removal networks by fourier convolution. By analyzing the PSNR and RMSE, the method used by the present invention greatly improves the pixel-level accuracy. This shows that the method of the invention is more accurate in restoring details, a feature that is critical for medical diagnosis.
Experimental example 2: contrast of different metal artifact removal methods under soft group window
TABLE 2 comparison of different metal artifact removal methods under soft tissue windows
Figure BDA0003494683260000072
/>
Figure BDA0003494683260000081
Soft tissue Windows are a commonly used display window in medicine, defined as [ -200,300] HU. Table 2 shows the performance of the invention under the soft tissue window. It is clear that the process of the invention has very significant advantages over other processes. Wherein the lift is particularly great, especially when the metal volume is large. The method used by the invention can effectively improve the clinical image quality.
Reference documents:
[1]Lin W A,Liao H,Peng C,et al.DuDoNet:Dual Domain Network for CT Metal Artifact Reduction[C]//2019IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2020.
[2]Zhang Y,Yu H.Convolutional Neural Network Based Metal Artifact Reduction in X-ray Computed Tomography[J].IEEE Transactions on Medical Imaging,2018:1-1.
[3]Yu L,Zhang Z,X Li,et al.Deep Sinogram Completion with Image Prior for Metal Artifact Reduction in CT Images[J].IEEE Transactions on Medical Imaging,2020.
[4]Wang T,Xia W,Y Huang,et al.DAN-Net:Dual-Domain Adaptive-Scaling Non-local Network for CT Metal Artifact Reduction[J].2021.
[5]Lyu Y,Lin W A,Lu J,et al.DuDoNet++:Encoding mask projection to reduce CT metal artifacts[J].2020.
[6]Suvorov R,Logacheva E,Mashikhin A,et al.Resolution-robust Large Mask Inpainting with Fourier Convolutions[J].2021.
[7]Kalender W A,Hebel R,Ebersberger J.Reduction of CT artifacts caused by metallic implants.[J].Radiology,1987,164(2):576-577.
[8]Meyer E,Raupach R,Lell M,et al.Normalized metal artifact reduction(NMAR)in computed tomography[J].Medical Physics,2010,37(10).
[9]KeY,Wang X,Le L,et al.DeepLesion:automated mining of large-scale lesion annotations and universal lesion detection with deep learning[J].Journal of Medical Imaging,2018,5(3):1-。

Claims (2)

1. a metal artifact removing method based on a double-domain Fourier neural network is characterized in that in a chord graph domain, a chord graph Fourier repairing network is adopted, a damaged area is repaired by utilizing an undamaged area of a full chord graph in the receptive field of the full chord graph by means of fast Fourier convolution; meanwhile, in an image domain, a Fourier U-Net is used for restoring an image from a local state to a global state through context information of the image domain; in addition, multi-window losses are introduced to help the network perceive accurate feedback in different dynamic ranges to get better results in clinical applications; the method comprises the following specific steps:
(1) In a chord graph field, utilizing a neural network based on fast Fourier convolution to realize global information interpolation so as to repair a chord graph damaged by metal; the specific process is as follows:
setting human body two-dimensional CT slicesThe attenuation coefficient distribution is: x = μ (i, j), where (i, j) represents a two-dimensional coordinate; by projection for setting up a string drawing S
Figure FDA0004072215670000012
Determine, i.e.>
Figure FDA0004072215670000013
Analogously by defining a back-projection process>
Figure FDA0004072215670000018
Reconstructing an image X from the chord graph S, i.e.
Figure FDA0004072215670000014
Is here->
Figure FDA0004072215670000015
Is->
Figure FDA0004072215670000016
The inverse operation of (1); when the metal implant X m When present, the chords are damaged along the metal traces; suppose the size of the chord graph is H s ×W s In which H is s And W s The number of detectors and the projection angle, respectively; then the present use->
Figure FDA0004072215670000017
To represent a binary metal trace, where M =1 corresponds to a damaged area of metal in the chord graph; thus, the chord graph S is influenced by the metal mc And the metal-free chord graph S is expressed as:
S mc ⊙(1-M)＝S⊙(1-M) (1)
wherein the symbol [ ] indicates element-by-element multiplication; the problem of removing the metal artifacts in the chord graph is to recover the damaged area marked by the metal track by using the information of the area which is not influenced by the metal; for the metal artifact problem of the chord graph, a chord graph recovery network FS-Net based on fast Fourier convolution is used, and the chord graph damaged by metal is accurately recovered on the chord graph by using a global receptive field; the chord graph recovery network firstly uses two down-sampling dimensionalities reduction, then uses six chord graph repair modules containing fast Fourier convolution to remove metal artifacts, and finally obtains a recovered chord graph through two up-sampling layers; in each chord graph restoration module, the input tensor is firstly divided into two branches, namely a local branch and a global branch; for local and global branches, three convolution modules are used to capture multi-scale information first; meanwhile, for the global branch, a fast Fourier convolution block is used for recovering a chord graph in a Fourier domain; the output of FS-Net is written as:
S r ＝FS-Net(S mc ,M) (2)
wherein S is r Is the chord graph after repair; finally, the restored chord graph is reconstructed through a Radon layer capable of back propagation, and the CT image X recovered from the angle of the chord graph s ：
X s ＝RL(S r ⊙M+S mc ⊙(1-M)) (3)
Measuring the difference of the chord map domain by using an L1 loss function, and measuring the difference of the image domain by using a smooth-L1 loss function so as to avoid overfitting and gradient explosion; thus, the loss of optimized FS-Net is defined as follows:
Figure FDA0004072215670000021
wherein S is gt And X gt A metal-free, real chord and image, respectively;
(2) In an image domain, firstly, removing metal artifacts in the image domain by using a local artifact removal network IU-Net; and then replacing the traditional U-Net by using U-Net comprising Fourier jump connection FR-Net to better remove secondary artifacts and full-image inconsistency, and comprising the following specific steps:
first, a U-Net is used to remove the metal artifacts primarily in the image domain, and the Net is written as:
X u ＝LU-Net(RL(S mc )) (5)
for this network, L1 loss is used as its loss function
Figure FDA0004072215670000022
Setting CT image-X with artifact removed from local and global angle by different method s And X u (ii) a Using information from the two roughly recovered images to obtain a deghosted accurate CT image using a fourier recovery network FR-Net; the FR-Net uses a U-Net framework, but uses local and global information to carry out reconstruction so as to further remove the artifact, and the network module is mainly connected through a jump based on Fourier; the input tensor from the encoder is first divided into two parts, one for the local branch and one for the global branch; for local branches, images are processed locally using convolutional layers, batch Normalization (BN) and ReLU activation functions; for global branches, applying fast fourier convolution to eliminate secondary artifacts and global-level inconsistencies in the fourier domain; wherein the input data is fed into a normal convolutional layer and a fast fourier convolution, respectively, to remove artifacts locally and globally; the network writes as:
X r ＝FR-Net(X s ,X u )+X u (7)
(3) Training by using a loss function with multiple window widths so as to improve the accuracy of the key window width in clinic;
training using a large window exceeding [ -1000,2000] HU, where HU stands for Hounsfield units; the metal artifact removal effect under different window widths is important, and therefore, a multi-window loss is adopted to force the network to pay attention to the clinically important windows; specifically, three commonly used windows are used, and the value ranges of the three windows are respectively: the full range window [ -1000,2000] HU; the pulmonary window [ -1000,200] hu and the soft tissue window [ -200,300] hu;
order to
Figure FDA0004072215670000023
Represents three CT windows, w =1,2,3; using three important image quality indicatorsThe structure and the details of the structure artifact removing picture are ensured to be correct, namely pixel level loss, edge loss and perception loss;
(3.1) pixel level loss: the L1 loss function is used to measure the pixel level difference for three CT windows:
Figure FDA0004072215670000024
(3.2) edge loss: the edge information is important for measuring boundary information in clinical diagnosis, and therefore a Sobel filter is used for extracting gradient information for comparison; the process is performed under a selected CT window, which is defined as:
Figure FDA0004072215670000031
(3.3) loss of perception
To ensure that the output of the network has a texture similar to the metal-free image, high-level features are extracted with perceptual loss for comparison; i.e., using a pre-trained VGG-16 network to form the feature extractor phi, specifically, connecting the images from three windows to form a three-channel RGB-like image, since VGG is pre-trained using natural images, the perceptual loss is defined as:
Figure FDA0004072215670000032
wherein
Figure FDA0004072215670000033
Represents the Frobenius norm; the multi-window loss function is then written as:
Figure FDA0004072215670000034
2. the method for removing the metal artifact based on the dual-domain Fourier neural network as claimed in claim 1, wherein the specific network structure used is as follows:
with respect to the chord graph repair network FS-Net: firstly, two down-sampling layers are used, and then six Fourier residual modules are used for removing artifacts; in each Fourier residual block, firstly, an input vector X epsilon R is input according to channels B×C×H×W Split into two local and global tensors X = { X l ,X g -to extract local and global features, respectively; wherein
Figure FDA0004072215670000035
In the form of a local tensor is used,
Figure FDA0004072215670000036
is a global tensor; b represents mini-batch size, C represents input channel number, H and W represent length and width of input vector, alpha 1 And alpha 2 To adjust the hyper-parameter, C = alpha is satisfied 1 +α 2 (ii) a Likewise, define the output vector as Y = Y l ,Y g Represents the extracted local and global features, respectively; then using three convolution modules f ll ，f gl ，f lg And a fast Fourier convolution module f g To remove artifacts on different local scales and globally: />
Y l ＝ f ll (X l )+f gl (X g ) (12)
Y g ＝ f lg (X l )+f g (X g ) (13)
Finally, two up-sampling layers corresponding to the down-sampling layers are used to make the input and output sizes consistent;
regarding local artifact removal network LU-Net of an image domain, a standard U-Net is used, and comprises four layers of down-sampling layers and four layers of up-sampling layers;
regarding the Fourier repair network FR-Net of the image domain, three layers of down-sampling layers and three layers of up-sampling layers are used, namely the structure is the same as that of the LU-Net; except that one fourier block is used for the hop connection; specifically, the system comprises a convolution module and a fast Fourier convolution module, wherein the convolution module comprises a convolution layer; the fast fourier convolution module includes fast fourier transform, convolution, batch normalization, RELU activation function, and inverse fast fourier transform.
CN202210108825.XA 2022-01-28 2022-01-28 Metal artifact removing method based on double-domain Fourier neural network Active CN114494498B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210108825.XA CN114494498B (en) 2022-01-28 2022-01-28 Metal artifact removing method based on double-domain Fourier neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210108825.XA CN114494498B (en) 2022-01-28 2022-01-28 Metal artifact removing method based on double-domain Fourier neural network

Publications (2)

Publication Number Publication Date
CN114494498A CN114494498A (en) 2022-05-13
CN114494498B true CN114494498B (en) 2023-04-18

Family

ID=81476889

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210108825.XA Active CN114494498B (en) 2022-01-28 2022-01-28 Metal artifact removing method based on double-domain Fourier neural network

Country Status (1)

Country Link
CN (1) CN114494498B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101821046B1 (en) * 2016-07-13 2018-01-23 연세대학교 산학협력단 Method and device for estimating metal artifacts of ct image and method for determining metal artifacts calibration
CN109472754A (en) * 2018-11-06 2019-03-15 电子科技大学 CT image metal artifact removing method based on image repair
CN111292386A (en) * 2020-01-15 2020-06-16 中国人民解放军战略支援部队信息工程大学 CT projection metal trace completion metal artifact correction method based on U-net
CN113538257A (en) * 2021-06-15 2021-10-22 复旦大学 Dual-domain U-net discriminator-based generation confrontation low-dose CT denoising method
CN113744155A (en) * 2021-09-04 2021-12-03 重庆大学 Rock and ore sample CT image metal artifact correction method based on triple convolution network

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101591381B1 (en) * 2014-10-30 2016-02-04 기초과학연구원 Method for reducing metal artifact in computed tomography
US11580678B2 (en) * 2020-03-02 2023-02-14 GE Precision Healthcare LLC Systems and methods for interpolation with resolution preservation

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101821046B1 (en) * 2016-07-13 2018-01-23 연세대학교 산학협력단 Method and device for estimating metal artifacts of ct image and method for determining metal artifacts calibration
CN109472754A (en) * 2018-11-06 2019-03-15 电子科技大学 CT image metal artifact removing method based on image repair
CN111292386A (en) * 2020-01-15 2020-06-16 中国人民解放军战略支援部队信息工程大学 CT projection metal trace completion metal artifact correction method based on U-net
CN113538257A (en) * 2021-06-15 2021-10-22 复旦大学 Dual-domain U-net discriminator-based generation confrontation low-dose CT denoising method
CN113744155A (en) * 2021-09-04 2021-12-03 重庆大学 Rock and ore sample CT image metal artifact correction method based on triple convolution network

Also Published As

Publication number Publication date
CN114494498A (en) 2022-05-13

Similar Documents

Publication Publication Date Title
Lu et al. Iterative reconstruction of low-dose CT based on differential sparse
US7653229B2 (en) Methods and apparatus for reconstruction of volume data from projection data
Yuan et al. Half2Half: deep neural network based CT image denoising without independent reference data
Wang et al. Medical image inpainting with edge and structure priors
CN113516586A (en) Low-dose CT image super-resolution denoising method and device
CN111091575B (en) Medical image segmentation method based on reinforcement learning method
Chi et al. CT image super-resolution reconstruction based on global hybrid attention
Li et al. Learning non-local perfusion textures for high-quality computed tomography perfusion imaging
Xia et al. Deep residual neural network based image enhancement algorithm for low dose CT images
Jie et al. Tri-modal medical image fusion based on adaptive energy choosing scheme and sparse representation
Wu et al. Masked joint bilateral filtering via deep image prior for digital X-ray image denoising
Mangalagiri et al. Toward generating synthetic CT volumes using a 3D-conditional generative adversarial network
CN114494498B (en) Metal artifact removing method based on double-domain Fourier neural network
Tran et al. Deep learning-based inpainting for chest X-ray image
Jin et al. Low-dose CT image restoration based on noise prior regression network
Lei et al. CT Image Denoising and Deblurring With Deep Learning: Current Status and Perspectives
Yu et al. RIRGAN: An end-to-end lightweight multi-task learning method for brain MRI super-resolution and denoising
Zhao et al. Medical images super resolution reconstruction based on residual network
Xu et al. MFGAN: Multi-modal feature-fusion for CT metal artifact reduction using GANs
Kornilov et al. Inpainting of ring artifacts on microtomographic images by 3D CNN
Li et al. Quad-Net: Quad-domain network for CT metal artifact reduction
Song et al. NeXtResUNet: A neural network for industrial CT image denoising
Li et al. A cascade-based dual-domain data correction network for sparse view CT image reconstruction
Xia et al. Dynamic controllable residual generative adversarial network for low-dose computed tomography imaging
Xu et al. Residual neural network for filter kernel design in filtered back-projection for ct image reconstruction

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant