CN111870245A - Cross-contrast-guided ultra-fast nuclear magnetic resonance imaging deep learning method - Google Patents
Cross-contrast-guided ultra-fast nuclear magnetic resonance imaging deep learning method Download PDFInfo
- Publication number
- CN111870245A CN111870245A CN202010627841.0A CN202010627841A CN111870245A CN 111870245 A CN111870245 A CN 111870245A CN 202010627841 A CN202010627841 A CN 202010627841A CN 111870245 A CN111870245 A CN 111870245A
- Authority
- CN
- China
- Prior art keywords
- magnetic resonance
- nuclear magnetic
- contrast
- image
- cross
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves
- A61B5/055—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
- G01R33/00—Arrangements or instruments for measuring magnetic variables
- G01R33/20—Arrangements or instruments for measuring magnetic variables involving magnetic resonance
- G01R33/44—Arrangements or instruments for measuring magnetic variables involving magnetic resonance using nuclear magnetic resonance [NMR]
- G01R33/48—NMR imaging systems
- G01R33/4818—MR characterised by data acquisition along a specific k-space trajectory or by the temporal order of k-space coverage, e.g. centric or segmented coverage of k-space
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
- G01R33/00—Arrangements or instruments for measuring magnetic variables
- G01R33/20—Arrangements or instruments for measuring magnetic variables involving magnetic resonance
- G01R33/44—Arrangements or instruments for measuring magnetic variables involving magnetic resonance using nuclear magnetic resonance [NMR]
- G01R33/48—NMR imaging systems
- G01R33/54—Signal processing systems, e.g. using pulse sequences ; Generation or control of pulse sequences; Operator console
- G01R33/56—Image enhancement or correction, e.g. subtraction or averaging techniques, e.g. improvement of signal-to-noise ratio and resolution
- G01R33/5608—Data processing and visualization specially adapted for MR, e.g. for feature analysis and pattern recognition on the basis of measured MR data, segmentation of measured MR data, edge contour detection on the basis of measured MR data, for enhancing measured MR data in terms of signal-to-noise ratio by means of noise filtering or apodization, for enhancing measured MR data in terms of resolution by means for deblurring, windowing, zero filling, or generation of gray-scaled images, colour-coded images or images displaying vectors instead of pixels
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention discloses a cross-contrast-guided ultra-fast nuclear magnetic resonance imaging deep learning method, which can reconstruct a high-quality nuclear magnetic resonance image from high k-space undersampled data acquired by nuclear magnetic resonance imaging equipment under the guidance of cross-contrast images. The method mainly comprises four steps of cross-contrast guided image reconstruction model construction, model-driven deep attention network training process and ultra-fast nuclear magnetic resonance imaging application. And adopting a plurality of groups of k-space undersampled data, corresponding full sampling data reconstruction images and guide nuclear magnetic resonance image training network parameters to enable output images of the network to approximate to the full sampling reconstruction images as much as possible. In application, k-space undersampled data and a guide nuclear magnetic resonance image are input, and the network output of the k-space undersampled data and the guide nuclear magnetic resonance image is a reconstructed high-quality nuclear magnetic resonance image.
Description
The technical field is as follows:
the invention belongs to the field of medical nuclear magnetic resonance imaging, and particularly relates to a cross-contrast-guided ultra-fast nuclear magnetic resonance imaging depth learning method which is used for reconstructing a high-quality nuclear magnetic resonance image from k-space highly undersampled data acquired by nuclear magnetic resonance equipment under the guidance of a cross-contrast image.
Background art:
the nuclear magnetic resonance imaging technology is a leading and nondestructive biomedical imaging technology. A typical magnetic resonance imaging protocol includes multiple contrast sequences of the same anatomy that provide complementary information to enhance clinical diagnosis. However, an important problem of the magnetic resonance imaging technology is that the data acquisition time is long and the imaging speed is slow.
Three types of existing fast mri techniques are briefly described below.
The compressive sensing nuclear magnetic resonance technology is a mainstream method for accelerating nuclear magnetic resonance imaging, and the nuclear magnetic resonance imaging speed is accelerated by undersampling sample data in a k-space and then reconstructing a clear nuclear magnetic resonance image based on a small amount of sample data. Traditional model-based compressive sensing nuclear magnetic resonance methods rely on regularization associated with image priors, such as total variation regularization [1,2], wavelet regularization [2,3], non-local regularization [4,5], and dictionary learning [6,7 ]. But it is challenging to manually design an optimal regularization. In recent years, the deep learning method is widely applied to compressive sensing nuclear magnetic resonance. It learns a mapping from a low-quality image reconstructed from undersampled data to a high-quality reconstructed image using a deep neural network based on training data [8,9,10,11,12,13 ]. However, these compressive sensing nmr methods only consider reconstructing nmr images using a single contrast sequence (e.g., T1WI or T2WI), and the acceleration factor is limited.
Another method to accelerate mri is to generate missing contrast mri from other contrast fully sampled data. Such methods either learn dictionaries or sparse representations of source contrast image blocks for target contrast [14,15] or learn mappings from source contrast to target contrast directly through deep neural networks [16,17,18,19 ]. However, this method has low reconstruction accuracy.
In clinical applications, some contrast sequences, such as T1WI, require shorter acquisition times to allow full sampling, while other sequences, such as T2WI and FLAIR, require longer acquisition times to accelerate imaging by undersampling. Recently, Xiang et al [20] proposed a cross-contrast guided nuclear magnetic resonance image reconstruction method to accelerate nuclear magnetic resonance imaging. And fusing the undersampled reconstructed T2WI image and the full sampling reconstructed T1WI guide image by adopting a Dense-Unet network, and outputting a reconstructed T2WI nuclear magnetic resonance image. However, the network does not introduce the nuclear magnetic resonance imaging mechanism and the domain knowledge into the design of the network structure, the network has no interpretability, a special network module is not elaborately designed aiming at the fusion problem, and the image reconstruction quality is not high.
Reference documents:
[1]Block K T,Uecker M,Frahm J.Undersampled radial MRI with multiplecoils.Iterative image reconstruction using a total variation constraint[J].Magnetic Resonance in Medicine,57(6):1086-1098,2007.
[2]Huang J,Chen C,Axel L.Fast multi-contrast MRI reconstruction[J].Magnetic resonance imaging,32(10):1344-1352,2014.
[3]Lustig M,Donoho D,Pauly J M.Sparse MRI:The application ofcompressed sensing for rapid MR imaging[J].Magnetic Resonance in Medicine,58(6):1182-1195,2007.
[4]Eksioglu E M.Decoupled algorithm for MRI reconstruction usingnonlocal block matching model:BM3D-MRI[J].Journal of Mathematical Imaging andVision,56(3):430-440,2016.
[5]Qu X,Hou Y,Lam F,et al.Magnetic resonance image reconstructionfrom undersampled measurements using a patch-based nonlocal operator[J].Medical image analysis,18(6):843-856,2014.
[6]Ravishankar S,Bresler Y.MR image reconstruction from highlyundersampled k-space data by dictionary learning[J].IEEE transactions onmedical imaging,30(5):1028-1041,2010.
[7]Zhan Z,Cai J F,Guo D,et al.Fast multiclass dictionaries learningwith geometrical directions in MRI reconstruction[J].IEEE Transactions onbiomedical engineering,63(9):1850-1861,2015.
[8]Wang S,Su Z,Ying L,et al.Accelerating magnetic resonance imagingvia deep learning[C].In International Symposium on Biomedical Imaging(ISBI).IEEE:514-517,2016.
[9]Lee D,Yoo J,Ye J C.Deep residual learning for compressed sensingMRI[C].In International Symposium on Biomedical Imaging(ISBI).IEEE:15-18,2017.
[10]Hammernik K,Klatzer T,Kobler E,et al.Learning a variationalnetwork for reconstruction of accelerated MRI data[J].Magnetic resonance inmedicine,79(6):3055-3071,2018.
[11]Meng N,Yang Y,Xu Z,et al.A Prior Learning Network for Joint Imageand Sensitivity Estimation in Parallel MR Imaging[C].In InternationalConference on Medical Image Computing and Computer-Assisted Intervention(MICCAI).Springer,Cham:732-740,2019.
[12]Schlemper J,Caballero J,Hajnal J V,et al.A deep cascade ofconvolutional neural networks for dynamic MR image reconstruction[J].IEEEtransactions on Medical Imaging,37(2):491-503,2017.
[13]Yang Y,Sun J,Li H,Xu Z.Deep ADMM-Net for compressive sensing MRI[C].Advances in neural information processing systems(NIPS),10-18,2016.
[14]Huang Y,Shao L,Frangi A F.Cross-modality image synthesis viaweakly coupled and geometry co-regularized joint dictionary learning[J].IEEEtransactions on medical imaging,37(3):815-827,2017.
[15]Roy S,Carass A,Prince J L.Magnetic resonance image example-basedcontrast synthesis[J].IEEE transactions on medical imaging,32(12):2348-2363,2013.
[16]Chartsias A,Joyce T,Giuffrida M V,et al.Multimodal MR synthesisvia modality-invariant latent representation[J].IEEE transactions on medicalimaging,37(3):803-814,2017.
[17]Dar S U H,Yurt M,Karacan L,et al.Image synthesis in multi-contrast MRI with conditional generative adversarial networks[J].IEEEtransactions on medical imaging,38(10):2375-2388,2019.
[18]Joyce T,Chartsias A,Tsaftaris S A.Robust multi-modal MR imagesynthesis[C].In International Conference on Medical Image Computing andComputer-Assisted Intervention(MICCAI).Springer,Cham:347-355,2017.
[19]Li H,Paetzold J C,Sekuboyina A,et al.DiamondGAN:Unified Multi-Modal Generative Adversarial Networks for MRI Sequences Synthesis[C].InInternational Conference on Medical Image Computing and Computer-AssistedIntervention(MICCAI).Springer,Cham:795-803,2019.
[20]Xiang L,Chen Y,Chang W,et al.Ultra-fast t2-weighted mrreconstruction using complementary t1-weighted information[C].InInternational Conference on Medical Image Computing and Computer-AssistedIntervention(MICCAI).Springer,Cham:215-223,2018.
the invention content is as follows:
the invention aims to provide a cross-contrast-guided ultra-fast nuclear magnetic resonance imaging deep learning method aiming at the defects and shortcomings of the existing fast nuclear magnetic resonance imaging technology. The invention reconstructs high quality nuclear magnetic resonance images from highly k-space undersampled data acquired by nuclear magnetic resonance imaging equipment under guidance of cross-contrast images. Because the acquired k-space data is highly undersampled and the data volume is far less than the full-sampling data volume, the nuclear magnetic resonance imaging equipment can realize ultra-fast imaging speed and simultaneously needs a reconstruction algorithm to achieve high nuclear magnetic resonance imaging precision.
In order to achieve the purpose, the invention adopts the following technical scheme to realize the purpose:
a cross-contrast-guided ultra-fast nuclear magnetic resonance imaging deep learning method comprises the following steps:
1) constructing a cross-contrast-guided ultra-fast nuclear magnetic resonance image reconstruction model: constructing a reconstruction model based on a nuclear magnetic resonance imaging mechanism and a compressive sensing theory;
2) constructing a model-driven deep attention network, wherein the construction method of the deep attention network comprises the following steps: developing and abstracting the calculation process of the half-quadratic splitting iterative algorithm for optimizing the ultra-fast nuclear magnetic resonance image reconstruction model into a depth attention network;
3) model-driven deep attention network training process: based on a training data set, learning the optimal parameters of the deep attention network by using an Adam optimization algorithm, so that the network output of the deep attention network when the highly undersampled data are input approaches a nuclear magnetic resonance image reconstructed by corresponding fully-sampled data;
4) performing a nuclear magnetic resonance imaging process by using the trained model-driven deep attention network: and inputting the k-space height undersampled data and the cross-contrast guided nuclear magnetic resonance image, wherein the output of the depth attention network is the reconstructed nuclear magnetic resonance image.
The invention has the further improvement that in the step 1), the cross-contrast-guided ultra-fast nuclear magnetic resonance image reconstruction model is a compressed sensing reconstruction model, and comprises a k-space undersampled data consistent term for modeling a nuclear magnetic resonance imaging mechanism and a cross-contrast prior term for modeling the correlation of two contrast images.
The invention is further improved in that the neighboring point operator operation and the image reconstruction operation for image fusion in the semi-quadratic splitting iterative algorithm in step 2) determines a model-driven depth composed of a cross-contrast fusion block and an image reconstruction block in step 3) the training data set is composed of a plurality of data sets, each data set being composed of k-space undersampled data ysCorresponding full sampling data reconstruction image
And cross-contrast guided nuclear magnetic resonance image xgComposition is carried out; objective function for deep attention network training
Is defined as:
wherein | | represents the number of elements in the training data set,
outputting an image for the depth attention network, wherein theta is a parameter of the depth attention network; calculating the gradient of the target function relative to the depth attention network parameters by adopting a back propagation algorithm, and then optimizing the depth attention network parameters by adopting an Adam algorithm based on a training data set to obtain the optimal parameter theta*。
The further improvement of the invention is that the specific application process of the deep attention network in the step 4) is as follows: when nuclear magnetic resonance imaging is carried out, k-space undersampled data and cross-contrast guided nuclear magnetic resonance image full-sampling data are collected through nuclear magnetic resonance equipment, then the k-space undersampled data and the guided nuclear magnetic resonance image are sent to a trained model-driven deep attention network, and an image output by the deep attention network is a reconstructed nuclear magnetic resonance image.
The invention has at least the following beneficial technical effects:
the invention provides a cross-contrast-guided ultra-fast nuclear magnetic resonance imaging deep learning method, which can reconstruct a high-quality nuclear magnetic resonance image from k-space undersampled data acquired by nuclear magnetic resonance imaging equipment under the guidance of cross-contrast images. Compared with the existing compressed sensing nuclear magnetic resonance method only considering a single-contrast image (such as a model-based compressed sensing nuclear magnetic resonance method and a deep learning-based compressed sensing nuclear magnetic resonance method), the method can achieve higher undersampling times, and has higher reconstruction precision and similar reconstruction speed. Compared with the image generation method, the quality of the nuclear magnetic resonance image reconstructed by the method is remarkably improved. Compared with the existing cross-contrast image guided nuclear magnetic resonance reconstruction depth attention network, the method has better interpretability and higher reconstruction precision.
In conclusion, the invention can be mainly used for realizing the rapid imaging function in the nuclear magnetic resonance imaging equipment and has important application value for the research, development and production of the nuclear magnetic resonance imaging equipment.
Description of the drawings:
FIG. 1 is a flow chart of an embodiment of the present invention.
FIG. 2 is a diagram of a model-driven deep attention network architecture.
Fig. 3 is a diagram of a cross-contrast fusion block network architecture.
Fig. 4 is a diagram of an example of magnetic resonance image reconstruction (1-dimensional cartesian sampling, 1/32 sampling rate).
The specific implementation mode is as follows:
in order to make the objects, technical solutions and advantages of the present invention more fresh, the present invention will be further described in detail with reference to the accompanying drawings and specific examples. These examples are merely illustrative and not restrictive of the invention.
As shown in fig. 1, the method for cross-contrast guided ultra-fast mri deep learning provided by the present invention includes the following steps:
the cross-contrast guided ultra-fast nuclear magnetic resonance image reconstruction model construction comprises the following steps:
given a nuclear magnetic resonance image to be reconstructed
(e.g., T2WI) of k-space sampled data
Cross-contrast guided nuclear magnetic resonance image reconstructed by full sampling
(e.g., T1WI) where N and U represent cardinality of the MRI image and k-space sampled data and U < N, based on MRI mechanisms and compressive sensing theory, the present invention designs the following MRI reconstruction models:
wherein
Is a Fourier transform, and the Fourier transform is,
is a sampling matrix of k-space. The first term is the data item which constrains the reconstructed nuclear magnetic resonance image x in k-spacesWith its undersampled data ysData consistency between. The second term is a cross-contrast prior term that models the magnetic resonance image xsAnd a guide image xgThe correlation between them.
Model-driven deep attention network construction:
the reconstruction model (1) can be efficiently solved by a semi-quadratic splitting algorithm. Introducing auxiliary nuclear magnetic resonance image zsLet z bes＝xsThe reconstruction model (1) is then equivalent to optimizing the energy model as follows:
with a penalty factor ρ → ∞ during optimization. The energy model (2) can estimate the unknown variable z by alternating iterationssAnd xsTo minimize. And solving the following two subproblems aiming at the k-th iteration.
Estimating z by neighbor operatorss(subproblem 1): given the step (k-1) of iterative generation of a nuclear magnetic resonance image
Auxiliary nuclear magnetic resonance image zsMay be based on guiding the nuclear magnetic resonance image xgUpdating is carried out by the following method:
the neighbor operator of the regularization term g (-) is defined as
Then auxiliary nmr image zsThe updating can be done by means of a neighborhood operator:
wherein the operators of neighboring points
Is determined by cross-contrast prior f (x)s,xg) Determined non-linear mapping in image xgWill input a nuclear magnetic resonance image under the guidance of
Is mapped as
Wherein xsInitialization by zero-filling:
estimating x by image reconstructions(subproblem 2): given the kth iteratively updated auxiliary nuclear magnetic resonance image
Reconstructed nuclear magnetic resonance image xsThe update may be by:
this subproblem has a closed form solution:
wherein Λ ═ MHM + diag (p) is a diagonal matrix.
The present invention expands and abstracts the above-mentioned iterative algorithm calculation processes (equations (4), (6)) into a deep attention network, i.e., a model-driven deep attention network. As shown in FIG. 2, the whole depth attention network comprises K operation units, each operation unit corresponds to two modules, namely a cross-contrast fusion block (F-B) and an image reconstruction block (R-B), and variable z in formula (4) and formula (6) is respectively realizedsAnd xsAnd (4) updating.
The present invention does not artificially design a cross-contrast prior f (x)s,xg) Instead, a learnable cross-contrast fusion block network is used instead of the neighborhood operators in equation (4):
thereby learning indirectly across contrast priors. And the cross-contrast fusion block network takes the two fused contrast images as input and outputs an updated auxiliary nuclear magnetic resonance image. As shown in fig. 3, the network is designed as a dual path deep convolutional neural network that contains a coding block, a feature fusion block, and a decoding block. The coding block is respectively from the guide image x by two sub-codersgAnd reconstructed image of step k-1 iteration (i.e. step k-1)
) Each sub-encoder consists of a convolution and four cascaded sub-block networks. Then the features from different coding network layers with different contrasts are connected in series, and further a decoding network comprising a convolution, five cascade sub-blocks, a multi-scale feature fusion (i.e. feature series connection), two convolutions and a Sigmoid operator is fused, and the decoder outputs an updated auxiliary coreMagnetic resonance imaging
Each sub-block network is designed with a hopping connection, a Channel Attention (CA) and a Space Attention (SA) module. Two attention modules are described in detail below.
Channel Attention (CA): the channel attention module is to make the network focus more on important channel features from different contrasts. First, the present invention converts the features represented by F into channel descriptors represented by G by means of channel-by-channel average pooling:
wherein FcIs a feature of the c-th channel, and H and W are the height and width of the feature. The channel descriptor G then passes through the cascaded network layers: convolution → ReLU layer → convolution → Sigmoid operator, so that the invention obtains the weights W of different channelsCA. The final weight is applied to the input features by element-wise multiplication:
spatial Attention (SA): the spatial attention module is to focus the network more on important spatial regions of the image, such as high frequency and heavy artifact regions. For feature F, through the cascaded network layers: convolution → ReLU layer → convolution → Sigmoid operator, the invention obtains the weight W of different pixelsSA. The weights are then applied to the input features by element-wise multiplication:
model-driven deep attention network training process:
to determine the optimal parameters of the model-driven depth attention network, the present invention constructs a training data set for a cross-contrast image-guided magnetic resonance imaging problem. The training data set is composed of a plurality of data sets, each data set is composed of k-space undersampled data of T2WI and corresponding full sampling data reconstruction imagesAnd a pilot nmr image of T1 WI. In practical construction, the invention firstly uses T2WI data of the nuclear magnetic resonance imaging equipment under the full sampling setting to reconstruct a nuclear magnetic resonance image corresponding to the full sampling data
The fully sampled data is then undersampled to obtain corresponding k-space sampled data ys. Reconstructing an image from k-space full-sampled data
As a standard reconstructed image, k-space sampling data y thereofsAnd a guide image xgAs the input of the depth attention network, the k-space sampling data, the standard reconstruction image and the guide image form a set of training data
Many sets of such training data constitute a deep attention network training set.
The present invention uses Mean Square Error (MSE) loss to train the network:
wherein, for the training data set, | | represents the number of elements in the training data set, ysK-space undersampled data acquired for a magnetic resonance imaging apparatus,
magnetic resonance image, x, reconstructed for corresponding full-sampled datagIn order to guide the nuclear magnetic resonance image,
for model-driven deep attention network output, Θ is a parameter of the deep attention network. Calculating the gradient of the objective function relative to the depth attention network parameters by adopting a back propagation algorithm, then optimizing the depth attention network parameters by adopting an Adam algorithm based on a training data set,obtaining the optimal parameter theta*。
Performing ultra-fast nuclear magnetic resonance imaging by using the trained model-driven deep attention network:
through the training process of the third step, the optimal parameters of the model-driven depth attention network can be determined, k-space undersampled data and a guided nuclear magnetic resonance image are input based on the trained depth attention network, and the output of the depth attention network is the reconstructed high-precision nuclear magnetic resonance image. Because the deep attention network parameter training process in the step three enables the output image of the deep attention network to be as close to the standard nuclear magnetic resonance image as possible, and the input of the deep attention network comprises the guide image of the full-sampling reconstruction, the trained deep attention network can still obtain a high-quality reconstructed image under the condition of extremely low data sampling rate.
In a numerical experiment, 379 brain multi-contrast nuclear magnetic resonance full-sampling reconstruction images are used, T2WI modal images are subjected to undersampling in k-space of the images according to different sampling rates, and therefore a brain nuclear magnetic resonance data set required by 379 experiments is obtained. 190 sets of data were selected as training data and 189 were used for testing. The k-space sampling pattern selects 1-dimensional cartesian sampling with sampling rates of 1/8, 1/16, and 1/32, respectively. In training, the present invention uses 2-dimensional slices to train the deep attention network, while in testing, the present invention tests reconstruction accuracy on a 3-dimensional volume of size 240 × 240 × 115. For objective evaluation of the different methods, the mean reconstruction accuracy over the test set was measured in terms of standard root mean square error (nRMSE) and peak signal to noise ratio (PSNR).
The model-driven deep attention network (MD-DAN) of the present invention was compared to other methods at different sampling rates, as shown in table 1. The contrast method includes a method of reconstructing T2WI from an undersampled T2WI modality (T2 → T2) as follows: Zero-Filling, DC-CNN, Dense-Unnet-R, ResNet-R, DPA-FusionNet-R; the method of generating T2WI from the T1WI modality (T1 → T2) is as follows: Dense-Unnet-S, ResNet-S, DPA-FusionNet-S; the method of generating T2WI from the undersampled T2WI modality and the guided T1WI modality (T1+ T2 → T2) is as follows: FCSA-MAT, Dense-Unet, ResNet. Where Dense-Unet-R (S), ResNet-R (S) are based on Dense-Unet (2 upsampling operations and 5 Dense blocks) and ResNet (9 convolutional residual blocks), respectively. DPA-fusion net-R (S) is a variant of cross-contrast fusion block network (DPA-fusion net) in fig. 3 for monomodal reconstruction, which represents an upper path ("-R") or a lower path ("-S") without cross-contrast fusion, respectively. The depth attention network designed by the invention achieves the best reconstruction precision under different sampling rates, and the result is greatly improved compared with other methods. Fig. 4 is a visualization result of a reconstructed image, and it can be seen that the method of the present invention achieves good reconstruction quality without significant artifacts.
Table one: comparison of different sampling rates in brain data test sets
Claims (5)
1. A cross-contrast-guided ultra-fast nuclear magnetic resonance imaging deep learning method is characterized by comprising the following steps:
1) constructing a cross-contrast-guided ultra-fast nuclear magnetic resonance image reconstruction model: constructing a reconstruction model based on a nuclear magnetic resonance imaging mechanism and a compressive sensing theory;
2) constructing a model-driven deep attention network, wherein the construction method of the deep attention network comprises the following steps: developing and abstracting the calculation process of the half-quadratic splitting iterative algorithm for optimizing the ultra-fast nuclear magnetic resonance image reconstruction model into a depth attention network;
3) model-driven deep attention network training process: based on a training data set, learning the optimal parameters of the deep attention network by using an Adam optimization algorithm, so that the network output of the deep attention network when the highly undersampled data are input approaches a nuclear magnetic resonance image reconstructed by corresponding fully-sampled data;
4) performing a nuclear magnetic resonance imaging process by using the trained model-driven deep attention network: and inputting the k-space height undersampled data and the cross-contrast guided nuclear magnetic resonance image, wherein the output of the depth attention network is the reconstructed nuclear magnetic resonance image.
2. The cross-contrast-guided ultra-fast MRI deep learning method according to claim 1, wherein in step 1), the cross-contrast-guided ultra-fast MRI image reconstruction model is a compressed sensing reconstruction model, and comprises a k-space undersampled data consistent term for modeling an MRI mechanism and a cross-contrast prior term for modeling the correlation between two contrast images.
3. The cross-contrast-guided ultrafast MRI deep learning method as claimed in claim 1, wherein the neighboring point operator operation and image reconstruction operation for image fusion in the semi-quadratic splitting iterative algorithm in step 2) determines a model-driven depth attention network composed of cross-contrast fusion blocks and image reconstruction blocks.
4. The cross-contrast-guided ultrafast MRI deep learning method as claimed in claim 1, wherein in said step 3), the training data set is composed of a plurality of data sets, each data set is composed of k-space undersampled data ysCorresponding full sampling data reconstruction image
And cross-contrast guided nuclear magnetic resonance image xgComposition is carried out; objective function for deep attention network training
Is defined as:
wherein | | represents the number of elements in the training data set,
outputting an image for the depth attention network, wherein theta is a parameter of the depth attention network; calculating the gradient of the target function relative to the depth attention network parameters by adopting a back propagation algorithm, and then optimizing the depth attention network parameters by adopting an Adam algorithm based on a training data set to obtain the optimal parameter theta*。
5. The cross-contrast-guided ultrafast MRI deep learning method according to claim 4, wherein the deep attention network of step 4) is specifically applied as follows: when nuclear magnetic resonance imaging is carried out, k-space undersampled data and cross-contrast guided nuclear magnetic resonance image full-sampling data are collected through nuclear magnetic resonance equipment, then the k-space undersampled data and the guided nuclear magnetic resonance image are sent to a trained model-driven deep attention network, and an image output by the deep attention network is a reconstructed nuclear magnetic resonance image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010627841.0A CN111870245B (en) | 2020-07-02 | 2020-07-02 | Cross-contrast-guided ultra-fast nuclear magnetic resonance imaging deep learning method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010627841.0A CN111870245B (en) | 2020-07-02 | 2020-07-02 | Cross-contrast-guided ultra-fast nuclear magnetic resonance imaging deep learning method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111870245A true CN111870245A (en) | 2020-11-03 |
| CN111870245B CN111870245B (en) | 2022-02-11 |
Family
ID=73149922
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010627841.0A Active CN111870245B (en) | 2020-07-02 | 2020-07-02 | Cross-contrast-guided ultra-fast nuclear magnetic resonance imaging deep learning method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111870245B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112950644A (en) * | 2021-03-17 | 2021-06-11 | 西北大学 | Deep learning-based neonatal brain image segmentation method and model construction method |
| CN113192151A (en) * | 2021-04-08 | 2021-07-30 | 广东工业大学 | MRI image reconstruction method based on structural similarity |
| CN113476029A (en) * | 2021-06-25 | 2021-10-08 | 陕西尚品信息科技有限公司 | Nuclear magnetic resonance imaging method based on compressed sensing |
| CN113842134A (en) * | 2021-11-09 | 2021-12-28 | 清华大学 | Double-sequence accelerated nuclear magnetic imaging optimization method based on double-path artificial neural network |
| CN113920213A (en) * | 2021-09-27 | 2022-01-11 | 深圳技术大学 | Multi-layer magnetic resonance imaging method and device based on long-distance attention model reconstruction |
| WO2022257090A1 (en) * | 2021-06-08 | 2022-12-15 | 苏州深透智能科技有限公司 | Magnetic resonance imaging method and related device |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103505207A (en) * | 2012-06-18 | 2014-01-15 | 山东大学威海分校 | Fast and effective dynamic MRI method based on compressive sensing technology |
| CN106373167A (en) * | 2016-11-15 | 2017-02-01 | 西安交通大学 | Compressed sensing nuclear magnetic resonance imaging method based on deep neural network |
| CN108090871A (en) * | 2017-12-15 | 2018-05-29 | 厦门大学 | A kind of more contrast MR image reconstruction methods based on convolutional neural networks |
| US20180286037A1 (en) * | 2017-03-31 | 2018-10-04 | Greg Zaharchuk | Quality of Medical Images Using Multi-Contrast and Deep Learning |
| CN109493394A (en) * | 2018-10-26 | 2019-03-19 | 上海东软医疗科技有限公司 | Method, method for reconstructing and the device of magnetic resonance imaging acquisition deep learning training set |
| US20190219654A1 (en) * | 2018-01-12 | 2019-07-18 | Korea Advanced Institute Of Science And Technology | Method for varying undersampling dimension for accelerating multiple-acquisition magnetic resonance imaging and device for the same |
| CN110151181A (en) * | 2019-04-16 | 2019-08-23 | 杭州电子科技大学 | Rapid magnetic resonance imaging method based on the U-shaped network of recurrence residual error |
| CN111028306A (en) * | 2019-11-06 | 2020-04-17 | 杭州电子科技大学 | AR2U-Net neural network-based rapid magnetic resonance imaging method |
-
2020
- 2020-07-02 CN CN202010627841.0A patent/CN111870245B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103505207A (en) * | 2012-06-18 | 2014-01-15 | 山东大学威海分校 | Fast and effective dynamic MRI method based on compressive sensing technology |
| CN106373167A (en) * | 2016-11-15 | 2017-02-01 | 西安交通大学 | Compressed sensing nuclear magnetic resonance imaging method based on deep neural network |
| US20180286037A1 (en) * | 2017-03-31 | 2018-10-04 | Greg Zaharchuk | Quality of Medical Images Using Multi-Contrast and Deep Learning |
| CN108090871A (en) * | 2017-12-15 | 2018-05-29 | 厦门大学 | A kind of more contrast MR image reconstruction methods based on convolutional neural networks |
| US20190219654A1 (en) * | 2018-01-12 | 2019-07-18 | Korea Advanced Institute Of Science And Technology | Method for varying undersampling dimension for accelerating multiple-acquisition magnetic resonance imaging and device for the same |
| CN109493394A (en) * | 2018-10-26 | 2019-03-19 | 上海东软医疗科技有限公司 | Method, method for reconstructing and the device of magnetic resonance imaging acquisition deep learning training set |
| CN110151181A (en) * | 2019-04-16 | 2019-08-23 | 杭州电子科技大学 | Rapid magnetic resonance imaging method based on the U-shaped network of recurrence residual error |
| CN111028306A (en) * | 2019-11-06 | 2020-04-17 | 杭州电子科技大学 | AR2U-Net neural network-based rapid magnetic resonance imaging method |
Non-Patent Citations (7)
| Title |
|---|
| LIYAN SUN等: "A deep information sharing network for multi-contrast compressed sensing MRI reconstruction", 《IEEE TRANSACTIONS ON IMAGE PROCESSING》 * |
| SHANSHAN WANG等: "Deep learing for fast MR imaging：a review for learning reconstruction from incomplete k-space data", 《INVITED REVIEW BIOMEDICAL SIGNAL PROCESSING AND CONTROL》 * |
| YAN YANG等: "ADMM-CSNet：a deep learning approach for image compressive sensing", 《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》 * |
| YAN YANG等: "Model-driven deep attention network for ultra-fast compressive sensing MRI guided by cross-contrast MR image", 《MICCAI2020》 * |
| YUBAO SUN等: "Learning non-locally regularized compressed sensing network with half-quadratic splitting", 《IEEE TRANSACTIONS ON MULTIMEDIA》 * |
| 肖韬辉等: "深度学习的快速磁共振成像及欠采样轨迹设计", 《中国图象图形学报》 * |
| 麻高超: "磁共振多对比度成像技术的研究", 《中国优秀硕士学位论文全文数据库医药卫生科技辑》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112950644A (en) * | 2021-03-17 | 2021-06-11 | 西北大学 | Deep learning-based neonatal brain image segmentation method and model construction method |
| CN112950644B (en) * | 2021-03-17 | 2024-04-05 | 西北大学 | Neonatal brain image segmentation method and model construction method based on deep learning |
| CN113192151A (en) * | 2021-04-08 | 2021-07-30 | 广东工业大学 | MRI image reconstruction method based on structural similarity |
| WO2022257090A1 (en) * | 2021-06-08 | 2022-12-15 | 苏州深透智能科技有限公司 | Magnetic resonance imaging method and related device |
| CN113476029A (en) * | 2021-06-25 | 2021-10-08 | 陕西尚品信息科技有限公司 | Nuclear magnetic resonance imaging method based on compressed sensing |
| CN113476029B (en) * | 2021-06-25 | 2024-02-02 | 陕西尚品信息科技有限公司 | Nuclear magnetic resonance imaging method based on compressed sensing |
| CN113920213A (en) * | 2021-09-27 | 2022-01-11 | 深圳技术大学 | Multi-layer magnetic resonance imaging method and device based on long-distance attention model reconstruction |
| CN113920213B (en) * | 2021-09-27 | 2022-07-05 | 深圳技术大学 | Multi-layer magnetic resonance imaging method and device based on long-distance attention model reconstruction |
| CN113842134A (en) * | 2021-11-09 | 2021-12-28 | 清华大学 | Double-sequence accelerated nuclear magnetic imaging optimization method based on double-path artificial neural network |
| CN113842134B (en) * | 2021-11-09 | 2024-04-12 | 清华大学 | Double-sequence acceleration nuclear magnetic imaging optimization method based on double-path artificial neural network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111870245B (en) | 2022-02-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111870245B (en) | Cross-contrast-guided ultra-fast nuclear magnetic resonance imaging deep learning method | |
| CN107610194B (en) | Magnetic resonance image super-resolution reconstruction method based on multi-scale fusion CNN | |
| CN104933683B (en) | A kind of non-convex low-rank method for reconstructing for magnetic resonance fast imaging | |
| Pawar et al. | Suppressing motion artefacts in MRI using an Inception‐ResNet network with motion simulation augmentation | |
| Cole et al. | Unsupervised MRI reconstruction with generative adversarial networks | |
| CN106780372A (en) | A kind of weight nuclear norm magnetic resonance imaging method for reconstructing sparse based on Generalized Tree | |
| CN109410289A (en) | A kind of high lack sampling hyperpolarized gas lung MRI method for reconstructing of deep learning | |
| KR102622283B1 (en) | Maxwell Parallel Imaging | |
| Merlet et al. | Parametric dictionary learning for modeling EAP and ODF in diffusion MRI | |
| Sudarshan et al. | Joint PET-MRI image reconstruction using a patch-based joint-dictionary prior | |
| Liu et al. | A deep framework assembling principled modules for CS-MRI: unrolling perspective, convergence behaviors, and practical modeling | |
| CN117011673B (en) | Electrical impedance tomography image reconstruction method and device based on noise diffusion learning | |
| Liu et al. | Online deep equilibrium learning for regularization by denoising | |
| Beetz et al. | Generating subpopulation-specific biventricular anatomy models using conditional point cloud variational autoencoders | |
| Kofler et al. | Learning regularization parameter-maps for variational image reconstruction using deep neural networks and algorithm unrolling | |
| Qian et al. | Physics-informed deep diffusion MRI reconstruction: Break the bottleneck of training data in artificial intelligence | |
| Cheng et al. | Model-based deep medical imaging: the roadmap of generalizing iterative reconstruction model using deep learning | |
| Cai et al. | Convolutional-recurrent neural networks approximate diffusion tractography from T1-weighted MRI and associated anatomical context | |
| CN112700508A (en) | Multi-contrast MRI image reconstruction method based on deep learning | |
| Gan et al. | Block coordinate plug-and-play methods for blind inverse problems | |
| CN114913262B (en) | Nuclear magnetic resonance imaging method and system with combined optimization of sampling mode and reconstruction algorithm | |
| Corda-D'Incan et al. | Syn-net for synergistic deep-learned pet-mr reconstruction | |
| Gan et al. | SS-JIRCS: Self-supervised joint image reconstruction and coil sensitivity calibration in parallel MRI without ground truth | |
| Alkan et al. | Learning to sample MRI via variational information maximization | |
| CN115170691A (en) | Magnetic resonance imaging reconstruction method based on deep learning and alternating direction multiplier method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |