CN113936073B - ATTISTANET compressed sensing magnetic resonance reconstruction method based on attention mechanism - Google Patents
ATTISTANET compressed sensing magnetic resonance reconstruction method based on attention mechanism Download PDFInfo
- Publication number
- CN113936073B CN113936073B CN202111290411.5A CN202111290411A CN113936073B CN 113936073 B CN113936073 B CN 113936073B CN 202111290411 A CN202111290411 A CN 202111290411A CN 113936073 B CN113936073 B CN 113936073B
- Authority
- CN
- China
- Prior art keywords
- istanet
- attention
- attistanet
- reconstruction
- attention module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 66
- 230000007246 mechanism Effects 0.000 title claims abstract description 27
- 230000006870 function Effects 0.000 claims description 28
- 210000004556 brain Anatomy 0.000 claims description 26
- 206010028980 Neoplasm Diseases 0.000 claims description 20
- 238000012549 training Methods 0.000 claims description 19
- 230000005012 migration Effects 0.000 claims description 15
- 238000013508 migration Methods 0.000 claims description 15
- 238000011176 pooling Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 5
- 230000003044 adaptive effect Effects 0.000 claims description 4
- 230000000694 effects Effects 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 4
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 2
- 238000012360 testing method Methods 0.000 abstract description 12
- 238000013461 design Methods 0.000 abstract description 3
- 238000005457 optimization Methods 0.000 description 5
- 230000009466 transformation Effects 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- 102100025752 CASP8 and FADD-like apoptosis regulator Human genes 0.000 description 3
- 101710100501 CASP8 and FADD-like apoptosis regulator Proteins 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 101000903698 Mus musculus B-cell lymphoma/leukemia 11B Proteins 0.000 description 2
- 230000001174 ascending effect Effects 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 208000003174 Brain Neoplasms Diseases 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011056 performance test Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/003—Reconstruction from projections, e.g. tomography
- G06T11/008—Specific post-processing after tomographic reconstruction, e.g. voxelisation, metal artifact correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Abstract
The invention discloses a compressed sensing magnetic resonance reconstruction method based on a focus mechanism ATTISTANET. The method aims to solve the problem that the ISTANet + model lacks attention to frequency and characteristic information contained in different areas and channels and cannot learn characteristics differently. The invention comprises the following steps: step one: ISTANet + building a network structure; ATTISTANET building a model; step two: introducing an attention module in ISTANet + networks, adding a channel attention module and a ATTISTANET of a space attention module before each reconstructed image x k, and carrying out channel and space feature recalibration on the features of the original K space data; step three: the loss function design, adopting a smooth average absolute loss function to replace symmetrical constraint loss in ISTANet + according to the thought of a greedy algorithm; step four: and (5) model testing. The invention is used for compressed sensing MRI reconstruction based on an attention mechanism.
Description
Technical Field
The invention relates to magnetic resonance image reconstruction, in particular to a ATTISTANET compressed sensing magnetic resonance reconstruction method based on an attention mechanism.
Background
Most of traditional CS-MRI reconstruction methods utilize structural sparsity as image prior, and solve the sparse regularization problem through iteration. These methods are based on the intrinsic properties of the image and the existing image formation model. However, such methods generally require not only manual setting of optimization parameters, but also complex iterative operations, which makes the image reconstruction time longer and difficult to meet the requirements of medical diagnosis.
In recent years, deep neural networks are gradually rising, and the optimal transformation in CS can be processed and manual setting of optimization parameters can be avoided by utilizing the strong feature extraction and generalization capability of the deep neural networks. The neural network is used for replacing fixed transformation to learn optimal sparse transformation, and optimal parameters are trained; in 2016, ADMMNet was proposed, which mapped the Alternate Direction Multiplier Method (ADMM) into a deep network, and learned the linear optimal transformation using cyclic convolution. After that, ISTANet is proposed, which maps the Iterative Shrink Threshold (ISTA) optimization method into a deep network, and implements nonlinear optimal transformation with two convolutional layers and a modified linear unit (ReLU) activation function, with a simpler structure. In addition, ISTANet +, introducing ResNet to reduce training difficulty, is also proposed. Because of the optimized heuristic design, ADMMNet, ISTANet, ISTANet + is superior to the common neural network in the aspects of compressed sensing reconstruction, convergence speed and the like. However, the ISTANet + model lacks attention to the frequency and feature information contained in the different regions and channels, and does not learn features differentially.
Disclosure of Invention
In order to overcome the above-mentioned drawbacks of the prior art, an object of the present invention is to propose a ATTISTANET compressed sensing magnetic resonance reconstruction method based on an attention mechanism to overcome the drawbacks of the prior art.
In order to achieve the above object, the present invention provides a ATTISTANET compressed sensing magnetic resonance reconstruction method based on an attention mechanism, which includes the following steps:
ISTANet + building a network structure;
H (-) and L (-) operators are added on the basis of ISTANet, wherein H k (-) and L k (-) are convolution kernels of 1 multiplied by 1, are linear operators for increasing the number of channels and reducing the number of channels respectively, and the difficulty of network training is reduced by using a residual structure;
constructing an MRI reconstruction model based on an attention module;
Introducing an attention module on the basis of ISTANet + networks, adding an improved channel attention module and a spatial attention module before each reconstructed image x k, and carrying out channel and spatial feature recalibration on the features of the original K space data by using an attention mechanism module;
Step three, designing a loss function;
replacing the symmetrical constraint loss in ISTANet + by a smooth average absolute loss function;
Model enhancement and model migration based on T2 tumor images;
The data-enhanced and model-migrated model was tested using the brain T2 tumor dataset and normal T1 brain image data. The ATTISTANET compressed sensing magnetic resonance reconstruction method based on the attention mechanism comprises the following specific processes:
H (-) and L (-) operators are added on the basis of ISTANet, wherein H k (-) and L k (-) are convolution kernels of 1×1, which are linear operators for increasing the number of channels and reducing the number of channels respectively, and a residual structure is used for reducing the difficulty of network training, and the iteration of the kth stage in the ISTANet + method is as follows:
rk＝xk-1-ρkΦT(Φxk-1-y) (1)
xk＝Lk(Fk)T(soft(Fk(Hk(rk)),θk))+rk (2)
Wherein x k represents a reconstructed image, r k represents an intermediate variable, and k, ρ, φ and y are respectively the number of stages, the step size, the measurement matrix and the observation signal; soft (·) represents a soft threshold function, θ represents a shrinkage threshold of the threshold function, F (·) represents a transform coefficient for a certain fixed transform F, T is a transposed symbol;
in the reconstruction of an MRI image, the adopted K space data features are scattered in different areas, and the equivalent processing of high-frequency and low-frequency parts in the K space data by ADMMNet, ISTANet + and other methods can reduce the extraction effect of a network on information of different areas and influence the identification capability of different types of features, so that the reconstruction accuracy is restricted from being improved.
The ATTISTANET compressed sensing magnetic resonance reconstruction method based on the attention mechanism comprises the following specific processes:
introducing different attention modules in ISTANet + networks, adding an improved channel attention module and a spatial attention module before each reconstructed image x k and reconstructed intermediate variable r k, and carrying out channel and spatial feature recalibration on the features of the original K space data by using the attention module, wherein after passing through the channel attention module and the spatial attention module, the expression is as follows:
Ck＝Lk(Fk)T(soft(Fk(Hk(rk)),θk)) (3)
ak＝sigmoid(relu(w(Ck))) (4)
sk＝sigmoid(f(ak)) (5)
xk＝Ck×sk+rk (6)
Wherein C is a feature to be noted, a and s are features processed by the channel attention module and the space attention module respectively, w is a weight scale of the multi-layer sensor, the features can be scaled, and f represents a convolution operation;
the adopted channel attention mechanism and the spatial attention mechanism can obtain richer characteristic information by the attention module, and the reconstruction accuracy is improved. The expression of the LSE function is as follows:
Wherein S ij represents the activation value of (i, j), S represents the pooling area, (i, j) is one point in the pooling area S, N is the total point number of the pooling area S, the pooling range can be from the maximum value (u → infinity) in S to the average value (u → 0) through the self-adaptive super-parameter u, the value of u can be updated iteratively along with the network, and the loss of characteristic information can be reduced through LSE.
The ATTISTANET compressed sensing magnetic resonance reconstruction method based on the attention mechanism comprises the following specific processes:
The symmetric constraint losses in ISTANet + are replaced with a smoothed average absolute loss function, ISTANet + introduces a symmetric constraint loss function due to the symmetry of the network. The loss function is as follows:
Ltotal(θ)＝Ldiscrepancy+γLconstraint (8)
wherein k, N b, N and The number of stages, the number of training total blocks, the size of each block and regularization parameters of ISTANet + respectively;
the formula (10) is replaced with a smoothed average absolute loss. Wherein δ is set to 0.5, the modified loss function is as follows:
The ISTANet + network comprises an adaptive and input initial layer, x k,rk is a reconstructed image and an intermediate reconstruction module respectively, the network comprises 1 to k stages of depth structures, the structures of each stage are identical, the forward transform F k consists of two linear convolution operators, the linear operators are separated by a ReLU function, and the backward transforms (F k)T and F k are structurally symmetrical, wherein F k and F k)T satisfy F k·(Fk)T =i, and I is an identity matrix.
The beneficial effects of the invention are that
1. The invention provides a reconstruction method based on an attention mechanism to solve the problem of the reconstruction precision of compressed sensing MRI, and designs a reconstruction method based on Log-Sum-Exp pooling (LSE) and Max-Pooling (CBAM) on the basis of denoising and detail recovery based on the strong learning capacity of CNN on the model structure. On the loss function, the smooth average absolute loss is introduced to replace the original symmetrical constraint loss, so that the problems of local optimum, unstable training and the like in the training process are solved. And the T2 tumor image is used for carrying out data enhancement on the original training set, and the model has better generalization capability through two training modes of data enhancement and model migration.
2. Compared with similar excellent methods FISTANet, ISTANet + and the like, the MRI reconstruction method has the advantages that the reconstructed image is more similar to the original image in subjective effect, more realistic brain and head MRI image texture details are provided, more original MRI image information is reserved, and the PSNR value and the SSIM value of the reconstructed image are greatly improved in objective indexes.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention. In the drawings:
FIG. 1 is a flow chart of the invention of ATTISTANET;
FIG. 2 is a schematic diagram of a layer of reconstruction architecture in ISTANet + networks;
FIG. 3 is a schematic view of a reconstruction structure of ATTISTANET according to the present invention;
FIG. 4 is a diagram showing a comparison of the channel attention module of CBAM and the channel attention module structure of the present invention;
FIG. 5 is a block diagram of a spatial attention module of the present invention;
FIG. 6 is a MRI reconstructed image of the brain for each reconstruction method at an undersampled rate of 30%;
FIG. 7 is a MRI reconstructed image of the brain for each reconstruction method at an undersampled rate of 25%;
FIG. 8 is a graph of PSNR and SSIM values for different addition ratios of tumors under model migration;
FIG. 9 is a graph of the results of PSNR and SSIM values for different tumor addition ratios under data enhancement;
Detailed Description
For further understanding of the structure and features of the present invention, reference should now be made in detail to the accompanying drawings of the preferred embodiments, which are included to provide a further understanding of the invention and are not intended to limit the invention to the embodiments described herein.
The first embodiment is as follows:
The ATTISTANET compressed sensing magnetic resonance reconstruction method based on an attention mechanism according to the present embodiment is characterized in that the method includes the following steps:
ISTANet + building a network structure;
H (-) and L (-) operators are added on the basis of ISTANet, wherein H k (-) and L k (-) are convolution kernels of 1 multiplied by 1, linear operators for increasing the number of channels and reducing the number of channels are respectively used, and a residual structure is used for reducing the difficulty of network training;
constructing an MRI reconstruction model based on an attention module;
Introducing an attention module in ISTANet + networks, adding an improved channel attention module and a spatial attention module before each reconstructed image x k, and carrying out channel and spatial feature recalibration on the features of the original K space data by using an attention mechanism module;
Step three, designing a loss function;
replacing the symmetrical constraint loss in ISTANet + by a smooth average absolute loss function;
Model enhancement and model migration based on T2 tumor images;
testing the model after data enhancement and model migration by using a brain T2 tumor data set and normal T1 brain image data; the second embodiment is as follows:
The present embodiment is further described in the first embodiment, which is a method for reconstructing ATTISTANET compressed sensing magnetic resonance based on an attention mechanism, wherein the specific process of the first step is as follows: h (-) and L (-) operators are added on the basis of ISTANet, wherein H k (-) and L k (-) are convolution kernels of 1×1, linear operators for increasing the number of channels and reducing the number of channels are respectively, and a residual structure is used for reducing the difficulty of network training, and the iteration of the kth stage in the ISTANet + method is as follows:
rk＝xk-1-ρkΦT(Φxk-1-y) (1)
xk＝Lk(Fk)T(soft(Fk(Hk(rk)),θk))+rk (2)
in the reconstruction of an MRI image, the adopted K space data features are scattered in different areas, and the equivalent processing of high-frequency and low-frequency parts in the K space data by ADMMNet, ISTANet + and other methods can reduce the extraction effect of a network on information of different areas and influence the identification capability of different types of features, so that the reconstruction accuracy is restricted from being improved.
And a third specific embodiment:
the present embodiment is further described in the first embodiment, which is a method for reconstructing ATTISTANET compressed sensing magnetic resonance based on an attention mechanism, wherein the specific process of the second step is as follows: introducing different attention modules in ISTANet + networks, adding an improved channel attention module and a spatial attention module before each reconstructed image x k is reconstructed and an intermediate variable r k, and carrying out channel and spatial feature recalibration on the features of the original K space data by using the attention module, wherein after passing through the channel attention module and the spatial attention module, the expression is as follows:
Ck＝Lk(Fk)T(soft(Fk(Hk(rk)),θk)) (3)
ak＝sigmoid(relu(w(Ck))) (4)
sk＝sigmoid(f(ak)) (5)
xk＝Ck×sk+rk (6)
The specific embodiment IV is as follows:
The embodiment is further described in the first embodiment, which is a method for reconstructing ATTISTANET compressed sensing magnetic resonance based on an attention mechanism, wherein the specific process of the third step is as follows: the symmetric constraint losses in ISTANet + are replaced with a smoothed average absolute loss function, ISTANet + introduces a symmetric constraint loss function due to the symmetry of the network. The loss function is as follows:
Ltotal(θ)＝Ldiscrepancy+γLconstraint (8)
wherein k, N b, N and The number of stages, the number of training total blocks, the size of each block and regularization parameters of ISTANet + respectively;
the formula (10) is replaced with a smoothed average absolute loss. Wherein δ is set to 0.5, the modified loss function is as follows:
Fifth embodiment:
This embodiment is a further illustration of a method for performing a ATTISTANET compressed-sensing magnetic resonance reconstruction based on an attention mechanism according to one embodiment, wherein the ISTANet + network includes an adaptive and input initial layer, x k,rk is a reconstructed image and an intermediate reconstruction module, respectively, and the network includes a depth structure of 1 to k stages, wherein the structure of each stage is identical, the forward transform F k is composed of two linear convolution operators, the linear operators are separated by a ReLU function, and the backward transforms (F k)T is structurally symmetrical to F k, wherein F k and F k)T satisfy F k·(Fk)T =i, where I is an identity matrix.
Performance test of reconstruction method:
(1) Parameter and data set settings
The test condition of the invention is CPU is Intel Core i7-9700k, the memory is 32G, the GPU is NVIDIA GeForce RTX 2070Super, the video memory is 8GB, the operating system built by the platform is Windows 10, the deep learning framework is Pytorch-GPU10.0, and the Python version is Python3.7.
Brain MRIT dataset using MICCAI2013 contest, MICCAI2018 contest MRIT dataset, and the head MRIT1 dataset of the empire university IXI database were tested. 100 groups of 3DMRIT brain images are selected from MICCAI2013 data sets, namely 13325 brain images with the size of 256 multiplied by 256 and 2DMRIT brain images with the size of 256 multiplied by 256 are selected, wherein 9829 brain images are used as training sets, and the total of 3495 brain images are used as test sets; 50 groups of 3DMRIT head images are selected from IXI data sets, 7293 brain images with the size of 256 multiplied by 256 and 2DMRIT brain images with the size of 256 multiplied by 256 are taken as training sets, and 2105 brain images with the size of 70 percent are taken as test sets;
The undersampled rates were set at 10%, 25%, 30% and 40%. For network optimization, adam optimization was used, the learning rate was set to 0.0001, the batch size was 64, and the number of stages was 9;
(2) Enhancing the data set;
The model parameters are prevented from being fitted by adopting a data enhancement mode, and the original data set is enhanced by adopting the brain tumor image, so that the addition mode is more in line with the requirements of actual MRI image reconstruction. Besides data enhancement, the invention also adopts a model migration mode to adjust parameters, and uses the brain T2 tumor data set and normal T1 brain image data to test the model after data enhancement and model migration. And a proper parameter adjusting mode is found through comparison of test results.
The invention selects 100 groups of 3DMRIT tumor images from MICCAI2018 data sets, wherein 1936 tumor images with the size of 256 multiplied by 256 of 2DMRIT tumor images. The T2 tumor dataset and MRIT1 dataset used in the present invention were used to expand at 10:1, 20:3, 5:1 ratios, respectively. In addition, a model migration mode is adopted, and parameters of the trained MRIT data set are finely adjusted by the T2 tumor data set according to the same proportion as data enhancement;
(3) Evaluating indexes and testing performance;
Under the same test conditions, the invention is compared with the test results of ISTANet, ISTANet +, FISTANet reconstructed MRI images. The invention tests the quality of the reconstructed MRI image from two aspects, namely an objective evaluation standard, adopts the structural similarity (Structural similarity index, SSIM) and the peak signal-to-noise ratio (PEAK SIGNAL to NoiseRatio, PSNR) as objective indexes for measuring the quality of the reconstructed MRI image, and adopts the Time (Time) as an index for measuring the speed of the reconstructed MRI; on the other hand, subjective visual effect is that whether the reconstructed image is good or not is judged by observing some texture details and a local difference image of the reconstructed image by human eyes;
Comparing the results of different methods shows that the method provided by the invention has higher objective index, and table 1 gives the quantitative results of all the methods of brain data under different undersampling rates, and the method provided by the patent has higher reconstruction accuracy. As can be seen in table 1, at each sampling rate, the objective index of the proposed method is close to or higher than that of other methods, especially at a sampling rate of 25% and above, the objective index of the proposed method is at least 1dB higher than that of other methods;
TABLE 1 brain MRI image objective index reconstructed at different sample rates
Table 2 shows the quantitative results of all methods for the header data at different undersampling rates. As can be seen from table 2, the method provided by the invention still has good performance under less training samples, and the objective index is still superior to other comparison methods;
TABLE 2 objective indicators of reconstructed head MRI images at different sample rates
Besides objective indexes, the difference between the method and other methods can be intuitively seen from the local images and the local difference images. Fig. 5 and fig. 6 are, from left to right, an original image, ISTANet, ISTANet +, FISTANet and an MRI image reconstructed by the method according to the present invention, and a corresponding partial image and a partial difference image are below each reconstructed image, and it is obvious from the partial images and the partial difference images of fig. 5 and fig. 6: compared with the MRI images reconstructed by other methods, the method has smaller gap with the original image, the texture details of the reconstructed brain MRI image are more vivid, and more original MRI image information is reserved.
Meanwhile, the invention also tests the proposed data enhancement and model migration and discusses the relation between the filling quantity and the reconstruction result. Table 3 shows that the results of data enhancement and model migration are significantly improved according to objective indexes, and the reconstruction results of tumor and normal brain images are improved in a data enhancement mode, while the parameters after adjustment have better reconstruction results of tumor images in a model migration mode.
TABLE 3 data enhancement of brain MRI image objective index for different modes with 25% sampling rate
Fig. 7 and 8 are trend graphs of objective indicators (PSNR and SSIM) at different ratios of model migration and data enhancement, respectively, as evident from fig. 7, with increasing tumor data. After model migration, the reconstruction result of the normal brain image shows a descending trend, and the reconstruction result of the tumor image shows an ascending trend; in fig. 8, after data enhancement, the reconstruction results of normal and tumor images are both in a steady or ascending trend. However, from a training time perspective, the training time for data enhancement is often tens of times greater than model migration. The two modes can prevent the model from being over fitted while the influence on the reconstruction accuracy of the image is less, and the robustness of the method is enhanced.
It should be noted that the foregoing summary and the detailed description are intended to demonstrate practical applications of the technical solution provided by the present invention, and should not be construed as limiting the scope of the present invention. Various modifications, equivalent alterations, or improvements will occur to those skilled in the art, and are within the spirit and principles of the invention. The scope of the invention is defined by the appended claims.
Claims (5)
1. A ATTISTANET compressed sensing magnetic resonance reconstruction method based on an attention mechanism is characterized in that: the method comprises the following steps:
step one: ISTANet + building a network structure;
H (-) and L (-) operators are added on the basis of ISTANet, wherein H k (-) and L k (-) are convolution kernels of 1 multiplied by 1, H k and L k are linear operators for increasing the number of channels and reducing the number of channels respectively, and a residual structure is used for reducing the difficulty of network training;
step two: constructing an MRI reconstruction model based on an attention module;
introducing different attention modules in ISTANet + networks, adding a channel attention module and a space attention module before each reconstructed image x k to obtain ATTISTANET algorithm, and carrying out channel and space feature recalibration on the features of the original K space data;
Step three: designing a loss function;
replacing the symmetrical constraint loss in ISTANet + by a smooth average absolute loss function;
Step four: model enhancement and model migration based on T2 tumor images;
The data-enhanced and model-migrated model was tested using the brain T2 tumor dataset and normal T1 brain image data.
2. A method of ATTISTANET compressed sensing magnetic resonance reconstruction based on an attention mechanism as defined in claim 1, wherein: the specific process of the first step is as follows: h (-) and L (-) operators are added on the basis of ISTANet, wherein H k (-) and L k (-) are convolution kernels of 1×1, which are linear operators for increasing the number of channels and reducing the number of channels respectively, and a residual structure is used for reducing the difficulty of network training, and the iteration of the kth stage in the ISTANet + method is as follows:
rk＝xk-1-ρkΦT(Φxk-1-y) (1)
xk＝Lk(Fk)T(soft(Fk(Hk(rk)),θk))+rk (2)
Wherein x k represents a reconstructed image, r k represents an intermediate variable, and k, ρ, Φ and y are the number of stages, the step size, the measurement matrix and the observation signal respectively; soft (·) represents a soft threshold function, θ represents a shrinkage threshold of the threshold function, F (·) represents a transform coefficient for a certain fixed transform F, T is a transposed symbol;
In the reconstruction of an MRI image, the used K space data features are scattered in different areas ADMMNet, ISTANet +, and the same processing of high-frequency and low-frequency parts in the K space data is performed by the algorithm, so that the extraction effect of a network on information of different areas is reduced, the recognition capability of different types of features is influenced, and the reconstruction precision is restrained from being improved.
3. A method of compressed aware magnetic resonance reconstruction of ATTISTANET based on an attention mechanism as claimed in claim 2, wherein: the specific process of the second step is as follows: introducing different attention modules in ISTANet + networks, adding a channel attention module and a space attention module before reconstructing an image x k and reconstructing an intermediate variable r k, and carrying out channel and space feature recalibration on the features of the original K space data by using the attention module, wherein after passing through the channel attention module and the space attention module, the expression is as follows:
Ck＝Lk(Fk)T(soft(Fk(Hk(rk)),θk)) (3)
ak＝sigmoid(relu(w(Ck))) (4)
sk＝sigmoid(f(ak)) (5)
xk＝Ck×sk+rk (6)
Wherein, C is the feature to be noted, a and s are the features processed by the channel attention module and the space attention module respectively, w is the weight scale of the multi-layer sensor, the feature can be scaled, and f represents a convolution operation;
The adopted channel attention mechanism and spatial attention mechanism can obtain richer characteristic information by an attention module, so that the reconstruction accuracy is improved, and the expression of the LSE function is as follows:
wherein S ij represents an activation value of (i, j), S represents a pooling area, (i, j) is a point in the pooling area S, N is a total point of the pooling area S, the pooling range can be iteratively updated along with the network from a maximum value (u- & gt infinity) to an average value (u- & gt 0) in the S through an adaptive super parameter u, and the loss of characteristic information can be reduced through LSE.
4. A method of ATTISTANET compressed sensing magnetic resonance reconstruction based on the attention mechanism as set forth in claim 3, wherein: the specific process of the third step is as follows: the idea according to the greedy algorithm replaces the symmetric constraint loss in ISTANet + with a smooth average absolute loss function that, due to the symmetry of the network,
ISTANet + introduces a loss function of the symmetry constraint, which is as follows:
Ltotal(θ)＝Ldiscrepancy+γLconstraint (8)
Wherein k, N b, N and γ are the number of stages of ISTANet +, the number of total training blocks, the size of each block and regularization parameters, respectively;
the formula (10) is replaced with a smoothed average absolute loss, where δ is set to 0.5, and the modified loss function is as follows:
5. The compressed sensing magnetic resonance reconstruction method based on the attention mechanism ATTISTANET as set forth in claim 4, wherein: ISTANet + the network comprises an adaptive and input initial layer, x k,rk is the reconstructed image and intermediate reconstruction module, respectively, the network comprises a1 to k stage depth structure, wherein the structure of each stage is identical, and the forward transform F k consists of two phases
Linear convolution operators, which are separated by ReLU functions, backward transforms (F k)T is structurally symmetric with F k, where F k and (F k)T satisfy F k·(Fk)T =i, I being an identity matrix).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111290411.5A CN113936073B (en) | 2021-11-02 | 2021-11-02 | ATTISTANET compressed sensing magnetic resonance reconstruction method based on attention mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111290411.5A CN113936073B (en) | 2021-11-02 | 2021-11-02 | ATTISTANET compressed sensing magnetic resonance reconstruction method based on attention mechanism |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113936073A CN113936073A (en) | 2022-01-14 |
| CN113936073B true CN113936073B (en) | 2024-05-14 |
Family
ID=79285255
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111290411.5A Active CN113936073B (en) | 2021-11-02 | 2021-11-02 | ATTISTANET compressed sensing magnetic resonance reconstruction method based on attention mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113936073B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108596994A (en) * | 2018-04-24 | 2018-09-28 | 朱高杰 | A kind of Diffusion-weighted imaging method being in harmony certainly based on deep learning and data |
| CN111161200A (en) * | 2019-12-22 | 2020-05-15 | 天津大学 | Human body posture migration method based on attention mechanism |
| CN111402137A (en) * | 2020-03-20 | 2020-07-10 | 南京信息工程大学 | Depth attention coding and decoding single image super-resolution algorithm based on perception loss guidance |
| CN112884851A (en) * | 2021-01-27 | 2021-06-01 | 武汉大学 | Deep compression sensing network for expanding iterative optimization algorithm |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11454689B2 (en) * | 2019-09-05 | 2022-09-27 | Canon Medical Systems Corporation | Magnetic resonance imaging apparatus, image processing apparatus, and image processing method |
| US11645791B2 (en) * | 2019-10-17 | 2023-05-09 | Rutgers, The State University Of New Jersey | Systems and methods for joint reconstruction and segmentation of organs from magnetic resonance imaging data |
-
2021
- 2021-11-02 CN CN202111290411.5A patent/CN113936073B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108596994A (en) * | 2018-04-24 | 2018-09-28 | 朱高杰 | A kind of Diffusion-weighted imaging method being in harmony certainly based on deep learning and data |
| CN111161200A (en) * | 2019-12-22 | 2020-05-15 | 天津大学 | Human body posture migration method based on attention mechanism |
| CN111402137A (en) * | 2020-03-20 | 2020-07-10 | 南京信息工程大学 | Depth attention coding and decoding single image super-resolution algorithm based on perception loss guidance |
| CN112884851A (en) * | 2021-01-27 | 2021-06-01 | 武汉大学 | Deep compression sensing network for expanding iterative optimization algorithm |
Non-Patent Citations (1)
| Title |
|---|
| 一种深度置信网络和迭代量化的图像检索方法;宋立新等;《信息与控制》;20191213;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113936073A (en) | 2022-01-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109191476B (en) | Novel biomedical image automatic segmentation method based on U-net network structure | |
| CN111754403B (en) | Image super-resolution reconstruction method based on residual learning | |
| CN110889853B (en) | Tumor segmentation method based on residual error-attention deep neural network | |
| CN110889852B (en) | Liver segmentation method based on residual error-attention deep neural network | |
| CN108805814B (en) | Image super-resolution reconstruction method based on multi-band deep convolutional neural network | |
| CN108090871A (en) | A kind of more contrast MR image reconstruction methods based on convolutional neural networks | |
| CN110276736B (en) | Magnetic resonance image fusion method based on weight prediction network | |
| CN116563204A (en) | Medical image segmentation method integrating multi-scale residual attention | |
| CN112164122A (en) | Rapid CS-MRI reconstruction method for generating countermeasure network based on depth residual error | |
| CN111487573A (en) | Enhanced residual error cascade network model for magnetic resonance undersampling imaging | |
| CN114972248A (en) | Attention mechanism-based improved U-net liver tumor segmentation method | |
| CN115375711A (en) | Image segmentation method of global context attention network based on multi-scale fusion | |
| CN114266939A (en) | Brain extraction method based on ResTLU-Net model | |
| Yan et al. | SMIR: A Transformer-Based Model for MRI super-resolution reconstruction | |
| CN117237196A (en) | Brain MRI super-resolution reconstruction method and system based on implicit neural characterization | |
| CN113936073B (en) | ATTISTANET compressed sensing magnetic resonance reconstruction method based on attention mechanism | |
| CN112541566B (en) | Image translation method based on reconstruction loss | |
| CN116993639A (en) | Visible light and infrared image fusion method based on structural re-parameterization | |
| CN116645283A (en) | Low-dose CT image denoising method based on self-supervision perceptual loss multi-scale convolutional neural network | |
| CN116630178A (en) | U-Net-based power frequency artifact suppression method for ultra-low field magnetic resonance image | |
| CN114723937A (en) | Method and system for classifying blood vessel surrounding gaps based on nuclear magnetic resonance image | |
| CN114494952A (en) | Breast MRI (magnetic resonance imaging) image time sequence generation method based on perception loss | |
| Pan et al. | Image Super-Resolution Model Based on Bypass Convolution and Feature Weighting | |
| CN114882138A (en) | Reconstruction method for multi-layer CS-MRI | |
| Zhou | Multithreshold Microbial Image Segmentation Using Improved Deep Reinforcement Learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |