CN111598966A - Magnetic resonance imaging method and device based on generation countermeasure network - Google Patents

Magnetic resonance imaging method and device based on generation countermeasure network Download PDF

Info

Publication number
CN111598966A
CN111598966A CN202010420192.7A CN202010420192A CN111598966A CN 111598966 A CN111598966 A CN 111598966A CN 202010420192 A CN202010420192 A CN 202010420192A CN 111598966 A CN111598966 A CN 111598966A
Authority
CN
China
Prior art keywords
image
mri
data
undersampled
current
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010420192.7A
Other languages
Chinese (zh)
Other versions
CN111598966B (en
Inventor
张贺晔
郭宜锋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
National Sun Yat Sen University
Original Assignee
National Sun Yat Sen University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by National Sun Yat Sen University filed Critical National Sun Yat Sen University
Priority to CN202010420192.7A priority Critical patent/CN111598966B/en
Publication of CN111598966A publication Critical patent/CN111598966A/en
Application granted granted Critical
Publication of CN111598966B publication Critical patent/CN111598966B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/003—Reconstruction from projections, e.g. tomography
    • G06T11/006—Inverse problem, transformation from projection-space into object-space, e.g. transform methods, back-projection, algebraic methods
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
    • G01R33/00—Arrangements or instruments for measuring magnetic variables
    • G01R33/20—Arrangements or instruments for measuring magnetic variables involving magnetic resonance
    • G01R33/44—Arrangements or instruments for measuring magnetic variables involving magnetic resonance using nuclear magnetic resonance [NMR]
    • G01R33/48—NMR imaging systems
    • G01R33/4818—MR characterised by data acquisition along a specific k-space trajectory or by the temporal order of k-space coverage, e.g. centric or segmented coverage of k-space
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
    • G01R33/00—Arrangements or instruments for measuring magnetic variables
    • G01R33/20—Arrangements or instruments for measuring magnetic variables involving magnetic resonance
    • G01R33/44—Arrangements or instruments for measuring magnetic variables involving magnetic resonance using nuclear magnetic resonance [NMR]
    • G01R33/48—NMR imaging systems
    • G01R33/54—Signal processing systems, e.g. using pulse sequences ; Generation or control of pulse sequences; Operator console
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
    • G01R33/00—Arrangements or instruments for measuring magnetic variables
    • G01R33/20—Arrangements or instruments for measuring magnetic variables involving magnetic resonance
    • G01R33/44—Arrangements or instruments for measuring magnetic variables involving magnetic resonance using nuclear magnetic resonance [NMR]
    • G01R33/48—NMR imaging systems
    • G01R33/54—Signal processing systems, e.g. using pulse sequences ; Generation or control of pulse sequences; Operator console
    • G01R33/56—Image enhancement or correction, e.g. subtraction or averaging techniques, e.g. improvement of signal-to-noise ratio and resolution
    • G01R33/5608—Data processing and visualization specially adapted for MR, e.g. for feature analysis and pattern recognition on the basis of measured MR data, segmentation of measured MR data, edge contour detection on the basis of measured MR data, for enhancing measured MR data in terms of signal-to-noise ratio by means of noise filtering or apodization, for enhancing measured MR data in terms of resolution by means for deblurring, windowing, zero filling, or generation of gray-scaled images, colour-coded images or images displaying vectors instead of pixels
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • High Energy & Nuclear Physics (AREA)
  • Condensed Matter Physics & Semiconductors (AREA)
  • Signal Processing (AREA)
  • Algebra (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)

Abstract

The application provides a magnetic resonance imaging method and device based on generation of a countermeasure network, comprising the following steps: establishing a corresponding relation between undersampled MRI data and image characteristics of an MRI image by utilizing the self-learning capability of an artificial neural network; specifically, determining the correlation between data segments corresponding to adjacent time sequences; determining a target spatial feature in the undersampled MRI data; determining a corresponding relation between the undersampled MRI data and the image characteristics of the MRI image according to the correlation and the target space characteristics; acquiring current undersampled MRI data of a current detected person; determining the image characteristics of the current MRI image corresponding to the current undersampled MRI data through the corresponding relation; specifically, determining image features of a current MRI image corresponding to current undersampled MRI data includes: and determining the image characteristics of the MRI image corresponding to the undersampled MRI data which is the same as the current undersampled MRI data in the corresponding relation as the image characteristics of the current MRI image. The generation of a large amount of residual noise during the reconstruction process is avoided.

Description

Magnetic resonance imaging method and device based on generation countermeasure network
Technical Field
The present application relates to the field of medical detection, and in particular, to a magnetic resonance imaging method and apparatus based on a generation countermeasure network.
Background
As a medical Imaging modality that is widely used in clinical applications, Magnetic Resonance Imaging (MRI) is capable of providing repeatable, non-invasive and quantitative measurements of tissue, as well as providing information including structure, anatomy and function. One of the major drawbacks that currently prevents wider application of MRI is the lengthy acquisition time, which is mainly related to its inherent acquisition speed. MRI acquisition data is acquired through frequency domain information of K-space (containing frequency information acquired line by line) rather than in the image domain. The lengthy acquisition time makes the MRI results susceptible to patient motion and physiological motion, such as heart beat, respiratory offset, and gastrointestinal peristalsis. In addition, for MRI coupled with contrast agent injection, lengthy acquisition times may result in too low a contrast ratio, which may result in poor quality or non-diagnostic images.
One existing fast MRI method is by undersampling the K-space data, however this approach typically results in a positive correlation between the acceleration factor and the undersampling rate. Donoho et al, IEEE journal of information treatises 2006, have proposed the application of compressed sensing-based MRI reconstruction methods (CS-MRI) to accelerate MRI procedures. The technique uses a small portion of data to reconstruct an image that is undersnyquist sampled. CS-MRI non-linearly optimizes undersampled data without significantly degrading the quality of the reconstructed image, assuming that the raw data is compressible. However, how to consolidate the speed of image reconstruction and the robustness of image quality preservation remains a very challenging problem in a CS-MRI based framework. In one aspect, CS-MRI attempts to solve an underdetermined equation to obtain the raw signal from limited undersampled data. This requires a non-linear optimization to solve for a common non-convex system, which typically involves iterative computations, which may result in extended reconstruction times. CS-MRI, on the other hand, may produce images of reduced image quality and low signal-to-noise ratio from random, highly undersampled k-space data. Furthermore, in addition to the large number of computations required for nonlinear optimization, CS-MRI requires that the acquisition matrix be independent of the sparse transformation matrix. In summary, the acceleration factor of CS-MRI is generally between 2 and 6.
The main disadvantages of the prior art are as follows: the end-to-end training mode ignores the correlation between adjacent two-dimensional slices in the MRI data sequence; the dynamic training mode lacks the k-space information which can not effectively utilize a single image; a large amount of residual noise is generated based on compressed perceptual reconstruction.
Disclosure of Invention
In view of the above, the present application is proposed to provide a magnetic resonance imaging method and apparatus based on generation of a countermeasure network that overcomes or at least partially solves the above problems, including:
a magnetic resonance imaging method based on generation countermeasure network is applied to imaging under-sampled MRI data acquired by a compressed sensing magnetic resonance imaging device, wherein the under-sampled MRI data comprises a plurality of data sections acquired and arranged according to time sequence, and the method comprises the following steps:
establishing a corresponding relation between undersampled MRI data and image characteristics of an MRI image by utilizing the self-learning capability of an artificial neural network; specifically, determining the correlation between the data segments corresponding to adjacent time sequences; determining a target spatial feature in the undersampled MRI data; determining a corresponding relation between the undersampled MRI data and image features of an MRI image according to the correlation and the target space features;
acquiring current undersampled MRI data of a current detected person;
determining the image characteristics of the current MRI image corresponding to the current undersampled MRI data according to the corresponding relation; specifically, determining image features of a current MRI image corresponding to the current undersampled MRI data includes: and determining the image characteristics of the MRI image corresponding to the undersampled MRI data which is the same as the current undersampled MRI data in the corresponding relation as the image characteristics of the current MRI image.
Further, the correspondence includes: a functional relationship; the undersampled MRI data is an input parameter of the functional relationship, and the image characteristics of the MRI image are output parameters of the functional relationship;
determining image features of a current MRI image corresponding to the current undersampled MRI data, further comprising:
and when the corresponding relation comprises a functional relation, inputting the current undersampled MRI data into the functional relation, and determining the output parameter of the functional relation as the image characteristic of the current MRI image.
Further, the step of establishing a correspondence between the undersampled MRI data and image features of the MRI image includes:
acquiring sample data for establishing a corresponding relation between the undersampled MRI data and image features of the MRI image;
analyzing the characteristics and the rules of the undersampled MRI data, and determining the network structure and the network parameters of the artificial neural network according to the characteristics and the rules;
training and testing the network structure and the network parameters using the sample data, and determining the correspondence of the undersampled MRI data to image features of the MRI image.
Further, the step of acquiring sample data for establishing a correspondence between the undersampled MRI data and image features of the MRI image includes:
collecting image features of the undersampled MRI data and the MRI images of different subjects;
analyzing the under-sampled MRI data, and selecting data related to the image characteristics of the MRI image as the under-sampled MRI data by combining with prestored expert experience information;
and taking the image characteristics of the MRI image and the data pair formed by the selected undersampled MRI data as sample data.
Further, training the network structure and the network parameters includes:
selecting a part of data in the sample data as a training sample, inputting the under-sampled MRI data in the training sample into the network structure, and training by a loss function of the network structure, an activation function and the network parameters to obtain an actual training result;
determining whether an actual training error between the actual training result and an image feature of a corresponding MRI image in the training sample satisfies a preset training error;
determining that the training of the network structure and the network parameters is completed when the actual training error meets the preset training error;
and/or the presence of a gas in the gas,
testing the network structure and the network parameters, comprising:
selecting another part of data in the sample data as a test sample, inputting the under-sampled MRI data in the test sample into the trained network structure, and testing by using the loss function, the activation function and the trained network parameters to obtain an actual test result;
determining whether an actual test error between the actual test result and an image feature of a corresponding MRI image in the test sample satisfies a set test error;
and when the actual test error meets the set test error, determining that the test on the network structure and the network parameters is finished.
Further, the loss function includes a combined image domain mean square error loss function, a frequency domain mean square error loss function, an image processing content aware loss function, and a countering loss function.
Further, training the network structure and the network parameters further includes:
when the actual training error does not meet the set training error, updating the network parameters through an error loss function of the network structure;
activating a function and the updated network parameters to retrain through the loss function of the network structure until the retrained actual training error meets the set training error;
and/or the presence of a gas in the gas,
testing the network structure and the network parameters, further comprising:
and when the actual test error does not meet the set test error, retraining the network structure and the network parameters until the retrained actual test error meets the set test error.
A magnetic resonance imaging device based on a generation countermeasure network is applied to imaging under-sampled MRI data acquired by a compressed sensing magnetic resonance imaging device, wherein the under-sampled MRI data comprises a plurality of data sections which are acquired and arranged according to a time sequence, and the magnetic resonance imaging device comprises:
the establishing module is used for establishing a corresponding relation between the undersampled MRI data and the image characteristics of the MRI image by utilizing the self-learning capability of the artificial neural network; specifically, determining the correlation between the data segments corresponding to adjacent time sequences; determining a target spatial feature in the undersampled MRI data; determining a corresponding relation between the undersampled MRI data and image features of an MRI image according to the correlation and the target space features;
the acquisition module is used for acquiring current undersampled MRI data of a current detected person;
a determining module, configured to determine, according to the correspondence, an image feature of a current MRI image corresponding to the current undersampled MRI data; specifically, determining image features of a current MRI image corresponding to the current undersampled MRI data includes: and determining the image characteristics of the MRI image corresponding to the undersampled MRI data which is the same as the current undersampled MRI data in the corresponding relation as the image characteristics of the current MRI image.
An apparatus comprising a processor, a memory and a computer program stored on the memory and executable on the processor, the computer program, when being executed by the processor, realizing the steps of the magnetic resonance imaging method based on generation of a countermeasure network as described above.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the magnetic resonance imaging method based on generation of a countermeasure network as described above.
The application has the following advantages:
in the embodiment of the application, the self-learning capability of the artificial neural network is utilized to establish the corresponding relation between the undersampled MRI data and the image characteristics of the MRI image; specifically, determining the correlation between the data segments corresponding to adjacent time sequences; determining a target spatial feature in the undersampled MRI data; determining a corresponding relation between the undersampled MRI data and image features of an MRI image according to the correlation and the target space features; acquiring current undersampled MRI data of a current detected person; determining the image characteristics of the current MRI image corresponding to the current undersampled MRI data according to the corresponding relation; specifically, determining image features of a current MRI image corresponding to the current undersampled MRI data includes: and determining the image characteristics of the MRI image corresponding to the undersampled MRI data which is the same as the current undersampled MRI data in the corresponding relation as the image characteristics of the current MRI image. Reconstruction of MRI images from more highly MRI undersampled k-space data is achieved; not only can better reconstruction results be realized, but also a large amount of residual noise can be avoided in the reconstruction process.
Drawings
In order to more clearly illustrate the technical solutions of the present application, the drawings needed to be used in the description of the present application will be briefly introduced below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive labor.
Fig. 1 is a flowchart illustrating steps of a magnetic resonance imaging method based on generation of a countermeasure network according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a generation network part in an artificial neural network based on a magnetic resonance imaging method for generating a countermeasure network according to an embodiment of the present application;
fig. 3 is a diagram illustrating a reconstructed image result using 10% k-space data based on qualitative visualization of a magnetic resonance imaging method for generating a countermeasure network according to an embodiment of the present application;
FIG. 4 is a graph illustrating PSNR distribution results of various methods at different noise levels under different undersampling rates of a magnetic resonance imaging method based on a generation countermeasure network according to an embodiment of the present application;
FIG. 5 is a graph illustrating residual noise results of different methods at different undersampling rates based on a magnetic resonance imaging method for generating a countermeasure network according to an embodiment of the present application;
fig. 6 is a block diagram of a magnetic resonance imaging apparatus based on generation of a countermeasure network according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a computer device according to an embodiment of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, the present application is described in further detail with reference to the accompanying drawings and the detailed description. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, a magnetic resonance imaging method based on a generative countermeasure network according to an embodiment of the present application is shown, and is applied to image under-sampled MRI data acquired by a compressive sensing magnetic resonance imaging apparatus, where the under-sampled MRI data includes a plurality of data segments acquired and arranged according to a time sequence, and includes:
s110, establishing a corresponding relation between undersampled MRI data and image characteristics of an MRI image by utilizing the self-learning capability of an artificial neural network; specifically, determining the correlation between the data segments corresponding to adjacent time sequences; determining a target spatial feature in the undersampled MRI data; determining a corresponding relation between the undersampled MRI data and image features of an MRI image according to the correlation and the target space features;
s120, acquiring current undersampled MRI data of a current detected person;
s130, determining the image characteristics of the current MRI image corresponding to the current undersampled MRI data according to the corresponding relation; specifically, determining image features of a current MRI image corresponding to the current undersampled MRI data includes: and determining the image characteristics of the MRI image corresponding to the undersampled MRI data which is the same as the current undersampled MRI data in the corresponding relation as the image characteristics of the current MRI image.
In the embodiment of the application, the self-learning capability of the artificial neural network is utilized to establish the corresponding relation between the undersampled MRI data and the image characteristics of the MRI image; specifically, determining the correlation between the data segments corresponding to adjacent time sequences; determining a target spatial feature in the undersampled MRI data; determining a corresponding relation between the undersampled MRI data and image features of an MRI image according to the correlation and the target space features; acquiring current undersampled MRI data of a current detected person; determining the image characteristics of the current MRI image corresponding to the current undersampled MRI data according to the corresponding relation; specifically, determining image features of a current MRI image corresponding to the current undersampled MRI data includes: and determining the image characteristics of the MRI image corresponding to the undersampled MRI data which is the same as the current undersampled MRI data in the corresponding relation as the image characteristics of the current MRI image. Reconstruction of MRI images from more highly MRI undersampled k-space data is achieved; not only can better reconstruction results be realized, but also a large amount of residual noise can be avoided in the reconstruction process.
Next, a magnetic resonance imaging method based on generation of the countermeasure network in the present exemplary embodiment will be further described.
As described in the above step S110, the self-learning capability of the artificial neural network is used to establish the corresponding relationship between the under-sampled MRI data and the image features of the MRI image; specifically, determining the correlation between the data segments corresponding to adjacent time sequences; determining a target spatial feature in the undersampled MRI data; and determining the corresponding relation between the undersampled MRI data and the image characteristics of the MRI image according to the correlation and the target space characteristics.
For example: and analyzing a display state rule corresponding to the image feature of the MRI image by utilizing an artificial neural network algorithm, and finding out a mapping rule between the undersampled MRI data of the examinee and the image feature of the MRI image through self-learning and self-adaptive characteristics of the artificial neural network.
For example: the method can utilize an artificial neural network algorithm, collect and collect the undersampled MRI data of a large number of different testees (including but not limited to one or more of age, weight, sex, disease state and the like), select the undersampled MRI data of a plurality of testees and the image characteristics of MRI images as sample data, learn and train the neural network, fit the relationship between the undersampled MRI data and the image characteristics of the MRI images by adjusting the weight between the network structure and the network nodes, and finally enable the neural network to accurately fit the corresponding relationship between the undersampled MRI data of different testees and the image characteristics of the MRI images.
It should be noted that, in any embodiment of the present application, the artificial neural network is a generation countermeasure network (GAN), where the GAN network includes three key components: Bi-ConvLSTM subnetworks, spatial attention Module (SAB), and WGAN-GP as evaluation functions.
Bi-ConvLSTM subnetworks: the redundancy of information in picture sequences is used to improve the MRI reconstruction results without encoding the a priori frequency and time domain information. K-space information can be sufficiently mined based on an end-to-end training mode of a U-Net generator, and time information supplement is carried out on each two-dimensional slice of each frame of the sequence through a Bi-ConvLSTM sub-network, so that the quality of generated images is improved.
Spatial attention module (SAB): the attention unit may distinguish important and unimportant features in the MRI reconstruction task. And generating a two-dimensional space attention diagram by the attention unit through the features obtained by the U-Net network learning, and fusing the attention diagram with the input features so as to learn to reconstruct the attention area.
WGAN-GP merit function: the WGAN model with a gradient penalty term (WGAN-gp) was applied as an evaluation function to significantly improve the stability of GAN.
In an embodiment, the correspondence includes: and (4) functional relation.
Preferably, the undersampled MRI data is an input parameter of the functional relationship, and the image characteristic of the MRI image is an output parameter of the functional relationship;
determining image features of a current MRI image corresponding to the current undersampled MRI data, further comprising:
and when the corresponding relation comprises a functional relation, inputting the current undersampled MRI data into the functional relation, and determining the output parameter of the functional relation as the image characteristic of the current MRI image.
Therefore, the flexibility and convenience of determining the image characteristics of the current MRI image can be improved through the corresponding relations in various forms.
For example: the artificial neural network algorithm can be utilized, a large number of under-sampled MRI data of different testees (including but not limited to one or more of age, sex, occupation and the like) are collected, under-sampled MRI data of a plurality of testees and image characteristics of MRI images are selected as sample data, the artificial neural network is learned and trained, the artificial neural network is enabled to fit the relationship between the image characteristics of the MRI images and the under-sampled MRI data by adjusting the weight between the network structure and network nodes, and finally the artificial neural network can accurately fit the corresponding relationship between the image characteristics of the MRI images of different testees and the under-sampled MRI data.
Referring to fig. 2, as an example, the artificial neural network mainly includes two parts, namely a deep attention generating network and a discriminating network, for generating the confrontation network, wherein the deep attention generating network includes three key modules: Bi-ConvLSTM subnetworks, spatial attention modules (SABs), and WGAN-GP as evaluation functions, wherein the deep attention generating network and SAB module framework are shown in FIG. 2.
In the Bi-ConvLSTM subnetwork, to achieve a highly undersampled reconstruction, a method has been proposed to encode a priori frequency and time domain information in the MRI sequence data, e.g., 2D MRI slices of 3D volumetric data. The end-to-end training approach can effectively utilize frequency domain information in the K-space domain, but correlation between adjacent two-dimensional slices in the MRI sequence data is easily neglected.
Let X be the characteristic representation of a two-dimensional series of MRI data slices in the entire three-dimensional space.
Figure BDA0002496665810000091
The ith slice, the ith iteration, is indicated. Need to be considered in the reconstruction process
Figure BDA0002496665810000092
And
Figure BDA0002496665810000093
is composed of
Figure BDA0002496665810000094
Providing the information. For this purpose, temporal and iterative dependencies are jointly exploited by the Bi-ConvLSTM sub-network. The Bi-ConvLSTM sub-network may be represented as:
Figure BDA0002496665810000095
Figure BDA0002496665810000096
Figure BDA0002496665810000097
in the formula (I), the compound is shown in the specification,
Figure BDA0002496665810000098
which represents the forward direction of the vehicle,
Figure BDA0002496665810000099
representing the backward direction. By Bi-CoThe nvLSTM subnetwork allows the neural network to learn the differences and correlations between successive MRI data slices. The output of the Bi-ConvLSTM layer is then connected in refinement to prevent data drift.
In the spatial attention module (SAB), the main purpose of adding the SAB is to improve expressiveness through an attention mechanism, i.e., to focus on important features, suppressing unnecessary features.
Specifically, the SAB is set after the first volume block, which is also propagated to the upsampling layer through skip connection. And carrying out average pool operation and maximum pool operation on the feature map obtained from the upper layer to generate a valid feature description. The feature map is then generated using the convolutional layer to achieve where the encoded image is emphasized or suppressed. The SAB calculates the attention map using all the features extracted by the upper layer.
Assume that the 2D mapping generated by the pool operation is
Figure BDA00024966658100000910
And
Figure BDA00024966658100000911
which are represented as the average pool feature and the maximum pool feature of the feature map, respectively. And then, the two characteristic graphs are superposed and convoluted through a standard convolution layer to obtain a two-dimensional space attention graph. Thus, the spatial attention map is calculated as:
Ms(F)＝σ(f7×7([Favg；Fmax]))
in the formula (f)7×7Representing a convolution operation with a filter size of 7 × 7, sigma representing a sigmoid function spatial attention finds the fundamental features across the entire spatial domain by computing the feature correlation across the channel domain.
Preferably, the network structure comprises a Bi-ConvLSTM network and a discrimination network; wherein the Bi-ConvLSTM network is used for determining correlation between the data segments corresponding to adjacent time sequences;
preferably, the network parameters include: at least one of the number of convolution layers, the number of cavity convolution layers, the number of BN layers, the type of activation function, the size of convolution kernel, the number of convolution kernels, the number of pooling layers, the number of upsampling layers, the number of output layers, the initial weight, and the offset value.
In an embodiment, a specific process of "establishing a correspondence between undersampled MRI data and image features of an MRI image" in step S110 may be further explained in conjunction with the following description.
The following steps are described: acquiring sample data for establishing a corresponding relation between the undersampled MRI data and image features of the MRI image;
in an advanced embodiment, a specific process of acquiring sample data for establishing a correspondence between the undersampled MRI data and image features of the MRI image may be further described in conjunction with the following description.
The following steps are described: collecting image features of the undersampled MRI data and the MRI images of different subjects;
for example: data collection: collecting undersampled MRI data of detected persons with different health conditions and corresponding image characteristics of MRI images; collecting undersampled MRI data of examinees of different ages and corresponding image characteristics of MRI images; and collecting undersampled MRI data of examinees of different genders and corresponding image characteristics of MRI images.
Therefore, the operation data are collected through multiple ways, the quantity of the operation data is increased, the learning capacity of the artificial neural network is improved, and the accuracy and the reliability of the determined corresponding relation are improved.
The following steps are described: analyzing the under-sampled MRI data, and selecting data related to the image characteristics of the MRI image as the under-sampled MRI data by combining with prestored expert experience information (for example, selecting the under-sampled MRI data which affects the image characteristics of the MRI image as input parameters, and using specified parameters as output parameters);
for example: the undersampled MRI data in the relevant data of the diagnosed examinee is used as an input parameter, and the image characteristics of the MRI image in the relevant data are used as output parameters.
The following steps are described: and taking the image characteristics of the MRI image and the data pair formed by the selected undersampled MRI data as sample data.
For example: and using part of the obtained input and output parameter pairs as training sample data and using part of the obtained input and output parameter pairs as test sample data.
Therefore, the collected under-sampled MRI data is analyzed and processed to further obtain sample data, the operation process is simple, and the reliability of the operation result is high.
The following steps are described: analyzing the characteristics and the rules of the undersampled MRI data, and determining the network structure and the network parameters of the artificial neural network according to the characteristics and the rules;
for example: the basic structure of the network, the number of input and output nodes of the network, the number of hidden nodes, the initial weight of the network and the like can be preliminarily determined by analyzing the image characteristics of the undersampled MRI data and the MRI image.
Optionally, the specific process of training the network structure and the network parameters in the step of using the sample data to train and test the network structure and the network parameters and determining the corresponding relationship between the undersampled MRI data and the image features of the MRI image may be further explained in conjunction with the following description.
Selecting a part of data in the sample data as a training sample, inputting the under-sampled MRI data in the training sample into the network structure, and training through a loss function of the network structure, an activation function and the network parameters to obtain an actual training result;
specifically, a loss function is minimized through a gradient descent algorithm, network parameters are updated, a current neural network model is trained, and an actual training result is obtained;
determining whether an actual training error between the actual training result and an image feature of a corresponding MRI image in the training sample satisfies a preset training error; determining that the training of the network structure and the network parameters is completed when the actual training error meets the preset training error;
specifically, when the actual training error satisfies the preset training error, and the currently trained model converges, it is determined that the training of the network structure and the network parameters is completed.
More optionally, training the network structure and the network parameters further includes:
when the actual training error does not meet the set training error, updating the network parameters through an error loss function of the network structure; activating a function and the updated network parameters to retrain through the loss function of the network structure until the retrained actual training error meets the set training error;
for example: and if the test error meets the requirement, finishing the network training test.
Therefore, the reliability of the network structure and the network parameters is further verified by using the test sample for testing the network structure and the network parameters obtained by training.
Optionally, a specific process of testing the network structure and the network parameters in the step of using the sample data to train and test the network structure and the network parameters and determining the corresponding relationship between the undersampled MRI data and the image features of the MRI image may be further explained in conjunction with the following description.
Selecting another part of data in the sample data as a test sample, inputting the under-sampled MRI data in the test sample into the trained network structure, and testing by using the loss function, the activation function and the trained network parameters to obtain an actual test result; determining whether an actual test error between the actual test result and an image feature of a corresponding MRI image in the test sample satisfies a set test error; and when the actual test error meets the set test error, determining that the test on the network structure and the network parameters is finished.
In an alternative example, the loss function comprises a combined image domain mean square error loss function LiMSEFrequency domain mean square error loss function LfMSEImage processing content perceptual loss function LVGGAnd a function of the penalty for fight LGEN。
Specifically, the overall loss function can be expressed as:
Figure BDA0002496665810000121
in the formula, α, β, γ represent hyper-parameters.
The loss of confrontation is as follows:
Figure BDA0002496665810000122
to improve perceptual quality, three different combinations of content loss and loss function are combined:
Figure BDA0002496665810000131
Figure BDA0002496665810000132
Figure BDA0002496665810000133
note that by using normalized mse (nmse) as the optimization function. However, using only NMSE as a loss of content may result in a perceptually non-uniform reconstruction and lack of consistent image detail. Therefore, to account for the perceptual similarity of images, NMSE and VGG loss (L) of frequency domain data is also addedVGG) As an additional constraint.
In an alternative example of this, the user may,
optionally, training the network structure and the network parameters may further include:
and when the actual training error does not meet the set training error, updating the network parameters through an error energy function of the network structure.
Retraining through the activation function of the network structure and the updated network parameters until the retrained actual training error meets the set training error.
Therefore, the network parameters are modified and retrained when the training errors are large, so that a more accurate and reliable network structure can be obtained, and a more accurate and reliable corresponding relation can be obtained.
Optionally, the testing the network structure and the network parameters may further include:
and when the actual test error does not meet the set test error, retraining the network structure and the network parameters until the retrained actual test error is slower than the set test error.
For example: and when the test error does not meet the requirement, repeating the steps and retraining the network.
Therefore, the network structure is retrained to be retested when the test error is large, so that the network structure which is more accurate and reliable is obtained, and the accuracy of determining the frosting state is improved.
As described in step S120 above, current under-sampled MRI data of the subject is obtained;
as described in step S130 above, the image feature of the current MRI image corresponding to the current undersampled MRI data is determined by the correspondence.
For example: under-sampled MRI data of a subject is identified in real-time.
Therefore, the image characteristics of the current MRI image are effectively identified according to the current undersampled MRI data based on the corresponding relation, so that accurate judgment basis is provided for the diagnosis of a tester, and the judgment result is good in accuracy.
In an alternative example, the determining of the image feature of the current MRI image corresponding to the undersampled MRI data in step S130 may include: and determining the image characteristics of the MRI image corresponding to the undersampled MRI data which is the same as the current undersampled MRI data in the corresponding relation as the image characteristics of the current MRI image.
In an optional example, the determining of the image feature of the current MRI image corresponding to the undersampled MRI data in step S130 may further include: when the corresponding relation can comprise a functional relation, inputting the current undersampled MRI data into the functional relation, and determining the output parameter of the functional relation as the image characteristic of the current MRI image.
Therefore, the image characteristics of the current MRI image are determined according to the current undersampled MRI data based on the corresponding relation or the functional relation, the determination mode is simple and convenient, and the reliability of the determination result is high.
For example, the artificial neural network model obtained by training is used to detect the image features of the MRI images of each sample in the test set.
In an alternative embodiment, the method may further include: and verifying whether the image characteristics of the current MRI image are consistent with the image characteristics of the actual MRI image.
Optionally, when a verification result that the image features of the current MRI image do not conform to the image features of the actual MRI image is received and/or it is determined that there is no undersampled MRI data in the correspondence that is the same as the current undersampled MRI data, at least one maintenance operation of updating, correcting, and relearning the correspondence may be performed.
For example: the device itself cannot know the image characteristics of the actual MRI image, and needs a feedback operation of a tester, that is, if the device intelligently judges the image characteristics of the MRI image, the tester can know the image characteristics by operating and feeding back the image characteristics which are not in accordance with the actual state.
And verifying whether the image features of the current MRI image are consistent with the image features of the actual MRI image (for example, displaying the image features of the actual MRI image through an AR display module to verify whether the determined image features of the current MRI image are consistent with the image features of the actual MRI image).
And when the image characteristics of the current MRI image are not consistent with the image characteristics of the actual MRI image and/or the corresponding relation does not have undersampled MRI data which is the same as the current undersampled MRI data, at least one maintenance operation of updating, correcting and relearning is carried out on the corresponding relation.
For example: the image characteristics of the current MRI image may be determined from the current undersampled MRI data according to the maintained correspondence. For example: and determining the image characteristics of the MRI image corresponding to the undersampled MRI data which is the same as the current undersampled MRI data in the maintained corresponding relation as the image characteristics of the current MRI image.
Therefore, the corresponding relation between the determined under-sampled MRI data and the image characteristics of the MRI image is maintained, and the accuracy and the reliability of determining the image characteristics of the MRI image are improved.
Referring to fig. 3-5, in one specific implementation, the experimental platform was an ubuntu16.04 server with 4 Tesla P40 (24 g of video memory per video card). The deep learning framework is the ensorlayer 1.7.0. The optimizer used in artificial neural network (DAWGAN) training of this embodiment is using Adam, and during the training, the initial learning rates of the generation network and the discrimination network are 0.001 and 0.001 respectively, and the decay rate is 0.96.
The performance of the network is measured using four criteria:
(1) PSNR (Peak Signal to Noise Ratio, PSNR) for measuring the image quality after processing: the higher the PSNR value, the better the model reconstruction effect.
(2) The Mean Opinion Score (MOS) of human perception is considered, which is the result of a domain expert evaluating the reconstruction and averaging its perceptual quality: the higher the MOS value is, the better the model reconstruction effect is.
(3) Normalized Mean absolute error between predicted and true values (NMSE): the lower the NMSE value, the better the model reconstruction.
(4) Normalized mutual information between predicted and true values (Structural Similarity Index, SSIM): the higher the SSIM value, the better the model reconstruction effect
The network of the present embodiment is compared with other existing methods. Other existing methods include Zero-filing, ADMM, DAGAN, and CRNN. The results of the experiment are shown in table 1.
Figure BDA0002496665810000161
TABLE 1
From the results of table one, it can be seen that the DAWGAN of the present embodiment performs best in PSNR and MOS. At 10 x and 3.3 x acceleration factors, the PSNR and MOS obtained by DAWGAN are significantly higher than other methods. Furthermore, as can be seen from fig. 3, the DAWGAN produced less noise in all simulation studies, while the other methods produced more noise. While CRNN and DAGAN may also suppress some noise, the reconstruction of the region of interest is less detailed than the DAWGAN reconstruction. Furthermore, the ADMM and zero-padding approach does not effectively suppress the remaining aliasing noise.
Optionally, reconstructing the image residual noise estimate, and testing the noise reduction effect of all models at different noise levels under different acceleration factors, so as to prove that the DAWGAN can significantly suppress the residual noise. To test the noise immunity of the different CS-MRI methods, white gaussian noise was added to the k-space data before undersampling. By applying the method for estimating image noise proposed by Liu et al in the 'IEEE image processing Association' of 2013, as shown in FIG. 4, the DAWGAN performs best at PSNR under different noise levels and under-sampling modes. Fig. 5 shows that the DAWGAN is effective at suppressing noise at different noise levels. As can be seen from the above results, the DAWGAN also exhibits good noise immunity at different noise levels, with a higher average signal-to-noise ratio than other approaches.
Alternatively, to prove that the various configurations of the DAWGAN in this embodiment are effective, ablation experiments were performed to prove the rationality of the network. In this part of the experiment, the submodels compared were: WGAN-GP + RNN, WGAN-GP + Attention and Attention + RNN, with results as in Table 2, the DAWGAN model outperformed other sub-model variants in all three indices.
Figure BDA0002496665810000171
TABLE 2
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
Referring to fig. 6, a magnetic resonance imaging apparatus based on a generation countermeasure network according to an embodiment of the present application is shown, and is applied to image under-sampled MRI data acquired by a compressive sensing magnetic resonance imaging apparatus, where the under-sampled MRI data includes a plurality of data segments acquired and arranged according to a time sequence, and includes:
the establishing module 610 is used for establishing a corresponding relation between the undersampled MRI data and the image characteristics of the MRI image by utilizing the self-learning capability of the artificial neural network; specifically, determining the correlation between the data segments corresponding to adjacent time sequences; determining a target spatial feature in the undersampled MRI data; determining a corresponding relation between the undersampled MRI data and image features of an MRI image according to the correlation and the target space features;
an obtaining module 620, configured to obtain current under-sampled MRI data of a current subject;
a determining module 630, configured to determine, according to the correspondence, an image feature of the current MRI image corresponding to the current undersampled MRI data; specifically, determining image features of a current MRI image corresponding to the current undersampled MRI data includes: and determining the image characteristics of the MRI image corresponding to the undersampled MRI data which is the same as the current undersampled MRI data in the corresponding relation as the image characteristics of the current MRI image.
In an embodiment, the correspondence includes: a functional relationship; the undersampled MRI data is an input parameter of the functional relationship, and the image characteristics of the MRI image are output parameters of the functional relationship;
determining image features of a current MRI image corresponding to the current undersampled MRI data, further comprising:
and when the corresponding relation comprises a functional relation, inputting the current undersampled MRI data into the functional relation, and determining the output parameter of the functional relation as the image characteristic of the current MRI image.
In one embodiment, the establishing module 610 includes:
the acquisition submodule is used for acquiring sample data for establishing a corresponding relation between the undersampled MRI data and image characteristics of the MRI image;
the analysis submodule is used for analyzing the characteristics and the rules of the under-sampled MRI data and determining the network structure and the network parameters of the artificial neural network according to the characteristics and the rules;
and the training submodule is used for training and testing the network structure and the network parameters by using the sample data and determining the corresponding relation between the undersampled MRI data and the image characteristics of the MRI image.
In one embodiment, the obtaining sub-module includes:
a collection sub-module for collecting the undersampled MRI data and image features of the MRI images of different subjects;
the analysis submodule is used for analyzing the under-sampled MRI data, and selecting data related to the image characteristics of the MRI image as the under-sampled MRI data by combining with prestored expert experience information;
and the sample data generation submodule is used for taking the image characteristics of the MRI image and the selected data pair formed by the undersampled MRI data as sample data.
In one embodiment of the present invention, the substrate is,
the training submodule includes:
a training result generation submodule, configured to select a part of data in the sample data as a training sample, input the under-sampled MRI data in the training sample to the network structure, and train through a loss function of the network structure, an activation function, and the network parameters to obtain an actual training result;
a training result error judgment submodule for determining whether an actual training error between the actual training result and the image feature of the corresponding MRI image in the training sample satisfies a preset training error;
a training completion determination submodule configured to determine that the training of the network structure and the network parameters is completed when the actual training error satisfies the preset training error;
and/or the presence of a gas in the gas,
a test sub-module for testing the network structure and the network parameters, the test sub-module comprising:
a test result generation submodule, configured to select another part of the sample data as a test sample, input the under-sampled MRI data in the test sample into the trained network structure, and perform a test with the loss function, an activation function, and the trained network parameters to obtain an actual test result;
the test result error judgment submodule is used for determining whether the actual test error between the actual test result and the image characteristics of the corresponding MRI image in the test sample meets the set test error;
and the test completion judging submodule is used for determining that the test on the network structure and the network parameters is completed when the actual test error meets the set test error.
In one embodiment, the loss function includes a combined image domain mean square error loss function, a frequency domain mean square error loss function, an image processing content aware loss function, and a countering loss function.
In one embodiment of the present invention, the substrate is,
the training submodule further comprises:
a network parameter updating submodule, configured to update the network parameter through an error loss function of the network structure when the actual training error does not meet the set training error;
the first retraining submodule is used for retraining the activation function and the updated network parameters through the loss function of the network structure until the actual training error after retraining meets the set training error;
and/or the presence of a gas in the gas,
the test submodule further comprises:
and the second retraining submodule is used for retraining the network structure and the network parameters when the actual test error does not meet the set test error until the retrained actual test error meets the set test error.
Referring to fig. 7, a computer device of the magnetic resonance imaging method based on generation of a countermeasure network is shown, which may specifically include the following:
the computer device 12 described above is embodied in the form of a general purpose computing device, and the components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.
Bus 18 represents one or more of any of several types of bus 18 structures, including a memory bus 18 or memory controller, a peripheral bus 18, an accelerated graphics port, and a processor or local bus 18 using any of a variety of bus 18 architectures. By way of example, such architectures include, but are not limited to, Industry Standard Architecture (ISA) bus 18, micro-channel architecture (MAC) bus 18, enhanced ISA bus 18, audio Video Electronics Standards Association (VESA) local bus 18, and Peripheral Component Interconnect (PCI) bus 18.
Computer device 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer device 12 and includes both volatile and nonvolatile media, removable and non-removable media.
The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)30 and/or cache memory 32. Computer device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (commonly referred to as "hard drives"). Although not shown in FIG. 7, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. The memory may include at least one program product having a set (e.g., at least one) of program modules 42, with the program modules 42 configured to carry out the functions of embodiments of the invention.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules 42, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.
Computer device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, camera, etc.), with one or more devices that enable a user to interact with computer device 12, and/or with any devices (e.g., network card, modem, etc.) that enable computer device 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Also, computer device 12 may communicate with one or more networks (e.g., a Local Area Network (LAN)), a Wide Area Network (WAN), and/or a public network (e.g., the Internet) via network adapter 20. As shown, the network adapter 20 communicates with the other modules of the computer device 12 via the bus 18. It should be appreciated that although not shown in FIG. 7, other hardware and/or software modules may be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units 16, external disk drive arrays, RAID systems, tape drives, and data backup storage systems 34, etc.
The processing unit 16 executes programs stored in the system memory 28 to execute various functional applications and data processing, such as implementing the magnetic resonance imaging method based on the generation countermeasure network provided by the embodiment of the present invention.
That is, the processing unit 16 implements, when executing the program,: establishing a corresponding relation between undersampled MRI data and image characteristics of an MRI image by utilizing the self-learning capability of an artificial neural network; specifically, determining the correlation between the data segments corresponding to adjacent time sequences; determining a target spatial feature in the undersampled MRI data; determining a corresponding relation between the undersampled MRI data and image features of an MRI image according to the correlation and the target space features; acquiring current undersampled MRI data of a current detected person; determining the image characteristics of the current MRI image corresponding to the current undersampled MRI data according to the corresponding relation; specifically, determining image features of a current MRI image corresponding to the current undersampled MRI data includes: and determining the image characteristics of the MRI image corresponding to the undersampled MRI data which is the same as the current undersampled MRI data in the corresponding relation as the image characteristics of the current MRI image.
In an embodiment of the present invention, the present invention further provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor, implements a magnetic resonance imaging method based on generation of a countermeasure network as provided in all embodiments of the present application:
that is, the program when executed by the processor implements: establishing a corresponding relation between undersampled MRI data and image characteristics of an MRI image by utilizing the self-learning capability of an artificial neural network; specifically, determining the correlation between the data segments corresponding to adjacent time sequences; determining a target spatial feature in the undersampled MRI data; determining a corresponding relation between the undersampled MRI data and image features of an MRI image according to the correlation and the target space features; acquiring current undersampled MRI data of a current detected person; determining the image characteristics of the current MRI image corresponding to the current undersampled MRI data according to the corresponding relation; specifically, determining image features of a current MRI image corresponding to the current undersampled MRI data includes: and determining the image characteristics of the MRI image corresponding to the undersampled MRI data which is the same as the current undersampled MRI data in the corresponding relation as the image characteristics of the current MRI image.
Any combination of one or more computer-readable media may be employed. The computer readable medium may be a computer-readable storage medium or a computer-readable signal medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPOM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
While preferred embodiments of the present application have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the true scope of the embodiments of the application.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.
The magnetic resonance imaging method and apparatus based on generation of countermeasure network provided by the present application are introduced in detail above, and specific examples are applied herein to illustrate the principles and embodiments of the present application, and the above descriptions of the embodiments are only used to help understanding the method and core ideas of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims (10)

1. A magnetic resonance imaging method based on generation countermeasure network is applied to imaging under-sampled MRI data acquired by a compressed sensing magnetic resonance imaging device, and the under-sampled MRI data comprises a plurality of data sections which are acquired and arranged according to time sequence, and is characterized by comprising the following steps:
establishing a corresponding relation between undersampled MRI data and image characteristics of an MRI image by utilizing the self-learning capability of an artificial neural network; specifically, determining the correlation between the data segments corresponding to adjacent time sequences; determining a target spatial feature in the undersampled MRI data; determining a corresponding relation between the undersampled MRI data and image features of an MRI image according to the correlation and the target space features;
acquiring current undersampled MRI data of a current detected person;
determining the image characteristics of the current MRI image corresponding to the current undersampled MRI data according to the corresponding relation; specifically, determining image features of a current MRI image corresponding to the current undersampled MRI data includes: and determining the image characteristics of the MRI image corresponding to the undersampled MRI data which is the same as the current undersampled MRI data in the corresponding relation as the image characteristics of the current MRI image.
2. The method of claim 1,
the corresponding relation comprises: a functional relationship; the undersampled MRI data is an input parameter of the functional relationship, and the image characteristics of the MRI image are output parameters of the functional relationship;
determining image features of a current MRI image corresponding to the current undersampled MRI data, further comprising:
and when the corresponding relation comprises a functional relation, inputting the current undersampled MRI data into the functional relation, and determining the output parameter of the functional relation as the image characteristic of the current MRI image.
3. The method of claim 1, wherein the step of establishing a correspondence between undersampled MRI data and image features of an MRI image comprises:
acquiring sample data for establishing a corresponding relation between the undersampled MRI data and image features of the MRI image;
analyzing the characteristics and the rules of the undersampled MRI data, and determining the network structure and the network parameters of the artificial neural network according to the characteristics and the rules;
training and testing the network structure and the network parameters using the sample data, and determining the correspondence of the undersampled MRI data to image features of the MRI image.
4. The method of claim 3, wherein the step of acquiring sample data for establishing a correspondence between the undersampled MRI data and image features of the MRI image comprises:
collecting image features of the undersampled MRI data and the MRI images of different subjects;
analyzing the under-sampled MRI data, and selecting data related to the image characteristics of the MRI image as the under-sampled MRI data by combining with prestored expert experience information;
and taking the image characteristics of the MRI image and the data pair formed by the selected undersampled MRI data as sample data.
5. The method according to any one of claims 4 to 5,
training the network structure and the network parameters, including:
selecting a part of data in the sample data as a training sample, inputting the under-sampled MRI data in the training sample into the network structure, and training by a loss function of the network structure, an activation function and the network parameters to obtain an actual training result;
determining whether an actual training error between the actual training result and an image feature of a corresponding MRI image in the training sample satisfies a preset training error;
determining that the training of the network structure and the network parameters is completed when the actual training error meets the preset training error;
and/or the presence of a gas in the gas,
testing the network structure and the network parameters, comprising:
selecting another part of data in the sample data as a test sample, inputting the under-sampled MRI data in the test sample into the trained network structure, and testing by using the loss function, the activation function and the trained network parameters to obtain an actual test result;
determining whether an actual test error between the actual test result and an image feature of a corresponding MRI image in the test sample satisfies a set test error;
and when the actual test error meets the set test error, determining that the test on the network structure and the network parameters is finished.
6. The method of claim 5,
the loss functions include a combined image domain mean square error loss function, a frequency domain mean square error loss function, an image processing content perception loss function, and an antagonistic loss function.
7. The method of claim 5,
training the network structure and the network parameters, further comprising:
when the actual training error does not meet the set training error, updating the network parameters through an error loss function of the network structure;
activating a function and the updated network parameters to retrain through the loss function of the network structure until the retrained actual training error meets the set training error;
and/or the presence of a gas in the gas,
testing the network structure and the network parameters, further comprising:
and when the actual test error does not meet the set test error, retraining the network structure and the network parameters until the retrained actual test error meets the set test error.
8. A magnetic resonance imaging device based on a generation countermeasure network is applied to imaging under-sampled MRI data acquired by a compressed sensing magnetic resonance imaging device, and the under-sampled MRI data comprises a plurality of data sections acquired and arranged according to a time sequence, and is characterized by comprising:
the establishing module is used for establishing a corresponding relation between the undersampled MRI data and the image characteristics of the MRI image by utilizing the self-learning capability of the artificial neural network; specifically, determining the correlation between the data segments corresponding to adjacent time sequences; determining a target spatial feature in the undersampled MRI data; determining a corresponding relation between the undersampled MRI data and image features of an MRI image according to the correlation and the target space features;
the acquisition module is used for acquiring current undersampled MRI data of a current detected person;
a determining module, configured to determine, according to the correspondence, an image feature of a current MRI image corresponding to the current undersampled MRI data; specifically, determining image features of a current MRI image corresponding to the current undersampled MRI data includes: and determining the image characteristics of the MRI image corresponding to the undersampled MRI data which is the same as the current undersampled MRI data in the corresponding relation as the image characteristics of the current MRI image.
9. An apparatus comprising a processor, a memory, and a computer program stored on the memory and capable of running on the processor, the computer program when executed by the processor implementing the method of any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.
CN202010420192.7A 2020-05-18 2020-05-18 Magnetic resonance imaging method and device based on generation countermeasure network Active CN111598966B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010420192.7A CN111598966B (en) 2020-05-18 2020-05-18 Magnetic resonance imaging method and device based on generation countermeasure network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010420192.7A CN111598966B (en) 2020-05-18 2020-05-18 Magnetic resonance imaging method and device based on generation countermeasure network

Publications (2)

Publication Number Publication Date
CN111598966A true CN111598966A (en) 2020-08-28
CN111598966B CN111598966B (en) 2023-04-18

Family

ID=72191544

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010420192.7A Active CN111598966B (en) 2020-05-18 2020-05-18 Magnetic resonance imaging method and device based on generation countermeasure network

Country Status (1)

Country Link
CN (1) CN111598966B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113592972A (en) * 2021-07-30 2021-11-02 哈尔滨工业大学（深圳） Magnetic resonance image reconstruction method and device based on multi-modal aggregation
JP7464138B2 (en) 2020-09-30 2024-04-09 日本電信電話株式会社 Learning device, learning method, and learning program

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017113205A1 (en) * 2015-12-30 2017-07-06 中国科学院深圳先进技术研究院 Rapid magnetic resonance imaging method and apparatus based on deep convolutional neural network
CN110517249A (en) * 2019-08-27 2019-11-29 中山大学 Imaging method, device, equipment and the medium of ultrasonic elastic image
CN110766769A (en) * 2019-10-23 2020-02-07 深圳先进技术研究院 Magnetic resonance image reconstruction method, device, equipment and medium
US20200105031A1 (en) * 2018-09-30 2020-04-02 The Board Of Trustees Of The Leland Stanford Junior University Method for Performing Magnetic Resonance Imaging Reconstruction with Unsupervised Deep Learning

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017113205A1 (en) * 2015-12-30 2017-07-06 中国科学院深圳先进技术研究院 Rapid magnetic resonance imaging method and apparatus based on deep convolutional neural network
US20200105031A1 (en) * 2018-09-30 2020-04-02 The Board Of Trustees Of The Leland Stanford Junior University Method for Performing Magnetic Resonance Imaging Reconstruction with Unsupervised Deep Learning
CN110517249A (en) * 2019-08-27 2019-11-29 中山大学 Imaging method, device, equipment and the medium of ultrasonic elastic image
CN110766769A (en) * 2019-10-23 2020-02-07 深圳先进技术研究院 Magnetic resonance image reconstruction method, device, equipment and medium

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7464138B2 (en) 2020-09-30 2024-04-09 日本電信電話株式会社 Learning device, learning method, and learning program
CN113592972A (en) * 2021-07-30 2021-11-02 哈尔滨工业大学（深圳） Magnetic resonance image reconstruction method and device based on multi-modal aggregation
CN113592972B (en) * 2021-07-30 2023-11-14 哈尔滨工业大学（深圳） Magnetic resonance image reconstruction method and device based on multi-mode aggregation

Also Published As

Publication number Publication date
CN111598966B (en) 2023-04-18

Similar Documents

Publication Publication Date Title
CN110400298B (en) Method, device, equipment and medium for detecting heart clinical index
Freiman et al. Improving CCTA‐based lesions' hemodynamic significance assessment by accounting for partial volume modeling in automatic coronary lumen segmentation
CN110858315A (en) Deep machine learning based magnetic resonance imaging quality classification considering less training data
CN112435341B (en) Training method and device for three-dimensional reconstruction network, and three-dimensional reconstruction method and device
CN111612756B (en) Coronary artery specificity calcification detection method and device
CN111598966B (en) Magnetic resonance imaging method and device based on generation countermeasure network
CN103810712B (en) Energy spectrum CT (computerized tomography) image quality evaluation method
CN111340794B (en) Quantification method and device for coronary artery stenosis
CN111179235A (en) Image detection model generation method and device, and application method and device
CN111445546A (en) Image reconstruction method and device, electronic equipment and storage medium
Marin et al. Numerical surrogates for human observers in myocardial motion evaluation from SPECT images
CN111612865B (en) MRI (magnetic resonance imaging) method and device for generating countermeasure network based on conditions
Mouches et al. Unifying brain age prediction and age-conditioned template generation with a deterministic autoencoder
CN111681297B (en) Image reconstruction method, computer device, and storage medium
Nabavi et al. Automatic multi-class cardiovascular magnetic resonance image quality assessment using unsupervised domain adaptation in spatial and frequency domains
CN112581385A (en) Diffusion kurtosis imaging tensor estimation method, medium and equipment based on multiple prior constraints
CN111968108A (en) CT intelligent imaging method, device and system based on intelligent scanning protocol
CN114557691A (en) Non-invasive blood lipid detection method and system based on multi-wavelength PPG (photoplethysmography) signals
CN114529519A (en) Image compressed sensing reconstruction method and system based on multi-scale depth cavity residual error network
CN114764766A (en) B-mode ultrasonic image denoising method based on FC-VoVNet and WGAN
CN110786839A (en) Method, device, equipment and medium for generating instantaneous waveform-free ratio
CN115251889B (en) Method for describing characteristics of dynamic connection network of functional magnetic resonance image
WO2024108438A1 (en) Motion artifact correction method for velocity encoding magnetic resonance imaging
EP4270309A1 (en) Image processing device and method
Kornaropoulos et al. Joint deformable image registration and ADC map regularization: application to DWI-based lymphoma classification

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant