WO2023045482A1 - 基于长距离注意力模型重建的多层磁共振成像方法及装置 - Google Patents

基于长距离注意力模型重建的多层磁共振成像方法及装置 Download PDF

Info

Publication number
WO2023045482A1
WO2023045482A1 PCT/CN2022/103707 CN2022103707W WO2023045482A1 WO 2023045482 A1 WO2023045482 A1 WO 2023045482A1 CN 2022103707 W CN2022103707 W CN 2022103707W WO 2023045482 A1 WO2023045482 A1 WO 2023045482A1
Authority
WO
WIPO (PCT)
Prior art keywords
embedding
magnetic resonance
layer
imaging
deep learning
Prior art date
Application number
PCT/CN2022/103707
Other languages
English (en)
French (fr)
Inventor
吕孟叶
Original Assignee
深圳技术大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳技术大学 filed Critical 深圳技术大学
Publication of WO2023045482A1 publication Critical patent/WO2023045482A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
    • G01R33/00—Arrangements or instruments for measuring magnetic variables
    • G01R33/20—Arrangements or instruments for measuring magnetic variables involving magnetic resonance
    • G01R33/44—Arrangements or instruments for measuring magnetic variables involving magnetic resonance using nuclear magnetic resonance [NMR]
    • G01R33/48—NMR imaging systems
    • G01R33/483—NMR imaging systems with selection of signals or spectra from particular regions of the volume, e.g. in vivo spectroscopy
    • G01R33/4833—NMR imaging systems with selection of signals or spectra from particular regions of the volume, e.g. in vivo spectroscopy using spatially selective excitation of the volume of interest, e.g. selecting non-orthogonal or inclined slices
    • G01R33/4835—NMR imaging systems with selection of signals or spectra from particular regions of the volume, e.g. in vivo spectroscopy using spatially selective excitation of the volume of interest, e.g. selecting non-orthogonal or inclined slices of multiple slices
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
    • G01R33/00—Arrangements or instruments for measuring magnetic variables
    • G01R33/20—Arrangements or instruments for measuring magnetic variables involving magnetic resonance
    • G01R33/44—Arrangements or instruments for measuring magnetic variables involving magnetic resonance using nuclear magnetic resonance [NMR]
    • G01R33/48—NMR imaging systems
    • G01R33/54—Signal processing systems, e.g. using pulse sequences ; Generation or control of pulse sequences; Operator console
    • G01R33/56—Image enhancement or correction, e.g. subtraction or averaging techniques, e.g. improvement of signal-to-noise ratio and resolution
    • G01R33/5608—Data processing and visualization specially adapted for MR, e.g. for feature analysis and pattern recognition on the basis of measured MR data, segmentation of measured MR data, edge contour detection on the basis of measured MR data, for enhancing measured MR data in terms of signal-to-noise ratio by means of noise filtering or apodization, for enhancing measured MR data in terms of resolution by means for deblurring, windowing, zero filling, or generation of gray-scaled images, colour-coded images or images displaying vectors instead of pixels
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/003—Reconstruction from projections, e.g. tomography
    • G06T11/005—Specific pre-processing for tomographic reconstruction, e.g. calibration, source positioning, rebinning, scatter correction, retrospective gating
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/003—Reconstruction from projections, e.g. tomography
    • G06T11/006—Inverse problem, transformation from projection-space into object-space, e.g. transform methods, back-projection, algebraic methods
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves 
    • A61B5/055—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves  involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/41—Medical
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2211/00—Image generation
    • G06T2211/40—Computed tomography
    • G06T2211/424—Iterative

Definitions

  • the invention relates to the technical fields of medical imaging and intelligent image processing, in particular to a multilayer magnetic resonance imaging method and a device for realizing the method.
  • Magnetic resonance imaging is of great value in clinical disease diagnosis and biomedical research due to its advantages of non-invasiveness and no ionizing radiation. However, its slow imaging speed and long scanning time hinder the development of magnetic resonance imaging. .
  • Parallel imaging technology including the recently developed simultaneous multi-slice magnetic resonance technology (which can be considered as 3D-based parallel imaging), reduces the number of spatial encodings in the imaging process and is expected to increase the imaging speed of magnetic resonance.
  • Another object of the present invention is to provide a multi-layer magnetic resonance imaging device based on long-distance attention model reconstruction, which solves the problems of poor magnetic resonance image quality, low diagnostic accuracy, slow imaging speed and magnetic The problem of low utilization of resonance machines.
  • the present invention provides a multi-layer magnetic resonance imaging method based on long-distance attention model reconstruction, including constructing a deep learning reconstruction model; performing data preprocessing on signals collected at the same time in multiple layers, using multi-layer magnetic Resonance images or K-space data are used as data input; learnable position embedding and imaging parameter embedding are obtained; preprocessed input data, position embedding and imaging parameter embedding are input to the deep learning reconstruction model; the deep learning reconstruction model outputs magnetic Results of resonance reconstructed images.
  • the input of the preprocessed input data, position embedding and imaging parameter embedding into the deep learning reconstruction model includes: the deep learning reconstruction model is a Transformer structure, and the Transformer structure uses a Transformer codec as The core, which includes the pre-convolution layer, the post-convolution layer and the learnable position embedding and imaging parameter embedding; after obtaining the learnable position embedding and imaging parameter embedding, the preprocessed data and the learnable The position embedding is added and input to the Transformer encoder of the deep learning reconstruction model; the output of the Transformer encoder and the learnable imaging parameters are embedded into the Transformer decoder of the deep learning reconstruction model; the result of the magnetic resonance reconstruction image is output.
  • a two-dimensional matrix is used to represent the multi-layer image, which includes: inserting zero values in the K space along the magnetic resonance readout direction, and expanding the field of view in the magnetic resonance readout direction , and then perform a fast Fourier transform to obtain an image of layer aliasing connected along the magnetic resonance readout direction; use multiple two-dimensional convolutional layers to extract the features of the image, form a feature tensor, and combine the feature tensor Divided into blocks.
  • the preprocessed data and the learnable position embedding are added and then input to the Transformer encoder of the deep learning reconstruction model, including: the convolution layer converts each Block unrolling, this data is fed into the Transformer encoder after summing with learnable positional embeddings.
  • the position embedding is obtained by transforming the coordinates of the block through a learnable embedding layer; or transforming the coordinates of the block through a fully connected layer.
  • the output of the Transformer encoder and the learnable imaging parameter embedding input into the Transformer decoder of the deep learning reconstruction model include inputting the output of the Transformer encoder and the learnable imaging parameter embedding into Transformer decoder, the output of the Transformer decoder is rearranged to form another feature tensor, and then the multi-layer image is generated through the convolutional layer.
  • the Transformer encoder includes the self-attention layer and the full connection layer.
  • the acquisition of imaging parameter embedding includes: scanning imaging information, encoding the scanned imaging information into a vector, inputting the vector into an embedding layer or a fully connected layer or constructing it in a fixed manner; wherein, the Imaging information can include imaging site, angle of slice, acquisition acceleration multiple, direction and distance of controllable aliasing, sequence type used, direction of phase encoding, FOV, TR time, TE time, pulse flip angle, scanned object, Scanning machines etc.
  • the input of the preprocessed input data, position embedding and imaging parameter embedding into the deep learning reconstruction model includes: using the original image or K-space data, imaging parameter embedding, and position embedding as three inputs , wherein the deep learning reconstruction model includes a convolutional layer and a fully connected layer; the original image or K-space data is processed by several layers of convolutional layers to form N1 feature channels; imaging parameters are embedded in several layers of convolutional layers After processing, N2 feature channels are formed; after splicing N1 feature channels and N2 feature channels, they are sent to several convolutional layers to form N3 feature channels; the position is embedded through the fully connected layer to form N3 output values, The N3 output values are added to the N3 feature channels, and processed by a convolutional neural network to obtain an output result of the magnetic resonance reconstruction image.
  • the deep learning reconstruction model includes a convolutional layer and a fully connected layer
  • the original image or K-space data is processed by several layers of convolutional layers to form N1 feature channels
  • the constructed deep learning reconstruction model is combined with the gradient-based data consistency update, and end-to-end iterative training is performed to obtain the optimal output result of the magnetic resonance reconstruction image.
  • the present invention provides a multi-layer magnetic resonance imaging device based on long-distance attention model reconstruction, including a model construction unit for building a deep learning reconstruction model; a data preprocessing unit for Data preprocessing is performed on multi-layer simultaneous acquisition signals, using multi-layer magnetic resonance images or K-space data as data input; embedding unit is used to obtain learnable position embedding and imaging parameter embedding; image reconstruction unit is used to The processed input data, position embedding and imaging parameter embedding are input to the deep learning reconstruction model; the output unit is used to output the result of the magnetic resonance reconstruction image.
  • the present invention uses a deep learning image reconstruction model, and adds imaging parameter embedding and spatial position embedding.
  • imaging prior information such as imaging sites, used sequences, etc.
  • long-distance related information in magnetic resonance data the present invention
  • the provided method can model efficiently and learn magnetic resonance domain knowledge better, so as to improve the noise and artifact problems of magnetic resonance reconstruction images.
  • the present invention can improve the quality of magnetic resonance images, improve the accuracy of doctor's diagnosis; accelerate the imaging speed, improve the utilization rate of magnetic resonance machines, have a wide range of applications, and do not need to fully collect the center of K space, and are widely applicable to gradient echo, spin echo, etc. Wave, echo planar and other imaging methods.
  • FIG. 1 is a flow chart of Embodiment 1 of the multi-slice magnetic resonance imaging method based on long-distance attention model reconstruction in the present invention.
  • FIG. 2 is a schematic diagram of using Transformer structure as a deep learning reconstruction model in Embodiment 1 of the multi-layer magnetic resonance imaging method based on long-distance attention model reconstruction in the present invention.
  • FIG. 3 is a schematic diagram of a reconstruction model using deep learning under an iterative expansion framework in an embodiment of the multi-layer magnetic resonance imaging method based on long-distance attention model reconstruction in the present invention.
  • Fig. 4 is a schematic diagram of an embodiment of a multi-layer magnetic resonance imaging device based on long-distance attention model reconstruction according to the present invention.
  • the multilayer magnetic resonance imaging method based on long-distance attention model reconstruction of the present invention comprises the following steps:
  • Step S1 constructing a deep learning reconstruction model.
  • Step S2 performing data preprocessing on the multi-layer simultaneously acquired signals, using multi-layer magnetic resonance images or K-space data as data input.
  • Step S3 acquiring learnable position embeddings and imaging parameter embeddings.
  • Step S4 input the preprocessed input data, position embedding and imaging parameter embedding into the deep learning reconstruction model.
  • step S5 the result of the MRI reconstruction image is output by the deep learning reconstruction model.
  • the preprocessed input data, position embedding, and imaging parameter embedding are input to the deep learning reconstruction model, specifically including: the deep learning reconstruction model is a Transformer structure, and the Transformer structure takes the Transformer codec as the core , which includes a pre-convolution layer, a post-convolution layer and a learnable position embedding and imaging parameter embedding; after obtaining the learnable position embedding and imaging parameter embedding, the preprocessed data and the learnable position After the embedding is added, it is input into the Transformer encoder of the deep learning reconstruction model; the output of the Transformer encoder and the learnable imaging parameters are embedded into the Transformer decoder of the deep learning reconstruction model; the result of the magnetic resonance reconstruction image is output.
  • the deep learning reconstruction model is a Transformer structure
  • the Transformer structure takes the Transformer codec as the core , which includes a pre-convolution layer, a post-convolution layer and a learnable position embedding and imaging parameter embedding
  • the preprocessed data and the learnable position After the embed
  • a two-dimensional matrix is used to represent the multi-layer image, which includes: inserting zero values in the K space along the magnetic resonance readout direction, expanding the field of view in the magnetic resonance readout direction (FOV), and then perform fast Fourier transform to obtain the image of layer aliasing connected along the magnetic resonance readout direction; use multiple two-dimensional convolutional layers to extract the features of the image, form a feature tensor, and combine the The feature tensor is divided into blocks.
  • the preprocessed data and the learnable position embedding are added and then input to the Transformer encoder of the deep learning reconstruction model, including: the convolution layer transforms each block in the form of 1-dimensional data Unfolded, this data is fed into the Transformer encoder after summing with learnable positional embeddings.
  • the above position embedding is obtained by transforming the coordinates of the block through a learnable embedding layer; or transforming the coordinates of the block through a fully connected layer.
  • the output of the Transformer encoder and the learnable imaging parameter embedding are input into the Transformer decoder of the deep learning reconstruction model, specifically including: inputting the output of the Transformer encoder and the learnable imaging parameter embedding into Transformer decoder, the output of the Transformer decoder is rearranged to form another feature tensor, and then the multi-layer image is generated through the convolutional layer.
  • the Transformer encoder includes the self-attention layer and the full connection layer.
  • the acquisition of imaging parameter embedding includes: scanning imaging information, encoding the scanned imaging information into a vector, inputting the vector into the embedding layer or fully connected layer or constructing it in a fixed way; wherein, the above imaging information can be Including imaging site, slice angle, acquisition acceleration multiple, controllable aliasing direction and distance, sequence type used, phase encoding direction, FOV, TR time, TE time, pulse flip angle, scanned object, scanning machine, etc. .
  • end-to-end iterative training is performed by combining the constructed deep learning reconstruction model with the gradient-based data consistency update to obtain the optimal output result of the magnetic resonance reconstruction image.
  • the gradient-based data can be conjugate gradient method, gradient descent method or other improved methods for iterative solution.
  • simultaneous multi-layer imaging can be regarded as parallel imaging in 3D
  • the reconstruction method suitable for simultaneous multi-layer imaging can be directly extended to 2D parallel imaging, so the following is mainly based on the reconstruction of simultaneous multi-layer imaging Give an example to illustrate the method.
  • the reconstruction of simultaneous multi-layer imaging data can correspond to the following optimization problem, such as formula (1):
  • A is the encoding operator, corresponding to simultaneous multi-layer sampling and coil sensitivity modulation in K-space
  • x is the multi-level MRI image to be reconstructed
  • b is the multi-channel K-space data collected
  • T(x) is the regularization constraint.
  • the Transformer codec as the core, including a pre-convolutional layer (convolutional neural network, CNN), a post-convolutional layer and a learnable position Embedding (positional embedding) and imaging parameter embedding (imaging parameter embedding).
  • the readout-concatenate preprocessing is performed on the multi-layer simultaneous acquisition signals, so that the multi-layer magnetic resonance image can be represented by a two-dimensional matrix, and the specific operation is to insert zero values in the K space along the magnetic resonance readout direction , expand the FOV in the magnetic resonance readout direction, and then perform fast Fourier transform to obtain an aliased image of slices connected along the magnetic resonance readout direction. Then, use multiple two-dimensional convolutional layers to extract features, form feature tensors, divide feature constants into small blocks (Patch), expand each block into one-dimensional, and add learnable position embeddings Input into the Transformer encoder.
  • K-spaces (instead of images) acquired at the same time by multiple layers may also be used as input.
  • the above position embedding can pass through a learnable embedding layer (such as tf.nn in the tensorflow framework) according to the coordinates (x, y, z) or (kx, ky, kz) of the block. .embedding_lookup) transformed. Alternatively, it can also be obtained by transforming the coordinates of the location through fully connected layers.
  • this positional embedding can also be directly constructed in a non-learned fixed way, for example using sin-cosine encoding.
  • the Transformer encoder can refer to the implementation in BERT (https://github.com/google-research/bert), which includes a self-attention layer and a fully connected layer that calculates all embedded correlations.
  • BERT https://github.com/google-research/bert
  • the output of the encoder is fed into the Transformer decoder together with a learnable imaging parameter embedding.
  • the output of the Transformer decoder is rearranged to form another feature tensor, which is then passed through a convolutional layer to generate a reconstructed multi-level image (the offset caused by controllable aliasing can be subsequently removed).
  • the above imaging parameter embedding is obtained in the following manner: firstly, the imaging information of the scan is encoded into a vector, and then the vector is input into the embedding layer or the fully connected layer or constructed in a fixed manner.
  • the above-mentioned imaging information may include the imaging site (head, neck, chest, upper abdomen, lower abdomen, elbow joint, knee joint, etc., each site is represented by an integer), the angle of the plane (using the three-plane included angle), acquisition acceleration (usually a decimal between 1 and 20), direction and distance of controllable aliasing, sequence type used (FSE, FLASH, EPI, FLAIR, etc., each sequence is represented by an integer ), the direction of phase encoding, FOV (field of view), TR time, TE time, pulse flip angle, the age, sex, height, weight of the scanned object, and the field strength, brand, model, etc. of the scanning machine.
  • these information can also be obtained through DICOM file information.
  • the above-mentioned deep learning reconstruction model can be used to process the input data, and the obtained output is the final reconstruction result.
  • the MoDL reconstruction framework https://github.com/hkaggarwal/modl
  • the reconstruction model can be trained on the training data set.
  • the training data set of this embodiment can be obtained in a variety of ways, it can be obtained from scanning on a real magnetic resonance machine, it can also be obtained from a large-scale public magnetic resonance data set (such as ADNI, HCP, etc.), or it can be obtained through algorithm simulation Obtained, for example, through bloch equation or directly using open source magnetic resonance simulation software such as mrilab (http://mrilab.sourceforge.net/). Of course, the above three methods can also be used in combination.
  • mrilab http://mrilab.sourceforge.net/
  • mrilab http://mrilab.sourceforge.net/
  • the above three methods can also be used in combination.
  • a comprehensive loss function of weighted average such as L1 loss, L2 loss, perceptual loss, and adversarial loss can be used, and the ADAM optimizer can be used for parameter update.
  • the present invention uses a deep learning image reconstruction model, and adds imaging parameter embedding and spatial position embedding.
  • imaging prior information such as imaging sites, used sequences, etc.
  • long-distance related information in magnetic resonance data the present invention
  • the provided method can model efficiently and learn magnetic resonance domain knowledge better, so as to improve the noise and artifact problems of magnetic resonance reconstruction images.
  • the present invention can improve the quality of magnetic resonance images, improve the accuracy of doctor's diagnosis; accelerate the imaging speed, improve the utilization rate of magnetic resonance machines, have a wide range of applications, and do not need to fully collect the center of K space, and are widely applicable to gradient echo, spin echo, etc. Wave, echo planar and other imaging methods.
  • inputting the preprocessed input data, position embedding, and imaging parameter embedding into the deep learning reconstruction model also includes: taking the original image or K-space data, imaging parameter embedding, and position embedding as three inputs , wherein the deep learning reconstruction model includes a convolutional layer and a fully connected layer; the original image or K-space data is processed by several layers of convolutional layers to form N1 feature channels; imaging parameters are embedded in several layers of convolutional layers After processing, N2 feature channels are formed; after splicing N1 feature channels and N2 feature channels, they are sent to several convolutional layers to form N3 feature channels; the position is embedded through the fully connected layer to form N3 output values, The N3 output values are added to the N3 feature channels, and processed by a convolutional neural network to obtain an output result of the magnetic resonance reconstruction image.
  • the deep learning reconstruction model includes a convolutional layer and a fully connected layer
  • the original image or K-space data is processed by several layers of convolutional layers to form N1 feature channels
  • the above-mentioned Transformer structure is not necessary.
  • the convolutional layer and the fully connected layer can be used, and the original image (or K-space) data, imaging parameter embedding, and position embedding can be used as three inputs, where , the original image (or K-space) data is processed by several layers of convolutional layers to form N1 feature channels, and the imaging parameter embedding is also processed by several layers of convolutional layers to form N2 feature channels.
  • the original image (or K-space ) data to form N1 feature channels and imaging parameters to form N2 feature channels are spliced, and then sent to several convolutional layers to form N3 feature channels.
  • N3 output values are also formed, and the N3 output values are added to the N3 feature channels, and then processed by a convolutional neural network (such as using resnet50, efficientnet, etc.) to obtain the reconstruction result .
  • a convolutional neural network such as using resnet50, efficientnet, etc.
  • a kind of multitask-based magnetic resonance reconstruction model training device provided by the present invention includes:
  • a model construction unit 10 configured to construct a deep learning reconstruction model.
  • the data preprocessing unit 20 is configured to perform data preprocessing on multi-layer simultaneously acquired signals, using multi-layer magnetic resonance images or K-space data as data input.
  • the embedding unit 30 is configured to acquire learnable position embeddings and imaging parameter embeddings.
  • the image reconstruction unit 40 is configured to input the preprocessed input data, position embedding and imaging parameter embedding into the deep learning reconstruction model.
  • the output unit 50 is configured to output the result of the magnetic resonance reconstruction image.
  • the preprocessed input data, position embedding, and imaging parameter embedding are input to the deep learning reconstruction model, specifically including: the deep learning reconstruction model is a Transformer structure, and the Transformer structure takes the Transformer codec as the core , which includes a pre-convolution layer, a post-convolution layer and a learnable position embedding and imaging parameter embedding; after obtaining the learnable position embedding and imaging parameter embedding, the preprocessed data and the learnable position After the embedding is added, it is input into the Transformer encoder of the deep learning reconstruction model; the output of the Transformer encoder and the learnable imaging parameters are embedded into the Transformer decoder of the deep learning reconstruction model; the result of the magnetic resonance reconstruction image is output.
  • the deep learning reconstruction model is a Transformer structure
  • the Transformer structure takes the Transformer codec as the core , which includes a pre-convolution layer, a post-convolution layer and a learnable position embedding and imaging parameter embedding
  • the preprocessed data and the learnable position After the embed
  • a two-dimensional matrix is used to represent the multi-layer image, which includes: inserting zero values in the K space along the magnetic resonance readout direction, expanding the field of view in the magnetic resonance readout direction, and then performing a fast Fourier transform to obtain the image of layer aliasing connected along the magnetic resonance readout direction; use multiple two-dimensional convolutional layers to extract the features of the image, form a feature tensor, and divide the feature tensor into blocks .
  • the preprocessed data and the learnable position embedding are added and then input to the Transformer encoder of the deep learning reconstruction model, including: the convolution layer transforms each block in the form of 1-dimensional data Unfolded, this data is fed into the Transformer encoder after summing with learnable positional embeddings.
  • the above position embedding is obtained by transforming the coordinates of the block through a learnable embedding layer; or transforming the coordinates of the block through a fully connected layer.
  • the output of the Transformer encoder and the learnable imaging parameter embedding are input into the Transformer decoder of the deep learning reconstruction model, specifically including: inputting the output of the Transformer encoder and the learnable imaging parameter embedding into Transformer decoder, the output of the Transformer decoder is rearranged to form another feature tensor, and then the multi-layer image is generated through the convolutional layer.
  • the Transformer encoder includes the self-attention layer and the full connection layer.
  • the acquisition of imaging parameter embedding includes: scanning imaging information, encoding the scanned imaging information into a vector, inputting the vector into the embedding layer or fully connected layer or constructing it in a fixed way; wherein, the above imaging information can include imaging parts, Slice angle, acquisition acceleration multiple, direction and distance of controllable aliasing, sequence type used, direction of phase encoding, FOV, TR time, TE time, pulse flip angle, scanned object, scanning machine, etc.
  • end-to-end iterative training is performed by combining the constructed deep learning reconstruction model with the gradient-based data consistency update to obtain the optimal output result of the magnetic resonance reconstruction image.
  • Example of a computer device Example of a computer device:
  • the computer device of this embodiment includes a processor, and when the processor executes the computer program, the steps in the above embodiment of the multi-layer magnetic resonance imaging method are realized.
  • a computer program can be divided into one or more modules, and one or more modules are stored in a memory and executed by a processor to implement the present invention.
  • One or more modules may be a series of computer program instruction segments capable of accomplishing specific functions, and the instruction segments are used to describe the execution process of the computer program in the computer device.
  • a computer device may include, but is not limited to, a processor, memory. Those skilled in the art can understand that the computer device may include more or less components, or combine certain components, or different components, for example, the computer device may also include input and output devices, network access devices, buses, and the like.
  • the processor can be a central processing unit (Central Processing Unit, CPU), or other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf Programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • the processor is the control center of the computer device, and uses various interfaces and lines to connect various parts of the entire computer device.
  • the memory can be used to store computer programs and/or modules, and the processor implements various functions of the computer device by running or executing the computer programs and/or modules stored in the memory and calling data stored in the memory.
  • the memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound receiving function, a sound conversion function, etc.) etc.; a data storage area Data created according to the use of the mobile phone (such as audio data, text data, etc.) and the like can be stored.
  • the memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disk, internal memory, plug-in hard disk, smart memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card , flash card (Flash Card), at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
  • non-volatile memory such as hard disk, internal memory, plug-in hard disk, smart memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card , flash card (Flash Card), at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
  • the integrated modules of the terminal equipment are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the present invention realizes all or part of the processes in the methods of the above embodiments, and can also be completed by instructing related hardware through a computer program.
  • the computer program can be stored in a computer-readable storage medium, and the computer program is in When executed by the processor, each step of the above-mentioned multi-layer magnetic resonance imaging method can be realized.
  • the computer program includes computer program code
  • the computer program code may be in the form of source code, object code, executable file or some intermediate form.
  • the computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access Memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.
  • ROM Read-Only Memory
  • RAM Random Access Memory
  • electrical carrier signal telecommunication signal and software distribution medium, etc.
  • the content contained on computer readable media may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer readable media does not include Electrical carrier signals and telecommunication signals.
  • the present invention provides a computer device and a storage medium, which include: one or more memories, and one or more processors.
  • the memory is used to store the program code and the intermediate data generated during the running of the program, the storage of the model output results and the storage of the model and model parameters;
  • the processor is used for the processor resources occupied by the code running and the multiple processors occupied by the training model resource.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • High Energy & Nuclear Physics (AREA)
  • Condensed Matter Physics & Semiconductors (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Mathematical Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Pure & Applied Mathematics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Mathematical Optimization (AREA)
  • Signal Processing (AREA)
  • Algebra (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Optics & Photonics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)

Abstract

本发明提供一种基于长距离注意力模型重建的多层磁共振成像方法及装置，其包括构建深度学习重建模型；对多层同时采集的信号进行数据预处理，使用多层磁共振图像或K空间数据作为数据输入，获取可学习的位置嵌入以及成像参数嵌入，将经过预处理后的输入数据、位置嵌入以及成像参数嵌入输入到深度学习重建模型，由深度学习重建模型输出磁共振重建图像的结果。本发明还提供实现上述方法的装置。应用本发明可以提高磁共振图像质量，提高医生诊断的准确率，加快成像速度以及提高磁共振机器利用率。

Description

基于长距离注意力模型重建的多层磁共振成像方法及装置 技术领域
本发明涉及医学成像、智能图像处理技术领域，尤其涉及一种多层磁共振成像方法以及实现该方法的装置。
背景技术
磁共振成像因其非创伤性、无电离辐射等优势，在临床疾病诊断和生物医学研究中有着巨大的价值，但是其成像速度较慢，过长的扫描时间等问题阻碍着磁共振成像的发展。
并行成像技术，包括近期发展出的同时多层磁共振技术(可以认为是基于3D的并行成像)，减少了成像过程中空间编码的次数，有望提高磁共振的成像速度。
然而，并行成像数据需要重建得到图像，此问题具有一定的病态性，随着加速倍数的增加，往往会伴随噪声的放大和伪影的残留，对临床诊断和数据分析可能产生潜在负面影响。
发明内容
本发明的主要目的是提供一种基于长距离注意力模型重建的多层磁共振成像方法，该方法解决了现有技术中磁共振图像质量不高，诊断准确率低，成像速度慢，磁共振机器利用率低的问题。
本发明的另一目的是提供一种基于长距离注意力模型重建的多层磁共振成像装置，该装置解决了现有技术中磁共振图像质量不高，诊断准确率低，成像速度慢，磁共振机器利用率低的问题。
为了实现上述主要目的，本发明提供的一种基于长距离注意力模型重建的多层磁共振成像方法，包括构建深度学习重建模型；对多层同时采集的信号进行数据预处理，使用多层磁共振图像或K空间数据作为数据输入；获取可学习的位置嵌入以及成像参数嵌入；将经过预处理后的输入数据、位置嵌入以及成像参数嵌入输入到深度学习重建模型；由深度学习重建模 型输出磁共振重建图像的结果。
进一步的方案中，所述将经过预处理后的输入数据、位置嵌入以及成像参数嵌入输入到深度学习重建模型，包括：所述深度学习重建模型为Transformer结构，该Transformer结构以Transformer编解码器为核心，其包含前置卷积层、后置卷积层和可学习的位置嵌入以及成像参数嵌入；在获取可学习的位置嵌入以及成像参数嵌入后，将经过预处理后的数据和可学习的位置嵌入相加后输入到深度学习重建模型的Transformer编码器中；将Transformer编码器的输出和可学习的成像参数嵌入输入到深度学习重建模型的Transformer解码器中；输出磁共振重建图像的结果。
更进一步的方案中，在使用多层磁共振图像作为数据输入时，使用二维矩阵表示多层图像，其包括：沿磁共振读出方向在K空间插入零值，扩大磁共振读出方向视野，然后进行快速傅里叶变换，得到沿磁共振读出方向连接的层面混叠的图像；使用多个二维卷积层，提取图像的特征，形成一个特征张量，并将该特征张量划分为区块。
更进一步的方案中，所述将经过预处理后的数据和可学习的位置嵌入相加后输入到深度学习重建模型的Transformer编码器中，包括：卷积层以1维数据的形式将每个区块展开，该数据与可学习的位置嵌入相加后输入到Transformer编码器中。
更进一步的方案中，所述位置嵌入由该区块所在坐标经过可学习的嵌入层变换得到；或者由该区块所在坐标经过全连接层变换得到。
更进一步的方案中，所述将Transformer编码器的输出和可学习的成像参数嵌入输入到深度学习重建模型的Transformer解码器中，包括将Transformer编码器的输出和可学习的成像参数嵌入一起输入到Transformer解码器，Transformer解码器的输出经过重排形成另一个特征张量，再经过卷积层生成重建的多个层面图像，其中，Transformer编码器包含计算所有嵌入相关性的自注意力层和全连接层。
更进一步的方案中，所述成像参数嵌入的获取包括：扫描成像信息， 将扫描的成像信息编码为一个向量，将向量输入到嵌入层或全连接层或通过固定方式构造得到；其中，所述成像信息可以包括成像部位、层面的角度、采集加速倍数、可控混叠的方向和距离、使用的序列类型、相位编码的方向、FOV、TR时间、TE时间、脉冲翻转角、被扫描对象、扫描机器等。
更进一步的方案中，所述将经过预处理后的输入数据、位置嵌入以及成像参数嵌入输入到深度学习重建模型，包括：将原始图像或K空间数据、成像参数嵌入、位置嵌入作为三个输入，其中，该深度学习重建模型包含卷积层和全连接层；将原始图像或K空间数据经过若干层的卷积层处理后形成N1个特征通道；将成像参数嵌入经过若干层的卷积层处理后形成N2个特征通道；将N1个特征通道和N2个特征通道相拼接后，再送入若干个卷积层，形成N3个特征通道；将位置嵌入经过全连接层后形成N3个输出值，将N3个输出值加到N3个特征通道上，经过卷积神经网络处理，从而得到磁共振重建图像的输出结果。
更进一步的方案中，对构建的深度学习重建模型和基于梯度的数据一致性更新结合，进行端对端迭代训练，获得磁共振重建图像的最优输出结果。
为了实现上述的另一目的，本发明提供的一种基于长距离注意力模型重建的多层磁共振成像装置，包括模型构建单元，用于构建深度学习重建模型；数据预处理单元，用于对多层同时采集的信号进行数据预处理，使用多层磁共振图像或K空间数据作为数据输入；嵌入单元，用于获取可学习的位置嵌入以及成像参数嵌入；图像重建单元，用于将经过预处理后的输入数据、位置嵌入以及成像参数嵌入输入到深度学习重建模型；输出单元，用于输出磁共振重建图像的结果。
由此可见，本发明使用深度学习图像重建模型，并加入成像参数嵌入和空间位置嵌入，对于成像的先验信息(如成像部位，使用的序列等)和磁共振数据中远距离相关信息，本发明提供的方法可以高效建模，更好的 学习磁共振领域知识，以改善磁共振重建图像的噪声和伪影问题。
所以，本发明可以提高磁共振图像质量，提高医生诊断的准确率；加快成像速度，提高磁共振机器利用率，应用范围广，无需对K空间中心满采，广泛适用于梯度回波，自旋回波、平面回波等成像方法。
附图说明
图1是本发明基于长距离注意力模型重建的多层磁共振成像方法实施例一的流程图。
图2是本发明基于长距离注意力模型重建的多层磁共振成像方法实施例一中以Transformer结构作为深度学习重建模型的原理图。
图3是本发明基于长距离注意力模型重建的多层磁共振成像方法实施例中一迭代展开框架下使用深度学习重建模型的原理图。
图4是本发明基于长距离注意力模型重建的多层磁共振成像装置实施例的原理图。
以下结合附图及实施例对本发明作进一步说明。
具体实施方式
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例的附图，对本发明实施例的技术方案进行清楚、完整地描述。显然，所描述的实施例是本发明的一部分实施例，而不是全部的实施例。基于所描述的本发明的实施例，本领域普通技术人员在无需创造性劳动的前提下所获得的所有其他实施例，都属于本发明保护的范围。
基于长距离注意力模型重建的多层磁共振成像方法第一实施例：
参见图1，本发明的基于长距离注意力模型重建的多层磁共振成像方法，包括以下步骤：
步骤S1，构建深度学习重建模型。
步骤S2，对多层同时采集的信号进行数据预处理，使用多层磁共振图像或K空间数据作为数据输入。
步骤S3，获取可学习的位置嵌入以及成像参数嵌入。
步骤S4，将经过预处理后的输入数据、位置嵌入以及成像参数嵌入输 入到深度学习重建模型。
步骤S5，由深度学习重建模型输出磁共振重建图像的结果。
在本实施例中，将经过预处理后的输入数据、位置嵌入以及成像参数嵌入输入到深度学习重建模型，具体包括：该深度学习重建模型为Transformer结构，该Transformer结构以Transformer编解码器为核心，其包含前置卷积层、后置卷积层和可学习的位置嵌入以及成像参数嵌入；在获取可学习的位置嵌入以及成像参数嵌入后，将经过预处理后的数据和可学习的位置嵌入相加后输入到深度学习重建模型的Transformer编码器中；将Transformer编码器的输出和可学习的成像参数嵌入输入到深度学习重建模型的Transformer解码器中；输出磁共振重建图像的结果。
在上述步骤S2中，在使用多层磁共振图像作为数据输入时，使用二维矩阵表示多层图像，其包括：沿磁共振读出方向在K空间插入零值，扩大磁共振读出方向视野(FOV)，然后进行快速傅里叶变换，得到沿磁共振读出方向连接的层面混叠的图像；使用多个二维卷积层，提取图像的特征，形成一个特征张量，并将该特征张量划分为区块。
在本实施例中，将经过预处理后的数据和可学习的位置嵌入相加后输入到深度学习重建模型的Transformer编码器中，包括：卷积层以1维数据的形式将每个区块展开，该数据与可学习的位置嵌入相加后输入到Transformer编码器中。
其中，上述位置嵌入由该区块所在坐标经过可学习的嵌入层变换得到；或者由该区块所在坐标经过全连接层变换得到。
在本实施例中，将Transformer编码器的输出和可学习的成像参数嵌入输入到深度学习重建模型的Transformer解码器中，具体包括：将Transformer编码器的输出和可学习的成像参数嵌入一起输入到Transformer解码器，Transformer解码器的输出经过重排形成另一个特征张量，再经过卷积层生成重建的多个层面图像，其中，Transformer编码器包含计算所有嵌入相关性的自注意力层和全连接层。
在上述步骤S2中，成像参数嵌入的获取包括：扫描成像信息，将扫描的成像信息编码为一个向量，将向量输入到嵌入层或全连接层或通过固定方式构造得到；其中，上述成像信息可以包括成像部位、层面的角度、采集加速倍数、可控混叠的方向和距离、使用的序列类型、相位编码的方向、FOV、TR时间、TE时间、脉冲翻转角、被扫描对象、扫描机器等。
在本实施例中，对构建的深度学习重建模型和基于梯度的数据一致性更新结合，进行端对端迭代训练，获得磁共振重建图像的最优输出结果。其中，基于梯度的数据可以是共轭梯度法、梯度下降法或其他一些迭代求解的改进方法。
在实际应用中，由于同时多层成像可以看作是3D情况下的并行成像，适用于同时多层成像的重建方法可以较为直接的推广到2D并行成像，故下面主要以同时多层成像的重建举例做方法说明。
根据线圈灵敏度编码理论，重建同时多层成像数据可以对应以下优化问题，如公式(1)：
x r＝argmin x‖Ax-b‖ 2+λ‖T(x)‖ 2  (1)
其中，A为编码算子，对应K空间同时多层采样和线圈灵敏度调制，x为待重建的多个层面的磁共振图像，b是采集的多通道K空间数据，T(x)是正则化约束。使用本实施例的深度学习重建模型实现T，如图2所示，以Transformer编解码器为核心，包含前置卷积层(convolutional neural network，CNN)、后置卷积层和可学习的位置嵌入(positional embedding)和成像参数嵌入(imaging parameter embedding)。
在本实施例中，首先对多层同时采集的信号进行readout-concatenate预处理，使得可以使用二维矩阵表示多层磁共振图像，其具体操作是沿磁共振读出方向在K空间插入零值，扩大磁共振读出方向FOV，然后进行快速傅里叶变换，得到沿磁共振读出方向连接的层面混叠的图像。然后，使用多个二维卷积层，提取特征，形成的特征张量，将特征常量划分为小的区块(Patch)，将每个区块展开成一维，和可学习的位置嵌入相加输入 到Transformer编码器中。当然，在另一些实施例中，也可以将多层同时采集的K空间(而不是图像)作为输入。
在本实施例中，上述的位置嵌入可以根据该区块所在坐标(x,y,z)或(kx，ky，kz)经过可学习的嵌入层(embedding layer，例如tensorflow框架中的tf.nn.embedding_lookup)变换得到的。或者，也可以通过所在坐标经过全连接层(fully connected layers)变换得到。当然，该位置嵌入也可以通过非学习的固定方式直接构造，例如使用正余弦编码。
在本实施例中，Transformer编码器可以参考BERT中的实现(https://github.com/google-research/bert)，其包含计算所有嵌入相关性的自注意力层和全连接层，将Transformer编码器的输出和可学习的成像参数嵌入(imaging parameter embedding)一起输入到Transformer解码器。Transformer解码器的输出经过重排形成另一个特征张量，再经过卷积层生成重建的多个层面图像(可控混叠引起的偏移可以随后去除)。
在本实施例中，上述的成像参数嵌入的获取方式如下：首先将该次扫描的成像信息编码为一个向量，再将向量输入到嵌入层或全连接层或通过固定方式构造得到。
其中，上述成像信息可以包括成像部位(头部、颈部、胸部、上腹部、下腹部、肘关节、膝关节等，每个部位用一个整数表示)，层面的角度(用和三个平面的夹角表示)，采集加速倍数(通常是介于1到20的小数)、可控混叠的方向和距离、使用的序列类型(FSE、FLASH、EPI、FLAIR等，每种序列使用一个整数表示)，相位编码的方向，FOV(视野大小)，TR时间、TE时间、脉冲翻转角、被扫描对象年龄、性别、身高、体重以及扫描机器的场强、品牌、型号等等。当然，在一些实施例中，这些信息也可以通过DICOM文件信息获取。
在本实施例中，可以使用上述深度学习重建模型处理输入数据，得到的输出即作为最终重建结果。
另外，可以结合MoDL重建框架(https://github.com/hkaggarwal/modl)， 将以上重建模型和基于梯度的数据一致性更新结合，进行端对端迭代，再得到最终输出图像，如图3所示，这样的可能优点在于可以使用更少的训练数据。当然，无论是采取何种方式，都要在训练数据集上进行训练重建模型。
本实施例的训练数据集可以通过多种方式获得，可以从真实磁共振机器上扫描获得的，也可以从大规模公开磁共振数据集(例如ADNI、HCP等)上取得，也可以通过算法仿真得到，例如通过bloch equation或者直接使用开源磁共振仿真软件如mrilab(http://mrilab.sourceforge.net/)。当然也可以混合使用以上三种方式。在训练时，可以采用L1损失、L2损失、感知损失、对抗性损失等加权平均的综合损失函数，可以使用ADAM优化器进行参数更新。
由此可见，本发明使用深度学习图像重建模型，并加入成像参数嵌入和空间位置嵌入，对于成像的先验信息(如成像部位，使用的序列等)和磁共振数据中远距离相关信息，本发明提供的方法可以高效建模，更好的学习磁共振领域知识，以改善磁共振重建图像的噪声和伪影问题。
所以，本发明可以提高磁共振图像质量，提高医生诊断的准确率；加快成像速度，提高磁共振机器利用率，应用范围广，无需对K空间中心满采，广泛适用于梯度回波，自旋回波、平面回波等成像方法。
基于长距离注意力模型重建的多层磁共振成像方法第二实施例：
在本实施例中，上述将经过预处理后的输入数据、位置嵌入以及成像参数嵌入输入到深度学习重建模型，还包括：将原始图像或K空间数据、成像参数嵌入、位置嵌入作为三个输入，其中，该深度学习重建模型包含卷积层和全连接层；将原始图像或K空间数据经过若干层的卷积层处理后形成N1个特征通道；将成像参数嵌入经过若干层的卷积层处理后形成N2个特征通道；将N1个特征通道和N2个特征通道相拼接后，再送入若干个卷积层，形成N3个特征通道；将位置嵌入经过全连接层后形成N3个输出值，将N3个输出值加到N3个特征通道上，经过卷积神经网络处 理，从而得到磁共振重建图像的输出结果。
可见，上述Transformer结构不是必须的，在本实施例当中，也可以只用卷积层和全连接层，可以将原始图像(或K空间)数据、成像参数嵌入、位置嵌入作为三个输入，其中，原始图像(或K空间)数据经过若干层的卷积层处理后形成N1个特征通道，成像参数嵌入也经过若干层的卷积层处理后形成N2个特征通道，将原始图像(或K空间)数据形成的N1个特征通道和成像参数形成的N2个特征通道相拼接后，再送入若干个卷积层，形成N3个特征通道。而位置嵌入经过全连接层后也形成N3个输出值，将N3个输出值加到N3个特征通道上，然后再经过卷积神经网络(例如使用resnet50，efficientnet等结构)的处理，得到重建结果。
基于多任务的磁共振重建模型训练装置实施例：
如图4所示，本发明提供的一种基于多任务的磁共振重建模型训练装置，包括：
模型构建单元10，用于构建深度学习重建模型。
数据预处理单元20，用于对多层同时采集的信号进行数据预处理，使用多层磁共振图像或K空间数据作为数据输入。
嵌入单元30，用于获取可学习的位置嵌入以及成像参数嵌入。
图像重建单元40，用于将经过预处理后的输入数据、位置嵌入以及成像参数嵌入输入到深度学习重建模型。
输出单元50，用于输出磁共振重建图像的结果。
在本实施例中，将经过预处理后的输入数据、位置嵌入以及成像参数嵌入输入到深度学习重建模型，具体包括：该深度学习重建模型为Transformer结构，该Transformer结构以Transformer编解码器为核心，其包含前置卷积层、后置卷积层和可学习的位置嵌入以及成像参数嵌入；在获取可学习的位置嵌入以及成像参数嵌入后，将经过预处理后的数据和可学习的位置嵌入相加后输入到深度学习重建模型的Transformer编码器中；将Transformer编码器的输出和可学习的成像参数嵌入输入到深度学习重 建模型的Transformer解码器中；输出磁共振重建图像的结果。
其中，在使用多层磁共振图像作为数据输入时，使用二维矩阵表示多层图像，其包括：沿磁共振读出方向在K空间插入零值，扩大磁共振读出方向视野，然后进行快速傅里叶变换，得到沿磁共振读出方向连接的层面混叠的图像；使用多个二维卷积层，提取图像的特征，形成一个特征张量，并将该特征张量划分为区块。
在本实施例中，将经过预处理后的数据和可学习的位置嵌入相加后输入到深度学习重建模型的Transformer编码器中，包括：卷积层以1维数据的形式将每个区块展开，该数据与可学习的位置嵌入相加后输入到Transformer编码器中。
其中，上述位置嵌入由该区块所在坐标经过可学习的嵌入层变换得到；或者由该区块所在坐标经过全连接层变换得到。
在本实施例中，将Transformer编码器的输出和可学习的成像参数嵌入输入到深度学习重建模型的Transformer解码器中，具体包括：将Transformer编码器的输出和可学习的成像参数嵌入一起输入到Transformer解码器，Transformer解码器的输出经过重排形成另一个特征张量，再经过卷积层生成重建的多个层面图像，其中，Transformer编码器包含计算所有嵌入相关性的自注意力层和全连接层。
其中，成像参数嵌入的获取包括：扫描成像信息，将扫描的成像信息编码为一个向量，将向量输入到嵌入层或全连接层或通过固定方式构造得到；其中，上述成像信息可以包括成像部位、层面的角度、采集加速倍数、可控混叠的方向和距离、使用的序列类型、相位编码的方向、FOV、TR时间、TE时间、脉冲翻转角、被扫描对象、扫描机器等。
在本实施例中，对构建的深度学习重建模型和基于梯度的数据一致性更新结合，进行端对端迭代训练，获得磁共振重建图像的最优输出结果。
计算机装置实施例：
本实施例的计算机装置包括处理器，处理器执行计算机程序时实现上 述多层磁共振成像方法实施例中的步骤。
例如，计算机程序可以被分割成一个或多个模块，一个或者多个模块被存储在存储器中，并由处理器执行，以完成本发明。一个或多个模块可以是能够完成特定功能的一系列计算机程序指令段，该指令段用于描述计算机程序在计算机装置中的执行过程。
计算机装置可包括，但不仅限于，处理器、存储器。本领域技术人员可以理解，计算机装置可以包括更多或更少的部件，或者组合某些部件，或者不同的部件，例如计算机装置还可以包括输入输出设备、网络接入设备、总线等。
例如，处理器可以是中央处理单元(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。处理器是计算机装置的控制中心，利用各种接口和线路连接整个计算机装置的各个部分。
存储器可用于存储计算机程序和/或模块，处理器通过运行或执行存储在存储器内的计算机程序和/或模块，以及调用存储在存储器内的数据，实现计算机装置的各种功能。例如，存储器可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序(例如声音接收功能、声音转换成文字功能等)等；存储数据区可存储根据手机的使用所创建的数据(例如音频数据、文本数据等)等。此外，存储器可以包括高速随机存取存储器，还可以包括非易失性存储器，例如硬盘、内存、插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)、至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。
存储介质实施例：
终端设备集成的模块如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明实现上述实施例方法中的全部或部分流程，也可以通过计算机程序来指令相关的硬件来完成，该计算机程序可存储于一计算机可读存储介质中，该计算机程序在被处理器执行时，可实现上述各个多层磁共振成像方法的各个步骤。
其中，计算机程序包括计算机程序代码，计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。计算机可读介质可以包括：能够携带计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、电载波信号、电信信号以及软件分发介质等。需要说明的是，计算机可读介质包含的内容可以根据司法管辖区内立法和专利实践的要求进行适当的增减，例如在某些司法管辖区，根据立法和专利实践，计算机可读介质不包括电载波信号和电信信号。
由此可见，本发明提供了一种计算机装置以及存储介质，其包括：一个或多个存储器，一个或多个处理器。存储器用于存储程序代码和程序运行过程中产生的中间数据、模型输出结果的储存和模型及模型参数的储存；处理器用于代码运行所占用的处理器资源和训练模型时占用的多个处理器资源。
需要说明的是，以上仅为本发明的优选实施例，但发明的设计构思并不局限于此，凡利用此构思对本发明做出的非实质性修改，也均落入本发明的保护范围之内。

Claims (10)

  1. 基于长距离注意力模型重建的多层磁共振成像方法，其特征在于，包括：
    构建深度学习重建模型；
    对多层同时采集的信号进行数据预处理，使用多层磁共振图像或K空间数据作为数据输入；
    获取可学习的位置嵌入以及成像参数嵌入；
    将经过预处理后的输入数据、位置嵌入以及成像参数嵌入输入到深度学习重建模型；
    由深度学习重建模型输出磁共振重建图像的结果。
  2. 根据权利要求1所述的多层磁共振成像方法，其特征在于，所述将经过预处理后的输入数据、位置嵌入以及成像参数嵌入输入到深度学习重建模型，包括：
    所述深度学习重建模型为Transformer结构，该Transformer结构以Transformer编解码器为核心，其包含前置卷积层、后置卷积层和可学习的位置嵌入以及成像参数嵌入；
    在获取可学习的位置嵌入以及成像参数嵌入后，将经过预处理后的数据和可学习的位置嵌入相加后输入到深度学习重建模型的Transformer编码器中；
    将Transformer编码器的输出和可学习的成像参数嵌入输入到深度学习重建模型的Transformer解码器中；
    输出磁共振重建图像的结果。
  3. 根据权利要求2所述的多层磁共振成像方法，其特征在于：
    在使用多层磁共振图像作为数据输入时，使用二维矩阵表示多层图像，其包括：沿磁共振读出方向在K空间插入零值，扩大磁共振读出方 向视野，然后进行快速傅里叶变换，得到沿磁共振读出方向连接的层面混叠的图像；
    使用多个二维卷积层，提取图像的特征，形成一个特征张量，并将该特征张量划分为区块。
  4. 根据权利要求3所述的多层磁共振成像方法，其特征在于，所述将经过预处理后的数据和可学习的位置嵌入相加后输入到深度学习重建模型的Transformer编码器中，包括：
    卷积层以1维数据的形式将每个区块展开，该数据与可学习的位置嵌入相加后输入到Transformer编码器中。
  5. 根据权利要求4所述的多层磁共振成像方法，其特征在于：
    所述位置嵌入由该区块所在坐标经过可学习的嵌入层变换得到；或者
    由该区块所在坐标经过全连接层变换得到。
  6. 根据权利要求5所述的多层磁共振成像方法，其特征在于，所述将Transformer编码器的输出和可学习的成像参数嵌入输入到深度学习重建模型的Transformer解码器中，包括：
    将Transformer编码器的输出和可学习的成像参数嵌入一起输入到Transformer解码器，Transformer解码器的输出经过重排形成另一个特征张量，再经过卷积层生成重建的多个层面图像，其中，Transformer编码器包含计算所有嵌入相关性的自注意力层和全连接层。
  7. 根据权利要求6所述的多层磁共振成像方法，其特征在于：
    所述成像参数嵌入的获取包括：扫描成像信息，将扫描的成像信息编码为一个向量，将向量输入到嵌入层或全连接层或通过固定方式构造得到；
    其中，所述成像信息可以包括成像部位、层面的角度、采集加速倍 数、可控混叠的方向和距离、使用的序列类型、相位编码的方向、FOV、TR时间、TE时间、脉冲翻转角、被扫描对象、扫描机器等。
  8. 根据权利要求1所述的多层磁共振成像方法，其特征在于，所述将经过预处理后的输入数据、位置嵌入以及成像参数嵌入输入到深度学习重建模型，包括：
    将原始图像或K空间数据、成像参数嵌入、位置嵌入作为三个输入，其中，该深度学习重建模型包含卷积层和全连接层；
    将原始图像或K空间数据经过若干层的卷积层处理后形成N1个特征通道；
    将成像参数嵌入经过若干层的卷积层处理后形成N2个特征通道；
    将N1个特征通道和N2个特征通道相拼接后，再送入若干个卷积层，形成N3个特征通道；
    将位置嵌入经过全连接层后形成N3个输出值，将N3个输出值加到N3个特征通道上，经过卷积神经网络处理，从而得到磁共振重建图像的输出结果。
  9. 根据权利要求1至8任一项所述的多层磁共振成像方法，其特征在于：
    对构建的深度学习重建模型和基于梯度的数据一致性更新结合，进行端对端迭代训练，获得磁共振重建图像的最优输出结果。
  10. 基于长距离注意力模型重建的多层磁共振成像装置，其特征在于，包括：
    模型构建单元，用于构建深度学习重建模型；
    数据预处理单元，用于对多层同时采集的信号进行数据预处理，使用多层磁共振图像或K空间数据作为数据输入；
    嵌入单元，用于获取可学习的位置嵌入以及成像参数嵌入；
    图像重建单元，用于将经过预处理后的输入数据、位置嵌入以及成像参数嵌入输入到深度学习重建模型；
    输出单元，用于输出磁共振重建图像的结果。
PCT/CN2022/103707 2021-09-27 2022-07-04 基于长距离注意力模型重建的多层磁共振成像方法及装置 WO2023045482A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111138087.5 2021-09-27
CN202111138087.5A CN113920213B (zh) 2021-09-27 2021-09-27 基于长距离注意力模型重建的多层磁共振成像方法及装置

Publications (1)

Publication Number Publication Date
WO2023045482A1 true WO2023045482A1 (zh) 2023-03-30

Family

ID=79236445

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/103707 WO2023045482A1 (zh) 2021-09-27 2022-07-04 基于长距离注意力模型重建的多层磁共振成像方法及装置

Country Status (3)

Country Link
US (1) US20230097417A1 (zh)
CN (1) CN113920213B (zh)
WO (1) WO2023045482A1 (zh)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102018217092A1 (de) * 2018-10-05 2020-04-09 Robert Bosch Gmbh Verfahren, künstliches neuronales Netz, Vorrichtung, Computerprogramm und maschinenlesbares Speichermedium zur semantischen Segmentierung von Bilddaten
CN113920213B (zh) * 2021-09-27 2022-07-05 深圳技术大学 基于长距离注意力模型重建的多层磁共振成像方法及装置
CN116741347B (zh) * 2023-05-12 2024-06-04 中山大学附属第一医院 一种病理图像patches提取与深度学习建模方法
CN116630386B (zh) * 2023-06-12 2024-02-20 新疆生产建设兵团医院 Cta扫描图像处理方法及其系统
US12105501B1 (en) * 2023-09-20 2024-10-01 Alexander Paul Eul Organ model production system
CN117315065B (zh) * 2023-09-26 2024-03-12 烟台大学 一种核磁共振成像精准加速重建方法以及系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200337648A1 (en) * 2019-04-24 2020-10-29 GE Precision Healthcare LLC Medical machine time-series event data processor
US20210082561A1 (en) * 2019-09-13 2021-03-18 RAD AI, Inc. Method and system for automatically generating a radiology impression
US20210133535A1 (en) * 2019-11-04 2021-05-06 Oracle International Corporation Parameter sharing decoder pair for auto composing
CN112862727A (zh) * 2021-03-16 2021-05-28 上海壁仞智能科技有限公司 一种跨模态图像转换方法及装置
US11045271B1 (en) * 2021-02-09 2021-06-29 Bao Q Tran Robotic medical system
CN113920213A (zh) * 2021-09-27 2022-01-11 深圳技术大学 基于长距离注意力模型重建的多层磁共振成像方法及装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11645791B2 (en) * 2019-10-17 2023-05-09 Rutgers, The State University Of New Jersey Systems and methods for joint reconstruction and segmentation of organs from magnetic resonance imaging data
US11120585B2 (en) * 2019-11-28 2021-09-14 Shanghai United Imaging Intelligence Co., Ltd. Systems and methods for image reconstruction
CN111870245B (zh) * 2020-07-02 2022-02-11 西安交通大学 一种跨对比度引导的超快速核磁共振成像深度学习方法
CN112150568A (zh) * 2020-09-16 2020-12-29 浙江大学 基于Transformer模型的磁共振指纹成像重建方法
CN113180633A (zh) * 2021-04-28 2021-07-30 济南大学 基于深度学习的mr影像肝癌术后复发风险预测方法及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200337648A1 (en) * 2019-04-24 2020-10-29 GE Precision Healthcare LLC Medical machine time-series event data processor
US20210082561A1 (en) * 2019-09-13 2021-03-18 RAD AI, Inc. Method and system for automatically generating a radiology impression
US20210133535A1 (en) * 2019-11-04 2021-05-06 Oracle International Corporation Parameter sharing decoder pair for auto composing
US11045271B1 (en) * 2021-02-09 2021-06-29 Bao Q Tran Robotic medical system
CN112862727A (zh) * 2021-03-16 2021-05-28 上海壁仞智能科技有限公司 一种跨模态图像转换方法及装置
CN113920213A (zh) * 2021-09-27 2022-01-11 深圳技术大学 基于长距离注意力模型重建的多层磁共振成像方法及装置

Also Published As

Publication number Publication date
CN113920213B (zh) 2022-07-05
US20230097417A1 (en) 2023-03-30
CN113920213A (zh) 2022-01-11

Similar Documents

Publication Publication Date Title
WO2023045482A1 (zh) 基于长距离注意力模型重建的多层磁共振成像方法及装置
Lin et al. Artificial intelligence for MR image reconstruction: an overview for clinicians
Du et al. Super-resolution reconstruction of single anisotropic 3D MR images using residual convolutional neural network
Hammernik et al. Learning a variational network for reconstruction of accelerated MRI data
US10852379B2 (en) Artifact reduction by image-to-image network in magnetic resonance imaging
CN110461228A (zh) 使用多对比度和深度学习改进医学图像的质量
de Leeuw Den Bouter et al. Deep learning-based single image super-resolution for low-field MR brain images
CN110992433A (zh) 用于利用空数据一致性来稀疏图像重建的系统和方法
Du et al. Accelerated super-resolution MR image reconstruction via a 3D densely connected deep convolutional neural network
WO2020114329A1 (zh) 磁共振快速参数成像方法及装置
Shende et al. A brief review on: MRI images reconstruction using GAN
Schaetz et al. Accelerated computing in magnetic resonance imaging: real‐time imaging using nonlinear inverse reconstruction
Wang et al. Unsupervised learning for cross-domain medical image synthesis using deformation invariant cycle consistency networks
Shen et al. Rapid reconstruction of highly undersampled, non‐Cartesian real‐time cine k‐space data using a perceptual complex neural network (PCNN)
CN106680748A (zh) 磁共振图像获取方法及磁共振系统
CN111157935A (zh) 磁共振成像方法、装置、存储介质及计算机设备
CN118037615A (zh) 一种肿瘤分割引导的磁共振图像翻译方法、系统、设备及介质
JP7300811B2 (ja) 医用情報処理装置、医用情報処理方法、およびプログラム
CN109544488B (zh) 一种基于卷积神经网络的图像合成方法
Mishra et al. Global U-net with amalgamation of inception model and improved kernel variation for MRI brain image segmentation
CN110226100A (zh) 用于磁共振成像的系统和方法
CN114119788A (zh) 一种基于对抗生成网络的多模态医学图像编码与生成方法
Forigua et al. SuperFormer: Volumetric transformer architectures for MRI super-resolution
Gulamhussene et al. Transfer-learning is a key ingredient to fast deep learning-based 4D liver MRI reconstruction
WO2024021796A1 (zh) 一种图像处理方法、装置、电子设备、存储介质及程序产品

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22871543

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22871543

Country of ref document: EP

Kind code of ref document: A1