WO2021097594A1

WO2021097594A1 - Quick imaging model training method and apparatus, and server

Info

Publication number: WO2021097594A1
Application number: PCT/CN2019/119097
Authority: WO
Inventors: 王珊珊; 郑海荣; 梁皓云; 刘新; 梁栋
Original assignee: 中国科学院深圳先进技术研究院
Priority date: 2019-11-18
Filing date: 2019-11-18
Publication date: 2021-05-27

Abstract

A quick imaging model training method and apparatus, and a server. The method comprises: during each instance of model iteration training, carrying out under-sampling, according to an under-sampling mask, on an image obtained by means of magnetic resonance scanning, to obtain training data (S110); inputting the training data into a quick imaging model, performing feature extraction on the training data by means of N multi-granularity attention modules and according to multi-scale information and an attention mechanism of the image, and fusing a feature map extracted by each multi-granularity attention module (S120); performing image reconstruction on the fused feature map, and outputting imaging data (S130); reversely calculating a gradient according to the imaging data and a target label so as to update parameters of the quick imaging model and the under-sampling mask by means of the gradient (S140); and performing forward calculation by using the updated parameters and the updated under-sampling mask to output the next piece of imaging data (S150). The method solves the problems of it not being possible to optimize an under-sampling mask and an imaging effect being poor.

Description

Training method, device and server of fast imaging model

Technical field

This application relates to the technical field of magnetic resonance scanning imaging, in particular to a training method, device and server for a fast imaging model.

Background technique

Due to its powerful functions, magnetic resonance imaging can provide a wealth of anatomical and functional information, making magnetic resonance imaging widely used in the medical field. In order to perform magnetic resonance imaging, the patient needs to be clinically scanned with magnetic resonance. During the scan, the patient needs to maintain a posture for a long time, resulting in a poor patient experience. Therefore, the speed of magnetic resonance imaging needs to be accelerated. In a real scene, the magnetic resonance scanner needs to sample the data at the Nyquist sampling frequency to ensure that the data can be restored without distortion.

In related technologies, the method of rapid magnetic resonance imaging is mainly constructed based on deep learning. The main steps of imaging using this method are to consider collecting only part of the data, sampling the data in a sampling method that does not meet the Nyquist sampling theorem (high-power retrospective under-sampling); performing zero-filling operations on the obtained under-sampling data Obtain a zero-filled image; input the zero-filled image into the deep learning network, and output the restored high-definition image after processing by the deep learning network. However, because the fast imaging method constructed based on deep learning cannot learn from the under-sampling mask used for data sampling, the under-sampling mask cannot be optimized; and the fast imaging method constructed based on deep learning only considers the channel attention, resulting in The imaging effect of the rapid imaging method is not good.

Summary of the invention

technical problem

One of the objectives of the embodiments of the present application is to provide a fast imaging model training method, device, and server, aiming to solve the problems of the inability to optimize the under-sampling mask and the long imaging time.

The solution to the problem

Technical solutions

In order to solve the above technical problems, the technical solutions adopted in the embodiments of this application are:

In the first aspect, a training method for a fast imaging model is provided, including:

In each iteration of the model training, under-sampling the image scanned by MRI according to the under-sampling mask to obtain training data;

The training data is input into a fast imaging model, the training data is extracted according to the multi-scale information of the image and the attention mechanism through N multi-granularity attention modules, and each of the multi-granularity attention modules is merged to extract Characteristic map of; N≥1;

Perform image reconstruction on the fused feature map and output imaging data;

Reversely calculate gradients according to the imaging data and target tags, so as to update the parameters of the fast imaging model and the under-sampling mask through the gradients;

The updated parameters and the updated under-sampling mask are used for forward calculation to output the next imaging data.

In one embodiment, the training data is input to a fast imaging model, and features are extracted from the training data according to the multi-scale information of the image and the attention mechanism through N multi-granularity attention modules, and each of them is merged. The feature maps extracted by the multi-granularity attention module include:

Extracting the initialization feature data of the training data;

For each of the multi-granularity attention modules, perform feature extraction on the initialization feature data according to preset image scales, and fuse the extracted feature maps;

Divide the fused feature map into several regional images with different attention weights through the multi-granularity attention mechanism;

Fusion of all the regional images to obtain a feature map after feature refinement.

In an embodiment, the step of calculating the gradient inversely according to the imaging data and the target label to update the parameters of the fast imaging model and the under-sampling mask through the gradient includes:

Calculate gradients backwards according to the imaging data and the target tag to obtain a gradient matrix;

Updating the attention weight given to the several regional images by the multi-granularity attention mechanism according to the gradient matrix.

In an embodiment, the step of calculating the gradient inversely according to the imaging data and the target label to update the parameters of the fast imaging model and the under-sampling mask through the gradient further includes:

Generating a continuous mask according to the under-sampling mask, and adding the continuous mask and the gradient matrix to obtain an updated continuous mask;

Binarize the updated continuous mask to obtain an updated under-sampling mask.

In one embodiment, the fast imaging model includes learning the convolutional layer of the under-sampling mask, and correspondingly setting the convolution kernel and parameters of the convolutional layer according to the under-sampling mask; the under-sampling mask The initial value of the film includes a preset number of low-frequency sampling strips and randomly sampled high-frequency sampling strips.

In an embodiment, the rule for binarizing the updated continuous mask is: set to 1 when any element in the updated continuous mask is greater than a preset threshold; when If any element in the updated continuous mask is less than the preset threshold, it is set to 0; wherein, the preset threshold is a preset percentage of the maximum value in the updated continuous mask; The preset percentage is set according to the imaging acceleration multiple.

In an embodiment, the under-sampling of the image scanned by magnetic resonance according to the under-sampling mask to obtain training data includes:

Performing under-sampling on the image scanned by magnetic resonance according to the under-sampling mask to obtain under-sampled K-space data;

Performing an inverse Fourier transform on the under-sampled K-space data to obtain under-sampled image domain data as the training data.

In one embodiment, inverse Fourier transform is performed on the image scanned by magnetic resonance to obtain full-sample image domain data, which is used as the target tag.

In the second aspect, a training device for a fast imaging model is provided, including:

The training data generation module is used for under-sampling the image scanned by MRI according to the under-sampling mask during each iteration of the model training to obtain training data;

The model training module is used to input the training data into the fast imaging model for model training to obtain imaging data;

The feature extraction module is used to input the training data into the fast imaging model, and perform feature extraction on the training data according to the multi-scale information and attention mechanism of the image through N multi-granularity attention modules, and fuse each of the multiple The feature map extracted by the granular attention module; N≥1;

The image fusion module is used to reconstruct the image of the fused feature map and output the imaging data;

The forward calculation module is used for forward calculation using the updated parameters and the updated under-sampling mask to output the next imaging data.

In an embodiment, the feature extraction module includes:

The feature extraction unit is used to extract the initialization feature data of the training data.

In an embodiment, each of the multi-granularity attention modules includes:

The multi-scale densely connected feature fusion unit is used to perform feature extraction on the initial feature data according to several preset image scales, and fuse several extracted feature maps;

The feature refinement unit based on the multi-granularity attention mechanism is used to segment the fused feature map into several regional images with different attention weights through the multi-granularity attention mechanism;

The fusion image unit is used to fuse all the region images to obtain a feature map after feature refinement.

In an embodiment, the parameter and under-sampling mask update module includes:

A gradient calculation unit, configured to reversely calculate a gradient according to the imaging data and the target tag to obtain a gradient matrix;

The model parameter update unit is configured to update the attention weight given to the plurality of regional images by the multi-granularity attention mechanism according to the gradient matrix.

In an embodiment, the parameter and under-sampling mask update module further includes:

An under-sampling mask updating unit, configured to generate a continuous mask according to the under-sampling mask, and add the continuous mask and the gradient matrix to obtain an updated continuous mask;

The mask binarization unit is used to binarize the updated continuous mask to obtain an updated under-sampling mask.

In a third aspect, a server is provided, including: a memory, a processor, and a computer program stored in the memory and capable of being run on the processor. When the processor executes the computer program, the computer program in the first aspect is implemented. Training method of fast imaging model.

The training method, device, and server of a fast imaging model provided by the embodiments of the application have the beneficial effect of: under-sampling the image scanned by magnetic resonance according to the under-sampling mask during each iteration of the model training to obtain training Data; input the training data into a fast imaging model, and perform feature extraction on the training data according to the multi-scale information and attention mechanism of the image through N multi-granularity attention modules, and fuse each of the multi-granularity attention modules The extracted feature map; N≥1; perform image reconstruction on the fused feature map, and output imaging data; the fast imaging model includes a neural network layer that learns the under-sampling mask; according to the imaging data and the target label Calculate the gradient backwards to update the parameters of the fast imaging model and the under-sampling mask through the gradient; use the updated parameters and the updated under-sampling mask to perform forward calculation to output the next Imaging data. By embedding the neural network that learns the under-sampling mask into the fast imaging model, iteratively trains together, and optimizes the under-sampling mask and model parameters according to the gradient of the imaging data and the target label inversely calculated, thereby improving the imaging rate of the fast imaging model. And the fast imaging model includes N multi-granularity attention modules to extract features of the training data according to the multi-scale information and attention mechanism of the image, and make full use of the multi-granularity information and regional attention of the image. Enhance the representation of features in the imaging data, thereby improving the imaging effect.

The beneficial effects of the invention

Brief description of the drawings

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings that need to be used in the embodiments or exemplary technical descriptions. Obviously, the accompanying drawings in the following description are only of the present application. For some embodiments, those of ordinary skill in the art can obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic flowchart of a method for training a fast imaging model provided in Embodiment 1 of the present application;

2 is a schematic structural diagram of a fast imaging model provided by Embodiment 1 of the present application;

FIG. 3 is a schematic structural diagram of a multi-granularity attention module provided in Embodiment 1 of the present application;

4 is a schematic structural diagram of a feature refinement part based on a multi-granularity attention mechanism provided by Embodiment 1 of the present application;

FIG. 5 is a schematic flowchart of a training method for a fast imaging model provided in Embodiment 2 of the present application;

6 is a schematic diagram of an embodiment of a magnetic resonance scanning imaging process provided in Embodiment 2 of the present application;

FIG. 7 is a schematic structural diagram of a training device for a fast imaging model provided in Embodiment 3 of the present application;

FIG. 8 is a schematic structural diagram of a server provided in Embodiment 4 of the present application.

Invention embodiment

Embodiments of the present invention

In order to make the purpose, technical solutions, and advantages of this application clearer, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the application, but not used to limit the application.

It should be noted that the full text of the application and the above-mentioned drawings in the term "including" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes Other steps or units inherent in these processes, methods, products or equipment. The terms "upper", "lower", "left", "right", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for ease of description, and do not indicate or imply the device referred to. Or the element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be construed as a limitation of the present application. For those of ordinary skill in the art, the specific meaning of the above terms can be understood according to specific conditions. The terms "first" and "second" are only used for ease of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features. The meaning of "plurality" means two or more than two, unless otherwise specifically defined.

In order to illustrate the technical solutions described in this application, detailed descriptions are given below in conjunction with specific drawings and embodiments.

Example one

As shown in FIG. 1, it is a schematic flowchart of a training method for a fast imaging model provided in Embodiment 1 of the present application. This embodiment can be applied to the application scenario of magnetic resonance scanning imaging. The method can be executed by a training device of a fast imaging model. The device can be a server, a smart terminal, a tablet or a PC, etc.; in this embodiment of the application, fast imaging is used. The training device of the model is explained as the main body of execution, and the method specifically includes the following steps:

S110: During each iteration of the model training, under-sampling the image scanned by the magnetic resonance according to the under-sampling mask to obtain training data;

During the MRI scan imaging process, the scan data, that is, the full sampling K-space data, is obtained. The magnetic resonance scanner needs to sample the scanned data at the Nyquist sampling frequency to generate images to ensure that the data can be restored without distortion. However, the process of sampling the scanned data at the Nyquist sampling frequency is slow, resulting in a long imaging time. In order to speed up the above-mentioned data sampling process and increase the imaging speed, it may be considered to collect only part of the scanned data, and sample the data at a sampling rate lower than the Nyquist sampling frequency, that is, under-sampling. There are many under-sampling methods in related technologies. The commonly used method is 1D random (one-dimensional random), which uses an under-sampling matrix whose number of columns is consistent with the length of the phase encoding direction of the K-space image (scan data), that is, the under-sampling mask. Multiply the scanned data to get an under-sampled image. Therefore, the data required for imaging can be obtained by under-sampling the scan data according to the preset under-sampling mask.

Specifically, during a certain iterative training of a model, if the iterative training process is the first model iterative training process, under-sampling the image scanned by magnetic resonance according to the preset initial value of the under-sampling mask to obtain training data; If this iterative training process is not the first model iterative training process, then the image scanned by the magnetic resonance is under-sampled according to the under-sampling mask updated after the previous iterative training to obtain training data.

S120. Input the training data into a fast imaging model, perform feature extraction on the training data according to the multi-scale information and attention mechanism of the image through N multi-granularity attention modules, and fuse each of the multi-granularity attention modules The extracted feature map; N≥1;

In an implementation example, as shown in FIG. 2, it is a schematic structural diagram of a fast imaging model. The fast imaging model can be a multi-granularity attention network, which mainly includes two parts: a feature extraction part 21 and a reconstruction part 22. In each iteration of the model training process, after the training data is input to the fast imaging model, the initial feature data of the training data can be extracted through a convolutional layer in the feature extraction part 21 first.

Specifically, the feature extraction part of the fast imaging model also includes N multi-granularity attention modules 23, where N≥1; and the parameters in each multi-granularity attention module 23 are different to add more nonlinear operations. Make the result more optimized. Optionally, N can be 5. The initial feature data extracted through a convolutional layer in the feature extraction part 21 is input to a multi-granularity attention module 23, and the initial feature data is feature extracted according to the multi-scale information of the image and the attention mechanism to obtain the feature image, and then The feature image is input to the next multi-granularity attention module 23 until N multi-granularity attention modules 23 are traversed. The feature extraction part of the fast imaging model also includes a connection layer, through which the feature maps extracted by each of the multi-granularity attention modules are fused together. Since the feature maps extracted by each of the multi-granularity attention modules produce feature maps of many channels, one convolutional layer in the feature extraction part needs to modify the number of channels of the feature maps. The feature map after the number of channels is modified also needs to perform global residual calculation to prevent the problem of gradient disappearance due to too deep model network layers and difficulty in training parameters. The calculated feature map is then input into the reconstruction part 22 in the fast imaging model to generate an image, which enhances the representation of the features in the generated image.

In an implementation example, the process for each multi-granularity attention module to perform feature extraction on the initial feature data according to the multi-scale information of the image and the attention mechanism is: for each of the multi-granularity attention modules, according to a preset Perform feature extraction on the initial feature data at several image scales, and fuse the extracted feature maps; divide the fused feature maps into several regional images with different attention weights through a multi-granularity attention mechanism; fuse all The area image is described, and the feature map after feature refinement is obtained.

Specifically, each multi-grained attention block may include two parts: a feature fusion part based on multi-scale dense connection and a feature refinement part based on a multi-grained attention mechanism; and each multi-grained attention block There is a local residual connection in the force module. As shown in Figure 3, it is a schematic diagram of the structure of the multi-granularity attention module. Since visual information of different scales will be helpful for imaging, the feature fusion part based on multi-scale dense connection performs feature extraction on the initial feature data according to several preset image scales, and fuse several extracted feature maps.

There are several units in the feature fusion part based on multi-scale dense connection, each unit has two paths, and each path is equipped with a convolutional layer. According to a number of preset image scales, each unit is The convolutional layer parameters are set. Optionally, there are 3 units in the feature fusion part based on multi-scale dense connection, and a convolution layer with a 3×3 convolution kernel and a convolution layer with a 5×5 convolution kernel may be used respectively. After the initial feature data is input based on the feature fusion part of multi-scale dense connection, the initial feature data is convolved through two convolutional layers in a unit, and then the outputs of the two convolutional layers are fused together through the connection layer, thus The feature maps containing visual information of different scales are integrated together; the feature images obtained by the fusion are input into the next unit in a densely connected manner to continue the convolution calculation, until traversing the 3 in the feature fusion part based on the multi-scale densely connected Units. Based on the feature fusion part of multi-scale dense connection, several feature maps extracted from the fusion are input into the feature refinement part based on the multi-granularity attention mechanism after convolution with a convolution kernel of 1X1.

Specifically, the feature refinement part based on the multi-granularity attention mechanism may include two parts: the squeeze excitation operation and the multi-granularity attention mechanism. As shown in Figure 4, it is a schematic diagram of the structure of the feature refinement part based on the multi-granularity attention mechanism. Input the feature map based on the feature refinement part of the multi-granularity attention mechanism through the multi-granular attention mechanism to divide the fused feature map into several regional images with different attention weights. The multi-granularity attention mechanism divides the input feature maps in a variety of preset ways, and each division way forms a corresponding number of regional feature maps. Optionally, three different image segmentation methods can be preset, which are S=1, S=2, and S=3, respectively. And the divided regional images are all given corresponding attention weight values, and the attention weight values of the regional images obtained by different division methods are not the same. Each regional image with attention weight also needs to go through the squeeze excitation operation, that is, go through the corresponding global pooling, and then go through two convolutional layers with weighted convolution kernels of 1X1 to obtain the learned channel weights W ₁ and W ₂ ; It also needs to go through activation function calculation and a dot product operation to get the final attention weight value. Optionally, the activation function may be a Sigmoid activation function. Fusion of all the regional images with the final attention weight value to obtain the feature map after feature refinement.

S130: Perform image reconstruction on the fused feature map, and output the imaging data;

Input the training data into the feature extraction part of the fast imaging model to fuse all the regional images with the final attention weight value to obtain the feature map after feature refinement, and the reconstruction part of the fast imaging model to image the feature map after feature refinement Reconstruction, and output the imaging data. Optionally, the reconstruction part may be composed of an up-sampling layer and a convolutional layer. Since the feature map after feature refinement is inconsistent with the final real image (target label), there is only half of the dimension. The feature map after feature refinement is up-sampling through the upsampling layer to restore the feature map to the real image (target label) Same size. The up-sampled feature map is then convolved by the convolutional layer to obtain imaging data.

S140. Calculate the gradient in reverse according to the imaging data and the target tag, so as to update the parameters of the fast imaging model and the under-sampling mask through the gradient;

In each round of model iterative training, when the fast imaging model performs imaging calculation on the input training data and the output imaging data, the parameters and under-sampling masks of the fast imaging model are determined based on the imaging data output from this model training. For optimization, the gradient can be calculated inversely based on the imaging data output from this model training and the preset target label, so as to update the parameters of the fast imaging model and the under-sampling mask according to the calculated gradient.

In an implementation example, the process of updating the parameters of the fast imaging model according to the gradient calculated inversely between the imaging data and the target tag may be: calculating the gradient inversely according to the imaging data and the target tag to obtain a gradient matrix; The gradient matrix updates the attention weight given to the several regional images by the multi-granularity attention mechanism.

S150. Use the updated parameters and the updated under-sampling mask to perform forward calculation to output the next imaging data.

In each round of model iterative training process, the parameters of the fast imaging model and the under-sampling mask are updated according to the output imaging data and the target label in reverse calculation of the gradient, and then the model iterative training process of this round is completed. The updated parameters and the under-sampling mask are used for forward calculation to perform the next round of model iterative training.

The training method of a fast imaging model provided by the embodiment of the application is to obtain training data by under-sampling the image scanned by magnetic resonance according to the under-sampling mask during each iteration of the model training; and input the training data The fast imaging model uses N multi-granularity attention modules to perform feature extraction on the training data according to the multi-scale information and attention mechanism of the image, and fuse the feature maps extracted by each of the multi-granularity attention modules; N≥ 1; Perform image reconstruction on the fused feature map and output imaging data; the fast imaging model includes a neural network layer that learns the under-sampling mask; the gradient is calculated inversely according to the imaging data and the target label to pass all The gradient updates the parameters of the fast imaging model and the under-sampling mask; the updated parameters and the updated under-sampling mask are used for forward calculation to output the next imaging data. By embedding the neural network that learns the under-sampling mask into the fast imaging model, iteratively trains together, and optimizes the under-sampling mask and model parameters according to the gradient of the imaging data and the target label inversely calculated, thereby improving the imaging rate of the fast imaging model. And the fast imaging model includes N multi-granularity attention modules to extract features of the training data according to the multi-scale information and attention mechanism of the image, and make full use of the multi-granularity information and regional attention of the image. Enhance the representation of features in the imaging data, thereby improving the imaging effect.

Example two

FIG. 5 is a schematic flowchart of the training method of the fast imaging model provided in the second embodiment of the present application. On the basis of the first embodiment, this embodiment also provides a method of implementing the learning of the under-sampling mask by embedding the neural network for learning the under-sampling mask into the fast imaging model together with iterative training. The method specifically includes:

S210: During each iteration of the model training, under-sampling the image scanned by the magnetic resonance according to the under-sampling mask to obtain training data;

In related technologies, a fast imaging model that can be constructed through deep learning generates images based on data obtained by under-sampling. If the imaging effect is not good, the parameters of the fast imaging model can be optimized through multiple iteration training. However, no matter how the fast imaging model is optimized, the under-sampling data of the input model is always obtained by under-sampling according to the initial under-sampling mask, and the under-sampling mask cannot be optimized at the same time according to the imaging effect. Since the under-sampling mask is related to the imaging speed of the fast imaging model, the under-sampling mask cannot be updated and optimized, which makes the imaging time long.

In order to solve the above problems, the neural network layer that learns the under-sampling mask is embedded in the fast imaging model. When the fast imaging model is iteratively trained, the image scanned by magnetic resonance is under-sampled according to the under-sampling mask to obtain training data. . The under-sampling mask can be iteratively trained with the fast imaging model to generate a learnable under-sampling mask.

In an implementation example, the fast imaging model includes a convolutional layer that learns an under-sampling mask, and the convolution kernel and parameters of the convolutional layer are correspondingly set according to the elements contained in the under-sampling mask; the initial value of the under-sampling mask includes A preset number of low-frequency sampling strips and randomly sampled high-frequency sampling strips. When the model is iteratively trained for the first time, the under-sampling method of sampling a certain number of middle parts of the scanned image and randomly sampling a part of the surroundings can be adopted. The initial value corresponding to the preset under-sampling mask includes a preset number of low-frequency samples Strips and randomly sampled high-frequency sampling strips. Specifically, the under-sampling mask is a binarization mask (that is, only two values of 0 and 1 are included), the element corresponding to the sampling bar in the under-sampling mask is "1", and the remaining elements are "0".

In an implementation example, the image scanned by magnetic resonance is under-sampled according to the under-sampling mask, and the specific process of obtaining training data may be: according to the under-sampling mask, the image scanned by magnetic resonance (full-sampled K-space Data) performing under-sampling to obtain under-sampled K-space data; performing inverse Fourier transform on the under-sampled K-space data to obtain under-sampled image domain data, which is used as the training data.

S220: Input the training data into a fast imaging model for model training, to obtain imaging data;

In each round of model iterative training process, as shown in Figure 6 is a schematic diagram of an embodiment of the magnetic resonance scanning imaging process. After under-sampling the image scanned by magnetic resonance according to the under-sampling mask, the training data will be obtained. Data is input to the fast imaging model for model training to obtain imaging data, that is, imaging calculations are performed according to the training data to generate reconstructed images.

S230. Calculate gradients inversely according to the imaging data and target tags, so as to update the parameters of the fast imaging model and the under-sampling mask through the gradients;

In an implementation example, the gradient is calculated backwards according to the imaging data output by the model training and the preset target label to update the under-sampling mask may be: the gradient is calculated backwards according to the imaging data and the target label, Obtain a gradient matrix; generate a continuous mask according to the under-sampling mask, add the continuous mask and the gradient matrix to obtain an updated continuous mask; combine the updated continuous mask Binarize to obtain the updated under-sampling mask.

Optionally, the full-sampling image domain data can be obtained by performing inverse Fourier transform on the image scanned by magnetic resonance, which can be used as a preset target label. According to the error between the target label and the output imaging data, the gradient is calculated inversely to obtain the gradient matrix, and then the current undersampling mask needs to be generated according to the current undersampling mask before updating the current undersampling mask according to the gradient matrix. If this model training process is the first model iterative training process, the current under-sampling mask is the initial value, and the sampling strip position of the continuous mask generated according to the current under-sampling mask can be preset to be consistent with the current under-sampling mask. The initial value of the sampling strip position comes from a uniform distribution U(0.5, 1), and the initial value of the non-sampling strip position comes from another uniform distribution U(0, 0.5). The generated continuous mask and the calculated gradient matrix are equal in size, and each element in the gradient matrix is the gradient that the corresponding element in the continuous mask needs to update. After that, the generated continuous mask and the gradient matrix are added to obtain the updated continuous mask; the updated continuous mask is binarized to obtain the updated under-sampling mask, so that the updated continuous mask can be obtained. The under-sampling mask of is used in the next round of fast imaging model training process.

In an implementation example, the rule for binarizing the updated continuous mask is: when any element in the updated continuous mask is greater than a preset threshold, set to 1; when the Any element in the updated continuous mask is set to 0 when it is less than the preset threshold; wherein, the preset threshold is a preset percentage of the maximum value in the updated continuous mask; The preset percentage is set according to the imaging acceleration multiple.

Specifically, the updated continuous mask is binarized according to the above-mentioned rules, and the value of the element greater than the preset threshold in the updated continuous mask is set to 1, and the value of the updated continuous mask is smaller than the preset threshold. Set the value of the threshold element to 0 to obtain the updated under-sampling mask. Wherein, the preset threshold is a preset percentage of the maximum value in the continuous mask after the update; and the preset percentage is set according to the imaging acceleration multiple. Optionally, the corresponding relationship between the imaging acceleration factor and the percentage may be that acceleration factor 4 corresponds to 25%, acceleration factor 8 corresponds to 12.5%, acceleration factor 12 corresponds to 8.3%, and acceleration factor 16 corresponds to 6.25%. In order to improve the imaging speed by updating and optimizing the under-sampling mask.

S240: Perform forward calculation using the updated parameters and the updated under-sampling mask to output the next imaging data.

Example three

FIG. 7 is a schematic structural diagram of a training device for a fast imaging model provided in Embodiment 3 of the present application. On the basis of

Embodiment

1 or 2, an embodiment of the present application also provides a training device 7, which includes:

The training data generation module 701 is used for under-sampling the image scanned by magnetic resonance according to the under-sampling mask during each iteration of the model training to obtain training data;

In an implementation example, during each model iteration training, the image scanned by the magnetic resonance is under-sampled according to the under-sampling mask, and when training data is obtained, the training data generation module 701 includes:

An under-sampling unit for under-sampling the image scanned by magnetic resonance according to the under-sampling mask to obtain under-sampled K-space data;

A data processing unit, configured to perform inverse Fourier transform on the under-sampled K-space data to obtain under-sampled image domain data as the training data;

The target label generating unit is configured to perform inverse Fourier transform on the image scanned by magnetic resonance to obtain full-sampled image domain data, which is used as the target label.

The feature extraction module 702 is used to input the training data into a fast imaging model, and perform feature extraction on the training data according to the multi-scale information and attention mechanism of the image through N multi-granularity attention modules, and fuse each of the training data. The feature map extracted by the multi-granularity attention module; N≥1;

The image fusion module 703 is configured to perform image reconstruction on the fused feature map, and output the imaging data.

In an implementation example, the feature extraction module 702 includes:

In an implementation example, each multi-granularity attention module includes:

The image fusion unit is used to fuse all the regional images to obtain a feature map after feature refinement.

The parameter and under-sampling mask update module 704 is configured to calculate a gradient inversely according to the imaging data and the target tag, so as to update the parameters of the fast imaging model and the under-sampling mask through the gradient;

In an implementation example, when the gradient is calculated inversely according to the imaging data and the target tag, so as to update the parameters of the fast imaging model and the under-sampling mask through the gradient, the parameter and under-sampling mask update module 704 include:

The forward calculation module 705 is configured to use the updated parameter and the updated under-sampling mask to perform forward calculation to output the next imaging data.

The training device for a fast imaging model provided by an embodiment of the present application obtains training data by under-sampling images scanned by magnetic resonance according to an under-sampling mask during each iteration of the model training; and inputting the training data The fast imaging model uses N multi-granularity attention modules to perform feature extraction on the training data according to the multi-scale information and attention mechanism of the image, and fuse the feature maps extracted by each of the multi-granularity attention modules; N≥ 1; Perform image reconstruction on the fused feature map and output imaging data; the fast imaging model includes a neural network layer that learns the under-sampling mask; the gradient is calculated inversely according to the imaging data and the target label to pass all The gradient updates the parameters of the fast imaging model and the under-sampling mask; the updated parameters and the updated under-sampling mask are used for forward calculation to output the next imaging data. By embedding the neural network that learns the under-sampling mask into the fast imaging model, iteratively trains together, and optimizes the under-sampling mask and model parameters according to the gradient of the imaging data and the target label inversely calculated, thereby improving the imaging rate of the fast imaging model. And the fast imaging model includes N multi-granularity attention modules to extract features of the training data according to the multi-scale information and attention mechanism of the image, and make full use of the multi-granularity information and regional attention of the image. Enhance the representation of features in the imaging data, thereby improving the imaging effect.

Example four

FIG. 8 is a schematic structural diagram of a server provided in Embodiment 4 of the present application. The server includes: a processor 1, a memory 2, and a computer program 3 stored in the memory 2 and running on the processor 1, such as a program for a training method of a fast imaging model. When the processor 1 executes the computer program 3, the steps in the above-mentioned fast imaging model training method embodiment are implemented, for example, steps S110 to S150 shown in FIG. 1.

Exemplarily, the computer program 3 may be divided into one or more modules, and the one or more modules are stored in the memory 2 and executed by the processor 1 to complete the application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 3 in the server. For example, the computer program 3 can be divided into a training data generation module, a feature extraction module, an image fusion module, a parameter and under-sampling mask update module, and a forward calculation module. The specific functions of each module are as follows:

The parameter and under-sampling mask update module is used to calculate a gradient inversely according to the imaging data and the target tag, so as to update the parameters of the fast imaging model and the under-sampling mask through the gradient;

The server may include, but is not limited to, a processor 1, a memory 2, and a computer program 3 stored in the memory 2. Those skilled in the art can understand that FIG. 8 is only an example of a server, and does not constitute a limitation on the server. It may include more or less components than those shown in the figure, or a combination of certain components, or different components, such as the The server may also include input and output devices, network access devices, buses, and so on.

The processor 1 may be a central processing unit (Central Processing Unit, CPU), other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (ASIC), Ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.

The storage 2 may be an internal storage unit of the server, such as a hard disk or memory of the server. The memory 2 may also be an external storage device, such as a plug-in hard disk equipped on a server, a smart memory card (Smart Media Card, SMC), a Secure Digital (SD) card, a flash memory card (Flash Card), etc. Further, the storage 2 may also include both an internal storage unit of the server and an external storage device. The memory 2 is used to store the computer program and other programs and data required by the fast imaging model training method. The memory 2 can also be used to temporarily store data that has been output or will be output.

Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the division of the above functional units and modules is used as an example. In practical applications, the above functions can be allocated to different functional units and modules as needed. Module completion, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist alone physically, or two or more units can be integrated into one unit. The above-mentioned integrated units can be hardware-based Formal realization can also be realized in the form of software functional units. In addition, the specific names of the functional units and modules are only used to facilitate distinguishing from each other, and are not used to limit the protection scope of the present application. For the specific working process of the units and modules in the foregoing system, reference may be made to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail or recorded in an embodiment, reference may be made to related descriptions of other embodiments.

A person of ordinary skill in the art may realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.

In the embodiments provided in this application, it should be understood that the disclosed device/terminal device and method may be implemented in other ways. For example, the device/terminal device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units. Or components can be combined or integrated into another system, or some features can be omitted or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated module/unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiments and methods, and can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed by the processor, it can implement the steps of the foregoing method embodiments. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electrical carrier signal, telecommunications signal, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately added or deleted according to the requirements of the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to the legislation and patent practice, the computer-readable medium Does not include electrical carrier signals and telecommunication signals.

The above are only optional embodiments of the application, and are not used to limit the application. For those skilled in the art, this application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in the scope of the claims of this application.

Claims

A method for training a fast imaging model, which is characterized in that it includes:

In each iteration of the model training, under-sampling the image scanned by MRI according to the under-sampling mask to obtain training data;

The training data is input into a fast imaging model, the training data is extracted according to the multi-scale information of the image and the attention mechanism through N multi-granularity attention modules, and each of the multi-granularity attention modules is merged to extract Characteristic map of; N≥1;

Perform image reconstruction on the fused feature map and output imaging data;

Reversely calculate gradients according to the imaging data and target tags, so as to update the parameters of the fast imaging model and the under-sampling mask through the gradients;

The updated parameters and the updated under-sampling mask are used for forward calculation to output the next imaging data.
The method for training a fast imaging model according to claim 1, wherein the training data is input to the fast imaging model, and the N multi-granularity attention modules are used to compare the training data according to the multi-scale information of the image and the attention mechanism. Perform feature extraction on the training data, and merge the feature maps extracted by each of the multi-granularity attention modules, including:

Extracting the initialization feature data of the training data;

For each of the multi-granularity attention modules, perform feature extraction on the initialization feature data according to preset image scales, and fuse the extracted feature maps;

Divide the fused feature map into several regional images with different attention weights through the multi-granularity attention mechanism;

Fusion of all the regional images to obtain a feature map after feature refinement.
The method for training a fast imaging model according to claim 2, wherein the gradient is calculated inversely according to the imaging data and the target label, so as to update the parameters of the fast imaging model and the under-developed image through the gradient. Sampling mask, including:

Calculate gradients backwards according to the imaging data and the target tag to obtain a gradient matrix;

According to the gradient matrix, the attention weight given to the several regional images by the multi-granularity attention mechanism is updated.
The method for training a fast imaging model according to claim 3, wherein the gradient is calculated inversely according to the imaging data and the target label, so as to update the parameters of the fast imaging model and the under-developed image through the gradient. The sampling mask also includes:

Generating a continuous mask according to the under-sampling mask, and adding the continuous mask and the gradient matrix to obtain an updated continuous mask;

Binarize the updated continuous mask to obtain an updated under-sampling mask.
The method for training a fast imaging model according to claim 4, wherein the fast imaging model comprises learning a convolutional layer of the under-sampling mask, and correspondingly setting the convolutional layer according to the under-sampling mask The convolution kernel and parameters; the initial value of the under-sampling mask includes a preset number of low-frequency sampling strips and randomly sampled high-frequency sampling strips.
The method for training a fast imaging model according to claim 4, wherein the rule for binarizing the updated continuous mask is: when any of the updated continuous masks When an element is greater than the preset threshold, set to 1; when any element in the updated continuous mask is less than the preset threshold, set to 0; wherein, the preset threshold is the updated The preset percentage of the maximum value in the continuous mask; the preset percentage is set according to the imaging acceleration multiple.
The method for training a fast imaging model according to any one of claims 1 to 6, wherein the under-sampling of the image scanned by magnetic resonance according to the under-sampling mask to obtain training data comprises:

Performing under-sampling on the image scanned by magnetic resonance according to the under-sampling mask to obtain under-sampled K-space data;

Performing an inverse Fourier transform on the under-sampled K-space data to obtain under-sampled image domain data as the training data.
The method for training a fast imaging model according to claim 7, wherein the inverse Fourier transform is performed on the image scanned by magnetic resonance to obtain fully sampled image domain data, which is used as the target label.
A training device for a fast imaging model, which is characterized in that it comprises:

The training data generation module is used for under-sampling the image scanned by MRI according to the under-sampling mask during each iteration of the model training to obtain training data;

The model training module is used to input the training data into the fast imaging model for model training to obtain imaging data;

The feature extraction module is used to input the training data into the fast imaging model, and perform feature extraction on the training data according to the multi-scale information and attention mechanism of the image through N multi-granularity attention modules, and fuse each of the multiple The feature map extracted by the granular attention module; N≥1;

The image fusion module is used to reconstruct the image of the fused feature map and output the imaging data;

The forward calculation module is used for forward calculation using the updated parameters and the updated under-sampling mask to output the next imaging data.
The training device for a fast imaging model according to claim 9, wherein the feature extraction module comprises:

The feature extraction unit is used to extract the initialization feature data of the training data.
The training device for a fast imaging model according to claim 10, wherein each of the multi-granularity attention modules comprises:

The multi-scale densely connected feature fusion unit is used to perform feature extraction on the initial feature data according to several preset image scales, and fuse several extracted feature maps;

The feature refinement unit based on the multi-granularity attention mechanism is used to segment the fused feature map into several regional images with different attention weights through the multi-granularity attention mechanism;

The fusion image unit is used to fuse all the region images to obtain a feature map after feature refinement.
The fast imaging model training device according to claim 11, wherein the parameter and under-sampling mask update module comprises:

A gradient calculation unit, configured to reversely calculate a gradient according to the imaging data and the target tag to obtain a gradient matrix;

The model parameter update unit is configured to update the attention weight given to the plurality of regional images by the multi-granularity attention mechanism according to the gradient matrix.
The fast imaging model training device according to claim 12, wherein the parameter and under-sampling mask update module further comprises:

An under-sampling mask updating unit, configured to generate a continuous mask according to the under-sampling mask, and add the continuous mask and the gradient matrix to obtain an updated continuous mask;

The mask binarization unit is used to binarize the updated continuous mask to obtain an updated under-sampling mask.
A server, characterized in that it includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to achieve The steps of the training method of the fast imaging model described in any one of 1 to 8 are required.