WO2022206021A1 - Procédé et appareil de génération de modèle de reconstruction d'image, procédé et appareil de reconstruction d'image, dispositif et support - Google Patents

Procédé et appareil de génération de modèle de reconstruction d'image, procédé et appareil de reconstruction d'image, dispositif et support Download PDF

Info

Publication number
WO2022206021A1
WO2022206021A1 PCT/CN2021/137623 CN2021137623W WO2022206021A1 WO 2022206021 A1 WO2022206021 A1 WO 2022206021A1 CN 2021137623 W CN2021137623 W CN 2021137623W WO 2022206021 A1 WO2022206021 A1 WO 2022206021A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
resolution
magnetic resonance
reconstruction
preset
Prior art date
Application number
PCT/CN2021/137623
Other languages
English (en)
Chinese (zh)
Inventor
郭木子
郑海荣
朱燕杰
梁栋
刘新
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Publication of WO2022206021A1 publication Critical patent/WO2022206021A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/003—Reconstruction from projections, e.g. tomography
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/41—Medical

Definitions

  • Embodiments of the present invention relate to the technical field of medical image processing, and in particular, to an image reconstruction model generation and image reconstruction method, apparatus, device, and medium.
  • Magnetic Resonance Imaging (MRI) technology is widely used in clinical diagnosis and medical research due to its non-invasive, non-radiation, good soft tissue contrast and imaging at any level.
  • cardiac magnetic resonance cine has been regarded as the imaging gold standard for assessing cardiac function.
  • high-resolution image acquisition is performed clinically, a long acquisition time is often required.
  • it is often difficult to obtain high-resolution cardiac magnetic resonance images in the actual clinical process due to the influence of patient tolerance and respiratory movement.
  • the methods for obtaining high-resolution magnetic resonance cardiac cine images mainly include image reconstruction methods based on interpolation, image reconstruction methods based on traditional machine learning, and image reconstruction methods based on deep learning.
  • the method based on the deep learning network has good processing ability for linear and nonlinear methods, and can reconstruct the magnetic resonance image with higher image quality.
  • the embodiments of the present invention provide an image reconstruction model generation and image reconstruction method, device, device and medium, so as to shorten the imaging time and reduce the complexity of the image reconstruction network, and at the same time improve the reconstructed image resolution. better image effects.
  • an embodiment of the present invention provides a method for generating an image reconstruction model, the method comprising:
  • the image that has undergone image interpolation processing is used as input data for model training, the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data, and the high-resolution image reconstruction model is trained.
  • the loss function of the model converges to a preset value, a target high-resolution image reconstruction model is generated.
  • a preliminary magnetic resonance reconstruction image is obtained, and the resolution of the preliminary magnetic resonance reconstruction image is increased by a preset ratio to reduce the resolution of the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image, including:
  • a discriminant image matching the constructed low-resolution image is extracted from the preliminary magnetic resonance reconstruction image, and the discriminant image and the constructed low-resolution image are input to the discriminator of the preset generative adversarial network , to train the preset generative adversarial network;
  • the generator of the preset generative adversarial network includes six convolution layers, and the convolution step size of the last layer of the six convolution layers is a multiple of the preset image resolution; the The discriminator of the preset Generative Adversarial Network consists of seven convolutional layers.
  • performing image interpolation processing on the low-resolution image includes:
  • the method before training the high-resolution image reconstruction model, the method further includes:
  • the image after image interpolation processing and the preliminary magnetic resonance reconstruction image are rotated or mirrored synchronously, and the image pair obtained after the rotation or mirror operation is used as the training sample data of the new high-resolution image reconstruction model.
  • a residual learning method is used to add the residual of the input data after passing through the convolution layer to the input data itself, and then calculate the sum of the obtained data.
  • the loss function between the described label data is used to add the residual of the input data after passing through the convolution layer to the input data itself, and then calculate the sum of the obtained data.
  • an embodiment of the present invention further provides an image reconstruction method, the method comprising:
  • the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor , to obtain the target MRI reconstructed image.
  • an embodiment of the present invention further provides a device for generating an image reconstruction model, the device comprising:
  • an image degradation module configured to obtain a preliminary magnetic resonance reconstruction image, and increase the resolution of the preliminary magnetic resonance image by a multiple according to a preset image resolution, reduce the resolution of the preliminary magnetic resonance reconstruction image, and obtain a corresponding low-resolution image
  • an image interpolation module configured to perform image interpolation processing on the low-resolution image, wherein the multiple of image interpolation is the preset image resolution enhancement multiple;
  • the model training module is used to use the image that has undergone image interpolation processing as input data for model training, use the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as label data, and train the high-resolution image reconstruction model.
  • the loss function of the high-resolution image reconstruction model converges to a preset value, the target high-resolution image reconstruction model is generated.
  • the image degradation module is specifically used for:
  • a discriminant image matching the constructed low-resolution image is extracted from the preliminary magnetic resonance reconstruction image, and the discriminant image and the constructed low-resolution image are input to the discriminator of the preset generative adversarial network , to train the preset generative adversarial network;
  • the generator of the preset generative adversarial network includes six convolution layers, and the convolution step size of the last layer of the six convolution layers is a multiple of the preset image resolution; the The discriminator of the preset Generative Adversarial Network consists of seven convolutional layers.
  • the image interpolation module is specifically used for:
  • the image reconstruction model generation device further includes a training sample enhancement module, which is used for, before training the high-resolution image reconstruction model, the image that has undergone image interpolation processing and the preliminary magnetic resonance reconstruction image, The rotation or mirroring operation is performed synchronously, and the image pair obtained after the rotation or mirroring operation is used as the training sample data for the new high-resolution image reconstruction model.
  • a training sample enhancement module which is used for, before training the high-resolution image reconstruction model, the image that has undergone image interpolation processing and the preliminary magnetic resonance reconstruction image, The rotation or mirroring operation is performed synchronously, and the image pair obtained after the rotation or mirroring operation is used as the training sample data for the new high-resolution image reconstruction model.
  • the model training module is further configured to, in the training process of the high-resolution image reconstruction model, adopt a residual learning method to combine the residual of the input data after passing through the convolution layer with the input. After the data itself is added, a loss function between the data and the label data is calculated.
  • an embodiment of the present invention further provides an image reconstruction device, the device comprising:
  • an image preprocessing module configured to obtain a preliminary magnetic resonance reconstruction image, and perform image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution increase multiple;
  • the image reconstruction module is used for inputting the preliminary magnetic resonance reconstruction image subjected to image interpolation processing to the preset resolution enhancement factor obtained by the image reconstruction model generation method described in any one of the embodiments.
  • the target image reconstruction model of the target magnetic resonance image reconstruction image is obtained.
  • an embodiment of the present invention further provides a computer device, the computer device comprising:
  • processors one or more processors
  • memory for storing one or more programs
  • the one or more processors When the one or more programs are executed by the one or more processors, the one or more processors implement the image reconstruction model generation method or the image reconstruction method provided by any embodiment of the present invention.
  • an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the image reconstruction model generation method or image provided by any embodiment of the present invention rebuild method.
  • a preliminary magnetic resonance reconstruction image is acquired, and the resolution of the preliminary magnetic resonance reconstruction image is increased by a multiple according to a preset image resolution to reduce the resolution of the preliminary magnetic resonance reconstruction image, so as to obtain a corresponding low-resolution image;
  • Perform image interpolation processing wherein the multiple of image interpolation is the preset image resolution enhancement multiple; the image that has undergone image interpolation processing is used as input data for model training, and the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used.
  • a high-resolution image reconstruction model is trained, and when the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated.
  • the problem of data dependence on the training of the magnetic resonance image reconstruction model in the prior art is solved, and the training data and label data are obtained by degrading the currently obtained magnetic resonance image itself, which can complete the network training of the image reconstruction model, and realizes the need for
  • a large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data is additionally collected to train the neural network, which can use the internal image to learn without a large amount of paired image data. It is an unsupervised learning method with fast imaging speed. At the same time, the hidden dangers such as poor learning effect caused by data deviation are eliminated.
  • FIG. 1 is a flowchart of a method for generating an image reconstruction model according to Embodiment 1 of the present invention
  • FIG. 2 is a schematic structural diagram of a generative adversarial network according to Embodiment 1 of the present invention.
  • FIG. 3 is a schematic diagram of a network structure of a discriminator in a generative adversarial network provided by Embodiment 1 of the present invention.
  • FIG. 4 is a schematic diagram of a generator network structure in a generative adversarial network according to Embodiment 1 of the present invention.
  • FIG. 5 is a schematic diagram of a network training process of an image reconstruction model according to Embodiment 1 of the present invention.
  • FIG. 6 is a flowchart of an image reconstruction method according to Embodiment 2 of the present invention.
  • FIG. 7 is a schematic diagram of an image reconstruction process according to Embodiment 2 of the present invention.
  • FIG. 9 is a schematic structural diagram of an apparatus for generating an image reconstruction model according to Embodiment 3 of the present invention.
  • FIG. 10 is a schematic structural diagram of an image reconstruction apparatus according to Embodiment 4 of the present invention.
  • FIG. 11 is a schematic structural diagram of a computer device according to Embodiment 5 of the present invention.
  • FIG. 1 is a flowchart of a method for generating an image reconstruction model according to Embodiment 1 of the present invention. This embodiment is applicable to the case of using low-resolution magnetic resonance images themselves to perform image reconstruction model training.
  • the method may be executed by an image reconstruction model generating apparatus, which may be implemented in software and/or hardware, and integrated into an electronic device with an application development function.
  • the image reconstruction model generation method includes the following steps:
  • the preliminary magnetic resonance reconstruction image is a magnetic resonance image that can be scanned and obtained by a current magnetic resonance imaging device, that is, an image with relatively low resolution and has not been reconstructed with improved resolution (super-resolution reconstruction).
  • the preset image resolution enhancement multiple is a multiple that is expected to be able to improve the resolution of the preliminary magnetic resonance reconstruction image. Exemplarily, if the resolution of the preliminary magnetic resonance reconstruction image is 128*128, it is hoped that the preliminary magnetic resonance reconstruction image can be reconstructed to obtain a high-resolution image with a resolution of 512*512; then, the preset image resolution
  • the enhancement factor can be obtained by dividing (512*512) by (128*128).
  • the preset image resolution enhancement factor is 16.
  • the resolution of the preliminary magnetic resonance reconstructed image is reduced according to the preset image resolution increase multiple to obtain a corresponding low-resolution image, and the purpose is to use the preliminary magnetic resonance reconstruction image itself as the training parameter of the image reconstruction network, Instead of matching the initial MR reconstruction image with the corresponding high-resolution high-quality MR reconstruction image. Therefore, there is no need to collect a large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data to train the neural network, which can reduce the difficulty of obtaining training samples for image reconstruction models.
  • the way to reduce the image resolution can be a down-sampling method to perform down-sampling processing on the preliminary magnetic resonance reconstructed image to obtain a corresponding low-resolution image.
  • a generative adversarial network can also be used to degrade the preliminary magnetic resonance image reconstruction image to obtain a low-resolution image whose resolution is reduced by a preset image resolution enhancement factor.
  • a kernel function can be determined for performing the same image degradation process on one or more preliminary MRI reconstruction images.
  • the preliminary magnetic resonance reconstruction image is input into the preset generative adversarial network as shown in FIG. 2 .
  • the generator G performs convolution and down-sampling processing on the preliminary magnetic resonance reconstruction image, and obtains a low-resolution image whose resolution is reduced by the preset image resolution enhancement multiple.
  • the obtained image can be regarded as the original magnetic resonance reconstruction image.
  • Fake version image F further, cut out image blocks with the same size and matching position as image F from the preliminary magnetic resonance reconstruction image, as real sample image T.
  • the discriminator analyzes the possibility that the image F is a real image pixel by pixel.
  • the goal of the generator is to make the output image F fool the discriminator as much as possible.
  • the two networks, the generator and the discriminator constantly adjust the parameters in the process of confrontation with each other. Finally, when the discriminator cannot judge whether the output result F of the generator is real, the network training is completed.
  • the image F and the image T are compared pixel by pixel, and every two pixels compared with each other are the probability value of the same pixel point, when the probability value of the year in the heat map is satisfied.
  • the probability value of the judgment is affirmative, it can be determined that the discriminator cannot judge whether the output result F of the generator is true.
  • all convolutional layer parameters of the generator in the trained preset generative adversarial network are convolved layer by layer to obtain the magnetic resonance image degradation kernel function; thus, each preliminary magnetic resonance reconstruction image can be respectively combined with the magnetic resonance image.
  • the image degradation kernel function performs convolution to obtain the corresponding low-resolution image.
  • the size and number of convolution kernels in the generator in the preset generative adversarial network can be changed and adjusted.
  • the generator includes six convolution kernels, and the size of each convolution kernel is: 5x5, 3x3, 1x1, 1x1, 1x1 and 1x1.
  • the structure of the discriminator in the preset Generative Adversarial Network may be the structure shown in FIG. 3 , which consists of 7 convolutional layers, and the convolution kernel sizes of each convolutional layer are: 7 ⁇ 7 and 1 ⁇ 1 respectively. , 1x1, 1x1, 1x1, 1x1 and 1x1.
  • the network structure of the generator is shown in Figure 4, which consists of 6 convolution layers, and the convolution kernel sizes are: 7x7, 5x5, 3x3, 1x1, 1x1, 1x1.
  • the value of the convolution step size of the last convolutional layer of the generator is the preset image resolution enhancement multiple, so as to achieve the effect of downsampling the input image.
  • the acquired preliminary magnetic resonance reconstruction image is a cardiac magnetic resonance cine image, which includes multiple frames of magnetic resonance images.
  • any frame of the cardiac magnetic resonance cine image can be taken as the input image.
  • image interpolation processing is performed on the low-resolution image, and the multiple of image interpolation is the preset image resolution enhancement multiple, so as to keep the size of the input image of the image reconstruction model consistent with the size of the output image, which can be shortened.
  • Model training time optimizing the model training process.
  • the interpolation method may adopt a bicubic interpolation (Bicubic) algorithm. This is because the bicubic interpolation can preserve more image details during the image enlargement process, and the enlarged image has the function of anti-aliasing. At the same time, the enlarged image has a more realistic effect than the source image.
  • Biubic bicubic interpolation
  • the image that has undergone image interpolation processing and the corresponding preliminary magnetic resonance reconstruction image can also be rotated or mirrored synchronously, and the image pair obtained after the rotation or mirroring operation can be used as a new high-resolution image.
  • Rate image reconstruction model training sample data For example, rotate the image pair at 0, 90, 180, and 270 degrees, and then perform mirror symmetry operations in the horizontal and vertical directions, respectively, to obtain 8 sets of training data, so that the training data can be obtained. enhanced.
  • the phases of the data are consistent, and no registration is required, thereby eliminating hidden dangers such as poor learning effects caused by data deviations.
  • a sufficient amount of model training sample data is constructed, and then the model training process can begin.
  • the image that has undergone image interpolation processing is used as the input data for model training, and the preliminary MRI reconstruction image corresponding to the low-resolution image is used as the label data.
  • the to-be-obtained preliminary magnetic resonance image is used as the label data, and the low-resolution image whose resolution is reduced by the preset image resolution enhancement multiple.
  • a high-resolution image that increases the resolution of the preliminary magnetic resonance reconstruction image by a preset image resolution can be obtained from the output of the image reconstruction model.
  • the image reconstruction network is a fully convolutional network with a total of 8 convolutional layers.
  • the convolution kernel size of the first layer is 3x3, the number of channels is f, and the second to seventh layers are
  • the size of the convolution kernel is 3x3, the number of channels is 64, and the size of the convolution kernel of the last layer is 3x3, and the number of channels is f.
  • f represents the number of image frames simultaneously input to the image reconstruction network.
  • the f value is 1.
  • the value of f is the number of image frames in the cardiac cine image.
  • the number of convolutional layers of the image reconstruction network can also be changed to 6 or more, and the size of the convolution kernel is not limited to 3x3, and the parameters can be adjusted according to the calculation requirements.
  • the model training process in Fig. 5 adopts a preferred deep learning method, that is, the residual learning method, which compares the residual of the image data of the input model after passing through the convolution layer with the input image data itself. After adding, calculate the loss function between and the label data.
  • the loss function may not only use the L1 loss function shown in FIG. 5 , but also use loss functions such as L2 loss and perceptual loss.
  • optimization algorithms such as Adam optimization algorithm, stochastic gradient descent algorithm or AdaGrad can also be used to optimize the network learning process.
  • conv represents the convolution kernel in the convolutional neural network
  • “.mat” is the abbreviation of the file format. All the parameters of the convolutional layers of the device are convolved layer by layer to obtain the degraded kernel function of the magnetic resonance image.
  • a corresponding low-resolution image is obtained by reducing the image resolution of the obtained preliminary magnetic resonance reconstruction image, and the resolution reduction factor is controllable; then, image interpolation processing is performed on the low-resolution image , where the multiple of image interpolation is the preset image resolution enhancement multiple; the image that has undergone image interpolation processing is used as input data for model training, and the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data,
  • the high-resolution image reconstruction model is trained, and when the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated.
  • the problem of data dependence on the training of the magnetic resonance image reconstruction model in the prior art is solved, and the training data and label data are obtained by degrading the currently obtained magnetic resonance image itself, which can complete the network training of the image reconstruction model, and realizes the need for
  • a large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data is additionally collected to train the neural network, which can use the internal image to learn without a large amount of paired image data. It is an unsupervised learning method with fast imaging speed. At the same time, the hidden dangers such as poor learning effect caused by data deviation are eliminated.
  • Fig. 6 is a flowchart of an image reconstruction method provided in Embodiment 2 of the present invention, and this embodiment can be applied to the situation of reconstructing a collected low-resolution medical image to obtain a high-resolution image.
  • the method may be performed by an image reconstruction apparatus, and the apparatus may be implemented in software and/or hardware, and integrated into a computer device with an application development function.
  • the image reconstruction method includes the following steps:
  • the preliminary magnetic resonance reconstruction image is a low-resolution image whose resolution needs to be improved. According to the resolution improvement requirement, the preliminary magnetic resonance reconstruction image can be interpolated and reconstructed to obtain a preprocessed image.
  • the multiple of image interpolation is the preset resolution enhancement multiple.
  • the target image reconstruction model is an image reconstruction model trained by the image reconstruction model generation method in the above-mentioned embodiment according to the requirement of increasing the preset resolution. Specifically, for the process of image reconstruction, reference may be made to the schematic diagram shown in FIG. 7 .
  • Fig. 8 shows the reconstructed image obtained by performing image reconstruction on the same preliminary magnetic resonance reconstructed image by different graphic reconstruction methods, wherein (a) NN is the result obtained by the nearest neighbor interpolation algorithm, (b) Bicubic is the double The result obtained by the cubic interpolation algorithm, (c) zero-padding is the result of transforming the image back to the image domain after the image is transformed to the Fourier domain, and then inversely transforming back to the image domain after zero-filling the surrounding area, (d) SR (Super Resolution) is the image of this example The results obtained by the reconstruction method. Intuitively, from the results, the image reconstruction results obtained by the image reconstruction method in this embodiment are clearer.
  • FIG. 9 is a schematic structural diagram of an image reconstruction model generating apparatus according to Embodiment 3 of the present invention. This embodiment is applicable to the case of using low-resolution magnetic resonance images themselves to perform image reconstruction model training.
  • the image reconstruction model generating apparatus includes an image degradation module 310 , an image interpolation module 320 and a model training module 330 .
  • the image degradation module 310 is used to obtain a preliminary magnetic resonance reconstruction image, and increase the resolution of the preliminary magnetic resonance image by a preset image resolution to reduce the resolution of the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image; the image interpolation module 320, for performing image interpolation processing on the low-resolution image, wherein the multiple of image interpolation is the preset image resolution enhancement multiple; model training module 330, for using the image subjected to image interpolation processing as model training
  • the input data of the high-resolution image reconstruction model is trained by using the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as the label data. When the loss function of the high-resolution image reconstruction model converges to the preset value , to generate the target high-resolution image reconstruction model.
  • the technical solution of this embodiment by acquiring a preliminary magnetic resonance reconstruction image, and increasing the resolution of the preliminary magnetic resonance image by a multiple, the resolution of the preliminary magnetic resonance reconstruction image is reduced, and a corresponding low-resolution image is obtained; Perform image interpolation processing on the low-resolution images, wherein the multiple of image interpolation is the preset image resolution improvement multiple; the image after image interpolation processing is used as the input data for model training, and the preliminary magnetic resonance imaging corresponding to the low-resolution image is used.
  • the reconstructed image is used as label data to train a high-resolution image reconstruction model, and when the loss function of the high-resolution image reconstruction model converges to a preset value, a target high-resolution image reconstruction model is generated.
  • the problem of data dependence on the training of the magnetic resonance image reconstruction model in the prior art is solved, and the training data and label data are obtained by degrading the currently obtained magnetic resonance image itself, which can complete the network training of the image reconstruction model, and realizes the need for
  • a large amount of paired low-resolution-high-resolution magnetic resonance parameter quantitative image data is additionally collected to train the neural network, which can use the internal image to learn without a large amount of paired image data. It is an unsupervised learning method with fast imaging speed. At the same time, the hidden dangers such as poor learning effect caused by data deviation are eliminated.
  • the image degradation module 310 is specifically used for:
  • a discriminant image matching the constructed low-resolution image is extracted from the preliminary magnetic resonance reconstruction image, and the discriminant image and the constructed low-resolution image are input to the discriminator of the preset generative adversarial network , to train the preset generative adversarial network;
  • the generator of the preset generative adversarial network includes six convolution layers, and the convolution step size of the last layer of the six convolution layers is a multiple of the preset image resolution; the The discriminator of the preset Generative Adversarial Network consists of seven convolutional layers.
  • the image interpolation module 320 is specifically used for:
  • the image reconstruction model generation device further includes a training sample enhancement module, which is used for, before training the high-resolution image reconstruction model, the image that has undergone image interpolation processing and the preliminary magnetic resonance reconstruction image, The rotation or mirroring operation is performed synchronously, and the image pair obtained after the rotation or mirroring operation is used as the training sample data for the new high-resolution image reconstruction model.
  • a training sample enhancement module which is used for, before training the high-resolution image reconstruction model, the image that has undergone image interpolation processing and the preliminary magnetic resonance reconstruction image, The rotation or mirroring operation is performed synchronously, and the image pair obtained after the rotation or mirroring operation is used as the training sample data for the new high-resolution image reconstruction model.
  • the model training module 330 is further configured to, in the training process of the high-resolution image reconstruction model, adopt a residual learning method to compare the residual of the input data after passing through the convolutional layer with the residual. After the input data itself is added, a loss function between the input data and the label data is calculated.
  • the image reconstruction model generation apparatus provided by the embodiment of the present invention can execute the image reconstruction model generation method provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.
  • FIG. 10 is a schematic structural diagram of an image reconstruction apparatus according to Embodiment 4 of the present invention. This embodiment can be applied to a situation in which a low-resolution medical image obtained by acquisition is reconstructed to obtain a high-resolution image.
  • the image reconstruction apparatus includes an image preprocessing module 410 and an image reconstruction module 420 .
  • the image preprocessing module 410 is used to obtain a preliminary magnetic resonance reconstruction image, and perform image interpolation processing on the preliminary magnetic resonance reconstruction image with a preset resolution increase multiple; the image reconstruction module 420 is used for image interpolation processing.
  • the preliminary magnetic resonance reconstruction image is input into the target image reconstruction model obtained by the image reconstruction model generation method described in any one of the embodiments, and the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor, and the target magnetic resonance image is obtained. Rebuild the image.
  • the image reconstruction apparatus provided by the embodiment of the present invention can execute the image reconstruction method provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.
  • FIG. 11 is a schematic structural diagram of a computer device according to Embodiment 5 of the present invention.
  • Figure 11 shows a block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention.
  • the computer device 12 shown in FIG. 11 is only an example, and should not impose any limitation on the function and scope of use of the embodiments of the present invention.
  • the computer device 12 may be any terminal device with computing capability connected to the magnetic resonance scanning imaging device, such as an intelligent controller, a server, a mobile phone and other terminal devices.
  • computer device 12 takes the form of a general-purpose computing device.
  • Components of computer device 12 may include, but are not limited to, one or more processors or processing units 16 , system memory 28 , and a bus 18 connecting various system components including system memory 28 and processing unit 16 .
  • Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of a variety of bus structures.
  • these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect ( PCI) bus.
  • Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including both volatile and nonvolatile media, removable and non-removable media.
  • System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and/or cache memory 32 .
  • Computer device 12 may further include other removable/non-removable, volatile/non-volatile computer system storage media.
  • storage system 34 may be used to read and write to non-removable, non-volatile magnetic media (not shown in FIG. 11, commonly referred to as a "hard drive”).
  • a disk drive for reading and writing to removable non-volatile magnetic disks (eg "floppy disks") and removable non-volatile optical disks (eg CD-ROM, DVD-ROM) may be provided or other optical media) to read and write optical drives.
  • each drive may be connected to bus 18 through one or more data media interfaces.
  • System memory 28 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of various embodiments of the present invention.
  • a program/utility 40 having a set (at least one) of program modules 42, which may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and programs Data, each or some combination of these examples may include an implementation of a network environment.
  • Program modules 42 generally perform the functions and/or methods of the described embodiments of the present invention.
  • Computer device 12 may also communicate with one or more external devices 14 (eg, keyboard, pointing device, display 24, etc.), may also communicate with one or more devices that enable a user to interact with computer device 12, and/or communicate with Any device (eg, network card, modem, etc.) that enables the computer device 12 to communicate with one or more other computing devices. Such communication may take place through input/output (I/O) interface 22 . Also, computer device 12 may communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet, through network adapters 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18 . It should be understood that, although not shown in FIG. 11, other hardware and/or software modules may be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tapes drives and data backup storage systems, etc.
  • the processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, for example, to realize the steps of an image reconstruction model generation method provided by the embodiment of the present invention, and the method includes:
  • the image that has undergone image interpolation processing is used as input data for model training, the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data, and the high-resolution image reconstruction model is trained.
  • the loss function of the model converges to a preset value, a target high-resolution image reconstruction model is generated.
  • the steps of an image reconstruction method provided by the embodiment of the present invention can also be implemented, and the method includes:
  • the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor , to obtain the target MRI reconstructed image.
  • the sixth embodiment provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the image reconstruction model generation method provided by any embodiment of the present invention, including:
  • the image that has undergone image interpolation processing is used as input data for model training, the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image is used as label data, and the high-resolution image reconstruction model is trained.
  • the loss function of the model converges to a preset value, a target high-resolution image reconstruction model is generated.
  • the steps of an image reconstruction method provided by the embodiment of the present invention can also be implemented, and the method includes:
  • the resolution of the magnetic resonance image is increased by the preset resolution enhancement factor , to obtain the target MRI reconstructed image.
  • the computer storage medium in the embodiments of the present invention may adopt any combination of one or more computer-readable mediums.
  • the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination of the above.
  • a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a propagated data signal in baseband or as part of a carrier wave, with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device .
  • Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
  • Computer program code for carrying out operations of the present invention may be written in one or more programming languages, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional A procedural programming language, such as the "C" language or similar programming language.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server.
  • the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or wide area network (WAN), or may be connected to an external computer (eg, through the Internet using an Internet service provider) connect).
  • LAN local area network
  • WAN wide area network
  • Internet service provider an external computer
  • modules or steps of the present invention can be implemented by a general-purpose computing device, and they can be centralized on a single computing device, or distributed on a network composed of multiple computing devices.
  • they may be implemented in program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be fabricated separately into individual integrated circuit modules, or a plurality of modules of them Or the steps are made into a single integrated circuit module to realize.
  • the present invention is not limited to any specific combination of hardware and software.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Biophysics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)

Abstract

Sont divulgués dans les modes de réalisation de la présente invention un procédé et un appareil de génération de modèle de reconstruction d'image, un procédé et un appareil de reconstruction d'image, ainsi qu'un dispositif et un support. Le procédé de génération de modèle de reconstruction d'image comprend : l'acquisition d'une image de résonance magnétique reconstruite préliminaire, et la réduction de la résolution de l'image de résonance magnétique reconstruite préliminaire selon un multiple d'augmentation de résolution d'image prédéfinie, de façon à obtenir une image à faible résolution ; la réalisation d'un traitement d'interpolation d'image sur l'image à faible résolution, un multiple pour l'interpolation d'image étant le multiple d'augmentation de résolution d'image prédéfinie ; et l'entraînement d'un modèle de reconstruction d'image à haute résolution en utilisant l'image ayant été soumise au traitement d'interpolation en tant que données d'entrée pour un entraînement de modèle et en utilisant l'image de résonance magnétique reconstruite préliminaire en tant que données d'étiquette, et lorsqu'une fonction de perte du modèle de reconstruction d'image à haute résolution converge vers une valeur prédéfinie, la génération d'un modèle de reconstruction d'image à haute résolution cible. Au moyen de la solution technique du présent mode de réalisation, des problèmes cachés tels qu'un mauvais effet d'apprentissage provoqué par un écart de données sont éliminés, et la résolution d'une image reconstruite peut être augmentée tout en raccourcissant le temps d'imagerie, ce qui permet d'obtenir un meilleur effet d'image.
PCT/CN2021/137623 2021-03-30 2021-12-13 Procédé et appareil de génération de modèle de reconstruction d'image, procédé et appareil de reconstruction d'image, dispositif et support WO2022206021A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110340196.9 2021-03-30
CN202110340196.9A CN115147502A (zh) 2021-03-30 2021-03-30 图像重建模型生成及图像重建方法、装置、设备和介质

Publications (1)

Publication Number Publication Date
WO2022206021A1 true WO2022206021A1 (fr) 2022-10-06

Family

ID=83403739

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/137623 WO2022206021A1 (fr) 2021-03-30 2021-12-13 Procédé et appareil de génération de modèle de reconstruction d'image, procédé et appareil de reconstruction d'image, dispositif et support

Country Status (2)

Country Link
CN (1) CN115147502A (fr)
WO (1) WO2022206021A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107610194A (zh) * 2017-08-14 2018-01-19 成都大学 基于多尺度融合cnn的磁共振图像超分辨率重建方法
CN110599401A (zh) * 2019-08-19 2019-12-20 中国科学院电子学研究所 遥感图像超分辨率重建方法、处理装置及可读存储介质
CN111353935A (zh) * 2020-01-03 2020-06-30 首都医科大学附属北京友谊医院 基于深度学习的磁共振成像优化方法及其设备
CN111583109A (zh) * 2020-04-23 2020-08-25 华南理工大学 基于生成对抗网络的图像超分辨率方法
US20200311926A1 (en) * 2019-03-27 2020-10-01 The General Hospital Corporation Super-resolution anatomical magnetic resonance imaging using deep learning for cerebral cortex segmentation

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107610194A (zh) * 2017-08-14 2018-01-19 成都大学 基于多尺度融合cnn的磁共振图像超分辨率重建方法
US20200311926A1 (en) * 2019-03-27 2020-10-01 The General Hospital Corporation Super-resolution anatomical magnetic resonance imaging using deep learning for cerebral cortex segmentation
CN110599401A (zh) * 2019-08-19 2019-12-20 中国科学院电子学研究所 遥感图像超分辨率重建方法、处理装置及可读存储介质
CN111353935A (zh) * 2020-01-03 2020-06-30 首都医科大学附属北京友谊医院 基于深度学习的磁共振成像优化方法及其设备
CN111583109A (zh) * 2020-04-23 2020-08-25 华南理工大学 基于生成对抗网络的图像超分辨率方法

Also Published As

Publication number Publication date
CN115147502A (zh) 2022-10-04

Similar Documents

Publication Publication Date Title
Hauptmann et al. Real‐time cardiovascular MR with spatio‐temporal artifact suppression using deep learning–proof of concept in congenital heart disease
US10387765B2 (en) Image correction using a deep generative machine-learning model
US11120582B2 (en) Unified dual-domain network for medical image formation, recovery, and analysis
JP7340107B2 (ja) 画像再構成方法、装置、デバイス、システムおよびコンピューター可読記憶媒体
de Leeuw Den Bouter et al. Deep learning-based single image super-resolution for low-field MR brain images
CN113870104A (zh) 超分辨率图像重建
CN115953494B (zh) 基于低剂量和超分辨率的多任务高质量ct图像重建方法
Zhu et al. Residual dense network for medical magnetic resonance images super-resolution
CN110807821A (zh) 一种图像重建方法和系统
WO2021102644A1 (fr) Procédé et appareil d'amélioration d'image, et dispositif terminal
US11967066B2 (en) Method and apparatus for processing image
CN105654425A (zh) 一种应用于医学x光图像的单幅图像超分辨率重建方法
CN114494022B (zh) 模型训练方法、超分辨率重建方法、装置、设备及介质
Yang et al. Generative Adversarial Networks (GAN) Powered Fast Magnetic Resonance Imaging--Mini Review, Comparison and Perspectives
Lu et al. A novel 3D medical image super-resolution method based on densely connected network
Li et al. Multi-scale residual denoising GAN model for producing super-resolution CTA images
CN111243052A (zh) 图像重建方法、装置、计算机设备和存储介质
US20230386067A1 (en) Systems and methods for segmenting 3d images
US20230079353A1 (en) Image correction using an invertable network
CN111047512A (zh) 图像增强方法、装置及终端设备
WO2022193379A1 (fr) Procédé et appareil de génération de modèle de reconstruction d'image, procédé et appareil de reconstruction d'image, dispositif et support
CN104504657B (zh) 磁共振弥散张量去噪方法和装置
CN116630634B (zh) 一种图像处理方法、装置、设备及存储介质
WO2024021796A1 (fr) Procédé et appareil de traitement d'image, dispositif électronique, support de stockage et produit de programme
WO2022206021A1 (fr) Procédé et appareil de génération de modèle de reconstruction d'image, procédé et appareil de reconstruction d'image, dispositif et support

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21934657

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21934657

Country of ref document: EP

Kind code of ref document: A1