CN115147502A

CN115147502A - Image reconstruction model generation and image reconstruction method, device, equipment and medium

Info

Publication number: CN115147502A
Application number: CN202110340196.9A
Authority: CN
Inventors: 郭木子; 郑海荣; 朱燕杰; 梁栋; 刘新
Original assignee: Shenzhen Institute of Advanced Technology of CAS
Current assignee: Shenzhen Institute of Advanced Technology of CAS
Priority date: 2021-03-30
Filing date: 2021-03-30
Publication date: 2022-10-04
Also published as: WO2022206021A1

Abstract

The embodiment of the invention discloses an image reconstruction model generation method, an image reconstruction device, an image reconstruction equipment and a medium, wherein the image reconstruction model generation method comprises the following steps: acquiring a preliminary magnetic resonance reconstruction image, and reducing the resolution of the preliminary magnetic resonance reconstruction image according to a preset image resolution improvement multiple to obtain a low-resolution image; carrying out image interpolation processing on the low-resolution image, wherein the image interpolation multiple is a preset image resolution improvement multiple; and taking the image after the interpolation processing as input data of model training, taking the preliminary magnetic resonance reconstructed image as label data, training the high-resolution image reconstructed model, and generating the target high-resolution image reconstructed model when a loss function of the high-resolution image reconstructed model converges to a preset value. The technical scheme of the embodiment realizes that hidden dangers such as poor learning effect and the like caused by data deviation are eliminated, the imaging time is shortened, meanwhile, the resolution ratio of the reconstructed image can be improved, and a better image effect is achieved.

Description

Image reconstruction model generation method, image reconstruction device, image reconstruction equipment and medium

Technical Field

The embodiment of the invention relates to the technical field of medical image processing, in particular to an image reconstruction model generation method, an image reconstruction model generation device, an image reconstruction method, an image reconstruction device, an image reconstruction equipment and a medium.

Background

Magnetic Resonance Imaging (MRI) technology is widely used in clinical diagnosis and medical research due to its characteristics of being non-invasive, non-radiative, good soft tissue contrast, imaging at any layer, and the like. At present, cardiac magnetic resonance cine has been considered as a standard for imaging for assessment of cardiac function. However, clinically, when high resolution image acquisition is performed, a long acquisition time is often required. Meanwhile, due to the influences of patient tolerance, respiratory motion and the like, in the actual clinical process, high-resolution cardiac magnetic resonance images are difficult to obtain.

At present, methods for obtaining a magnetic resonance cardiac cine image with high resolution mainly include an image reconstruction method based on interpolation, an image reconstruction method based on conventional machine learning, and an image reconstruction method based on deep learning. The method based on the deep learning network has good processing capability for both linear and nonlinear methods, and can reconstruct and obtain a magnetic resonance image with higher image quality.

However, the existing method for performing super-resolution image reconstruction based on deep learning does not solve the problem of data dependency, and a large number of low-resolution images and high-resolution images in a pair are still required to be learned. However, from the perspective of practical operation and patient ethics, as well as the specificity of medical images, paired high-resolution and low-resolution cine images are difficult to obtain. Meanwhile, for film imaging with different resolutions, different field strengths and different phases, standardized registration is required, and the process is often inaccurate due to the influence of machine and geometric deformation. Therefore, the existing super-resolution method based on deep learning is easily influenced by data deviation, and the difficulty of large-scale clinical popularization is increased.

Disclosure of Invention

The embodiment of the invention provides an image reconstruction model generation method, an image reconstruction device and a medium, which are used for shortening the imaging time, reducing the complexity of an image reconstruction network, improving the resolution of a reconstructed image and achieving a better image effect.

In a first aspect, an embodiment of the present invention provides an image reconstruction model generation method, where the method includes:

acquiring a preliminary magnetic resonance reconstruction image, and reducing the resolution of the preliminary magnetic resonance reconstruction image according to a preset image resolution improvement multiple to obtain a corresponding low-resolution image;

performing image interpolation processing on the low-resolution image, wherein the image interpolation multiple is the preset image resolution improvement multiple;

and training a high-resolution image reconstruction model by taking the image subjected to image interpolation as input data of model training and taking a preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as label data, and generating the target high-resolution image reconstruction model when a loss function of the high-resolution image reconstruction model converges to a preset value.

Optionally, obtaining a preliminary magnetic resonance reconstructed image, and reducing the resolution of the preliminary magnetic resonance reconstructed image according to a preset image resolution improvement multiple to obtain a corresponding low-resolution image, including:

inputting the preliminary magnetic resonance reconstruction image into a generator of a preset generation countermeasure network, and performing convolution and down-sampling processing on the preliminary magnetic resonance reconstruction image by the generator to obtain a constructed low-resolution image with resolution reduced by the resolution improvement multiple of the preset image;

extracting a distinguishing image matched with the constructed low-resolution image from the preliminary magnetic resonance reconstruction image, and inputting the distinguishing image and the constructed low-resolution image into a discriminator of the preset generation countermeasure network so as to train the preset generation countermeasure network;

performing layer-by-layer convolution on all convolution layer parameters of a generator in a preset generation countermeasure network after training is completed to obtain a magnetic resonance image degradation kernel function;

and convolving the preliminary magnetic resonance reconstruction image with the magnetic resonance image degradation kernel function to obtain a corresponding low-resolution image.

Preferably, the generator of the preset generation countermeasure network comprises six convolutional layers, and the convolution step length of the last layer of the six convolutional layers is the preset image resolution improvement multiple; the preset generation countermeasure network arbiter comprises seven convolutional layers.

Alternatively to this, the first and second parts may, the image interpolation processing is performed on the low-resolution image, the method comprises the following steps:

and performing double cubic interpolation processing of the preset image resolution improvement multiple on the low-resolution image.

Optionally, before training the high resolution image reconstruction model, the method further includes:

and synchronously performing rotation or mirror image operation on the image subjected to the image interpolation processing and the preliminary magnetic resonance reconstruction image, and taking the image pair obtained after the rotation or mirror image operation as training sample data of a new high-resolution image reconstruction model.

Optionally, in the training process of the high-resolution image reconstruction model, a residual error learning manner is adopted, and after a residual error of the input data after passing through the convolutional layer is added to the input data, a loss function between the input data and the label data is calculated.

In a second aspect, an embodiment of the present invention further provides an image reconstruction method, where the method includes:

acquiring a preliminary magnetic resonance reconstruction image, and performing image interpolation processing of a preset resolution improvement multiple on the preliminary magnetic resonance reconstruction image;

and inputting the preliminary magnetic resonance reconstructed image subjected to image interpolation into a target image reconstructed model obtained by the image reconstructed model generating method according to any embodiment, wherein the target image reconstructed model is obtained by increasing the resolution of the magnetic resonance image by the preset resolution increasing times, so as to obtain a target magnetic resonance reconstructed image.

In a third aspect, an embodiment of the present invention further provides an image reconstruction model generation apparatus, where the apparatus includes:

the image degradation module is used for acquiring a preliminary magnetic resonance reconstruction image, reducing the resolution of the preliminary magnetic resonance reconstruction image according to a preset image resolution improvement multiple, and obtaining a corresponding low-resolution image;

the image interpolation module is used for carrying out image interpolation processing on the low-resolution image, wherein the image interpolation multiple is the preset image resolution improvement multiple;

and the model training module is used for training the high-resolution image reconstruction model by taking the image subjected to image interpolation as input data of model training and taking the preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as label data, and generating the target high-resolution image reconstruction model when a loss function of the high-resolution image reconstruction model converges to a preset value.

Optionally, the image degradation module is specifically configured to:

Optionally, the image interpolation module is specifically configured to:

Optionally, the image reconstruction model generation device further includes a training sample enhancement module, configured to perform rotation or mirror image operation on the image subjected to the image interpolation processing and the preliminary magnetic resonance reconstructed image synchronously before training the high-resolution image reconstruction model, and use an image pair obtained after the rotation or mirror image operation as new training sample data of the high-resolution image reconstruction model.

Optionally, the model training module is further configured to, in a training process of the high-resolution image reconstruction model, calculate a loss function between the input data and the label data after adding a residual error of the input data after passing through the convolutional layer to the input data by using a residual error learning method.

In a fourth aspect, an embodiment of the present invention further provides an image reconstruction apparatus, where the apparatus includes:

the image preprocessing module is used for acquiring a preliminary magnetic resonance reconstruction image and performing image interpolation processing of preset resolution improvement times on the preliminary magnetic resonance reconstruction image;

and the image reconstruction module is used for inputting the preliminary magnetic resonance reconstruction image subjected to image interpolation processing into a target image reconstruction model which is obtained by the image reconstruction model generation method according to any embodiment and is used for increasing the resolution of the magnetic resonance image by the preset resolution increasing multiple to obtain a target magnetic resonance reconstruction image.

In a fifth aspect, an embodiment of the present invention further provides a computer device, where the computer device includes:

one or more processors;

a memory for storing one or more programs;

when executed by the one or more processors, cause the one or more processors to implement an image reconstruction model generation method or an image reconstruction method as provided by any of the embodiments of the present invention.

In a sixth aspect, embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, where the program is executed by a processor to implement the image reconstruction model generation method or the image reconstruction method provided in any embodiment of the present invention.

The embodiment of the invention has the following advantages or beneficial effects:

in the embodiment of the invention, by acquiring the preliminary magnetic resonance reconstruction image and increasing the multiple according to the preset image resolution, reducing the resolution of the preliminary magnetic resonance reconstruction image to obtain a corresponding low-resolution image; performing image interpolation processing on the low-resolution image, wherein the image interpolation multiple is the preset image resolution improvement multiple; and training a high-resolution image reconstruction model by taking the image subjected to image interpolation as input data of model training and taking a preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as label data, and generating the target high-resolution image reconstruction model when a loss function of the high-resolution image reconstruction model converges to a preset value. The problem of dependence of training on data of a magnetic resonance image reconstruction model in the prior art is solved, the training data and the label data are obtained by degrading the currently acquired magnetic resonance image, network training of the image reconstruction model can be completed, the neural network can be trained without additionally collecting a large amount of low-resolution-high-resolution magnetic resonance parameter quantitative image data which are well paired, learning can be performed by using the inside of the image, and a large amount of image data are not required to be paired.

Drawings

Fig. 1 is a flowchart of an image reconstruction model generation method according to an embodiment of the present invention;

fig. 2 is a schematic diagram of a structure of a generative confrontation network according to an embodiment of the present invention;

fig. 3 is a schematic network structure diagram of an arbiter in a generation countermeasure network according to an embodiment of the present invention;

fig. 4 is a schematic diagram of a network structure of a generator in a generative countermeasure network according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a training process of an image reconstruction model network according to an embodiment of the present invention;

fig. 6 is a flowchart of an image reconstruction method according to a second embodiment of the present invention;

FIG. 7 is a schematic diagram of an image reconstruction process according to a second embodiment of the present invention;

fig. 8 is a comparison graph of magnetic resonance image reconstruction effects performed by different image reconstruction methods according to a second embodiment of the present invention;

fig. 9 is a schematic structural diagram of an image reconstruction model generation apparatus according to a third embodiment of the present invention;

fig. 10 is a schematic structural diagram of an image reconstruction apparatus according to a fourth embodiment of the present invention;

fig. 11 is a schematic structural diagram of a computer device according to a fifth embodiment of the present invention.

Detailed Description

The present invention will be described in further detail with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not to be construed as limiting the invention. It should be further noted that, for the convenience of description, only some structures related to the present invention are shown in the drawings, not all of them.

Example one

Fig. 1 is a flowchart of an image reconstruction model generation method according to an embodiment of the present invention, the embodiment is applicable to the case of performing image reconstruction model training by using the low-resolution magnetic resonance image itself. The method can be executed by an image reconstruction model generation device, which can be implemented by software and/or hardware, and is integrated in an electronic device with application development function.

As shown in fig. 1, the image reconstruction model generation method includes the following steps:

and S110, acquiring a preliminary magnetic resonance reconstruction image, and reducing the resolution of the preliminary magnetic resonance reconstruction image according to a preset image resolution improvement multiple to obtain a corresponding low-resolution image.

The preliminary magnetic resonance reconstructed image is a magnetic resonance image which can be obtained by scanning with the existing magnetic resonance imaging equipment, that is, an image which has a relatively low resolution and has not been subjected to resolution-improved reconstruction (super-resolution reconstruction). The preset image resolution improvement factor is a factor which is expected to improve the resolution of the preliminary magnetic resonance reconstruction image. For example, if the resolution of the preliminary magnetic resonance reconstructed image is 128 × 128, it is desirable to reconstruct the preliminary magnetic resonance reconstructed image to obtain a high resolution image with a resolution of 512 × 512; then, the predetermined image resolution improvement factor is obtained by dividing (512 × 512) by (128 × 128), and the predetermined image resolution improvement factor is 16.

In this embodiment, the resolution of the preliminary magnetic resonance reconstructed image is reduced according to the preset image resolution improvement multiple to obtain a corresponding low-resolution image, so that the preliminary magnetic resonance reconstructed image itself is used as a training parameter of the image reconstruction network, rather than the preliminary magnetic resonance reconstructed image is matched with a corresponding high-resolution high-quality magnetic resonance reconstructed image. Therefore, a large amount of matched low-resolution-high-resolution magnetic resonance parameter quantitative image data does not need to be collected to train the neural network, and the difficulty in obtaining the training sample of the image reconstruction model can be reduced.

Specifically, the manner of reducing the image resolution may be a down-sampling method, and the down-sampling processing is performed on the preliminary magnetic resonance reconstructed image to obtain a corresponding low-resolution image.

In a preferred embodiment, the generation countermeasure network may be further utilized to perform a degradation process on the preliminary magnetic resonance reconstructed image, so as to obtain a low-resolution image with a resolution reduced by a preset image resolution improvement multiple. During the training of the generation of the countermeasure network, a kernel function may be determined for performing the same image degradation process on one or more preliminary mr reconstruction images. Specifically, first, a preliminary magnetic resonance reconstructed image is input into a preset generation countermeasure network as shown in fig. 2. Firstly, performing convolution and down-sampling processing on a preliminary magnetic resonance reconstructed image by a generator G to obtain a constructed low-resolution image with resolution reduced by a preset image resolution improvement multiple, and regarding the obtained image as a forged version image F of the preliminary magnetic resonance reconstructed image; furthermore, an image block which has the same size as the image F and is matched with the image F in position is cut out from the preliminary magnetic resonance reconstruction image and is used as a real sample image T. After the image T and the image F are input into the discriminator D together, the discriminator analyzes the possibility that the image F is a real image pixel by pixel according to the image T. The goal of the generator is to make the output image F spoof the discriminator as much as possible. And continuously adjusting parameters in the mutual confrontation process of the generator and the discriminator, and finally finishing network training if the discriminator cannot judge whether the output result F of the generator is real time. In a hot map (heat map) output by the discriminator, an image F and an image T are compared pixel by pixel, every two pixels which are compared with each other are the probability value of the same pixel, and when the probability values of the year in the hot map meet the probability value of positive judgment, whether the output result F of the generator is real can be determined. Further, performing layer-by-layer convolution on all convolution layer parameters of the generator in the trained preset generation countermeasure network to obtain a magnetic resonance image degradation kernel function; therefore, each preliminary magnetic resonance reconstruction image can be convolved with the magnetic resonance image degradation kernel function respectively to obtain a corresponding low-resolution image. In this embodiment, the size and number of convolution kernels in a generator in the preset generation countermeasure network can be changed and adjusted, the generator includes six convolution kernels, and the size of each convolution kernel is: 5x5, 3x3, 1x1 and 1x1. In a preferred embodiment, the structure of the arbiter in the pre-set generation countermeasure network may be the structure shown in fig. 3, and is composed of 7 convolutional layers, and the sizes of the convolutional kernels of the convolutional layers are respectively: 7x7, 1x1 and 1x1. The network structure of the generator is shown in fig. 4, and is composed of 6 convolutional layers, and the sizes of the convolutional cores are respectively: 7x7, 5x5, 3x3, 1x1. The convolution step value of the last layer of convolution layer of the generator is a preset image resolution improvement multiple, so that the effect of performing down-sampling on the input image is achieved.

It should be noted that, if the magnetic resonance imaging object is a heart, the acquired preliminary magnetic resonance reconstructed image is a cardiac magnetic resonance cine image, which includes a plurality of frames of magnetic resonance images. When the generation countermeasure network training is carried out, any frame in the cardiac magnetic resonance film image can be taken as an input image.

And S120, carrying out image interpolation processing on the low-resolution image, wherein the image interpolation multiple is the preset image resolution improvement multiple.

In this step, the image interpolation processing is performed on the low-resolution image, and the image interpolation multiple is a preset image resolution improvement multiple, so that the size of the input image of the image reconstruction model is consistent with the size of the output image, the model training time can be shortened, and the model training process is optimized.

Specifically, in this embodiment, the interpolation method may adopt a Bicubic (Bicubic) algorithm. The image magnification method is characterized in that the image magnification device comprises a double cubic interpolation device, a source image and a reference image, wherein the double cubic interpolation device is used for carrying out image magnification on the image.

In a preferred embodiment, the image subjected to the image interpolation processing and the corresponding preliminary magnetic resonance reconstructed image may be synchronously rotated or mirrored, and an image pair obtained after the rotation or mirroring operation is used as new training sample data of the high-resolution image reconstruction model. For example, 8 sets of training data can be obtained by performing rotation operations of 0 degrees, 90 degrees, 180 degrees, and 270 degrees on the image pair and performing mirror symmetry operations in the horizontal and vertical directions, respectively, so that the training data can be enhanced. Moreover, the phases of the image pairs subjected to synchronous processing are consistent, and registration is not needed, so that the hidden dangers of poor learning effect and the like caused by data deviation are eliminated.

S130, taking the image subjected to image interpolation processing as input data of model training, taking a preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as tag data, training a high-resolution image reconstruction model, and generating a target high-resolution image reconstruction model when a loss function of the high-resolution image reconstruction model converges to a preset value.

After the steps are performed, a sufficient number of model training sample data are constructed, so that the model training process can be started. And taking the image subjected to image interpolation processing as input data of model training, and taking a preliminary magnetic resonance reconstructed image corresponding to the low-resolution image as tag data. The acquired preliminary magnetic resonance image is used as label data, and the resolution is reduced to form a low-resolution image with the preset image resolution improvement multiple. Then, when the preliminary magnetic resonance reconstructed image is input into the trained image reconstruction model, a high-resolution image for improving the resolution of the preliminary magnetic resonance reconstructed image by a preset image resolution improvement multiple can be obtained through the output of the image reconstruction model.

Specifically, the process shown in fig. 5 may be referred to for the training process of the target image reconstruction model. In the process shown in fig. 5, the image reconstruction network is a full convolution network, and there are 8 convolution layers in total, where the convolution kernel size of the first layer is 3x3, the number of channels is f, the convolution kernels of the second to seventh layers are 3x3, the number of channels is 64, the convolution kernel size of the last layer is 3x3, and the number of channels is f. Where f represents the number of frames of images simultaneously input into the image reconstruction network. For an image, the f-number is 1. And when the acquired preliminary magnetic resonance reconstruction image is a cardiac cine image, the value of f is the number of image frames in the cardiac cine image. The number of convolution layers of the image reconstruction network can be changed to 6 or more, meanwhile, the size of the convolution kernel is not limited to 3x3, and parameters can be adjusted according to calculation requirements.

Further, in the model training process in fig. 5, a preferred deep learning method, that is, a residual learning method, is adopted, and after a residual of the image data of the input model after passing through the convolutional layer is added to the input image data itself, a loss function between the input model and the tag data is calculated. Here, it should be noted that the loss function may be not only the L1 loss function shown in fig. 5, but also a loss function such as L2 loss, perceptual loss, or the like. Further, optimization algorithms such as an Adam optimization algorithm, a random gradient descent algorithm or an AdaGrad algorithm can be adopted to optimize the network learning process.

It should be noted here that, in the structure diagram of each network, conv represents a convolution kernel in a convolutional neural network, and "mat" is an abbreviation of a file format, and in this embodiment, represents a magnetic resonance image degradation kernel function obtained by performing layer-by-layer convolution on all convolutional layer parameters of a generator in a pre-set generation countermeasure network.

According to the technical scheme of the embodiment, the resolution of the obtained preliminary magnetic resonance reconstruction image is reduced to obtain a corresponding low-resolution image, and the resolution reduction times can be controlled; then, carrying out image interpolation processing on the low-resolution image, wherein the image interpolation multiple is the preset image resolution improvement multiple; and training a high-resolution image reconstruction model by taking the image subjected to image interpolation as input data of model training and taking a preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as label data, and generating the target high-resolution image reconstruction model when a loss function of the high-resolution image reconstruction model converges to a preset value. The problem of dependence of training on data of a magnetic resonance image reconstruction model in the prior art is solved, the training data and the label data are obtained by degrading the currently acquired magnetic resonance image, network training of the image reconstruction model can be completed, the neural network can be trained without additionally collecting a large amount of low-resolution-high-resolution magnetic resonance parameter quantitative image data which are well paired, learning can be performed by using the inside of the image, and a large amount of image data are not required to be paired.

Example two

Fig. 6 is a flowchart of an image reconstruction method according to a second embodiment of the present invention, which is applicable to reconstructing a low-resolution medical image obtained by acquisition to obtain a high-resolution image. The method may be performed by an image reconstruction apparatus, which may be implemented by means of software and/or hardware, integrated in a computer device having application development functionality.

As shown in fig. 6, the image reconstruction method includes the steps of:

s210, acquiring a preliminary magnetic resonance reconstruction image, and performing image interpolation processing of preset resolution improvement multiple on the preliminary magnetic resonance reconstruction image.

The preliminary magnetic resonance reconstructed image is a low-resolution image needing resolution improvement, and interpolation reconstruction can be carried out on the preliminary magnetic resonance reconstructed image according to the resolution improvement requirement to obtain a preprocessed image. And the multiple of the image interpolation is the preset resolution improvement multiple.

And S220, inputting the preliminary magnetic resonance reconstructed image subjected to the image interpolation processing into a target image reconstruction model which is obtained by the image reconstruction model generating method of any embodiment and is used for increasing the resolution of the magnetic resonance image by the preset resolution increasing multiple to obtain a target magnetic resonance reconstructed image.

The target image reconstruction model is an image reconstruction model trained by the image reconstruction model generation method in the above embodiment according to the requirement of the preset resolution improvement multiple. Specifically, the process of image reconstruction can refer to the schematic diagram shown in fig. 7.

Further, fig. 8 shows reconstructed images obtained by image reconstructing the same preliminary magnetic resonance reconstructed image by using different image reconstruction methods, where (a) NN is a result obtained by a nearest neighbor interpolation algorithm, (b) Bicubic is a result obtained by a Bicubic interpolation algorithm, (c) zero-padding is a result obtained by performing inverse transformation on the image domain after the image is transformed into a fourier domain and performing zero padding on the periphery, and (d) SR (Super Resolution) is a result obtained by the image reconstruction method of this embodiment. The image reconstruction result obtained by the image reconstruction method is clearer in visual results.

According to the technical scheme, the low-resolution image is input into the image reconstruction model which is trained in advance, so that the image resolution can be improved by a preset multiple, and a reconstructed high-resolution image is obtained; the problems of long image reconstruction time and low imaging speed in the prior art are solved, and the high-resolution image with clear edges can be obtained in a short time through an image reconstruction network with a simple structure without a large amount of calculation.

EXAMPLE III

Fig. 9 is a schematic structural diagram of an image reconstruction model generation apparatus according to a third embodiment of the present invention, which is applicable to a case where a low-resolution magnetic resonance image is used to perform image reconstruction model training.

As shown in fig. 9, the image reconstruction model generation apparatus includes an image degradation module 310, an image interpolation module 320, and a model training module 330.

The image quality degradation module 310 is configured to obtain a preliminary magnetic resonance reconstructed image, and reduce a resolution of the preliminary magnetic resonance reconstructed image according to a preset image resolution improvement multiple to obtain a corresponding low-resolution image; an image interpolation module 320, configured to perform image interpolation processing on the low-resolution image, where a multiple of the image interpolation is the preset image resolution improvement multiple; the model training module 330 is configured to train a high-resolution image reconstruction model by using an image subjected to image interpolation as input data for model training and using a preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as tag data, and generate a target high-resolution image reconstruction model when a loss function of the high-resolution image reconstruction model converges to a preset value.

According to the technical scheme of the embodiment, the resolution of the preliminary magnetic resonance reconstruction image is reduced by obtaining the preliminary magnetic resonance reconstruction image and increasing the multiple according to the preset image resolution, so that a corresponding low-resolution image is obtained; performing image interpolation processing on the low-resolution image, wherein the image interpolation multiple is the preset image resolution improvement multiple; and training a high-resolution image reconstruction model by taking the image subjected to image interpolation as input data of model training and taking a preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as label data, and generating the target high-resolution image reconstruction model when a loss function of the high-resolution image reconstruction model converges to a preset value. The problem of dependence of training on data of a magnetic resonance image reconstruction model in the prior art is solved, the training data and the label data are obtained by degrading the currently acquired magnetic resonance image, network training of the image reconstruction model can be completed, the neural network can be trained without additionally collecting a large amount of low-resolution-high-resolution magnetic resonance parameter quantitative image data which are well paired, learning can be performed by using the inside of the image, and a large amount of image data are not required to be paired.

Optionally, the image degradation module 310 is specifically configured to:

performing layer-by-layer convolution on all convolution layer parameters of a generator in the trained preset generation countermeasure network to obtain a magnetic resonance image degradation kernel function;

Optionally, the image interpolation module 320 is specifically configured to:

Optionally, the image reconstruction model generating device further includes a training sample enhancement module, configured to synchronously perform rotation or mirror image operation on the image subjected to the image interpolation processing and the preliminary magnetic resonance reconstructed image before training the high-resolution image reconstruction model, and use an image pair obtained after the rotation or mirror image operation as training sample data of a new high-resolution image reconstruction model.

Optionally, the model training module 330 is further configured to, in the training process of the high-resolution image reconstruction model, calculate a loss function between the input data and the label data after adding a residual of the input data after passing through the convolutional layer to the input data by using a residual learning method.

The image reconstruction model generation device provided by the embodiment of the invention can execute the image reconstruction model generation method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.

Example four

Fig. 10 is a schematic structural diagram of an image reconstruction apparatus according to a fourth embodiment of the present invention, which is applicable to reconstructing a low-resolution medical image obtained by acquisition to obtain a high-resolution image.

As shown in fig. 10, the image reconstruction apparatus includes an image preprocessing module 410 and an image reconstruction module 420.

The image preprocessing module 410 is configured to acquire a preliminary magnetic resonance reconstructed image, and perform image interpolation processing with a preset resolution improvement multiple on the preliminary magnetic resonance reconstructed image; the image reconstruction module 420 is configured to input the preliminary magnetic resonance reconstructed image subjected to the image interpolation processing into a target image reconstruction model obtained by the image reconstruction model generation method according to any embodiment, where the resolution of the magnetic resonance image is increased by the preset resolution increase factor, so as to obtain a target magnetic resonance reconstructed image.

The image reconstruction device provided by the embodiment of the invention can execute the image reconstruction method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.

EXAMPLE five

Fig. 11 is a schematic structural diagram of a computer device according to a fifth embodiment of the present invention. FIG. 11 illustrates a block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention. The computer device 12 shown in fig. 11 is only an example and should not bring any limitation to the function and the scope of use of the embodiments of the present invention. The computer device 12 may be any terminal device with computing capability, such as a terminal device of an intelligent controller and server, a mobile phone, etc., connected to the mri apparatus.

As shown in FIG. 11, computer device 12 is in the form of a general purpose computing device. The components of computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.

Bus 18 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, such architectures include, but are not limited to, industry Standard Architecture (ISA) bus, micro-channel architecture (MAC) bus, enhanced ISA bus, video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

Computer device 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer device 12 and includes both volatile and nonvolatile media, removable and non-removable media.

The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and/or cache memory 32. The computer device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 11, and commonly referred to as a "hard drive"). Although not shown in FIG. 11, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.

A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.

Computer device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), with one or more devices that enable a user to interact with computer device 12, and/or with any devices (e.g., network card, modem, etc.) that enable computer device 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Also, computer device 12 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) through network adapter 20. As shown, network adapter 20 communicates with the other modules of computer device 12 via bus 18. It should be understood that although not shown in FIG. 11, other hardware and/or software modules may be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

The processing unit 16 executes programs stored in the system memory 28 to perform various functional applications and data processing, for example, to implement steps of an image reconstruction model generation method provided by the embodiment of the present invention, the method including:

and training a high-resolution image reconstruction model by taking the image subjected to image interpolation as input data of model training and taking a preliminary magnetic resonance reconstruction image corresponding to the low-resolution image as tag data, and generating a target high-resolution image reconstruction model when a loss function of the high-resolution image reconstruction model is converged to a preset value.

Or, the steps of the image reconstruction method provided by the embodiment of the present invention may also be implemented, where the method includes:

and inputting the preliminary magnetic resonance reconstructed image subjected to image interpolation into a target image reconstruction model which is obtained by the image reconstruction model generating method of any embodiment and is used for increasing the resolution of the magnetic resonance image by the preset resolution increasing multiple to obtain a target magnetic resonance reconstructed image.

Example six

The sixth embodiment provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements an image reconstruction model generation method according to any embodiment of the present invention, including:

Or, steps of an image reconstruction method provided in this embodiment may also be implemented, where the method includes:

Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer-readable storage medium may be, for example but not limited to: an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

It will be understood by those skilled in the art that the modules or steps of the invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of computing devices, and optionally they may be implemented by program code executable by a computing device, such that it may be stored in a memory device and executed by a computing device, or it may be separately fabricated into various integrated circuit modules, or it may be fabricated by fabricating a plurality of modules or steps thereof into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.

It is to be noted that the foregoing description is only exemplary of the invention and that the principles of the technology may be employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. An image reconstruction model generation method, comprising:

2. The method of claim 1, wherein obtaining a preliminary magnetic resonance reconstructed image, and reducing a resolution of the preliminary magnetic resonance reconstructed image according to a preset image resolution improvement multiple to obtain a corresponding low resolution image comprises:

3. The method of claim 2, wherein the generator of the preset generated countermeasure network comprises six convolutional layers, and a convolution step size of a last layer of the six convolutional layers is the preset image resolution improvement multiple; the preset generation countermeasure network arbiter comprises seven convolutional layers.

4. The method according to any one of claims 1 to 3, wherein the image interpolation processing on the low resolution image includes:

5. The method of any of claims 1-3, wherein prior to training the high resolution image reconstruction model, the method further comprises:

6. The method according to any one of claims 1 to 3, wherein in the training process of the high resolution image reconstruction model, a residual learning method is adopted, and after adding the residual of the input data after passing through the convolution layer to the input data, a loss function between the input data and the label data is calculated.

7. An image reconstruction method, comprising:

inputting the preliminary magnetic resonance reconstructed image subjected to image interpolation into a target image reconstruction model obtained by the image reconstruction model generating method according to any one of claims 1 to 6, wherein the target image reconstruction model is obtained by increasing the resolution of the magnetic resonance image by the preset resolution increasing factor, so as to obtain a target magnetic resonance reconstructed image.

8. An image reconstruction model generation apparatus, comprising:

9. An image reconstruction apparatus, comprising:

an image reconstruction module, configured to input the preliminary magnetic resonance reconstructed image subjected to the image interpolation processing into a target image reconstruction model obtained by the image reconstruction model generation method according to any one of claims 1 to 6, where the resolution of the magnetic resonance image is increased by the preset resolution increase factor, so as to obtain a target magnetic resonance reconstructed image.

10. A computer device, characterized in that the computer device comprises:

one or more processors;

a memory for storing one or more programs;

when executed by the one or more processors, cause the one or more processors to implement the image reconstruction model generation method or the image reconstruction method of any of claims 1-7.

11. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the image reconstruction model generation method or the image reconstruction method according to any one of claims 1 to 7.