WO2023116633A1 - 细节保真多尺度深度学习磁共振动态图像重建方法 - Google Patents

细节保真多尺度深度学习磁共振动态图像重建方法 Download PDF

Info

Publication number
WO2023116633A1
WO2023116633A1 PCT/CN2022/140071 CN2022140071W WO2023116633A1 WO 2023116633 A1 WO2023116633 A1 WO 2023116633A1 CN 2022140071 W CN2022140071 W CN 2022140071W WO 2023116633 A1 WO2023116633 A1 WO 2023116633A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
fidelity
scale
magnetic resonance
detail
Prior art date
Application number
PCT/CN2022/140071
Other languages
English (en)
French (fr)
Inventor
王珊珊
郑海荣
邹娟
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Publication of WO2023116633A1 publication Critical patent/WO2023116633A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/30—Assessment of water resources

Definitions

  • the present invention relates to the technical field of medical imaging, in particular to a detail fidelity multi-scale deep learning magnetic resonance dynamic image reconstruction method, device, equipment and storage medium thereof.
  • High-resolution magnetic resonance imaging can clearly display the anatomical structure information of the scanned object, which has important clinical applications.
  • the clinical application is limited. Undersampling the K-space data can effectively reduce the scanning time, but the undersampling scan cannot directly produce high-resolution images. Therefore, the realization of magnetic resonance imaging It is an urgent problem to be solved to scan at a high acceleration multiple and improve the resolution of magnetic resonance images.
  • high-resolution images are generated using image super-resolution technology.
  • This type of method is divided into two steps.
  • the first step is to reconstruct the low-resolution image from the collected low-resolution K-space information.
  • the second step is to use image super-resolution technology to post-process the low-resolution image into a high-resolution one. image.
  • reconstruction and super-resolution techniques are two separate steps, and the original K-space information does not directly participate in the super-resolution step, resulting in errors generated during the reconstruction process will accumulate in the super-resolution process, affecting high-resolution images the quality of.
  • the embodiment of the present application provides a detailed fidelity multi-scale deep learning magnetic resonance dynamic image reconstruction method, the method includes: determining the image to be reconstructed; inputting the image to be reconstructed into the image reconstruction model, the The image reconstruction model includes an image super-resolution unit and a bidirectional long-short-term memory convolutional network unit; in the image reconstruction model, a multi-scale super-resolution method is used to form a high-detail fidelity image; Correlation, using a bidirectional long-short-term memory convolutional network to merge adjacent frame information.
  • the determining the image to be reconstructed includes: determining an undersampled image.
  • the multi-scale super-resolution method used in the image reconstruction model to form a high-detail fidelity image includes: performing N-1 times of maximum pooling on the intermediate result of the first iterative reconstruction of the undersampled image Perform the first super-resolution technique on the obtained image to obtain the first residual image; perform N-2 maximum pooling operations on the intermediate result of the second iterative reconstruction, and compare the obtained image with the first residual image
  • the images are added to obtain the first iterative detail fidelity image; the 1/(N-2)2 scale supervision of the reconstructed image is carried out on the first iterative detail fidelity image, and the above operations are repeated in turn to obtain a full-scale high detail fidelity image.
  • the merging adjacent frame information by using a bidirectional long-short-term memory convolutional network includes: merging temporal and spatial dependencies by using a bidirectional long-short-term memory convolutional network.
  • the embodiment of the present application also provides a detailed fidelity multi-scale deep learning magnetic resonance dynamic image reconstruction device, the device includes: a determination unit, used to determine the image to be reconstructed; an input unit, used to input the image to be reconstructed
  • the reconstructed image is input into the image reconstruction model, which includes an image super-resolution unit and a bidirectional long-short-term memory convolution network unit; a formation unit is used to form a high-resolution image using a multi-scale super-resolution device in the image reconstruction model.
  • the determining the image to be reconstructed includes: determining an undersampled image.
  • the use of a multi-scale super-resolution device in the image reconstruction model to form a high-detail fidelity image includes: performing N-1 times of maximum pooling on the intermediate result of the first iterative reconstruction of the undersampled image Perform the first super-resolution technique on the obtained image to obtain the first residual image; perform N-2 maximum pooling operations on the intermediate result of the second iterative reconstruction, and compare the obtained image with the first residual image
  • the images are added to obtain the first iterative detail fidelity image; the 1/(N-2)2 scale supervision of the reconstructed image is carried out on the first iterative detail fidelity image, and the above operations are repeated in turn to obtain a full-scale high detail fidelity image.
  • the merging adjacent frame information by using a bidirectional long-short-term memory convolutional network includes: merging temporal and spatial dependencies by using a bidirectional long-short-term memory convolutional network.
  • the embodiment of the present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and operable on the processor.
  • the processor executes the program, it implements the The method described in any one of the descriptions of the examples.
  • the embodiment of the present application also provides a computer device, a computer-readable storage medium, on which a computer program is stored, and the computer program is used for: when the computer program is executed by a processor, the computer program according to the present application is implemented.
  • a computer device a computer-readable storage medium, on which a computer program is stored, and the computer program is used for: when the computer program is executed by a processor, the computer program according to the present application is implemented.
  • the detail-fidelity multi-scale deep learning magnetic resonance dynamic image reconstruction method realizes the reconstruction from high-multiple undersampling data to high-detail fidelity images.
  • Existing techniques either require the undersampled data to be undersampled by an undersampling factor of about 10, or the details of the reconstructed image are not clear enough.
  • the technical scheme proposed by the present invention reconstructs a high-detail fidelity image very similar to a full-sampling image when the under-sampling multiple is greater than 10.
  • FIG. 1 shows a schematic flow diagram of a detailed fidelity multi-scale deep learning magnetic resonance dynamic image reconstruction method provided by an embodiment of the present application
  • Fig. 2 shows an exemplary structural block diagram of a detailed fidelity multi-scale deep learning magnetic resonance dynamic image reconstruction device 200 according to an embodiment of the present application
  • FIG. 3 shows a schematic structural diagram of a computer system suitable for implementing a terminal device according to an embodiment of the present application
  • FIG. 4 shows another schematic flowchart provided by the embodiment of the present application.
  • first and second are used for descriptive purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features.
  • the features defined as “first” and “second” may explicitly or implicitly include at least one of these features.
  • “plurality” means at least two, such as two, three, etc., unless otherwise specifically defined.
  • the first feature may be in direct contact with the first feature or the first and second feature may be in direct contact with the second feature through an intermediary. touch.
  • “above”, “above” and “above” the first feature on the second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature.
  • “Below”, “beneath” and “beneath” the first feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is less horizontally than the second feature.
  • FIG. 1 shows a schematic flow chart of a detailed fidelity multi-scale deep learning magnetic resonance dynamic image reconstruction method provided by an embodiment of the present application.
  • the method includes:
  • Step 110 determining the image to be reconstructed
  • Step 120 inputting the image to be reconstructed into an image reconstruction model, which includes an image super-resolution unit and a bidirectional long-short-term memory convolutional network unit;
  • Step 130 using a multi-scale super-resolution method in the image reconstruction model to form a high-detail fidelity image; for a movie magnetic resonance image, using the time correlation of the image sequence, using a bidirectional long-short-term memory convolution network to merge adjacent frame information.
  • the reconstruction from high-multiple under-sampled data to high-detail and high-fidelity images is realized.
  • Existing techniques either require the undersampled data to be undersampled by an undersampling factor of about 10, or the details of the reconstructed image are not clear enough.
  • the technical scheme proposed by the present invention reconstructs a high-detail fidelity image very similar to a full-sampling image when the under-sampling multiple is greater than 10.
  • determining an image to be reconstructed in the present application includes: determining an undersampled image.
  • the multi-scale super-resolution method used in the image reconstruction model in this application to form a high-detail fidelity image includes: performing N-1 maximum Pooling operation, perform the first super-resolution technique on the obtained image to obtain the first residual image; perform N-2 maximum pooling operations on the intermediate result of the second iterative reconstruction, and compare the obtained image with the first residual image
  • the difference images are added to obtain the first iterative detail fidelity image; the 1/(N-2)2 scale supervision of the reconstructed image is carried out on the first iterative detail fidelity image, and the above operations are repeated in turn to obtain the full-scale high-resolution image.
  • Detail fidelity images are performed by: performing N-1 maximum Pooling operation, perform the first super-resolution technique on the obtained image to obtain the first residual image; perform N-2 maximum pooling operations on the intermediate result of the second iterative reconstruction, and compare the obtained image with the first residual image
  • the difference images are added to obtain the first iterative detail fidelity image; the 1/(N-2)2 scale supervision of the
  • combining information of adjacent frames using a bidirectional long-short-term memory convolutional network in this application includes: using a bidirectional long-term short-term memory convolutional network to combine temporal and spatial dependencies.
  • the multi-scale detail fidelity network proposed in this application is used in the method of movie magnetic resonance imaging, including the reconstruction-super-resolution process and the bidirectional long-short-term memory convolution network process.
  • This reconstruction model is an end-to-end deep learning method.
  • the network inputs under-sampled images, outputs reconstructed high-detail fidelity images, and multi-supervises the network with multi-scale images of fully sampled images.
  • the specific implementation steps are as follows:
  • the present invention adds N-1 times of multi-scale super-resolution technology in the N times of iterative process of reconstructing the cascaded network.
  • the specific steps are: perform N-1 times of maximum pooling operations on the intermediate results of the first iterative reconstruction.
  • the image is subjected to the first super-resolution technique to obtain the first residual image
  • N-2 maximum pooling operations are performed on the intermediate result of the second iterative reconstruction
  • the obtained image is added to the first residual image to obtain the second Iterate the detail fidelity image once, and supervise the 1/(N-2)2 scale of the final reconstructed image on this image.
  • the subsequent iteration process performs the above operations in sequence to obtain the final output full-size high-detail fidelity image.
  • the convolution part of the long-term short-term memory convolutional network structure not only considers the spatial correlation of the image, but also considers the temporal dependence between adjacent image frames in the long-term short-term memory method .
  • the unidirectional long-short-term memory convolutional network structure can only capture the historical information of the sequence. Heart movement is a continuous dynamic process, and the future information of the sequence is also an important reference for reconstructing sequence images. Therefore, in the network of the present application, a bidirectional LSTM convolutional network is used between the layers of the 3D convolutional neural network during the reconstruction process to incorporate temporal and spatial dependencies.
  • the basic idea is to use two long-short-term memory convolutional network structures in the hidden layer to model the sequence in the forward and reverse directions, and then connect their outputs.
  • FIG. 2 shows an exemplary structural block diagram of a detailed fidelity multi-scale deep learning magnetic resonance dynamic image reconstruction apparatus 200 according to an embodiment of the present application.
  • the device includes:
  • a determining unit 210 configured to determine an image to be reconstructed
  • the input unit 220 is used to input the image to be reconstructed into the image reconstruction model, which includes an image super-resolution unit and a bidirectional long-short-term memory convolutional network unit;
  • the forming unit 230 is used to form a high-detail and fidelity image using a multi-scale super-resolution device in the image reconstruction model; for a movie magnetic resonance image, using the temporal correlation of the image sequence, a two-way long-short-term memory convolution network is used to merge adjacent frame information.
  • the units or modules recorded in the device 200 correspond to the steps in the method described with reference to FIG. 1 . Therefore, the operations and features described above for the method are also applicable to the device 200 and the units contained therein, and will not be repeated here.
  • the apparatus 200 may be pre-implemented in the browser of the electronic device or other security applications, and may also be loaded into the browser of the electronic device or its security applications by downloading or other means.
  • the corresponding units in the apparatus 200 may cooperate with the units in the electronic device to implement the solutions of the embodiments of the present application.
  • FIG. 3 shows a schematic structural diagram of a computer system 300 suitable for implementing a terminal device or a server according to an embodiment of the present application.
  • a computer system 300 includes a central processing unit (CPU) 301 that can execute programs according to programs stored in a read-only memory (ROM) 302 or loaded from a storage section 308 into a random-access memory (RAM) 303 Instead, various appropriate actions and processes are performed.
  • CPU 301, ROM 302 and RAM 303 are connected to each other through a bus 304 .
  • An input/output (I/O) interface 305 is also connected to the bus 304 .
  • the following components are connected to the I/O interface 305: an input section 306 including a keyboard, a mouse, etc.; an output section 307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 308 including a hard disk, etc. and a communication section 309 including a network interface card such as a LAN card, a modem, or the like.
  • the communication section 309 performs communication processing via a network such as the Internet.
  • a drive 310 is also connected to the I/O interface 305 as needed.
  • a removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 310 as necessary so that a computer program read therefrom is installed into the storage section 308 as necessary.
  • an embodiment of the present disclosure includes a detail fidelity multi-scale deep learning magnetic resonance dynamic image reconstruction method, which includes a computer program tangibly embodied on a machine-readable medium, the computer program including a method for executing the method shown in FIG. 1 .
  • the program code for the method may be downloaded and installed from a network via communication portion 309 and/or installed from removable media 311 .
  • each block in a flowchart or block diagram may represent a module, program segment, or part of code that includes one or more logical functions for implementing specified executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.
  • the units or modules involved in the embodiments described in the present application may be implemented by means of software or by means of hardware.
  • the described units or modules may also be set in a processor.
  • a processor includes a first sub-region generation unit, a second sub-region generation unit, and a display region generation unit.
  • the names of these units or modules do not constitute limitations on the units or modules themselves in some cases, for example, the display area generation unit can also be described as "used to generate The cell of the display area of the text".
  • the present application also provides a computer-readable storage medium, which may be the computer-readable storage medium contained in the aforementioned devices in the above-mentioned embodiments; computer-readable storage media stored in the device.
  • the computer-readable storage medium stores one or more programs, and the aforementioned programs are used by one or more processors to execute the text generation method applied to transparent window envelopes described in this application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Biophysics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)

Abstract

一种细节保真多尺度深度学习磁共振动态图像重建方法、装置、设备及其存储介质，该方法包括：确定待重建图像；将待重建图像输入至图像重建模型中，图像重建模型中包括图像超分辨率单元和双向长短期记忆卷积网络单元；在图像重建模型中采用多尺度超分辨率方法形成高细节保真图像；对于电影磁共振图像，利用图像序列的时间相关性，采用双向长短期记忆卷积网络合并相邻帧信息。实现了从高倍数欠采样数据到高细节保真图像的重建。

Description

细节保真多尺度深度学习磁共振动态图像重建方法 技术领域
本发明涉及医学影像技术领域，具体涉及一种细节保真多尺度深度学习磁共振动态图像重建方法、装置、设备及其存储介质。
 背景技术
高分辨率磁共振成像可以清晰的表现出扫描对象的解剖结构信息，在临床中有着重要的应用。然而，由于磁共振扫描扫描时间长，限制了在临床的应用，对K空间数据进行欠采样可以有效的减少扫描时间，但是欠采样扫描并不能直接产生高分辨率图像，所以，实现磁共振成像的高加速倍数扫描，并提高磁共振图像分辨率是亟需解决的一个问题。
目前，解决这个问题的方案主要有两种:其一是从压缩感知重建的角度出发，利用磁共振图像的稀疏性等先验信息，对K空间进行高度欠采样，节省扫描时间，再利用基于压缩感知重建算法重建出高分辨率图像。这类方法包括传统的重建方法（如稀疏约束重建算法、压缩感知SENSE算法等）和基于深度学习的神经网络方法（如ADMM-Net、ISTA-Net和Variational-Net）。然而，这种策略下的重建方法在高加速倍数（大于10倍）时，难以重建出高分辨率的图像。其二是基于计算机视觉的超分辨技术，在采集和重建的低分辨率图像的基础上，利用图像的超分辨率技术生成高分辨率图像。该类方法分为两个步骤，第一步是对采集到的低分辨率K空间信息进行低分辨率图像重建，第二步是利用图像超分辨技术对低分辨率图像后处理成高分辨率图像。然而，这种策略下重建和超分辨率技术是两个分开的步骤，原始K空间信息不直接参与超分辨步骤，导致重建过程中产生的误差会累积到超分辨过程中，影响高分辨率图像的质量。
 技术问题
鉴于现有技术中的上述缺陷或不足，期望提供一种细节保真多尺度深度学习磁共振动态图像重建方法、装置、设备及其存储介质。
 技术解决方案
第一方面，本申请实施例提供了一种细节保真多尺度深度学习磁共振动态图像重建方法，该方法包括：确定待重建图像；将所述待重建图像输入至图像重建模型中，所述图像重建模型中包括图像超分辨率单元和双向长短期记忆卷积网络单元；在图像重建模型中采用多尺度超分辨率方法形成高细节保真图像；对于电影磁共振图像，利用图像序列的时间相关性，采用双向长短期记忆卷积网络合并相邻帧信息。
在其中一个实施例中，所述确定待重建图像包括：确定欠采图像。
在其中一个实施例中，所述在图像重建模型中采用多尺度超分辨率方法形成高细节保真图像，包括：对欠采图像的第一次迭代重建的中间结果进行N-1次最大池化操作，对所得到图像进行第一次超分技术得到第一个残差图像；对第二次迭代重建的中间结果进行N-2次最大池化操作，对所得图像与第一个残差图像相加，得到第一次迭代细节保真图像；对第一次迭代细节保真图像进行重建图像的1/(N-2)2尺度的监督，依次重复上述操作，得到全尺寸的高细节保真图像。
在其中一个实施例中，所述采用双向长短期记忆卷积网络合并相邻帧信息，包括：采用双向长短期记忆卷积网络合并时间和空间的依赖性。
第二方面，本申请实施例还提供了一种细节保真多尺度深度学习磁共振动态图像重建装置，该装置包括：确定单元，用于确定待重建图像；输入单元，用于将所述待重建图像输入至图像重建模型中，所述图像重建模型中包括图像超分辨率单元和双向长短期记忆卷积网络单元；形成单元，用于在图像重建模型中采用多尺度超分辨率装置形成高细节保真图像；对于电影磁共振图像，利用图像序列的时间相关性，采用双向长短期记忆卷积网络合并相邻帧信息。
在其中一个实施例中，所述确定待重建图像包括：确定欠采图像。
在其中一个实施例中，所述在图像重建模型中采用多尺度超分辨率装置形成高细节保真图像，包括：对欠采图像的第一次迭代重建的中间结果进行N-1次最大池化操作，对所得到图像进行第一次超分技术得到第一个残差图像；对第二次迭代重建的中间结果进行N-2次最大池化操作，对所得图像与第一个残差图像相加，得到第一次迭代细节保真图像；对第一次迭代细节保真图像进行重建图像的1/(N-2)2尺度的监督，依次重复上述操作，得到全尺寸的高细节保真图像。
在其中一个实施例中，所述采用双向长短期记忆卷积网络合并相邻帧信息，包括：采用双向长短期记忆卷积网络合并时间和空间的依赖性。
第三方面，本申请实施例还提供了一种计算机设备，包括存储器、处理器以及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述程序时实现如本申请实施例描述中任一所述的方法。
第四方面，本申请实施例还提供了一种计算机设备一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序用于：所述计算机程序被处理器执行时实现如本申请实施例描述中任一所述的方法。
 有益效果
本发明的有益效果：
本发明提供的细节保真多尺度深度学习磁共振动态图像重建方法，实现了从高倍数欠采样数据到高细节保真图像的重建。现有技术要么要求欠采样数据为大约低于10的欠采倍数，要么所重建图像的细节不够清晰。本发明所提技术方案，在欠采样倍数大于10时，重建出与全采样图像很相近的高细节保真图像。
 附图说明
通过阅读参照以下附图所作的对非限制性实施例所作的详细描述，本申请的其它特征、目的和优点将会变得更明显：
图1示出了本申请实施例提供的细节保真多尺度深度学习磁共振动态图像重建方法的流程示意图；
图2示出了根据本申请一个实施例的细节保真多尺度深度学习磁共振动态图像重建装置200的示例性结构框图；
图3示出了适于用来实现本申请实施例的终端设备的计算机系统的结构示意图；
图4示出了本申请实施例提供的又一流程示意图。
 本发明的实施方式
为使本发明的上述目的、特征和优点能够更加明显易懂，下面结合附图对本发明的具体实施方式做详细的说明。在下面的描述中阐述了很多具体细节以便于充分理解本发明。但是本发明能够以很多不同于在此描述的其它方式来实施，本领域技术人员可以在不违背本发明内涵的情况下做类似改进，因此本发明不受下面公开的具体实施例的限制。
在本发明的描述中，需要理解的是，术语“中心”、“纵向”、“横向”、“长度”、“宽度”、“厚度”、“上”、“下”、“前”、“后”、“左”、“右”、“竖直”、“水平”、“顶”、“底”、“内”、“外”、“顺时针”、“逆时针”、“轴向”、“径向”、“周向”等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本发明和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本发明的限制。
此外，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。在本发明的描述中，“多个”的含义是至少两个，例如两个，三个等，除非另有明确具体的限定。
在本发明中，除非另有明确的规定和限定，术语“安装”、“相连”、“连接”、“固定”等术语应做广义理解，例如，可以是固定连接，也可以是可拆卸连接，或成一体；可以是机械连接，也可以是电连接；可以是直接相连，也可以通过中间媒介间接相连，可以是两个元件内部的连通或两个元件的相互作用关系，除非另有明确的限定。对于本领域的普通技术人员而言，可以根据具体情况理解上述术语在本发明中的具体含义。
在本发明中，除非另有明确的规定和限定，第一特征在第二特征“上”或“下”可以是第一和第二特征直接接触，或第一和第二特征通过中间媒介间接接触。而且，第一特征在第二特征“之上”、“上方”和“上面”可是第一特征在第二特征正上方或斜上方，或仅仅表示第一特征水平高度高于第二特征。第一特征在第二特征“之下”、“下方”和“下面”可以是第一特征在第二特征正下方或斜下方，或仅仅表示第一特征水平高度小于第二特征。
需要说明的是，当元件被称为“固定于”或“设置于”另一个元件，它可以直接在另一个元件上或者也可以存在居中的元件。当一个元件被认为是“连接”另一个元件，它可以是直接连接到另一个元件或者可能同时存在居中元件。本文所使用的术语“垂直的”、“水平的”、“上”、“下”、“左”、“右”以及类似的表述只是为了说明的目的，并不表示是唯一的实施方式。
请参考图1，图1示出了本申请实施例提供的细节保真多尺度深度学习磁共振动态图像重建方法的流程示意图。
如图1所示，该方法包括：
步骤110，确定待重建图像；
步骤120，将所述待重建图像输入至图像重建模型中，所述图像重建模型中包括图像超分辨率单元和双向长短期记忆卷积网络单元；
步骤130，在图像重建模型中采用多尺度超分辨率方法形成高细节保真图像；对于电影磁共振图像，利用图像序列的时间相关性，采用双向长短期记忆卷积网络合并相邻帧信息。
采用上述技术方案，实现了从高倍数欠采样数据到高细节保真图像的重建。现有技术要么要求欠采样数据为大约低于10的欠采倍数，要么所重建图像的细节不够清晰。本发明所提技术方案，在欠采样倍数大于10时，重建出与全采样图像很相近的高细节保真图像。
在一些实施例中，本申请中的确定待重建图像包括：确定欠采图像。
在一些实施例中，本申请中的在图像重建模型中采用多尺度超分辨率方法形成高细节保真图像，包括：对欠采图像的第一次迭代重建的中间结果进行N-1次最大池化操作，对所得到图像进行第一次超分技术得到第一个残差图像；对第二次迭代重建的中间结果进行N-2次最大池化操作，对所得图像与第一个残差图像相加，得到第一次迭代细节保真图像；对第一次迭代细节保真图像进行重建图像的1/(N-2)2尺度的监督，依次重复上述操作，得到全尺寸的高细节保真图像。
在一些实施例中，本申请中的采用双向长短期记忆卷积网络合并相邻帧信息，包括：采用双向长短期记忆卷积网络合并时间和空间的依赖性。
参考图4所示，本申请提出的多尺度细节保真网络用于电影磁共振成像方法包括重建-超分过程和双向长短期记忆卷积网络过程，此重建模型是端到端的深度学习方法，网络输入欠采图像，输出重建高细节保真图像，以全采样图像的多尺度图像对网络进行多监督。具体实现步骤如下：
本发明在重建级联网络的N次迭代过程中加入N-1次多尺度超分技术，具体步骤为：对第一次迭代重建的中间结果进行N-1次最大池化操作，对所得到图像进行第一次超分技术得到第一个残差图像，对第二次迭代重建的中间结果进行N-2次最大池化操作，对所得图像与第一个残差图像相加，得到第一次迭代细节保真图像，对这个图像进行最后重建图像的1/(N-2)2尺度的监督，后面的迭代过程依次进行上述操作，得到最终输出的全尺寸的高细节保真图像
不同于标准的传统的长短期记忆方法，长短期记忆卷积网络结构中的卷积部分不仅考虑了图像的空间相关性，长短期记忆方法部分还考虑了相邻图像帧之间的时间依赖性。而单向的长短期记忆卷积网络结构只能捕获序列的历史信息，心脏运动是连续的动态过程，序列的未来信息对重建序列图像也有重要参考作用。因此，在本申请的网络中，重建过程中在三维卷积神经网络的层之间运用了双向长短期记忆卷积网络来合并时间和空间的依赖性。其基本思想是在隐层使用两个长短期记忆卷积网络结构分别按照正向和反向对序列建模，然后将他们的输出连接起来。
进一步地，参考图2，图2示出了根据本申请一个实施例的细节保真多尺度深度学习磁共振动态图像重建装置200的示例性结构框图。
如图2所示，该装置包括：
确定单元210，用于确定待重建图像；
输入单元220，用于将所述待重建图像输入至图像重建模型中，所述图像重建模型中包括图像超分辨率单元和双向长短期记忆卷积网络单元；
形成单元230，用于在图像重建模型中采用多尺度超分辨率装置形成高细节保真图像；对于电影磁共振图像，利用图像序列的时间相关性，采用双向长短期记忆卷积网络合并相邻帧信息。
应当理解，装置200中记载的诸单元或模块与参考图1描述的方法中的各个步骤相对应。由此，上文针对方法描述的操作和特征同样适用于装置200及其中包含的单元，在此不再赘述。装置200可以预先实现在电子设备的浏览器或其他安全应用中，也可以通过下载等方式而加载到电子设备的浏览器或其安全应用中。装置200中的相应单元可以与电子设备中的单元相互配合以实现本申请实施例的方案。
下面参考图3，其示出了适于用来实现本申请实施例的终端设备或服务器的计算机系统300的结构示意图。
如图3所示，计算机系统300包括中央处理单元（CPU）301，其可以根据存储在只读存储器（ROM）302中的程序或者从存储部分308加载到随机访问存储器（RAM）303中的程序而执行各种适当的动作和处理。在RAM 303中，还存储有系统300操作所需的各种程序和数据。CPU 301、ROM 302以及RAM 303通过总线304彼此相连。输入/输出（I/O）接口305也连接至总线304。
以下部件连接至I/O接口305：包括键盘、鼠标等的输入部分306；包括诸如阴极射线管（CRT）、液晶显示器（LCD）等以及扬声器等的输出部分307；包括硬盘等的存储部分308；以及包括诸如LAN卡、调制解调器等的网络接口卡的通信部分309。通信部分309经由诸如因特网的网络执行通信处理。驱动器310也根据需要连接至I/O接口305。可拆卸介质311，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在驱动器310上，以便于从其上读出的计算机程序根据需要被安装入存储部分308。
特别地，根据本公开的实施例，上文参考图1描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种细节保真多尺度深度学习磁共振动态图像重建方法，其包括有形地包含在机器可读介质上的计算机程序，所述计算机程序包含用于执行图1的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信部分309从网络上被下载和安装，和/或从可拆卸介质311被安装。
附图中的流程图和框图，图示了按照本发明各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，前述模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本申请实施例中所涉及到的单元或模块可以通过软件的方式实现，也可以通过硬件的方式来实现。所描述的单元或模块也可以设置在处理器中，例如，可以描述为：一种处理器包括第一子区域生成单元、第二子区域生成单元以及显示区域生成单元。其中，这些单元或模块的名称在某种情况下并不构成对该单元或模块本身的限定，例如，显示区域生成单元还可以被描述为“用于根据第一子区域和第二子区域生成文本的显示区域的单元”。
作为另一方面，本申请还提供了一种计算机可读存储介质，该计算机可读存储介质可以是上述实施例中前述装置中所包含的计算机可读存储介质；也可以是单独存在，未装配入设备中的计算机可读存储介质。计算机可读存储介质存储有一个或者一个以上程序，前述程序被一个或者一个以上的处理器用来执行描述于本申请的应用于透明窗口信封的文本生成方法。
以上描述仅为本申请的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本申请中所涉及的发明范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离前述发明构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本申请中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。

Claims (10)

  1. 一种细节保真多尺度深度学习磁共振动态图像重建方法，其特征在于，该方法包括：
    确定待重建图像；
    将所述待重建图像输入至图像重建模型中，所述图像重建模型中包括图像超分辨率单元和双向长短期记忆卷积网络单元；
    在图像重建模型中采用多尺度超分辨率方法形成高细节保真图像；对于电影磁共振图像，利用图像序列的时间相关性，采用双向长短期记忆卷积网络合并相邻帧信息。
  2. 根据权利要求1所述的细节保真多尺度深度学习磁共振动态图像重建方法，其特征在于，所述确定待重建图像包括：
    确定欠采图像。
  3. 根据权利要求2所述的细节保真多尺度深度学习磁共振动态图像重建方法，其特征在于，所述在图像重建模型中采用多尺度超分辨率方法形成高细节保真图像，包括：
    对欠采图像的第一次迭代重建的中间结果进行N-1次最大池化操作，对所得到图像进行第一次超分技术得到第一个残差图像；
    对第二次迭代重建的中间结果进行N-2次最大池化操作，对所得图像与第一个残差图像相加，得到第一次迭代细节保真图像；
    对第一次迭代细节保真图像进行重建图像的1/(N-2)2尺度的监督，依次重复上述操作，得到全尺寸的高细节保真图像。
  4. 根据权利要求3所述的细节保真多尺度深度学习磁共振动态图像重建方法，其特征在于，所述采用双向长短期记忆卷积网络合并相邻帧信息，包括：
    采用双向长短期记忆卷积网络合并时间和空间的依赖性。
  5. 一种细节保真多尺度深度学习磁共振动态图像重建装置，其特征在于，该装置包括：
    确定单元，用于确定待重建图像；
    输入单元，用于将所述待重建图像输入至图像重建模型中，所述图像重建模型中包括图像超分辨率单元和双向长短期记忆卷积网络单元；
    形成单元，用于在图像重建模型中采用多尺度超分辨率装置形成高细节保真图像；对于电影磁共振图像，利用图像序列的时间相关性，采用双向长短期记忆卷积网络合并相邻帧信息。
  6. 根据权利要求5所述的细节保真多尺度深度学习磁共振动态图像重建装置，其特征在于，所述确定待重建图像包括：
    确定欠采图像。
  7. 根据权利要求6所述的细节保真多尺度深度学习磁共振动态图像重建装置，其特征在于，所述在图像重建模型中采用多尺度超分辨率装置形成高细节保真图像，包括：
    对欠采图像的第一次迭代重建的中间结果进行N-1次最大池化操作，对所得到图像进行第一次超分技术得到第一个残差图像；
    对第二次迭代重建的中间结果进行N-2次最大池化操作，对所得图像与第一个残差图像相加，得到第一次迭代细节保真图像；
    对第一次迭代细节保真图像进行重建图像的1/(N-2)2尺度的监督，依次重复上述操作，得到全尺寸的高细节保真图像。
  8. 根据权利要求7所述的细节保真多尺度深度学习磁共振动态图像重建装置，其特征在于，所述采用双向长短期记忆卷积网络合并相邻帧信息，包括：
    采用双向长短期记忆卷积网络合并时间和空间的依赖性。
  9. 一种计算机设备，包括存储器、处理器以及存储在存储器上并可在处理器上运行的计算机程序，其特征在于，所述处理器执行所述程序时实现如权利要求1-4中任一所述的方法。
  10. 一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序用于：
    所述计算机程序被处理器执行时实现如权利要求1-4中任一所述的方法。
PCT/CN2022/140071 2021-12-21 2022-12-19 细节保真多尺度深度学习磁共振动态图像重建方法 WO2023116633A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111576151.8A CN116342722A (zh) 2021-12-21 2021-12-21 细节保真多尺度深度学习磁共振动态图像重建方法
CN202111576151.8 2021-12-21

Publications (1)

Publication Number Publication Date
WO2023116633A1 true WO2023116633A1 (zh) 2023-06-29

Family

ID=86893463

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/140071 WO2023116633A1 (zh) 2021-12-21 2022-12-19 细节保真多尺度深度学习磁共振动态图像重建方法

Country Status (2)

Country Link
CN (1) CN116342722A (zh)
WO (1) WO2023116633A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109658469A (zh) * 2018-12-13 2019-04-19 深圳先进技术研究院 一种基于深度先验学习的头颈联合成像方法和装置
CN109712208A (zh) * 2018-12-13 2019-05-03 深圳先进技术研究院 基于深度学习的大视野磁共振扫描图像重建方法和装置
CN110766768A (zh) * 2019-10-23 2020-02-07 深圳先进技术研究院 一种磁共振图像重建方法、装置、设备和介质
CN110969577A (zh) * 2019-11-29 2020-04-07 北京交通大学 一种基于深度双重注意力网络的视频超分辨率重建方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109658469A (zh) * 2018-12-13 2019-04-19 深圳先进技术研究院 一种基于深度先验学习的头颈联合成像方法和装置
CN109712208A (zh) * 2018-12-13 2019-05-03 深圳先进技术研究院 基于深度学习的大视野磁共振扫描图像重建方法和装置
CN110766768A (zh) * 2019-10-23 2020-02-07 深圳先进技术研究院 一种磁共振图像重建方法、装置、设备和介质
CN110969577A (zh) * 2019-11-29 2020-04-07 北京交通大学 一种基于深度双重注意力网络的视频超分辨率重建方法

Also Published As

Publication number Publication date
CN116342722A (zh) 2023-06-27

Similar Documents

Publication Publication Date Title
Reed et al. Dynamic ct reconstruction from limited views with implicit neural representations and parametric motion fields
Zheng et al. Multi-contrast brain MRI image super-resolution with gradient-guided edge enhancement
US20230206396A1 (en) Image super-resolution reconstructing
CN111353940A (zh) 一种基于深度学习迭代上下采样的图像超分辨率重建方法
CN111861884A (zh) 一种基于深度学习的卫星云图超分辨率重建方法
CN106618571A (zh) 一种磁共振成像方法和系统
Muhammad et al. Multi-scale Xception based depthwise separable convolution for single image super-resolution
CN110717958A (zh) 一种图像重建方法、装置、设备及介质
CN116485655A (zh) 一种基于扩散模型的ct图像超分辨率处理方法
Guan et al. Magnetic resonance imaging reconstruction using a deep energy‐based model
Wu et al. Deep learning-based image super-resolution restoration for mobile infrared imaging system
Sui et al. Simultaneous image reconstruction and lesion segmentation in accelerated MRI using multitasking learning
Zhou et al. CloudBrain-ReconAI: An online platform for MRI reconstruction and image quality evaluation
CN112700508A (zh) 一种基于深度学习的多对比度mri图像重建方法
WO2023116633A1 (zh) 细节保真多尺度深度学习磁共振动态图像重建方法
CN114972033B (zh) 一种提高光学相干层析图像纵向分辨率的自监督方法
CN116797541A (zh) 一种基于Transformer的肺部CT图像超分辨率重建方法
CN110728732A (zh) 一种图像重建方法、装置、设备及介质
WO2022193378A1 (zh) 图像重建模型生成方法及装置、图像重建方法及装置、设备、介质
CN112669400B (zh) 基于深度学习预测和残差架构的动态mr重建方法
WO2022155990A1 (zh) 一种基于自监督学习的视频盲超分辨率重建方法及系统
CN114757826A (zh) 一种基于多特征的pocs图像超分辨率重建方法
Xie et al. Bidirectionally aligned sparse representation for single image super-resolution
Guo et al. Semi‐supervised super‐resolution of diffusion‐weighted images based on multiple references
WO2023115339A1 (zh) 基于物理的数据增强对比学习表示成像方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22909945

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE