WO2020135015A1 - 一种医学成像模型的建立方法、装置、设备及存储介质 - Google Patents

一种医学成像模型的建立方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2020135015A1
WO2020135015A1 PCT/CN2019/124239 CN2019124239W WO2020135015A1 WO 2020135015 A1 WO2020135015 A1 WO 2020135015A1 CN 2019124239 W CN2019124239 W CN 2019124239W WO 2020135015 A1 WO2020135015 A1 WO 2020135015A1
Authority
WO
WIPO (PCT)
Prior art keywords
loss function
model
image
space
image domain
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/124239
Other languages
English (en)
French (fr)
Inventor
梁栋
王珊珊
柯子文
刘新
郑海荣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Institute of Advanced Technology of CAS
Original Assignee
Shenzhen Institute of Advanced Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Institute of Advanced Technology of CAS filed Critical Shenzhen Institute of Advanced Technology of CAS
Publication of WO2020135015A1 publication Critical patent/WO2020135015A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning

Definitions

  • the present application relates to the technical field of magnetic resonance imaging, for example, to a method, device, equipment and storage medium for establishing a medical imaging model.
  • Magnetic resonance cardiac imaging is a non-invasive imaging technique that can provide rich spatial and temporal information for clinical diagnosis. Due to the physical and hardware constraints of magnetic resonance, magnetic resonance cardiac film imaging is often accompanied by the disadvantages of long scanning time and slow imaging speed. Therefore, on the premise of ensuring the imaging quality, accelerated magnetic resonance cardiac film imaging is particularly important.
  • accelerated magnetic resonance cardiac film imaging include parallel imaging, compressed sensing technology, and deep learning methods.
  • TGRAPPA dynamic generalized automatic calibration partial parallel acquisition
  • TENSE adaptive sensitivity coding
  • ktFOCUSS focal underdetermination system
  • kt SLR dynamic redundancy
  • L+S low rank sparse matrix
  • Embodiments of the present invention provide a method, device, equipment, and storage medium for establishing a medical imaging model, so as to restrain the intermediate results of the network, and more accurately and quickly reconstruct the K-space under-sampled medical images.
  • An embodiment of the present invention provides a method for establishing a medical imaging model.
  • the method includes:
  • the original neural network includes a frequency domain network FDN and an image domain network SDN, and the frequency domain network FDN and the image domain network SDN are connected by an inverse Fourier transform IFFT;
  • the model loss function includes a main loss Function and at least one additional loss function, the additional loss function including a k-space loss function and/or an image domain loss function.
  • An embodiment of the present invention also provides a device for establishing a medical imaging model.
  • the device includes:
  • the training sample acquisition module is set to acquire K-space undersampled data of medical images as training samples
  • a training module configured to input the training samples into the pre-built original neural network for training, and optimize the original neural network according to a model loss function
  • the target medical imaging model determination module is configured to stop training the machine learning model when the model loss function converges, and use the trained original neural network as the target medical imaging model;
  • the original neural network includes a frequency domain network FDN and an image domain network SDN, and the frequency domain network FDN and the image domain network SDN are connected by an inverse Fourier transform IFFT;
  • the model loss function includes a main loss Function and at least one additional loss function, the additional loss function including a k-space loss function and/or an image domain loss function.
  • An embodiment of the present invention also provides a device, which includes:
  • One or more processors are One or more processors;
  • Memory for storing one or more programs
  • the one or more processors When the one or more programs are executed by the one or more processors, the one or more processors implement the method for establishing a medical imaging model according to any one of the embodiments of the present invention.
  • An embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored.
  • the program is executed by a processor, the method for establishing a medical imaging model according to any of the embodiments of the present invention is implemented.
  • the technical solution of the embodiment of the present invention acquires K-space under-sampled data of medical images as training samples, and can directly learn the mapping relationship from under-sampled images to fully sampled images. Furthermore, the training samples are input to the pre-built original neural network for training, and the original neural network is optimized according to the model loss function, so that the neural network more accurately characterizes the input samples. Furthermore, when the model loss function converges, the training of the machine learning model is stopped, and the trained original neural network is used as the target medical imaging model for reconstruction of K-space undersampled data.
  • the model loss function includes Main loss function and at least one additional loss function, the additional loss function includes a k-space loss function and/or image domain loss function, fully combines the frequency domain and image domain information, and directly learns the mapping relationship from the under-captured image to the full-captured image
  • the network information of different depths can be used more fully.
  • the above technical solution solves the traditional parallel imaging or compressed sensing technology, which does not use big data priors, is time-consuming and cumbersome to adjust parameters, and the neural network method based on deep learning cannot fully utilize the frequency domain information and cannot accurately undersample.
  • the problem of image reconstruction is realized, which can learn the characteristics of the frequency domain and the image domain at the same time, fully combine the information of the frequency domain and the image domain, and constrain the reconstruction results of different network depths, which can make full use of the network information of different depths. Reconstructing under-sampled medical images accurately can avoid time-consuming iterative solving steps and tedious parameter adjustment process.
  • FIG. 1a is a flowchart of a method for establishing a medical imaging model provided in Embodiment 1 of the present invention
  • FIG. 1b is a schematic structural diagram of an original neural network provided in Embodiment 1 of the present invention.
  • FIG. 2a is a flowchart of a method for establishing a medical imaging model provided in Embodiment 2 of the present invention
  • Example 2b is a comparison of results of different magnetic resonance cardiac film reconstruction methods provided in Example 2 of the present invention.
  • FIG. 3 is a schematic structural diagram of an apparatus for establishing a medical imaging model provided in Embodiment 3 of the present invention.
  • Embodiment 4 is a schematic structural diagram of a device provided in Embodiment 4 of the present invention.
  • FIG. 1a is a flowchart of a method for establishing a medical imaging model according to Embodiment 1 of the present invention.
  • This embodiment is applicable to the case of establishing a medical imaging model, and is particularly suitable for establishing an imaging model of K-space under-sampled data.
  • the method may be executed by a device for establishing a medical imaging model, and the device may be implemented by hardware and/or software.
  • the device may be integrated into a device (such as a computer) for execution, and specifically includes the following steps:
  • Step 101 Acquire K-space under-sampled data of a medical image as a training sample.
  • the medical image may be a magnetic resonance cardiac movie image.
  • K-space is the dual space of ordinary space under Fourier transform. It is mainly used in the imaging analysis of magnetic resonance imaging. Others such as the design of RF waveforms in magnetic resonance imaging and the preparation of the initial state in quantum computing also use the concept of k-space.
  • K-space under-sampled data refers to under-sampled K-space data.
  • This step obtains K-space undersampled data for model training.
  • Step 102 Input the training samples into the pre-built original neural network for training, and optimize the original neural network according to the model loss function.
  • the original neural network includes a frequency domain network FDN and an image domain network SDN, and the frequency domain network FDN and the image domain network SDN are connected by an inverse Fourier transform IFFT;
  • the model loss function includes a main loss function and At least one additional loss function, the additional loss function including a k-space loss function and/or an image domain loss function.
  • the additional loss function includes at least one of a k-space loss function and an image domain loss function.
  • the frequency domain network includes a first preset number of frequency domain modules Fnet, wherein each frequency domain module Fnet includes a second preset number of three-dimensional convolution layers 3D Conv and a frequency domain data consistency layer KDC .
  • the frequency domain data consistency layer KDC in the frequency domain module does not have to be included, and the frequency domain module Fnet may only include the second preset number of three-dimensional convolution layers 3D Conv.
  • the training sample is used as the input of the first three-dimensional convolutional layer of the first frequency domain module
  • the output result of the last three-dimensional convolutional layer of the frequency domain module is used as the input of the frequency domain data consistency layer KDC of the frequency domain module, and the output result of the frequency domain data consistency layer KDC is used as the frequency domain module's output Output result
  • the output result of the previous frequency domain module is used as the input of the next frequency domain module, and the output result of the last frequency domain module is used as the output result of the frequency domain network.
  • the output result of the frequency domain network FDN undergoes the inverse Fourier transform IFFT as the input of the image domain network SDN.
  • each frequency domain module contains L 3D convolutional layers (3DConv) and one frequency domain data is consistent Layer (KDC).
  • KDC 3D convolutional layers
  • the first frequency domain module (m 1):
  • ⁇ represents a nonlinear activation function, which can be a nonlinear activation function commonly used in neural network models.
  • KDC is used to perform frequency domain data consistency operations, the formula is as follows:
  • the final output of the frequency domain network FDN is Correct Then the inverse Fourier transform can be used to obtain the image domain data S 0 , which is also the input of the image domain network, as shown in the following formula (10).
  • the image domain network SDN includes a third preset number of image domain modules Snet, wherein each image domain module includes a fourth preset number of three-dimensional convolution layers 3D Conv, an image domain data consistency layer IDC And a residual connection.
  • the image domain module may only contain a fourth preset number of three-dimensional convolutional layers 3D Conv and a residual connection, but not IDC.
  • the output result of the frequency domain network after inverse Fourier transform IFFT is used as the input of the first three-dimensional convolution layer of the first image domain module;
  • the output result of the image domain data consistency layer IDC is used as the output result of the image domain module;
  • the output result of the previous image domain module is used as the input of the next image domain module, and the output result of the last image domain module is used as the output result of the image domain network.
  • the first image domain module (n 1):
  • IDC is used to perform data domain data consistency operations, the formula is as follows:
  • IDC has more conversion between the frequency domain and the image domain than KDC, that is, formula (21), and then the image domain data consistency operation is performed by formulas (22)-(23). ⁇ is used to control the degree of data consistency. It is the result of S n for IDC.
  • the frequency domain network (FDN) and the image domain network (SDN) After the frequency domain network (FDN) and the image domain network (SDN), the result of reconstructing the K-space undersampled data of medical images can be obtained.
  • the frequency domain network is used to predict the fully sampled k-space
  • the image domain network is used to extract image features
  • the two networks are connected by inverse Fourier transform.
  • both the frequency domain network and the image domain network use a data consistency layer to correct k-space data.
  • the k-space loss function is calculated according to the following formula:
  • Kloss represents k-space loss
  • K f represents full-sampled k-space data corresponding to under-sampled K-space data of medical images
  • M represents the number of frequency-domain modules
  • Each frequency domain module calculates the corresponding k-space loss.
  • the reconstruction results of different depths are constrained.
  • the image domain loss function is calculated according to the following formula:
  • Sloss represents the image domain loss
  • S represents the fully sampled medical image corresponding to the K-space undersampled data of the medical image
  • S n the output of the nth image domain module
  • N represents the number of image domain modules
  • ⁇ n the weight loss function of the image field n images corresponding domain module weight beta]
  • S n in the formula represents the output of the n-th image domain module, which needs to be adjusted according to whether there is an IDC.
  • the main loss function is calculated according to the following formula:
  • Ploss represents the main loss
  • S represents S is the fully sampled medical image corresponding to the K-space undersampled data of the medical image
  • S N represents the output of the Nth image domain module. If KDC and IDC are included, then
  • the method further includes:
  • Step 103 When the model loss function converges, stop training the machine learning model, and use the trained original neural network as the target medical imaging model.
  • the original neural network obtained after the training samples are trained as the target medical imaging model can be used to reconstruct the K-space under-sampled data of medical images.
  • an original neural network model according to an embodiment of the present invention is shown in FIG. 1b.
  • the input of the network is the k-space undersampled data k u
  • the output is the reconstructed medical image S N (the output of the image domain network, when KDC and IDC exist, it is ).
  • the original neural network consists of a frequency domain network (FDN) and an image domain network (SDN), which are connected by an inverse Fourier transform (IFFT).
  • FDN frequency domain network
  • SDN image domain network
  • IFFT inverse Fourier transform
  • Fnet frequency domain modules
  • the output of the last image domain module and the fully sampled medical image corresponding to the K-space undersampled data of the medical image are operated to obtain the main loss function Ploss.
  • the model loss function is obtained for the k-space loss function, the image domain loss function and the main loss function. Recon in the figure represents the reconstructed image.
  • the technical solution of the embodiment of the present invention acquires K-space under-sampled data of medical images as training samples, and can directly learn the mapping relationship from under-sampled images to fully sampled images. Furthermore, the training samples are input to the pre-built original neural network for training, and the original neural network is optimized according to the model loss function, so that the neural network more accurately characterizes the input samples. Furthermore, when the model loss function converges, the training of the machine learning model is stopped, and the trained original neural network is used as the target medical imaging model for reconstruction of K-space undersampled data.
  • the model loss function includes Main loss function and at least one additional loss function, the additional loss function includes a k-space loss function and/or image domain loss function, fully combines the frequency domain and image domain information, and directly learns the mapping relationship from the under-captured image to the full-captured image
  • the network information of different depths can be used more fully.
  • the above technical solution solves the traditional parallel imaging or compressed sensing technology, which does not use big data priors, is time-consuming and cumbersome to adjust parameters, and the neural network method based on deep learning cannot fully utilize the frequency domain information and cannot accurately undersample.
  • the problem of image reconstruction is realized, which can learn the characteristics of the frequency domain and the image domain at the same time, fully combine the information of the frequency domain and the image domain, and constrain the reconstruction results of different network depths, which can make full use of the network information of different depths. Reconstructing under-sampled medical images accurately can avoid time-consuming iterative solving steps and tedious parameter adjustment process.
  • FIG. 2a is a flowchart of a method for establishing a medical imaging model according to Embodiment 2 of the present invention.
  • the weighting of at least one of the additional loss functions and Summing the main loss function to obtain a model loss function including: weighting the k-space loss function and the image domain loss function respectively; weighting the main loss function, the weighted k-space loss function, and the weighted image domain
  • the loss function is summed to obtain the model loss function.
  • the method of this embodiment further includes: acquiring K-space under-sampled data to be imaged; inputting the K-space under-sampled data to be imaged into the target medical imaging model after training To get reconstructed medical images.
  • Step 201 Acquire K-space under-sampled data of a medical image as a training sample.
  • Step 202 Input the training samples to the pre-built original neural network for training, and optimize the original neural network according to the model loss function.
  • the original neural network includes a frequency domain network FDN and an image domain network SDN, and the frequency domain network FDN and the image domain network SDN are connected by an inverse Fourier transform IFFT;
  • the model loss function includes a main loss Function and at least one additional loss function, the additional loss function including a k-space loss function and/or an image domain loss function.
  • the weighting at least one of the additional loss functions and performing a sum operation with the main loss function to obtain a model loss function includes: separately weighting the k-space loss function and the image domain loss function; Summing the main loss function, the weighted k-space loss function and the weighted image domain loss function to obtain the model loss function.
  • model loss function can be expressed by the following formula:
  • Tloss is the model loss calculated by the above formula
  • S represents the fully sampled medical image corresponding to the K-space undersampled data of the medical image
  • S n represents the output of the nth image domain module
  • N represents the number of image domain modules
  • ⁇ n represents the weight of the image domain loss function corresponding to the n-th image domain module.
  • K f represents the fully sampled k-space data corresponding to the K-space under-sampled data of the medical image
  • M represents the number of frequency-domain modules
  • ⁇ m represents the weight of the k-space loss function corresponding to the m-th frequency domain module.
  • the calculation formula of the above-mentioned Tloss is a calculation formula of the loss of a single training sample.
  • the above formula is used to calculate and sum the losses of each training sample separately as the loss of the model.
  • Step 203 Determine whether the model loss function converges. If yes, go to step 204, if no, go back to step 202.
  • the preset threshold can be set based on empirical values.
  • Step 204 Stop training the machine learning model, and use the trained original neural network as the target medical imaging model.
  • Step 205 Acquire K-space under-sampled data to be imaged.
  • Step 206 Input the K-space under-sampled data to be imaged into the target medical imaging model after training to obtain a reconstructed medical image.
  • the method of the embodiments of the present invention is compared with the current mainstream compressed sensing and deep learning methods.
  • the results of different magnetic resonance cardiac film reconstruction methods are compared, as shown in Figure 2b.
  • Figure 2b shows a comparison of the results of different magnetic resonance cardiac film reconstruction methods.
  • the methods are: the time-frequency sparsity of the focal underdetermination system ktFOFOSS, the dynamic redundancy Kalki method kt SLR, the low-rank sparse matrix L+S, the magnetic resonance dynamic imaging D5C5 based on the cascaded convolution network, and this embodiment The method provided.
  • the technical solution of this embodiment obtains a model loss function by summing at least one of the additional loss function and the main loss function, including: separately weighting the k-space loss function and the image domain loss function The sum of the main loss function, the weighted k-space loss function and the weighted image domain loss function are summed to obtain the model loss function, which constrains the reconstruction results of different depths and provides more training for the entire network.
  • each supervision loss function provides guidance for network training.
  • FIG. 3 is a schematic structural diagram of a medical imaging model building device provided in Embodiment 3 of the present invention.
  • the apparatus for establishing a medical imaging model provided by an embodiment of the present invention can execute the method for establishing a medical imaging model provided by any embodiment of the present invention.
  • the specific structure of the apparatus is as follows: a training sample acquisition module 31, a training module 32, and target medical imaging Model determination module 33.
  • the training sample acquisition module 31 is set to acquire K-space under-sampled data of medical images as training samples.
  • the training module 32 is configured to input the training samples to the pre-built original neural network for training, and optimize the original neural network according to a model loss function.
  • the target medical imaging model determination module 33 is configured to stop training the machine learning model when the model loss function converges, and use the trained original neural network as the target medical imaging model.
  • the original neural network includes a frequency domain network FDN and an image domain network SDN, and the frequency domain network FDN and the image domain network SDN are connected by an inverse Fourier transform IFFT;
  • the model loss function includes a main loss Function and at least one additional loss function, the additional loss function including a k-space loss function and/or an image domain loss function.
  • the technical solution of the embodiment of the present invention acquires K-space under-sampled data of medical images as training samples, and can directly learn the mapping relationship from under-sampled images to fully sampled images. Furthermore, the training samples are input to the pre-built original neural network for training, and the original neural network is optimized according to the model loss function, so that the neural network more accurately characterizes the input samples. Furthermore, when the model loss function converges, the training of the machine learning model is stopped, and the trained original neural network is used as the target medical imaging model for reconstruction of K-space undersampled data.
  • the model loss function includes Main loss function and at least one additional loss function, the additional loss function includes k-space loss function and/or image domain loss function, fully combines the frequency domain and image domain information, and directly learns the mapping relationship from the under-captured image to the full-captured image
  • the network information of different depths can be used more fully.
  • the above technical solution solves the traditional parallel imaging or compressed sensing technology, which does not use big data priors, is time-consuming and cumbersome to adjust parameters, and the neural network method based on deep learning cannot fully utilize the frequency domain information and cannot accurately undersample.
  • the problem of image reconstruction is realized, which can learn the characteristics of the frequency domain and the image domain at the same time, fully combine the information of the frequency domain and the image domain, and constrain the reconstruction results of different network depths, which can make full use of the network information of different depths. Reconstructing under-sampled medical images accurately can avoid time-consuming iterative solving steps and tedious parameter adjustment process.
  • the training module 32 may be specifically set as follows: the frequency domain network includes a first preset number of frequency domain modules Fnet, and each frequency domain module Fnet includes a second preset number of three-dimensional convolutions Layer 3D Conv and a frequency domain data consistency layer KDC.
  • the training module 32 may be specifically configured to calculate the k-space loss function according to the following formula:
  • Kloss represents k-space loss
  • K f represents full-sampled k-space data corresponding to under-sampled K-space data of medical images
  • M represents the number of frequency-domain modules
  • ⁇ m represents the weight of the k-space loss function corresponding to the m-th frequency domain module.
  • the training module 32 may specifically be set as follows: the image domain network SDN includes a third preset number of image domain modules Snet, wherein each image domain module contains a fourth preset number of three-dimensional Convolutional layer 3D Conv, an image domain data consistency layer IDC and a residual connection.
  • the training module 32 may be specifically configured to calculate the image domain loss function according to the following formula:
  • Sloss represents the image domain loss
  • S represents the fully sampled medical image corresponding to the K-space undersampled data of the medical image
  • S n represents the output of the nth image domain module
  • N represents the number of image domain modules
  • ⁇ n represents the The weight of the image domain loss function corresponding to n image domain modules.
  • the device for establishing a medical imaging model further includes a weighting module.
  • the weighting module is configured to weight at least one of the additional loss functions and perform a sum operation with the main loss function to obtain a model loss function.
  • the weighting module may be specifically configured to: add the weighted at least one of the additional loss functions and perform a sum operation with the main loss function to obtain a model loss function, including:
  • the device for establishing the medical imaging model further includes: a reconstruction module.
  • the reconstruction module is set to obtain K-space under-sampled data to be imaged
  • the K-space under-sampled data to be imaged is input into the target medical imaging model after training to obtain a reconstructed medical image.
  • the apparatus for establishing a medical imaging model provided by an embodiment of the present invention can execute the method for establishing a medical imaging model provided by any embodiment of the present application, and has functional modules and beneficial effects corresponding to the execution method.
  • the device includes a processor 40, a memory 41, an input device 42, and an output device 43; the number of processors 40 in the device may be One or more, one processor 40 is taken as an example in FIG. 4; the processor 40, the memory 41, the input device 42 and the output device 43 in the device may be connected through a bus or other means, and FIG. 4 is taken as an example through a bus connection .
  • the memory 41 is a computer-readable storage medium that can be used to store software programs, computer executable programs, and modules, such as program instructions/modules corresponding to the method of establishing a medical imaging model in the embodiment of the present invention (for example, the The training sample acquisition module 31, the training module 32 and the target medical imaging model determination module 33 in the establishment device).
  • the processor 40 executes various functional applications and data processing of the device by running software programs, instructions, and modules stored in the memory 41, that is, implementing the above-described method of establishing a medical imaging model.
  • the memory 41 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system and application programs required by at least one function; the storage data area may store data created according to the use of the terminal, and the like.
  • the memory 41 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices.
  • the memory 41 may further include memories remotely provided with respect to the processor 40, and these remote memories may be connected to the device through a network. Examples of the aforementioned network include, but are not limited to, the Internet, intranet, local area network, mobile communication network, and combinations thereof.
  • the input device 42 can be used to receive the K-space under-sampled data of the input medical image and generate signal input related to the user settings and function control of the device.
  • the output device 43 may include a display device such as a display screen.
  • Embodiment 5 of the present invention also provides a storage medium containing computer-executable instructions, which when executed by a computer processor is used to perform a method for establishing a medical imaging model, the method includes:
  • the original neural network includes a frequency domain network FDN and an image domain network SDN, and the frequency domain network FDN and the image domain network SDN are connected by an inverse Fourier transform IFFT;
  • the model loss function includes a main loss Function and at least one additional loss function, the additional loss function including a k-space loss function and/or an image domain loss function.
  • a storage medium containing computer-executable instructions provided by an embodiment of the present invention is not limited to the method operations described above, but can also execute the medical imaging model provided by any embodiment of the present application. Related operations in the establishment method.
  • the present application can be implemented by software and necessary general hardware, and of course can also be implemented by hardware, but in many cases the former is a better embodiment .
  • the technical solutions of the present application can essentially be embodied in the form of software products, and the computer software products can be stored in computer-readable storage media, such as computer floppy disks, Read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), flash memory (FLASH), hard disk or optical disc, etc., including several instructions to make a computer device (which can be a personal computer, Server, or network equipment, etc.) to execute the method described in each embodiment of the present application.
  • a computer device which can be a personal computer, Server, or network equipment, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Epidemiology (AREA)
  • Medical Informatics (AREA)
  • Primary Health Care (AREA)
  • Public Health (AREA)
  • Image Analysis (AREA)

Abstract

一种医学成像模型的建立方法、装置、设备及存储介质。该方法包括：获取医学图像的K空间欠采样数据作为训练样本(101)；将训练样本输入至预先构建的原始神经网络进行训练，并根据模型损失函数优化原始神经网络(102)；当模型损失函数收敛时，停止训练机器学习模型，将训练完成的原始神经网络作为目标医学成像模型(103)；其中，原始神经网络包括频率域网络和图像域网络；模型损失函数包括主损失函数以及至少一个附加损失函数，附加损失函数包括k空间损失函数和/或图像域损失函数。

Description

一种医学成像模型的建立方法、装置、设备及存储介质
本申请要求在2018年12月27日提交中国专利局、申请号为201811654047.4的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本申请涉及磁共振成像技术领域，例如涉及一种医学成像模型的建立方法、装置、设备及存储介质。
背景技术
磁共振心脏电影成像是一种非侵入式的成像技术，能够为临床诊断提供丰富的空间和时间信息。由于磁共振物理及硬件的制约，磁共振心脏电影成像往往伴随着扫描时间长及成像速度慢等缺点。因此，如何在保证成像质量的前提下，加速磁共振心脏电影成像尤为重要。
相关技术中，常用的加速磁共振心脏电影成像的方法，包括并行成像、压缩感知技术、深度学习的方法等。例如，动态广义自动校准部分并行采集(TGRAPPA)、利用时间滤波器的自适应敏感度编码(TSENSE)、利用时间频率稀疏性的焦欠定系统(k-t FOCUSS)、利用动态冗余的卡尔基方法(k-t SLR)、低秩稀疏矩阵(L+S)等。在磁共振心脏电影成像领域，基于级联卷积网络的磁共振动态成像(D5C5)及卷积递归神经网络(CRNN)取得了良好的重建效果。这两种方法利用神经网络，直接学习从欠采图像到全采图像的映射关系。
传统的并行成像或者压缩感知技术，没有利用大数据先验，并且这种迭代优化方法往往是耗时的且参数较难选择。而基于深度学习的神经网络方法(D5C5、CRNN)也存在明显的不足。这两种深度模型只运用了一个损失函数用于训练整个网络，网络的中间结果并没有加以约束，无法更准确地优化神经 网络模型，最终无法更准确地对磁共振心脏电影成像进行重建。
发明内容
本发明实施例提供了一种医学成像模型的建立方法、装置、设备及存储介质，以实现对网络的中间结果加以约束，更准确、更快速地对K空间欠采样医学图像进行重建。
本发明实施例提供了一种医学成像模型的建立方法，该方法包括：
获取医学图像的K空间欠采样数据作为训练样本；
将所述训练样本输入至预先构建的所述原始神经网络进行训练，并根据模型损失函数优化所述原始神经网络；
当所述模型损失函数收敛时，停止训练所述机器学习模型，将训练完成的所述原始神经网络作为目标医学成像模型；
其中，所述原始神经网络包括频率域网络FDN以及图像域网络SDN，所述频率域网络FDN与所述图像域网络SDN之间通过傅里叶逆变换IFFT连接；所述模型损失函数包括主损失函数以及至少一个附加损失函数，所述附加损失函数包括k空间损失函数和/或图像域损失函数。
本发明实施例还提供了一种医学成像模型的建立装置，该装置包括：
训练样本获取模块，设置为获取医学图像的K空间欠采样数据作为训练样本；
训练模块，设置为将所述训练样本输入至预先构建的所述原始神经网络进行训练，并根据模型损失函数优化所述原始神经网络；
目标医学成像模型确定模块，设置为当所述模型损失函数收敛时，停止训练所述机器学习模型，将训练完成的所述原始神经网络作为目标医学成像模型；
其中，所述原始神经网络包括频率域网络FDN以及图像域网络SDN，所述频率域网络FDN与所述图像域网络SDN之间通过傅里叶逆变换IFFT连接；所述模型损失函数包括主损失函数以及至少一个附加损失函数，所述附加损失函数包括k空间损失函数和/或图像域损失函数。
本发明实施例还提供了一种设备，该设备包括：
一个或多个处理器；
存储器，用于存储一个或多个程序，
当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现本发明实施例中任一所述的医学成像模型的建立方法。
本发明实施例还提供了一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现本发明实施例中任一所述的医学成像模型的建立方法。
本发明实施例的技术方案获取医学图像的K空间欠采样数据作为训练样本，能够直接学习从欠采样图像到全采样图像的映射关系。进而，将所述训练样本输入至预先构建的所述原始神经网络进行训练，并根据模型损失函数优化所述原始神经网络，使神经网络更加准确地表征输入的样本。进而，当所述模型损失函数收敛时，停止训练所述机器学习模型，将训练完成的所述原始神经网络作为目标医学成像模型，用于K空间欠采样数据的重建。进而，其中，所述原始神经网络包括频率域网络FDN以及图像域网络SDN，所述频率域网络FDN与所述图像域网络SDN之间通过傅里叶逆变换IFFT连接；所述模型损失函数包括主损失函数以及至少一个附加损失函数，所述附加损失函数包括k空间损失函数和/或图像域损失函数，充分结合频率域与图像域信息，直接学习从欠采图像到全采图像的映射关系，对不同网络深度的重建结果加以约束，能够 更加充分地利用不同深度的网络信息。上述技术方案解决了传统的并行成像或者压缩感知技术，没有利用大数据先验，耗时且调参繁琐、基于深度学习的神经网络方法无法充分地利用频率域信息，无法更准确地对欠采样的图像进行重建的问题，实现能够同时学习频率域与图像域特征，充分结合频率域与图像域信息，对不同网络深度的重建结果加以约束，能够更加充分地利用不同深度的网络信息，快速、准确地对欠采样的医学图像进行重建，能够避免耗时的迭代求解步骤以及繁琐的调参过程。
附图说明
图1a是本发明实施例一中提供的一种医学成像模型的建立方法的流程图；
图1b是本发明实施例一中提供的一种原始神经网络的结构示意图；
图2a是本发明实施例二中提供的一种医学成像模型的建立方法的流程图；
图2b是本发明实施例二中提供的不同磁共振心脏电影重建方法的结果比较；
图3是本发明实施例三中提供的一种医学成像模型的建立装置的结构示意图；
图4是本发明实施例四中的提供的一种设备的结构示意图。
具体实施方式
下面结合附图和实施例对本申请作进一步的详细说明。可以理解的是，此处所描述的具体实施例仅仅用于解释本申请，而非对本申请的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与本申请相关的部分而非全部结构。
实施例一
图1a为本发明实施例一提供的医学成像模型的建立方法的流程图，本实施例可适用于建立医学成像模型的情况，尤其适用于建立K空间欠采样数据的成像模型。该方法可以由医学成像模型的建立装置来执行，该装置可以由硬件和/或软件来实现，该装置可集成于设备(例如计算机)中来执行，具体包括如下步骤：
步骤101、获取医学图像的K空间欠采样数据作为训练样本。
示例性地，医学图像可以是磁共振心脏电影图像。K空间是寻常空间在傅利叶转换下的对偶空间，主要应用在磁振造影的成像分析，其他如磁振造影中的射频波形设计，以及量子计算中的初始态准备亦用到k空间的概念。
K空间欠采样数据是指欠采样的K空间数据。
该步骤获取K空间欠采样数据用于模型的训练。
步骤102、将所述训练样本输入至预先构建的所述原始神经网络进行训练，并根据模型损失函数优化所述原始神经网络。
所述原始神经网络包括频率域网络FDN以及图像域网络SDN，所述频率域网络FDN与所述图像域网络SDN之间通过傅里叶逆变换IFFT连接；所述模型损失函数包括主损失函数以及至少一个附加损失函数，所述附加损失函数包括k空间损失函数和/或图像域损失函数。
所述附加损失函数包括k空间损失函数和图像域损失函数的至少一个。
可选地，所述频率域网络包括第一预设数量的频率域模块Fnet，其中，每个频率域模块Fnet包含第二预设数量的三维卷积层3D Conv以及一个频率域数据一致层KDC。
频率域模块中的频率域数据一致层KDC不是必须包括的，频率域模块Fnet可以只包含第二预设数量的三维卷积层3D Conv。
可选地，将所述训练样本作为第一个频率域模块的第一个三维卷积层的输入；
将所述频率域模块的前一个三维卷积层的输出结果作为所述频率域模块的下一个三维卷积层的输入；
将所述频率域模块的最后一个三维卷积层的输出结果作为所述频率域模块频率域数据一致层KDC的输入，将所述频率域数据一致层KDC的输出结果作为所述频率域模块的输出结果；
将前一频率域模块的输出结果作为下一个频率域模块的输入，将最后一个频率域模块的输出结果作为所述频率域网络的输出结果。
频率域网络FDN的输出结果经过傅里叶逆变换IFFT的结果作为图像域网络SDN的输入。
假设频率域网络由M个频率域模块Fnet m(Fnet m,m＝1,...,M)构成，每个频率域模块包含L个3维卷积层(3DConv)及一个频率域数据一致层(KDC)。频率域网络的前向过程可由如下公式表示：
第一个频率域模块(m＝1)：
Figure PCTCN2019124239-appb-000001
后续的频率域模块(m＝2,...,M)
Figure PCTCN2019124239-appb-000002
其中，σ表示非线性激活函数，可以是神经网络模型中常用的非线性激活函数。k u表示输入的医学图像的K空间欠采样数据，
Figure PCTCN2019124239-appb-000003
分别是第m个频率域模块中第l个卷积层的卷积核和偏置项，l＝1,...,L,m＝1,...,M。
Figure PCTCN2019124239-appb-000004
表示第m个频率域模块中第l个卷积层的输出。每个频率域模块除了最后一个卷积层，其余每个频率域模块的所有卷积层均由非线性激活函数σ进行激活。经过卷积层进行提取特征后，即得到
Figure PCTCN2019124239-appb-000005
后，利用频率域数据一致层KDC来纠正网络预测的k空间，如公式(9)所示。
KDC用于执行频率域数据一致操作，公式如下：
Figure PCTCN2019124239-appb-000006
Figure PCTCN2019124239-appb-000007
表示对
Figure PCTCN2019124239-appb-000008
进行纠正的结果。令所有已采集的医学图像的K空间欠采样数据坐标构成的集合为Ω。如果k空间坐标(k x,k y)在集合Ω内，则
Figure PCTCN2019124239-appb-000009
将通过真实采集的k空间点进行纠正。λ用于控制数据一致的程度，如果λ→∞，可以直接将实际采样点去替代
Figure PCTCN2019124239-appb-000010
对应的点。
频率域网络FDN最终的输出是
Figure PCTCN2019124239-appb-000011
对
Figure PCTCN2019124239-appb-000012
再进行傅里叶逆变换便可以得到图像域的数据S 0，它也是图像域网络的输入，如以下公式(10)。
Figure PCTCN2019124239-appb-000013
如果频率域模块不包括KDC,则
Figure PCTCN2019124239-appb-000014
作为频率域网络FDN最终的输出。
可选地，所述图像域网络SDN包括第三预设数量的图像域模块Snet，其中， 每个图像域模块包含第四预设数量的三维卷积层3D Conv、一个图像域数据一致层IDC以及一个残差连接。
图像域模块可以只包含第四预设数量的三维卷积层3D Conv以及一个残差连接，不包含IDC。
可选地，将所述频率域网络的输出结果经过傅里叶逆变换IFFT后的结果作为第一个图像域模块的第一个三维卷积层的输入；
将所述图像域模块的前一个三维卷积层的输出结果作为所述图像域模块的下一个三维卷积层的输入；
将所述图像域模块的最后一个三维卷积层的输出结果与所述图像域模块的第一个三维卷积层的输入进行求和运算后，输入所述图像域模块图像域数据一致层IDC，将所述图像域数据一致层IDC的输出结果作为所述图像域模块的输出结果；
将前一图像域模块的输出结果作为下一个图像域模块的输入，将最后一个图像域模块的输出结果作为所述图像域网络的输出结果。
图像域网络的前向过程可由如下公式表示，假设图像域网络包含N个图像域模块Snet n(Snet n,n＝1,...,N)：
第一个图像域模块(n＝1)：
Figure PCTCN2019124239-appb-000015
后续的图像域模块(n＝2,...,N)
Figure PCTCN2019124239-appb-000016
IDC用于执行图像域数据一致操作，公式如下：
Figure PCTCN2019124239-appb-000017
Figure PCTCN2019124239-appb-000018
Figure PCTCN2019124239-appb-000019
Figure PCTCN2019124239-appb-000020
分别是第n个图像域模块中第l个卷积层的卷积核和偏置项，l＝1,...,L,n＝1,...,N。
Figure PCTCN2019124239-appb-000021
是第n个图像域模块中第l个卷积层的输出。除了最后一个(第L个)卷积层，其余所有卷积层均由非线性激活函数σ进行激活。经过卷积层进行提取特征后，引入残差学习(通过残差连接实现)，公式(19)中的S n是残差学习(即残差连接)的结果。然后对S n进行图像域数据一致操作(IDC)。IDC比KDC多了频率域与图像域之间转换，即公式(21)，然后通过式(22)-(23)进行图像域数据一致操作。λ用于控制数据一致的程度。
Figure PCTCN2019124239-appb-000022
是对S n进行IDC后的结果。
经过频率域网络(FDN)及图像域网络(SDN)之后，可以得到对医学图像的K空间欠采样数据进行重建的结果。
频率域网络用于预测全采样的k空间，图像域网络用于提取图像特征，两个网络通过傅里叶逆变换进行连接。同时，频率域网络和图像域网络均使用了数据一致层，用于纠正k空间数据。
可选地，根据如下公式计算所述k空间损失函数：
Figure PCTCN2019124239-appb-000023
其中，Kloss表示k空间损失，K f表示医学图像的K空间欠采样数据对应的全采样k空间数据，
Figure PCTCN2019124239-appb-000024
表示第m个频率域模块的输出，M表示频率域模块的数量，α m表示第m个频率域模块对应的k空间损失函数的权重，m＝1,…,M。每个频率域模块都会计算相应的k空间损失，M个频率域模块的损失相加构成频率域网络损失(k空间损失)，可以Kloss m(m＝1,…,M)表示，能够实现对不同深度的重建结果进行约束。
可以理解的是，公式中的
Figure PCTCN2019124239-appb-000025
(第m个频率域模块的输出)需要根据是否有KDC进行调整。
可选地，根据如下公式计算所述图像域损失函数：
Figure PCTCN2019124239-appb-000026
其中，Sloss表示图像域损失，S表示医学图像的K空间欠采样数据对应的全采样的医学图像，S n表示第n个图像域模块的输出，N表示图像域模块的数量，β n表示第n个图像域模块对应的图像域损失函数的权重，可以根据重建结果对β n进行设置，n＝1,…,N-1。每个图像域模块都会计算相应的损失，用Lloss n(n＝1,…,N-1)表示，N-1个图像域模块的损失函数相加构成图像域损失函数。
可以理解的是，公式中的S n表示第n个图像域模块的输出，需要根据是否有IDC进行调整。
可选地，根据如下公式计算所述主损失函数：
Figure PCTCN2019124239-appb-000027
其中，Ploss表示主损失，S表示S是医学图像的K空间欠采样数据对应的全采样的医学图像，S N表示第N个图像域模块的输出，如果包含KDC和IDC,则
Figure PCTCN2019124239-appb-000028
可选地，所述方法还包括：
将至少一项所述附加损失函数加权后与所述主损失函数进行求和运算得到模型损失函数。
步骤103、当所述模型损失函数收敛时，停止训练所述机器学习模型，将训练完成的所述原始神经网络作为目标医学成像模型。
其中，训练样本经过训练后得到的原始神经网络作为目标医学成像模型，可以用于对医学图像的K空间欠采样数据进行重建。
示例性地，本发明实施例的一种原始神经网络模型如图1b所示。网络的输入是K空间欠采样数据k u，输出是重建的医学图像S N(图像域网络的输出，当存在KDC和IDC时，为
Figure PCTCN2019124239-appb-000029
)。原始神经网络由频率域网络(FDN)及图像域网络(SDN)组成，两者通过傅里叶逆变换(IFFT)连接。其中频率域网络由M个频率域模块(Fnet)构成，每个频率域模块包含第二预设数量的3维卷积层(3D Conv)及1个频率域数据一致层(也可以不包括频率域数据一致层)，每个频率域模块的损失用Kloss m(m＝1,…,M)表示，M个频率域模块的损失相加构成频率域网络损失(k空间损失)。
图像域网络由N个图像域模块(Snet)构成，每个图像域模块包含第四预设数量的3维卷积层(3D Conv)、一个图像域数据一致层(也可以不包括图像域数据一致层)及一个残差连接，每个图像域模块的损失用Lloss n，(n＝1,…,N-1)表示，N-1个图像域模块的损失函数相加构成图像域损失函数。最后一个图像域模块的输出与医学图像的K空间欠采样数据对应的全采样的医学图像进行运算 得到主损失函数Ploss。对k空间损失函数、图像域损失函数和主损失函数得到模型损失函数。图中的Recon表示重建后图像。
本发明实施例的技术方案获取医学图像的K空间欠采样数据作为训练样本，能够直接学习从欠采样图像到全采样图像的映射关系。进而，将所述训练样本输入至预先构建的所述原始神经网络进行训练，并根据模型损失函数优化所述原始神经网络，使神经网络更加准确地表征输入的样本。进而，当所述模型损失函数收敛时，停止训练所述机器学习模型，将训练完成的所述原始神经网络作为目标医学成像模型，用于K空间欠采样数据的重建。进而，其中，所述原始神经网络包括频率域网络FDN以及图像域网络SDN，所述频率域网络FDN与所述图像域网络SDN之间通过傅里叶逆变换IFFT连接；所述模型损失函数包括主损失函数以及至少一个附加损失函数，所述附加损失函数包括k空间损失函数和/或图像域损失函数，充分结合频率域与图像域信息，直接学习从欠采图像到全采图像的映射关系，对不同网络深度的重建结果加以约束，能够更加充分地利用不同深度的网络信息。上述技术方案解决了传统的并行成像或者压缩感知技术，没有利用大数据先验，耗时且调参繁琐、基于深度学习的神经网络方法无法充分地利用频率域信息，无法更准确地对欠采样的图像进行重建的问题，实现能够同时学习频率域与图像域特征，充分结合频率域与图像域信息，对不同网络深度的重建结果加以约束，能够更加充分地利用不同深度的网络信息，快速、准确地对欠采样的医学图像进行重建，能够避免耗时的迭代求解步骤以及繁琐的调参过程。
实施例二
图2a为本发明实施例二提供的一种医学成像模型的建立方法的流程图，本 实施例在上述实施例的基础上，可选是所述将至少一项所述附加损失函数加权后与所述主损失函数进行求和运算得到模型损失函数，包括：分别对k空间损失函数和图像域损失函数进行加权；将所述主损失函数、加权后的k空间损失函数以及加权后的图像域损失函数进行求和运算得到模型损失函数。
在此基础上，一些实施例中，本实施例的方法还包括：获取待成像的K空间欠采样数据；将所述待成像的K空间欠采样数据输入训练完成的所述目标医学成像模型中，得到重建医学图像。
如图2a所示，具体包括以下步骤：
步骤201、获取医学图像的K空间欠采样数据作为训练样本。
步骤202、将所述训练样本输入至预先构建的所述原始神经网络进行训练，并根据模型损失函数优化所述原始神经网络。
其中，所述原始神经网络包括频率域网络FDN以及图像域网络SDN，所述频率域网络FDN与所述图像域网络SDN之间通过傅里叶逆变换IFFT连接；所述模型损失函数包括主损失函数以及至少一个附加损失函数，所述附加损失函数包括k空间损失函数和/或图像域损失函数。
可选地，将至少一项所述附加损失函数加权后与所述主损失函数进行求和运算得到模型损失函数。
可选地，所述将至少一项所述附加损失函数加权后与所述主损失函数进行求和运算得到模型损失函数，包括：分别对k空间损失函数和图像域损失函数进行加权；将所述主损失函数、加权后的k空间损失函数以及加权后的图像域损失函数进行求和运算得到模型损失函数。
例如，模型损失函数可以用如下公式进行表示：
Figure PCTCN2019124239-appb-000030
其中，Tloss为通过上述公式计算得到的模型损失，S表示医学图像的K空间欠采样数据对应的全采样的医学图像，S n表示第n个图像域模块的输出，N表示图像域模块的数量，β n表示第n个图像域模块对应的图像域损失函数的权重。K f表示医学图像的K空间欠采样数据对应的全采样k空间数据，
Figure PCTCN2019124239-appb-000031
表示第m个频率域模块的输出，M表示频率域模块的数量，α m表示第m个频率域模块对应的k空间损失函数的权重。
可以理解的是，上述Tloss的计算公式为单个训练样本的损失计算公式，当多个训练样本同时训练时，使用上述公式分别对各训练样本计算损失并求和，作为模型的损失。
步骤203、判断所述模型损失函数是否收敛。若是，执行步骤204，若否，返回执行步骤202。
即判断模型损失函数计算的损失是否达到预设阈值。预设阈值可以根据经验值设置。
步骤204、停止训练所述机器学习模型，将训练完成的所述原始神经网络作为目标医学成像模型。
步骤205、获取待成像的K空间欠采样数据。
步骤206、将所述待成像的K空间欠采样数据输入训练完成的所述目标医学成像模型中，得到重建医学图像。
以磁共振心脏电影成像为例，为了展示本发明实施例对磁共振心脏电影成像的有效性，将本发明实施例的方法与目前主流的压缩感知及深度学习方法进行对比。不同磁共振心脏电影重建方法的结果比较，如图2b所示。图2b表示了不同磁共振心脏电影重建方法的结果比较。方法分别为：时间频率稀疏性的焦欠定系统k-t FOCUSS、动态冗余的卡尔基方法k-t SLR、低秩稀疏矩阵L+S、基 于级联卷积网络的磁共振动态成像D5C5以及本实施例提供的方法。(a)表示全采样图像，(b)表示零填充图像，(c)表示采样模板及时间维度的展开，(d)表示k-t FOCUSS的重建结果，(e)表示k-t SLR的重建结果，(f)表示L+S的重建结果，(g)表示D5C5的重建结果，(h)本实施例提出的方法的重建结果；(i)，(j)，(k)，(l)和(m)分别是(d)，(e)，(f)，(g)和(h)对应的的重建结果与全采样图像(a)间的残差图。其中，残差越小，表示重建效果越好。从实验结果可以看出，本发明实施例的方法对磁共振心脏电影成像具有最好的重建结果。这充分说明了本发明实施例的方法的有效性。
本实施例的技术方案通过所述将至少一项所述附加损失函数加权后与所述主损失函数进行求和运算得到模型损失函数，包括：分别对k空间损失函数和图像域损失函数进行加权；将所述主损失函数、加权后的k空间损失函数以及加权后的图像域损失函数进行求和运算得到模型损失函数，实现对不同深度的重建结果进行约束，为整个网络的训练提供了多个监督，每个监督损失函数都为网络的训练提供指导。进而，获取待成像的K空间欠采样数据；将所述待成像的K空间欠采样数据输入训练完成的所述目标医学成像模型中，得到重建医学图像，实现更快速、更准确地对K空间欠采样数据进行重建。
实施例三
图3是本发明实施例三中提供的一种医学成像模型的建立装置的结构示意图。本发明实施例所提供的医学成像模型的建立装置可执行本发明任意实施例所提供的医学成像模型的建立方法，该装置的具体结构如下：训练样本获取模块31、训练模块32和目标医学成像模型确定模块33。
训练样本获取模块31，设置为获取医学图像的K空间欠采样数据作为训练 样本。
训练模块32，设置为将所述训练样本输入至预先构建的所述原始神经网络进行训练，并根据模型损失函数优化所述原始神经网络。
目标医学成像模型确定模块33，设置为当所述模型损失函数收敛时，停止训练所述机器学习模型，将训练完成的所述原始神经网络作为目标医学成像模型。
其中，所述原始神经网络包括频率域网络FDN以及图像域网络SDN，所述频率域网络FDN与所述图像域网络SDN之间通过傅里叶逆变换IFFT连接；所述模型损失函数包括主损失函数以及至少一个附加损失函数，所述附加损失函数包括k空间损失函数和/或图像域损失函数。
本发明实施例的技术方案获取医学图像的K空间欠采样数据作为训练样本，能够直接学习从欠采样图像到全采样图像的映射关系。进而，将所述训练样本输入至预先构建的所述原始神经网络进行训练，并根据模型损失函数优化所述原始神经网络，使神经网络更加准确地表征输入的样本。进而，当所述模型损失函数收敛时，停止训练所述机器学习模型，将训练完成的所述原始神经网络作为目标医学成像模型，用于K空间欠采样数据的重建。进而，其中，所述原始神经网络包括频率域网络FDN以及图像域网络SDN，所述频率域网络FDN与所述图像域网络SDN之间通过傅里叶逆变换IFFT连接；所述模型损失函数包括主损失函数以及至少一个附加损失函数，所述附加损失函数包括k空间损失函数和/或图像域损失函数，充分结合频率域与图像域信息，直接学习从欠采图像到全采图像的映射关系，对不同网络深度的重建结果加以约束，能够更加充分地利用不同深度的网络信息。上述技术方案解决了传统的并行成像或者压缩感知技术，没有利用大数据先验，耗时且调参繁琐、基于深度学习的神 经网络方法无法充分地利用频率域信息，无法更准确地对欠采样的图像进行重建的问题，实现能够同时学习频率域与图像域特征，充分结合频率域与图像域信息，对不同网络深度的重建结果加以约束，能够更加充分地利用不同深度的网络信息，快速、准确地对欠采样的医学图像进行重建，能够避免耗时的迭代求解步骤以及繁琐的调参过程。
在上述技术方案的基础上，训练模块32，具体可设置为：所述频率域网络包括第一预设数量的频率域模块Fnet，每个频率域模块Fnet包含第二预设数量的三维卷积层3D Conv以及一个频率域数据一致层KDC。
在上述技术方案的基础上，训练模块32，具体可设置为：根据如下公式计算所述k空间损失函数：
Figure PCTCN2019124239-appb-000032
其中，Kloss表示k空间损失，K f表示医学图像的K空间欠采样数据对应的全采样k空间数据，
Figure PCTCN2019124239-appb-000033
表示第m个频率域模块的输出，M表示频率域模块的数量，α m表示第m个频率域模块对应的k空间损失函数的权重。
在上述技术方案的基础上，训练模块32，具体可设置为：所述图像域网络SDN包括第三预设数量的图像域模块Snet，其中，每个图像域模块包含第四预设数量的三维卷积层3D Conv、一个图像域数据一致层IDC以及一个残差连接。
在上述技术方案的基础上，训练模块32，具体可设置为：根据如下公式计算所述图像域损失函数：
Figure PCTCN2019124239-appb-000034
其中，Sloss表示图像域损失，S表示医学图像的K空间欠采样数据对应的全采样的医学图像，S n表示第n个图像域模块的输出，N表示图像域模块的数 量，β n表示第n个图像域模块对应的图像域损失函数的权重。
在上述技术方案的基础上，医学成像模型的建立装置还包括加权模块。加权模块，设置为将至少一项所述附加损失函数加权后与所述主损失函数进行求和运算得到模型损失函数。
在上述技术方案的基础上，加权模块，具体可设置为：所述将至少一项所述附加损失函数加权后与所述主损失函数进行求和运算得到模型损失函数，包括：
分别对k空间损失函数和图像域损失函数进行加权；
将所述主损失函数、加权后的k空间损失函数以及加权后的图像域损失函数进行求和运算得到模型损失函数。
在上述技术方案的基础上，医学成像模型的建立装置还包括：重建模块。
重建模块，设置为获取待成像的K空间欠采样数据；
将所述待成像的K空间欠采样数据输入训练完成的所述目标医学成像模型中，得到重建医学图像。
本发明实施例所提供的医学成像模型的建立装置可执行本申请任一实施例所提供的医学成像模型的建立方法，具备执行方法相应的功能模块和有益效果。
实施例四
图4为本发明实施例四提供的一种设备的结构示意图，如图4所示，该设备包括处理器40、存储器41、输入装置42和输出装置43；设备中处理器40的数量可以是一个或多个，图4中以一个处理器40为例；设备中的处理器40、存储器41、输入装置42和输出装置43可以通过总线或其他方式连接，图4中以通过总线连接为例。
存储器41作为一种计算机可读存储介质，可用于存储软件程序、计算机可执行程序以及模块，如本发明实施例中的医学成像模型的建立方法对应的程序指令/模块(例如，医学成像模型的建立装置中的训练样本获取模块31、训练模块32和目标医学成像模型确定模块33)。处理器40通过运行存储在存储器41中的软件程序、指令以及模块，从而执行设备的各种功能应用以及数据处理，即实现上述的医学成像模型的建立方法。
存储器41可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序；存储数据区可存储根据终端的使用所创建的数据等。此外，存储器41可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在一些实例中，存储器41可进一步包括相对于处理器40远程设置的存储器，这些远程存储器可以通过网络连接至设备。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
输入装置42可用于接收输入的医学图像的K空间欠采样数据，以及产生与设备的用户设置以及功能控制有关的信号输入。输出装置43可包括显示屏等显示设备。
实施例五
本发明实施例五还提供一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行一种医学成像模型的建立方法，该方法包括：
获取医学图像的K空间欠采样数据作为训练样本；
将所述训练样本输入至预先构建的所述原始神经网络进行训练，并根据模 型损失函数优化所述原始神经网络；
当所述模型损失函数收敛时，停止训练所述机器学习模型，将训练完成的所述原始神经网络作为目标医学成像模型；
其中，所述原始神经网络包括频率域网络FDN以及图像域网络SDN，所述频率域网络FDN与所述图像域网络SDN之间通过傅里叶逆变换IFFT连接；所述模型损失函数包括主损失函数以及至少一个附加损失函数，所述附加损失函数包括k空间损失函数和/或图像域损失函数。
当然，本发明实施例所提供的一种包含计算机可执行指令的存储介质,其计算机可执行指令不限于如上所述的方法操作，还可以执行本申请任一实施例所提供的医学成像模型的建立方法中的相关操作。
通过以上关于实施方式的描述，所属领域的技术人员可以清楚地了解到，本申请可借助软件及必需的通用硬件来实现，当然也可以通过硬件实现，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对相关技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如计算机的软盘、只读存储器(Read-Only Memory,ROM)、随机存取存储器(Random Access Memory,RAM)、闪存(FLASH)、硬盘或光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述的方法。
值得注意的是，上述医学成像模型的建立装置的实施例中，所包括的各个单元和模块只是按照功能逻辑进行划分的，但并不局限于上述的划分，只要能够实现相应的功能即可；另外，各功能单元的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。
注意，上述仅为本申请的可选实施例及所运用技术原理。本领域技术人员会理解，本申请不限于这里所述的可选实施例，对本领域技术人员来说能够进行各种明显的变化、重新调整和替代而不会脱离本申请的保护范围。因此，虽然通过以上实施例对本申请进行了较为详细的说明，但是本申请不仅仅限于以上实施例，在不脱离本申请构思的情况下，还可以包括更多其他等效实施例，而本申请的范围由所附的权利要求范围决定。

Claims (11)

  1. 一种医学成像模型的建立方法，包括：
    获取医学图像的K空间欠采样数据作为训练样本；
    将所述训练样本输入至预先构建的所述原始神经网络进行训练，并根据模型损失函数优化所述原始神经网络；
    当所述模型损失函数收敛时，停止训练所述机器学习模型，将训练完成的所述原始神经网络作为目标医学成像模型；
    其中，所述原始神经网络包括频率域网络FDN以及图像域网络SDN，所述频率域网络FDN与所述图像域网络SDN之间通过傅里叶逆变换IFFT连接；所述模型损失函数包括主损失函数以及至少一个附加损失函数，所述附加损失函数包括k空间损失函数和/或图像域损失函数。
  2. 根据权利要求1所述的方法，其中，所述频率域网络包括第一预设数量的频率域模块Fnet，其中，每个频率域模块Fnet包含第二预设数量的三维卷积层3D Conv以及一个频率域数据一致层KDC。
  3. 根据权利要求2所述的方法，其中，根据如下公式计算所述k空间损失函数：
    Figure PCTCN2019124239-appb-100001
    其中，Kloss表示k空间损失，K f表示医学图像的K空间欠采样数据对应的全采样k空间数据，
    Figure PCTCN2019124239-appb-100002
    表示第m个频率域模块的输出，M表示频率域模块的数量，α m表示第m个频率域模块对应的k空间损失函数的权重。
  4. 根据权利要求1所述的方法，其中，所述图像域网络SDN包括第三预设数量的图像域模块Snet，其中，每个图像域模块包含第四预设数量的三维卷积层3D Conv、一个图像域数据一致层IDC以及一个残差连接。
  5. 根据权利要求4所述的方法，其中，根据如下公式计算所述图像域损失函数：
    Figure PCTCN2019124239-appb-100003
    其中，Sloss表示图像域损失，S表示医学图像的K空间欠采样数据对应的全采样的医学图像，S n表示第n个图像域模块的输出，N表示图像域模块的数量，β n表示第n个图像域模块对应的图像域损失函数的权重。
  6. 根据权利要求1所述的方法，还包括：
    将至少一项所述附加损失函数加权后与所述主损失函数进行求和运算得到模型损失函数。
  7. 根据权利要求6所述的方法，其中，所述将至少一项所述附加损失函数加权后与所述主损失函数进行求和运算得到模型损失函数，包括：
    分别对k空间损失函数和图像域损失函数进行加权；
    将所述主损失函数、加权后的k空间损失函数以及加权后的图像域损失函数进行求和运算得到模型损失函数。
  8. 根据权利要求1所述的方法，还包括：
    获取待成像的K空间欠采样数据；
    将所述待成像的K空间欠采样数据输入训练完成的所述目标医学成像模型中，得到重建医学图像。
  9. 一种医学成像模型的建立装置，包括：
    训练样本获取模块，设置为获取医学图像的K空间欠采样数据作为训练样本；
    训练模块，设置为将所述训练样本输入至预先构建的所述原始神经网络进 行训练，并根据模型损失函数优化所述原始神经网络；
    目标医学成像模型确定模块，设置为当所述模型损失函数收敛时，停止训练所述机器学习模型，将训练完成的所述原始神经网络作为目标医学成像模型；
    其中，所述原始神经网络包括频率域网络FDN以及图像域网络SDN，所述频率域网络FDN与所述图像域网络SDN之间通过傅里叶逆变换IFFT连接；所述模型损失函数包括主损失函数以及至少一个附加损失函数，所述附加损失函数包括k空间损失函数和/或图像域损失函数。
  10. 一种设备，包括：
    一个或多个处理器；
    存储器，用于存储一个或多个程序，
    当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如权利要求1-8中任一项所述的医学成像模型的建立方法。
  11. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，该程序被处理器执行时实现如权利要求1-8中任一项所述的医学成像模型的建立方法。
PCT/CN2019/124239 2018-12-27 2019-12-10 一种医学成像模型的建立方法、装置、设备及存储介质 Ceased WO2020135015A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811654047.4 2018-12-27
CN201811654047.4A CN111383742A (zh) 2018-12-27 2018-12-27 一种医学成像模型的建立方法、装置、设备及存储介质

Publications (1)

Publication Number Publication Date
WO2020135015A1 true WO2020135015A1 (zh) 2020-07-02

Family

ID=71126819

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/124239 Ceased WO2020135015A1 (zh) 2018-12-27 2019-12-10 一种医学成像模型的建立方法、装置、设备及存储介质

Country Status (2)

Country Link
CN (1) CN111383742A (zh)
WO (1) WO2020135015A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112015659A (zh) * 2020-09-02 2020-12-01 三维通信股份有限公司 基于网络模型的预测方法及装置
CN113888659A (zh) * 2021-09-29 2022-01-04 南方科技大学 压缩矩阵拆分设计方法、装置、设备及存储介质
CN115222628A (zh) * 2022-07-25 2022-10-21 腾讯科技（深圳）有限公司 一种图像处理方法、装置、设备及存储介质
CN116503685A (zh) * 2023-02-23 2023-07-28 中国人民解放军国防科技大学 基于层次域先验的光学-isar图像转换方法及装置

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112116674A (zh) * 2020-08-13 2020-12-22 香港大学 图像重建方法、装置、终端及存储介质
CN112749802B (zh) * 2021-01-25 2024-02-09 深圳力维智联技术有限公司 神经网络模型的训练方法、装置以及计算机可读存储介质
CN112967185A (zh) * 2021-02-18 2021-06-15 复旦大学 基于频率域损失函数的图像超分辨率算法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007027893A2 (en) * 2005-08-30 2007-03-08 The Regents Of The University Of California, Santa Cruz Kernel regression for image processing and reconstruction
CN106373109A (zh) * 2016-08-31 2017-02-01 南方医科大学 一种医学图像模态合成的方法
CN107064845A (zh) * 2017-06-06 2017-08-18 深圳先进技术研究院 基于深度卷积网的一维部分傅里叶并行磁共振成像方法
CN107507148A (zh) * 2017-08-30 2017-12-22 南方医科大学 基于卷积神经网络去除磁共振图像降采样伪影的方法
CN107633486A (zh) * 2017-08-14 2018-01-26 成都大学 基于三维全卷积神经网络的结构磁共振图像去噪方法

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103646410B (zh) * 2013-11-27 2016-06-08 中国科学院深圳先进技术研究院 磁共振快速参数成像方法和系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007027893A2 (en) * 2005-08-30 2007-03-08 The Regents Of The University Of California, Santa Cruz Kernel regression for image processing and reconstruction
CN106373109A (zh) * 2016-08-31 2017-02-01 南方医科大学 一种医学图像模态合成的方法
CN107064845A (zh) * 2017-06-06 2017-08-18 深圳先进技术研究院 基于深度卷积网的一维部分傅里叶并行磁共振成像方法
CN107633486A (zh) * 2017-08-14 2018-01-26 成都大学 基于三维全卷积神经网络的结构磁共振图像去噪方法
CN107507148A (zh) * 2017-08-30 2017-12-22 南方医科大学 基于卷积神经网络去除磁共振图像降采样伪影的方法

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112015659A (zh) * 2020-09-02 2020-12-01 三维通信股份有限公司 基于网络模型的预测方法及装置
CN113888659A (zh) * 2021-09-29 2022-01-04 南方科技大学 压缩矩阵拆分设计方法、装置、设备及存储介质
CN115222628A (zh) * 2022-07-25 2022-10-21 腾讯科技（深圳）有限公司 一种图像处理方法、装置、设备及存储介质
CN116503685A (zh) * 2023-02-23 2023-07-28 中国人民解放军国防科技大学 基于层次域先验的光学-isar图像转换方法及装置

Also Published As

Publication number Publication date
CN111383742A (zh) 2020-07-07

Similar Documents

Publication Publication Date Title
WO2020135015A1 (zh) 一种医学成像模型的建立方法、装置、设备及存储介质
WO2020135014A1 (zh) 医学成像模型的建立方法、装置、设备及存储介质
US11880972B2 (en) Tissue nodule detection and tissue nodule detection model training method, apparatus, device, and system
CN111714124B (zh) 磁共振电影成像方法、装置、成像设备及存储介质
WO2020134826A1 (zh) 磁共振并行成像方法及相关设备
CN117011673B (zh) 基于噪声扩散学习的电阻抗层析成像图像重建方法和装置
WO2018099321A1 (zh) 一种基于广义树稀疏的权重核范数磁共振成像重建方法
CN110675321A (zh) 一种基于渐进式的深度残差网络的超分辨率图像重建方法
WO2017113205A1 (zh) 一种基于深度卷积神经网络的快速磁共振成像方法及装置
CN110942496B (zh) 基于螺旋桨采样和神经网络的磁共振图像重建方法及系统
WO2022193379A1 (zh) 图像重建模型生成及装置、图像重建方法及装置、设备、介质
CN117670703A (zh) 一种基于多任务联合训练的磁共振图像恢复方法
CN108010094A (zh) 一种磁共振图像重建方法和装置
WO2020119581A1 (zh) 磁共振参数成像方法、装置、设备及存储介质
CN104569880A (zh) 一种磁共振快速成像方法及系统
WO2022193378A1 (zh) 图像重建模型生成方法及装置、图像重建方法及装置、设备、介质
CN111681297B (zh) 图像重建方法、计算机设备和存储介质
CN111243052A (zh) 图像重建方法、装置、计算机设备和存储介质
CN110570487B (zh) 欠采样模型生成方法、图像重建方法、装置和计算机设备
US20230367850A1 (en) Systems and methods for mri data processing
CN110728732A (zh) 一种图像重建方法、装置、设备及介质
CN111161370A (zh) 一种基于ai的人体多核dwi联合重建方法
JP2021530780A (ja) 画像変換に基づく病巣判定方法、装置、コンピュータ機器及び記憶媒体
CN112530003B (zh) 一种三维人手重建方法、装置及电子设备
CN112530004B (zh) 一种三维点云重建方法、装置及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19902769

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 12/11/2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19902769

Country of ref document: EP

Kind code of ref document: A1