WO2022199135A1 - 一种基于深度学习的仰卧位和俯卧位乳腺图像配准方法 - Google Patents

一种基于深度学习的仰卧位和俯卧位乳腺图像配准方法 Download PDF

Info

Publication number
WO2022199135A1
WO2022199135A1 PCT/CN2021/137313 CN2021137313W WO2022199135A1 WO 2022199135 A1 WO2022199135 A1 WO 2022199135A1 CN 2021137313 W CN2021137313 W CN 2021137313W WO 2022199135 A1 WO2022199135 A1 WO 2022199135A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
registration
network
deformation field
moving image
Prior art date
Application number
PCT/CN2021/137313
Other languages
English (en)
French (fr)
Inventor
欧阳效芸
谢耀钦
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Publication of WO2022199135A1 publication Critical patent/WO2022199135A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
    • G06T3/147—Transformations for image registration, e.g. adjusting or mapping for alignment of images using affine transformations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
    • G06T3/153—Transformations for image registration, e.g. adjusting or mapping for alignment of images using elastic snapping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30068—Mammography; Breast
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • the invention relates to the technical field of medical image processing, and more particularly, to a breast image registration method based on deep learning in the supine position and the prone position.
  • Supine and prone breast images refer to the patient's supine and prone positions when the medical image is taken. Because the breast tissue is a soft tissue and the patient's position changes, the shape of the breast varies greatly in different positions, which makes the registration of breast images in the supine and prone positions more difficult. Supine and prone breast image registration has potential applications in breast cancer diagnosis, surgery, and postoperative radiation therapy.
  • the existing deep learning-based supine and prone breast image registration methods use a multi-network cascade to decompose the large deformation registration problem into several small deformation registration problems, that is, using several registration networks, each Each network learns a part of the deformation, and then combines the learning results of multiple networks to obtain the final registration result.
  • This scheme is an end-to-end registration method consisting of an affine registration network and three elastic registration networks.
  • the loss function of the affine network is the normalized cross-correlation loss function between the fixed image and the moving image after affine transformation
  • the loss function of the elastic registration network is the regularization loss function of the deformation field.
  • the normalized loss function between the final transformed moving image and the fixed image is calculated. Therefore, in the existing breast configuration methods, the number of networks is large, and the amount of calculation and parameters is large. However, due to the limited training data set, it is prone to overfitting, and the generated deformation field has many inconsistencies. actual deformation.
  • the purpose of the present invention is to overcome the above-mentioned defects of the prior art, and to provide a breast image registration method based on deep learning in the supine position and the prone position, which has fast registration speed and can reduce the occurrence of unrealistic deformations in the deformation field. .
  • a deep learning-based breast image registration method for supine and prone positions includes the following steps:
  • the registration network includes an affine registration network, a first spatial transformation network, an elastic registration network and a second spatial transformation network;
  • the registration network calculate the deformation field and the loss function value between the fixed image and the transformed moving image, until the set total loss function satisfies the optimization convergence condition, where the fixed image is a supine or prone breast image, The moving image is a prone or supine breast image in a different position from the fixed image;
  • the affine registration network takes the fixed image IF and the moving image IM as input to perform affine registration, and outputs the deformation field ⁇ 1 ;
  • the first spatial transformation network takes the deformation field ⁇ 1 and the moving image IM as the input, and the output is Deformation field transformed moving image I ′M ;
  • the elastic registration network takes the fixed image IF and the moving image I ′M as inputs for local registration, and in the upsampling structure, each upsampling layer outputs A deformation field, the deformation field output by the last up-sampling layer is marked as ⁇ 25 ;
  • the second spatial transformation network takes the combined deformation field ⁇ 1 ⁇ 25 and the moving image IM as input, and obtains the transformed moving image I′′ M .
  • a breast image registration method includes: inputting the breast image to be registered into the above-mentioned trained deep learning registration network obtained according to the present invention to obtain a registered image.
  • the present invention has the advantages that the existing supine and prone breast image registration methods have the problems of complex modeling, slow registration speed, low registration accuracy and individual differences.
  • the invention proposes a breast image registration method for supine and prone positions based on deep learning, which has the advantages of fast registration speed, simple model, good generalization performance and less unrealistic deformation.
  • FIG. 1 is a flowchart of a deep learning-based breast image registration method for supine and prone positions according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of a breast image registration process based on deep learning in a supine position and a prone position according to an embodiment of the present invention
  • FIG. 3 is a schematic diagram of the overall architecture of a supine and prone breast image registration network according to an embodiment of the present invention
  • FIG. 4 is a schematic diagram of a supine and prone breast image affine registration network according to an embodiment of the present invention
  • FIG. 5 is a schematic diagram of an elastic registration network for supine and prone breast images according to one embodiment of the present invention.
  • the invention proposes a deep learning-based registration method for supine and prone breast images.
  • the preprocessed fixed images and moving images are input into a registration network, and a deformation field is generated through the registration network.
  • the deformation field is applied to the moving image to obtain the transformed moving image.
  • the generated deformation field, as well as the fixed image and the transformed moving image are input into multiple loss functions, and then the loss function values are input into the deep learning optimizer, which optimizes the network parameters until the set conditions are met.
  • the provided deep learning-based breast image registration method in supine position and prone position includes the following steps.
  • step S1 the data sets of breast images in the supine and prone positions are preprocessed to obtain a data set.
  • step S1 includes:
  • Step S11 segment the breast tissue from the collected image.
  • the breast tissue is firstly segmented.
  • step S12 the voxel interval of the image is adjusted to reduce the increase in registration difficulty caused by different voxel intervals in different body positions.
  • Step S13 rotating the breast segmented image in the supine position to reduce the difficulty of registration.
  • step S14 the segmented image is cut to reduce the proportion of the background.
  • Step S15 normalize the image, and normalize the voxel value to [0,1].
  • step S16 an enhancement operation is performed on the data to enrich the training samples.
  • any supine position image and any prone position image of the same individual are combined in the channel dimension.
  • the combination sequence can be either the supine position image as the image on the first channel or the prone position image as the first channel image.
  • the image on the first channel is the fixed image IF
  • the image on the second channel is the moving image IM
  • the final dataset is obtained.
  • 2500 combined images are the training set
  • 100 combined images are the validation set
  • 400 combined images are the test set.
  • the test set data is composed of data from two patients, which never appeared in the training set and validation set.
  • Step S2 input the preprocessed data set into the deep learning registration network, and output the deformation field between the two images, the deformation field represents the moving distance of the voxel on the moving image, and the obtained deformation field is applied to the moving image, Get the transformed moving image.
  • step S2 includes:
  • Step S21 constructing a deep learning registration network including a spatial transformation network, an affine configuration network (or affine network) and an elastic registration network, so as to adopt a multi-resolution registration strategy.
  • the registration network also includes an affine registration network.
  • the overall architecture of the network is shown in Figure 3, which includes two spatial transformation networks (hereinafter referred to as the first spatial transformation network and the second spatial transformation network, respectively), an affine registration network, and an elastic registration network.
  • Step S22 input the preprocessed fixed image IF and moving image IM into the affine registration network, perform affine registration, and output the deformation field ⁇ 1 .
  • the deformation field ⁇ 1 represents the moving distance of the voxel on the moving image IM .
  • Step S23 the first space transformation network takes the deformation field ⁇ 1 and the moving image IM as input, and outputs the moving image I′ M after the moving image is transformed by the deformation field.
  • step S24 the fixed image IF and the moving image I′ M transformed by the affine registration network are input to the elastic registration network for local registration.
  • the elastic registration network contains an upsampling structure in which each upsampling layer outputs a deformation field.
  • the deformation field output by the last upsampling layer is labeled ⁇ 25 .
  • the deformation field ⁇ 25 represents the moving distance of the voxels on the transformed moving image I ′M .
  • step S25 the obtained deformation fields ⁇ 1 and ⁇ 25 are combined to obtain a combined deformation field ⁇ 1 ⁇ 25 .
  • step S26 the combined deformation field ⁇ 1 ⁇ 25 and the moving image IM are input into the second spatial transformation network to obtain the transformed moving image I′′ M .
  • the affine registration network mainly includes an input module, a downsampling module, an affine transformation parameter output module and a full-graphic variable field module, as shown in FIG. 4 , where rectangles with different colors represent different types of operations, The numbers in the rectangles indicate the number of channels.
  • the input module refers to reading the moving image and the fixed image first, then combining the moving image and the fixed image on the channel and inputting it into the input layer of the network.
  • the downsampling module consists of a series of convolution operations to reduce the image size.
  • the operation order of the downsampling operation is a convolution operation with a convolution kernel of 3*3*3 and a stride of 1, an activation operation, and a convolution operation with a convolution kernel of 3*3*3 and a stride of 2.
  • Product operation, one activation operation and 4 residual operations, 4 convolution kernels are 3*3*3 convolution operations with stride 2, and 4 activation operations appear alternately.
  • the residual operation is two activation operations and two convolution kernels of 3*3*3, and convolution operations with a stride of 1 appear alternately, and finally add the input of the residual network and the output of the second convolution operation. as the output of the residual operation.
  • the affine transformation parameter output module refers to further processing the output of the downsampling module, including a convolution operation with a convolution kernel of 3*3*3, a stride of 1, an activation operation, and an output of 9 numbers and 3 numbers convolution operation.
  • the first 9 numbers represent the shearing, scaling, and rotation parameters in the affine transformation parameters.
  • the last 3 numbers are translation parameters.
  • the full-image deformation field is obtained through the 12 affine transformation parameters obtained above, which is the full-image deformation field module.
  • the size of the deformation field is the same as the size of the fixed image. Because it is a point in three-dimensional space, the position change of each point corresponds to the position change in three directions, so the channel dimension of the deformation field is 3.
  • the structure of the elastic registration network is shown in Figure 5, where rectangles with different colors represent different types of operations, and each upsampling operation outputs a deformation field.
  • the basic structure of the elastic registration network includes an input module, a down-sampling module and an up-sampling module.
  • the input module refers to reading the moving image and the fixed image first, then combining the moving image and the fixed image on the channel and inputting it into the input layer of the network.
  • the downsampling module includes: a convolution operation with a convolution kernel of 3*3*3 and a stride of 1, an activation operation, a convolution operation with a convolution kernel of 3*3*3 and a stride of 2, a convolution operation with a convolution kernel of 3*3*3 and a stride of 2.
  • Activation operation and 4 residual operations, 4 convolution kernels are 3*3*3, convolution operation with stride 2 and 4 activation operations alternate. Then there is a convolution operation with a convolution kernel of 3*3*3, a stride of 1, and an activation operation.
  • the upsampling module includes 4 slightly complex upsampling operations and a simple upsampling process.
  • the complex upsampling process includes a transposed convolution operation, which combines the output of the transposed convolution operation and the output of the same layer of downsampling in the channel dimension, and then passes through a convolution kernel of 1*1*1 and a stride of 1.
  • the convolution operation of an activation operation, a convolution operation with a convolution kernel of 3*3*3, a stride of 1, and an activation operation.
  • the simple upsampling process includes a transposed convolution operation, which combines the output of the transposed convolution operation with the output of the same layer of downsampling in the channel dimension, and then passes through a convolution kernel of 3*3*3 and a stride of 1.
  • the convolution operation of an activation operation.
  • the transposed convolution outputs a deformation field of the same size as the fixed image. For example, there are 5 layers of upsampling operations in total, so 5 deformation fields are output. Only the displacement field of the last upsampling output is the deformation field that the elastic registration network needs to find finally.
  • the deformation fields output during other upsampling processes are the corresponding deformation fields at lower image resolutions, in order to allow the network to learn transformations at different spatial resolutions.
  • these lower resolution displacement fields are then subjected to transposed convolution operations to a fixed image size.
  • the first output after the complex upsampling operation, after the transposed convolution operation with a convolution kernel of 4*4*4 and a stride of 16 outputs a deformation field of the same size as the fixed image.
  • the output of the second complex upsampling operation is a transposed convolution operation with a convolution kernel of 4*4*4 and a stride of 8 to output a deformation field of the same size as the fixed image.
  • the third output after the complex upsampling operation, the transposed convolution operation with a convolution kernel of 4*4*4 and a stride of 4 outputs a deformation field of the same size as the fixed image.
  • the fourth output after the complex upsampling operation, the transposed convolution operation with a convolution kernel of 4*4*4 and a stride of 2 outputs a deformation field of the same size as the fixed image.
  • the last upsampling layer is a simple upsampling operation that outputs a deformation field of the same size as the fixed image. Adopting this operation is beneficial to the breast image registration task with large deformation, and also enables the network to be fully trained and accelerates the convergence speed.
  • step S3 the obtained deformation field, the transformed moving image and the fixed image are input into a plurality of loss functions to obtain the value of the loss function.
  • step S3 includes:
  • Step S31 setting the loss function of the registration network.
  • the total loss function of the registration network is shown in formula (1), where L sim represents the normalized cross-correlation loss function, L smooth represents the regularization loss function of the deformation field, ⁇ 21 , ⁇ 22 , ⁇ 23 , ⁇ 24 , ⁇ 25 is the deformation field output by each of the five upsampling layers, respectively.
  • ⁇ 1 , ⁇ 2 , ⁇ 3 , ⁇ 4 , ⁇ 5 are the weight coefficients
  • IF , IM represent the gray value of the fixed image and the moving image
  • I′ M is the moving image transformed by the affine registration network Image gray value
  • ⁇ 1 is the deformation field output by the reflection network.
  • the specific form of the normalized cross-correlation loss function is as formula (2), where and represents the average gray value of the corresponding image, p represents the point in the image, and ⁇ represents the dimension of the image.
  • the specific form of the regularization loss function L smooth of the deformation field is as in formula (3), where ⁇ represents the deformation field parameter, represents the derivative of the deformation field in the x-axis direction, represents the derivative of the deformation field in the y-axis direction, Represents the derivative of the deformation field in the z-axis direction.
  • Step S32 the loss function of the affine registration network is the normalized cross-correlation loss function between the fixed image IF and the transformed image I ′M .
  • the loss function of the elastic registration network includes the regularization loss function of the deformation field generated by each upsampling layer and the normalization loss function between the deformed moving image and the fixed image, and the weights are different.
  • formula (1) is in the form of a loss function in which 5 upsampling layers are set as an example, and can be further extended to a general form including more upsampling layers.
  • weight parameters can be set according to the image resolution size corresponding to each upsampling layer, for example, for lower resolution images, set lower weight values.
  • the registration network learning is constrained, so that the network can be fully trained, the convergence speed is faster, and the generated deformation field is more realistic.
  • Step S4 input the value of the loss function into the deep learning optimizer, and use the optimizer to update the parameters in the network.
  • step S5 steps S2-S4 are executed cyclically, and the network is optimized until the set conditions are met, and a trained registration network is obtained.
  • Step S6 input the test set into the trained registration network to test the registration performance of the network.
  • step S6 includes:
  • Step S61 visualize the moving image, the fixed image, and the moving image transformed by the deformation field, and evaluate the registration performance of the registration network from the image aspect.
  • Step S62 calculate the normalized cross-correlation value and normalized mutual information value between the fixed image and the moving image transformed by the deformation field, and evaluate the registration performance of the registration network from the aspect of image similarity.
  • Step S63 first obtain the binarized images of the foreground and background of the fixed image and the transformed moving image, and then calculate the dice value between the two binarized images.
  • Step S64 calculate the Jacobian value of the deformation field, and evaluate whether the deformation field generated by the registration network conforms to the reality.
  • the present invention uses a simpler network structure to realize the registration of breast images in the supine and prone positions. Registration, with fewer parameters, reduces the chance of overfitting and makes registration faster.
  • the present invention adopts a multi-resolution strategy to allow the network to learn the spatial deformation under different spatial resolutions.
  • the multi-resolution strategy introduces a variety of loss functions to reduce the unrealistic deformation in the generated deformation field, making the final deformation field more realistic.
  • the present invention may be a system, method and/or computer program product.
  • the computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the present invention.
  • a computer-readable storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Non-exhaustive list of computer readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM) or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically coded devices, such as printers with instructions stored thereon Hole cards or raised structures in grooves, and any suitable combination of the above.
  • RAM random access memory
  • ROM read only memory
  • EPROM erasable programmable read only memory
  • flash memory static random access memory
  • SRAM static random access memory
  • CD-ROM compact disk read only memory
  • DVD digital versatile disk
  • memory sticks floppy disks
  • mechanically coded devices such as printers with instructions stored thereon Hole cards or raised structures in grooves, and any suitable combination of the above.
  • Computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (eg, light pulses through fiber optic cables), or through electrical wires transmitted electrical signals.
  • the computer readable program instructions described herein may be downloaded to various computing/processing devices from a computer readable storage medium, or to an external computer or external storage device over a network such as the Internet, a local area network, a wide area network, and/or a wireless network.
  • the network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • a network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .
  • the computer program instructions for carrying out the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or instructions in one or more programming languages.
  • Source or object code written in any combination including object-oriented programming languages, such as Smalltalk, C++, Python, etc., and conventional procedural programming languages, such as the "C" language or similar programming languages.
  • the computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement.
  • the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through the Internet connect).
  • LAN local area network
  • WAN wide area network
  • custom electronic circuits such as programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs)
  • FPGAs field programmable gate arrays
  • PDAs programmable logic arrays
  • Computer readable program instructions are executed to implement various aspects of the present invention.
  • These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processor of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.
  • These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium on which the instructions are stored includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.
  • Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more functions for implementing the specified logical function(s) executable instructions.
  • the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented in dedicated hardware-based systems that perform the specified functions or actions , or can be implemented in a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are all equivalent.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

一种基于深度学习的仰卧位和俯卧位乳腺图像配准方法。该方法包括：构建深度学习配准网络，该配准网络包含仿射配准网络、第一空间变换网络、弹性配准网络和第二空间变换网络，其中在弹性配准网络的上采样结构中，每一个上采样层都输出一个变形场；训练所述配准网络，计算变形场以及固定图像和变换后的移动图像之间的损失函数值，直到设定的总损失函数满足优化收敛条件，其中固定图像是仰卧位或俯卧位乳腺图像，移动图像是与固定图像不同体位的俯卧位或仰卧位乳腺图像。该方法具有配准速度快，模型简单，泛化性能强并产生了较少的不符合实际的变形。

Description

一种基于深度学习的仰卧位和俯卧位乳腺图像配准方法 技术领域
本发明涉及医学图像处理技术领域，更具体地，涉及一种基于深度学习的仰卧位和俯卧位乳腺图像配准方法。
背景技术
图像配准的目的是求取两幅图像之间的一种或者一系列变换，使得两幅图像中对应点达到空间位置上的一致。这两幅图像分别称作固定图像和移动图像。仰卧位和俯卧位乳腺图像是指在拍摄医学图像时病人处于仰卧位和俯卧位的体位。由于乳腺组织是一个软组织以及病人体位的变化，使得不同体位下乳腺的形状发生很大的变化，从而使得仰卧位和俯卧位的乳腺图像配准难度加大。仰卧位和俯卧位的乳腺图像配准在乳腺癌的诊断，手术以及术后的放射治疗中都具有潜在的应用。
现有基于深度学习的仰卧位和俯卧位的乳腺图像配准方法采用多网络级联的方式将大形变配准问题分解成若干个小形变配准问题，也就是利用若干个配准网络，每个网络学习一部分形变，然后将多个网络的学习结果进行组合得到最终的配准结果。这种方案是端到端的配准方法，包含仿射配准网络和三个弹性配准网络。仿射网络的损失函数是固定图像和经过仿射变换后的移动图像之间的归一化互相关损失函数，弹性配准网络损失函数是变形场的正则化损失函数。最后再计算最终得到的变换后的移动图像和固定图像之间的归一化损失函数。因此，在现有乳腺配置方法中，网络个数较多，计算量和参数量较多，而由于训练数据集有限，容易出现过拟合的情况，而且生成的变形场存在较多的不符合实际的变形。
发明内容
本发明的目的是克服上述现有技术的缺陷，提供一种基于深度学习的仰卧位和俯卧位乳腺图像配准方法，配准速度快，并能够减少变形场中不符合实际的变形情况的出现。
根据本发明的第一方面，提供一种基于深度学习的仰卧位和俯卧位乳腺图像配准方法。该方法包括以下步骤：
构建深度学习配准网络，该配准网络包含仿射配准网络、第一空间变换网络、弹性配准网络和第二空间变换网络；
训练所述配准网络，计算变形场以及固定图像和变换后的移动图像之间的损失函数值，直到设定的总损失函数满足优化收敛条件，其中固定图像是仰卧位或俯卧位乳腺图像，移动图像是与固定图像不同体位的俯卧位或仰卧位乳腺图像；
其中仿射配准网络以固定图像I F和移动图像I M为输入进行仿射配准，输出变形场φ 1；第一空间变换网络以变形场φ 1和移动图像I M作为输入，输出经变形场变换后的移动图像I′ M；弹性配准网络以固定图像I F和移动图像I′ M为输入，用于进行局部配准，并且在上采样结构中，每一个上采样层都输出一个变形场，最后的上采样层输出的变形场标记为φ 25；第二空间变换网络以组合变形场φ 1οφ 25和移动图像I M为输入，得到变换后的移动图像I″ M。
根据本发明的第二方面，提供一种乳腺图像配准方法。该方法包括：将待配准的乳腺图像输入上述根据本发明获得的经训练深度学习配准网络，获得配准图像。
与现有技术相比，本发明的优点在于，针对现有的仰卧位和俯卧位乳腺图像配准方法具有建模复杂，配准速度慢，配准精度不高和个体差异性等问题，本发明提出一种基于深度学习的仰卧位和俯卧位的乳腺图像配准方法，具有配准速度快，模型简单，泛化性能好并产生了较少的不符合实际的变形。
通过以下参照附图对本发明的示例性实施例的详细描述，本发明的其它特征及其优点将会变得清楚。
附图说明
被结合在说明书中并构成说明书的一部分的附图示出了本发明的实施例，并且连同其说明一起用于解释本发明的原理。
图1是根据本发明一个实施例的基于深度学习的仰卧位和俯卧位乳腺图像配准方法的流程图；
图2是根据本发明一个实施例的基于深度学习的仰卧位和俯卧位的乳腺图像配准过程示意图；
图3是根据本发明一个实施例的仰卧位和俯卧位乳腺图像配准网络的总体架构示意图；
图4是根据本发明一个实施例的仰卧位和俯卧位乳腺图像仿射配准网络的示意图；
图5是根据本发明一个实施例的仰卧位和俯卧位乳腺图像弹性配准网络的示意图。
具体实施方式
现在将参照附图来详细描述本发明的各种示例性实施例。应注意到：除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值不限制本发明的范围。
以下对至少一个示例性实施例的描述实际上仅仅是说明性的，决不作为对本发明及其应用或使用的任何限制。
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下，所述技术、方法和设备应当被视为说明书的一部分。
在这里示出和讨论的所有例子中，任何具体值应被解释为仅仅是示例性的，而不是作为限制。因此，示例性实施例的其它例子可以具有不同的值。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步讨论。
本发明提出一种基于深度学习的仰卧位和俯卧位乳腺图像的配准方法，首先将经过预处理后的固定图像和移动图像输入到配准网络中，通过配准网络生成变形场。然后将变形场作用于移动图像，得到变换后的移动图像。将生成的变形场以及固定图像和变换后的移动图像输入到多个损失函数中，然后将损失函数值输入到深度学习优化器中，优化网络参数，直到满足设定的条件。
具体地，结合图1和图2所示，所提供的基于深度学习的仰卧位和俯 卧位乳腺图像的配准方法包括以下步骤。
步骤S1，对仰卧位和俯卧位乳腺图像数据集进行预处理，获得数据集。
在一个实施例中，步骤S1包括：
步骤S11，从采集的图像中分割出乳腺组织。
由于获取的图像中包含较多的组织和器官，而本发明的目标是乳腺组织的配准，因此首先将乳腺组织分割出来。
步骤S12，调整图像的体素间隔，减少因为不同体位下体素间隔不同带来的配准难度的提高。
步骤S13，旋转仰卧位的乳腺分割图像，降低配准难度。
步骤S14，对分割出来的图像进行剪切，减少背景所占比例。
步骤S15，对图像进行归一化处理，将体素值归一化到[0,1]。
步骤S16，对数据进行增强操作，以丰富训练样本。
由于采集到的数据集较少，优选地，还包括数据增强操作。即对经过了步骤S11-S15处理的数据进行随机的弹性变形。变换后将同一个体的任意一幅仰卧位图像和任意一幅俯卧位图像在通道维度上组合，组合顺序可以是仰卧位图像作为第一通道上的图像也可以是俯卧位图像作为第一通道上的图像，总之第一通道上的图像为固定图像I F，第二通道上的图像为移动图像I M，得到了最终的数据集。例如，2500幅组合图像为训练集，100幅组合图像为验证集，400幅组合图像为测试集。测试集数据是由两个病人的数据组成，从未出现在训练集和验证集中。
步骤S2，将预处理的数据集输入到深度学习配准网络，输出两幅图像之间的变形场，该变形场表示移动图像上体素的移动距离，将得到的变形场作用于移动图像，得到变换后的移动图像。
在一个实施例中，参见图3所示，步骤S2包括：
步骤S21，构建包含空间变换网络、仿射配置网络(或称仿射网络)和弹性配准网络的深度学习配准网络，以采用多分辨率的配准策略。
由于仰卧位和俯卧位的乳腺图像的形变较大，使用简单的配准网络进行配准得到的配准效果并不是很好。因此，优选地，采用了多分辨率的配准策略。为了尽可能减少预处理步骤以及实现真正的端到端配准，因此配准网 络中还包含了仿射配准网络。网络的总体架构如图3所示，其包含两个空间变换网络(在下文中分别称为第一空间变换网络和第二空间变换网络)、仿射配准网络以及一个弹性配准网络。
步骤S22，将预处理好的固定图像I F和移动图像I M输入到仿射配准网络中，进行仿射配准，输出变形场φ 1。该变形场φ 1表示移动图像I M上体素的移动距离。
步骤S23，第一空间变换网络将变形场φ 1和移动图像I M作为输入，输出移动图像经过变形场变换后的移动图像I′ M。
步骤S24，将固定图像I F和经过仿射配准网络变换后的移动图像I′ M输入到弹性配准网络，进行局部配准。弹性配准网络包含上采样结构，在上采样结构中，每一个上采样层都输出一个变形场。最后的上采样层输出的变形场标记为φ 25。变形场φ 25表示变换后的移动图像I′ M上体素的移动距离。
步骤S25，将得到的变形场φ 1和φ 25进行组合，得到组合变形场φ 1οφ 25。
步骤S26，将组合变形场φ 1οφ 25和移动图像I M输入到第二空间变换网络中，得到变换后的移动图像I″ M。
在一个实施例中，仿射配准网络主要包括输入模块，下采样模块，仿射变换参数输出模块和全图形变场模块，如图4所示，其中不同颜色的长方形表示不同类型的操作，长方形中的数字表示通道数。
具体地，输入模块是指先读取移动图像和固定图像，再将移动图像和固定图像在通道上进行组合然后输入到网络的输入层中。
下采样模块包含一系列的卷积操作将图像尺寸减小。例如，下采样操作的运算顺序是一个卷积核为3*3*3，步长为1的卷积运算，一个激活运算，一个卷积核为3*3*3，步长为2的卷积运算，一个激活运算以及4个残差运算，4个卷积核为3*3*3，步长为2的卷积运算，4个激活运算交替出现。残差运算是两个激活运算和两个卷积核为3*3*3，步长为1的卷积运算交替出现，最后将残差网络的输入和第二个卷积运算的输出相加作为残差运算的输出。
仿射变换参数输出模块是指进一步处理下采样模块的输出，包含一个卷积核为3*3*3，步长为1的卷积运算，一个激活运算以及输出为9个数和 3个数的卷积运算。前9个数表示的是仿射变换参数中的剪切，缩放，旋转参数。后3个数为平移参数。通过前面求出的12个仿射变换参数求出全图变形场，即为全图变形场模块。变形场的大小和固定图像的大小相同。因为是三维空间上的点，所以每个点的位置变化对应的是三个方向上的位置变化，因此变形场的通道维度为3。
在一个实施例中，弹性配准网络的结构如图5所示，其中不同颜色的长方形表示不同类型的操作，每个上采样操作都会输出变形场。弹性配准网络的基本结构包含输入模块，下采样模块和上采样模块。
具体地，输入模块是指先读取移动图像和固定图像，再将移动图像和固定图像在通道上进行组合然后输入到网络的输入层中。
下采样模块包括：一个卷积核为3*3*3，步长为1的卷积运算，一个激活运算，一个卷积核为3*3*3，步长为2的卷积运算，一个激活运算以及4个残差运算，4个卷积核为3*3*3，步长为2的卷积运算和4个激活运算交替出现。接着是一个卷积核为3*3*3，步长为1的卷积运算和一个激活运算。
上采样模块包括4个稍微复杂上采样操作以及一个简单的上采样过程。复杂的上采样过程包括转置卷积操作，将转置卷积操作的输出和同一层下采样的输出在通道维度组合起来，再经过一个卷积核为1*1*1，步长为1的卷积运算，一个激活运算，一个卷积核为3*3*3，步长为1的卷积运算，一个激活运算。简单的上采样过程包括转置卷积操作，将转置卷积操作的输出和同一层下采样的输出在通道维度组合起来，再经过一个卷积核为3*3*3，步长为1的卷积运算，一个激活运算。在上采样过程中，每一个上采样经过复杂的上采样操作后，经过转置卷积输出和固定图像相同大小的变形场。例如，总共有5层上采样操作，因此输出了5个变形场。只有最后一个上采样输出的位移场才是弹性配准网络最终需要求出的变形场。其他上采样过程中输出的变形场是较低图像分辨率下对应的变形场，目的是让网络学习不同空间分辨率下的变换。为了计算方便，于是使这些较低分辨率的位移场经过转置卷积操作到固定图像大小。从下往上数，第一个经过复杂上采样操作后的输出，经过卷积核为4*4*4，步长为16的转置卷积操作输出和固定图像相同大小 的变形场。第二个经过复杂上采样操作后的输出，经过卷积核为4*4*4，步长为8的转置卷积操作输出和固定图像相同大小的变形场。第三个经过复杂上采样操作后的输出，经过卷积核为4*4*4，步长为4的转置卷积操作输出和固定图像相同大小的变形场。第四个经过复杂上采样操作后的输出，经过卷积核为4*4*4，步长为2的转置卷积操作输出和固定图像相同大小的变形场。最后一个上采样层是经过简单的上采样操作输出的和固定图像相同大小的变形场。采用此操作有利于大形变的乳腺图像配准任务，也使得网络得到充分的训练，加快收敛速度。
步骤S3，将得到的变形场和变换后的移动图像以及固定图像输入到多个损失函数中，得到损失函数的值。
在一个实施例中，步骤S3包括：
步骤S31，设置配准网络的损失函数。
例如，配准网络的总损失函数如公式(1)，其中L sim表示归一化互相关损失函数，L smooth表示变形场的正则化损失函数，φ 21,φ 22,φ 23,φ 24,φ 25分别是五个上采样层中，每一个上采样层输出的变形场。λ 1,λ 2,λ 3,λ 4,λ 5是权重系数，I F，I M表示的是固定图像和移动图像的灰度值，I′ M是经过仿射配准网络变换后的移动图像灰度值，φ 1是反射网络输出的变形场。
在一个实施例中，归一化互相关损失函数具体形式如公式(2)，其中
Figure PCTCN2021137313-appb-000001
和
Figure PCTCN2021137313-appb-000002
表示相应图像的平均灰度值，p表示图像中的点，Ω表示图像的维度。
在一个实施例中，变形场的正则化损失函数L smooth的具体形式如公式(3)，其中θ表示变形场参数，
Figure PCTCN2021137313-appb-000003
表示变形场在x轴方向的导数，
Figure PCTCN2021137313-appb-000004
表示变形场在y轴方向的导数，
Figure PCTCN2021137313-appb-000005
表示变形场在z轴方向的导数。
步骤S32，仿射配准网络的损失函数是固定图像I F和变换后的图像I′ M之间的归一化互相关损失函数。
步骤S33，弹性配准网络的损失函数包括了每一个上采样层生成的变形场的正则化损失函数和变形后的移动图像和固定图像之间的归一化损失函数，权重各不相同。
Figure PCTCN2021137313-appb-000006
Figure PCTCN2021137313-appb-000007
Figure PCTCN2021137313-appb-000008
需要说明的是，公式(1)是以设置5个上采样层为例的损失函数形式，可以进一步扩展为包含更多上采样层的通用形式。并且可根据每个上采样层对应的图像分辨率大小设置权重参数，例如，对于较低分辨率图像，设置较低的权重值。
在此步骤中，通过多分辨策略，约束配准网络学习，使得网络能够得到充分的训练，收敛速度更快，并且生成的变形场更符合实际。
步骤S4，将损失函数的值输入到深度学习优化器中，利用优化器更新网络中的参数。
步骤S5，循环执行步骤S2-S4，对网络进行优化，直到满足设定的条件，得到训练好的配准网络。
步骤S6，将测试集输入到训练好的配准网络中，测试网络的配准性能。
在一个实施例中，步骤S6包括：
步骤S61，可视化移动图像，固定图像，以及经过变形场变换后的移动图像，从图像方面评估配准网络的配准性能。
步骤S62，计算固定图像和经过变形场变换后的移动图像之间的归一化互相关值和归一化互信息值，从图像相似度方面评估配准网络的配准性能。
步骤S63，先得到固定图像和变换后的移动图像的前景和背景的二值化图像，再计算这两幅二值化图像之间的dice值。
步骤S64，计算变形场的雅克比行列式值，评价配准网络生成的变形 场是否符合实际。
综上所述，相比于目前基于深度学习的多网络级联的仰卧位和俯卧位乳腺图像配准方法而言，本发明使用更简单的网络结构实现了仰卧位和俯卧位的乳腺图像的配准，参数量较少，降低了出现过拟合的几率并且配准速度更快。此外，考虑到仰卧位和俯卧位的乳腺图像之间存在较大的形变，本发明采用了一种多分辨率的策略，让网络学习不同空间分辨率下的空间变形。多分辨率策略引入了多种损失函数，减少了生成的变形场中包含不符合实际的变形，使得最终得到的变形场更符合实际。
本发明可以是系统、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本发明的各个方面的计算机可读程序指令。
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是――但不限于――电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中 的计算机可读存储介质中。
用于执行本发明操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++、Python等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本发明的各个方面。
这里参照根据本发明实施例的方法、装置(系统)和计算机程序产品的流程图和/或框图描述了本发明的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上 执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本发明的多个实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。对于本领域技术人员来说公知的是，通过硬件方式实现、通过软件方式实现以及通过软件和硬件结合的方式实现都是等价的。
以上已经描述了本发明的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。本发明的范围由所附权利要求来限定。

Claims (10)

  1. 一种基于深度学习的仰卧位和俯卧位乳腺图像配准方法，包括以下步骤：
    构建深度学习配准网络，该配准网络包含仿射配准网络、第一空间变换网络、弹性配准网络和第二空间变换网络；
    训练所述配准网络，计算变形场以及固定图像和变换后的移动图像之间的损失函数值，直到设定的总损失函数满足优化收敛条件，其中固定图像是仰卧位或俯卧位乳腺图像，移动图像是与固定图像不同体位的俯卧位或仰卧位乳腺图像；
    其中：仿射配准网络以固定图像I F和移动图像I M为输入进行仿射配准，输出变形场φ 1；第一空间变换网络以变形场φ 1和移动图像I M作为输入，输出经变形场变换后的移动图像I′ M；弹性配准网络以固定图像I F和移动图像I′ M为输入，用于进行局部配准，并且在上采样结构中，每一个上采样层都输出一个变形场，最后的上采样层输出的变形场标记为φ 25；第二空间变换网络以组合变形场φ 1οφ 25和移动图像I M为输入，得到变换后的移动图像I″ M。
  2. 根据权利要求1所述的方法，其特征在于，所述弹性配准网络包含输入模块、下采样模块和上采样模块，输入模块用于读取移动图像和固定图像，并将移动图像和固定图像在通道上进行组合后输入到弹性配准网络的输入层；下采样模块包括卷积运算、激活运算和残差运算；上采样模块包括多个上采样层，每一上采样层输出和固定图像相同大小的变形场，且将最后一个上采样层输出的位移场作为所述弹性配准网络最终的变形场，而其余上采样层输出的变形场对应不同图像分辨率下的变形场。
  3. 根据权利要求2所述的方法，其特征在于，所述配准网络的总损失函数表示为：
    Figure PCTCN2021137313-appb-100001
    其中，L sim表示归一化互相关损失函数，L smooth表示变形场的正则化损失函数，φ 21,φ 22,φ 23,φ 24,φ 25分别是五个上采样层中，每一个上采样层输出的变形场， λ 1,λ 2,λ 3,λ 4,λ 5是权重系数，I F和I M分别表示固定图像和移动图像的灰度值，I′ M是经过仿射配准网络变换后的移动图像灰度值，φ 1是仿射配准网络输出的变形场。
  4. 根据权利要求3所述的方法，其特征在于，所述归一化互相关损失函数表示为：
    Figure PCTCN2021137313-appb-100002
    其中，
    Figure PCTCN2021137313-appb-100003
    表示图像的平均灰度值，p表示图像中的点，Ω表示图像的维度。
  5. 根据权利要求3所述的方法，其特征在于，所述变形场的正则化损失函数表示为：
    Figure PCTCN2021137313-appb-100004
    其中，θ表示变形场参数，
    Figure PCTCN2021137313-appb-100005
    表示变形场在x轴方向的导数，
    Figure PCTCN2021137313-appb-100006
    表示变形场在y轴方向的导数，
    Figure PCTCN2021137313-appb-100007
    表示变形场在z轴方向的导数，p表示图像中的点，Ω表示图像的维度。
  6. 根据权利要求1所述的方法，其特征在于，所述仿射配准网络包括输入模块、下采样模块、仿射变换参数输出模块和全图变形场模块，所述输入模块用于将数据集读取到所述仿射配准网络的输入层；所述下采样模块用于降低输入层图像的尺寸，包括卷积操作、激活处理和残差运算；所述仿射变换参数输出模块用于处理下采样模块的输出，以输出仿射变换参数；所述全图变形场模块用于利用仿射变换参数求出全图变形场。
  7. 根据权利要求3所述的方法，其特征在于，根据每个上采样层对应的图像分辨率大小设置权重参数。
  8. 一种乳腺图像配准方法，包括：将待配准的乳腺图像输入根据权利要求1至7任一项所述方法获得的经训练的深度学习配准网络，获得配准图像。
  9. 一种计算机可读存储介质，其上存储有计算机程序，其中，该程序被处理器执行时实现根据权利要求1至8中任一项所述方法的步骤。
  10. 一种计算机设备，包括存储器和处理器，在所述存储器上存储有能够在处理器上运行的计算机程序，其特征在于，所述处理器执行所述程序时实现权利要求1至8中任一项所述的方法的步骤。
PCT/CN2021/137313 2021-03-26 2021-12-12 一种基于深度学习的仰卧位和俯卧位乳腺图像配准方法 WO2022199135A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110327737.4A CN112907439B (zh) 2021-03-26 2021-03-26 一种基于深度学习的仰卧位和俯卧位乳腺图像配准方法
CN202110327737.4 2021-03-26

Publications (1)

Publication Number Publication Date
WO2022199135A1 true WO2022199135A1 (zh) 2022-09-29

Family

ID=76109232

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/137313 WO2022199135A1 (zh) 2021-03-26 2021-12-12 一种基于深度学习的仰卧位和俯卧位乳腺图像配准方法

Country Status (2)

Country Link
CN (1) CN112907439B (zh)
WO (1) WO2022199135A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116958217A (zh) * 2023-08-02 2023-10-27 德智鸿(上海)机器人有限责任公司 一种mri与ct多模态3d自动配准方法及装置
CN118279364A (zh) * 2024-06-03 2024-07-02 青岛山大齐鲁医院（山东大学齐鲁医院（青岛）） 一种mri影像与cbct影像的配准方法

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112907439B (zh) * 2021-03-26 2023-08-08 中国科学院深圳先进技术研究院 一种基于深度学习的仰卧位和俯卧位乳腺图像配准方法
CN113450397B (zh) * 2021-06-25 2022-04-01 广州柏视医疗科技有限公司 基于深度学习的图像形变配准方法
CN113643332B (zh) * 2021-07-13 2023-12-19 深圳大学 图像配准方法、电子设备及可读存储介质
CN113870327B (zh) * 2021-09-18 2024-05-21 大连理工大学 基于预测多层次变形场的医学图像配准方法
CN114359356A (zh) * 2021-12-28 2022-04-15 上海联影智能医疗科技有限公司 图像配准模型的训练方法、图像配准方法、设备及介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100135544A1 (en) * 2005-10-25 2010-06-03 Bracco Imaging S.P.A. Method of registering images, algorithm for carrying out the method of registering images, a program for registering images using the said algorithm and a method of treating biomedical images to reduce imaging artefacts caused by object movement
CN105389815A (zh) * 2015-10-29 2016-03-09 武汉联影医疗科技有限公司 一种乳腺图像配准方法及装置
CN108738300A (zh) * 2016-02-29 2018-11-02 皇家飞利浦有限公司 用于医学乳房图像的校正的设备、成像系统和方法
CN110599528A (zh) * 2019-09-03 2019-12-20 济南大学 一种基于神经网络的无监督三维医学图像配准方法及系统
CN112907439A (zh) * 2021-03-26 2021-06-04 中国科学院深圳先进技术研究院 一种基于深度学习的仰卧位和俯卧位乳腺图像配准方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9808213B2 (en) * 2014-08-11 2017-11-07 Canon Kabushiki Kaisha Image processing apparatus, image processing method, medical image diagnostic system, and storage medium
JP6433190B2 (ja) * 2014-08-11 2018-12-05 キヤノン株式会社 画像処理装置、画像処理方法、およびプログラム
CN110827335B (zh) * 2019-11-01 2020-10-16 北京推想科技有限公司 乳腺影像配准方法和装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100135544A1 (en) * 2005-10-25 2010-06-03 Bracco Imaging S.P.A. Method of registering images, algorithm for carrying out the method of registering images, a program for registering images using the said algorithm and a method of treating biomedical images to reduce imaging artefacts caused by object movement
CN105389815A (zh) * 2015-10-29 2016-03-09 武汉联影医疗科技有限公司 一种乳腺图像配准方法及装置
CN108738300A (zh) * 2016-02-29 2018-11-02 皇家飞利浦有限公司 用于医学乳房图像的校正的设备、成像系统和方法
CN110599528A (zh) * 2019-09-03 2019-12-20 济南大学 一种基于神经网络的无监督三维医学图像配准方法及系统
CN112907439A (zh) * 2021-03-26 2021-06-04 中国科学院深圳先进技术研究院 一种基于深度学习的仰卧位和俯卧位乳腺图像配准方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
OUYANG XIAOYUN; LIANG XIAOKUN; XIE YAOQIN: "Preliminary Feasibility Study of Imaging Registration Between Supine and Prone Breast CT in Breast Cancer Radiotherapy Using Residual Recursive Cascaded Networks", IEEE ACCESS, vol. 9, 28 December 2020 (2020-12-28), USA , pages 3315 - 3325, XP011829501, DOI: 10.1109/ACCESS.2020.3047829 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116958217A (zh) * 2023-08-02 2023-10-27 德智鸿(上海)机器人有限责任公司 一种mri与ct多模态3d自动配准方法及装置
CN116958217B (zh) * 2023-08-02 2024-03-29 德智鸿(上海)机器人有限责任公司 一种mri与ct多模态3d自动配准方法及装置
CN118279364A (zh) * 2024-06-03 2024-07-02 青岛山大齐鲁医院（山东大学齐鲁医院（青岛）） 一种mri影像与cbct影像的配准方法

Also Published As

Publication number Publication date
CN112907439A (zh) 2021-06-04
CN112907439B (zh) 2023-08-08

Similar Documents

Publication Publication Date Title
WO2022199135A1 (zh) 一种基于深度学习的仰卧位和俯卧位乳腺图像配准方法
Yin et al. [Retracted] U‐Net‐Based Medical Image Segmentation
Tang et al. CT-realistic data augmentation using generative adversarial network for robust lymph node segmentation
Xia et al. Automatic 3D atrial segmentation from GE-MRIs using volumetric fully convolutional networks
WO2022193750A1 (zh) 一种基于深度学习的乳腺图像配准方法
JP2022191354A (ja) 画像解析における解剖学的構造のセグメンテーションのためのシステム及び方法
CN111008688A (zh) 网络训练期间使用环路内数据增加的神经网络
WO2023060746A1 (zh) 一种基于超分辨率的小图像多目标检测方法
CN111557020A (zh) 基于完全卷积神经网络的心脏cta解剖结构分割系统
CN113450396B (zh) 基于骨骼特征的三维/二维图像配准方法及装置
WO2022151586A1 (zh) 一种对抗配准方法、装置、计算机设备及存储介质
WO2021102644A1 (zh) 图像增强方法、装置及终端设备
Pandey et al. Segmentation of liver lesions with reduced complexity deep models
CN111091010A (zh) 相似度确定、网络训练、查找方法及装置和存储介质
CN110570394A (zh) 医学图像分割方法、装置、设备及存储介质
Zhong et al. Deep action learning enables robust 3D segmentation of body organs in various CT and MRI images
Li et al. Category guided attention network for brain tumor segmentation in MRI
Sarma et al. Harnessing clinical annotations to improve deep learning performance in prostate segmentation
Anas et al. Ct scan registration with 3d dense motion field estimation using lsgan
CN114283110A (zh) 用于医学图像的图像处理方法、装置、设备及存储介质
US20230410483A1 (en) Medical imaging analysis using self-supervised learning
WO2024041058A1 (zh) 一种随访病例数据的处理方法、装置、设备及存储介质
Gao et al. Consistency based co-segmentation for multi-view cardiac MRI using vision transformer
Muksimova et al. Enhancing medical image denoising with innovative teacher–student model-based approaches for precision diagnostics
CN115908811A (zh) 一种基于Transformer和卷积注意力机制的CT图像分割方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21932742

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21932742

Country of ref document: EP

Kind code of ref document: A1