WO2021164418A1 - Tubular structure segmentation map fracture restoration system for three-dimensional image based on deep learning network - Google Patents

Tubular structure segmentation map fracture restoration system for three-dimensional image based on deep learning network Download PDF

Info

Publication number
WO2021164418A1
WO2021164418A1 PCT/CN2020/139471 CN2020139471W WO2021164418A1 WO 2021164418 A1 WO2021164418 A1 WO 2021164418A1 CN 2020139471 W CN2020139471 W CN 2020139471W WO 2021164418 A1 WO2021164418 A1 WO 2021164418A1
Authority
WO
WIPO (PCT)
Prior art keywords
deep learning
map
learning network
segmentation
target organ
Prior art date
Application number
PCT/CN2020/139471
Other languages
French (fr)
Chinese (zh)
Inventor
赖耀明
魏军
沈烁
Original Assignee
广州柏视医疗科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广州柏视医疗科技有限公司 filed Critical 广州柏视医疗科技有限公司
Publication of WO2021164418A1 publication Critical patent/WO2021164418A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G06T2207/10012—Stereo images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30101—Blood vessel; Artery; Vein; Vascular

Definitions

  • the invention relates to the field of medical images, in particular to a pipe structure segmentation map fracture repair system based on a deep learning network of three-dimensional images.
  • an embodiment of the present invention provides a system for repairing a fracture of a tubular structure segmentation map based on a three-dimensional image of a deep learning network.
  • an embodiment of the present invention provides a method for repairing a fracture of a segmented map of a tubular structure of a three-dimensional image based on a deep learning network, which includes the following steps:
  • Step S1 Collect three-dimensional image data and perform preprocessing
  • Step S2 Re-sampling the pre-processed 3D image data to generate multiple candidate 3D data of the region of interest;
  • Step S3 Input the three-dimensional data of all candidate sense regions into the target segmentation deep learning network to obtain a target organ segmentation map in each region of interest;
  • Step S4 Perform distance transformation on the target organ segmentation map to obtain a distance transformation map, and input the target organ segmentation map and the distance transformation map into the fracture repair deep learning network to obtain a fracture repair map for each region of interest, and then combine all the fracture repair maps Stitch the combined result to the complete target organ segmentation result.
  • the preprocessing of the three-dimensional image data in step S1 includes the following steps: pixel value normalization and interpolation.
  • c is the window level
  • w is the window width
  • x is the pixel value
  • the interpolation method is: first parse the original three-dimensional image, obtain the image Spacing attributes (z 0 , y 0 , x 0 ); among them, the target image Spacing is (z 1 , y 1 , x 1 ), and calculate the Scaling factor (z 0 /z 1 , y 0 /y 1 , x 0 /x 1 ), and then according to the scaling factor in each direction, bilinear interpolation is used to obtain the interpolated three-dimensional data.
  • step S2 the method of resampling in step S2 is: taking the interpolated three-dimensional data according to the step size of 32 ⁇ 48 ⁇ 48, and extracting the three-dimensional image of the local patches with the size of 64 ⁇ 96 ⁇ 96 and overlapping each other.
  • the target segmentation deep learning network uses the convolution residual block and downsampling alternately 4 times to extract high-dimensional features, and then alternately performs 4 times through the convolution residual block and upsampling to restore the resolution of the original input image Rate.
  • the feature map of the target segmentation deep learning network after upsampling is fused with the low-level feature map of the same resolution.
  • the target segmentation deep learning network uses up-sampling feature maps of target organs of different depths to perform feature fusion.
  • the fracture repair deep learning network uses the convolution residual block and downsampling alternately once to extract high-dimensional features, and then alternates the convolution residual block and upsampling once to restore the original input image resolution .
  • an embodiment of the present invention provides a three-dimensional image-based deep learning network-based pipe structure segmentation map fracture repair system, including:
  • Preprocessing module used to preprocess 3D image data
  • Image resampling module used to resample the preprocessed 3D image data to generate multiple candidate 3D data of the region of interest;
  • Target organ extraction module used to input the three-dimensional data of all candidate sense regions into the target segmentation deep learning network to obtain the target organ segmentation map in each region of interest;
  • Fracture repair module used to perform distance transformation on the target organ segmentation map to obtain the distance transformation map.
  • the target organ segmentation map and the distance transformation map are input into the fracture repair deep learning network to obtain the fracture repair map for each region of interest.
  • Fracture repair images are stitched together to complete the segmentation result of the target organ.
  • the embodiment of the present invention provides a three-dimensional image-based tubular structure segmentation map fracture repair system based on a deep learning network, which has the following advantages: the three-dimensional image of the tubular structure segmentation map fracture repair method and system uses the distance transformation information of the fracture as the fracture location The identification information can realize automatic fracture repair without detecting the fracture position in advance, which is simple and effective, and improves the segmentation effect.
  • FIG. 1 is a flowchart of a method for repairing a fracture of a tubular structure segmentation map of a three-dimensional image based on a deep learning network according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of the structure of a deep learning network for target segmentation in a method provided by an embodiment of the present invention
  • FIG. 3 is a schematic diagram of the structure of a fracture repair deep learning network in a method provided by an embodiment of the present invention
  • Fig. 4 is a schematic diagram of a fracture repair system for a tubular structure segmentation diagram of a three-dimensional image based on a deep learning network provided by an embodiment of the present invention.
  • FIG. 1 is a flowchart of a method for repairing a fracture of a three-dimensional image of a tubular structure based on a deep learning network provided by an embodiment of the present invention. As shown in FIG. The method for repairing the structure segmentation diagram fracture includes the following steps:
  • Step S1 Collect three-dimensional image data and perform preprocessing
  • step S1 of the embodiment of the present invention the preprocessing of the three-dimensional image data includes the following steps: pixel value normalization and interpolation.
  • the method of pixel value normalization is:
  • c is the window level
  • w is the window width
  • x is the pixel value
  • the interpolation method is: first parse the original three-dimensional image, obtain the image Spacing attribute (z 0 , y 0 , x 0 ); among them, the target image Spacing is (z 1 , y 1 , x 1 ), and calculate the zoom factor ( z 0 /z 1 , y 0 /y 1 , x 0 /x 1 ), and then according to the scaling factors in each direction, bilinear interpolation is used to obtain interpolated three-dimensional data.
  • Step S2 Re-sampling the pre-processed 3D image data to generate multiple candidate 3D data of the region of interest;
  • the re-sampling method is: taking the interpolated three-dimensional data according to the step size of 32 ⁇ 48 ⁇ 48, and extracting the three-dimensional image of the local patch with the size of 64 ⁇ 96 ⁇ 96 and overlapping each other.
  • Step S3 Input the three-dimensional data of all candidate sense regions into the target segmentation deep learning network to obtain a target organ segmentation map in each region of interest;
  • Step S4 Perform distance transformation on the target organ segmentation map to obtain a distance transformation map, and input the target organ segmentation map and the distance transformation map into the fracture repair deep learning network to obtain a fracture repair map for each region of interest, and then combine all the fracture repair maps Stitch the combined result to the complete target organ segmentation result.
  • the deep learning network of the three-dimensional image arteriovenous segmentation method adopts the two-stage recognition of the target segmentation deep learning network and the fracture repair deep learning network, that is, first use the target segmentation deep learning network to extract the target organ to obtain the target organ segmentation map, and then The target organ segmentation map is transformed by distance to obtain the distance transformation map, and the target organ segmentation map and the distance transformation map are input into the fracture repair deep learning network for fracture repair.
  • the target segmentation deep learning network and the fracture repair deep learning network are both output probability maps.
  • the target segmentation deep learning network uses the convolution residual block and downsampling alternately 4 times to extract high-dimensional features, and then alternately performs 4 times through the convolution residual block and upsampling to restore to The resolution of the original input image.
  • the up-sampled feature map of the target segmentation deep learning network is merged with the low-level feature map of the same resolution.
  • the target segmentation deep learning network uses up-sampling for feature fusion of target organ feature maps of different depths.
  • the fracture repair deep learning network uses the convolution residual block and downsampling alternately once to extract high-dimensional features, and then alternates the convolution residual block and upsampling once to restore the original input image Resolution.
  • the fracture repair deep learning network uses dilated convolution in the 4th, 5th, 6th, 7th, and 8th residual blocks; and uses skip connections to supplement low-level Detailed information.
  • FIG. 4 is a schematic structural diagram of a pipe structure segmentation map fracture repair system based on a deep learning network based on a three-dimensional image of a deep learning network provided by an embodiment of the present invention, and the system includes:
  • Preprocessing module used to preprocess 3D image data
  • Image resampling module used to resample the preprocessed 3D image data to generate multiple candidate 3D data of the region of interest;
  • Target organ extraction module used to input the three-dimensional data of all candidate sense regions into the target segmentation deep learning network to obtain the target organ segmentation map in each region of interest;
  • Fracture repair module used to perform distance transformation on the target organ segmentation map to obtain the distance transformation map.
  • the target organ segmentation map and the distance transformation map are input into the fracture repair deep learning network to obtain the fracture repair map for each region of interest.
  • Fracture repair images are stitched together to complete the segmentation result of the target organ.
  • the fracture repair system of the tubular structure segmentation map based on the three-dimensional image of the deep learning network uses the distance transformation information of the fracture as the fracture location identification information, and can realize automatic fracture without detecting the fracture location in advance.
  • the repair is simple and effective and improves the segmentation effect.
  • the device embodiments described above are merely illustrative.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network units.
  • Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments. Those of ordinary skill in the art can understand and implement it without creative work.
  • each implementation manner can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware.
  • the above technical solution essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM/RAM, magnetic A disc, an optical disc, etc., include several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiment.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • Quality & Reliability (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

A tubular structure segmentation map fracture restoration system for a three-dimensional image based on a deep learning network, comprising: a preprocessing module, configured to preprocess three-dimensional image data; an image resampling module, configured to resample the preprocessed three-dimensional image data, and generate three-dimensional data of a plurality of candidate regions of interest; a target organ extraction module, configured to input the three-dimensional data of all the candidate regions of interest into a target segmentation deep learning network, to obtain a target organ segmentation map in each region of interest; and a fracture restoration module, configured to perform distance transform on the target organ segmentation map to obtain a distance transform map, input the target organ segmentation map and the distance transform map to a fracture restoration deep learning network to obtain a fracture restoration map of each region of interest, and splice all the fracture restoration maps together to obtain the complete target organ segmentation result. The system can realize automatic fracture restoration, improving the segmentation effect.

Description

基于深度学习网络的三维影像的管状结构分割图断裂修复系统Tubular structure segmentation map fracture repair system based on 3D image of deep learning network 技术领域Technical field
本发明涉及医疗图像领域，尤其涉及一种基于深度学习网络的三维影像的管状结构分割图断裂修复系统。The invention relates to the field of medical images, in particular to a pipe structure segmentation map fracture repair system based on a deep learning network of three-dimensional images.
背景技术Background technique
在正常人体解剖结构里，存在很多管状结构的器官，例如动脉、静脉、支气管、神经等等。在疾病的诊断与治疗过程中，这些器官重建非常重要，具有重要的临床价值。In normal human anatomy, there are many organs with tubular structures, such as arteries, veins, bronchi, nerves and so on. In the diagnosis and treatment of diseases, the reconstruction of these organs is very important and has important clinical value.
目前针对不同的管状器官，在自动重建后的分割图像中，会出现噪声、断裂等异常情况，严重影响疾病的诊断与治疗。At present, for different tubular organs, abnormal conditions such as noise and breakage will appear in the segmented images after automatic reconstruction, which seriously affects the diagnosis and treatment of diseases.
公开于该背景技术部分的信息仅仅旨在增加对本实用新型的总体背景的理解，而不应当被视为承认或以任何形式暗示该信息构成已为本领域一般技术人员所公知的现有技术。The information disclosed in the background section is only intended to increase the understanding of the general background of the present utility model, and should not be regarded as an acknowledgement or any form of suggestion that the information constitutes the prior art known to those of ordinary skill in the art.
发明内容Summary of the invention
针对现有技术存在的问题，本发明实施例提供一种基于深度学习网络的三维影像的管状结构分割图断裂修复系统。In view of the problems in the prior art, an embodiment of the present invention provides a system for repairing a fracture of a tubular structure segmentation map based on a three-dimensional image of a deep learning network.
第一方面，本发明实施例提供一种基于深度学习网络的三维影像的管状结构分割图断裂修复方法，包括以下步骤：In the first aspect, an embodiment of the present invention provides a method for repairing a fracture of a segmented map of a tubular structure of a three-dimensional image based on a deep learning network, which includes the following steps:
步骤S1：采集三维影像数据，并进行预处理；Step S1: Collect three-dimensional image data and perform preprocessing;
步骤S2：对预处理后的三维影像数据进行重采样，生成多个候选的感兴趣区域的三维数据；Step S2: Re-sampling the pre-processed 3D image data to generate multiple candidate 3D data of the region of interest;
步骤S3：将所有候选感区域的三维数据输入目标分割深度学习网络，得到每一个感兴趣区域内的目标器官分割图；Step S3: Input the three-dimensional data of all candidate sense regions into the target segmentation deep learning network to obtain a target organ segmentation map in each region of interest;
步骤S4：将目标器官分割图进行距离变换得到距离变换图，将目标器官分割图与距离变换图一起输入断裂修复深度学习网络，得到每一个感兴趣区域的断裂修复图，将所有的断裂修复图拼接组合在一起的到完整的目标器官分割结果。Step S4: Perform distance transformation on the target organ segmentation map to obtain a distance transformation map, and input the target organ segmentation map and the distance transformation map into the fracture repair deep learning network to obtain a fracture repair map for each region of interest, and then combine all the fracture repair maps Stitch the combined result to the complete target organ segmentation result.
进一步地，步骤S1中对三维图像数据的预处理包括以下步骤：像素值规范化和插值。Further, the preprocessing of the three-dimensional image data in step S1 includes the following steps: pixel value normalization and interpolation.
进一步地，像素值规范化的方法为：Further, the method for normalizing pixel values is:
min＝c-w/2min=c-w/2
max＝c+w/2max=c+w/2
if x＜min then x＝0if x＜min then x=0
if x＞max then x＝1if x＞max then x=1
Figure PCTCN2020139471-appb-000001
Figure PCTCN2020139471-appb-000001
其中c为窗位，w为窗宽，x为像素值。Where c is the window level, w is the window width, and x is the pixel value.
进一步地，插值的方法为：先解析原始三维图像，获取图像Spacing属性(z 0,y 0,x 0)；其中，目标图像Spacing为(z 1,y 1,x 1)，计算各个方向的缩放因子(z 0/z 1,y 0/y 1,x 0/x 1)，再根据各方向的缩放因子，利用双线性插值，得到插值后的三维数据。 Further, the interpolation method is: first parse the original three-dimensional image, obtain the image Spacing attributes (z 0 , y 0 , x 0 ); among them, the target image Spacing is (z 1 , y 1 , x 1 ), and calculate the Scaling factor (z 0 /z 1 , y 0 /y 1 , x 0 /x 1 ), and then according to the scaling factor in each direction, bilinear interpolation is used to obtain the interpolated three-dimensional data.
进一步地，步骤S2中的重采样的方法为：将插值后的三维数据，按照步长为32×48×48，提取大小为64×96×96且有相互重叠的局部片块三维图像。Further, the method of resampling in step S2 is: taking the interpolated three-dimensional data according to the step size of 32×48×48, and extracting the three-dimensional image of the local patches with the size of 64×96×96 and overlapping each other.
进一步地，目标分割深度学习网络利用卷积残差块和降采样交替进行4次，提取高维特征，然后再通过卷积残差块和上采样交替进行4次，恢复到原来输入图像的分辨率。Further, the target segmentation deep learning network uses the convolution residual block and downsampling alternately 4 times to extract high-dimensional features, and then alternately performs 4 times through the convolution residual block and upsampling to restore the resolution of the original input image Rate.
进一步地，目标分割深度学习网络在上采样后的特征图与低层次同分辨率的特征图进行融合。Further, the feature map of the target segmentation deep learning network after upsampling is fused with the low-level feature map of the same resolution.
进一步地，目标分割深度学习网络将不同深度的目标器官特征图利用上 采样进行特征融合。Further, the target segmentation deep learning network uses up-sampling feature maps of target organs of different depths to perform feature fusion.
进一步地，断裂修复深度学习网络利用卷积残差块和降采样交替进行1次，提取高维特征，然后再通过卷积残差块和上采样交替进行1次，恢复原来输入图像的分辨率。Furthermore, the fracture repair deep learning network uses the convolution residual block and downsampling alternately once to extract high-dimensional features, and then alternates the convolution residual block and upsampling once to restore the original input image resolution .
第二方面，本发明实施例提供一种基于深度学习网络的三维影像的管状结构分割图断裂修复系统，包括：In a second aspect, an embodiment of the present invention provides a three-dimensional image-based deep learning network-based pipe structure segmentation map fracture repair system, including:
预处理模块：用于对三维图像数据进行预处理；Preprocessing module: used to preprocess 3D image data;
图像重采样模块：用于对预处理后的三维影像数据进行重采样，生成多个候选的感兴趣区域的三维数据；Image resampling module: used to resample the preprocessed 3D image data to generate multiple candidate 3D data of the region of interest;
目标器官提取模块：用于将所有候选感区域的三维数据输入目标分割深度学习网络，得到每一个感兴趣区域内的目标器官分割图；Target organ extraction module: used to input the three-dimensional data of all candidate sense regions into the target segmentation deep learning network to obtain the target organ segmentation map in each region of interest;
断裂修复模块：用于将目标器官分割图进行距离变换得到距离变换图，将目标器官分割图与距离变换图一起输入断裂修复深度学习网络，得到每一个感兴趣区域的断裂修复图，将所有的断裂修复图拼接组合在一起的到完整的目标器官分割结果。Fracture repair module: used to perform distance transformation on the target organ segmentation map to obtain the distance transformation map. The target organ segmentation map and the distance transformation map are input into the fracture repair deep learning network to obtain the fracture repair map for each region of interest. Fracture repair images are stitched together to complete the segmentation result of the target organ.
本发明实施例提供的一种基于深度学习网络的三维影像的管状结构分割图断裂修复系统具有如下优点：该三维影像的管状结构分割图断裂修复方法及系统利用断裂处的距离变换信息作为断裂位置识别信息，无需事先检测断裂位置，即可实现自动断裂修复，简单而有效，提高了分割效果。The embodiment of the present invention provides a three-dimensional image-based tubular structure segmentation map fracture repair system based on a deep learning network, which has the following advantages: the three-dimensional image of the tubular structure segmentation map fracture repair method and system uses the distance transformation information of the fracture as the fracture location The identification information can realize automatic fracture repair without detecting the fracture position in advance, which is simple and effective, and improves the segmentation effect.
附图说明Description of the drawings
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.
图1为本发明实施例提供的基于深度学习网络的三维影像的管状结构分割图断裂修复方法流程图；FIG. 1 is a flowchart of a method for repairing a fracture of a tubular structure segmentation map of a three-dimensional image based on a deep learning network according to an embodiment of the present invention;
图2为本发明实施例提供的方法中目标分割深度学习网络结构示意图；2 is a schematic diagram of the structure of a deep learning network for target segmentation in a method provided by an embodiment of the present invention;
图3为本发明实施例提供的方法中断裂修复深度学习网络结构示意图；FIG. 3 is a schematic diagram of the structure of a fracture repair deep learning network in a method provided by an embodiment of the present invention;
图4本发明实施例提供的基于深度学习网络的三维影像的管状结构分割图断裂修复系统的原理图。Fig. 4 is a schematic diagram of a fracture repair system for a tubular structure segmentation diagram of a three-dimensional image based on a deep learning network provided by an embodiment of the present invention.
具体实施方式Detailed ways
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。In order to make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.
除非另有其它明确表示，否则在整个说明书和权利要求书中，术语“包括”或其变换如“包含”或“包括有”等等将被理解为包括所陈述的元件或组成部分，而并未排除其它元件或其它组成部分。Unless otherwise expressly stated otherwise, throughout the specification and claims, the term "comprising" or its transformations such as "including" or "including" will be understood to include the stated elements or components, and not Other elements or other components are not excluded.
图1为本发明实施例提供的一种基于深度学习网络的三维影像的管状结构分割图断裂修复方法流程图，如图1所示，本发明提出的一种基于深度学习网络的三维影像的管状结构分割图断裂修复方法，包括下述步骤：FIG. 1 is a flowchart of a method for repairing a fracture of a three-dimensional image of a tubular structure based on a deep learning network provided by an embodiment of the present invention. As shown in FIG. The method for repairing the structure segmentation diagram fracture includes the following steps:
步骤S1：采集三维影像数据，并进行预处理；Step S1: Collect three-dimensional image data and perform preprocessing;
本发明实施例的步骤S1中，对三维图像数据的预处理包括以下步骤：像素值规范化和插值。In step S1 of the embodiment of the present invention, the preprocessing of the three-dimensional image data includes the following steps: pixel value normalization and interpolation.
其中，像素值规范化的方法为：Among them, the method of pixel value normalization is:
min＝c-w/2min=c-w/2
max＝c+w/2max=c+w/2
if x＜min then x＝0if x＜min then x=0
if x＞max then x＝1if x＞max then x=1
Figure PCTCN2020139471-appb-000002
Figure PCTCN2020139471-appb-000002
其中c为窗位，w为窗宽，x为像素值。Where c is the window level, w is the window width, and x is the pixel value.
插值的方法为：先解析原始三维图像，获取图像Spacing属性(z 0,y 0,x 0)；其中，目标图像Spacing为(z 1,y 1,x 1)，计算各个方向的缩放因子(z 0/z 1,y 0/y 1,x 0/x 1)， 再根据各方向的缩放因子，利用双线性插值，得到插值后的三维数据。 The interpolation method is: first parse the original three-dimensional image, obtain the image Spacing attribute (z 0 , y 0 , x 0 ); among them, the target image Spacing is (z 1 , y 1 , x 1 ), and calculate the zoom factor ( z 0 /z 1 , y 0 /y 1 , x 0 /x 1 ), and then according to the scaling factors in each direction, bilinear interpolation is used to obtain interpolated three-dimensional data.
步骤S2：对预处理后的三维影像数据进行重采样，生成多个候选的感兴趣区域的三维数据；Step S2: Re-sampling the pre-processed 3D image data to generate multiple candidate 3D data of the region of interest;
本发明实施例的步骤S2中，重采样方法为：将插值后的三维数据，按照步长为32×48×48，提取大小为64×96×96且有相互重叠的局部片块三维图像。In step S2 of the embodiment of the present invention, the re-sampling method is: taking the interpolated three-dimensional data according to the step size of 32×48×48, and extracting the three-dimensional image of the local patch with the size of 64×96×96 and overlapping each other.
步骤S3：将所有候选感区域的三维数据输入目标分割深度学习网络，得到每一个感兴趣区域内的目标器官分割图；Step S3: Input the three-dimensional data of all candidate sense regions into the target segmentation deep learning network to obtain a target organ segmentation map in each region of interest;
步骤S4：将目标器官分割图进行距离变换得到距离变换图，将目标器官分割图与距离变换图一起输入断裂修复深度学习网络，得到每一个感兴趣区域的断裂修复图，将所有的断裂修复图拼接组合在一起的到完整的目标器官分割结果。Step S4: Perform distance transformation on the target organ segmentation map to obtain a distance transformation map, and input the target organ segmentation map and the distance transformation map into the fracture repair deep learning network to obtain a fracture repair map for each region of interest, and then combine all the fracture repair maps Stitch the combined result to the complete target organ segmentation result.
该三维影像动静脉分割方法的深度学习网络采用了目标分割深度学习网络与断裂修复深度学习网络的两阶段的识别，即先用目标分割深度学习网络进目标器官提取，得到目标器官分割图，再将目标器官分割图进行距离变换得到距离变换图，将目标器官分割图与距离变换图一起输入断裂修复深度学习网络进行断裂修复。The deep learning network of the three-dimensional image arteriovenous segmentation method adopts the two-stage recognition of the target segmentation deep learning network and the fracture repair deep learning network, that is, first use the target segmentation deep learning network to extract the target organ to obtain the target organ segmentation map, and then The target organ segmentation map is transformed by distance to obtain the distance transformation map, and the target organ segmentation map and the distance transformation map are input into the fracture repair deep learning network for fracture repair.
其中，目标分割深度学习网络与断裂修复深度学习网络都是输出概率图。Among them, the target segmentation deep learning network and the fracture repair deep learning network are both output probability maps.
其中，如图2所示，目标分割深度学习网络利用卷积残差块和降采样交替进行4次，提取高维特征，然后再通过卷积残差块和上采样交替进行4次，恢复到原来输入图像的分辨率。Among them, as shown in Figure 2, the target segmentation deep learning network uses the convolution residual block and downsampling alternately 4 times to extract high-dimensional features, and then alternately performs 4 times through the convolution residual block and upsampling to restore to The resolution of the original input image.
其中，为了能够补充低层次的与位置相关的信息，目标分割深度学习网络在上采样后的特征图与低层次同分辨率的特征图进行融合。Among them, in order to be able to supplement the low-level location-related information, the up-sampled feature map of the target segmentation deep learning network is merged with the low-level feature map of the same resolution.
其中，为了加强不同尺寸的目标器官的关注，目标分割深度学习网络将不同深度的目标器官特征图利用上采样进行特征融合。Among them, in order to strengthen the attention of target organs of different sizes, the target segmentation deep learning network uses up-sampling for feature fusion of target organ feature maps of different depths.
如图3所示，断裂修复深度学习网络利用卷积残差块和降采样交替进行1次，提取高维特征，然后再通过卷积残差块和上采样交替进行1次，恢复原来输入图像的分辨率。As shown in Figure 3, the fracture repair deep learning network uses the convolution residual block and downsampling alternately once to extract high-dimensional features, and then alternates the convolution residual block and upsampling once to restore the original input image Resolution.
其中，为了扩大感受野并捕抓多尺度特征，断裂修复深度学习网络在第4、5、6、7、8的残差块里，使用了dilated卷积；并利用skip connection，补充低层次的细节信息。Among them, in order to expand the receptive field and capture multi-scale features, the fracture repair deep learning network uses dilated convolution in the 4th, 5th, 6th, 7th, and 8th residual blocks; and uses skip connections to supplement low-level Detailed information.
基于上述任一实施例，图4为本发明实施例提供的一种基于深度学习网络的三维影像的管状结构分割图断裂修复系统的结构示意图，该系统包括：Based on any of the foregoing embodiments, FIG. 4 is a schematic structural diagram of a pipe structure segmentation map fracture repair system based on a deep learning network based on a three-dimensional image of a deep learning network provided by an embodiment of the present invention, and the system includes:
预处理模块：用于对三维图像数据进行预处理；Preprocessing module: used to preprocess 3D image data;
图像重采样模块：用于对预处理后的三维影像数据进行重采样，生成多个候选的感兴趣区域的三维数据；Image resampling module: used to resample the preprocessed 3D image data to generate multiple candidate 3D data of the region of interest;
目标器官提取模块：用于将所有候选感区域的三维数据输入目标分割深度学习网络，得到每一个感兴趣区域内的目标器官分割图；Target organ extraction module: used to input the three-dimensional data of all candidate sense regions into the target segmentation deep learning network to obtain the target organ segmentation map in each region of interest;
断裂修复模块：用于将目标器官分割图进行距离变换得到距离变换图，将目标器官分割图与距离变换图一起输入断裂修复深度学习网络，得到每一个感兴趣区域的断裂修复图，将所有的断裂修复图拼接组合在一起的到完整的目标器官分割结果。Fracture repair module: used to perform distance transformation on the target organ segmentation map to obtain the distance transformation map. The target organ segmentation map and the distance transformation map are input into the fracture repair deep learning network to obtain the fracture repair map for each region of interest. Fracture repair images are stitched together to complete the segmentation result of the target organ.
综上所述，本发明实施例提供的基于深度学习网络的三维影像的管状结构分割图断裂修复系统利用断裂处的距离变换信息作为断裂位置识别信息，无需事先检测断裂位置，即可实现自动断裂修复，简单而有效，提高了分割效果。In summary, the fracture repair system of the tubular structure segmentation map based on the three-dimensional image of the deep learning network provided by the embodiment of the present invention uses the distance transformation information of the fracture as the fracture location identification information, and can realize automatic fracture without detecting the fracture location in advance. The repair is simple and effective and improves the segmentation effect.
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下，即可以理解并实施。The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments. Those of ordinary skill in the art can understand and implement it without creative work.
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，上述技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可 读存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。Through the description of the above implementation manners, those skilled in the art can clearly understand that each implementation manner can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM/RAM, magnetic A disc, an optical disc, etc., include several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiment.
最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions recorded in the foregoing embodiments are modified, or some of the technical features thereof are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (1)

  1. 一种基于深度学习网络的三维影像的管状结构分割图断裂修复系统，其特征在于，包括：A three-dimensional image-based tubular structure segmentation map fracture repair system based on a deep learning network is characterized in that it includes:
    预处理模块：用于对三维图像数据进行预处理；Preprocessing module: used to preprocess 3D image data;
    图像重采样模块：用于对预处理后的三维影像数据进行重采样，生成多个候选的感兴趣区域的三维数据；Image resampling module: used to resample the preprocessed 3D image data to generate multiple candidate 3D data of the region of interest;
    目标器官提取模块：用于将所有候选感区域的三维数据输入目标分割深度学习网络，得到每一个感兴趣区域内的目标器官分割图；所述目标分割深度学习网络利用卷积残差块和降采样交替进行4次，提取高维特征，然后再通过卷积残差块和上采样交替进行4次，恢复到原来输入图像的分辨率；所述目标分割深度学习网络在上采样后的特征图与低层次同分辨率的特征图进行融合；所述目标分割深度学习网络将不同深度的目标器官特征图利用上采样进行特征融合Target organ extraction module: used to input the three-dimensional data of all candidate sense regions into the target segmentation deep learning network to obtain the target organ segmentation map in each region of interest; the target segmentation deep learning network uses convolution residual blocks and reduction Sampling is performed alternately 4 times, high-dimensional features are extracted, and then convolution residual blocks and upsampling are performed alternately 4 times to restore the original input image resolution; the target segmentation deep learning network's feature map after upsampling Fuse with low-level feature maps of the same resolution; the target segmentation deep learning network uses up-sampling to perform feature fusion of target organ feature maps of different depths
    断裂修复模块：用于将目标器官分割图进行距离变换得到距离变换图，将目标器官分割图与距离变换图一起输入断裂修复深度学习网络，得到每一个感兴趣区域的断裂修复图，将所有的断裂修复图拼接组合在一起的到完整的目标器官分割结果；所述断裂修复深度学习网络利用卷积残差块和降采样交替进行1次，提取高维特征，然后再通过卷积残差块和上采样交替进行1次，恢复原来输入图像的分辨率。Fracture repair module: used to perform distance transformation on the target organ segmentation map to obtain the distance transformation map. The target organ segmentation map and the distance transformation map are input into the fracture repair deep learning network to obtain the fracture repair map for each region of interest. Fracture repair images are spliced and combined together to obtain a complete target organ segmentation result; the fracture repair deep learning network uses convolution residual blocks and downsampling alternately once, extracts high-dimensional features, and then passes convolution residual blocks Alternate with upsampling once to restore the resolution of the original input image.
PCT/CN2020/139471 2020-02-18 2020-12-25 Tubular structure segmentation map fracture restoration system for three-dimensional image based on deep learning network WO2021164418A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
CN202011457993.7 2020-02-18
CN202010098343.1A CN111260670B (en) 2020-02-18 2020-02-18 Tubular structure segmentation graph fracture repairing method and system of three-dimensional image based on deep learning network
CN202011457993.7A CN112508969B (en) 2020-02-18 2020-02-18 Tubular structure segmentation graph fracture repair system of three-dimensional image based on deep learning network

Publications (1)

Publication Number Publication Date
WO2021164418A1 true WO2021164418A1 (en) 2021-08-26

Family

ID=70952766

Family Applications (2)

Application Number Title Priority Date Filing Date
PCT/CN2020/139468 WO2021164417A1 (en) 2020-02-18 2020-12-25 Deep learning network based fracture repairing method for three-dimensional image tubular structure segmentation diagram
PCT/CN2020/139471 WO2021164418A1 (en) 2020-02-18 2020-12-25 Tubular structure segmentation map fracture restoration system for three-dimensional image based on deep learning network

Family Applications Before (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/139468 WO2021164417A1 (en) 2020-02-18 2020-12-25 Deep learning network based fracture repairing method for three-dimensional image tubular structure segmentation diagram

Country Status (2)

Country Link
CN (2) CN111260670B (en)
WO (2) WO2021164417A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113808146A (en) * 2021-10-18 2021-12-17 山东大学 Medical image multi-organ segmentation method and system
CN114049359A (en) * 2021-11-22 2022-02-15 北京航空航天大学 Medical image organ segmentation method

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111260670B (en) * 2020-02-18 2021-02-19 广州柏视医疗科技有限公司 Tubular structure segmentation graph fracture repairing method and system of three-dimensional image based on deep learning network

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109377458A (en) * 2018-09-30 2019-02-22 数坤（北京）网络科技有限公司 A kind of restorative procedure and device of coronary artery segmentation fracture
US20190221314A1 (en) * 2018-01-12 2019-07-18 Siemens Medical Solutions Usa, Inc. Integrated precision medicine by combining quantitative imaging techniques with quantitative genomics for improved decision making
CN110765856A (en) * 2019-09-12 2020-02-07 南京邮电大学 Convolution-based low-quality finger vein image edge detection algorithm
CN111260670A (en) * 2020-02-18 2020-06-09 广州柏视医疗科技有限公司 Tubular structure segmentation graph fracture repairing method and system of three-dimensional image based on deep learning network

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6785409B1 (en) * 2000-10-24 2004-08-31 Koninklijke Philips Electronics, N.V. Segmentation method and apparatus for medical images using diffusion propagation, pixel classification, and mathematical morphology
US20080159604A1 (en) * 2005-12-30 2008-07-03 Allan Wang Method and system for imaging to identify vascularization
CN103295195B (en) * 2013-05-16 2017-07-07 深圳市旭东数字医学影像技术有限公司 The enhanced method of vascular and its system of soft image
CN106204733B (en) * 2016-07-22 2024-04-19 青岛大学附属医院 Liver and kidney CT image combined three-dimensional construction system
JP7184489B2 (en) * 2016-09-18 2022-12-06 イェダ リサーチ アンド ディベロップメント カンパニー リミテッド Systems and methods for generating 3D images based on fluorescent illumination
KR101754291B1 (en) * 2017-04-04 2017-07-06 이현섭 Medical image processing system and method for personalized brain disease diagnosis and status determination
US10709399B2 (en) * 2017-11-30 2020-07-14 Shenzhen Keya Medical Technology Corporation Methods and devices for performing three-dimensional blood vessel reconstruction using angiographic images
CN109523507B (en) * 2018-09-26 2023-09-19 苏州六莲科技有限公司 Method and device for generating lesion image and computer readable storage medium
CN109448005B (en) * 2018-10-31 2019-12-27 数坤（北京）网络科技有限公司 Network model segmentation method and equipment for coronary artery
CN109583576B (en) * 2018-12-17 2020-11-06 上海联影智能医疗科技有限公司 Medical image processing device and method
CN109903292A (en) * 2019-01-24 2019-06-18 西安交通大学 A kind of three-dimensional image segmentation method and system based on full convolutional neural networks
CN109785325A (en) * 2019-01-30 2019-05-21 陕西中医药大学 A method of the Multimodal medical image based on deep learning
CN109934812B (en) * 2019-03-08 2022-12-09 腾讯科技（深圳）有限公司 Image processing method, image processing apparatus, server, and storage medium
CN109934816B (en) * 2019-03-21 2021-05-11 数坤（北京）网络科技有限公司 Method and device for complementing model and computer readable storage medium
CN109961446B (en) * 2019-03-27 2021-06-01 深圳视见医疗科技有限公司 CT/MR three-dimensional image segmentation processing method, device, equipment and medium
CN110288524B (en) * 2019-05-09 2020-10-30 广东启迪图卫科技股份有限公司 Deep learning super-resolution method based on enhanced upsampling and discrimination fusion mechanism
CN110287956B (en) * 2019-06-13 2021-05-25 北京理工大学 Automatic matching method and device for blood vessel central lines
CN110599465B (en) * 2019-08-28 2022-07-26 上海联影智能医疗科技有限公司 Image positioning method and device, computer equipment and storage medium
CN110570446A (en) * 2019-09-20 2019-12-13 河南工业大学 Fundus retina image segmentation method based on generation countermeasure network

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190221314A1 (en) * 2018-01-12 2019-07-18 Siemens Medical Solutions Usa, Inc. Integrated precision medicine by combining quantitative imaging techniques with quantitative genomics for improved decision making
CN109377458A (en) * 2018-09-30 2019-02-22 数坤（北京）网络科技有限公司 A kind of restorative procedure and device of coronary artery segmentation fracture
CN110765856A (en) * 2019-09-12 2020-02-07 南京邮电大学 Convolution-based low-quality finger vein image edge detection algorithm
CN111260670A (en) * 2020-02-18 2020-06-09 广州柏视医疗科技有限公司 Tubular structure segmentation graph fracture repairing method and system of three-dimensional image based on deep learning network

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113808146A (en) * 2021-10-18 2021-12-17 山东大学 Medical image multi-organ segmentation method and system
CN113808146B (en) * 2021-10-18 2023-08-18 山东大学 Multi-organ segmentation method and system for medical image
CN114049359A (en) * 2021-11-22 2022-02-15 北京航空航天大学 Medical image organ segmentation method
CN114049359B (en) * 2021-11-22 2024-04-16 北京航空航天大学 Medical image organ segmentation method

Also Published As

Publication number Publication date
CN111260670B (en) 2021-02-19
CN112508969B (en) 2021-12-07
CN111260670A (en) 2020-06-09
CN112508969A (en) 2021-03-16
WO2021164417A1 (en) 2021-08-26

Similar Documents

Publication Publication Date Title
WO2021164418A1 (en) Tubular structure segmentation map fracture restoration system for three-dimensional image based on deep learning network
CN108765363B (en) Coronary artery CTA automatic post-processing system based on artificial intelligence
CN112950651B (en) Automatic delineation method of mediastinal lymph drainage area based on deep learning network
CN109063710B (en) 3D CNN nasopharyngeal carcinoma segmentation method based on multi-scale feature pyramid
WO2022057078A1 (en) Real-time colonoscopy image segmentation method and device based on ensemble and knowledge distillation
US8194936B2 (en) Optimal registration of multiple deformed images using a physical model of the imaging distortion
CN110544205B (en) Image super-resolution reconstruction method based on visible light and infrared cross input
CN107358576A (en) Depth map super resolution ratio reconstruction method based on convolutional neural networks
CN110110808B (en) Method and device for performing target labeling on image and computer recording medium
CN110120055B (en) Fundus fluorography image non-perfusion area automatic segmentation method based on deep learning
CN113012155A (en) Bone segmentation method in hip image, electronic device, and storage medium
KR20190105180A (en) Apparatus for Lesion Diagnosis Based on Convolutional Neural Network and Method thereof
US11830193B2 (en) Recognition method of intracranial vascular lesions based on transfer learning
Sreelakshmy et al. An automated deep learning model for the cerebellum segmentation from fetal brain images
CN115471470A (en) Esophageal cancer CT image segmentation method
CN116228785A (en) Pneumonia CT image segmentation method based on improved Unet network
CN113221945B (en) Dental caries identification method based on oral panoramic film and dual attention module
Zheng et al. Multi-strategy mutual learning network for deformable medical image registration
CN107993277B (en) priori knowledge-based reconstruction method of artificial bone repair model of damaged part
CN111325756A (en) Three-dimensional image artery and vein segmentation method and system based on deep learning network
CN111918611A (en) Abnormal display control method for chest X-ray image, abnormal display control program, abnormal display control device, and server device
CN114170221A (en) Method and system for confirming brain diseases based on images
Ahmed et al. Edge detection for roof images using transfer learning
CN117726822B (en) Three-dimensional medical image classification segmentation system and method based on double-branch feature fusion
CN117911705B (en) Brain MRI (magnetic resonance imaging) tumor segmentation method based on GAN-UNet variant network

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20920359

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20920359

Country of ref document: EP

Kind code of ref document: A1