WO2021128230A1 - Deep learning-based medical image processing method and system, and computer device - Google Patents

Deep learning-based medical image processing method and system, and computer device Download PDF

Info

Publication number
WO2021128230A1
WO2021128230A1 PCT/CN2019/128924 CN2019128924W WO2021128230A1 WO 2021128230 A1 WO2021128230 A1 WO 2021128230A1 CN 2019128924 W CN2019128924 W CN 2019128924W WO 2021128230 A1 WO2021128230 A1 WO 2021128230A1
Authority
WO
WIPO (PCT)
Prior art keywords
deep learning
data
learning model
interest
region
Prior art date
Application number
PCT/CN2019/128924
Other languages
French (fr)
Chinese (zh)
Inventor
刘非
Original Assignee
上海昕健医疗技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海昕健医疗技术有限公司 filed Critical 上海昕健医疗技术有限公司
Priority to PCT/CN2019/128924 priority Critical patent/WO2021128230A1/en
Publication of WO2021128230A1 publication Critical patent/WO2021128230A1/en

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00Apparatus for radiation diagnosis, e.g. combined with radiation therapy equipment
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/40Image enhancement or restoration by the use of histogram techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation

Definitions

  • the present invention relates to the field of medical image processing, in particular to a medical image processing method, system and computer equipment based on deep learning.
  • Accurate medical image segmentation is a solid foundation for subsequent doctors to diagnose and formulate surgical plans. If the results of image segmentation are biased, it will affect the safety and effectiveness of the operation. Therefore, efficient and high-precision image segmentation methods are essential. However, the current image segmentation for complex parts is still unable to obtain good results.
  • the ankle includes 26 bones such as the calcaneus, talus, navicular, internal cuneiform, intermediate cuneiform, external cuneiform, cuboid, five metatarsals, and phalanges (including 14 bones in the phalanges).
  • the bones are closely connected, the space between the bones at the bone joints is small, and the gray scale of the bone joint is similar to the gray scale of the surrounding tissues, which makes the segmentation of the bone and joint difficult.
  • Using traditional methods to segment these 26 bones is difficult to achieve satisfactory results. It is easy to cause incomplete segmentation at the bone joints, leakage of segmentation boundaries, etc., which requires cumbersome subsequent editing and processing; manual calibration usually requires a lot of cost.
  • the manpower, material resources and time required, and the calibration results of different people have certain differences, which will affect the accuracy of bone and joint segmentation.
  • the purpose of the present invention is to provide a medical image processing method, system and computer equipment based on deep learning that can quickly and accurately segment images of complex parts.
  • the present invention provides a deep learning model design method, including the following steps: S11, collecting two-dimensional medical image data containing the part to be segmented; S12, labeling the image data of the region of interest to obtain the initial label Data set; S13, divide the initial labeled data set into a training set and a test set; S14, select and design a deep learning model to obtain an initial deep learning model; the specific steps are as follows: select a deep learning model; according to the selected deep learning Model, add an adaptive layer to the first layer of the network to obtain a deep learning design model; according to the deep learning design model, add a custom loss function loss to the tail layer of the network to obtain the initial deep learning model; S15.
  • Adjust Deep learning initial model hyperparameters training the deep learning initial model according to the obtained training set data to obtain a deep learning training model; S16, testing the deep learning training model according to the obtained test set data , Calculate the test coefficient; S17, determine whether the test coefficient reaches a fixed parameter value or more, the judgment result is yes, then the model parameters after the deep learning training meet the requirements, the training ends, and the final deep learning model is obtained; otherwise, deep learning training After the model parameters do not meet the requirements, return to step S15.
  • the activation function of the adaptive layer is Where x is the input of the adaptive layer, and ⁇ is the adaptive threshold of x; the loss function Where ⁇ is a constant, which is a custom hyperparameter; m represents the segmentation category; Vol(A_perd i ) represents the predicted volume of the i-th category of bone; Vol(A_true i ) represents the i-th category of bone artificial The marked volume; Vol(A_pred_true i ) represents the correct volume predicted by the i-th category; Vol(A_pred i ⁇ A_true i ) represents the volume obtained after the merging of the bone predicted by the i-th category with the artificially labeled bone.
  • the method for selecting a deep learning model includes the following steps: S31. Divide the training set data into N pieces, where N-1 pieces are used as training data and 1 piece is used as verification data, where N is an integer greater than 1; S32 Select a deep learning training model as the preliminary selected deep learning model; S33. According to the obtained training data, train the preliminary selected deep learning model, and verify the preliminary selected deep learning model according to the obtained verification data.
  • step S34 non-repetitively select another piece of the N training set data in step S31 as the verification data, and the remaining N-1 pieces as the training data, repeat the operation of step S33; S35, steps S33 and S34 repeat After N times, record the average of the errors recorded for N times to obtain the average error of the selected deep learning model; S36, select other deep learning models as the preliminary selected deep learning models, and repeat steps S33 to S35 to obtain other deep learning The average error of the model; S37. Select the deep learning model with the smallest average error as the final selected deep learning model.
  • the calculation steps of the adaptive threshold ⁇ are as follows: S41, traverse the input data of the first layer of the network, and calculate the average value ⁇ ; S42, use the average value ⁇ as the threshold to divide the input data of the first layer of the network into the foreground and the background.
  • a medical image processing method based on deep learning which is a deep learning model designed based on the deep learning model design method of claim 1, comprising the following steps: reading two-dimensional medical image data; Preprocess the three-dimensional medical image data to obtain the initial data; extract the data of the region of interest according to the obtained initial data; scale the data of the region of interest to a fixed size through the interpolation method to obtain the scaled region of interest data; The scaled region of interest data is standardized to obtain input data of a deep learning model; the final deep learning model is run to obtain an initial segmentation result; the initial segmentation result is post-processed to obtain a final segmentation result.
  • the preprocessing method is filtering processing.
  • a medical image processing system based on deep learning comprising: a reading module for reading two-dimensional medical image data; a preprocessing module for preprocessing the read two-dimensional medical image data, Obtain training data; an extraction module for extracting data of the region of interest according to the obtained training data; a scaling module for scaling the data of the region of interest to a fixed size through an interpolation method to obtain the scaled region of interest data ; Standardization module, used to standardize the scaled region of interest data to obtain input data of the deep learning model; Run module, used to run the final deep learning model to obtain the initial segmentation results; Post-processing module, used to The initial segmentation result is post-processed to obtain the final segmentation result.
  • a computer device includes a memory and a processor, and a computer program is stored in the memory.
  • the processor executes the computer program, the following steps are implemented: reading two-dimensional medical image data; Perform preprocessing of the two-dimensional medical image data of the two-dimensional medical image to obtain training data; extract the region of interest data according to the obtained training data; scale the region of interest data to a fixed size through an interpolation method to obtain the scaled region of interest data Standardize the scaled region of interest data to obtain the input data of the deep learning model; run the final deep learning model to obtain the initial segmentation result; post-process the initial segmentation result to obtain the final segmentation result.
  • a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented: reading two-dimensional medical image data; preprocessing the read two-dimensional medical image data, Obtain the training data; extract the region of interest data according to the obtained training data; scale the region of interest data to a fixed size by interpolation method to obtain the scaled region of interest data; convert the scaled region of interest Standardize data processing to obtain the input data of the deep learning model; run the final deep learning model to obtain the initial segmentation result;
  • the medical image processing method, system and computer equipment based on deep learning of the present invention adopts a fully automatic segmentation algorithm, which can automatically and finely separate the calcaneus, talus, navicular bone, and internal cuneiform from the entire ankle within tens of seconds 26 bones, including the middle cuneiform, the external cuneiform, the cuboid, the first metatarsal, the second metatarsal, the third metatarsal, the fourth metatarsal, the fifth metatarsal and the phalanges, for the doctor’s reference. , Perform a specific analysis of the patient to obtain the diagnosis result, thereby reducing the workload of the doctor, improving the doctor's work efficiency and the accuracy of the diagnosis.
  • Fig. 1 is a schematic diagram of a method for designing a deep learning model for medical image processing according to the present invention.
  • Figure 2 is a schematic diagram of the process of selecting and designing the deep learning model in Figure 1 to obtain the initial deep learning model.
  • Fig. 3 is a schematic diagram of the process of selecting the deep learning model in Fig. 2.
  • Fig. 4 is a schematic flowchart of the calculation method of the adaptive threshold ⁇ in Fig. 2.
  • Fig. 5 is a schematic flowchart of a medical image processing method based on deep learning of the present invention.
  • FIG. 6 is a schematic flowchart of the method for normalizing the zoomed region of interest data in FIG. 5.
  • the first embodiment provides a deep learning model design method for medical image processing, including the following steps:
  • step S17 Determine whether the test coefficient reaches a fixed parameter value or more, and the judgment result is yes, then the model parameters after the deep learning training meet the requirements, and the training ends, and the final deep learning model is obtained; otherwise, the model parameters after the deep learning training do not meet the requirements , Return to step S15.
  • the part to be divided in step S11 is the ankle part
  • the number of the two-dimensional medical image data is more than 100 sets
  • the size of each set of data may be 512*512 pixels*100 sheets.
  • the marking of the image data of the region of interest in step S12 is to organize some qualified physicians to manually outline the calcaneus, talus, navicular bone, internal cuneiform, and intermediate cuneiform in the image data of the region of interest.
  • 26 bones including the external cuneiform, cuboid, first metatarsal, second metatarsal, third metatarsal, fourth metatarsal, fifth metatarsal, and phalanges, to obtain the initial labeled data set.
  • step S13 the initial labeled data set is divided into a training set and a test set, and the division operation may be performed in a random allocation manner, and the allocation ratio of the training set and the test set may be 80% and 20%.
  • the test coefficient in step S16 is the index DICE coefficient of the test set data segmentation result; the fixed parameter value in step S17 is 90%.
  • the method of selecting and designing a deep learning model in step S14 to obtain an initial deep learning model includes the following steps:
  • the deep learning model includes but not limited to FCN, Res-VNet, U-Net, and V-Net models, and the method for selecting the deep learning model is to pass the training set data through N-fold cross-validation To evaluate the robustness of the selected deep learning model, choose a deep learning model with better robustness;
  • an adaptive layer is added to the first layer of the network to obtain a deep learning design model;
  • the activation function of the adaptive layer is Among them, x is the input of the adaptive layer, and ⁇ is the adaptive threshold of x;
  • the method for selecting a deep learning model in step S21 includes the following steps:
  • step S34 Non-repetitively select another piece of data in the N training set data in step S31 as verification data, and the remaining N-1 pieces as training data, and repeat the operation of step S33;
  • steps S33 and S34 are repeated N times, the average value of the errors recorded for N times is recorded to obtain the average error of the selected deep learning model;
  • the value range of N is 3-10, preferably, the value range of N is 5-10, and more preferably, the value of N in the first embodiment is 10.
  • step S22 the calculation steps of the adaptive threshold ⁇ in step S22 are as follows:
  • the threshold value set in step S45 has a value range of 0.01-10.
  • the threshold value set in the first embodiment is 0.1.
  • the second embodiment provides a medical image processing method based on the deep learning model designed in the first embodiment, including the following steps:
  • S54 Scale the data of the region of interest to a fixed size by an interpolation method to obtain the data of the region of interest after scaling, and the fixed size may be 512*512 pixels;
  • S55 Standardize the scaled region of interest data to obtain input data of the deep learning model
  • the preprocessing method in step S52 is filtering processing, such as median filtering, mean filtering, Gaussian filtering, bilateral filtering or other filtering methods.
  • the method of extracting the region of interest data in step S53 includes: automatically searching all connected regions of the image according to the obtained initial data and according to a set threshold, and growing all connected regions into a separate container; according to the size of each connected region Judge the area of the patient’s ankle with the direction, and extract the area of the ankle as the area of interest data.
  • the method of post-processing the initial segmentation result in step S57 is to modify part of the over-segmented and under-segmented regions, filter out small mis-segmented impurities, and process them through algorithms such as morphological expansion, erosion, and filling. Obtain the final segmentation result.
  • the method for standardizing the zoomed region of interest data in step S55 includes the following steps:
  • the present invention adopts a medical image processing method based on deep learning and a fully automatic segmentation algorithm, which can automatically and finely separate the calcaneus, talus, navicular bone, internal cuneiform, and intermediate cuneiform from the entire ankle within tens of seconds.
  • 26 bones including the external cuneiform, cuboid, first metatarsal, second metatarsal, third metatarsal, fourth metatarsal, fifth metatarsal, and phalanges, for the doctor’s reference.
  • the diagnosis results are analyzed, thereby reducing the workload of the doctors and improving the working efficiency of the doctors and the accuracy of the diagnosis.
  • the third embodiment provides a medical image processing system based on deep learning, and the system includes:
  • a reading module for reading two-dimensional medical image data the two-dimensional medical image data being CT image data complying with the DICOM3.0 standard;
  • the preprocessing module is used to preprocess the read two-dimensional medical image data to obtain training data
  • the extraction module is used to extract the data of the region of interest according to the obtained training data
  • a scaling module configured to scale the region of interest data to a fixed size through an interpolation method to obtain scaled region of interest data, the fixed size may be 512*512 pixels;
  • the standardization module is used to standardize the scaled region of interest data to obtain input data of the deep learning model
  • the running module is used to run the final deep learning model obtained in the first embodiment to obtain the initial segmentation result
  • the post-processing module is used to perform post-processing on the initial segmentation result to obtain the final segmentation result.
  • the various modules in the medical image processing system based on deep learning can be implemented in whole or in part by software, hardware, and a combination thereof.
  • the foregoing modules may be embedded in the form of hardware or independent of the processor in the computer device, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the foregoing modules.
  • the fourth embodiment provides a computer device, the computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when the processor executes the computer program:
  • the fifth embodiment provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:
  • S84 Scale the data of the region of interest to a fixed size by an interpolation method to obtain the data of the region of interest after scaling, and the fixed size may be 512*512 pixels;

Abstract

A deep learning-based medical image processing method and system, and a computer device. A deep learning model design method comprises the following steps: collecting image data (S11); annotating the image data (S12); dividing an initial annotated data set (S13); selecting and designing a deep learning model (S14); adjusting a deep learning initial model hyperparameter and training a deep learning initial model (S15); testing a deep learning trained model (S16); and determining whether a test coefficient reaches a fixed parameter value or more (S17). According to the method, a complex part image can be segmented in a quick and high-precision manner, thereby improving working efficiency and diagnosis accuracy of doctors.

Description

基于深度学习的医学图像处理方法、系统以及计算机设备Medical image processing method, system and computer equipment based on deep learning 技术领域Technical field
本发明涉及医学图像处理领域,尤其涉及一种基于深度学习的医学图像处理方法、系统以及计算机设备。The present invention relates to the field of medical image processing, in particular to a medical image processing method, system and computer equipment based on deep learning.
背景技术Background technique
精确的医学图像分割是后续医生进行诊断、制定手术方案的坚实基础,若图像分割结果存在偏差,将影响手术的安全性及有效性,因此,高效、高精度的图像分割方法至关重要。然而,目前针对复杂部位的图像分割仍然无法获得良好的效果。Accurate medical image segmentation is a solid foundation for subsequent doctors to diagnose and formulate surgical plans. If the results of image segmentation are biased, it will affect the safety and effectiveness of the operation. Therefore, efficient and high-precision image segmentation methods are essential. However, the current image segmentation for complex parts is still unable to obtain good results.
例如,脚踝部分包括跟骨、距骨、舟骨、内楔骨、中间楔骨、外楔骨、骰骨、五块跖骨以及趾骨(趾骨中包含14块骨)等26块骨头。这些部位之间,各块骨骼紧密相连,骨关节处骨与骨间的腔隙较小,且骨关节灰度与周边组织灰度相似,使得骨关节分割难度较大。采用传统的方法对这26块骨头进行分割,难以达到满意效果,容易造成骨关节处的分割不完全,分割边界泄露等情况,需要繁琐的后续编辑处理;使用人工标定的方式,通常需要耗费大量的人力、物力和时间,而且不同人标定的结果具有一定差异性,将影响骨关节分割的精确度。For example, the ankle includes 26 bones such as the calcaneus, talus, navicular, internal cuneiform, intermediate cuneiform, external cuneiform, cuboid, five metatarsals, and phalanges (including 14 bones in the phalanges). Among these parts, the bones are closely connected, the space between the bones at the bone joints is small, and the gray scale of the bone joint is similar to the gray scale of the surrounding tissues, which makes the segmentation of the bone and joint difficult. Using traditional methods to segment these 26 bones is difficult to achieve satisfactory results. It is easy to cause incomplete segmentation at the bone joints, leakage of segmentation boundaries, etc., which requires cumbersome subsequent editing and processing; manual calibration usually requires a lot of cost. The manpower, material resources and time required, and the calibration results of different people have certain differences, which will affect the accuracy of bone and joint segmentation.
目前常用的基于最大流最小分割思想的方法、分水岭方法等,对于脚踝多目标的分割需要人工添加大量的前景与背景标记;基于模板匹配或配准的方法,不仅计算耗时长,而且分割效果难以保证。Currently commonly used methods based on the idea of maximum flow and minimum segmentation, watershed methods, etc., for the segmentation of ankle multi-targets, a large number of foreground and background markers need to be manually added; methods based on template matching or registration not only take a long time to calculate, but also the segmentation effect is difficult. Guarantee.
因此,确有必要提供一种基于深度学习的医学图像处理方法、系统以及计算机设备,以克服现有技术中存在的缺陷。Therefore, it is indeed necessary to provide a medical image processing method, system and computer equipment based on deep learning to overcome the defects in the prior art.
发明内容Summary of the invention
本发明的目的在于提供一种能够快速、高精度地分割出复杂部位图像的基于深度学习的医学图像处理方法、系统以及计算机设备。The purpose of the present invention is to provide a medical image processing method, system and computer equipment based on deep learning that can quickly and accurately segment images of complex parts.
为实现上述目的,本发明提供了一种深度学习模型设计方法,包括如下步骤:S11、收集含有待分割部位的二维医学图像数据;S12、对感兴趣区域的图像数据进行标注,获得初始标注数据集;S13、将所述初始标注数据集划分为训练集和测试集;S14、选择和设计深度学习模型,获得深度学习初始模型;具体步骤如下:选择深度学习模型;根据所选择的深度学习模型,在网络的首层添加一个自适应层,获得深度学习设计模型;根据所述深度学习设计模型,在网络的尾层添加一个自定义的损失函数loss,获得深度学习初始模型;S15、调整深度学习初始模型超参数,根据获得的训练集数据,对所述深度学习初始模型进行训练,获得深度学习训练后模型;S16、根据获得的测试集数据,对所述深度学习训练后模型进行测试,计算获得测试系数;S17、判断所述测试系数是否达到一固定参数值以上,判断结果为是,则深度学习训练后模型参数符合要求,训练结束,获得深度学习最终模型;否则,深度学习训练后模型参数不符合要求,返回步骤S15。In order to achieve the above objective, the present invention provides a deep learning model design method, including the following steps: S11, collecting two-dimensional medical image data containing the part to be segmented; S12, labeling the image data of the region of interest to obtain the initial label Data set; S13, divide the initial labeled data set into a training set and a test set; S14, select and design a deep learning model to obtain an initial deep learning model; the specific steps are as follows: select a deep learning model; according to the selected deep learning Model, add an adaptive layer to the first layer of the network to obtain a deep learning design model; according to the deep learning design model, add a custom loss function loss to the tail layer of the network to obtain the initial deep learning model; S15. Adjust Deep learning initial model hyperparameters, training the deep learning initial model according to the obtained training set data to obtain a deep learning training model; S16, testing the deep learning training model according to the obtained test set data , Calculate the test coefficient; S17, determine whether the test coefficient reaches a fixed parameter value or more, the judgment result is yes, then the model parameters after the deep learning training meet the requirements, the training ends, and the final deep learning model is obtained; otherwise, deep learning training After the model parameters do not meet the requirements, return to step S15.
进一步地,所述自适应层的激活函数为
Figure PCTCN2019128924-appb-000001
其中,x为自适应层的输入,τ为x的自适应阈值;所述损失函数
Figure PCTCN2019128924-appb-000002
其中ω是一个常量,为自 定义的超参数;m表示分割的类别;Vol(A_perd i)表示第i个类别的骨所预测到的体积;Vol(A_true i)表示第i个类别的骨人工标记的体积;Vol(A_pred_true i)表示第i个类别预测正确的体积;Vol(A_pred i∪A_true i)表示第i个类别预测的骨与人工标记的骨相并之后得到的体积。
Further, the activation function of the adaptive layer is
Figure PCTCN2019128924-appb-000001
Where x is the input of the adaptive layer, and τ is the adaptive threshold of x; the loss function
Figure PCTCN2019128924-appb-000002
Where ω is a constant, which is a custom hyperparameter; m represents the segmentation category; Vol(A_perd i ) represents the predicted volume of the i-th category of bone; Vol(A_true i ) represents the i-th category of bone artificial The marked volume; Vol(A_pred_true i ) represents the correct volume predicted by the i-th category; Vol(A_pred i ∪A_true i ) represents the volume obtained after the merging of the bone predicted by the i-th category with the artificially labeled bone.
进一步地,选择深度学习模型的方法包括如下步骤:S31、将所述训练集数据分为N份,其中N-1份作为训练数据,1份作为验证数据,其中N为大于1的整数;S32、选取一个深度学习训练模型,作为初步选择的深度学习模型;S33、根据获得的训练数据,对初步选择的深度学习模型进行训练,根据获得的验证数据,对初步选择的深度学习模型进行验证,同时记录误差;S34、非重复地选取步骤S31中N份训练集数据中的另一份数据作为验证数据,其余N-1份作为训练数据,重复步骤S33的操作;S35、步骤S33和S34重复N次后,记录N次所记录误差的平均值,获得所选择的深度学习模型的平均误差;S36、挑选其他深度学习模型作为初步选择的深度学习模型,重复步骤S33~S35,获得其他深度学习模型的平均误差;S37、选取平均误差最小的深度学习模型,作为最终所选择的深度学习模型。Further, the method for selecting a deep learning model includes the following steps: S31. Divide the training set data into N pieces, where N-1 pieces are used as training data and 1 piece is used as verification data, where N is an integer greater than 1; S32 Select a deep learning training model as the preliminary selected deep learning model; S33. According to the obtained training data, train the preliminary selected deep learning model, and verify the preliminary selected deep learning model according to the obtained verification data. Record the error at the same time; S34, non-repetitively select another piece of the N training set data in step S31 as the verification data, and the remaining N-1 pieces as the training data, repeat the operation of step S33; S35, steps S33 and S34 repeat After N times, record the average of the errors recorded for N times to obtain the average error of the selected deep learning model; S36, select other deep learning models as the preliminary selected deep learning models, and repeat steps S33 to S35 to obtain other deep learning The average error of the model; S37. Select the deep learning model with the smallest average error as the final selected deep learning model.
进一步地,自适应阈值τ的计算步骤如下:S41、遍历网络首层的输入数据,计算平均值μ;S42、用所述平均值μ作为阈值将网络首层输入数据分割为前景、背景两个部分,分别计算所述前景部分与所述背景部分的均值t_fore、t_back;S43、根据获得的前景部分均值、背景部分均值,计算新的阈值μ_new,所述阈值μ_new=(t_fore+t_back)/2;S44、根据获得的新的阈值μ_new,计算获得diff,所述diff=μ_new-μ,并将μ_new的值赋值给μ;S45、重复步骤S42~S44,直至diff小于设定的阈值迭代停止,获得自适应阈值τ。Further, the calculation steps of the adaptive threshold τ are as follows: S41, traverse the input data of the first layer of the network, and calculate the average value μ; S42, use the average value μ as the threshold to divide the input data of the first layer of the network into the foreground and the background. Part, calculate the mean values t_fore and t_back of the foreground part and the background part respectively; S43, calculate a new threshold μ_new according to the obtained mean value of the foreground part and the mean value of the background part, the threshold μ_new=(t_fore+t_back)/2 S44, calculate the diff according to the obtained new threshold μ_new, the diff=μ_new-μ, and assign the value of μ_new to μ; S45, repeat steps S42 to S44 until the diff is less than the set threshold and the iteration stops, Obtain an adaptive threshold τ.
一种基于深度学习的医学图像处理方法,其是基于权利要求1所述的深度学习模型设计方法所设计的深度学习模型,包括如下步骤:读取二维医学图像数据;对读取后的二维医学图像数据进行预处理,获得初始数据;根据获得的初始数据,提取感兴趣区域数据;将所述感兴趣区域数据通过插值方法缩放到固定尺寸大小,获得缩放后的感兴趣区域数据;将所述缩放后的感兴趣区域数据标准化处理,获得深度学习模型的输入数据;运行所述深度学习最终模型,得到初始分割结果;对所述初始分割结果进行后处理,获得最终分割结果。A medical image processing method based on deep learning, which is a deep learning model designed based on the deep learning model design method of claim 1, comprising the following steps: reading two-dimensional medical image data; Preprocess the three-dimensional medical image data to obtain the initial data; extract the data of the region of interest according to the obtained initial data; scale the data of the region of interest to a fixed size through the interpolation method to obtain the scaled region of interest data; The scaled region of interest data is standardized to obtain input data of a deep learning model; the final deep learning model is run to obtain an initial segmentation result; the initial segmentation result is post-processed to obtain a final segmentation result.
进一步地,所述预处理的方法为滤波处理。Further, the preprocessing method is filtering processing.
进一步地,将所述缩放后的感兴趣区域数据标准化处理的方法包括如下步骤:根据所述缩放后的感兴趣区域数据,获得缩放后的感兴趣区域数据的像素值x i;根据获得的像素值x i计算其均值μ和方差σ;根据获得的均值μ和方差σ,计算获得标准化后的像素值y i,所述
Figure PCTCN2019128924-appb-000003
将所述标准化后的像素值y i赋值给像素值x i作为标准化处理后的像素值,即令x i=y i
Further, the method for standardizing the scaled region of interest data includes the following steps: obtaining the pixel value x i of the scaled region of interest data according to the scaled region of interest data; and according to the obtained pixels Calculate the mean value μ and variance σ of the value x i ; calculate and obtain the normalized pixel value y i according to the obtained mean value μ and variance σ, the
Figure PCTCN2019128924-appb-000003
Assign the normalized pixel value yi to the pixel value x i as the normalized pixel value, that is, let x i =y i .
一种基于深度学习的医学图像处理系统,所述系统包括:读取模块,用于读取二维医学图像数据;预处理模块,用于对读取后的二维医学图像数据进行预处理,获得训练数据;提取模块,用于根据获得的训练数据,提取感兴趣区域数据;缩放模块,用于将所述感兴趣区域数据通过插值方法缩放到固定尺寸大小,获得缩放后的感兴趣区域数据;标准化模块,用于将所述缩放后的感兴趣区域数据标准化处理,获得深度学习模型的输入数据;运行模块,用于运行深度学习最终模型,得到初始分割结果;后处理模块,用于对所述初始分割结果进行后处理,获得最终分割结果。A medical image processing system based on deep learning, the system comprising: a reading module for reading two-dimensional medical image data; a preprocessing module for preprocessing the read two-dimensional medical image data, Obtain training data; an extraction module for extracting data of the region of interest according to the obtained training data; a scaling module for scaling the data of the region of interest to a fixed size through an interpolation method to obtain the scaled region of interest data ; Standardization module, used to standardize the scaled region of interest data to obtain input data of the deep learning model; Run module, used to run the final deep learning model to obtain the initial segmentation results; Post-processing module, used to The initial segmentation result is post-processed to obtain the final segmentation result.
一种计算机设备,所述计算机设备包括存储器和处理器,所述存储器中存储有计算机程序,所述处理器执行所述计算机程序时实现以下步骤:读取二维医学图像数据;对读取后的二维医学图像数据进行预处理,获得训练数据;根据获得的训练数据,提取感兴趣区域数据;将所述感兴趣区域数据通过插值方法缩放到固定尺寸大小,获得缩放后的感兴趣区域数据;将所述缩放后的感兴趣区域数据标准化处理,获得深度学习模型的输入数据;运行深度学习最终模型,得到初始分割结果;对所述初始分割结果进行后处理,获得最终分割结果。A computer device includes a memory and a processor, and a computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented: reading two-dimensional medical image data; Perform preprocessing of the two-dimensional medical image data of the two-dimensional medical image to obtain training data; extract the region of interest data according to the obtained training data; scale the region of interest data to a fixed size through an interpolation method to obtain the scaled region of interest data Standardize the scaled region of interest data to obtain the input data of the deep learning model; run the final deep learning model to obtain the initial segmentation result; post-process the initial segmentation result to obtain the final segmentation result.
一种计算机可读存储介质,其上存储有计算机程序,所述计算机程序被处理器执行时实现以下步骤:读取二维医学图像数据;对读取后的二维医学图像数据进行预处理,获得训练数据;根据获得的训练数据,提取感兴趣区域数据;将所述感兴趣区域数据通过插值方法缩放到固定尺寸大小,获得缩放后的感兴趣区域数据;将所述缩放后的感兴趣区域数据标准化处理,获得深度学习模型的输入数据;运行深度学习最终模型,得到初始分割结果;A computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented: reading two-dimensional medical image data; preprocessing the read two-dimensional medical image data, Obtain the training data; extract the region of interest data according to the obtained training data; scale the region of interest data to a fixed size by interpolation method to obtain the scaled region of interest data; convert the scaled region of interest Standardize data processing to obtain the input data of the deep learning model; run the final deep learning model to obtain the initial segmentation result;
对所述初始分割结果进行后处理,获得最终分割结果。Perform post-processing on the initial segmentation result to obtain the final segmentation result.
本发明的基于深度学习的医学图像处理方法、系统以及计算机设备,采用全自动分割算法,能够在几十秒内全自动地从整个脚踝精细地分出跟骨、距骨、舟骨、内楔骨、中间楔骨、外楔骨、骰骨、第一跖骨、第二跖骨、第三跖骨、第四跖骨、第五跖骨以及趾骨等26块骨头,以供医生参考,医生根据分割重建后的结果,对患者进行具体分析得到诊断结果,从而减少医生的工作量,提高医生工作效率和诊断的准确度。The medical image processing method, system and computer equipment based on deep learning of the present invention adopts a fully automatic segmentation algorithm, which can automatically and finely separate the calcaneus, talus, navicular bone, and internal cuneiform from the entire ankle within tens of seconds 26 bones, including the middle cuneiform, the external cuneiform, the cuboid, the first metatarsal, the second metatarsal, the third metatarsal, the fourth metatarsal, the fifth metatarsal and the phalanges, for the doctor’s reference. , Perform a specific analysis of the patient to obtain the diagnosis result, thereby reducing the workload of the doctor, improving the doctor's work efficiency and the accuracy of the diagnosis.
附图说明Description of the drawings
图1为本发明用于医学图像处理的深度学习模型设计的方法流程示意图。Fig. 1 is a schematic diagram of a method for designing a deep learning model for medical image processing according to the present invention.
图2为图1中的选择和设计深度学习模型,获得深度学习初始模型的流程示意图。Figure 2 is a schematic diagram of the process of selecting and designing the deep learning model in Figure 1 to obtain the initial deep learning model.
图3为图2中选择深度学习模型的流程示意图。Fig. 3 is a schematic diagram of the process of selecting the deep learning model in Fig. 2.
图4为图2中自适应阈值τ的计算方法流程示意图。Fig. 4 is a schematic flowchart of the calculation method of the adaptive threshold τ in Fig. 2.
图5为本发明基于深度学习的医学图像处理方法的流程示意图。Fig. 5 is a schematic flowchart of a medical image processing method based on deep learning of the present invention.
图6为图5中将所述缩放后的感兴趣区域数据标准化处理的方法流程示意图。FIG. 6 is a schematic flowchart of the method for normalizing the zoomed region of interest data in FIG. 5.
具体实施方式Detailed ways
为使本发明的上述目的、特征和优点能够更为明显易懂,下面结合附图对本发明的具体实施例作详细的说明。In order to make the above objectives, features and advantages of the present invention more obvious and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
实施例一:Example one:
如图1所示,本实施例一提供了一种用于医学图像处理的深度学习模型设计方法,包括如下步骤:As shown in Figure 1, the first embodiment provides a deep learning model design method for medical image processing, including the following steps:
S11、收集含有待分割部位的二维医学图像数据;S11. Collect two-dimensional medical image data containing the part to be segmented;
S12、对感兴趣区域的图像数据进行标注,获得初始标注数据集;S12. Annotate the image data of the region of interest to obtain an initial annotated data set;
S13、将所述初始标注数据集划分为训练集和测试集;S13. Divide the initial labeled data set into a training set and a test set;
S14、选择和设计深度学习模型,获得深度学习初始模型;S14. Select and design a deep learning model to obtain an initial deep learning model;
S15、调整深度学习初始模型超参数,根据获得的训练集数据,对所述深度学习初始模型进行训练,获得深度学习训练后模型;S15. Adjust the hyperparameters of the deep learning initial model, and train the deep learning initial model according to the obtained training set data to obtain the deep learning trained model;
S16、根据获得的测试集数据,对所述深度学习训练后模型进行测试,计算获得测试系数;S16. Test the model after the deep learning training according to the obtained test set data, and calculate a test coefficient;
S17、判断所述测试系数是否达到一固定参数值以上,判断结果为是,则深度学习训练后模型参数符合要求,训练结束,获得深度学习最终模型;否则,深度学习训练后模型参数不符合要求,返回步骤S15。S17. Determine whether the test coefficient reaches a fixed parameter value or more, and the judgment result is yes, then the model parameters after the deep learning training meet the requirements, and the training ends, and the final deep learning model is obtained; otherwise, the model parameters after the deep learning training do not meet the requirements , Return to step S15.
本实施例一中,步骤S11中所述待分割部位为脚踝部位,所述二维医学图像数据的数量为100套以上,每套数据的大小可以是512*512像素*100张。In the first embodiment, the part to be divided in step S11 is the ankle part, the number of the two-dimensional medical image data is more than 100 sets, and the size of each set of data may be 512*512 pixels*100 sheets.
步骤S12中所述对感兴趣区域的图像数据进行标注,是组织一些有资质的医师手工勾画所述感兴趣区域的图像数据中的跟骨、距骨、舟骨、内楔骨、中间楔骨、外楔骨、骰骨、第一跖骨、第二跖骨、第三跖骨、第四跖骨、第五跖骨以及趾骨等26块骨头,获得初始标注数据集。The marking of the image data of the region of interest in step S12 is to organize some qualified physicians to manually outline the calcaneus, talus, navicular bone, internal cuneiform, and intermediate cuneiform in the image data of the region of interest. 26 bones including the external cuneiform, cuboid, first metatarsal, second metatarsal, third metatarsal, fourth metatarsal, fifth metatarsal, and phalanges, to obtain the initial labeled data set.
步骤S13中将所述初始标注数据集划分为训练集和测试集,可以采用随机分配方式进行划分操作,所述训练集和测试集的分配比例可以是80%和20%。In step S13, the initial labeled data set is divided into a training set and a test set, and the division operation may be performed in a random allocation manner, and the allocation ratio of the training set and the test set may be 80% and 20%.
步骤S16中所述测试系数为测试集数据分割结果的指标DICE系数;步骤S17中所述一固定参数值为90%。The test coefficient in step S16 is the index DICE coefficient of the test set data segmentation result; the fixed parameter value in step S17 is 90%.
如图2所示,步骤S14中选择和设计深度学习模型,获得深度学习初始模型的方法,包括如下步骤:As shown in Figure 2, the method of selecting and designing a deep learning model in step S14 to obtain an initial deep learning model includes the following steps:
S21、选择深度学习模型,所述深度学习模型包括但不限于FCN、Res-VNet、U-Net、V-Net模型,所述选择深度学习模型的方法是将训练集数据通过N-fold交叉验证来评测所选择的深度学习模型的鲁棒性,选择鲁棒性较好的深度学习模型;S21. Select a deep learning model, the deep learning model includes but not limited to FCN, Res-VNet, U-Net, and V-Net models, and the method for selecting the deep learning model is to pass the training set data through N-fold cross-validation To evaluate the robustness of the selected deep learning model, choose a deep learning model with better robustness;
S22、根据所选择的深度学习模型,在网络的首层添加一个自适应层,获得深度学习设计模型;所述自适应层的激活函数为
Figure PCTCN2019128924-appb-000004
其中,x为自适应层的输入,τ为x的自适应阈值;
S22. According to the selected deep learning model, an adaptive layer is added to the first layer of the network to obtain a deep learning design model; the activation function of the adaptive layer is
Figure PCTCN2019128924-appb-000004
Among them, x is the input of the adaptive layer, and τ is the adaptive threshold of x;
S23、根据所述深度学习设计模型,在网络的尾层添加一个自定义的损失函数loss,获得深度学习初始模型;所述损失函数
Figure PCTCN2019128924-appb-000005
其中ω是一个常量,为自定义的超参数;m表示分割的类别,例如:定义跟骨为类别1、距骨为类别2、舟骨为类别3,以此类推;Vol(A_perd i)表示第i个类别的骨所预测到的体积;Vol(A_true i)表示第i个类别的骨人工标记的体积;Vol(A_pred_true i)表示第i个类别预测正确的体积;Vol(A_pred i∪A_true i)表示第i个类别预测的骨与人工标记的骨相并之后得到的体积。
S23. Add a custom loss function loss to the tail layer of the network according to the deep learning design model to obtain an initial deep learning model; the loss function
Figure PCTCN2019128924-appb-000005
Where ω is a constant, which is a custom hyperparameter; m represents the segmentation category, for example: define the calcaneus as category 1, talus as category 2, navicular bone as category 3, and so on; Vol(A_perd i ) represents the first i categories predicted to bone volume; vol (A_true i) represents the volume of the i-th category labeled artificial bone; vol (A_pred_true i) denotes the i th class prediction correct volume; vol (a_pred i ∪A_true i ) Represents the volume obtained after combining the predicted bone of the i-th category with the artificially labeled bone.
如图3所示,步骤S21中所述选择深度学习模型的方法包括如下步骤:As shown in Fig. 3, the method for selecting a deep learning model in step S21 includes the following steps:
S31、将所述训练集数据分为N份,其中N-1份作为训练数据,1份作为验证数据,其中N为大于1的整数;S31. Divide the training set data into N pieces, where N-1 pieces are used as training data, and 1 piece is used as verification data, where N is an integer greater than 1.
S32、选取一个深度学习训练模型,作为初步选择的深度学习模型;S32. Select a deep learning training model as the preliminary selected deep learning model;
S33、根据获得的训练数据,对初步选择的深度学习模型进行训练,根据获得的验证数据,对初步选择的深度学习模型进行验证,同时记录误差;S33. Train the preliminarily selected deep learning model according to the obtained training data, verify the preliminarily selected deep learning model according to the obtained verification data, and record the error at the same time;
S34、非重复地选取步骤S31中N份训练集数据中的另一份数据作为验证数据,其余N-1份作为训练数据,重复步骤S33的操作;S34. Non-repetitively select another piece of data in the N training set data in step S31 as verification data, and the remaining N-1 pieces as training data, and repeat the operation of step S33;
S35、步骤S33和S34重复N次后,记录N次所记录误差的平均值,获得所选择的深度学习模型的平均误差;After S35, steps S33 and S34 are repeated N times, the average value of the errors recorded for N times is recorded to obtain the average error of the selected deep learning model;
S36、挑选其他深度学习模型作为初步选择的深度学习模型,重复步骤S33~S35,获得其他深度学习模 型的平均误差;S36. Select other deep learning models as the preliminary selected deep learning models, and repeat steps S33 to S35 to obtain the average error of other deep learning models;
S37、选取平均误差最小的深度学习模型,作为最终所选择的深度学习模型。S37. Select the deep learning model with the smallest average error as the final selected deep learning model.
其中,N的取值范围为3~10,优选的,N的取值范围为5~10,更优选的,本实施例一中N的取值为10。Wherein, the value range of N is 3-10, preferably, the value range of N is 5-10, and more preferably, the value of N in the first embodiment is 10.
如图4所示,步骤S22中自适应阈值τ的计算步骤如下:As shown in Fig. 4, the calculation steps of the adaptive threshold τ in step S22 are as follows:
S41、遍历网络首层的输入数据,计算平均值μ;S41. Traverse the input data of the first layer of the network and calculate the average value μ;
S42、用所述平均值μ作为阈值将网络首层输入数据分割为前景、背景两个部分,分别计算所述前景部分与所述背景部分的均值t_fore、t_back;S42. Use the average value μ as a threshold value to divide the first layer input data of the network into two parts, a foreground part and a background part, and calculate the mean values t_fore and t_back of the foreground part and the background part respectively;
S43、根据获得的前景部分均值、背景部分均值,计算新的阈值μ_new,所述阈值μ_new=(t_fore+t_back)/2;S43. Calculate a new threshold μ_new according to the obtained average value of the foreground part and the average value of the background part, the threshold μ_new=(t_fore+t_back)/2;
S44、根据获得的新的阈值μ_new,计算获得diff,所述diff=μ_new-μ,并将μ_new的值赋值给μ,即令μ=μ_new;S44. Calculate diff according to the obtained new threshold μ_new, the diff=μ_new-μ, and assign the value of μ_new to μ, that is, let μ=μ_new;
S45、重复步骤S42~S44,直至diff小于设定的阈值迭代停止,获得所述激活函数中的自适应阈值τ,所述自适应阈值τ为得到的最终的μ值。S45. Repeat steps S42 to S44 until the diff is less than the set threshold and the iteration stops, and the adaptive threshold τ in the activation function is obtained, and the adaptive threshold τ is the obtained final μ value.
其中,步骤S45中所述设定的阈值,其取值范围为0.01~10,优选的,本实施例一中所述设定的阈值为0.1。Wherein, the threshold value set in step S45 has a value range of 0.01-10. Preferably, the threshold value set in the first embodiment is 0.1.
实施例二:Embodiment two:
如图5所示,本实施例二提供了一种基于实施例一所设计的深度学习模型的医学图像处理方法,包括如下步骤:As shown in Figure 5, the second embodiment provides a medical image processing method based on the deep learning model designed in the first embodiment, including the following steps:
S51、读取二维医学图像数据,所述二维医学图像数据为符合DICOM3.0标准的CT图像数据;S51. Reading two-dimensional medical image data, where the two-dimensional medical image data is CT image data conforming to the DICOM3.0 standard;
S52、对读取后的二维医学图像数据进行预处理,获得初始数据;S52. Preprocess the read two-dimensional medical image data to obtain initial data;
S53、根据获得的初始数据,提取感兴趣区域数据;S53. Extract the region of interest data according to the obtained initial data;
S54、将所述感兴趣区域数据通过插值方法缩放到固定尺寸大小,获得缩放后的感兴趣区域数据,所述固定尺寸可以是512*512像素;S54. Scale the data of the region of interest to a fixed size by an interpolation method to obtain the data of the region of interest after scaling, and the fixed size may be 512*512 pixels;
S55、将所述缩放后的感兴趣区域数据标准化处理,获得深度学习模型的输入数据;S55: Standardize the scaled region of interest data to obtain input data of the deep learning model;
S56、运行实施例一所获得的深度学习最终模型,得到初始分割结果;S56: Run the final deep learning model obtained in the first embodiment to obtain an initial segmentation result;
S57、对所述初始分割结果进行后处理,获得最终分割结果。S57. Perform post-processing on the initial segmentation result to obtain a final segmentation result.
步骤S52中的预处理方法为滤波处理,如中值滤波、均值滤波、高斯滤波、双边滤波或其他滤波方法。The preprocessing method in step S52 is filtering processing, such as median filtering, mean filtering, Gaussian filtering, bilateral filtering or other filtering methods.
步骤S53中提取感兴趣区域数据的方法包括:根据获得的初始数据,根据设定的阈值,自动搜索图像所有连通域,并将所有连通域各自生长到单独容器中;根据每个连通域的大小和方位判断出病人脚踝部位的区域,并提取出脚踝部位区域作为感兴趣区域数据。The method of extracting the region of interest data in step S53 includes: automatically searching all connected regions of the image according to the obtained initial data and according to a set threshold, and growing all connected regions into a separate container; according to the size of each connected region Judge the area of the patient’s ankle with the direction, and extract the area of the ankle as the area of interest data.
步骤S57中对所述初始分割结果进行后处理的方法,是修改部分过分割和欠分割的区域,对小的误分割杂质进行滤除,并通过形态学膨胀、腐蚀、填充等算法进行处理,获得最终分割结果。The method of post-processing the initial segmentation result in step S57 is to modify part of the over-segmented and under-segmented regions, filter out small mis-segmented impurities, and process them through algorithms such as morphological expansion, erosion, and filling. Obtain the final segmentation result.
如图6所示,步骤S55中将所述缩放后的感兴趣区域数据标准化处理的方法包括如下步骤:As shown in FIG. 6, the method for standardizing the zoomed region of interest data in step S55 includes the following steps:
S61、根据所述缩放后的感兴趣区域数据,获得缩放后的感兴趣区域数据的像素值x i S61. Obtain a pixel value x i of the zoomed region of interest data according to the zoomed region of interest data;
S62、根据获得的像素值x i计算其均值μ和方差σ; S62: Calculate the mean value μ and variance σ of the obtained pixel value x i;
S63、根据获得的均值μ和方差σ,计算获得标准化后的像素值y i,所述
Figure PCTCN2019128924-appb-000006
S63. Calculate and obtain a standardized pixel value y i according to the obtained mean value μ and variance σ, and
Figure PCTCN2019128924-appb-000006
S64、将所述标准化后的像素值y i赋值给像素值x i作为标准化处理后的像素值,即令x i=y iS64. Assign the normalized pixel value yi to the pixel value x i as the normalized pixel value, that is, set x i =y i .
由于病人的个体化差异以及医生对二维影像信息观察的局限性,难免会发生判断失误的情形。本发明采用基于深度学习的医学图像处理方法,采用全自动分割算法,能够在几十秒内全自动地从整个脚踝精细地分出跟骨、距骨、舟骨、内楔骨、中间楔骨、外楔骨、骰骨、第一跖骨、第二跖骨、第三跖骨、第四跖骨、第五跖骨以及趾骨等26块骨头,以供医生参考,医生根据分割重建后的结果,对患者进行具体分析得到诊断结果,从而减少医生的工作量,提高医生工作效率和诊断的准确度。Due to the individual differences of patients and the limitations of doctors' observation of two-dimensional image information, it is inevitable that judgment errors will occur. The present invention adopts a medical image processing method based on deep learning and a fully automatic segmentation algorithm, which can automatically and finely separate the calcaneus, talus, navicular bone, internal cuneiform, and intermediate cuneiform from the entire ankle within tens of seconds. 26 bones, including the external cuneiform, cuboid, first metatarsal, second metatarsal, third metatarsal, fourth metatarsal, fifth metatarsal, and phalanges, for the doctor’s reference. The diagnosis results are analyzed, thereby reducing the workload of the doctors and improving the working efficiency of the doctors and the accuracy of the diagnosis.
实施例三:Example three:
本实施例三提供了一种基于深度学习的医学图像处理系统,所述系统包括:The third embodiment provides a medical image processing system based on deep learning, and the system includes:
读取模块,用于读取二维医学图像数据,所述二维医学图像数据为符合DICOM3.0标准的CT图像数据;A reading module for reading two-dimensional medical image data, the two-dimensional medical image data being CT image data complying with the DICOM3.0 standard;
预处理模块,用于对读取后的二维医学图像数据进行预处理,获得训练数据;The preprocessing module is used to preprocess the read two-dimensional medical image data to obtain training data;
提取模块,用于根据获得的训练数据,提取感兴趣区域数据;The extraction module is used to extract the data of the region of interest according to the obtained training data;
缩放模块,用于将所述感兴趣区域数据通过插值方法缩放到固定尺寸大小,获得缩放后的感兴趣区域数据,所述固定尺寸可以是512*512像素;A scaling module, configured to scale the region of interest data to a fixed size through an interpolation method to obtain scaled region of interest data, the fixed size may be 512*512 pixels;
标准化模块,用于将所述缩放后的感兴趣区域数据标准化处理,获得深度学习模型的输入数据;The standardization module is used to standardize the scaled region of interest data to obtain input data of the deep learning model;
运行模块,用于运行实施例一所获得的深度学习最终模型,得到初始分割结果;The running module is used to run the final deep learning model obtained in the first embodiment to obtain the initial segmentation result;
后处理模块,用于对所述初始分割结果进行后处理,获得最终分割结果。The post-processing module is used to perform post-processing on the initial segmentation result to obtain the final segmentation result.
所述基于深度学习的医学图像处理系统中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各个模块可以硬件形式内嵌于或独立于计算机设备中的处理器中,也可以软件形式存储于计算机设备中的存储器中,以便于处理器调用执行以上各个模块对应的操作。The various modules in the medical image processing system based on deep learning can be implemented in whole or in part by software, hardware, and a combination thereof. The foregoing modules may be embedded in the form of hardware or independent of the processor in the computer device, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the foregoing modules.
本实施例三提供的所述基于深度学习的医学图像处理系统,其实现原理和技术效果与上述方法实施例二类似,在此不再赘述。The implementation principles and technical effects of the medical image processing system based on deep learning provided in the third embodiment are similar to those in the second embodiment of the method, and will not be repeated here.
实施例四:Embodiment four:
本实施例四提供了一种计算机设备,所述计算机设备包括存储器和处理器,所述存储器中存储有计算机程序,所述处理器执行所述计算机程序时实现以下步骤:The fourth embodiment provides a computer device, the computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when the processor executes the computer program:
S71、读取二维医学图像数据,所述二维医学图像数据为符合DICOM3.0标准的CT图像数据;S71. Reading two-dimensional medical image data, where the two-dimensional medical image data is CT image data conforming to the DICOM3.0 standard;
S72、对读取后的二维医学图像数据进行预处理,获得训练数据;S72. Preprocess the read two-dimensional medical image data to obtain training data;
S73、根据获得的训练数据,提取感兴趣区域数据;S73. Extract the region of interest data according to the obtained training data;
S74、将所述感兴趣区域数据通过插值方法缩放到固定尺寸大小,获得缩放后的感兴趣区域数据,所述固定尺寸可以是512*512像素;S74. Scale the data of the region of interest to a fixed size by an interpolation method to obtain the data of the region of interest after scaling, and the fixed size may be 512*512 pixels;
S75、将所述缩放后的感兴趣区域数据标准化处理,获得深度学习模型的输入数据;S75. Standardize the scaled region of interest data to obtain input data of the deep learning model;
S76、运行实施例一所获得的深度学习最终模型,得到初始分割结果;S76. Run the final deep learning model obtained in the first embodiment to obtain an initial segmentation result;
S77、对所述初始分割结果进行后处理,获得最终分割结果。S77. Perform post-processing on the initial segmentation result to obtain a final segmentation result.
本实施例四提供的所述计算机设备,其实现原理和技术效果与上述方法实施例二类似,在此不再赘述。The implementation principles and technical effects of the computer device provided in the fourth embodiment are similar to those in the second embodiment of the method, and will not be repeated here.
实施例五Example five
本实施例五提供了一种计算机可读存储介质,其上存储有计算机程序,所述计算机程序被处理器执行时实现以下步骤:The fifth embodiment provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:
S81、读取二维医学图像数据,所述二维医学图像数据为符合DICOM3.0标准的CT图像数据;S81. Read two-dimensional medical image data, where the two-dimensional medical image data is CT image data that meets the DICOM3.0 standard;
S82、对读取后的二维医学图像数据进行预处理,获得训练数据;S82. Preprocess the read two-dimensional medical image data to obtain training data;
S83、根据获得的训练数据,提取感兴趣区域数据;S83. Extract the region of interest data according to the obtained training data;
S84、将所述感兴趣区域数据通过插值方法缩放到固定尺寸大小,获得缩放后的感兴趣区域数据,所述固定尺寸可以是512*512像素;S84. Scale the data of the region of interest to a fixed size by an interpolation method to obtain the data of the region of interest after scaling, and the fixed size may be 512*512 pixels;
S85、将所述缩放后的感兴趣区域数据标准化处理,获得深度学习模型的输入数据;S85. Standardize the scaled region of interest data to obtain input data of the deep learning model;
S86、运行实施例一所获得的深度学习最终模型,得到初始分割结果;S86: Run the final deep learning model obtained in the first embodiment to obtain an initial segmentation result;
S87、对所述初始分割结果进行后处理,获得最终分割结果。S87. Perform post-processing on the initial segmentation result to obtain a final segmentation result.
本实施例五提供的所述计算机可读存储介质,其实现原理和技术效果与上述方法实施例二类似,在此不再赘述。The implementation principle and technical effect of the computer-readable storage medium provided in the fifth embodiment are similar to those of the second method described above, and will not be repeated here.
综上,以上仅为本发明的较佳实施例而已,不应以此限制本发明的范围,即凡是依本发明的权利要求书及本发明说明书内容所作的简单的等效变化与修饰,均应仍属本发明专利涵盖的范围内。In summary, the above are only preferred embodiments of the present invention, and should not be used to limit the scope of the present invention. That is, all simple equivalent changes and modifications made in accordance with the claims of the present invention and the contents of the description of the present invention are all It should still fall within the scope of the invention patent.

Claims (10)

  1. 一种深度学习模型设计方法,其特征在于,包括如下步骤:A method for designing a deep learning model is characterized in that it includes the following steps:
    S11、收集含有待分割部位的二维医学图像数据;S11. Collect two-dimensional medical image data containing the part to be segmented;
    S12、对感兴趣区域的图像数据进行标注,获得初始标注数据集;S12. Annotate the image data of the region of interest to obtain an initial annotated data set;
    S13、将所述初始标注数据集划分为训练集和测试集;S13. Divide the initial labeled data set into a training set and a test set;
    S14、选择和设计深度学习模型,获得深度学习初始模型;具体步骤如下:选择深度学习模型;根据所选择的深度学习模型,在网络的首层添加一个自适应层,获得深度学习设计模型;根据所述深度学习设计模型,在网络的尾层添加一个自定义的损失函数loss,获得深度学习初始模型;S14. Select and design a deep learning model to obtain an initial deep learning model; the specific steps are as follows: select a deep learning model; add an adaptive layer to the first layer of the network according to the selected deep learning model to obtain a deep learning design model; In the deep learning design model, a custom loss function loss is added to the tail layer of the network to obtain an initial deep learning model;
    S15、调整深度学习初始模型超参数,根据获得的训练集数据,对所述深度学习初始模型进行训练,获得深度学习训练后模型;S15. Adjust the hyperparameters of the deep learning initial model, and train the deep learning initial model according to the obtained training set data to obtain the deep learning trained model;
    S16、根据获得的测试集数据,对所述深度学习训练后模型进行测试,计算获得测试系数;S16. Test the model after the deep learning training according to the obtained test set data, and calculate a test coefficient;
    S17、判断所述测试系数是否达到一固定参数值以上,判断结果为是,则深度学习训练后模型参数符合要求,训练结束,获得深度学习最终模型;否则,深度学习训练后模型参数不符合要求,返回步骤S15。S17. Determine whether the test coefficient reaches a fixed parameter value or more, and the judgment result is yes, then the model parameters after the deep learning training meet the requirements, and the training ends, and the final deep learning model is obtained; otherwise, the model parameters after the deep learning training do not meet the requirements , Return to step S15.
  2. 如权利要求1所述的深度学习模型设计方法,其特征在于,所述自适应层的激活函数为
    Figure PCTCN2019128924-appb-100001
    Figure PCTCN2019128924-appb-100002
    其中,x为自适应层的输入,τ为x的自适应阈值;
    The deep learning model design method of claim 1, wherein the activation function of the adaptive layer is
    Figure PCTCN2019128924-appb-100001
    Figure PCTCN2019128924-appb-100002
    Among them, x is the input of the adaptive layer, and τ is the adaptive threshold of x;
    所述损失函数
    Figure PCTCN2019128924-appb-100003
    其中ω是一个常量,为自定义的超参数;m表示分割的类别;Vol(A_perd i)表示第i个类别的骨所预测到的体积;Vol(A_true i)表示第i个类别的骨人工标记的体积;Vol(A_pred_true i)表示第i个类别预测正确的体积;Vol(A_pred i∪A_true i)表示第i个类别预测的骨与人工标记的骨相并之后得到的体积。
    The loss function
    Figure PCTCN2019128924-appb-100003
    Where ω is a constant, which is a custom hyperparameter; m represents the segmentation category; Vol(A_perd i ) represents the predicted volume of the i-th category of bone; Vol(A_true i ) represents the i-th category of bone artificial The marked volume; Vol(A_pred_true i ) represents the correct volume predicted by the i-th category; Vol(A_pred i ∪A_true i ) represents the volume obtained after the merging of the bone predicted by the i-th category with the artificially labeled bone.
  3. 如权利要求1所述的深度学习模型设计方法,其特征在于,选择深度学习模型的方法包括如下步骤:The method for designing a deep learning model according to claim 1, wherein the method for selecting a deep learning model comprises the following steps:
    S31、将所述训练集数据分为N份,其中N-1份作为训练数据,1份作为验证数据,其中N为大于1的整数;S31. Divide the training set data into N pieces, where N-1 pieces are used as training data, and 1 piece is used as verification data, where N is an integer greater than 1.
    S32、选取一个深度学习训练模型,作为初步选择的深度学习模型;S32. Select a deep learning training model as the preliminary selected deep learning model;
    S33、根据获得的训练数据,对初步选择的深度学习模型进行训练,根据获得的验证数据,对初步选择的深度学习模型进行验证,同时记录误差;S33. Train the preliminarily selected deep learning model according to the obtained training data, verify the preliminarily selected deep learning model according to the obtained verification data, and record the error at the same time;
    S34、非重复地选取步骤S31中N份训练集数据中的另一份数据作为验证数据,其余N-1份作为训练数据,重复步骤S33的操作;S34. Non-repetitively select another piece of data in the N training set data in step S31 as verification data, and the remaining N-1 pieces as training data, and repeat the operation of step S33;
    S35、步骤S33和S34重复N次后,记录N次所记录误差的平均值,获得所选择的深度学习模型的平均误差;After S35, steps S33 and S34 are repeated N times, the average value of the errors recorded for N times is recorded to obtain the average error of the selected deep learning model;
    S36、挑选其他深度学习模型作为初步选择的深度学习模型,重复步骤S33~S35,获得其他深度学习模型的平均误差;S36. Select other deep learning models as the preliminary selected deep learning models, and repeat steps S33 to S35 to obtain the average error of other deep learning models;
    S37、选取平均误差最小的深度学习模型,作为最终所选择的深度学习模型。S37. Select the deep learning model with the smallest average error as the final selected deep learning model.
  4. 如权利要求2所述的深度学习模型设计方法,其特征在于,自适应阈值τ的计算步骤如下:The method for designing a deep learning model according to claim 2, wherein the steps of calculating the adaptive threshold τ are as follows:
    S41、遍历网络首层的输入数据,计算平均值μ;S41. Traverse the input data of the first layer of the network and calculate the average value μ;
    S42、用所述平均值μ作为阈值将网络首层输入数据分割为前景、背景两个部分,分别计算所述前景部分与所述背景部分的均值t_fore、t_back;S42. Use the average value μ as a threshold value to divide the first layer input data of the network into two parts, a foreground part and a background part, and calculate the mean values t_fore and t_back of the foreground part and the background part respectively;
    S43、根据获得的前景部分均值、背景部分均值,计算新的阈值μ_new,所述阈值μ_new=(t_fore+t_back)/2;S43. Calculate a new threshold μ_new according to the obtained average value of the foreground part and the average value of the background part, the threshold μ_new=(t_fore+t_back)/2;
    S44、根据获得的新的阈值μ_new,计算获得diff,所述diff=μ_new-μ,并将μ_new的值赋值给μ;S44. According to the obtained new threshold μ_new, calculate to obtain diff, the diff=μ_new-μ, and assign the value of μ_new to μ;
    S45、重复步骤S42~S44,直至diff小于设定的阈值迭代停止,获得自适应阈值τ。S45. Repeat steps S42 to S44 until the diff is less than the set threshold and the iteration stops, and the adaptive threshold τ is obtained.
  5. 一种基于深度学习的医学图像处理方法,其是基于权利要求1所述的深度学习模型设计方法所设计的深度学习模型,其特征在于,包括如下步骤:A medical image processing method based on deep learning, which is a deep learning model designed based on the deep learning model design method of claim 1, characterized in that it comprises the following steps:
    读取二维医学图像数据;Read two-dimensional medical image data;
    对读取后的二维医学图像数据进行预处理,获得初始数据;Preprocess the read two-dimensional medical image data to obtain initial data;
    根据获得的初始数据,提取感兴趣区域数据;According to the obtained initial data, extract the data of the region of interest;
    将所述感兴趣区域数据通过插值方法缩放到固定尺寸大小,获得缩放后的感兴趣区域数据;Scaling the data of the region of interest to a fixed size by an interpolation method to obtain the data of the region of interest after scaling;
    将所述缩放后的感兴趣区域数据标准化处理,获得深度学习模型的输入数据;Standardize the scaled region of interest data to obtain input data of the deep learning model;
    运行所述深度学习最终模型,得到初始分割结果;Run the final deep learning model to obtain an initial segmentation result;
    对所述初始分割结果进行后处理,获得最终分割结果。Perform post-processing on the initial segmentation result to obtain the final segmentation result.
  6. 如权利要求5所述的基于深度学习的医学图像处理方法,其特征在于,所述预处理的方法为滤波处理。The medical image processing method based on deep learning according to claim 5, wherein the preprocessing method is filtering processing.
  7. 如权利要求5所述的基于深度学习的医学图像处理方法,其特征在于,将所述缩放后的感兴趣区域数据标准化处理的方法包括如下步骤:The medical image processing method based on deep learning according to claim 5, wherein the method for standardizing the scaled region of interest data comprises the following steps:
    根据所述缩放后的感兴趣区域数据,获得缩放后的感兴趣区域数据的像素值x i Obtaining the pixel value x i of the zoomed region of interest data according to the zoomed region of interest data;
    根据获得的像素值x i计算其均值μ和方差σ; Calculate the mean value μ and variance σ of the obtained pixel value x i;
    根据获得的均值μ和方差σ,计算获得标准化后的像素值y i,所述
    Figure PCTCN2019128924-appb-100004
    According to the obtained mean value μ and variance σ, the normalized pixel value y i is calculated, the
    Figure PCTCN2019128924-appb-100004
    将所述标准化后的像素值y i赋值给像素值x i作为标准化处理后的像素值,即令x i=y iAssign the normalized pixel value yi to the pixel value x i as the normalized pixel value, that is, let x i =y i .
  8. 一种基于深度学习的医学图像处理系统,其特征在于,所述系统包括:A medical image processing system based on deep learning, characterized in that the system includes:
    读取模块,用于读取二维医学图像数据;Reading module for reading two-dimensional medical image data;
    预处理模块,用于对读取后的二维医学图像数据进行预处理,获得训练数据;The preprocessing module is used to preprocess the read two-dimensional medical image data to obtain training data;
    提取模块,用于根据获得的训练数据,提取感兴趣区域数据;The extraction module is used to extract the data of the region of interest according to the obtained training data;
    缩放模块,用于将所述感兴趣区域数据通过插值方法缩放到固定尺寸大小,获得缩放后的感兴趣区域数据;A scaling module, configured to scale the data of the region of interest to a fixed size through an interpolation method, to obtain the data of the region of interest after scaling;
    标准化模块,用于将所述缩放后的感兴趣区域数据标准化处理,获得深度学习模型的输入数据;The standardization module is used to standardize the scaled region of interest data to obtain input data of the deep learning model;
    运行模块,用于运行深度学习最终模型,得到初始分割结果;The running module is used to run the final deep learning model to get the initial segmentation result;
    后处理模块,用于对所述初始分割结果进行后处理,获得最终分割结果。The post-processing module is used to perform post-processing on the initial segmentation result to obtain the final segmentation result.
  9. 一种计算机设备,所述计算机设备包括存储器和处理器,所述存储器中存储有计算机程序,其特征在于,所述处理器执行所述计算机程序时实现以下步骤:A computer device, the computer device includes a memory and a processor, and a computer program is stored in the memory, and is characterized in that the processor implements the following steps when the processor executes the computer program:
    读取二维医学图像数据;Read two-dimensional medical image data;
    对读取后的二维医学图像数据进行预处理,获得训练数据;Preprocess the read two-dimensional medical image data to obtain training data;
    根据获得的训练数据,提取感兴趣区域数据;According to the obtained training data, extract the data of the region of interest;
    将所述感兴趣区域数据通过插值方法缩放到固定尺寸大小,获得缩放后的感兴趣区域数据;Scaling the data of the region of interest to a fixed size by an interpolation method to obtain the data of the region of interest after scaling;
    将所述缩放后的感兴趣区域数据标准化处理,获得深度学习模型的输入数据;Standardize the scaled region of interest data to obtain input data of the deep learning model;
    运行深度学习最终模型,得到初始分割结果;Run the final deep learning model to get the initial segmentation result;
    对所述初始分割结果进行后处理,获得最终分割结果。Perform post-processing on the initial segmentation result to obtain the final segmentation result.
  10. 一种计算机可读存储介质,其上存储有计算机程序,其特征在于,所述计算机程序被处理器执行时实现以下步骤:A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the following steps when being executed by a processor:
    读取二维医学图像数据;Read two-dimensional medical image data;
    对读取后的二维医学图像数据进行预处理,获得训练数据;Preprocess the read two-dimensional medical image data to obtain training data;
    根据获得的训练数据,提取感兴趣区域数据;According to the obtained training data, extract the data of the region of interest;
    将所述感兴趣区域数据通过插值方法缩放到固定尺寸大小,获得缩放后的感兴趣区域数据;Scaling the data of the region of interest to a fixed size by an interpolation method to obtain the data of the region of interest after scaling;
    将所述缩放后的感兴趣区域数据标准化处理,获得深度学习模型的输入数据;Standardize the scaled region of interest data to obtain input data of the deep learning model;
    运行深度学习最终模型,得到初始分割结果;Run the final deep learning model to get the initial segmentation result;
    对所述初始分割结果进行后处理,获得最终分割结果。Perform post-processing on the initial segmentation result to obtain the final segmentation result.
PCT/CN2019/128924 2019-12-27 2019-12-27 Deep learning-based medical image processing method and system, and computer device WO2021128230A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/128924 WO2021128230A1 (en) 2019-12-27 2019-12-27 Deep learning-based medical image processing method and system, and computer device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/128924 WO2021128230A1 (en) 2019-12-27 2019-12-27 Deep learning-based medical image processing method and system, and computer device

Publications (1)

Publication Number Publication Date
WO2021128230A1 true WO2021128230A1 (en) 2021-07-01

Family

ID=76573811

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/128924 WO2021128230A1 (en) 2019-12-27 2019-12-27 Deep learning-based medical image processing method and system, and computer device

Country Status (1)

Country Link
WO (1) WO2021128230A1 (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102800095A (en) * 2012-07-17 2012-11-28 南京特雷多信息科技有限公司 Lens boundary detection method
CN105631479A (en) * 2015-12-30 2016-06-01 中国科学院自动化研究所 Imbalance-learning-based depth convolution network image marking method and apparatus
CN108921227A (en) * 2018-07-11 2018-11-30 广东技术师范学院 A kind of glaucoma medical image classification method based on capsule theory
CN110097554A (en) * 2019-04-16 2019-08-06 东南大学 The Segmentation Method of Retinal Blood Vessels of convolution is separated based on intensive convolution sum depth
US20190365341A1 (en) * 2018-05-31 2019-12-05 Canon Medical Systems Corporation Apparatus and method for medical image reconstruction using deep learning to improve image quality in position emission tomography (pet)
CN110599502A (en) * 2019-09-06 2019-12-20 江南大学 Skin lesion segmentation method based on deep learning

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102800095A (en) * 2012-07-17 2012-11-28 南京特雷多信息科技有限公司 Lens boundary detection method
CN105631479A (en) * 2015-12-30 2016-06-01 中国科学院自动化研究所 Imbalance-learning-based depth convolution network image marking method and apparatus
US20190365341A1 (en) * 2018-05-31 2019-12-05 Canon Medical Systems Corporation Apparatus and method for medical image reconstruction using deep learning to improve image quality in position emission tomography (pet)
CN108921227A (en) * 2018-07-11 2018-11-30 广东技术师范学院 A kind of glaucoma medical image classification method based on capsule theory
CN110097554A (en) * 2019-04-16 2019-08-06 东南大学 The Segmentation Method of Retinal Blood Vessels of convolution is separated based on intensive convolution sum depth
CN110599502A (en) * 2019-09-06 2019-12-20 江南大学 Skin lesion segmentation method based on deep learning

Similar Documents

Publication Publication Date Title
CN107767376B (en) X-ray bone age prediction method and system based on deep learning
CN110111313B (en) Medical image detection method based on deep learning and related equipment
CN109754387B (en) Intelligent detection and positioning method for whole-body bone imaging radioactive concentration focus
CN108464840B (en) Automatic detection method and system for breast lumps
CN110059697B (en) Automatic lung nodule segmentation method based on deep learning
Liu et al. A framework of wound segmentation based on deep convolutional networks
CN109671068B (en) Abdominal muscle labeling method and device based on deep learning
CN108062749B (en) Identification method and device for levator ani fissure hole and electronic equipment
CN111179227A (en) Mammary gland ultrasonic image quality evaluation method based on auxiliary diagnosis and subjective aesthetics
US20230230241A1 (en) System and method for detecting lung abnormalities
CN111652838B (en) Thyroid nodule positioning and ultrasonic report error correction method based on target detection network
US20220335600A1 (en) Method, device, and storage medium for lesion segmentation and recist diameter prediction via click-driven attention and dual-path connection
CN116386902B (en) Artificial intelligent auxiliary pathological diagnosis system for colorectal cancer based on deep learning
CN113763340A (en) Automatic grading method based on multitask deep learning ankylosing spondylitis
Tang et al. One click lesion RECIST measurement and segmentation on CT scans
CN114565572A (en) Cerebral hemorrhage CT image classification method based on image sequence analysis
Liu et al. Automated classification and measurement of fetal ultrasound images with attention feature pyramid network
Zeng et al. TUSPM-NET: A multi-task model for thyroid ultrasound standard plane recognition and detection of key anatomical structures of the thyroid
Tang et al. Lesion segmentation and RECIST diameter prediction via click-driven attention and dual-path connection
CN111667457B (en) Automatic identification method, system, terminal and storage medium for vertebral body information based on medical image
Gordon et al. Evaluation of uterine cervix segmentations using ground truth from multiple experts
CN111476802A (en) Medical image segmentation and tumor detection method and device based on dense convolution model and readable storage medium
WO2021128230A1 (en) Deep learning-based medical image processing method and system, and computer device
CN116228660A (en) Method and device for detecting abnormal parts of chest film
CN115564750A (en) Intraoperative frozen slice image identification method, intraoperative frozen slice image identification device, intraoperative frozen slice image identification equipment and intraoperative frozen slice image storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19957775

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19957775

Country of ref document: EP

Kind code of ref document: A1