WO2023024882A1 - 基于深度学习的股骨髓腔形态识别方法、装置及存储介质 - Google Patents

基于深度学习的股骨髓腔形态识别方法、装置及存储介质 Download PDF

Info

Publication number
WO2023024882A1
WO2023024882A1 PCT/CN2022/110972 CN2022110972W WO2023024882A1 WO 2023024882 A1 WO2023024882 A1 WO 2023024882A1 CN 2022110972 W CN2022110972 W CN 2022110972W WO 2023024882 A1 WO2023024882 A1 WO 2023024882A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
dimensional medical
medical image
femur
pixel point
Prior art date
Application number
PCT/CN2022/110972
Other languages
English (en)
French (fr)
Inventor
张逸凌
刘星宇
Original Assignee
北京长木谷医疗科技有限公司
张逸凌
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京长木谷医疗科技有限公司, 张逸凌 filed Critical 北京长木谷医疗科技有限公司
Publication of WO2023024882A1 publication Critical patent/WO2023024882A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/08—Projecting images onto non-planar surfaces, e.g. geodetic screens
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30008—Bone

Definitions

  • the present application relates to the technical field of femoral medullary cavity shape recognition, in particular to a deep learning-based femoral medullary cavity shape recognition method, device and storage medium.
  • Artificial hip replacement refers to a prosthesis similar to human bone joints made of biocompatible materials, which replaces joints or joint planes damaged by diseases or injuries, thereby relieving joint pain, correcting deformed prostheses, and improving joints. activity function.
  • the morphological characteristics of the femoral medullary cavity play an important role in guiding the design of the artificial hip prosthesis.
  • the stability of the artificial hip prosthesis after implantation is directly related to the matching degree of the prosthesis and the medullary cavity.
  • the embodiment of the present application provides a deep learning-based femoral medullary cavity shape recognition method, device, and storage medium to solve the problem that only the coronal diameter or sagittal diameter at the key position of the femur is calculated in the related art, which cannot fully and accurately reflect the The problem of the type of medullary cavity morphology.
  • the first aspect of the embodiment of the present application provides a method for recognizing the shape of the femoral medullary cavity based on deep learning, including:
  • processing the two-dimensional medical image of the femoral region to obtain a three-dimensional medical image of the femur includes:
  • the pixel point with the largest X-axis coordinate value Based on the coordinates of the several pixel points, respectively determine the pixel point with the largest X-axis coordinate value, the pixel point with the smallest X-axis coordinate value, the pixel point with the largest Y-axis coordinate value, the pixel point with the smallest Y-axis coordinate value, and the pixel point with the smallest Y-axis coordinate value.
  • the pixel point with the largest X-axis coordinate value the pixel point with the smallest X-axis coordinate value, the pixel point with the largest Y-axis coordinate value, the pixel point with the smallest Y-axis coordinate value, the pixel point with the largest Z-axis coordinate value, and the Z-axis
  • the pixel point with the smallest coordinate value is used to determine the three-dimensional medical image of the femur.
  • the pre-training process of the image segmentation network model includes:
  • An image segmentation network model is trained based on the training data set, verification data set, and test data set in combination with neural network algorithms and deep learning.
  • an image segmentation network model is trained based on the training data set, verification data set, and test data set in combination with neural network algorithms and deep learning, including:
  • Roughly segment the training data set through the first image segmentation network model perform multiple downsampling on the two-dimensional medical images in the training data set to identify each two-dimensional medical image through the processing of the convolutional layer and the pooling layer.
  • the deep features of the medical image multiple upsampling is performed on the down-sampled two-dimensional medical image, so as to reversely store the deep feature into the two-dimensional medical image through the processing of the up-sampling layer and the convolution layer; utilize
  • the Adam classification optimizer carries out image rough classification processing, obtains image rough segmentation result; Wherein, activation function is all provided with after described each convolutional layer;
  • Perform fine segmentation processing on the rough image segmentation result through the second image segmentation model filter the feature point data with preset reliability from the deep features, and perform bilinear interpolation calculation on the feature point data, based on the calculated The feature point data identify the category of the deep feature, and obtain the final image segmentation result;
  • the inputting the perspective image of the femur into the VGG classifier network to obtain the type of the medullary cavity of the femur output by the VGG classifier network includes :
  • the types of the medullary cavity of the femur output by the VGG classifier network include: normal type, champagne type, and chimney type.
  • the VGG classifier network includes 16 hidden layers, including: 13 convolutional layers and 3 fully connected layers.
  • a device for recognizing the shape of the femoral medullary cavity based on deep learning including:
  • an acquisition module configured to acquire a two-dimensional medical image of the hip joint
  • the segmentation module is configured to perform image segmentation processing on the two-dimensional medical image of the hip joint based on a pre-trained image segmentation network model, and obtain a two-dimensional medical image of the femoral region based on the image segmentation result;
  • the image processing module is configured to process the two-dimensional medical image of the femoral region to obtain a three-dimensional medical image of the femur; in the direction of the coronal plane of the patient, transform the three-dimensional medical image of the femur through orthographic projection to obtain the an orthographic projection image; performing perspective processing on the orthographic projection image of the femur to obtain a perspective image of the femur;
  • the classification module is configured to input the perspective image of the femur into the VGG classifier network, so as to obtain the type of the medullary cavity shape of the femur output by the VGG classifier network.
  • the image processing module is further configured to perform the following steps, including:
  • the pixel point with the largest X-axis coordinate value Based on the coordinates of the several pixel points, respectively determine the pixel point with the largest X-axis coordinate value, the pixel point with the smallest X-axis coordinate value, the pixel point with the largest Y-axis coordinate value, the pixel point with the smallest Y-axis coordinate value, and the pixel point with the smallest Y-axis coordinate value.
  • the pixel point with the largest X-axis coordinate value the pixel point with the smallest X-axis coordinate value, the pixel point with the largest Y-axis coordinate value, the pixel point with the smallest Y-axis coordinate value, the pixel point with the largest Z-axis coordinate value, and the Z-axis
  • the pixel point with the smallest coordinate value is used to determine the three-dimensional medical image of the femur.
  • the third aspect of the embodiments of the present application provides a readable storage medium, the computer program is stored in the readable storage medium, and the computer program is used to realize the first aspect and the first aspect of the present application when executed by a processor Various possible designs of the method.
  • an electronic device including a memory, a processor, and a computer program stored on the memory and operable on the processor, and the processor implements the program when executing the program. Steps of the method according to the first aspect of the present application and various possible designs of the first aspect.
  • the present application provides a deep learning-based femoral medullary cavity shape recognition method, device, and storage medium, which can accurately identify and classify the types of femoral medullary cavity shape using deep learning technology based on two-dimensional medical images of the hip joint.
  • Deep learning techniques apart from traditional methods is their ability to generate features inside the human body.
  • Deep learning techniques consist of multiple layers of neural networks that are trained on large amounts of data to give a ground-truth description, which is then used to predict segmentation on a test dataset to identify the medullary canal morphology of the femur.
  • the result of identifying the shape of the medullary cavity is more accurate, and the speed is faster, flexible and efficient, which provides a new method for the analysis of the morphological characteristics of the femoral medullary cavity, and then provides data support for the scientific design of hip joint prosthesis.
  • Fig. 1 is the flow chart of the first embodiment of the femoral medullary canal shape recognition method based on deep learning provided by the embodiment of the present application;
  • Fig. 2 is the structural diagram of the image segmentation network model provided by the embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of the VGG classifier network provided by the embodiment of the present application.
  • Fig. 4 is the morphological classification diagram of the femoral medullary cavity provided by the embodiment of the present application.
  • FIG. 5 is a structural diagram of the first embodiment of the deep learning-based femoral canal shape recognition device provided in the embodiment of the present application;
  • FIG. 6 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • serial numbers of the processes do not mean the order of execution, and the execution order of the processes should be determined by their functions and internal logic, rather than by the implementation order of the embodiments of the present application.
  • the implementation process constitutes no limitation.
  • B corresponding to A means that B is associated with A, and according to A It is possible to determine B. Determining B from A does not mean determining B from A alone, B can also be determined from A and/or other information.
  • the matching between A and B means that the similarity between A and B is greater than or equal to a preset threshold.
  • This application provides a method for recognizing the shape of the femoral medullary cavity based on deep learning, as shown in Figure 1, its flow chart, including:
  • Step S110 acquiring a two-dimensional medical image of the hip joint.
  • the CT two-dimensional medical image dataset of the hip joint which includes several CT two-dimensional medical images of the patient's hip joint, and uses at least one of manual labeling and automatic labeling
  • the method is to mark the femoral region of the CT two-dimensional medical image, and divide the marked CT two-dimensional medical image into a training set, a verification set and a view measurement set according to a preset ratio. For example, it can be divided according to the ratio of 6:2:2.
  • Step S120 performing image segmentation processing on the 2D medical image of the hip joint based on the pre-trained image segmentation network model, and obtaining a 2D medical image of the femur region based on the image segmentation result.
  • step S120 the pre-trained image segmentation network model: pointrend neural network, unet convolutional neural network is used for segmentation processing, that is, firstly, the unet convolutional neural network is used as the backbone network, and the above-mentioned CT 2 of the femoral region is marked. Dimensional medical images are roughly segmented; then the results of the rough segmentation are accurately segmented using the pointrend neural network.
  • Step S130 processing the two-dimensional medical image of the femoral region to obtain a three-dimensional medical image of the femur; in the direction of the patient's coronal plane, transforming the three-dimensional medical image of the femur through orthographic projection to obtain an orthographic projection image of the femur; Perform perspective processing on the orthographic projection image of the femur to obtain a perspective image of the femur.
  • Step S140 input the fluoroscopic image of the femur into the VGG classifier network, so as to obtain the type of the medullary cavity shape of the femur output by the VGG classifier network.
  • the VGG classifier network used is VGG16, all of which are 3x3 small convolution kernels and 2x2 pooling kernels, and the performance can be improved by continuously deepening the network. It contains 16 hidden layers, including: 13 convolutional layers and 3 fully connected layers (FC-4096, FC-4096, FC-1000); the convolutional layer is 5 convolutional segments, respectively: con3- 64*2, con3-128*2, con3-256*3, con3-512*3, con3-512*3; at the same time, the end of each convolution segment will be connected with a largest pooling layer maxpool, which is used to reduce the image size of.
  • VGG16 all of which are 3x3 small convolution kernels and 2x2 pooling kernels, and the performance can be improved by continuously deepening the network. It contains 16 hidden layers, including: 13 convolutional layers and 3 fully connected layers (FC-4096, FC-4096, FC-1000); the convolutional layer is 5 convolutional segments, respectively: con3- 64*2, con3-128*2, con3-2
  • step S140 the specific VGG classifier network training process is as follows: first input a 256*256*3 picture, and perform two convolutions + Relu through 64 3*3 convolution kernels, and the size of the convolution becomes 256 *256*64; through max pooling, the size of the pooling unit is 2*2, and the size after pooling becomes 128*128*64; after 128 3*3 unilateral convolution kernels do two convolutions + relu, The size becomes 128*128*128; through 2*2 max pooling, the size becomes 64*64*128; after 256 3*3 convolution kernels do three convolutions + relu, the size becomes 64* 64*256; through 2*2 max pooling, the size becomes 32*32*256; after 512 3*3 convolution kernels do three convolutions + relu, the size becomes 32*32*512; through 2*2 max pooling pooling, the size becomes 16*16*512; through 512 3*3 convolution kernels to do three convolutions +
  • processing the two-dimensional medical image of the femoral region to obtain the three-dimensional medical image of the femur includes: performing three-dimensional reconstruction on the two-dimensional medical image of the femoral region to obtain the three-dimensional medical image of the femoral region Image; based on the three-dimensional medical image of the femoral region, obtain the coordinates of several pixel points contained in the pixel point set of the femoral region; based on the coordinates of the several pixel points, respectively determine the pixel point with the largest X-axis coordinate value , the pixel point with the smallest X-axis coordinate value, the pixel point with the largest Y-axis coordinate value, the pixel point with the smallest Y-axis coordinate value, the pixel point with the largest Z-axis coordinate value, and the pixel point with the smallest Z-axis coordinate value; according to the X Pixel with the largest axis coordinate value, pixel with the smallest X-axis coordinate value, pixel with the largest Y
  • V contains the pixel point set of all femoral regions
  • Xmin, Xmin, Xmax, Ymin and Ymax are found in the Y coordinates of all pixels
  • Zmin and Zmax are found in the Z coordinates of all pixels
  • the three-dimensional femur can be determined by Xmin, Xmax, Ymin, Ymax, Zmin and Zmax spatial extent.
  • the pre-training process of the image segmentation network model includes:
  • An image segmentation network model is trained based on the training data set, verification data set, and test data set in combination with neural network algorithms and deep learning.
  • the CT medical images of the hip joints of multiple patients will be obtained first, and the femoral region will be marked manually or automatically, and then the multiple CT images marked with the femoral region will be combined according to the ratio of 6:2:2.
  • the image is divided into training set, validation set, and test set as the input of the image segmentation network model.
  • the image segmentation network model is trained based on the training data set, verification data set, and test data set in combination with neural network algorithms and deep learning, including:
  • Roughly segment the training data set through the first image segmentation network model perform multiple downsampling on the two-dimensional medical images in the training data set to identify each two-dimensional medical image through the processing of the convolutional layer and the pooling layer.
  • the deep features of the medical image multiple upsampling is performed on the downsampled two-dimensional medical image, so as to reversely store the deep feature into the two-dimensional medical image through the processing of the upsampling layer and the convolution layer; use
  • the Adam classification optimizer carries out image rough classification processing, obtains image rough segmentation result; Wherein, activation function is all provided with after described each convolutional layer;
  • Perform fine segmentation processing on the rough image segmentation result through the second image segmentation model filter the feature point data with preset reliability from the deep features, and perform bilinear interpolation calculation on the feature point data, based on the calculated The feature point data identify the category of the deep feature, and obtain the final image segmentation result;
  • the step of performing image segmentation processing on the two-dimensional medical image of the hip joint based on the pre-trained image segmentation network model to obtain the two-dimensional medical image of the femoral region it mainly includes: using the first image processing sub-model unet Carry out rough segmentation and use the second image processing sub-model pointrend for precise segmentation.
  • the specific network structure is shown in Figure 2. The specific process is as follows:
  • Rough segmentation using the first image processing sub-model unet specifically refers to: use the unet network as the backbone network to roughly segment the input two-dimensional image, use 4 times of downsampling to learn the deep features of the image in the first stage, and then perform 4 upsampling to re-store the feature map into the image, where each downsampling layer includes 2 convolutional layers and 1 pooling layer, the convolution kernel size is 3*3, and the convolution in the pooling layer The kernel size is 2*2, and the number of convolution kernels in each convolutional layer is 128, 256, 512; each upsampling layer includes 1 upsampling layer and 2 convolutional layers, where the convolutional layer The size of the convolution kernel is 3*2, the size of the convolution kernel in the upsampling layer is 2*2, and the number of convolution kernels in each upsampling layer is 512, 256, 128.
  • the background pixel value of the data label is set to 0, the femur is set to 1, the training batch_size is set to 6, the learning rate is set to 1e -4 , and the classification optimizer uses the Adam classification optimizer.
  • the loss function is DICE loss.
  • the original image of the training set and the femur/tibia/fibula/patella are sent to the network for training. According to the change of the loss function during the training process, the size of the training batch is adjusted, and finally the rough segmentation results of each part are obtained. .
  • Using the second image processing sub-model pointrend for accurate segmentation specifically refers to: use pointrend to accurately segment the results, first the goal of pixel selection is to select a series of potential feature points to prepare for the next judgment, here we select The basis is that the classification confidence is close to 0.5 in the rough segmentation results (the confidence of a point in the binary classification task will approach 0 or 1, and the confidence near 0.5 means that the network is very uncertain about the classification of this point) , usually such points are points close to the edge of the object.
  • the second step we perform feature extraction on the points selected in the previous step, and the rough segmentation network comes with a feature extractor (feature extractor). We only need to select the features of the selected points at the corresponding positions in the feature extractor. .
  • the types of femoral medullary cavity morphology output by the VGG classifier network include: normal type, champagne type, and chimney type.
  • the application provides a deep learning-based femoral medullary cavity shape recognition method, device, and storage medium, which can accurately identify and classify the types of femoral medullary cavity shape based on two-dimensional medical images of the hip joint using deep learning technology .
  • Deep learning techniques apart from traditional methods is their ability to generate features inside the human body. Deep learning techniques consist of multiple layers of neural networks that are trained on large amounts of data to give a ground-truth description, which is then used to predict segmentation on a test dataset to identify the medullary canal morphology of the femur.
  • the result of identifying the shape of the medullary cavity is also more accurate, and the speed is faster, flexible and efficient, which provides a new method for the analysis of the morphological characteristics of the femoral medullary cavity, and then provides data support for the scientific design of hip joint prosthesis.
  • Embodiments of the present application also provide a deep learning-based femoral medullary cavity shape recognition device, as shown in Figure 5, including:
  • An acquisition module 51 configured to acquire a two-dimensional medical image of the hip joint
  • the segmentation module 52 is configured to perform image segmentation processing on the two-dimensional medical image of the hip joint based on a pre-trained image segmentation network model, and obtain a two-dimensional medical image of the femoral region based on the image segmentation result;
  • the image processing module 53 is configured to process the two-dimensional medical image of the femur region to obtain a three-dimensional medical image of the femur; in the direction of the patient's coronal plane, transform the three-dimensional medical image of the femur through orthographic projection to obtain a femoral orthographic projection image of the femur; performing perspective processing on the orthographic projection image of the femur to obtain a perspective image of the femur;
  • the classification module 54 is configured to input the perspective image of the femur into a VGG classifier network, so as to obtain the type of the medullary cavity shape of the femur output by the VGG classifier network.
  • the image processing module 53 is further configured to perform the following steps, including:
  • the pixel point with the largest X-axis coordinate value Based on the coordinates of the several pixel points, respectively determine the pixel point with the largest X-axis coordinate value, the pixel point with the smallest X-axis coordinate value, the pixel point with the largest Y-axis coordinate value, the pixel point with the smallest Y-axis coordinate value, and the pixel point with the smallest Y-axis coordinate value.
  • the pixel point with the largest X-axis coordinate value the pixel point with the smallest X-axis coordinate value, the pixel point with the largest Y-axis coordinate value, the pixel point with the smallest Y-axis coordinate value, the pixel point with the largest Z-axis coordinate value, and the Z-axis
  • the pixel point with the smallest coordinate value is used to determine the three-dimensional medical image of the femur.
  • the present application also provides a readable storage medium, wherein a computer program is stored in the readable storage medium, and when the computer program is executed by a processor, it is used to realize the above-mentioned femoral marrow based on deep learning
  • a cavity shape recognition method comprising:
  • the readable storage medium may be a computer storage medium, or a communication medium.
  • Communication media includes any medium that facilitates transfer of a computer program from one place to another.
  • Computer storage media can be any available media that can be accessed by a general purpose or special purpose computer.
  • a readable storage medium is coupled to the processor such that the processor can read information from, and write information to, the readable storage medium.
  • the readable storage medium can also be a component of the processor.
  • the processor and the readable storage medium may be located in application specific integrated circuits (Application Specific Integrated Circuits, ASIC). Additionally, the ASIC may be located in the user equipment.
  • ASIC Application Specific Integrated Circuits
  • the processor and the readable storage medium can also exist in the communication device as discrete components.
  • the readable storage medium may be read only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage devices, among others.
  • the present application also provides a program product, which includes execution instructions, and the execution instructions are stored in a readable storage medium.
  • At least one processor of the device may read the execution instruction from the readable storage medium, and the at least one processor executes the execution instruction so that the device implements the methods provided in the foregoing various implementation manners.
  • the electronic device may include: a processor (processor) 610, a communication interface (Communications Interface) 620, a memory (memory) 630, and a communication bus 640, wherein, The processor 610 , the communication interface 620 , and the memory 630 communicate with each other through the communication bus 640 .
  • the processor 610 can call the logic instructions in the memory 630 to execute the above-mentioned method for recognizing the shape of the femoral medullary cavity based on deep learning, and the method includes:
  • the processor can be a central processing unit (English: Central Processing Unit, CPU), and can also be other general-purpose processors, digital signal processors (English: Digital Signal Processor, DSP )wait.
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • the steps of the methods disclosed in this application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Molecular Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

本申请提供一种基于深度学习的股骨髓腔形态识别方法、装置及存储介质，包括：获取髋关节的二维医学图像；基于预先训练好的图像分割网络对髋关节的二维医学图像进行图像分割处理，基于图像分割结果得到股骨区域的二维医学图像；对股骨区域的二维医学图像进行处理，得到股骨的三维医学图像；在患者的冠状面方向上，将股骨的三维医学图像通过正投影变换得到股骨的正投影图像；对股骨的正投影图像进行透视处理，得到股骨的透视图像；将股骨的透视图像输入VGG分类器网络，以获得VGG分类网络输出的股骨的髓腔形态的种类。本申请提供的股骨髓腔形态识别方法，能够基于髋关节的二维医学图像使用深度学习技术对股骨的髓腔形态的种类进行精准识别。

Description

基于深度学习的股骨髓腔形态识别方法、装置及存储介质
相关申请的交叉引用
本申请要求于2021年08月24日提交的申请号为202110974202.6，名称为“基于深度学习的股骨髓腔形态识别方法、装置及存储介质”的中国专利申请的优先权，其通过引用方式全部并入本文。
技术领域
本申请涉及股骨髓腔形态识别技术领域，尤其涉及一种基于深度学习的股骨髓腔形态识别方法、装置及存储介质。
背景技术
人工髋关节置换是指用生物相容性良好的材料制成的类似人体骨关节的假体，置换被疾病或损伤所破坏的关节或关节平面，从而缓解关节疼痛、矫正畸形假体、改善关节的活动功能。股骨髓腔形态特征对人工髋关节假体设计有重要指导作用，人工髋关节假体植入后的稳定性与假体和髓腔的匹配程度有直接关系。
若股骨髓腔形态特征的分析方法不合理，则会导致髋关节假体和人体匹配程度不理想，严重的可能会出现人工髋关节置换手术失败；而且，由于股骨髓腔形态特征有很大的个体差异，正确地描述股骨髓腔形态显得十分关键。
相关技术通常在描述股骨髓腔形态学特征参数时，只计算股骨各关键部位的长度(如冠状径或者矢状径)，然而单纯的只计算股骨关键位置处的冠状径或者矢状径并不能全面准确地反映股骨髓腔的形态。因此，如何寻找出合理的股骨髓腔形态识别方法对选择合适的髋关节假体及手术实施方案有重要指导作用。
发明内容
本申请实施例提供一种基于深度学习的股骨髓腔形态识别方法、装置及 存储介质，用以解决相关技术中只计算股骨关键位置处的冠状径或者矢状径，并不能全面准确地反映股骨的髓腔形态的种类的问题。
本申请实施例的第一方面，提供一种基于深度学习的股骨髓腔形态识别方法，包括：
获取髋关节的二维医学图像；
基于预先训练好的图像分割网络模型对所述髋关节的二维医学图像进行图像分割处理，基于图像分割结果得到股骨区域的二维医学图像；
对所述股骨区域的二维医学图像进行处理，得到股骨的三维医学图像；
在患者的冠状面方向上，将所述股骨的三维医学图像通过正投影变换得到股骨的正投影图像；
对所述股骨的正投影图像进行透视处理，得到所述股骨的透视图像；
将所述股骨的透视图像输入VGG分类器网络，以获得所述VGG分类器网络输出的股骨的髓腔形态的种类。
可选地，在第一方面的一种可能实现方式中，对所述股骨区域的二维医学图像进行处理，得到股骨的三维医学图像，包括：
对所述股骨区域的二维医学图像进行三维重建，得到所述股骨区域的三维医学图像；
基于所述股骨区域的三维医学图像，获得所述股骨区域的像素点点集中包含的若干个像素点的坐标；
基于所述若干个像素点的坐标，分别确定X轴坐标值最大的像素点、X轴坐标值最小的像素点、Y轴坐标值最大的像素点、Y轴坐标值最小的像素点、Z轴坐标值最大的像素点和Z轴坐标值最小的像素点；
根据所述X轴坐标值最大的像素点、X轴坐标值最小的像素点、Y轴坐标值最大的像素点、Y轴坐标值最小的像素点、Z轴坐标值最大的像素点和Z 轴坐标值最小的像素点，确定所述股骨的三维医学图像。
可选地，在第一方面的一种可能实现方式中，所述图像分割网络模型的预先训练过程包括：
获取髋关节的二维医学图像数据集，其中，所述二维医学图像数据集中包含有多个二维医学图像；
标注出各个所述二维医学图像中的股骨区域；
将经过标注后的各个二维医学图像按照预设比例划分为训练数据集、验证数据集和测试数据集；
基于所述训练数据集、验证数据集、测试数据集并结合神经网络算法和深度学习训练出图像分割网络模型。
可选地，在第一方面的一种可能实现方式中，基于所述训练数据集、验证数据集、测试数据集并结合神经网络算法和深度学习训练出图像分割网络模型，包括：
通过第一图像分割网络模型对所述训练数据集进行粗分割处理：对所述训练数据集中的二维医学图像执行多次下采样，以通过卷积层和池化层的处理识别各二维医学图像的深层特征；对进行下采样后的二维医学图像执行多次上采样，以通过上采样层和卷积层的处理反向存储所述深层特征至所述二维医学图像中；利用Adam分类优化器进行图像粗分类处理，获得图像粗分割结果；其中，所述各卷积层后均设置有激活函数；
通过第二图像分割模型对所述图像粗分割结果进行精分割处理：从所述深层特征中筛选预设置信度的特征点数据，对所述特征点数据进行双线性插值计算，基于计算后的特征点数据识别所述深层特征的所属类别，获得最终的图像分割结果；
基于所述最终的图像分割结果以及所述训练数据集、验证数据集和测试数据集计算损失函数；
基于所述损失函数调整所述图像分割网络模型的参数，直至所述图像分割网络模型训练成功。
可选地，在第一方面的一种可能实现方式中，所述将所述股骨的透视图像输入VGG分类器网络，以获得所述VGG分类器网络输出的股骨的髓腔形态的种类，包括：
所述VGG分类器网络输出的股骨的髓腔形态的种类包括：正常型、香槟型、烟囱型。
可选地，在第一方面的一种可能实现方式中，所述VGG分类器网络包含16个隐藏层，包括：13个卷积层和3个全连接层。
本申请实施例的第二方面，提供一种基于深度学习的股骨髓腔形态识别装置，包括：
获取模块，被配置为获取髋关节的二维医学图像；
分割模块，被配置为基于预先训练好的图像分割网络模型对所述髋关节的二维医学图像进行图像分割处理，基于图像分割结果得到股骨区域的二维医学图像；
图像处理模块，被配置为对所述股骨区域的二维医学图像进行处理，得到股骨的三维医学图像；在患者的冠状面方向上，将所述股骨的三维医学图像通过正投影变换得到股骨的正投影图像；对所述股骨的正投影图像进行透视处理，得到所述股骨的透视图像；
分类模块，被配置为将所述股骨的透视图像输入VGG分类器网络，以获得所述VGG分类器网络输出的股骨的髓腔形态的种类。
可选地，在第二方面的一种可能实现方式中，所述图像处理模块，还用于执行以下步骤，包括：
对所述股骨区域的二维医学图像进行三维重建，得到所述股骨区域的三维医学图像；
基于所述股骨区域的三维医学图像，获得所述股骨区域的像素点点集中包含的若干个像素点的坐标；
基于所述若干个像素点的坐标，分别确定X轴坐标值最大的像素点、X轴坐标值最小的像素点、Y轴坐标值最大的像素点、Y轴坐标值最小的像素点、Z轴坐标值最大的像素点和Z轴坐标值最小的像素点；
根据所述X轴坐标值最大的像素点、X轴坐标值最小的像素点、Y轴坐标值最大的像素点、Y轴坐标值最小的像素点、Z轴坐标值最大的像素点和Z轴坐标值最小的像素点，确定所述股骨的三维医学图像。
本申请实施例的第三方面，提供一种可读存储介质，所述可读存储介质中存储有计算机程序，所述计算机程序被处理器执行时用于实现本申请第一方面及第一方面各种可能设计的所述方法。
本申请实施例的第四方面，提供一种电子设备，包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述程序时实现如本申请第一方面及第一方面各种可能设计的所述方法的步骤。
本申请提供的一种基于深度学习的股骨髓腔形态识别方法、装置及存储介质，能够基于髋关节的二维医学图像使用深度学习技术对股骨的髓腔形态的种类进行精确识别和分类。深度学习技术与传统方法不同的是其能够在人体内部生成特征。深度学习技术由多层神经网络组成，这些神经网络在大量数据上进行训练，给出真实情况描述，然后用于预测测试数据集上的分段，从而识别股骨的髓腔形态。识别髓腔形态的结果较为精准，且速度较快，灵活、高效等特点，为股骨髓腔形态特征的分析提供了新方法，进而为科学地设计髋关节假体提供了数据支持。
附图说明
图1为本申请实施例提供的基于深度学习的股骨髓腔形态识别方法的第一种实施方式的流程图；
图2为本申请实施例提供的图像分割网络模型结构图；
图3为本申请实施例提供的VGG分类器网络的结构示意图；
图4为本申请实施例提供的股骨髓腔形态分类图；
图5为本申请实施例提供的基于深度学习的股骨髓腔形态识别装置的第一种实施方式的结构图；
图6为本申请实施例提供的一种电子设备的结构示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”“第四”等(如果存在)是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施。
应当理解，在本申请的各种实施例中，各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
应当理解，在本申请中，“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
应当理解，在本申请中，“多个”是指两个或两个以上。“和/或”仅仅是 一种描述关联对象的关联关系，表示可以存在三种关系，例如，和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。字符“/”一般表示前后关联对象是一种“或”的关系。“包含A、B和C”、“包含A、B、C”是指A、B、C三者都包含，“包含A、B或C”是指包含A、B、C三者之一，“包含A、B和/或C”是指包含A、B、C三者中任1个或任2个或3个。
应当理解，在本申请中，“与A对应的B”、“与A相对应的B”、“A与B相对应”或者“B与A相对应”，表示B与A相关联，根据A可以确定B。根据A确定B并不意味着仅仅根据A确定B，还可以根据A和/或其他信息确定B。A与B的匹配，是A与B的相似度大于或等于预设的阈值。
取决于语境，如在此所使用的“若”可以被解释成为“在……时”或“当……时”或“响应于确定”或“响应于检测”。
下面以具体地实施例对本申请的技术方案进行详细说明。下面这几个具体的实施例可以相互结合，对于相同或相似的概念或过程可能在某些实施例不再赘述。
本申请提供一种基于深度学习的股骨髓腔形态识别方法，如图1所示其流程图，包括：
步骤S110、获取髋关节的二维医学图像。
在本步骤中，需要获取髋关节的CT二维医学图像数据集，CT二维医学图像数据集中包括若干个患者髋关节的CT二维医学图像，并采用手动标注、自动标注中至少之一的方式对该CT二维医学图像进行股骨区域的标注，并且将标注后的CT二维医学图像按照预设比例划分为训练集、验证集和测视集。例如，可以按照6:2:2的比例划分。
将CT二维医学图像的DICOM数据转换成JPG格式的图片，将标注后的CT二维医学图像转换成png格式的图片，保存后作为图像分割网络模型的输入。
步骤S120、基于预先训练好的图像分割网络模型对所述髋关节的二维医学图像进行图像分割处理，基于图像分割结果得到股骨区域的二维医学图像。
在步骤S120中，通过预先训练好的图像分割网络模型：pointrend神经网络、unet卷积神经网络进行分割处理，即：先利用unet卷积神经网络作为主干网络，对上述标注过股骨区域的CT二维医学图像进行粗分割；然后对粗分割的结果使用pointrend神经网络进行精确分割。
步骤S130、对所述股骨区域的二维医学图像进行处理，得到股骨的三维医学图像；在患者的冠状面方向上，将所述股骨的三维医学图像通过正投影变换得到股骨的正投影图像；对所述股骨的正投影图像进行透视处理，得到所述股骨的透视图像。
步骤S140、将所述股骨的透视图像输入VGG分类器网络，以获得所述VGG分类器网络输出的股骨的髓腔形态的种类。
在步骤S140中，所使用的VGG分类器网络为VGG16,使用的全部都是3x3的小卷积核和2x2的池化核，可以通过不断加深网络来提升性能。其中包含16个隐藏层，包括：13个卷积层和3个全连接层(FC-4096、FC-4096、FC-1000)；卷积层为为5个卷积段，分别为：con3-64*2、con3-128*2、con3-256*3、con3-512*3、con3-512*3；同时每个卷积段的结尾都会连接一个最大的池化层maxpool,用于缩小图片的尺寸。
在步骤S140中，具体VGG分类器网络训练过程为：首先输入256*256*3的图片，经过64个3*3的卷积核做两次卷积+Relu，卷积后的尺寸变为256*256*64；通过max pooling，池化单元尺寸为2*2，池化后的尺寸变为128*128*64；经过128个3*3单侧卷积核做两次卷积+relu，尺寸变为128*128*128；通过2*2的max pooling池化，尺寸变为64*64*128；经过256个3*3的卷积核做三次卷积+relu，尺寸变为64*64*256；通过2*2的max pooling池化，尺寸变为32*32*256；经过512个3*3的卷积核做三次卷积+relu，尺寸变为32*32*512；通过2*2的max pooling池化，尺寸变为16*16*512；通过512个3*3的卷积核做三次卷积+relu，尺寸变为16*16*512；通过2*2的max pooling池化，尺寸变为8*8*512；与两层1*1*4096，一层1*1*1000 进行全连接(前两组全连接，每组都是fc-relu-dropout，最后一个全连接仅有fc)，最后通过softmax分类器输出预测结果。
在一个实施例中，对所述股骨区域的二维医学图像进行处理，得到股骨的三维医学图像，包括：对所述股骨区域的二维医学图像进行三维重建，得到所述股骨区域的三维医学图像；基于所述股骨区域的三维医学图像，获得所述股骨区域的像素点点集中包含的若干个像素点的坐标；基于所述若干个像素点的坐标，分别确定X轴坐标值最大的像素点、X轴坐标值最小的像素点、Y轴坐标值最大的像素点、Y轴坐标值最小的像素点、Z轴坐标值最大的像素点和Z轴坐标值最小的像素点；根据所述X轴坐标值最大的像素点、X轴坐标值最小的像素点、Y轴坐标值最大的像素点、Y轴坐标值最小的像素点、Z轴坐标值最大的像素点和Z轴坐标值最小的像素点，确定所述股骨的三维医学图像。
在对股骨区域图像进行提取，得到股骨图像的过程中，首先需要定义股骨的分割区域为V，其中V包含了所有股骨区域的像素点点集；其次在所有像素点的X坐标中分别找到Xmin、Xmax,在所有像素点的Y坐标中分别找到Ymin、Ymax,在所有像素点的Z坐标中分别找到Zmin、Zmax,则通过Xmin、Xmax、Ymin、Ymax,、Zmin、Zmax就可以确定股骨的三维空间范围。
在一个实施例中，图像分割网络模型的预先训练过程包括：
获取髋关节的二维医学图像数据集，其中，所述二维医学图像数据集中包含有多个二维医学图像；
标注出各个所述二维医学图像中的股骨区域；
将经过标注后的各个二维医学图像按照预设比例划分为训练数据集、验证数据集和测试数据集；
基于所述训练数据集、验证数据集、测试数据集并结合神经网络算法和深度学习训练出图像分割网络模型。
在本步骤中，会先获取多个患者髋关节的CT医学图像，以手动或者自 动标注的方式对其进行标注股骨区域，然后按照6：2:2的比例将多张标注过股骨区域的CT图像划分为训练集、验证集、测试集，以作为图像分割网络模型的输入。
在一些实施例中，基于所述训练数据集、验证数据集、测试数据集并结合神经网络算法和深度学习训练出图像分割网络模型，包括：
通过第一图像分割网络模型对所述训练数据集进行粗分割处理：对所述训练数据集中的二维医学图像执行多次下采样，以通过卷积层和池化层的处理识别各二维医学图像的深层特征；对进行下采样后的二维医学图像执行多次上采样，以通过上采样层和卷积层的处理反向存储所述深层特征至所述二维医学图像中；利用Adam分类优化器进行图像粗分类处理，获得图像粗分割结果；其中，所述各卷积层后均设置有激活函数；
通过第二图像分割模型对所述图像粗分割结果进行精分割处理：从所述深层特征中筛选预设置信度的特征点数据，对所述特征点数据进行双线性插值计算，基于计算后的特征点数据识别所述深层特征的所属类别，获得最终的图像分割结果；
基于所述最终的图像分割结果以及所述训练数据集、验证数据集和测试数据集计算损失函数；
基于所述损失函数调整所述图像分割网络模型的参数，直至所述图像分割网络模型训练成功。
在上述基于预先训练好的图像分割网络模型对所述髋关节的二维医学图像进行图像分割处理，以得到股骨区域的二维医学图像的步骤中，主要包括：利用第一图像处理子模型unet进行粗分割和使用第二图像处理子模型pointrend进行精确分割，具体网络结构如图2所示，具体过程如下：
“利用第一图像处理子模型unet进行粗分割”具体是指：利用unet网络作为主干网络，对输入的二维图像进行粗分割，第一阶段使用4次下采样学习图像的深层特征，然后进行4次上采样以将特征图重新存储到图像中，其中每个下采样层中包括2个卷积层和1个池化层，卷积核大小为3*3，池化 层中的卷积核大小为2*2，每个卷积层中的卷积核的个数为128，256，512；每个上采样层中包括1个上采样层和2个卷积层，其中卷积层的卷积核大小为3*2，上采样层中的卷积核大小为2*2，每个上采样层中的卷积核个数为512，256，128。最后一次上采样结束后设有一个的dropout层，droupout率设置为0.7。所有的卷积层后面都设有激活函数为relu函数。最终获得股骨的粗分割预测结果，它们的结果均为0-1之间的预测概率值。
在上述粗分割网络模型训练的过程中，数据标签的背景像素值设置为0，股骨为1，训练的batch_size为6，学习率设置为1e -4，分类优化器使用Adam分类优化器，使用的损失函数为DICE loss，将训练集原图和股骨/胫骨/腓骨/髌骨分别送入网络进行训练，根据训练过程中损失函数的变化，调整训练批次的大小，最终得到各个部分的粗分割结果。
“使用第二图像处理子模型pointrend进行精确分割”具体是指：使用pointrend精确分割结果，首先像素选择的目标是挑选出一系列潜在的特征点来为接下来的判断做准备，在此我们挑选的依据是粗略分割的结果中分类置信度接近0.5的点(二分类任务中一个点的置信度会趋近于0或者1，置信度在0.5附近则代表网络对这个点的分类很不确定)，通常这类点都是接近物体边缘的点。第二步我们对上步挑选出的点进行特征提取，而粗略分割网络就自带特征提取器(feature extractor)，我们只需要将所选点在特征提取器中相应位置的特征选取出来即可。这些点的特征通过双线性插值Bilinear计算，使用一个小型的分类器去判断这个点属于哪个类别。这其实是等价于用一个1*1的卷积来预测，但是对于置信度接近于1或者0的点并不计算。这样我们就可以对所有不确定的像素点逐个进行分类，从而提高分割的精准度。
在上述精确分割网络模型训练过程中，进入pointrend模块后，先会使用双线性插值上采样前一步分割预测结果，然后在这个更密集的特征图中选择N个最不确定的点，比如概率接近0.5的点。然后计算这N个点的特征表示并且预测它们的labels，这个过程一直被重复，直到上采样到需要的大小。对于每个选定点的逐点特征表示，使用简单的多层感知器进行逐点预测，因为MLP预测的是各点的分割label，所以可以使用Unet粗分割任务中的loss来训练。训练后输出就是股骨区域。
如图3所示的VGG分类器网络的结构示意图，如图4所示的股骨髓腔形态分类图，在一个实施例中，所述将所述股骨的透视图像输入VGG分类器网络，以获得所述VGG分类器网络输出的股骨的髓腔形态的种类，股骨的髓腔形态的种类包括：正常型、香槟型、烟囱型。
最终，本申请提供的一种基于深度学习的股骨髓腔形态识别方法、装置及存储介质，能够基于髋关节的二维医学图像使用深度学习技术对股骨的髓腔形态的种类进行精确识别和分类。深度学习技术与传统方法不同的是其能够在人体内部生成特征。深度学习技术由多层神经网络组成，这些神经网络在大量数据上进行训练，给出真实情况描述，然后用于预测测试数据集上的分段，从而识别股骨的髓腔形态。识别髓腔形态的结果也较为精准，且速度较快，灵活、高效等特点，为股骨髓腔形态特征的分析提供了新方法，进而为科学地设计髋关节假体提供了数据支持。
本申请的实施例还提供一种基于深度学习的股骨髓腔形态识别装置，如图5所示，包括：
获取模块51，被配置为获取髋关节的二维医学图像；
分割模块52，被配置为基于预先训练好的图像分割网络模型对所述髋关节的二维医学图像进行图像分割处理，基于图像分割结果得到股骨区域的二维医学图像；
图像处理模块53，被配置为对所述股骨区域的二维医学图像进行处理，得到股骨的三维医学图像；在患者的冠状面方向上，将所述股骨的三维医学图像通过正投影变换得到股骨的正投影图像；对所述股骨的正投影图像进行透视处理，得到所述股骨的透视图像；
分类模块54，被配置为将所述股骨的透视图像输入VGG分类器网络，以获得所述VGG分类器网络输出的股骨的髓腔形态的种类。
在一个实施例中，所述图像处理模块53，还被配置为执行以下步骤，包括：
对所述股骨区域的二维医学图像进行三维重建，得到所述股骨区域的三维医学图像；
基于所述股骨区域的三维医学图像，获得所述股骨区域的像素点点集中包含的若干个像素点的坐标；
基于所述若干个像素点的坐标，分别确定X轴坐标值最大的像素点、X轴坐标值最小的像素点、Y轴坐标值最大的像素点、Y轴坐标值最小的像素点、Z轴坐标值最大的像素点和Z轴坐标值最小的像素点；
根据所述X轴坐标值最大的像素点、X轴坐标值最小的像素点、Y轴坐标值最大的像素点、Y轴坐标值最小的像素点、Z轴坐标值最大的像素点和Z轴坐标值最小的像素点，确定所述股骨的三维医学图像。
在一些实施例中，本申请还提供了一种可读存储介质，所述可读存储介质中存储有计算机程序，所述计算机程序被处理器执行时用于实现上述的基于深度学习的股骨髓腔形态识别方法，该方法包括：
获取髋关节的二维医学图像；
基于预先训练好的图像分割网络模型对所述髋关节的二维医学图像进行图像分割处理，基于图像分割结果得到股骨区域的二维医学图像；
对所述股骨区域的二维医学图像进行处理，得到股骨的三维医学图像；
在患者的冠状面方向上，将所述股骨的三维医学图像通过正投影变换得到股骨的正投影图像；
对所述股骨的正投影图像进行透视处理，得到所述股骨的透视图像；
将所述股骨的透视图像输入VGG分类器网络，以获得所述VGG分类器网络输出的股骨的髓腔形态的种类。
其中，可读存储介质可以是计算机存储介质，也可以是通信介质。通信介质包括便于从一个地方向另一个地方传送计算机程序的任何介质。计算机存储介质可以是通用或专用计算机能够存取的任何可用介质。例如，可读存 储介质耦合至处理器，从而使处理器能够从该可读存储介质读取信息，且可向该可读存储介质写入信息。当然，可读存储介质也可以是处理器的组成部分。处理器和可读存储介质可以位于专用集成电路(Application Specific Integrated Circuits，ASIC)中。另外，该ASIC可以位于用户设备中。当然，处理器和可读存储介质也可以作为分立组件存在于通信设备中。可读存储介质可以是只读存储器(ROM)、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。
本申请还提供一种程序产品，该程序产品包括执行指令，该执行指令存储在可读存储介质中。设备的至少一个处理器可以从可读存储介质读取该执行指令，至少一个处理器执行该执行指令使得设备实施上述的各种实施方式提供的方法。
本申请实施例还提供一种电子设备，如图6所示，该电子设备可以包括：处理器(processor)610、通信接口(Communications Interface)620、存储器(memory)630和通信总线640，其中，处理器610，通信接口620，存储器630通过通信总线640完成相互间的通信。处理器610可以调用存储器630中的逻辑指令，以执行上述的基于深度学习的股骨髓腔形态识别方法，该方法包括：
获取髋关节的二维医学图像；
基于预先训练好的图像分割网络模型对所述髋关节的二维医学图像进行图像分割处理，基于图像分割结果得到股骨区域的二维医学图像；
对所述股骨区域的二维医学图像进行处理，得到股骨的三维医学图像；
在患者的冠状面方向上，将所述股骨的三维医学图像通过正投影变换得到股骨的正投影图像；
对所述股骨的正投影图像进行透视处理，得到所述股骨的透视图像；
将所述股骨的透视图像输入VGG分类器网络，以获得所述VGG分类器网络输出的股骨的髓腔形态的种类。
在上述终端或者服务器的实施例中，应理解，处理器可以是中央处理单元(英文：Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器(英文：Digital Signal Processor，DSP)等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请所公开的方法的步骤可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。
最后应说明的是：以上各实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述各实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。

Claims (10)

  1. 一种基于深度学习的股骨髓腔形态识别方法，包括：
    获取髋关节的二维医学图像；
    基于预先训练好的图像分割网络模型对所述髋关节的二维医学图像进行图像分割处理，基于图像分割结果得到股骨区域的二维医学图像；
    对所述股骨区域的二维医学图像进行处理，得到股骨的三维医学图像；
    在患者的冠状面方向上，将所述股骨的三维医学图像通过正投影变换得到股骨的正投影图像；
    对所述股骨的正投影图像进行透视处理，得到所述股骨的透视图像；
    将所述股骨的透视图像输入VGG分类器网络，以获得所述VGG分类器网络输出的股骨的髓腔形态的种类。
  2. 根据权利要求1所述的基于深度学习的股骨髓腔形态识别方法，其中，对所述股骨区域的二维医学图像进行处理，得到股骨的三维医学图像，包括：
    对所述股骨区域的二维医学图像进行三维重建，得到所述股骨区域的三维医学图像；
    基于所述股骨区域的三维医学图像，获得所述股骨区域的像素点点集中包含的若干个像素点的坐标；
    基于所述若干个像素点的坐标，分别确定X轴坐标值最大的像素点、X轴坐标值最小的像素点、Y轴坐标值最大的像素点、Y轴坐标值最小的像素点、Z轴坐标值最大的像素点和Z轴坐标值最小的像素点；
    根据所述X轴坐标值最大的像素点、X轴坐标值最小的像素点、Y轴坐标值最大的像素点、Y轴坐标值最小的像素点、Z轴坐标值最大的像素点和Z轴坐标值最小的像素点，确定所述股骨的三维医学图像。
  3. 根据权利要求1所述的基于深度学习的股骨髓腔形态识别方法，其中，所述图像分割网络模型的预先训练过程包括：
    获取髋关节的二维医学图像数据集，其中，所述二维医学图像数据集中包含有多个二维医学图像；
    标注出各个所述二维医学图像中的股骨区域；
    将经过标注后的各个二维医学图像按照预设比例划分为训练数据集、验证数据集和测试数据集；
    基于所述训练数据集、验证数据集、测试数据集并结合神经网络算法和 深度学习训练出图像分割网络模型。
  4. 根据权利要求3所述的基于深度学习的股骨髓腔形态识别方法，其中，基于所述训练数据集、验证数据集、测试数据集并结合神经网络算法和深度学习训练出图像分割网络模型，包括：
    通过第一图像分割网络模型对所述训练数据集进行粗分割处理：对所述训练数据集中的二维医学图像执行多次下采样，以通过卷积层和池化层的处理识别各二维医学图像的深层特征；对进行下采样后的二维医学图像执行多次上采样，以通过上采样层和卷积层的处理反向存储所述深层特征至所述二维医学图像中；利用Adam分类优化器进行图像粗分类处理，获得图像粗分割结果；其中，所述各卷积层后均设置有激活函数；
    通过第二图像分割模型对所述图像粗分割结果进行精分割处理：从所述深层特征中筛选预设置信度的特征点数据，对所述特征点数据进行双线性插值计算，基于计算后的特征点数据识别所述深层特征的所属类别，获得最终的图像分割结果；
    基于所述最终的图像分割结果以及所述训练数据集、验证数据集和测试数据集计算损失函数；
    基于所述损失函数调整所述图像分割网络模型的参数，直至所述图像分割网络模型训练成功。
  5. 根据权利要求1所述的基于深度学习的股骨髓腔形态识别方法，其中，所述将所述股骨的透视图像输入VGG分类器网络，以获得所述VGG分类器网络输出的股骨的髓腔形态的种类，包括：
    所述VGG分类器网络输出的股骨的髓腔形态的种类包括：正常型、香槟型、烟囱型。
  6. 根据权利要求5所述的基于深度学习的股骨髓腔形态识别方法，其中，所述VGG分类器网络包含16个隐藏层，包括：13个卷积层和3个全连接层。
  7. 一种基于深度学习的股骨髓腔形态识别装置，包括：
    获取模块，被配置为获取髋关节的二维医学图像；
    分割模块，被配置为基于预先训练好的图像分割网络模型对所述髋关节的二维医学图像进行图像分割处理，基于图像分割结果得到股骨区域的二维医学图像；
    图像处理模块，被配置为对所述股骨区域的二维医学图像进行处理，得到股骨的三维医学图像；在患者的冠状面方向上，将所述股骨的三维医学图像通过正投影变换得到股骨的正投影图像；对所述股骨的正投影图像进行透视处理，得到所述股骨的透视图像；
    分类模块，被配置为将所述股骨的透视图像输入VGG分类器网络，以获得所述VGG分类器网络输出的股骨的髓腔形态的种类。
  8. 根据权利要求7所述的基于深度学习的股骨髓腔形态识别装置，其中，所述图像处理模块，还用于执行以下步骤，包括：
    对所述股骨区域的二维医学图像进行三维重建，得到所述股骨区域的三维医学图像；
    基于所述股骨区域的三维医学图像，获得所述股骨区域的像素点点集中包含的若干个像素点的坐标；
    基于所述若干个像素点的坐标，分别确定X轴坐标值最大的像素点、X轴坐标值最小的像素点、Y轴坐标值最大的像素点、Y轴坐标值最小的像素点、Z轴坐标值最大的像素点和Z轴坐标值最小的像素点；
    根据所述X轴坐标值最大的像素点、X轴坐标值最小的像素点、Y轴坐标值最大的像素点、Y轴坐标值最小的像素点、Z轴坐标值最大的像素点和Z轴坐标值最小的像素点，确定所述股骨的三维医学图像。
  9. 一种电子设备，包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述程序时实现如权利要求1至6任一项所述的基于深度学习的股骨髓腔形态识别方法的步骤。
  10. 一种可读存储介质，所述可读存储介质中存储有计算机程序，所述计算机程序被处理器执行时用于实现权利要求1至6任一项所述的基于深度学习的股骨髓腔形态识别方法的步骤。
PCT/CN2022/110972 2021-08-24 2022-08-08 基于深度学习的股骨髓腔形态识别方法、装置及存储介质 WO2023024882A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110974202.6A CN113689402B (zh) 2021-08-24 2021-08-24 基于深度学习的股骨髓腔形态识别方法、装置及存储介质
CN202110974202.6 2021-08-24

Publications (1)

Publication Number Publication Date
WO2023024882A1 true WO2023024882A1 (zh) 2023-03-02

Family

ID=78581775

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/110972 WO2023024882A1 (zh) 2021-08-24 2022-08-08 基于深度学习的股骨髓腔形态识别方法、装置及存储介质

Country Status (2)

Country Link
CN (1) CN113689402B (zh)
WO (1) WO2023024882A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116934708A (zh) * 2023-07-20 2023-10-24 北京长木谷医疗科技股份有限公司 胫骨平台内外侧低点计算方法、装置、设备及存储介质

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113689402B (zh) * 2021-08-24 2022-04-12 北京长木谷医疗科技有限公司 基于深度学习的股骨髓腔形态识别方法、装置及存储介质
CN113870261B (zh) * 2021-12-01 2022-05-13 杭州柳叶刀机器人有限公司 用神经网络识别力线的方法与系统、存储介质及电子设备
CN114494183B (zh) * 2022-01-25 2024-04-02 哈尔滨医科大学附属第一医院 一种基于人工智能的髋臼半径自动测量方法及系统
CN114419618B (zh) * 2022-01-27 2024-02-02 北京长木谷医疗科技股份有限公司 基于深度学习的全髋关节置换术前规划系统
CN115131301B (zh) * 2022-06-15 2023-04-18 北京长木谷医疗科技有限公司 基于深度学习的智能识别骨关节炎的方法及系统
CN115131300B (zh) * 2022-06-15 2023-04-07 北京长木谷医疗科技有限公司 一种基于深度学习的骨性关节炎智能三维诊断方法及系统
CN116309636B (zh) * 2023-02-21 2024-07-23 北京长木谷医疗科技股份有限公司 基于多任务神经网络模型的膝关节分割方法、装置及设备
CN116630427B (zh) * 2023-07-24 2023-10-31 杭州键嘉医疗科技股份有限公司 一种ct图像中髋骨和股骨自动定位的方法及装置
CN117437459B (zh) * 2023-10-08 2024-03-22 昆山市第一人民医院 基于决策网络实现用户膝关节髌骨软化状态分析方法
CN117689683B (zh) * 2024-02-01 2024-05-03 江苏一影医疗设备有限公司 一种双腿膝关节运动状态图像处理方法、设备及存储介质
CN118279248B (zh) * 2024-03-18 2024-10-11 中国人民解放军总医院第六医学中心 一种基于图像处理技术的股骨近端中pct的提取方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190105009A1 (en) * 2017-10-10 2019-04-11 Holo Surgical Inc. Automated segmentation of three dimensional bony structure images
CN111179350A (zh) * 2020-02-13 2020-05-19 张逸凌 基于深度学习的髋关节图像处理方法及计算设备
CN111402230A (zh) * 2020-03-16 2020-07-10 徐州医科大学 基于Monte Carlo方法的股骨髓腔形态特征分析方法
CN112634283A (zh) * 2020-12-22 2021-04-09 华中科技大学 使用小样本图像训练的髋关节分割模型建立方法及其应用
CN113076987A (zh) * 2021-03-29 2021-07-06 北京长木谷医疗科技有限公司 骨赘识别方法、装置、电子设备及存储介质
CN113689402A (zh) * 2021-08-24 2021-11-23 北京长木谷医疗科技有限公司 基于深度学习的股骨髓腔形态识别方法、装置及存储介质

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1437685A3 (de) * 2003-01-13 2004-07-21 Philips Intellectual Property & Standards GmbH Verfahren zum Segmentieren einer dreidimensionalen Struktur
US10687856B2 (en) * 2007-12-18 2020-06-23 Howmedica Osteonics Corporation System and method for image segmentation, bone model generation and modification, and surgical planning
CN104657984B (zh) * 2015-01-28 2018-10-16 复旦大学 三维超声乳腺全容积图像感兴趣区域的自动提取方法
CN105869149B (zh) * 2016-03-24 2019-04-16 大连理工大学 基于主向量分析的断骨截面分割及断骨模型配准方法
CN107958223B (zh) * 2017-12-13 2020-09-18 北京小米移动软件有限公司 人脸识别方法及装置、移动设备、计算机可读存储介质
US10796475B2 (en) * 2017-12-13 2020-10-06 Carestream Health, Inc. Bone segmentation and display for 3D extremity imaging
CN110009666B (zh) * 2018-01-05 2021-01-22 北京柏惠维康科技有限公司 一种机器人空间配准中建立匹配模型的方法及装置
CN111583240B (zh) * 2020-05-09 2023-05-26 北京天智航医疗科技股份有限公司 一种股骨端前后轴线的确定方法、装置和计算机设备
CN111714145B (zh) * 2020-05-27 2022-07-01 浙江飞图影像科技有限公司 基于弱监督分割的股骨颈骨折检测方法及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190105009A1 (en) * 2017-10-10 2019-04-11 Holo Surgical Inc. Automated segmentation of three dimensional bony structure images
CN111179350A (zh) * 2020-02-13 2020-05-19 张逸凌 基于深度学习的髋关节图像处理方法及计算设备
CN111402230A (zh) * 2020-03-16 2020-07-10 徐州医科大学 基于Monte Carlo方法的股骨髓腔形态特征分析方法
CN112634283A (zh) * 2020-12-22 2021-04-09 华中科技大学 使用小样本图像训练的髋关节分割模型建立方法及其应用
CN113076987A (zh) * 2021-03-29 2021-07-06 北京长木谷医疗科技有限公司 骨赘识别方法、装置、电子设备及存储介质
CN113689402A (zh) * 2021-08-24 2021-11-23 北京长木谷医疗科技有限公司 基于深度学习的股骨髓腔形态识别方法、装置及存储介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116934708A (zh) * 2023-07-20 2023-10-24 北京长木谷医疗科技股份有限公司 胫骨平台内外侧低点计算方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN113689402A (zh) 2021-11-23
CN113689402B (zh) 2022-04-12

Similar Documents

Publication Publication Date Title
WO2023024882A1 (zh) 基于深度学习的股骨髓腔形态识别方法、装置及存储介质
EP3961500A1 (en) Medical image detection method based on deep learning, and related device
WO2022257344A1 (zh) 图像配准融合方法及装置、模型训练方法及电子设备
WO2022257345A1 (zh) 医学图像融合方法及系统、模型训练方法及存储介质
CN111062955A (zh) 一种肺部ct影像数据分割方法及系统
CN113076987B (zh) 骨赘识别方法、装置、电子设备及存储介质
WO2023029896A1 (zh) 基于强化学习的髋臼杯位置调整方法、装置及存储介质
WO2024001140A1 (zh) 一种椎体亚区域分割方法、装置及存储介质
WO2022247173A1 (zh) 图像识别及模型训练的方法、关节位置识别的方法
CN110570394A (zh) 医学图像分割方法、装置、设备及存储介质
CN113643353B (zh) 眼底图像的血管管径的增强分辨率的测量方法
CN114445904A (zh) 基于全卷积神经网络的虹膜分割方法和装置、介质和设备
Abbas et al. Lower leg bone fracture detection and classification using faster RCNN for X-rays images
Thangam et al. Skeletal Bone Age Assessment-Research Directions.
CN116563533A (zh) 基于目标位置先验信息的医学图像分割方法及系统
CN118229724A (zh) 人像抠图过程中不透明度图的获取方法及装置
CN117710760A (zh) 残差的注意神经网络用于胸部x线病灶检测的方法
CN113077418A (zh) 基于卷积神经网络的ct影像骨骼分割方法及装置
Yadollahi et al. The use of combined illumination in segmentation of orthodontic bodies
CN115252233B (zh) 基于深度学习的全髋关节置换术中髋臼杯的自动化规划方法
CN114332128B (zh) 医学图像处理方法及其装置、电子设备和计算机存储介质
CN115410032A (zh) 基于自监督学习的octa图像分类结构训练方法
CN114419061A (zh) 一种肺部动静脉血管分割方法及系统
CN114359308A (zh) 一种基于边缘响应与非线性损失的主动脉夹层分割方法
Al-Eiadeh Automatic lung field segmentation using robust deep learning criteria

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22860228

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 18.06.2024)