WO2021068933A1 - 一种基于深度学习网络的椎弓根钉手术路径自动规划方法 - Google Patents

一种基于深度学习网络的椎弓根钉手术路径自动规划方法 Download PDF

Info

Publication number
WO2021068933A1
WO2021068933A1 PCT/CN2020/120168 CN2020120168W WO2021068933A1 WO 2021068933 A1 WO2021068933 A1 WO 2021068933A1 CN 2020120168 W CN2020120168 W CN 2020120168W WO 2021068933 A1 WO2021068933 A1 WO 2021068933A1
Authority
WO
WIPO (PCT)
Prior art keywords
network
surgical
path
spine
surgical path
Prior art date
Application number
PCT/CN2020/120168
Other languages
English (en)
French (fr)
Inventor
刘文勇
蔡东阳
王再跃
谭保森
Original Assignee
北京航空航天大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京航空航天大学 filed Critical 北京航空航天大学
Publication of WO2021068933A1 publication Critical patent/WO2021068933A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • A61B2034/101—Computer-aided simulation of surgical operations
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • A61B2034/108—Computer aided selection or customisation of medical implants or cutting guides
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]

Definitions

  • the invention relates to the technical field of medical engineering, in particular to a method for automatically planning a pedicle screw surgical path based on a deep learning network.
  • Surgical path planning is the key core step to realize intelligent orthopedic surgery (especially computer-assisted or robot-assisted orthopedic surgery under image guidance).
  • Traditional surgical path planning methods mostly rely on the experience of the surgeon (doctor).
  • the surgeon's physiological state, subjective judgment and other factors significantly affect the quality of the path planning, and the stability and operation efficiency of the surgical planning result cannot be effectively guaranteed. Therefore, seeking a more efficient and intelligent surgical path planning method is becoming a research and application hotspot of intelligent orthopedic surgery.
  • the purpose of the present invention is to provide a spine surgery path planning method based on deep learning, which can automatically plan the linear path required for spinal pedicle screw implantation operations, reduce the burden on the surgeon, and ensure the accuracy of path planning Performance and stability, and promote the precise and intelligent development of intelligent orthopedic surgery.
  • a method for automatic planning of pedicle screw surgery path based on deep learning network including the following steps:
  • Step 1 Establish an abstract expression of the surgical path of the spine pedicle screw, and express the surgical path as two discrete points: the surgical entry point and the surgical direction point;
  • Step 2 Establish a dataset of pedicle screw surgery path; the dataset uses three-dimensional CT images as the original image, and annotates the spine segmentation image and the location of the key points of the surgery path;
  • Step 3 Establish a segmentation network for the shape of the vertebrae; segment the shape and position of each vertebral column on the three-dimensional CT image;
  • Step 4 Establish a positioning network for key points of surgery; locate key points of the surgical path on the vertebral column;
  • Step 5 According to the pre-trained network model, automatically segment the spine vertebral blocks on the 3D CT image, and automatically locate the key points of the surgical path; plan the surgical path according to the key points of the surgical path; at the same time, adopt quantitative and qualitative methods Method to evaluate the rationality of path planning.
  • the key point on the surgical path is the entry point and direction point calibrated by the surgeon, wherein the surgical entry point is located in the depression of the transverse process of the pedicle and is used to locate the entry position of the pedicle screw ;
  • the operation direction point is located at the centroid of the narrowest section of the lamina, and is used to constrain the direction of the pedicle screw.
  • the method of "semi-automatic annotation + manual fine adjustment" is used to label the spine segmentation map.
  • Amira or other medical image processing software is used to semi-automatically mark the contour of the vertebral block, and then the refined segmentation is performed manually; The key points of the surgical path are marked manually.
  • the network is divided into two parts: an encoder and a decoder, where the encoder is responsible for extracting the features of the three-dimensional CT image, and the decoder is responsible for generating images of spine segmentation; between the encoder and the decoder
  • this network uses six-layer "downsampling + cross-layer connection" to replace the four-layer downsampling in 3D-Unet; at the same time, it uses residual
  • the difference convolution structure replaces the traditional convolution, which makes the network deeper and can better extract image information; in terms of the loss function, the Dice loss function and the softmax loss function are combined to train the network, from the image level and the pixel level.
  • the level constrains the degree of convergence of the network.
  • the regression network is constructed using the "convolutional layer + fully connected layer” method; the network is divided into two parts: a feature extraction network and a key point location network; among them, the feature extraction network is responsible for extracting three-dimensional CT
  • the feature information of the image, and the input of the surgical key point positioning network is the feature extracted by the feature extraction network, and the output is six values, representing the three coordinates of the X, Y, and Z axes of the entry point and the direction point.
  • the positions of the spine vertebral block and the key points of the operation can be calculated according to the pre-trained network model.
  • the connection and segmentation between the key points The intersection point of the surface of the vertebral block is used as the entry point of the pedicle screw, and the path planning is made based on the entry point according to the direction of the line of the key points.
  • the quality of the spine segmentation can be evaluated according to the IOU (Intersection over Union) index, and the positioning of the key points of the surgical path can be evaluated according to the mean square error loss; From the perspective of qualitative evaluation, the quality of surgical path planning can be judged according to the subjective evaluation of the surgeon.
  • IOU Intersection over Union
  • An automatic planning method for pedicle screw surgery path based on deep learning network including:
  • the data set uses a three-dimensional CT image as the original image, and annotated the spine segmentation image and the key point position of the surgical path;
  • An automatic surgical path planning network is established according to the surgical entry point and surgical direction point of the surgical path and the data set;
  • the automatic surgical path planning network includes a vertebral block shape segmentation network and a surgical key point positioning network;
  • the vertebral block shape The segmentation network takes the data set as input, and segments the shape and position of each spinal vertebral block on the three-dimensional CT image as output;
  • the operation key point positioning network takes the operation entry point and the operation direction point as well as the The data set is input, and the position of the key point of the surgical path is the output;
  • the trained operation path automatic planning network is used to determine the needle path of the spine pedicle screw operation and the geometric shape of the vertebral block of the spine.
  • the present invention discloses the following technical effects:
  • the invention provides a method for spine surgery path planning based on deep learning, which abstracts the spine pedicle screw surgery path as a straight line and discretely expresses it as two specific key points of the surgery path.
  • the shape segmentation network of the spine is constructed, and the segmentation accuracy of the spine data set is better than that of the traditional algorithm.
  • the key point (path point) positioning network is constructed, and the automatic planning of the surgical path based on "neural network preliminary planning + path point relocation calculation" is realized.
  • the invention uses deep learning to realize the automatic and intelligent planning of the surgical path of the spine pedicle screw.
  • FIG. 1 is a schematic diagram of the overall flow of a method for planning a spinal surgery path based on deep learning provided by the present invention
  • FIG. 2 is a schematic diagram of the structure of the shape segmentation network provided by the present invention.
  • Fig. 3 is a schematic structural diagram of the key point positioning network provided by the present invention.
  • the purpose of the present invention is to provide a spine surgery path planning method based on deep learning, which can automatically plan the linear path required for spinal pedicle screw implantation operations, reduce the burden on the surgeon, and ensure the accuracy of path planning Performance and stability, and promote the precise and intelligent development of intelligent orthopedic surgery.
  • Figure 1 is a schematic diagram of the overall flow of a method for planning a spine surgery path based on deep learning provided by the present invention.
  • Figure 1 shows a method for automatically planning a pedicle screw surgery path based on a deep learning network, including the following steps:
  • Step 1 Establish an abstract expression of the surgical path of the spine pedicle screw, and express the surgical path as two discrete points: the surgical entry point and the surgical direction point.
  • the key points on the surgical path are the entry point and the direction point calibrated by the surgeon, wherein the surgical entry point is located in the depression of the transverse process of the pedicle, and is used to locate the entry position of the pedicle screw; the surgical direction point It is located at the centroid of the narrowest section of the lamina and is used to constrain the direction of the pedicle screw.
  • Step 2 Establish a pedicle screw surgery path data set; the data set uses three-dimensional CT images as the original image, and annotates the spine segmentation image and the key points of the surgery path.
  • the spine segmentation map is marked with the method of "semi-automatic marking + manual fine adjustment".
  • Amira or other medical image processing software is used to semi-automatically mark the contour of the vertebral mass, and then manually perform refined segmentation; mark the key points of the surgical path Marking is done manually.
  • Step 3 Establish a segmentation network for the shape of the vertebrae; segment the shape and position of each vertebral column on the three-dimensional CT image.
  • the network is divided into two parts: the encoder and the decoder, where the encoder is responsible for extracting the features of the three-dimensional CT image, and the decoder is responsible for generating images of the spine segmentation; between the encoder and the decoder, the design span Layer connection for information exchange; different from the traditional 3D-Unet, the network adopts the six-layer "downsampling + cross-layer connection" method, instead of the four-layer downsampling in 3D-Unet; at the same time, the residual convolution structure is used instead of the traditional Convolution makes the network deeper and can better extract image information; in terms of loss function, the Dice loss function is combined with the softmax loss function to train the network, and the convergence of the network is carried out at the image level and the pixel level. constraint.
  • Step 4 Establish a positioning network for key points of surgery; locate key points of the surgical path on the vertebral column.
  • the regression network is constructed by "convolutional layer + fully connected layer"; the network is divided into two parts, the feature extraction network and the key point location network; among them, the feature extraction network is responsible for extracting the feature information of the 3D CT image , And the input of the surgical key point positioning network is the feature extracted by the feature extraction network, and the output is six values, representing the three coordinates of the X, Y, and Z axes of the entry point and the direction point.
  • Step 5 According to the pre-trained network model, automatically segment the spine vertebrae on the 3D CT image, and automatically locate the key points of the surgical path; plan the surgical path according to the key points of the surgical path; that is, according to the key points of the surgical path and the key points of the surgical path.
  • the segmented shape of the vertebral block automatically plans the surgical path.
  • quantitative and qualitative methods are used to evaluate the rationality of path planning.
  • the positions of the spine vertebral block and the key points of the operation can be calculated according to the pre-trained network model.
  • the connection between the key points and the segmentation of the surface of the vertebral block The intersection point is used as the entry point of the pedicle screw, and the path planning is made based on the entry point in accordance with the direction of the line of the key points.
  • the quality of the spine segmentation can be evaluated according to the IOU index, and the positioning of the key points of the surgical path can be evaluated according to the mean square error loss; in the perspective of qualitative evaluation, it can be evaluated according to the operator The subjective evaluation to judge the quality of the surgical path planning.
  • the present invention also provides an automatic planning method for pedicle screw surgery path based on deep learning network, including:
  • S2 Determine the surgical path according to the characteristics.
  • S3 Determine an operation entry point and an operation direction point of the operation path according to the geometric characteristics of the operation path.
  • S4 Obtain a data set of the surgical path of the spine pedicle screw; the data set uses a three-dimensional CT image as the original image, and annotates the spine segmentation image and the key point position of the surgical path.
  • S5 Establish an automatic surgical path planning network according to the surgical entry point and surgical direction point of the surgical path and the data set;
  • the automatic surgical path planning network includes a vertebral block shape segmentation network and a surgical key point positioning network;
  • the vertebra The block shape segmentation network takes the data set as input, and segments the shape and position of each spine vertebral block on the three-dimensional CT image as output;
  • the operation key point positioning network takes the operation entry point and operation direction point and The data set is input, and the position of the key point of the surgical path is the output.
  • the trained surgical path automatic planning network is used to determine the needle insertion path of the spine pedicle screw operation and the geometric shape of the vertebral block of the spine.
  • this example uses a three-dimensional CT image data set obtained from a professional medical institution.
  • the data set contains twenty-one sets of spinal CT images, and each CT image contains five vertebral blocks of the spine and lumbar vertebrae (L1 -L5), the image format is nii format, each image contains three dimensions, respectively representing the three directions in the human body.
  • An automatic planning method of surgical path based on deep learning includes the following five steps:
  • the surgical path is abstractly expressed as a straight line, and the linear surgical path is discretely expressed as the operation entry point and the operation direction point according to its geometric characteristics.
  • the main components of the network are a convolution residual module and a deconvolution module.
  • F(x) is composed of convolutional layer, batch normalization layer, ReLU layer, random inactivation layer, convolution layer, and batch normalization layer.
  • the deconvolution module is mainly composed of deconvolution layers, and the final output is a 128 ⁇ 128 ⁇ 128 spine segmentation image.
  • the segmentation image is divided into six categories, representing five spine vertebrae and background regions.
  • step four train the key point positioning network of surgery based on deep learning.
  • the main components of the network are convolutional layer, batch normalization layer, ReLU layer, pooling layer, and fully connected layer. Among them, the output of the last fully connected layer is six values, representing the X and the direction points of the entry point and the direction point. Three coordinates of Y and Z axis.
  • the initial learning rate is set to 0.01, and the linear learning rate is reduced, and the learning rate is reduced by 10 times every 200 iterations; the Adam optimizer is used for optimization, and the initial momentum is 0.95.
  • the initial learning rate is set to 0.001, and the SGD (stochastic gradient descent) optimizer is used for optimization; at the same time, the initial rate of the random inactivation layer is 0.8.
  • the network automatically outputs the needle path of the pedicle screw operation, and can output the geometric shape of the spine vertebral block.
  • the present invention can realize the automatic planning of end-to-end spinal surgery path, specifically:
  • the present invention realizes an expression mode of the surgical path, abstracting the surgical path of the spine pedicle screw as a straight line, and discretely expressing it as two key points of the surgical path.
  • the present invention uses the proposed spine shape segmentation network to achieve segmentation accuracy superior to traditional algorithms on the spine data set.
  • the present invention adopts the proposed key point regression network to realize the automatic positioning of the key points of the surgical path.
  • the present invention proposes a method for automatically planning the surgical path, which uses deep learning to realize the automatic and intelligent planning of the surgical path of the spine pedicle screw.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Surgery (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Robotics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本发明公开一种基于深度学习的脊柱椎弓根钉手术路径规划方法。该方法包含：将脊柱椎弓根钉手术路径表达为直线型的手术路径，并定义出手术入点与手术方向点；建立包括脊柱分割数据集与手术路径关键点数据集的手术路径规划数据集；采用编码器和解码器结构、Dice损失与softmax损失共同监督网络的方式，来设计分割脊柱的网络；采用卷积网络与全连接网络结合，L1损失与均方误差损失共同监督网络的方式，来设计手术路径点定位的网络；采用训练好的网络对脊柱CT图像进行自动分割并自动定位手术路径关键点，通过关键点重定位来建立手术路径，并采用主观评价与客观评价两种方式对手术路径的规划进行评价。该方法可以自动规划出脊柱椎弓根钉手术的入钉路径。

Description

一种基于深度学习网络的椎弓根钉手术路径自动规划方法
本申请要求于2019年10月11日提交中国专利局、申请号为2019109669856、发明名称为“一种基于深度学习网络的椎弓根钉手术路径自动规划方法”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及医学工程技术领域，特别是涉及一种基于深度学习网络的椎弓根钉手术路径自动规划方法。
背景技术
手术路径规划是实现智能骨科手术(特别是图像引导下的计算机辅助或者机器人辅助骨科手术)的关键核心步骤。传统的手术路径规划方法多依赖于术者(医生)经验，术者的生理状态、主观判断等因素显著影响路径规划的质量，手术规划结果的稳定性和操作效率无法得到有效保证。因此，寻求更高效、智能的手术路径规划方式，正在成为智能骨科手术的研究和应用热点。
早期的脊柱手术路径规划方法主要是由术者在软件交互界面上进行手动或半自动的交互式操作，其中的交互界面和操作方法带有明显的工程特征，对医生而言并不友好。临床上迫切需要更加自动化的、鲁棒的手术路径规划方法，来优化图像引导脊柱机器人手术的操作流程，提升人机协作效率。机器学习特别是深度学习方法的出现，为手术路径的自动规划提供了一种智能化解决手段。借助计算机强大的计算能力，深度学习能够很好地整合手术路径中多种因素的影响，建立起端到端的手术路径规划模型。
发明内容
本发明的目的是提供一种基于深度学习的脊柱手术路径规划方法，该方法可以自动地规划脊柱椎弓根钉植入手术所需的直线型路径，减轻术者的负担，确保路径规划的准确性和稳定性，促进智能骨科手术的精准化与 智能化发展。
为达到上述目的，本发明的技术方案为：
一种基于深度学习网络的椎弓根钉手术路径自动规划方法，包括以下步骤：
步骤一：建立脊柱椎弓根钉手术路径的抽象表达，将手术路径表达为两个离散点：手术入点与手术方向点；
步骤二：建立椎弓根钉手术路径数据集；数据集采用三维CT图像作为原始图像，并标注出脊柱分割图像以及手术路径关键点位置；
步骤三：建立椎块形状分割网络；在三维CT图像上分割出每个脊柱椎块的形状和位置；
步骤四：建立手术关键点定位网络；在脊柱椎块上定位出手术路径关键点的位置；
步骤五：根据预训练的网络模型，自动分割出三维CT图上的脊柱椎块，并自动定位出手术路径关键点的位置；根据手术路径关键点规划出手术路径；同时，采用定量和定性的方法评估路径规划的合理性。
可选的，所述步骤一中，手术路径上的关键点为术者标定的入点和方向点，其中，手术入点位于椎弓根横突凹陷处，用于定位椎弓根钉进入位置；手术方向点位于椎板最狭窄切面的形心处，用于约束椎弓根钉方向。
可选的，所述步骤二中，标注脊柱分割图采用“半自动标注+手动精细调整”的方式，首先用Amira或其它医学图像处理软件半自动标注出椎块轮廓，然后手动进行精细化分割；标注手术路径关键点采用全手动方式进行标注。
可选的，所述步骤三中，网络分为两部分：编码器和解码器，其中，编码器负责提取三维CT图像的特征，解码器负责生成脊柱分割的图像；在编码器和解码器之间，设计跨层连接进行信息互通；不同于传统的3D-Unet，本网络采用六层“下采样+跨层连接”的方式，来代替3D-Unet中的四层下采样；同时，采用残差卷积结构来代替传统卷积，使网络更深，能够更好地提取图像信息；在损失函数方面，采用Dice损失函数与softmax 损失函数相结合的方式训练网络，从图像级和像素级两个层次对网络的收敛程度进行约束。
可选的，所述步骤四中，回归网络采用“卷积层+全连接层”的方式构建；网络分为两部分：特征提取网络和关键点定位网络；其中，特征提取网络负责提取三维CT图像的特征信息，而手术关键点定位网络的输入为特征提取网络提取的特征，输出为六个值，分别代表入点和方向点的X、Y、Z轴的三个坐标。
可选的，所述步骤五中，脊柱椎块与手术关键点的位置可以根据预训练好的网络模型计算出，在得出脊柱手术路径关键点后，通过关键点之间的连线与分割椎块表面的交点作为椎弓根钉的入点，并在入点的基础上按照关键点连线的方向做出路径规划。
可选的，所述步骤五中，在手术路径规划完成后，可以根据IOU(Intersection over Union)指标评估脊柱分割的好坏，同时可以根据均方误差损失评估手术路径关键点定位的好坏；从定性评价的角度，可以根据术者的主观评价来判断手术路径规划的好坏。
一种基于深度学习网络的椎弓根钉手术路径自动规划方法，包括:
获取脊柱椎弓根钉手术的特征；
根据所述特征确定手术路径；
根据所述手术路径的几何特征确定所述手术路径的手术入点和手术方向点；
获取脊柱椎弓根钉手术路径的数据集；所述数据集采用三维CT图像作为原始图像，并标注出脊柱分割图像以及手术路径关键点位置；
根据所述手术路径的手术入点和手术方向点以及所述数据集建立手术路径自动规划网络；所述手术路径自动规划网络包括椎块形状分割网络和手术关键点定位网络；所述椎块形状分割网络以所述数据集为输入，以在三维CT图像上分割出每个脊柱椎块的形状和位置为输出；所述手术关键点定位网络以所述手术入点和手术方向点以及所述数据集为输入，以手术路径关键点的位置为输出；
获取待确定的脊柱三维CT图像；
根据所述脊柱三维CT图像，利用训练好的手术路径自动规划网络确定脊柱椎弓根钉手术的进针路径以及脊柱椎块的几何形状。
根据本发明提供的具体实施例，本发明公开了以下技术效果：
本发明所提供的一种基于深度学习的脊柱手术路径规划方法，将脊柱椎弓根钉手术路径抽象为一条直线，并离散表达为两个具体的手术路径关键点。构建脊柱的形状分割网络，在脊柱数据集上达到了优于传统算法的分割精度。构建关键点(路径点)定位网络，实现了基于“神经网络初步规划+路径点重定位计算”的手术路径自动规划。本发明利用深度学习实现了脊柱椎弓根钉手术路径的自动化、智能化规划。
说明书附图
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本发明所提供的一种基于深度学习的脊柱手术路径规划方法的整体流程示意图；
图2为本发明所提供的形状分割网络的结构示意图；
图3为本发明所提供的关键点定位网络的结构示意图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
本发明的目的是提供一种基于深度学习的脊柱手术路径规划方法，该方法可以自动地规划脊柱椎弓根钉植入手术所需的直线型路径，减轻术者 的负担，确保路径规划的准确性和稳定性，促进智能骨科手术的精准化与智能化发展。
为使本发明的上述目的、特征和优点能够更加明显易懂，下面结合附图和具体实施方式对本发明作进一步详细的说明。
图1为本发明所提供的一种基于深度学习的脊柱手术路径规划方法的整体流程示意图，如图1所示一种基于深度学习网络的椎弓根钉手术路径自动规划方法，包括以下步骤：
步骤一：建立脊柱椎弓根钉手术路径的抽象表达，将手术路径表达为两个离散点：手术入点与手术方向点。
所述步骤一中，手术路径上的关键点为术者标定的入点和方向点，其中，手术入点位于椎弓根横突凹陷处，用于定位椎弓根钉进入位置；手术方向点位于椎板最狭窄切面的形心处，用于约束椎弓根钉方向。
步骤二：建立椎弓根钉手术路径数据集；数据集采用三维CT图像作为原始图像，并标注出脊柱分割图像以及手术路径关键点位置。
所述步骤二中，标注脊柱分割图采用“半自动标注+手动精细调整”的方式，首先用Amira或其它医学图像处理软件半自动标注出椎块轮廓，然后手动进行精细化分割；标注手术路径关键点采用全手动方式进行标注。
步骤三：建立椎块形状分割网络；在三维CT图像上分割出每个脊柱椎块的形状和位置。
所述步骤三中，网络分为两部分：编码器和解码器，其中，编码器负责提取三维CT图像的特征，解码器负责生成脊柱分割的图像；在编码器和解码器之间，设计跨层连接进行信息互通；不同于传统的3D-Unet，网络采用六层“下采样+跨层连接”的方式，代替3D-Unet中的四层下采样；同时，采用残差卷积结构代替传统卷积，使网络更深，能够更好地提取图像信息；在损失函数方面，采用Dice损失函数与softmax损失函数相结合的方式训练网络，从图像级和像素级两个层次对网络的收敛程度进行约束。
步骤四：建立手术关键点定位网络；在脊柱椎块上定位出手术路径关 键点的位置。
所述步骤四中，回归网络采用“卷积层+全连接层”的方式构建；网络分为两部分，特征提取网络与关键点定位网络；其中，特征提取网络负责提取三维CT图像的特征信息，而手术关键点定位网络的输入为特征提取网络提取的特征，输出为六个值，分别代表入点和方向点的X、Y、Z轴的三个坐标。
步骤五：根据预训练的网络模型，自动分割出三维CT图上的脊柱椎块，并自动定位出手术路径关键点的位置；根据手术路径关键点规划出手术路径；即根据手术路径关键点和分割出的椎块形状，自动规划出手术路径。同时，采用定量和定性的方法评估路径规划的合理性。
所述步骤五中，脊柱椎块与手术关键点的位置可以根据预训练好的网络模型计算出，在得出脊柱手术路径关键点后，通过关键点之间的连线与分割椎块表面的交点作为椎弓根钉的入点，并在入点的基础上按照关键点连线的方向做出路径规划。
所述步骤五中，在手术路径规划完成后，可以根据IOU指标评估脊柱分割的好坏，同时可以根据均方误差损失评估手术路径关键点定位的好坏；在定性评价角度，可以根据术者的主观评价来判断手术路径规划的好坏。
本发明还提供一种基于深度学习网络的椎弓根钉手术路径自动规划方法，包括:
S1，获取脊柱椎弓根钉手术的特征。
S2，根据所述特征确定手术路径。
S3，根据所述手术路径的几何特征确定所述手术路径的手术入点和手术方向点。
S4，获取脊柱椎弓根钉手术路径的数据集；所述数据集采用三维CT图像作为原始图像，并标注出脊柱分割图像以及手术路径关键点位置。
S5，根据所述手术路径的手术入点和手术方向点以及所述数据集建立手术路径自动规划网络；所述手术路径自动规划网络包括椎块形状分割 网络和手术关键点定位网络；所述椎块形状分割网络以所述数据集为输入，以在三维CT图像上分割出每个脊柱椎块的形状和位置为输出；所述手术关键点定位网络以所述手术入点和手术方向点以及所述数据集为输入，以手术路径关键点的位置为输出。
S6，获取待确定的脊柱三维CT图像。
S7，根据所述脊柱三维CT图像，利用训练好的手术路径自动规划网络确定脊柱椎弓根钉手术的进针路径以及脊柱椎块的几何形状。
作为一个具体的实施例，本实例采用从专业医疗机构获取的三维CT图像数据集，该数据集包含二十一组脊柱CT图像，每个CT图像中均包含脊柱腰椎的五个椎块(L1-L5)，图像格式为nii格式，每个图像包含三个维度，分别表示人体内的三个方向。
一种基于深度学习的手术路径自动规划方式包含以下五个步骤：
一、根据脊柱椎弓根钉手术的特点将手术路径抽象表达为一条直线，并根据其几何特征将直线型手术路径离散表达为手术入点与手术方向点。
二、采用Amira或其它医学图像处理软件对脊柱CT图像进行标注。针对脊柱分割的问题，采用“半自动标注+手动精细调整”方式，获取脊柱CT图像上的五个椎块的分割标签；针对手术路径关键点定位问题，采用在软件上手动标注关键点坐标的方式，获取手术路径关键点标签。脊柱CT分割图像采用nii格式存储，而关键点坐标采用txt文件保存。
三、训练基于深度学习的椎块形状分割网络。在将图像输入进网络之前，对图像进行数据增强操作，包括增加高斯噪声、随机旋转、水平翻转和垂直翻转等，得到多张128×128×128的图像作为网络的输入。网络的主要构成为卷积残差模块和反卷积模块，卷积残差模块形成残差函数y＝F(x)+x。其中：F(x)由卷积层、批标准化层、ReLU层、随机失活层、卷积层、批标准化层组成。反卷积模块主要由反卷积层组成，最后输出为一张128×128×128大小的脊柱分割图像，分割图像分为六类，分别代表五个脊柱椎块以及背景区域。
四、训练基于深度学习的手术关键点定位网络。脊柱的三维CT图像进入网络前，同样需要进行与步骤四一样的数据增强操作。网络的主要构成为卷积层、批标准化层、ReLU层、池化层以及全连接层等，其中：最 后一层全连接层的输出为六个值，分别代表入点和方向点的X、Y、Z轴的三个坐标。
所述步骤三中，初始学习率设为0.01，并采用线性学习率下降的方式，每200次迭代学习率减小10倍；采用Adam优化器进行优化，初始动量为0.95。
所述步骤四中，初始学习率设为0.001，采用SGD(随机梯度下降)优化器进行优化；同时，随机失活层的初始速率为0.8。
五、根据步骤三和步骤四训练出来的神经网络，针对新输入的脊柱三维CT图像，网络自动输出椎弓根钉手术的进针路径，并能够输出脊柱椎块的几何形状。
综上所述，本发明能够实现端到端的脊柱手术路径自动规划，具体来说：
(1)本发明实现了一种手术路径的表达方式，将脊柱椎弓根钉手术路径抽象为一条直线，并离散表达为两个手术路径关键点。
(2)本发明采用所提的脊柱形状分割网络，在脊柱数据集上达到了优于传统算法的分割精度。
(3)本发明采用所提的关键点回归网络，实现了手术路径关键点的自动定位。
(4)本发明提出了一种自动规划手术路径的方法，利用深度学习实现了脊柱椎弓根钉手术路径的自动化、智能化规划。
本说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似部分互相参见即可。
本文中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想；同时，对于本领域的一般技术人员，依据本发明的思想，在具体实施方式及应用范围上均会有改变之处。综上所述，本说明书内容不应理解为对本发明的限制

Claims (8)

  1. 一种基于深度学习网络的椎弓根钉手术路径自动规划方法，其特征在于，包括以下步骤：
    步骤一：建立脊柱椎弓根钉手术路径的抽象表达，将手术路径表达为两个离散点：手术入点与手术方向点；
    步骤二：建立椎弓根钉手术路径数据集；数据集采用三维CT图像作为原始图像，并标注出脊柱分割图像以及手术路径关键点位置；
    步骤三：建立椎块形状分割网络；在三维CT图像上分割出每个脊柱椎块的形状和位置；
    步骤四：建立手术关键点定位网络；在脊柱椎块上定位出手术路径关键点的位置；
    步骤五：根据预训练的网络模型，自动分割出三维CT图上的脊柱椎块，并自动定位出手术路径关键点的位置；根据手术路径关键点规划出手术路径；同时，采用定量和定性的方法评估路径规划的合理性。
  2. 根据权利要求1所述的基于深度学习网络的椎弓根钉手术路径自动规划方法，其特征在于，所述步骤一中，手术路径上的关键点为术者标定的入点和方向点，其中，手术入点位于椎弓根横突凹陷处，用于定位椎弓根钉进入位置；手术方向点位于椎板最狭窄切面的形心处，用于约束椎弓根钉方向。
  3. 根据权利要求1所述的基于深度学习网络的椎弓根钉手术路径自动规划方法，其特征在于，所述步骤二中，标注脊柱分割图采用“半自动标注+手动精细调整”的方式，首先用Amira或其它医学图像处理软件半自动标注出椎块轮廓，然后手动进行精细化分割；标注手术路径关键点采用全手动方式进行标注。
  4. 根据权利要求1所述的基于深度学习网络的椎弓根钉手术路径自动规划方法，其特征在于，所述步骤三中，网络分为两部分：编码器和解码器，其中，编码器负责提取三维CT图像的特征，解码器负责生成脊柱分割的图像；在编码器和解码器之间，设计跨层连接进行信息互通；不同于传统的3D-Unet，网络采用六层“下采样+跨层连接”的方式，代替3D-Unet中的四层下采样；同时，采用残差卷积结构代替传统卷积，使网络更深， 能够更好地提取图像信息；在损失函数方面，采用Dice损失函数与softmax损失函数相结合的方式训练网络，从图像级和像素级两个层次对网络的收敛程度进行约束。
  5. 根据权利要求1所述的基于深度学习网络的椎弓根钉手术路径自动规划方法，其特征在于，所述步骤四中，回归网络采用“卷积层+全连接层”的方式构建；网络分为两部分，特征提取网络与关键点定位网络；其中，特征提取网络负责提取三维CT图像的特征信息，而手术关键点定位网络的输入为特征提取网络提取的特征，输出为六个值，分别代表入点和方向点的X、Y、Z轴的三个坐标。
  6. 根据权利要求1所述的基于深度学习网络的椎弓根钉手术路径自动规划方法，其特征在于，所述步骤五中，脊柱椎块与手术关键点的位置可以根据预训练好的网络模型计算出，在得出脊柱手术路径关键点后，通过关键点之间的连线与分割椎块表面的交点作为椎弓根钉的入点，并在入点的基础上按照关键点连线的方向做出路径规划。
  7. 根据权利要求1所述的基于深度学习网络的椎弓根钉手术路径自动规划方法，其特征在于，所述步骤五中，在手术路径规划完成后，可以根据IOU指标评估脊柱分割的好坏，同时可以根据均方误差损失评估手术路径关键点定位的好坏；在定性评价角度，可以根据术者的主观评价来判断手术路径规划的好坏。
  8. 一种基于深度学习网络的椎弓根钉手术路径自动规划方法，其特征在于，包括:
    获取脊柱椎弓根钉手术的特征；
    根据所述特征确定手术路径；
    根据所述手术路径的几何特征确定所述手术路径的手术入点和手术方向点；
    获取脊柱椎弓根钉手术路径的数据集；所述数据集采用三维CT图像作为原始图像，并标注出脊柱分割图像以及手术路径关键点位置；
    根据所述手术路径的手术入点和手术方向点以及所述数据集建立手术路径自动规划网络；所述手术路径自动规划网络包括椎块形状分割网络 和手术关键点定位网络；所述椎块形状分割网络以所述数据集为输入，以在三维CT图像上分割出每个脊柱椎块的形状和位置为输出；所述手术关键点定位网络以所述手术入点和手术方向点以及所述数据集为输入，以手术路径关键点的位置为输出；
    获取待确定的脊柱三维CT图像；
    根据所述脊柱三维CT图像，利用训练好的手术路径自动规划网络，采用“神经网络初步规划+路径点重定位计算”的方式，确定脊柱椎弓根钉手术的进针路径。
PCT/CN2020/120168 2019-10-11 2020-10-10 一种基于深度学习网络的椎弓根钉手术路径自动规划方法 WO2021068933A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910966985.6A CN110738681A (zh) 2019-10-11 2019-10-11 一种基于深度学习网络的椎弓根钉手术路径自动规划方法
CN201910966985.6 2019-10-11

Publications (1)

Publication Number Publication Date
WO2021068933A1 true WO2021068933A1 (zh) 2021-04-15

Family

ID=69268771

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/120168 WO2021068933A1 (zh) 2019-10-11 2020-10-10 一种基于深度学习网络的椎弓根钉手术路径自动规划方法

Country Status (2)

Country Link
CN (1) CN110738681A (zh)
WO (1) WO2021068933A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3919017A1 (en) * 2020-06-03 2021-12-08 Globus Medical, Inc. Machine learning system for navigated spinal surgeries
CN114159142A (zh) * 2021-09-30 2022-03-11 国家康复辅具研究中心 基于人工智能深度学习的胫骨外固定器安装辅助导板设计方法
CN114944216A (zh) * 2022-05-22 2022-08-26 北京航空航天大学 一种融合解剖及力学特性的脊柱手术直线型路径自动规划方法
CN116958128A (zh) * 2023-09-18 2023-10-27 中南大学 基于深度学习的医学图像自动定位方法

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110738681A (zh) * 2019-10-11 2020-01-31 北京航空航天大学 一种基于深度学习网络的椎弓根钉手术路径自动规划方法
CN111598948B (zh) * 2020-04-03 2024-02-20 上海嘉奥信息科技发展有限公司 基于深度学习的ct影像椎弓根植钉通道规划方法和系统
CN112155729B (zh) * 2020-10-15 2021-11-23 中国科学院合肥物质科学研究院 手术穿刺路径智能自动化规划方法及系统和医疗系统
CN112914696B (zh) * 2021-03-22 2022-10-18 深圳市第二人民医院(深圳市转化医学研究院) 一种基于物联网的颅脑术后引流方法、系统及可读存储介质
CN113240661B (zh) * 2021-05-31 2023-09-26 平安科技（深圳）有限公司 基于深度学习的腰椎骨分析方法、装置、设备及存储介质
CN113781496B (zh) * 2021-08-06 2024-02-27 北京天智航医疗科技股份有限公司 基于cbct脊椎图像的椎弓根螺钉通道自动规划系统和方法
CN114372970B (zh) * 2022-01-04 2024-02-06 杭州三坛医疗科技有限公司 一种手术参考信息生成方法及装置
CN114288018A (zh) * 2022-01-04 2022-04-08 青岛大学附属医院 一种机器人辅助镜下融合技术方法
CN114913124B (zh) * 2022-04-13 2023-04-07 中南大学湘雅医院 一种用于肿瘤手术的切缘路径生成方法、系统及存储介质
CN115331082B (zh) * 2022-10-13 2023-02-03 天津大学 追踪声源的路径生成方法、模型的训练方法及电子设备
CN116211458B (zh) * 2022-12-12 2023-10-03 高峰医疗器械(无锡)有限公司 种植体规划方法、装置、设备及存储介质
CN116473673B (zh) * 2023-06-20 2024-02-27 浙江华诺康科技有限公司 内窥镜的路径规划方法、装置、系统和存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106691600A (zh) * 2016-11-21 2017-05-24 胡磊 一种脊柱椎弓根钉植入定位装置
CN107157579A (zh) * 2017-06-26 2017-09-15 苏州铸正机器人有限公司 一种脊柱椎弓根螺钉植入路径规划方法
US20190133791A1 (en) * 2017-11-07 2019-05-09 Howmedica Osteonics Corp. Robotic System For Shoulder Arthroplasty Using Stemless Implant Components
CN110738681A (zh) * 2019-10-11 2020-01-31 北京航空航天大学 一种基于深度学习网络的椎弓根钉手术路径自动规划方法

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101625766A (zh) * 2009-08-03 2010-01-13 深圳先进技术研究院 医学图像处理方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106691600A (zh) * 2016-11-21 2017-05-24 胡磊 一种脊柱椎弓根钉植入定位装置
CN107157579A (zh) * 2017-06-26 2017-09-15 苏州铸正机器人有限公司 一种脊柱椎弓根螺钉植入路径规划方法
US20190133791A1 (en) * 2017-11-07 2019-05-09 Howmedica Osteonics Corp. Robotic System For Shoulder Arthroplasty Using Stemless Implant Components
CN110738681A (zh) * 2019-10-11 2020-01-31 北京航空航天大学 一种基于深度学习网络的椎弓根钉手术路径自动规划方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CAI DONGYANG; WANG ZAIYUE; LIU YAJUN; ZHANG QI; HAN XIAOGUANG; LIU WENYONG: "Automatic Path Planning for Navigated Pedicle Screw Surgery Based on Deep Neural Network", 2019 WRC SYMPOSIUM ON ADVANCED ROBOTICS AND AUTOMATION (WRC SARA), IEEE, 21 August 2019 (2019-08-21), pages 62 - 67, XP033673501, DOI: 10.1109/WRC-SARA.2019.8931805 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3919017A1 (en) * 2020-06-03 2021-12-08 Globus Medical, Inc. Machine learning system for navigated spinal surgeries
CN114159142A (zh) * 2021-09-30 2022-03-11 国家康复辅具研究中心 基于人工智能深度学习的胫骨外固定器安装辅助导板设计方法
CN114944216A (zh) * 2022-05-22 2022-08-26 北京航空航天大学 一种融合解剖及力学特性的脊柱手术直线型路径自动规划方法
CN116958128A (zh) * 2023-09-18 2023-10-27 中南大学 基于深度学习的医学图像自动定位方法
CN116958128B (zh) * 2023-09-18 2023-12-26 中南大学 基于深度学习的医学图像自动定位方法

Also Published As

Publication number Publication date
CN110738681A (zh) 2020-01-31

Similar Documents

Publication Publication Date Title
WO2021068933A1 (zh) 一种基于深度学习网络的椎弓根钉手术路径自动规划方法
Reitinger et al. Liver surgery planning using virtual reality
JP7489732B2 (ja) 深層学習に基づく脊椎ｍｒｉ映像キーポイント検出方法
CN109063710A (zh) 基于多尺度特征金字塔的3d cnn鼻咽癌分割方法
CN111768382B (zh) 一种基于肺结节生长形态的交互式分割方法
CN107665497A (zh) 一种医学图像中计算心胸比的方法
JPWO2020056086A5 (zh)
CN108309334B (zh) 一种脊柱x线影像的数据处理方法
CN108109170B (zh) 医学图像扫描方法及医学影像设备
CN110766694B (zh) 一种三维医学图像的交互式分割方法
CN111767952B (zh) 一种可解释的肺结节良恶性分类方法
CN113643790A (zh) 一种脊椎的置换建模方法及系统
CN112349392A (zh) 一种人体颈椎医学图像处理系统
CN113724206A (zh) 一种基于自监督学习的眼底图像血管分割方法及系统
CN115222937A (zh) 一种脊柱侧弯检测方法及装置
CN110136113A (zh) 一种基于卷积神经网络的阴道病理图像分类方法
CN115689971A (zh) 基于深度学习的椎弓根植钉通道规划方法及装置
CN108846342A (zh) 一种唇裂手术标志点识别系统
CN112562070A (zh) 基于模板匹配的颅缝早闭手术切割坐标生成系统
CN112008982A (zh) 模型打印装置
CN116747017A (zh) 脑出血手术规划系统及方法
CN114693981A (zh) 一种膝关节特征点自动识别方法
Ye et al. Puzzlefixer: A visual reassembly system for immersive fragments restoration
CN113975662A (zh) 一种基于数据挖掘技术的鼻咽癌精准放疗平台
CN113269816A (zh) 一种分区域递进式脑图像弹性配准方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20873473

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20873473

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 20873473

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 21.11.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 20873473

Country of ref document: EP

Kind code of ref document: A1