CN116196099A - Cardiovascular intervention operation path planning method, system, storage medium and terminal - Google Patents
Cardiovascular intervention operation path planning method, system, storage medium and terminal Download PDFInfo
- Publication number
- CN116196099A CN116196099A CN202310154582.8A CN202310154582A CN116196099A CN 116196099 A CN116196099 A CN 116196099A CN 202310154582 A CN202310154582 A CN 202310154582A CN 116196099 A CN116196099 A CN 116196099A
- Authority
- CN
- China
- Prior art keywords
- path
- blood vessel
- image
- algorithm
- planning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 230000002526 effect on cardiovascular system Effects 0.000 title claims abstract description 38
- 238000003860 storage Methods 0.000 title claims abstract description 14
- 210000004204 blood vessel Anatomy 0.000 claims abstract description 61
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 50
- 238000001914 filtration Methods 0.000 claims abstract description 30
- 238000013136 deep learning model Methods 0.000 claims abstract description 17
- 238000007781 pre-processing Methods 0.000 claims abstract description 15
- 230000011218 segmentation Effects 0.000 claims abstract description 11
- 238000005452 bending Methods 0.000 claims abstract description 10
- 230000002146 bilateral effect Effects 0.000 claims abstract description 9
- 238000002583 angiography Methods 0.000 claims description 19
- 230000015654 memory Effects 0.000 claims description 17
- 238000013152 interventional procedure Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 10
- 238000009472 formulation Methods 0.000 claims description 6
- 230000002068 genetic effect Effects 0.000 claims description 6
- 239000000203 mixture Substances 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 6
- 238000010845 search algorithm Methods 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000003745 diagnosis Methods 0.000 abstract description 6
- 238000012545 processing Methods 0.000 description 7
- 238000012986 modification Methods 0.000 description 6
- 230000004048 modification Effects 0.000 description 6
- 230000002792 vascular Effects 0.000 description 5
- 238000000605 extraction Methods 0.000 description 4
- 230000003902 lesion Effects 0.000 description 3
- 210000000056 organ Anatomy 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 101100295091 Arabidopsis thaliana NUDT14 gene Proteins 0.000 description 1
- 235000002566 Capsicum Nutrition 0.000 description 1
- 208000024172 Cardiovascular disease Diseases 0.000 description 1
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 1
- 239000006002 Pepper Substances 0.000 description 1
- 235000016761 Piper aduncum Nutrition 0.000 description 1
- 235000017804 Piper guineense Nutrition 0.000 description 1
- 244000203593 Piper nigrum Species 0.000 description 1
- 235000008184 Piper nigrum Nutrition 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000005284 excitation Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000011946 reduction process Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 150000003839 salts Chemical class 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000001356 surgical procedure Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 231100000216 vascular lesion Toxicity 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/30—Surgical robots
- A61B34/32—Surgical robots operating autonomously
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A61B2034/101—Computer-aided simulation of surgical operations
- A61B2034/105—Modelling of the patient, e.g. for ligaments or bones
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A61B2034/107—Visualisation of planned trajectories or target regions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20028—Bilateral filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30101—Blood vessel; Artery; Vein; Vascular
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- Multimedia (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Robotics (AREA)
- Animal Behavior & Ethology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Molecular Biology (AREA)
- Databases & Information Systems (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Apparatus For Radiation Diagnosis (AREA)
Abstract
The invention discloses a cardiovascular interventional operation path planning method, a cardiovascular interventional operation path planning system, a storage medium and a terminal. The method comprises the steps of preprocessing an image by sequentially utilizing median filtering, bilateral filtering and Gaussian filtering; and carrying out blood vessel semantic segmentation by using a deep learning model, identifying focus points by using a YOLO V4 model, judging and scoring the blood vessel bending and the blood vessel width by using a knowledge graph, and finally carrying out path planning by using a path planning algorithm. The invention can realize the task autonomous operation from diagnosis and identification to planning operation and then to operation implementation under the supervision of doctors, greatly simplifies the whole workflow of the interventional operation from operation planning to operation execution, and can output the operation path map at the current moment in real time.
Description
Technical Field
The invention relates to the technical field of intelligent medical assistance, in particular to a cardiovascular interventional operation path planning method, a cardiovascular interventional operation path planning system, a cardiovascular interventional operation path planning storage medium and a cardiovascular interventional operation terminal.
Background
The minimally invasive interventional operation mode can achieve a good treatment effect on most cardiovascular diseases, and due to the fact that the cardiovascular structure is complex, reasonable preoperative planning can greatly improve the success rate of operation and the operation efficiency, and reduce the existing risks.
The vascular path planning technology can help doctors to carry out operation planning by generating a path from an operation entrance to a lesion, and provides important reference basis for the doctors to carry out interventional operation.
Along with the popularization of the application of cardiovascular interventional operation robots, the path planning of the vascular interventional operation is also an important research direction of operation automation, and the operation robots can automatically execute the operation along the planned path given by the system.
Meanwhile, in the complicated and complicated vascular topological structure, the factors such as the size of a surgical instrument, the length of a vascular path, the bending degree and the like are comprehensively considered, and an optimal surgical path from a surgical entrance to a patient is necessarily existed. Therefore, in order to shorten the operation time, reduce the operation risk and improve the operation effect, a globally optimal or suboptimal operation path is searched in the three-dimensional geometrical space of the blood vessel, and the method has important positive significance for cardiovascular interventional operation.
The Chinese patent application No. 202210639634.6 discloses a surgical path planning method, system, device, medium and surgical operating system, the method comprising: constructing a digital twin human model based on the associated parameters of the target object; acquiring current contour information of a target object to construct a human body three-dimensional model; registering the digital twin human body model and the human body three-dimensional model to obtain a registration result; determining an initial surgical path plan for a target organ in the digital twin phantom; and converting the initial operation path planning into a target operation path planning corresponding to the target organ in the human body three-dimensional model. The method can plan the operation path by utilizing a digital twin technology, but lacks screening of the optimal path, has a defect in path planning based on operation efficiency, is suitable for an operation object with a large size such as an organ, and has low applicability to the operation object with a blood vessel scale.
Disclosure of Invention
Due to the defects in the prior art, the invention provides a cardiovascular intervention operation path planning method, a cardiovascular intervention operation path planning system, a storage medium and a cardiovascular intervention operation path planning terminal, which can be combined with a cardiovascular intervention operation robot to realize task autonomous operation from diagnosis and identification to planning operation to operation implementation under the supervision of doctors, so that the whole workflow of intervention operation from operation planning to operation execution is greatly simplified, and an operation path map at the current moment can be output in real time.
To achieve the above object, in one aspect, the present invention provides a cardiovascular interventional operation path planning method, which is characterized by comprising the following steps:
s1, acquiring angiographic images;
step S2, preprocessing angiographic images, comprising the following substeps:
s2.1, performing median filtering on the acquired angiography image;
step S2.2, bilateral filtering is carried out on the image processed in the step S2.1;
step S2.3, performing Gaussian filtering on the image processed in the step S2.2;
s3, segmenting the image processed in the step S2 by using a deep learning model, identifying and marking the blood vessel outline, and generating a blood vessel outline image;
s4, identifying focus points of the original angiography image by utilizing a YOLO V4 model, and marking;
s5, combining the blood vessel contour image, and judging and scoring the characteristics of blood vessel bending, blood vessel width and the like of the original angiography image by utilizing a knowledge graph;
step S6, a doctor manually selects a starting point of an operation path on the blood vessel contour image, the focus point identified in the step S4 is used as an end point of the operation path, and a depth-first search algorithm is used for judging whether a path which can be conducted by an operation exists or not; if no path capable of being conducted exists, a doctor is required to participate in planning of the operation and formulation and adjustment of a specific operation scheme; if a path capable of being conducted exists, obtaining a globally optimal path for operation by using a path planning algorithm according to the judgment result of the step S5;
steps S2 to S3 and S4 may be performed synchronously.
Preferably, in the step S3, the deep learning model is a resunet++ model, a ResUNet model, or a UNet model.
Preferably, in the step S4, the structure of the YOLO V4 model is divided into three parts: a backbone network, a neck network, and a header structure; the main network is a CSPDarknet53 network structure, and the CSPDarknet53 network comprises five residual modules with stacking times of 1, 2, 8 and 4 respectively; the neck network adopts an SPP module and a PANet network; the SPP module adopts the maximum pooling of 13×13, 9×9, 5×5 and 1×1 different pooling core sizes to process the final output characteristic layer of the CSPDarknet53 network; the head structure adopts three-scale output for detecting large, medium and small targets respectively for heads with the sizes of 1/8, 1/16 and 1/32 of the original input size.
Preferably, in the step S6, the path planning algorithm is Dijstra algorithm, astar algorithm or genetic algorithm.
Preferably, in the step S6, the path planning algorithm is a Dijstra algorithm, and the Dijstra algorithm includes the following steps:
s6.1, setting a weighted adjacent matrix AS to represent a weighted directed network, wherein AS [ i, j ] represents a weight value on an arc < vi, vj >, and arcs [ j, k ] represents the length of an edge between a vertex k and a vertex j;
if < vi, vj > does not exist, then set AS [ i, j ] to ≡; s is the set of end points of the shortest path from v, and the initial state of the set is an empty set; then, starting from v to the remaining vertices or endpoints vi on the graph, the initial value of the shortest path length that may be reached is:
DIST[i]=AS[v0,i]
step S6.2, selecting vj such that
DIST[j]=Min{DIST[i]|vi∈V-S}
vj is the end point of the shortest path from v obtained at present; order the
S=S∪{j}
S6.3, modifying the shortest path length from V to any vertex vk on the set V-S; if it is
D[j]+arcs[j,k]<D[k]
Then modify Dk to
D[k]=D[j]+arcs[j,k]
Step S6.4, repeating the steps S6.2 and S6.3 for n-1 times, and obtaining the shortest path from v to the rest vertexes on the graph as a sequence increasing according to the path length.
In another aspect, the present invention provides a cardiovascular interventional procedure path planning system, comprising:
the image preprocessing module is used for preprocessing the acquired angiography image by sequentially utilizing median filtering, bilateral filtering and Gaussian filtering methods;
the blood vessel contour segmentation module is used for segmenting the image processed by the image preprocessing module by using a deep learning model, identifying and marking the blood vessel contour, and generating a blood vessel contour image;
the focal point identification module is used for identifying and marking focal points of the original angiography image by utilizing the YOLO V4 model;
the blood vessel feature appraising module is used for appraising the features of blood vessel bending, blood vessel width and the like of the original angiography image by combining the blood vessel contour image and utilizing a knowledge graph;
the path planning module is used for judging whether a path which can be conducted by the operation exists or not; if no path capable of being conducted exists, a doctor is required to participate in planning of the operation and formulation and adjustment of a specific operation scheme; and if a path capable of being conducted exists, obtaining a globally optimal path for operation by using a path planning algorithm according to the judgment result of the blood vessel characteristic judgment module.
Preferably, in the vessel contour segmentation module, the deep learning model is a resune++ model, a resune model or a UNet model.
Preferably, in the path planning module, the path planning algorithm is Dijstra algorithm, astar algorithm or genetic algorithm.
In a further aspect, the present invention provides a computer storage medium having a computer program, characterized in that the computer program, when executed by a processor, implements a cardiovascular interventional procedure path planning method as described above.
In a final aspect, the present invention provides an intelligent terminal, which is characterized by comprising: one or more memories and one or more processors;
the one or more memories are used for storing computer programs;
the one or more processors are coupled to the memory for executing the computer program to perform the cardiovascular interventional procedure path planning method as described above.
Compared with the prior art, the invention has the following advantages or beneficial effects:
(1) The invention integrates the feature information of the blood vessel contour segmentation module, the focus point identification module and the blood vessel feature judgment module, utilizes the corresponding algorithm to plan the optimal path, can be combined with a cardiovascular intervention operation robot to realize the task autonomous operation from diagnosis identification to planning operation to operation implementation under the supervision of doctors, greatly simplifies the whole working flow of intervention operation from operation planning to operation execution, and can output the operation path map at the current moment in real time;
(2) The invention adopts the Resunet++ model with excellent performance in the medical image segmentation field and the mature YOLO V4 model in the target detection field, thereby ensuring the reliability and stability of the effective information extraction of the image;
(3) According to the invention, the Dijstra algorithm is used in the path selection, so that the calculation speed of path search is improved, the forming speed of an interventional operation planning method is improved, and the interventional operation efficiency is improved.
Drawings
The invention and its features, aspects and advantages will become more apparent from the detailed description of non-limiting embodiments with reference to the following drawings.
FIG. 1 is a flow chart showing the steps of a method for planning a path for a cardiovascular interventional procedure according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a cardiovascular interventional procedure path planning system according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a path planning result of a central vascular interventional procedure according to an embodiment of the present invention.
Detailed Description
The invention will now be further described with reference to the accompanying drawings and specific examples, which are not intended to limit the invention.
The order of execution of the operations, steps, and the like in the methods shown in the claims, the specification, and the drawings may be performed in any order unless otherwise specified, as long as the output of the preceding process is not used in the following process. The use of the description of "first," "second," etc. for convenience of description does not imply that the operations must be performed in such order.
In the following detailed description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that well-known algorithms and models are not shown in detail in order to avoid obscuring the principles of the present invention.
Furthermore, the terms "comprises," "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements but may, optionally, include other steps or elements not listed or inherent to such process, method, article, or apparatus.
Example 1
Referring to fig. 1, the present embodiment provides a cardiovascular interventional operation path planning method, which includes the following steps:
and S1, acquiring angiographic images.
Step S2, preprocessing angiographic images, comprising the following substeps:
and step S2.1, performing median filtering on the acquired angiography image.
The basic principle of the median filter is: the median value of the pixel values in a neighborhood of the pixel to be processed is selected to replace the pixel to be processed. The main function of the median filter is to make the gray value of a certain pixel relatively close to the pixels in the surrounding area, so that isolated noise points are eliminated, and the median filter can well eliminate salt and pepper noise. The median filter can effectively protect the boundary information of the image while eliminating noise, and can not cause great blurring to the image.
And step S2.2, carrying out bilateral filtering on the image processed in the step S2.1.
The bilateral filtering is a nonlinear filtering method, is a compromise combining the image proximity and pixel value similarity, and can retain the edge information of the original image while filtering noise.
And step S2.3, performing Gaussian filtering on the image processed in the step S2.2.
Gaussian filtering is a linear smoothing filtering, is suitable for eliminating Gaussian noise, and is widely applied to a noise reduction process of image processing. Gaussian filtering is a process of weighted averaging over the entire image, where the value of each pixel is obtained by weighted averaging itself and other pixel values in the neighborhood.
And step S3, segmenting the image processed in the step S2 by using a deep learning model, identifying and marking the blood vessel outline, and generating a blood vessel outline image.
As a preferred technical scheme, further: in step S3, the deep learning model is a resunet++ model, a ResUNet model, or a UNet model.
As an example, step S3 uses a resunet++ deep learning model to segment the preprocessed angiography image of the blood vessel, identify the contour of the blood vessel, and mark it.
Resunet++ is an improvement over Resunet, which combines the advantages of ResNet and UNet, and continues to introduce squeeze and excitation block, ASPP and actionblock over Resunet. The ResUnet++ not only comprises the characteristic that ResNet is easy to optimize, but also can improve the accuracy by increasing a considerable depth, and the residual block uses jump connection, so that the gradient vanishing problem caused by increasing the depth in the deep neural network is relieved; meanwhile, the method has the characteristics of simplicity, high efficiency, easy understanding, easy construction, suitability for the field of semantic segmentation and capability of training from a small data set.
S4, identifying focus points of the original angiography image by utilizing a YOLO V4 model, and marking; further, the YOLO V4 deep learning model is utilized to identify and mark the corresponding vascular lesion type and lesion part of the original contrast image. The corresponding lesion information is automatically marked as the end point of the path in the system and is provided for the following optimal path planning as a reference.
YOLO V4 is a relatively mature deep learning model applied to the field of target detection, and we can use the model to realize the focus recognition and marking module.
As a preferred technical scheme, further: in step S4, the YOLO V4 model structure is divided into three parts: a backbone network, a neck network, and a header structure; the trunk network is a trunk feature extraction network, and the Yolo V4 refers to the CSPNet structure and is merged into the Darknet53, so that the CSPDarknet53 network structure is provided, and the performance of the trunk network is greatly improved. CSPDarknet53 contains five residual modules with stacking times of 1, 2, 8, 4, respectively.
The neck network is also called an enhanced feature extraction network, and YOLO V4 employs SPP modules and a PANet network as the neck network. SPP adopts the maximum pooling of 13x13, 9x9, 5x5 and 1x1 different pooling core sizes to process the final output characteristic layer of CSPDarknet53, so that the receptive field of the CSPDarknet53 is widened, the extraction capacity of the characteristic map is enhanced, and the overfitting is effectively prevented.
The head structure of the YOLO V4 adopts three-scale output for detecting targets with different sizes, and the heads with the sizes of 1/8, 1/16 and 1/32 of the original input sizes are used for detecting large, medium and small targets respectively. The depth of the head structure represents the bounding box offset, confidence, class, and prior box, and each scale output has three different sized prior boxes.
And S5, combining the blood vessel contour image, and judging the characteristics of blood vessel bending, blood vessel width and the like of the original angiography image by utilizing a knowledge graph.
As an example, in step S5, the original angiographic image will be used to score based on the knowledge graph, and the experienced doctor will score according to the difficulty, with high score and low score, mainly based on the indexes such as the width of the blood vessel and the bending degree of the blood vessel, which affect the interventional operation; the highest score of difficulty was 10 and the lowest score was 0.
The system can match the corresponding images of the blood vessels with the corresponding scores in focus diagnosis to form corresponding knowledge maps.
Step S6, a doctor manually selects a starting point of an operation path on the blood vessel contour image, the focus point identified in the step S4 is used as an end point of the operation path, and a depth-first search algorithm is used for judging whether a path which can be conducted by an operation exists or not; if no path capable of being conducted exists, a doctor is required to participate in planning of the operation and formulation and adjustment of a specific operation scheme; if a path capable of being conducted exists, obtaining a globally optimal path for operation by using a path planning algorithm according to the judgment result of the step S5.
As a preferred technical scheme, further: in step S6, the path planning algorithm is Dijstra algorithm, astar algorithm or genetic algorithm.
As an example, a DFS algorithm (depth-first search algorithm) is used to determine whether a path is on. The DFS algorithm is an algorithm for traversing or searching a tree or graph, in this embodiment a depth-first search of a scaled directed graph. Each scaled node of the graph is traversed along its depth, searching as deep as possible for each branch of the directed graph. When the edge of the node v has been searched or the node does not meet the condition during searching, the searching will trace back to the starting node of the edge where the node v is found. The entire process iterates until all nodes are accessed.
The decision of the DFS algorithm is performed based on the path in the image marked by Resunet++, and whether the starting point and the end point are conducted or not is determined. In practical use, the starting point is selected by a doctor according to the actual condition of the operation, and the end point is obtained by comprehensively judging the suspicious position detected by the YOLO V4 with the aid of the doctor. Starting from the start point, all paths which can reach the end point are walked through according to the depth priority principle by using a DFS algorithm, and marking and storing are carried out.
And screening and determining an optimal path by using a Dijstra algorithm, wherein the Dijstra algorithm is mainly used for obtaining the shortest path from one point to other points.
The specific steps of the Dijstra algorithm are as follows:
s1, a weighted adjacent matrix AS is used for representing a weighted directed network, AS [ i, j ] represents a weight value on an arc < vi, vj >, and arcs [ j, k ] represents the length of an edge between a vertex k and a vertex j.
If < vi, vj > does not exist, AS [ i, j ] is set to ≡. S is the set of endpoints for which the shortest path from v has been found, its initial state is the empty set. Then, starting from v to the remaining vertices (endpoints) vi on the graph, the initial value of the shortest path length that is possible is:
DIST[i]=AS[v0,i]
s2, selecting vj so that
DIST[j]=Min{DIST[i]|vi∈V-S}
vj is the end point of the shortest path currently found from v. Order the
S=S∪{j}
S3, modifying the shortest path length from V to any vertex vk on the set V-S. If it is
D[j]+arcs[j,k]<D[k]
Then modify Dk to
D[k]=D[j]+arcs[j,k]
S4, repeating the operations S2 and S3 for n-1 times. The shortest path from v to the remaining vertices on the graph is thus found as an increasing sequence of path lengths.
It is understood that steps S2 to S3 and step S4 may be performed simultaneously.
An example result of a specific algorithm program is shown in fig. 2. According to the cardiovascular interventional operation path planning method, the task automation operation from diagnosis identification to planning operation to operation implementation can be realized under the supervision of doctors, the interventional operation whole workflow from operation planning to operation execution is greatly simplified, and the operation path map at the current moment can be output in real time.
Example 2
Referring to fig. 3, the present embodiment provides a cardiovascular interventional procedure path planning system, including:
the image preprocessing module 100 is used for preprocessing the acquired angiography image by sequentially utilizing median filtering, bilateral filtering and Gaussian filtering methods;
the blood vessel contour segmentation module 110 is used for segmenting the image processed by the image preprocessing module by using a deep learning model, identifying and marking the blood vessel contour, and generating a blood vessel contour image;
the focal point identification module 120 performs focal point identification on the original angiography image by using the YOLO V4 model and marks the focal point;
the blood vessel feature appraising module 130 is used for appraising the features of blood vessel bending, blood vessel width and the like of the original angiography image by combining the blood vessel contour image and utilizing a knowledge graph;
a path planning module 140, configured to determine whether a path capable of conducting a surgery exists; if no path capable of being conducted exists, a doctor is required to participate in planning of the operation and formulation and adjustment of a specific operation scheme; and if a path capable of being conducted exists, obtaining a globally optimal path for operation by using a path planning algorithm according to the judgment result of the blood vessel characteristic judgment module.
As a preferred technical scheme, further: in the blood vessel contour segmentation module, the deep learning model is a Resunet++ model, a Resunet model or a UNet model.
As a preferred technical scheme, further: in the path planning module, the path planning algorithm is Dijstra algorithm, astar algorithm or genetic algorithm.
Firstly, an original X-ray contrast image is acquired in the operation process, and the image is subjected to noise reduction treatment in a preprocessing module 100; and extracting relevant characteristic information from the blood vessel contour segmentation module 110 and the focus point identification module 120 respectively. Meanwhile, the blood vessel feature judgment module 130 judges the relevant features of the blood vessel in the original image, and finally in the path planning module 140, the information is integrated, and the optimal path is marked by using the corresponding calculation rules to assist the doctor in performing cardiovascular interventional operation.
Example 3
The present embodiment provides a computer storage medium having a computer program which, when executed by a processor, implements the cardiovascular interventional procedure path planning method as described in embodiment 1.
Those of ordinary skill in the art will appreciate that all or a portion of the steps in the various methods of the above embodiments may be implemented by a program that instructs associated hardware, and that the program may be stored on a computer readable storage medium. The computer-readable storage medium may include, but is not limited to, floppy diskettes, optical disks, CD-ROMs (compact disk-read only memories), magneto-optical disks, ROMs (read-only memories), RAMs (random access memories), EPROMs (erasable programmable read only memories), EEPROMs (electrically erasable programmable read only memories), magnetic or optical cards, flash memory, or other type of media/machine-readable medium suitable for storing machine-executable instructions. The computer readable storage medium may be an article of manufacture that is not accessed by a computer device or may be a component used by an accessed computer device.
Example 4
The embodiment provides an intelligent terminal, including: one or more memories and one or more processors;
the one or more memories are used for storing computer programs;
the one or more processors are connected to the memory for running the computer program to perform the cardiovascular interventional procedure path planning method as described in embodiment 1.
Optionally, the memory may include, but is not limited to, high speed random access memory, nonvolatile memory. Such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices; the processor 42 may include, but is not limited to, a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but also digital signal processors (Digital Signal Processing, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field-programmable gate arrays (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
Alternatively, the processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, abbreviated as CPU), a network processor (Network Processor, abbreviated as NP), and the like; but also digital signal processors (Digital Signal Processing, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field-programmable gate arrays (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
In summary, the invention provides a cardiovascular interventional operation path planning method, a cardiovascular interventional operation path planning system, a storage medium and a terminal. The method comprises the steps of preprocessing an image by sequentially utilizing median filtering, bilateral filtering and Gaussian filtering; and carrying out blood vessel semantic segmentation by using a deep learning model, identifying focus points by using a YOLO V4 model, judging and scoring the blood vessel bending and the blood vessel width by using a knowledge graph, and finally carrying out path planning by using a path planning algorithm. The invention can realize the task autonomous operation from diagnosis and identification to planning operation and then to operation implementation under the supervision of doctors, greatly simplifies the whole workflow of the interventional operation from operation planning to operation execution, and can output the operation path map at the current moment in real time.
Those skilled in the art will understand that the skilled person can implement the modification in combination with the prior art and the above embodiments, and this will not be repeated here. Such modifications do not affect the essence of the present invention, and are not described herein.
The preferred embodiments of the present invention have been described above. It is to be understood that the invention is not limited to the specific embodiments described above, wherein devices and structures not described in detail are to be understood as being implemented in a manner common in the art; any person skilled in the art can make many possible variations and modifications to the technical solution of the present invention or modifications to equivalent embodiments without departing from the scope of the technical solution of the present invention, using the methods and technical contents disclosed above, without affecting the essential content of the present invention. Therefore, any simple modification, equivalent variation and modification of the above embodiments according to the technical substance of the present invention still fall within the scope of the technical solution of the present invention.
Those of ordinary skill in the art will appreciate that the elements of the various examples described in connection with the present embodiments, i.e., the algorithm steps, can be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
Claims (10)
1. A method for planning a path of a cardiovascular interventional procedure, comprising the steps of:
s1, acquiring angiographic images;
step S2, preprocessing angiographic images, comprising the following substeps:
s2.1, performing median filtering on the acquired angiography image;
step S2.2, bilateral filtering is carried out on the image processed in the step S2.1;
step S2.3, performing Gaussian filtering on the image processed in the step S2.2;
s3, segmenting the image processed in the step S2 by using a deep learning model, identifying and marking the blood vessel outline, and generating a blood vessel outline image;
s4, identifying focus points of the original angiography image by utilizing a YOLO V4 model, and marking;
s5, combining the blood vessel contour image, and judging and scoring the characteristics of blood vessel bending, blood vessel width and the like of the original angiography image by utilizing a knowledge graph;
step S6, a doctor manually selects a starting point of an operation path on the blood vessel contour image, the focus point identified in the step S4 is used as an end point of the operation path, and a depth-first search algorithm is used for judging whether a path which can be conducted by an operation exists or not; if no path capable of being conducted exists, a doctor is required to participate in planning of the operation and formulation and adjustment of a specific operation scheme; if a path capable of being conducted exists, obtaining a globally optimal path for operation by using a path planning algorithm according to the judgment result of the step S5;
steps S2 to S3 and S4 may be performed synchronously.
2. The method according to claim 1, wherein in the step S3, the deep learning model is a resune++ model, a resune model or a une model.
3. The method according to claim 1 or 2, wherein in the step S4, the YOLO V4 model structure is divided into three parts: a backbone network, a neck network, and a header structure; the main network is a CSPDarknet53 network structure, and the CSPDarknet53 network comprises five residual modules with stacking times of 1, 2, 8 and 4 respectively; the neck network adopts an SPP module and a PANet network; the SPP module adopts the maximum pooling of 13×13, 9×9, 5×5 and 1×1 different pooling core sizes to process the final output characteristic layer of the CSPDarknet53 network; the head structure adopts three-scale output for detecting large, medium and small targets respectively for heads with the sizes of 1/8, 1/16 and 1/32 of the original input size.
4. A method of planning a path for a cardiovascular intervention according to claim 1 or 2, wherein in step S6, the path planning algorithm is Dijstra algorithm, astar algorithm or genetic algorithm.
5. A method of planning a path for a cardiovascular intervention according to claim 1 or 2, wherein in step S6, the path planning algorithm is a Dijstra algorithm, the Dijstra algorithm comprising the steps of:
s6.1, setting a weighted adjacent matrix AS to represent a weighted directed network, wherein AS [ i, j ] represents a weight value on an arc < vi, vj >, and arcs [ j, k ] represents the length of an edge between a vertex k and a vertex j;
if < vi, vj > does not exist, then set AS [ i, j ] to ≡; s is the set of end points of the shortest path from v, and the initial state of the set is an empty set; then, starting from v to the remaining vertices or endpoints vi on the graph, the initial value of the shortest path length that may be reached is:
DIST[i]=AS[v0,i]
step S6.2, selecting vj such that
DIST[j]=Min{DIST[i]|vi∈V-S}
vj is the end point of the shortest path from v obtained at present; order the
S=S∪{j}
S6.3, modifying the shortest path length from V to any vertex vk on the set V-S; if it is
D[j]+arcs[j,k]<D[k]
Then modify Dk to
D[k]=D[j]+arcs[j,k]
Step S6.4, repeating the steps S6.2 and S6.3 for n-1 times, and obtaining the shortest path from v to the rest vertexes on the graph as a sequence increasing according to the path length.
6. A cardiovascular interventional procedure path planning system, comprising:
the image preprocessing module is used for preprocessing the acquired angiography image by sequentially utilizing median filtering, bilateral filtering and Gaussian filtering methods;
the blood vessel contour segmentation module is used for segmenting the image processed by the image preprocessing module by using a deep learning model, identifying and marking the blood vessel contour, and generating a blood vessel contour image;
the focal point identification module is used for identifying and marking focal points of the original angiography image by utilizing the YOLO V4 model;
the blood vessel feature appraising module is used for appraising the features of blood vessel bending, blood vessel width and the like of the original angiography image by combining the blood vessel contour image and utilizing a knowledge graph;
the path planning module is used for judging whether a path which can be conducted by the operation exists or not; if no path capable of being conducted exists, a doctor is required to participate in planning of the operation and formulation and adjustment of a specific operation scheme; and if a path capable of being conducted exists, obtaining a globally optimal path for operation by using a path planning algorithm according to the judgment result of the blood vessel characteristic judgment module.
7. The cardiovascular interventional procedure path planning system of claim 6, wherein the vessel contour segmentation module wherein the deep learning model is a resunet++ model, a ResUNet model or a UNet model.
8. The cardiovascular interventional procedure path planning system of claim 6, wherein the path planning algorithm in the path planning module is Dijstra algorithm, astar algorithm or genetic algorithm.
9. A computer storage medium having a computer program, characterized in that the computer program, when executed by a processor, implements a cardiovascular intervention procedure path planning method according to any of claims 1 to 5.
10. An intelligent terminal, characterized by comprising: one or more memories and one or more processors;
the one or more memories are used for storing computer programs;
the one or more processors being connected to the memory for running the computer program to perform the cardiovascular interventional procedure path planning method of any one of claims 1 to 5.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310154582.8A CN116196099A (en) | 2023-02-23 | 2023-02-23 | Cardiovascular intervention operation path planning method, system, storage medium and terminal |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310154582.8A CN116196099A (en) | 2023-02-23 | 2023-02-23 | Cardiovascular intervention operation path planning method, system, storage medium and terminal |
Publications (1)
Publication Number | Publication Date |
---|---|
CN116196099A true CN116196099A (en) | 2023-06-02 |
Family
ID=86514345
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202310154582.8A Pending CN116196099A (en) | 2023-02-23 | 2023-02-23 | Cardiovascular intervention operation path planning method, system, storage medium and terminal |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN116196099A (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116473673A (en) * | 2023-06-20 | 2023-07-25 | 浙江华诺康科技有限公司 | Path planning method, device, system and storage medium for endoscope |
CN116912430A (en) * | 2023-09-14 | 2023-10-20 | 北京唯迈医疗设备有限公司 | Device for constructing three-dimensional digital twin system of remote intervention operating room |
CN117393107A (en) * | 2023-12-12 | 2024-01-12 | 北京唯迈医疗设备有限公司 | Iterative learning method and system for automatic surgical intervention robot and storage medium |
CN117481805A (en) * | 2023-11-30 | 2024-02-02 | 贵州医科大学 | Preoperative planning method and system for vascular intervention |
CN117530775A (en) * | 2024-01-09 | 2024-02-09 | 华中科技大学同济医学院附属协和医院 | Magnetic control intervention control method and system based on artificial intelligence and CT |
-
2023
- 2023-02-23 CN CN202310154582.8A patent/CN116196099A/en active Pending
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116473673A (en) * | 2023-06-20 | 2023-07-25 | 浙江华诺康科技有限公司 | Path planning method, device, system and storage medium for endoscope |
CN116473673B (en) * | 2023-06-20 | 2024-02-27 | 浙江华诺康科技有限公司 | Path planning method, device, system and storage medium for endoscope |
CN116912430A (en) * | 2023-09-14 | 2023-10-20 | 北京唯迈医疗设备有限公司 | Device for constructing three-dimensional digital twin system of remote intervention operating room |
CN116912430B (en) * | 2023-09-14 | 2023-12-15 | 北京唯迈医疗设备有限公司 | Device for constructing three-dimensional digital twin system of remote intervention operating room |
CN117481805A (en) * | 2023-11-30 | 2024-02-02 | 贵州医科大学 | Preoperative planning method and system for vascular intervention |
CN117481805B (en) * | 2023-11-30 | 2024-04-26 | 贵州医科大学 | Preoperative planning method and system for vascular intervention |
CN117393107A (en) * | 2023-12-12 | 2024-01-12 | 北京唯迈医疗设备有限公司 | Iterative learning method and system for automatic surgical intervention robot and storage medium |
CN117393107B (en) * | 2023-12-12 | 2024-03-15 | 北京唯迈医疗设备有限公司 | Iterative learning method and system for automatic surgical intervention robot and storage medium |
CN117530775A (en) * | 2024-01-09 | 2024-02-09 | 华中科技大学同济医学院附属协和医院 | Magnetic control intervention control method and system based on artificial intelligence and CT |
CN117530775B (en) * | 2024-01-09 | 2024-04-30 | 华中科技大学同济医学院附属协和医院 | Magnetic control intervention control method and system based on artificial intelligence and CT |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN116196099A (en) | Cardiovascular intervention operation path planning method, system, storage medium and terminal | |
WO2022141882A1 (en) | Lesion recognition model construction apparatus and system based on historical pathological information | |
CN110807764A (en) | Lung cancer screening method based on neural network | |
CN113112609A (en) | Navigation method and system for lung biopsy bronchoscope | |
CN111242959B (en) | Target area extraction method of multi-mode medical image based on convolutional neural network | |
CN111144486B (en) | Heart nuclear magnetic resonance image key point detection method based on convolutional neural network | |
CN113538471B (en) | Plaque segmentation method, plaque segmentation device, computer equipment and storage medium | |
CN112348883B (en) | Interventional instrument endpoint real-time positioning system, method and device in vascular interventional operation | |
CN111178369A (en) | Medical image identification method and system, electronic device and storage medium | |
CN109003280B (en) | Method for segmenting intima in blood vessel by two-channel intravascular ultrasonic image | |
CN114820520A (en) | Prostate image segmentation method and intelligent prostate cancer auxiliary diagnosis system | |
CN115082493A (en) | 3D (three-dimensional) atrial image segmentation method and system based on shape-guided dual consistency | |
CN115205520A (en) | Gastroscope image intelligent target detection method and system, electronic equipment and storage medium | |
CN117409002A (en) | Visual identification detection system for wounds and detection method thereof | |
JP7187680B2 (en) | LINE STRUCTURE EXTRACTION APPARATUS AND METHOD, PROGRAM AND TRAINED MODEL | |
Kusumaningtyas et al. | Feature extraction for application of heart abnormalities detection through iris based on mobile devices | |
CN107194922A (en) | A kind of extracting method of intravascular ultrasound image sequence key frame | |
CN113706684A (en) | Three-dimensional blood vessel image reconstruction method, system, medical device and storage medium | |
CN116012328A (en) | Method and device for detecting cavity branch point, electronic equipment and readable storage medium | |
Zhou et al. | Automatic segmentation of liver from CT scans with CCP–TSPM algorithm | |
CN114010227B (en) | Right ventricle characteristic information identification method and device | |
WO2022227193A1 (en) | Liver region segmentation method and apparatus, and electronic device and storage medium | |
CN114081625B (en) | Navigation path planning method, system and readable storage medium | |
CN115908224A (en) | Training method of target detection model, target detection method and training device | |
CN107464243B (en) | Aortic valve positioning method, device and equipment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |