CN117530775A - Magnetic control intervention control method and system based on artificial intelligence and CT - Google Patents

Magnetic control intervention control method and system based on artificial intelligence and CT Download PDF

Info

Publication number
CN117530775A
CN117530775A CN202410028563.5A CN202410028563A CN117530775A CN 117530775 A CN117530775 A CN 117530775A CN 202410028563 A CN202410028563 A CN 202410028563A CN 117530775 A CN117530775 A CN 117530775A
Authority
CN
China
Prior art keywords
virtual
vessel shape
intervention
magnetic
real
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202410028563.5A
Other languages
Chinese (zh)
Other versions
CN117530775B (en
Inventor
张进祥
陈标
马冰清
蔡丞俊
吕新宇
程星
高翾
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tongji Medical College of Huazhong University of Science and Technology
Original Assignee
Tongji Medical College of Huazhong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tongji Medical College of Huazhong University of Science and Technology filed Critical Tongji Medical College of Huazhong University of Science and Technology
Priority to CN202410028563.5A priority Critical patent/CN117530775B/en
Publication of CN117530775A publication Critical patent/CN117530775A/en
Application granted granted Critical
Publication of CN117530775B publication Critical patent/CN117530775B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2051—Electromagnetic tracking systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2065—Tracking using image or pattern recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30101—Blood vessel; Artery; Vein; Vascular

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Software Systems (AREA)
  • Surgery (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Geometry (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Graphics (AREA)
  • Public Health (AREA)
  • Animal Behavior & Ethology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Robotics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Veterinary Medicine (AREA)
  • Artificial Intelligence (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

The invention relates to the technical field of intelligent medical treatment, in particular to a magnetic control intervention control method and system based on artificial intelligence and CT, comprising the following steps: obtaining a blood vessel shape-changing calibration image through an artificial intelligent recognition model; obtaining a three-dimensional model of the blood vessel shape by a three-dimensional reconstruction technology; performing virtual construction of a digital twin platform by using a Gaussian-Kelvin projection method to obtain a blood vessel shape-shifting digital twin platform comprising a virtual magnetic intervention guide wire and a virtual blood vessel shape-shifting three-dimensional model; based on the vessel shape-moving digital twin platform, a virtual magnetic control intervention path of a virtual magnetic intervention guide wire is built in real time in a virtual vessel shape-moving three-dimensional model. The invention ensures that the virtual space and the physical space are synchronous in real time, the magnetic control intervention guide wire can obtain the intervention path update which is suitable for the real structure of the blood vessel in real time from the virtual space of the digital twin platform, and the planning of the intervention path and the magnetic control of the intervention running are separated, so that the mutual occupation of resources is avoided.

Description

Magnetic control intervention control method and system based on artificial intelligence and CT
Technical Field
The invention relates to the technical field of intelligent medical treatment, in particular to a magnetic control intervention control method based on artificial intelligence and CT.
Background
In the current medical environment, magnetic navigation technology is increasingly applied to medical practice. In the prior art, magnetic navigation techniques have been applied to interventional procedures. For example: in cardiac intervention operation, the magnetic navigation technology can control a magnetic field by using a computer interface, and guide and position a puncture guide wire and a catheter under the guidance of DSA, so that the precision of fine intervention operation is greatly improved, and the radiation dose of a patient and medical staff in the operation process can be greatly reduced.
At present, the existing interventional operation path usually uses some simple three-dimensional environments or virtual environments which do not exist in the real world to test the effect of a path planning algorithm, the vascular shape structure in the real environment is complex and various, and the path planning algorithm which is developed and tested in the simple three-dimensional environments or the virtual environments which do not exist in the real world is difficult to meet the actual travelling requirement of a magnetic interventional guide wire, so that the interventional treatment effect is influenced. In addition, the calculation and storage resources of the onboard computer of the magnetic navigator are limited, so that the magnetic navigator cannot adapt to the change of the real condition of the blood vessel in real time, and the situation that the magnetic interventional guide wire is controlled to perform the real-time path planning update while the interventional guide wire is controlled to perform the interventional operation is difficult to realize, and the accuracy and the safety of the magnetic interventional guide wire can be influenced due to the occupation of the mutual resources.
Disclosure of Invention
The invention aims to provide a magnetic control intervention control method and a magnetic control intervention control system based on artificial intelligence and CT (computed tomography), which are used for solving the technical problems that the magnetic control intervention process is controlled by a computer in the whole course, the magnetic intervention guide wire intervention advancing is controlled, the real-time path planning updating is performed, the occupation of the mutual resources of a computer program is avoided, and the accuracy and the safety of the magnetic control intervention are influenced. The invention not only can realize more accurate interventional operation, but also can greatly reduce the technical effect of radiation contact of interventional medical staff.
In order to solve the technical problems, the invention specifically provides the following technical scheme:
a magnetic control intervention control method based on artificial intelligence and CT comprises the following steps:
acquiring an angiographic CT image;
calibrating the vessel shape: performing calibration processing on image pixels representing blood vessels in an angiography CT image according to blood vessel shape variation through an artificial intelligent identification model to obtain a blood vessel shape variation calibration image;
reconstructing a three-dimensional model of the blood vessel shape: performing physical reconstruction of a three-dimensional space based on the blood vessel shape-shifting calibration image by a three-dimensional reconstruction technology to obtain a blood vessel shape-shifting three-dimensional model;
Constructing a vessel shape-changing digital twin platform: performing virtual construction of a digital twin platform according to the magnetic intervention guide wire and the blood vessel shape-changing three-dimensional model by using a Gaussian-Kriging projection method to obtain the blood vessel shape-changing digital twin platform comprising the virtual magnetic intervention guide wire and the virtual blood vessel shape-changing three-dimensional model;
based on a vessel shape-changing digital twin platform, a DSA system is utilized to build a virtual magnetic control intervention path of a virtual magnetic intervention guide wire in real time in a virtual vessel shape-changing three-dimensional model;
the vessel shape-moving digital twin platform feeds back the virtual magnetic control intervention path to the magnetic navigation instrument in real time, and the magnetic intervention guide wire is controlled by the magnetic navigation instrument in real time to perform real-time intervention movement according to the virtual magnetic control intervention path.
As a preferable scheme of the invention, the calibration method for the blood vessel shape-changing calibration image comprises the following steps:
randomly selecting a plurality of angiography CT images as sample images;
marking image pixels representing blood vessels in the sample image as target pixels;
taking the sample image as an input item of a YOLO V3 neural network, taking the target pixel as an output item of the YOLO V3 neural network, and carrying out mapping learning on the input item of the YOLO V3 neural network and the output item of the YOLO V3 neural network by utilizing the YOLO V3 neural network to obtain the artificial intelligent recognition model;
Identifying image pixels representing blood vessels in each angiography CT image by using the artificial intelligent model, and smoothly linking the image pixels representing the blood vessels by using a B-spline curve smoothing method to obtain calibration lines representing the shape of the blood vessels;
and taking the angiography CT image with the calibration lines as the blood vessel shape-changing calibration image.
As a preferable scheme of the invention, the reconstruction of the three-dimensional model of the blood vessel shape is realized by using a three-dimensional reconstruction technology 3DMax, and a magnetic control intervention starting point and a magnetic control intervention end point are marked on the three-dimensional model of the blood vessel shape by using the three-dimensional reconstruction technology 3 DMax.
As a preferred embodiment of the present invention, the method for constructing a digital twin platform for vessel shape correction includes:
developing a digital twin platform by utilizing Unity3D, and loading and displaying the magnetic intervention guide wire and blood vessel shape-changing three-dimensional model in the digital twin platform to obtain the virtual magnetic intervention guide wire and virtual blood vessel shape-changing three-dimensional model;
mapping physical coordinates of the magnetic intervention guide wire to virtual coordinates of the virtual magnetic intervention guide wire in the digital twin platform by using a Gaussian-Kriging projection method, and mapping physical coordinates of the blood vessel shape-moving three-dimensional model to virtual coordinates of the virtual blood vessel shape-moving three-dimensional model in the digital twin platform;
Establishing an interaction channel for data interaction between the magnetic intervention physical guide wire and the virtual magnetic intervention guide wire;
and constructing the vessel shape-changing digital twin platform.
As a preferred scheme of the invention, the method for carrying out coordinate mapping on physical coordinates of the magnetic intervention guide wire and the blood vessel shape three-dimensional model by using the Gauss-Gauss projection method comprises the following steps:
selecting a plurality of first control points from a blood vessel shape three-dimensional model, recording three-dimensional coordinates of the first control points, finding positions of the selected first control points corresponding to observation points in a real environment, recording longitude and latitude coordinates of the observation points, and converting the longitude and latitude coordinates containing height information recorded by each observation point into three-dimensional coordinates in a Cartesian coordinate system through Gauss-Krueger projection orthometric formula projection;
estimating coordinate system deviation of a blood vessel shape three-dimensional model center point from longitude and latitude coordinate projection conversion to a Cartesian coordinate system origin by using a least square method according to the three-dimensional coordinates of a plurality of first control points and the three-dimensional coordinates of the corresponding observation points after the longitude and latitude coordinate projection conversion;
the three-dimensional coordinates of each model point in the physical space where the three-dimensional model of the blood vessel is located are obtained by adding the three-dimensional coordinates obtained by projection conversion of the longitude and latitude coordinates of each model point in the physical space where the three-dimensional model of the blood vessel is located through a Gaussian-Kelvin projection orthometric formula and the deviations of the coordinate system, and the three-dimensional coordinates of the corresponding point in the virtual space where the three-dimensional model of the virtual blood vessel is located are obtained under the Cartesian coordinate system;
Subtracting the coordinate system deviation from the three-dimensional coordinates of each model point in the virtual space where the virtual vessel shape three-dimensional model is located under the Cartesian coordinate system, and obtaining mathematical correlation of longitude and latitude coordinates of corresponding points in the physical space where the vessel shape three-dimensional model is located through the projection conversion of a Gaussian-Kelvin projection inverse solution formula, so that the physical coordinates of the vessel shape three-dimensional model are mapped to the virtual coordinates of the virtual vessel shape three-dimensional model in the digital twin platform;
selecting a plurality of second control points from the magnetic interventional guide wire, recording three-dimensional coordinates of the second control points, finding positions of the selected second control points corresponding to the observation points in a real environment, recording longitude and latitude coordinates of the observation points, and converting the longitude and latitude coordinates containing the height information recorded by each observation point into three-dimensional coordinates in a Cartesian coordinate system through Gauss-Krueger projection orthometric formula projection;
estimating coordinate system deviation of the magnetic intervention guide wire center point from the longitude and latitude coordinate projection conversion to the origin of a Cartesian coordinate system by using a least square method according to the three-dimensional coordinates of the plurality of second control points and the three-dimensional coordinates of the corresponding observation points after the longitude and latitude coordinate projection conversion;
The three-dimensional coordinates of each model point in the physical space where the magnetic interventional guide wire is located are obtained by adding the three-dimensional coordinates obtained by projection conversion of the Gauss-Gauss projection orthometric formula and the deviations of the coordinate system, and the three-dimensional coordinates of the corresponding point in the virtual space where the virtual magnetic interventional guide wire is located are obtained under the Cartesian coordinate system;
after the coordinate system deviation is subtracted from the three-dimensional coordinate of each model point in the virtual space where the magnetic intervention guide wire is located under the Cartesian coordinate system, the mathematical correlation of the longitude and latitude coordinates of the corresponding point in the physical space where the magnetic intervention guide wire is located is obtained after the projection conversion of the Gaussian-Kelvin projection inverse solution formula, and the mapping of the physical coordinate of the magnetic intervention guide wire to the virtual coordinate of the virtual magnetic intervention guide wire in the digital twin platform is realized.
As a preferable scheme of the invention, the method for constructing the virtual magnetic control intervention path in real time comprises the following steps:
the vessel shape-changing digital twin platform is based on a virtual vessel shape-changing three-dimensional model, and a magnetic control intervention basic virtual path is planned at a magnetic control intervention starting point and a magnetic control intervention ending point by utilizing a path planning algorithm;
the vessel shape-moving digital twin platform feeds back the magnetic control intervention basic virtual path to the magnetic navigation instrument in real time through the interaction channel, and the magnetic intervention guide wire is controlled by the magnetic navigation instrument in real time to perform real-time intervention movement according to the magnetic control intervention basic virtual path;
Acquiring real-time physical coordinates of the magnetic control intervention guide wire in the process of performing real-time intervention movement along a virtual path of a magnetic control intervention foundation by using a DSA system, acquiring a real-time DSA image of the magnetic control intervention guide wire at the real-time physical coordinates, and simultaneously transmitting the real-time DSA image and the real-time physical coordinates to a blood vessel shape-moving digital twin platform through an interactive channel;
according to the real-time physical coordinates, the vessel shape-changing digital twin platform acquires angiography CT images at virtual coordinates corresponding to the real-time physical coordinates in the virtual vessel shape-changing three-dimensional model, and marks the angiography CT images as priori CT images;
the vessel shape-shifting digital twin platform utilizes a twin neural network to carry out real-time detection on the suitability of path planning based on a real-time DSA image and a priori CT image, so as to obtain the suitability of path planning;
the vessel shape-moving digital twin platform carries out the identification of the update sites on the virtual vessel shape-moving three-dimensional model according to the path planning applicability, and the update sites in the virtual vessel shape-moving three-dimensional model are obtained;
at an update site in the virtual vessel shape three-dimensional model, updating the virtual vessel shape three-dimensional model by using a real-time DSA image corresponding to the update site;
The vessel shape-changing digital twin platform is used for planning a magnetic control intervention updating virtual path at an updating site and a magnetic control intervention terminal point by utilizing a path planning algorithm based on the updated virtual vessel shape-changing three-dimensional model;
the vessel shape-moving digital twin platform feeds the magnetic control intervention updating virtual path back to the magnetic navigation instrument in real time through the interaction channel, and the magnetic intervention guide wire is controlled by the magnetic navigation instrument in real time to perform real-time intervention movement according to the magnetic control intervention updating virtual path.
As a preferred embodiment of the present invention, a method for detecting path planning suitability includes:
inputting the real-time DSA image into a first CNN network structure in the twin neural network, and outputting the vascular characteristics in the real-time DSA image by the first CNN network structure in the twin neural network;
inputting the real-time CT image into a second CNN network structure in the twin neural network, and outputting vessel characteristics in the prior CT image by the second CNN network structure in the twin neural network;
detecting path planning applicability by using a loss function of a twin neural network, wherein the path planning applicability has the following expression: match_gold= - (1-Low) r log (Loss); in the formula, match_gold is path planning applicability, loss is a Loss function of the twin neural network, and r is an artificial regulation parameter;
The artificial regulation parameters are used for adding artificial will in the updating of the virtual vessel shape-changing three-dimensional model;
wherein, the loss function of the twin neural network is: loss=mse (out 1, out 2); where Loss is a Loss function, out1 is an output of a first CNN network structure in the twin neural network, out2 is an output of a second CNN network structure in the twin neural network, MSE is a mean square error function, and MSE (out 1, out 2) is a mean square error between out1 and out 2.
As a preferable scheme of the invention, the method for identifying the update site in the virtual blood vessel shape three-dimensional model comprises the following steps:
comparing the path planning suitability with a preset threshold, wherein,
when the path planning applicability is greater than or equal to a preset threshold, calibrating virtual coordinates corresponding to the prior CT image as update sites in the virtual vessel shape three-dimensional model;
and when the path planning applicability is smaller than a preset threshold, calibrating the virtual coordinates corresponding to the prior CT image as non-updated sites in the virtual blood vessel walking three-dimensional model.
As a preferred embodiment of the present invention, the method for updating the three-dimensional model of the virtual vessel shape by using the corresponding real-time DSA image at the update site includes:
And replacing the blood vessel characteristics of the corresponding real-time DSA image at the update site with the blood vessel characteristics of the corresponding priori CT image at the update site in the virtual blood vessel shape three-dimensional model to obtain an updated virtual blood vessel shape three-dimensional model.
As a preferred scheme of the invention, the invention provides a magnetic control intervention control system based on artificial intelligence and CT, which is applied to the magnetic control intervention control method based on artificial intelligence and CT, and comprises the following steps:
the image acquisition unit is used for acquiring angiography CT images and DSA images;
the image processing unit is used for calibrating image pixels representing blood vessels in the angiography CT image according to blood vessel shape variation through the artificial intelligent recognition model to obtain a blood vessel shape variation calibration image;
the three-dimensional reconstruction unit is used for carrying out physical reconstruction of a three-dimensional space based on the blood vessel shape-shifting calibration image by a three-dimensional reconstruction technology to obtain a blood vessel shape-shifting three-dimensional model;
the platform building unit is used for virtually building the digital twin platform according to the magnetic intervention guide wire and the blood vessel shape-shifting three-dimensional model by utilizing a Gaussian-Krueger projection method to obtain the blood vessel shape-shifting digital twin platform comprising the virtual magnetic intervention guide wire and the virtual blood vessel shape-shifting three-dimensional model;
The platform running unit is connected with the DSA system and is used for building a virtual magnetic control intervention path of the virtual magnetic intervention guide wire in a virtual blood vessel shape three-dimensional model in real time based on the blood vessel shape digital twin platform by utilizing the DSA system;
the magnetic control intervention unit comprises a magnetic navigation instrument, is used for receiving a virtual magnetic control intervention path fed back by the blood vessel shape-changing digital twin platform, and controls the magnetic intervention guide wire to perform real-time intervention movement according to the virtual magnetic control intervention path in real time;
the platform operation unit builds a virtual magnetic control intervention path of a virtual magnetic intervention guide wire in real time in a virtual blood vessel shape-changing three-dimensional model by using a DSA system, and the platform operation unit comprises:
the vessel shape-changing digital twin platform is based on a virtual vessel shape-changing three-dimensional model, and a magnetic control intervention basic virtual path is planned at a magnetic control intervention starting point and a magnetic control intervention ending point by utilizing a path planning algorithm;
the vessel shape-moving digital twin platform feeds back the magnetic control intervention basic virtual path to the magnetic navigation instrument in real time through the interaction channel, and the magnetic intervention guide wire is controlled by the magnetic navigation instrument in real time to perform real-time intervention movement according to the magnetic control intervention basic virtual path;
acquiring real-time physical coordinates of the magnetic control intervention guide wire in the process of performing real-time intervention movement along a virtual path of a magnetic control intervention foundation by using a DSA system, acquiring a real-time DSA image of the magnetic control intervention guide wire at the real-time physical coordinates, and simultaneously transmitting the real-time DSA image and the real-time physical coordinates to a blood vessel shape-moving digital twin platform through an interactive channel;
According to the real-time physical coordinates, the vessel shape-changing digital twin platform acquires angiography CT images at virtual coordinates corresponding to the real-time physical coordinates in the virtual vessel shape-changing three-dimensional model, and marks the angiography CT images as priori CT images;
the vessel shape-shifting digital twin platform utilizes a twin neural network to carry out real-time detection on the suitability of path planning based on a real-time DSA image and a priori CT image, so as to obtain the suitability of path planning;
the vessel shape-moving digital twin platform carries out the identification of the update sites on the virtual vessel shape-moving three-dimensional model according to the path planning applicability, and the update sites in the virtual vessel shape-moving three-dimensional model are obtained;
at an update site in the virtual vessel shape three-dimensional model, updating the virtual vessel shape three-dimensional model by using a real-time DSA image corresponding to the update site;
the vessel shape-changing digital twin platform is used for planning a magnetic control intervention updating virtual path at an updating site and a magnetic control intervention terminal point by utilizing a path planning algorithm based on the updated virtual vessel shape-changing three-dimensional model;
the vessel shape-moving digital twin platform feeds the magnetic control intervention updating virtual path back to the magnetic navigation instrument in real time through the interaction channel, and the magnetic intervention guide wire is controlled by the magnetic navigation instrument in real time to perform real-time intervention movement according to the magnetic control intervention updating virtual path.
Compared with the prior art, the invention has the following beneficial effects:
according to the invention, the virtual construction of the digital twin platform is carried out according to the magnetic intervention guide wire and the blood vessel shape-changing three-dimensional model, so that the blood vessel shape-changing digital twin platform comprising the virtual magnetic intervention guide wire and the virtual blood vessel shape-changing three-dimensional model is obtained, a great amount of tests can be carried out in a virtual space close to a real environment in a development and test stage of magnetic intervention path planning, the safety of the magnetic intervention guide wire during intervention movement in the real environment is effectively improved, and the real-time synchronization of the virtual space and the physical space is realized, so that the magnetic intervention guide wire can obtain the intervention path update which is suitable for the real structure of the blood vessel in real time from the virtual space of the digital twin platform, the planning of the intervention path and the magnetic control of intervention running are separated, the mutual occupation of resources is avoided, the airborne computer pressure of the magnetic intervention navigator is effectively reduced, the planning accuracy and the safety of the magnetic intervention path are also improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It will be apparent to those of ordinary skill in the art that the drawings in the following description are exemplary only and that other implementations can be obtained from the extensions of the drawings provided without inventive effort.
FIG. 1 is a flowchart of a magnetic control intervention control method based on artificial intelligence and CT provided by an embodiment of the invention;
FIG. 2 is a block diagram of a control system according to an embodiment of the present invention;
fig. 3 is a calibration image of vessel shape deformation according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
As shown in fig. 1, the invention provides a magnetic control intervention control method based on artificial intelligence and CT, comprising the following steps: acquiring an angiographic CT image;
calibrating the vessel shape: performing calibration processing on image pixels representing blood vessels in an angiography CT image according to blood vessel shape variation through an artificial intelligent identification model to obtain a blood vessel shape variation calibration image;
reconstructing a three-dimensional model of the blood vessel shape: performing physical reconstruction of a three-dimensional space based on the blood vessel shape-shifting calibration image by a three-dimensional reconstruction technology to obtain a blood vessel shape-shifting three-dimensional model;
Constructing a vessel shape-changing digital twin platform: performing virtual construction of a digital twin platform according to the magnetic intervention guide wire and the blood vessel shape-shifting three-dimensional model by using a Gaussian-Kriging projection method to obtain the blood vessel shape-shifting digital twin platform comprising the virtual magnetic intervention guide wire and the virtual blood vessel shape-shifting three-dimensional model;
based on a vessel shape-changing digital twin platform, a DSA system is utilized to build a virtual magnetic control intervention path of a virtual magnetic intervention guide wire in real time in a virtual vessel shape-changing three-dimensional model;
the vessel shape-moving digital twin platform feeds back the virtual magnetic control intervention path to the magnetic navigation instrument in real time, and the magnetic intervention guide wire is controlled by the magnetic navigation instrument in real time to perform real-time intervention movement according to the virtual magnetic control intervention path.
In order to avoid resource occupation in two processes of planning an intervention path and magnetic control of intervention traveling in the magnetic control intervention control method, a digital twin platform is built by utilizing a digital twin technology to plan the intervention path, so that a magnetic control navigator only needs to conduct magnetic control of intervention traveling according to the intervention path planned by the digital twin platform, and the efficiency of magnetic control intervention traveling can be improved while the pressure of an onboard computer of the magnetic control navigator is effectively reduced.
Specifically, the digital twin platform is utilized to plan the intervention path, so that the magnetic control intervention path planning can carry out a large number of tests in a virtual space close to the real environment in a development and test stage, and the safety of the magnetic control intervention guide wire during the intervention advancing movement in the real environment is effectively improved.
Further, data interaction is carried out between the blood vessel virtual model and the real intervention scene in the virtual space of the digital twin platform, real-time interaction is presented by the interaction, and the digital twin platform carries out self-adaptive updating of the blood vessel virtual model according to the real-time interaction information, so that the self-adaptive updating of the intervention path is realized, the adaptively matching of the intervention path according to the real blood vessel shape and structure in the intervention occurrence process is ensured, the correction updating of the intervention path is realized, and the accuracy of magnetic control intervention is improved.
In order to construct a digital twin platform for planning an intervention path, an artificial intelligent model capable of identifying vessel shape after CT radiography is trained by utilizing a neural network, so that vessel identification accuracy after radiography is realized, a data base is provided for planning the intervention path of the digital twin platform, the suitability of the intervention path planning of the digital twin platform to a real scene is further ensured, namely, the accuracy of the intervention path planning is improved, and the method specifically comprises the following steps:
As shown in fig. 3, the calibration method for the blood vessel shape-changing calibration image comprises the following steps:
randomly selecting a plurality of angiography CT images as sample images;
marking image pixels representing blood vessels in the sample image as target pixels;
taking the sample image as an input item of a YOLO V3 neural network, taking a target pixel as an output item of the YOLO V3 neural network, and carrying out mapping learning on the input item of the YOLO V3 neural network and the output item of the YOLO V3 neural network by utilizing the YOLO V3 neural network to obtain an artificial intelligent recognition model;
identifying image pixels representing blood vessels in each angiography CT image by using an artificial intelligent model, and smoothly linking the image pixels representing the blood vessels by using a B-spline curve smoothing method to obtain calibration lines representing the shape of the blood vessels;
the angiographic CT image with the calibration lines is taken as a blood vessel shape calibration image, and the black lines in FIG. 3 are blood vessel shapes.
The invention utilizes a digital twin technology to build a digital twin platform, and specifically comprises the following steps:
the reconstruction of the three-dimensional model of the blood vessel shape is realized by using a three-dimensional reconstruction technology 3DMax, and a magnetic control intervention starting point and a magnetic control intervention end point are marked on the three-dimensional model of the blood vessel shape by using the three-dimensional reconstruction technology 3 DMax.
The method for building the vessel shape-changing digital twin platform comprises the following steps:
developing a digital twin platform by utilizing Unity3D, and loading and displaying the magnetic intervention guide wire and the blood vessel shape-shifting three-dimensional model in the digital twin platform to obtain a virtual magnetic intervention guide wire and a virtual blood vessel shape-shifting three-dimensional model;
mapping physical coordinates of the magnetic intervention guide wire to virtual coordinates of the virtual magnetic intervention guide wire in the digital twin platform by using a Gaussian-Kriging projection method, and mapping physical coordinates of the blood vessel shape-moving three-dimensional model to virtual coordinates of the virtual blood vessel shape-moving three-dimensional model in the digital twin platform;
establishing an interaction channel for data interaction between the magnetic intervention physical guide wire and the virtual magnetic intervention guide wire;
and constructing a blood vessel shape-changing digital twin platform.
The method for carrying out coordinate mapping on physical coordinates of the magnetic intervention guide wire and the blood vessel shape-changing three-dimensional model by using the Gauss-Gauss projection method comprises the following steps:
selecting a plurality of first control points from a blood vessel shape three-dimensional model, recording three-dimensional coordinates of the first control points, finding positions of the selected first control points corresponding to observation points in a real environment, recording longitude and latitude coordinates of the observation points, and converting the longitude and latitude coordinates containing height information recorded by each observation point into three-dimensional coordinates in a Cartesian coordinate system through Gauss-Krueger projection orthometric formula projection;
Estimating coordinate system deviation of a blood vessel shape three-dimensional model center point from longitude and latitude coordinate projection conversion to a Cartesian coordinate system origin by using a least square method according to the three-dimensional coordinates of a plurality of first control points and the three-dimensional coordinates of the corresponding observation points after the longitude and latitude coordinate projection conversion;
the three-dimensional coordinates of each model point in the physical space where the three-dimensional model of the blood vessel is located are obtained by adding the three-dimensional coordinates obtained by projection conversion of the longitude and latitude coordinates of each model point in the physical space where the three-dimensional model of the blood vessel is located through a Gaussian-Kelvin projection orthometric formula and the deviations of the coordinate system, and the three-dimensional coordinates of the corresponding point in the virtual space where the three-dimensional model of the virtual blood vessel is located are obtained under the Cartesian coordinate system;
subtracting the coordinate system deviation from the three-dimensional coordinates of each model point in the virtual space where the virtual vessel shape three-dimensional model is located under the Cartesian coordinate system, and obtaining mathematical correlation of longitude and latitude coordinates of corresponding points in the physical space where the vessel shape three-dimensional model is located through the projection conversion of a Gaussian-Kelvin projection inverse solution formula, so that the physical coordinates of the vessel shape three-dimensional model are mapped to the virtual coordinates of the virtual vessel shape three-dimensional model in the digital twin platform;
selecting a plurality of second control points from the magnetic interventional guide wire, recording three-dimensional coordinates of the second control points, finding positions of the selected second control points corresponding to the observation points in a real environment, recording longitude and latitude coordinates of the observation points, and converting the longitude and latitude coordinates containing the height information recorded by each observation point into the three-dimensional coordinates in a Cartesian coordinate system through Gauss-Krueger projection orthometric formula projection;
Estimating coordinate system deviation of the magnetic intervention guide wire center point from the longitude and latitude coordinate projection conversion to the origin of a Cartesian coordinate system by using a least square method according to the three-dimensional coordinates of the plurality of second control points and the three-dimensional coordinates of the corresponding observation points after the longitude and latitude coordinate projection conversion;
the three-dimensional coordinates of each model point in the physical space where the magnetic interventional guide wire is located are obtained by adding the three-dimensional coordinates obtained by projection conversion of the Gauss-Gauss projection orthometric formula and the deviations of the coordinate system, and the three-dimensional coordinates of the corresponding point in the virtual space where the virtual magnetic interventional guide wire is located are obtained under the Cartesian coordinate system;
after the coordinate system deviation is subtracted from the three-dimensional coordinate of each model point in the virtual space where the magnetic intervention guide wire is located under the Cartesian coordinate system, the mathematical correlation of the longitude and latitude coordinates of the corresponding point in the physical space where the magnetic intervention guide wire is located is obtained after the projection conversion of the Gaussian-Kelvin projection inverse solution formula, and the mapping of the physical coordinate of the magnetic intervention guide wire to the virtual coordinate of the virtual magnetic intervention guide wire in the digital twin platform is realized.
The Gaussian-Creuger projection method ensures that the virtual blood vessel shape-moving three-dimensional model on the digital twin platform is synchronized in terms of shape and coordinates, thereby facilitating data interaction between the digital twin platform and an intervention real scene, namely realizing synchronous mapping of the virtual space and the real space, further ensuring that the digital twin platform can carry out adaptive correction according to the real condition of the blood vessel and ensuring the planning accuracy of the intervention path.
According to the invention, data interaction is carried out between a blood vessel virtual model and a real intervention scene in a virtual space of a digital twin platform, real-time interaction is presented by interaction, and the digital twin platform carries out self-adaptive updating of the blood vessel virtual model according to real-time interaction information, so that the self-adaptive updating of an intervention path is realized, and the method comprises the following specific steps:
the method for constructing the virtual magnetic control intervention path in real time comprises the following steps:
the vessel shape-changing digital twin platform is based on a virtual vessel shape-changing three-dimensional model, and a magnetic control intervention basic virtual path is planned at a magnetic control intervention starting point and a magnetic control intervention ending point by utilizing a path planning algorithm;
the vessel shape-moving digital twin platform feeds back the magnetic control intervention basic virtual path to the magnetic navigation instrument in real time through the interaction channel, and the magnetic intervention guide wire is controlled by the magnetic navigation instrument in real time to perform real-time intervention movement according to the magnetic control intervention basic virtual path;
acquiring real-time physical coordinates of the magnetic control intervention guide wire in the process of performing real-time intervention movement along a virtual path of a magnetic control intervention foundation by using a DSA system, acquiring a real-time DSA image of the magnetic control intervention guide wire at the real-time physical coordinates, and simultaneously transmitting the real-time DSA image and the real-time physical coordinates to a blood vessel shape-moving digital twin platform through an interactive channel;
The vessel shape-changing digital twin platform obtains angiography CT images at virtual coordinates corresponding to the real-time physical coordinates in the virtual vessel shape-changing three-dimensional model according to the real-time physical coordinates, and marks the angiography CT images as priori CT images;
the vessel shape-shifting digital twin platform utilizes a twin neural network to carry out real-time detection on the suitability of path planning based on a real-time DSA image and a priori CT image, so as to obtain the suitability of path planning;
the vessel shape-moving digital twin platform carries out the identification of the update sites on the virtual vessel shape-moving three-dimensional model according to the path planning applicability, and the update sites in the virtual vessel shape-moving three-dimensional model are obtained;
at an update site in the virtual vessel shape three-dimensional model, updating the virtual vessel shape three-dimensional model by using a real-time DSA image corresponding to the update site;
the vessel shape-changing digital twin platform is used for planning a magnetic control intervention updating virtual path at an updating site and a magnetic control intervention terminal point by utilizing a path planning algorithm based on the updated virtual vessel shape-changing three-dimensional model;
the vessel shape-moving digital twin platform feeds the magnetic control intervention updating virtual path back to the magnetic navigation instrument in real time through the interaction channel, and the magnetic intervention guide wire is controlled by the magnetic navigation instrument in real time to perform real-time intervention movement according to the magnetic control intervention updating virtual path.
The method for detecting the applicability of the path planning comprises the following steps:
inputting the real-time DSA image into a first CNN network structure in the twin neural network, and outputting the vascular characteristics in the real-time DSA image by the first CNN network structure in the twin neural network;
inputting the real-time CT image into a second CNN network structure in the twin neural network, and outputting the blood vessel characteristics in the prior CT image by the second CNN network structure in the twin neural network;
and detecting path planning applicability by using a loss function of the twin neural network, wherein the path planning applicability has the following expression:
Match_goal=-(1-Loss) r log (Loss); in the formula, match_gold is path planning applicability, loss is a Loss function of the twin neural network, and r is an artificial regulation parameter;
the artificial regulation parameters are used for adding artificial will in the updating of the virtual vessel shape-changing three-dimensional model;
the loss function of the twin neural network is as follows:
loss=mse (out 1, out 2); where Loss is a Loss function, out1 is an output of a first CNN network structure in the twin neural network, out2 is an output of a second CNN network structure in the twin neural network, MSE is a mean square error function, and MSE (out 1, out 2) is a mean square error between out1 and out 2.
When the intervention route is planned, the DSA image displaying the current blood vessel shape and structure is acquired in real time, whether the current intervention route is applicable to the current blood vessel intervention is adaptively distinguished from CT images displaying the blood vessel shape and structure in the DSA image and the three-dimensional model through the twin neural network, the intervention route is re-planned according to the applicability detection result, namely the intervention route is corrected, the intervention accuracy is ensured, real-time data represented by the real-time DSA image and priori knowledge represented by the CT image are matched, the route planning correction is completed, the whole-process control of the magnetic control intervention process is realized by a computer, and the intervention progress of the magnetic intervention guide wire is controlled while the real-time route planning update is performed. Therefore, the invention realizes the self-adaptive intervention path planning correction and enhances the planning robustness of the intervention path.
Specifically, the invention utilizes the twin neural network to identify that the current blood vessel shape and structure (blood vessel characteristics) are different from the blood vessel shape and structure at the same position in the virtual blood vessel shape three-dimensional model, namely, the change detection of the blood vessel shape and structure is realized, the blood vessel shape and structure is changed or the blood vessel shape and structure in the prior virtual blood vessel shape three-dimensional model is constructed inaccurately, and the current intervention path is not applicable, so that the intervention path is required to be corrected/re-planned, thereby realizing the self-adaptive identification of the update site of the intervention path, further self-adaptively correcting the intervention path, ensuring the real-time accuracy of magnetic control intervention, and maintaining the accuracy of magnetic control intervention in the whole intervention process.
The invention utilizes the loss function of the twin neural network to quantify the route planning applicability, realizes that the larger the loss function is, the higher the change degree of the shape and the structure of the blood vessel is compared with the change degree of the virtual blood vessel in the three-dimensional model of the walking shape, so that the lower the route planning applicability is, the lower the change degree of the shape and the structure of the blood vessel is compared with the change degree of the virtual blood vessel in the three-dimensional model of the walking shape is, the higher the route planning applicability is, the real situation is met, and the self-adaptive identification of the updating site is realized.
And the manual regulation and control parameters are added in the path planning applicability, the intervention path can be manually determined to be re-planned by adjusting r, the manual intention is added, and under the condition that the intervention path is manually expected to be corrected, the self-adaptive control and the manual control can be combined by manually regulating and controlling the correction of the intervention path, so that the needs of various scenes are met. When manual regulation is not needed, r can be assigned to 1, and when manual regulation is needed, r is assigned to a value which is larger than 1 and meets the condition that the KPI is larger than a preset threshold, and the method is specifically determined according to a real-scene application scene.
The method for identifying the update site in the virtual blood vessel shape-changing three-dimensional model comprises the following steps:
comparing the path planning suitability with a preset threshold, wherein,
When the applicability of the path planning is greater than or equal to a preset threshold, the virtual coordinates corresponding to the prior CT image are marked as update sites in the virtual vessel shape three-dimensional model;
and when the path planning applicability is smaller than a preset threshold, calibrating the virtual coordinates corresponding to the prior CT image as non-updated sites in the virtual blood vessel walking three-dimensional model.
The method for updating the virtual blood vessel shape three-dimensional model by using the corresponding real-time DSA image at the updating site comprises the following steps:
and replacing the blood vessel characteristics of the corresponding real-time DSA image at the update site with the blood vessel characteristics of the corresponding priori CT image at the update site in the virtual blood vessel shape three-dimensional model to obtain an updated virtual blood vessel shape three-dimensional model.
As shown in fig. 2, the present invention provides a magnetic control intervention control system based on artificial intelligence and CT, which is characterized in that the magnetic control intervention control system is applied to a magnetic control intervention control method based on artificial intelligence and CT as claimed in any one of claims 1 to 9, and the magnetic control intervention control system comprises:
the image acquisition unit is used for acquiring angiography CT images and DSA images;
the image processing unit is used for calibrating image pixels representing blood vessels in the angiography CT image according to blood vessel shape variation through the artificial intelligent recognition model to obtain a blood vessel shape variation calibration image;
The three-dimensional reconstruction unit is used for carrying out physical reconstruction of a three-dimensional space based on the blood vessel shape-shifting calibration image by a three-dimensional reconstruction technology to obtain a blood vessel shape-shifting three-dimensional model;
the platform building unit is used for virtually building the digital twin platform according to the magnetic intervention guide wire and the blood vessel shape-shifting three-dimensional model by utilizing a Gaussian-Krueger projection method to obtain the blood vessel shape-shifting digital twin platform comprising the virtual magnetic intervention guide wire and the virtual blood vessel shape-shifting three-dimensional model;
the platform running unit is connected with the DSA system and is used for building a virtual magnetic control intervention path of the virtual magnetic intervention guide wire in a virtual blood vessel shape three-dimensional model in real time based on the blood vessel shape digital twin platform by utilizing the DSA system;
the magnetic control intervention unit comprises a magnetic navigation instrument and is used for receiving a virtual magnetic control intervention path fed back by the blood vessel shape-changing digital twin platform and controlling the magnetic intervention guide wire to perform real-time intervention movement according to the virtual magnetic control intervention path.
The platform operation unit builds a virtual magnetic control intervention path of a virtual magnetic intervention guide wire in real time in a virtual blood vessel shape three-dimensional model by using a DSA system, and the platform operation unit comprises:
the vessel shape-changing digital twin platform is based on a virtual vessel shape-changing three-dimensional model, and a magnetic control intervention basic virtual path is planned at a magnetic control intervention starting point and a magnetic control intervention ending point by utilizing a path planning algorithm;
The vessel shape-moving digital twin platform feeds back the magnetic control intervention basic virtual path to the magnetic navigation instrument in real time through the interaction channel, and the magnetic intervention guide wire is controlled by the magnetic navigation instrument in real time to perform real-time intervention movement according to the magnetic control intervention basic virtual path;
acquiring real-time physical coordinates of the magnetic control intervention guide wire in the process of performing real-time intervention movement along a virtual path of a magnetic control intervention foundation by using a DSA system, acquiring a real-time DSA image of the magnetic control intervention guide wire at the real-time physical coordinates, and simultaneously transmitting the real-time DSA image and the real-time physical coordinates to a blood vessel shape-moving digital twin platform through an interactive channel;
the vessel shape-changing digital twin platform obtains angiography CT images at virtual coordinates corresponding to the real-time physical coordinates in the virtual vessel shape-changing three-dimensional model according to the real-time physical coordinates, and marks the angiography CT images as priori CT images;
the vessel shape-shifting digital twin platform utilizes a twin neural network to carry out real-time detection on the suitability of path planning based on a real-time DSA image and a priori CT image, so as to obtain the suitability of path planning;
the vessel shape-moving digital twin platform carries out the identification of the update sites on the virtual vessel shape-moving three-dimensional model according to the path planning applicability, and the update sites in the virtual vessel shape-moving three-dimensional model are obtained;
At an update site in the virtual vessel shape three-dimensional model, updating the virtual vessel shape three-dimensional model by using a real-time DSA image corresponding to the update site;
the vessel shape-changing digital twin platform is used for planning a magnetic control intervention updating virtual path at an updating site and a magnetic control intervention terminal point by utilizing a path planning algorithm based on the updated virtual vessel shape-changing three-dimensional model;
the vessel shape-moving digital twin platform feeds the magnetic control intervention updating virtual path back to the magnetic navigation instrument in real time through the interaction channel, and the magnetic intervention guide wire is controlled by the magnetic navigation instrument in real time to perform real-time intervention movement according to the magnetic control intervention updating virtual path.
According to the invention, the virtual construction of the digital twin platform is carried out according to the magnetic intervention guide wire and the blood vessel shape-changing three-dimensional model, so that the blood vessel shape-changing digital twin platform comprising the virtual magnetic intervention guide wire and the virtual blood vessel shape-changing three-dimensional model is obtained, a great amount of tests can be carried out in a virtual space close to a real environment in a development and test stage of magnetic intervention path planning, the safety of the magnetic intervention guide wire during intervention movement in the real environment is effectively improved, and the real-time synchronization of the virtual space and the physical space is realized, so that the magnetic intervention guide wire can obtain the intervention path update which is suitable for the real structure of the blood vessel in real time from the virtual space of the digital twin platform, the planning of the intervention path and the magnetic control of intervention running are separated, the mutual occupation of resources is avoided, the airborne computer pressure of the magnetic intervention navigator is effectively reduced, the planning accuracy and the safety of the magnetic intervention path are also improved.
The invention not only can realize more accurate interventional operation, but also can greatly reduce the radiation contact of interventional medical staff.
The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application, the scope of which is defined by the claims. Various modifications and equivalent arrangements may be made to the present application by those skilled in the art, which modifications and equivalents are also considered to be within the scope of the present application.

Claims (10)

1. The magnetic control intervention control method based on artificial intelligence and CT is characterized by comprising the following steps:
acquiring an angiographic CT image;
calibrating the vessel shape: performing calibration processing on image pixels representing blood vessels in an angiography CT image according to blood vessel shape variation through an artificial intelligent identification model to obtain a blood vessel shape variation calibration image;
reconstructing a three-dimensional model of the blood vessel shape: performing physical reconstruction of a three-dimensional space based on the blood vessel shape-shifting calibration image by a three-dimensional reconstruction technology to obtain a blood vessel shape-shifting three-dimensional model;
constructing a vessel shape-changing digital twin platform: performing virtual construction of a digital twin platform according to the magnetic intervention guide wire and the blood vessel shape-changing three-dimensional model by using a Gaussian-Kriging projection method to obtain the blood vessel shape-changing digital twin platform comprising the virtual magnetic intervention guide wire and the virtual blood vessel shape-changing three-dimensional model;
Based on a vessel shape-changing digital twin platform, a DSA system is utilized to build a virtual magnetic control intervention path of a virtual magnetic intervention guide wire in real time in a virtual vessel shape-changing three-dimensional model;
the vessel shape-moving digital twin platform feeds back the virtual magnetic control intervention path to the magnetic navigation instrument in real time, and the magnetic intervention guide wire is controlled by the magnetic navigation instrument in real time to perform real-time intervention movement according to the virtual magnetic control intervention path.
2. The magnetic control intervention control method based on artificial intelligence and CT as set forth in claim 1, wherein: the calibration method for the blood vessel shape calibration image comprises the following steps:
randomly selecting a plurality of angiography CT images as sample images;
marking image pixels representing blood vessels in the sample image as target pixels;
taking the sample image as an input item of a YOLO V3 neural network, taking the target pixel as an output item of the YOLO V3 neural network, and carrying out mapping learning on the input item of the YOLO V3 neural network and the output item of the YOLO V3 neural network by utilizing the YOLO V3 neural network to obtain the artificial intelligent recognition model;
identifying image pixels representing blood vessels in each angiography CT image by using the artificial intelligent model, and smoothly linking the image pixels representing the blood vessels by using a B-spline curve smoothing method to obtain calibration lines representing the shape of the blood vessels;
And taking the angiography CT image with the calibration lines as the blood vessel shape-changing calibration image.
3. The magnetic control intervention control method based on artificial intelligence and CT as claimed in claim 2, wherein the method comprises the following steps: the reconstruction of the three-dimensional model of the blood vessel shape is realized by using a three-dimensional reconstruction technology 3DMax, and a magnetic control intervention starting point and a magnetic control intervention end point are marked on the three-dimensional model of the blood vessel shape by using the three-dimensional reconstruction technology 3 DMax.
4. A magnetic control intervention control method based on artificial intelligence and CT as claimed in claim 3, wherein: the method for building the vessel shape-changing digital twin platform comprises the following steps:
developing a digital twin platform by utilizing Unity3D, and loading and displaying the magnetic intervention guide wire and blood vessel shape-changing three-dimensional model in the digital twin platform to obtain the virtual magnetic intervention guide wire and virtual blood vessel shape-changing three-dimensional model;
mapping physical coordinates of the magnetic intervention guide wire to virtual coordinates of the virtual magnetic intervention guide wire in the digital twin platform by using a Gaussian-Kriging projection method, and mapping physical coordinates of the blood vessel shape-moving three-dimensional model to virtual coordinates of the virtual blood vessel shape-moving three-dimensional model in the digital twin platform;
Establishing an interaction channel for data interaction between the magnetic intervention physical guide wire and the virtual magnetic intervention guide wire;
and constructing the vessel shape-changing digital twin platform.
5. The magnetic control intervention control method based on artificial intelligence and CT as set forth in claim 4, wherein: the method for carrying out coordinate mapping on physical coordinates of the magnetic intervention guide wire and the blood vessel shape-changing three-dimensional model by using the Gauss-Gauss projection method comprises the following steps:
selecting a plurality of first control points from a blood vessel shape three-dimensional model, recording three-dimensional coordinates of the first control points, finding positions of the selected first control points corresponding to observation points in a real environment, recording longitude and latitude coordinates of the observation points, and converting the longitude and latitude coordinates containing height information recorded by each observation point into three-dimensional coordinates in a Cartesian coordinate system through Gauss-Krueger projection orthometric formula projection;
estimating coordinate system deviation of a blood vessel shape three-dimensional model center point from longitude and latitude coordinate projection conversion to a Cartesian coordinate system origin by using a least square method according to the three-dimensional coordinates of a plurality of first control points and the three-dimensional coordinates of the corresponding observation points after the longitude and latitude coordinate projection conversion;
the three-dimensional coordinates of each model point in the physical space where the three-dimensional model of the blood vessel is located are obtained by adding the three-dimensional coordinates obtained by projection conversion of the longitude and latitude coordinates of each model point in the physical space where the three-dimensional model of the blood vessel is located through a Gaussian-Kelvin projection orthometric formula and the deviations of the coordinate system, and the three-dimensional coordinates of the corresponding point in the virtual space where the three-dimensional model of the virtual blood vessel is located are obtained under the Cartesian coordinate system;
Subtracting the coordinate system deviation from the three-dimensional coordinates of each model point in the virtual space where the virtual vessel shape three-dimensional model is located under the Cartesian coordinate system, and obtaining mathematical correlation of longitude and latitude coordinates of corresponding points in the physical space where the vessel shape three-dimensional model is located through the projection conversion of a Gaussian-Kelvin projection inverse solution formula, so that the physical coordinates of the vessel shape three-dimensional model are mapped to the virtual coordinates of the virtual vessel shape three-dimensional model in the digital twin platform;
selecting a plurality of second control points from the magnetic interventional guide wire, recording three-dimensional coordinates of the second control points, finding positions of the selected second control points corresponding to the observation points in a real environment, recording longitude and latitude coordinates of the observation points, and converting the longitude and latitude coordinates containing the height information recorded by each observation point into three-dimensional coordinates in a Cartesian coordinate system through Gauss-Krueger projection orthometric formula projection;
estimating coordinate system deviation of the magnetic intervention guide wire center point from the longitude and latitude coordinate projection conversion to the origin of a Cartesian coordinate system by using a least square method according to the three-dimensional coordinates of the plurality of second control points and the three-dimensional coordinates of the corresponding observation points after the longitude and latitude coordinate projection conversion;
The three-dimensional coordinates of each model point in the physical space where the magnetic interventional guide wire is located are obtained by adding the three-dimensional coordinates obtained by projection conversion of the Gauss-Gauss projection orthometric formula and the deviations of the coordinate system, and the three-dimensional coordinates of the corresponding point in the virtual space where the virtual magnetic interventional guide wire is located are obtained under the Cartesian coordinate system;
after the coordinate system deviation is subtracted from the three-dimensional coordinate of each model point in the virtual space where the magnetic intervention guide wire is located under the Cartesian coordinate system, the mathematical correlation of the longitude and latitude coordinates of the corresponding point in the physical space where the magnetic intervention guide wire is located is obtained after the projection conversion of the Gaussian-Kelvin projection inverse solution formula, and the mapping of the physical coordinate of the magnetic intervention guide wire to the virtual coordinate of the virtual magnetic intervention guide wire in the digital twin platform is realized.
6. The magnetic control intervention control method based on artificial intelligence and CT as set forth in claim 5, wherein the magnetic control intervention control method is characterized in that: the method for constructing the virtual magnetic control intervention path in real time comprises the following steps:
the vessel shape-changing digital twin platform is based on a virtual vessel shape-changing three-dimensional model, and a magnetic control intervention basic virtual path is planned at a magnetic control intervention starting point and a magnetic control intervention ending point by utilizing a path planning algorithm;
The vessel shape-moving digital twin platform feeds back the magnetic control intervention basic virtual path to the magnetic navigation instrument in real time through the interaction channel, and the magnetic intervention guide wire is controlled by the magnetic navigation instrument in real time to perform real-time intervention movement according to the magnetic control intervention basic virtual path;
acquiring real-time physical coordinates of the magnetic control intervention guide wire in the process of performing real-time intervention movement along a virtual path of a magnetic control intervention foundation by using a DSA system, acquiring a real-time DSA image of the magnetic control intervention guide wire at the real-time physical coordinates, and simultaneously transmitting the real-time DSA image and the real-time physical coordinates to a blood vessel shape-moving digital twin platform through an interactive channel;
according to the real-time physical coordinates, the vessel shape-changing digital twin platform acquires angiography CT images at virtual coordinates corresponding to the real-time physical coordinates in the virtual vessel shape-changing three-dimensional model, and marks the angiography CT images as priori CT images;
the vessel shape-shifting digital twin platform utilizes a twin neural network to carry out real-time detection on the suitability of path planning based on a real-time DSA image and a priori CT image, so as to obtain the suitability of path planning;
the vessel shape-moving digital twin platform carries out the identification of the update sites on the virtual vessel shape-moving three-dimensional model according to the path planning applicability, and the update sites in the virtual vessel shape-moving three-dimensional model are obtained;
At an update site in the virtual vessel shape three-dimensional model, updating the virtual vessel shape three-dimensional model by using a real-time DSA image corresponding to the update site;
the vessel shape-changing digital twin platform is used for planning a magnetic control intervention updating virtual path at an updating site and a magnetic control intervention terminal point by utilizing a path planning algorithm based on the updated virtual vessel shape-changing three-dimensional model;
the vessel shape-moving digital twin platform feeds the magnetic control intervention updating virtual path back to the magnetic navigation instrument in real time through the interaction channel, and the magnetic intervention guide wire is controlled by the magnetic navigation instrument in real time to perform real-time intervention movement according to the magnetic control intervention updating virtual path.
7. The magnetic control intervention control method based on artificial intelligence and CT as set forth in claim 6, wherein: the method for detecting the applicability of the path planning comprises the following steps:
inputting the real-time DSA image into a first CNN network structure in the twin neural network, and outputting the vascular characteristics in the real-time DSA image by the first CNN network structure in the twin neural network;
inputting the real-time CT image into a second CNN network structure in the twin neural network, and outputting vessel characteristics in the prior CT image by the second CNN network structure in the twin neural network;
Detecting path planning applicability by using a loss function of a twin neural network, wherein the path planning applicability has the following expression: match_gold= - (1-Low) r log (Loss); in the formula, match_gold is path planning applicability, loss is a Loss function of the twin neural network, and r is an artificial regulation parameter;
the artificial regulation parameters are used for adding artificial will in the updating of the virtual vessel shape-changing three-dimensional model;
wherein, the loss function of the twin neural network is:
loss=mse (out 1, out 2); where Loss is a Loss function, out1 is an output of a first CNN network structure in the twin neural network, out2 is an output of a second CNN network structure in the twin neural network, MSE is a mean square error function, and MSE (out 1, out 2) is a mean square error between out1 and out 2.
8. The magnetic control intervention control method based on artificial intelligence and CT as set forth in claim 7, wherein: the method for identifying the update sites in the virtual blood vessel shape-changing three-dimensional model comprises the following steps:
comparing the path planning suitability with a preset threshold, wherein,
when the path planning applicability is greater than or equal to a preset threshold, calibrating virtual coordinates corresponding to the prior CT image as update sites in the virtual vessel shape three-dimensional model;
And when the path planning applicability is smaller than a preset threshold, calibrating the virtual coordinates corresponding to the prior CT image as non-updated sites in the virtual blood vessel walking three-dimensional model.
9. The magnetic control intervention control method based on artificial intelligence and CT as set forth in claim 8, wherein: the method for updating the virtual blood vessel shape three-dimensional model by using the corresponding real-time DSA image at the updating site comprises the following steps:
and replacing the blood vessel characteristics of the corresponding real-time DSA image at the update site with the blood vessel characteristics of the corresponding priori CT image at the update site in the virtual blood vessel shape three-dimensional model to obtain an updated virtual blood vessel shape three-dimensional model.
10. A magnetic control intervention control system based on artificial intelligence and CT, characterized in that it is applied to a magnetic control intervention control method based on artificial intelligence and CT as described in any one of claims 1 to 9, and the magnetic control intervention control system comprises:
the image acquisition unit is used for acquiring angiography CT images and DSA images;
the image processing unit is used for calibrating image pixels representing blood vessels in the angiography CT image according to blood vessel shape variation through the artificial intelligent recognition model to obtain a blood vessel shape variation calibration image;
The three-dimensional reconstruction unit is used for carrying out physical reconstruction of a three-dimensional space based on the blood vessel shape-shifting calibration image by a three-dimensional reconstruction technology to obtain a blood vessel shape-shifting three-dimensional model;
the platform building unit is used for virtually building the digital twin platform according to the magnetic intervention guide wire and the blood vessel shape-shifting three-dimensional model by utilizing a Gaussian-Krueger projection method to obtain the blood vessel shape-shifting digital twin platform comprising the virtual magnetic intervention guide wire and the virtual blood vessel shape-shifting three-dimensional model;
the platform running unit is connected with the DSA system and is used for building a virtual magnetic control intervention path of the virtual magnetic intervention guide wire in a virtual blood vessel shape three-dimensional model in real time based on the blood vessel shape digital twin platform by utilizing the DSA system;
the magnetic control intervention unit comprises a magnetic navigation instrument, is used for receiving a virtual magnetic control intervention path fed back by the blood vessel shape-changing digital twin platform, and controls the magnetic intervention guide wire to perform real-time intervention movement according to the virtual magnetic control intervention path in real time;
the platform operation unit builds a virtual magnetic control intervention path of a virtual magnetic intervention guide wire in real time in a virtual blood vessel shape-changing three-dimensional model by using a DSA system, and the platform operation unit comprises:
the vessel shape-changing digital twin platform is based on a virtual vessel shape-changing three-dimensional model, and a magnetic control intervention basic virtual path is planned at a magnetic control intervention starting point and a magnetic control intervention ending point by utilizing a path planning algorithm;
The vessel shape-moving digital twin platform feeds back the magnetic control intervention basic virtual path to the magnetic navigation instrument in real time through the interaction channel, and the magnetic intervention guide wire is controlled by the magnetic navigation instrument in real time to perform real-time intervention movement according to the magnetic control intervention basic virtual path;
acquiring real-time physical coordinates of the magnetic control intervention guide wire in the process of performing real-time intervention movement along a virtual path of a magnetic control intervention foundation by using a DSA system, acquiring a real-time DSA image of the magnetic control intervention guide wire at the real-time physical coordinates, and simultaneously transmitting the real-time DSA image and the real-time physical coordinates to a blood vessel shape-moving digital twin platform through an interactive channel;
according to the real-time physical coordinates, the vessel shape-changing digital twin platform acquires angiography CT images at virtual coordinates corresponding to the real-time physical coordinates in the virtual vessel shape-changing three-dimensional model, and marks the angiography CT images as priori CT images;
the vessel shape-changing digital twin platform utilizes twin neural network, and is based on real-time DSA image andpriori CTThe image carries out real-time detection on the path planning applicability to obtain the path planning applicability;
the vessel shape-moving digital twin platform carries out the identification of the update sites on the virtual vessel shape-moving three-dimensional model according to the path planning applicability, and the update sites in the virtual vessel shape-moving three-dimensional model are obtained;
At an update site in the virtual vessel shape three-dimensional model, updating the virtual vessel shape three-dimensional model by using a real-time DSA image corresponding to the update site;
the vessel shape-changing digital twin platform is used for planning a magnetic control intervention updating virtual path at an updating site and a magnetic control intervention terminal point by utilizing a path planning algorithm based on the updated virtual vessel shape-changing three-dimensional model;
the vessel shape-moving digital twin platform feeds the magnetic control intervention updating virtual path back to the magnetic navigation instrument in real time through the interaction channel, and the magnetic intervention guide wire is controlled by the magnetic navigation instrument in real time to perform real-time intervention movement according to the magnetic control intervention updating virtual path.
CN202410028563.5A 2024-01-09 2024-01-09 Magnetic control intervention control method and system based on artificial intelligence and CT Active CN117530775B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410028563.5A CN117530775B (en) 2024-01-09 2024-01-09 Magnetic control intervention control method and system based on artificial intelligence and CT

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410028563.5A CN117530775B (en) 2024-01-09 2024-01-09 Magnetic control intervention control method and system based on artificial intelligence and CT

Publications (2)

Publication Number Publication Date
CN117530775A true CN117530775A (en) 2024-02-09
CN117530775B CN117530775B (en) 2024-04-30

Family

ID=89794186

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410028563.5A Active CN117530775B (en) 2024-01-09 2024-01-09 Magnetic control intervention control method and system based on artificial intelligence and CT

Country Status (1)

Country Link
CN (1) CN117530775B (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111784751A (en) * 2020-06-16 2020-10-16 北京理工大学 3D/2D registration-based guide wire 3D simulation tracking method and device
CN112155729A (en) * 2020-10-15 2021-01-01 中国科学院合肥物质科学研究院 Intelligent automatic planning method and system for surgical puncture path and medical system
CN112270740A (en) * 2020-10-10 2021-01-26 孟自力 Guide wire training model for interventional operation and construction method thereof
US20210145525A1 (en) * 2019-11-15 2021-05-20 Magic Leap, Inc. Viewing system for use in a surgical environment
WO2022048984A1 (en) * 2020-09-02 2022-03-10 Koninklijke Philips N.V. Medical intervention control based on device type identification
KR102398522B1 (en) * 2021-02-08 2022-05-17 광주과학기술원 Image registration method for coronary artery intervention and electronic device performing the same
CN115005981A (en) * 2022-06-07 2022-09-06 武汉联影智融医疗科技有限公司 Surgical path planning method, system, equipment, medium and surgical operation system
CN115227394A (en) * 2022-07-07 2022-10-25 大连理工大学 Robot minimally invasive vascular interventional operation danger early warning method based on digital twinning
US20220378514A1 (en) * 2021-05-25 2022-12-01 The Asan Foundation Device and method for detecting guidewire
CN116196099A (en) * 2023-02-23 2023-06-02 同济大学 Cardiovascular intervention operation path planning method, system, storage medium and terminal

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210145525A1 (en) * 2019-11-15 2021-05-20 Magic Leap, Inc. Viewing system for use in a surgical environment
CN111784751A (en) * 2020-06-16 2020-10-16 北京理工大学 3D/2D registration-based guide wire 3D simulation tracking method and device
WO2022048984A1 (en) * 2020-09-02 2022-03-10 Koninklijke Philips N.V. Medical intervention control based on device type identification
CN112270740A (en) * 2020-10-10 2021-01-26 孟自力 Guide wire training model for interventional operation and construction method thereof
CN112155729A (en) * 2020-10-15 2021-01-01 中国科学院合肥物质科学研究院 Intelligent automatic planning method and system for surgical puncture path and medical system
KR102398522B1 (en) * 2021-02-08 2022-05-17 광주과학기술원 Image registration method for coronary artery intervention and electronic device performing the same
US20220378514A1 (en) * 2021-05-25 2022-12-01 The Asan Foundation Device and method for detecting guidewire
CN115005981A (en) * 2022-06-07 2022-09-06 武汉联影智融医疗科技有限公司 Surgical path planning method, system, equipment, medium and surgical operation system
CN115227394A (en) * 2022-07-07 2022-10-25 大连理工大学 Robot minimally invasive vascular interventional operation danger early warning method based on digital twinning
CN116196099A (en) * 2023-02-23 2023-06-02 同济大学 Cardiovascular intervention operation path planning method, system, storage medium and terminal

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
李在娟等: "基于路径的血管介入手术电磁跟踪的配准算法", 华中科技大学学报(自然科学版), vol. 41, no. 1, 31 January 2014 (2014-01-31), pages 316 - 319 *
杨健等: "DSA血管三维重建技术分析与展望", 中国生物医学工程学报, vol. 24, no. 06, 31 December 2005 (2005-12-31), pages 655 - 660 *

Also Published As

Publication number Publication date
CN117530775B (en) 2024-04-30

Similar Documents

Publication Publication Date Title
KR101864380B1 (en) Surgical image data learning system
KR102250164B1 (en) Method and system for automatic segmentation of vessels in medical images using machine learning and image processing algorithm
CN111680447A (en) Blood flow characteristic prediction method, blood flow characteristic prediction device, computer equipment and storage medium
KR20190088375A (en) Surgical image data learning system
CN111862046B (en) Catheter position discrimination system and method in heart coronary wave silhouette
CN115005981A (en) Surgical path planning method, system, equipment, medium and surgical operation system
CN111612778B (en) Preoperative CTA and intraoperative X-ray coronary artery registration method
CN115345938B (en) Global-to-local-based head shadow mark point positioning method, equipment and medium
CN112734776A (en) Minimally invasive surgical instrument positioning method and system
CN113662573A (en) Mammary gland focus positioning method, device, computer equipment and storage medium
CN117530775B (en) Magnetic control intervention control method and system based on artificial intelligence and CT
CN111784751B (en) 3D/2D registration-based guide wire 3D simulation tracking method and device
CN115689971A (en) Pedicle screw implantation channel planning method and device based on deep learning
CN113066111A (en) Automatic positioning method for cardiac mitral valve vertex based on CT image
CN113077499A (en) Pelvis registration method, pelvis registration device and pelvis registration system
CN116350346A (en) Space motion trail planning method for robot based on space scanning digital twin
EP4270313A1 (en) Registering projection images to volumetric images
Gil et al. Intraoperative extraction of airways anatomy in videobronchoscopy
KR102250173B1 (en) Method and system for automatic segmentation of vessels in medical images using machine learning and image processing algorithm
CN112085698A (en) Method and device for automatically analyzing left and right breast ultrasonic images
CN116831730A (en) Real-time puncture guiding evaluation system based on soft tissue deformation field
CN111627554A (en) Fracture image automatic classification system based on deep convolutional neural network
CN117994346A (en) Digital twinning-based puncture instrument detection method, system and storage medium
US20220230319A1 (en) System and method for training a machine learning model and for providing an estimated interior image of a patient
CN113012036B (en) Human motion style migration method and system based on generative flow model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant