CN113796992A - Automatic valve stent conveying system based on image segmentation - Google Patents
Automatic valve stent conveying system based on image segmentation Download PDFInfo
- Publication number
- CN113796992A CN113796992A CN202111130985.6A CN202111130985A CN113796992A CN 113796992 A CN113796992 A CN 113796992A CN 202111130985 A CN202111130985 A CN 202111130985A CN 113796992 A CN113796992 A CN 113796992A
- Authority
- CN
- China
- Prior art keywords
- valve
- delivery device
- stent
- conveying device
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
- A61F2/00—Filters implantable into blood vessels; Prostheses, i.e. artificial substitutes or replacements for parts of the body; Appliances for connecting them with the body; Devices providing patency to, or preventing collapsing of, tubular structures of the body, e.g. stents
- A61F2/02—Prostheses implantable into the body
- A61F2/24—Heart valves ; Vascular valves, e.g. venous valves; Heart implants, e.g. passive devices for improving the function of the native valve or the heart muscle; Transmyocardial revascularisation [TMR] devices; Valves implantable in the body
- A61F2/2442—Annuloplasty rings or inserts for correcting the valve shape; Implants for improving the function of a native heart valve
- A61F2/2466—Delivery devices therefor
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/30—Surgical robots
- A61B34/35—Surgical robots for telesurgery
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A61B2034/101—Computer-aided simulation of surgical operations
Abstract
The invention relates to an automatic valve stent conveying system based on image segmentation, which consists of the following 3 parts: remote control software, a mechanical arm and a support conveying device; the remote control software comprises a parameter setting page, an ultrasonic image preselection mark page and a process control page, and remote connections are respectively established with the conveying device and the mechanical arm through Wi-Fi; the six-axis mechanical arm is adopted as the mechanical arm, and the position of the valve support is adjusted in the operation process; the stent delivery device is mainly used for carrying the compressed valve stent into the heart to complete release and fixation. Its advantages are: the device can monitor the position of the conveying device, the activity state of the mitral valve, the release stage of the artificial stent and the like in real time, feed corresponding information back to the controller and the mechanical arm, complete the implantation and release of the cardiac valve stent in a highly automatic and staged manner, solve the problems of low controllability, accuracy and safety of the existing valve conveying system, and protect operators from radiation hazard.
Description
Technical Field
The invention relates to the technical field of medical instruments, in particular to an automatic valve stent conveying system based on image segmentation.
Background
Mitral insufficiency (MR) is the most common valvular disease, with an incidence of about 1.7% in the general population. Statistically, about 2000 million people suffering from severe mitral insufficiency in China are present. Severe MR will severely impair cardiac function, increasing the risk of heart failure, thereby threatening the health and longevity of the person. In severe MR patients without surgery, mortality is as high as 50% within 5 years, and 90% of surviving patients will progress to the stage of heart failure.
In current clinical practice, the focus of research is transcatheter mitral valve placement (TMVI). TMVI is a minimally invasive surgery in which a prosthetic valve is placed into the heart with the aid of an interventional technique to complete valve replacement, and has the characteristics of small trauma and quick recovery. Among them, the delivery system is an important part for TMVI operation, and mainly plays a role in carrying the compressed valve stent into the heart to complete release and fixation. The quality of the delivery system determines the success or failure of the TMVI procedure.
In the prior art, the TMVI procedure is fully manual. Before operation, the doctor performs mitral valve annulus analysis through CT to determine the best approach and related information. During the operation, the doctor needs to perform valve implantation under the guidance of left ventricular esophageal ultrasonography, and meanwhile, the doctor needs to perform judgment by means of Digital Subtraction Angiography (DSA). However, when valve replacement is performed under DSA, the operator must be exposed to radiation for a long period of time, which is harmful to health. In addition, the process of implanting the mitral valve stent is complex, the manual operation easily causes the problems of unstable control and inaccurate positioning, and the prior art cannot achieve pre-adjustment due to different anatomical parameters of patients.
Chinese patent documents: CN201721010385.5, application date 20170814, patent names: a mitral valve stent delivery system. Disclosed is a mitral valve stent delivery system comprising: a handle provided with a first button; the outer tube is movably arranged on the handle and is connected with the first button, and the first button controls the outer tube to slide on the handle; the inner pipe is positioned in the outer pipe, the first end of the inner pipe is fixed on the handle, and a gap is formed between the inner pipe and the outer pipe; a tapered tip disposed at the second end of the inner tube; valve support connection structure, including a plurality of bar structures, bar structure's first end is fixed on interior pipe, and the second end is the free end, and its free end department is equipped with the clamping part that is used for fixed mitral valve support.
In the mitral valve stent delivery system in the patent document, when the first button is in an open state, the valve stent connecting structure exposes out of the outer tube, and the free end of the strip-shaped structure is in a state of being bent in a direction away from the inner tube, so that the delivery system of the utility model can effectively ensure the stable loading of the valve stent and accurately release the valve stent in the process of delivering the valve stent, and effectively improve the success rate of surgery; however, the valve stent delivery system is still operated manually, so that the problems of unstable control and inaccurate positioning are easily caused. The present applicant filed an invention with a patent name (publication No. 2021.07.16) of: an electric bracket conveying device based on remote control. This electric stent conveying system solves on the present mitral valve replacement technique, the operation is complicated, the precision is low, the problem that the operation failure rate is high, practically reduce the patient wound, reduce the art person degree of difficulty, improve the operation success rate, reduce postoperative complication, can simplify the operation process, guarantee accurate positioning and accurate release that the mitral valve was put into, make the contribution for the development of mitral valve replacement technique, but it needs manual operation equally in operation process, the valve support that does not reach automatic level is carried.
In summary, there is a need for an automated system for delivering a valve stent based on image segmentation, which can monitor the position of a delivery device, the activity of a mitral valve, the release stage of a stent prosthesis, etc. in real time, and feed corresponding information back to a controller and a mechanical arm, so as to complete the insertion and release of the valve stent in a highly automated and staged manner, solve the problems of low controllability, accuracy and safety of the existing valve delivery system, and simultaneously protect the operator from radiation hazard. There are no reports on this system.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, and provides an automatic valve stent conveying system based on image segmentation, which can monitor the position of a conveying device, the activity state of a mitral valve, the release stage of a manual stent and the like in real time, feed corresponding information back to a controller and a mechanical arm, complete the implantation and release of a heart valve stent in a highly automatic and staged manner, solve the problems of low controllability, accuracy and safety of the conventional valve conveying system, and protect operators from radiation hazards.
In order to achieve the purpose, the invention adopts the technical scheme that:
an automatic valve stent delivery system based on image segmentation is composed of the following 3 parts: remote control software, a mechanical arm and a support conveying device;
the remote control software mainly has 2 functions; firstly, setting before operation; the operator is allowed to preset surgical parameters according to the individual condition of each patient and to set a pre-selected frame required for image segmentation according to the identified object. Secondly, intervention in operation; because the surgical procedure is highly automated when using the system, remote control software provides the possibility for manual intervention; an echocardiogram and DSA image monitoring window is arranged in the interface, so that an operator can find out an accident situation in time in the operation process, and carry out operations such as pause, adjustment and the like, thereby enhancing the controllability and safety of the operation;
the mechanical arm is used for clamping the stent conveying device and can adjust the position of the valve stent in the operation process;
the stent delivery device mainly plays a role in carrying the compressed valve stent into the heart to complete release and fixation.
The embodiment of the specification provides a method for conveying an automatic valve stent conveying system based on image segmentation, which comprises the following steps:
analyzing required information from two-dimensional and three-dimensional DICOM data input by an external ultrasonic device, and displaying a two-dimensional echocardiogram and a three-dimensional echocardiogram in a control interface of remote control software in real time;
before operation, the position of the ultrasonic probe needs to be adjusted, and a long-axis section of the left atrium beside the sternum is adopted, so that the anterior cuspid valve and the posterior cuspid valve of the mitral valve are completely displayed, and the top of the left atrium can be more completely seen;
after the operation is started, making a small chest wall incision (2-3cm), cutting the pericardium, performing purse-string suture at the apex of the heart, performing apex puncture by using a puncture sheath tube, placing a soft guide wire, reversely crossing the mitral valve to enter the left atrium, establishing a delivery passage, replacing the soft guide wire with a hard guide wire, and placing a delivery device into the left heart along the guide wire; the operator can then leave the operating table, enter a remote control room, and perform remote fine manipulation of the valve replacement.
After entering the remote control room, the operator needs to set the personalized parameters of the patient on the parameter setting page of the remote control software, so that the operation process can be adjusted according to the personal condition of the patient; in addition, the surgeon must set pre-selection boxes that select the approximate locations of the left atrium, the anterior cusp of the mitral valve, the posterior cusp of the mitral valve, and the delivery device. The preselected boxes are oval shaped, allowing for overlap between the preselected boxes.
Further, the C-V model is automatically adopted to process the two-dimensional echocardiogram in real time.
Further, the first stage of the conveyor set-in is entered. According to the processing result, when the head end of the stent conveying device is far away from the mitral valve, the controller controls the conveying device to move forwards at a higher speed; the first stage of delivery device placement ends when the tip of the stent delivery device is positioned near the mitral valve.
Further, entering a second stage of placing the conveying device; according to the processing result, when the mitral valve is relaxed, the stent delivery device moves forwards with small amplitude; when the mitral valve contracts, the stent delivery device stops moving; when the head end of the stent delivery device reaches the target location in the left atrium, the second stage of delivery device placement is ended;
after the conveying device is placed in, the identification of the two-dimensional echocardiogram is suspended; the operator must manually frame the mitral annulus and the delivery device tip in a three-dimensional echocardiogram so that the system automatically identifies the exact contour of the mitral annulus and the exact contour of the delivery device tip.
Further, the three-dimensional echocardiogram is automatically processed in real time by adopting a C-V model.
Further, the prosthetic valve is adjusted to an appropriate release position according to the processing result, and information is sent to the controller of the delivery device to release the atrial side of the valve. After the release is finished, the conveying device moves backwards to enable the artificial valve to be tightly attached to the original valve leaf of the mitral valve.
Further, the information is sent to a controller, and the delivery device is controlled to completely release the valve;
after valve release is complete, identification of the three-dimensional echocardiogram is suspended. The operator must reset the pre-selection frame on the two-dimensional echocardiogram, and frame the delivery device, the anterior cusp valve and the posterior cusp valve of the artificial valve respectively.
Further, the C-V model is automatically adopted to process the two-dimensional echocardiogram in real time.
Further, a first phase of withdrawal of the delivery device is entered; according to the processing result, when the head end of the stent delivery device is not completely withdrawn from the left atrium, the stent delivery device moves backwards with a small amplitude; the first phase of delivery device withdrawal is complete when the tip of the stent delivery device is fully withdrawn from the left atrium.
Further, a second stage of withdrawal of the delivery device; according to the processing result, the controller controls the conveying device to move backwards at a higher speed until the conveying device is completely withdrawn;
after the delivery device is completely withdrawn from the heart, the automatic delivery process is ended; the operator enters the operating room from the remote control room, and the operation can be completed after the pressurizing and suturing treatment is carried out on the incision of the cardiac apex.
The invention has the advantages that:
1. remote control, protection operator's safety: the operator is allowed to perform related operations in the remote control room through remote control software, and the operator is prevented from being in the radiation environment of the operating room for a long time.
2. Accurate release, improve the success rate of operation: different delivery strategies are designed according to different stages of the valve delivery process, and accurate automatic release is realized by matching with the heart pulsation by means of an image segmentation technology.
3. Reasonable manual intervention, and improvement of operation reliability and safety: a manual control mode is designed in remote control software, and a manual control panel is arranged, so that a doctor is allowed to remotely control manually, and accidents are prevented.
4. Based on an image segmentation technology, the highly automated TMVI operation is realized, the problems of low controllability, accuracy and safety of the existing valve conveying system are solved, meanwhile, operators are protected from radiation hazard, and remote operation becomes possible; in particular, the system may also be applied to other valve replacement procedures, as well as interventional procedures for vascular surgery and hepatobiliary surgery, among others.
Drawings
Fig. 1 is a schematic view of a parameter setting page of remote control software of an automated valve stent delivery system based on image segmentation according to the present invention.
Fig. 2 is a schematic diagram of an ultrasound image pre-selection mark page of the remote control software of the automated valve stent delivery system based on image segmentation provided by the invention.
Fig. 3 is a schematic process control page of the remote control software of the automated valve stent delivery system based on image segmentation according to the present invention.
Fig. 4 is a flowchart of a method for delivering an automated valve stent delivery system based on image segmentation according to the present invention.
Fig. 5 is a flow chart of the real-time identification of the preselected mark object based on the C-V model provided by the invention.
FIG. 6 is a flow chart of an algorithm of a C-V model according to the present invention.
Fig. 7 is a schematic parameter definition diagram (1) of a heart implantation process of a delivery device according to the present invention.
Fig. 8 is a parameter definition diagram (2) of a heart implantation process of a delivery device according to the present invention.
Fig. 9 is a schematic diagram illustrating parameter definition of a position adjustment process of a conveying device according to the present invention.
Fig. 10 is a parameter definition diagram of a process of withdrawing the heart from the delivery device provided by the invention.
Detailed Description
The invention is further described with reference to the following examples and with reference to the accompanying drawings.
Example 1
In order to make those skilled in the art better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification. It is to be understood that the described embodiments are merely a few embodiments of the present application and not all embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any inventive step based on the embodiments of the present disclosure, shall fall within the scope of protection of the present application.
The automatic valve stent conveying system based on image segmentation provided by the specification consists of the following 3 parts: remote control software, a mechanical arm and a support conveying device;
the remote control software is compiled by adopting a C # language by taking Visual Studio 2019 as a development platform; the software respectively establishes remote connection with the conveying device and the mechanical arm through Wi-Fi and carries out automatic or manual motion control; the automatic control is based on image segmentation, and a C-V (Chan-Vese) algorithm is adopted, so that the real-time monitoring of the position of a conveying device, the activity state of a mitral valve, the release stage of an artificial stent and the like is realized; the software is mainly composed of three pages: the method comprises the following steps of (1) controlling a page, an ultrasonic image preselection mark page and a parameter setting page;
referring to fig. 1, on the parameter setting page, the operator can set the ideal moving speed and the critical stopping distance of the transportation device at different transportation stages according to experience and preoperative analysis; after all the parameters are set, clicking 'save' to ensure that the setting is successful, and operating the system according to the group of parameters until the next modification; in addition, the operator can save the parameters set at this time as default parameters through a 'set default' button, and automatically fill all the parameters through a 'restore default' button when the next operation is performed;
as shown in fig. 2, in the ultrasound image preselection mark page, when the ultrasound data is accessed, the page displays the image captured by the ultrasound probe in real time; before the identification process is started, the operator can move and mark the preselection frames of the left atrium, the anterior mitral valve of the mitral valve, the posterior mitral valve of the mitral valve and the delivery device through a mouse; after marking is finished, clicking a 'confirmation' button below to submit; if drawing errors occur in the midway, a 'redrawing' button can be clicked to mark again;
as shown in fig. 3, the control page can monitor the surgical procedure and allow the operator to properly intervene in the surgical procedure to avoid the occurrence of unexpected dangerous situations and ensure the safety and reliability of the surgical procedure; in the monitoring window at the upper left corner, a two-dimensional echocardiogram, a three-dimensional echocardiogram or a DSA image can be selected to be viewed; wherein, the two-dimensional echocardiogram and the three-dimensional echocardiogram can select whether to display the identification result or not; in the state detection module at the upper right corner, the connection of the mechanical arm, the conveying device and the remote control software can be carried out, and the connection state can be checked; when the connection state shows that the connection is already connected and the working state shows that the connection is ready, the system works normally; when the system is used, the operation process is highly automated, so that the mechanical arm and the conveying device are respectively and independently provided with a manual control button below the page; in the manual control module of the mechanical arm, the plane of the end effector is the plane of the mitral valve annulus, namely the plane vertical to the motion direction of the conveying device; absolute coordinate plane system vertical plane;
the mechanical arm can adopt a common six-axis mechanical arm and is used for clamping the stent conveying device and adjusting the position of the valve stent in the operation process; the six-axis mechanical arm meets six degrees of freedom in a three-dimensional space, and theoretically, the tail end of the six-axis mechanical arm can reach any position and angle in the space, so that the six-axis mechanical arm is suitable for work in almost any track;
the stent conveying device is mainly used for carrying the compressed valve stent into the heart to complete release and fixation;
fig. 4 is a flow chart of the automated valve stent delivery system based on image segmentation provided in the present specification; the process comprises the following steps:
step S101: analyzing required information from two-dimensional and three-dimensional DICOM data input by an external ultrasonic device, and displaying a two-dimensional echocardiogram and a three-dimensional echocardiogram in a control interface of remote control software in real time;
the real-time position of the stent conveying device in the heart can be monitored by a two-dimensional and three-dimensional echocardiogram through an image segmentation method; the two-dimensional echocardiogram, namely utilize the echo signal that the ultrasonic wave reflects from the human body, make up the sectional image in the form of the light spot, can be from two-dimensional space clear, direct-viewing real-time display heart every structural form, spatial position and continuous relation, etc., suitable for the inspection of various cardiovascular diseases; the three-dimensional images reconstructed from the series of two-dimensional images can provide accurate anatomical information, so that the three-dimensional echocardiogram can be used as a supplement of the two-dimensional echocardiogram and provide accurate references in three-dimensional forms, spatial position relations and the like;
DICOM is an international standard protocol for medical image and related information transmission, and is widely applied to cardiovascular imaging and radiation diagnosis and treatment equipment such as ultrasound, X-ray, CT and the like; in the DICOM standard, the composition format and the exchange method of medical images and related information thereof are defined in detail; by using the standard, people can complete the input and output work of image data by establishing an interface on the medical imaging equipment;
in the DICOM data structure standard, a data set is defined to hold all information for a single diagnosis (patient information + image data); depending on the content, the data elements of a DICOM file can be roughly classified into four categories, department, Series, and Image; the Image data comprises information such as the size and height of an Image, and a medical Image picture is stored in a pixel array form, wherein the first byte in the array represents a pixel at the upper left corner of the Image, and the last byte represents a pixel at the lower right corner of the Image;
the remote control software reads and analyzes DICOM data, extracts required information such as image size, height, width, byte number per pixel and the like, and displays the two-dimensional and three-dimensional echocardiograms in a control page in real time;
step S102: acquiring patient personalized parameters set by a preoperative operator so that the operation process can be adjusted according to the personal condition of the patient;
recording the ideal advancing speed of the conveying device in the first stage as V1 and the critical stopping distance as D1; the ideal forward speed of the second stage is V2, and the critical stopping distance is D2; recording the ideal withdrawing speed of the first stage of withdrawing the conveying device as V3 and the critical stopping distance as D3; the ideal withdrawal speed in the second stage is V4; recording the ideal movement speed of the conveying device when the position of the artificial valve is adjusted as V5;
step S201: entering an automatic conveying process; the operator needs to set a preselection frame in the two-dimensional echocardiogram, namely the frame selects the approximate positions of the left atrium, the anterior cusp of the mitral valve, the posterior cusp of the mitral valve and the delivery device; the pre-selection frames are oval, the pre-selection frames are required to be attached to the edge of the target as far as possible, and the pre-selection frames are allowed to be overlapped;
step S202: taking four pre-selection frames marked by an operator as an initialization boundary of a first frame image, and adopting a C-V model to segment the two-dimensional echocardiogram in real time;
the C-V (Chan-Vese) model is a region-based level set method, has high running speed, is insensitive to noise and the state of an initial evolution curve, and has good segmentation effect aiming at images with obviously different pixel average values of a background and a segmentation object; the algorithm adopts a level set method to express an evolution curve, constructs an energy function and solves the Euler-Lagrange equation, namely an evolution equation; finally, the evolution curve begins to evolve under the action of the energy function, and when the value of the energy function reaches the minimum value, the image can be segmented; the specific segmentation steps are as follows:
step S202-1: determining an initial evolution curve;
because the C-V model has the advantage of insensitivity to the initial evolution curve, a preselected frame manually marked by an operator is directly selected as the initial evolution curve of the first frame image; in addition, considering the continuity of the video, for other frames except the first frame, the output result of each frame of image is taken as the initial evolution curve of the next frame of image;
specifically, in order to reduce the number of iterations and improve the accuracy of segmentation, the operator should attach the elliptical preselected frame to the recognition object as much as possible when manually labeling the preselected frame; after the manual marking is finished, whether the marking is complete or has a wrong marking is automatically checked according to the relative position of the preselected frames, and the specific content marked by each preselected frame is determined: according to the human body structure and the placement requirement of the ultrasonic probe, the relative positions are sequentially a left atrium, an anterior cusp valve, a posterior cusp valve and a conveying device from top to bottom and from left to right;
step S202-2: constructing an energy function;
the energy function is composed of a length constraint term, an area constraint term and a fidelity term; the length constraint term and the area constraint term are used for ensuring the smoothness of the evolution curve in the evolution process; the fidelity term is related to the gray values inside and outside the evolution curve and is the power of curve evolution;
in the formula, C represents the evolution curve u0(x, y) represents a gray value at a certain point in the image to be identified, inside (C) represents the inside of the evolution curve, and outside (C) represents the outside of the evolution curve; parameter C1、C2Respectively representing the average gray values, mu, v, lambda, inside and outside the evolution curve1、λ2Coefficients representing a length constraint term, an area constraint term, and a fidelity term, respectively; in general, μ ≧ 0, v ≧ 0, λ are taken1、λ2＞0；
Writing the energy function described above with respect to curve C as an energy function with respect to the level set function Φ (x, y); when (x, y) is located inside the evolution curve, phi (x, y) > 0; when (x, y) is outside the evolution curve, φ (x, y) < 0;
wherein Ω represents an image to be recognized, Hε(z) and deltaε(z) is the regularized hai (Heaviside) function and Dirac (Dirac) function;
minimizing the above energy function yields C1、C2Expression (c):
step S202-3: obtaining an evolution equation;
aiming at the energy function of the C-V model, solving the Euler-Lagrange equation:
φ(O，x，y)＝φ0(x，y)inΩ，
step S202-4: iterative solution and verification;
according to the preset iteration times, repeatedly solving; the contour obtained by iteration is checked, and the percentage of the number of the pixel points in the preselection frame to the total number of the pixel points is counted; if the result is more than 80%, the identification is considered to be correct, otherwise, the iteration is considered to be failed; the frame with the iteration failure is tried to be solved again, and if the iteration failure still occurs, the frame is discarded;
at the moment, the task of identifying a single outline of a single frame image is completed; in the system, each frame of image has a plurality of outlines to be identified, so the outlines in each frame are identified in series according to the sequence from top to bottom and from left to right;
step S203: calculating each parameter of the putting-in process of the conveying device according to the four identification profiles in each frame of image;
specifically, the contour line of the left atrium is recorded as a curve S1, the minimum circumscribed rectangles of the posterior cusp, the anterior cusp and the delivery device are respectively drawn and are respectively recorded as R1, R2 and R3;
identifying 5 measuring points; the lower right corner of the minimum circumscribed rectangle R1 of the posterior cusp is a measuring point 1, the lower left corner of the minimum circumscribed rectangle R2 of the anterior cusp is a measuring point 2, the middle point of the side, close to the atrial end, of the minimum circumscribed rectangle R3 of the conveying device is a measuring point 3, and the middle point of the side, far away from the atrial end, of the minimum circumscribed rectangle R3 of the conveying device is a measuring point 4; determining a straight line according to the measuring point 3 and the measuring point 4, and intersecting the left atrium contour line S1 with two points, wherein the upper point is marked as a measuring point 5;
4 distances are calculated; recording the distance between the measuring point 1 and the measuring point 2 as L1, the distance between the measuring point 3 and the measuring point 5 as L2, the distance between the measuring point 1 and the measuring point 3 as L3 and the distance between the measuring point 2 and the measuring point 3 as L4;
in particular, during the first period of time, the change in size of L1 is constantly calculated and recorded; recording the size of L1 as diastolic distance OL when the mitral valve is in diastole, and recording the size of L1 as systolic distance CL when the mitral valve is in systole;
step S204: entering a first stage of putting in a conveying device; when the head end of the stent conveying device is far away from the mitral valve, the controller controls the conveying device to move forwards at a higher speed; when the head end of the stent delivery device is positioned near the mitral valve, the first stage of delivery device placement is ended;
in this phase, when the distances L3 and L4 are both greater than the critical distance D1 at all times (i.e., whether the valve is in diastole or in systole), information is sent to the controller to control the delivery device to move forward at a constant speed at the speed V1; when detecting that the distance L3 or L4 is less than D1, sending a stop message that the delivery device has reached position 1, is located near the valve, and the first stage of motion is over;
step S205: entering a second stage of placing the conveying device; when the mitral valve is relaxed, the stent delivery device moves forwards with small amplitude; when the mitral valve contracts, the stent delivery device stops moving; when the head end of the stent delivery device reaches the target location in the left atrium, the second stage of delivery device placement is ended;
in the stage, when detecting that L1 is approximately equal to the diastolic distance OL and L2 is greater than the critical distance D2, sending corresponding information to the controller to control the conveying device to move forwards at a constant speed of V2; when it is detected that L1 is approximately equal to the retraction distance CL, the transport apparatus needs to stop moving even if L2 is greater than the threshold distance D2; when it is detected that L2 is less than the threshold distance D2, the conveyor stops moving immediately, regardless of the value of L1, marking the end of the second phase;
step S301: after the conveying device is placed in, the identification of the two-dimensional echocardiogram is suspended; the operator needs to manually frame the mitral valve ring and the head end of the conveying device in the three-dimensional echocardiogram so that the system can automatically identify the accurate contour of the mitral valve ring and the accurate contour of the head end of the conveying device;
adjusting the visual angle of the three-dimensional echocardiogram to enable the mitral valve ring to be parallel to the observation plane, namely the outer sheath of the conveying device is vertical to the observation plane; due to the characteristics of three-dimensional ultrasonic cardiogram three-dimensional imaging, when the valve is in diastole, the outlines of the mitral valve ring and the conveying device are easy to distinguish; therefore, the system firstly judges the activity state of the valve frame by frame, extracts the image frame of the valve diastole state and then identifies the contour of the mitral valve ring and the conveying device;
specifically, the active state of the valve is first determined frame by frame; converting the image into a gray scale image, wherein the operator marks a mitral valve annulus preselection frame as C'; according to the characteristics of the three-dimensional echocardiogram, when the valve is relaxed, the image in C' is darker in color and smaller in gray value; when the valve contracts, the image in C' is lighter in color and larger in gray value; therefore, in one period, the average gray value in C' is gradually increased from minimum to maximum and then decreased to minimum; according to the rule, the average gray value of the pixels C' in one period is averaged over time and is recorded as a reference value E; after the reference value E is determined, calculating the average gray value of the pixels in C' frame by frame and comparing the average gray value with E, and when the gray value is smaller than E, considering that the valve is in a diastole state;
extracting an image frame of a valve diastole state, and segmenting a mitral valve ring contour and a conveying device contour by adopting the C-V model;
step S302: calculating parameters of the coaxial adjustment process of the artificial valve and the mitral valve ring according to the D-shaped mitral valve ring and the head end of the conveying device in each frame of image, and adjusting the position according to the parameters;
identifying 4 measuring points; recording the contour line of the mitral valve ring as a curve S2, and the central pixel point of the head end of the stent conveying device is A; drawing a horizontal line valve-intersected contour line S2 at a measuring point 6 and a measuring point 7 after the point A, and drawing a vertical line valve-intersected contour line S2 at a measuring point 8 and a measuring point 9;
4 distances are calculated; calculating the distance between the point A and the measuring point 6 and recording the distance as L5, calculating the distance between the point A and the measuring point 7 and recording the distance as L6, calculating the distance between the point A and the measuring point 8 and recording the distance as L7, and calculating the distance between the point A and the measuring point 9 and recording the distance as L8;
further, the distance information of the L5, the L6, the L7 and the L8 is sent to the mechanical arm, and the mechanical arm is controlled by remote control software to perform coaxial adjustment of the artificial valve and the mitral valve annulus in a plane (namely, the plane of the mitral valve annulus) which is perpendicular to the moving direction of the conveying device; tying the optimal position for valve release when the prosthetic valve and mitral annulus are coaxial;
step S303: after the artificial valve is adjusted to a proper release position, sending information to a controller of the conveying device to release the atrium side of the valve; after the release is finished, the conveying device moves backwards to enable the artificial valve to be tightly attached to the original valve leaf of the mitral valve;
in this stage, the viewing angle of the three-dimensional echocardiogram is adjusted so that the plane of the mitral annulus is at a certain angle with the observation plane; at this time, the operator needs to manually select the identification frame again, and the identification frame should include the mitral valve annulus and the artificial valve;
solving the pixel variance in the identification frame; when the variance is large, corresponding information is sent to the controller, and the conveying device is controlled to move backwards at a constant speed V5; when the variance is small, traversing all pixel points in the identification frame, if the gray values of all the pixel points are large, considering that the artificial valve and the mitral valve leaflet are tightly attached, and stopping the movement of the conveying device;
step S304: sending the information to a controller, and controlling a conveying device to completely release the valve;
step S401: after the valve release is completed, the identification of the three-dimensional echocardiogram is suspended; the operator needs to reset the preselection frame on the two-dimensional echocardiogram, and respectively frame out the delivery device and the anterior cusp valve and the posterior cusp valve of the artificial valve;
step S402: three pre-selection frames marked by an operator are taken as the initialization boundary of the first frame image, and a C-V model is adopted to process the two-dimensional echocardiogram in real time;
step S403: calculating each parameter of the withdrawing process of the conveying device according to the three identification profiles in each frame of image;
specifically, the minimum circumscribed rectangles of the posterior cusp of the prosthetic valve, the anterior cusp of the prosthetic valve and the delivery device are respectively drawn and are respectively marked as R1 ', R2 ' and R3 ';
identifying 3 measuring points; the lower right corner of the minimum circumscribed rectangle R1 'of the posterior cusp of the prosthetic valve is a measuring point 1', the lower left corner of the minimum circumscribed rectangle R2 'of the anterior cusp of the prosthetic valve is a measuring point 2', and the middle point of the side, close to the atrial end, of the minimum circumscribed rectangle R3 'of the conveying device is a measuring point 3';
calculating 3 distances; recording the distance between the measuring point 1 ' and the measuring point 2 ' as L1 ', the distance between the measuring point 1 ' and the measuring point 3 ' as L3 ', and the distance between the measuring point 2 and the measuring point 3 as L4 ';
in particular, during the initial period of time, the change in size of L1' is constantly calculated and recorded; recording the size of L1 ' when the artificial valve is in diastole as the artificial valve diastole distance OL ', and recording the size of L1 when the artificial valve is in systole as the artificial valve systole distance CL ';
step S404: a first stage of withdrawal into the delivery device; when the head end of the stent delivery device is not completely withdrawn from the left atrium, the stent delivery device moves backwards with a small amplitude; the first phase of delivery device withdrawal ends when the tip of the stent delivery device is fully withdrawn from the left atrium;
in the stage, when detecting that L1 'is approximately equal to the diastolic distance OL', corresponding information is sent to the controller, and the conveying device is controlled to move backwards at a constant speed V3; when it is detected that measuring point 1 ' and measuring point 2 ' are both located above measuring point 3 ' and the sizes of L3 ' and L4 ' are both greater than critical distance D3, it is indicated that the delivery device has completely withdrawn from the left atrium, and the first stage of withdrawal of the delivery device is finished;
step S405: a second stage of withdrawal into the delivery device; in the stage, the controller controls the conveying device to move backwards at a higher speed until the conveying device is completely withdrawn;
in this stage, the controller controls the delivery device to move backward at a constant speed of V4 until the delivery device is no longer detected in the two-dimensional echocardiogram, and the second stage of withdrawal of the delivery device is ended; the automatic conveying process is finished;
the present specification describes examples of use of an automated valve stent delivery system based on image segmentation; in an example, the delivery device automatically performs transcatheter mitral valve placement under guidance of esophageal ultrasound; the automatic operation executed by the ultrasonic guiding and conveying device and the mechanical arm enables an operator to remotely operate, avoids radiation injury, improves the controllability and safety of the operation, and has higher social value.
In the description, the description of the use example of the system is performed according to basic operation steps, and parts with the same principle among the steps are not repeated for avoiding repetition, and reference can be made to the description above; in the system, part of related programming algorithms are implemented at the bottom layer, and the generality of the programming algorithms is considered, so that the programming algorithms are not described in detail, and the requirements are met by adopting a conventional method.
The description is only one specific use example of the system, and the system can also be applied to other valve replacement operations, interventional treatment of vascular surgery and hepatobiliary surgery, and the like.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and additions can be made without departing from the principle of the present invention, and these should also be considered as the protection scope of the present invention.
Claims (10)
1. An automatic valve stent conveying system based on image segmentation is characterized by comprising 3 parts of remote control software, a mechanical arm and a stent conveying device.
2. The system of claim 1, wherein the remote control software comprises a parameter setting page, an ultrasound image preselection mark page, and a process control page, and establishes remote connections with the conveying device and the robotic arm through Wi-Fi, respectively, for automatic or manual motion control.
3. The remote control software of claim 2, wherein the parameter settings page contains 3 sections of delivery device placement, prosthetic valve stent position adjustment, delivery device withdrawal; the conveying device embedding part comprises 2 stages, and each stage has two parameters of an ideal movement speed and a critical stopping distance; the position adjusting part of the artificial valve support comprises 1 adjusting speed parameter; the delivery device withdrawal portion also includes 2 stages, the first stage including both the ideal motion speed and the critical stopping distance parameters, and the second stage including only the ideal motion speed parameter.
4. The remote control software of claim 2, wherein the ultrasound image preselection mark page displays in real time the image captured by the esophageal ultrasound probe, allowing the operator to use a mouse to draw an oval pre-selected box at any position for automatic image segmentation.
5. The remote control software of claim 2, wherein the process control page comprises an echocardiogram and DSA image display module, a delivery device and robot arm status monitoring module, a delivery device automatic or manual control module, a robot arm automatic or manual control module.
6. A method for delivering an automated valve stent delivery system based on image segmentation, comprising the steps of:
acquiring image data and patient personalized parameters;
placing the delivery device into the heart;
adjusting a proper release position, and releasing the artificial valve stent;
the delivery device is withdrawn from the heart.
7. The method of claim 6, wherein the image data is obtained by analyzing all image information from two-dimensional and three-dimensional DICOM data inputted from an external ultrasound device; and acquiring the patient personalized parameters, namely reading and storing the patient personalized parameters set by the operator.
8. The method of claim 6, wherein the delivery device is placed into the heart, the left atrium, the anterior mitral valve of the mitral valve, the posterior mitral valve of the mitral valve, and the delivery device are segmented by image segmentation, the placement of the delivery device into the heart is divided into two stages by defining 3 minimum bounding rectangles, 5 measuring points, and 4 distances, and the delivery device is automatically placed into the heart while avoiding traction and damage to tissues under the valve.
9. The method of claim 6, wherein the delivery device is withdrawn from the heart, the anterior valve prosthesis, the posterior valve prosthesis and the delivery device are segmented using image segmentation techniques, and the withdrawal of the delivery device from the heart is divided into two stages by defining 3 minimum bounding rectangles, 4 measurement points and 4 distances, and the delivery device is automatically withdrawn completely from the heart while avoiding traction and damage to the tissue under the valve.
10. The method of claim 6, wherein adjusting the appropriate release position to release the prosthetic valve stent comprises the steps of:
adjusting the position such that the delivery device is coaxial with the mitral annulus;
releasing the atrial side of the prosthetic valve stent;
withdrawing the conveying device to make the artificial valve support and the original valve leaf of the mitral valve tightly fit;
fully releasing the prosthetic valve stent;
the position of the conveying device is adjusted to be coaxial with the mitral valve annulus, namely the position of the conveying device is automatically adjusted to be coaxial with the mitral valve annulus by defining 4 measuring points and 4 distances by adopting an image segmentation technology;
the withdrawing conveying device enables the artificial valve support to be tightly attached to the original valve leaflets of the mitral valve, namely the artificial valve support is tightly attached to the original valve leaflets of the mitral valve through pixel gray value detection according to the characteristics of the three-dimensional echocardiogram.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111130985.6A CN113796992A (en) | 2021-09-26 | 2021-09-26 | Automatic valve stent conveying system based on image segmentation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111130985.6A CN113796992A (en) | 2021-09-26 | 2021-09-26 | Automatic valve stent conveying system based on image segmentation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113796992A true CN113796992A (en) | 2021-12-17 |
Family
ID=78938587
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111130985.6A Pending CN113796992A (en) | 2021-09-26 | 2021-09-26 | Automatic valve stent conveying system based on image segmentation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113796992A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103093477A (en) * | 2013-02-08 | 2013-05-08 | 河北大学 | Aortic valve fast segmentation method based on esophageal echocardiography |
| CN105264573A (en) * | 2013-06-07 | 2016-01-20 | 皇家飞利浦有限公司 | Planning an implantation of a cardiac implant |
| CN107660137A (en) * | 2015-04-02 | 2018-02-02 | 卡尔迪亚韦弗公司 | Method and apparatus for treating valvular heart disease |
| CN113116606A (en) * | 2021-05-27 | 2021-07-16 | 上海市同济医院 | Electric bracket conveying system based on remote control |
-
2021
- 2021-09-26 CN CN202111130985.6A patent/CN113796992A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103093477A (en) * | 2013-02-08 | 2013-05-08 | 河北大学 | Aortic valve fast segmentation method based on esophageal echocardiography |
| CN105264573A (en) * | 2013-06-07 | 2016-01-20 | 皇家飞利浦有限公司 | Planning an implantation of a cardiac implant |
| CN107660137A (en) * | 2015-04-02 | 2018-02-02 | 卡尔迪亚韦弗公司 | Method and apparatus for treating valvular heart disease |
| CN113116606A (en) * | 2021-05-27 | 2021-07-16 | 上海市同济医院 | Electric bracket conveying system based on remote control |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210338097A1 (en) | Apparatus and methods for mapping a sequence of images to a roadmap image | |
| US20170360383A1 (en) | System and method for positioning an artificial heart valve at the position of a malfunctioning valve of a heart through a percutaneous route | |
| US9144394B2 (en) | Apparatus and methods for determining a plurality of local calibration factors for an image | |
| US8252049B2 (en) | Method for therapy of heart valves with a robot-based X-ray device | |
| JP2018531721A (en) | Communication of aortic valve status | |
| US20070233238A1 (en) | Devices for Imaging and Navigation During Minimally Invasive Non-Bypass Cardiac Procedures | |
| US20140094693A1 (en) | Accounting for skipped imaging locations during movement of an endoluminal imaging probe | |
| JP2009213892A (en) | Method and apparatus for conducting an interventional procedure involving heart valves | |
| EP1779779A2 (en) | Method and system for delivering a medical device to a selected position within a lumen | |
| CN104053404A (en) | Surgical navigation for repair of heart valve leaflets | |
| US11793484B2 (en) | Method, device and system for intracavity probe procedure planning | |
| CN112739268A (en) | Image guidance for implanted lead extraction | |
| EP3673854B1 (en) | Correcting medical scans | |
| CN105992556B (en) | Medical viewing system with observation angle and optimizing function | |
| US11497566B2 (en) | Loose mode for robot | |
| JP6222801B2 (en) | Medical image processing apparatus, X-ray diagnostic apparatus, and medical image processing program | |
| CN113796992A (en) | Automatic valve stent conveying system based on image segmentation | |
| US20120296196A1 (en) | Method for assisting a person performing a minimally invasive intervention with a catheter involving a puncture of a septum and x-ray device | |
| US20230009891A1 (en) | Augmented Imaging For Valve Repair | |
| CN114283179A (en) | Real-time fracture far-near end space pose acquisition and registration system based on ultrasonic images | |
| CN114748141A (en) | Puncture needle three-dimensional pose real-time reconstruction method and device based on X-ray image | |
| CN115089294B (en) | Interventional operation navigation method | |
| EP4230169A1 (en) | Medical system | |
| EP3454293B1 (en) | Method and apparatus for enhancement of bronchial airways representations using vascular morphology | |
| CN114449952A (en) | Signal triggered interventional device control |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |