CN110164557B - Method for simulating and simulating soft tissue surgery path planning by using implicit surface algorithm - Google Patents
Method for simulating and simulating soft tissue surgery path planning by using implicit surface algorithm Download PDFInfo
- Publication number
- CN110164557B CN110164557B CN201910216440.3A CN201910216440A CN110164557B CN 110164557 B CN110164557 B CN 110164557B CN 201910216440 A CN201910216440 A CN 201910216440A CN 110164557 B CN110164557 B CN 110164557B
- Authority
- CN
- China
- Prior art keywords
- model
- soft tissue
- tissue
- simulating
- simulation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/003—Navigation within 3D models or images
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/50—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for simulation or modelling of medical disorders
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/41—Medical
Abstract
The invention relates to a method for simulating and simulating soft tissue surgery path planning by using an implicit surface algorithm, which comprises the following steps: s1, acquiring medical image data of soft tissue parts of a specific case; s2: establishing a three-dimensional geometric model; s3: acquiring instrument characteristic data of an excision operation; s4: based on the design of a human-computer interaction interface, tracking and positioning the operation of a surgical instrument real object or a model of the soft tissue resection operation, calculating the relative position of the surgical instrument real object or the model and an anatomical model, and detecting the operation amount of the resection instrument at each moment; s5: based on an implicit surface algorithm, applying the operation amount to a soft tissue model, and calculating an action amount corresponding to the current operation amount according to the characteristics of the instrument, wherein the action amount comprises an effective ablation area and an ablation profile of the ultrasonic knife on the tissue; s6: and editing and modifying the tissue model according to the calculated action quantity, and displaying dynamic changes including path extension of the section and wound boundary contour extension in the operation. The preoperative preview method and the preoperative preview device can give a user more comprehensive and real preoperative preview experience.
Description
Technical Field
The invention relates to a method for simulating and simulating soft tissue surgery path planning by using an implicit surface algorithm, belonging to the technical field of computer surgery simulation.
Background
The present invention relates to the following similar prior art:
1. the inventor, Shenyuhe, is responsible for developing a prostate minimally invasive surgery simulator [1, 2 ].
2. National research council of canada (NRC) brain surgery simulator: craniocerebral surgery simulator (2012 reports on a method similar to that in 1) [3]
3. Bone surgery simulation [4] for orthopedics or dentistry and the like: a method similar to 1.
4. Liver surgery planning system of japan bubal university: the old method, same field, same operation [5-7 ].
5. Simulation technique of liver resection surgery at european INRIA institute: old procedure, same surgery simulation [8, 9 ].
6. Simulator of prostate surgery of the university of zurich, switzerland/VirtaMed: the old method, the later stage, imitates 1.
7. Typical liver surgery Path planning software MeVisLab [10 ].
[1]Shen，Y.，Konchada，V.，Zhang，N.，Jain，S.，Zhou，X.，Burke，D.，Wong，C.，Carson，C.，Roehrborn，C.，and Sweet，R.，2011，“Laser Surgery Simulation Platform：Toward Full-Procedure Training and Rehearsal for Benign Prostatic Hyperplasia(BPH)Therapy，”Stud.Health Technol.Inform.，163，pp.574-580.
[2]Aydin，A.，Raison，N.，Khan，M.S.，Dasgupta，P.，and Ahmed，K.，2016，“Simulation-Based Training and Assessment in Urological Surgery，”Nat.Rev.Urol.，13(9)，pp.503-519.
[3]Delorme，S.，Laroche，D.，DiRaddo，R.，and F.Del Maestro，R.，2012，“NeuroTouch：A Physics-Based Virtual Simulator for Cranial Microneurosurgery Training，”Oper.Neurosurg.，71，pp.ons32-ons42.
[4]Pflesser，B.，Petersik，A.，Tiede，U.，Hihne，K.H.，and Leuwer，R.，2002，“Volume Cutting for Virtual Petrous Bone Surgery，”Comput.Aided Surg.，7(2)，pp.74-83.
[5]Oshiro，Y.，and Ohkohchi，N.，2017，“Three-Dimensional Liver Surgery Simulation：Computer-Assisted Surgical Planning with Three-Dimensional Simulation Software and Three-Dimensional Printing＜sup/＞，”Tissue Eng.Part A，23(11-12)，pp.474-480.
[6]Oshiro，Y.，2015，“Novel 3-Dimensional Virtual Hepatectomy Simulation Combined with Real-Time Deformation，”World J.Gastroenterol.，21(34)，p.9982.
[7]Enzaki，Y.，Yano，H.，Oshiro，Y.，Kim，J.，Kim，S.，Iwata，H.，and Ohkohchi，N.，2015，“Development of the Haptic Device for a Hepatectomy Simulator，”Haptic Interaction，H.Kajimoto，H.Ando，and K.-U.Kyung，eds.，Springer Japan，Tokyo，pp.231-235.
[8]Courtecuisse，H.，Allard，J.，Kerfriden，P.，Bordas，S.P A.，Cotin，S.，and Duriez，C.，2014，“Real-Time Simulation of Contact and Cutting of Heterogeneous Soft-Tissues，”Med.Image Anal.，18(2)，pp.394-410.
[9]Courtecuisse，H.，Jung，H.，Allard，J.，Duriez，C.，Lee，D.Y.，and Cotin，S.，2010，“GPU-Based Real-Time Soft Tissue Deformation with Cutting and Haptic Feedback，”Prog.Biophys.Mol.Biol.，103(2-3)，pp.159-168.
[10]“MeVisLab：History”[Online].Available：https：//www.mevislab.de/mevislab/history/.[Accessed：15-Dec-2017].
[11]Lang，H.，2005，“Impact of Virtual Tumor Resection and Computer-Assisted Risk Analysis on Operation Planning and Intraoperative Strategy in Major Hepatic Resection，”Arch.Surg.，140(7)，p.629.
[12]Muhler，K.，Tietjen，C.，Ritter，F.，and Preim，B.，2010，“The Medical Exploration Toolkit：An Efricient Support for Visual Computing in Surgical Planning and Training，”IEEE Trans.Vis.Comput.Graph.，16(1)，pp.133-146.
[13]“
Vision-Vital Images”[Online].Available：http：//www.vitalimages.com/vitrea-vision/.[Accessed：15-Dec-2017].
[14]Reitinger，B.，Borbik，A.，Beichel，R.，and Schmalstieg，D.，2006，“Liver Surgery Planning Using Virtual Reality，”IEEE Comput.Graph.Appl.，26(6)，pp.36-47.
[15]Marescaux，J.，Clément，J.-M.，Tassetti，V.，Koehl，C.，Cotin，S.，Russier，Y.，Mutter，D.，Delingette，H.，and Ayache，N.，1998，“Virtual Reality Applied to Hepatic Surgery Simulation：The Next Revolution：，”Ann.Surg.，228(5)，pp.627-634.
Surgical planning and surgical simulation training are two different fields, and have different requirements and emphasis points. On the other hand, there are some techniques that can be used in common for both technical components.
In the field of surgical planning, traditional computer surgical planning software (e.g., reference [10]) mainly uses medical imaging software technologies such as computer visualization (visualization), three-dimensional anatomical model reconstruction, etc. to generate a two-dimensional image or a three-dimensional graph corresponding to a surgical object from clinical image data such as tomography of a specific case; and inputting numerical values or calibration points, lines and surfaces through interfaces such as a keyboard, a mouse and the like, and carrying out preoperative planning and analytical evaluation on the operation path or results and indexes (such as a reference document [10-13 ]). Recent technological advances include integrating physical models such as 3D printing, and various virtual reality (VR, AR, MR) technologies to improve human-machine interface and data presentation capabilities of planning systems (e.g., reference [14 ]).
In the field of operation simulation, an operation simulator is used as simulation exercise equipment for actual operation skills, and a safe and efficient skill learning and training mode is provided for a trainer. These simulator systems differ from surgical planning systems mainly in that, due to the different purposes of application: (1) in order to simulate the interaction process of surgical equipment with human tissue, these systems emphasize that the virtual patient model has real-time and dynamic simulation capabilities, as well as the realism of the human-computer interaction experience. These dynamic models are typically based on physical principles and mathematical models such as computer graphics, computational mechanics, etc. (2) A preselected set of typical case models is generated without the need to make a specific virtual patient model for each specific patient: the current state of the art is not capable of generating fully automatic models for interactive simulation of surgical procedures, which is not necessary for training subjects.
Although the related technologies are yet to be perfected, some researchers in medical simulation (e.g. ref 15) in early days have begun to discuss the concept of cross-domain technological innovation and application — the possibility of planning liver surgery using the technological approach of surgical simulation.
Tissue isolation and resection: organ tissues are changed in a plurality of ways during operations, wherein the separation of different tissues or the same tissue is an important change process in a large number of surgical subjects. Devices for separating tissue and the corresponding separation modes are various, for example: the cutting, shearing and clamping tools in the traditional open type operation and minimally invasive operation instruments, various electric knives, laser, grinding and suction, ultrasonic knives and other instruments and respectively corresponding mechanical cutting, passive stripping, grinding and drilling, gasification, ablation, emulsification and the like. The scientific and technological method for carrying out simulation calculation and reproduction on tissue separation is the technical key point of the text and the invention.
Comparing the basic principle and the method of the separation and excision simulation modeling: the basic calculation and simulation contents for realizing separation change and excision operation mainly comprise the steps of establishing a three-dimensional anatomical model and demonstrating excision effect by editing and modifying the model. The three-dimensional anatomical model can be created by applying computer visualization algorithm to medical tomography image data, and can also be made and processed by computer three-dimensional modeling software. The geometric model of the three-dimensional model is usually a mesh structure, such as a polygonal mesh surface or a polyhedral mesh, and may also directly perform volume rendering (volumetric rendering) on volume elements (voxels) of the medical image data. Thus, by changing the structure of the geometry in the model, the effect of the separation and excision of the tissue in the operation can be shown. The mesh modeling and planning algorithm (meshing) belongs to the field of computational geometry (computational geometry) and computer graphics research.
In the above prior art with 1-7 items being similar, the classification and comparison of the algorithm for planning and editing the three-dimensional geometric model are as follows:
the old method refers to a traditional "explicit" method for modifying geometric model elements by direct editing, for example, subdividing (subdividing) a triangle into a plurality of triangles according to tangent trajectories.
New and similar methods refer to the class of computer visualization algorithms for extracting and constructing implicit surfaces (implicit surfaces) from three-dimensional arrays of volume elements (Voxel), hereinafter referred to as "implicit" methods. The method of the present invention is of this type.
Prior art 7 is an example of conventional liver resection path planning software.
The prior art 4 and 5 adopt the old method as the excision algorithm, and the results show that the display method has poor applicability in such applications. Prior art 4 is a liver surgery planning system integrated with VR interface, which plans an incision path in a short straight line subdivision grid model. The prior art 5 is used for liver surgery simulation, a resection process is simulated by specific surgical instruments, and the application prospect of the technology in the liver surgery planning direction is mentioned; however, the segmentation method of the display mesh model is adopted, so that the liver section effect is rough, and the method is not suitable for the application of surgical planning.
The defects of the prior art are as follows: the keyboard and mouse operation interface used by the traditional operation planning software has poor applicability and no similarity with the actual operation interface.
The mode of inputting the parameters of the preset points, lines and planes to perform the resection planning is abstract and simplified for the actual surgical resection operation and the change process, and lacks the planning and experience for the intermediate steps. The main functions of such surgical planning systems are limited to the analysis and evaluation of surgical results, and do not have preview and experience functions.
The explicit method of directly segmenting geometric model elements is not beneficial to planning paths for fine and complicated minimally invasive surgeries such as 'partial hepatectomy'. The existing explicit resection method has low model resolution, and the simulated cutting block size is often larger than the actual situation. Simply increasing the resolution will quickly increase the amount of computation beyond the real-time response capability of the simulation system. The gradual, random and repeated excision operation is continuously subdivided in the local part of the model to form an intricate and complex structure, so that the error rate of the explicit algorithm is increased, the reliability is reduced, and the complexity of the boundary of the section is difficult to control.
The grid modeling of a specific case has low efficiency, complex process, incapability of being completed fully automatically and long time. The process of establishing a dynamic simulation model of the visceral organs of the virtual patient in the operation simulator is more complicated. These two major technical difficulties limit the feasibility and practicality of the concept of using a surgical simulator as a surgical planning system.
Disclosure of Invention
The invention aims to solve the defects of the prior art and provides a method for simulating and simulating soft tissue surgery path planning by using an implicit surface algorithm. The soft tissue includes at least any one of liver tissue, lung tissue and brain tissue.
The surgical path planning method can simultaneously achieve the following aims:
the method is used for the ablation surgical planning of soft tissues and other parts.
And (3) accurately planning a path, and accurately displaying section boundaries of various cutting modes according to the characteristics of the cutting instrument.
Progressive surgical planning, each operation step corresponds to an intermediate result, and the specific forming process of the resection path is completely shown.
And (3) performing human-computer interactive dynamic operation planning, calculating and updating the virtual model in real time according to the operation amount at each moment, and generating a corresponding tangent plane/wound surface.
By adopting partial technical method in the field of operation simulation, the operation function of operation simulation can be selectively realized, for example, a simulation operation interface can be integrated. On the other hand, because the technical invention focuses on solving the requirement of planning the operation path of a specific case, rather than training the operation skill, the necessary organ tissue dynamic simulation model in the operation simulation system can be simplified or used as an option to ensure the feasibility of the scheme when necessary.
The volume element model composed of medical image data can be directly used, so that the task of reconstructing the mesh geometric model becomes an option rather than a necessary link.
The technical scheme adopted by the invention for solving the technical problems is as follows:
the implicit surface algorithm is used for simulating a method for planning a soft tissue surgery path:
s1: acquiring medical image data of soft tissue parts of a specific case;
s2: establishing a three-dimensional geometric model, specifically generating a grid model of a soft tissue structure by using a visualization algorithm, or adopting volume rendering data of medical image software as a volume voxel model;
algorithms such as Marching Cubes are commonly used for three-dimensional model reconstruction of medical image data. Many open source tools such as VTKs and commercial software carried by clinical imaging equipment have these functions;
the volume rendering data is from a medical imaging device. Many open source tools such as VTKs and commercial software carried by clinical imaging equipment have these functions. The application of the invention is for surgical planning, the initial data being case images (e.g. CT/MRI) from the surgical object.
S3: acquiring instrument characteristic data of the resection, wherein the instrument characteristic data comprises the corresponding relation between the outline of an instrument mechanical structure and the relevant parameter setting of instrument power and the tissue resection or ablation amount, and the acquisition mode is actually measured or provided by an instrument manufacturer;
s4: based on the design of a human-computer interaction interface of an integrated operation simulator, the operation of a real object or a model of the soft tissue surgical instrument is tracked and positioned, the relative position of the real object or the model and an anatomical model is calculated, and the operation amount of the resection instrument at each moment is detected;
the operation interface of the operation simulator designed by adopting computer simulation has the functions of tracking and positioning. Meanwhile, the surgical robot system also has the function of positioning and detecting surgical instruments. The operation planning function module can be integrated into a surgical robot system or a computer-assisted surgery system, and can also be integrated into an operation simulator and medical image diagnosis equipment;
s5: based on an implicit surface algorithm, applying the operation amount to a soft tissue model, and calculating an action amount corresponding to the current operation amount according to the characteristics of the instrument, wherein the action amount comprises an effective ablation area and an ablation contour of the surgical instrument on the tissue;
specifically, the ablation and excision action on the liver tissue and the damage action on the blood vessel and related tissues are calculated according to the relevant factors of the mechanical contour of the cutter body of the surgical instrument and the distribution characteristics of ultrasonic energy in the cutter body. By adopting an implicit surface algorithm, a doctor can generate different wound surface contours corresponding to different action quantities according to each operation, and the overall resection path is continuously modified and updated very reliably;
s6: editing and modifying the tissue model according to the calculated action quantity, and displaying dynamic changes including path extension of a section and extension of a wound boundary contour in an operation;
the change of the boundary contour is automatically generated by the simulation method: a new wound surface generated by using an implicit surface algorithm and integrated with the existing three-dimensional model; wherein, the position of the newly added wound or the changed position of the existing wound is determined by the step S4; the shape of the boundary contour and the size of the amount of ablation action corresponding thereto are determined in step S5.
S7: based on other simulation functions which are realized by the integrated operation simulator and comprise soft tissue deformation and bleeding hemostasis, the excision process of a specific case is displayed in real time in a man-machine interaction operation mode.
The invention can be integrated with a conventional computer surgery simulator, and mainly aims to use a human-computer interaction interface of the computer surgery simulator. The method comprises the following steps: the system needs to have real-time/interactive computing capabilities.
Preferably, for product design considerations such as cost, if the computational efficiency of a computer fails to meet the criteria of real-time response, the secondary simulation function can be eliminated: for example, neglecting special effects such as bleeding and the like, simplifying simulation on tissue deformation (citing some existing rapid algorithms), realizing only a surgical path planning function by a software module, and providing a man-machine interactive operation interface for the software module.
The core content of the surgical path planning is that a gradual generation and change process of a wound surface is realized on a case model. In addition, the prior art provides a practical example for the design and implementation of the simulation functions of the virtual surgical scene and the integration of these functions with the ablation path simulation calculation module.
Preferably, in step S1, the medical image data is X-ray tomography (CT) data or magnetic resonance tomography (MRI) data.
The invention has the beneficial effects that: the incidence rate of soft tissue diseases is high. For example, our country is a large country with typical liver diseases. Liver tissue resection is extremely challenging: the liver blood flow is large, a large number of arteriovenous ducts and bile ducts with different sizes are arranged in the liver, the structure is complicated, the operation difficulty is large, and the risk is high. The invention carries out the planning of the soft tissue operation path in a man-machine interactive operation simulation mode. Compared with the traditional operation planning software, the preoperative preview experience can be more comprehensive and real for the user. On the basis of the existing minimally invasive surgery simulator, the section is generated by using the implicit method, the section generation method is more accurate than the section generation method in the existing soft tissue surgery simulator adopting the explicit method, the actual requirements of accurate planning, analysis and preview of the surgical resection path can be met, and the method has practical value. The invention can be used for the grid model data generated by visual software, can omit the modeling link, can be directly used for the specific case data provided by medical imaging equipment, and obviously saves the time cost for the application of the operation planning technology in clinical practice.
Drawings
FIG. 1 is prior liver surgery planning software of reference [11] -a line drawing shows the cutting results;
FIG. 2 is a section effect-display subdivision scheme in the existing liver surgery simulator of reference [8 ];
FIG. 3 is a section effect-display subdivision scheme in the existing liver surgery simulator of reference [9 ];
FIG. 4 is a straight line cutting effect-explicit subdivision of the prior liver surgery planning VR system of reference [5-7 ];
FIG. 5 shows the effect of the present invention in simulating tissue, such as liver, using an implicit ablation method, with controllable ablation path boundaries.
Detailed Description
The technical solution of the present invention is further specifically described below by way of specific examples in conjunction with the accompanying drawings.
The invention provides a method for simulating and simulating soft tissue surgery path planning by using an implicit surface algorithm, wherein liver tissue is taken as an example of soft tissue, and the method comprises the following steps:
s1: acquiring medical image data of a liver part of a specific case;
s2: establishing a three-dimensional geometric model, specifically generating a grid model of a liver tissue structure by using a visualization algorithm, or adopting volume rendering data of medical image software as a volume voxel model;
algorithms such as Marching Cubes are commonly used for three-dimensional model reconstruction of medical image data. Many open source tools such as VTKs and commercial software carried by clinical imaging equipment have these functions;
the volume rendering data is from a medical imaging device. Many open source tools such as VTKs and commercial software carried by clinical imaging equipment have these functions. The application of the invention is for surgical planning, i.e. the initial data, i.e. case images (e.g. CT/MRI) from the surgical object.
S3: acquiring instrument characteristic data of the resection, wherein the instrument characteristic data comprises the corresponding relation between the outline of an instrument mechanical structure and the relevant parameter setting of instrument power and the tissue resection or ablation amount, and the acquisition mode is actually measured or provided by an instrument manufacturer;
s4: based on the design of a human-computer interaction interface of the integrated operation simulator, the operation of a real object or a model of the liver resection operation surgical instrument is tracked and positioned, the relative position of the real object or the model and an anatomical model is calculated, and the operation amount of the resection instrument at each moment is detected;
the operation interface of the operation simulator designed by adopting computer simulation has the functions of tracking and positioning. Meanwhile, the surgical robot system also has the function of positioning and detecting surgical instruments. The operation planning function module can be integrated into a surgical robot system or a computer-assisted surgery system, and can also be integrated into an operation simulator and medical image diagnosis equipment;
s5: based on an implicit surface algorithm, applying the operation amount to a liver tissue model, and calculating an action amount corresponding to the current operation amount according to the characteristics of the instrument, wherein the action amount comprises an effective ablation area and an ablation contour of the surgical instrument on the tissue;
specifically, the ablation and excision action on the liver tissue and the damage action on the blood vessel and related tissues are calculated according to the relevant factors of the mechanical contour of the cutter body of the surgical instrument and the distribution characteristics of ultrasonic energy in the cutter body. By adopting an implicit surface algorithm, a doctor can generate different wound surface contours corresponding to different action quantities according to each operation, and the overall resection path is continuously modified and updated very reliably;
s6: editing and modifying the tissue model according to the calculated action quantity, and displaying dynamic changes including path extension of a section and extension of a wound boundary contour in an operation;
the change of the boundary contour is automatically generated by the simulation method: a new wound surface, generated by using an implicit surface algorithm and integrated with the existing three-dimensional model; wherein, the position of the newly added wound or the changed position of the existing wound is determined by the step S4; the shape of the boundary contour and the size of the amount of ablation action corresponding thereto are determined in step S5.
S7: based on other simulation functions which are realized by the integrated operation simulator and comprise soft tissue deformation and bleeding hemostasis, the excision process of a specific case is displayed in real time in a man-machine interaction operation mode.
The invention can be integrated with a conventional computer surgery simulator, and mainly aims to use a human-computer interaction interface of the computer surgery simulator. The method comprises the following steps: the system needs to have real-time/interactive computing capabilities.
Specifically, for product design considerations such as cost, if the computational efficiency of a computer fails to meet the criteria of real-time response, the secondary simulation function can be eliminated: for example, neglecting special effects such as bleeding and the like, simplifying simulation on tissue deformation (citing some existing rapid algorithms), realizing only a surgical path planning function by a software module, and providing a man-machine interactive operation interface for the software module.
The core content of the surgical path planning is to realize the progressive generation and change process of the wound surface on a case model. In addition, the prior art provides a practical example for the design and implementation of the simulation functions of the virtual surgical scene and the integration of these functions with the ablation path simulation calculation module.
In step S1, the medical image data is X-ray tomography (CT) data or magnetic resonance tomography (MRI) data.
The system flow based on the method of the invention is shown as follows: medical image data (CT tomography and magnetic resonance), a volume element (voxel) three-dimensional anatomical model of organ tissues of an operation part is reconstructed by using medical visualization technologies such as image segmentation (segmentation), or a grid (mesh) model is generated by using the visual three-dimensional reconstruction technology for the volume element model, an interactive model is established by using an operation simulation modeling technology, and the interactive model is dynamically displayed on a display device by using rendering technologies in the fields of computer graphics and visualization;
the interactive model, the separation excision simulation and the generated operation path belong to the components of the main program of the real-time simulation system (including other common technical components, and review documents related to an operation simulator or a virtual operation system can be consulted); the real-time simulation updates the interactive model according to the operation quantity detected and collected by the human-computer interaction interface and the operation path which is gradually generated and extended;
the separation and excision simulation based on the method of the invention is shown as follows: isolated resection data can be obtained from experimental surgery or estimated from Molecular Dynamics Simulations. (note: the molecular dynamics simulation does not achieve the operation efficiency of the real-time simulation, and the obtained result is the same as the experimental data and can be used as the prepared reference data.) according to the operation amount, the action amount of the separation and the excision is obtained through fitting, the wound surface generation module gradually calculates the new wound surface formed by the operation at the current moment according to the action amount and the interaction model at the previous moment, and the operation path is updated in a gradual mode.
In the invention, the content of the related technology composition is as follows:
1. medical image data: medical image data of a surgical site of a specific case is acquired. Preferably, the image may be a clinical X-ray tomography (CT) image or a Magnetic Resonance Imaging (MRI) image.
2. Visualization: here, a three-dimensional anatomical model of an organ tissue in a surgical site is extracted and reconstructed mainly by using a technique such as image segmentation (segmentation) in medical visualization, and the boundary, internal structure, and relative positional relationship between models of each model are specified.
3. A body element model: a three-dimensional model of organ tissue composed of numerous volume elements (voxels).
4. Visualization: here, visualization techniques are used to generate mesh models from the volume element models. Implicit surface algorithms are a type of algorithms in visualization technology, and can extract, display and modify isosurface (isosurface) from body elements.
5. Grid model: a two-dimensional or three-dimensional geometric model composed of polygons or polyhedrons, and surface maps, material rendering properties and the like.
6. Modeling: response and change rules of the organ tissues in the operation process are defined according to the characteristics of morphological structures, physical properties and the like of the organ tissues, and a solving mechanism is established on a three-dimensional model by a series of algorithms of computer simulation (refer to a review of related surgery simulation literature). The morphological structure may be defined by a mesh model or a volume element model.
7. And (3) interaction model: and simulating, updating and displaying the simulation model of the corresponding dynamic change of the organ tissues in real time according to the input of the operator in the operation process.
8. Separating and cutting data: the isolated ablation data reflects the distribution of ablation energy and the effect volume in the subject tissue. The data can be obtained by experiments or by simulation calculation such as Molecular Dynamics Simulations. Our field of application is mainly the precise path planning of minimally invasive resection. Different from the traditional mechanical excision tools such as a scalpel and a scissors in open surgery, in modern minimally invasive surgery, an ablation excision instrument with controllable energy is generally adopted for important separation excision operation, for example, ultrasonic energy, laser energy and the like are applied to local tissues to generate ablation effects such as liquefaction, gasification and the like, so that fine separation excision operation is realized.
8. Separation and excision simulation: fitting the separation and excision data, calculating the action amount of the separation and excision according to the operation amount of an operator, and generating a wound surface.
9. A surgical path: as the surgical procedure progresses, an overall separating resection boundary contour is formed gradually (progressive) from the generated wound surface on the interactive model. The operation path includes a trace left after the organ tissue is separated and excised and also includes operation path information.
10. Rendering: and using a computer graphic display card product, calling and writing a three-dimensional rendering program, or calling and writing a volume rendering program in a visualization technology, and displaying the appearance and the structure of the three-dimensional model.
11. Displaying: the display may use a flat panel display, a three-dimensional holographic display, or a Head Mounted Display (HMD) such as Virtual Reality (VR), Augmented Reality (AR), etc., as desired.
12. A human-computer interaction interface: user interface equipment for a user to perform interactive operation with a virtual surgical scene generally consists of a surgical instrument model and a motion detection sensor or motion tracking equipment.
13. Operation amount: measurement of surgical procedures by sensors or motion tracking devices.
14. Fitting: from the known isolated resection data, the change of the currently measured surgical procedure quantity to the interactive model is calculated so as to match the known data as much as possible.
15. Acting amount: the change in the current interaction model.
16. Generating a wound surface: and editing and modifying the interactive model according to the calculated effect of the current separation and excision operation, and adding a new operation wound surface on the interactive model.
Surgical procedure volume tracking detection- > calculation of ablation effect volume- > editing of dissection algorithm (explicit or implicit surface) - > generation of ablation wound-surface- > updating of surgical path.
The invention firstly proposes to use an artificial intelligent machine learning technology to carry out separation excision data fitting, and calculate the action quantity of the currently measured operation quantity on the interactive model by using the known separation excision data so as to lead the action quantity to be consistent with the known reference data.
The target is as follows: the volume of tissue ablated and its spatial distribution are estimated given the variables of ablation instrument power, relative position/distance to the tissue, etc.
Compared with the existing method, the method can accurately determine the ablation boundary contour and the operation path formed by the ablation boundary contour.
In the case of gradual, progressive (progressive) changes in the boundary, the amount of separate ablation effect per session can be defined as the difference between the existing boundary and the new boundary. In three-dimensional space, the total amount of this difference is a volumetric amount. To completely determine the amount of separation ablation, we determine not only this total amount, but also the distribution of this volume over the existing boundaries. Therefore, we define the "acting amount" as the distribution of the volume along the boundary surface, i.e. the volume corresponding to each point on the boundary surface.
Two general types of methods for machine learning, such as Support Vector Machine (SVM) methods, neural network methods, can be applied to this problem of the present invention.
1.1 data and features of machine learning
Data: the isolated ablation data reflects the distribution of ablation energy and the effect volume in the subject tissue. The data can be obtained by experiments or by simulation calculation such as Molecular Dynamics Simulations.
Energy is emitted from the instrument, spreading around. The tissue located in the effective region is affected by the energy and changes in physical state.
The experiment can measure the effect of energy on the tissue sample. For example, after laser energy is absorbed by tissue, heat is generated in the tissue and some solid tissue is vaporized; after the ultrasonic energy is absorbed by the tissue, some of the solid tissue is emulsified. After the specimen is subjected to processing such as reverse molding and slicing, the range (position and volume) of the phase change of the tissue in gaseous state, liquid state and the like can be measured.
Simulation can calculate the amount of energy contribution to the tissue model. For example, molecular dynamics simulations can calculate the distribution of energy, temperature, etc. values in the tissue, and can therefore calculate the range of solid tissue to reach gaseous and liquid phase transition conditions.
Is characterized in that:
power of
Coordinates/distances
Time of action
Tissue differentiation
Target: and judging and estimating the position/range of the tissue where the phase change such as gasification ablation occurs.
The problems are that: this fitting problem can be defined as a Regression (Regression) problem and also as a Classification (Classification) problem.
Solving the regression problem, namely determining the distribution of ablation energy in the tissue according to an experimental result or molecular dynamics simulation, and determining the liquefaction or gasification range of the tissue according to an energy distribution result.
And solving the classification problem, namely directly dividing the liquefaction or gasification range according to experiment or molecular dynamics simulation data. For example, using the direct results of molecular dynamics simulations, we can classify the virtual tissue of the interaction model as belonging to the tissue that was not separately excised, but still remains, and as belonging to the tissue that was separately excised at the current time period. In this way, the problem of solving the contribution amount is converted into a binary problem in machine learning.
The geometric model of the three-dimensional model is usually a mesh structure, such as a polygonal mesh surface or a polyhedral mesh, and may also directly perform volume rendering (volumetric rendering) on volume elements (voxels) of the medical image data.
Computer three-dimensional graphics technologies and devices have been designed based on the surface rendering of mesh models.
At present, the surface rendering effect of the grid model has more special effect options and is faster. The volume rendering needs to preprocess the volume element model, and the fidelity of the surface rendering of the grid model cannot be achieved at present.
The volume rendered model more fully includes structural information of the three-dimensional solid model, such as anatomical structures inside the model boundaries. When a complex structure is constructed, the voxel model is simpler and more reliable than a grid model.
According to the technical scheme, a separation and excision algorithm based on the implicit surface is applied, and not only can a grid model be used, but also a body element model formed by medical image data can be directly used. The task of reconstructing the geometric model of the mesh becomes an option rather than a necessary step. The technical scheme can realize path planning by interactive operation simulation on deformable soft tissue models such as liver and the like.
The process of reconstructing the grid model and then constructing the interactive simulation model is complex, and many computer simulation algorithms have strict requirements on grid quality, so that professionals are often required to manually correct and optimize grid reconstruction and planning results for many times, and test and verification are carried out. The reconstruction of the grid model for each clinical specific case (patient-specific) cannot be completed fully automatically, and the process is complex; the process of establishing a dynamic simulation model of the visceral organs and tissues of the virtual patient in the operation simulator is more complicated. These two major technical difficulties limit the feasibility and practicality of the concept of using a surgical simulator as a surgical planning system.
The two methods of grid model reconstruction and volume element model volume rendering are commonly used in the medical image application field of clinical diagnosis such as radiology department; the separation and excision module in the existing operation simulation technology product generally uses a mesh reconstruction model, and a voxel model is only used for 'rigid body' models such as dentistry, orthopedics and the like. The main technical problem to be solved when being applied to the soft tissue model is to perform the separation excision and the deformation of the soft tissue and simultaneously perform the simulation by using a voxel model.
Early efforts by the inventors of the present application and other prior art examples show that implicit surface algorithms can be used to implement soft tissue surgery simulation systems.
The technical scheme further provides a simulation and path planning system for constructing the soft tissue separation resection by directly using the voxel model, and provides an option for directly jumping from the voxel model to the interactive model.
The implicit surface algorithm of the application can edit and modify voxel model data.
The volume element data, or called volume data, is data corresponding to 8 vertices (sampling points) of a cube occupied by each volume element (voxel) in a three-dimensional sampling space in which the volume element model is located. The data values acquired by the medical imaging device usually reflect some physical values and tissue properties corresponding to the vertex positions, such as magnetic resonance amplitude, X-ray transmission distribution, ultrasonic transmission distribution, and tissue density derived from transformation, biological material properties, and the like. These data are modified and updated in the system by the detachment excision simulation module.
The implicit surface algorithm is used as a visualization algorithm, isosurfaces can be dynamically extracted from the voxel data, the isosurfaces can be constructed by a grid surface model, the processing flow of the grid model is converted, and simulation and path planning options based on the grid model are adopted. The construction steps of the grid model can be skipped, and 3, 6 and 7 modules in the system are skipped, namely the volume elements and the volume rendering options.
The application provides the following related technical scheme and solutions for the new design based on voxel rendering: problem of voxel model over-resolution:
if the resolution of some medical image original data is too high relative to the real-time simulation load of soft tissue model deformation and separation excision, the calculated amount of a system real-time module can be reduced by common optimization technical means such as reducing sampling rate (down sampling), adaptive resolution (adaptive resolution) and the like.
Whether the resolution of the volume element model is too low or not, whether the volume rendering has no visual boundary surface in the grid model or not and whether the definition of the grid model can be achieved or not.
The voxel model and the volume rendering are widely used for clinical medical image diagnosis equipment, comprise operation planning software for keyboard and mouse marking operation, meet the requirement standard of clinical work and have practical value.
And adopting the body elements and the body rendering options to judge whether the full-automatic operation path planning or the full-automatic operation simulation of any specific case is realized.
The original body element image data is still in a medical tomography format, and the reconstructed three-dimensional body element model needs to be subjected to visualization processing such as image segmentation. This reconstruction technique is currently in general at a semi-automated stage.
The main advantages of the voxel rendering option: the complexity and the time cost of a modeling process are obviously reduced, and the modeling efficiency and the practicability of preoperative path planning are improved.
Fig. 1 is a drawing showing a cutting result of the existing liver surgery planning software of reference [11], fig. 2 is a section effect-display subdivision method in the existing liver surgery simulator of reference [8], fig. 3 is a section effect-display subdivision method in the existing liver surgery simulator of reference [9], fig. 4 is a straight line cutting effect-display subdivision method of the existing liver surgery planning VR system of reference [5-7], and fig. 5 is an effect of performing simulation of tissues such as a liver and the like by the implicit ablation method of the present invention-controllable boundary of an ablation path. The invention carries out the planning of the soft tissue operation path in a man-machine interactive operation simulation mode. Compared with the traditional operation planning software, the preoperative preview experience can be more comprehensive and real for the user. On the basis of the existing minimally invasive surgery simulator, the section is generated by using the implicit method, the section generation method is more accurate than the section generation method in the existing soft tissue surgery simulator adopting the explicit method, the actual requirements of accurate planning, analysis and preview of the surgical resection path can be met, and the method has practical value. The invention can be used for the grid model data generated by visual software, can omit the modeling link, can be directly used for the specific case data provided by medical imaging equipment, and obviously saves the time cost for the application of the operation planning technology in clinical practice.
The above-described embodiments are only preferred embodiments of the present invention, and are not intended to limit the present invention in any way, and other variations and modifications may be made without departing from the spirit of the invention as set forth in the claims.
Claims (7)
1. The method for simulating and simulating the soft tissue surgery path planning by using the implicit surface algorithm is characterized by comprising the following steps of:
step S1, acquiring medical image data of soft tissue parts of a specific case; the medical image data is X-ray tomography data or magnetic resonance tomography data;
step S2: establishing a three-dimensional geometric anatomical model;
step S3: acquiring surgical instrument characteristic data of an excision operation;
step S4: based on the design of a human-computer interaction interface of an integrated operation simulator, the operation of a surgical instrument object or a model of the soft tissue resection operation is tracked and positioned, the relative position of the surgical instrument object or the model and an anatomical model is calculated, and the operation amount of the surgical instrument at each moment is detected;
step S5: based on an implicit surface algorithm, applying the operation amount to a tissue model of the soft tissue, and calculating an action amount corresponding to the current operation amount according to the characteristics of the surgical instrument, wherein the action amount comprises an effective ablation area and an ablation contour of the surgical instrument on the soft tissue;
step S6: editing and modifying the tissue model according to the calculated action quantity, and displaying dynamic changes including path extension of a section and extension of a wound boundary contour in an operation;
step S7: based on other simulation functions which are realized by the integrated operation simulator and comprise soft tissue deformation and bleeding hemostasis, the excision process of a specific case is displayed in real time in a man-machine interaction operation mode.
2. The method for simulating soft tissue surgical path planning using implicit surface algorithm of claim 1, wherein the step S2 of creating a three-dimensional geometric model includes using a visualization algorithm to generate a mesh model of the tissue structure of the soft tissue, or using volume rendering data provided by the medical imaging software as a voxel model.
3. The method for simulating soft tissue surgery path planning according to the implicit surface algorithm of claim 1 or 2, wherein the surgical instrument feature data in step S3 includes the corresponding relationship between the contour of the mechanical structure of the surgical instrument and the power related parameter setting of the surgical instrument and the soft tissue ablation or ablation amount, and the obtaining manner is the actual measurement or is provided by the instrument manufacturer or is obtained by simulation calculation.
4. The method for simulating soft tissue surgery path planning using implicit surface algorithm according to claim 1 or 2, wherein the change of the boundary contour in step S6 is automatically generated by simulation method: a new wound surface, generated by using an implicit surface algorithm and integrated with the existing three-dimensional model; wherein, the position of the newly added wound or the changed position of the existing wound is determined by the step S4; the shape of the boundary contour and the size of the amount of ablation action corresponding thereto are determined in step S5.
5. The method for simulating soft tissue surgery path planning using implicit surface algorithm according to claim 1 or 2, wherein the display presented in step S6 can use a flat display, a three-dimensional holographic display, or a virtual reality, augmented display head-mounted display as required.
6. The method for simulating soft tissue surgery path planning according to the implicit surface algorithm of claim 1 or 2, wherein the calculation of the action quantity in step S5 is performed by fitting the separation and excision data using an artificial intelligence machine learning method, and the action quantity of the currently measured surgery operation quantity on the anatomical model is calculated from the known separation and excision data.
7. A method for emulating simulated soft tissue surgical path planning according to an implicit surface algorithm according to claim 1 or 2, characterized in that the soft tissue comprises at least any of liver tissue, lung tissue and brain tissue.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910216440.3A CN110164557B (en) | 2019-07-08 | 2019-07-08 | Method for simulating and simulating soft tissue surgery path planning by using implicit surface algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910216440.3A CN110164557B (en) | 2019-07-08 | 2019-07-08 | Method for simulating and simulating soft tissue surgery path planning by using implicit surface algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110164557A CN110164557A (en) | 2019-08-23 |
| CN110164557B true CN110164557B (en) | 2022-05-31 |
Family
ID=67638991
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910216440.3A Active CN110164557B (en) | 2019-07-08 | 2019-07-08 | Method for simulating and simulating soft tissue surgery path planning by using implicit surface algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110164557B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112843506B (en) * | 2019-11-28 | 2023-07-04 | 重庆西山科技股份有限公司 | Surgical system and ultrasonic suction knife system |
| CN111067618A (en) * | 2019-12-31 | 2020-04-28 | 华科精准（北京）医疗科技有限公司 | Simulation method and device for laser interstitial thermotherapy |
| CN112614152B (en) * | 2020-12-22 | 2022-03-25 | 电子科技大学 | Soft tissue surface motion tracking method based on depth feature learning |
| CN113796952B (en) * | 2021-11-18 | 2022-03-18 | 北京智愈医疗科技有限公司 | Tissue resection system and cutting parameter determination method thereof |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102117378A (en) * | 2009-12-31 | 2011-07-06 | 苏州瑞派宁科技有限公司 | Hepatic tumor comprehensive surgical planning analogy method and system thereof based on three-dimensional multimode images |
| CN104537938A (en) * | 2014-12-21 | 2015-04-22 | 合肥德铭电子有限公司 | Endoscope simulative training system capable of achieving flexible force feedback |
| JP6351138B2 (en) * | 2013-03-13 | 2018-07-04 | 国立大学法人 筑波大学 | Diagnostic imaging support program |
| CN108492693A (en) * | 2018-03-28 | 2018-09-04 | 盛玉涛 | A kind of laparoscopic surgery simulated training system shown based on computer aided medicine |
| CN109243614A (en) * | 2018-09-11 | 2019-01-18 | 深圳先进技术研究院 | A kind of Surgery Simulation methods, devices and systems |
-
2019
- 2019-07-08 CN CN201910216440.3A patent/CN110164557B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102117378A (en) * | 2009-12-31 | 2011-07-06 | 苏州瑞派宁科技有限公司 | Hepatic tumor comprehensive surgical planning analogy method and system thereof based on three-dimensional multimode images |
| JP6351138B2 (en) * | 2013-03-13 | 2018-07-04 | 国立大学法人 筑波大学 | Diagnostic imaging support program |
| CN104537938A (en) * | 2014-12-21 | 2015-04-22 | 合肥德铭电子有限公司 | Endoscope simulative training system capable of achieving flexible force feedback |
| CN108492693A (en) * | 2018-03-28 | 2018-09-04 | 盛玉涛 | A kind of laparoscopic surgery simulated training system shown based on computer aided medicine |
| CN109243614A (en) * | 2018-09-11 | 2019-01-18 | 深圳先进技术研究院 | A kind of Surgery Simulation methods, devices and systems |
Non-Patent Citations (1)
| Title |
|---|
| "虚拟腹腔镜肝囊肿开窗手术的分析与设计";向泽波;《华中科技大学硕士学位论文》;20170515(第5期);第15-35页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110164557A (en) | 2019-08-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110164557B (en) | Method for simulating and simulating soft tissue surgery path planning by using implicit surface algorithm | |
| US11547499B2 (en) | Dynamic and interactive navigation in a surgical environment | |
| Tolsdorff et al. | Virtual reality: a new paranasal sinus surgery simulator | |
| Nakao et al. | Physics-based interactive volume manipulation for sharing surgical process | |
| Özgür et al. | Preoperative liver registration for augmented monocular laparoscopy using backward–forward biomechanical simulation | |
| CN101393654A (en) | Computer assistant organ operation simulating system | |
| CN103345568A (en) | Method and system for surgical planning based on three-dimensional model | |
| CN101477706B (en) | Simulated operation planning method based on medical image | |
| CN106806021A (en) | A kind of VR surgery simulation systems and method based on human organ 3D models | |
| Wang et al. | A review of virtual cutting methods and technology in deformable objects | |
| Cheng et al. | A novel haptic interactive approach to simulation of surgery cutting based on mesh and meshless models | |
| Pettersson et al. | Simulation of patient specific cervical hip fracture surgery with a volume haptic interface | |
| Pflesser et al. | Specification, modelling and visualization of arbitrarily shaped cut surfaces in the volume model | |
| Shi et al. | A new volumetric geometric model for cutting procedures in surgical simulation | |
| Zhang et al. | Research on registration and navigation technology of augmented reality for ex-vivo hepatectomy | |
| Pflesser et al. | Volume based planning and rehearsal of surgical interventions | |
| Chen et al. | Computer-aided liver surgical planning system using CT volumes | |
| Preim et al. | 3D-Interaction Techniques for Planning of Oncologic Soft Tissue Operations. | |
| Selmi et al. | 3D interactive ultrasound image deformation for realistic prostate biopsy simulation | |
| Shi et al. | A new geometric combination of cutting and bleeding modules for surgical simulation systems | |
| Ribeiro et al. | A 3D computed tomography based tool for orthopedic surgery planning | |
| WO2018030015A1 (en) | Resection process estimation device and resection process navigation system | |
| Neubauer et al. | Advanced virtual corrective osteotomy | |
| Bao et al. | Virtual Surgical Instruments and Surgical Simulation | |
| Sierra et al. | Pathology design for surgical training simulators |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |