CN114241156A - Device for simulating soft tissue deformation and simulation system - Google Patents
Device for simulating soft tissue deformation and simulation system Download PDFInfo
- Publication number
- CN114241156A CN114241156A CN202111533643.9A CN202111533643A CN114241156A CN 114241156 A CN114241156 A CN 114241156A CN 202111533643 A CN202111533643 A CN 202111533643A CN 114241156 A CN114241156 A CN 114241156A
- Authority
- CN
- China
- Prior art keywords
- soft tissue
- deformation
- module
- simulating
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 210000004872 soft tissue Anatomy 0.000 title claims abstract description 165
- 238000004088 simulation Methods 0.000 title claims abstract description 36
- 238000005520 cutting process Methods 0.000 claims abstract description 31
- 238000000034 method Methods 0.000 claims abstract description 31
- 230000008569 process Effects 0.000 claims abstract description 19
- 238000009877 rendering Methods 0.000 claims abstract description 18
- 238000006073 displacement reaction Methods 0.000 claims description 38
- 239000011159 matrix material Substances 0.000 claims description 35
- 238000013334 tissue model Methods 0.000 claims description 33
- 238000004364 calculation method Methods 0.000 claims description 26
- 238000001514 detection method Methods 0.000 claims description 14
- 238000012800 visualization Methods 0.000 claims description 14
- 238000005192 partition Methods 0.000 claims description 10
- 230000003993 interaction Effects 0.000 claims description 8
- 239000000463 material Substances 0.000 claims description 7
- 238000004891 communication Methods 0.000 claims description 6
- 230000002452 interceptive effect Effects 0.000 claims description 5
- 238000012216 screening Methods 0.000 claims description 4
- 230000011218 segmentation Effects 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 11
- 238000004422 calculation algorithm Methods 0.000 abstract description 8
- 238000005457 optimization Methods 0.000 abstract description 7
- 210000003484 anatomy Anatomy 0.000 description 9
- 238000002591 computed tomography Methods 0.000 description 7
- 210000000988 bone and bone Anatomy 0.000 description 6
- 239000010410 layer Substances 0.000 description 6
- 230000000007 visual effect Effects 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 3
- 238000002595 magnetic resonance imaging Methods 0.000 description 3
- 239000002356 single layer Substances 0.000 description 3
- 238000001356 surgical procedure Methods 0.000 description 3
- 239000003086 colorant Substances 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000001815 facial effect Effects 0.000 description 2
- 238000007654 immersion Methods 0.000 description 2
- 210000003205 muscle Anatomy 0.000 description 2
- 230000002980 postoperative effect Effects 0.000 description 2
- 210000001519 tissue Anatomy 0.000 description 2
- 208000002847 Surgical Wound Diseases 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 230000005284 excitation Effects 0.000 description 1
- 210000004907 gland Anatomy 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000004400 mucous membrane Anatomy 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 239000004576 sand Substances 0.000 description 1
- 210000003491 skin Anatomy 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A61B2034/108—Computer aided selection or customisation of medical implants or cutting guides
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Computer Graphics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Surgery (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Robotics (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Geometry (AREA)
- Processing Or Creating Images (AREA)
Abstract
The application discloses a device and a simulation system for simulating soft tissue deformation. This application accomplishes better balance to simulation system's real-time with the reality of rendering through local deformation algorithm and art district grid optimization, can realize the puncture and the cutting simulation of the arbitrary region of soft tissue moreover to have the effect that puncture site, cutting path are free, carry out quick emulation to the produced soft tissue deformation process of virtual operation moreover, and then realize lifelike vision and render and mechanical feedback, in order to accord with the demand of virtual operation.
Description
Technical Field
The application relates to the technical field of virtual reality, in particular to a device and a simulation system for simulating soft tissue deformation.
Background
Virtual Reality (VR) technology has interactivity, immersion, and imagination. The craniomaxillofacial surgery simulation system developed based on VR technology can assist clinical practice in a plurality of aspects, such as surgical scheme design, postoperative facial form prediction, surgical teaching, operation training and the like. Compared with the traditional clinical teaching and practice process, the virtual surgery simulation system has the advantages of high efficiency, economy and convenience, and has wide development prospect in the medical field.
The majority of craniomaxillofacial surgery simulation systems developed at present only have bone tissue models and operation simulation. For individual simulation systems with soft tissue models, the soft tissue models also do not have the geometric modeling of real biomechanical features. The selection and manipulation of surgical incisions is the most common and challenging step in craniomaxillofacial surgery. The soft tissues of the face have more complex and delicate anatomy and the incision procedure is more tricky than other soft tissue organs. In order to simulate the face incision operation more truly and increase the clinical practicability, the simulation of the operation incision operation is realized by means of a face soft tissue physical model with real biomechanical characteristics, which is very important.
Soft tissue deformation caused by the surgical operation is represented as deformation, destruction and reconstruction of a mesh on the physical model of the soft tissue of the face. If the simulation accuracy of visual rendering and force feedback rendering is to be improved, the generated calculation scale is large, and further the time required by calculation is increased; if the real-time performance of the simulation system on the visual feedback and force feedback rendering of the operation is required to be satisfied, the accuracy of the calculation is required to be lost so as to reduce the calculation scale.
Therefore, for the simulation of soft tissue deformation, research and development personnel are always dedicated to a method which is both true and real-time.
Disclosure of Invention
The utility model provides a purpose for, this application embodiment provides a device and simulation system for simulating soft tissue deformation, it accomplishes better balance to simulation system's real-time nature and rendering authenticity through local deformation algorithm and operation district grid optimization, and can realize the puncture and the cutting simulation in the arbitrary region of soft tissue, thereby have puncture site, the effect of cutting route freedom, and carry out quick simulation to the soft tissue deformation process that virtual operation produced, and then realize lifelike vision rendering and mechanical feedback, in order to accord with virtual operation's demand.
According to a first aspect of the present application, an embodiment of the present application provides a device for simulating soft tissue deformation. The device comprises: the collision detection module is used for judging whether the virtual surgical instrument collides with the three-dimensional soft tissue physical model; when collision is judged, position information and depth information of a collision node are obtained; the grid deformation module is used for calculating an influence area of the three-dimensional soft tissue physical model based on the position information of the collision node; establishing a rigidity matrix for all triangular units in the influence area; inputting collision node depth information to the stiffness matrix to calculate displacement of the remaining nodes of the affected area; and rendering the mesh deformation of the influence area based on the displacement amounts of all the nodes of the influence area.
Optionally, in some embodiments, the apparatus for simulating soft tissue deformation further comprises: a soft tissue physics module comprising a geometric modeling unit and a physical modeling unit; the geometric modeling unit is used for acquiring medical digital image communication data of a virtual object, and performing threshold screening, interactive segmentation and construction on the basis of the scanned image data to obtain a three-dimensional soft tissue model and obtain a grid structure of the three-dimensional soft tissue model; the physical modeling unit is used for setting boundary conditions for the three-dimensional soft tissue model to obtain a virtual partition; acquiring mechanical characteristics of different partitions; and fitting the mechanical characteristics of the different partitions to corresponding partitions of the three-dimensional soft tissue model to obtain the three-dimensional soft tissue physical model.
Optionally, in some embodiments, the geometric modeling unit is further configured to determine a surgical region and a non-surgical region of the three-dimensional soft tissue model according to a preset target, and perform an increment operation on a mesh of the surgical region and a decrement operation on a mesh of the non-surgical region according to a preset rule.
Optionally, in some embodiments, the apparatus for simulating soft tissue deformation further comprises: an instrument modeling module to obtain laser scan image data of a virtual surgical instrument; rendering a three-dimensional virtual surgical instrument model based on laser scan image data of the virtual surgical instrument.
Optionally, in some embodiments, the virtual surgical instrument and the three-dimensional soft tissue model are each comprised of triangular patches.
Optionally, in some embodiments, the apparatus for simulating soft tissue deformation further comprises a puncture operating module and a force feedback obtaining module; the puncture operation module is used for controlling the virtual surgical instrument to move towards the three-dimensional soft tissue physical model; after the collision detection module determines that collision occurs, calculating the feedback force of the collision node based on the depth information of the collision node; judging whether the feedback force is greater than the preset maximum puncture strength; when the feedback force is judged to be smaller than the maximum puncture strength, determining that the grid of the three-dimensional soft tissue physical model is not damaged; when the feedback force is judged to be larger than the maximum puncture strength, determining that the grid of the three-dimensional soft tissue physical model is punctured; judging the inclination angle of the virtual surgical instrument; when the inclination angle of the virtual surgical instrument is judged to be within a first preset angle range, determining the operation of the virtual surgical instrument on the three-dimensional soft tissue physical model to be a puncture operation; the force feedback obtaining module is used for obtaining feedback force of the collision node and outputting the feedback force to the force feedback device based on the obtained feedback force.
Optionally, in some embodiments, the device for simulating soft tissue deformation further comprises a cutting operation module; the cutting operation module is used for determining that the operation of the virtual surgical instrument on the three-dimensional soft tissue physical model is cutting operation when the inclination angle of the virtual surgical instrument is judged to be in a second preset angle range; the mesh deformation module is also used for deconstructing the constraint relation of meshes of the cutting part of the three-dimensional soft tissue physical model; reconstructing the cut grids of the three-dimensional soft tissue physical model; and updates the color and texture of the cut mesh.
Optionally, in some embodiments, the mesh deformation module includes an influence region calculation unit, and the influence region calculation unit is configured to define a stiffness matrix establishment range and obtain calculation parameters of the influence region, where the calculation parameters include a preset displacement error, an elastic modulus of a material of the three-dimensional soft tissue physical model, and a surface skin thickness of the three-dimensional soft tissue physical model; and calculating to obtain the influence area of the three-dimensional soft tissue physical model based on the calculation parameters.
Optionally, in some embodiments, the mesh deformation module comprises a displacement amount calculation unit for constructing a triangular cell stiffness matrix; obtaining the input quantity of the stiffness matrix based on the depth information; and solving the stiffness matrix based on the input quantity of the stiffness matrix to obtain the displacement quantity of all nodes of the affected area.
Optionally, in some embodiments, the device for simulating soft tissue deformation further comprises a procedure assistance module for simulating an auxiliary procedure when performing the virtual surgery; wherein the secondary procedure includes performing punctuation operations to form a virtual surgical path plan.
Optionally, in some embodiments, the apparatus for simulating soft tissue deformation further comprises a visualization module for displaying the mesh deformation of the affected area, and displaying whether the mesh of the three-dimensional soft tissue physical model is broken, and displaying the color and texture of the mesh of the cut three-dimensional soft tissue physical model.
According to a second aspect of the present application, embodiments of the present application provide a simulation system comprising a device for simulating soft tissue deformation, a display device, a force feedback device and an auxiliary input device according to any of the embodiments of the present application; wherein the display device is connected with a visualization module of the device for simulating soft tissue deformation and is used for visually interacting with the visualization module; the force feedback device is connected with the force feedback obtaining module and used for performing tactile interaction with the force feedback obtaining module; the auxiliary input device is connected with the process auxiliary module and used for carrying out auxiliary input interaction with the process auxiliary module.
The embodiment of the application provides a device for simulating soft tissue deformation and a simulation system, and the real-time performance and rendering reality of the simulation system are well balanced through a local deformation algorithm and surgical area grid optimization. The application a device for simulating soft tissue deformation can realize the simulation of puncture and cutting of any region of soft tissue, thereby having the effect of free puncture site and cutting path, and quickly simulating the soft tissue deformation process generated by virtual surgery to realize vivid visual rendering and mechanical feedback so as to meet the requirement of the virtual surgery. In addition, the device can be repeatedly used, so that the cost of operation training can be reduced, and the success rate of operations can be improved.
Drawings
The technical solution and other advantages of the present application will become apparent from the detailed description of the embodiments of the present application with reference to the accompanying drawings.
Fig. 1 is a block diagram of an apparatus for simulating soft tissue deformation according to an embodiment of the present application.
FIG. 2 is a schematic diagram of edge distances of a triangular mesh.
FIG. 3 is a schematic illustration of the impact zone created by a collision node.
Fig. 4 is a schematic view showing the process of the puncturing operation and the cutting operation.
Fig. 5A is a comparison of the effect graph of the puncturing operation and the corresponding grid change graph.
Fig. 5B is a comparison of the effect graph after the frontal skin cutting operation and the corresponding grid change graph.
Fig. 5C is a comparison of the effect graph after the skin incision operation in the masseter region and the corresponding grid change graph.
FIG. 5D is a comparison of an effect graph after a chin skin cutting operation and a corresponding grid change graph.
Fig. 6 is a schematic diagram of a simulation system according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "first", "second" and "first" are used herein for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", may explicitly or implicitly include one or more of the described features. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.
In the description of the present application, it is to be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; may be mechanically connected, may be electrically connected or may be in communication with each other; either directly or indirectly through intervening media, either internally or in any other relationship. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art as appropriate.
The following disclosure provides many different embodiments or examples for implementing different features of the application. In order to simplify the disclosure of the present application, specific example components and arrangements are described below. Of course, they are merely examples and are not intended to limit the present application. Moreover, the present application may repeat reference numerals and/or letters in the various examples, such repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed.
The embodiment of the application provides a device for simulating soft tissue deformation, which comprises: the collision detection module is used for judging whether the virtual surgical instrument collides with the three-dimensional soft tissue physical model; when collision is judged, position information and depth information of a collision node are obtained; the grid deformation module is used for calculating an influence area of the three-dimensional soft tissue physical model based on the position information of the collision node; establishing a rigidity matrix for all nodes in the influence area; inputting depth information to the stiffness matrix to calculate displacement of the remaining nodes of the affected area; and rendering the mesh deformation of the influence area based on the displacement amounts of all the nodes of the influence area.
The device for simulating soft tissue deformation achieves good balance on the real-time performance and rendering reality of the simulation system through a local deformation algorithm and operative region grid optimization. Further, this application a device for simulating soft tissue deformation can realize the puncture of the arbitrary region of soft tissue and the simulation of cutting to have the effect that puncture site, cutting path are free, carry out quick simulation to the soft tissue deformation process that virtual operation produced moreover, in order to realize lifelike vision and render and mechanical feedback, in order to accord with the demand of virtual operation. In addition, the device can be repeatedly used, so that the cost of operation training can be reduced, and the success rate of operations can be improved.
The functions of the various modules of the apparatus 100 for simulating soft tissue deformation will be further described below in conjunction with the drawings.
As shown in fig. 1, the apparatus 100 for simulating soft tissue deformation includes a soft tissue physics module 110, a collision detection module 120, a mesh deformation module 130, an instrument modeling module 140, a puncture operation module 150, a cutting operation module 160, a force feedback obtaining module 170, a procedure assistance module 180, and a visualization module 190.
In particular, the soft tissue physics module 110 includes a geometric modeling unit 111 and a physics modeling unit 112. The geometric modeling unit 111 is configured to obtain medical digital image communication data of a virtual object, perform threshold screening, interactive segmentation and construction based on the scanned image data to obtain a three-dimensional soft tissue model, and obtain a mesh of the three-dimensional soft tissue model. The virtual object refers to a virtual human body, and the following description is the same. The medical digital image communication data (DICOM data) may be CT (computed tomography) data or MRI (magnetic resonance imaging) data. Note that the three-dimensional soft tissue model may be generated from CT data or MRI data. Whereas a three-dimensional bone tissue model can only be generated from CT data. . In the present embodiment, the CT data may be spiral CT data, i.e., spiral computed tomographics. The cranial vertex to the infracervical region of the target subject was photographed with the layer thickness set to 1.25mm to obtain the relevant data and stored as DICOM format data. The three-dimensional soft tissue model refers to a facial model of a human body, and may refer to other parts of the human body in other embodiments.
The geometric modeling unit 111 may perform threshold value screening based on the MITK (medical imaging interactive kit) class library according to the characteristic that the gray value ranges of bone tissues and soft tissues in the obtained image data are different, so as to preliminarily segment the bone tissues and soft tissues in the electronic computed tomography image. Then, interactive segmentation is performed to erase and fill in the parts of the image data where the bone tissue or soft tissue does not conform to the actual anatomical structure. Finally, a three-dimensional soft tissue model of the face (which is a surface model) can be constructed based on the MC algorithm (the algorithm for extracting iso-surfaces from three-dimensional discrete data fields). In the process of constructing the three-dimensional soft tissue model of the face, the three-dimensional soft tissue model of the craniofacial part can be generated in the highest quality three-dimensional mode by adopting a triangular patch mesh, and a storage dimension STL format file is output to prepare for subsequent physical modeling and the like. It should be noted that, in this embodiment, the three-dimensional soft tissue model is formed by a plurality of meshes, and each mesh is a triangular patch. Of course, in some other embodiments, each grid may also be a quadrilateral patch, but is not limited thereto.
In some embodiments, the geometric modeling unit 111 is further configured to divide a mesh of the three-dimensional soft tissue model of the face, i.e., an optimization process of the triangular patch density. For example, based on a preset target, a surgical region and a non-surgical region of the three-dimensional soft tissue model are determined. The preset target may be, but is not limited to, a frontal region, a masseter region, a chin region. For example, the preset target is a masseter region, so that the grid of the operative region can be subjected to incremental refinement, and the grid of the non-operative region is subjected to grid decrement optimization, so that the real-time performance of calculation and the accuracy of grid deformation described below are ensured.
The physical modeling unit 112 is configured to set boundary conditions for the three-dimensional soft tissue model to obtain a virtual partition; acquiring mechanical characteristics of different partitions; and fitting the mechanical characteristics of the different partitions to corresponding partitions of the three-dimensional soft tissue model to obtain the three-dimensional soft tissue physical model. The mechanical properties are actual mechanical properties, that is, information on mechanical properties obtained in an actual surgical operation.
Specifically, the soft tissue includes various anatomical structures such as skin, muscle, mucous membrane, gland and the like, the layers are rich, and the biological and mechanical characteristics of tissues in different anatomical structures and different layers are different, if all the anatomical structures and the tissues in different layers are modeled one by one, the generated calculation scale is very large, and the real-time performance of the device for simulating the soft tissue deformation is difficult to meet. Therefore, in the embodiment of the present application, the soft tissue structure is simplified into a single-layer soft tissue, and boundary conditions are set for the single-layer soft tissue to obtain a virtual partition, that is, a partition for simulating a real anatomy. Since the segmented face has soft tissues with a certain thickness, including skin, muscle, etc., the specific way of simplifying the segmented face into a single-layer soft tissue is to treat all regions of the three-dimensional soft tissue model according to the skin, and the parameters added later are only parameters of the skin.
The boundary conditions described above may be set by the world coordinate values in the algorithm as a partition rule that makes the segmented region contain as much as possible all of the desired anatomical structures and no undesired anatomical structures (e.g., bone tissue). The boundary conditions define anatomical regions for the three-dimensional soft tissue model, and different mechanical characteristics can be set for different regions, for example, fitting of a cubic polynomial of a puncture force equation or definition of cutting feedback force (Y, Z axis) parameters can be realized.
Furthermore, the physical modeling unit 112 is used for assigning material properties of the three-dimensional soft tissue model and for locking the deformation state of the soft tissue.
With continued reference to fig. 1, the instrument modeling module 140 is configured to acquire laser scan image data of a virtual surgical instrument; a three-dimensional virtual surgical instrument model, such as a virtual scalpel, is rendered based on laser scan image data of the virtual surgical instrument. As described above, the virtual surgical instrument and the three-dimensional soft tissue model are each composed of triangular patches. The mesh formed by the triangular patches may be referred to as a triangular mesh, as follows.
The collision detection module 120 is configured to determine whether a virtual surgical instrument collides with the three-dimensional soft tissue physical model; when it is determined that a collision occurs, position information and depth information of a collision node are obtained.
Specifically, when the virtual surgical instrument performs a puncturing or cutting operation, the collision detection module 120 detects a collision node where a collision occurs between the virtual surgical instrument and the three-dimensional soft tissue physical model in a frame cycle to obtain position information and depth information of the collision node, so as to obtain input conditions required by soft tissue mesh deformation and force feedback. The frame cycle means real-time change of a mesh for displaying the three-dimensional soft tissue physical model by each frame of a screen of the display device. In some embodiments, the collision detection comprises a rough detection phase and a detailed detection phase, wherein the detailed detection phase comprises two levels of a step-by-step refinement layer and a precise intersection layer, and adopts a mode based on an axis alignment bounding box structure.
The mesh deformation module 130 is configured to calculate an influence region of the three-dimensional soft tissue physical model based on the position information of the collision node; establishing a rigidity matrix for all triangular units in the influence area; inputting depth information of collision nodes into the stiffness matrix to calculate displacement of the remaining nodes of the affected area; and rendering the mesh deformation of the influence area based on the displacement amounts of all the nodes of the influence area.
Specifically, the mesh deformation module 130 may calculate the influence region of the three-dimensional soft tissue physical model according to the position information of the collision node. If one node on the triangular grid is displaced and the displacement is a fixed amount, or the displacement of other nodes caused by the action of an external force on the node is smaller than a preset error, the node is regarded as affecting other nodes, and an area formed by the affected nodes is defined as an affected area. At this time, the size of the affected area can be characterized by minN. It should be noted that, the distance between two nodes on the triangular mesh is:
lmin＝min(lv1,v2)＝min(ΣEi)
in the above formulav1,v2Representing the edge distance between two nodes, i.e. node V1To node V2The shortest one of the plurality of sides. Sigma EiRepresents the sum of the lengths of the edges, wherein i ≧ 1.
When the triangle shape of the triangular mesh is not greatly distorted and the side length difference is not large, the lminCan be represented as node V1To node V2The minimum number of edges in between corresponds to the sum of the side lengths, i.e.
As described above, the size of the area of influence may be characterized by minN. As shown in fig. 2, if minN is 2, this node only affects a region composed of nodes that are directly connected to the node and are connected to two edges. As shown in FIG. 3, node V0DI, where minN is 3.
The mesh deformation module 130 includes an influence region calculation unit, which is configured to define (i.e., limit) a stiffness matrix establishment range and obtain calculation parameters of an influence region, where the calculation parameters include a preset displacement error, an elastic modulus of a material of the three-dimensional soft tissue physical model, and a surface skin thickness of the three-dimensional soft tissue physical model; and calculating to obtain the influence area of the three-dimensional soft tissue physical model based on the calculation parameters. The area of influence can then be derived from the position information (e.g. point coordinates) of the collision node and by means of the area of influence calculation unit. It should be noted that the area of influence DI can be expressed by the following formula:
DI＝fdi(ε,Emd), where ε represents a predetermined bitError shift, EmThe elastic modulus of the material representing the three-dimensional soft tissue physical model, and d the surface skin thickness of the three-dimensional soft tissue physical model.
From the above formula, the larger the displacement or external force is, the larger the influence area is; the larger the modulus of elasticity, the larger the area of influence.
Further, after the influence region is obtained, a stiffness matrix may be established for all the triangular cells within the influence region, and displacement amounts of all the nodes of the influence region are calculated based on the depth information of the collision node (i.e., displacement amounts of the junction, described below).
In this embodiment, the mesh deformation module 130 includes a displacement amount calculation unit, which is used to construct a triangular cell stiffness matrix; and obtaining the input quantity of the stiffness matrix based on the depth information, and solving the stiffness matrix based on the input quantity of the stiffness matrix to obtain the displacement quantity of all nodes of the affected area. Further, each triangle unit has a stiffness matrix, and a stiffness matrix solution result of one triangle unit (for example, displacement amounts of the other two vertices of the triangle) can be used as an input amount of the stiffness matrix of the adjacent triangle unit, so that displacement amounts of all nodes in the affected area can be obtained through calculation.
It should be noted that the stiffness matrix of the triangular unit is defined as KeWhich represents the resistance to deformation K of a single triangular unite＝λDKDK。
The triangular mesh S is composed of n triangular units, so that the rigidity matrix K of the triangular mesh S in the three-dimensional spacesAnd forming a 3 n-order matrix by the stiffness matrixes of the n triangular units. That is, each triangular cell establishes a stiffness matrix. The rigidity matrix of the whole triangular mesh is obtained by superposing the rigidity matrices of all triangular patches forming the mesh, and a three-order matrix is formed because the three-dimensional matrix is in a three-dimensional space.
Assuming that there exists a set of junction points in the triangular meshes S and S', and the junction points have an initial displacement (for example, displacement occurs due to external force, specifically, the virtual surgical instrument collides with the three-dimensional soft tissue physical model due to stress, so that three meshes in the three-dimensional soft tissue physical model are displaced), the initial displacement is converted into geometric constraint to respectively solve the deformation of the two triangular meshes. Depending on the nature of the bond site, the actual force at the initial displacement of the bond site on each of the two triangular meshes may be calculated. If the calculated forces are of the same magnitude and opposite directions, the initial displacement of the bond site is the correct coordinated deformation position and the calculation is complete. Otherwise, adjusting the position value (namely adjusting the position value of the connecting point) according to the calculated resultant force magnitude and direction of the acting force to obtain a new displacement vector, and repeating the process to carry out iterative solution.
Specifically, assume that there exists a set of junction points in the triangular mesh S and the triangular mesh S', where the corresponding node of a junction point in the triangular mesh S is { V } - { V ═ V }1,V2,...,VnIn the triangular mesh S ', the corresponding node is { V } ' - { V '1,V′2,...,V′n}. When the constraint (i.e. the deformation constraint, the influence between nodes) is established, the initial displacement value of each node is taken as the midpoint of the corresponding node in S and S', i.e. the initial displacement value
{V1,V2,...,VnThe initial displacement values of are:
{V′1,V′2,...,V′nthe initial displacement values of are:
using { δ }iAnd { δ }'.iThe value of (a) is iteratively calculated, and whether the magnitude of the acting force at the junction point satisfies the following formula is judged:
where ε is the convergence error.
If the condition is satisfied, the current displacement value { δ }iAnd { δ }'.iIs the actual displacement value for the node. Otherwise, the iterative displacement value at the relevant junction in S and S' is calculated as follows:
wherein KsFor the stiffness matrix at the relevant node, calculate { F }i+1And { F }'i+1And continuing the iteration process until the iteration error judgment condition is met.
The network deformation module is further configured to render the mesh deformation of the affected area through the visualization module 190 after obtaining the displacement amounts of all the nodes of the affected area.
With continued reference to fig. 1 and (b), (c), and (d) of fig. 4, the puncturing operation module 150 is configured to control the virtual surgical instrument to move toward the three-dimensional soft tissue physical model; after the collision detection module 120 determines that a collision occurs, calculating a feedback force of the collision node based on the depth information of the collision node; judging whether the feedback force is greater than the preset maximum puncture strength; when the feedback force is judged to be smaller than the maximum puncture strength, determining that the grid of the three-dimensional soft tissue physical model is not damaged; when the feedback force is judged to be larger than the maximum puncture strength, determining that the grid of the three-dimensional soft tissue physical model is punctured; judging the inclination angle of the virtual surgical instrument; and when the inclination angle of the virtual surgical instrument is judged to be a first preset angle range, determining that the operation of the virtual surgical instrument on the three-dimensional soft tissue physical model is a puncture operation, wherein the first preset angle range is 90 +/-5 degrees.
In this embodiment, the procedure assistance module 180 may be invoked to simulate an assistance procedure in performing a virtual surgery prior to performing the lancing operation. Wherein the auxiliary process may include performing a punctuation operation to form a virtual surgical path plan, as shown in fig. 4 (a). By using the procedure auxiliary module 180, the simulation performance of the simulation system can be further improved, thereby providing powerful support for surgical plan design, postoperative surface type prediction and operation training.
The force feedback obtaining module is used for obtaining feedback force of the collision node and outputting the feedback force to the force feedback device based on the obtained feedback force. The force feedback device can simulate corresponding force, vibration or passive motion to be transmitted to the user, and the excitation can help the user sense objects in the virtual environment from the sense of touch, directly sense the interaction between the force and the model, and therefore the immersion is increased. The force feedback device can be connected with electronic equipment such as a computer through a network cable interface, an open-source OpenHaptics driving tool kit is installed, after parameters are debugged, the force feedback device can be used for conducting proxy rendering on the calculated feedback force, and the calculated feedback force is displayed through the display device.
As shown in (e) of fig. 4, the cutting operation module 160 is configured to determine that the operation of the virtual surgical instrument on the three-dimensional soft tissue physical model is a cutting operation when the tilt angle of the virtual surgical instrument is determined to be a second preset angle range, where the second preset angle range is 45 ° ± 5 °.
At this time, the mesh deformation module 130 is further configured to determine whether a collision occurs between the virtual surgical instrument and the three-dimensional soft tissue physical model through the collision detection module 120 in a frame cycle when the cutting operation is performed, obtain position information and depth information of a collision node when it is determined that the collision occurs, and continuously deconstruct a constrained relationship of meshes of a cut portion of the three-dimensional soft tissue physical model, reconstruct a cut mesh of the three-dimensional soft tissue physical model, and update colors and textures of the cut mesh.
In this embodiment, the apparatus for simulating soft tissue deformation further comprises a visualization module 190, and the visualization module 190 is configured to display deformation of the mesh of the affected area, and display whether the mesh of the three-dimensional soft tissue physical model is damaged, and display color and texture of the mesh of the cut three-dimensional soft tissue physical model. In this way, after the cutting operation is performed, the triangular mesh of the cut portion in the three-dimensional soft tissue physical model has different colors and materials after being rendered, so that the cutting effect is more consistent with the real situation, and is convenient for the user to observe, as shown in fig. 5A, 5B, 5C and 5D. Further, the visualization module 190 may perform setting of color, illumination, material properties on the three-dimensional soft tissue model through OPENGL tool library, and update the soft tissue in real time at the network node of the deformed region, so that the user may have a visual interaction with the device for simulating the deformation of the soft tissue through a display device (e.g., a display, a projection device, etc.), thereby improving the user experience.
In addition, through the cooperation of the process auxiliary module 180, the mesh deformation module 130, the cutting operation module 160 and the visualization module 190, an arc-shaped incision can be realized, thereby achieving the effect of free cutting path.
Based on the same inventive concept, the present application provides a simulation system 1000, as shown in fig. 6, wherein the simulation system 1000 includes the device 100 for simulating soft tissue deformation, the display device 300, the force feedback device 200 and the auxiliary input device 400 according to any of the embodiments of the present application. Wherein a display device 300 is connected with the visualization module 190 of the device for simulating soft tissue deformation 100 for visual interaction with the visualization module 190; the force feedback device 200 is connected to the force feedback acquisition module 170 for haptic interaction with the force feedback acquisition module 170; the auxiliary input device 400 is coupled to the process assistance module 180 for interacting with auxiliary inputs of the process assistance module 180.
The embodiment of the application provides a device for simulating soft tissue deformation and a simulation system, and the real-time performance and rendering reality of the simulation system are well balanced through a local deformation algorithm and surgical area grid optimization. This application the device can realize the puncture of the arbitrary region of soft tissue and the simulation of cutting to have the effect that puncture site, cutting path are free, carry out quick emulation to the produced soft tissue deformation process of virtual operation moreover, with the visual rendering and the mechanical feedback that realize lifelike, in order to accord with the demand of virtual operation. In addition, the device can be repeatedly used, so that the cost of operation training can be reduced, and the success rate of operations can be improved.
In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
The device and the simulation system for simulating soft tissue deformation provided by the embodiment of the application are introduced in detail, specific examples are applied in the description to explain the principle and the implementation of the application, and the description of the embodiment is only used for helping to understand the technical scheme and the core idea of the application; those of ordinary skill in the art will understand that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications or substitutions do not depart from the spirit and scope of the present disclosure as defined by the appended claims.
Claims (12)
1. An apparatus for simulating soft tissue deformation, the apparatus comprising:
the collision detection module is used for judging whether the virtual surgical instrument collides with the three-dimensional soft tissue physical model; when collision is judged, position information and depth information of a collision node are obtained;
the grid deformation module is used for calculating an influence area of the three-dimensional soft tissue physical model based on the position information of the collision node; establishing a rigidity matrix for all triangular units in the influence area; inputting collision node depth information to the stiffness matrix to calculate displacement of the remaining nodes of the affected area; and rendering the mesh deformation of the influence area based on the displacement amounts of all the nodes of the influence area.
2. The apparatus for simulating a soft tissue deformation of claim 1 wherein the apparatus for simulating a soft tissue deformation further comprises: a soft tissue physics module comprising a geometric modeling unit and a physical modeling unit; the geometric modeling unit is used for acquiring medical digital image communication data of a virtual object, and performing threshold value screening, interactive segmentation and construction on the basis of the medical digital image communication data to obtain a three-dimensional soft tissue model and obtain a grid structure of the three-dimensional soft tissue model; the physical modeling unit is used for setting boundary conditions for the three-dimensional soft tissue model to obtain a virtual partition; and fitting the mechanical characteristics of the different partitions to corresponding partitions of the three-dimensional soft tissue model to obtain the three-dimensional soft tissue physical model.
3. The apparatus for simulating soft tissue deformation of claim 2, wherein the geometric modeling unit is further configured to determine a surgical area and a non-surgical area of the three-dimensional soft tissue model according to a preset target, and perform an increment operation on the mesh of the surgical area and a decrement operation on the mesh of the non-surgical area according to a preset rule.
4. The apparatus for simulating soft tissue deformation of claim 2 wherein the apparatus for simulating soft tissue deformation further comprises: an instrument modeling module to obtain laser scan image data of a virtual surgical instrument; rendering a three-dimensional virtual surgical instrument model based on laser scan image data of the virtual surgical instrument.
5. The apparatus for simulating soft tissue deformation of claim 4 wherein the virtual surgical instrument and the three-dimensional soft tissue model are each comprised of triangular patches.
6. The apparatus for simulating soft tissue deformation of claim 1 wherein the apparatus for simulating soft tissue deformation further comprises a puncture manipulation module and a force feedback acquisition module; the puncture operation module is used for controlling the virtual surgical instrument to move towards the three-dimensional soft tissue physical model; after the collision detection module determines that collision occurs, calculating the feedback force of the collision node based on the depth information of the collision node; judging whether the feedback force is greater than the preset maximum puncture strength; when the feedback force is judged to be smaller than the maximum puncture strength, determining that the grid of the three-dimensional soft tissue physical model is not damaged; when the feedback force is judged to be larger than the maximum puncture strength, determining that the grid of the three-dimensional soft tissue physical model is punctured; judging the inclination angle of the virtual surgical instrument; when the inclination angle of the virtual surgical instrument is judged to be within a first preset angle range, determining the operation of the virtual surgical instrument on the three-dimensional soft tissue physical model to be a puncture operation; the force feedback obtaining module is used for obtaining feedback force of the collision node and outputting the feedback force to the force feedback device based on the obtained feedback force.
7. The apparatus for simulating soft tissue deformation of claim 6 further comprising a cutting operation module; the cutting operation module is used for determining that the operation of the virtual surgical instrument on the three-dimensional soft tissue physical model is cutting operation when the inclination angle of the virtual surgical instrument is judged to be in a second preset angle range; the mesh deformation module is also used for deconstructing the constraint relation of meshes of the cutting part of the three-dimensional soft tissue physical model; reconstructing the cut grids of the three-dimensional soft tissue physical model; and updates the color and texture of the cut mesh.
8. The apparatus for simulating soft tissue deformation according to claim 1, wherein the mesh deformation module includes an influence region calculation unit, the influence region calculation unit is configured to define a stiffness matrix establishing range and obtain calculation parameters of an influence region, and the calculation parameters include a preset displacement error, an elastic modulus of a material of the three-dimensional soft tissue physical model, and a surface skin thickness of the three-dimensional soft tissue physical model; and calculating to obtain the influence area of the three-dimensional soft tissue physical model based on the calculation parameters.
9. The apparatus for simulating soft tissue deformation of claim 1 wherein the mesh deformation module includes a displacement amount calculation unit for constructing a triangular cell stiffness matrix; obtaining the input quantity of the stiffness matrix based on the depth information; and solving the stiffness matrix based on the input quantity of the stiffness matrix to obtain the displacement quantity of all nodes of the affected area.
10. The apparatus for simulating a soft tissue deformation of claim 1 wherein the apparatus for simulating a soft tissue deformation further comprises a procedure assistance module for simulating an assistance procedure in performing a virtual surgery; wherein the secondary procedure includes performing punctuation operations to form a virtual surgical path plan.
11. The apparatus for simulating soft tissue deformation of claim 1 further comprising a visualization module for displaying mesh deformation of the affected area and displaying whether the mesh of the three-dimensional soft tissue physical model is broken and displaying the color and texture of the mesh of the cut three-dimensional soft tissue physical model.
12. A simulation system, comprising the device for simulating soft tissue deformation of any one of claims 1 to 11, a display device, a force feedback device and an auxiliary input device; wherein the display device is connected with a visualization module of the device for simulating soft tissue deformation and is used for visually interacting with the visualization module; the force feedback device is connected with the force feedback obtaining module and used for performing tactile interaction with the force feedback obtaining module; the auxiliary input device is connected with the process auxiliary module and used for carrying out auxiliary input interaction with the process auxiliary module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111533643.9A CN114241156A (en) | 2021-12-15 | 2021-12-15 | Device for simulating soft tissue deformation and simulation system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111533643.9A CN114241156A (en) | 2021-12-15 | 2021-12-15 | Device for simulating soft tissue deformation and simulation system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114241156A true CN114241156A (en) | 2022-03-25 |
Family
ID=80756255
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111533643.9A Pending CN114241156A (en) | 2021-12-15 | 2021-12-15 | Device for simulating soft tissue deformation and simulation system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114241156A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115019877A (en) * | 2022-08-05 | 2022-09-06 | 上海华模科技有限公司 | Method and device for modeling and updating biological tissue model and storage medium |
| WO2024072689A1 (en) * | 2022-09-26 | 2024-04-04 | Intuitive Surgical Operations, Inc. | Systems and methods for determining a force applied to an anatomical object within a subject based on a deformable three-dimensional model |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103400023A (en) * | 2013-06-28 | 2013-11-20 | 华北水利水电大学 | Soft tissue deformation simulation method |
| CN105303605A (en) * | 2015-10-26 | 2016-02-03 | 哈尔滨理工大学 | Orthopedic surgery operation simulation system on the basis of force feedback |
| US20160196645A1 (en) * | 2013-09-03 | 2016-07-07 | Universite Grenoble Alpes | Image processing method based on the finite element method for directly solving inverse problems in structural mechanics |
| CN105913718A (en) * | 2016-07-08 | 2016-08-31 | 哈尔滨理工大学 | Thread lift plastic surgery simulation system |
| CN107330972A (en) * | 2017-06-28 | 2017-11-07 | 华中科技大学鄂州工业技术研究院 | Simulate the real-time soft tissue deformation method and system of biomechanics characteristic |
-
2021
- 2021-12-15 CN CN202111533643.9A patent/CN114241156A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103400023A (en) * | 2013-06-28 | 2013-11-20 | 华北水利水电大学 | Soft tissue deformation simulation method |
| US20160196645A1 (en) * | 2013-09-03 | 2016-07-07 | Universite Grenoble Alpes | Image processing method based on the finite element method for directly solving inverse problems in structural mechanics |
| CN105303605A (en) * | 2015-10-26 | 2016-02-03 | 哈尔滨理工大学 | Orthopedic surgery operation simulation system on the basis of force feedback |
| CN105913718A (en) * | 2016-07-08 | 2016-08-31 | 哈尔滨理工大学 | Thread lift plastic surgery simulation system |
| CN107330972A (en) * | 2017-06-28 | 2017-11-07 | 华中科技大学鄂州工业技术研究院 | Simulate the real-time soft tissue deformation method and system of biomechanics characteristic |
Non-Patent Citations (1)
| Title |
|---|
| 孙剑, 柴建云, 唐泽圣: "手术模拟中软组织变形的实时有限元新算法", 系统仿真学报, no. 05, 20 October 2000 (2000-10-20), pages 553 - 556 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115019877A (en) * | 2022-08-05 | 2022-09-06 | 上海华模科技有限公司 | Method and device for modeling and updating biological tissue model and storage medium |
| CN115019877B (en) * | 2022-08-05 | 2022-11-04 | 上海华模科技有限公司 | Method and device for modeling and updating biological tissue model and storage medium |
| WO2024072689A1 (en) * | 2022-09-26 | 2024-04-04 | Intuitive Surgical Operations, Inc. | Systems and methods for determining a force applied to an anatomical object within a subject based on a deformable three-dimensional model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Nienhuys et al. | A surgery simulation supporting cuts and finite element deformation | |
| US9330502B2 (en) | Mixed reality simulation methods and systems | |
| CN114241156A (en) | Device for simulating soft tissue deformation and simulation system | |
| US10529132B2 (en) | Method for real-time cutting of digital organ based on metaball model and hybrid driving method | |
| US8149237B2 (en) | Information processing apparatus and program | |
| CN109308739A (en) | A kind of soft tissue Minimally Invasive Surgery training method based on virtual reality | |
| JP2010017421A (en) | Method for creating living body data model and its device and data structure of living body data model, and data storing device of living body data model and method for dispersing load of three-dimensional data model and its device | |
| Kim et al. | Haptic interaction and volume modeling techniques for realistic dental simulation | |
| CN108766579B (en) | Virtual brain surgery simulation method based on high-fusion augmented reality | |
| Bielser et al. | Open surgery simulation | |
| Jayasudha et al. | Soft tissues deformation and removal simulation modelling for virtual surgery | |
| CN109243614B (en) | Operation simulation method, device and system | |
| Nienhuys et al. | Supporting cuts and finite element deformation in interactive surgery simulation | |
| CN115457008A (en) | Real-time abdominal puncture virtual simulation training method and device | |
| Al-Khlaifah et al. | A survey of modeling approaches for medical simulators | |
| CN102034272B (en) | Generating method of individualized maxillofacial soft tissue hexahedral mesh | |
| CN114267219B (en) | Virtual operation simulation device and simulation system | |
| CN113379929A (en) | Bone tissue repair virtual reality solution method based on physical simulation | |
| Chen et al. | A displacement driven real-time deformable model for haptic surgery simulation | |
| CN117113452A (en) | Muscle model, construction method thereof and motion simulation method | |
| Lim et al. | On the use of meshfree methods and a geometry based surgical cutting algorithm in multimodal medical simulations | |
| JP2003141566A (en) | Method of simulation for cutting three-dimensional object | |
| Chai et al. | Hybrid FEM for deformation of soft tissues in surgery simulation | |
| Delacruz et al. | Medical manikin augmented reality simulation (M2ARS) | |
| Guo et al. | Efficient collision detection with a deformable model of an abdominal aorta |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |