CN114404039B - Tissue drift correction method and device for three-dimensional model, electronic equipment and storage medium - Google Patents
Tissue drift correction method and device for three-dimensional model, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114404039B CN114404039B CN202111654380.7A CN202111654380A CN114404039B CN 114404039 B CN114404039 B CN 114404039B CN 202111654380 A CN202111654380 A CN 202111654380A CN 114404039 B CN114404039 B CN 114404039B
- Authority
- CN
- China
- Prior art keywords
- dimensional model
- position information
- tissue
- spatial position
- points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A61B2034/101—Computer-aided simulation of surgical operations
- A61B2034/105—Modelling of the patient, e.g. for ligaments or bones
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2065—Tracking using image or pattern recognition
Landscapes
- Health & Medical Sciences (AREA)
- Surgery (AREA)
- Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Robotics (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Molecular Biology (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Image Processing (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Abstract
The embodiment of the application discloses a tissue drift correction method and device of a three-dimensional model, electronic equipment and a storage medium. The method comprises the following steps: establishing a three-dimensional model based on the acquired medical image data, and extracting a plurality of mark points of a tissue structure in the skull in the three-dimensional model; acquiring spatial position information of a plurality of body surface feature points by using a tracking system, performing rigid registration with a three-dimensional model, and establishing a space-to-model conversion matrix; extracting a plurality of mark points of a tissue structure in the skull in the three-dimensional model, acquiring first space position information of the plurality of mark points, and matching second space position information of the plurality of mark points with the first space position information according to a conversion matrix; calculating the displacement relation between the intra-operation position and the pre-operation position of each mark point to obtain a non-rigid matching relation for drift correction of the three-dimensional model; and correcting the three-dimensional model by using the non-rigid matching relation. The technical scheme of the application can effectively solve the problem that tissue deformation causes three-dimensional digital model drift in operation.
Description
Technical Field
The present invention relates to the field of medical image processing, and in particular, to a method and apparatus for correcting tissue drift of a three-dimensional model, an electronic device, and a storage medium.
Background
With the increasing processing power and efficiency of computer systems, it has become possible to use computer-assisted medical procedures. Under the support of a computer software and hardware system, the prior art can realize an operation navigation system for accurately positioning, feeding back and guiding the position of a surgical knife in the operation process by combining the front-edge technologies such as a digital scanning technology, a microsurgery technology, a stereotactic detection technology and the like, and the technology has extremely important significance for minimally invasive operation or fine operation (such as neurosurgery).
Taking brain surgery as an example, the existing surgical navigation system generally performs complete scanning on the brain of a patient before surgery to pre-establish a three-dimensional digital model, and during the surgery, the position of a surgical knife and the situation of peripheral brain tissues are displayed in the three-dimensional digital model in real time through detection and positioning of the surgical knife so as to help a doctor observe the progress of the surgery and guide subsequent actions. However, in the actual operation execution process, when external protectors (such as dura mater and the like) of brain tissues are opened, the brain tissues can change in position and shape due to intracranial pressure change, gravity, cerebrospinal fluid loss and other factors, so that the three-dimensional structure of the brain tissues in operation is greatly different from that of the brain tissues before operation, and a three-dimensional model established before operation cannot continuously and accurately guide the position of a scalpel.
In order to solve the problem, the prior art further provides some means for registration correction, a target tissue (brain tissue) is obtained through a three-dimensional automatic segmentation algorithm, the segmented brain tissue is gridded, a physical model of the brain tissue is established on the basis of an on-line elasticity theory by endowing each grid unit with corresponding biomechanical properties, finite element calculation is carried out in combination with the physical model, deformation of any position of the whole brain tissue is obtained, and the three-dimensional model is corrected to reduce errors. Or under the condition of not considering the cost, the high-precision operation navigation can be realized by reconstructing the three-dimensional model in real time through intra-operation imaging.
However, the inventor finds that in the process of implementing the technical scheme related to the embodiment of the invention, the prior art has at least the following problems: for landmark points in the three-dimensional model, although there have been some people to form landmark points by collecting the sulcus features of the cerebral cortex; however, in fact, due to the lack of clear and clear structures of the gray matter of the cerebral cortex and the individual differences, the clinical staff feedback that the sulcus of a part of people is shallow, even the expert with abundant experience can not always recognize important structures and sulcus, and reliable mark points are difficult to obtain by taking the sulcus as a target surface for scanning; further, when the biomechanical model is adopted for correction in the prior art, the ditch back mark points are all positioned on the surface of the brain tissue and can not provide effective registration support because of the large volume and nonuniform structure of the brain tissue, so that the correction effect is poor. For real-time intraoperative imaging, on one hand, the cost of related equipment is high, and on the other hand, a great amount of time is needed for multiple real-time intraoperative imaging and modeling, so that the surgical efficiency is seriously influenced, the surgical risk is increased, and the actual effect is also not ideal.
Disclosure of Invention
Aiming at the technical problems in the prior art, the embodiment of the application provides a tissue drift correction method, a device, electronic equipment and a computer readable storage medium of a three-dimensional model, so as to solve the problem of rapid and reliable registration of brain tissue deformation in operation to the three-dimensional digital model.
A first aspect of an embodiment of the present application provides a method for correcting tissue drift of a three-dimensional model, including:
establishing a three-dimensional model based on the acquired medical image data;
the method comprises the steps of acquiring spatial position information of at least three body surface feature points by using a first data acquisition unit under the assistance of a tracking system, and carrying out rigid registration with the three-dimensional model to establish a conversion matrix from a real space to the three-dimensional model; the body surface feature points can be the corners of eyes, nose tips and the like, and can also be bone nails or markers stuck on the surface of the skin;
selecting at least four mark points of a tissue structure in a skull in the three-dimensional model, recording the spatial position information of the mark points in the three-dimensional model as first spatial position information, acquiring the spatial position information of at least three mark points again by using a second data acquisition unit under the assistance of a tracking system after brain tissue is deformed, converting the spatial position information into the spatial position information of the mark points in the three-dimensional model as second spatial position information through the conversion matrix, and matching the second spatial position information of the mark points with the first position information to obtain a non-rigid matching relationship;
and calibrating the three-dimensional model by using the non-rigid matching relation to obtain a calibrated three-dimensional model.
In some embodiments, the process of acquiring the first spatial location information may be performed after the conversion matrix of the three-dimensional model is established and before the second data acquisition unit acquires the spatial location information of at least three of the marker points.
In some embodiments, the medical image data includes one or more of: electron computed tomography (CT, computed Tomography), magnetic resonance imaging (MRI, magnetic Resonance Imaging), X-ray, C-arm, and positron emission computed tomography (PET, positron Emission Tomography).
In some embodiments, the method is a three-dimensional model of the brain, the body surface feature points include facial feature points, such as nasal tips, corners of the eye, etc., markers affixed to the surface, and structures fixedly attached to the skull, such as bone nails, etc., and the internal tissue structure is a brain tissue structure, including vascular structures.
In some embodiments, the extracting the landmark points of the internal tissue structure in the three-dimensional model comprises:
and processing the three-dimensional model according to a deep learning algorithm, segmenting and extracting features to obtain a vascular tissue structure and a plurality of vascular marker points.
In some embodiments, the method further comprises:
the calibrated three-dimensional model is verified using the probe and at least one marker point that was not used in the process of calibrating the three-dimensional model.
In some embodiments, the method wherein the second data acquisition unit acquires spatial location information of the plurality of marker points using at least one of a probe, a laser point cloud, and a contactless laser ultrasound.
In some embodiments, the vascular structure comprises a deep vascular structure of brain tissue, the spatial position information of the deep vascular of brain tissue is acquired by using contactless ultrasonic waves, and then the transformation matrix obtained through registration of the body surface feature points is transformed into the second spatial position information of the deep vascular.
A second aspect of embodiments of the present application provides a tissue drift correction device for a three-dimensional model, including:
the three-dimensional modeling module is used for establishing a three-dimensional model based on the acquired medical image data;
the tracking module is used for acquiring the space position information under the coordinate system of the tracking system;
the registration module is used for carrying out rigid registration on the space position information of at least three body surface feature points acquired by the first data acquisition unit under the assistance of the tracking module and the three-dimensional model, and establishing a registration relation from an actual space to the three-dimensional model;
the model correction module is used for selecting at least four mark points of a tissue structure in a skull in the three-dimensional model, recording the spatial position information of the mark points in the three-dimensional model as first spatial position information, acquiring the spatial position information of at least three mark points again after brain tissue deformation by using a second data acquisition unit, converting the spatial position information of the mark points in the three-dimensional model into second spatial position information by using the conversion matrix, matching the second spatial position information of the mark points with the first position information to obtain a non-rigid matching relation, and correcting the three-dimensional model by using the non-rigid matching relation.
In some embodiments, the device wherein the three-dimensional model is a three-dimensional model of the brain, the body surface feature points are fixation structures and/or devices at the skull and/or face, and the internal tissue structure is a vascular structure.
In some embodiments, the three-dimensional modeling module includes:
and the marker point extraction module is used for processing the three-dimensional model according to a deep learning algorithm, dividing and extracting features to obtain a vascular tissue structure and a plurality of vascular marker points.
In some embodiments, the apparatus further comprises:
and the verification module is used for verifying the correction matrix and/or the corrected three-dimensional model by using the probe and/or at least one mark point out of the mark points.
In some embodiments, the apparatus obtains spatial location information of the plurality of marker points using at least one of a probe, a laser point cloud, and a contactless laser ultrasound.
A third aspect of the embodiments of the present application provides an electronic device, including:
a memory and one or more processors;
wherein the memory is communicatively coupled to the one or more processors, and instructions executable by the one or more processors are stored in the memory, which when executed by the one or more processors, are operable to implement the methods as described in the previous embodiments.
A fourth aspect of the embodiments provides a computer-readable storage medium having stored thereon computer-executable instructions which, when executed by a computing device, are operable to implement the method of the previous embodiments.
A fifth aspect of the embodiments of the present application provides a computer program product comprising a computer program stored on a computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, are operable to carry out the method as described in the previous embodiments.
A sixth aspect of the embodiments of the present application provides a surgical navigation system including a tissue drift correction device of a three-dimensional model as described above, and a data acquisition unit, an image processing unit, and a display unit.
In some embodiments, the data acquisition unit comprises at least one of X-ray, CT, MRI, conventional ultrasound, probe, laser point cloud, and contactless laser ultrasound.
According to the technical scheme, consistency of the three-dimensional digital model is improved through two times of registration and correction of the internal and external double tissue mark points, so that the problem of three-dimensional digital model drift caused by brain tissue deformation in operation is effectively solved.
Drawings
The features and advantages of the present application will be more clearly understood by reference to the accompanying drawings, which are schematic and should not be interpreted as limiting the application in any way, in which:
FIG. 1 is a flow diagram of a method for tissue drift correction of a three-dimensional model, according to some embodiments of the present application;
FIG. 2 is a block diagram of a three-dimensional model tissue drift correction device according to some embodiments of the present application;
fig. 3 is a schematic diagram of an electronic device, according to some embodiments of the present application.
Detailed Description
In the following detailed description, numerous specific details are set forth by way of examples in order to provide a thorough understanding of the relevant disclosure. However, it will be apparent to one of ordinary skill in the art that the present application may be practiced without these specific details. It should be appreciated that the terms "system," "apparatus," "unit," and/or "module" are used herein to describe various elements, components, portions, or assemblies in a sequential order. However, these terms may be replaced with other expressions if the other expressions can achieve the same purpose.
It will be understood that when a device, unit, or module is referred to as being "on," "connected to," or "coupled to" another device, unit, or module, it can be directly on, connected to, or coupled to, or in communication with the other device, unit, or module, or intervening devices, units, or modules may be present unless the context clearly indicates an exception. For example, the term "and/or" as used herein includes any and all combinations of one or more of the associated listed items.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the scope of the present application. As used in the specification and in the claims, the terms "a," "an," "the," and/or "the" are not specific to a singular, but may include a plurality, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" are intended to cover only those features, integers, steps, operations, elements, and/or components that are explicitly identified, but do not constitute an exclusive list, as other features, integers, steps, operations, elements, and/or components may be included.
These and other features and characteristics of the present application, as well as the methods of operation and functions of the related elements of structure, the combination of parts and economies of manufacture, may be better understood with reference to the following description and the accompanying drawings, all of which form a part of this specification. It is to be expressly understood, however, that the drawings are for the purpose of illustration and description only and are not intended as a definition of the limits of the application. It will be understood that the figures are not drawn to scale.
Various block diagrams are used herein to illustrate various modifications of embodiments according to the present application. It should be understood that the preceding or following structures are not intended to limit the present application. The protection scope of the present application is subject to the claims.
The existing operation navigation system can not effectively solve the problem that tissue deformation in operation affects the precision of a pre-built three-dimensional digital model, and particularly the existing ditch-back characteristic mark point mode can not achieve a good correction effect due to large volume and nonuniform structure of brain tissue. In view of this, the embodiment of the application provides a tissue drift correction method for a three-dimensional model, which improves the consistency of the three-dimensional digital model through the two registration and correction of the internal and external double tissue mark points, thereby effectively solving the problem of the three-dimensional digital model drift caused by brain tissue deformation in the operation. In one embodiment of the present application, as shown in fig. 1, the method for correcting the tissue drift of the three-dimensional model includes the steps of:
s101, establishing a three-dimensional model based on medical image data (CT, MRI, PET and the like) containing brain information;
s102, acquiring spatial position information of at least three body surface feature points by using a tracking system (a probe and a laser point cloud), registering with the three-dimensional model, and establishing a conversion matrix from a real space to the three-dimensional model;
s103, selecting at least four mark points of a tissue structure in the skull in the three-dimensional model; the method comprises the steps that a selected point cloud which can be a tissue structure in a skull is used, first space position information of the mark points in the tracking system or the three-dimensional model is recorded, after brain tissue is deformed, three-dimensional space position information of at least three mark points is obtained again by using the tracking system (a probe and a laser point cloud), the space position information of the mark points in the three-dimensional model is converted into second space position information through the conversion matrix, and the second space position information of the mark points is matched with the first position information in the tracking system or the three-dimensional model to obtain a non-rigid matching relation;
and S104, calibrating the three-dimensional model by using the non-rigid matching relation to obtain a calibrated three-dimensional model. Wherein in embodiments of the present application the three-dimensional model is preferably a three-dimensional model of the brain, the body surface feature points are preferably features and/or devices at the skull and/or face etc., such as the corners of the eyes, nose or (implanted) bone nails etc., and the internal tissue structure is preferably a vascular structure. Therefore, the technical scheme is preferably applied to the surgical navigation of brain surgery, and the consistency of the three-dimensional digital model and the actual tissue structure is improved by using the two registration and correction of the external tissue and the internal tissue, so that the problem that the three-dimensional structure deformation of brain tissue in the surgery greatly influences the precision of the three-dimensional digital model and reduces the surgical navigation performance is solved.
Optionally, the non-rigid registration adopts a position information mode based on a three-dimensional model to register the corresponding information of the blood vessel structure of the three-dimensional model, and the corresponding information of the blood vessel structure after cerebral drift is kept consistent,
optionally, the non-rigid registration process includes:
extracting characteristic points of the cerebral vascular structure to obtain first spatial position information of the characteristic points, forming a source curved surface by taking the characteristic points as source points and edges,
then obtaining second space position information of the feature points as sampling points to form a target curved surface;
a local affine transformation is used on the source surface to align it with the vessel after brain drift.
Optionally, the resultant expression mode of the non-rigid registration is a deformation map, specifically including: affine matrix corresponding to each vertex.
Optionally, the affine transformation process of the deformation graph includes: alignment errors, transformation regularization of the vascular structure as a whole and deviations between transformation matrix and rotation matrix,
the influence result of the affine transformation process corresponding to the alignment error is registration validity;
the influence result of the affine transformation process corresponding to the transformation regularization of the whole vascular structure is local consistency;
the result of the influence of the affine transformation process corresponding to the deviation between the transformation matrix and the rotation matrix is local stiffness.
Optionally, the affine transformation result obtaining manner of the deformation graph includes: weighting the consideration factors except the alignment error according to actual requirements, adding all the weighted consideration factors and the alignment error, taking the minimum value as affine transformation result of the transformation graph,
the consideration factors other than alignment error include: and the integral conversion regularization of the source curved surface and the deviation between the transformation matrix and the rotation matrix.
Optionally, the source surface supports a local affine transformation process, and the specific operation process is described above.
Further, in a preferred embodiment of the present application, in the step S101, the three-dimensional model is processed according to a deep learning algorithm, and features are segmented and extracted to obtain a vascular tissue structure and a plurality of vascular landmark points. In the preferred embodiment of the application, the brain blood vessel marker points are mainly used as correction bases, and the robustness of the method in the embodiment of the application is high due to the characteristics of wide distribution range, large differentiation from surrounding tissues and the like of blood vessels.
Optionally, in the step S101, the process of obtaining the vascular tissue structure and the plurality of vascular markers may be further performed in the step S103,
and selecting the execution sequence of the process of obtaining the vascular tissue structure and the plurality of vascular marker points according to actual conditions.
Optionally, the segmentation and feature extraction process uses a 3D Attention U-Net based deep learning model for vessel segmentation;
the 3D Attention U-Net uses jump-connection in structure so as to keep and fuse more detail features (low-level) in the final prediction graph, and the quality of image detail retention;
and by means of a attentive mechanism, the vessels can be better focused than the background (air or other tissue, etc.) to make the prediction graph result more accurate.
Optionally, a model of Models Genesis is selected for pre-training, and the model Genesis improves the spatial feature learning effect of the vascular structure, so that the effect of adapting the model to the vascular segmentation task is improved.
Optionally, the pre-training process includes:
the Models Genesis performs image transformation on the input 3D image, and inputs the 3DAttention U-Net model;
the 3D Attention U-Net restores the transformed 3D image and trains the same.
Optionally, the 3D image content includes brain tissue anatomy, gray scale distribution, and spatial structure of blood vessels.
Optionally, the image transformation content of the 3D image includes: nonlinear transformation, local pixel reconstruction, and image edge blurring.
Optionally, the 3D Attention U-Net training content includes: brain tissue anatomy, gray scale distribution, and spatial structure of blood vessels.
Preferably, in an embodiment of the present application, at least one of a probe and a laser point cloud is used to obtain the spatial position information of the plurality of marker points. For the body surface feature points, the spatial position information is easy to collect, and the probe and the laser point cloud are used for detecting the surface in real time to obtain the body surface feature points; and if the marker points of the internal tissue structure are only positioned on the surface, the laser point cloud can be used for detecting the marker points on the surface, so that the marker points and the spatial position information thereof can be obtained (the spatial position information of the marker points on the surface can also comprise information of a plurality of directions because the human tissue has radian).
Optionally, feature identification of the blood vessel is achieved by scanning the influence, and the identification features of the blood vessel include: vessel color, vessel continuity, and vessel boundaries.
Optionally, the mask of the blood vessel is acquired through image processing, so that a point cloud of the blood vessel is obtained.
However, in the preferred embodiment of the present application, the marker points may be deep tissues, and these surface detection means cannot obtain the ideal spatial position information of deep tissues, which is very expensive in surgery if large-scale equipment (such as X-ray machine, CT machine, MRI, etc.) is used for acquisition; in order to avoid adverse effects on the operation, even contact detection means (such as B ultrasonic and color ultrasonic) should not be used. To address this problem, in a preferred embodiment of the present application, contactless laser ultrasound is used to acquire spatial location information of tissue deep marker points. The principle of the laser ultrasonic mode is as follows, the laser ultrasonic mode transmits laser pulse with specific wavelength to human body, and penetrates the skin to be absorbed by vascular tissues; the vascular tissue is quickly expanded and relaxed by laser heating, is quickly cooled and restored by body temperature, and the process is repeated when the next pulse arrives, so that the generated mechanical vibration forms sound waves. On the basis, another laser wave with the same wavelength is further designed, and acoustic wave signals returned from a human body are received at a certain distance, so that imaging is completed. In this way, the laser ultrasonic technique can scan from a place within half a meter from the human body to obtain information of tissues such as muscles, fat and bones inside the human body, and the method can penetrate at least 6 cm below the skin, and can effectively detect the structure of the tissues at the inner depth in a non-contact manner, thereby enabling deformation correction of deep brain tissues.
In addition, to further enhance the accuracy of the surgical navigation system, in a preferred embodiment of the present application, the correction matrix and/or the corrected three-dimensional model is verified using a probe and/or at least one marker point out of the plurality of marker points. Wherein the checking using the probe comprises: and detecting at least one check point (any point except the mark point used for registration) on the surface or the surface of the internal tissue structure by using a probe, and checking the reliability of the correction matrix and/or the corrected three-dimensional model according to the spatial position information of the check point and the consistency of the position information of the check point in the corrected three-dimensional model. Verifying using at least one marker point other than the plurality of marker points includes: and extracting at least one unused mark point, detecting the spatial position information of the mark point, and checking the reliability of the correction matrix and/or the corrected three-dimensional model according to the consistency of the position information and the real position of the spatial position information in the corrected three-dimensional model.
In the embodiment of the application, the primary registration of the body surface feature points is rigid registration, has small deformation before and during operation, is easy to identify and process, and is mainly used for determining the registration relationship before operation. The internal tissue structure mark points such as blood vessels and the like are larger in deformation before and during operation, uncertainty exists, and secondary registration for calculating displacement relation of the internal tissue structure mark points is non-rigid registration and is used for determining tissue drift degree of real deformation. The three-dimensional model is converted by using the correction matrix T2 to obtain a new three-dimensional model, so that navigation is continued by using the conversion relation T1.
FIG. 2 is a schematic diagram of an apparatus for tissue drift correction of a three-dimensional model shown in accordance with some embodiments of the present application. As shown in fig. 2, the apparatus 200 includes:
a three-dimensional modeling module 210 for creating a three-dimensional model based on the acquired medical image data;
the tracking module 220 is configured to collect spatial location information under a tracking system coordinate system; the registration module 230 performs rigid registration on the spatial position information of at least three body surface feature points acquired by the first data acquisition unit under the assistance of the tracking module and the three-dimensional model, and establishes a registration relationship from an actual space to the three-dimensional model;
the model correction module 240 selects at least four marker points of the tissue structure in the skull in the three-dimensional model, records the spatial position information of the marker points in the three-dimensional model as first spatial position information, acquires second spatial position information of at least three marker points again after brain tissue deformation by using a second data acquisition unit, matches the second spatial position information of the marker points with the first position information to obtain a non-rigid matching relationship, and corrects the three-dimensional model by using the non-rigid matching relationship. The method and the device for correcting the tissue drift of the three-dimensional model are mainly applied to a surgical navigation system, and obviously further comprise hardware units such as acquisition, processing, projection and imaging modules of medical image data, wherein the hardware units are generally realized by adopting existing hardware units in the prior art.
In some embodiments, the device wherein the three-dimensional model is a three-dimensional model of the brain, the body surface feature points are fixation structures and/or devices at the skull and/or face, and the internal tissue structure is a vascular structure.
In some embodiments, the three-dimensional modeling module includes:
and the marker point extraction module is used for processing the three-dimensional model according to a deep learning algorithm, dividing and extracting features to obtain a vascular tissue structure and a plurality of vascular marker points.
In some embodiments, the apparatus further comprises:
and the verification module is used for verifying the correction matrix and/or the corrected three-dimensional model by using the probe and/or at least one mark point out of the mark points.
In some embodiments, the apparatus obtains spatial location information of the plurality of marker points using at least one of a probe, a laser point cloud, and a contactless laser ultrasound.
Referring to fig. 3, a schematic diagram of an electronic device according to an embodiment of the present application is provided. As shown in fig. 3, the electronic device 500 includes:
memory 530 and one or more processors 510;
wherein the memory 530 is communicatively coupled to the one or more processors 510, and a communication bus 540, wherein program instructions 532 executable by the one or more processors are stored in the memory 530, and wherein the program instructions 532 are executed by the one or more processors 510 to cause the one or more processors 510 to perform the steps of the method embodiments described above. Further, the electronic device 500 may also interact with external devices through the communication interface 520.
In some embodiments, the positioning navigation system may be an optical surgical navigation system, an electromagnetic navigation system, or the like;
the optical surgical navigation system mainly comprises: an infrared surgical navigation system and a visible surgical navigation system;
the infrared optical surgical navigation system includes: the system comprises a host computer, an infrared tracking camera system and a probe with a positioning marker.
In some embodiments, the infrared optical surgical navigation system may be used for three-dimensional model rectification, specifically including:
the host receives the medical image and builds a three-dimensional model;
acquiring at least three body surface feature points fixedly connected with the surface of the skull by using a probe, and acquiring the spatial positions of the body surface feature points under the coordinate system of the infrared tracking camera system;
the infrared optical surgery navigation system registers the space position of the bone nail with the three-dimensional model, and establishes a conversion matrix corresponding to the actual space and the three-dimensional model one by one;
selecting at least four brain tissue vascular bifurcation positions as marker points, and displaying the positions of the marker points in the three-dimensional model as first space position information;
after the brain tissue is deformed, the probe acquires the spatial positions of at least three marker points after the brain tissue is deformed, and the spatial positions are converted into the three-dimensional model according to the conversion matrix, and the marker points after the brain tissue is deformed are converted into the coordinate positions in the three-dimensional model to serve as second spatial position information;
the host establishes a non-rigid matching relationship according to the first spatial position information and the second spatial position information of the mark point;
the host computer then calibrates the three-dimensional model through the established non-rigid matching relationship, so that the three-dimensional model is consistent with the deformed brain tissue structure.
Optionally, the body surface feature points include: the corners of the eyes, the nose, the bone nails and the markers stuck on the surface of the skin.
Optionally, the verification may be performed by using other marker points that are not used in the establishment of the non-rigid matching relationship, recording a theoretical coordinate position of the marker point in the three-dimensional model, then acquiring a spatial position of the marker point under tracking of the infrared tracking camera system by using a probe, calculating a position in the three-dimensional model coordinate system by using the transformation matrix as an actual coordinate position, and then calculating a difference between the theoretical position coordinate position and the actual coordinate position as a parameter for evaluating the non-rigid matching relationship.
An embodiment of the present application provides a computer-readable storage medium having stored therein computer-executable instructions that, when executed, perform the steps of the above-described method embodiments.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the apparatus and modules described above may refer to corresponding descriptions in the foregoing method and/or apparatus embodiments, and are not repeated herein.
While the subject matter described herein is provided in the general context of operating systems and application programs that execute in conjunction with the execution of a computer system, those skilled in the art will recognize that other implementations may also be performed in combination with other types of program modules. Generally, program modules include routines, programs, components, data structures, and other types of structures that perform particular tasks or implement particular abstract data types. Those skilled in the art will appreciate that the subject matter described herein may be practiced with other computer system configurations, including hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and the like, as well as distributed computing environments that have tasks performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.
Those of ordinary skill in the art will appreciate that the elements and method steps of the examples described in connection with the embodiments disclosed herein can be implemented as electronic hardware, or as a combination of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or a part of the technical solution, or in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiments of the present application. Whereas the foregoing computer-readable storage media includes physical volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Computer-readable storage media includes, but is not limited to, U disk, removable hard disk, read-Only Memory (ROM), random access Memory (RAM, random Access Memory), erasable programmable Read-Only Memory (EPROM), electrically erasable programmable Read-Only Memory (EEPROM), flash Memory or other solid state Memory technology, CD-ROM, digital Versatile Disks (DVD), HD-DVD, blue-Ray or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium capable of storing the desired information and that can be accessed by a computer.
In summary, the disclosure provides a method and apparatus for correcting tissue drift of a three-dimensional model, an electronic device and a computer readable storage medium thereof. Through the technical scheme of the embodiment of the application, the consistency of the three-dimensional digital model is improved through the two times of registration and correction of the internal and external double tissue mark points, so that the problem of three-dimensional digital model drift caused by brain tissue deformation in operation is effectively solved.
It is to be understood that the above-described embodiments of the present application are merely illustrative of or explanation of the principles of the present application and are in no way limiting of the present application. Accordingly, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present application are intended to be included within the scope of the present application. Furthermore, the appended claims are intended to cover all such changes and modifications that fall within the scope and boundary of the appended claims, or equivalents of such scope and boundary.
Claims (13)
1. A computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a computing device, are operable to implement a method of tissue drift correction of a three-dimensional model, comprising:
establishing a three-dimensional model based on medical image data containing head information;
the method comprises the steps of acquiring spatial position information of at least three body surface feature points by using a first data acquisition unit under the assistance of a tracking system, registering with the three-dimensional model, and establishing a conversion matrix from a real space to the three-dimensional model;
selecting at least four mark points of a tissue structure in a skull in the three-dimensional model, recording the spatial position information of the mark points in the three-dimensional model as first spatial position information, acquiring the spatial position information of at least three mark points again by using a second data acquisition unit under the assistance of a tracking system after brain tissue is deformed, converting the spatial position information into the spatial position information of the mark points in the three-dimensional model as second spatial position information through the conversion matrix, and matching the second spatial position information of the mark points with the first spatial position information to obtain a non-rigid matching relation;
and calibrating the three-dimensional model by using the non-rigid matching relation to obtain a calibrated three-dimensional model.
2. The computer readable storage medium of claim 1, wherein in the tissue drift correction method of the three-dimensional model, the intra-skull tissue structure comprises a vascular structure.
3. The computer-readable storage medium of claim 1, wherein the marker points comprise: intracranial specific anatomical structures and vascular landmark points that can be identified by a physician.
4. The computer-readable storage medium of claim 1, wherein the method further comprises:
the calibrated three-dimensional model is verified using the probe and at least one marker point that was not used in the process of calibrating the three-dimensional model.
5. The computer-readable storage medium of claim 1, wherein the second data acquisition unit acquires the spatial location information of the marker point using at least one of a probe, a laser point cloud, and a contactless laser ultrasound.
6. The computer readable storage medium of claim 2, wherein the vascular structure comprises a deep vascular structure of brain tissue, and wherein the positional information of the marker points of the deep vascular structure of brain tissue is acquired using contactless laser ultrasound.
7. A tissue drift correction device for a three-dimensional model, comprising:
the three-dimensional modeling module is used for establishing a three-dimensional model based on medical image data containing head information;
the tracking module is used for acquiring the space position information under the coordinate system of the tracking system,
the registration module is used for carrying out rigid registration on the spatial position information of at least three body surface feature points acquired by the first data acquisition unit under the assistance of the tracking module and the three-dimensional model, establishing a conversion matrix from an actual space to the three-dimensional model, and recording the spatial position information of the mark points;
the model correction module is used for selecting at least four mark points of a tissue structure in the skull in the three-dimensional model, recording the spatial position information of the mark points in the three-dimensional model as first spatial position information, acquiring second spatial position information of at least three mark points again after brain tissue deformation by using a second data acquisition unit, matching the second spatial position information of the mark points with the first spatial position information to obtain a non-rigid matching relation, and correcting the three-dimensional model by using the non-rigid matching relation.
8. The apparatus as recited in claim 7, further comprising:
and the marker point extraction module is used for processing the three-dimensional model according to a deep learning algorithm, dividing and extracting features to obtain a vascular tissue structure and vascular marker points.
9. The apparatus of claim 7, wherein the apparatus further comprises:
and the verification module is used for verifying the correction matrix and/or the corrected three-dimensional model by using the probe and/or at least one mark point which is not used by the registration module.
10. The apparatus of claim 7, wherein the apparatus obtains the second spatial location information of the marker point using at least one of a probe, a laser point cloud, and a contactless laser ultrasound.
11. The apparatus of claim 7, wherein the vascular structure further comprises a deep vascular structure of brain tissue, and wherein the spatial location information of the marker points of the deep vascular structure of brain tissue is acquired using contactless laser ultrasound.
12. A surgical navigation system, characterized in that it comprises a tissue drift correction device of the three-dimensional model according to any one of claims 7-11.
13. A surgical robotic system comprising a tissue drift correction device of the three-dimensional model of any one of claims 7-11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111654380.7A CN114404039B (en) | 2021-12-30 | 2021-12-30 | Tissue drift correction method and device for three-dimensional model, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111654380.7A CN114404039B (en) | 2021-12-30 | 2021-12-30 | Tissue drift correction method and device for three-dimensional model, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114404039A CN114404039A (en) | 2022-04-29 |
| CN114404039B true CN114404039B (en) | 2023-05-05 |
Family
ID=81268668
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111654380.7A Active CN114404039B (en) | 2021-12-30 | 2021-12-30 | Tissue drift correction method and device for three-dimensional model, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114404039B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115375855B (en) * | 2022-10-25 | 2023-01-10 | 四川公路桥梁建设集团有限公司 | Engineering project visualization method and device, electronic equipment and readable medium |
| CN116630206B (en) * | 2023-07-20 | 2023-10-03 | 杭州安劼医学科技有限公司 | Positioning method and system for rapid registration |
Family Cites Families (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7327865B2 (en) * | 2004-06-30 | 2008-02-05 | Accuray, Inc. | Fiducial-less tracking with non-rigid image registration |
| US8989349B2 (en) * | 2004-09-30 | 2015-03-24 | Accuray, Inc. | Dynamic tracking of moving targets |
| ATE553456T1 (en) * | 2005-02-03 | 2012-04-15 | Bracco Imaging Spa | METHOD AND COMPUTER PROGRAM PRODUCT FOR REGISTRATION OF BIOMEDICAL IMAGES WITH REDUCED OBJECT MOTION-RELEASED IMAGING ARTIFACTS |
| CN101916443B (en) * | 2010-08-19 | 2012-10-17 | 中国科学院深圳先进技术研究院 | Processing method and system of CT image |
| US10290076B2 (en) * | 2011-03-03 | 2019-05-14 | The United States Of America, As Represented By The Secretary, Department Of Health And Human Services | System and method for automated initialization and registration of navigation system |
| WO2014032041A1 (en) * | 2012-08-24 | 2014-02-27 | Old Dominion University Research Foundation | Method and system for image registration |
| JP2015529529A (en) * | 2012-09-13 | 2015-10-08 | コーニンクレッカ フィリップス エヌ ヴェ | Breast surgery guidance based on breast MR images and radioactive markers |
| EP2929510A1 (en) * | 2012-12-10 | 2015-10-14 | The Cleveland Clinic Foundation | Image fusion with automated compensation for brain deformation |
| CN103908344A (en) * | 2012-12-31 | 2014-07-09 | 复旦大学 | Tractive straightening method based on surgical navigation system |
| CN103337065B (en) * | 2013-05-22 | 2016-03-02 | 西安电子科技大学 | The non-rigid registration method of mouse three-dimensional CT image |
| CN103700101B (en) * | 2013-12-19 | 2017-02-08 | 华东师范大学 | Non-rigid brain image registration method |
| CN103955963B (en) * | 2014-04-30 | 2017-05-10 | 崔岩 | Digital human body three-dimensional reconstruction method and system based on Kinect device |
| WO2016095167A2 (en) * | 2014-12-18 | 2016-06-23 | 复旦大学 | Traction deformation correction method based on surgical navigation system |
| GB201506842D0 (en) * | 2015-04-22 | 2015-06-03 | Ucl Business Plc And Schooling Steven | Locally rigid vessel based registration for laparoscopic liver surgery |
| CN106420055A (en) * | 2016-02-15 | 2017-02-22 | 复旦大学 | Brain tissue deformation correction system based on wireless transmission |
| DK3554631T3 (en) * | 2016-12-13 | 2022-05-30 | Novocure Gmbh | TREATMENT OF PATIENTS WITH TTFIELDS WITH ELECTRODEPOSITIONS OPTIMIZED USING DEFORMABLE TEMPLATES |
| KR102370490B1 (en) * | 2020-03-13 | 2022-03-04 | 숭실대학교 산학협력단 | Method for modifying medical images according to modification of virtual object, recording medium and device for performing the method |
| CN111489425B (en) * | 2020-03-21 | 2022-07-22 | 复旦大学 | Brain tissue surface deformation estimation method based on local key geometric information |
| CN113648060B (en) * | 2021-05-14 | 2024-02-27 | 上海交通大学 | Ultrasonic guided soft tissue deformation tracking method, device, storage medium and system |
| CN113298854B (en) * | 2021-05-27 | 2022-02-01 | 广州柏视医疗科技有限公司 | Image registration method based on mark points |
-
2021
- 2021-12-30 CN CN202111654380.7A patent/CN114404039B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN114404039A (en) | 2022-04-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Alam et al. | Medical image registration in image guided surgery: Issues, challenges and research opportunities | |
| CN110325141B (en) | Image integration device and image integration method | |
| US11862348B2 (en) | Systems and methods for using generic anatomy models in surgical planning | |
| EP2081494B1 (en) | System and method of compensating for organ deformation | |
| US11547488B2 (en) | Systems and methods for performing intraoperative image registration | |
| CN109646089B (en) | Intelligent positioning system and method for spinal cord body surface puncture access point based on multi-mode medical fusion image | |
| US10147190B2 (en) | Generation of a patient-specific anatomical atlas | |
| CN111494009B (en) | Image registration method and device for surgical navigation and surgical navigation system | |
| CN114404039B (en) | Tissue drift correction method and device for three-dimensional model, electronic equipment and storage medium | |
| US9757202B2 (en) | Method and system of determining probe position in surgical site | |
| CN105931237A (en) | Image calibration method and system | |
| US10390892B2 (en) | System and methods for updating patient registration during surface trace acquisition | |
| Ebrahimi et al. | Using surface markers for MRI guided breast conserving surgery: a feasibility survey | |
| CN114652443A (en) | Ultrasonic operation navigation system and method, storage medium and device | |
| EP3292835B1 (en) | Ent image registration | |
| CN115775611B (en) | Puncture operation planning system | |
| US20220354579A1 (en) | Systems and methods for planning and simulation of minimally invasive therapy | |
| Bergmeier et al. | Workflow and simulation of image-to-physical registration of holes inside spongy bone | |
| Dekomien et al. | System architecture for intraoperative ultrasound registration in image-based medical navigation | |
| CN114631887A (en) | Method and device for correcting tissue deformation based on blood vessels | |
| CN114451994A (en) | Method and device for correcting breathing real-time image of flexible tissue in abdominal cavity | |
| US20240206973A1 (en) | Systems and methods for a spinal anatomy registration framework | |
| Tomaževič | 3D/2D Registration of medical images | |
| CN116563297A (en) | Craniocerebral target positioning method, device and storage medium | |
| CN116509543A (en) | Composite surgical navigation device, method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |