WO2017107116A1 - Navigation system for minimally invasive operation - Google Patents

Abstract

A navigation system for minimally invasive operation comprises: a pre-operative data module (100) for acquiring a 3D model of the human body; an intra-operative data module (200) for acquiring a pose code, and looking up a corresponding pre-operative time according to the pose code; a positioning module (300) for acquiring position data of operative instrument and human body positioners (301, 201, 302); a navigation module (400) for receiving the 3D model of the human body and the position data of the operative instrument and human body positioners (301, 201, 302), acquiring a model of the operative instrument according to the position data of the operative instrument positioner (301), and displaying the model of the operative instrument and the 3D model of the human body on the basis of the position data of the human body positioners (201, 302) using an augmented reality display device; an intra-operative imaging module (500) for acquiring imaging data of a surgical site when abnormity occurs; and the navigation module (400) for adjusting the 3D model of the human body according to the imaging data. The solution can effectively superimpose a fixed model generated prior to the operation upon the body parts moves with natural respiration in real time during the operation and display the result, thus solving the problem in which the minimally invasive operation is affected by abnormal human respiration during the operation.

Description

Minimally invasive surgery navigation system Technical field

The invention relates to the field of medical device technology, in particular to a minimally invasive surgery navigation system.

Background technique

In traditional surgery, doctors can directly observe the anatomy of the surgical site by performing large-scale wounds on the surface of the human body, which can cause great damage to the patient's body. Emerging minimally invasive surgery is a procedure that allows surgeons to perform surgery without the need for large wounds to the patient, primarily through endoscopes and various imaging techniques. Compared with traditional surgery, minimally invasive surgery can be performed without causing a large wound to the patient, and the damage to the patient is greatly reduced, which is a great advancement in medicine. However, minimally invasive surgery also has higher technical requirements. Because minimally invasive surgery can not directly visually observe the surgical site, doctors must rely on the surgical navigation system to obtain the spatial position and posture information of the human surgical site and surgical instruments, and the stability of the minimally invasive surgical navigation system. Whether it will directly affect the results of minimally invasive surgery, and the ease with which doctors obtain information during surgery can also affect the quality of surgery. Therefore, the effective operation of minimally invasive surgery navigation system is the key to the success of minimally invasive surgery.

The main technical difficulties of the minimally invasive surgery navigation system are summarized as follows:

1 device tracking system: intraoperative tracking of surgical equipment and other surgical equipment and real-time posture of the human body;

2 preoperative modeling: used to determine the physiological structure of the human body in real time during the operation (dynamic process);

3 intraoperative images: used for intraoperative doctors' reference, through the intraoperative interim angiography to obtain real-time image information of the human body during surgery, to avoid the error of the doctor due to the real-time state of the human body and the pre-operative human body modeling. Accidental damage caused by wrong guidance;

4 Human Machine Interface: It is used by doctors to obtain surgical related information and human-computer interaction during surgery.

At present, the most common minimally invasive surgical navigation system is called "lobster" system, and the "lobster" system is based on the biological form of lobster according to the concept of bionics. The "Lobster" minimally invasive surgery navigation system mainly includes the following main parts: the robotic arm console, the binocular tracker, the auxiliary robot arm and the surgical instrument. The mechanical arm console is used for controlling the movement of the mechanical arm, and the real-time spatial positioning information about the object provided by the binocular tracker is used to indicate the moving target of the auxiliary mechanical arm, and the display on the console is used for real-time operation of the doctor during the operation. State observation (mainly the position of the human body, surgical instruments and movement) and the next surgical planning; the binocular tracker tracks the human body by marking points that are attached to the tracking object (the latter is described in the following NDI products) Auxiliary robotic arm and hand The real-time spatial pose of the instrument, and the relevant information is provided to the robotic arm console for analysis and processing; the surgical instrument is clamped by the auxiliary mechanical arm, which can help the doctor to perform auxiliary operation on the patient.

In the patent of "Augmented Endoscopic Minimally Invasive Surgery Navigation System Based on Augmented Reality Technology", the application number is 200910243116.7, it is mentioned that an infrared tracking camera, a three-dimensional scanner, a three-dimensional stereoscopic display and a connected nasal endoscope connected to a computer are used. Reconstructing the intracranial tissue, blood vessels and skin structure of the patient, and superimposing the reconstructed model image rendered by the computer in real time with the real-time image taken by the nasal endoscope according to the endoscopic posture obtained by the infrared tracking camera. In the middle of the surgery for the doctor to enhance the display type of surgery (using a three-dimensional display for rendering), the operation diagram shown in Figure 1.

However, this patent has the following disadvantages: it is suitable for surgery on a relatively fixed human body part of the skull, and surgery for a human body part with real-time exercise volume affected by natural respiration cannot be applied because the program cannot fix a pre-operative computer-generated fixed model. It is effectively superimposed and displayed with the body part that changes with natural breathing in real time; and when the abnormality of the human body breathing occurs during surgery, the organ displacement state during breathing will be greatly different from that obtained before surgery, and the minimally invasive The surgery has an impact.

Summary of the invention

The embodiment of the invention provides a minimally invasive surgery navigation system, which can effectively superimpose and display a fixed model generated by a computer before surgery and a human body part that changes with natural breathing in real time; There is a big difference between the organ displacement status and the preoperative image, which has an impact on minimally invasive surgery.

The minimally invasive surgical navigation system includes:

The preoperative data module is configured to reconstruct four-dimensional CT data at different times in a respiratory cycle collected before surgery to obtain a three-dimensional human body model at different times in a breathing cycle;

The intraoperative data module is configured to collect the posture of the human body locator during the operation by using a tracker, and encode the posture of the human body locator to obtain a pose encoding; and find the pose encoding in the pose lookup table. Corresponding preoperative time;

a positioning module, configured to collect position data of the surgical instrument positioner and position data of the human body positioner through the tracker, and transmit the position data of the surgical instrument positioner and the position data of the human body positioner to the navigation module;

a navigation module, configured to receive and store the pre-operative time, obtain and display a three-dimensional model of the human body from the pre-operative data module according to the pre-operative time; receive the position data of the surgical instrument positioner and the human body locator The position data is obtained according to the position data of the surgical instrument positioner; based on the position data of the human body positioner, the surgical instrument model and the human body three-dimensional model are displayed by using the augmented reality display device.

Intraoperative angiography module is used for real-time angiography of the surgical site when the body posture is abnormal during surgery, obtaining angiographic data, and transmitting the angiographic data to the navigation module;

The navigation module is further configured to adjust and display the three-dimensional human body model according to the contrast data.

In one embodiment, the intraoperative contrast module is specifically configured to determine an abnormality of the posture of the human body during the operation when Δx _{0 is} greater than or equal to Δx;

Where Δx ₀ is the error distance generated by the n body positioner,

i=1, 2, . . . , n, Δx _i is the error distance generated by the i-th body positioner, and Δx is a preset critical error distance.

In one embodiment, the intraoperative contrast module performs real-time imaging of the surgical site through a C-wall or ultrasound device.

In one embodiment, the minimally invasive surgical navigation system further includes:

An independent navigation device for acquiring position data of the human body positioner when the body positioner exceeds the tracking range of the tracker or when there is occlusion between the body positioner and the tracker;

Or, for acquiring the position data of the surgical instrument positioner when the surgical instrument positioner exceeds the tracking range of the tracker or when there is occlusion between the surgical instrument positioner and the tracker.

In one embodiment, the independent navigation device includes a gyroscope, an accelerometer, and a wireless communication module;

The gyroscope is configured to acquire real-time spatial angular acceleration data of a human body locator and/or a surgical instrument locator;

The accelerometer is configured to acquire acceleration data in three coordinate directions of a real-time space of a human body positioner and/or a surgical instrument positioner;

The wireless communication module is configured to transmit acceleration data acquired by the gyroscope and the accelerometer to the navigation module in real time.

In an embodiment, the independent navigation device further includes:

Power supply for independent powering of gyroscopes, accelerometers, and wireless communication modules.

In one embodiment, the gyroscope is a spatial three-phase gyroscope.

In one embodiment, the accelerometer is a spatial three-way acceleration sensor.

In one embodiment, the body positioner and surgical instrument positioner are optical positioners.

In one embodiment, the tracker is an optical tracker.

In the embodiment of the present invention, the three-dimensional model of the human body at different times in the breathing cycle is obtained by the pre-operative data module; the pose of the human body locator is encoded by the intraoperative data module to obtain the pose encoding, and the pose lookup table is obtained. Finding a pre-operative moment corresponding to the pose encoding; receiving a pre-operative moment through the navigation module, and finding a three-dimensional model of the human body at a corresponding moment in the pre-operative data module according to the pre-operative time; according to the position data of the received surgical instrument locator Obtaining a surgical instrument model; using the augmented reality display device to display the surgical instrument model and the human body three-dimensional model based on the position data of the human body locator, so that the computer-generated fixed model and the human body part that changes with natural breathing in real time cannot be effectively performed. Superimposed and displayed problems; when abnormality occurs in the human body during the operation, real-time angiography is performed on the surgical site through the intraoperative contrast module to obtain angiographic data; the navigation module adjusts and displays the three-dimensional model of the human body according to the angiographic data, so that the solution can be solved. The problem of minimally invasive surgery is caused by the abnormal position of the organ resulting in a large difference between the organ displacement condition and the preoperative image.

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present invention. Other drawings may also be obtained from those of ordinary skill in the art in light of the inventive work. In the drawing:

1 is a diagram showing the operation of a nasal endoscopic minimally invasive surgery navigation system based on augmented reality technology according to an embodiment of the present invention;

2 is a schematic structural view of a minimally invasive surgery navigation system according to an embodiment of the present invention;

3 is a system operation diagram of a minimally invasive surgery navigation system according to an embodiment of the present invention;

4 is a diagram showing the composition of a Marker with an independent navigation device in an embodiment of the present invention;

FIG. 5 is a working mechanism diagram of a minimally invasive surgery navigation system according to an embodiment of the present invention.

detailed description

The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. The illustrative embodiments of the present invention and the description thereof are intended to explain the present invention, but are not intended to limit the invention.

In the existing minimally invasive surgery navigation technology, since the fixed model generated by the pre-operative computer and the human body part that changes with the natural breathing in real time cannot be effectively superimposed and displayed, it is only suitable for the relatively fixed human body part surgery of the skull. Surgery for a human body part with real-time exercise volume affected by natural respiration is not applicable; and when an abnormal situation occurs in the human body posture (which can be said to be breathing) during surgery, the organ displacement condition and the pre-operative image are generated. The difference is that it affects minimally invasive surgery. If the fixed model generated by the pre-operative computer can be effectively superimposed and displayed with the body part that changes with natural breathing in real time, and when the abnormal posture of the human body posture (which can be said to be breathing) is performed during the operation, the surgical site can be re-executed. By angiography, the problems in the prior art described above can be solved. Based on this, the present invention proposes a minimally invasive surgical navigation system.

2 is a schematic structural view of a minimally invasive surgery navigation system according to an embodiment of the present invention. As shown in FIG. 2, the minimally invasive surgery navigation system includes:

The pre-operative data module 100 is configured to reconstruct four-dimensional CT data at different times in a respiratory cycle collected before surgery to obtain a three-dimensional human body model at different times in a breathing cycle;

The intraoperative data module 200 is configured to collect the posture of the human body locator during the operation by using a tracker, and encode the posture of the human body locator to obtain a pose encoding; and find the posture in the pose lookup table. Coding the corresponding preoperative time;

The positioning module 300 is configured to collect position data of the surgical instrument positioner and position data of the human body positioner through the tracker, and transmit the position data of the surgical instrument positioner and the position data of the human body positioner to the navigation module;

The navigation module 400 is configured to receive and store the pre-operative time, obtain and display a three-dimensional human body model from the pre-operative data module according to the pre-operative time, and receive the position data and the human body positioning of the surgical instrument positioner. Position data of the device, obtaining a surgical instrument model according to the position data of the surgical instrument positioner; displaying the surgical instrument model and the human body three-dimensional model using the augmented reality display device based on the position data of the human body positioner;

The intraoperative contrast module 500 is configured to perform real-time angiography on the surgical site when the posture of the human body is abnormal during the operation, obtain angiographic data, and send the angiographic data to the navigation module;

The navigation module 400 is further configured to adjust the three-dimensional human body model according to the contrast data.

In a specific implementation, the pre-operative data module 100 includes a 3D database 101, a file lookup table 102, and an input and output module 103.

During the patient's breathing, the human organs will slowly shift with the breathing. The organs of the human body will also move for one cycle in each breathing cycle. Therefore, 4D image acquisition (4D CT) is performed for each respiratory cycle, and the patient can be breathed. 3D image information at each intermittent moment in the process (similar to the production of cartoons).

The specific function of the preoperative data module 100 is: responsible for reconstructing 4D CT (generally taking one breathing cycle, 0.1s interval), and the reconstruction effect is that the human reconstructs a corresponding time 3D data every 0.1 s in a breathing cycle, and according to The imaging time is performed on the 3D database 101; the file lookup table 102 is to locate the location of the file to be extracted in the 3D database 101 according to the recording time provided by the intraoperative data module 200; the input and output module 103 can be used to perform data entry, and request data extraction. Provide file output function after searching.

In specific implementation, the intraoperative data module 200 includes a body locator 201, a tracker, a pose encoding module 203, and a pose lookup table 204.

The tracker tracks the state of the body locator 201, and the pose is encoded by the pose encoding module 203. The code is used to find the pre-recorded time corresponding to the state code of the body locator 201 at this time based on the pose lookup table 204. Pre-stored in the pose lookup table 204 is a code sequence generated by continuously tracking the state of the human body locator 201 in synchronization with the dynamic CT scan, and the recording time corresponding to each code is consistent with the pre-operative contrast time (of course here) All system time systems are required to be fully synchronized).

Specifically, the number of the human body positioners 201 is three or more, and the position locator is used. The position locator is preferably a patch external locator, and the patch external locator is preferably a magnetic patch locator or an optical patch locator. Prior to surgery, the body positioner 201 should be attached as much as possible to the ribs or sternum that are subject to greater changes in breathing, such as under the chest.

Correspondingly, the tracker is preferably a magnetic tracker or an optical tracker.

In a specific implementation, the positioning module 300 includes a surgical instrument positioner 301, a human body positioner 302, an augmented reality display device positioner 303, and a tracker.

The specific function of the positioning module 300 is: tracking the surgical instrument positioner 301 through the tracker to obtain the position data of the surgical instrument positioner; tracking the body positioner 302 through the tracker to obtain the position data of the body positioner; and tracking the augmented reality through the tracker The device locator 303 obtains location data of the augmented reality device locator. The positioning module 300 transmits the position data of the surgical instrument positioner and the position data of the human body positioner to the navigation module for correlation operation and display of the navigation module.

The augmented reality display device locator 303 is tracked by the tracker to obtain the position data of the augmented reality display device and transmitted to the navigation module for related operations and display of the navigation module. Since the existing augmented reality display device itself may also have a positioning function, the augmented reality display device locator 303 can also be cancelled.

Wherein, the tracker is preferably a magnetic tracker or an optical tracker, and the surgical instrument positioner 301 and the body positioner 302 are preferably magnetic or optical positioners. The body positioner 302 is mounted in an area that does not move with the human body (such as a hip or shoulder joint).

The human body positioner 201 and the human body positioner 302 described above are collectively referred to as a human body positioner.

In a specific implementation, the navigation module 400 is configured to receive and store the pre-operative time, obtain and display the human body three-dimensional model from the pre-operative data module 100 according to the pre-operative time; and receive the position data of the surgical instrument locator sent by the positioning module 300. And the position data of the human body positioner, the surgical instrument model is obtained according to the position data of the surgical instrument positioner; based on the position data of the human body positioner, the surgical instrument model and the human body three-dimensional model are displayed by using the augmented reality display device.

Specifically, when noise is mixed in the process of image acquisition, the navigation module 400 also needs to perform denoising operation and image enhancement operation on the image. The navigation module 400 also needs to delineate the lesions and the vascular nerves or organs that need attention in the nearby surgery in the three-dimensional model of the human body before surgery.

The augmented reality display module included in the navigation module 400 is used to display the surgical instrument model and the human body three-dimensional model. The augmented reality display module is mainly an augmented reality display device, and the wearable smart glasses or augmented reality (AR) is a technique for calculating the position and angle of the camera image in real time and adding corresponding images. The goal is to put the virtual world on the screen and interact with the helmet. The effect of the display is that the images are delineated, denoised and enhanced to coincide with the real scene, so that doctors can quickly obtain real-time and intuitive surgery. Navigation information.

In the specific implementation, the intraoperative contrast module 500 is used for real-time angiography of the surgical site when the abnormal posture of the human body during the operation (which can be said to be breathing or other conditions), obtaining angiographic data, and transmitting the angiographic data to the navigation. Module.

In the actual situation, abnormal breathing may occur in the human body, resulting in a large difference in the organ displacement during breathing and the preoperative impact. At this time, it is necessary to use the help of intraoperative angiography guidance, using C-arm or ultrasound. The device illuminates the surgical area and provides the doctor with the most realistic intraoperative data. In this process, it is necessary to make a quantitative judgment on the abnormal situation. The determination method is: there is an n body positioner (or called a marker point), and the error distance generated by the n body positioner is Δx ₀ , that is,

i=1,2,...,n,Δx _i is the error distance generated by the i-th human body locator, and the critical error distance is Δx. When Δx _{0 is} greater than or equal to Δx, it is determined that the critical error distance is exceeded. Intraoperative angiography is required for abnormalities in human breathing. Here, the critical error distance Δx is set according to the number of locator arrays according to the specific surgery. In the cranial surgery, since the respiratory influence is small, the error source is mainly the navigation error. When 10 points are marked, Δx should be set to 1 cm. Left and right, that is, the average navigation error of each marker point cannot exceed 1 mm. In abdominal surgery, due to the large mobility, when the number of marked points is 10, Δx should be set to about 3 cm.

At this time, the navigation module 400 is configured to adjust and display the three-dimensional human body model according to the contrast data.

3 is a system operation diagram of a minimally invasive surgery navigation system according to an embodiment of the present invention. As shown in FIG. 3, the specific execution flow of the minimally invasive surgery navigation system of the present invention is as follows:

First, preoperative preparation:

The human body locator 302 and the human body locator 201 are attached to the patient's body, and the 4D-CT (Four Dimensional Computed Tomography) scan and the posture tracking of the human body locator 201 are simultaneously performed, and the CT data is according to the imaging time. Reconstruction, when each time interval (recommended interval 0.1s) is reconstructed The engraved three-dimensional volume data model, the real-time tracking of the positioner pose sequence is encoded and archived by corresponding time intervals. The three-dimensional volume data is processed by the preoperative operation planning, and the navigation module 400 of the system is used to outline the lesions and the vascular nerves or organs that need attention in the vicinity of the operation before being stored in the 3D database 101. The pose state combination information of the human body positioner 201 is placed in the pose lookup table 203.

The encoding method of the positioner sequence of the locator can be customized. Here, an encoding method of four pose locators is provided: since the pose has directionality, the four locators here can be identified and numbered separately. A1, A2, A3, A4, this encoding method is coded according to "A1-A2", "A2-A3", "A3-A4" mode, where "A?-A?" represents two locators. The spatial orientation relationship between the simplest ones can directly use the coordinate transformation matrix, but this coding method is inconvenient to retrieve and can be stored using the "xyz" spatial angle recording method. The advantage is that there is no accurate data matching. It can be replaced with a data that is closest to the nearest one, resulting in a search result that is closest to the state, avoiding search failures due to discontinuous sampling, and this closest search result reflects the maximum error of 0.05s in medical image acquisition time. , is completely acceptable.

Second, during surgery:

The tracker of the intraoperative data module 200 tracks the pose state of the human body locator 201, and the pose is encoded by the pose encoding module 203. Using the code, the pose corresponding to the state code of the human body locator 201 is searched according to the pose lookup table 204. Before time

The navigation module 400 receives and stores the pre-operative time, and the file lookup table 102 of the pre-operative data module 100 locates the location of the file to be extracted in the 3D database 101 according to the pre-operative time. The input and output module 103 transmits the found human body three-dimensional model data file to the navigation module 400;

The positioning module 300 uses the tracker to calculate the real-time position data (position and posture coordinates) of the surgical instrument positioner 301, the human body positioner 302, and the augmented reality device 303 in a unified coordinate system, and transmits the real-time positioning information to the navigation. Module 400;

The navigation module 400 receives the position data of the surgical instrument positioner and the position data of the human body positioner transmitted from the positioning module 300, obtains the surgical instrument model according to the position data of the surgical instrument positioner, and uses the augmented reality display based on the position data of the human body positioner. The device displays the surgical instrument model and the human body 3D model. Image denoising and image enhancement operations are also required. Finally, the processed image is displayed by the augmented reality display device. If the display module uses wearable smart glasses or an augmented reality helmet, the effect of the display is to delineate and go through the image. The noise and enhanced images coincide with the real scene, allowing doctors to quickly access real-time and intuitive surgical navigation information;

When the abnormality of the human body posture (breathing) occurs during the operation, the intraoperative imaging module 500 (mainly using a C-arm or an ultrasound device) performs real-time imaging on the surgical site, and transmits the contrast data to the navigation module 400;

The navigation module 400 adjusts and displays the three-dimensional human body model according to the contrast data, and continues the operation according to the adjusted three-dimensional human body model.

In the specific implementation, in the patent of "Augmented Reality Technology-based Endoscopic Minimally Invasive Surgery Navigation System" of application number 200910243116.7, there is also the following problem: using the intra-operative tracking device based on infrared tracking, there is no occlusion of the tracking optical path If the situation exceeds the tracking area temporarily, if the situation occurs during the operation, the infrared tracking camera cannot give valid real-time tracking data, which will lead to the failure of the surgical navigation system.

In view of the above problems in the prior art, the system of the present invention further includes an independent navigation device, which can be performed when the tracked device (ie, the human body locator/surgical instrument locator) exceeds the tracking range and occlusion of the binocular tracker. Intermittent effective pose information tracking. As shown in FIG. 4, the independent navigation device includes: a power source, a gyroscope, an accelerometer, and a wireless communication module. among them:

The power supply is used to independently supply power to the gyroscope, accelerometer, and wireless communication module;

The gyroscope is used as an angular acceleration sensor to acquire the real-time spatial angular acceleration data of the Marker;

The accelerometer is used as a spatial acceleration sensor to acquire acceleration data in three coordinate directions of the Marker real-time space;

The wireless communication module is used to transmit the data acquired by the gyroscope and the accelerometer in real time (the relevant data is received by the navigation system host).

The navigation system host must have a wireless receiving module that wirelessly connects to the wireless communication module, which is used to collect data of the gyroscope and the accelerometer in real time. The navigation system host uses the data of the gyroscope and the accelerometer to calculate the relative spatial angle change and spatial position change of the Marker relative to the initial position of the motion. Specifically, the data of the gyroscope can be obtained by the second integral to obtain a relative spatial angle change, and similarly, the relative spatial position change can be obtained by performing second integration on the data of the accelerometer. In this way, when the optical path between the Tracker and the Marker is occluded, the Marker spatial pose of the occluded time is used as the starting spatial pose, and the real-time spatial pose of the Marker can be obtained by using the real-time data of the gyroscope and the accelerometer. .

In a specific implementation, the gyro refers to an angular motion detecting device that uses a momentum moment sensitive housing of a high speed rotating body to orbit the one or two axes orthogonal to the axis of rotation with respect to the inertia space. The gyroscope is an inertial navigation. It can accurately measure the angular acceleration of motion. With the accelerometer, it can measure the acceleration and velocity of the motion. Multiply the speed by the time to obtain the distance of motion. Therefore, in the inertial navigator that requires the highest aircraft missiles, the high-performance gyroscope is one of the most important components. The accuracy of the gyroscope determines the safety of the flight and the ability to accurately hit the target.

Gyroscopes are widely used. For example, most of today's smartphones are also equipped with low-cost gyroscopes. The main problem of gyroscopes is that they cannot be navigated for a long time in real time, because the gyroscope error will gradually become tired as time goes by. Accumulation, resulting in more and more navigation errors. Therefore, the current aircraft and the like generally use inertial navigation (gyro navigation) combined with GPS navigation (satellite navigation). Using low-cost gyroscopes and accelerometers as sensors can perform more accurate navigation in a short period of time, but when the error accumulates at a certain stage, it needs to be calibrated (to eliminate accumulated error), thus ensuring the accuracy of motion.

The use of low-cost sensors to set up independent navigation on the tracked device (Marker) not only enables practical continuous navigation, but also enables cost reduction. The specific ideas are as follows:

The navigation system in the surgical system includes: a binocular tracker (Tracker), and the patient and the surgical instrument are fixed with the tracked device (hereinafter collectively referred to as Marker). The following highlights the working principle of Marker and the working mechanism of the surgical navigation system:

The structure diagram of Marker's working principle is shown in Figure 4:

The analysis is as follows: The traditional Marker is an optical positioning lattice consisting of four reflective spheres with a specific configuration in the same plane. Before tracking navigation, the Marker configuration needs to be input to the Tracker. The Tracker calculates the optical positioning lattice by tracking the optical positioning lattice. Spatial location and posture. If the light path between the Tracker and the optical positioning point is occluded or the positioning point moves out of the Tracker tracking range, the Tracker will lose track of the Marker, causing the navigation system to fail. The invention focuses on the tracking implementation of the Marker spatial position and posture in this case. In the present invention, the independent navigation device is fixed to the independent navigation device by using a low-cost sensor (gyroscope + acceleration sensor). On Marker, Marker is able to independently provide its own spatial position and attitude when the optical path between Tracker and Marker is occluded (need to predict the spatial pose of the moment when using independent navigation devices).

The biggest problem with low-cost sensors compared to optical trackers is that their accuracy degrades over time (sensor drift problems), so it is not possible to rely on Marker's independent navigation devices for long-term spatial pose data. That is to say, if the occlusion time is too long, the spatial pose accuracy obtained by the data of the independent navigation device 500 will become worse and worse until the clinical requirements are not met. As for the length of time allowed for occlusion, depending on the stability of the selected sensor (gyroscope, accelerometer), the better the performance, the longer the precise spatial pose data can be provided, and the longer the occlusion time is allowed. Fortunately, clinical occlusion generally only lasts for a short period of time (half less than 1 minute), and Marker's movement during surgery is not large, so use a lower cost sensor (100 yuan grade) Meet clinical requirements (sub-millimeter level).

Another issue that needs to be addressed is the working mechanism of the surgical navigation system, as shown in Figure 5 below:

When the system works normally, the system preferentially uses the navigation data of the Tracker (high precision and stability). When the optical path is blocked, the navigation system host records the spatial pose data provided by the last time of the Tracker at the occlusion time. As the starting pose of Marker, and using the navigation data provided by Marker's independent navigation device from this moment, and based on this (secondary integral), the Marker real-time spatial pose is calculated.

Because Marker's independent navigation devices have limited time-preserving time, the Marker and Tracker optical paths need to be connected at specific times, and it must be ensured that each optical path is not interrupted for a specific period of time. To put it simply, the Tracker is placed in a position that it will not touch (keeping the world coordinate system fixed), and the Tracker is used to perform the Marker's spatial pose calibration.

The specific length of time for this particular time depends on the actual measurement of the sensor used. The measurement methods are as follows:

Setting spatial position accuracy separately requires distance and spatial angle accuracy requires angle. Starting from a starting position, the optical path between the Tracker and the Marker is kept unobstructed during the process. The navigation system host uses the real-time spatial pose data (x ₁ , y ₁ , z ₁ , θ _x0 , θ _y0 , θ _z0 ) tracked by the Tracker as the reference, and simultaneously utilizes the spatial pose obtained by the simultaneous operation of the Mracker independent navigation device. The data (x ₁ , y ₁ , z ₁ , θ _x1 , θ _y1 , θ _z1 ) is calculated in error with the reference value.

The error calculation is defined as follows:

When Error_dis tan ce ≥ dis tan ce or Error_angle ≥ angle, it is determined that the Marker independent navigation device is invalid and needs to be recalibrated. The time from the start time to the time of failure is the longest calibration time T, that is, the Marker must be calibrated once for each time period of t(t<=T). Note: The optical path is blocked for a maximum duration of no more than T.

In the specific implementation, in the patent of "Augmented Endoscopic Minimally Invasive Surgery Navigation System Based on Augmented Reality Technology", the application number is 200910243116.7, there is also the following problem: in the operation, usually the surgery is performed by the surgeon, and there is a group The expert group conducts remote online consultations, and the nurses perform corresponding auxiliary actions according to the doctor's instructions. The surgeon needs to view the lesion area image (usually an endoscopic image) through a three-dimensional display and view the results of the intraoperative imaging device (C-arm, ultrasound, etc.) (display) for the next surgical operation. The doctor cannot view the actual operation screen and the navigation screen at the same time. If there is a certain deviation in the navigation, the situation cannot be found in real time. In addition, the navigation system does not provide a means for real-time communication by the peripheral expert group, and it is impossible to communicate between the doctor and the rear expert if necessary.

In the system of the present invention, the above problem can be solved by the augmented reality display module 404 (selecting wearable smart glasses or augmented reality helmet) in the navigation module 400.

In recent years, with the release of "Google Glass" smart glasses by Google, there has been a wave of development of smart glasses. The "Google Glass" smart glasses combines a smartphone, a GPS, and a camera to present real-time information in front of the user and enable voice control. Because "Google Glass" is a wearable device, it can be worn lightly, without obscuring the real view, without affecting the normal movement of the human body, and providing a transparent display, and can also effectively make voice calls and video recordings. stand by. "Google Glass" is the ideal device for human-computer interaction between doctors in surgery.

The surgeon can view the transmitted intraoperative angiography or preoperative image through "Google Glass", can perform routine surgery without changing the angle of view, and can perform surgery through the camera on "Google Glass" during the operation. The images are transmitted in real time to online expert panel members who can provide real-time surgical guidance or advice to physicians via online audio. The entire surgical procedure can also be recorded from the perspective of the doctor, and has been used for postoperative insufficiency or for surgical training. In the present invention, the navigation information is presented to the doctor through the smart glasses, and the smart glasses communicate with other devices by means of wireless access to TCP/IP (navigation images and related data transmission), and can perform real-time voice transmission.

It should be pointed out that the development of smart glasses is in full swing. Augmented reality glasses with wireless network transmission function (starting with Google glasses) and virtual reality glasses (Oculus VR) have been greatly developed. Realistic glasses can be easily transplanted into clinical surgical navigation.

In summary, compared with the patent of "Augmented Reality Technology-based Endoscopic Minimally Invasive Surgery Navigation System" of Application No. 200910243116.7, the main improvement work of the present invention has three points:

1) Preoperative image modeling method and improvement of intraoperative human tracking mode: 4D CT images are used to attach external marker points to the human body, and the preoperative images corresponding to the intraoperative human body model will be tracked by tracking the marker group. Quickly correspond, and provide the spatial positional relationship between the surgical instrument and the human computer model for the surgeon to refer to during surgery. Not only can it be applied to the operation of a part of the body that does not move with the breathing movement, but also the operation of a part of the body with a large displacement of the breathing movement;

In addition, an intraoperative contrast module has been added. When an abnormality occurs in the human posture (or breathing), the surgical site can be re-imaged during the operation, and the three-dimensional model of the human body can be adjusted according to the contrast data to continue the operation.

2) Improvements to the tracking range and occlusion situation beyond the binocular tracker: the introduction of independent navigation devices based on gyroscopes and accelerometers, so that the binocular tracker is removed from the tracking range due to occlusion or tracking of the surgical instrument The system can acquire the real-time position of the surgical instrument during independent navigation to ensure the uninterrupted operation of the surgical navigation system;

3) The intraoperative interaction is more convenient. The smart glasses can be used to provide doctors with real-time surgical operation pictures and related prompts. The real-time intraoperative images can be transmitted to the expert group assisting the operation through the wireless network. Both doctors and expert groups can communicate in real time via voice. (The current level of development of 3D display technology is very low, and it is still not up to the medical level).

Obviously, those skilled in the art should understand that the above modules or steps of the embodiments of the present invention can be implemented by a general computing device, which can be concentrated on a single computing device or distributed in multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device such that they may be stored in the storage device by the computing device and, in some cases, may be different from The steps shown or described are performed sequentially, or they are separately fabricated into individual integrated circuit modules, or a plurality of modules or steps thereof are fabricated into a single integrated circuit module. Thus, embodiments of the invention are not limited to any specific combination of hardware and software.

The above described specific embodiments of the present invention are further described in detail, and are intended to be illustrative of the embodiments of the present invention. All modifications, equivalent substitutions, improvements, etc., made within the spirit and scope of the invention are intended to be included within the scope of the invention.

Claims (10)

1. A minimally invasive surgical navigation system, comprising:

   The preoperative data module is configured to reconstruct four-dimensional CT data at different times in a respiratory cycle collected before surgery to obtain a three-dimensional human body model at different times in a breathing cycle;

   The intraoperative data module is configured to collect the posture of the human body locator during the operation by using a tracker, and encode the posture of the human body locator to obtain a pose encoding; and find the pose encoding in the pose lookup table. Corresponding preoperative time;

   a positioning module, configured to collect position data of the surgical instrument positioner and position data of the human body positioner through the tracker, and transmit the position data of the surgical instrument positioner and the position data of the human body positioner to the navigation module;

   a navigation module, configured to receive and store the pre-operative time, obtain and display a three-dimensional model of the human body from the pre-operative data module according to the pre-operative time; receive the position data of the surgical instrument positioner and the human body locator Position data, obtaining a surgical instrument model according to the position data of the surgical instrument positioner; displaying the surgical instrument model and the human body three-dimensional model using the augmented reality display device based on the position data of the human body positioner;

   Intraoperative angiography module is used for real-time angiography of the surgical site when the body posture is abnormal during surgery, obtaining angiographic data, and transmitting the angiographic data to the navigation module;

   The navigation module is further configured to adjust and display the three-dimensional human body model according to the contrast data.
2. The minimally invasive surgery navigation system according to claim 1, wherein the intraoperative contrast module is specifically configured to determine an abnormality of the posture of the human body during the Δx _{0 is} greater than or equal to Δx;

   Where Δx ₀ is the error distance generated by the n body positioner,

   

   Δx _i is the error distance generated by the ith body locator, and Δx is the critical error distance.
3. The minimally invasive surgical navigation system of claim 1 wherein the intraoperative contrast module performs real-time imaging of the surgical site using a C-wall or ultrasound device.
4. The minimally invasive surgery navigation system of claim 1 further comprising:

   An independent navigation device for acquiring position data of the human body positioner when the body positioner exceeds the tracking range of the tracker or when there is occlusion between the body positioner and the tracker;

   Or, for acquiring the position data of the surgical instrument positioner when the surgical instrument positioner exceeds the tracking range of the tracker or when there is occlusion between the surgical instrument positioner and the tracker.
5. A minimally invasive surgical navigation system according to claim 4, wherein said independent navigation device comprises a gyroscope, an accelerometer and a wireless communication module;

   The gyroscope is configured to acquire real-time spatial angular acceleration data of a human body locator and/or a surgical instrument locator;

   The accelerometer is configured to acquire acceleration data in three coordinate directions of a real-time space of a human body positioner and/or a surgical instrument positioner;

   The wireless communication module is configured to transmit acceleration data acquired by the gyroscope and the accelerometer to the navigation module in real time.
6. The minimally invasive surgical navigation system of claim 5, further comprising:

   Power supply for independent powering of gyroscopes, accelerometers, and wireless communication modules.
7. The minimally invasive surgical navigation system of claim 5 wherein said gyroscope is a spatial three-phase gyroscope.
8. The minimally invasive surgical navigation system of claim 5 wherein said accelerometer is a spatial three-way acceleration sensor.
9. The minimally invasive surgical navigation system of claim 1 or 4, wherein the body positioner and the surgical instrument positioner are optical positioners.
10. The minimally invasive surgical navigation system of claim 1 or 4, wherein the tracker is an optical tracker.

Images