JPH11197159A - Operation supporting system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an operation supporting system capable of consistently operating operation simulation and operation navigation and raising the level of an operation plan and intervention guidance. SOLUTION: Operation plan data 2 which are the output of an operation simulation function 11 are utilized for the intervention guidance of an operation navigation function 12 and operation evaluation data 3 which are the output of the operation navigation function 12 are fed back to the operation simulation function 11 and utilized for the operation plan of the next time. A virtual patient model including function information as well is utilized and a unified user interface is utilized by the entire system. Thus, the intervention guidance as the operation plan and an appropriate operation plan based on the operation result are made possible.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a surgical operation support system which mainly supports planning and execution of surgical treatment in medical practice.

[0002]

2. Description of the Related Art With the aging of society, it has become an important issue in the field of medical care how to improve the treatment effect without lowering the quality of life of patients. In the field of surgery, a preoperative plan that determines the optimal surgical procedure and approach direction before surgery, and an intraoperative guide to grasp the situation around the surgical field and the relationship between the affected area and surgical instruments during surgery and perform surgery as planned. Two points are particularly important. In preoperative planning, a physician plans an operation based on various surgical procedures and approaches based on the patient's condition.
At this time, if various options can be virtually tried, a more reliable plan can be made.
This is called an operation simulation. For example, Japan ME
Academic Journal BME, Vol. 11, No. 8 (199
7), p. As described in “Surgical simulation using high-dimensional images” described in 18-23, X-ray CT and M
Creates a model of an organ or the like in a virtual space from a medical three-dimensional image such as RI, calculates the optimal approach direction and endoscope visual field prediction before surgery, and uses technologies such as virtual reality (virtual reality). It has been practiced to present it to doctors in an easy-to-understand manner.

On the other hand, in order to safely and reliably approach a target site during intraoperative guidance, it is necessary for a doctor to always grasp the condition around the affected part, the positional relationship between the affected part and a surgical instrument, and the like. By measuring this positional relationship with a sensor or the like and presenting it to the physician in real time, an accurate operation as planned can be performed. This is called surgical navigation.
In this field, for example, in an operation called stereotactic brain surgery, research has mainly been conducted on measuring the position and direction of a surgical instrument with a sensor or the like and displaying the tip position of a puncture needle on a preoperative image. Also, the ME Society of Japan Magazine BME, Vol. 1
1, No. 8 (1997) pp. 3-8 "3D image navigation (Volumegra
ph) ”, 3 of the lesions reconstructed from preoperative images
Attempts have also been made to use augmented reality (augmented reality) technology, which shows a two-dimensional image superimposed on the patient's head using a half mirror.

[0004]

First, in the prior art including the above example, the surgical simulation and the surgical navigation have been performed separately. For example, no matter how optimal an operation plan could be made in a surgery simulation, it was a surgery without any guidance in performing it, or it was a completely separate navigation system,
It was difficult to perform as planned in the surgical simulation. On the other hand, no matter how accurately the surgical navigation can guide the operator, if the surgical plan is incorrect or the patient suddenly changes unexpectedly, the guide must be abandoned. The conventional surgical navigation guides on the assumption that the plan is absolutely correct, so that even if the assumption is broken, there is a problem that it is difficult to directly apply the lesson to the next surgical plan.

[0005] Second, in the known techniques including the above example, a model of a patient / affected part used in a surgical simulation is:
Shape information obtained from medical images and mechanical information calculated based on mechanical characteristics, that is, physical models have been used. Therefore, surgery simulations are limited to physical ones such as determination of puncture direction and prediction of organ deformation, and it is difficult to plan surgery based on changes in blood flow due to surgery and functional predictions such as optimal anesthesia volume. There was a problem.

Third, in the prior art including the above-described example, since the user interface of the surgery simulation and the user interface of the surgery navigation are separate, it is difficult to train the intraoperative operation itself before the operation. There was a problem.

[0007] It is an object of the present invention to solve the above-mentioned problems of the prior art and to provide a surgery support system capable of operating surgery simulation and surgery navigation consistently and enhancing surgery planning and intervention guidance.

[0008]

In the operation support system of the present invention, a simulation for intervention by a doctor is performed by using preoperative patient data such as images and test results as input, and an operation plan such as an optimum operation condition and an optimum approach is performed. A surgical simulation function that outputs data, a surgical navigation function that outputs intraoperative patient data such as images and measured values, and intraoperative guidance for physician intervention, and outputs surgical evaluation data such as adopted surgical conditions and actual approaches. The operation plan data, which is the output of the operation simulation, is used as part of the operation navigation input, and the operation evaluation data obtained by the operation navigation is used as part of the input of the operation simulation for the next and subsequent operations. It has characteristics, not only physical information such as patient shape and mechanical characteristics, but also cardiopulmonary function information and Using the virtual patient model, including the function information such as the good response information, keep common user interface to be used in each stage of surgical navigation and surgical simulation.

According to the present invention, the operation simulation and the operation navigation can be operated consistently, the operation plan and the intervention guidance can be advanced, and the appropriate operation plan can be performed based on the operation guidance and the operation result according to the operation plan.

[0010]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An operation support system 1 according to the present invention
This will be described with reference to FIG. The surgery support system 1 performs a virtual patient construction process 101 for constructing a virtual patient for predicting a result of intervention of the doctor 6 in surgery by using preoperative patient data 41 such as a preoperative image and a preoperative examination result. Next, an operation simulation execution process 102 is performed in which the virtual intervention data 42, which is an intervention of a virtual doctor, is input, the result is predicted based on the information of the virtual patient, and the patient prediction data 43 is output. Virtual intervention data 4 obtained here
Surgery planning process 103 for selecting the optimal combination of virtual intervention data 42 from the set of patient prediction data 43
I do. Finally, the constructed virtual patient information and the set of the optimal virtual intervention data 42 such as the optimal operation conditions and the optimal approach are output as the operation plan data 2.
This is the operation simulation function 11 in the operation support system 1.

The surgery support system 1 receives intraoperative patient data 44 such as an intraoperative image and an intraoperative measured value and the above-mentioned operation plan data 2 and inputs the data of the virtual patient constructed by the preoperative information into an actual operation. A virtual patient correction process 104 for approaching the patient 5 is performed. Next, doctor intervention data 46 such as, for example, the tip position and direction of the puncture needle, obtained by measuring the intervention action 45 actually performed by the doctor 6 on the patient by a sensor, image recognition, and the like, and the above-mentioned patient operation With data 44 as input,
A surgical navigation execution process 105 for outputting intervention guidance data 47 for guiding the intervention action 45 of the doctor 6 to proceed as planned is performed. Intraoperative data 44 obtained here
Then, a surgical result evaluation process 106 for analyzing the doctor intervention data 46 and the intervention guidance data 47 is performed. Eventually, the corrected virtual patient information and the intervention action 45 that was actually adopted and worked, the intervention action 45 that was adopted but the expected effect was not obtained, and the unexpected reaction of the patient Then, a set of the patient's intraoperative data 44 and the doctor's intervention data 46, which extracted the intervention actions 45 and the like that could not be adopted, is output as the surgery evaluation data 3. This is the surgical navigation function 12
It is.

Further, it is assumed that the operation evaluation data 3 is used as input to the operation simulation function 11 for virtual patient construction processing.

As described above, the operation plan data 2 which is the output of the operation simulation function 11 is used as the input of the operation navigation function 12 and the operation navigation function 12 is used.
The operation evaluation data 3 which is the output of the operation simulation is used as an input of the operation simulation function 11 and thereby, it is possible to perform an accurate operation navigation according to the operation plan drafted by the operation simulation, and to obtain the operation navigation result. Making use of the obtained surgical evaluation, it has become possible to perform a more reliable surgical simulation.

Further, in the surgery support system 1, it is possible to construct and use a virtual patient including not only physical information such as the shape and mechanical characteristics of the patient 5 but also functional information such as cardiopulmonary function information and anesthesia reaction information. This enabled surgery simulation and surgical navigation based on functional information such as changes in blood flow due to surgery and optimal anesthesia volume.

Further, in the above-mentioned operation support system, the user interface of the input / output means 13 used in each stage of the operation simulation and the operation navigation is made common, so that the practical operation training by the operation simulation similar to the operation operation can be performed. In addition, it was possible to promptly and reliably guide intervention through surgical navigation while training. In addition, each function 1 in the surgery support system 1
Each of the processes 101, 102, 103 and each of the processes 106, 105, 104 in the steps 1 and 12 are actually executed by the common processing unit 14, and the storage unit 15 is also commonly used in the functions 11 and 12. .

Next, an example of application to laparoscopic cholecystectomy, which has been rapidly developed in recent years, as an excellent invasive technique with little invasion will be described. In laparoscopic cholecystectomy, (1) correctly position the puncture, (2) grasp the bile duct running at the junction of the gallbladder and bile duct in three dimensions, (3) avoid large vessels such as the hepatic artery, etc. If the common bile duct or hepatic artery is accidentally injured, it is necessary to immediately open the laparotomy. To solve these issues, consider using the following methods.

FIG. 2 shows an example of the configuration of the surgery support system. In this example, the integrated processing device 141 is configured by a virtual patient database storage device 151 connected directly or indirectly via the local area network 1001 or the Internet 1002, and an input interface and an output interface. The input interface is a surgical data glove 13 with a position / angle sensor.
1, a laparoscope 132 with a sensor, a surgical instrument 133, and the like. In this case, the input data includes the position and angle of the physician's hand, the movement of each finger joint, the position and
There are angles and operation states. As an output interface, a large two-dimensional or three-dimensional display 134, an HMD (head mounted display) 13 with a sensor,
Use 5 etc. The HMD 135 also has a function of inputting a position, an angle, and the like to the system. The data output in this case includes various information such as images and numerical values based on the virtual patient data during the surgery simulation, and various information based on both the virtual patient data and the actual intraoperative data during the surgery navigation.

Before the operation, the general organs and blood vessels near the operation field are acquired by MRI / CT images, and the running of the bile duct is acquired by DIC-CT (cholangiography) and DIC (cholangiography) fluoroscopic images. Also, virtual patient data is constructed using the parameters of the functional model obtained by performing specialized tests for biochemical tests and simulations. Although it is technically difficult to model all information of the human body, relatively various simulations can be performed by combining circulatory function models, respiratory function models, metabolic function models, anesthesia function models, etc. Becomes Also, in the conventional organ shape model, etc., a bottom-up model construction that reconstructs the model from image data was performed, but in this example, a standard comprehensive model of a virtual patient is prepared and the image data A top-down model construction for creating a virtual patient corresponding to the target patient by changing the parameters of the standard model based on the test data and the test data may be performed.

When the preparation is completed, a surgical simulation is performed. First, while a three-dimensional image is displayed and the doctor grasps the operation field, the computer automatically calculates the optimal puncture direction.
The final decision is made by the doctor. In addition, the operation simulation is provided with a function for calculating the optimal amount of anesthesia and the like, so that anesthesia planning can be performed almost automatically. As a summary of the preoperative plan, rehearsals for virtual surgery are performed using the same instruments as in the actual operation, and problems are identified.

Here, an example of intraoperative navigation will be described with reference to FIG. The display screen 71 of the conventional laparoscope shown in FIG. 3A has a narrow visual field (round visual field), and there is a risk of erroneous recognition of the common bile duct and the gallbladder bile duct. Therefore, the shape model of the virtual patient constructed before surgery was matched with the laparoscopic image,
As shown in FIG. 3B, a 3D image outside the visual field range is complemented and provided to the operator as a display screen 72 of a large 3D display. In addition to displaying not only the surface of the organ but also blood vessels and gallstones inside the organ and displaying the names of important blood vessels and the like in letters, it is possible to reduce blind spots and avoid injuries to blood vessels and the like. It is also effective to add an assist function to the character display so as to present the target site or to issue a warning to prevent mistakes.

The surgical evaluation data is fed back to the surgical simulation, and the more the cases are repeated, the more reliable the virtual patient can be constructed, the more appropriate the preoperative plan, the more accurate the postoperative prediction, and the more realistic simulation. More accurate navigation becomes possible.

Next, as a second embodiment of the present invention, application to a vascular catheter treatment for treating an embolus or a vascular malformation by inserting a catheter into a blood vessel will be described with reference to FIG.
Prior to the operation, an optimal approach route is determined using a surgical simulation function based on images such as contrast images and MRI. During the operation, the position of the catheter tip is
It is detected by a fluoroscopic image or the like, and guidance data is presented on a display screen 81 of a large three-dimensional display using a human body map screen 812, 813 like a current car navigation system. By introducing an endoscope-integrated catheter, insertion while viewing the endoscopic image is possible during the operation, and by recognizing the image during the operation, it is possible to determine which direction to proceed, as in the guidance screen 811. In addition, it is also possible to use a method in which the information is inserted into the display screen 81 and presented by characters or voice.

The present system can be applied not only to minimally invasive surgery but also to general laparotomy and craniotomy procedures by switching the user interface according to the procedure. FIG. 5 shows a third embodiment of the present invention applied to a general surgical procedure. Physician 6 / assistant 60 is a transmissive HMD1
35 is attached. From the position and angle of the HMD135,
The relative position with respect to the affected part is calculated, and the data of the patient / affected part and the virtual patient are overlaid and presented. For example, before the abdomen is opened, it is possible to three-dimensionally grasp the position of the affected part and the positional relationship of the nearby large blood vessels.

A surgical data glove 131 is attached to the hand, and the position and angle of the hand and the movement of the finger are monitored. Surgical instruments used should also be equipped with sensors to warn of dangerous movements.

FIG. 5 is a view of the actual operation in which the actual patient 5 has the patient 5, but an operation simulation for only the virtual patient with the exact same device configuration is also possible. The virtual patient faithfully predicts and responds to the patient's response in the surgical simulation, such as incision of the skin of the virtual patient in response to the movement of the hand of the doctor 6 and the movement of the surgical instrument, exposing the affected area, and reducing blood pressure due to a large amount of bleeding if hemostasis is not performed. By doing so, the difference between the virtual operation and the actual operation is significantly reduced for the doctor 6. Therefore,
A thorough planning by virtual surgery like a real operation, and quick implementation by a real operation like a virtual operation are possible.

The operation simulation function of the present system can be used for a teleconference in which a pre-arrangement of operations and the like is performed between remote locations, and for training and education of students, trainees, and the like. The surgical navigation function is also useful for managing parts that cannot be accessed by human hands, for robotic surgery in which minute treatments and the like are performed using a dedicated robot, and for remote surgery that performs such operations between remote locations.

Further, the operation support system according to the present invention has a possibility to be developed outside the surgical field. If the function model of the virtual patient is enhanced, this system can be applied to the internal medicine and pharmaceutical fields, and can be developed into a comprehensive medical support system not limited to surgery.

On the other hand, the present system can be easily applied to entertainment such as games and theme parks. With the aging of society, interest in healthcare and healthcare is increasing at a very high rate. If everyone can learn the correct medical knowledge while having fun, it will contribute to a great social contribution.

[0029]

According to the operation support system of the present invention, by using the operation plan data obtained by the operation simulation for the operation navigation, it is possible to guide the intervention exactly as planned, and further to obtain the operation navigation. By using the obtained surgery evaluation data for surgery simulation, a more appropriate preoperative plan can be made. By repeating this process repeatedly, the planning and execution of surgery can be performed safely, reliably and promptly with advanced advanced techniques. There is a remarkable effect of becoming something.

Further, by constructing a virtual patient model including not only physical information of a patient / affected part but also functional information, a functional surgery simulation can be performed. There is a remarkable effect that can be applied.

Further, by adopting a user interface common to the operation simulation and the operation navigation, it is possible to perform practical skill training by operation simulation as if performing actual operation and prompt and reliable intervention guidance by operation navigation while performing operation. Has a remarkable effect.

The present system is useful for teleconferences, training and education, robotic surgery and telesurgery, and has a wide and deep potential of being easily applied to entertainment such as games and theme parks. ing.

[Brief description of the drawings]

FIG. 1 is a conceptual diagram illustrating a basic configuration and a basic function of a surgery support system according to the present invention.

FIG. 2 is a diagram showing a configuration example of an embodiment of a surgery support system of the present invention.

FIG. 3 is a diagram illustrating an example in which the surgery support system of the present invention is applied to laparoscopic cholecystectomy.

FIG. 4 is a diagram illustrating an example in which the surgery support system of the present invention is applied to vascular catheter treatment.

FIG. 5 is a view for explaining an example in which the surgery support system of the present invention is applied to a general operation method.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Surgery support system, 2 ... Surgery plan data, 3 ... Surgery evaluation data, 5 ... Patient, 6 ... Doctor, 11 ... Surgery simulation function, 12 ... Surgery navigation function, 13 ...
Input / output means, 14 processing means, 15 storage means, 41 ...
Preoperative patient data, 42 ... virtual intervention data, 43 ... patient prediction data, 44 ... patient intraoperative data, 45 ... doctor intervention data, 46 ... doctor intervention data, 47 ... intervention guidance data, 6
0: assistant, 71: display screen of conventional laparoscope, 72: display screen of large three-dimensional display in application to laparoscopic cholecystectomy, 81: display of large three-dimensional display in application to vascular catheter treatment Display screen, 101 ...
Virtual patient construction processing, 102: Surgery simulation execution processing, 103: Surgery planning processing, 104: Virtual patient correction processing, 105: Surgery navigation execution processing, 106
... Surgical result evaluation processing, 131 ... Surgical data glove with position / angle sensor, 132 ... Stereolaparoscopy with position / angle sensor, 133 ... Forceps with position / angle sensor, 1
34 ... Large three-dimensional display, 135 ... Transmissive liquid crystal stereo HMD with position / angle sensor, 141 ... Integrated control processor, 151 ... Storage device for virtual patient database, 811 ... Guidance screen using intraoperative image, 812 ...
Guidance screen using human body map, 813: Enlarged screen of catheter tip, 1001: Local area network, 1002: Internet.

Claims (3)

[Claims] 1. An operation simulation function for performing simulation for intervention of a physician by using preoperative patient data such as images and examination results as input and outputting operation plan data such as optimal operation conditions and an optimal approach, and an image and measurement function It has a surgical navigation function that outputs intraoperative patient data such as values and intraoperative guidance for doctor intervention, and outputs surgical evaluation data such as adopted surgical conditions and actual approaches. A surgery support system characterized in that data is used as a part of an input for a surgical navigation, and surgical evaluation data obtained by the surgical navigation is used as a part of an input for a subsequent surgical simulation. 2. The operation support system according to claim 1, wherein a virtual patient model including not only physical information such as the shape and mechanical characteristics of the patient but also functional information such as cardiopulmonary function information and anesthesia response information is used. Characterized surgery support system. 3. The operation support system according to claim 1, wherein a user interface used in each stage of operation simulation and operation navigation is shared.