JP2007029232A - System for supporting endoscopic operation - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve safety of an endoscopic operation by providing to an operator information regarding various levels of risks. <P>SOLUTION: This system for supporting an endoscopic operation displays a movable range of a manipulator in which an operator can operate a surgical instrument. The system comprises a position detection section 31 for calculating the positional relationship between a driving region of the manipulator and the object organ with which a part of the manipulator may come in contact, a processing section 321 for setting a warning level among various warning levels on the basis of an approaching or abutting condition corresponding to the calculated positional relationship, and a display section 322 for informing visually the set warning level to the operator. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention displays a target organ in a medical diagnostic image photographed by a medical image diagnostic apparatus, and displays the target organ, a driving region of an endoscope and a manipulator in association with each other, and safely supports an endoscopic surgical operation. It is about technology.

  Surgery is shifting from conventional laparotomy or conventional endoscopic surgery to endoscopic surgery in which a doctor trained using, for example, a fine manipulator performs surgery. This endoscopic operation is expected as a minimally invasive operation because it does not require a conventional laparotomy. In addition, by adopting the master-slave system, it is also expected as telemedicine that allows surgery even if a doctor does not go to the place where the patient is.

Endoscopic surgery is performed while confirming the state of the patient's body using a medical image diagnostic apparatus (here, MRI apparatus) (for example, Patent Document 1).
JP 2001-104333 A

  Patent Document 1 mentions the clarification of the movable range of the surgical instrument in the medical diagnostic image and the warning display when it is out of the movable range, but merely displays the movable range of the manipulator. There is no mention of providing the surgeon with warning information corresponding to various levels of risk or ultimately shutting down the manipulator to ensure safety.

  Therefore, an object of the present invention is to improve the safety of endoscopic surgery by providing the surgeon with information corresponding to various levels of risk.

  In order to achieve the above object, an endoscopic surgical operation support system according to the present invention is an endoscopic surgical operation support system that displays a movable range of a manipulator capable of operating a surgical instrument, wherein the manipulator drive region and the manipulator Position calculating means for calculating a positional relationship with a target organ that may partially touch, and warning level setting means for setting an approach or contact condition according to the calculated positional relation as several levels of warning levels; And an informing means for informing the operator of the set warning level.

  According to the present invention, the safety of endoscopic surgery can be improved.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Note that components having the same function are denoted by the same reference symbols throughout the drawings for describing the embodiments of the present invention, and the repetitive description thereof will be omitted.

  In FIG. 1, an endoscopic surgical system includes a medical image diagnostic apparatus 1, a rigid endoscope or a flexible endoscope (hereinafter abbreviated as `` endoscopic apparatus '') 2, a medical image diagnostic apparatus 1 and The image superimposing apparatus 3 connected to the endoscope apparatus 2 so as to be able to transmit signals, the master apparatus 4 installed on the surgeon side, and the slave apparatus 5 installed on the patient side.

  The medical image diagnostic apparatus 1 is an MRI apparatus or an X-ray CT apparatus, for example, and acquires a diagnostic image inside the patient such as a tomogram from the outside of the patient. The endoscope apparatus 2 inserts a probe into a small hole or the like opened by a surgical instrument in a patient, and acquires an image of the inside of the patient at a position where the probe is inserted. The image superimposing apparatus 3 performs organ extraction and superimposition processing on the operation target using the endoscopic image from the endoscopic apparatus 2 and the medical diagnostic image from the medical image diagnostic apparatus 1. The master device 4 is directly operated by the surgeon. The slave device 5 operates the endoscope and the manipulator reflecting the operator's operation.

First, the function of the endoscopic surgery system will be described with reference to FIG.
This endoscopic surgery system enhances the operator's visual information and enhances the quality and quantity of information by strengthening the operator's visual information by superimposing medical diagnostic images on the endoscopic image as one of the intraoperative guide imaging techniques. Reality: augmented reality).

  For this reason, the image superimposing device 3 is the position detecting device 31, and the intraoperative image position information of the medical image diagnostic device 1 (MRI device in the first embodiment) and the position information of the endoscope position manipulator of the endoscope device 2 are displayed. get. Here, the position detection device 31 detects a position from an optical / magnetic position detection device or a position detection marker displayed on a medical diagnostic image. The image processing unit 32 acquires an intraoperative image and an endoscopic image at the same time, performs scale conversion between the acquired images, and performs registration (registration) in the processing unit 321. If the two images are isotropic, they are superimposed as they are, and are superimposed on the display unit 322. The image superimposing apparatus 3 also has a system controller function for controlling the entire system.

The term “superimposition” as defined in this specification means that the positions of all the pixels are exactly the same, and is a structure that can be displayed and referred to in superimposition as required.
On the other hand, if the two images are anisotropic, one image is subjected to distortion correction processing, and both images are converted to isotropic images or converted to anisotropic images having the same distortion. As a distortion correction method, a distortion correction method using Hilbert transform or a distortion correction method using collinearity conditions may be used. In the case of a medical diagnostic image in which the distortion amount is analyzed in advance, the distortion amount calculated in advance Alternatively, distortion correction may be performed.

  In the preprocessing performed by the image superimposing device 3, a region is extracted in advance from the medical diagnostic image by using a region growing method or the like for an organ into which an endoscope is inserted, an organ to be operated, a blood vessel or a nerve fiber that is desired to avoid damage due to surgery. . The area extracted data is transferred to the master / slave device and used when setting a manipulator driving area described later.

  Master-slave devices can be generally classified into two control methods: a unilateral control method and a bilateral control method.

  In the unilateral control method, a command is sent from the master device to the slave device, but the subsequent operation on the object is not related to the master device, and only the slave device is affected. That is, the mechanical work information of the slave device is not fed back to the master device.

  In the bilateral control system, the slave device that receives a command from the master device and the operation on the object act on both the master device and the slave device. Therefore, tactile / force information by contact between the slave device and the object is presented to the operator who handles the master device. However, feedback by the operator's visual information is performed by either control method, and a command can be issued again from that information.

  In this embodiment, the manipulator and the object are limited to the motion and driving area by the area calculated from the superimposed image instead of the tactile / force sensation, and can be applied to both the unilateral control system and the bilateral control system.

Next, the configuration of a unilateral control master / slave device will be described with reference to FIG.
The master-slave device includes the master device 4 (motor 41, position detection unit 42), the motor control unit 6 (position control unit 61, force control unit 62) of the motor 41, the slave device 5 (actuator 51), and image superposition. The apparatus 3 (the position detection unit 31, the image superimposition processing unit 321 and the image superimposition display unit 322 of the slave device 5) and the actuator control unit 7 (position control unit 71, force control unit 72) are included.

  The master device 4 includes an operation unit (not shown) operated by the surgeon, a motor 41 for applying a driving force equivalent to the reaction force from the slave device 5 to the operation unit, and the surgeon against the driving force. A position detection unit 42 that detects a position where the operation unit is operated, and the detected position information L1 is transmitted to the motor control unit 6 and the actuator control unit 7.

  The slave device 5 includes an endoscope 51 and an actuator 51 that controls the manipulator of the slave device 5 that operates according to a command from the master device 4. The position information L2 of the endoscope and manipulator of the slave device 5 is input to the motor control unit and the actuator control unit using the position detection unit 31 of the image superimposition processing unit.

  The position information L1 and L2 are detected by the position detection units 42 and 52 and input to the image processing unit 321. The switching unit 63 of the motor control unit 6 and the switching unit 73 of the actuator control unit 7 selectively switch the position control unit or the force control unit according to the control signals C1 and C2 input from the image superimposition processing unit 3. In the initial state when the apparatus is activated, the switching units 63 and 73 are selected on the position control unit side.

  The image processing unit 321 is a motor control unit for controlling the driving of the endoscope and manipulator by determining contact between the position information L1 and L2 detected by the position detecting units 42 and 52 and the driving region of the endoscope and manipulator. Commands are given to 6 and actuator controller 7. When the manipulator is present in the drive region, the master side device 4 transmits information from the position control unit 61 of the motor control unit 6 to the motor 41 so that the shift amounts of L1 and L2 are equal.

  The slave side device 5 transmits information from the position control unit 71 of the actuator control unit 7 to the actuator 51 so that the L1 and L2 shift amounts are equal. When the manipulator is likely to reach the drive restriction region (region where the manipulator may come into contact with the target organ), the master side device 4 sends the control signal C1 to the switching unit 63 of the motor control unit 6 at the level of the reaching position. To the motor 41, and the slave side device 5 transmits the control signal C2 to the switching unit 73 of the actuator control unit 7 and operates from the force control unit 72. The information before moving the part is transmitted to the motor 51. As a result, the master-slave device is held with the information before the operation unit movement command.

Further, in order to increase the safety of the master-slave device, a mechanism is provided for forcibly stopping the slave side when the operation amount of the operation unit on the master side cannot maintain a safe distance from the target organ. In this forced stop mechanism, the actual value of the positional relationship (distance) between the transmitted manipulator and the target organ is always measured by a sensor in use, and the sensor output and the safety distance are compared and determined by the position control unit 71. When the measured value reaches a predetermined amount, for example 110%, with respect to the safety distance, the manipulator is forcibly stopped. As the forced stop mechanism itself, a publicly known one that can be interrupted by a signal to drive the manipulator is used. This signal needs only one bit indicating whether the actual measurement value is less than or equal to the predetermined amount. In this control example, when the measured value is equal to or smaller than a predetermined amount, the gear for driving the manipulator is removed, and when the predetermined value is exceeded, the connection of the gear is restored.
Accordingly, the manipulator can be controlled (driven and stopped) based on the region set by the image superimposing processing unit 3.

  Next, FIG. 4, FIG. 5, FIG. 6 and FIG. 7 show one embodiment of the driving region setting method and display method of the endoscopic surgery support system set by the image superimposing processing unit 3.

FIG. 4 (a) shows an example in which a virtual image 91 of a surgical instrument such as a forceps or a manipulator is superimposed and displayed on the superimposed image of the actual endoscopic image 8 and the virtual endoscopic image 92. In this display example, an image to be superimposed is distorted and superimposed in accordance with the distortion of the endoscopic image. The virtual endoscopic image 92 is obtained by obtaining a plurality of tomographic images in the body axis direction of a subject with an X-ray CT apparatus or an MRI apparatus and obtaining the obtained tomographic images (for example, see Patent Document 2).
Thus, if the surgeon is familiar with endoscopic surgery, the safety of endoscopic surgery can be improved using familiar images.

  Further, in the superimposed display, only the actual endoscopic image 8 can be displayed by switching whether to display the virtual endoscopic image 92 or not. Thereby, even when an operator only needs an endoscope, the safety of endoscopic surgery can be improved.

  FIG. 4B shows an example in which the drive areas 931 of all joints of the manipulator are displayed. In this example, a sphere indicating the rotation range of the manipulator is displayed superimposed. As a result, the surgeon can immediately know the movable range of each joint of the manipulator by associating it with the endoscopic image, so that the safety of endoscopic surgery can be improved in consideration of the movement of each joint of the manipulator. it can.

  Fig. 4 (c) shows the positional relationship (distance) between the drive region 931 and the target organ 8 and the contact condition to which the weight based on the severity is added. Based on the product, the drive restriction area display 932 by the contact between the drive area and the target organ is set. As a result, the surgeon can immediately know where the symptom is severely associated, and can prioritize the treatment to efficiently support the operation. The safety of the operation can be improved.

In FIG. 4 (d), when a bi-literal master slave device is employed, it is possible to transmit a tactile sensation and reaction force to the master device simultaneously with the display of the drive restriction area. In that case, a signal for applying a load to the motor from the force control unit of the motor control unit 6 may be transmitted as a reaction force. In the drive restriction area 932, a warning level for approaching and touching the manipulator drive area and the drive restriction area is displayed in color in the 2D 94 and 3D images 933 according to the contact conditions. Instead of color display or simultaneously, it may be emitted as a warning sound or vibration. Further, the warning sound and the vibration may be emitted simultaneously. The two-dimensional image 94 may indicate the strength of warning as a bar graph or a pie graph.
Thereby, since attention can be alerted at the contact level according to the positional relationship (distance) between the surgical instrument and the target organ, the safety of endoscopic surgery can be improved by the alerting.

FIG. 5 (a) performs color display 98 on the endoscopic image of a warning level indicating that the manipulator 8 has approached or contacted the drive region or the drive restriction region depending on the contact condition. The color display 98 is color-coded according to the possibility of contact shown in the figure.
As a result, since the contact level corresponding to the positional relationship (distance) between the surgical instrument and the target organ is displayed in direct correspondence with the endoscopic image, the corresponding display can improve the safety of endoscopic surgery. it can.

FIG. 5 (b) displays organ information 99 in which an alarm level indicating that the manipulator 8 is approaching or contacting the driving area or the driving restriction area depending on the contact condition is developed on the endoscopic image. For organ information 99, a 3D image is created from a medical image acquired by an MRI device, etc., and the created 3D image is distorted according to the coordinate system of the endoscopic image, and the distorted 3D image is endoscopically viewed. Superimposed on the mirror image.
Thereby, since the positional relationship (distance) between the surgical instrument and the target organ is intuitively displayed, the safety of endoscopic surgery can be improved by the intuitive display.

FIG. 5 (c) corresponds to the fact that when the manipulator 8 is virtually moved, the virtually moved manipulator 8 approaches and contacts the drive area or the drive restriction area depending on the contact condition. That is, in the drive restriction area 932, a warning level for approaching and touching the manipulator drive area or the drive restriction area depending on the contact condition is displayed in two dimensions 94 in color.
As a result, since the positional relationship (distance) between the surgical instrument and the target organ in a virtually moved state is displayed, attention can be drawn by the virtual moved display, which can improve the safety of endoscopic surgery. it can.

FIG. 5 (d) shows the movement of the manipulator 8 that is virtually moved so as to maintain a safe distance that does not approach the target organ or the like when the manipulator 8 is virtually moved. That is, the arrow 91 indicating the movement in which the manipulator 8 maintains the safe distance before and after the movement is displayed.
As a result, since the safe range of the positional relationship (distance) between the surgical instrument and the target organ in the virtually moved state is displayed, the operator can operate with peace of mind by displaying the safe range. The safety of the operation can be improved.

The drive restriction area is an example shown in a three-dimensional image as shown in FIGS. 4 (b) (c) (d), and on the axial image 95, as shown in FIGS. 6 (a) (b) (c), on the sagittal image 96, It can be displayed superimposed on the coronal image 97. At the same time, the manipulator is virtually projected and displayed on the cross section of the drive restriction region and the cut surface 934.
Thereby, since it can grasp | ascertain three-dimensionally from the surgical instrument and the image of a multifaceted direction, the safety | security of endoscopic surgery can be improved by the three-dimensional grasp.

  While checking the weight by severity on the monitor 110 as shown in Fig. 7, set a free curve, straight line, rectangle etc. 103 for each extracted organ (104 for each object) and change it according to the severity in the organ There is also. Severity weighting is set by the user freely using the mouse 105 by the graphical user interface 100 as shown in FIG. Therefore, as shown in FIG. 8 (a), a nonlinear weight coefficient 111 is given to each complexly intertwined target in the organ to set the degree of danger. Further, as shown in FIG. 8 (b), a weight is set to a rectangle 112 by setting a large weight on a part (for example, a blood vessel or a nerve) where contact should be avoided in the operation or setting a small weight on the part to be operated. Thus, it is possible to avoid inadvertent collision between the manipulator and the target organ and accurately position the target organ, and the safety of the endoscopic surgery can be improved by the positioning.

  FIG. 9 shows a configuration of a master-slave device of a bilateral control system. The signal added to FIG. 3 is a signal F1, which is transmitted to the motor control unit 6 and the actuator control unit 7, and converted by the force control units 62 and 72, respectively, to drive and control the motor 41 and the actuator 51.

  In the present embodiment, information necessary for assisting endoscopic surgery can be obtained from the operator's eyes and ears, and the manipulator is driven and stopped in a backup manner. For example, the following application examples can be considered by superimposing virtual information on a real endoscope.

(a) Simulation information obtained by image processing can be added to a virtual endoscopic image.
By displaying, hiding, or combining and displaying a virtual endoscopic image to which information has been added, for example, it is possible to understand the extent of lesion spread and the degree of infiltration that are difficult to determine with the actual endoscopic image alone. In addition, even an organ that can be confirmed with a real endoscopic image may be difficult to see with water or bubbles during surgery, but information can be temporarily supplemented with a superimposed virtual endoscopic image. Since the virtual endoscopic image has position information in the real space before conversion, an accurate three-dimensional position can be calculated on the real endoscopic image. From these advantages, safer and more accurate endoscopic surgery can be expected.
(b) It is possible to intuitively grasp an accurate positional relationship of an organ that cannot be visually recognized only by an endoscope, and an endoscopic surgical operation with high visibility can be expected.
(c) Since the surgical instrument and virtual endoscopic image are accurately superimposed, only the virtual manipulator is moved during the operation, and the drive shown in (a), (b) and Fig. 4 is performed in the same environment as the real endoscope. By visually informing the surgeon of information such as restricted areas, the near-future preoperative situation can be simulated, and safe endoscopic surgery can be performed.
(d) In addition, the position information L1 from the position detection unit of the master device is input to the image superimposition processing unit in FIG. 9, and the position information L1 ′ and L2 ′ obtained by simulation are output from the image superimposition processing unit. By changing, the simulation information can be reflected in the actual manipulator. If simulation is not performed, information input from the position detection unit may be output as it is to L1 ′ and L2 ′.

An example of the operation of the master / slave device in the manipulator at that time will be described with reference to FIGS.
First, the image superimposing apparatus 3 determines whether or not to end the image superimposing apparatus 3 image superimposing apparatus 3 image superimposing process and the master slave apparatus control process (S1).
If the control process is not finished, the image superimposing device 3 determines whether or not to lock the slave device in order to perform the surgery simulation (S2).
In this case, it is assumed that the real endoscopic image, the virtual endoscopic image, and the virtual manipulator image are accurately superimposed. The lock mechanism is a mechanism that separates the real endoscopic image and the virtual endoscopic image.When the lock mechanism is invalid (off), L1 and L2 are acquired, and control is performed so that the slave device operates following the operation of the master device. Perform (S3).

The lock mechanism is enabled (ON) / disabled (OFF) by an operator's input. The input device may take any form as long as it is a mechanism for sending an input signal, such as a mouse or a foot switch. When the lock mechanism is valid, L1 and L2 are stored in order to reproduce the movement of the master device after the simulation (S4).
The operation unit of the master device 4 operates only a virtual manipulator and a virtual endoscopic image. The slave device is in a stopped state. L1 keeps the position information of the master device at any time, and L2 keeps only the position of the slave when the lock mechanism is activated. Next, the image superimposing device 3 determines the state transition of the lock mechanism. When the lock mechanism transitions from invalid to valid or from valid to invalid, when the lock mechanism is not valid in the initial state or when the lock mechanism is released, the process proceeds to S6 (S5).
When the operation simulation is not performed, the position information L1 is detected by the position detection unit 42 of the master device 4, the position information L2 is detected by the position detection unit 31 of the image superimposing device 3, and the motor control unit 6 and the actuator are detected via the processing unit 321. Transmit to the control unit 7 as L1 ′ and L2 ′. In the master side device, information is transmitted from the position control unit 61 of the motor control unit 6 to the motor 41 so that the shift amounts of L1 ′ and L2 ′ are equal. On the slave side, information is transmitted from the position control unit 71 of the actuator control unit 7 to the actuator 51 so that the shift amounts of L1 ′ and L2 ′ are equal (S7). When L1 'is not in the drive restriction area set by the image superimposition process, L1' and L2 'are input to the motor control unit 6 and actuator control 7, and the motor operates and the master device can be moved without load. It is possible (S6).

At that time, the image superimposing device 3 drives the slave side by the size of L1 and L2 (S7).
The image superimposing device 3 determines whether or not L1 is a driving warning area, and if it is a driving warning area, the process proceeds to S11, and if not, the process shifts to S9 (S8).
The image superimposing device 3 causes the motor control unit 6 to operate the motor 41 (S9)

The image superimposing device 3 smoothly transmits the movement of the master device to the slave device 5, and operates the actuator 51 connected to the slave device 5 (S10). These series of steps are performed as needed until the surgery is completed.
Next, when L1 is the drive restriction region, a load is supplied to the motor 41 by the force control units 62 and 72 according to the force control signal F1 from the image superimposition processing unit.
The image superimposing device 3 causes the motor control unit 6 to operate according to the load supplied to the motor 41 (S11).
The image superimposing device 3 also applies a load to the actuator 51 (S12).
The load applied to the motor 41 and the actuator 51 gradually increases, and finally the master slave device stops.

Next, when the lock mechanism transits from invalid to valid and a surgical simulation is possible, the slave device stops and only the master device can be moved (S13).
Similar to when the lock mechanism is disabled, the image superimposing device 3 determines whether to operate the motor when L1 is outside the drive restriction area or to apply a load when L1 is inside the drive restriction area. (S14).
The image superimposing apparatus 3 operates the motor 41 because L1 exists outside the drive restriction region (S15).

The image superimposing apparatus 3 applies a load to the motor 41 because L1 exists outside the drive restriction region (S16).
The image superimposing device 3 determines whether or not to reproduce the result of the manipulator operation simulation when the lock mechanism transitions from valid to invalid.If the determination result does not reproduce the simulation, the image superimposing device 3 proceeds to S1 and executes the simulation. When reproducing, the process proceeds to S18 (S17).
The image superimposing apparatus 3 determines the validity / invalidity of the lock mechanism assuming that the lock mechanism becomes invalid during reproduction. If the result of the determination is invalid (off), the process proceeds to S1, and if valid (on), the process proceeds to S19 (S18).

The image superimposing device 3 reads out L1 and L2 stored during simulation reproduction (S19).
The image superimposing apparatus 3 repeats the processes of S6 to S12 until there is no error in the shift amounts of L1 and L2 (S20).
When L1 and L2 become equal, the image superimposing device 3 updates the data of L1 as the next L1 (S21).
In such an operation, the same processing is repeated until the endoscope and manipulator are moved from the patient's body surface to the affected part.

In this way, the movement of the virtual image of the forceps or the endoscopic surgical manipulator is reflected on the actual forceps or the endoscopic surgical manipulator. Note that the safety mechanism that stops the actual forceps and the endoscopic surgical manipulator according to the warning level works during the simulation and during the actual realization of the simulation result.
Further, when it is desired to approach the organ to be operated, the safety mechanism can be released by enlarging the drive restriction region.

  Further, in this embodiment, instead of the optical / magnetic position detection device as the position detection device 31 in FIG. 2, a marker 120 is attached to a forceps or a manipulator as shown in FIG. Can be detected to detect the movement of forceps or an endoscopic surgical manipulator. Naturally, the position detection method of FIG. 11 can also be used in the system configuration of FIG.

The marker may be a substance in which a substance photographed with high or low brightness by an X-ray CT apparatus or MRI apparatus is included in a capsule photographed with low or high brightness.
Further, if the marker is limited to the MRI apparatus, the marker may include a coil used for the Active Tracking System.

In addition, if a coordinate group consisting of three or more markers is defined as a “tool”, it can be used as a different tool by changing the placement of the markers for each tool attached to each forceps or endoscopic surgical manipulator (that is, the forceps or manipulator It may be recognized as a tool for each part).
Once the above tool is recognized by the medical image diagnostic apparatus, a cross-sectional image 130 of the tool or a volume image including the cross-sectional image is taken as shown in FIG. Tools can be extracted.
Then, the reference point (for example, the center of gravity) is calculated from the coordinates of the plurality of markers of the tool, and the position and direction of the forceps and the endoscopic surgical manipulator are calculated from the relative position of the forceps and the endoscopic surgical manipulator from the reference point. can do. Since the position of the forceps or the endoscopic surgical manipulator can be found on the volume data of the medical diagnostic image taken before or during the operation, it can be used in the application example of FIG.

Further, if the image superimposing device 3 causes the master device to perform visual feedback and force feedback, this embodiment can be realized by a simple system including a medical image diagnostic device and an endoscope device.
This is because visual feedback and force feedback to the operator are generated only from image information obtained from the medical image diagnostic apparatus. This method is endoscopic surgery that guides the endoscope with intraoperative medical diagnostic images because force information is not fed back to the slave device, but there are advantages in that the system can be simplified and normal surgical techniques are not changed. is there.

  According to this embodiment, even if the surgeon's master is careless, dangerous contact between the target organ and the manipulator can be avoided, and safe endoscopic surgery can be performed.

  The endoscopic surgical operation support system of the present invention displays a target organ in a medical diagnostic image photographed by a medical image diagnostic apparatus in association with the target organ and the driving area of the endoscope and manipulator. Safe support for endoscopic surgical operation.

The apparatus block diagram of the endoscopic surgery operation assistance system of this embodiment. The functional block diagram of FIG. FIG. 3 is a functional block diagram of the master slave device of FIG. FIG. 2 is a diagram showing an example of a superimposed display of surgical instruments and images of the endoscopic surgical operation support system of FIG. FIG. 5 is a diagram showing an example of superimposed display in a form different from FIG. FIG. 6 is a diagram showing an example of superimposed display in a form different from those in FIGS. The figure which shows the example of the user interface which changes and sets the display range, such as the seriousness of an affected part. The figure which shows the example of the change setting of FIG. FIG. 3 is a functional block diagram of a master / slave device different from FIG. 2 is a flowchart for explaining an operation example of the endoscopic surgical operation support system in FIG. The figure which shows the example which further superimposed and displayed the marker on the image.

Explanation of symbols

  31 Position detection unit, 321 processing unit, 322 display unit

Claims (11)

In a manipulator operation support system that displays a movable range of a manipulator capable of operating a surgical instrument and image information of a patient in association with each other,
Position calculating means for calculating a positional relationship between a driving region of the manipulator and a target organ with which a part of the manipulator may contact;
Warning level setting means for setting the approach or contact condition according to the calculated positional relationship as a warning level of several stages;
Informing means for informing the operator of the set warning level;
An endoscopic surgical operation support system comprising:   The endoscopic surgical operation support system according to claim 1, further comprising a mechanism that stops the manipulator in accordance with a warning level set by the warning level setting means.   The endoscopic surgical operation support system according to any one of claims 1 and 2, wherein the notifying means notifies the operator by a display indicating the warning level.   The endoscopic surgery operation support system according to any one of claims 1 to 3, wherein the notifying means notifies the surgeon by at least one of a sound and a sound indicating the warning level.   The endoscopic surgical operation support system according to any one of claims 1 to 4, wherein the notifying unit notifies the operator by vibration indicating the warning level.   The endoscopic surgical operation support system according to claim 3, wherein the display in the notification unit displays the image information of the patient and the surgical instrument on the image.   The display in the said alerting | reporting means displays the said patient's image information on an endoscopic image and a virtual endoscopic image, and displays the said surgical instrument on the superimposed image further. Endoscopic operation support system.   The endoscope according to claim 3, wherein the display in the notification unit displays the image information of the patient and the surgical instrument in an overlapping manner, and further displays a joint drive region of the surgical instrument in an overlapping manner. Surgery operation support system.   The display in the notification means displays the image information of the patient and the surgical instrument in an overlapping manner, and further displays the affected part in different display forms based on the severity. Endoscopic operation support system.   4. The internal view according to claim 3, wherein the display in the notification means displays the image information of the patient and the surgical instrument in an overlapping manner, and further displays an index indicating the degree of contact with the target organ. Mirror surgery operation support system.   The endoscopic surgical operation support according to any one of claims 3 and 6, wherein the display in the notification means displays the image information of the patient and the simulation of the surgical instrument on the image. system.

Images