JP2005021354A - Surgery remotely supporting apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To support a surgery by simply providing an appropriate instruction from remote apparatus relative to a live endoscopy image at a low cost on a real time basis. <P>SOLUTION: This surgery remotely supporting apparatus 1 comprises a VE image creating device 19 and a support information creating device 20. The VE image creating device 19 obtains insertion amount data and insertion tilt angle data of a rigid scope 2 from a surgery system 2 via a communication line 100 on a real time basis and creates a virtual endoscopy image (VE image), or a virtual image on a real time basis with the endoscope image picked up by the rigid scope 2 and whose visual line direction is accorded with the endoscope image, on the basis of the insertion amount data, the insertion tilt angle data and the CT image. The support information creating device 20 creates support image to be transmitted to the surgery system 2 by referring to the VE image. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a remote surgery support apparatus that remotely supports surgery using an image.
[0002]
[Prior art]
In recent years, diagnosis based on images has been widely performed. For example, three-dimensional virtual image data in a subject is obtained by capturing a tomographic image of the subject using an X-ray CT (Computed Tomography) apparatus or the like. The diagnosis of an affected area has been performed using the virtual image data.
[0003]
In the CT apparatus, by continuously feeding the subject in the body axis direction while continuously rotating the X-ray irradiation / detection, a helical continuous scan (helical scan) is performed on the three-dimensional region of the subject. A three-dimensional virtual image is created from tomographic images of successive slices in a three-dimensional region.
[0004]
One such 3D image is a 3D image of the lung bronchi. The three-dimensional image of the bronchus is used to three-dimensionally grasp the position of an abnormal part suspected of lung cancer, for example. In order to confirm the abnormal portion by biopsy, a bronchoscope is inserted and a biopsy needle, biopsy forceps, or the like is taken out from the distal end portion and a tissue sample is taken.
[0005]
In a duct in the body having a multi-stage branch such as the bronchi, when the location of the abnormal part is close to the periphery of the branch, it is difficult to correctly reach the target site in a short time, for example, In Japanese Patent Application Laid-Open No. 2000-135215, etc., a three-dimensional image of a pipeline in the subject is created based on image data of a three-dimensional region of the subject, and a target along the pipeline is created on the three-dimensional image. By obtaining a path to a point, creating a virtual endoscopic image of the duct along the path based on the image data, and displaying the virtual endoscopic image, the bronchoscope An apparatus for navigating to a target site has been proposed.
[0006]
By the way, in diagnosis using an internal organ in the abdominal region as a subject, conventionally, in order to make a diagnosis while creating a three-dimensional virtual image of the subject mainly in the abdominal region and displaying the same, as described above. Image analysis software has been put to practical use.
[0007]
An image system using this kind of image analysis software is used for diagnosis by doctors to grasp changes in the lesion of a subject in the patient's abdominal region, etc. in advance while viewing the virtual image before surgery. It is common to be done on a desk, usually.
[0008]
[Patent Document 1]
Japanese Patent Laid-Open No. 2000-135215
[Problems to be solved by the invention]
However, when an instruction is to be provided from a remote facility using this image system, parameters such as the line-of-sight direction must be input to the image system in accordance with the endoscope live image, which is inconvenient.
[0010]
The present invention has been made in view of the above circumstances, and supports surgery by providing appropriate instructions from a remote facility for live endoscopic images in a simple, inexpensive and real-time manner. An object of the present invention is to provide a remote operation support device that can perform the operation.
[0011]
[Means for Solving the Problems]
The remote operation support device according to the present invention is for a surgical system that performs an operation performed under observation by an endoscopic image captured by an endoscope that is inserted into the body and capable of specifying a visual field direction. In a remote operation support apparatus for remotely providing support information via a communication line, a CT image storage means for storing a plurality of CT image data in the body cavity, and the inside of the body cavity of the endoscope from the operation system Data input means for inputting the insertion position into the endoscope, the insertion amount data of the endoscope and the insertion inclination angle data of the endoscope through the communication line, the insertion position data, the insertion amount data, and the insertion Virtual image construction means for constructing a virtual endoscopic image synchronized with the endoscopic image in real time from the plurality of CT image data based on tilt angle data.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings.
[0013]
FIGS. 1 to 12 relate to a first embodiment of the present invention, FIG. 1 is a configuration diagram showing configurations of a remote surgery support device and a surgery system, and FIG. 2 is a diagram showing a usage state of the rigid endoscope of FIG. 3 is a diagram showing a configuration of the rigid endoscope of FIG. 2, FIG. 4 is a diagram showing a configuration of a main part of the trocar of FIG. 2, and FIG. 5 is a flowchart showing a processing flow of the remote surgery support apparatus and the surgery system of FIG. FIG. 6 is a second flowchart showing the processing flow of the remote surgery support apparatus and the surgery system in FIG. 1, and FIG. 7 is a diagram showing a VE display screen displayed on the VE image display monitor in FIG. 8 is a diagram showing a support image generated by the support information generating device of FIG. 1, FIG. 9 is a diagram showing a first example of an endoscopic image displayed on the endoscopic image display monitor of FIG. 1, and FIG. FIG. 11 is a diagram showing a VE display screen displayed corresponding to the endoscopic image of FIG. Shows a second example of an endoscopic image displayed on the endoscope image display monitor, FIG. 12 is a diagram showing a VE display screen displayed in response to the endoscopic image of FIG. 11.
[0014]
As shown in FIG. 1, the remote operation support device 1 according to the present embodiment is arranged in a support room far from the operation room, and remotely supports the operation system 2 arranged in the operation room via a communication line 100. Device.
[0015]
The surgical system 2 is configured by arranging a rigid endoscope 3, a system controller 4, a CCU 5, a light source device 6, an insufflator 7, an electric scalpel 8, an ultrasonic treatment device 9, a VTR 10, a support information reproducing device 18 and the like in an operating room. The remote operation support device 1 is configured by arranging the VE image generation device 19 and the support information generation device 20 outside the operation room, and the operation system 2 and the remote operation support device 1 are connected via the communication line 100. Yes.
[0016]
First, details of the surgical system 2 will be described. The imaging signal imaged by the imaging unit 11 of the rigid endoscope 2 is transmitted to the CCU 5 and subjected to image processing, and then output to the VTR 10 and the system controller 4 that record the image.
[0017]
The system controller 4 stores a communication I / F unit 12 for transmitting / receiving setting information to / from each device of the CCU 5, the light source device 6, the insufflator 7, the electric knife 8, the ultrasonic treatment device 9, and the VTR 10, and various programs. A memory 13, a display I / F unit 15 that displays an image signal from the CCU 5 on the endoscope image display monitor 14, and a CPU 16 that controls these units.
[0018]
A remote controller 17 is connected to the CPU 16 of the system controller 4 via the communication I / F unit 12, and various data can be input by the remote controller 17.
[0019]
The rigid endoscope 3 is provided with an insertion amount detection unit 21 for detecting the insertion amount of the rigid endoscope 3 and an inclination angle sensor 22 for detecting the insertion inclination angle of the rigid endoscope 3, and insertion amount data detected by the insertion amount detection unit 21. The insertion angle data detected by the inclination angle sensor 22 is input to the CPU 16 via the communication I / F unit 12 of the system controller 4, and the CPU 16 receives the insertion amount data and the insertion angle data via the communication I / F unit 12. Is output to the information transfer I / F unit 24, and transmitted to the information transfer I / F unit 25 of the remote surgery support apparatus 1 through the communication line 100 by the information transfer I / F unit 24.
[0020]
The support information reproducing device 18 receives information from the support information generating device 20 of the remote operation support device 1 input by the information transfer I / F unit 25 of the remote operation support device 1 and the information transfer I / F unit 24 via the communication line 100. An apparatus for reproducing support information including support image information and support audio information, a video I / F unit 31 for inputting support image information, and an endoscopic image based on image information input by the video I / F unit 31 A display I / F unit 33 that displays a support image composed of + support information on the monitor 32 and a voice I / F unit 35 that inputs the support voice information and reproduces it by the speaker 34.
[0021]
As shown in FIG. 2, the rigid endoscope 3 is inserted into the body of a patient 39 together with a treatment tool 38 such as an electric knife 6 and an ultrasonic treatment device 9 via trocars 36 and 37.
[0022]
As shown in FIG. 3, the rigid endoscope 3 includes a TV camera 40 on the insertion base end side, and a tilt angle sensor 22 is provided on the grip 41 on the insertion base end side. This inclination angle sensor 22 measures the insertion inclination angle of the rigid endoscope 3 by a gyro etc. and outputs it to the system controller 4.
[0023]
Further, as shown in FIG. 4, an insertion amount detection unit 21 is provided on the proximal end side of the trocar 36 that guides the insertion portion 42 of the rigid endoscope 3 into the body of the patient 39. A roller 43 that contacts the outer peripheral surface of the insertion portion 42 and rotates according to the insertion of the insertion portion 42, and a rotary encoder 44 that detects the amount of rotation of the roller 43 and outputs the rotation amount of the insertion portion 42 to the system controller 4. .
[0024]
Next, details of the remote surgery support apparatus 1 will be described. The VE image generation device 19 obtains the insertion amount data and insertion inclination angle data of the rigid endoscope 2 in real time from the surgical system 2 via the communication line 100, and inserts the insertion amount data, the insertion inclination angle data, and the CT apparatus (not shown). The virtual endoscopy image (VE image) which is a virtual image in which the gaze direction coincides with the endoscopic image captured by the rigid endoscope 2 based on the CT image obtained by (1). The support information generation device 20 is a device that generates a support image to be transmitted to the surgery system 2 with reference to the VE image.
[0025]
Specifically, as shown in FIG. 1, the VE image generation device 19 stores a recording unit 51 that stores a CT image DB (database) constructed from a plurality of CT images, and various programs. Communication that receives the insertion amount data detected by the insertion amount detection unit 21 and the insertion inclination angle data detected by the inclination angle sensor 22 from the surgical system 2 transmitted through the memory 52 and the information transfer I / F unit 25 I / F unit 53, VE image construction unit 54 that constructs a VE image based on the insertion amount data, the insertion tilt angle data, and the CT image of the CT image DB obtained by communication I / F unit 53, and VE image construction unit 54 includes a display I / F unit 56 for displaying the VE image constructed by the VE image display monitor 55, and a CPU 57 for controlling these units, in order to input various data to the CPU 57. Keyboard 58 and mouse 59 are connected.
[0026]
The support information generation apparatus 20 includes a video I / F unit 61 that receives an endoscopic image from the CCU 5 transmitted via the information transfer I / F units 24 and 25 and the communication line 100, and a video I / F unit 61. An endoscope image input unit 62 that converts the obtained endoscope image into digital endoscope image data, an arrow image construction unit 63 that constructs an arrow image to be superimposed on the endoscope image data, and an endoscope An image composition unit 64 that superimposes the arrow image from the arrow image construction unit 63 on the endoscope image data from the image input unit 62 to generate a composite image, and positional information of the arrow image to be superimposed on the endoscope image data From the communication I / F unit 66 that receives the instruction information from the instruction information input unit 65 that inputs the voice, the voice I / F unit 68 that inputs the voice data from the microphone 67 that inputs the instruction voice, and the image synthesis unit 64 Endoscopic image that is a composite image of the image + support A display I / F unit 70 that displays a support image made up of information on the monitor 69, a memory 71 that stores various programs, and a CPU 72 that controls these units, and a voice I / F unit 68 are provided. The audio data and the display I / F unit 70 output a support image to the support information reproducing device 18 of the surgical system 2 via the information transfer I / F units 24 and 25 and the communication line 100.
[0027]
The operation of the present embodiment configured as described above will be described. As shown in FIGS. 5 and 6, in step S <b> 1, the CPU 16 of the system controller 4 is an insertion position of the rigid endoscope 3 into the body of the patient 39 using the remote controller 17 connected to the system controller 4 of the surgical system 2. (Endoscope) Enter the coordinates of the insertion point. This coordinate system coincides with the coordinate system of the CT image.
[0028]
In step S2, the CPU 16 measures and inputs the insertion tilt angle data of the rigid endoscope 3 by the tilt angle sensor 22, and in step S3, input information including the coordinate data of the insertion point and the insertion tilt angle data is input to the remote surgery support device 1. The image is transferred to the VE image generation device 19.
[0029]
In step S4, the CPU 57 of the VE image generation device 19 receives and inputs the input information composed of the coordinate data of the insertion point and the insertion inclination angle data when the VE image generation device 19 of the remote surgery support device 1 receives and inputs the input information including the coordinate data of the insertion point and the insertion inclination angle data. Endoscope) The magnification / line-of-sight direction of the VE image is determined based on the coordinate data of the insertion point and the insertion tilt angle data. In step S6, the VE image construction unit 54 generates a VE image based on the magnification / line-of-sight direction, and displays the display I / The VE image is displayed on the VE image display monitor 55 by the F unit 56.
[0030]
This VE image is displayed in the VE image display area 102 of the VE display screen 101 displayed on the VE image display monitor 55 as shown in FIG. The VE display screen 101 includes a VE image display area 102 that displays a VE image generated by the VE image construction unit 34, a two-dimensional image display area 103 that displays a plurality of two-dimensional CT images related to the VE image, and a rigid endoscope. An insertion point display field 104 for displaying two insertion points (x0, y0, z0), a start / stop button 105 for instructing start and stop of tracking, a display magnification change input unit 106 for changing the display magnification, and the like. Is done.
[0031]
Then, with reference to this VE image, in step S7, in the support information generation device 20, the CPU 72 displays a support image 110 in which an arrow image 109 indicating the affected part position as shown in FIG. The support image 110 is generated and displayed on the monitor 69 via the display I / F unit 70.
[0032]
In step S <b> 7, not only the generation / display of the support image 110 but also the support voice to the operating room is input by the microphone 67. Then, the CPU 72 transmits the generated support image data and the input support voice data to the surgery system 2 via the communication line 100.
[0033]
In the surgery system 2 to which the support image data and the input support sound data are transmitted, the support information generation device 20 displays the support image 110 on the monitor 32 and the support sound is reproduced by the speaker 34 in step S8.
[0034]
In this way, when the first support image 110 is displayed and tracking (following of the VE image to the live image) is started in step S9, the CPU 16 of the system controller 4 uses the tilt angle sensor 22 to detect the rigid endoscope 3 in step S10. In step S11, the CPU 16 measures the insertion amount data of the rigid endoscope 3 by the insertion amount detection unit 21.
[0035]
In step S12, the CPU 16 determines whether there is a change in either the insertion amount data or the insertion inclination angle data. If there is no change, the process returns to step S10. In step S13 in which there is a change, the CPU 16 The input information including the insertion inclination angle data is transferred to the VE image generation device 19 of the remote surgery support device 1.
[0036]
In step S14, the CPU 57 of the VE image generation device 19 receives and inputs the input information including the insertion amount data and the insertion inclination angle data when the VE image generation device 19 of the remote surgery support device 1 receives and inputs the insertion amount data and the insertion inclination angle data. The magnification / gaze direction of the VE image is determined based on the insertion tilt angle data. In step S16, the VE image construction unit 54 generates a VE image based on the magnification / gaze direction, and the display I / F unit 56 converts the VE image into the VE image. It is displayed on the display monitor 55.
[0037]
Then, with reference to the VE image, in step S17, in the support information generation device 20, the CPU 72 generates a support image 110 in which an arrow image indicating the position of the affected part is superimposed on the endoscope image data and a display I / F. The support image 110 including the endoscopic image + support information is displayed on the monitor 69 via the unit 70.
[0038]
In step S <b> 17, not only the generation and display of the support image 110, but also support sound for the operating room is input by the microphone 67. Then, the CPU 72 transmits the generated support image data and the input support voice data to the surgery system 2 via the communication line 100.
[0039]
In the surgery system 2 to which the support image data and the input support sound data are transmitted, the support information 110 is displayed on the monitor 32 by the support information generation device 20 and the support sound is reproduced by the speaker 34 in step S18.
[0040]
Then, the CPU 16 of the system controller 4 determines whether or not there is a support end instruction from the remote controller 17 in step S19, and if not, returns to step S10, and if there is, ends the processing.
[0041]
With the processing in steps S10 to S19, when the live endoscopic image 14a as shown in FIG. 9 is displayed on the endoscopic image display monitor 14 in the VE image generating device 19, for example, on the VE display screen 101. FIG. 10 shows a real-time blood vessel placement virtual image 102a in which the organ portion is erased as shown in FIG. 10 and displayed in the VE image display area 102 in the same size (magnification) as the line-of-sight direction of the live endoscopic image 14a. The
[0042]
In addition, when the rigid endoscope 2 is tilted from the state of FIG. 9 and a live endoscope image 14b as shown in FIG. 11 is displayed on the endoscope image display monitor 14, it follows (tracks) and performs live. A real-time blood vessel placement virtual image 102b in which, for example, an organ portion is deleted as shown in FIG. 12, which matches the line-of-sight direction (magnification) of the endoscopic image 14b, is displayed in the VE image display area 102.
[0043]
As described above, according to the present embodiment, the communication line 100 is used to transmit the insertion amount and the insertion inclination angle of the rigid endoscope 3 to the support room at a location apart from the operating room, so that the remote support room can be provided. Since a supervising doctor provides a support image and sound to a surgeon in the operating room by referring to a VE image that is tracked (followed up) in real time with a live endoscopic image, it is appropriate for the surgeon easily and inexpensively. Can provide procedure support.
[0044]
FIG. 13 is a block diagram showing the configuration of the remote surgery support apparatus and the surgery system according to the second embodiment of the present invention.
[0045]
Since the second embodiment is almost the same as the first embodiment, only different points will be described, and the same components are denoted by the same reference numerals and description thereof will be omitted.
[0046]
As shown in FIG. 13, the surgery system 2 includes a VE image display device 202 that inputs VE image data generated by the VE image generation device 19 of the remote surgery support device 1 and displays the VE image data on the VE image display monitor 201. The VE image display device 202 includes a video I / F unit 203 that inputs VE image data via the information transfer I / F units 24 and 25 and the communication line 100, and a VE image display monitor based on the input VE image data. And a display I / F unit 204 to be displayed on 201. Other configurations are the same as those of the first embodiment.
[0047]
In this embodiment, in step S7 or 17 described in the first embodiment, not only the support image data and the input support voice data are transmitted to the surgical system 2 via the communication line 100 but also the VE image. Data is also transmitted to the surgical system 2 via the communication line 100, and a VE image is displayed on the VE image display monitor 201 provided in the operating room. Other operations are the same as those in the first embodiment.
[0048]
According to the present embodiment, in addition to the effects of the first embodiment, an operator in the operating room can also refer to the VE image displayed on the VE image display monitor 201. Can be made stronger.
[0049]
FIG. 14 is a configuration diagram showing configurations of a remote surgery support apparatus and a surgery system according to the third embodiment of the present invention.
[0050]
Since the third embodiment is almost the same as the second embodiment, only different points will be described, and the same components are denoted by the same reference numerals and description thereof will be omitted.
[0051]
In the present embodiment, a VE image generation device 301 having a configuration equivalent to the VE image generation device 19 provided in the remote surgery support device 1 is provided in the surgical system for the VE image displayed on the VE image display monitor 201. .
[0052]
Similar to the VE image generation device 19, the VE image generation device 301 includes a recording unit 351 that stores a CT image DB (database) constructed from a plurality of CT images, and a memory 352 that stores various programs. The communication I / F unit 353 that receives the insertion amount data detected by the insertion amount detection unit 21 and the insertion inclination angle data detected by the inclination angle sensor 22 from the system controller 4 and the insertion obtained by the communication I / F unit 353 A VE image construction unit 354 that constructs a VE image based on the amount data, the insertion tilt angle data, and the CT image of the CT image DB, and a display I / I that displays the VE image constructed by the VE image construction unit 354 on the VE image display monitor 201. The keyboard 3 is composed of an F unit 356 and a CPU 357 for controlling these units, and inputs various data to the CP 3U 57. 8 and a mouse 359 are connected. Other configurations and operations are the same as those of the second embodiment.
[0053]
According to the present embodiment, in addition to the effects of the first embodiment, the VE image generation device 301 generates a VE image to be displayed on the VE image display monitor 201. Therefore, the remote operation support device via the communication line 100 Since there is no need to transmit a VE image from one VE image generation device 19, the communication traffic of the communication line 100 can be reduced, and the communication environment can be greatly improved.
[0054]
The present invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the scope of the present invention.
[0055]
【The invention's effect】
As described above, according to the present invention, it is possible to support surgery by providing an appropriate instruction from a remote facility for a live endoscopic image in a simple, inexpensive and real-time manner. is there.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a remote surgery support apparatus and a surgical system according to a first embodiment of the present invention. FIG. 2 is a diagram showing a usage state of the rigid endoscope of FIG. FIG. 4 is a diagram showing a configuration of a main part of the trocar of FIG. 2. FIG. 5 is a first flowchart showing a processing flow of the remote surgery support device and the surgery system of FIG. FIG. 7 is a diagram showing a VE display screen displayed on the VE image display monitor of FIG. 1. FIG. 8 is a diagram showing the support of FIG. FIG. 9 is a diagram showing a support image generated by the information generating device. FIG. 9 is a diagram showing a first example of an endoscopic image displayed on the endoscopic image display monitor of FIG. 1. FIG. FIG. 11 is a diagram showing a VE display screen displayed corresponding to an image. FIG. 12 is a diagram showing a second example of an endoscopic image displayed on the screen. FIG. 12 is a diagram showing a VE display screen displayed corresponding to the endoscopic image of FIG. 11. FIG. FIG. 14 is a block diagram showing the configuration of a remote surgery support device and a surgical system according to the embodiment. FIG. 14 is a block diagram showing the configuration of the remote surgery support device and the surgical system according to the third embodiment of the present invention. ]
DESCRIPTION OF SYMBOLS 1 ... Remote surgery assistance apparatus 2 ... Surgery system 3 ... Rigid endoscope 4 ... System controller 5 ... CCU
6 ... light source device 7 ... pneumo-abdominal device 8 ... electric knife 9 ... ultrasonic treatment device 10 ... VTR
DESCRIPTION OF SYMBOLS 11 ... Imaging part 18 ... Support information reproduction apparatus 19 ... VE image generation apparatus 20 ... Support information generation apparatus 21 ... Insertion amount detection part 22 ... Inclination angle sensor 51 ... Recording part (CT image DB)
52 ... Memory 53 ... Communication I / F unit 54 ... VE image construction unit 55 ... VE image display monitor 56 ... Display I / F unit 57 ... CPU
100: Communication line

Claims (4)

An endoscope capable of specifying the visual field direction is inserted into the body, and support information is transmitted via a communication line to a surgical system for performing a surgery performed under observation by an endoscopic image captured by the endoscope. In a remote operation support device that provides remotely,
CT image storage means for storing a plurality of CT image data in the body cavity;
Data input means for inputting the insertion position of the endoscope into the body cavity, the insertion amount data of the endoscope and the insertion inclination angle data of the endoscope from the surgical system via the communication line;
Virtual image construction means for constructing a virtual endoscopic image synchronized with the endoscopic image in real time from the plurality of CT image data based on the insertion position data, the insertion amount data, and the insertion inclination angle data. A telesurgical support device. Image input means for inputting the endoscopic image from the surgical system via the communication line;
The support information is generated based on the virtual endoscopic image constructed by the virtual image construction means, the support information is added to the endoscopic image input by the image input means, and the communication line is connected to the surgical system. The operation support apparatus according to claim 1, further comprising support information output means for outputting via the operation information. The surgery support apparatus according to claim 2, wherein the support information added to the endoscopic image is superimposition support image data superimposed on the endoscopic image. The support information added to the endoscopic image is voice support information related to the endoscopic image input by the image input unit based on the virtual endoscopic image. 3. The surgery support apparatus according to 3.

Images