JP2015066021A - Three-dimensional navigation image generation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a 3D navigation image generation device that enables easy reference to an operation plan created in a preoperative conference, etc. according to a situation changing every moment during an operation.SOLUTION: A 3D navigation image generation device includes: an operation field 3D image forming device 3 for acquiring an operation field 3D image showing an affected part under treatment; a navigation 3D image generation device 6 for generating a navigation 3D image in which a marker that gives an instruction on an operation procedure is given to a volume rendering 3DCG image of the affected part generated prior to the operation; a 3Din3D image combining device 9 for generating a 3Din3D image that displays a navigation 3D image formed in correspondence with the depth and the display range of the operation field 3D image by fitting it into the operation field 3D image; and a 3D image display device 11. An image is displayed in which the navigation 3D image is fitted into a part of the operation field 3D image.

Description

  The present invention generates a 3D navigation image that adds a navigation image for assisting surgery to a stereoscopic image of an affected part for an operation performed while observing the stereoscopic image of the affected part, such as a surgical robot, a surgical microscope, and a 3D endoscope. Relates to the device.

As a technique for assisting surgery, there is a method in which a fluorescent light-emitting substance such as indocyanine green that collects in an affected area to be operated is applied to clearly display the affected area with a fluorescent display. As a result, the target position to be treated is clarified, and there is an effect of preventing the remaining affected area to be removed. However, if the affected area is injured, the luminescent material is scattered, which may make it difficult to identify the treatment target position.
There is also a method of navigating the laparotomy position and the like by acquiring a CT or X-ray photograph of the affected area before the operation and projecting the image on the affected area of the patient to clarify the processing target position. However, this method is not always effective due to difficulty in aligning the entity and the projected image, variation in the affected area during surgery, difficulty in identifying the image and the entity, and the like.

In addition, before performing the patient's surgery, a 3D image converted to 3DCG is generated from the CT image of the affected area using a volume rendering device, and the image is downloaded to a tablet-type display device. There is also a technique for performing treatment with reference to the display device. However, there is a problem that it is difficult to communicate between the practitioner and the operation assistant, and it is difficult to display an appropriate video image during the operation.
In addition, there are many cases where a preoperative conference is performed before a patient's surgery is performed and a surgical plan is made using a 3D CG image of the affected area and then the operation is started. It was impossible to refer to the surgical plan as an image during the operation, and it was necessary to rely on the memory of the surgeon.

  Patent Document 1 discloses a surgical instrument guided surgery support device. The reference point is registered on the 3D volume image obtained by the medical imaging device, and the coordinate axis of the 3D volume image is corrected in accordance with the deviation of the surgical tool position and the reference point position in the cross-sectional image obtained during the operation. By doing so, it is described that 3D navigation follows the movement of the subject.

JP 2012-227714 A

  Therefore, the problem to be solved by the present invention is to make it possible to easily refer to a surgical plan designed by human preoperative surgery etc. together with a stereoscopic image (3D video) of an affected part in accordance with a situation that changes every time during the operation. A navigation video generation apparatus is provided.

  The 3D navigation video generation apparatus according to the present invention instructs a surgical procedure to a surgical field 3D video forming apparatus that acquires a surgical field 3D video that displays an affected area during a procedure, and a volume rendering 3DCG video of the affected area that is generated before surgery. A navigation 3D video generation device that generates a navigation 3D video with a marker, and a navigation 3D video that corresponds to the depth of the surgical field 3D video and that corresponds to the display range, and is embedded in a part of the surgical field 3D video. A 3Din3D video synthesizing device for generating 3Din3D video and a 3D video display device, and displaying a video in which a navigation 3D video is inserted into an operative field 3D video.

  The D navigation image generation device of the present invention is applied to a 3D image display device in a surgical support robot such as a surgery using a 3D microscope or a 3D endoscope, or a Davinci surgical system (trade name). The navigation 3D video is displayed in the 3D video at the same time, so the correspondence between the affected area video during the operation and the navigation video can be easily judged, and the surgeon keeps an eye on the video of the surgical field during the operation. You can receive navigation.

Further, by configuring the navigation 3D video so that the display position, size, or display area of the navigation 3D video can be adjusted, the navigation 3D video can be arranged so as not to obstruct the operative field 3D video, thereby supporting the surgery more accurately. it can.
Furthermore, by adjusting the depth of the navigation 3D video so as to correspond to the depth of the surgical field 3D video, navigation with less discomfort to the surgeon can be performed.

  In addition, when 3Din3D video is formed with side-by-side parallax 3D video signals, conventional video display devices and video signals used for television broadcasting can be applied. And broadcast devices can be used to easily record and play back and broadcast video.

  By using the 3D navigation video generation device of the present invention, it is possible to perform a precise treatment while referring to the navigation video instead of performing the treatment while restoring the previous conference result by relying on the memory. Therefore, since the position of the affected part and the position of the surgical site can be accurately grasped and the operation can be performed according to the plan, the operation time can be shortened and the success rate of the operation can be improved. As a result, minimal invasiveness is realized, which contributes to shortening the recovery period and hospitalization period.

Furthermore, the 3D navigation video generation apparatus of the present invention can also input and display a surgical procedure that responds to the situation based on the video during the operation, and can be used for remote coaching.
In addition, the 3D navigation video generation apparatus of the present invention has a recording and recording function, can produce high-level educational video that reproduces and displays a surgical situation, and can contribute to raising the level of medical technology.

It is a block diagram which shows the principle of the 3D navigation image | video production apparatus which concerns on one Example of this invention. It is a conceptual diagram which shows the example of the stereo image obtained by the 3D navigation image | video production apparatus which concerns on a present Example. It is process drawing which shows the stereo image structure procedure in the 3D navigation image | video production apparatus which concerns on a present Example. It is a photograph which shows the example of the stereoscopic video display image comprised by the present Example.

  The 3D navigation video generation apparatus of the present invention forms and displays a 3Din3D video in which a navigation 3D video is inserted into an operative field 3D video obtained by imaging an affected area displayed on a video device that supports surgery such as a microscope or an endoscope. It is. The navigation 3D image is usually a 3D image in which a position serving as an index is indicated in accordance with a surgical procedure on a stereoscopic image of an affected area acquired before surgery. As the stereoscopic image of the affected area, for example, a volume rendering 3DCG image created from a cross-sectional image such as a CT image or an NMR image can be used.

  When the navigation 3D image is inserted into the 3D image and converted into the 3Din 3D image, the operation field 3D image to be acquired is scheduled according to the planned surgical procedure. Then, the range and size of the navigation 3D video to be displayed in the same field of view are determined in correspondence with the planned surgical field 3D video, and the display position on the screen is determined. At this time, it is preferable to adjust the depth of the navigation 3D image to be displayed in accordance with the depth of the operative field 3D image. The depth can be adjusted by adjusting the amount of left-right parallax.

Hereinafter, with reference to the drawings, a 3D navigation video generation device of the present invention will be described in detail based on an embodiment.
FIG. 1 is a diagram illustrating a configuration of a 3D navigation video generation apparatus according to an embodiment of the present invention.

The 3D navigation video generation apparatus according to the present embodiment includes an operative field 3D video formation apparatus 3, a volume rendering generation apparatus 6, a 3Din3D video synthesis apparatus 9, and a video display apparatus 11. The operative field 3D image forming apparatus 3 is connected to a left-eye camera 1 and a right-eye camera 2 for inputting a 3D image, and images a diseased part during surgery to form a stereoscopic image. The three-dimensional video signal is sent to the input monitor 4 and displayed as a surgical field 3D video.
The surgical field 3D video signal is also sent to the 3Din3D video synthesis device 9.

  A CT imaging device 5 and a console 7 are connected to the volume rendering generation device 6. The CT imaging device 5 forms a cross-sectional image of the affected area and outputs an image signal. An NMR (nuclear magnetic resonance apparatus) or the like can be used instead of the CT imaging apparatus. The image output of the CT imaging device 5 is subjected to volume rendering by the volume rendering generation device 6 to generate a 3DCG image signal, which is displayed on the operation monitor 8 as a stereoscopic image.

The CT imaging device 5 acquires an X-ray CT image related to the affected area of the patient before the operation, and sends it to the volume rendering generation device 6. In the volume rendering generation device 6, the affected part is formed into a 3D image for three-dimensional display from the X-ray CT image, and the volume rendering image is displayed on the operation monitor 8.
The volume rendering image can be reconstructed any number of times based on a cross-sectional image such as an X-ray CT image acquired before surgery. In addition, an appropriate volume rendering image can be selected and stored from several volume rendering images having different display forms, and can be displayed as necessary.

  A surgeon, a surgical assistant, or a surgical instructor can formulate an appropriate surgical plan by performing a prior simulation or conference based on the volume rendering image of the affected area displayed on the operation monitor 8. it can.

  The console 7 uses a touch panel, a keyboard, a personal computer controller, a joystick, etc., and instructs the video display range and size based on the volume rendering image displayed on the operation monitor 8 and rotates the video. It is used to place a pointer as an index in the volume rendering image of the affected area and to specify the position and posture of the surgical instrument. Further, the depth amount of the video can be adjusted by manipulating the parallax amount between the left-eye video and the right-eye video.

  In this way, a navigation 3D image is formed by placing an appropriate index on the volume rendering image and instructing a surgical method or a surgical position in accordance with a surgical plan, and is stored in the volume rendering generation device 6. The navigation 3D video can be extracted and transmitted to the operation monitor 8 or the 3Din3D video composition device 9 as necessary.

  The 3Din3D video synthesizing device 9 receives the operative field 3D video signal acquired in real time and the navigation 3D video signal formed before the operation, and the 3Din3D in which the navigation 3D video is overwritten and displayed on a part of the operative field 3D video. Generate video. The video signal of 3Din3D video can be transmitted to the 3D video display device 11 and displayed as 3D video.

FIG. 2 is a diagram showing a concept of 3Din3D video formed by the 3Din3D video synthesizing device 9. In the 3Din3D video, the navigation 3D video is inserted into a part of the operative field 3D video.
The operative field 3D image can be controlled so as to acquire an image that can accurately support the operation by adjusting the imaging position, the imaging direction, the enlargement / reduction of the image, and the like.

  The display area of the navigation 3D image can be changed in size or positioned by moving in the horizontal direction or the vertical direction by operating a controller or the like. Therefore, it is possible to arrange the navigation 3D image having an appropriate size at an appropriate position while avoiding the target portion displayed in the operative field 3D image.

  In addition, the navigation 3D image can be an image suitable for navigating the operation by arbitrarily rotating the image by a manual operation or scaling the image while maintaining the aspect ratio of the screen. In addition, by associating the depth amount of the navigation 3D image with the depth amount of the surgical field 3D image, the corresponding portions of both images give an equivalent stereoscopic sensation so as to support smooth treatment. preferable.

  The navigation 3D image can be displayed in time series in advance in accordance with the surgical plan, but it can also be controlled by selecting fast forward, rewind, stop, etc. by operating the controller. It is also possible to select a marker attached in advance and jump to that position. In this way, navigation suitable for the surgical field becomes possible.

FIG. 3 is a process diagram showing a procedure for constructing a 3Din3D video in the 3D navigation video generation device.
The left operative field image 31 acquired by the left-eye camera 1 and the right operative field image 32 acquired by the right-eye camera 2 are respectively compressed in the lateral direction and arranged side by side into an operative field 3D image 33 that can be transmitted as a TV image signal. Metamorphosis.

  A left navigation image 34 and a right navigation image 35 related to the navigation 3D image formed by adding various indexes to the 3D image generated by volume rendering from the cross-sectional image of the target area are part of the operative field 3D image. The image is compressed by the same amount for display on the left and right sides, and arranged at corresponding positions on the left and right in the operative field 3D video arranged side by side, to obtain a navigation 3D video 36 arranged side by side. It is preferable that the portion other than the video display area in the navigation 3D video is blackened so that no noise is generated when the navigation 3D video is overlaid with the surgical field 3D video.

The operative field 3D image 33 and the navigation 3D image 36 are overlapped on the same surface to become a 3Din3D image 37. At this time, it is preferable to prevent the navigation 3D image from becoming difficult to see by blackening a portion of the operative field 3D image 33 where the image portion of the navigation 3D image 36 is filled.
Further, it is preferable to provide a buffer band due to blackening or the like at the boundary between the operative field 3D image 33 and the navigation 3D image 36 so as to suppress eye fatigue due to stereoscopic viewing.

FIG. 4 is a photograph illustrating one frame of the 3Din3D video 37 thus configured.
The 3Din3D video 37 is separated into left and right by the 3D video display device 11 and displayed as a left 3D video 38 and a right 3D video 39, respectively, and is observed as a parallax image with a left eye and a right eye of a person by means of stereoscopic glasses or the like. Recognized as a video. The 3D video display device may be capable of autostereoscopic viewing.

  The 3D image display device 11 may be installed not only in a place where the surgeon can observe but also in a classroom or conference place for observation by a person other than the surgeon. Note that the 3D video display device 11 is not limited to various 3D glasses-type or naked-eye type display devices, and may be a display device that can be projected onto a screen and displayed in 3D for simultaneous observation.

Alternatively, the 3Din3D video generation device 9 may be controlled via the remote coaching transmission device 10 to control the navigation 3D video so that an instruction can be given remotely from outside the operating room.
Further, the video signal of the 3Din3D video 37 may be sent to the recording / recording apparatus 12 for recording as necessary so that the video signal can be edited and reproduced.

  The 3D navigation video generation apparatus of the present embodiment can simultaneously display a navigation 3D video corresponding to the depth information represented by the surgical field 3D video in the field of view of the surgical field 3D video representing the stereoscopic video of the affected area. The operating surgeon can observe the video of the affected area and the navigation video without feeling uncomfortable, and can keep an eye on the surgical field when referring to the navigation video.

Further, the navigation 3D image can be moved to an appropriate position or the display area can be enlarged or reduced so as not to disturb the image of the affected area. In addition, smooth operation can be supported by obtaining a navigation 3D video that does not give a sense of incongruity to the depth information represented by the surgical field 3D video.
For this reason, it greatly contributes to performing the surgery reliably and safely, shortens the operation time, suppresses the invasion by the operation, and improves the quality of the operation.

In the 3D navigation video generation apparatus of the present embodiment, the surgeon or the operation assistant operates the console, and the navigation 3D video necessary for the currently operating surgical field or necessary for the operation to proceed. Can be selected and displayed, so that smooth treatment can be performed.
According to the surgical plan, marker points can be embedded in the navigation 3D video in advance, and the navigation video can be developed and changed according to the progress of the operation.

Side-by-side 3Din3D video can be transmitted remotely and recorded and recorded, enabling remote coaching to guide surgery from a remote location during the procedure, and use it as an educational material that represents the actual state of surgery. You can also.
Needless to say, when a 3D navigation video cannot be obtained, even if the navigation video is a 2D video, embedding it in the 3D video of the operative field can produce a much greater support effect than before. Absent.

  The present invention provides a 3D navigation video generation device capable of performing accurate navigation and coaching in a place where surgery is performed based on 3D-displayed surgical field images, which greatly contribute to medicine and medicine. It is.

DESCRIPTION OF SYMBOLS 1 Left-eye camera 2 Right-eye camera 3 Surgery field 3D image formation apparatus 4 Input monitor 5 CT imaging device 6 Volume rendering production device 7 Console 8 Operation monitor 9 3Din 3D image composition device 10 Remote coaching transmission device 11 3D image display device 12 Recording recording Device 31 Left operation field image 32 Right operation field image 33 Operation field 3D image 34 Left navigation image 35 Right navigation image 36 Navigation 3D image 37 3Din 3D image 38 Left 3D image 39 Right 3D image

JP 2009-279209 A

Claims (5)

An operative field 3D image forming device for acquiring an operative field 3D image for displaying an affected area during treatment;
A navigation 3D image generation device for generating a navigation 3D image in which a marker for instructing a surgical procedure is added to a volume rendering 3DCG image of an affected area generated before surgery;
A 3Din3D video synthesizing device that generates a 3Din3D video that displays a navigation 3D video that corresponds to the depth of the surgical field 3D video and that corresponds to the display range and that is embedded in a part of the surgical field 3D video;
A 3D video display device,
A 3D navigation video generation device for displaying a video in which a navigation 3D video is inserted into an operation field 3D video on the 3D video display device.   The 3D navigation video generation apparatus according to claim 1, wherein the 3Din3D video is formed by a 3D video signal of a two-parallax stereoscopic display system in a side-by-side format.   3. The 3D according to claim 1, wherein a recording / recording device is further connected to the 3Din3D video composition device, and the recording / recording device records the video signal of the 3Din3D video and can reproduce it when necessary. Navigation video generation device.   The 3Din3D video composition device is further connected to a remote coaching transmission device, and the 3Din3D video can be operated via the remote coaching transmission device when necessary. The 3D navigation video generation device according to item 1.   The volume rendering generation device is further connected to a console for instructing the enlargement / reduction of the display area of the navigation 3D video inserted in the operative field 3D video, horizontal movement and positioning, vertical movement and positioning. The 3D navigation video generation device according to any one of claims 1 to 4, wherein the 3D navigation video generation device is provided.