JP2015220643A - Stereoscopic observation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a stereoscopic observation device that is capable of easily switching a stereoscopic image display and a planar image display of an observation site according to a user's instruction.SOLUTION: A stereoscopic observation device has a first imaging part for picking up a first observation image from a first direction, a second imaging part for picking up a second observation image from a second direction different from the first direction, a housing for holding the first imaging part and the second imaging part integrally, a parallax image generator 51 for generating a first parallax image and a second parallax image on the basis of the first observation image and the second observation image, a switching instruction accepting part 52 for accepting a user's switching instruction whether a stereoscopic image is displayed or a planar image is displayed, and an image composite part 53 for displaying the stereoscopic image by displaying the first parallax image and the second parallax image on 3D display devices 33, 18 when the stereoscopic image display is instructed, and displaying the planar image on the 3D display devices 33, 18 when the planar image display is instructed.

Description

  Embodiments described herein relate generally to a stereoscopic observation apparatus.

  Recently, two images having parallax with respect to the observation object are captured, and the captured images are guided to the left and right eyes of the user so that the user can perceive a stereoscopic image of the observation object. Stereoscopic observation technology has been developed.

  An observation apparatus using this kind of stereoscopic observation technology can be applied to the medical field, for example, an endoscope or a stereomicroscope that is required to perform a delicate and advanced technique under observation of a fine surgical site. It is suitable for a surgical operation using

JP 2001-46399 A

  However, if the user continues to perceive the stereoscopic image, the user may feel a heavy burden on the eyes and the efficiency of the procedure may be reduced. On the other hand, if the observation apparatus is configured to confirm the observation object with a planar image, the information before and after the depth direction of the observation object obtained by the stereoscopic image is lost.

  In order to solve the above-described problem, a stereoscopic observation apparatus according to an embodiment of the present invention includes a first imaging unit that captures a first observation image of an observation site of a subject from a first direction, and the first imaging unit. A second imaging unit that captures a second observation image of the observation site from a second direction different from the direction, a housing that integrally holds the first imaging unit and the second imaging unit, and the first observation A parallax image generation unit that generates a first parallax image and a second parallax image based on the image and the second observation image, and a switch that receives a switching instruction from the user to display the observation site as a stereoscopic image or a planar image When instructed to display the stereoscopic image by the instruction receiving unit, the first parallax image and the second parallax image are displayed on the 3D display device to display the stereoscopic image, while the planar image is displayed. Ru , In which and an image synthesizing unit for displaying a plane image based on either the first parallax image and the second parallax image to the 3D display device.

1 is an overall configuration diagram showing an example of a stereoscopic observation apparatus according to a first embodiment of the present invention. The block diagram which shows roughly the example of an internal structure of a microscope part. FIG. 3 is a configuration diagram illustrating an example of a first imaging unit and a second imaging unit according to the first embodiment. The flowchart which shows an example of the procedure at the time of switching easily the stereo image display and planar image display of an observation site | part according to a user's instruction | indication by an image process part. FIG. 2 is a block diagram schematically showing an internal configuration example of an image processing unit according to the first embodiment. Explanatory drawing which shows an example of the instruction | indication reception image for receiving the user instruction | indication with respect to the display content. (A) is explanatory drawing which shows the example in case a some eyepiece part is provided in a microscope part, (b) is explanatory drawing which shows the example in case an eyepiece part and 3D display part are provided in a microscope part. The whole block diagram which shows an example of the stereoscopic observation apparatus which concerns on 2nd Embodiment of this invention. Explanatory drawing which expands and shows the IX part of FIG. FIG. 9 is a block diagram schematically showing an internal configuration example of an image processing unit according to a second embodiment. The figure for demonstrating the focus adjustment process of the image process part which concerns on 2nd Embodiment.

  An embodiment of a stereoscopic observation apparatus according to the present invention will be described with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is an overall configuration diagram illustrating an example of a stereoscopic observation apparatus 10 according to the first embodiment of the present invention.

  The stereoscopic observation apparatus 10 has a microscope unit 11 as shown in FIG. The microscope unit 11 according to the present embodiment is a so-called stereo microscope.

  The microscope unit 11 images the subject O from the two directions of the first direction 12 and the second direction 13. The stereoscopic observation apparatus 10 according to this embodiment performs image processing on image data captured from two directions inside the microscope unit 11 and image processing using an image processing unit 15 provided inside the base 14. The microscope unit 11 and the base 14 are connected by an articulated arm 16. An input unit 17 and a monitor 3D display device 18 are disposed on the base 14.

  The input unit 17 is configured by a general input device such as a mouse, a trackball, a keyboard, a touch panel, a numeric keypad, and a voice input microphone, and outputs an operation input signal corresponding to a user operation to the image processing unit 15. To do. In addition, when using a microphone as the input part 17, a microphone converts the audio | voice input by the user into a digital audio | voice signal. The image processing unit 15 performs an operation corresponding to the voice input by the user by performing voice recognition processing on the digital voice signal.

  Note that the image processing unit 15 according to the present embodiment is provided in at least one of the inside of the microscope unit 11 and the inside of the base 14. In the following description, an example in which the image processing units 15m and 15 having the same image processing function are provided both inside the microscope unit 11 and inside the base 14 will be described.

  The microscope unit 11 has a housing 20. The case 20 is provided with an eyepiece 21 for a user to observe an image displayed inside the microscope unit 11. The eyepiece unit 21 includes a left-eye observation unit 21L and a right-eye observation unit 21R.

  Further, the housing 20 is provided with a changeover switch 22 for giving a user an instruction to switch between stereoscopic image display and flat image display, and a focus adjusting unit 23 for giving a user an instruction to adjust the focus of the display image. Is done. The housing 20 is provided with a handle 24 for changing the relative positional relationship between the microscope unit 11 and the subject O by moving the microscope unit 11.

  FIG. 2 is a block diagram schematically illustrating an internal configuration example of the microscope unit 11.

  The microscope unit 11 includes an image processing unit 15m, a housing 20, an eyepiece unit 21, a changeover switch 22, a focus adjustment unit 23, a first imaging unit 31, a second imaging unit 32, a built-in 3D display device 33, and a drive. The unit 34 and the distance sensor 35 are included.

  FIG. 3 is a configuration diagram illustrating an example of the first imaging unit 31 and the second imaging unit 32 according to the first embodiment. FIG. 3 shows an example in which the first imaging unit 31 and the second imaging unit 32 are configured by a confocal optical system.

  The first imaging unit 31 captures an image of an observation site of the subject O (hereinafter referred to as a first observation image) from the first direction 12. The first imaging unit 31 includes a first direction optical system 41 having an optical axis along the first direction 12 and a first detection unit 42. When configuring a confocal optical system, the first direction optical system 41 has a light source 43 as shown in FIG.

  The second imaging unit 32 captures an image of an observation site (hereinafter referred to as a second observation image) from a second direction 13 different from the first direction 12. The second imaging unit 32 includes a second direction optical system 44 having an optical axis along the second direction 13 and a second detection unit 45. The second direction optical system 44 has a light source 46. The first direction optical system 41 and the second direction optical system 44 constitute a so-called magnifying optical system or wide angle optical system.

  The first imaging unit 31 and the second imaging unit 32 are integrally held by the housing 20. Note that the description of the configuration of the second imaging unit 32 equivalent to the configuration of the first imaging unit 31 is omitted.

  As the light source 43, for example, a halogen lamp or a laser can be used. The light emitted from the light source 43 becomes a planar illumination light beam through the illumination lens. This light is focused on the subject-side condensing point while being projected onto the subject O via the polarization beam splitter.

  Of the reflected light from the surface of the observation site, the reflected light at the subject-side condensing point is optically conjugate with the subject-side condensing point by the objective lens and the imaging lens (image-side condensing point). ). The light that has passed through the aperture provided at the image-side condensing point is incident on the image sensor that constitutes the first detection unit 42.

  The first detection unit 42 is configured by a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and sends a signal corresponding to the intensity of irradiated light to the inside of the image processing unit 15 m and the base 14. To the image processing unit 15.

  The eyepiece unit 21 includes a left-eye observation unit 21L and a right-eye observation unit 21R attached to the housing 20. The left-eye observation unit 21L and the right-eye observation unit 21R correspond to the display directions of the left-eye parallax image (first parallax image) and the right-eye parallax image (second parallax image) displayed on the built-in 3D display device 33, respectively. It is provided at the position. The user can perceive a stereoscopic image based on the first parallax image and the second parallax image displayed on the built-in 3D display device 33 by looking at the built-in 3D display device 33 via the eyepiece unit 21.

  The changeover switch 22 is provided at a position that can be operated by the user, receives a change instruction from the user to display the observation site as a stereoscopic image or a flat image, and gives this instruction information to the image processing units 15 and 15m. FIG. 1 shows an example in which the changeover switch 22 is attached to the side surface of the housing 20. The switching instruction may be given to the image processing units 15 and 15m via the input unit 17.

  The focus adjustment unit 23 is provided at a position operable by the user, receives a focus adjustment instruction for an image displayed on the built-in 3D display device 33 from the user, and gives the instruction information to the image processing units 15 and 15m. FIG. 1 shows an example in which the focus adjustment unit 23 is attached to the side surface of the housing 20. The focus adjustment instruction may be given to the image processing units 15 and 15m via the input unit 17.

  The monitor 3D display device 18 may be a glasses-type 3D display or a naked-eye 3D display that does not require dedicated glasses. In the eyeglass system, two parallax images, a left-eye parallax image (first parallax image) and a right-eye parallax image (second parallax image) output from the image processing units 15 and 15m, are converted into glasses with a polarizing filter or a liquid crystal shutter. By separating with attached glasses, the viewer feels the depth of the three-dimensional object.

  A naked-eye 3D display has two parallax images, a parallax image for a left eye and a parallax image for a right eye, or a parallax image group further decomposed in a multi-parallax (for example, 9 parallax) direction with a cylindrical lens called a lenticular lens. Sort in the direction. As a result, it is possible to give a viewer a sense of depth of a three-dimensional object without using dedicated glasses. In the case of a naked-eye 3D display, when an observer moves around the display device, it is possible to give the observer a stereoscopic effect as if he / she observed an actual three-dimensional object. .

  The built-in 3D display device 33 may also be a glasses-based 3D display or a naked-eye 3D display that does not require dedicated glasses. The built-in 3D display device 33 is expected by the user via the eyepiece unit 21. For this reason, when the built-in 3D display device 33 is a glasses system, a polarizing filter or a liquid crystal shutter may be provided for the eyepiece unit 21.

  The image processing units 15 and 15m are constituted by a storage medium such as a CPU, a RAM, and a ROM. The CPU of the image processing unit 15 loads an image generation program stored in a storage medium such as a ROM and data necessary for executing this program into the RAM, and displays a stereoscopic image display and a plane of the observation site according to this program. A process for easily switching the image display in accordance with a user instruction is executed.

  The RAMs of the image processing units 15 and 15m provide a work area for temporarily storing programs and data executed by the CPU. The storage medium such as the ROM of the image processing units 15 and 15m stores various data necessary for executing the image processing program.

  A storage medium such as a ROM has a configuration including a recording medium readable by a CPU, such as a magnetic or optical recording medium or a semiconductor memory, and a part of programs and data in the storage medium. Or you may comprise so that all may be downloaded via an electronic network.

  Here, an outline of the operation of the image processing unit 15m according to the present embodiment will be briefly described.

  FIG. 4 is a flowchart illustrating an example of a procedure when the image processing unit 15m easily switches between the stereoscopic image display and the planar image display of the observation site in accordance with a user instruction. In FIG. 4, reference numerals with numbers added to S indicate steps in the flowchart.

  First, in step S1, the image processing unit 15m acquires a first observation image and a second observation image captured from the first direction 12 and the second direction 13 of the observation region, respectively.

  Next, in step S2, the image processing unit 15m should display a stereoscopic image (3D display) on the built-in 3D display device 33 and the monitor 3D display device 18, or display a planar image (2D display). Or a switching instruction from the user. If 3D display is to be performed, the process proceeds to step S3. On the other hand, if 2D display is to be performed, the process proceeds to step S4.

  Next, in step S <b> 3, the image processing unit 15 m displays at least the image of the observation region on the built-in 3D display device 33 and the monitor 3D display device 18 as a stereoscopic image.

  On the other hand, in step S4, the image processing unit 15m displays at least the image of the observation region on the built-in 3D display device 33 and the monitor 3D display device 18 as a planar image. When displaying as a planar image, the parallax image for the left eye (first parallax image) and the parallax image for the right eye (second parallax image) are taken by either the first imaging unit 31 or the second imaging unit 32. Become.

  Next, in step S5, the image processing unit 15m determines whether or not to end a series of procedures (for example, whether or not a processing end instruction has been received from the user via the input unit 17). Returning to step S1, if it should be terminated, the series of procedures is terminated.

  In step S2-4, the built-in 3D display device 33 and the monitor 3D display device 18 may display images of different dimensions in accordance with a user instruction. When the stereoscopic observation apparatus 10 includes the image processing unit 15, the operation related to the monitor 3D display device 18 may be executed by the image processing unit 15 in the processes of steps S <b> 3 and S <b> 4. When the stereoscopic observation apparatus 10 includes the image processing unit 15 and does not include the image processing unit 15m, the above-described procedure is executed by the image processing unit 15, and the image processing unit 15 includes the monitor 3D display device 18 and the built-in 3D. Display control of the display device 33 may be performed.

  With the above procedure, the stereoscopic image display and the planar image display of the observation site can be easily switched according to the user's instruction.

  For this reason, according to the stereoscopic observation apparatus 10 according to the present embodiment, the user can easily change the observation site at any timing while considering both the burden on the eyes of the user and the necessity of the depth information of the observation site. Since the planar image display and the stereoscopic image display of the image can be switched, it is very convenient.

  In addition, since the depth information of the observation site can be easily obtained by the stereoscopic image display as compared with the case where only the planar image display is possible, the user can perform an accurate and quick procedure and the subject to be processed The burden of O can be reduced. In addition, compared to a case where only stereoscopic image display is possible, the user can easily switch to planar image display when he / she feels fatigue of eyes due to stereoscopic image display, so that the burden on the user can be greatly reduced.

  Next, details of the image processing unit 15m according to the present embodiment will be described.

  FIG. 5 is a block diagram schematically illustrating an internal configuration example of the image processing unit 15m according to the first embodiment. A part or all of the internal configuration may be configured as a CPU function realization unit, or may be configured by hardware logic such as a circuit. Further, the configuration of the image processing unit 15 is the same as the configuration of the image processing unit 15m, and thus the description thereof is omitted.

  FIG. 6 is an explanatory diagram showing an example of an instruction reception image 50 for receiving a user instruction for display content.

  The image processing unit 15m includes at least a parallax image generation unit 51, a switching instruction reception unit 52, and an image composition unit 53 as a basic configuration.

  The parallax image generation unit 51 acquires the first observation image obtained by the first imaging unit 31 and the second observation image obtained by the second imaging unit 32 (see step S1 in FIG. 4), and these observation images. Based on this, a left-eye parallax image (first parallax image) and a right-eye parallax image (second parallax image) are generated.

  Further, the first imaging unit 31 and the second imaging unit 32 may be configured to be capable of wide-angle imaging using a wide-angle imaging lens such as a wide-angle lens or a fisheye lens. In this case, the parallax image generation unit 51 corrects distortion caused by the wide-angle imaging lens, and then generates a left-eye parallax image and a right-eye parallax image based on the first observation image and the second observation image. (Distortion correction processing).

  The switching instruction receiving unit 52 receives a switching instruction from the user via the changeover switch 22 or the input unit 17 to display the observation site as a stereoscopic image or a planar image (see step S2 and FIG. 6 in FIG. 4).

  When instructed to display a stereoscopic image, the image composition unit 53 displays the left-eye parallax image and the right-eye parallax image on the built-in 3D display device 33 and the monitor 3D display device 18 to display the stereoscopic image (FIG. 4). Step S3). Further, when instructed to display a planar image, the image composition unit 53 displays a planar image based on either the left-eye parallax image or the right-eye parallax image on the built-in 3D display device 33 and the monitor 3D display device 18. (See step S4 in FIG. 4).

  Further, when instructed to display a stereoscopic image, the image composition unit 53 may display only a part of the stereoscopic image and display another part of the planar image (partial stereoscopic display processing). Specifically, the image compositing unit 53 uses the corresponding regions of the first parallax image and the second parallax image for the predetermined region of the imaging region to use the built-in 3D display device 33 and the monitor 3D display device 18. A stereoscopic image is displayed on the screen, and a planar image based on either the first parallax image or the second parallax image is displayed on the built-in 3D display device 33 and the monitor 3D display device 18 for an area other than the predetermined area of the imaging area. Let

  For example, when the first imaging unit 31 and the second imaging unit 32 are configured to be capable of wide-angle imaging, it is preferable to display a stereoscopic image only in the vicinity of the imaging region center where distortion is unlikely to occur. In this case, the parallax image generation unit 51 may not perform distortion correction.

  Next, focus adjustment will be briefly described. When a focus adjustment instruction is received from the user via the focus adjustment unit 23 or the input unit 17, the image processing units 15 and 15m can perform focus adjustment by software image processing.

  Furthermore, the image processing units 15 and 15m according to the present embodiment include a drive control unit 54, and can perform focus adjustment by hardware adjustment when the focus adjustment instruction is an instruction for a stereoscopic image.

  Specifically, the drive unit 34 is controlled by the drive control unit 54 to change the relative positional relationship between the first direction optical system 41 and the second direction optical system 44. When the drive control unit 54 receives a focus adjustment instruction for the stereoscopic image from the user via the focus adjustment unit 23 or the input unit 17, the drive control unit 54 changes the angle θ formed by the first direction 12 and the second direction 13 to change the stereoscopic image. The drive unit 34 is controlled to adjust the focus of the image.

  According to the drive control unit 54, the stereoscopic image can be focused by changing the relative positional relationship between the first direction optical system 41 and the second direction optical system 44. It should be noted that a part of the built-in 3D display device 33 and the monitor 3D display device 18 is specified as a region of interest, where the part is located in the left-eye parallax image and the right-eye parallax image, and the left-eye parallax image and The focus adjustment may be performed semi-automatically using position shift information in the parallax image for the right eye.

  Subsequently, a superimposed display of the stereoscopic image of the observation site and the planar image and the rendering image will be described.

  The image processing unit 15m may be configured so that a rendering image based on external medical three-dimensional image data (volume data) can be superimposed and displayed on a stereoscopic image and a planar image of the observation site. In this case, the image processing unit 15m further includes a network connection unit 55 and a rendering unit 56.

  The network connection unit 55 implements various information communication protocols according to the form of the network 100. The network connection unit 55 connects the stereoscopic observation apparatus 10 and another apparatus such as the modality 101 according to these various protocols. Here, the network 100 means an entire information communication network using telecommunications technology. In addition to a wireless / wired LAN such as a hospital basic LAN (Local Area Network) and an Internet network, a telephone communication line network, an optical fiber communication network. Cable communication networks and satellite communication networks.

  The stereoscopic observation apparatus 10 receives medical volume data (medical three-dimensional image data) from the modality 101 and the image server 102 connected via the network 100. Volume data and reconstructed image data output from the modality 101 are received via the network 100 and stored in a storage medium (not shown).

  The modality 101 is a medical image diagnostic apparatus such as an X-ray CT (Computed Tomography) apparatus, an MRI (Magnetic Resonance Imaging) apparatus, an ultrasonic diagnostic apparatus, or an X-ray diagnostic apparatus, and is obtained by imaging an object O. It can be configured by a device capable of generating volume data (medical three-dimensional image data) based on projection data.

  The image server 102 is a server for long-term storage of images provided in, for example, a PACS (Picture Archiving and Communication System), and is an X-ray CT apparatus and a magnetic resonance imaging apparatus connected via the network 100. In addition, volume data generated by the modality 101 such as an ultrasonic diagnostic apparatus and an X-ray diagnostic apparatus, a reconstructed image, and the like are stored.

  The rendering unit 56 generates a volume rendering image of the observation site based on the volume data of the subject O acquired via the network 100. The image composition unit 53 superimposes the volume rendering image on the stereoscopic image and the planar image, and displays them on the built-in 3D display device 33 and the monitor 3D display device 18.

  More specifically, the image synthesis unit 53 performs alignment of the first parallax image, the second parallax image, and the volume rendering image by associating feature points such as bones, brain tables, and blood vessels. The feature points may be specified by an input operation by the user on the displayed image, or may be automatically set by the image composition unit 53 extracting pre-registered feature points from each image.

  In addition, the rendering unit 56 generates a volume rendering image so as to highlight the attention location when the information on the attention location related to the technique (such as a surgical target region or a dangerous region such as a blood vessel) is attached to the volume data. May be.

  Further, the rendering unit 56 may generate volume rendering images corresponding to the first direction 12 and the second direction 13 based on the volume data so that the volume rendering image can be stereoscopically displayed. In this case, when instructed to display a stereoscopic image, the image synthesis unit 53 synthesizes the first parallax image and the second parallax image and the volume rendering image in the corresponding direction, respectively, and the built-in 3D display device 33 and A 3D image is displayed on the monitor 3D display device 18. Note that a stereoscopic image of the volume rendering image may be superimposed on the planar image of the observation site in accordance with a user instruction.

  Further, the image composition unit 53 may give a predetermined transmittance (transparency) to the volume rendering image. This transmittance may be set by the user via the input unit 17.

When displaying the rendered image in a superimposed manner, the user exists in the back of the surface of the observation site obtained from the rendering image in addition to the information obtained from the surface image of the observation site captured by the first imaging unit 31 and the second imaging unit 32 Information on the part to be performed can be obtained. In addition, when the attention location obtained from the volume data is highlighted in the rendering image, the user can easily obtain information such as blood vessels and danger sites that exist in the back of the surface of the observation site.
Next, the display of distance information will be described.

  The image processing unit 15m may be configured so that images (distance information images) indicating various distances can be superimposed and displayed on the stereoscopic image and the planar image of the observation site. In this case, the image processing unit 15m further includes a distance calculation unit 57 and a distance information image generation unit 58, and displays a distance information image based on the output of the distance sensor 35. When the display of the distance information image is unnecessary, the stereoscopic observation apparatus 10 may not include the distance sensor 35, the distance calculation unit 57, and the distance information image generation unit 58.

  The distance sensor 35 is provided in the vicinity of the distal end of the first direction optical system 41 and the second direction optical system 44 on the subject O side. The distance sensor 35 outputs a signal corresponding to the distance from the end of the housing 20 on the subject side to a predetermined position on the surface of the observation site.

  The distance calculation unit 57 uses the output of the distance sensor 35 to determine the distance from the subject-side end of the housing 20 to a predetermined position on the surface of the observation site (for example, the subject-side condensing point on the brain surface). . The distance information image generation unit 58 generates an image indicating the distance to the surface. The image composition unit 53 superimposes an image indicating the distance to the surface on the stereoscopic image and the planar image, and displays them on the built-in 3D display device 33 and the monitor 3D display device 18.

  In addition, when the image processing unit 15m includes the rendering unit 56, and information on the attention location related to the procedure is attached to the volume data, the distance calculation portion 57 extracts the attention location in the rendered image, and the surface of the observation region. The distance from the predetermined position to the point of interest may be obtained (see FIG. 6). In this case, the distance information image generation unit 58 generates an image indicating the distance to this point of interest. The image composition unit 53 superimposes an image indicating the distance to the target location on the stereoscopic image and the planar image, and causes the built-in 3D display device 33 and the monitor 3D display device 18 to display them.

  Further, the distance calculation unit 57 may acquire the first observation image and the second observation image from the first detection unit 42 and the second detection unit 45. In this case, the distance calculation unit 57 extracts a predetermined portion (the tip of the catheter, forceps, a marker, etc.) of the treatment device (catheter, forceps, etc.) included in the first observation image and the second observation image, and A distance from a predetermined position on the surface to a predetermined position is obtained. In this case, the distance information image generation unit 58 generates an image indicating the distance to the predetermined portion of the treatment device. The image composition unit 53 superimposes an image indicating the distance to the predetermined location of the treatment device on the stereoscopic image and the planar image, and displays the superimposed image on the built-in 3D display device 33 and the monitor 3D display device 18.

  According to the distance sensor 35, the distance calculation unit 57, and the distance information image generation unit 58, the user can easily know the distance from the end of the housing 20 on the subject side to a predetermined position on the surface of the observation site. In addition, when displaying an image indicating the distance to the point of interest in the rendered image, the user can easily grasp the distance to the point of interest at a position that cannot be confirmed from the stereoscopic image and the surface image of the observation site. it can. Further, for example, when displaying an image indicating the distance from the tip of the forceps to the surface of the observation site, the user can prevent the adverse effect of unexpectedly damaging the surface of the observation site.

  In addition, when the distance obtained by the distance calculation unit 57 is equal to or smaller than a predetermined threshold, the image processing unit 15m further outputs an audio output and a buzzer output via a speaker, and the built-in 3D display device 33 and the monitor 3D display device 18. A warning unit 59 that warns the user by at least one of warning indications for the above may be further provided. When performing warning display, for example, the warning unit 59 may display character information indicating that the built-in 3D display device 33 and the monitor 3D display device 18 are dangerous. A frame surrounding the position may be displayed. At this time, the frame or the area inside the frame may be painted in red or the like, or the frame, the area within the frame, or the entire image may be blinked.

  According to the stereoscopic observation apparatus 10 according to the present embodiment, the stereoscopic image display and the planar image display of the observation site can be easily switched according to a user instruction.

  Note that the microscope unit 11 may be configured so that a plurality of users can simultaneously observe an observation site. FIG. 7A is an explanatory diagram showing an example in which a plurality of eyepieces 21 are provided in the microscope unit 11, and FIG. 7B is a case in which the eyepieces 21 and the 3D display unit 60 are provided in the microscope unit 11. It is explanatory drawing which shows the example of. When the image composition unit 53 outputs the same image to each of the plurality of eyepiece units 21 and the 3D display unit 60, it is possible to simultaneously observe the observation site by a plurality of users.

(Second Embodiment)
Next, a second embodiment of the stereoscopic observation apparatus according to the present invention will be described.

  FIG. 8 is an overall configuration diagram showing an example of a stereoscopic observation apparatus 10A according to the second embodiment of the present invention. In addition, about the same structure as the stereoscopic observation apparatus 10 which concerns on 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  As shown in FIG. 1, the stereoscopic observation apparatus 10 </ b> A includes a microscope unit 11 </ b> A. The microscope unit 11A according to the present embodiment is a so-called endoscope.

  The microscope unit 11A has a housing 20A. The housing 20A is provided with a changeover switch 22A for giving an instruction for the user to switch between stereoscopic image display and planar image display. The trunk of the housing 20A functions as a handle 24A. A part of the housing 20 </ b> A forms an insertion portion 70.

  The image processing unit 15 </ b> A according to the present embodiment is provided not in the microscope unit 11 but in the base 14. The base 14 is provided with a light source 43 </ b> A, and light emitted from the light source 43 </ b> A is guided to a light guide material such as an optical fiber to illuminate an observation site of the subject O from the distal end of the insertion portion 70.

  FIG. 9 is an explanatory diagram showing the IX portion of FIG. 8 in an enlarged manner. As shown in FIG. 9, at the distal end of the insertion portion 70 of the microscope unit 11A (endoscope), the wide-angle imaging lens 71 is disposed at the distal end of the first direction optical system 41A and the distal end of the second direction optical system 44A, respectively. And 72 are attached. When the insertion portion 70 is formed thin, it is preferable to use the wide-angle imaging lenses 71 and 72 as described above. In this case, the first direction optical system 41A and the second direction optical system 44A are configured by a light guide material such as an optical fiber, for example.

  FIG. 10 is a block diagram schematically illustrating an internal configuration example of the image processing unit 15A according to the second embodiment. A part or all of the internal configuration may be configured as a CPU function realization unit, or may be configured by hardware logic such as a circuit.

  As illustrated in FIG. 10, the stereoscopic observation apparatus 10 </ b> A according to the present embodiment may not include the focus adjustment unit 23, the built-in 3D display device 33, the drive unit 34, and the drive control unit 54.

  The parallax image generation unit 51A corrects distortion caused by the wide-angle imaging lenses 71 and 72, and is based on the first observation image and the second observation image output from the first detection unit 42A and the second detection unit 45A, respectively. A first parallax image and a second parallax image are generated.

  The image processing unit 15A according to the present embodiment includes a distortion correction process, a partial stereoscopic display process, a stereoscopic image of an observation site, a superimposed display process of a planar image and a rendering image, a distance information display process, and a warning process according to the first embodiment. Since these are the same as the image processing units 15 and 15m according to FIG.

  A focus adjustment process of the image processing unit 15A according to the present embodiment will be described.

  FIG. 11 is a diagram for explaining the focus adjustment process of the image processing unit 15A according to the second embodiment.

  When the insertion portion 70 is formed thin, it is difficult to change the first direction 12 and the second direction 13 by hardware. Therefore, the parallax image generation unit 51A according to the present embodiment uses a virtual third image that is different from both the first direction 12 and the second direction 13 by using at least one of the first observation image and the second observation image. A third parallax image corresponding to the case where the observation site is imaged from the direction 80 is generated. Then, when instructed to display a stereoscopic image, the image synthesis unit 53A displays either the first parallax image or the second parallax image and the third parallax image on the monitor 3D display device 18 to display the stereoscopic image. Let

  The stereoscopic observation apparatus 10A according to the present embodiment has the same effects as the stereoscopic observation apparatus 10 according to the first embodiment. In addition, the stereoscopic observation apparatus 10A according to the present embodiment can perform focus adjustment by generating a parallax image corresponding to a case where an observation site is imaged from the virtual third direction 80. For this reason, the length of the insertion portion 70 can be reduced, and is particularly suitable for an endoscope.

  Note that the method of performing the focus adjustment by generating a parallax image corresponding to the case where the observation site is imaged from the virtual third direction 80 is the software focus of the stereoscopic observation apparatus 10 according to the first embodiment. It may be used in adjustment.

  In addition, although some embodiment of this invention was described, these embodiment is shown as an example and is not intending limiting the range of invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

10, 10A Stereoscopic observation device 11, 11A Microscope unit 12 First direction 13 Second direction 15, 15m, 15A Image processing unit 17 Input unit 18 Monitor 3D display device 20, 20A Housing 21 Eyepiece 21L For left eye Observation unit 21R Right-eye observation unit 31 First imaging unit 32 Second imaging unit 33 Built-in 3D display device 34 Drive unit 35 Distance sensor 41, 41A First direction optical system 44, 44A Second direction optical system 51, 51A Parallax image generation Unit 52 switching instruction receiving unit 53, 53A image composition unit 54 drive control unit 56 rendering unit 57 distance calculation unit 59 warning unit 70 insertion unit 71, 72 wide-angle imaging lens 80 third direction O subject

Claims (14)

A first imaging unit that images a first observation image of an observation site of the subject from a first direction;
A second imaging unit that images a second observation image of the observation region from a second direction different from the first direction;
A housing that integrally holds the first imaging unit and the second imaging unit;
A parallax image generation unit that generates a first parallax image and a second parallax image based on the first observation image and the second observation image;
A switching instruction receiving unit that receives a switching instruction to display the stereoscopic image display or planar image display of the observation site from a user;
When instructed to display the stereoscopic image, the first parallax image and the second parallax image are displayed on the 3D display device to display the stereoscopic image, while when instructed to display the planar image, An image compositing unit that causes the 3D display device to display a planar image based on one of the one parallax image and the second parallax image;
Stereoscopic observation device with A signal according to the distance from the subject-side end of the housing to a predetermined position on the surface of the observation site provided near the subject-side tip of the optical system of the first imaging unit and the second imaging unit A distance sensor that outputs
A distance calculation unit for obtaining a distance from a subject-side end of the housing to a predetermined position on the surface of the observation site using an output of the distance sensor;
Further comprising
The image composition unit
An image indicating a distance from a subject-side end of the housing to a predetermined position on the surface of the observation site is superimposed on the stereoscopic image and the planar image and displayed on the 3D display device;
The stereoscopic observation apparatus according to claim 1. A rendering unit that generates a volume rendering image of the observation site based on the volume data of the subject;
Further comprising
The image composition unit
The volume rendering image is superimposed on the stereoscopic image and the planar image and displayed on the 3D display device.
The stereoscopic observation apparatus according to claim 1 or 2. The rendering unit
Generating a volume rendering image corresponding to each of the first direction and the second direction based on the volume data;
The image composition unit
When instructed to display the stereoscopic image, the first parallax image and the second parallax image and the volume rendering image in the corresponding direction are respectively combined and displayed on the 3D display device to display the stereoscopic image. Display,
The stereoscopic observation apparatus according to claim 3. The distance calculation unit
Extracting a point of interest in the rendered image, obtaining a distance from the point of interest to a predetermined position on the surface of the observation site,
The image composition unit
An image indicating a distance from the target location in the rendering image to a predetermined position on the surface of the observation site is superimposed on the stereoscopic image and the planar image and displayed on the 3D display device.
The stereoscopic observation apparatus according to any one of claims 2 to 4. The distance calculation unit
Extracting a predetermined portion of the treatment device included in the first observation image and the second observation image, and determining a distance from the predetermined portion to a predetermined position on the surface of the observation portion;
The image composition unit
An image indicating a distance from the predetermined location of the treatment device to a predetermined position on the surface of the observation site is superimposed on the stereoscopic image and the planar image and displayed on the 3D display device.
The stereoscopic observation apparatus according to any one of claims 2 to 5. A warning unit that warns the user by at least one of an audio output and a buzzer output via a speaker and a warning display for the 3D display device when the distance obtained by the distance calculation unit is equal to or less than a predetermined threshold;
The stereoscopic observation apparatus according to any one of claims 2 to 6, further comprising: Each of the first imaging unit and the second imaging unit is
A wide-angle imaging lens is attached and held integrally with the housing,
The parallax image generation unit
Correcting distortion caused by the wide-angle imaging lens to generate a first parallax image and a second parallax image based on the first observation image and the second observation image;
The stereoscopic observation apparatus according to any one of claims 1 to 7. The parallax image generation unit
Corresponding to a case where the observation site is imaged from a virtual third direction different from both the first direction and the second direction using at least one of the first observation image and the second observation image. A third parallax image is generated,
The image composition unit
When instructed to display the stereoscopic image, the stereoscopic image is displayed by displaying either the first parallax image or the second parallax image and the third parallax image on the 3D display device;
The stereoscopic observation apparatus according to any one of claims 1 to 8. The image composition unit
When instructed to display the stereoscopic image, for a predetermined region of the imaging region, the stereoscopic image is displayed on the 3D display device using the corresponding regions of the first parallax image and the second parallax image. And the 3D display device displays a planar image based on one of the first parallax image and the second parallax image for an area other than the predetermined area of the imaging area.
The stereoscopic observation apparatus according to any one of claims 1 to 9. The 3D display device built in the housing;
A left-eye observation unit attached to the housing and provided at a position corresponding to each display direction of the first parallax image and the second parallax image displayed on the 3D display device built in the housing. And an eyepiece composed of an observation part for the right eye,
The stereoscopic observation apparatus according to any one of claims 1 to 10, further comprising: The first imaging unit and the second imaging unit are:
A first directional optical system and a second directional optical system having optical axes along the first direction and the second direction, respectively.
A drive unit for changing a relative positional relationship between the first direction optical system and the second direction optical system;
When a focus adjustment instruction for the stereoscopic image is received from the user via the input unit, the drive unit is controlled to adjust the focus by changing the relative positional relationship between the first direction and the second direction. A drive control unit;
The stereoscopic observation apparatus according to claim 11, further comprising: The housing is
An insertion part constituting an endoscope,
Each of the first imaging unit and the second imaging unit is
A wide-angle imaging lens is attached and held integrally with the insertion part,
The parallax image generation unit
Correcting distortion caused by the wide-angle imaging lens to generate a first parallax image and a second parallax image based on the first observation image and the second observation image;
The stereoscopic observation apparatus according to any one of claims 1 to 10. The 3D display device for a monitor provided outside the housing;
Further comprising
The image composition unit
Displaying the stereoscopic image and the planar image on the 3D display device for a monitor provided outside the housing;
The stereoscopic observation apparatus according to any one of claims 1 to 13.

Images