JP5818265B2 - Stereoscopic endoscope device - Google Patents

Description

  The present invention relates to a stereoscopic endoscope apparatus, and in particular, captures a parallax image by an imaging unit provided at a distal end portion of an insertion unit to be inserted into a body cavity, and a stereoscopic image (3D image) of a site to be observed in the body cavity. The present invention relates to a stereoscopic endoscope apparatus capable of displaying

  The stereoscopic endoscope system includes a pair of left and right imaging units (imaging means) including an imaging optical system and a solid-state imaging device at a distal end portion of an insertion unit of an endoscope apparatus that is inserted into a body cavity. The part to be observed is photographed, and left and right parallax images are obtained as stereoscopic images (3D images) for stereoscopic viewing. The 3D image is displayed in 3D on the 3D display device, and the right eye image is observed with the observer's right eye, and the left eye image is observed with the observer's left eye, so that the object is observed. The part is observed three-dimensionally.

  In Patent Document 1, in a stereoscopic endoscope apparatus, it is impossible to confirm the state of the peripheral portion or the position of a treatment tool only by observing a site to be observed with a narrow-angle 3D image. It is disclosed that 2D images can be displayed. According to this, a 2D image with a standard angle of view is taken by one of the left and right imaging units, and a wide-angle 2D image is taken by the other imaging unit. In the wide-angle two-dimensional mode, a wide-angle 2D image is displayed, and in the three-dimensional mode, a 3D image is displayed with a 2D image with a standard angle of view and a 2D image cut out from the wide-angle 2D image. As a result, a 3D image and a wide-angle 2D image are displayed.

  Patent Document 2 discloses a three-dimensional endoscope having a photographing unit for photographing a wide-angle 2D image separately from a pair of left and right photographing units for photographing a 3D image. 3D images and wide-angle 2D images are displayed.

Japanese Patent Laid-Open No. 9-5643 JP 2003-334160 A

  By the way, as in the stereoscopic endoscope apparatus of Patent Document 2, it is possible to provide an imaging unit for capturing a wide-angle 2D image in addition to a pair of left and right imaging units for capturing a 3D image. This is not preferable because it leads to an increase in the diameter and size of the tip portion.

  On the other hand, when the 3D image and the wide-angle 2D image can be displayed by the left and right parallax images captured by the pair of left and right imaging units as in Patent Document 1, the 3D image and There is an advantage that a wide-angle 2D image can be displayed.

  Here, in order to increase the quality of 3D images and 2D images, it is necessary to increase the resolution, and in the case of moving images, it is also necessary to increase the frame rate.

  However, when the resolution and the frame rate are increased, there is a problem that the amount of data that must be processed per unit time increases and the processing load (compression, recording, transmission, etc.) of the system increases.

  On the other hand, when observing a 3D image and a 2D image, the resolution of an image that does not cause dissatisfaction with the image quality is higher for a 3D image than for a 2D image. 3D images are higher than 2D images in order to make the image feel natural and smooth.

  In addition, the 3D image is more useful than the 2D image for the observer, and it is important that the 3D image can be displayed as a higher quality and smooth moving image.

  Therefore, it is desired to improve the quality of the 3D image by increasing the resolution and frame rate of the 3D image while suppressing an increase in the processing load of the system as much as possible. In Patent Document 1, no proposal is made regarding the improvement of the quality of such a 3D image.

  Further, it is desirable that the 2D image is a wide-angle image in order to obtain as wide a range of information as possible. In that case, it is conceivable to use an image pickup device having a large light receiving surface. However, in order to maintain a high quality 3D image, the pixel density of the image pickup device cannot be reduced and the number of pixels increases. Therefore, there is a problem that the processing load of the system increases.

  Accordingly, it is desired to widen the 2D image while suppressing an increase in the processing load of the system as much as possible.

  Further, it is conceivable to use an optical method as a method for improving the quality of 3D images and widening the angle of 2D images without increasing the processing load on the system. For example, it is conceivable to improve the performance of the optical system to increase the resolution of an image (optical image) formed on the imaging surface, or to use an optical system having a wide angle as much as possible.

  However, when a part of a 2D image acquired by one photographing unit is cut out as a 3D image (one parallax image of the 3D image) as in Patent Document 1, there are the following problems. .

  When the optical system of the imaging unit has a wide angle, the resolution of an image (optical image) formed on the imaging plane by the optical system varies greatly depending on the position on the imaging plane due to the influence of aberration, distortion, and the like. In particular, such a phenomenon is conspicuous when an optical system having a simple configuration for miniaturization or an inexpensive optical system for cost reduction.

  Therefore, the resolution of an optical image captured as a 3D image may be low, and there is a problem that it is difficult to achieve both a wide angle of the 2D image and a high quality of the 3D image. Therefore, it is desired to achieve both widening of the 2D image and high quality of the 3D image by an optical method.

  The present invention has been made in view of such circumstances, and in a stereoscopic endoscope apparatus that can form a 3D image and a wide-angle 2D image by using parallax images captured by a pair of imaging units, an increase in the processing load of the system is increased. It is an object of the present invention to provide a stereoscopic endoscope apparatus that can improve the quality of a 3D image and widen the angle of a 2D image while suppressing as much as possible.

  In order to achieve the above object, a stereoscopic endoscope apparatus according to one aspect of the present invention captures a subject in a first visual field range and forms a stereoscopic 3D image and a planar 2D image. A first imaging unit for acquiring a first image and a subject in a second visual field range that partially overlaps the first visual field range, and forming a 3D image for stereoscopic viewing and a 2D image for planar viewing. A stereoscopic endoscope apparatus having a second imaging unit that acquires two images, wherein the first imaging unit and the second imaging unit are for stereoscopic vision among image regions in the first image and the second image. The resolution of the image of the 3D area that forms the 3D image is set higher than the resolution of the image of the 2D area that forms only the 2D image for planar view.

  According to the present invention, it is possible to reduce the processing load on 2D image data and increase the resolution and frame rate of the 3D image, and to improve the quality of the 3D image without increasing the processing load on the system. be able to. Further, it is possible to increase the area of the light receiving surface for capturing the 2D image without increasing the processing load of the system, and it is possible to widen the angle of the 2D image.

  In addition, in the form in which the resolution of the image of the 3D area is higher than the resolution of the image of the 2D area, the resolution of the optical image formed by the photographing optical system is set higher than the 2D area as described below. Each of the forms, the form in which the 3D area is made higher than the 2D area with respect to the pixel pitch of the light receiving surface of the image sensor, and the form in which the 3D area is made higher than the 2D area with respect to the density of pixels for effectively acquiring pixel data from the image sensor Is included.

  In the present invention, the first photographing unit includes a first photographing optical system that forms an optical image of the subject on the first imaging surface, and the second photographing unit includes the optical image of the subject on the second imaging surface. The first imaging optical system and the second imaging optical system pick up an image of a 3D area among the first imaging plane and the area in the second imaging plane. The resolution of the optical image of the area is set higher than the resolution of the image of the 2D area by increasing the resolution of the image of the 3D area by increasing the resolution of the optical image of the area where the image of the 2D area is captured. be able to.

  More specifically, the first imaging optical system and the second imaging optical system are 3D in an area including the position where the resolution of the optical image is maximum among the areas in the first imaging plane and the second imaging plane. It is possible to form an optical image in the visual field range for capturing an image. Further, the first photographing optical system and the second photographing optical system can be a fisheye lens of an equidistant projection method, an equal solid angle projection method, or an orthographic projection method.

  According to the present embodiment, even when the resolution of the optical image varies depending on the position on the imaging surface due to distortion or aberration of the photographing optical system, the 3D is surely compared with the case where the resolution is not taken into consideration. The quality of the image can be improved.

  In the present invention, the first photographing unit includes a first photographing optical system that forms an optical image of a subject, and a first image pickup device that picks up an optical image formed by the first photographing optical system. The imaging unit includes a second imaging optical system that forms an optical image of a subject, and a second imaging element that images the optical image formed by the second imaging optical system. The first imaging element and the second imaging element The element may have a form in which a pixel pitch of a region for capturing an image of a 3D region is made smaller than a pixel pitch of a region for capturing an image of a 2D region.

  By making the pixel pitch of the image sensor different between the 3D region and the 2D region as in this embodiment, the resolution of the 3D image can be made higher than the resolution of the 2D image.

  In the present invention, the first photographing unit includes a first photographing optical system that forms an optical image of a subject, and a first image pickup device that picks up an optical image formed by the first photographing optical system. The second photographing unit includes a second photographing optical system that forms an optical image of a subject, and a second imaging element that picks up an optical image formed by the second photographing optical system. Among the areas within the light receiving surface of the two image sensor, the pixel density for effectively acquiring pixel data from the area for capturing the image of the 3D area, and the pixel for effectively acquiring the pixel data from the area for capturing the image of the 2D area The pixel density changing means for making the density higher than the above density can be provided.

  As in this embodiment, the resolution of the 3D image can be made higher than the resolution of the 2D image depending on the density of pixels that effectively acquire pixel data from the region of the light receiving surface of the image sensor. Specifically, the pixel density can be changed (reduced) by thinning processing or binning processing.

  According to the present invention, it is possible to increase the quality of a 3D image and widen the angle of a 2D image while suppressing an increase in the processing load of the system as much as possible.

Overall configuration diagram showing an outline of the appearance of a stereoscopic endoscope system to which the present invention is applied Front view showing the distal end of the endoscope from the distal end side The block diagram which showed the structure relevant to the process part which displays a 3D image and a wide-angle 2D image as an endoscope image in a stereoscopic endoscope system. Diagram showing the photographic optical system and image sensor of the photographic unit and their visual field ranges The figure which illustrated the right image and left image which were image | photographed as a parallax image by the right imaging | photography part and the left imaging | photography part It is the figure which illustrated the screen composition in the 3D display device of the display image (the display image for right eyes and the display image for left eyes) generated by the display image generation part, and illustrated the display area which displays an endoscope image Figure It is the figure which illustrated the screen composition in the 3D display device of the display image (the display image for right eyes and the display image for left eyes) generated by the display image generation part, and illustrated the display area which displays an endoscope image Figure It is the figure which illustrated the screen composition in the 3D display device of the display image (the display image for right eyes and the display image for left eyes) generated by the display image generation part, and illustrated the display area which displays an endoscope image Figure The figure which showed the outline | summary of the process which produces | generates the right display image and left display image of the 3D image & wide-angle 2D image display area in FIG. The figure which showed the modification of the process which produces | generates the right display image and left display image of the 3D image & wide-angle 2D image display area in FIG. The figure which showed the outline | summary of the process which produces | generates the right display image and left display image of the 3D image display area 160 in FIG.7 and FIG.8 (A). The figure which showed the outline | summary of the process which produces | generates the right display image and left display image of the wide-angle 2D image display area 170 in FIG.7 and FIG.8 (B). The figure which showed the right image and left image which are taken in by the right imaging | photography part and the left imaging | photography part in the modification 1. The figure which showed the right image and the left image which were taken in by the right imaging | photography part and the left imaging | photography part in the image 1 which showed the pixel structure of the right image and left image IL taken in by the imaging | photography part in the modification 2, 3. The figure which showed the structure of the imaging | photography part in the modification 2. The figure which showed the pixel pitch in the light-receiving surface of the image sensor in the modification 3. The figure which showed the structure of the imaging | photography optical system and image sensor of the imaging | photography part in the modification 4. Diagram illustrating the resolution characteristics of the photographic optical system Diagram illustrating the resolution characteristics of the photographic optical system The figure which showed the form of the photographing part where a part of the lens of the photographing optical system is shared The figure which showed the form of the photographing part where all the lenses of the photographing optical system are shared

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is an overall configuration diagram showing an outline of an external appearance of a stereoscopic endoscope system to which the present invention is applied. A stereoscopic endoscope system 10 shown in the figure is a system capable of capturing and displaying a stereoscopic image (3D image) for stereoscopic viewing. Except for the configuration and processing related to imaging, the imaging system is not greatly different from a known endoscope system that captures and displays a normal 2D image (an image for planar view) instead of a 3D image. Hereinafter, the configuration and processing related mainly to imaging will be described, and other configurations and processing can be configured in the same manner as any known endoscope system.

  As shown in the figure, the stereoscopic endoscope system 10 of the present embodiment includes a stereoscopic endoscope device 12 (hereinafter referred to as an endoscope 12), a processor device 14 to which the endoscope 12 is connected, and a light source device. 16 and a 3D display device 18 connected to the processor device 14.

  An endoscope 12 is inserted into the body cavity of a patient, and a pair of left and right parallax images (right image and left image) for displaying a 3D image of a desired observation site are captured by the endoscope 12. It has become. The observation site is irradiated with illumination light supplied from the light source device 16 to the endoscope 12. These parallax images are used not only as 3D images but also as images for displaying wide-angle 2D images (wide-angle 2D images).

  The parallax image photographed by the endoscope 12 is captured by the processor device 14 and subjected to necessary processing to be formed into a display image for displaying a 3D image or a 2D image. Then, the display image is output to the 3D display device 18, and a 3D image or a wide-angle 2D image is displayed on the 3D display device 18 as an endoscopic image. By observing the endoscopic image displayed on the 3D display device 18, the practitioner can observe the site to be observed in the body cavity three-dimensionally or planarly.

  The endoscope 12 includes an insertion portion 20 that can be inserted into a body cavity of a patient (subject), an operation portion 22 that is held by a practitioner and that performs various operations, and the endoscope 12 that includes a processor device 14 and a light source. And a universal cord 24 connected to the device 16.

  The insertion portion 20 is formed in an elongated shape with the longitudinal axis 20a in the longitudinal direction as a central axis, and has an outer peripheral surface that is substantially circular in a cross section orthogonal to the longitudinal axis 20a. The insertion portion 20 includes a distal end portion 30, a bending portion 32, and a flexible portion 34.

  The distal end portion 30 is provided at the distal end of the insertion portion 20, and has a distal end surface 30a substantially orthogonal to the longitudinal axis 20a. As will be described in detail later, the distal end portion 30 includes a component of an imaging unit that images an observed site in a body cavity located in front of the distal end surface 30a, and a light source for the observed site captured by the imaging unit. The components of the illumination unit that emits illumination light from the device 16 are accommodated and held in a hard member.

  The bending portion 32 is connected to the proximal end side of the distal end portion 30 and can be actively bent in the vertical and horizontal directions by the turning operation of the angle knob 40 of the operation portion 22. By bending the bending portion 32, the direction of the distal end portion 30 in the body cavity can be changed to adjust the direction of the site to be observed where imaging is performed by the imaging portion of the distal end portion 30.

  The flexible portion 34 is connected to the proximal end side of the bending portion 32 and is connected to the distal end of the operation portion 22, and connects the proximal end of the bending portion 32 to the distal end of the operation portion 22. The soft portion 34 is soft and flexible, and the soft portion 34 is passively bent according to the shape of the insertion path into the body cavity, etc., so that the distal end portion 30 is moved into the desired position in the body cavity. It can be inserted and placed at the position. Inside the flexible portion 34 and the bending portion 32, a cable, a light guide, and the like connected to the photographing portion and the illumination portion of the distal end portion 30 are disposed.

  The operation unit 22 is connected to the proximal end side of the insertion unit 20, and the operation unit 22 is provided with operation members of the endoscope 12 such as the angle knob 40 and the air / water supply button 42. ing. The practitioner can perform various operations of the endoscope 12 by holding the operation unit 22 and operating an operation member provided in the operation unit 22.

  In addition, a treatment instrument insertion port 44 is provided on the distal end side of the operation unit 22. The treatment instrument insertion port 44 communicates with the distal end portion 30 (a treatment instrument outlet 54 described later) through a treatment instrument channel (pipe) that is inserted through the insertion portion 20. Therefore, by inserting a desired treatment tool from the treatment tool insertion port 44, the treatment tool is led out from the treatment tool lead-out port 54 of the distal end portion 30, and a treatment corresponding to the type of the treatment tool is observed in the body cavity. Etc. can be applied.

  The universal cord 24 extends from the operation unit 22, and a composite type connector 26 is provided at the end. The universal cord 24 is connected to the processor device 14 and the light source device 16 by the connector 26.

  Inside the universal cord 24, a cable, a light guide, and the like that are inserted through the insertion unit 20 and the operation unit 22 from the imaging unit and the illumination unit of the distal end 30 are inserted and arranged. The cable connected to the imaging unit The (signal line) is connected to the processor device 14 via the connector 26, and the light guide connected to the illumination unit is connected to the light source device 16 via the connector 26.

  As a result, the imaging signal of the parallax image captured by the imaging unit of the distal end portion 30 is transmitted to the processor device 14 through the cable, and the illumination light emitted from the light source device 16 is transmitted to the light guide to transmit the distal end portion 30. It is emitted from the illumination part.

  FIG. 2 is a front view showing the distal end portion 30 of the endoscope 12 from the distal end surface 30a side. As shown in the figure, the distal end portion 30 is provided with an imaging unit 50 that captures an observation site on the front side of the distal end surface 30a and an illumination unit 52 that emits illumination light that illuminates the observation site. ing.

  The imaging unit 50 has a pair of imaging units 50R and 50L (right imaging unit 50R and left imaging unit 50L) arranged side by side on the left and right sides, and by these imaging units 50R and 50L, the same site to be observed in the body cavity A pair of left and right parallax images taken from different positions are taken.

  On the distal end surface 30a, observation windows 62R and 62L for capturing light from the observation site are arranged in the imaging units 50R and 50L, respectively, and an illumination window 94 for emitting illumination light from the illumination unit 52 to the observation site is arranged. It is installed. Further, on the distal end surface 30a, a treatment instrument outlet 54 through which the treatment instrument inserted through the treatment instrument insertion port 44 of the insertion section 20 and inserted through the treatment instrument channel is led out from the distal end face 30a, and the air supply / An air supply / water supply nozzle 56 is provided to inject cleaning water or air toward the observation windows 62R and 62L by operating the water supply button 42.

  FIG. 3 is a block diagram illustrating a configuration related to a processing unit that displays a 3D image or a wide-angle 2D image as an endoscopic image in the stereoscopic endoscope system 10.

  As shown in the figure, the endoscope 12 (tip portion 30) is provided with a pair of right and left right photographing portions 50R (first photographing portion) and left photographing portion 50L (second photographing portion) as photographing portions 50. The right photographing unit 50R and the left photographing unit 50L are arranged symmetrically with respect to a predetermined center plane 50a. Each of the imaging units 50R and 50L includes imaging optical systems 60R and 60L (first imaging optical system and second imaging optical system), CCD or MOS type (CMOS) image sensors (imaging elements) 70R and 70L (first imaging). Element and second imaging device), analog signal processing units (AFE) 80R and 80L, transmission units 82R and 82L, and the like.

  The photographing optical systems 60R and 60L of the right photographing unit 50R and the left photographing unit 50L have the same characteristics, and are configured by a plurality of lenses although a detailed configuration is omitted. The light from the part to be observed facing the distal end surface 30a illuminated by the illumination light from the illumination unit 52 shown in FIG. 2 is incident on these photographing optical systems 60R and 60L, and the connection of the photographing optical systems 60R and 60L. An image of the site to be observed is formed on the light receiving surfaces of the image sensors 70R and 70L by image action.

  The images formed on the light receiving surfaces of the image sensors 70R and 70L are captured (photoelectric conversion) as a pair of left and right parallax images by the image sensors 70R and 70L, and are output from the image sensors 70R and 70L as imaging signals. . The parallax image obtained by being imaged by the image sensor 70R of the right photographing unit 50R is called a right image (first image), and the parallax image obtained by being imaged by the image sensor 70L of the left photographing unit 50L is a left image (first image). 2 images).

  FIG. 4 is a diagram illustrating the photographing optical systems 60R and 60L and the image sensors 70R and 70L of the photographing units 50R and 50L, and their visual field ranges.

  In the drawing, the imaging optical system 60R and the image sensor 70R of the right imaging unit 50R and the imaging optical system 60L and the image sensor 70L of the left imaging unit 50L are symmetrical with respect to the center plane 50a perpendicular to the paper surface. Has been placed.

  The photographing optical system 60R and the image sensor 70R of the right photographing unit 50R photograph a subject in the visual field range VFR (first visual field range) as a right image, and the photographing optical system 60L and the image sensor 70L of the left photographing unit 50L include the visual field range. A VFL subject is photographed as a left image, and the visual field range VFR and the visual field range VFL are symmetrical with respect to the central plane 50a, and overlap in the central portion.

  The reference plane RP in the figure is a plane orthogonal to the center plane 50a, and is orthogonal to the longitudinal axis 20a of the insertion portion 20, for example. It is assumed that the position is a predetermined distance L from the distal end surface 30a of the distal end portion 30 and is in focus by the photographing optical systems 60R and 60L.

  The reference plane RP is a position of an object point that is recognized as being present at a position on the screen when a 3D image is displayed on the screen of the 3D display device 18 by a pair of left and right parallax images captured by the imaging units 50R and 50L. Is shown. That is, the object points on the reference plane RP are displayed so as to exist on the screen in the 3D image. The distance L from the tip surface 30a of the reference plane RP is desirably 8 cm to 10 cm, which is a focus position in a general endoscope. Further, it is desirable that about 1 cm before and after the reference plane RP is in focus as the range of the depth of field.

  In the reference plane RP, a region where the visual field range VFR and the visual field range VFL overlap is indicated by a visual field range 3DR from the left end of the visual field range VFR and a visual field range 3DL from the right end of the visual field range VFL. In the reference plane RP, among the regions where the visual field range VFR and the visual field range VFL do not overlap, the region of only the visual field range VFR is indicated by the visual field range 2DR from the right end of the visual field range VFR, and the region of the visual field range VFL only A visual field range 2DL from the left end of the visual field range VFL is shown.

  In the following, when the visual field ranges VFR, VFL, 3DR, and 3DL are referred to individually, the visual field range VFR is the right full visual field range VFR, the visual field range VFL is the left full visual field range VFL, and the visual field range 3DR is the right 3D visual field range. 3DR, the visual field range 3DL is referred to as a left 3D visual field range 3DL, and the visual field range in which the right full visual field range VFR and the left full visual field range VFL overlap on the reference plane RP (that is, the right 3D visual field range 3DR and the left 3D visual field range 3DL are The combined visual field range is referred to as 3D visual field range 3DR & 3DL. When the visual field ranges 2DR and 2DL are referred to individually, the visual field range 2DR is referred to as a right 2D visual field range 2DR, and the visual field range 2DL is referred to as a left 2D visual field range 2DL.

  The subjects O1, O2, and O3 shown in the figure illustrate objects existing on the reference plane RP, the subject O1 exists in the 3D visual field range 3DR & 3DL, and the subject O2 exists in the right 2D visual field range 2DR. The object O3 exists in the left 2D visual field range 2DL. The subjects O4 and O5 indicate subjects that exist in the 3D visual field range 3DR & 3DL but do not exist on the reference plane RP. The subject O4 exists at a position closer to the front end surface 30a than the reference plane RP. 3 shows a subject that is located farther from the front end surface 30a than the reference plane RP.

  When these subjects O1 to O5 exist, the right image IR and the left image IL as shown in the upper part of FIG. 4 and FIGS. 5A and 5B are respectively obtained by the right photographing unit 50R and the left photographing unit 50L. Are taken as a pair of left and right parallax images. The right image IR includes the images of the subjects O1, O2, O4, and O5 among the subjects O1 to O5, and these images are respectively shown as the subject images O1R, O2, O4R, and O5R. The left image IL includes images of subjects O1, O3, O4, and O5 among the subjects O1 to O5, and these images are shown as subject images O1L, O3L, O4L, and O5L, respectively.

  The right image IR and the left image IL shown in the upper part of FIG. 4 are a right 3D visual field range that is a visual field range where the right full visual field range VFR of the right photographing unit 50R and the full visual field range VFL of the left photographing unit 50L overlap each other on the reference plane RP. 3DR and the image of the left 3D visual field range 3DL are superimposed and shown as an image of the 3D visual field range 3DR & 3DL.

  When the right image IR and the left image IL are displayed in such a positional relationship, the image of the subject existing in the 3D visual field range 3DR & 3DL on the reference plane RP is the subject image O1R in the right image IR that is the image of the subject O1. The subject images O1L and O1L shown together with the subject image O1L in the left image IL are displayed at the same position.

  On the other hand, the image of the subject that exists in the 3D visual field range 3DR & 3DL but is closer to the tip surface 30a than the reference plane RP is the subject image O4R and the left image IL in the right image IR that is the image of the subject O4. As shown in the subject image O4L in FIG. 5, the image is separated and displayed at different positions on the left side in the right image IR and on the right side in the left image IL.

  On the contrary, the image of the subject that exists in the 3D visual field range 3DR & 3DL but is farther from the front end surface 30a than the reference plane RP is the subject image O5R in the right image IR that is the image of the subject O5. As in the subject image O5L in the left image IL, the right image IR is separated and displayed at different positions on the right side and the left image IL on the left side.

  Further, the image of the subject existing in the right 2D visual field range 2DR in the region where the right full visual field range VFR and the left full visual field range VFL do not overlap is the right image IR which is an image of the subject O2 existing on the reference plane RP. Is displayed only in the right 2D visual field range 2DR of the right image IR. Similarly, an image of a subject that exists in the left 2D visual field range 2DL in a region where the right full visual field range VFR and the left full visual field range VFL do not overlap each other is an object O3 that exists in the left 2D visual field range 2DL on the reference plane RP. Like the subject image O3 in the left image IL which is the image of the left image IL, it is displayed only in the left 2D visual field range 2DL of the left image IL.

  Therefore, in the 3D display device 18, the right image IR of the right full visual field range VFR photographed by the right photographing unit 50R and the left image IL of the left full visual field range VFL photographed by the left photographing unit 50L are shown in the upper part of FIG. By displaying as a 3D image so that the image of the right 3D field of view range 3DR and the image of the left 3D field of view range 3DL overlap as shown above, subjects such as subjects O1, O4, and O5 that exist in the region of the 3D field of view range 3DR & 3DL Can be stereoscopically viewed by the observer. A subject existing on the reference plane RP such as the subject O1 is recognized as a subject existing on the screen of the 3D display device 18, and a subject existing near the reference plane RP such as the subject O4 is displayed in the 3D display device 18. Is recognized as a subject existing at a position farther than the screen of the 3D display device 18 and is recognized as a subject present at a position farther than the screen of the 3D display device 18 such as the subject O5.

  In addition, since a subject existing in an area captured by only one of the imaging units 50R and 50L, such as the right 2D visual field range 2DR and the left 2D visual field range 2DL, can also be displayed as a 2D image, the 3D visual field range 3DR & 3DL It is possible to observe the state of the site to be observed having a wider field of view than the 3D image.

  Returning to FIG. 3, as described above, the right image IR photographed by the photographing optical system 60R and the image sensor 70R of the right photographing unit 50R, and the left image photographed by the photographing optical system 60L and the image sensor 70L of the left photographing unit 50L. IL is output as an analog imaging signal from the image sensors 70R and 70L, respectively, and taken into analog signal processing units (AFE) 80R and 80L.

  In the AFE 80R and 80L, analog signal processing such as correlated double sampling (CDS), automatic gain (AGC), and analog / digital conversion (A / D) is performed on the imaging signal, and the AFE 80R, The data is output from 80L to the transmitters 82R and 82L.

  In the transmission units 82R and 82L, parallel / serial conversion processing or the like is performed on the imaging signals, and the serial signals are transmitted to cables (signal lines) connected to the imaging units 50R and 50L. The cable is inserted into the insertion unit 20, the operation unit 22, and the universal cord 24 and connected to the reception unit 100 of the processor device 14, and the imaging signal sent to the cable is received by the reception unit 100. The

  Here, in the present embodiment, assuming that the endoscopic image is displayed as a moving image on the 3D display device 18, the right image IR and the left image IL are frame images constituting the moving image in each of the image sensors 70R and 70L. The imaging signals of the right image IR and the left image IL, which are continuously captured and sequentially captured as frame images, are sequentially transmitted from the transmission units 82R and 82L to the reception unit 100 as serial digital signals for each frame image. It is like that.

  In addition, it is desirable to use a high-speed digital transmission technique that performs data transmission using, for example, a low voltage operation signal (LVDS) or the like for transmission of the imaging signal from the transmission units 82R and 82L to the reception unit 100. Further, the imaging signals of the right image IR and the left image IL may be transmitted in parallel through different signal lines, or may be alternately transmitted through a common signal line.

  The processor device 14 includes a receiving unit 100, an image processing unit 102, a display image generating unit 104, a display control unit 106, and the like as processing units for displaying an endoscopic image.

  In the receiving unit 100, the imaging signals of the right image IR and the left image IL transmitted from the right imaging unit 50R and the left imaging unit 50L of the endoscope 12 and transmitted as serial digital signals via the cable are received. These image signals are subjected to parallel / serial conversion processing or the like. As a result, the image pickup signal is restored to a parallel digital signal before being converted in the transmission units 82R and 82L and is taken into the image processing unit 102.

  The image processing unit 102 performs digital image processing such as color separation, color interpolation, gain correction, white balance adjustment, gamma correction, contour enhancement processing, and brightness adjustment processing. As a result, the imaging signals of the right image IR and the left image IL are generated as image data suitable for display, and the image data is captured by the display image generation unit 104.

  Note that the image processing unit 102 sequentially generates image data of the right image IR and the left image IL, and the image data is temporarily stored in a memory (not shown) and sequentially updated to the latest image data. The display image generation unit 104 is configured to sequentially fetch the latest image data from the memory.

  In the display image generation unit 104, the image data of the right-eye display image and the left-eye display image displayed on the entire screen of the 3D display device 18 are sequentially captured from the image processing unit 102 of the right image IR and the left image IL. Generated using image data. The display image for the right eye indicates an image of the entire screen visually recognized by the observer's right eye, and a part or all of the right image IR is included as an endoscopic image. The display image for the left eye shows an image of the entire screen visually recognized by the left eye of the observer, and a part or all of the left image IL is included as an endoscopic image therein. The image data of the display image for the right eye and the display image for the left eye is sequentially updated together with the update of the image data of the right image IR and the left image IL, and is output to the display control unit 106. Details of the processing of the display image generation unit 104 will be described later.

  In the display control unit 106, the video signal for causing the 3D display device 18 to display the right-eye display image and the left-eye display image generated by the display image generation unit 104 is supplied to the right side from the display image generation unit 104. It is generated based on the image data of the eye display image and the left eye display image. The video signal is generated in a standard that can be supported by the 3D display device 18. The video signal is output to the 3D display device 18.

  In the 3D display device 18, an image is displayed according to the video signal from the display control unit 106, and the right-eye display image and the left-eye display image generated by the display image generation unit 104 are the right eye and the left eye, respectively. It is displayed so that it may be visually recognized by one corresponding eye among eyes. Then, by displaying and updating the right image IR and the left image IL included in each of the right-eye display image and the left-eye display image, the endoscopic image is displayed as a moving image, and the 3D of the site to be observed is displayed. Images and 2D images are displayed.

  Here, as a known 3D display device, a display image for the right eye and a display image for the left eye are alternately displayed on the screen of one monitor, and through shutter glasses that are alternately opened and closed in synchronization with this. The right eye display image and the left eye display image on the screen of one monitor (the frame sequential method) or the method of observing the display image for the right eye with the right eye and the display image for the left eye with the left eye Are alternately displayed in units of scanning lines, for example, a right eye display image is observed with the right eye and a left eye display image is observed with the left eye via polarization filter glasses having different polarization directions on the left and right sides (polarization method) ) And a display that displays different images depending on the angle at which the screen is viewed by arranging a minute lens on the screen of a single monitor. Observe the display image for the right eye with the left eye and the left eye with the left eye A method that allows the display image to be observed (integral imaging method) and a fine vertical-striped light-shielding object placed on the screen of one monitor so that different images can be displayed depending on the viewing angle. A system that displays a display image for the right eye and a display image for the left eye on the object, and allows the right eye to observe the display image for the right eye and the left eye to observe the display image for the left eye even with the naked eye (parallax barrier method) Is known. Further, the right eye display image and the left eye display image are displayed on each of the two monitors for the right eye and the left eye like a head-mounted display, and the right eye display image is observed with the right eye. A method of observing a display image for the left eye with the left eye is also known. As the 3D display device 18 of the present embodiment, any system can be employed. Moreover, it is good also as a structure which displays a 3D image and a 2D image on another monitor.

  Next, the processing content of the display image generation unit 104 will be specifically described.

  6 to 8 are diagrams exemplifying screen configurations on the 3D display device 18 of the display images (the display image for the right eye and the display image for the left eye) generated by the display image generation unit 104. FIG. It is the figure which illustrated the display area which displays an image. 6 has a 3D image & wide-angle 2D image display area 150 as an endoscopic image display area. In the 3D image & wide-angle 2D image display area 150, the right image IR and the left image IL photographed by the photographing units 50R and 50L as shown in FIGS. 5A and 5B are shown in the upper part of FIG. In this way, the image is displayed so that the image in the right 3D visual field range 3DR and the image in the left 3D visual field range 3DL overlap. As a result, the images of the 3D visual field range 3DR & 3DL are displayed in 3D as 3D images, and the images of the right 2D visual field range 2DR and the left 2D visual field range 2DL on both sides thereof are displayed in 2D as wide-angle 2D images. Although detailed description is omitted, characters and images related to other information such as patient information and state information of the endoscope 12 are displayed in an area other than the 3D image & wide-angle 2D image display area 150 in the screen. The same applies to regions other than the display area of the endoscopic image in the form of FIGS.

  In the screen configuration in the form of FIG. 7, a 3D image display area 160 and a wide-angle 2D image display area 170 are individually provided in the screen as display areas for endoscopic images. The 3D image display area 160 includes a right 3D visual field range 3DR image and a left 3D visual field range among the right image IR and the left image IL photographed by the photographing units 50R and 50L as shown in FIGS. This is an area where only the 3DL image is displayed at the same position as the 3D visual field range 3DR & 3DL image shown in the upper part of FIG. As a result, the 3D visual field range 3DR & 3DL image is displayed in 3D as a 3D image. The visual field range displayed as a 3D image in the 3D image display area 160 may be a range that is expanded by a certain amount to the left and right than the 3D visual field range 3DR & 3DL.

  On the other hand, the wide-angle 2D image display area 170 is an area for 2D-displaying the images of the 3D visual field range 3DR & 3DL and the right 2D visual field range 2DR and the left 2D visual field range 2DL on the left and right sides as a wide-angle 2D image, as will be described in detail later. It is.

  The form of FIG. 8 is a screen having a 3D image display area 160 similar to FIG. 7 as an endoscope image display area as shown in FIG. 7A and the same as FIG. 7 as shown in FIG. The screen has two configurations including a screen having a wide-angle 2D image display area 170, and the screen is switched according to the type of display mode. When the 3D image display mode is selected as the display mode, a screen having a 3D image display area 160 is displayed as shown in FIG. 5A, and when the wide-angle 2D image display mode is selected, A screen having a wide-angle 2D image display area 170 as shown in (B) is displayed. The display mode may be switched by an observer by operating an input device (not shown) connected to the processor device 14 or by operating an operation member provided on the operation unit 22 of the endoscope 12. it can.

  Note that the screen configurations shown in FIGS. 8A and 8B can also be applied as a screen configuration when a 3D image and a wide-angle 2D image are displayed on different monitors.

  In addition to the endoscopic image display areas shown in FIGS. 6 to 8, a display area for 2D display of only the right image IR or only the left image IL as a 2D image may be provided.

  Furthermore, the image configurations shown in FIGS. 6 to 8 are merely examples, and any type of display area (3D image & wide-angle 2D image display area 150, 3D image display area 160, wide-angle 2D image display area 170, etc.) It is good also as a screen structure which has any one or several types of display area of these, and you may make it switch those screens by selection of a display mode. In addition, when a plurality of types of display areas are displayed in one screen, they are not arranged in different areas, but a so-called PinP display arrangement in which other display areas are arranged in a part of one display area. It is also good.

  The display image generation unit 104 in FIG. 3 generates a right eye display image and a left eye display image corresponding to the screen configurations illustrated in FIGS. 6 to 8. At that time, the image processing unit 102 sequentially generates an image of the display area of the endoscopic image in the display image for the right eye and an image of the display area of the endoscopic image in the display image for the left eye. It is generated using the latest image data and image data of the left image IL. With respect to the processing for generating the endoscopic image display area, a case in which the 3D image & wide-angle 2D image display area 150, the 3D image display area 160, and the wide-angle 2D image display area 170 illustrated in FIGS. This is illustrated in Further, the image other than the display area of the endoscopic image is generated by acquiring image data or character data generated by a processing unit such as a CPU (not shown), but detailed description thereof is omitted.

  In the following, the image of the display area of the endoscopic image in the display image for the right eye is referred to as the right display image of the display area, and similarly the image of the display area of the endoscopic image in the display image for the left eye is This is called the left display image of the display area.

  FIG. 9 is a diagram showing an outline of processing for generating the right display image and the left display image in the 3D image & wide-angle 2D image display area 150 in FIG. 6, and FIG. 9 (A) shows the 3D image & wide-angle 2D image. The right display image generation process in the display area 150 is shown, and FIG. 5B shows the left display image generation process in the 3D image & wide-angle 2D image display area 150.

  As shown in FIGS. 4A and 4B, the 3D image & wide-angle 2D image display area 150 includes an area 150M for displaying an image of the 3D visual field range 3DR & 3DL (see FIG. 4) in the center, and a right side of the area 150M. The region 150R displays an image in the right 2D visual field range 2DR (see FIG. 4) and the region 150L displays an image in the left 2D visual field range 2DL (see FIG. 4) on the left side of the region 150M.

  First, the right display image of the 3D image & wide-angle 2D image display area 150 will be described. As shown in FIG. 3A, the 3D view range 3DR & 3DL and the right 2D view range 2DR of the 3D image & wide-angle 2D image display area 150 are shown. In the regions 150M and 150R, the right image IR (the image of the right 3D visual field range 3DR and the image of the right 2D visual field range 2DR of the right image IR) captured by the right photographing unit 50R is fitted. At this time, using the latest image data of the right image IR sequentially generated by the image processing unit 102, image data of the images of the regions 150M and 150R of the 3D visual field range 3DR & 3DL and the right 2D visual field range 2DR is generated. At that time, thinning processing or interpolation processing is performed on the image data of the right image IR so as to match the resolution of the 3D image & wide-angle 2D image display area 150 on the screen of the 3D display device 18. .

  Further, for example, a non-pattern black image is fitted in the region 150L of the left 2D visual field range 2DL of the 3D image & wide-angle 2D image display area 150.

  As a result, a right display image 150DR as shown in FIG. 5A is generated as the right display image of the 3D image & wide-angle 2D image display area 150.

  Next, the left display image of the 3D image & wide-angle 2D image display area 150 will be described. As shown in FIG. 5B, the 3D image range 3DR & 3DL and the left 2D view range 2DL area of the 3D image & wide-angle 2D image display area 150 150M and 150L are fitted with a left image IL (an image of the left 3D visual field range 3DL and an image of the left 2D visual field range 2DL of the left image IL) captured by the left photographing unit 50L. At this time, the latest image data of the left image IL sequentially generated by the image processing unit 102 is used to generate image data of the images of the regions 150M and 150L of the 3D visual field range 3DR & 3DL and the left 2D visual field range 2DL. In addition, thinning processing or interpolation processing is performed on the image data of the left image IL so as to match the resolution of the 3D image & wide-angle 2D image display area 150 on the screen of the 3D display device 18.

  Further, for example, a non-pattern black image is fitted in the region 2OR of the right 2D visual field range 2DR of the 3D image & wide-angle 2D image display area 150.

  As a result, a left display image 150DL as shown in FIG. 5B is generated as the left display image of the 3D image & wide-angle 2D image display area 150.

  The right display image 150DR and the left display image 150DL generated as described above are displayed in the 3D image & wide-angle 2D image display area 150 on the screen of the 3D display device 18, and the right display image 150DR is displayed by the observer. By displaying the left display image 150DL so as to be visible only with the right eye and only with the left eye of the observer, the 3D visual field range 3DR & 3DL region 150M as a 3D image that can be stereoscopically viewed 3D display. In addition, an image of the region 150R in the right 2D visual field range 2DR and the region 150L in the left 2D visual field range 2DL is displayed as a 2D image with a wide field angle range.

  In the form shown in FIG. 9, the image of the region 150L of the left 2D visual field range 2DL in the right display image 150DR of the 3D image & wide-angle 2D image display area 150 is a black image, and the 3D image & wide-angle 2D image display area Although the image of the region 150R in the right 2D visual field range 2DR in the 150 left display images 150DL is a black image, the present invention is not limited to this. For example, a modification of FIG. 10 shown in the same drawing configuration as FIG. 9 may be used.

  As shown in FIG. 10A, the image of the left 2D visual field range 2DL of the left image IL is fitted in the region 150L of the left 2D visual field range 2DL in the right display image 150DR of the 3D image & wide-angle 2D image display area 150. . As a result, a right display image 150DR as shown in FIG. 5A is generated as the right display image of the 3D image & wide-angle 2D image display area 150.

  On the other hand, as shown in FIG. 10B, the right 2D visual field range 2DR image of the right image IR is present in the region 150R of the right 2D visual field range 2DR in the left display image 150DL of the 3D image & wide-angle 2D image display area 150. It is inserted. As a result, a left display image 150DL as shown in FIG. 5B is generated as the left display image of the 3D image & wide-angle 2D image display area 150.

  According to this, in both the right display image 150DR and the left display image 150DL of the 3D image & wide-angle 2D image display area 150, the imaging unit 50R is divided into the region 150R of the right 2D visual field range 2DR and the region 150L of the left 2D visual field range 2DL. Since the images captured at 50L are displayed, the images in those areas can be viewed with both eyes, and a wide-angle 2D image can be observed without a sense of incongruity.

  FIG. 11 is a diagram showing an outline of processing for generating the right display image and the left display image of the 3D image display area 160 in FIGS. 7 and 8A, and FIG. 11A shows the 3D image display area. 160 shows a right display image generation process, and FIG. 10B shows a left display image generation process of the 3D image display area 160.

  As shown in FIGS. 4A and 4B, the 3D image display area 160 is formed only by an area for displaying an image in the 3D visual field range 3DR & 3DL (see FIG. 4).

  First, the right display image in the 3D image display area 160 will be described. As shown in FIG. 3A, the right image captured by the right imaging unit 50R is displayed in the 3D image display area 160 (the 3D visual field range 3DR & 3DL region). The image of the right 3D visual field range 3DR in the right image IR of the entire visual field range VFR is fitted. At this time, image data of the image in the 3D image display area 160 is generated using the latest image data of the right image IR sequentially generated by the image processing unit 102, and at that time, the image data on the screen of the 3D display device 18 is generated. Thinning processing, interpolation processing, or the like is performed on the image data of the right image IR so as to conform to the resolution of the 3D image display area 160.

  As a result, a right display image 160DR as shown in FIG. 5A is generated as the right display image of the 3D image display area 160. That is, an image of the right 3D visual field range 3DR in the right image IR is generated as the right display image of the 3D image display area 160.

  Next, the left display image of the 3D image display area 160 will be described. As shown in FIG. 5B, the 3D image display area 160 (the 3D visual field range 3DR & 3DL region) has an entire left image captured by the left imaging unit 50L. The image of the left 3D visual field range 3DL in the left image IL of the visual field range VFL is fitted. At this time, the image data of the image in the 3D image display area 160 is generated using the latest image data of the left image IL sequentially generated by the image processing unit 102. At that time, the image data on the screen of the 3D display device 18 is generated. Thinning processing, interpolation processing, or the like is performed on the image data of the left image IL so as to conform to the resolution of the 3D image display area 160.

  As a result, a left display image 160DL as shown in FIG. 5B is generated as the left display image of the 3D image display area 160. That is, the image of the left 3D visual field range 3DL in the left image IL is generated as the left display image of the 3D image display area 160.

  The right display image 160DR and the left display image 160DL generated as described above are displayed in the 3D image display area 160 on the screen of the 3D display device 18, and the right display image 160DR is only for the right eye of the observer. By displaying the left display image 160DL so as to be visible and visible only with the left eye of the observer, the 3D image 3D image display area 160 (3D visual field range 3DR & 3DL image) can be viewed stereoscopically. 3D display as an image.

  FIG. 12 is a diagram showing an outline of processing for generating the right display image and the left display image in the wide-angle 2D image display area 170 in FIGS. 7 and 8B, and FIG. 12A shows the wide-angle 2D image. The right display image generation process of the display area 170 is shown, and FIG. 5B shows the left display image generation process of the wide-angle 2D image display area 170.

  As shown in FIGS. 9A and 9B, the wide-angle 2D image display area 170 is similar to the 3D image & wide-angle 2D image display area 150 of FIG. 9 in the 3D visual field range 3DR & 3DL (see FIG. 4) in the center. An area 170M for displaying an image, an area 170R for displaying an image in the right 2D visual field range 2DR (see FIG. 4) on the right side of the area 170M, and an image in the left 2D visual field range 2DL (see FIG. 4) on the left side of the area 170M. The display area 170L is formed.

  First, the right display image of the wide-angle 2D image display area 170 will be described. As shown in FIG. 5A, the 3D visual field range 3DR & 3DL and the right 2D visual field range 2DR areas 170M and 170R of the wide-angle 2D image display area 170 are shown in FIG. The right image IR (the image of the right 3D visual field range 3DR and the image of the right 2D visual field range 2DR of the right image IR) captured by the right photographing unit 50R is fitted. At this time, using the latest image data of the right image IR sequentially generated by the image processing unit 102, image data of the regions 170M and 170R in the 3D visual field range 3DR & 3DL and the right 2D visual field range 2DR is generated. In addition, thinning processing or interpolation processing is performed on the image data of the right image IR so as to match the resolution of the wide-angle 2D image display area 170 on the screen of the 3D display device 18.

  On the other hand, the image of the left 2D visual field range 2DL of the left image IL is fitted into the region 170L of the left 2D visual field range 2DL of the wide-angle 2D image display area 170. At this time as well, image data of the image of the region 170L of the left 2D visual field range 2DL is generated using the latest image data of the left image IL sequentially generated by the image processing unit 102. At this time, the 3D display device The image data of the left image IL is subjected to thinning processing or interpolation processing so as to match the resolution of the wide-angle 2D image display area 170 on the 18 screens.

  As a result, a right display image 170DR as shown in FIG. 5A is generated as the right display image of the wide-angle 2D image display area 170.

  The left display image in the wide-angle 2D image display area 170 is generated by a process that completely matches the above-described right display image 170DR, and the left display image 170DL is completely the same as the right display image 170DR as shown in FIG. Matched images.

  The right display image 170DR and the left display image 170DL generated as described above are displayed in the wide-angle 2D image display area 170 on the screen of the 3D display device 18, and the right display image 170DR is the right eye of the observer. When the left display image 170DL is displayed so that it can be visually recognized only by the left eye of the observer, an image that completely matches the right eye and the left eye is visually recognized. Therefore, the image in the entire range of the wide-angle 2D image display area 170 is displayed as a 2D image in the wide-angle range.

  The monitor (screen) that displays the image in the wide-angle 2D image display area 170 may be a monitor that displays a 2D image different from the monitor that displays a 3D image. There is no need to generate the right display image and the left display image (the display image for the right eye and the display image for the left eye for the entire screen) in the image display area 170, and only one of them can be generated and displayed on the 2D display device. That's fine.

  In FIG. 12, in both the right display image 170DR and the left display image 170DL, the image of the right 3D visual field range 3DR of the right image IR is compared to the 3D visual field range 3DR & 3DL region 170M of the wide-angle 2D image display area 170. Instead, an image of the left 3D visual field range 3DL of the left image IL may be inserted, or an image of the right 3D visual field range 3DR of the right image IR and an image of the left 3D visual field range 3DL of the left image IL may be used instead. You may make it insert the image produced | generated using these images.

  As described above, the case where the right image IR and the left image IL photographed by the photographing units 50R and 50L are taken into the processor device 14 and displayed on the 3D display device 18 as a 3D image or a wide-angle 2D image has been described in the above embodiment. However, the right image IR and the left image IL captured by the processor device 14, or a 3D image or a wide-angle 2D image formed by these can be stored in the storage unit.

  Next, assuming that the above embodiment is a basic embodiment, a modification for improving the quality of a 3D image and widening the angle of a 2D image in the 3D display device 18 with respect to the basic embodiment will be described. .

  The present invention is configured such that the resolution of the 3D image is higher than the resolution of the 2D image. This reduces the processing burden on 2D image data and reduces the processing burden on the system on 3D image data when compared with a configuration having the same resolution on the assumption that the processing burden on the system is not increased. As an increase, the resolution and frame rate of the 3D image can be increased, and the quality of the 3D image can be improved.

  In addition, the area of the light receiving surface of the image sensor that captures the 2D image is increased (the area of the light receiving surface that captures the 3D image is changed), assuming that the system processing burden for the 3D image data and the system processing burden for the 2D image data are not changed. 2D image can be widened.

  Further, it is possible to obtain both effects by changing the ratio of the effect of improving the quality of these 3D images and the effect of widening the angle of 2D images.

  The present invention also includes a configuration in which the resolution of the 3D image is higher than the resolution of the 2D image by an optical configuration. In this case, there is no relation to the increase or decrease in the processing load of the system, and the optical image of the area where the 3D image is captured on the imaging surface on which the optical image of the subject is formed by the photographing optical system, ie, the light receiving surface of the image sensor. Is configured to be higher than the resolution of the optical image in the area where the 2D image is captured. Accordingly, even when the resolution of the optical image varies depending on the position on the imaging surface due to distortion or aberration of the photographing optical system, the 3D image can be reliably increased in comparison with the case where the resolution is not taken into consideration. Quality can be improved.

  In particular, when a small lens, a simple configuration lens, or an inexpensive lens is used as the photographing optical system, the variation in the resolution of the optical image according to the position on the imaging surface becomes significant. It is effective to do.

  Further, even when a wide-angle lens such as a fish-eye lens is used as a photographing optical system in order to widen the 2D image, the change in the resolution of the optical image according to the position on the imaging surface becomes significant, so that the 3D image This is more effective in achieving both high quality and a wide-angle 2D image.

  The present invention is based on the premise that even if the resolution of the 2D image is lowered, the deterioration of the image quality is smaller than that of the 3D image. It is desirable to make the resolution of the 2D image lower than the resolution of the 3D image.

  First, as a modification example that expands the application range of the present invention, an embodiment (an embodiment of Modification Example 1) for making the frame rate of a 3D image higher than the frame rate of a 2D image will be described. In the first modification, the quality of the 3D image can be improved regardless of the resolution of the 3D image or the 2D image. The embodiment of the first modification and the embodiment of the present invention described below will be described. Can be applied to a stereoscopic endoscope system without interfering with each other.

  In the basic embodiment, as shown in FIG. 5 (A), the right image IR of the right full visual field range VFR composed of the right 3D visual field range 3DR and the right 2D visual field range 2DR in the right photographing unit 50R is one frame image. As shown in FIG. 5B, in the left photographing unit 50L, the left image IL of the left entire visual field range VFL composed of the left 3D visual field range 3DL and the left 2D visual field range 2DL is used as one frame image. I try to capture it.

  On the other hand, in the embodiment of the first modification, the right imaging unit 50R sequentially captures the entire image of the right full visual field range VFR and captures the image of only the right 3D visual field range 3DR at a specific ratio in order. The left photographing unit 50L also sequentially captures the entire image of the left full visual field range VFL and captures the image of the left 3D visual field range 3DL at a specific ratio.

  FIG. 13 shows the right image IR and the left image IL captured by each of the photographing units 50R and 50L in the embodiment of the first modification with respect to the photographing state of the photographing units 50R and 50L shown in FIG. FIG.

  As shown in FIG. 5A, in the right photographing unit 50R, as the right image IR, as in the basic embodiment, the right image IR1 composed of the entire image of the right full visual field range VFR and the 3D image are used. The right image IR2 made up of the image of only the right 3D visual field range 3DR is captured. The right image IR1 and the right image IR2 are captured at a specific rate. For example, the right image IR1 and the right image IR2 may be alternately captured at a ratio of 1: 1, or the right image IR1, the right image IR2, and the right image IR2 may be captured at a ratio of 1: 2. May be.

  Similarly, as shown in FIG. 5B, in the left photographing unit 50L, as the left image IL, as in the basic embodiment, the left image IL1 composed of the entire image of the left full visual field range VFL and 3D A left image IL2 consisting of an image of only the left 3D visual field range 3DL used for image display is captured. The capture of the left image IL1 and the left image IL2 is performed at the same rate as the capture of the right image IR1 and the right image IR2. Further, the capturing of the right image IR1 and the capturing of the left image IL1 are performed in synchronization, and the capturing of the right image IR2 and the capturing of the left image IL2 are performed in synchronization. However, these captures do not necessarily have to be performed synchronously, and the capture rate of the right image IR1 and the right image IR2 may not match the capture rate of the left image IL1 and the left image IL2. Good.

  Here, the capturing of the right image IR1 and the left image IL1 is performed by reading out the pixel data of the entire light receiving units of the image sensors 70R and 70L, whereas the capturing of the right image IR2 and the left image IL2 is performed by the image sensor. By reading out some pixel data of the light receiving units 70R and 70L, or after reading out the pixel data of the entire light receiving units of the image sensors 70R and 70L, a predetermined processing circuit in the photographing units 50R and 50L This is done by extracting pixel data.

  Further, depending on the relationship between the arrangement of the photographing optical systems 60R and 60L and the image sensors 70R and 70L and the visual field ranges VFR and VFL, the right image IR1 and the left image IL1 are similar to the right image IR2 and the left image IL2, respectively. There may be a case in which some pixel data of the light receiving units of the sensors 70R and 70L are read out or a part of the pixel data is extracted after reading out the pixel data of the entire light receiving units of the image sensors 70R and 70L. obtain.

  According to such capturing of the right image IR (right image IR1, right image IR2) and left image IL (left image IL1, left image IL2), the right 3D visual field range 3DR and the left 3D used for displaying the 3D image are displayed. The image of the visual field range 3DL is obtained from all of the right image IR1, the right image IR2, the left image IL, and the left image IL2 that are sequentially captured, whereas the right 2D visual field range 2DR used for displaying the 2D image and the left The image of the 2D visual field range 2DL is obtained only from the right image 1R1 and the left image 1L1. Therefore, the frame rate of the 3D image (3D video) can be made higher than the frame rate of the 2D image (2D video), and the quality of the 3D image can be improved.

  Next, in order to capture the right image IR and the left image IL as in the first modification example, the frame rate for the 3D image (3D video) is made higher than the frame rate of the 2D image (2D video) effectively. A display image generation process in the display image generation unit 104 will be described.

  First, a process for generating the right display image and the left display image of the 3D image & wide-angle 2D image display area 150 in FIG. 6 will be described. Since the right display image and the left display image are generated by the same processing, only the right display image generation processing will be described in detail here.

  The right images IR1 and IR2 shown in FIG. 13A sequentially taken in by the right photographing unit 50R as described above are transmitted to the processor device 14 and are sequentially generated and displayed as image data for display by the image processing unit 102. The image is generated by the image generation unit 104.

  When the image data of the right image 1R1 is obtained by the image processing unit 102, the display image generation unit 104 uses the image data in the basic embodiment shown in FIG. 9A or 10A. The same process as the generation process is performed. As a result, the right display image 150DR shown in FIG. 9A or 10A is generated. The right display image 150DR is the latest one.

  On the other hand, when the image data of the right image 1R2 is obtained by the image processing unit 102, only the image data of the image of the 3D visual field range 3DR & 3DL with respect to the latest right display image 150DR already generated by the display image generation unit 104, It is replaced with the image data of the right image IR2. As a result, a new right display image 150DR in which only the image of the 3D visual field range 3DR & 3DL is updated is generated, and the right display image 150DR is updated.

  Such generation processing is repeated each time the image processing unit 102 obtains image data of the right image 1R1 or the right image IR2.

  Similarly, for the left display image, when the image data of the left image IL1 is obtained, the left display image 150DL is obtained by the same processing as the generation processing in the basic embodiment shown in FIG. 9B or FIG. When the image data of the left image 1L2 is obtained, only the image of the 3D visual field range 3DR & 3DL of the latest left display image 150DL is updated.

  The right display image 150DR and the left display image 150DL that are sequentially generated in this way are displayed in the 3D image & wide-angle 2D image display area 150 on the screen of the 3D display device 18, thereby displaying the 3D image as a 3D image. The frame rate of the images in the visual field range 3DR & 3DL is substantially higher than the frame rates of the images in the right 2D visual field range 2DR and the left 2D visual field range 2DL displayed as 2D images. As a result, the quality of the 3D image is improved.

  Next, a process for generating the right display image and the left display image of the 3D image display area 160 in FIGS. 7 and 8A will be described. Since the right display image and the left display image are generated by the same processing, only the right display image generation processing will be described in detail here.

  The right images IR1 and IR2 shown in FIG. 13A sequentially taken in by the right photographing unit 50R as described above are transmitted to the processor device 14 and are sequentially generated and displayed as image data for display by the image processing unit 102. The image is generated by the image generation unit 104.

  In the display image generation unit 104, when the image data of the right image 1R1 is obtained by the image processing unit 102, the same processing as the generation processing in the basic embodiment shown in FIG. And the right display image 160DR of the 3D image display area 160 is generated.

  On the other hand, when the image data of the right image 1R2 is obtained by the image processing unit 102, the image data of the right image 1R2 is used as the image data of the right display image 160DR in the 3D image display area 160, and the right image IR2 is displayed as it is. An image 160DR is obtained.

  Such generation processing is repeated each time the image processing unit 102 obtains image data of the right image 1R1 or the right image IR2.

  Similarly, for the left display image, when the image data of the left image IL1 is obtained, the left display image 160DL is generated by the same process as the generation process in the basic embodiment shown in FIG. When 1L2 image data is obtained, the left image 1L2 becomes the left display image 160DL.

  The right display image 150DR and the left display image 150DL that are sequentially generated in this way are displayed in the 3D image display area 160 on the screen of the 3D display device 18, whereby the 3D image display area 160 displayed as a 3D image. Are updated every time the right image 1R1 and the right image IR2 are captured (each time the left image 1L1 and the left image 1L2 are captured). The display of the 3D image in the 3D image display area 160 is performed together with or switched to the display of the 2D image in the wide-angle 2D image display area 170 displayed in another area. Comparison with the frame rate will be described later.

  Next, a process for generating the right display image and the left display image of the wide-angle 2D image display area 170 in FIGS. 7 and 8B will be described. Since the right display image and the left display image are generated as the same image by the same processing, only the right display image generation processing will be described in detail here.

  The right images IR1 and IR2 shown in FIG. 13A sequentially taken in by the right photographing unit 50R as described above are transmitted to the processor device 14 and are sequentially generated and displayed as image data for display by the image processing unit 102. The image is generated by the image generation unit 104.

  In the display image generation unit 104, when the image data of the right image 1R1 is obtained by the image processing unit 102, the same processing as the generation processing in the basic embodiment shown in FIG. And the right display image 170DR of the wide-angle 2D image display area 170 is generated. The right display image 170DR is the latest. Note that the image data of the left image IL1 is also used to generate the right display image 170DR as shown in FIG.

  On the other hand, when the image data of the right image 1R2 is obtained by the image processing unit 102, only the image data of the image of the 3D visual field range 3DR & 3DL with respect to the latest right display image 170DR already generated by the display image generation unit 104, It is replaced with the image data of the right image IR2. As a result, a new right display image 170DR in which only the image of the 3D visual field range 3DR & 3DL is updated is generated, and the right display image 150DR is updated.

  Such generation processing is repeated each time the image processing unit 102 obtains image data of the right image 1R1 or the right image IR2.

  The right display image 170DR is updated only when the image data of the right image 1R1 is obtained, and the right display image 170DR is not changed when the image data of the right image IR2 is obtained. Good.

  For the left display image, a left display image 170DL having the same image as the right display image 170DR is generated by the same processing as the right display image 170DR.

  The right display image 170DR and the left display image 170DL that are sequentially generated in this way are displayed in the wide-angle 2D image display area 170 on the screen of the 3D display device 18, so that the entire range of the wide-angle 2D image display area 170 is displayed. The image is displayed as a 2D image with a wide field angle range. At this time, at least the images of the right 2D visual field range 2DR and the left 2D visual field range 2DL in the wide-angle 2D image display area 170 are updated every time the right image IR1 (left image 1L1) is captured. Therefore, the frame rate of the image in the 3D image display area 160 displayed as the 3D image is higher than the frame rate of the image in the wide-angle 2D image display area 170 displayed as a 2D image. As a result, the quality of the 3D image is improved.

  Next, an embodiment of the present invention in which the resolution of the 3D image is higher than the resolution of the 2D image (embodiments of Modification 2 and Modification 3) will be described.

  FIGS. 14A and 14B are image diagrams illustrating pixel configurations of the right image IR and the left image IL captured by the imaging units 50R and 50L in the second and third embodiments. In the second modification, the imaging units 50R and 50L, for example, continuously capture the right image IR in the right visual field range VFR and continuously capture the left image IL in the left visual field range VFL, as in the basic embodiment. Shall.

  At this time, the right image IR has a right 2D visual field range 2DR used for displaying a 2D image rather than a resolution of the image of the right 3D visual field range 3DR used for displaying the 3D image, as shown in FIG. The image is captured so that the resolution of the image is lower. Here, in FIGS. 2A and 2B, white pixels indicate pixels that are effectively captured as pixel data of the right image IR, and black pixels are effectively captured as pixel data of the right image IR. Indicates pixels that are not.

  Further, as shown in FIG. 5B, the left image IL has a left 2D visual field range 2DL used for displaying a 2D image rather than a resolution of an image of the left 3D visual field range 3DL used for displaying the 3D image. The image is captured with a lower resolution.

  Thereby, the resolution of the 3D image becomes higher than the resolution of the 2D image, and as described above, it is possible to improve the quality of the 3D image and widen the angle of the 2D image.

  FIG. 15 shows the configuration of the imaging units 50R and 50L in Modification 2 for capturing the right image IR and the left image IL as shown in FIGS. 14 (A) and 14 (B). In addition, the same code | symbol as FIG. 3 is attached | subjected to the component of the same effect | action in the imaging | photography parts 50R and 50L shown in FIG. 3, or a similar effect | action, and description is abbreviate | omitted.

  As shown in the drawing, the pixel data read out from the image sensor 70R as the imaging signal in the right photographing unit 50R is subjected to analog signal processing by the AFE 80R, and then processed as digital pixel data in the processing unit 200R (pixel density changing means). ).

  The processing unit 200R outputs the pixel data in the right 3D visual field range 3DR of the right image IR out of the sequentially input digital pixel data to the transmission unit 82R as it is, and thins out the pixel data in the right 2D visual field range 2DR. Processing is performed to output only a part of the pixel data (for example, pixel data of pixels at regular intervals among the pixels arranged in the horizontal direction) to the transmission unit 82R.

  Accordingly, the pixel density at which pixel data is effectively captured from the light receiving surface of the image sensor 70R as the pixel data of the image in the right 2D visual field range 2DR is the same as the light receiving surface of the image sensor 70R using the image in the right 3D visual field range 3DR as pixel data. The right 2D used for displaying the 2D image is lower than the density of the pixels that are effectively captured from the image, and the resolution of the image in the right 3D visual field range 3DR used for displaying the 3D image as shown in FIG. The right image IR having a lower resolution of the image in the visual field range 2DR is captured by the right photographing unit 50R.

  Similarly, the processing unit 200L outputs the pixel data in the left 3D visual field range 3DL of the left image IL among the digital pixel data sequentially input to the transmission unit 82L as it is, and outputs the pixel data in the left 2D visual field range 2DL. Then, a thinning process is performed, and only a part of the pixel data (for example, pixel data of pixels arranged at regular intervals among the pixels arranged in the horizontal direction) is output to the transmission unit 82L.

  As a result, the pixel density at which pixel data is effectively captured from the light receiving surface of the image sensor 70L as the pixel data of the image in the left 2D visual field range 2DL is the light receiving surface of the image sensor 70L using the image in the left 3D visual field range 3DL as the pixel data. The left 2D used for displaying the 2D image is lower than the density of the pixels that are effectively captured from the image, and the resolution of the image of the left 3D visual field range 3DL used for displaying the 3D image as shown in FIG. The left image IL having a lower resolution of the image in the visual field range 2DL is captured by the left photographing unit 50L.

  The processing units 200R and 200L do not reduce the number of pixels (pixel density) by thinning out the pixel data of the right 2D visual field range 2DR and the left 2D visual field range 2DL, but from a plurality of pixel data close to each other by binning processing. Pixel data for one pixel may be generated to reduce the number of pixels (pixel density), and the generated pixel data may be output to the transmission units 82R and 82L.

  Further, the processing units 200R and 200L may be provided in front of the AFEs 80R and 80L, or may be performed within the AFEs 80R and 80L.

  Further, when the image sensors 70R and 70L have a structure for outputting pixel data at the position instructed from the driving units 202R and 202L in the figure, the right 2D visual field range 2DR and It is possible to perform the thinning process by outputting only a part of the pixel data of all the pixel data in the left 2D visual field range 2DL.

  In addition, when the image sensors 70R and 70L have the function of performing the above binning process according to an instruction from the driving units 202R and 202L, the right 2D visual field is displayed in the image sensors 70R and 70L according to the instruction from the driving units 202R and 202L. A binning process may be performed on the pixel data in the range 2DR and the left 2D visual field range 2DL and output.

  FIG. 16 shows the pixel pitch (pixel density) on the light receiving surfaces of the image sensors 70R and 70L in Modification 3 for capturing the right image IR and the left image IL as shown in FIGS. 14 (A) and 14 (B).

  Pixels in a region 230R that captures an image in the right 3D visual field range 3DR of the right image IR and a region 230L that captures an image in the left 3D visual field range 3DL of the left image IL on the light receiving surfaces (light receiving units) of the image sensors 70R and 70L. The pitch is consistent. Enlarged view 234 is an enlarged view of the same area within these regions 230R and 230L. As shown in the enlarged view 234, the pixels (light receiving elements) 238, 238,... In the regions 230R, 230L are arranged, for example, at the same pixel pitch P1 in the vertical direction and the horizontal direction.

  On the other hand, on the light receiving surfaces of the image sensors 70R and 70L, the pixel pitch of the region 232R that captures an image in the right 2D visual field range 2DR of the right image IR and the region 232L that captures an image in the left 2D visual field range 2DL of the left image IL is Match. The enlarged view 236 is an enlarged view of the same area within the regions 232R and 232L. As shown in the enlarged view 236, the pixels (light receiving elements) 240, 240,... In the regions 232R, 232L are arranged, for example, at the same pixel pitch P2 in the vertical direction and the horizontal direction.

  When comparing the pixel pitch P1 in the regions 230R and 230L and the pixel pitch P2 in the regions 232R and 232L, the pixel pitch P2 is larger than the pixel pitch P1. That is, the pixel density of the regions 230R and 230L is larger than the pixel density of the regions 232R and 232L.

  As described above, as the image sensors 70R and 70L, by using an image sensor having a larger pixel pitch in the region of the light receiving surface that captures the 2D image than in the region of the light receiving surface that captures the 3D image, FIG. As shown, the right image IR is lower in the resolution of the image in the right 2D visual field range 2DR used for displaying the 2D image than the image in the right 3D visual field range 3DR used for displaying the 3D image. It is taken in by 50R. Further, as shown in FIG. 14B, the resolution of the image in the left 2D visual field range 2DL used for displaying the 2D image is lower than the resolution of the image in the left 3D visual field range 3DL used for displaying the 3D image. The left image IL is captured by the left imaging unit 50L.

  As in the basic embodiment, the right image capturing unit 50R continuously captures the right image IR of the right full visual field range VFR, and the left image capturing unit 50L continuously captures the left image IL of the left full visual field range VFL. In the case where it is carried out manually, the capturing of the right image IR and the left image IL of Modifications 2 and 3 can be applied to the capturing of the right image IR and the left image IL as they are. In that case, the right display image and the left display image of the display area of the endoscopic image are generated by the same process as the generation process according to the type of the display area described in the basic embodiment, and are displayed on the 3D display device 18. Is displayed.

  On the other hand, as in the embodiment of the first modification described with reference to FIG. 13 and the like, the right imaging unit 50R captures the right image IR1 of the right full visual field range VFR and the right image IR2 of only the right 3D visual field range 3DR. Are sequentially performed at a specific ratio, and the left imaging unit 50L sequentially captures the left image IL1 of the left entire visual field range VFL and the left image IL2 of only the left 3D visual field range 3DL at a specific ratio. The capturing of the right image IR and the left image IL according to the second and third modifications can be applied to capturing the right image IR1 and the left image 1L1.

  In that case, the right display image and the left display image of the display area of the endoscopic image are generated by the same process as the generation process according to the type of the display area described in the embodiment of the first modification, and the 3D display device 18 is displayed.

  Further, when the right image IR and the left image IL are fetched and the display image generation processing of the display area of the endoscope image is performed as in the first modification, the right image IR1 and the left image 1L1 are fetched. On the other hand, the resolution of the image used for displaying the 3D image as well as the resolution of the image used for displaying the 2D image, such as the right image IR and the left image IL of Modifications 2 and 3, that is, the right image A mode is also conceivable in which the resolution of the entire range of IR1 and left image 1L1 is lowered by thinning processing, binning processing, or the like as in the second modification.

  Next, an embodiment of Modification 4 in which the resolution of the 3D image is higher than the resolution of the 2D image by an optical configuration will be described.

FIG. 17 shows the configuration of the imaging optical systems 60R and 60L and the image sensors 70R and 70L of the imaging units 50R and 50L in the fourth modification by directly applying the various visual field ranges in the basic embodiment shown in FIG. It is a figure. Note that the figure in which the right image IR and the left image IL are overlapped in a region displayed as a 3D image at the top of the subjects O1 to O5 and the top of the figure is also exactly the same as FIG. 4, as shown in FIG. The optical axis OR and the optical axis OL of the 60R and the photographing optical system 60L intersect with each other on the reference plane RP and are arranged in a symmetrical direction. That is, photographing is performed so that the optical axis OR and the optical axis OL intersect at the center position of a range where the right 3D visual field range 3DR and the left 3D visual field range 3DL for photographing a 3D image overlap on the reference plane RP (3D visual field range 3DR & 3DL). The orientations of the optical system 60R and the photographing optical system 60L are set.

  On the other hand, the image sensor 70R and the image sensor 70L are arranged at positions where the central axes of their light receiving surfaces are shifted from the optical axis OR and the optical axis OL, respectively. The image sensor 70R is disposed such that the light receiving surface thereof is positioned to restrict the left end of the right full visual field range VFR, that is, the left end of the right 3D visual field range 3DR that captures a 3D image in a predetermined viewing direction. Similarly, the image sensor 70L is arranged such that the light receiving surface thereof is positioned to restrict the right end of the left full visual field range VFL, that is, the right end of the left 3D visual field range 3DL for capturing a 3D image in a predetermined viewing direction. Yes.

  Here, FIG. 18 and FIG. 19 are diagrams illustrating the resolution characteristics of the imaging optical systems 60R and 60L having the same characteristics. For example, when the right 2D field-of-view range 2DR and the left 2D field-of-view range 2DL in FIG. 17 are enlarged in order to capture a wide-angle 2D image, an equidistance projection method such as a fisheye lens as the imaging optical systems 60R and 60L, an equal solid angle projection It is preferable to use a wide-angle lens of a method or an orthogonal projection method. FIG. 18 shows the resolution characteristics of the photographing optical systems 60R and 60L in that case.

  As shown in the figure, the photographing optical systems 60R and 60L form, for example, vertical stripes at regular intervals on the reference plane RP as images of vertical stripes on the imaging plane IP. According to this, the distance between the vertical stripes becomes narrower as the distance from the optical axes OR and OL in the horizontal direction (the direction orthogonal to the optical axes OR and OL) increases on the imaging plane IP. Therefore, the photographing optical systems 60R and 60L have the highest resolution at the positions of the optical axes OR and OL on the imaging plane IP as shown in the lowermost graph in the drawing, and the resolution decreases as the distance from the optical axes OR and OL increases. It has the characteristic to do.

  Note that the resolution is determined including other factors such as aberration and distortion. Further, when a wide-angle lens such as an equidistant projection method such as a fisheye lens is used as the photographing optical systems 60R and 60L, the right image IR and the right image IR obtained by the photographing units 50R and 50L in the image processing unit 102 and the like of the processor device 14 are used. It is desirable to perform processing for correcting the left image IL to an image having no distortion obtained by the perspective projection method.

  On the other hand, it is also possible to use a perspective projection type wide-angle lens (or a lens with a normal angle of view) as the photographing optical systems 60R and 60L. FIG. 19 shows the resolution characteristics of the photographing optical systems 60R and 60L in that case.

  As shown in the figure, the photographing optical systems 60R and 60L form, for example, vertical stripes at regular intervals on the reference plane RP as images of vertical stripes on the imaging plane IP. According to this, when the photographing optical systems 60R and 60L are of the perspective projection method, ideally, the image formed on the image plane IP also has vertical stripes with a constant interval. Therefore, the resolution does not change depending on the projection method as shown in FIG. However, in the case of a wide-angle lens, a lens with a simple configuration, etc., aberration and distortion are large, and as shown in the lowermost graph of FIG. 18, at the positions of the optical axes OR and OL on the image plane IP as in FIG. The resolution is the highest, and the resolution decreases as the distance from the optical axes OR and OL increases.

  FIG. 17 shows a configuration suitable when the photographing optical systems 60R and 60L having the characteristics shown in FIGS. 18 and 19 are used, and the resolutions around the optical axes OR and OL of the photographing optical systems 60R and 60L. Light in the center of the 3D visual field range 3DR & 3DL on the reference plane RP so that an image of the subject in the right 3D visual field range 3DR and the left 3D visual field range 3DL taken as a 3D image is formed in the region of the imaging plane IP having a high The photographing optical systems 60R and 60L are arranged with the axes OR and OL directed. As a result, the image of the subject in the right 2D visual field range 2DR and the left 2D visual field range 2DL photographed as a 2D image is formed on the region of the image plane IP having a lower resolution.

  On the other hand, in the image sensors 70R and 70L, when the center of the light receiving surface is arranged at the position of the optical axes OR and OL, the right 2D visual field range 2DR and the left 2D visual field range 2DL are narrowed, and the angle of view of the 2D image is narrowed. Thus, the image sensor 70R is arranged so that the light receiving surface is arranged in the region of the imaging plane IP on which the subject in the right range from the left end of the right 3D visual field range 3DR taken as a 3D image is formed. It is arranged so that the center is on the left side in the figure with respect to the optical axis OR. However, it is possible to obtain the same effect as shifting the light receiving surface of the image sensor 70R by effectively capturing an image of a part of the light receiving surface as the right image IR. It is not necessary to adjust the visual field range for capturing the right image IR at the position (the same applies to the image sensor 70L).

  Similarly, the image sensor 70L is configured so that the light receiving surface is arranged in the region of the imaging plane IP where the subject in the range on the left side of the right end of the left 3D visual field range 3DL captured as a 3D image is formed. It is arranged so that the center is on the right side in the figure with respect to the optical axis OL.

  According to the configuration of the imaging units 50R and 50L configured as described above, the image of the right 3D visual field range 3DR of the right image IR and the left 3D visual field range of the left image IL that are captured as 3D images by the imaging units 50R and 50L. A 3DL image can be obtained with higher resolution than an image in the right 2D visual field range 2DR of the right image IR captured as a 2D image and an image in the left 2D visual field range 2DL of the left image IL. Therefore, the quality of the 3D image can be improved.

  In addition, since a wide-angle lens such as a fisheye lens can be used as the photographing optical systems 60R and 60L while maintaining a high quality 3D image, it is possible to widen the 2D image. In FIG. 17, the right 2D visual field range 2DR and the left 2D visual field range 2DL in which the 2D image is captured are illustrated as if they coincide with those visual field ranges illustrated in FIG. Can be.

  18 and 19, the resolution characteristics of the imaging optical systems 60R and 60L are such that the resolution is maximized at the positions of the optical axes OR and OL on the imaging plane IP. The imaging optical systems 60R and 60L are arranged so that their optical axes OR and OL intersect at the center of the visual field range (3D visual field range 3DR & 3DL) for photographing a 3D image on the reference plane RP, but the arrangement is limited to such an arrangement. It is not something.

  For example, when the imaging optical systems 60R and 60L have a mountain-shaped characteristic that maximizes the resolution at positions other than the optical axes OR and OL on the imaging plane IP, the imaging optical system 60R and 60L may have a maximum resolution on the imaging plane IP. The photographic optical systems 60R and 60L are arranged (or configured) so that an image of the 3D visual field range 3DR & 3DL is formed in the region including the position, and the resolution around the position on the imaging plane IP where the resolution is maximum. What is necessary is just to make it form the image of the subject in the field of view to be photographed as a 3D image in the region on the high imaging plane IP.

  In all the above embodiments, the photographing optical systems 60R and 60L are described as being configured by independent lenses. However, some lenses of the photographing optical systems 60R and 60L are shared as shown in FIG. The photographic optical system 300 having a configuration as described above or the photographic optical system 302 having a configuration in which all the lenses of the photographic optical systems 60R and 60L are shared as shown in FIG. In these embodiments, a configuration in which the resolution of the 3D image is higher than the resolution of the 2D image can be applied as in the third modification.

  That is, in FIG. 20, on the imaging plane IPR on which a subject image is formed as a parallax image for the right eye by the photographing optical system 300 and on the imaging plane IPL on which a subject image is formed as a parallax image for the left eye. The image of the subject in the area to be photographed as a 3D image may be formed in the area where the resolution is high. That is, when the center of the field of view taken as a 3D image is a point 3DP on the reference plane RP, the image of the point 3DP is close to the maximum point of the resolution of the mountain distribution on each of the imaging plane IPR and the imaging plane IPL. What is necessary is just to make it image-form. The same applies to the photographing optical system 302 in FIG.

  The configuration of the fourth modification embodiment can be applied to both the basic embodiment and the first to third modification embodiments.

DESCRIPTION OF SYMBOLS 10 ... Stereoscopic endoscope system, 12 ... Stereoscopic endoscope apparatus (endoscope), 14 ... Processor apparatus, 16 ... Light source, 18 ... 3D display apparatus, 20 ... Insertion part, 22 ... Operation part, 24 ... Universal cord 30 ... distal end portion, 30a ... distal end surface, 32 ... curved portion, 34 ... flexible portion, 50 ... photographing portion, 50R ... right photographing portion, 50L ... left photographing portion, 52 ... illumination portion, 60R, 60L ... photographing optical system , 70R, 70L ... image sensor, 80R, 80L ... analog signal processing unit (AFE), 82R, 82L ... transmission unit, 100 ... reception unit, 102 ... image processing unit, 104 ... display image generation unit, 106 ... display control unit 150 ... 3D image & wide-angle 2D image display area, 160 ... 3D image display area, 170 ... wide-angle 2D image display area, 200R, 200L ... processing unit, 202R, 202L ... drive unit, RP ... reference , VFR ... right full field of view range, VFL ... left full field of view range, 3DR ... right 3D field of view range, 3DL ... left 3D field of view range, 3DR & 3DL ... 3D field of view range, 2DR ... right 2D field of view range, 2DL ... left 2D field of view range

Claims (8)

A first photographing unit that photographs a subject in a first visual field range and obtains a first image for forming a stereoscopic 3D image and a planar 2D image;
A second photographing unit that photographs a subject in a second visual field range that partially overlaps the first visual field range, and obtains a second image for forming the stereoscopic 3D image and the planar 2D image. A stereoscopic endoscope device comprising:
The first photographing unit and the second photographing unit are configured to determine a resolution of an image in a 3D region forming the stereoscopic 3D image among the image regions in the first image and the second image in the plan view. The stereoscopic endoscope apparatus which makes it higher than the resolution of the image of the 2D area | region which forms only 2D image for images. The first photographing unit includes a first photographing optical system that forms an optical image of a subject on a first imaging surface,
The second imaging unit includes a second imaging optical system that forms an optical image of a subject on a second imaging surface,
The first photographing optical system and the second photographing optical system have a resolution of an optical image of a region where an image of the 3D region is picked up among regions in the first imaging surface and the second imaging surface. The three-dimensional image according to claim 1, wherein the resolution of the image of the 3D area is made higher than the resolution of the image of the 2D area by making the resolution higher than the resolution of the optical image of the area where the image of the 2D area is captured. Endoscopic device. The first photographing unit includes a first photographing optical system that forms an optical image of a subject, and a first imaging element that picks up an optical image formed by the first photographing optical system,
The second photographing unit includes a second photographing optical system that forms an optical image of a subject, and a second imaging element that picks up an optical image formed by the second photographing optical system,
The first imaging device and the second imaging device have a pixel pitch of a region for capturing an image of the 3D region, which is smaller than a pixel pitch of a region for capturing an image of the 2D region, in the region within the light receiving surface. The stereoscopic endoscope apparatus according to claim 1 or 2. The first photographing unit includes a first photographing optical system that forms an optical image of a subject, and a first imaging element that picks up an optical image formed by the first photographing optical system,
The second photographing unit includes a second photographing optical system that forms an optical image of a subject, and a second imaging element that picks up an optical image formed by the second photographing optical system,
Of the areas in the light receiving surface of the first imaging element and the second imaging element, the density of pixels that effectively acquire pixel data from the area that captures the image of the 3D area is captured, and the image of the 2D area is captured. The stereoscopic endoscope apparatus according to claim 1, further comprising: a pixel density changing unit that increases the density of pixels that effectively acquire pixel data from a region.   The first imaging optical system and the second imaging optical system include the 3D image in an area including a position where the resolution of the optical image is maximum among the areas in the first imaging plane and the second imaging plane. The stereoscopic endoscope apparatus according to claim 2, wherein an optical image in a visual field range for photographing is formed.   The stereoscopic endoscope apparatus according to claim 2 or 5, wherein the first imaging optical system and the second imaging optical system are fish-eye lenses of an equidistant projection method, an equal solid angle projection method, or an orthographic projection method.   The stereoscopic endoscope apparatus according to claim 4, wherein the pixel density changing unit reduces the density of pixels that effectively acquire pixel data from a region that captures an image of the 2D region by thinning processing.   The stereoscopic endoscope apparatus according to claim 4, wherein the pixel density changing unit reduces the density of pixels that effectively acquire pixel data from an area where an image of the 2D area is captured by binning processing.

Images