JP2012015819A - Stereoscopic image imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a monocular stereoscopic image imaging apparatus which can take a two-dimensional image in high quality when a stereoscopic image cannot be taken.SOLUTION: A stereoscopic image imaging apparatus includes an imaging device in which a first pixel belonging to a first pixel group and a second pixel belonging to a second pixel group adjacent to the first pixel constitute a pair and the first pixel and the second pixel constituting the pair are subjected to a pupil division to receive light with different incident angles from the same object; a monocular imaging lens part disposed in front of the imaging device; three-dimensional image generation means to individually perform processing on each picked-up image signal of the first pixel and the second pixel constituting the pair to generate image data for a right eye and image data for a left eye; two-dimensional image generation means to perform processing on each picked-up image signal of the first pixel and the second pixel constituting the pair by adding pixels to generate a plane image of the object; and imaging mode switching means (S37) to select three-dimensional image generation means or two-dimensional image generation means according to a focus position of the imaging lens part from a focal distance and the object.

Description

  The present invention relates to a stereoscopic image capturing apparatus that captures a stereoscopic image of a subject.

  Stereo image pickup devices capable of capturing a stereoscopic image of a subject (three-dimensional image: hereinafter also referred to as a 3D image) have begun to spread, and television devices and personal computer monitor devices that can display a stereoscopic image have become widespread. Have begun to do.

  A conventional stereoscopic image pickup apparatus is equipped with two image pickup units as described in, for example, Patent Documents 1 and 2 below, and the shooting lens systems of each image pickup unit are provided side by side on the left and right sides of the front of the camera housing. ing. Then, a subject image for the right eye is photographed through the right photographing lens system, and a subject image for the left eye is photographed through the left photographing lens system.

  A compound-eye stereoscopic image capturing apparatus having two image capturing units captures a monocular two-dimensional image (planar image: hereinafter also referred to as a 2D image) because each image capturing unit includes an expensive photographing lens system and image sensor. There is a problem that the cost of the image pickup unit is doubled as compared to the camera that performs.

  Therefore, Patent Document 3 proposes a stereoscopic imaging apparatus that has two left and right imaging lens systems, but uses only one imaging element. Patent Document 4 proposes a monocular three-dimensional image capturing apparatus in which a photographing lens system is shared and only one system is used.

  In these three-dimensional image pickup devices, right-eye image data is incident on about half of the many pixels arranged in a two-dimensional array on one image pickup device, and the left-eye is applied to the remaining half of the pixels. The pupil is divided so that image data is incident.

JP 2010-114777 A JP 2008-167066 A Japanese Patent Laid-Open No. 2003-7994 JP 10-42314 A

  When photographing a subject image, any subject image cannot always be photographed as a stereoscopic image. For example, a landscape image of a distant view at infinity cannot be taken as a stereoscopic image because the parallax between the left and right eyes cannot be taken.

  The range that can be captured as a stereoscopic image is a range in which a certain amount of parallax can be obtained between the right-eye image and the left-eye image, and even if a subject image that cannot be captured as a stereoscopic image is reproduced as a stereoscopic image, a high-quality stereoscopic image can be obtained. It does not become an image.

  An image at a boundary between whether or not a subject image of a foreground or a middle scene that can be reproduced as a stereoscopic image does not become a clear stereoscopic image even if the image is reproduced as a stereoscopic image. In this case, the image is reproduced as an image close to a planar image (2D image). However, since the display is a three-dimensional image, the entire image becomes an unclear two-dimensional image.

  A conventional monocular stereoscopic image capturing apparatus considers only a case where a stereoscopic image is captured, and does not consider how to deal with a case where a stereoscopic image cannot be captured.

  An object of the present invention is to provide a monocular stereoscopic image capturing apparatus that can capture a high-quality two-dimensional image when a stereoscopic image cannot be captured.

In the stereoscopic image capturing apparatus according to the present invention, the first pixel group and the second pixel group are formed on the same plane, and the first pixel belonging to the first pixel group and the second pixel adjacent to the first pixel are included. A second pixel belonging to a pixel group constitutes a pair, and the first pixel and the second pixel constituting the pair are pupil-divided to receive light of different incident angles from the same subject;
A system of taking lens units placed in front of the image sensor and imaging incident light from the subject on the light receiving surface of the image sensor;
The captured image signal of each of the first pixel and the second pixel constituting the pair is processed separately, and the first image for the right eye is obtained from the captured image signal of the first pixel of each of the first pixel groups. 3D image generating means for generating subject image data and generating second subject image data for the left eye from the captured image signal of the second pixel of each of the second pixel groups;
2D image generation means for adding and processing each captured image signal of the first pixel and the second pixel constituting the pair to generate a planar image of the subject as third subject image data;
Depending on the focal length of the photographic lens unit and the focus position to the subject, the 3D image generating means generates the first and second subject image data for the right eye and the left eye, or And a photographing mode switching means for causing the 2D image generating means to generate the third subject image data.

  According to the present invention, when a high-quality stereoscopic image (3D image) of a subject cannot be captured, the stereoscopic imaging apparatus side makes a determination and captures a high-quality planar image (2D image). It is possible to capture a high-quality subject image under any shooting condition.

1 is an external perspective view of a monocular stereoscopic image capturing apparatus according to an embodiment of the present invention. FIG. 2 is a functional block diagram of the stereoscopic image capturing apparatus illustrated in FIG. 1. It is a surface schematic diagram of the image sensor shown in FIG. It is a figure which shows only the color filter arrangement | sequence of the image pick-up element shown in FIG. It is a figure which shows the positional relationship of the microlens and light shielding film opening of the image pick-up element shown in FIG. It is a figure which shows the parallax between A group pixel and B group pixel which have the light shielding film opening shown in FIG. It is a figure which shows the pair pixel which performs the pixel addition for producing | generating a 2D image with the image pick-up element of FIG. It is a figure which shows the signal read order when performing pixel addition output with the image pick-up element of FIG. It is a figure which shows the signal read order when outputting the picked-up image signal of a stereo image with the image pick-up element of FIG. FIG. 10 is a diagram showing a signal reading order instead of FIG. 9. It is a figure which shows the example of arrangement | positioning of A group pixel and B group pixel in a square pixel arrangement | sequence. It is a figure which shows the example of arrangement | positioning of only A group pixel (a) in FIG. 11, and only B group pixel (b). It is a figure which shows another example of arrangement | positioning of the A group pixel and B group pixel in a square pixel arrangement | sequence. It is a figure which shows the actual three primary color filter arrangement | sequence and light shielding film opening in FIG. It is a figure which shows another example of arrangement | positioning of the A group pixel and B group pixel in a square pixel arrangement | sequence. It is a figure which shows the actual 3 primary color filter arrangement | sequence and light shielding film opening in FIG. It is a figure which illustrates the method of the pupil division by a micro lens. It is a flowchart which shows the basics of the imaging | photography process procedure in one Embodiment of this invention. FIG. 3 is a functional block diagram of a stereoscopic image capturing apparatus according to an embodiment instead of FIG. 2. It is a flowchart which shows the imaging process sequence in the stereo image imaging device of FIG. It is a flowchart which shows the detail of an imaging | photography mode determination process step among the flowcharts of FIG. It is a flowchart which concerns on another embodiment of embodiment of FIG. It is explanatory drawing of the branch process of 2D mode / 3D mode in the process step of FIG. It is a flowchart which shows the imaging | photography process procedure which concerns on another embodiment of this invention.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is an external perspective view of a monocular stereoscopic image capturing apparatus according to an embodiment of the present invention. In this stereoscopic image capturing apparatus 10, a lens barrel 13 that accommodates a monocular photographing lens system 12 is attached to the front portion of a camera housing 11 so as to be retractable. A release button 14 is provided, and a liquid crystal display unit (display unit 25 in FIG. 2) not shown in FIG. 1 is provided on the back of the camera housing 11.

  FIG. 2 is a functional block diagram of the stereoscopic image capturing apparatus 10 shown in FIG. On the back of the photographic lens system 12, an image sensor 100, which will be described in detail later, is disposed via a shutter 16 and a diaphragm (iris) (not shown). Captured image data corresponding to the subject light image formed on the light receiving surface of the image sensor 100 through the photographing lens system 12 is converted into digital data by the analog-digital (AD) converter 17 and output to the bus 18.

  The bus 18 includes a central control unit (CPU) 21 that performs overall control of the entire stereoscopic image capturing apparatus 10, an operation unit 22 including operation buttons including a shutter release button 14, a DSP 21, and the like. The image processing unit 23 that performs well-known image processing on the captured image data based on the instruction, the video encoder 24 that converts the captured image data that has been subjected to the image processing into display data, and the video encoder 24 A driver 26 for displaying captured image data on the liquid crystal display unit 25, a main memory 27, and a media control unit 28 are connected, and a recording medium (memory card) 29 is detachably attached to the media control unit 28.

  The liquid crystal display unit 25 is, for example, a scan backlight type liquid crystal display unit, and is a display unit that can display right-eye image data and left-eye image data and display a stereoscopic image of a subject.

  A device control unit 31 is connected to the CPU 21. The device control unit 31 performs drive control of the image sensor 100 according to an instruction from the CPU 21, performs opening / closing control of the shutter 16, and performs focus position control and zoom position control of the photographing lens system 12.

  FIG. 3 is a schematic surface view of the image sensor 100 shown in FIG. In the solid-state imaging device 100 of the present embodiment, a plurality of photodiodes (photoelectric conversion elements: pixels) 33 are arranged in a two-dimensional array on the surface portion of a semiconductor substrate. In the example shown in the figure, a so-called honeycomb pixel array is formed in which even-numbered pixel rows are shifted by 1/2 pixel pitch with respect to odd-numbered pixel rows, and the light-receiving surface area of each pixel is a square pixel array. Can be wider than the case.

  A vertical charge transfer path (VCCD) 34 is formed along each pixel column composed of pixels 33, and a line memory (LM) 35 is formed along the transfer direction end of each vertical charge transfer path 34. A horizontal charge transfer path (HCCD) 36 is formed in parallel with 35, and a voltage value signal corresponding to the charge amount of the transferred signal charge is output to the end of the horizontal charge transfer path 36 in the transfer direction as a captured image signal. An amplifier 37 is provided.

  The vertical charge transfer path 34 is composed of a buried channel formed in the semiconductor substrate and a series of transfer electrode films stacked on the buried channel via an insulating film formed on the surface of the semiconductor substrate.

  The line memory 35 includes a signal charge temporary storage buffer 35a for each vertical charge transfer path 34, and a signal sent from the vertical charge transfer path 34 as described in, for example, Japanese Patent Application Laid-Open No. 2006-157624. By holding the charge temporarily and controlling the timing when the signal charge is transferred to the horizontal charge transfer path 36, it has a function of performing pixel addition (mixing of signal charges) in the horizontal direction.

  “R”, “r”, “G”, “g”, “B”, and “b” shown in FIG. 3 indicate the color of the color filter (R, r = red, G, g = green, B, b = blue). . There is no distinction between R and r colors, and G, g and B, b are the same.

  In the solid-state imaging device 100 according to the present embodiment, RGB color filters are Bayer-arranged in each pixel of the odd-numbered pixel rows (hereinafter also referred to as group A pixels), and each pixel ( Hereinafter, rgb color filters are arranged in a Bayer array. FIG. 4 is a diagram illustrating only the arrangement of the color filters RGBrgb stacked on each pixel of FIG. The A group pixel and the B group pixel provided on the same plane on average are provided in a one-to-one correspondence with each other, so the A group pixel and the B group pixel have the same number of pixels.

  A readout pulse is applied to the readout electrode / transfer electrode V2 among the vertical transfer electrodes V1 to V8 in FIG. 3 to read out the signal charges of B □ G □ B □ G □ to the vertical charge transfer path 34, which is divided into two stages. When the readout pulse is applied to the readout electrode / transfer electrode V4 after the partial transfer, the signal charges of g, b, g,... Are read out at the above-mentioned position □, and the signal charges of bBgGbBgG. It will be. By transferring this to the line memory 35 and controlling the transfer timing to the horizontal charge transfer path as described above, two pixels of the same color of b + B, g + G, b + B, and g + G can be added.

  Microlenses are stacked on each pixel 33 (on the color filter) shown in FIG. 3, and a light-shielding film opening is provided above the light receiving surface of each pixel 33 (under the color filter). However, these are not shown in FIG.

  FIG. 5 is a diagram showing a positional relationship between the microlens (circular shape) 41 and the light shielding film opening for each pixel. In the A group pixel, the light shielding film opening 42a is eccentrically provided on the left side of the microlens 41 with respect to the microlens 41 (when the solid-state imaging device 100 is viewed from the subject side). Further, in the B group pixel, the light shielding film opening 42b is eccentrically provided on the right side with respect to the center of the microlens with respect to the microlens 41.

  As can be seen from FIGS. 3 and 4, the A group pixel and the B group pixel are provided by shifting the pixel pitch by 1/2 in both the vertical direction and the horizontal direction, so that the same color pixels (R and r, G and g , B and b) are arranged obliquely adjacent to each other. When two pixels of the same color that are diagonally adjacent to each other are paired pixels, as shown in FIG. 6, one light shielding film opening 42a of the paired pixels is eccentric to the left side, and the other light shielding film opening 42b is eccentric to the right side. become.

  Since one pixel and the other pixel constituting the pair pixel have light-shielding film openings that are shifted in opposite directions with respect to the center of the microlens, incident light from the same subject incident through the photographing lens 12 is incident. The corners are limited to be in opposite directions.

  As a result, when a subject image is captured using the solid-state imaging device 100, the captured image signal of the group A pixels received through the light shielding film opening 42a becomes an image obtained by viewing the subject with the right eye and is received through the light shielding film opening 42b. The captured image signal of the group B pixels is an image obtained by viewing the subject with the left eye, and parallax occurs as shown in the lower part of FIG.

  A portion of the subject that is in focus between the A group pixel and the B group pixel is in a focused state at the same position, and forms an image on each of the A group pixel and the B group pixel. In a portion of the subject that is not in focus, a blurred image is created at a position shifted left and right by the A group pixel (right eye image) and the B group pixel (left eye image).

  Since this blurred image changes as a left-right shift amount (parallax) according to the difference in the subject distance with respect to the in-focus distance, the captured image of the A group pixel and the captured image of the B group pixel are used as the left and right images. A stereoscopic image can be reproduced.

  When reproducing a stereoscopic image, the image signal processing unit 23 in FIG. 2 processes the captured image signal of each pixel of the A group pixel as a single captured image data for the right eye, stores it in the memory card 29, and displays the display unit. 25, the right-eye picked-up image data is displayed, the picked-up image signal of each pixel of the B group pixel is processed as a single left-eye picked-up image data, stored in the memory card 29, and displayed on the display unit 25 for the left eye The captured image data is displayed, and a stereoscopic image of the subject is displayed on the display unit 25.

  FIG. 7 is an explanatory diagram when a 2D image is captured by the solid-state imaging device 100. When a 2D image is captured, two pixels of the same color that form a pair of A group pixel and B group pixel are added and output. By adding the paired pixels indicated by ellipses in FIG. 7 using the line memory 35 of FIG. 3 described above, 2D captured image data can be obtained.

  The captured image signal of the A group pixel and the captured image signal of the B group pixel generated for the stereoscopic image are shifted in position between the left and right blurred images in the A group and the B group. Occurs. However, when pixel addition is performed as in the embodiment, the blurred portion is synthesized by addition and becomes a single 2D image with no distinction between the A group and B group pixels, so a high-quality 2D image can be acquired. It becomes possible.

  Further, since the solid-state imaging device 100 of the embodiment has the color filter array of FIGS. 3 and 4, the color array of the 2D captured image data obtained by pixel addition is a Bayer array, and image processing for an existing Bayer array is performed. The technology can be used, and image processing becomes easy.

  FIG. 8 is an explanatory diagram of a signal reading order when an A + B pixel addition signal (2D image signal) is output using the line memory 35. When the 2D captured image signal is pixel-added and output inside the solid-state image sensor 100, the line memory 35 is used for pixel addition of the A group pixel row and the B group pixel row that are adjacent in the vertical direction. Output for each row (A + B).

  3 performs image processing on the (A + B) pixel addition signal, generates 2D image data, displays the data on the display unit 25, and stores it in the memory card 29.

  FIG. 9 is an explanatory diagram of a signal reading order when a 3D image signal is output. Without adding the pixels, the A group pixel row, the B group pixel row, the A group pixel row,... Are alternately output, and the image processing unit 23 performs image processing on the captured image signal of the A group pixel and uses the right eye. Image data is generated and stored in the memory card 29, and image processing is performed on the captured image signal of the group B pixels to generate image data for the left eye, which is stored in the memory 29. Then, the right-eye and left-eye images are displayed on the display unit 25 as stereoscopic images.

  FIG. 10 is an explanatory diagram of a signal reading order different from that in FIG. 9 when a 3D image signal is output. In FIG. 9, the A group pixel row, the B group pixel row, the A group pixel row,... Are alternately output to generate the right eye image data and the left eye image data. The group pixels are read first for all pixel rows, and then the group B pixels are read for all pixel rows. Even in this case, image processing becomes easy as in FIG.

  The number of pixels of the captured image signal of the A group pixel output in the signal readout order of FIG. 9 or FIG. 10 is half of all the pixels, and the number of pixels of the captured image signal of the B group pixel is also half of the total number of pixels. On the other hand, the number of pixels obtained by adding two pixels for 2D image generation is also half of the total number of pixels. That is, since the 2D image of this embodiment has the same number of pixels as the pixel resolution of the 3D image, it is possible to obtain a clear 2D image without reducing the resolution.

  In the embodiment described above, an example in which two pixels are added in the solid-state imaging device 100 has been described. However, the captured image signals of all the pixels of the A group pixel are read, the captured image signals of all the pixels of the B group pixel are read, and image processing is performed. The unit 23 may add 2 pixels of the same color of the paired pixels to generate 2D image data.

  In the above-described embodiment, the solid-state imaging device 100 using a charge coupled device (CCD) as a signal readout circuit has been described. However, a solid-state imaging device using a transistor circuit such as a CMOS type as the signal readout circuit is also applicable. . When the signal readout circuit is a MOS transistor circuit, for example, when the signal readout order is as shown in FIG. 9, there is an advantage that almost no timing difference in exposure time can be eliminated between the A group pixel and the B group pixel.

  FIG. 11 is an explanatory diagram of a pixel arrangement of the A group pixel and the B group pixel. In the embodiment described with reference to FIGS. 3 and 4, the pixels in the odd-numbered rows have a pixel array (so-called honeycomb pixel array) in which the pixels in the even-numbered rows are shifted by 1/2 pixel pitch. It may be a square array.

  When the pixel array is a square array, as shown in FIG. 12A, a pixel at one checkered position is set as an A group pixel, and a pixel at the other checkered position is set at B as shown in FIG. As the group pixel, the pixel array shown in FIG. 11 can be used. In this case, the paired two pixels may be the A pixel B pixel adjacent in the horizontal direction or the A pixel B pixel adjacent in the vertical direction.

  FIG. 13 is a diagram showing a pixel arrangement of A group pixels and B group pixels according to another embodiment when the pixel arrangement is a square arrangement. In this embodiment, odd-numbered columns of pixels are group A pixels, and even-numbered columns of pixels are group B pixels.

  FIG. 14 is an explanatory diagram of a pixel in which the three primary color filters RGBrgb are mounted in the pixel array of FIG. A pixel and B pixel are adjacent in the horizontal direction, and two pixels of the same color adjacent in the horizontal direction are paired pixels. For example, the upper left G pixel in FIG. 14 (A pixel in FIG. 13) and the g pixel adjacent in the horizontal direction (B pixel in FIG. 13) are paired pixels, and the light shielding is eccentric to the A pixel in the left direction. A film opening 42a is provided, and the B pixel is provided with a light shielding film opening 42b eccentric to the right.

  FIG. 15 is an array diagram of another embodiment of a group A pixel and a group B pixel in a square array. In the present embodiment, the pixels in the odd rows are the A group pixels, and the pixels in the even rows are the B group pixels.

  FIG. 16 is an explanatory diagram of a pixel on which the three primary color filters RGBrgb are mounted in the pixel array of FIG. A pixel and B pixel are adjacent in the vertical direction, and two pixels of the same color adjacent in the vertical direction are paired pixels. For example, the upper left G pixel in FIG. 16 (A pixel in FIG. 15) and the g pixel adjacent in the vertical direction (B pixel in FIG. 15) are paired pixels, and the light shielding is eccentric to the A pixel in the left direction. A film opening 42a is provided, and the B pixel is provided with a light shielding film opening 42b eccentric to the right.

  By using the color filter array and the pair pixel array shown in FIGS. 14 and 16, the color array as a result of adding two pixels becomes a Bayer array, so that an image processing technique for an existing Bayer array can be applied, and image processing is easy. Become.

  In the embodiment described above, for example, as shown in FIG. 14, the light shielding film openings of the paired pixels are decentered in the opposite directions to the left and right, so that the paired pixels can be divided into pupils and a parallax image can be captured. However, the pupil division can be realized without decentering the light shielding film opening.

  For example, as shown in the plan view of FIG. 17A and the cross-sectional view of FIG. 17B, the opening 52a of the light shielding film 52 of the paired pixels 51a and 51b is opened over the entire light receiving surface of each of the pixels 51a and 51b. The oval (elliptical) microlens 53 is mounted on the pair of pixels 51a and 51b so that incident light having different incident angles from the same subject is incident on each of the pair of pixels 51a and 51b. do it.

  FIG. 18 is a flowchart showing the basics of the imaging processing procedure in the stereoscopic image imaging apparatus 10 shown in FIG. When imaging processing is started and imaging (exposure of each pixel) is performed, first, in step S11, it is determined whether or not the imaging is performed in the 3D mode.

  In the case of imaging in the 3D mode, the process proceeds from step S11 to step S12, and the captured image signal of each pixel of the A group pixel and the captured image signal of each pixel of the B group pixel are read out separately. Similarly, image data for the right eye is generated from the captured image signal of the A group pixel, and image data for the left eye is generated from the captured image signal of the B group pixel. In the next step S13, the image data for the right eye and the left eye are recorded in the memory card 29, and this process is terminated.

  If the result of determination in step S11 is that imaging is not in 3D mode, processing proceeds from step S11 to step S14, 2D image data is generated using a pixel addition signal of a pair pixel of the A group pixel and the B group pixel, In step S13, the 2D image data is stored in the memory card 29, and this process ends.

  The determination as to whether the image is captured in the 3D mode or the 2D mode in step S11 may be made, for example, based on the image capture mode instructed by the user of the stereoscopic image capturing apparatus 10 from the operation unit 22.

  However, whether or not the subject is within a distance where a stereoscopic image can be taken depends on the magnification of the telephoto lens at that time or the presence or absence of macro photography. It is difficult. Therefore, an algorithm that automatically determines whether or not to perform imaging in the 3D mode is installed in the stereoscopic imaging apparatus so that a non-expert can easily shoot a high-quality 3D image. When it is not possible to take a picture, a high-quality 2D image can be taken.

  FIG. 19 is a functional block diagram of a stereoscopic image capturing apparatus 20 according to another embodiment instead of FIG. Since the basic configuration is the same as that of the stereoscopic image capturing apparatus 10 shown in FIG. 2 and only the internal configuration of the photographic lens system 12 is different, the same members are denoted by the same reference numerals and the description thereof is omitted. The photographic lens system 12 of this embodiment includes a focal length variable mechanism and focal length detector 12a, and a focus position variable mechanism and focus position detector 12b.

  In the stereoscopic imaging device 20 shown in FIG. 19, based on the detection results of these variable mechanisms & detection units 12a and 12b, as shown in FIG. 20, it is automatically determined whether the imaging device side takes an image in the 3D mode or the 2D mode. To decide.

  In FIG. 20, when imaging processing is started and imaging (exposure of each pixel) is performed, first, the focal length f is detected in step S1, and the focus position x is detected in the next step S2. Then, in the next step S3, the CPU 21 determines whether the photographing mode (imaging device drive mode or image processing mode) is the 3D mode or the 2D mode, and thereafter, the same steps S11 and S12 as in FIG. , S13, S14.

  Details of the processing procedure of step S3 are shown in FIG. In this embodiment, the shooting mode is set to the 3D mode using three threshold values: a threshold value a for determining the size of the focal length f and threshold values b and c (c ≦ b) for determining the size of the focus position x. Whether the 2D mode is set is automatically determined.

  When the processing procedure proceeds to step S3, it is first determined in step S31 whether f <a. When f <a, that is, when shooting at a wider angle than the focal length threshold value a, the process proceeds to step S32, where it is determined whether x> c.

  That is, when the focus position x is farther than the threshold value c, it is determined that the subject cannot take a parallax angle as a stereoscopic image, the process proceeds to step S33, the shooting mode is set to the 2D mode, and the process proceeds to step S11.

  As a result of the determination in step S31, when f ≧ a, that is, in the case of telephoto shooting from the focal length a, the process proceeds to step S34, where it is determined whether x> b. When the focus position x is larger than the threshold value b, that is, when the focus position x is far from the threshold value b, it is difficult to capture a stereoscopic image, so the process proceeds to step S33 and the shooting mode is set to the 2D mode.

  As a result of the determination in step S34, when x ≦ b, that is, when the focus position x is closer to the threshold value b, the parallax angle for capturing a stereoscopic image can be obtained, so the process proceeds to step S35 and the shooting mode is set to the 3D mode. As shown in FIG.

  As described above, according to the embodiment, since c <b, it is easy to perform shooting in 3D mode in the case of telephoto shooting, and it is easy to perform shooting in 2D mode in the case of wide-angle shooting.

  FIG. 22 is a flowchart showing the detailed processing procedure of another embodiment of step S3 of FIG. In the present embodiment, when the process proceeds to step S3, first, the aperture value F is detected in step S36. Then, in the next step S37, it is determined how much the parallax angle θ is taken, and when the parallax angle θ is smaller than the threshold d (determination result is No), the process proceeds to step S33 to set the shooting mode to the 2D mode, When the parallax angle θ is equal to or greater than the threshold value d (determination result is Yes), the process proceeds to step S35 to set the shooting mode to the 3D mode.

  FIG. 23 is a diagram illustrating how to obtain the parallax angle θ in the determination process performed in step S37. Since the focal length is f with respect to the aperture value F, the size Y of the lens opening at the aperture position is Y = f × (1 / F).

  On the other hand, the distance (focus position) from the lens to the subject is x, and if the convergence angle (parallax angle) when the subject is viewed from the aperture position is θ, Y = x × tan θ.

When Y is eliminated from both equations, the convergence angle θ is θ = tan ⁻¹ [f × (1 / F) / x].
Represented as:

  If the parallax angle θ is smaller than the predetermined threshold d, it is determined that a stereoscopic image cannot be captured, and the 2D mode is set in step S33. If the parallax angle θ is larger than the threshold d, it is determined that a stereoscopic image can be captured. Proceeding to step S35, photographing in 3D mode is performed.

  In this embodiment, only one threshold value d for determining the magnitude of the parallax angle θ is set, but this threshold value d takes a plurality of values d1, d2,... Depending on the magnitude of the focal length f. It may be determined whether or not it is suitable for photographing a stereoscopic image according to the distance f.

  In the embodiment described above, pixel addition is performed when shooting in the 2D mode. When this pixel addition is performed, the amount of signal increases, so the sensitivity increases, which is advantageous for low-illuminance photography. Therefore, when the shooting scene is dark and noise is conspicuous in the 3D mode, the shooting mode may be automatically switched to the 2D mode. This processing procedure is shown in FIG.

  In FIG. 24, when imaging processing is started and imaging (exposure of each pixel) is performed, first, an exposure value E is detected in step S4. Then, in the next step S5, it is determined whether or not the exposure value E is smaller than the threshold value Th. If E <Th, it is determined that the noise is increased because the scene is dark, and the process proceeds to step S33, the shooting mode is set to the 2D mode, and the process proceeds to step S11.

  If the result of determination in step S5 is E ≧ Th, it is determined that there is little noise because of a bright scene, the process proceeds to step S35, the shooting mode is set to the 3D mode, and the process proceeds to step S11.

  As mentioned above, although each embodiment was described separately, it is also possible to set it as the embodiment which used several embodiment mentioned above together. For example, even if step S5 in the embodiment of FIG. 24 is performed immediately before step 35 of FIGS. 21 and 22 and it is determined that shooting is performed in 3D mode in the upstream determination step, it is automatically performed in the case of a dark scene. Then, take a picture in 2D mode.

As described above, in the stereoscopic imaging device according to the embodiment, the first pixel group and the second pixel group are formed on the same plane, and the first pixel belonging to the first pixel group and the first pixel are included in the first pixel group. The adjacent second pixels belonging to the second pixel group form a pair, and the first pixel and the second pixel forming the pair are divided into pupils to receive light of different incident angles from the same subject. An image sensor;
A system of taking lens units placed in front of the image sensor and imaging incident light from the subject on the light receiving surface of the image sensor;
The captured image signal of each of the first pixel and the second pixel constituting the pair is processed separately, and the first image for the right eye is obtained from the captured image signal of the first pixel of each of the first pixel groups. 3D image generating means for generating subject image data and generating second subject image data for the left eye from the captured image signal of the second pixel of each of the second pixel groups;
2D image generation means for adding and processing each captured image signal of the first pixel and the second pixel constituting the pair to generate a planar image of the subject as third subject image data;
Depending on the focal length of the photographic lens unit and the focus position to the subject, the 3D image generating means generates the first and second subject image data for the right eye and the left eye, or And a shooting mode switching unit that causes the 2D image generation unit to generate the third subject image data.

  In addition, the shooting mode switching unit of the stereoscopic imaging apparatus according to the embodiment causes the 2D image generation unit to generate the third subject image data when the wide-angle shooting has a focal length shorter than the threshold, and the focal length is longer than the threshold. In the telephoto shooting, the 3D image generation means generates the first and second subject image data.

  In addition, the shooting mode switching unit of the stereoscopic image capturing apparatus according to the embodiment causes the 2D image generation unit to generate the third subject image data when the focus position is more than a threshold, and when the focus position is not more than the threshold. The 3D image generation means generates the first and second subject image data.

Further, the photographing mode switching means of the stereoscopic image capturing apparatus of the embodiment has a parallax angle θ of θ = tan ⁻¹ [fx × (1 /) where F is the aperture value, x is the focus position, and f is the focal length. F) / x]
And the 3D image generation unit generates the first and second subject image data when d ≦ θ with respect to the threshold d, and the dD θ generates the third subject when the d> θ. Image data is generated.

  The stereoscopic image capturing apparatus according to the embodiment is characterized in that the threshold d for determining the parallax angle θ takes a plurality of values depending on the magnitude of the focal length f.

  In the stereoscopic image capturing apparatus according to the embodiment, even when the shooting mode switching unit determines that the 3D image generation unit generates the first and second subject image data, the exposure value is equal to or less than a predetermined value. The 2D image generation means generates the third subject image data.

  In the stereoscopic image capturing apparatus according to the embodiment, when the 2D image generating unit generates the third subject image data, the captured image signals of the first pixel and the second pixel constituting the pair are obtained. A signal obtained by adding pixels is read from the image sensor and processed.

  In the stereoscopic image capturing apparatus according to the embodiment, when the 3D image generation unit generates the first and second subject image data, the first pixel group and the second pixel group are alternately arranged for each pixel row. A signal is read out.

  In the stereoscopic image capturing apparatus according to the embodiment, when the 3D image generation unit generates the first and second subject image data, the second pixel group is read after all the signals of the first pixel group are read. The signal is read out.

  In the stereoscopic image capturing apparatus according to the embodiment, the number of pixels of the first pixel group and the second pixel group is the same.

  In the stereoscopic image capturing apparatus according to the embodiment, each pixel of the image sensor is provided with a color filter in which the color array resulting from the pixel addition is a Bayer array.

  In addition, the stereoscopic image capturing apparatus according to the embodiment is characterized in that the pixel array of the image sensor is arranged so that even-numbered pixel rows are shifted by 1/2 pixel pitch with respect to odd-numbered pixel rows.

  In the stereoscopic image capturing apparatus according to the embodiment, the pixel array of the image sensor is a square lattice array.

  In the stereoscopic image capturing apparatus according to the embodiment, a color filter is Bayer arranged in the first pixel group, and a color filter is Bayer arranged in the second pixel group.

  Further, the pupil division of the stereoscopic image capturing apparatus according to the embodiment is performed by shifting a light shielding film opening position of the first pixel and a light shielding film opening position of the second pixel to the left and right.

  In addition, the pupil division of the stereoscopic image capturing apparatus according to the embodiment includes a single microlens that is common to the first pixel and the second pixel that form the pair, and the first pixel and the second pixel that pass through the microlens. This is performed by changing the incident angle of light incident on each pixel.

  According to the embodiment described above, since the camera side automatically selects the optimum shooting mode (3D shooting mode or 2D shooting mode) according to the shooting conditions of the subject, it is better for the user to take a picture in the 3D shooting mode. Therefore, it is possible to always obtain a high-quality subject image without determining whether it is better to shoot in the 2D shooting mode.

  Since the stereoscopic imaging device according to the present invention is monocular, it can be manufactured at a low cost, and since it captures a high-quality planar image when it is inappropriate for capturing a stereoscopic image, it is an easy-to-use 3D camera, Useful as a stereo camera.

10, 20 Stereoscopic image pickup device 12 Shooting lens 14 Shutter button 21 CPU
23 Image processor 33, 51a, 51b Pixel (photodiode)
34 Vertical charge transfer path (VCCD)
35 Line memory 36 Horizontal charge transfer path (HCCD)
41, 53 Microlenses 42a, 42b Light-shielding film opening 100 Image sensor

Claims (16)

A first pixel group and a second pixel group are formed on the same plane, and a first pixel belonging to the first pixel group and a second pixel belonging to the second pixel group adjacent to the first pixel are An image sensor configured to form a pair, the first pixel and the second pixel forming the pair are pupil-divided to receive light of different incident angles from the same subject;
A system of taking lens units placed in front of the image sensor and imaging incident light from the subject on the light receiving surface of the image sensor;
The captured image signal of each of the first pixel and the second pixel constituting the pair is processed separately, and the first image for the right eye is obtained from the captured image signal of the first pixel of each of the first pixel groups. 3D image generating means for generating subject image data and generating second subject image data for the left eye from the captured image signal of the second pixel of each of the second pixel groups;
2D image generation means for adding and processing each captured image signal of the first pixel and the second pixel constituting the pair to generate a planar image of the subject as third subject image data;
Depending on the focal length of the photographic lens unit and the focus position to the subject, the 3D image generating means generates the first and second subject image data for the right eye and the left eye, or A stereoscopic image capturing apparatus comprising: a shooting mode switching unit that causes the 2D image generation unit to generate the third subject image data.   2. The stereoscopic image capturing apparatus according to claim 1, wherein the shooting mode switching unit causes the 2D image generation unit to generate the third subject image data at the time of wide-angle shooting having a focal length shorter than a threshold value, and the threshold value. A stereoscopic image capturing apparatus that causes the 3D image generation means to generate the first and second subject image data during telephoto shooting with a longer focal length.   3. The stereoscopic image capturing apparatus according to claim 1, wherein the shooting mode switching unit causes the 2D image generation unit to generate the third subject image data when a focus position is beyond a threshold, and the focus is switched. A stereoscopic image capturing apparatus that causes the 3D image generation means to generate the first and second subject image data when the position is not more than the threshold. 4. The stereoscopic image capturing apparatus according to claim 1, wherein the shooting mode switching unit has a parallax angle when an aperture value is F, a focus position is x, and a focal length is f. 5. θ is θ = tan ⁻¹ [f × (1 / F) / x]
And the 3D image generation unit generates the first and second subject image data when d ≦ θ with respect to the threshold d, and the dD θ generates the third subject when the d> θ. A stereoscopic image capturing device for generating image data.   5. The stereoscopic image capturing apparatus according to claim 4, wherein the threshold d for determining the parallax angle θ takes a plurality of values depending on the magnitude of the focal length f.   6. The stereoscopic image capturing apparatus according to claim 1, wherein the shooting mode switching unit determines that the 3D image generation unit generates the first and second subject image data. In this case, when the exposure value is equal to or smaller than the predetermined value, the stereoscopic image capturing apparatus causes the 2D image generation unit to generate the third subject image data.   7. The stereoscopic image capturing apparatus according to claim 1, wherein when the 2D image generation unit generates the third subject image data, the first image forming the pair is formed. A stereoscopic image capturing apparatus that reads and processes a signal obtained by pixel-adding each captured image signal of a pixel and the second pixel from the image sensor.   8. The stereoscopic image capturing apparatus according to claim 1, wherein when the 3D image generation unit generates the first and second subject image data, the first pixel group. And a stereoscopic image capturing apparatus that reads signals from the second pixel group alternately for each pixel row.   8. The stereoscopic image capturing apparatus according to claim 1, wherein when the 3D image generation unit generates the first and second subject image data, the first pixel group. A three-dimensional image pickup device that reads out the signals of the second pixel group after reading out all of the signals.   10. The stereoscopic image capturing apparatus according to claim 1, wherein each of the first pixel group and the second pixel group has the same number of pixels. 11.   11. The stereoscopic image capturing apparatus according to claim 1, wherein each pixel of the image sensor is provided with a color filter in which a color array resulting from the pixel addition is a Bayer array. Stereo imaging device.   12. The stereoscopic image capturing apparatus according to claim 1, wherein the pixel array of the image sensor has an even number of pixel rows by a ½ pixel pitch with respect to an odd number of pixel rows. A three-dimensional image pickup device arranged in a shifted manner.   The stereoscopic image capturing apparatus according to claim 1, wherein a pixel array of the image sensor is a square lattice array.   The stereoscopic image capturing apparatus according to claim 13, wherein a color filter is Bayer arrayed in the first pixel group, and a color filter is Bayer arrayed in the second pixel group.   15. The stereoscopic image capturing apparatus according to claim 1, wherein the pupil division is performed by changing a position of the light shielding film opening of the first pixel and a position of the light shielding film opening of the second pixel. A three-dimensional image pickup device that is moved by shifting the   15. The stereoscopic image capturing apparatus according to claim 1, wherein the pupil division includes a single microlens that is common to the first pixel and the second pixel that form the pair. A three-dimensional image pickup apparatus that is configured to make different incident angles of light incident on the first pixel and the second pixel through the microlens.

Images