WO2013047080A1

WO2013047080A1 - Three-dimensional imaging device

Info

Publication number: WO2013047080A1
Application number: PCT/JP2012/072091
Authority: WO
Inventors: 裕樹鈴木
Original assignee: 富士フイルム株式会社
Priority date: 2011-09-29
Filing date: 2012-08-31
Publication date: 2013-04-04

Abstract

Provided is an imaging device that can obtain a plurality of image data having parallax and can hold down production costs. An imaging device (10) has an imaging optical system and an imaging element (100) that receives light that has passed through this imaging lens system. The imaging optical system includes an imaging lens (1) and an aperture (2). The imaging element (100) includes pairs of a first pixel cell and a second pixel cell (10, 11) that perform photoelectric conversion of respective luminous flux that has passed through different areas of the imaging lens (1), and these pairs are arranged two-dimensionally. A digital signal processing unit (17) of the imaging device (10) processes the captured image signals from each of the pixel cells (10, 11) and generates three-dimensional image data from the captured image data obtained by processing the captured image signal of the pixel cells (10) and the captured image data obtained by processing the captured image signal of the pixel cells (11). Even with an aperture (2) of any aperture value from open to a small aperture, the center of gravity of sensitivity for the pixel cells (10, 11) and the maximum sensitivity for the pixel cells (10, 11) coincide.

Description

Stereoscopic imaging device

The present invention relates to a stereoscopic image capturing apparatus capable of capturing a stereoscopic image.

Stereo image pickup devices that can pick up a stereoscopic image of a subject have begun to become popular, and television devices and personal computer monitor devices that can display stereoscopic images in a stereoscopic manner are also becoming popular.

Some conventional stereoscopic image pickup devices are equipped with two image pickup units, and the photographing lens systems of the respective image pickup units are provided side by side on the left and right of the front part of the camera housing. This stereoscopic image capturing apparatus captures a subject image for the right eye through the right photographic lens system and images a subject image for the left eye through the left photographic lens system.

A stereoscopic image capturing apparatus having two image capturing units includes an expensive photographing lens system and image capturing element in each image capturing unit. For this reason, the cost of the imaging unit is doubled compared to a general camera that captures a two-dimensional image.

Patent Document 1 discloses a stereoscopic image capturing apparatus capable of capturing a stereoscopic image with a single photographing lens and a single image sensor. In this stereoscopic image pickup device, a liquid crystal shutter provided between the image pickup element and the photographing lens blocks the right half of the optical path of the light collected by the photographing lens and the light collected by the photographing lens. Switch between the states that block the left half of the optical path. In each state, a three-dimensional image is generated from the right image and the left image captured by the image sensor. According to this stereoscopic image capturing apparatus, since a stereoscopic image can be captured by a single photographing lens and a single image sensor, the cost can be reduced.

JP 2001-61165 A

However, the imaging device described in Patent Document 1 needs to perform imaging twice in order to obtain a stereoscopic image. For this reason, when the subject is moving, an image difference other than parallax occurs between the right-eye image and the left-eye image, and a high-quality stereoscopic image cannot be generated. Further, since it is necessary to provide an unnecessary liquid crystal shutter in a general camera, the cost increases accordingly.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a stereoscopic image capturing apparatus that can simultaneously obtain a plurality of imaging data with parallax and that can reduce manufacturing costs. To do.

A stereoscopic image capturing apparatus according to a first aspect of the present invention includes a single photographing optical system including a photographing lens and a diaphragm that adjusts the amount of transmitted light by changing an aperture, and a single light receiving light that has passed through the photographing optical system. A stereoscopic image capturing apparatus including an imaging element, the imaging element having a first pixel cell and a second pixel cell that photoelectrically convert light beams that have passed through different areas of the photographing optical system, respectively, It is configured to generate a first image obtained from the output value of the pixel cell and a second image obtained from the output value of the second pixel cell. In this stereoscopic image capturing apparatus, the amount of light shielding in the direction orthogonal to the pupil division direction is larger than the amount of light shielding in the pupil division direction.

A stereoscopic image capturing apparatus according to a second aspect of the present invention includes a single photographing optical system including a photographing lens, a diaphragm with a fixed aperture and a light amount adjusting unit, and a single imaging element that receives light passing through the photographing optical system. The imaging device includes a first pixel cell and a second pixel cell that photoelectrically convert light beams that have passed through different areas of the imaging optical system, and the first pixel cell A first image obtained from the output value and a second image obtained from the output value of the second pixel cell are configured to generate a first aperture and a first aperture that can be taken in and out of the optical path of the photographing lens. The first diaphragm includes a maximum sensitivity of each of the first pixel cell and the second pixel cell in the incident angle range in the pupil division direction of light incident on the image sensor, and the first diaphragm. Incident angle range in the pupil division direction of light incident on the image sensor via The center of gravity of the area surrounded by the sensitivity distribution waveform of each of the first pixel cell and the second pixel cell (hereinafter referred to as the sensitivity center of gravity) is combined, and the second diaphragm is inserted through the second diaphragm. The incident angles that are the centroids of sensitivity of the first pixel cell and the second pixel cell in the incident angle range in the pupil division direction of the light incident on the image sensor are the first pixel cell and the second pixel cell. Each pixel cell has an aperture that is smaller than the angle of incidence for maximum sensitivity, processes the output values of the first and second pixel cells, and processes the output values of the first pixel cell First captured image data for generating the first image obtained in this way, and second captured image data for generating the second image obtained by processing the output value of the second pixel cell A stereoscopic image generation processing unit for generating the output value of the first pixel cell and the second In the two-dimensional image generation processing unit that generates two-dimensional image data using the output value of the cell and the photographing mode for obtaining stereoscopic image data, the photographing mode for obtaining the two-dimensional image data by inserting the first aperture into the optical path Then, an image pickup control unit that inserts the second diaphragm into the optical path and causes the image pickup device to pick up an image is provided. The first invention is adopted in order not to shield the sensitivity center of gravity.

According to the present invention, it is possible to provide a stereoscopic image capturing apparatus capable of simultaneously obtaining a plurality of imaging data with parallax and suppressing manufacturing cost.

The figure which shows schematic structure of the stereo image imaging device for describing one Embodiment of this invention 2D schematic diagram showing a schematic configuration of the solid-state imaging device 100 shown in FIG. Diagram for explaining the aperture of a general configuration The figure which shows the sensitivity with respect to the incident angle of the light in the row direction X of each of the pixel cell 10 and the pixel cell 11 shown in FIG. The figure explaining the change of the incident angle range by the aperture_diaphragm | restriction 2 mounted in the digital camera shown in FIG. The figure which shows the specific structural example of the aperture_diaphragm | restriction 2 with the change of an incident angle range as shown in FIG. The figure explaining the modification of the change of the incident angle range by the aperture_diaphragm | restriction 2 mounted in the digital camera shown in FIG. The figure which shows the specific structural example of the aperture_diaphragm | restriction 2 with a change of an incident angle range as shown in FIG. The figure explaining another modification of the change of the incident angle range by the aperture_diaphragm | restriction 2 mounted in the digital camera shown in FIG. The figure which shows the specific structural example of the aperture_diaphragm | restriction 2 with a change of an incident angle range as shown in FIG. The figure which shows schematic structure of the digital camera for describing another embodiment of this invention The figure which shows the structure of the aperture_diaphragm | restriction 2 'mounted in the digital camera shown in FIG.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing a schematic configuration of a stereoscopic image capturing apparatus for explaining an embodiment of the present invention. Examples of the stereoscopic image pickup apparatus include a digital camera and a digital video camera, an electronic endoscope, an imaging module mounted on a camera-equipped mobile phone, and the like. Here, a digital camera will be described as an example.

The image pickup system of the digital camera shown in the figure has a single shooting optical system including a shooting lens 1 such as a focus lens and a zoom lens and a diaphragm 2 provided behind the shooting lens 1 and light passing through the shooting optical system. And a solid-state imaging device 100 such as a CCD image sensor or a CMOS image sensor for receiving light.

The system control unit 11 that controls the entire electric control system of the digital camera controls the flash light emitting unit 12 and the light receiving unit 13. Further, the system control unit 11 controls the lens driving unit 8 to adjust the position of the focus lens included in the photographing lens 1 or adjust the position of the zoom lens included in the photographing lens 1. Further, the system control unit 11 adjusts the exposure amount by controlling the aperture amount of the aperture 2 via the aperture drive unit 9.

Further, the system control unit 11 drives the solid-state image sensor 100 via the image sensor driving unit 10 and causes the solid-state image sensor 100 to output a subject image captured by the solid-state image sensor 100 through the imaging optical system as a captured image signal. . An instruction signal from the user is input to the system control unit 11 through the operation unit 14.

The electric control system of the digital camera further includes an analog signal processing unit 6 and an A / D conversion circuit 7. The analog signal processing unit 6 performs analog signal processing such as correlated double sampling processing connected to the output of the solid-state imaging device 100. The A / D conversion circuit converts the RGB color signal output from the analog signal processing unit 6 into a digital signal. The analog signal processing unit 6 and the A / D conversion circuit 7 are controlled by the system control unit 11.

Furthermore, the electric control system of the digital camera includes a main memory 16 and a memory control unit 15 connected to the main memory 16. Furthermore, the electric control system of the digital camera includes a digital signal processing unit 17, a compression / decompression processing unit 18, an external memory control unit 20 to which a detachable recording medium 21 is connected, and a liquid crystal mounted on the back of the camera. And a display control unit 22 to which the display unit 23 is connected. The digital signal processing unit 17 performs interpolation calculation, gamma correction calculation, RGB / YC conversion processing, and the like to generate captured image data. The compression / decompression processing unit 18 compresses the captured image data generated by the digital signal processing unit 17 into the JPEG format or decompresses the compressed image data.

The memory control unit 15, the digital signal processing unit 17, the compression / decompression processing unit 18, the external memory control unit 20, and the display control unit 22 are connected to each other by a control bus 24 and a data bus 25, and commands from the system control unit 11. Controlled by. The liquid crystal display unit 23 is configured to display two captured image data with parallax so as to be stereoscopically viewed.

FIG. 2 is a schematic plan view showing a schematic configuration of the solid-state imaging device 100 shown in FIG.

The solid-state imaging device 100 is two-dimensionally arranged in a row direction X and a column direction Y orthogonal to the row direction X (in the example of FIG. 2, a square lattice shape), and has two types of pixel cells (10, 11). The two types of pixel cells include a pixel cell 10 that detects one of a pair of light beams that have passed through different pupil regions of the photographic lens 1 and a pixel cell 11 that detects the other of the pair of light beams. The same number of these two types of pixel cells is provided. Note that the solid-state imaging device 100 is not provided with a pixel cell that can detect both a pair of light beams that have passed through the pupil region of the photographing lens 1.

In the example of FIG. 2, in the pixel cell 10, a light receiving region (a portion indicated by a white rectangle in FIG. 2) that receives light and performs photoelectric conversion is eccentric to the left with respect to the center of the pixel cell 10. It has a configuration. A region other than the light receiving region of the pixel cell 10 (a hatched region in FIG. 2) is shielded from light by a light shielding film.

Further, the pixel cell 11 has a light receiving region (a portion indicated by a white rectangle in FIG. 2) that receives light and performs photoelectric conversion with respect to the center of the pixel cell 11. The configuration is eccentric in the right direction opposite to the eccentric direction (left direction). A region other than the light receiving region of the pixel cell 11 (a hatched region in FIG. 2) is shielded from light by a light shielding film.

Each of the

pixel cells

10 and 11 is formed by, for example, decentering the center of the opening formed in the light shielding film provided above the semiconductor substrate on which the photodiode is formed with respect to the center of the pixel cell.

The

pixel cells

10 and 11 have a configuration in which the center position of the photodiode formed in the semiconductor substrate is decentered with respect to the center of the pixel cell (the photodiode is formed only on the right half or the left half of the pixel cell). There may be. The

pixel cells

10 and 11 may be configured as long as pupil division can be performed in the row direction X by the pixel cell 10 and the pixel cell 11, and a well-known configuration can be adopted.

In the pixel cell array shown in FIG. 2, rows composed of a plurality of pixel cells 10 arranged in the row direction X and rows composed of a plurality of pixel cells 11 arranged in the row direction X are alternately arranged in the column direction Y. It is an array. That is, in the row of pixel cells of the solid-state imaging device 100, the pixel cells 10 are arranged in the odd rows and the pixel cells 11 are arranged in the even rows.

In each pixel cell, one of three types of color filters is mounted in the light receiving area. In FIG. 2, a color filter that transmits red has a letter “R” in the light receiving area of the pixel cell provided in the light receiving area. Further, a color filter that transmits green has a letter “G” in the light receiving area of the pixel cell provided in the light receiving area. Further, a color filter that transmits blue is marked with a letter “B” in the light receiving area of the pixel cell provided in the light receiving area.

As shown in FIG. 2, the array of color filters mounted on the pixel cells 10 of the solid-state image sensor 100 is a Bayer array, and the array of color filters mounted on the pixel cells 11 of the solid-state image sensor 100 is also a Bayer array. It has become.

When the pixel cell 10 and the pixel cell 11 that detect light of the same color adjacent to each other in the column direction Y are a pair of pixel cells that perform pupil division, the solid-state imaging device 100 uses a pair that detects red light and green light. A pair to be detected and a pair to detect blue light are arranged in a Bayer shape as a whole.

In the digital camera shown in FIG. 1, the digital signal processing unit 17 processes captured image signals that are a set of signals output from the pixel cells 10 of the solid-state imaging device 100 to generate right-eye image data, The captured image signal that is a set of signals output from the pixel cell 11 is processed to generate image data for the left eye. Then, the digital signal processing unit 17 generates stereoscopic image data in a format that can be stereoscopically reproduced from these two image data, and records this on the recording medium 21. The system control unit 11 causes the display unit 23 to display a stereoscopic image based on the stereoscopic image data.

As described above, the digital camera shown in FIG. 1 uses the solid-state imaging device 100 in which pairs of pixel cells divided into pupils each including the pixel cell 10 and the pixel cell 11 adjacent to the pixel cell 10 are two-dimensionally arranged. Two image data (right-eye image data and left-eye image data) having parallax can be generated by a single photographing optical system and a single image sensor. Also, this digital camera can generate two pieces of image data with parallax by one shooting without providing a liquid crystal shutter, unlike the image pickup apparatus described in Patent Document 1.

As illustrated in FIG. 2, in the configuration in which pupil division is performed in the pixel cells included in the solid-state imaging device 100, it is preferable to use the diaphragm having the general configuration illustrated in FIG. 3 as the diaphragm 2 illustrated in FIG. 1. May not obtain a good parallax. Hereinafter, the reason will be described.

FIG. 3 is a diagram for explaining a general aperture stop. The aperture 200 shown in FIG. 3 forms an aperture 200a that passes all light that passes through the pupil region H of the photographing lens 1 when opened, and the aperture value from the fully open to the minimum aperture (hereinafter referred to as a small aperture). In the description, an opening 200b that passes only light that passes through the central portion of the pupil region H is formed.

FIG. 4 is a diagram showing the sensitivity of each of the pixel cell 10 and the pixel cell 11 shown in FIG. 2 to the incident angle of light in the row direction X. The horizontal axis of FIG. 4 indicates the incident angle when the incident angle when light is incident on the pixel cell perpendicularly is 0 °. This incident angle is obtained by adding the angle formed between the straight line perpendicular to the paper surface and the paper surface in FIG. 2 to the right, and the straight line perpendicular to the paper surface to the left. The angle between the straight line and the paper is negative. The vertical axis in FIG. 4 indicates the sensitivity of the

pixel cells

10 and 11 at each incident angle.

The sensitivity distribution of the pixel cell 10 has a waveform indicated by reference numeral 10a, and the sensitivity distribution of the pixel cell 11 has a waveform indicated by reference numeral 11a. The waveform indicated by reference numeral 11a and the waveform indicated by reference numeral 10a have substantially the same shape, and the incident angles at which the respective maximum sensitivities, that is, peak sensitivities (portions surrounded by circles in FIG. 4) are incident angles. They are located at substantially the same distance in the opposite directions from the 0 ° position.

In order to obtain a good parallax between the captured image signal obtained from the pixel cell 10 and the captured image signal obtained from the pixel cell 11, the pixel cell 10 has a light incident angle range determined by the F value of the diaphragm 200. The centroid of the area surrounded by the sensitivity distribution waveform 10a (11a) of (11), that is, the incident angle serving as the sensitivity centroid coincides with the incident angle serving as the peak sensitivity of the pixel cell 10 (11) shown in FIG. It is preferable.

Note that the incident angle θG, which is the sensitivity centroid of the pixel cell 10 (11), can be obtained by the following equation, for example.

In the above equation, θ represents the value of each incident angle in the incident angle range determined by the F value of the stop, and I (θ) represents the sensitivity of the pixel cell 10 (11) at the incident angle θ.

4 shows the range of the incident angle of light to the solid-state imaging device 100 in the row direction X when the diaphragm 200 illustrated in FIG. 3 is opened. Further, reference numeral 42 denotes a range of incident angles of light to the solid-state imaging device 100 in the row direction X when the diaphragm 200 is a small diaphragm (the most narrowed state). Further, reference numeral 41 denotes a range of incident angles of light to the solid-state imaging device 100 in the row direction X when the diaphragm 200 is between the full aperture and the small diaphragm.

As shown in FIG. 4, the sensitivity centroid of the

pixel cells

10 and 11 when light in the incident angle range 40 is incident is a portion surrounded by a circle (solid line and broken line) in the figure. It almost agrees with the peak sensitivity.

However, the sensitivity centroid of the

pixel cells

10 and 11 when light in the incident angle range 41 is incident is a portion surrounded by □ (solid line and broken line) in the figure, and the pixel cell when light in the incident angle range 42 is incident. The sensitivity centroids 10 and 11 are surrounded by Δ (solid line and broken line) in the figure, and the sensitivity centroids of the

pixel cells

10 and 11 become closer to each other as the aperture 200 is reduced.

That the sensitivity center of gravity is close means that the pupil division performance (parallax separation performance) in the row direction X of the pixel cell 10 and the pixel cell 11 is lowered. Therefore, when the diaphragm 200 as shown in FIG. 3 is used, the pupil division performance is reduced on the small diaphragm side, and the captured image signal obtained from the pixel cell 10 and the captured image signal obtained from the pixel cell 11 have a parallax. It becomes difficult and a three-dimensional effect is impaired.

Therefore, in the present embodiment, when the light shielding amount is increased from the fully open position to the small diaphragm as the diaphragm 2 of the digital camera shown in FIG. 1, the pupil division direction (row direction X) is larger than the light shielding amount in the pupil division direction (row direction X). A diaphragm that forms a diaphragm shape that increases the amount of light shielded in the direction orthogonal to the row direction X) is used.

That is, in the present embodiment, the

pixel cells

10 and 11 in the incident angle range of the pupil division direction (row direction X) of the light incident on the solid-state imaging device 100 at the respective aperture values from the open position to the small stop are used as the stop 2. Of each of the

pixel cells

10 and 11 in the incident angle range of the light in the pupil division direction (row direction X) of the solid-state imaging device 100 is coincident (not completely coincident, both are In other words, a device that changes the shape of an aperture through which light is incident on the solid-state imaging device 100 is used.

For example, as the diaphragm 2, one in which the incident angle range of light in the row direction X of the solid-state imaging device 100 changes as shown in FIG. 5 according to the F value is used.

The

waveforms

10a and 11a shown in FIG. 5 are the same as those shown in FIG. In FIG. 5, an incident angle range 50 indicates a range of incident angles of light to the solid-state imaging device 100 in the row direction X when the diaphragm 2 is opened. The incident angle ranges 52a and 52b indicate the range of the incident angle of light to the solid-state imaging device 100 in the row direction X when the stop 2 is a small stop. Incident angle ranges 51a and 51b indicate the range of incident angles of light to the solid-state imaging device 100 in the row direction X when the diaphragm 2 is between the full aperture and the small diaphragm.

The sensitivity centroid of the

pixel cells

10 and 11 in the incident angle range 50 shown in FIG. 5 is a portion surrounded by a circle (solid line and broken line) in the figure, and substantially coincides with the peak sensitivity of the

pixel cells

10 and 11.

Further, the sensitivity centroid of the pixel cell 10 in the incident angle range 51a shown in FIG. 5 is a portion surrounded by a circle (solid line) in the figure, and substantially coincides with the peak sensitivity of the pixel cell 10. Further, the sensitivity centroid of the pixel cell 11 in the incident angle range 51b shown in FIG. 5 is a portion surrounded by a circle (broken line) in the figure, and substantially coincides with the peak sensitivity of the pixel cell 11.

Further, the sensitivity centroid of the pixel cell 10 in the incident angle range 52a shown in FIG. 5 is a portion surrounded by a circle (solid line) in the figure, and substantially coincides with the peak sensitivity of the pixel cell 10. Further, the sensitivity centroid of the pixel cell 11 in the incident angle range 52b shown in FIG. 5 is a portion surrounded by a circle (broken line) in the figure, and substantially coincides with the peak sensitivity of the pixel cell 11.

Next, a specific configuration of the diaphragm 2 having a change in the incident angle range as shown in FIG. 5 is shown in FIG.

The diaphragm 2 shown in FIG. 6 includes a movable diaphragm member 2a and a diaphragm member 2b. FIG. 6A in FIG. 6 shows a state when the diaphragm 2 is open. As shown in FIG. 6A, the aperture 2 forms one opening 60 with respect to the pupil region H of the photographing lens 1 when opened. FIG. 6B to FIG. 6D show a state when the diaphragm is gradually narrowed from the state of FIG. 6A, and shows a state when the FIG. 6D is a small diaphragm.

As shown in FIG. 6B, the aperture 2 is in a state in which two

apertures

60a and 60b arranged in the row direction X with respect to the pupil region H are formed when the aperture 2 is reduced to some extent from the open position. Further, when the diaphragm 2 is further reduced, the

openings

60a and 60b become smaller like FIG. 6C and FIG. 6D, respectively.

The incident angle range of light incident on the solid-state imaging device 100 through the opening 60 in the case of FIG. 6A is as indicated by reference numeral 50 in FIG. In addition, the incident angle range of the light incident on the solid-state image sensor 100 through the opening 60a in the case of FIG. 6B is as indicated by the reference numeral 51a in FIG. 5, and the light incident on the solid-state image sensor 100 through the opening 60b. The incident angle range is indicated by the reference numeral 51b in FIG.

In addition, the incident angle range of the light incident on the solid-state image sensor 100 through the opening 60a in the case of FIG. 6D is as indicated by the reference numeral 52a in FIG. 5, and the light incident on the solid-state image sensor 100 through the opening 60b. The incident angle range is indicated by reference numeral 52b in FIG.

As the diaphragm 2, a diaphragm whose incident angle range corresponding to the F value changes as shown in FIG. 7 can be used.

The

waveforms

10a and 11a shown in FIG. 7 are the same as those shown in FIG. In FIG. 7, incident angle ranges 70 a and 70 b indicate ranges of incident angles of light to the solid-state imaging device 100 in the row direction X when the diaphragm 2 is opened. Incident angle ranges 72a and 72b indicate ranges of incident angles of light to the solid-state imaging device 100 in the row direction X when the diaphragm 2 is a small diaphragm. Incident angle ranges 71a and 71b indicate the range of incident angles of light to the solid-state imaging device 100 in the row direction X when the diaphragm 2 is between the open and small diaphragms.

The sensitivity centroid of the pixel cell 10 in the incident angle range 70a shown in FIG. 7, the sensitivity centroid of the pixel cell 10 in the incident angle range 71a shown in FIG. 7, and the sensitivity centroid of the pixel cell 10 in the incident angle range 72a shown in FIG. The sensitivity centroids are the portions surrounded by circles (solid lines) in the figure, and substantially coincide with the peak sensitivity of the image cell 10.

Also, the sensitivity centroid of the pixel cell 11 in the incident angle range 70b shown in FIG. 7, the sensitivity centroid of the pixel cell 11 in the incident angle range 71b shown in FIG. 7, and the pixel cell in the incident angle range 72b shown in FIG. The sensitivity centroid of 11 is a portion surrounded by a circle (broken line) in the figure, and substantially coincides with the peak sensitivity of the image cell 11.

Next, a specific configuration of the diaphragm 2 having a change in the incident angle range as shown in FIG. 7 is shown in FIG.

The diaphragm 2 shown in FIG. 8 includes a movable diaphragm member 2a and a diaphragm member 2b. FIG. 8A in FIG. 8 shows a state when the diaphragm 2 is open. As shown in FIG. 8A, the aperture 2 forms two

openings

80a and 80b with respect to the pupil region H of the photographing lens 1 when opened. 8 show the state when the diaphragm 2 is gradually reduced from the state of FIG. 8A, and shows the state when the FIG. 8D is a small stop.

As shown in FIG. 8B, when the diaphragm 2 is narrowed to a certain extent from the open state, the size of each of the

openings

80a and 80b becomes small, and the distance in the row direction X between the

openings

80a and 80b becomes large. Further, when the diaphragm 2 is further reduced, the

openings

80a and 80b become smaller as in FIG. 8C and FIG. 8D, and the distance in the row direction X between the two openings becomes larger.

In the state of FIG. 8A, the incident angle range of light incident on the solid-state image sensor 100 through the opening 80a is as indicated by the reference numeral 70a in FIG. 7, and the incident light incident on the solid-state image sensor 100 through the opening 80b. The angular range is indicated by reference numeral 70b in FIG.

In the state of FIG. 8C, the incident angle range of the light incident on the solid-state image sensor 100 through the opening 80a is as indicated by reference numeral 71a in FIG. 7, and the incident light incident on the solid-state image sensor 100 through the opening 80b. The angular range is indicated by reference numeral 71b in FIG.

In the state of FIG. 8D, the incident angle range of the light incident on the solid-state image sensor 100 through the opening 80a is as indicated by the reference numeral 72a in FIG. 7, and the incident light incident on the solid-state image sensor 100 through the opening 80b. The angular range is indicated by reference numeral 72b in FIG.

Further, as the aperture stop 2, a diaphragm whose incident angle range corresponding to the F value changes as shown in FIG. 9 can be used.

The

respective waveforms

10a and 11a shown in FIG. 9 are the same as those shown in FIG. In FIG. 9, an incident angle range 90 indicates a range of incident angles of light to the solid-state imaging device 100 in the row direction X when the diaphragm 2 is opened. An incident angle range 92 indicates a range of incident angles of light to the solid-state imaging device 100 in the row direction X when the diaphragm 2 is a small diaphragm. An incident angle range 91 indicates a range of incident angles of light to the solid-state imaging device 100 in the row direction X when the diaphragm 2 is between the full aperture and the small diaphragm.

The sensitivity centroid of the pixel cell 10 in each of the incident angle ranges 90, 91, and 92 shown in FIG. 9 is a portion surrounded by a circle (solid line) in the figure, and substantially coincides with the peak sensitivity of the pixel cell 10. In addition, the sensitivity centroid of the pixel cell 11 in each of the incident angle ranges 90, 91, and 92 shown in FIG. 9 is a portion surrounded by a circle (broken line) in the figure, and almost coincides with the peak sensitivity of the pixel cell 11. To do.

Next, a specific configuration of the diaphragm 2 having a change in the incident angle range as shown in FIG. 9 is shown in FIG.

The diaphragm 2 shown in FIG. 10 is constituted by a movable diaphragm member 2a and a diaphragm member 2b. FIG. 10A in FIG. 10 shows a state when the diaphragm 2 is open. As shown in FIG. 10A, the aperture 2 forms one opening 101 larger than the pupil region H with respect to the pupil region H of the photographing lens 1 when opened. FIG. 10B and FIG. 10C show a state when the diaphragm 2 is gradually reduced from the state of FIG. 10A, and show a state when the FIG.

As shown in FIG. 10B, when the diaphragm 2 is squeezed to some extent from the open position, the length of the opening 101 in the column direction Y contracts, and the opening 101 becomes a shape elongated in the row direction X. That is, the diaphragm 2 forms an opening 101 having a longitudinal shape that crosses the pupil region H in the row direction X.

Further, when the diaphragm 2 is further reduced from the state of FIG. 10B, the length in the column direction Y of the openings 101 is further reduced. In any F value, the diaphragm 2 forms at least an opening in a portion overlapping the pupil region H in a line segment that bisects the pupil region H in the column direction Y.

In the state of FIG. 10A, the incident angle range of light incident on the solid-state imaging device 100 via the opening 101 is as indicated by reference numeral 90 in FIG. In the state of FIG. 10B, the incident angle range of light incident on the solid-state imaging device 100 through the opening 101 is as indicated by reference numeral 91 in FIG. The incident angle range of light incident on the solid-state imaging device 100 through the opening 101 in the state of FIG. 10C is as indicated by reference numeral 92 in FIG.

As described above, by using the diaphragm 2 that changes the aperture as shown in FIGS. 5, 7, and 9, the sensitivity centroids and the peak sensitivities of the

pixel cells

10 and 11 substantially match at any F value. Regardless of the value, good parallax can be obtained. In particular, according to the configuration of the diaphragm shown in FIG. 10, all the light from the end to the end in the row direction X of the subject to be photographed can be incident on the solid-state imaging device 100. Therefore, it is possible to generate a stereoscopic image that can provide a good stereoscopic effect even in a special shooting scene in which the main subject spreads in the horizontal direction, such as a group photo.

FIG. 11 is a diagram showing a schematic configuration of a digital camera for explaining another embodiment of the present invention. The digital camera shown in FIG. 11 is obtained by changing the aperture 2 of the digital camera shown in FIG. 1 to an aperture 2 ′ and adding an ND filter 3 and an ND drive unit 19.

The aperture 2 ′ includes a first aperture for a two-dimensional imaging mode that can be inserted into and removed from the optical path of the imaging lens 1 and a second aperture for a stereoscopic imaging mode.

The two-dimensional imaging mode is a mode in which the digital signal processing unit 17 generates and records one captured image data (two-dimensional image data) using each captured image signal of the pixel cell 10 and the pixel cell 11. . In the two-dimensional imaging mode, the system control unit 11 that also functions as an imaging control unit retracts the second aperture from the optical path of the imaging lens 1 and inserts the first aperture in the optical path, and then the solid-state imaging device Control is performed to capture an image by 100. Further, in the stereoscopic shooting mode, the system control unit 11 causes the solid-state imaging device 100 to perform imaging with the first diaphragm retracted from the optical path of the photographing lens 1 and the second diaphragm inserted in the optical path. Take control.

The first diaphragm included in the diaphragm 2 ′ has an incident angle (absolute as a sensitivity centroid) of each of the

pixel cells

10 and 11 in the incident angle range in the pupil division direction of light incident on the solid-state imaging device 100 through the diaphragm. (Value) has an opening whose value is smaller than the incident angle (absolute value) that is the peak sensitivity of each of the

pixel cells

10 and 11 in the pupil division direction of the solid-state imaging device 100.

FIG. 12A shows a configuration example of the first aperture in FIG. An aperture 200 shown in FIG. 12A has an opening 200 a formed at the center of the pupil region H of the photographic lens 1. The incident angle range of light incident on the solid-state imaging device 100 through the opening 200a is the same as the incident angle range 42 shown in FIG. 4, for example. That is, when imaging is performed by the solid-state imaging device 100 through the diaphragm 200, the incident angle serving as the sensitivity centroid of each of the

pixel cells

10 and 11 approaches 0 °, and the pupil division performance is degraded.

Therefore, parallax hardly occurs between the captured image signal obtained from the pixel cell 10 and the captured image signal obtained from the pixel cell 11. Therefore, the digital signal processing unit 17 that also functions as a two-dimensional image generation processing unit generates one captured image data by using these two captured image signals (for example, an output signal between pixel cells constituting a pair). The captured image signal after the addition is processed to generate one captured image data), and natural captured image data without blur can be generated.

The second diaphragm included in the diaphragm 2 ′ includes the sensitivity centroid of each of the

pixel cells

10 and 11 in the incident angle range in the pupil division direction of light incident on the solid-state image sensor 100 through the diaphragm, and the solid-state image sensor 100. Of the

pixel cells

10 and 11 in the pupil division direction are provided with apertures that substantially coincide with each other.

A configuration example of the second aperture is shown in FIG. 12B and FIG. 12C in FIG. The diaphragm 201 shown in FIG. 12B is formed so that the two

openings

201a and 201b arranged in the row direction X overlap with the pupil region H of the photographing lens 1. Further, the stop 202 shown in FIG. 12C is formed so that the opening 202a long in the row direction X overlaps the pupil region H of the photographing lens 1.

The incident angle range of light incident on the solid-state imaging device 100 through the aperture 201a of the diaphragm 201 is the same as the incident angle ranges 52a and 52b shown in FIG. Further, the incident angle range of light incident on the solid-state imaging device 100 through the opening 202a of the diaphragm 202 is the same as the incident angle range 90 to 92 shown in FIG. 9, for example.

When imaging is performed by the solid-state imaging device 100 through the diaphragm 201 or the diaphragm 202, the sensitivity centroids and the peak sensitivities of the

pixel cells

10 and 11 substantially coincide with each other, and the pupil division performance is optimized. Therefore, a good parallax occurs between the captured image signal obtained from the pixel cell 10 and the captured image signal obtained from the pixel cell 11. Accordingly, the digital signal processing unit 17 that also functions as a stereoscopic image generation processing unit generates stereoscopic image data having a favorable stereoscopic effect by generating one stereoscopic image data using the two captured image signals. be able to.

In the digital camera shown in FIG. 11, since the aperture area of the diaphragm 2 ′ is fixed in each photographing mode, the amount of light incident on the solid-state imaging device 100 cannot be adjusted with the diaphragm 2 ′. Therefore, an ND filter 3 as a light amount adjustment unit is provided between the diaphragm 2 ′ and the solid-state image sensor 100.

The ND filter 3 is composed of, for example, an electrochromic element that can electrically control light transmittance. The system control unit 11 issues a command to the ND filter driving unit 19, and the ND filter driving unit 19 adjusts the light transmittance of the ND filter 3 in accordance with this command. As described above, the digital camera shown in FIG. 11 adjusts the amount of light incident on the solid-state imaging device 100 by the ND filter 3.

As described above, according to the digital camera shown in FIG. 11, the solid-state imaging device 100 having the configuration shown in FIG. 2 is mounted, and the shape of the diaphragm 2 ′ is changed according to the shooting mode. It is possible to switch between shooting and two-dimensional shooting. The aperture and the ND filter are mounted on a general digital camera. For this reason, according to the digital camera shown in FIG. 11, it is possible to realize a function of performing both stereoscopic photography and two-dimensional photography at a low cost.

Note that the arrangement of the pixel cells of the solid-state imaging device 100 mounted on the digital camera shown in FIGS. 1 and 11 is not limited to that shown in FIG. For example, in FIG. 2, the even-numbered rows in which the pixel cells 11 are arranged may be a so-called honeycomb arrangement in which the odd-numbered rows in which the pixel cells 10 are arranged are shifted in the row direction X by 1/2 of the pixel cell arrangement pitch of each row. .

Further, each pixel cell of the solid-state imaging device 100 may be a solid-state imaging device for monochrome imaging without mounting a color filter.

Further, the positions of the

diaphragms

2 and 2 ′ may be in front of the photographing lens 1.

As described above, the following items are disclosed in this specification.

The disclosed stereoscopic image capturing apparatus includes a single photographing optical system including a photographing lens and a diaphragm that adjusts the amount of transmitted light by changing an aperture amount, and a single photographing that receives light passing through the photographing optical system. The image pickup device includes a first pixel cell and a second pixel cell that photoelectrically convert light beams that have passed through different areas of the photographing optical system, and the first pixel. It is configured to generate a first image obtained from the output value of the cell and a second image obtained from the output value of the second pixel cell. Is characterized in that the light shielding amount in the direction orthogonal to the pupil division direction is larger than the light shielding amount in the pupil division direction.

In the disclosed stereoscopic image capturing apparatus, the aperture forms one opening with respect to the pupil region of the photographing lens when opened, and the aperture value up to the minimum aperture is in the pupil division direction with respect to the pupil region. One longitudinally shaped opening is formed.

In the disclosed stereoscopic image capturing apparatus, the aperture forms two apertures arranged in the pupil division direction with respect to the pupil region of the photographing lens at any aperture value from the maximum aperture to the minimum aperture.

In the disclosed stereoscopic image capturing apparatus, the aperture forms one opening with respect to the pupil region of the photographing lens when opened, and the aperture value up to the minimum aperture is in the pupil division direction with respect to the pupil region. Two side-by-side openings are formed.

In the disclosed stereoscopic imaging device, a color filter is mounted on the first pixel cell arranged in two dimensions, and a color filter of the same color is mounted on the second pixel cell adjacent to the first pixel cell. Is done.

In the disclosed stereoscopic image pickup device, a color filter is mounted in a Bayer array in a first pixel cell arranged in two dimensions, and a color filter is arranged in a Bayer array in a second pixel cell arranged in two dimensions. Installed.

In the disclosed stereoscopic image capturing apparatus, pupil division is performed by shifting the position of the light shielding film opening of the first pixel cell and the position of the light shielding film opening of the second pixel cell in the opposite directions.

The disclosed stereoscopic image capturing apparatus includes a single photographing optical system including a photographing lens, a diaphragm having a fixed aperture amount, and a light amount adjusting unit, and a single imaging element that receives light passing through the photographing optical system. The imaging device includes a first pixel cell and a second pixel cell that photoelectrically convert light beams that have passed through different regions of the imaging optical system, and the first pixel cell A first image obtained from the output value and a second image obtained from the output value of the second pixel cell are configured to generate a first aperture and a first aperture that can be taken in and out of the optical path of the photographing lens. The first diaphragm includes a maximum sensitivity of each of the first pixel cell and the second pixel cell in the incident angle range in the pupil division direction of light incident on the image sensor, and the first diaphragm. Incident angle range in the pupil division direction of light incident on the image sensor via The center of the area surrounded by the sensitivity distribution waveform of each of the first pixel cell and the second pixel cell in the first pixel cell and the second diaphragm is a pupil of light incident on the image sensor through the second diaphragm The size of the incident angle that is the center of gravity of the area surrounded by the sensitivity distribution waveforms of the first pixel cell and the second pixel cell in the incident angle range in the division direction is the first pixel cell and the second pixel. Each cell has an aperture smaller than the angle of incidence for maximum sensitivity, processes the output values of the first and second pixel cells, and processes the output values of the first pixel cell. First captured image data for generating the first image obtained in this way and second captured image data for generating the second image obtained by processing the output value of the second pixel cell Stereo image generation processing unit to be generated, output value of first pixel cell and second pixel In the 2D image generation processing unit for generating 2D image data using the output value of the image and in the shooting mode for obtaining stereoscopic image data, in the shooting mode for obtaining 2D image data by inserting the first aperture in the optical path And an imaging control unit that inserts a second diaphragm into the optical path and causes the imaging device to perform imaging.

DESCRIPTION OF SYMBOLS 100 Solid-state image sensor 1 Shooting lens 2 Diaphragm 10a Sensitivity distribution waveform 11a with respect to the incident angle of the pixel cell 10 Sensitivity distribution waveform 50 with respect to the incident angle of the pixel cell 11 Incident angle range 51 when the aperture 2 is open 51 Incident angle range 52 when the aperture is 2 Incident angle range when the aperture 2 is a small aperture

Claims

A single photographic optical system including a photographic lens and an aperture that adjusts the amount of transmitted light by changing the aperture;
A single image pickup device that receives light passing through the photographing optical system, and a stereoscopic image pickup device comprising:
The imaging element includes a first pixel cell and a second pixel cell that photoelectrically convert light beams that have passed through different regions of the photographing optical system, and is obtained from an output value of the first pixel cell. And a second image obtained from the output value of the second pixel cell,
3. The stereoscopic image pickup apparatus according to claim 1, wherein when the diaphragm increases the light shielding amount from the open position, the light shielding amount in the direction orthogonal to the pupil division direction is larger than the light shielding amount in the pupil division direction.
The stereoscopic image capturing apparatus according to claim 1,
The aperture forms one aperture with respect to the pupil region of the photographing lens when opened, and the aperture value up to the minimum aperture is 1 with a shape that is long in the pupil division direction with respect to the pupil region. Stereoscopic imaging device that forms two openings.
The stereoscopic image capturing apparatus according to claim 1,
The three-dimensional image pickup apparatus in which the aperture forms two apertures arranged in the pupil division direction with respect to the pupil region of the photographing lens at any aperture value from the maximum aperture to the minimum aperture.
The stereoscopic image capturing apparatus according to claim 1,
The aperture forms one aperture with respect to the pupil region of the photographic lens when opened, and two apertures arranged in the pupil division direction with respect to the pupil region at the aperture value up to the minimum aperture. A stereoscopic image pickup device to be formed.
The stereoscopic image capturing apparatus according to any one of claims 1 to 4,
A color filter is mounted on the first pixel cell arranged two-dimensionally,
A stereoscopic image pickup device in which a color filter of the same color is mounted on the second pixel cell adjacent to the first pixel cell.
A stereoscopic image capturing apparatus according to any one of claims 1 to 5,
A stereoscopic image in which a color filter is mounted in a Bayer array in the first pixel cell arranged in the two-dimensional array, and a color filter is mounted in a Bayer array in the second pixel cell arranged in the two-dimensional array. Imaging device.
The stereoscopic image capturing apparatus according to any one of claims 1 to 6,
The pupil division is a stereoscopic image pickup device that is performed by shifting a light shielding film opening position of the first pixel cell and a light shielding film opening position of the second pixel cell in the opposite directions.
A single photographing optical system including a photographing lens, a diaphragm with a fixed aperture, and a light amount adjustment unit;
A single image pickup device that receives light passing through the photographing optical system, and a stereoscopic image pickup device comprising:
The imaging element includes a first pixel cell and a second pixel cell that photoelectrically convert light beams that have passed through different regions of the photographing optical system, and is obtained from an output value of the first pixel cell. And a second image obtained from the output value of the second pixel cell,
The aperture includes a first aperture and a second aperture that can be taken in and out of the optical path of the photographic lens,
The first diaphragm has a maximum sensitivity of each of the first pixel cell and the second pixel cell in an incident angle range in a pupil division direction of light incident on the image sensor, and the first diaphragm. The center of gravity of the area surrounded by the respective sensitivity distribution waveforms of the first pixel cell and the second pixel cell in the incident angle range in the pupil division direction of the light incident on the image sensor
The second diaphragm is a sensitivity distribution of each of the first pixel cell and the second pixel cell in an incident angle range in the pupil division direction of light incident on the image sensor through the second diaphragm. An opening having a size of an incident angle that is a center of gravity of an area surrounded by a waveform that is smaller than an incident angle that is a maximum sensitivity of each of the first pixel cell and the second pixel cell;
A first for processing the output value of each of the first pixel cell and the second pixel cell and generating the first image obtained by processing the output value of the first pixel cell A stereoscopic image generation processing unit that generates second captured image data for generating the second image obtained by processing the captured image data and the output value of the second pixel cell;
A two-dimensional image generation processing unit that generates two-dimensional image data using the output value of the first pixel cell and the output value of the second pixel cell;
In the imaging mode for obtaining stereoscopic image data, the first diaphragm is inserted into the optical path, and in the imaging mode for obtaining the two-dimensional image data, the second diaphragm is inserted into the optical path, and imaging is performed by the imaging device. A stereoscopic image capturing apparatus comprising: an imaging control unit to be performed.