WO2012169301A1

WO2012169301A1 - Image pickup device imaging three-dimensional moving image and two-dimensional moving image, and image pickup apparatus mounting image pickup device

Info

Publication number: WO2012169301A1
Application number: PCT/JP2012/061675
Authority: WO
Inventors: 智行河合; 宗之大島; 太田　毅; 遠藤　宏; 井上　知己
Original assignee: 富士フイルム株式会社
Priority date: 2011-06-06
Filing date: 2012-05-07
Publication date: 2012-12-13
Also published as: JP2014158062A

Abstract

An image pickup device includes first group pixels and second group pixels which are arranged horizontally and vertically so as to be displaced by 1/2 pixel pitch, and is provided with color filters (RGB) stacked on the first group pixels and color filters (rgb) stacked on the second group pixels. Each pixel of the first group pixels, and the pixel belonging to the second group pixels, which is obliquely adjacent to the each pixel and is stacked with the color filter, the color of which is the same as the each pixel, are defined as paired pixels, and the paired pixels horizontally and vertically corresponding at the interval of (2n + 1) (n = 1, 2, …) among the paired pixels are configured by phase difference detection pixels. The paired pixels having light-shielding film openings (2a, 2b) are the phase difference detection pixel pairs.

Description

Image pickup device for picking up three-dimensional moving image and plane moving image, and image pickup apparatus equipped with the image pickup device

The present invention relates to an imaging device having a structure suitable for capturing a stereoscopic moving image and a planar moving image, and an imaging device mounted on the imaging device.

As a conventional imaging device that captures a stereoscopic image (hereinafter also referred to as a 3D image), for example, there is a device described in Patent Document 1 below. This conventional imaging apparatus includes two cameras, and can capture the same subject with left and right cameras and reproduce a stereoscopic image of the subject. However, since the conventional stereoscopic image capturing apparatus needs to prepare two cameras, there is a problem that the cost is increased and the apparatus is further increased in size.

Therefore, a conventional technique described in Patent Document 2 below has been proposed as an imaging apparatus that can capture a stereoscopic image of a subject with a single camera. This stereoscopic image pickup apparatus divides a plurality of pixels (photoelectric conversion elements: photodiodes) two-dimensionally arranged on the surface of a solid-state image pickup device into two groups, and light from an object viewed from the right side enters one group. The other group is made to receive light viewed from the left side of the subject.

According to this prior art, a single camera (imaging device) can capture a stereoscopic image of a subject, but it cannot capture a two-dimensional image (planar image, hereinafter also referred to as a 2D image) of the subject. There is.

There have been many demands for imaging devices in recent years, and both 3D moving images and 2D moving images that are difficult to realize with the prior art described in Patent Documents 1 and 2 are not limited to low cost and downsizing. A video shooting function that can be used is required.

Japanese Unexamined Patent Publication No. 11-341522 Japanese Unexamined Patent Publication No. 2003-7994

An object of the present invention is to provide an image pickup device that can suitably process both 3D moving image shooting and 2D moving image shooting, and an image pickup apparatus mounted on the image pickup device.

The imaging device according to the present invention includes a first group pixel that includes a plurality of pixels arranged in a two-dimensional array and is configured by one pixel row of odd or even rows and the other pixel row of odd or even rows. The second group pixels to be arranged are shifted by a ½ pixel pitch in the horizontal direction and the vertical direction, the Bayer array color filter stacked on the first group pixels, and the second group pixels stacked on the second group pixels. An image pickup device including a color filter having a Bayer array, wherein each of the pixels of the first group pixel is obliquely adjacent to the pixel and the color filter of the same color as the pixel is stacked. A pixel belonging to a pixel is a pair pixel, and among the pair of pixels, a pair pixel corresponding to an interval of 2n + 1 (n = 1, 2,...) In the horizontal direction and the vertical direction is configured by a phase difference detection pixel. To do.

The imaging apparatus according to the present invention alternately captures an image signal of the phase difference detection pixel from the pixel row of the first group pixel and the pixel row of the second group pixel at the time of capturing a stereoscopic moving image. And an image sensor driving unit for reading.

In the imaging apparatus of the present invention, when imaging a planar moving image, the imaging element driving unit outputs the captured image signals of the paired pixels other than the phase difference detection pixels to the pixel rows of the first group pixels and the second group pixels. Are alternately read from the pixel rows.

According to the present invention, it is possible to capture both a high-quality stereoscopic moving image and a planar moving image with a single image sensor.

1 is an external perspective view of an imaging apparatus (digital camera) according to an embodiment of the present invention. It is a functional block block diagram of the imaging device shown in FIG. It is a surface schematic diagram of the solid-state image sensor shown in FIG. It is explanatory drawing when a 3D moving image is imaged with the solid-state image sensor shown in FIG. It is principle explanatory drawing which can image | photograph 3D image with a phase difference detection pixel. It is explanatory drawing which reads 2D moving image data with the solid-state image sensor of FIG. It is a flowchart which shows the process sequence when producing | generating 2D moving image data. It is a figure which shows the addition pixel position when producing | generating a highly sensitive 2D moving image by 4 pixel addition. It is a figure which shows the addition pixel position when producing | generating a highly sensitive 2D moving image by 8 pixel addition. It is a figure which shows the addition pixel position when producing | generating a highly sensitive 2D moving image by 16 pixel addition. It is explanatory drawing when reading 2D moving image data of HD image quality from the solid-state image sensor of FIG. It is a figure which shows the addition pixel position when producing | generating 2D moving image of HD image quality by 4 pixel addition. It is a flowchart which shows the process sequence which produces | generates 2D moving image of HD image quality according to the brightness of a photographing scene.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is an external perspective view of an imaging apparatus according to an embodiment of the present invention. In this monocular imaging device (digital camera) 10, a lens barrel 13 that houses a photographing lens 12 is attached to a front portion of a camera housing 11 so as to be retractable. A shutter release button 14 is provided at the right end of the upper surface of the camera casing 11, and a liquid crystal display section (display section 28 in FIG. 2) not shown in FIG. 1 is provided at the back of the camera casing 11. .

FIG. 2 is a functional block configuration diagram of the imaging apparatus 10 shown in FIG. The imaging device 10 includes a photographic lens 12 and a CMOS solid-state imaging device 21 that is placed on the back of the photographic lens 12 and disposed on the imaging plane thereof. The photographing lens 12 and the solid-state solid-state imaging device 21 are driven by instructions from a system control unit (CPU) 29 described later.

The solid-state imaging device 21 is a CMOS type image sensor in this embodiment, but may be a CCD type or other type of solid-state imaging device.

The imaging apparatus 10 of the present embodiment further includes a digital signal processing unit 26 that takes in a digital captured image signal output from the solid-state image sensor 21 and performs interpolation processing, white balance correction, RGB / YC conversion processing, and the like, and a captured image signal A compression / decompression processing unit 27 that compresses image data in JPEG format or the like, a display unit 28 that displays menus and displays through images and captured images, and overall control of the entire digital camera A system control unit (CPU) 29, an internal memory 30 such as a frame memory, and a media interface (I / F) unit 31 that performs interface processing between a recording medium 32 that stores JPEG image data and the like. And a bus 40 connected to each other.

Furthermore, an operation unit 33 for inputting an instruction from the user is connected to the system control unit 29, and a flash 25 that emits light in response to an instruction from the system control unit 29 is connected.

The imaging device 10 captures a high-definition 2D still image of a subject, a 2D moving image or a 3D moving image with a resolution lower than that of the high-definition still image based on a selection instruction from the operation unit 33 of the user. Then, based on these selection instructions, drive control and image processing of the solid-state imaging device 21 are performed.

FIG. 3 is a schematic diagram of the surface of the solid-state imaging device 21, showing a pixel array and a color filter array on each pixel. Each square 2 inclined 45 degrees obliquely represents each pixel (photoelectric conversion element) formed on the semiconductor substrate, and R (= r), G (= g), B described thereon (= B) indicates the colors (R, r = red, G, g = green, B, b = blue) of the three primary color filters stacked on each pixel 2. Since the signal readout circuit, vertical scanning circuit, horizontal scanning circuit, and the like may be the same as those of a normal CMOS image sensor, illustration is omitted.

In the solid-state imaging device 21 of the present embodiment, even-numbered pixel rows are arranged by being shifted by 1/2 pixel pitch with respect to odd-numbered pixel rows. If only even pixel rows are viewed, each pixel 2 is arranged in a square grid, and further, three primary color filters are Bayer arranged, and even if only odd pixel rows are viewed, each pixel 2 is arranged in a square lattice. The three primary color filters are arranged in a Bayer array.

The color filter on each pixel in the even line is indicated by uppercase RGB, and the color filter on each pixel in the odd line is indicated by lowercase rgb. Each pixel in either the even row or the odd row (even row in the illustrated example) is hereinafter also referred to as a group A (or first group) pixel, and the other of the even row or the odd row (in the illustrated example, the odd row). Each pixel is also referred to as a B group (or second group) pixel. As shown in the figure, since the color filter array is a Bayer array for both the A group pixel and the B group pixel, the A group pixel and the B group pixel that are diagonally adjacent to each other are the same color pixel, and constitute a pair pixel.

In the solid-state imaging device 21 of the present embodiment, among all the pixel pairs, pair pixels existing at discrete and periodic positions are used as pair pixels of phase difference detection pixels. A phase difference detection pixel (also referred to as a focus detection pixel) refers to a pair of pixels obtained by pupil division. In the example shown in the drawing, the light

shielding film openings

2a and 2b are located at the center of each of the pair pixels. It is configured to be eccentric in the opposite direction.

For a normal pixel (a pixel that is not a phase difference detection pixel), as shown in a dotted rectangular frame at the upper left of FIG. The membrane opening 1 is opened. On the other hand, the light

shielding film openings

2a and 2b of the paired pixels of the phase difference detection pixel are smaller than the light shielding film opening 1, and in the example shown in the drawing, the light shielding film opening 2a opens only to the left half with respect to the opening 1. The opening 2b has a configuration in which only the right half of the opening 1 is opened. In FIG. 3, only one light-shielding film opening 1 is shown, but it is also provided in other normal pixels (the illustration is omitted because the figure becomes complicated). In this embodiment, the microlens is also stacked on the phase difference detection pixel.

The opening center position of the light shielding film opening 2a is eccentric to the left with respect to the center of the pixel, and the opening center position of the light shielding film opening 2b is eccentric to the right with respect to the center of the pixel. As shown in FIG. 1, a lens 12 is disposed in front of the solid-state imaging device 21, and light passing through the lens 12 and received by the pixel through the light-shielding film opening 2 a is on the left side of the lens 12. From the direction of viewing the subject with the left eye, and the light received by the pixels through the light shielding film opening 2b is the light from the right side of the lens 12, that is, the subject viewed with the right eye. The light coming from the direction is the main.

That is, the captured image signal of the pixel having the light shielding film opening 2a among the paired pixels constituting the phase difference detection pixel becomes an image obtained by viewing the subject with the left eye, and the captured image signal of the pixel having the light shielding film opening 2b represents the subject. The image is seen with the right eye. Therefore, a combination of the two makes it possible to reproduce a stereoscopic image of the subject.

The pupil division method illustrated in FIG. 3 is configured by decentering the light

shielding film openings

2a and 2b. In addition to this, as described in, for example, Japanese Patent Application Laid-Open No. 2007-281296, it is commonly used for paired pixels. A configuration may be adopted in which a single microlens or the like is mounted, and the incident angles of the paired pixels to each pixel are different.

The paired pixels of the phase difference detection pixel of the present embodiment are one pair every three pairs in the vertical direction and the horizontal direction as a phase difference detection pixel pair, but one pair every five pairs may be a phase difference detection pixel pair. good. Generally, one pair is used as a phase difference detection pixel pair for every [2n + 1] pairs (n = 1, 2,...). Thus, by determining the arrangement position of the phase difference detection pixel pair, the color filter array of the phase difference detection pixel pair is also a Bayer array having the three primary colors of R, G, and B.

FIG. 4 shows a pair of pixels for reading a picked-up image signal when a solid-state moving image 21 is picked up by the solid-state image pickup device 21 shown in FIG. 3, and a thick line frame is shown in FIG. Only the captured image signal detected by each pixel of the phase difference detection pixel pair indicated by is read out.

Shooting is performed by, for example, rolling shutter driving, and when reading a picked-up image signal from the solid-state image pickup device 21, reading is performed in order of horizontal lines in order of A group pixel → B group pixel → A group pixel → B group pixel →.

The captured image signal of the pixel mounting the light shielding film opening 2a is processed as image data of the subject viewed with the left eye, and the captured image signal of the pixel mounting the light shielding film opening 2b is imaged of the subject viewed with the right eye. A stereoscopic moving image of the subject can be recorded by performing image processing as data, associating the subject image after the image processing, and recording the captured image data for each frame in the memory. Of course, it can also be recorded as a stereoscopic still image.

Since the captured image signal for the left eye and the captured image signal for the right eye read from the solid-state image sensor 21 are signals read from the Bayer array pixels, the image processing is performed for the existing Bayer array. An image processing engine can be used, and cost reduction can be achieved.

FIG. 5 is a diagram for explaining the principle that a stereoscopic image of a subject can be picked up by a phase difference detection pixel pair. The phase difference detection pixel pairs are arranged at a required interval in the horizontal direction, and a change in the captured image signal obtained through the light shielding film opening 2a is defined as a signal f (x), and a change in the captured image signal obtained through the light shielding film opening 2b is represented as a signal Let g (x).

The signal f (x) and the signal g (x) are obtained as a result of receiving incident light from the same subject on the same horizontal line. The signals f (x) and g (x) have the same waveform shifted in the horizontal direction. The amount of phase difference. This phase difference amount is a parallax according to the distance to the subject, and the right-eye captured image signal and the left-eye captured image signal having this phase difference amount are read out from the solid-state image sensor 21, so An image can be generated.

Next, a case where a 2D moving image is captured using the imaging device 10 will be described. As described above, the imaging apparatus 10 can capture 3D moving images, but some users may prefer 2D moving images. In some cases, the monitor or television receiver to be played back is not for 3D image playback and only supports 2D images. Alternatively, it may be required to simultaneously capture both a 3D moving image and a 2D moving image. In view of this, the imaging apparatus 10 of the present embodiment is equipped with not only 3D but also a 2D moving image shooting function.

FIG. 6 is an explanatory diagram of the first embodiment for capturing a 2D moving image. Of course, since the solid-state imaging device 21 is the same, the pixel arrangement and the color filter arrangement shown in FIG. 6 are the same as those in FIGS. In the present embodiment, normal pixels surrounded by a thick black frame shown in FIG. 6 are used when capturing a 2D moving image.

The pixel pair surrounded by the thick black frame in FIG. 6 is a pixel adjacent to the phase difference detection pixel pair surrounded by the thick black frame in FIG. 4 in the horizontal direction, and is a normal pixel pair. In the present embodiment, a moving image is generated using an addition average value of captured image signals detected by each pixel of the normal pixel pair. By using the addition average value, S / N is improved.

When both a 3D moving image and a 2D moving image are shot, in this embodiment, a 3D moving image and a 2D moving image can be shot simultaneously using pixels adjacent in the horizontal direction. Since the above-described average value is used as data for generating a 2D moving image, each pixel position becomes an addition centroid position of a pair pixel, and a position closest to each pixel position of the 3D image. For this reason, 3D moving images and 2D moving images having the same resolution and the same resolution can be obtained.

Further, since 3D moving image data and 2D moving image data can be read from the same horizontal line from the solid-state imaging device 21, it is possible to simultaneously read the 3D moving image and the 2D moving image in a short time. Similarly to the image processing of 3D moving images, the image processing of 2D moving image data can use a Bayer array image processing engine because the color array of the read captured image signal is a Bayer array. Can be achieved.

FIG. 7 is a flowchart showing a processing procedure when performing 2D moving image shooting according to the second embodiment. In the embodiment of FIG. 6, the 2D moving image is generated using the addition average value of the normal pixel pair. However, the present invention is not limited to this, and the 2D moving image is generated using only the captured image signal of one of the paired pixels. Alternatively, the 2D moving image may be generated by adding signals of both captured image signals of the paired pixels outside the solid-state imaging device. It is better to decide which one to use depending on the brightness of the shooting scene.

In the present embodiment, the brightness of the photographic scene is determined in step S1, and if the brightness is equal to or greater than the predetermined threshold value α, the process proceeds to step S2, and the captured image signal of one of the paired pixels is employed, or the pair The addition average value of the captured image signals of the two pixels of the pixels is adopted, and the process proceeds to step S4.

If the result of determination in step S1 is that the scene brightness is darker than the threshold value α, the process proceeds to step S3 to increase the sensitivity, and the captured image signals of the two pixels of the paired pixels are added to step S4. In step S4, 2D moving image data is generated, and this process ends.

Note that it is also possible to shoot a moving image with a wide dynamic range when shooting only a 2D moving image instead of shooting a 3D moving image and a 2D moving image at the same time. By adjusting the reset signal applied to the pixel signal readout circuit, the timing of the readout signal, etc., the exposure time of the B group pixel is shortened relative to the exposure time of the A group pixel, and one of the paired pixels ( If the exposure time is changed between the (A group pixel) and the other (B group pixel), a 2D moving image with a wide dynamic range can be obtained by adding both captured image signals.

8 to 10 are diagrams illustrating a method of reading a captured image signal for 2D moving images from the solid-state image sensor 21 according to the third embodiment of the present invention. In the embodiment of FIG. 7, when the shooting scene is dark and low in illuminance, two pixels of the paired pixels are added to increase the sensitivity. However, when the shooting scene is darker, it is necessary to further increase the number of added pixels. FIGS. 8 to 10 are diagrams showing the pixel positions to be added when the number of added pixels is increased.

That is, when the shooting scene has normal brightness, a 2D moving image is generated using only the addition average value of the paired pixels described in FIG. 6 and the captured image signal of one pixel, but the shooting scene is darker. In this case, as shown in FIG. 8, for example, as image data at a pixel position indicated by an ellipse, two normal pixel pairs (indicated by bold black frames) adjacent to the left and right in the horizontal direction of the phase difference detection pixel position indicated by the ellipse. The imaged image signals of a total of 4 pixels are added to obtain image data at the ellipse center position.

If darker, add 8 pixels or 10 pixels. In the case of performing 8-pixel addition, the four paired pixel positions are, for example, the positions shown in FIG. That is, as the image data at the phase difference detection pixel position indicated by the ellipse, the picked-up image signals of four normal color pixel pairs on the top, bottom, left, and right are selected as addition targets.

When performing 10-pixel addition, in addition to 8-pixel addition, a phase difference detection pixel indicated by an ellipse is also added. Since the phase difference detection pixel is an addition of the left eye and the right eye, if it is added in pairs, it can be used in the same way as a normal pixel.

If darker, add 16 pixels or add 18 pixels. FIG. 10 shows eight pair pixel positions where 16 pixel addition is performed. That is, as the image data at the phase difference detection pixel position indicated by an ellipse, a captured image signal of four pairs of pixels in an oblique direction is added in addition to the top, bottom, left, and right in FIG. When performing 18-pixel addition, in addition to 16-pixel addition, a phase difference detection pixel pair indicated by an ellipse is also added in the same manner as described above.

In FIG. 10, only eight pair pixel positions for one phase difference detection pixel position are shown. However, when eight pair pixel positions for each phase difference detection pixel position are taken, all normal pixel pairs are added. Will be selected. That is, since a 2D moving image is generated using captured image signals of all normal pixels, it is possible to suppress image quality deterioration factors such as false colors and jaggy.

FIG. 11 is a diagram for explaining a method of reading a captured image signal for 2D moving images from a solid-state image sensor according to the fourth embodiment. In the second and third embodiments, the case where a 2D moving image having the same resolution as that of the 3D moving image is generated has been described. However, in addition to the 3D moving image, a 2D moving image having an increased horizontal resolution may be desired. For example, there is a case where an HD quality moving image having an aspect ratio of 16: 9 is desired.

The aspect ratio of an image captured by the imaging apparatus 10 is normally 4: 3, but there are cases where it is desired to view a moving image on a large screen of a large television receiver having an aspect ratio of 16: 9. In such a case, the reading method of the following fourth embodiment may be adopted, and selection is made on the menu screen of the imaging apparatus 10.

In FIG. 11, a pair of normal pixels indicated by a thick black frame is selected to read a captured image signal for a 2D image. In this case, pixels are not thinned out in the horizontal line, and pixels in the vertical line are thinned out without reading the horizontal line in which the phase difference detection pixel exists. Shooting is performed by, for example, rolling shutter drive, and when a captured image signal is read out from the solid-state image sensor 21, reading is performed in the order of horizontal lines in the order of A group pixel → B group pixel → A group pixel → B group pixel →.

As in the above-described embodiment, a value obtained by averaging the captured image signals of the paired pixels is used, and the signal at the paired pixel position is used as a signal for one pixel. By thinning out pixels in this way, the color array of the picked-up image signal as a result of the averaging is a Bayer array, and it is possible to use an image processing engine for the Bayer array.

Next, a fifth embodiment will be described. In the fifth embodiment, the 2D moving image data is not generated by using the average value of the paired pixels, but one of the paired pixels is selected and 2D is selected as described in FIG. 2D moving image data is generated by generating moving image data or performing two-pixel addition of paired pixels. The pixel addition is preferably performed by the digital signal processing unit 26 in FIG. 2 after reading out the captured image signal from the solid-state image sensor 21. As a result, the generation speed of 2D moving image data is improved, or highly sensitive 2D moving image data can be obtained.

FIG. 12 is a view showing a captured image signal reading method for 2D moving images according to the sixth embodiment. Also in the present embodiment, signal readout is not performed for horizontal lines in which phase difference detection pixels exist, and horizontal lines in which only other normal pixels exist are read out. Then, in this embodiment, when the shooting scene is dark and the sensitivity is insufficient even with the two-pixel addition of the fifth embodiment described above, as shown in FIG. 12, two sets of the same color sandwiching the non-read line in the vertical direction. Four pairs of pixels are added. In FIG. 12, the pair pixels are surrounded by an ellipse, and the pair pixel ellipses to be added are connected by lines. Thereby, it is possible to shoot a 2D moving image with high sensitivity and HD image quality.

FIG. 13 is a flowchart showing a processing procedure for variably controlling the pixel addition number according to the brightness of the shooting scene. First, it is determined whether or not the brightness of the shooting scene is brighter than the first threshold value α1, and if it is bright, the process proceeds to step S12, and as described with reference to FIG. 11, only the captured image signal of one pixel of the paired pixels is obtained. Or, an addition average value of captured image signals of paired pixels is adopted. Then, the process proceeds to the next step S13, where a 2D moving image of HD image quality is generated using the adopted captured image signal, and this process is terminated.

If the result of determination in step S11 is that the brightness is darker than the first threshold value α1, the process proceeds to step S14, where it is determined whether or not the photographed scene is brighter than the second threshold value α2 that is darker than α1. If it is brighter than the second threshold value α2, the process proceeds from step S14 to step S15, the captured image signals of the two pixels of the pair pixel are added, and the process proceeds to step S13.

If the result of determination in step S14 is that the brightness of the photographic scene is darker than the second threshold value α2, the process proceeds to step S16, the four-pixel addition of the captured image signal described in FIG. 12 is performed, and the process proceeds to step S13.

As described above, it is possible to shoot a 2D moving image of HD image quality with a sensitivity within a required range regardless of whether the brightness of the shooting scene is bright or dark.

When taking a high-definition 2D still image using this solid-state imaging device 21, the imaged image signals of all the pixels are read out. Since the solid-state imaging device 21 can acquire two Bayer array images (captured images by group A pixels and captured images by group B pixels) in which the pixel array (array of captured image signals) is shifted by half a pitch in the vertical and horizontal directions. A high-definition 2D still image can be generated using the two pieces of image data.

In this case, since the light

shielding film openings

2a and 2b of the phase difference detection pixel are narrower than the light shielding film opening 1 of the normal pixel, the sensitivity is low. Therefore, the captured image signal of the phase difference detection pixel is corrected by interpolation calculation using data of surrounding normal pixels. Alternatively, the gain of the amplification process performed by the digital signal processing unit 26 of FIG. 2 on the captured image signal of the phase difference detection pixel is corrected to be higher than the amplification rate of the captured image signal of the normal pixel.

When capturing a still image, when the subject image is generated by adding the captured image signals of the pair pixels with the same exposure time for the A group pixel and the B group pixel, a high-definition subject image is obtained as two images in a square array. Can be generated. When the A group pixel and the B group pixel are pixel-added between the paired pixels, the signal arrangement becomes one square arrangement and a highly sensitive image can be generated. Further, when image processing is performed on still image data, since the basic color filter array is a Bayer array in the solid-state imaging device 21 of the present embodiment, an image processing engine for the Bayer array can be used, thereby reducing costs. it can. In this case as well, it is necessary to correct the captured image signal of the phase difference detection pixel as described above.

It is also possible to photograph a high-definition (number of pixels that is 1/2 of the actual number of pixels) wide dynamic range using the solid-state imaging device 21. The reset signal applied to the signal readout circuit of each pixel, the timing of the readout signal, etc. are adjusted, and the exposure time of the B group pixel is shortened with respect to the exposure time of the A group pixel. Then, by reading out the captured image signal from all the pixels and adding the captured image signals of the paired pixels, the dynamic range of the captured image is widened. Also in this case, it is necessary to correct the detection signal of the phase difference detection pixel as described above.

As described above, according to the above-described embodiment, when a 3D image is captured, image processing is performed using only the captured image signal of the phase difference detection pixel, and when a 2D image that is not a still image is captured, Since image processing is performed using only captured image signals of normal pixels other than the phase difference detection pixels, both are imaged together, both are read out from the solid-state image sensor 21, and then are distinguished by image processing. Can be stored in a recording medium. In addition, when capturing a 2D moving image, the number of pixel additions can be controlled according to the brightness of the shooting scene, so that a moving image with a required sensitivity can be generated.

In addition, as in the above-described embodiment, when adding pixels, by setting the added barycentric position as the position of the phase difference detection pixel, it is possible to eliminate phase difference unevenness in the screen and obtain a high-quality moving image. It becomes possible.

When a 3D moving image having an aspect ratio of 16: 9 is picked up using an image sensor on a light receiving surface with an aspect ratio of 4: 3, a phase difference detection pixel pair is set only within the range of the aspect ratio of 16: 9 on the light receiving surface. Just place it.

As described above, in the imaging device of the embodiment, the plurality of pixels are arranged in a two-dimensional array, and the first group pixel configured by one pixel row of an odd row or an even row and an odd row or an even row. A semiconductor substrate in which a second group pixel constituted by the other pixel row is disposed with a ½ pixel pitch shift in the horizontal direction and the vertical direction; a Bayer array color filter stacked on the first group pixel; An image pickup device comprising a Bayer array color filter stacked on a second group pixel, wherein each pixel of the first group pixel and the color filter of the same color as the pixel adjacent to the pixel are stacked The pixel belonging to the second group pixel is set as a pair pixel, and among the pair of pixels, pair pixels corresponding to 2n + 1 (n = 1, 2,...) Intervals in the horizontal direction and the vertical direction are phase difference detection pixels. Characterized by constructionAs a result, the Bayer array can be used as an image processing engine for image processing of still images and moving images.

In the imaging device according to the embodiment, the phase difference detection pixel included in the first group pixel acquires one of an image for the right eye or an image for the left eye, and the position included in the second group pixel. The phase difference detection pixel is configured to acquire the other of the right-eye image and the left-eye image.

Further, the image pickup device of the embodiment is characterized in that n = 1. Thereby, the number of pixels of the 3D moving image can be maximized with respect to the number of pixels of the image sensor.

In addition, the imaging apparatus according to the embodiment includes the imaging element described above, and the captured image signal of the phase difference detection pixel when the stereoscopic moving image is captured, the pixel row of the first group pixel, and the pixel row of the second group pixel. And an image sensor driving unit that alternately reads from the image sensor. As a result, the difference in exposure time between the first group pixel and the second group pixel can be minimized, and post-processing can be simplified side by side.

In the imaging apparatus according to the embodiment, when a planar moving image is captured, the captured image signal of the paired pixels other than the phase difference detection pixel is transmitted to the pixel group row of the first group pixel and the second group. It is characterized by alternately reading from pixel rows of pixels. This makes it possible to generate a 2D moving image while avoiding the influence of the phase difference pixels.

In addition, the imaging apparatus according to the embodiment uses a captured image signal of any one of the paired pixels other than the phase difference detection pixels or a captured image signal of two pixels. An image processing unit that generates the planar moving image using the addition average value is provided. As a result, a 2D moving image can be quickly obtained, and a 2D moving image with good S / N can be obtained.

In addition, the imaging apparatus according to the embodiment includes an image processing unit that generates the planar moving image by adding both the captured image signals of the paired pixels other than the phase difference detection pixels when the shooting scene is dark. It is characterized by. Thereby, a highly sensitive 2D moving image can be obtained.

In addition, in the imaging device of the embodiment, when the shooting scene is further dark, the image processing unit captures image signals of a plurality of pixels constituting the adjacent pair pixels of the same color among the pair pixels other than the phase difference detection pixels. Are added to generate the planar moving image. This makes it possible to capture a 2D moving image with higher sensitivity.

Further, the imaging apparatus according to the embodiment is characterized in that the planar moving image is captured simultaneously with the stereoscopic moving image. This makes it possible to capture two types of moving images in a short time.

In addition, the phase difference detection pixels of the imaging apparatus of the embodiment are arranged in an aspect ratio of 16: 9 in the imaging device, and the planar moving image is vertical without performing pixel thinning in the horizontal direction of the imaging device. Only the pixel thinning in the direction is performed to obtain an image with an aspect ratio of 16: 9. As a result, a large screen moving image can be captured by a television receiver having an aspect ratio of 16: 9.

In addition, when capturing a high-definition planar still image, the imaging apparatus according to the embodiment reads a captured image signal from all pixels of the image sensor and corrects the captured image signal of the phase difference detection pixel to generate a high-definition planar still image. It is characterized by generating. Since normal pixels are arranged around the phase difference detection pixels, correction can be performed easily and reliably, and a high-definition still image can be acquired.

According to the embodiment described above, it is possible to acquire a moving image that meets a user's request in a short time and at a low cost.

The imaging apparatus according to the present invention can capture not only a still image but also a 3D moving image and a 2D moving image, and is useful as a digital camera or the like with a moving image shooting function.

Although the present invention has been described in detail and with reference to specific embodiments, it will be apparent to those skilled in the art that various changes and modifications can be made without departing from the spirit and scope of the invention.
This application is based on a Japanese patent application filed on June 6, 2011 (Japanese Patent Application No. 2011-126643), the contents of which are incorporated herein by reference.

1 Light-shielding film opening of normal pixel 2 Pixel (photoelectric conversion element)
2a, 2b Light-shielding film opening 10 of phase difference detection pixel Imaging device 12 Shooting lens 26 Digital signal processing unit 29 System control unit

Claims

A first group of pixels formed by arranging one or more odd rows or even rows of pixels and a second group of pixels comprising the other row of odd rows or even rows; Are arranged with a half pixel pitch shifted in the horizontal and vertical directions, a Bayer array color filter stacked on the first group pixel, and a Bayer array color filter stacked on the second group pixel. A pair of each pixel of the first group pixel and a pixel belonging to the second group pixel that is obliquely adjacent to the pixel and in which the color filters of the same color as the pixel are stacked. An image sensor in which pair pixels corresponding to 2n + 1 (n = 1, 2,...) Intervals in the horizontal direction and the vertical direction are configured by phase difference detection pixels.
2. The imaging device according to claim 1, wherein the phase difference detection pixel included in the first group pixel acquires one of an image for a right eye or an image for a left eye and is included in the second group pixel. The imaging device configured to acquire the other of the right-eye image or the left-eye image.
The image pickup device according to claim 1 or 2, wherein n = 1.
The imaging device according to any one of claims 1 to 3, and a captured image signal of the phase difference detection pixel at the time of imaging a stereoscopic moving image, the pixel row of the first group pixel and the second group pixel An imaging device comprising: an imaging element driving unit that alternately reads from pixel rows.
5. The imaging device according to claim 4, wherein when imaging a planar moving image, the imaging element driving unit outputs a captured image signal of the paired pixels other than the phase difference detection pixels and a pixel row of the first group pixel and the pixel group An imaging device that alternately reads out the pixel rows of the second group pixels.
The imaging apparatus according to claim 5, wherein an imaging image signal of any one of the paired pixels other than the phase difference detection pixels is used, or imaging of two pixels is performed. An imaging apparatus including an image processing unit that generates the planar moving image by using an average value of image signals.
The image processing unit according to claim 5, wherein when the shooting scene is dark, the planar moving image is generated by adding both of the captured image signals of the paired pixels other than the phase difference detection pixels. An imaging apparatus comprising:
The imaging apparatus according to claim 7, wherein when the shooting scene is further dark, the image processing unit includes a plurality of pixels constituting a pair of adjacent pixels of the same color among the pair of pixels other than the phase difference detection pixels. An imaging apparatus that generates the planar moving image by adding captured image signals.
The imaging apparatus according to claim 4, wherein the planar moving image is captured simultaneously with the stereoscopic moving image.
9. The imaging apparatus according to claim 4, wherein the phase difference detection pixels are arranged in an aspect ratio of 16: 9 in the imaging device, and the planar moving image is An image pickup apparatus that performs only pixel thinning in the vertical direction without thinning out pixels in the horizontal direction of the image pickup device, and forms an image with an aspect ratio of 16: 9.
11. The imaging device according to claim 4, wherein when a high-definition planar still image is captured, a captured image signal is read from all pixels of the image sensor and imaging of the phase difference detection pixel is performed. An imaging device that corrects an image signal to generate a high-definition flat still image.