JP6652666B2 - Image sensor - Google Patents

Description

  The present invention relates to an image sensor in which focus detection pixels for performing focus detection by a phase difference method are arranged on an imaging surface.

  2. Description of the Related Art An image sensor in which focus detection pixels for performing focus detection by a phase difference method are arranged in a part of the image sensor has been conventionally known. For example, in Patent Document 1, as a focus detection pixel, a right aperture pixel (abbreviated as an R pixel) in which a subject luminous flux from the right enters the pixel, and a subject luminous flux from the left enter the pixel. An image sensor in which left-opening pixels (abbreviated as L-pixels) are arranged in the horizontal direction is disclosed.

JP 2013-257494 A

  In Patent Document 1, since the R pixel and the L pixel are only arranged in the horizontal direction, for example, when a subject having a change in contrast only in the horizontal direction is inclined by 45 degrees or more from the horizontal direction, the contrast change Distance cannot be measured for the subject for which is detected. Therefore, an upper aperture pixel (abbreviated as T pixel) in which the subject light flux from the upper side is incident on the pixel, and a lower aperture pixel (abbreviated as B pixel) in which the subject light flux from the lower side is incident on the pixel. ) May be arranged in the vertical direction so that the distance can be measured even for the subject inclined at 45 degrees or more.

  However, when the T pixel and the B pixel are arranged at the position of the G pixel (the pixel on which the green filter is arranged) at the same density as the R pixel and the L pixel, the AF performance in both the left-right direction and the up-down direction is compatible. It is difficult.

  The present invention has been made in view of such circumstances, and arranges pixel rows with upper and lower apertures in addition to pixel rows with left and right apertures for focus detection, thereby achieving both AF performance in the horizontal direction and the vertical direction. It is an object of the present invention to provide an imaging device that can perform the above-described operations.

In order to achieve the above object, an image pickup device according to a first aspect of the present invention is an image pickup device having a plurality of image pickup pixels and a plurality of focus detection pixels in which the position of an opening of a light receiving unit is shifted from the image pickup pixels. The focus detection pixels of a first focus detection pixel group that includes a plurality of focus detection pixels that shift the position of the opening in the direction of 1 and that have different positions of the openings correspond to the first color filters of the imaging pixels. In the position, the focus detection pixels having different positions of the opening are arranged at a first pixel pitch in the first direction, the openings are shifted in a second direction different from the first direction, and the positions of the openings are shifted. The focus detection pixels of a second focus detection pixel group including a plurality of focus detection pixels different from each other are located at positions corresponding to a second color filter different in color from the first color filter of the imaging pixels. Focus detection images with different aperture positions Respectively disposed at a second pixel pitch in the second direction for each, a position corresponding to the second color filter of the imaging pixel, the position of the second focus detection pixels are arranged Is provided with the first color filter .

  In the imaging device according to a second aspect, in the first aspect, the focus detection pixels of the first focus detection pixel group having different positions of the openings are alternately arranged in the second direction at the first pixel pitch. And the focus detection pixels of the second focus detection pixel group having different positions of the openings are alternately arranged at the second pixel pitch in the first direction.

Imaging device according to the third invention, in any one of the first or second aspect of the present invention, the first color filter in green, the second color filter is a blue or red.

  According to the present invention, it is possible to provide an image sensor in which pixel rows of upper and lower apertures for focus detection are arranged and which can achieve both AF performance in the horizontal direction and the vertical direction.

FIG. 2 is a plan view showing a pixel array of the image sensor according to the embodiment of the present invention. FIG. 2 is a plan view illustrating an arrangement relationship between RL pixels and BT pixels in the image sensor according to the embodiment of the present invention. FIG. 4 is a diagram illustrating mixed reading in the image sensor according to the embodiment of the present invention. FIG. 9 is a diagram illustrating a modified example of mixed reading in the image sensor according to the embodiment of the present invention. FIG. 9 is a diagram illustrating a modified example of mixed reading in the image sensor according to the embodiment of the present invention. FIG. 4 is a diagram illustrating islanding of RL pixels in the image sensor according to the embodiment of the present invention. FIG. 3 is a diagram illustrating pixel reading in the image sensor according to the embodiment of the present invention. FIG. 1 is a block diagram mainly showing an electrical configuration of a digital camera to which an image sensor according to an embodiment of the present invention is applied. 5 is a flowchart illustrating an operation of the digital camera to which the imaging device according to the embodiment of the present invention is applied.

  Hereinafter, an example in which an embodiment of the present invention is applied to an image sensor will be described. The image sensor according to the present embodiment generally has a pixel light-blocking type image plane phase difference AF function, and has horizontal pixel rows (RL pixel rows) arranged at intervals of 4 pixel pitch or more (for details, see FIG. Further, vertical pixel columns (TB pixel columns) are also arranged at intervals of 4 pixel pitch or more. The TB pixel columns are arranged at intervals obtained by rotating the RL pixel columns by exactly 90 degrees (see FIG. 2B for details).

  Either the TB pixel or the RL pixel is arranged at the Bb pixel, and at the time of the mixed reading, the focus detection pixel (either the TB pixel or the RL pixel) arranged at the Bb pixel is skipped and read (for details, see FIG. (See FIG. 3). Either the TB pixel or the RL pixel is arranged at the Rr pixel, and at the time of the mixed reading, the focus detection pixel (either the TB pixel or the RL pixel) arranged at the Rr pixel is skipped and read (see FIG. 4 for details). reference).

  The TB pixel or the RL pixel arranged in the G pixel is converted into an island (a region where the TB pixel or the RL pixel does not exist in the G pixel is set) (see FIG. 5 for details). The spacing between the islands secures a spacing corresponding to the number of pixels that is at least twice the number of lines in the vertical direction necessary for pixel mixing during mixed reading. Further, skip reading of TB pixels is also performed in an area where no TB pixels exist (see FIG. 6 for details).

  FIG. 1 shows an arrangement of imaging pixels and focus detection pixels in the imaging element 21. In FIG. 1, a focus detection pixel (R pixel) having an opening on the right side and a focus detection pixel (L pixel) having an opening on the left side are arranged at a pitch of 4 pixels in the horizontal direction (horizontal direction). This is an example. In FIG. 1, the pitch is four pixel pitches. This is because it is difficult to maintain the image quality performance when the focus detection pixels are arranged at a pitch of less than 4 pixels.

  In FIG. 1, an R pixel or an L pixel is arranged at a part of a position where a G pixel (an imaging pixel having a G filter) is arranged in the case of an ordinary Bayer array imaging element that does not include a focus detection pixel. ing. When no R pixel or L pixel is arranged, two G pixels (imaging pixels on which a green filter is arranged) are arranged opposite to each other within a range of 2 × 2 pixels, and the other G pixels are arranged opposite to each other. One Bb pixel (an imaging pixel on which a blue filter is arranged) and one Rr pixel (an imaging pixel on which a red filter is arranged) are arranged at positions.

  Then, in FIG. 1, positions (x1, y1), (x5, y1), (x9, y1), (x9, y1), (x13) where the G pixels are arranged in the case of a normal Bayer array imaging device that does not include the focus detection pixels. , Y1),..., (X3, y9), (x7, y9), (x11, y9), (x15, y9),. (X5, y5), (x9, y5), (x13, y5), ..., (x3, y13), (x7, y13), (x11, y13), (x15, y13), ... L pixels are arranged. As described above, the R pixel and the L pixel are arranged at every four pixels in the horizontal direction (horizontal direction) at a part of the position where the G pixel of the normal imaging device that does not include the focus detection pixel is arranged. .

  In performing focus detection, the phase difference may be calculated using one pixel each of the R pixel and the L pixel. However, in the present embodiment, the phase difference is calculated within a predetermined range in the vertical direction (vertical direction) of the image sensor 21. The outputs of the R pixels (in Rsumarea) are added and treated as image data of one pixel. Similarly, outputs of L pixels within a predetermined range (within Lsumarea) in the vertical direction (vertical direction) of the image sensor 21 are added and handled as one pixel of image data. A phase difference is calculated from a lateral (horizontal) change in the added value of the R pixel and the added value of the L pixel within this predetermined range.

  Further, within a predetermined range (Rsumarea and Lsumarea), R pixels at different positions in the y direction are arranged by being shifted by two pixels in the x direction. That is, R pixels at positions (x1, y1), (x5, y1),... Are shifted by two pixels in the horizontal direction, and (x3, y9), (x7, y9),. Pixels are arranged. This is to ensure AF accuracy by arranging focus detection pixels more densely with respect to the sampling pitch (four pixels in the example of FIG. 1). The same arrangement is applied to the L pixels.

  Next, the arrangement of T pixels and B pixels will be described with reference to FIG. FIG. 2A is the same as the pixel array shown in FIG. Rotate clockwise 90 degrees centering on the lower right corner O in FIG. 2A and arrange the B pixel and T pixel at the position of the Bb pixel. To a B pixel and a T pixel by one pixel to obtain the pixel arrangement shown in FIG. Then, when the R pixel and the L pixel shown in FIG. 2A and the T pixel and the B pixel shown in FIG. 2B are overlapped, as shown in FIG. 2C, the RL pixel (the R pixel and the L pixel A pixel arrangement of an abbreviated name when indicating both pixels and a TB pixel (an abbreviated name when indicating both a T pixel and a B pixel) is obtained.

  For example, the R pixel at the position (x1, y1) is rotated by 90 degrees around the position O (position (x16, y1)), and diagonally left down to correspond to the Bb pixel in the arrangement of FIG. After moving by one pixel, the position becomes (x15, y2), and the B pixel is arranged at this position. Further, the L pixel at the position (x1, y5) is rotated by 90 degrees around the position O (position x12, y1), and one pixel diagonally downward to the left to correspond to the Bb pixel in the arrangement of FIG. When moved, the position becomes (x11, y2), and a T pixel is arranged at this position.

  As described above, in the pixel arrangement (shown in FIG. 2C) in the present embodiment, the horizontal pixel rows of RL pixels are arranged at intervals of 4 pixel pitch or less, and the vertical pixel rows of TB pixels are RL pixels. The pixel pitch is the same as that of the columns, and the RL pixel columns are arranged at intervals just rotated 90 degrees. The position of the TB pixel is originally at the position of the Bb pixel, and is replaced with a green or transparent color filter instead of the blue filter.

  Therefore, the imaging element 21 according to the present embodiment includes a plurality of imaging pixels and a plurality of focus detection pixels (RL pixels and TB pixels) in which the position of the opening of the light receiving unit is shifted with respect to the imaging pixels. I have. Then, a first focus detection pixel (for example, RL pixel) whose opening position is shifted in a first direction (for example, horizontal direction) is changed to a position (for example, G corresponding to the first color filter of the imaging pixel). Second focus detection in which pixels are arranged at a first pixel pitch (for example, at least four pixel pitches) and the aperture is shifted in a second direction (for example, a vertical direction) different from the first direction. Pixels (for example, TB pixels) at positions corresponding to a second color filter different from the first color filter of the imaging pixels (for example, Gb pixel positions, Gr pixel positions) at a second pixel pitch. (For example, 4 pixel pitch or more).

  In this case, the first color filter may be green, the second color filter may be blue (see FIG. 3, which will be described later, or a red filter may be used instead of the blue filter). May be blue, and the second color filter may be red (see FIGS. 4A and 4B to be described later, the first color filter may be red, and the second color filter may be blue). Further, focus detection pixels (for example, RL pixels) corresponding to the first color filter are arranged in a plurality of areas included in the effective light receiving area of the image pickup device (see FIG. 5 for details), and are used for the second color filter. The corresponding focus detection pixels (for example, TB pixels) are arranged over the entire light receiving area of the image sensor (see FIG. 6 for details).

Next, reading of focus detection pixels will be described. Since the focus detection pixel has the following two characteristics that are different from those of a normal imaging pixel, a correction process is performed to ensure the image quality of a still image.
(I) Since the focus detection pixels are shielded from light by about 30% to 80% as compared with the normal imaging pixels, the amount of light is different from that of the imaging pixels (the amount of light is about 30% to 80%). Decrease)
(Ii) The phase is shifted when the out-of-focus subject image is incident on the image sensor surface. (The defocus amount is detected using this characteristic, but the phase shift poses a problem from the viewpoint of image quality. ).

  Basically, a correction process is required for the RL pixel and the TB pixel for the two different characteristics described above. Since this correction method is described in detail in JP-A-2013-257494 and JP-A-2013-106124, the description is omitted here.

  As described above, correction processing is performed on pixel data at the time of still image shooting, but mixed reading is performed at the time of reading out pixel data during live view display or moving image recording, and still image shooting is performed. The processing different from the case of is performed. This mixed reading will be described with reference to FIG.

  As described above, in the present embodiment, the TB pixel is arranged at the position of the Bb pixel in the case of an image sensor that does not include a focus detection pixel. At the time of the mixed reading, the TB pixel of the focus detection pixel is skipped and read. This is because Bb pixels have a high contribution as color information, but have a low contribution as luminance information. Therefore, even if some Bb pixel information is missing, the image quality is sufficiently established in the case of a moving image. It is. As described above, the RL pixel is subjected to the mixed reading, but the TB pixel is skipped during the pixel reading, so that the image quality of the moving image can be secured without performing a complicated correction process. In addition, since the RL pixels of the focus detection pixels are mixed and read out for the G pixel, the correction of the light amount (i) and the correction of the phase shift (ii) are performed as in the case of the still image. As for the correction of the light amount, the light amount is corrected using a light amount ratio between a mixed read output of G pixels not including RL pixels and a mixed read output of G pixels including RL pixels in a non-mixed region described later.

  The example shown in FIG. 3 shows a pixel block mixed with a pixel arrangement for explaining pixel mixing at the time of four-pixel mixed reading. Although the pixel arrangement shown in FIG. 3 is substantially the same as the pixel arrangement shown in FIG. 2C, the coordinate system is changed for convenience of explanation. The correspondence between the coordinates in FIG. 3A and FIG. 2C is as follows. The correspondence from FIG. 3A to FIG. 2C is indicated by →, (x1, y1) → (x9, y16), (x1, y16) → (x9, y1), (x8, y1) → (X16, y16), (x8, y16) → (x16, y1). The four-pixel mixed readout is a method in which a total of four pixels, that is, two pixels of the same color in the horizontal direction for two lines of the same color in the vertical direction are mixed and read. Note that the example shown in FIG. 3 shows a case where data is read from the lower line (pixel) in the figure.

  FIG. 3A shows a pixel arrangement in a partial area of the image sensor 21 for explaining the four-pixel mixed read operation. When the pixels of the image sensor 21 are divided into areas A, B, C, and D as shown in FIG. 3A, the arrangement of the image sensor that does not include a normal focus detection pixel in each area is as follows. , RL pixels and TB pixels are different.

  FIG. 3B shows the area A (the rectangular range surrounded by the positions (x1, y13), (x4, y13), (x1, y16), and (x4, y16)) in FIG. It is an enlarged view. In the case of performing the four-pixel mixed reading, the data of each RGB pixel (red, green, and blue pixels in the case of the Bayer array) for the pixels (pixel blocks) in this range is obtained by the following equations (1) to (4).

  As shown in FIG. 3B, the coordinates of the coordinates in each area are represented by Gr (0,0) at the pixel position at the lower left corner, and Rr (1,0), Gr in the horizontal order. (2, 0) and Rr (3, 0), and the upper part in the vertical direction is represented by Bb (0, 3). The method of obtaining the coordinates is the same in FIGS. 3 (c), 3 (d) and 3 (e).

Gr_mix1 = {Gr (0,0): R + Gr (2,0) + Gr (0,2) + Gr (2,2)} / 4 (1)
Rr_mix1 = {Rr (1,0) + Rr (3,0) + Rr (1,2) + Rr (3,2)} / 4 (2)
Gb_mix1 = {Gb (1,1) + Gb (3,1) + Gb (1,3) + Gb (3,3)} / 4 (3)
Bb_mix1 = {Bb (0,1) + Bb (0,3) + Bb (2,3)} / 3 (4)

  As can be seen from FIG. 3B, the Gr mixed pixel (green mixed pixel), the Gb mixed pixel (green mixed pixel), and the Rr mixed pixel (red mixed pixel) are pixel data of four pixels of the same color in the area A. Is added and divided by 4 to calculate a mixed value (see equations (1) to (3)). In this case, as shown in Expression (1), when obtaining the Gr mixed pixel data, the R pixel (position Gr (0, 0)) is also added. On the other hand, the Bb mixed pixel (blue mixed pixel) adds the pixel data of three pixels of the same color in the area A as shown in Expression (4), and makes the T pixel in Bb (2, 1) an addition target. Instead, the mixture value is calculated using only the other three same-color pixels.

  FIG. 3C shows an area B (a rectangular range surrounded by positions (x5, y13), (x8, y13), (x5, y16), and (x8, y16)) in FIG. It is an enlarged view. When performing mixed readout for pixels in this range, the data of each of the RGB pixels (red, green, and blue pixels in the case of the Bayer array) is obtained by the following equations (5) to (8).

Gr_mix2 = {Gr (0,0): R + Gr (2,0) + Gr (0,2) + Gr (2,2)} / 4 (5)
Rr_mix2 = {Rr (1,0) + Rr (3,0) + Rr (1,2) + Rr (3,2)} / 4 (6)
Gb_mix2 = {Gb (1,1) + Gb (3,1) + Gb (1,3) + Gb (3,3)} / 4 (7)
Bb_mix2 = {Bb (0,1) + Bb (0,3) + Bb (2,3)} / 3 (8)

  FIG. 3D shows an area C (a rectangular range surrounded by positions (x9, y13), (x12, y13), (x9, y16), and (x12, y16)) in FIG. It is an enlarged view. When performing mixed readout for pixels in this range, the data of each RGB pixel (red, green, and blue pixels in the case of the Bayer array) is obtained by the following equations (9) to (12).

Gr_mix3 = {Gr (0,0): R + Gr (2,0) + Gr (0,2) + Gr (2,2)} / 4 (9)
Rr_mix3 = {Rr (1,0) + Rr (3,0) + Rr (1,2) + Rr (3,2)} / 4 (10)
Gb_mix3 = {Gb (1,1) + Gb (3,1) + Gb (1,3) + Gb (3,3)} / 4 (11)
Bb_mix3 = {Bb (0,1) + Bb (0,3) + Bb (2,1)} / 3 (12)

  FIG. 3E shows an area D (a rectangular range surrounded by positions (x13, y13), (x16, y13), (x13, y16), and (x16, y16)) in FIG. It is an enlarged view. When performing mixed readout for pixels in this range, the data of each RGB pixel (red, green, and blue pixels in the case of the Bayer array) is obtained by the following equations (13) to (16).

Gr_mix4 = {Gr (0,0): R + Gr (2,0) + Gr (0,2) + Gr (2,2)} / 4 (13)
Rr_mix4 = {Rr (1,0) + Rr (3,0) + Rr (1,2) + Rr (3,2)} / 4 (14)
Gb_mix4 = {Gb (1,1) + Gb (3,1) + Gb (1,3) + Gb (3,3)} / 4 (15)
Bb_mix4 = {Bb (0,1) + Bb (0,3) + Bb (2,1)} / 3 (16)

  3 (c) to 3 (e) are the same as FIG. 3 (b), and include a Gr mixed pixel (green mixed pixel), a Gb mixed pixel (green mixed pixel), and an Rr mixed pixel (red mixed pixel). Calculates a mixed value by adding pixel data of four pixels of the same color in the areas B to D and dividing by four. In this case, when obtaining the Gr mixed pixel data, the R pixel (position Gr (0, 0)) is also to be added (see equations (5), (9), and (13)). On the other hand, the Bb mixed pixel (blue mixed pixel) adds three pixel data of the same color in the area B, and does not add the B pixel in Bb (2, 1) to the other three same color pixels. Only the mixture value is calculated (see equation (8)). Further, in the areas C and D, the Bb mixed pixel (blue mixed pixel) adds the pixel data of three pixels of the same color, and does not add the T pixel or the B pixel in Bb (2, 3) to the addition target. The mixture value is calculated using only the other three same-color pixels (see equations (12) and (16)).

  Next, a modified example in which the RL pixel is arranged at the position of the Rr pixel and the TB pixel is arranged at the position of the Bb pixel will be described with reference to FIGS. 4A and 4B. In the present embodiment shown in FIGS. 1 to 3, the RL pixel is arranged at the position of the G pixel, and the TB pixel is arranged at the position of the Bb pixel. However, other than this arrangement, for example, it is also possible to arrange the RL pixel at the position of the Rr pixel and arrange the TB pixel at the position of the Bb pixel. In this case, the image quality can be ensured by skipping the focus detection pixels (RL pixels and TB pixels) during the mixed reading of both the RL pixels and the TB pixels. In this case, it is not necessary to set a gap between islands in the RL pixel described later with reference to FIG.

  FIG. 4A shows a pixel arrangement in the image sensor 21, in which the RL pixel is arranged at the position of the Rr pixel and the TB pixel is arranged at the position of the Bb pixel. The pixel arrangement in FIG. 4A is different from the pixel arrangement in FIG. 3A in that the RL pixel is moved to the right (in the x direction) by one pixel to be positioned at the Rr pixel. The example shown in FIG. 4B shows a pixel block for pixel mixture at the time of 6-pixel mixture readout. The 6-pixel mixed readout is a method of mixing and reading a total of 6 pixels, that is, 3 lines of the same color in the horizontal direction for 2 lines in 3 lines of the same color in the adjacent vertical direction. For the explanation of the six-pixel mixed reading, the pixels of the image sensor 21 are divided into areas E, F, G, and H as shown in FIG. 4A, and the enlarged views of each are shown in FIGS. 4B (a), (b), and (c). (D). Each area is 6 × 6 pixels, and focus detection pixels (RL pixels and TB pixels) are not arranged at the position of the G pixel, but the RL pixel is located at a part of the Rr pixel and one of the Bb pixels. TB pixels are arranged at the positions of the sections.

  FIG. 4B (a) is an enlarged view of an area E (a rectangular area surrounded by positions (x1, y1), (x6, y1), (x1, y6), and (x6, y6)) in FIG. 4A. is there. For example, in a case where six pixels are mixed and read within a thick line frame for pixels in this range, the data of each RGB pixel (red, green, and blue pixels in the Bayer array) is expressed by the following equations (17) to (19). ). In the six-pixel mixed reading, pixels of two lines out of three adjacent lines of the same color are mixed, but pixel values of the same color of the remaining one line are not mixed (not read).

  As shown in FIG. 4B (a), the coordinates of the coordinates in each area are represented by Gr (0,0) at the pixel position at the lower left corner, and Rr (1,0), Gr in the horizontal right direction. (2,0), Rr (3,0), Gr (4,0), Rr (5,0), and the upper part in the vertical direction is represented by Bb (0,5). Note that the way of obtaining the coordinates is the same in FIGS. 4B (b), (c), and (d).

Gr_mix5 = {Gr (0,0) + Gr (2,0) + Gr (4,0) + Gr (0,2) + Gr (2,2) + Gr (4,2)} / 6 (17)
In the case of the 6-pixel mixed reading, the pixel values of the Gr pixels on the line of Y = 4 are not mixed.
Rr_mix5 = {Rr (3,0) + Rr (1,2) + Rr (3,2) + Rr (5,2)} / 4 (18)
In the case of the 6-pixel mixed reading, the pixel values of the Rr pixels in the line of Y = 4 are not mixed.
Bb_mix5 = {Bb (0,3) + Bb (2,3) + Bb (4,3) + Bb (0,5) + Bb (4,5)} / 5 (19)
In the case of the 6-pixel mixed reading, the pixel values of the Bb pixels in the line of Y = 1 are not mixed.

  As can be seen from Expression (17), the Gr mixed pixel (green mixed pixel) calculates a mixed value by adding pixel data of six G pixels of the same color (G) in the area E and dividing by 6. are doing. On the other hand, the Rr mixed pixel (red mixed pixel) is obtained by adding the pixel data of the four Rr pixels in the thick frame in the area E and dividing by 4, as shown in Expression (18). The Rr mixture value is calculated. In the case of an ordinary Bayer array image pickup device that does not include a focus detection pixel, Rr pixels are also arranged in Rr (1,0) and Rr (5,0), and their pixel values are also mixed. , The mixing is skipped because the R pixels are arranged.

  The Bb mixed pixel (blue mixed pixel) is obtained by adding the pixel data of the five Bb pixels in the thick frame in the area E and dividing by 5 as shown in Expression (19). Is calculated. In the case of an ordinary Bayer array image pickup device that does not include focus detection pixels, Bb pixels are also arranged in Bb (2, 5) and their pixel values are mixed, but a T pixel is arranged in this position. It has been skipped for mixing.

  FIG. 4B (b) is an enlarged view of an area F (a rectangular range surrounded by positions (x7, y1), (x12, y1), (x7, y6), (x12, y6)) in FIG. 4A. is there. For example, in a case where six pixels are mixed and read in a thick line frame for pixels in this range, the data of each RGB pixel (red, green, and blue pixels in the Bayer array) is expressed by the following equations (20) to (22). ).

Gr_mix6 = {Gr (0,0) + Gr (2,0) + Gr (4,0) + Gr (0,2) + Gr (2,2) + Gr (4,2)} / 6 (20)
In the case of the 6-pixel mixed reading, the pixel values of the Gr pixels on the line of Y = 4 are not mixed.
Rr_mix6 = {Rr (1,0) + Rr (5,0) + Rr (1,2) + Rr (3,2) + Rr (5,2)} / 5 (21)
Bb_mix6 = {Bb (0,3) + Bb (2,3) + Bb (2,5) + Bb (4,5)} / 4 (22)
In the case of the 6-pixel mixed reading, the pixel values of the Bb pixels in the line of Y = 1 are not mixed.

  As can be seen from Expression (20), also in the area F, the Gr mixed pixel (green mixed pixel) is obtained by adding pixel data of six G pixels of the same color (G) in the area F and dividing by 6. , And the mixture value is calculated. On the other hand, the Rr mixed pixel (red mixed pixel) is obtained by adding the pixel data of the five Rr pixels in the thick frame in the area F and dividing by 5, as shown in Expression (21). The Rr mixture value is calculated. In the case of a normal Bayer array imaging device that does not include focus detection pixels, Rr pixels are also arranged in Rr (3,0) and their pixel values are mixed, but R pixels are arranged in this position. It has been skipped for mixing.

  The Bb mixed pixel (blue mixed pixel) is obtained by adding the pixel data of the four Bb pixels in the thick frame in the area F and dividing by 4 as shown in Expression (22). Is calculated. In the case of an ordinary Bayer array imaging device that does not include a focus detection pixel, Bb pixels are also arranged in Bb (4,3) and Bb (0,5), and the pixel values of the pixels are also mixed. Since T pixels or B pixels are arranged at this position, the mixing is skipped.

  FIG. 4B (c) is an enlarged view of an area G (a rectangular range surrounded by positions (x13, y1), (x18, y1), (x13, y6), and (x18, y6)) in FIG. 4A. is there. For example, in a case where six pixels are read out in a bold frame for pixels in this range, the data of each of the RGB pixels (red, green, and blue pixels in the Bayer array) are expressed by the following equations (23) to (25). ).

Gr_mix7 = {Gr (0,0) + Gr (2,0) + Gr (4,0) + Gr (0,2) + Gr (2,2) + Gr (4,2)} / 6 (23)
In the case of the 6-pixel mixed reading, the pixel values of the Gr pixels on the line of Y = 4 are not mixed.
Rr_mix7 = {Rr (3,0) + Rr (1,2) + Rr (3,2) + Rr (5,2)} / 4 (24)
In the case of the 6-pixel mixed reading, the pixel values of the Rr pixels in the line of Y = 4 are not mixed.
Bb_mix7 = {Bb (0,3) + Bb (4,3) + Bb (0,5) + Bb (2,5) + Bb (4,5)} / 5 (25)
In the case of the 6-pixel mixed reading, the pixel values of the Bb pixels in the line of Y = 1 are not mixed.

  As can be seen from Expression (23), also in the area G, the Gr mixed pixel (green mixed pixel) is obtained by adding pixel data of six G pixels of the same color (G) in the area G and dividing by 6. , And the mixture value is calculated. On the other hand, the Rr mixed pixel (red mixed pixel) is obtained by adding the pixel data of the four Rr pixels in the thick frame in the area G and dividing by 4, as shown in Expression (24). The Rr mixture value is calculated. In the case of an ordinary Bayer array image pickup device that does not include a focus detection pixel, Rr pixels are also arranged in Rr (1,0) and Rr (5,0), and their pixel values are also mixed. , The mixing is skipped because the R pixels are arranged.

  The Bb mixed pixel (blue mixed pixel) is obtained by adding the pixel data of the five Bb pixels in the thick frame in the area F and dividing by 5, as shown in Expression (25). Is calculated. In the case of a normal Bayer array imaging device that does not include focus detection pixels, Bb pixels are also arranged in Bb (2, 3) and their pixel values are mixed, but B pixels are arranged in this position. It has been skipped for mixing.

  FIG. 4B (d) is an enlarged view of an area H (a rectangular area surrounded by positions (x19, y1), (x24, y1), (x19, y6), and (x24, y6)) in FIG. 4A. is there. For example, in the case of performing the mixed readout of six pixels in a thick line frame for the pixels in this range, the data of each RGB pixel (red, green, and blue pixels in the case of the Bayer array) is expressed by the following equations (26) to (28). Find more.

Gr_mix8 = {Gr (0,0) + Gr (2,0) + Gr (4,0) + Gr (0,2) + Gr (2,2) + Gr (4,2)} / 6 (26)
In the case of the 6-pixel mixed reading, the pixel values of the Gr pixels on the line of Y = 4 are not mixed.
Rr_mix8 = {Rr (1,0) + Rr (5,0) + Rr (1,2) + Rr (3,2) + Rr (5,2)} / 5 (27)
In the case of the 6-pixel mixed reading, the pixel values of the Rr pixels in the line of Y = 4 are not mixed.
Bb_mix8 = {Bb (0,3) + Bb (2,3) + Bb (4,3) + Bb (2,5)} / 4 (28)
In the case of the 6-pixel mixed reading, the pixel values of the Bb pixels in the line of Y = 1 are not mixed.

  As can be seen from Expression (26), also in the area H, the Gr mixed pixel (green mixed pixel) is obtained by adding the pixel data of the six G pixels of the same color (G) in the area H and dividing by 6. , And the mixture value is calculated. On the other hand, the Rr mixed pixel (red mixed pixel) is obtained by adding the pixel data of the five Rr pixels in the thick frame in the area H and dividing by 4, as shown in Expression (27). The Rr mixture value is calculated. In the case of an ordinary Bayer array imaging device that does not include focus detection pixels, Rr pixels are also arranged in Rr (3,0) and their pixel values are mixed, but B pixels are arranged in this position. It has been skipped for mixing.

  The Bb mixed pixel (blue mixed pixel) is obtained by adding the pixel data of five Bb pixels in the thick frame in the area H and dividing by 4 as shown in Expression (28). Is calculated. In a normal Bayer array imaging device that does not include focus detection pixels, Bb pixels are also arranged in Bb (0,5) and Bb (4,5), and their pixel values are also mixed. , T pixels or B pixels are arranged, so that the mixing is skipped.

  Next, with reference to FIG. 5, the islanding in which the RL pixel does not exist in the Gr pixel will be described. In the present embodiment, the RL pixels are arranged at some positions of the G pixels. However, an area where no RL pixels exist is set in a predetermined area in the image sensor 21 (this is called an island). Called). That is, RL pixels and TB pixels are arranged in the area 21a of FIG. 5A, RL pixels are not arranged in the area 21b, and only TB pixels are arranged as focus detection pixels (the area G pixels are arranged in the area 21a and the area 21b).

  When capturing a still image, the pixel data of the RL pixel disposed at the position of the G pixel needs to be corrected as described above. For example, as described in JP-A-2013-257494, a light amount ratio between a normal imaging pixel and an R pixel or an L pixel is calculated, and a correction process is performed using the light amount ratio. Even in the mixed reading, in order to calculate and correct this light amount ratio, it is necessary to read without mixing the RL pixels even in the mixed reading state and generate information in which only the pixel values of the imaging pixels are mixed. . Therefore, it is necessary to provide an area in which pixel mixture readout can be performed without mixing pixel values of focus detection pixels.

  For this reason, in the mixed reading, the non-mixed area 21b in which the RL pixel does not exist is provided, and the pixel mixed reading can be performed without mixing the RL pixels. The number of pixels corresponding to the width of the non-mixed area 21b in the vertical direction in FIG. 5A requires at least the number of vertical lines necessary for 2 × pixel mixing. For example, in the case of an image sensor capable of performing 6-pixel mixed reading, as described above, a total of 6 pixels of three horizontal same-color pixels of two adjacent vertical same-color three lines are mixed. To perform a 6-pixel mixed read. Therefore, since the number of pixels corresponding to the width of the non-mixed area 21b is 2 × 3 (line) = 6 pixels, 6 or more pixels are required.

  Similarly, in the case of an image sensor capable of performing mixed readout of 10 pixels, the number of pixels of the non-mixed area 21b needs to be 10 or more. The 10-pixel mixed readout method is a method of mixing and reading out a total of 10 pixels of 5 pixels of the same color in the horizontal direction for 2 lines out of 5 lines of the same color in the adjacent vertical direction. Accordingly, since the number of pixels corresponding to the width of the non-mixed area 21b is 2 × 5 (lines) = 10 pixels, 10 or more pixels are required. In addition, the non-mixed area 21b requires at least three places at the center and above and below the center to properly correct the overlap of the center of the image. The larger the number, the higher the image quality. Can be dignified.

  Here, for example, if the TB pixel is arranged at the position of the G pixel in the same manner as the RL pixel, even if an attempt is made to set a gap area where no focus detection pixel exists, the TB pixel cannot be arranged. This is because the TB pixels need to be arranged long in the vertical direction in order to detect the phase difference in the vertical direction (see the area 21c in FIG. 5B), so that the islands for the TB pixels cannot be arranged. . In other words, the region where the region 21c and the region 21b intersect in FIG. 5B means that the TB pixels need to be arranged for the above-described reason, and cannot be the regions where the focus detection pixels are not arranged. Then, it is not possible to appropriately perform the correction processing on the focus detection pixels around the intersecting region, and there is a problem that the image quality is deteriorated.

  However, in the present embodiment, as described with reference to FIG. 3, the TB pixels are arranged at the positions of some of the Bb pixels, and the TB pixels do not exist at the positions of the G pixels. . Therefore, only the RL pixels are made to correspond to the positions of the G pixels (see the area 21a), and an area where the RL pixels are not made (see the non-mixed area 21b) is provided, thereby enabling islanding and arranging the G pixels. RL pixels can be corrected.

  As described above, in the present embodiment, the focus detection pixels corresponding to the first color filter (for example, the green filter) are arranged in a plurality of areas included in the effective area of the image sensor, and are provided between the plurality of areas. The area of the area where the focus detection pixels corresponding to the first color filter are not arranged in the area of, and the width of the area where the focus detection pixels corresponding to the first color filter are not arranged is the width of the pixel line necessary for pixel mixture readout. It is made to correspond to more than twice the number.

  Next, with reference to FIG. 6, mixed reading in a region where no TB pixel exists will be described. In FIG. 6, a region 21d is a region where normal imaging pixels are arranged. The imaging pixels are arranged over the entire area (area 21d) of the imaging area (within the Bayer pixel) of the imaging element 21. The TB pixels are arranged in a range (for example, 80% × 80%) of a part of the region 21e in the region 21d. Therefore, in FIG. 6, both the imaging pixel and the TB pixel are arranged in the area 21e, and only the imaging pixel is arranged in the area 21d.

  Since the TB pixels are arranged only in the region 21e, the skip reading of the TB pixels at the time of the mixed reading only needs to be performed only in the region 21e, and the skip of the TB pixels is performed in the region of the region 21d except the region 21e. There is no need to read. However, in the present embodiment, the skip reading of the TB pixel is performed also in the area other than the area 21e in the area 21d. In other words, the mixed operation of the mixed reading (Equations (2), (4), (6), (8), etc.) described with reference to FIG. 3 is performed not only in the area 21e but also in the area 21d except for the area 21e. The same calculation method is used in the region where the calculation is performed. This is to ensure the image quality performance by unifying all the reading methods in the screen and not providing a boundary line.

  Next, a digital camera incorporating the image sensor 21 according to the present embodiment will be described with reference to FIG. FIG. 7 is a block diagram mainly showing an electrical configuration of the camera according to the embodiment of the present invention. The camera according to the present embodiment includes an interchangeable lens barrel 10 and a camera body 20. In the present embodiment, the interchangeable lens barrel 10 and the camera body 20 are configured separately, but it is needless to say that the interchangeable lens barrel 10 and the camera body 20 may be configured integrally as a general compact camera.

  A photographic lens 11 is arranged in the interchangeable lens barrel 10. The photographing lens 11 includes a plurality of optical lenses for forming an optical image of the subject S. An actuator 12 and a lens control unit 13 are provided in the interchangeable lens barrel 10. The lens control unit 13 receives the defocus direction and the defocus amount from the AF calculation unit 23 in the camera body 20, and controls the actuator 12 based on the information. The actuator 12 moves the photographing lens 11 in the optical axis direction to perform focusing.

  In the camera body 20, an imaging element 21, an image processing unit 22, an AF calculation unit 23, a recording unit 24, and a display unit 25 are provided.

  The image sensor 21 is arranged on the optical axis of the photographing lens 11 and near the image forming position of the subject image. The imaging device 21 includes a plurality of pixels having a photoelectric conversion unit that converts a subject image (optical image) into an electric signal. That is, in the image sensor 21, the photodiodes constituting each pixel are two-dimensionally arranged in a matrix, and each photodiode generates a photoelectric conversion current according to the amount of received light. Charge is stored by a capacitor connected to the diode. In front of each pixel, a Bayer array RGB filter is arranged. These multiple photodiodes correspond to the multiple pixels described above.

  Further, as described with reference to FIGS. 1 to 4A and 4B, the plurality of pixels of the image sensor 21 are focus detection pixels (RL pixels, TB, TB) configured to limit the incident direction of a light beam incident on the pixels. Pixel) and an imaging pixel configured such that a light beam incident on the pixel is not restricted more than the focus detection pixel. The imaging element 21 outputs pixel values output from the focus detection pixel and the imaging pixel to the image processing unit 22 and the AF calculation unit 23.

  The image processing unit 22 inputs a mixed pixel value from an imaging pixel and a focus detection pixel (but does not include a TB pixel and only an RL pixel), and outputs an image for a live view display image and an image for a moving image recording image. Perform processing. Further, pixel values from the imaging pixel and the focus detection pixel are input, and image processing for recording a still image is performed. Further, the image processing unit 22 outputs the image data processed for recording to the recording unit 24, and outputs the image data processed for live view display to the display unit 25.

  The recording unit 24 includes an electrically rewritable nonvolatile memory, and inputs and records image data for recording. The display unit 25 inputs image data for live view display and image data for reproduction, and displays a live view image and a reproduced image based on the image data on a display panel such as an LCD or an organic EL.

  The AF calculation unit 23 receives a pixel value from a focus detection pixel (RL pixel, TB pixel) among the pixel values, and calculates a defocus direction and a defocus amount by a phase difference AF method.

  Note that the pixel values may be mixed at the time of reading from the image sensor 21, and the image processing unit 22 or the AF operation unit 23 may use the pixel values read from the image sensor 21 to perform digital operation. May be used to perform pixel mixing.

  In the present embodiment, a function of a pixel mixing and reading unit that mixes and reads pixel signals is provided by cooperation of any one or any of the imaging element 21, the image processing unit 22, the AF calculation unit 23, and the like. . The pixel mixture readout unit mixes and reads out the outputs of the imaging pixels corresponding to the second color filter (for example, when mixing and reading out the Bb pixels or Rr pixels), the second focus detection pixel (For example, T pixels and B pixels) are read out without mixing (for example, see FIGS. 3, 4A, and 4B). In addition, when mixing and reading the outputs of the imaging pixels corresponding to the first color filter (for example, G), the pixel mixture readout unit outputs the first focus detection pixels (for example, RL pixels). And read them out.

  Next, the operation of the camera will be described using the flowchart shown in FIG. This flowchart is executed by controlling each unit in the camera by a control unit such as a CPU (Central Processor Unit) provided in the image processing unit 22 or the AF calculation unit 23 according to a program stored in a nonvolatile memory. Is done.

  When the power of the camera is turned on, the flowchart shown in FIG. 8 starts. When the process starts, first, a through image (image data for live view display) is captured (S1). Here, as described with reference to FIG. 3 or FIG. 4, mixed readout is performed from the normal image pickup pixel and the focus detection pixel (excluding the TB pixel) from the image pickup device 1.

  After capturing the through image, AF pixel correction processing is performed next (S3). Here, the correction process is performed on the RL pixels of the focus detection pixels. That is, since the aperture of the focus detection pixel is restricted, the pixel value becomes small. Correction is performed so that the level is substantially the same as that of a normal imaging pixel. Since the output value of the RL pixel as the focus detection pixel is included in the pixel mixed output value of the G pixel, the mixed pixel output value of the G pixel in the non-mixed pixel area 21b shown in FIG. (There is no mixing) and a correction process using correction data based on an image height position according to the optical characteristics of the photographing lens 11 and the like.

  After performing the AF pixel correction processing, a live view display (live view display) is performed next (S5). Here, the image processing unit 22 displays a through image on the display unit 25 using the image data read in step S1 and corrected in step S3.

  After displaying the through image, it is determined whether or not the first release is pressed down (S7). Here, whether the release button is half-pressed, that is, whether or not the first release is pressed down, is determined based on the state of a switch that is turned on / off by the half-press operation of the release button. If the result of this determination is that the first release has not been depressed, processing returns to step S1.

  On the other hand, if the result of determination in step S7 is that the first release has been depressed, exposure for AF is performed (S9). Here, exposure control is performed so that the focus detection pixels (RL pixels and TB pixels) are properly exposed, and the sum of the pixel values of the RL pixels and TB pixels (see FIG. 1) is read from the image sensor 21.

  After the AF exposure, the amount of defocus is detected (S11). Here, the photographing lens 11 is calculated using the sum value of the pixel values of the focus detection pixels (RL pixel and TB pixel) (the sum value of the output of the R pixel in Rsumarea, the sum value of the output of the L pixel in Lsumarea). The defocus direction (defocus direction) and the defocus amount (defocus amount) are calculated.

  After detecting the amount of defocus, it is next determined whether or not the camera is in focus (S13). Here, the determination is made based on whether or not the defocus amount calculated in step S11 falls within a predetermined range (a range in which focusing can be considered).

  If the result of determination in step S13 is that the camera is not in focus, focus lens driving is performed (S15). Here, the lens controller 13 moves the photographing lens 11 to the in-focus position via the actuator 12 based on the focus shift amount and the focus shift direction calculated in step S11. When the focus lens is driven, the process returns to step S1.

  If the result of determination in step S13 is that the camera is in focus, it is determined whether or not the second release is pressed down (S17). The photographer observes the through image and, when judging that it is a photo opportunity, fully presses the release button, that is, depresses the second release. Therefore, in this step, the determination is made based on the state of the switch which is turned on / off by the full-press operation of the release button. If the result of this determination is that the second release has not been depressed, processing returns to step S1.

  If the result of determination in step S17 is that the second release is pressed down, main exposure is performed (S19). Here, the exposure operation by the image sensor 21 is performed according to a predetermined exposure control value. When this exposure is completed, the pixel values of all the pixels (normal imaging pixels and focus detection pixels) are read from the imaging element 21, and the image processing unit 22 generates image data of a still image. In generating the image data, the pixel values from the focus detection pixels are subjected to a correction process. When the image data is generated, it is recorded in the recording unit 24.

  When the main exposure is completed, it is next determined whether or not the camera is turned off (S21). Here, the determination is made based on the state of the power switch of the camera. If the result of this determination is not power off, processing returns to step S1. On the other hand, when the power is off, after the termination processing, the power is turned off.

  As described above, the image sensor according to one embodiment of the present invention includes a plurality of imaging pixels and a plurality of focus detection pixels in which the position of the opening of the light receiving unit is shifted from the imaging pixels. Then, first focus detection pixels (for example, RL pixels) whose openings are shifted in the first direction are arranged at a first pixel pitch, and openings are formed in a second direction different from the first direction. The shifted second focus detection pixels (for example, TB pixels) are arranged at a second pixel pitch. For this reason, even if pixel rows with upper and lower apertures are arranged in addition to the pixel rows with left and right apertures for focus detection, it is possible to achieve both image quality and AF performance during a moving image in which mixed reading is performed.

  In one embodiment of the present invention, the first direction and the second direction are orthogonal to each other, and the first pixel pitch and the second pixel pitch are equal (for example, 4 pixel pitches). The arrangement of the second focus detection pixels (for example, TB pixels) matches the arrangement of the first focus detection pixels (for example, RL pixels) rotated by 90 degrees (see FIG. 3). ). By arranging such focus detection pixels, it is possible to make AF performance such as focus detection accuracy in the first direction and the second direction equal.

  In the image pickup device according to the embodiment of the present invention, the pixel pitch is four pixel pitches, but is not limited thereto. Although the pixels are arranged along two orthogonal directions, the present invention is not limited to this. The arrangement of the RL pixel and the TB pixel to the RGB pixels is not limited to the illustrated example.

  Further, in one embodiment of the present invention, a digital camera has been described as a device for shooting, but the camera may be a digital single-lens reflex camera or a compact digital camera, such as a video camera or a movie camera. A camera for moving images may be used, and a camera built in a mobile phone, a smartphone, a personal digital assistant (PDA), a personal computer (PC), a tablet computer, a game machine, or the like may be used. In any case, any device may be used as long as the device incorporates the image sensor.

  Further, among the techniques described in the present specification, the control mainly described in the flowchart is often settable by a program, and may be stored in a recording medium or a recording unit. The recording method of the recording medium and the recording unit may be recorded at the time of shipping the product, may use a distributed recording medium, or may be downloaded via the Internet.

  Further, even if the operation flow in the claims, the specification, and the drawings is described using words expressing the order of “first”, “next”, etc. for convenience, in places not particularly described, It does not mean that it is essential to carry out in this order.

  The present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the gist thereof at the stage of implementation. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the above embodiments. For example, some components of all the components shown in the embodiment may be deleted. Further, components of different embodiments may be appropriately combined.

DESCRIPTION OF SYMBOLS 10 ... Interchangeable lens barrel, 11 ... Photographing lens, 12 ... Actuator, 13 ... Lens control part, 20 ... Camera body, 21 ... Image sensor, 21a-21e ... Area, 22: Image processing unit, 23: AF calculation unit, 24: Recording unit, 25: Display unit

Claims (3)

In an image sensor having a plurality of imaging pixels and a plurality of focus detection pixels in which the position of the opening of the light receiving unit is shifted from the imaging pixels,
The position of the opening is shifted in the first direction, and the focus detection pixels of the first focus detection pixel group including a plurality of focus detection pixels having different positions of the openings correspond to the first color filters of the imaging pixels. At positions where the apertures are located at a first pixel pitch in the first direction for each focus detection pixel having a different position of the aperture,
The aperture is shifted in a second direction different from the first direction, and the focus detection pixels of a second focus detection pixel group including a plurality of focus detection pixels having different positions of the openings are replaced with the focus detection pixels of the imaging pixels. At a position corresponding to a second color filter having a different color from that of the first color filter, the focus detection pixels having different positions of the openings are disposed at a second pixel pitch in the second direction, respectively.
The first color filter is arranged at a position corresponding to the second color filter of the imaging pixel and at a position where the second focus detection pixel is arranged. element.   The focus detection pixels of the first focus detection pixel group having different positions of the openings are alternately arranged in the second direction at the first pixel pitch, and the positions of the openings of the second focus detection pixel group are arranged. The image pickup device according to claim 1, wherein focus detection pixels different from each other are alternately arranged at the second pixel pitch in the first direction. The first color filter in green, the imaging device according to claim 1 or 2, the second color filter is characterized by a blue or red.

Images