JP2010139624A - Imaging element and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To suppress image deterioration to the minimum by being detected up to a large defocusing amount. <P>SOLUTION: An imaging element for arranging at least one row of a focus detection pixel row for arranging first focus detection pixels 313 receiving one light flux and a second focus detection pixels 314 receiving the other light flux in one row out of a pair of light fluxes passing a pair of regions of an emission pupil of an optical system forming images on an imaging plane on a part of arrangement of an imaging pixel two-dimensionally arranged on the imaging plane is formed by dividing array of the first focus detection pixels 313 and the second focus detection pixels 314 into a plurality of the regions; alternately arranging the first focus detection pixels 313 and the second focus detection pixels 314 in at least one region in the plurality of the regions; and alternately arranging the second focus detection pixels 314 and one or a plurality of imaging pixels without arranging the first focus detection pixel 313 in at least another region. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging element and an imaging apparatus.

An image sensor is provided, which includes an image sensor in which an array of a plurality of focus detection pixels that generate a pair of image signals corresponding to a pair of images formed by a pair of light beams passing through an optical system are mixed in the array of imaging pixels. A focus detection function of a so-called pupil division phase difference detection system that generates an image signal based on the output of the pixel and detects the focus adjustment state of the optical system based on the shift amount of the pair of image signals generated by the focus detection pixel. An imaging device provided is known (see, for example, Patent Document 1).
In such an imaging apparatus, the focus detection pixel is configured by a microlens and a photoelectric conversion unit disposed behind the microlens, and a pair of image signals generated by the array of focus detection pixels is relatively shifted. The amount of deviation is calculated on the basis of the degree of correlation, and focus detection is performed by converting the amount of deviation into a defocus amount. Further, when generating image data, pixel data for imaging at the focus detection pixel position is obtained by interpolation based on pixel data of imaging pixels arranged around the focus detection pixel.
JP 2007-270312 A

  However, in order to detect a large defocus amount, the dift range of a pair of image signals must also be increased, and thus a large number of focus detection pixels must be arranged, and as a result, pixel data to be interpolated Therefore, the image quality tends to deteriorate in proportion to the number of interpolated pixel data.

(1) The invention of claim 1 has passed through a pair of regions of the exit pupil of the optical system that forms an image on the imaging surface, in a part of the array of imaging pixels arrayed two-dimensionally on the imaging surface An image sensor in which at least one focus detection pixel array in which a first focus detection pixel that receives one light beam and a second focus detection pixel that receives the other light beam in a pair is arranged in a line. The first focus detection pixels and the second focus detection pixels are divided into a plurality of regions, and the first focus detection pixels and the second focus detection pixels are alternately arranged in at least one of the plurality of regions. In the other at least one region, the second focus detection pixels and one or a plurality of imaging pixels are alternately arranged without arranging the first focus detection pixels.
(2) According to a second aspect of the present invention, in the imaging device according to the first aspect, an imaging pixel having a green filter, an imaging pixel having a red filter, and an imaging pixel having a blue filter are arranged in a Bayer array on the imaging surface. The first focus detection pixel is arranged at the position of the image pickup pixel having the red filter or the blue filter in the Bayer array, and the second focus detection pixel is set to the image pickup pixel having the green filter in the Bayer array. Placed in position.
(3) In the image pickup device according to claim 1 or 2, the first focus detection pixel and the second focus detection pixel each include a microlens and a photoelectric conversion unit.
(4) According to a fourth aspect of the present invention, in the imaging device according to any one of the first to third aspects, a second focus detection pixel and one or a plurality of imaging pixels in a plurality of regions are alternately arranged. This area is an area including the end of the focus detection pixel column.
(5) According to the invention of claim 5, in the imaging device according to any one of claims 1 to 4, the first focus detection pixels and the second focus detection pixels of the plurality of regions are alternately arranged. The region is a central region of the focus detection pixel row.
(6) The invention according to claim 6 is the second focus detection pixel for the first data string generated by the array of the imaging element according to any one of claims 1 to 5 and the first focus detection pixel. By shifting the second data string generated by the arrangement of the first and second positions, a shift amount calculating means for calculating the shift amount of the pair of images formed by the pair of light beams, and the shift amount between the imaging surface and the focusing surface of the optical system. Conversion means for converting to a defocus amount between them, and output data of the virtual imaging pixels at the positions of the first focus detection pixel and the second focus detection pixel, around the first focus detection pixel and the second focus detection pixel. Interpolation means for performing interpolation based on output data of the imaging pixels.
(7) According to a seventh aspect of the present invention, in the imaging apparatus according to the sixth aspect, the shift amount calculating means shifts the second data string with respect to the first data string by an arrangement pitch unit of the second focus detection pixels. To calculate the degree of correlation between the first data string and the second data string, and the relative shift amount between the first data string and the second data string according to the shift amount indicating the highest degree of correlation. To do.
(8) The invention according to claim 8 is the imaging apparatus according to claim 7, wherein the deviation amount calculating means is based on the degree of correlation between the shift amount showing the highest degree of correlation and the shift amount before and after the shift amount. The shift amount is interpolated by the following amount, and the shift amount between the first data string and the second data string is calculated based on the interpolated shift amount.
(9) According to a ninth aspect of the present invention, in the imaging apparatus according to any one of the sixth to eighth aspects, the interpolation means outputs the output data of the virtual imaging pixel at the position of the second focus detection pixel. When an image pickup pixel exists between the second focus detection pixel and the second focus detection pixel when interpolating with the output data of the image pickup pixels around the bifocal detection pixel, the output data of the image pickup pixel is also used. To interpolate.

  According to the present invention, it is possible to detect a large defocus amount and minimize the degree of image degradation in an imaging element in which pupil-divided focus detection pixels are mixed with imaging pixels.

  As an image pickup apparatus including the image pickup device according to the embodiment, a lens interchangeable digital still camera will be described as an example. FIG. 1 is a cross-sectional view of a camera showing the configuration of the camera of one embodiment. A digital still camera 201 according to an embodiment includes an interchangeable lens 202 and a camera body 203, and the interchangeable lens 202 is attached to the camera body 203 via a mount unit 204. An interchangeable lens 202 having various photographing optical systems can be attached to the camera body 203 via a mount unit 204.

  The interchangeable lens 202 includes a lens 209, a zooming lens 208, a focusing lens 210, an aperture 211, a lens drive control device 206, and the like. The lens drive control device 206 includes a microcomputer (not shown), a memory, a drive control circuit, and the like. The lens drive control device 206 includes drive control for adjusting the focus of the focusing lens 210 and the aperture diameter of the aperture 211, zooming lens 208, and focusing. In addition to detecting the state of the lens 210 and the aperture 211, the lens information is transmitted and the camera information is received through communication with a body drive control device 214 described later. The aperture 211 forms an aperture having a variable aperture diameter at the center of the optical axis in order to adjust the amount of light and the amount of blur.

  The camera body 203 includes an imaging element 212, a body drive control device 214, a liquid crystal display element drive circuit 215, a liquid crystal display element 216, an eyepiece lens 217, a memory card 219, and the like. In the imaging element 212, imaging pixels are two-dimensionally arranged, and focus detection pixels are incorporated in portions corresponding to focus detection positions. Details of the image sensor 212 will be described later.

  The body drive control device 214 includes a microcomputer, a memory, a drive control circuit, and the like. The adjustment is repeated, and image signal processing and recording, camera operation control, and the like are performed. The body drive control device 214 communicates with the lens drive control device 206 via the electrical contact 213 to receive lens information and send camera information (defocus amount, aperture value, etc.).

  The liquid crystal display element 216 functions as an electric view finder (EVF). The liquid crystal display element driving circuit 215 displays a through image by the imaging element 212 on the liquid crystal display element 216, and the photographer can observe the through image through the eyepiece lens 217. The memory card 219 is an image storage that stores an image captured by the image sensor 212.

  A subject image is formed on the light receiving surface of the image sensor 212 by the light beam that has passed through the interchangeable lens 202. This subject image is photoelectrically converted by the image sensor 212, and an image signal and a focus detection signal are sent to the body drive control device 214.

  The body drive control device 214 calculates the defocus amount based on the focus detection signal from the focus detection pixel of the image sensor 212 and sends the defocus amount to the lens drive control device 206. The body drive control device 214 processes the image signal from the image sensor 212 to generate an image, stores the image in the memory card 219, and sends the through image signal from the image sensor 212 to the liquid crystal display element drive circuit 215. The through image is displayed on the liquid crystal display element 216. Further, the body drive control device 214 sends aperture control information to the lens drive control device 206 to control the aperture of the aperture 211.

  The lens drive controller 206 updates the lens information according to the focusing state, zooming state, aperture setting state, aperture opening F value, and the like. Specifically, the positions of the zooming lens 208 and the focusing lens 210 and the aperture value of the aperture 211 are detected, and lens information is calculated according to these lens positions and aperture values, or a lookup prepared in advance. Lens information corresponding to the lens position and aperture value is selected from the table.

  The lens drive control device 206 calculates a lens drive amount based on the received defocus amount, and drives the focusing lens 210 to the in-focus position according to the lens drive amount. Further, the lens drive control device 206 drives the diaphragm 211 in accordance with the received diaphragm value.

  FIG. 2 is a diagram showing a focus detection position on the photographing screen of the interchangeable lens 202, and a region (focus detection) in which a focus detection pixel array on the image sensor 212 described later samples an image on the photographing screen when focus detection is performed. An example of an area and a focus detection position is shown. In this example, focus detection areas 101 to 103 are arranged at the center and three locations above and below the rectangular shooting screen 100. Focus detection pixels are linearly arranged in the longitudinal direction of the focus detection area indicated by a rectangle. Note that, here, the focus detection areas 101 to 103 are rectangular extending in the vertical direction in the figure, but are not limited thereto. You may make it the rectangle extended in the left-right direction or the diagonal direction.

  FIG. 3 is a front view showing a detailed configuration of the image sensor 212, and shows an enlarged vicinity of the focus detection area 101 on the image sensor 212. Since the focus detection pixel array is long, the vicinity of the upper end, the vicinity of the center, and the vicinity of the lower end of the array are shown separately. In the image sensor 212, the imaging pixels 310 are densely arranged in a two-dimensional square lattice pattern (Bayer array including green pixels, red pixels, and blue pixels), and a position for focus detection is provided at a position corresponding to the focus detection area 101. The focus detection pixels 313 and 314 are alternately arranged on a straight line in the vertical direction. However, only the focus detection pixel 314 is arranged in the vicinity of the upper end and the lower end of the focus detection pixel array, and the imaging pixel 310 (blue pixel) is arranged at the pixel position where the focus detection pixel 313 is to be arranged. The focus detection pixels 313 and 314 are arranged at positions where blue pixels and green pixels of the imaging pixels are to be arranged, respectively. Although not shown, the configuration in the vicinity of the focus detection areas 102 and 103 is the same as the configuration shown in FIG.

  As illustrated in FIG. 4, the imaging pixel 310 includes a microlens 10, a photoelectric conversion unit 11, and a color filter (not shown). There are three types of color filters, red (R), green (G), and blue (B), and the respective spectral sensitivities have the characteristics shown in FIG. In the image pickup device 212, image pickup pixels 310 having respective color filters are arranged in a Bayer array.

  As shown in FIG. 5A, the focus detection pixel 313 includes the microlens 10 and the photoelectric conversion unit 13, and the photoelectric conversion unit 13 has a rectangular shape. Further, the focus detection pixel 314 includes the microlens 10 and the photoelectric conversion unit 14 as illustrated in FIG. 5B, and the photoelectric conversion unit 14 has a rectangular shape. When the focus detection pixel 313 and the focus detection pixel 314 are displayed with the microlens 10 superimposed, the photoelectric conversion units 13 and 14 are arranged in the vertical direction.

  The focus detection pixels 313 and 314 are not provided with a color filter to increase the amount of light, and the spectral characteristics of the focus detection pixels 313 and 314 include the spectral sensitivity of a photodiode that performs photoelectric conversion and the spectral characteristics of an infrared cut filter (not shown). Spectral characteristics (see FIG. 7). That is, the spectral characteristics are obtained by adding the spectral characteristics of the green pixel, the red pixel, and the blue pixel shown in FIG. 6, and the light wavelength region of the sensitivity includes the light wavelength regions of the sensitivity of the green pixel, the red pixel, and the blue pixel. ing.

  The focus detection pixels 313 and 314 for focus detection are arranged in a column where B and G of the imaging pixel 310 should be arranged. The focus detection pixels 313 and 314 for focus detection are arranged in a column in which B and G of the imaging pixel 310 are to be arranged because interpolation processing for obtaining an image signal for imaging at the position of the focus detection pixel is performed. This is because the interpolation error of the blue pixel is less noticeable than the interpolation error of the red pixel due to human visual characteristics when an interpolation error occurs in FIG.

  The photoelectric conversion unit 11 of the imaging pixel 310 is designed so as to receive all the light beams that pass through the exit pupil diameter (for example, F1.0) of the brightest interchangeable lens by the microlens 10. In addition, the photoelectric conversion units 13 and 14 of the focus detection pixels 313 and 314 are designed in such a shape that the microlens 10 receives light beams that pass through a pair of predetermined regions of the exit pupil of the interchangeable lens.

  FIG. 8 is a cross-sectional view of the imaging pixel 310. In the imaging pixel 310, the microlens 10 is disposed in front of the photoelectric conversion unit 11 for imaging, and the shape of the photoelectric conversion unit 11 is projected forward by the microlens 10. The photoelectric conversion unit 11 is formed on the semiconductor circuit substrate 29. A color filter (not shown) is arranged between the microlens 10 and the photoelectric conversion unit 11.

  FIG. 9A is a cross-sectional view of the focus detection pixel 313. In the focus detection pixel 313 disposed in the focus detection area 101 at the center of the screen, the microlens 10 is disposed in front of the photoelectric conversion unit 13, and the shape of the photoelectric conversion unit 13 is projected forward by the microlens 10. The photoelectric conversion unit 13 is formed on the semiconductor circuit substrate 29, and the microlens 10 is integrally and fixedly formed thereon by a semiconductor image sensor manufacturing process. Note that the cross-sectional structure of the focus detection pixels 313 arranged in the focus detection areas 102 and 103 at the top and bottom of the screen is the same as the cross-sectional structure shown in FIG.

  FIG. 9B is a cross-sectional view of the focus detection pixel 314. In the focus detection pixel 314 disposed in the focus detection area 101 in the center of the screen, the microlens 10 is disposed in front of the photoelectric conversion unit 14, and the shape of the photoelectric conversion unit 14 is projected forward by the microlens 10. The photoelectric conversion unit 14 is formed on the semiconductor circuit substrate 29, and the microlens 10 is integrally and fixedly formed thereon by the manufacturing process of the semiconductor image sensor. Note that the cross-sectional structure of the focus detection pixels 314 arranged in the focus detection areas 102 and 103 at the top and bottom of the screen is the same as the cross-sectional structure shown in FIG.

  FIG. 10 shows a configuration of a pupil division type phase difference detection type focus detection optical system using a microlens. The focus detection pixel portion is shown in an enlarged manner. In the figure, reference numeral 90 denotes an exit pupil set at a distance d forward from the microlens arranged on the planned imaging plane of the interchangeable lens 202 (see FIG. 1). This distance d is a distance determined according to the curvature and refractive index of the microlens, the distance between the microlens and the photoelectric conversion unit, and is referred to as a distance measuring pupil distance in this specification. 91 is an optical axis of the interchangeable lens, 10 is a microlens, 11, 13 and 14 are photoelectric conversion units, 310 is an imaging pixel, 313 and 314 are focus detection pixels, 71 is a photographing light beam, and 73 and 74 are focus detection light beams. .

  Reference numeral 95 denotes an area of the photoelectric conversion unit 11 projected by the microlens 10. Reference numeral 93 denotes an area of the photoelectric conversion unit 13 projected by the microlens 10 and is referred to as a distance measuring pupil in this specification. Similarly, 94 is an area of the photoelectric conversion unit 14 projected by the microlenses 10b and 10d, and is called a distance measuring pupil in this specification. In FIG. 10, in order to make the explanation easy to understand, the regions are indicated by 93, 94, and 95 elliptical regions. However, in reality, the shape of the photoelectric conversion unit is an enlarged projection shape.

  FIG. 10 schematically illustrates four pixels (focus detection pixels 313, 314, and 2 imaging pixels 310) adjacent to the imaging optical axis, but other focus detection pixels and imaging pixels in the focus detection area 101. In addition, also in the focus detection pixels and the imaging pixels in the focus detection areas 102 and 103 in the periphery of the screen, each photoelectric conversion unit transmits the light flux that arrives at each microlens from the corresponding region 95 and distance measurement pupils 93 and 94, respectively. It is configured to receive light. The arrangement direction of the focus detection pixels is made to coincide with the arrangement direction of the pair of distance measuring pupils, that is, the arrangement direction of the pair of photoelectric conversion units.

  The microlens 10 is disposed in the vicinity of the planned imaging plane of the interchangeable lens 202 (see FIG. 1), and the shape of the photoelectric conversion units 11, 13, and 14 disposed behind the microlens 10 is measured from the microlens 10. The projection is performed on the exit pupil 90 separated by the distance pupil distance d, and the projection shape forms a region 95 and distance measurement pupils 93 and 94. That is, in the focus detection pixel, the microlens in each focus detection pixel is matched with the projection shape (ranging pupils 93 and 94) of the photoelectric conversion unit of each focus detection pixel on the exit pupil 90 at the projection distance d. The relative positional relationship of the photoelectric conversion units is determined, and thereby the projection direction of the photoelectric conversion unit in each focus detection pixel is determined.

  The photoelectric conversion unit 13 outputs a signal corresponding to the intensity of the image formed on the microlens 10 by the light beam 73 passing through the distance measuring pupil 93 and directed to the microlens 10 of the focus detection pixel 313. Further, the photoelectric conversion unit 14 outputs a signal corresponding to the intensity of the image formed on the microlens 10 by the light flux 74 that passes through the distance measuring pupil 94 and travels toward the microlens 10 of the focus detection pixel 314. Similarly, the photoelectric conversion unit 11 outputs a signal corresponding to the intensity of the image formed on the microlens 10 by the light beam 92 passing through the region 95 and directed to the microlens 10 of the imaging pixel.

  A large number of the two types of focus detection pixels described above are arranged in a straight line, and the output of the photoelectric conversion unit of each pixel is grouped into a distance measurement pupil 93 and an output group corresponding to the distance measurement pupil 94, thereby measuring the distance measurement pupil 93 and the measurement pupil 93. Information on the intensity distribution of the pair of images formed on the pixel array by the focus detection light beams that respectively pass through the distance pupil 94 is obtained. By applying an image shift detection calculation process (correlation calculation process, phase difference detection process), which will be described later, to this information, an image shift amount of a pair of images is detected by a so-called pupil division type phase difference detection method. Further, by performing a conversion operation according to the center-of-gravity interval of the pair of distance measuring pupils on the image shift amount, the current imaging plane with respect to the planned imaging plane (the focal point corresponding to the position of the microlens array on the planned imaging plane) The deviation (defocus amount) of the imaging plane at the detection position is calculated.

  FIG. 11 is a flowchart illustrating an imaging operation of the digital still camera (imaging device) according to the embodiment. When the power of the camera is turned on in step 100, the body drive control device 214 starts the imaging operation after step 110. In step 110, the image pickup pixel data is read out and displayed on the electronic viewfinder. In subsequent step 120, a pair of image data corresponding to the pair of images is read from the focus detection pixel array. The focus detection area is assumed to be selected in advance by the photographer using one of the focus detection areas 101 to 103 using a focus detection area selection member (not shown).

  In step 130, an image shift detection calculation process (correlation calculation process) described later is performed based on the read pair of image data, and the image shift amount is calculated and converted into a defocus amount. In step 140, it is checked whether or not the focus is close, that is, whether or not the absolute value of the calculated defocus amount is within a predetermined value. If it is determined that the lens is not in focus, the process proceeds to step 150, where the defocus amount is transmitted to the lens drive control device 206, and the focusing lens 210 of the interchangeable lens 202 is driven to the focus position. Then, it returns to step 110 and repeats the above-mentioned operation.

  Even when focus detection is impossible, the process branches to this step, a scan drive command is transmitted to the lens drive control device 206, and the focusing lens 210 of the interchangeable lens 202 is driven to scan from infinity to the nearest. Then, it returns to step 110 and repeats the above-mentioned operation.

  If it is determined in step 140 that the focus is close to the in-focus state, the process proceeds to step 160, where it is determined whether or not a shutter release has been performed by operating a shutter button (not shown). If it is determined that the shutter release has not been performed, the process returns to step 110 to repeat the above-described operation. On the other hand, if it is determined that the shutter release has been performed, the process proceeds to step 170, where an aperture adjustment command is transmitted to the lens drive control device 206, and the aperture value of the interchangeable lens 202 is controlled to a control F value (F set by the photographer or automatically). Value). When the aperture control is finished, the image sensor 212 is caused to perform an imaging operation, and image data is read from the image pickup pixel 310 and all the focus detection pixels 313 and 314 of the image pickup element 212.

  In step 180, pixel data of each pixel position in the focus detection pixel column is subjected to pixel interpolation based on data of imaging pixels around the focus detection pixel. In the subsequent step 190, image data composed of the imaged pixel data and the interpolated data is stored in the memory card 219, and the process returns to step 110 to repeat the above-described operation.

  Next, details of the image shift detection calculation process (correlation calculation process) in step 130 of FIG. 11 will be described below.

At the time of image shift detection, in the focus detection areas 101 to 103, the arrangement of the focus detection pixels 313 existing only in the central portion becomes the reference data row, while the focus detection pixels 314 existing in the upper end portion, the central portion, and the lower end portion. Is an auxiliary data string (see FIG. 3). A reference data string (A1n, where n = 0 to N) generated by the array of focus detection pixels 313 is fixed, and a sub data string (A2n, where the array of focus detection pixels 314 is generated with respect to this reference data string. n = −m to N + m) are sequentially shifted by the data interval (data pitch) of the data sequence, and the correlation amount in each shift is calculated by the following equation (1).
C (k) = Σ | A1n−A2n + k | (1)
In the equation (1), the Σ operation performs integration from 0 to N for n. The shift amount k is an integer, and the calculation of equation (1) is performed in the range of k = −m to + m.

  FIG. 12 is a graph showing the waveform of the reference data string and the sub data string with the data value on the vertical axis and the data position on the horizontal axis, where ○ indicates the reference data string corresponding to the focus detection pixel 313, ● indicates a sub-data string corresponding to the focus detection pixel 314. In FIG. 12, the state of the calculation of the above equation (1) will be briefly described. The range α of the reference data string is fixed, and according to the shift amount k (k = −4, −3... +3). Thus, the calculation of equation (1) is performed while changing the range of the sub data string to β1, β2,.

As shown in FIG. 13A, the calculation result of the expression (1) shows that the correlation amount C (k) is minimal at a shift amount having a high correlation between a pair of data (k = kj = 2 in FIG. 13A). (The smaller the value, the higher the degree of correlation). The shift amount x that gives the minimum value C (x) with respect to the continuous correlation amount is obtained by using the three-point interpolation method according to the following equations (2) to (5).
x = kj + D / SLOP (2),
C (x) = C (kj) − | D | (3),
D = {C (kj-1) -C (kj + 1)} / 2 (4),
SLOP = MAX {C (kj + 1) -C (kj), C (kj-1) -C (kj)} (5)

Whether or not the shift amount x calculated by the equation (2) is reliable is determined as follows. As shown in FIG. 13B, when the degree of correlation between a pair of data is low, the value of the interpolated minimum value C (x) of the correlation amount increases. Therefore, when C (x) is equal to or greater than a predetermined threshold value, it is determined that the calculated shift amount has low reliability, and the calculated shift amount x is canceled. Alternatively, in order to normalize C (x) with the contrast of data, when the value obtained by dividing C (x) by SLOP that is proportional to the contrast is equal to or greater than a predetermined value, the reliability of the calculated shift amount Is determined to be low, and the calculated shift amount x is canceled. Alternatively, when SLOP that is a value proportional to the contrast is equal to or less than a predetermined value, it is determined that the subject has low contrast and the reliability of the calculated shift amount is low, and the calculated shift amount x is canceled. As shown in FIG. 13C, when the correlation between the pair of data is low and there is no drop in the correlation amount C (k) between the shift ranges kmin to kmax, the minimum value C (x) is obtained. In such a case, it is determined that the focus cannot be detected.

If it is determined that the calculated shift amount x is reliable, it is converted into the image shift amount shft by the following equation (6).
shft = PY · (x−0.5) (6)
In Expression (6), PY is a detection pitch (actual interval between the focus detection pixels 313 or 314), and -0.5 is arranged with the focus detection pixel 313 and the focus detection pixel 314 separated by a half detection pitch. This is an offset amount for correcting this. The image shift amount calculated by the equation (6) is multiplied by a predetermined conversion coefficient a to be converted into a defocus amount def.
def = a · shft (7)

The pair of images detected by the focus detection pixels can maintain image shift detection accuracy with respect to the light amount balance when the distance measuring pupil is displaced by the aperture of the lens and the light amount balance is lost. Such correlation calculation may be performed.
C (k) = Σ | A1n · A2n + 1 + k−A2n + k · A1n + 1 | (8)
In equation (8), the Σ operation performs integration from 0 to N−1 for n.

  Details of the interpolation calculation of the image data at the focus detection pixel position in step 180 in FIG. 11 will be described with reference to FIGS. 14 and 15. In the following description, processing focusing on one focus detection pixel will be described, but the same processing is applied to calculation of image data at all focus detection pixel positions.

FIG. 14 is an enlarged view of a region including a focus detection pixel (shaded portion) in the central portion of the focus detection pixel array, and data of imaging pixels (B11 to B11) corresponding to a region of 7 rows and 5 columns in pixel unit conversion. B75) is shown corresponding to the pixel layout. B13a, G23a, B33a, G43a, B53a, G63a, and B73a are data of each imaging pixel (imaging pixel data to be interpolated) when a virtual imaging pixel is arranged at the focus detection pixel position. In the following description, the processing when B33a and G43a are interpolated will be described as a representative, but each imaging when the imaging pixels are arranged at other focus detection pixel positions by applying the same processing. Pixel data can be interpolated.
B33a = (B11 + B31 + B51 + B15 + B35 + B55) / 6,
G43a = (G32 + G41 + G52 + G34 + G45 + G54) / 6 (9)

FIG. 15 is an enlarged view of a region including focus detection pixels (shaded portions) at the upper end portion or the lower end portion of the focus detection pixel array, and data of imaging pixels corresponding to a region of 7 rows and 5 columns in terms of pixel units. (B11 to B75) are shown corresponding to the layout of the pixels. G23a, G43a, and G63a are data of each imaging pixel (imaging pixel data to be interpolated) when a virtual imaging pixel is arranged at the focus detection pixel position. In the following description, the process when G43a is interpolated will be described as a representative. However, by applying the same process, it is assumed that the image pickup pixels are arranged at other focus detection pixel positions. Data can be interpolated.
G43a = (G32 + G41 + G52 + G34 + G45 + G54) / 6 (10)

  Thus, in one embodiment, as shown in FIG. 3, a pair of focus detection pixels are alternately arranged at the center of the focus detection pixel array, and a pair of focus detection pixels are arranged at the upper end and the lower end. Only one is arranged. In the image shift detection calculation, the data rows of the focus detection pixels arranged in the central portion, the upper end portion, and the lower end portion are relatively shifted with reference to the data row of the focus detection pixels arranged only in the central portion. Therefore, a large defocus amount (a large image shift amount) can be detected, and image data at the focus detection pixel positions at the upper end portion and the lower end portion (virtual image pickup pixel when assuming that the image pickup pixels are arranged) In the pixel interpolation for obtaining the output data), it is only necessary to interpolate the pixel data at one focus detection pixel position, so that deterioration of image quality can be suppressed. Furthermore, since the focus detection pixels arranged at the upper end portion and the lower end portion are arranged at the positions of the green pixels having a high arrangement density in the Bayer arrangement, compared with the case where the focus detection pixels are arranged at the positions of the blue pixels and the red pixels, Since the imaged pixel data close to the pixel position to be interpolated can be used for pixel interpolation, it is possible to further suppress the degradation of image quality.

<< Other Embodiments of the Invention >>
In the embodiment described above, the pair of focus detection pixels is arranged at the position where the green pixels and the blue pixels in the Bayer array are arranged, but the green pixels and the red pixels in the Bayer array are arranged in the pair of focus detection pixels. You may arrange | position in the arranged position.

  The arrangement of the focus detection areas in the image sensor is not limited to that shown in FIG. 2, and the focus detection areas can be arranged in the diagonal direction and in other positions in the horizontal and vertical directions.

  In the focus detection pixel array shown in FIG. 3, one focus detection pixel 314 and imaging pixel of the pair of focus detection pixels 313 and 314 are alternately arranged at the end of the focus detection pixel array. By further thinning out the focus detection pixels 314 at the ends of the pixels and arranging the imaging pixels instead, the density of the interpolation pixel data at the ends of the focus detection pixel array is lowered and the interpolation accuracy of the interpolation pixel data is improved. By doing so, it is possible to further suppress degradation of image quality.

  FIG. 16 shows a configuration of an imaging element 212A according to a modification. In FIG. 16, only the configuration of the end of the focus detection pixel array corresponding to the image sensor 212 shown in FIG. 3 is shown. The focus detection pixels 314 and the imaging pixels are alternately arranged at the end of the focus detection pixel array, but the focus detection pixels 314 are further thinned out at the “endmost part”, and focus detection is performed on one of the four pixels. Pixel 314 is arranged. Imaging pixels are arranged between the focus detection pixels 314 according to the Bayer array.

FIG. 17 is an enlarged view of a region including the focus detection pixel (shaded portion) in the “endmost part” of the focus detection pixel array, and the imaging pixels corresponding to the region of 10 rows and 5 columns in terms of pixel units. Data (B11 to G105) are shown corresponding to the layout of the pixels. G43a and G83a are data of each imaging pixel (imaging pixel data to be interpolated) when a virtual imaging pixel is arranged at the focus detection pixel position. In the following description, the process when G43a is interpolated will be described as a representative. However, by applying the same process, it is assumed that the image pickup pixels are arranged at other focus detection pixel positions. Data can be interpolated.
G43a = (G32 + G41 + G52 + G23 + G63 + G34 + G45 + G54) / 8 (11)

In the equation (11), the green pixel interpolation data G43a is the green pixel data in the four directions around the pixel position (the green pixel data in the horizontal direction (G41, G45), the green pixel data in the vertical direction (G23, G63), Interpolated from the green pixel data (G32, G54) in the 45 ° upward slanting direction and the green pixel data (G34, G52) in the 45 ° slanting upward leftward direction, so that highly accurate interpolation is possible regardless of the directionality of the image. Can be expected.
In the image shift detection calculation, in the case of a large shift amount (that is, a large defocus amount) that uses the sub-data at the extreme end of the focus detection pixel array as described above, the high frequency component of the image is reduced due to the defocus. Since many frequency components remain, even if the image is sampled by thinning out the focus detection pixels, there is no possibility that the image shift detection accuracy is greatly lowered. In the shift value in which the thinned sub data must also be used, the image shift detection calculation of the equation (1) is performed between the thinned data of both the reference data and the sub data.

  In the above-described embodiment and its modification, both the pair of focus detection pixels are alternately arranged at the center of the focus detection pixel array, and only one of the pair of focus detection pixels is provided at the upper end and the lower end. Although the arrangement example is shown, the area where both the pair of focus detection pixels are alternately arranged is not necessarily the central area. For example, the focus detection pixel array may be divided into two regions, both of the pair of focus detection pixels may be alternately arranged in one region, and only one of the pair of focus detection pixels may be disposed in the other region. . Alternatively, the focus detection pixel array is divided into three or more areas, both of the pair of focus detection pixels are alternately arranged in one or a plurality of areas, and only one of the pair of focus detection pixels is arranged in the remaining area. It's okay. However, in order to enable precise focus detection in the vicinity of the in-focus state, it is desirable to set a region where both of the pair of focus detection pixels are alternately arranged in the center.

  In the case where the focus detection pixel array is divided into such regions, in the case of image shift detection calculation, the focus array arranged in all regions with reference to a data string of focus detection pixels arranged only in one or a plurality of regions. What is necessary is just to relatively shift the data string of the detection pixels. As a result, a large defocus amount (a large image shift amount) can be detected, and pixel interpolation for obtaining image data at the focus detection pixel position (output data of the virtual imaging pixel when the imaging pixel is arranged) is obtained. Since only pixel data at one focus detection pixel position needs to be interpolated, degradation of image quality can be suppressed.

  In the data interpolation calculation of the virtual imaging pixel described above, an example using the simple average of the imaging pixel data of the same color as the virtual imaging pixel existing around the virtual imaging pixel is shown. A weighted average may be performed according to a weighting factor corresponding to a positional relationship (direction, distance) between a typical imaging pixel and an imaging pixel used for interpolation. In addition, the directionality of the image around the virtual imaging pixel is determined using the data of the imaging pixels around the virtual imaging pixel, and the weighted average is calculated according to the weighting coefficient according to the determined directionality. May be performed.

  In the above-described embodiment and its modification, an example in which the imaging pixels are RGB Bayer arrays has been described. However, for example, a pixel array other than a Bayer array such as a complementary color filter array may be used.

  In the focus detection pixels 313 and 314 shown in FIG. 5 of the embodiment described above, an example in which the shape of the photoelectric conversion unit is rectangular is shown, but the shape of the photoelectric conversion unit of the focus detection pixel is not limited to these, Other shapes may be used. For example, the shape of the photoelectric conversion unit of the focus detection pixel can be a semicircle, an ellipse, or a polygon.

  In the focus detection pixels 313 and 314 shown in FIG. 5 of the above-described embodiment, pupil division is performed by projecting the shape of the photoelectric conversion units 13 and 14 forward by the microlens 10. , 14 is the same as the shape of the photoelectric conversion unit 11 of the imaging pixel 310, and a mask having the same opening as the shape of the photoelectric conversion units 13, 14 shown in FIG. The focus detection pixel may be configured so that the shape of the mask is projected forward by the microlens 10. In this case, the photoelectric conversion units 13 and 14 receive the light flux that has passed through the opening of the mask.

  Further, in the imaging device 212 shown in FIG. 3 and the imaging device 212A shown in FIG. 16, the example in which the imaging pixels and the focus detection pixels are arranged in a dense square lattice arrangement is shown, but a dense hexagonal lattice arrangement may be used.

  In the above-described embodiment, the focus detection operation by the pupil division method using the microlens has been described. However, the present invention is not limited to the focus detection of such a method, and is disclosed in Japanese Patent Laid-Open No. 2008-015157. The present invention can also be applied to a pupil detection type phase difference detection type focus detection apparatus using a polarizing element. Further, a micro lens may be provided only in the focus detection pixel.

  Note that the imaging apparatus is not limited to a digital still camera or a film still camera in which an interchangeable lens is mounted on the camera body as described above. For example, the present invention can also be applied to a lens-integrated digital still camera, film still camera, or video camera. Furthermore, the present invention can be applied to a small camera module built in a mobile phone, a surveillance camera, a visual recognition device for a robot, an in-vehicle camera, and the like.

  In the above-described embodiments and their modifications, all combinations of the embodiments and the modifications are possible.

  According to the above-described embodiment and its modifications, the following operational effects can be achieved. First, one of a pair of light fluxes that has passed through a pair of regions of the exit pupil of the interchangeable lens 202 that forms an image on the imaging surface, in a part of the array of imaging pixels that are two-dimensionally arranged on the imaging surface. The focus detection pixels 313 include at least one focus detection pixel array in which focus detection pixels 313 that receive one light beam and focus detection pixels 314 that receive the other light beam are arranged in a line. And the focus detection pixels 314 are divided into a plurality of regions, the focus detection pixels 313 and the focus detection pixels 314 are alternately arranged in at least one of the plurality of regions, and the focus is detected in the other at least one region. The focus detection pixels 314 and one or more imaging pixels were alternately arranged without arranging the detection pixels 313. Accordingly, it is possible to detect a large defocus amount and minimize the degree of image degradation in an imaging element in which pupil-divided focus detection pixels are mixed with imaging pixels.

  Next, the focus detection pixel 313 is arranged at the position of the image pickup pixel having the red filter or the blue filter in the Bayer array, and the focus detection pixel 314 is set to the image pickup pixel having the green filter in the Bayer array. Since it is arranged at the position, compared with the case where the focus detection pixel 314 is arranged at the position of the blue imaging pixel or the red imaging pixel, the data of the imaging pixel close to the pixel position to be interpolated can be used for pixel interpolation. Can be further suppressed.

Cross-sectional view of the camera showing the configuration of the camera of one embodiment The figure which shows the focus detection position on the imaging | photography screen of the interchangeable lens 202. Front view showing a detailed configuration of the image sensor 212 The figure which shows the structure of the imaging pixel 310 The figure which shows the structure of the focus detection pixel 313,314. The figure which shows the spectral characteristic of the imaging pixel 310 The figure which shows the spectral characteristics of the focus detection pixels 313 and 314 Sectional drawing which shows the structure of the imaging pixel 310 Sectional drawing which shows the structure of focus detection pixel 313,314 The figure which shows the structure of the focus detection optical system of the pupil division type phase difference detection method using a micro lens The flowchart which shows the imaging operation of the digital still camera (imaging device) of one embodiment Graph showing the waveform of the reference data row and the sub data row with the data value on the vertical axis and the data position on the horizontal axis Diagram for explaining the reliability of focus detection results The figure for demonstrating the detail of the interpolation calculation of the image data of the focus detection pixel position in step 180 of FIG. The figure for demonstrating the detail of the interpolation calculation of the image data of the focus detection pixel position in step 180 of FIG. The figure which shows the structure of the image pick-up element 212A of a modification. Enlarged view of the region including the focus detection pixel (shaded portion) at the “endmost part” of the focus detection pixel array.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10; Micro lens, 11, 13, 14; Photoelectric conversion part, 201; Camera, 202; Interchangeable lens, 203; Camera body, 212, 212A; Image sensor, 214; Body drive control device, 310; 314: focus detection pixel

Claims (9)

One of the pair of luminous fluxes that has passed through a pair of regions of the exit pupil of the optical system that forms an image on the imaging plane in a part of the array of imaging pixels arranged two-dimensionally on the imaging plane. An image sensor in which at least one focus detection pixel array in which a first focus detection pixel that receives light and a second focus detection pixel that receives the other light beam is arranged in a row is arranged,
The focus detection pixel row is divided into a plurality of regions, and the first focus detection pixels and the second focus detection pixels are alternately arranged in at least one of the plurality of regions, and at least one other In the region, the second focus detection pixels and one or a plurality of the imaging pixels are alternately arranged without arranging the first focus detection pixels. The imaging device according to claim 1,
On the imaging surface, the imaging pixels having a green filter, the imaging pixels having a red filter, and the imaging pixels having a blue filter are arranged in a Bayer array,
The first focus detection pixel is disposed at a position of the imaging pixel having a red filter or a blue filter in a Bayer array,
The imaging element, wherein the second focus detection pixel is arranged at a position of the imaging pixel having a green filter in a Bayer array. The imaging device according to claim 1 or 2,
Each of the first focus detection pixel and the second focus detection pixel includes a microlens and a photoelectric conversion unit, respectively. The imaging device according to any one of claims 1 to 3,
The imaging element, wherein a region in which the second focus detection pixels and one or a plurality of the imaging pixels are alternately arranged in the plurality of regions is a region including an end of the focus detection pixel row. In the image sensor according to any one of claims 1 to 4,
The imaging element, wherein a region where the first focus detection pixels and the second focus detection pixels are alternately arranged in the plurality of regions is a central region of the focus detection pixel row. The image sensor according to any one of claims 1 to 5,
A pair of images formed by the pair of light beams by shifting the second data string generated by the array of second focus detection pixels with respect to the first data string generated by the array of first focus detection pixels. A deviation amount calculating means for calculating a deviation amount of
Conversion means for converting the shift amount into a defocus amount between the imaging surface and a focusing surface of the optical system;
Output data of virtual imaging pixels at the positions of the first focus detection pixels and the second focus detection pixels are interpolated by output data of the imaging pixels around the first focus detection pixels and the second focus detection pixels. An imaging apparatus comprising: an interpolation means for performing The imaging device according to claim 6,
The shift amount calculation means shifts the second data string with respect to the first data string by a shift amount in an arrangement pitch unit of the second focus detection pixels, and the first data string, the second data string, And an relative shift amount between the first data string and the second data string according to a shift amount indicating the highest correlation degree. The imaging apparatus according to claim 7,
The deviation amount calculation means interpolates the shift amount by an amount equal to or less than the arrangement pitch unit based on the correlation amount between the shift amount showing the highest degree of correlation and the shift amount before and after the shift amount, and based on the interpolated shift amount An image pickup apparatus that calculates a shift amount between the first data string and the second data string. In the imaging device according to any one of claims 6 to 8,
The interpolation means detects the second focus detection when interpolating the output data of the virtual imaging pixel at the position of the second focus detection pixel with the output data of the imaging pixel around the second focus detection pixel. When the imaging pixel exists between a pixel and the second focus detection pixel, the imaging apparatus performs interpolation using output data of the imaging pixel.

Images