JP2008040087A - Imaging device, imaging apparatus and focus detecting device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately reproduce image data located on the positions of focus detection pixels. <P>SOLUTION: An imaging device is disclosed in which a plurality of kinds of imaging devices 310a, 310b and 310c having different spectral sensitivity characteristics are two-dimensionally and regularly arrayed to image an image formed by an optical system, the imaging device is characterized in that among the imaging devices, the imaging devices 311a and 311b arranged along the prescribed direction are configured as the focus detection pixels, and that in the regular array, the imaging pixels and the focus detection pixels are arrayed in a coexistent manner on the pixel array in the prescribed direction so that the imaging pixel having the same spectral sensitivity characteristics as that of the imaging pixel located in the position of the focus detection pixel may be arranged between the plurality of focus detection pixels. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging element, an imaging apparatus, and a focus detection apparatus.

  A color imaging apparatus that uses an imaging device having focus detection pixels (hereinafter referred to as AF pixels) and interpolates image data of AF pixels from image data of imaging pixels around the AF pixels. Is known (see, for example, Patent Document 1).

Prior art documents related to the invention of this application include the following.
JP 2000-305010 A

However, the above-described conventional color imaging device has the following problems.
When a thin line pattern 402 (shown by a broken line in the drawing) in which the green component or the blue component is different from the background green component or the blue component is superimposed on the AF pixel row of the image sensor shown in FIG. In the interpolation of the image data according to the equations (2) and (2), the image data on the AF pixel column has the same color component as the background, and the fine line pattern 402 cannot be reproduced as the image data.
Further, when not only the fine line pattern but also the edge pattern is superimposed on the AF pixel column, the edge pattern cannot be faithfully reproduced by the conventional image data interpolation, and noise such as false color may occur.

(1) The invention of claim 1 is an imaging device that regularly arranges a plurality of types of imaging pixels having different spectral sensitivity characteristics in a two-dimensional manner and images an image formed by an optical system. A plurality of pixels for imaging along the predetermined direction are configured as focus detection pixels, and there are a plurality of imaging pixels having the same spectral sensitivity characteristics as the imaging pixels located at the positions of the focus detection pixels in a regular arrangement The imaging pixels and the focus detection pixels are mixedly arranged on the pixel array in a predetermined direction so as to be arranged between the focus detection pixels.
(2) An invention according to claim 2 is an image pickup device for picking up an image formed by an optical system by regularly arranging two or more types of image pickup pixels having different spectral sensitivity characteristics in two dimensions. A part of the pixels for detection along the predetermined direction is configured as a focus detection pixel, and all types of imaging pixels on the pixel array in the predetermined direction in the regular array are arranged between the plurality of focus detection pixels. In order to be arranged, the imaging pixels and the focus detection pixels are mixedly arranged on the pixel array in a predetermined direction.
(3) In the image pickup device according to the third aspect, a plurality of types of imaging pixels are arranged in a square, and the predetermined direction is a direction of each side or diagonal line of the square arrangement.
(4) In the image pickup device according to the fourth aspect, the focus detection pixels receive a pair of light fluxes passing through different pupil portions along a predetermined direction in the exit pupil of the optical system.
(5) In the imaging device according to the fifth aspect, the focus detection pixel includes a microlens and a photoelectric conversion unit, and the photoelectric conversion unit and the exit pupil of the optical system are conjugate with respect to the microlens.
(6) In the image pickup device according to the sixth aspect, the focus detection pixel includes a pair of photoelectric conversion units, and each photoelectric conversion unit receives a pair of light fluxes passing through different pupil portions of the optical system.
(7) In the imaging device according to the seventh aspect, the focus detection pixel receives one of a pair of light beams passing through different pupil portions of the optical system by the photoelectric conversion unit.
(8) In the image pickup device according to the eighth aspect, a plurality of pairs of focus detection pixels that respectively receive a pair of light beams passing through different pupil portions of the optical system are arranged.
(9) In the image pickup device according to the ninth aspect, the focus detection pixels have different spectral sensitivity characteristics from the image pickup pixels.
(10) In the image pickup device according to the tenth aspect, the focus detection pixels have the same spectral sensitivity characteristics as the image pickup pixels located at the positions of the focus detection pixels in a regular arrangement.
(11) The imaging device according to an eleventh aspect includes three types of imaging pixels that receive RGB light.
(12) The image pickup device according to the twelfth aspect includes a Bayer array of three types of image pickup pixels that receive RGB.
(13) In the imaging device according to the thirteenth aspect, the imaging pixel includes a microlens and a photoelectric conversion unit, and the photoelectric conversion unit and the exit pupil of the optical system are conjugate with respect to the microlens.
(14) The image pickup device according to the fourteenth aspect makes the optical characteristics of the microlens of the imaging pixel different from the optical characteristics of the microlens of the focus detection pixel.
(15) A focus detection apparatus according to a fifteenth aspect detects a focus adjustment state of an optical system based on the image sensor according to any one of claims 1 to 14 and output data of focus detection pixels of the image sensor. And a focus detection means.
(16) An image pickup apparatus according to a sixteenth aspect is based on the image pickup element according to any one of the first to fourteenth aspects and output data of an image pickup pixel in the vicinity of the focus detection pixel of the image pickup element. Interpolating means for interpolating the image data at the pixel position.

  According to the present invention, it is possible to accurately reproduce the image data at the position of the focus detection pixel and prevent deterioration of the image quality.

  An embodiment in which an image sensor according to an embodiment of the present invention is applied to a digital still camera will be described. FIG. 1 shows the configuration of an embodiment. A digital still camera 201 according to an embodiment includes an interchangeable lens 202 and a camera body 203, and the interchangeable lens 202 is attached to the camera body 203 by a mount unit 204.

  The interchangeable lens 202 includes a lens drive control device 206, a zooming lens 208, a lens 209, a focusing lens 210, a diaphragm 211, and the like. The lens drive control device 206 includes peripheral components such as a microcomputer and a memory. The lens drive control device 206 controls driving of the focusing lens 210 and the aperture 211, detects the state of the aperture 211, the zooming lens 208 and the focusing lens 210, and body drive control described later. Transmission of lens information to the device 214 and reception of camera information are performed.

  The camera body 203 includes an imaging element 212, a body drive control device 214, a liquid crystal display element drive circuit 215, a liquid crystal display element 216, an eyepiece lens 217, a memory card 219, and the like. Pixels, which will be described later, are arranged in a two-dimensional manner on the imaging element 212, and are arranged on the planned imaging plane of the interchangeable lens 202 to capture a subject image formed by the interchangeable lens 202. Although details will be described later, focus detection pixels (hereinafter referred to as AF pixels) are arranged at predetermined focus detection positions of the image sensor 212.

  The body drive control device 214 includes a microcomputer and peripheral components such as a memory. The body drive control device 214 reads the image signal from the image sensor 212, corrects the image signal, detects the focus adjustment state of the interchangeable lens 202, and outputs from the lens drive control device 206. It receives lens information, transmits camera information (defocus amount), and controls the operation of the entire digital still camera. The body drive control device 214 and the lens drive control device 206 communicate via the electrical contact portion 213 of the mount portion 204 to exchange various information.

  The liquid crystal display element driving circuit 215 drives a liquid crystal display element 216 of a liquid crystal viewfinder (EVF: electrical viewfinder). The photographer can observe an image displayed on the liquid crystal display element 216 via the eyepiece lens 217. The memory card 219 is removable from the camera body 203 and is a portable storage medium that stores and stores image signals.

  The subject image formed on the image sensor 212 through the interchangeable lens 202 is photoelectrically converted by the image sensor 212 and the output is sent to the body drive controller 214. The body drive control device 214 calculates the defocus amount at a predetermined focus detection position based on the output data of the AF pixels on the image sensor 212 and sends this defocus amount to the lens drive control device 206. The body drive control device 214 stores an image signal generated based on the output of the image sensor 212 in the memory card 219 and sends the image signal to the liquid crystal display element drive circuit 215 to display an image on the liquid crystal display element 216. Let

  The camera body 203 is provided with operation members (not shown) (shutter buttons, focus detection position setting members, etc.), and the body drive control device 214 detects operation state signals from these operation members. The corresponding operations (imaging operation, focus detection position setting operation, image processing operation) are controlled.

  The lens drive control device 206 changes the lens information according to the focusing state, zooming state, aperture setting state, aperture opening F value, and the like. Specifically, the lens drive control device 206 monitors the positions of the lenses 208 and 210 and the diaphragm position of the diaphragm 211, calculates lens information according to the monitor information, or monitors from a lookup table prepared in advance. Select lens information according to the information. The lens drive control device 206 calculates a lens drive amount based on the received defocus amount, and drives the focusing lens 210 to a focal point by a drive source such as a motor (not shown) based on the lens drive amount.

  FIG. 2 shows a focus detection position on the photographing screen. Focus detection areas 101 to 105 are arranged at five locations on the photographing screen 100. These focus detection areas 101 to 105 indicate areas in which AF pixels of the image sensor 212 (to be described later) sample subject images on the photographing screen 100, and are used for AF in the longitudinal direction of the focus detection areas 101 to 105 indicated by rectangles. Pixels are arranged linearly. The photographer manually selects a focus detection area from a plurality of focus detection areas 101 to 105 by a focus detection area selection operation member (not shown) according to the shooting composition.

  FIG. 3 shows an array of imaging pixels of the image sensor 212. The imaging pixels that are two-dimensionally arranged on the substrate of the imaging element 212 are provided with a Bayer array color filter as shown in FIG. 3 shows an arrangement of color filters for four pixels (2 × 2) of image pickup pixels. On the image pickup element 212, there are four pixel image pickup pixel units having the Bayer arrangement shown in FIG. Arranged in two dimensions. In the Bayer array, two pixels of the green filter (G) are arranged on a diagonal line, and a blue filter (B) and a red filter (R) are arranged on the diagonal line one pixel at a time. In the Bayer array, the density of green pixels is higher than the density of red pixels and blue pixels.

  FIG. 8 shows spectral characteristics of the imaging green pixel, imaging red pixel, and imaging blue pixel of the imaging device 212. This spectral characteristic is a spectral characteristic that combines the spectral sensitivity of each color filter shown in FIG. 3, the spectral sensitivity of a photodiode (not shown) that performs photoelectric conversion, and the spectral characteristic of an infrared cut filter (not shown). The pixels, red pixels, and blue pixels have different light wavelength regions (having different spectral sensitivities) in order to perform color separation. In particular, the spectral characteristic of the green pixel has a sensitivity close to the peak of relative luminous sensitivity as shown in FIG.

  Note that a color filter is not disposed in the AF pixel to increase the amount of light, and the spectral characteristics thereof are the spectral sensitivity of a photodiode (not shown) that performs photoelectric conversion and the spectral characteristic of an infrared cut filter (not shown). The spectral characteristics shown in FIG. The spectral characteristic of the AF pixel is a spectral characteristic obtained by adding the spectral characteristics of the green pixel, red pixel, and blue pixel shown in FIG. 8, and the light wavelength region of the sensitivity is the sensitivity of the green pixel, red pixel, and blue pixel. It covers the optical wavelength region.

  FIG. 5 is a front view showing a detailed configuration of the image sensor 212. In FIG. 5, the vicinity of one focus detection area on the image sensor 212 is shown enlarged. The image sensor 212 includes imaging pixels (green pixel 310a, blue pixel 310b, red pixel 310c) and AF pixels (AF pixel 311a replacing the green pixel and AF pixel 311b replacing the blue pixel). The As shown in FIG. 6, the imaging pixels 310a, 310b, and 310c include a microlens 10, a photoelectric conversion unit 11, and a color filter (not shown). As shown in FIG. 7, the AF pixels 311 a and 311 b include a microlens 10 and a pair of photoelectric conversion units 12 and 13.

  The photoelectric conversion unit 11 of the imaging pixels 310a, 310b, and 310c is designed so as to receive all the light beams that pass through the exit pupil (for example, F1.0) of the bright interchangeable lens by the microlens 10. The pair of photoelectric conversion units 12 and 13 of the AF pixels 311a and 311b are designed in such a shape that the microlens 10 receives all the light beams that pass through a specific exit pupil (for example, F2.8) of the interchangeable lens.

  As shown in FIG. 5, the imaging pixels 310a, 310b, and 310c arranged two-dimensionally are provided with RGB Bayer color filters. The AF pixels 311a and 311b are arranged on a straight line in a row (column) where the green pixels 310a and the blue pixels 310b of the imaging pixels are to be arranged, and the AF pixels 311a and 311b are separated by two pixels. , One green pixel 310a and one blue pixel 310b are arranged there.

  By arranging the AF pixels 311a and 311b in the row (column) where the green pixels 310a and the blue pixels 310b of the imaging pixels are to be arranged, the pixel signal at the position of the AF pixel 311 is interpolated by pixel interpolation described later. Even if some errors occur, it can be made inconspicuous to the human eye. This is because the human eye is more sensitive to red than blue and because the density of green pixels is higher than that of blue and red pixels, the contribution to image degradation for a single pixel defect of green pixels is small.

  The AF pixels 311a and 311b are arranged in the focus detection area. By arranging the green pixel 310a and the blue pixel 310b between the AF pixels 311a and 311b, the image data of the positions of the AF pixels 311a and 311b is interpolated between the image data of the green pixel 310a and the blue pixel 310b. The image data can be favorably interpolated even when an image pattern such as an edge or a thin line is superimposed on the array of AF pixel columns.

  FIG. 11 is a cross-sectional view of the imaging pixels 310a, 310b, and 310c. In the imaging pixels 310 a, 310 b, and 310 c, the microlens 10 is disposed in front of the imaging photoelectric conversion unit 11, and the photoelectric conversion unit 11 is projected forward by the microlens 10. The photoelectric conversion unit 11 is formed on the semiconductor circuit substrate 29. The color filter (not shown) described above is disposed between the microlens 10 and the photoelectric conversion unit 11.

  FIG. 12 is a cross-sectional view of the AF pixels 311a and 311b. In the AF pixels 311a and 311b, the microlens 10 is disposed in front of the focus detection photoelectric conversion units 12 and 13, and the photoelectric conversion units 12 and 13 are projected forward by the microlens 10. The photoelectric conversion units 12 and 13 are formed on the semiconductor circuit substrate 29.

  Here, the principle of focus detection by the pupil division type phase difference detection method will be described with reference to FIG. In the figure, reference numeral 90 denotes an exit pupil set at a distance d4 in front of the microlens arranged on the planned imaging plane of the interchangeable lens. The distance d4 is a distance determined according to the curvature and refractive index of the microlens and the distance between the microlens and the photoelectric conversion unit, and is hereinafter referred to as a distance measuring pupil distance. 91 is an optical axis of the interchangeable lens, 50 and 60 are microlenses, 52 and 53 are a pair of photoelectric conversion units for focus detection pixels, 61 is a photoelectric conversion unit for imaging pixels, 72 and 73 are focus detection light fluxes, and 74 is This is the imaging light flux. Reference numeral 92 denotes a region of the photoelectric conversion unit 52 projected by the microlens 50 (hereinafter referred to as a ranging pupil), and reference numeral 93 denotes a region of the photoelectric conversion unit 53 projected by the microlens 50 (hereinafter referred to as a ranging pupil). , 94 is a region (hereinafter referred to as a distance measuring pupil) of the photoelectric conversion unit 61 projected by the micro lens 60.

  In FIG. 13, an AF pixel (having a microlens 50 and a pair of photoelectric conversion units 52 and 53) on an optical axis 91 and an adjacent imaging pixel (having a microlens 60 and a photoelectric conversion unit 61). In other AF pixels as well, the pair of photoelectric conversion units receive light beams coming from the pair of distance measuring pupils 92 and 93 to the microlenses, and in other imaging pixels. In addition, the photoelectric conversion units receive the light beams coming from the pair of regions 94 to the respective microlenses. The arrangement direction of the AF pixels is matched with the arrangement direction of the pair of distance measuring pupils.

  The microlens 50 is disposed in the vicinity of the planned imaging plane of the optical system, and the shape of the pair of photoelectric conversion units 52 and 53 disposed behind the microlens 50 disposed on the optical axis 91 is the microlens 50. Is projected onto the exit pupil 90 separated by the projection distance d4, and the projection shape forms distance measuring pupils 92 and 93. That is, the projection direction of each pixel is determined so that the projection shapes (ranging pupils 92 and 93) of the photoelectric conversion units of the focus detection pixels coincide on the exit pupil 90 at the projection distance d4. In the AF pixel, with respect to the microlens, the photoelectric conversion unit and the exit pupil of the interchangeable lens (imaging optical system) are conjugate.

  The photoelectric conversion unit 52 outputs a signal corresponding to the intensity of the image formed on the microlens 50 by the focus detection light beam 72 that passes through the distance measuring pupil 92 and faces the microlens 50. On the other hand, the photoelectric conversion unit 53 outputs a signal corresponding to the intensity of the image formed on the microlens 50 by the focus detection light beam 73 that passes through the distance measuring pupil 93 and faces the microlens 50. A large number of such AF pixels are arranged in a straight line, and the output of the pair of photoelectric conversion units of each pixel is collected into an output group corresponding to the distance measurement pupil 92 and the distance measurement pupil 93, thereby measuring the distance measurement pupil 92 and the measurement pupil 92. Information on the intensity distribution of a pair of images formed on the AF pixel row by the focus detection light beams that pass through the distance pupils 93 is obtained.

  By applying an image shift detection calculation process (correlation process, phase difference detection process) to be described later to this information, the image shift amount of a pair of images is detected by a so-called pupil division method. Further, by multiplying the image shift amount by a predetermined conversion coefficient, the deviation of the current imaging plane (the imaging plane at the focus detection position corresponding to the position of the microlens array on the planned imaging plane) with respect to the planned imaging plane (Defocus amount) can be calculated.

  The microlens 60 is disposed in the vicinity of the planned imaging plane of the optical system, and the shape of the photoelectric conversion unit 61 disposed behind the microlens 60 is on the exit pupil 90 separated from the microlens 60 by the projection distance d4. The projected shape forms a region 94. That is, the projection direction of each pixel is determined so that the projection shape (region 94) of the photoelectric conversion unit of each imaging pixel matches on the exit pupil 90 at the projection distance d4. The photoelectric conversion unit 61 outputs a signal corresponding to the intensity of the image formed on the microlens 60 by the imaging light flux 74 that passes through the region 94 and travels toward the microlens 60.

  By arranging a large number of such imaging pixels two-dimensionally and detecting the output of the photoelectric conversion unit of each pixel, information (image data) relating to the two-dimensional light intensity distribution of the image on the screen can be obtained. .

  FIG. 14 is a front view showing a projection relationship on the exit pupil plane. The circumscribed circle of the distance measuring pupils 92 and 93 obtained by projecting a pair of photoelectric conversion units of the AF pixels onto the exit pupil plane 90 by the microlens is a predetermined aperture F value (ranging pupil F value) when viewed from the imaging plane. Here, it is F2.8). In addition, a region 94 in which the photoelectric conversion unit of the imaging pixel is projected onto the exit pupil plane 90 by the microlens is a wide region including the distance measuring pupils 92 and 93. The area on the distance measuring pupil plane 90 corresponding to the brightest aperture (F1) is 95, and the area on the distance measuring pupil plane 90 corresponding to the dark aperture (F5.6) is 96.

  Since the area 94 is wider than the areas 95 and 96, the light flux received by the imaging pixels is limited by the aperture, and the data of the imaging pixels changes according to the aperture value. The distance measuring pupils 92 and 93 are narrower than the region 95 and wider than the region 96. Therefore, if the aperture value is brighter than F2.8, the AF pixel data does not change even if the aperture value changes. If the aperture value is darker than F2.8, the AF pixel data does not change. Varies depending on the aperture value.

  When the imaging pixel data value / focus detection pixel data value is graphed as an output ratio coefficient (normalized as 1 when the aperture value is F2.8), the aperture value is F2.8 as shown in FIG. When the aperture value is darker, the constant value is 1 regardless of the aperture value. When the aperture value is brighter than F2.8, the aperture value increases as the aperture value becomes brighter.

  FIG. 16 is a flowchart illustrating the operation of the digital still camera (imaging device) according to the embodiment. The body drive control device 214 repeatedly executes this operation while the camera is powered on. When the power of the camera is turned on in step 100, the process proceeds to step 110, where the data of the imaging pixels are read out and displayed on the electronic viewfinder. When thinning out and reading out the data of the imaging pixels, the display quality can be improved by performing the thinning out reading with a setting that does not include AF pixels as much as possible.

  In step 120, data is read from the AF pixel column. Here, it is assumed that the focus detection area is selected by the user using a selection operation member (not shown). Next, in step 130, an image shift detection calculation process is performed based on a pair of image data corresponding to the AF pixel row, an image shift amount is calculated, and a defocus amount is further calculated.

Here, the image shift detection calculation process (correlation algorithm) will be described with reference to FIG. Assuming that a pair of data corresponding to a certain AF pixel column is ei, fi (where i = 1 to m), first, a correlation amount C (L) is obtained by a differential correlation algorithm expressed by the following equation.
C (L) = Σ | e (i + L) −f (i) | (1)
In the equation (1), Σ represents a total operation of i = p to q. L is an integer, and is a relative shift amount with the detection pitch of a pair of data as a unit. Further, the range of L is Lmin to Lmax (-5 to +5 in the figure). Further, the range of the parameter i is from p to q, and is determined so as to satisfy the condition of 1 ≦ p <q ≦ m.

As shown in FIG. 17A, the calculation result of the equation (4) shows that the correlation amount C (L) is obtained when the shift amount L = kj (kj = 2 in FIG. 17A) with a high correlation between a pair of data. Be minimized. Next, a shift amount x that gives a minimum value C (L) min = C (x) with respect to a continuous correlation amount is obtained using a three-point interpolation method according to equations (2) to (5).
x = kj + D / SLOP (2),
C (x) = C (kj) − | D | (3),
D = {C (kj-1) -C (kj + 1)} / 2 (4),
SLOP = MAX {C (kj + 1) -C (kj), C (kj-1) -C (kj)} (5)
In addition, the defocus amount DEF of the subject image plane with respect to the planned image formation plane can be obtained from the following equation using the shift amount x obtained by the equation (1).
DEF = KX · PY · x (6)
In Equation (6), PY is a detection pitch, and KX is a conversion coefficient determined by the size of the opening angle of the center of gravity of the pair of distance measuring pupils.

  Whether or not the calculated defocus amount DEF is reliable is determined as follows. As shown in FIG. 17B, when the degree of correlation between a pair of data is low, the value of the minimum value C (X) of the interpolated correlation amount is large. Therefore, when C (X) is equal to or greater than a predetermined value, it is determined that the reliability is low. Alternatively, in order to normalize C (X) with the contrast of data, it is determined that the reliability is low when the value obtained by dividing C (X) by SLOP which is a value proportional to the contrast is equal to or greater than a predetermined value. Alternatively, if SLOP that is proportional to the contrast is equal to or less than a predetermined value, it is determined that the subject has low contrast and the reliability of the calculated defocus amount DEF is low.

  As shown in FIG. 17C, when the correlation between the pair of data is low and there is no drop in the correlation amount C (L) between the shift ranges Lmin to Lmax, the minimum value C (X) is obtained. In such a case, it is determined that the focus cannot be detected. When focus detection is possible, the defocus amount is calculated by multiplying the calculated image shift amount by a predetermined conversion coefficient.

  In step 140, it is checked whether or not the focus is close, that is, whether the calculated absolute value of the defocus amount is within a predetermined value. If it is determined that the lens is not in focus, the process proceeds to step 150, the defocus amount is transmitted to the lens drive control device 216, and the focusing lens 210 of the interchangeable lens 202 is driven to the focus position based on the defocus amount. Returning to 110, the above-described operation is repeated. If the focus cannot be detected, the process branches to step 150, a scan drive command is transmitted to the lens drive control device 216, and the focusing lens 210 of the interchangeable lens 202 is driven to scan from infinity to close, and the process returns to step 110. Repeat the above operation.

  On the other hand, if it is determined that the focus is close, the process proceeds to step 160, where it is determined whether or not a shutter release has been performed by operating a shutter button (not shown), and if it is determined that the shutter release has not been performed, the process returns to step 110. The above operation is repeated. If it is determined that the shutter release has been performed, the process proceeds to step 170, an aperture adjustment command is transmitted to the lens drive control device 216, and the aperture value of the interchangeable lens 202 is set to the control F value (the F value set by the user or automatically). Set. When the aperture control is completed, the image sensor 212 performs an image capturing operation, and image data is read from the image capturing pixels of the image sensor 212 and all the AF pixels.

  In step 180, the pixel data at each pixel position in the AF pixel column is interpolated based on the AF pixel data and the surrounding imaging pixel data. This interpolation process will be described later. In the next step 190, image data composed of the pixel data for imaging and the interpolated data is stored in the memory card, and the process returns to step 110 to repeat the above-described operation.

  Next, image data interpolation processing will be described. FIG. 18 is a diagram illustrating a schematic configuration of the image sensor 212 illustrated in FIG. 5 according to an embodiment. A green pixel G310a, a blue pixel B310b, and a red pixel R310c are arranged in a Bayer array. In a row corresponding to a part of the focus detection area, AF pixels A are arranged in a row, and two imaging pixels G and B in between. It is arranged across An AF pixel A311a is arranged at a position where the green pixel G should be, and an AF pixel A311b is arranged at a position where the blue pixel B should be.

FIG. 19 is a diagram illustrating image data interpolation processing for the image sensor 212 according to the embodiment. In the figure, the green pixel G data of the imaging pixel is Gn, the red pixel R data is Rn, the blue pixel B data is Bn, and the focus detection pixel A data is An (n = 1, 2,... ). Note that although the interpolation process will be described by paying attention to a specific pixel, the same process is performed on all the AF pixels arranged at the position of the green pixel. The image data I (A1) at the position of the AF pixel (data A1) arranged in the green pixel is interpolated by the following equation.
I (A1) = (G2 + G4 + G5 + G6 + G7 + G9 + G11 + G12) / 8 (7)
The image data G11 and G12 reflect the image pattern 400 even when the image pattern 400 such as an edge or a thin line is superimposed on the AF pixel array by performing the interpolation calculation on the image data as in the equation (7). Therefore, a good image interpolation result can be obtained.

  As a method for calculating the image data I (A1) at the position of the AF pixel by performing statistical processing, not only a simple average calculation as shown in the equation (7) but also a direction perpendicular to the AF pixel. Pixel data at a plurality of peripheral positions may be obtained by linear interpolation, interpolation by a second-order or higher-order multi-order expression, or median processing.

Further, the focus detection pixel data A1 may be used for the interpolation calculation of the image data.
I (A1) = {k1 · (G2 + G4 + G5 + G6 + G7 + G9 + G11 + G12) / 8} + k2 · ks · A1 (8)
In the equation (8), the coefficient ks is an output ratio coefficient shown in FIG. 15, and an output ratio coefficient corresponding to a control F value (aperture F value) at the time of imaging is selected. The coefficients k1 and k2 are weighting coefficients when the data interpolated from the surrounding pixel data and the AF pixel data are mixed, and satisfy k1 + k2 = 1. Since the AF pixel data A1 is data corresponding to the luminance at the position of the AF pixel, the high-frequency component of the image is obtained by using the AF pixel data A1 for the interpolation calculation of the image data as shown in the equation (8). Reproducibility can be improved.

FIG. 20 is a diagram illustrating image data interpolation processing for the image sensor 212 according to one embodiment. In the figure, the green pixel data of the imaging pixel is Gn, the red pixel data is Rn, the blue pixel data is Bn, and the AF pixel data is An (n = 1, 2,...). In the following description, the interpolation process will be described by focusing on a specific pixel, but the same process is performed on all AF pixels arranged at the positions of the blue pixels. The image data I (A21) at the position of the AF pixel (data A21) arranged at the blue pixel position is interpolated by the following equation.
I (A21) = (B21 + B22 + B23 + B24 + B25 + B26 + B27 + B28) / 8 (9)
By interpolating the image data with the equation (9), even when the image pattern 400 such as an edge or a thin line is superimposed on the AF pixel array, the image data B27 and B28 become data reflecting the image pattern 400. A good image interpolation result can be obtained.

Similarly to the equation (9), the AF pixel data A21 may be used for interpolation of image data.
I (A21) = {k1 · (B21 + B22 + B23 + B24 + B25 + B26 + B27 + B28) / 8} + k2 · ks · A21 (10)

<< Modification of Embodiment >>
FIG. 21 is a front view showing a detailed configuration of an imaging element 212A according to a modification. FIG. 22 is a schematic configuration diagram of the image sensor 212A. In the imaging device 212 shown in FIG. 5, two imaging pixels (green pixel and blue pixel) are arranged between the AF pixels 311, whereas in FIGS. 21 and 22, 3 between the AF pixels 311. Two imaging pixels (two green pixels and one blue pixel) are arranged. The AF pixel 311a is disposed at the position of the green pixel 310a.

  Also in the configuration of the image sensor 212A shown in FIGS. 21 and 22, the image data of the AF pixel arranged at the position of the green pixel can be interpolated by the equations (10) and (11). In this image sensor 212A, since the AF pixel is arranged by replacing only the green pixel, the blue component and the red component of the image can be detected without being affected by the arrangement of the AF pixel, and the color reproducibility of the image is improved. .

  Although the quality of the image data is further improved by further increasing the AF pixel interval, it is desirable that the AF pixel interval is narrow from the viewpoint of focus detection accuracy. However, in any case, it is a minimum essential requirement that an imaging pixel used for interpolation of image data is arranged between the AF pixel and the AF pixel.

  FIG. 23 is a front view showing a detailed configuration of an image sensor 212B according to another modification. FIG. 24 is a diagram showing a schematic configuration of the image sensor 212B. In the image sensor 212 shown in FIG. 5, the AF pixels 311 are arranged in the horizontal direction (rows composed of green pixels and blue pixels). However, in the image sensor 212B shown in FIGS. 23 and 24, the AF pixels 311c are diagonally left. Arranged in the 45-degree direction. The AF pixels 311 are arranged in place of green pixels, and are arranged every other pixel in the arrangement direction. The AF pixel 311c has the same configuration of the photoelectric conversion unit as the AF pixel 311a shown in FIG. 7 is rotated 45 degrees clockwise.

FIG. 25 is a diagram illustrating image data interpolation processing for the image sensor 212B illustrated in FIGS. In the figure, the green pixel G data of the imaging pixel is Gn, the red pixel R data is Rn, the blue pixel B data is Bn, and the focus detection pixel A data is An (n = 1, 2,... ). In the following description, the interpolation process will be described by focusing on a specific pixel, but the same process is performed on all AF pixels arranged at the position of the green pixel. The image data I (A2) at the position of the AF pixel (data A2) arranged at the position of the green pixel is interpolated by the following equation.
I (A2) = (G1 + G3 + G4 + G5 + G6 + G7 + G8 + G10) / 8 (11)
By interpolating the image data with the equation (11), even when the image pattern 401 such as an edge or a thin line is superimposed on the AF pixel array, the image data G3 and G8 become data reflecting the image pattern 400. A good image interpolation result can be obtained.

Alternatively, the image data may be interpolated using the AF pixel data A2 by the following equation.
I (A2) = {k1 · (G1 + G3 + G4 + G5 + G6 + G7 + G8 + G10) / 8} + k2 · ks · A2 (12)

  In the AF pixel arrangement of the image sensor 212B shown in FIGS. 23 and 24, the AF pixel interval can be made narrower than the AF pixel arrangement shown in FIG. 5, so that improvement in image quality and focus detection accuracy are expected. it can.

  FIG. 26 is a front view illustrating a detailed configuration of an imaging element 212C according to another modification. In the image sensor 212 shown in FIG. 5, the image sensor 212A of the modification shown in FIG. 21, and the image sensor 212B of the other modification shown in FIG. 23, the AF pixel 311 has a pair of photoelectric conversion units in one pixel. In the imaging device 212C shown in FIG. 26, the AF pixels 313 and 314 include one photoelectric conversion unit in one pixel. In FIG. 26, an AF pixel 313 and an AF pixel 314 are paired and correspond to the AF pixel 311a of the imaging elements 211, 211A, and 211B described above.

  FIG. 27 is a diagram showing a configuration of the AF pixel shown in FIG. As shown in (a), the AF pixel 313 includes the microlens 10 and the photoelectric conversion unit 16. Further, as shown in (b), the focus detection pixel 314 includes the microlens 10 and the photoelectric conversion unit 17. These photoelectric conversion units 16 and 17 are projected onto the exit pupil of the interchangeable lens by the microlens 10 to form distance measuring pupils 92 and 93 shown in FIG. Therefore, a pair of image data used for focus detection can be obtained by the AF pixels 313 and 314. By providing one photoelectric conversion unit in the AF pixel, it is possible to prevent complication of the readout circuit configuration of the image sensor.

  FIG. 28 is a front view showing a detailed configuration of an imaging element 212D of another modification. This image sensor 212D is obtained by replacing the AF pixel 311a of the image sensor 212A shown in FIG. 21 with an AF pixel 313 and an AF pixel 314.

  FIG. 29 is a diagram illustrating a detailed configuration of an image sensor 212E according to another modification. In the imaging elements 212 to 212D described above, an example in which the array of imaging pixels is a Bayer array is shown. However, in this imaging element 212E, the green pixels G, blue pixels B, and red pixels R are arranged in the order of GBGR in the horizontal direction. It has an array. The AF pixel A is disposed at the position of the green pixel G, and three pixels of the green pixel G, the blue pixel B, and the red pixel R are disposed between the AF pixel and the AF pixel. The present invention can also be applied to such a pixel arrangement.

  In the imaging elements 212 to 212D described above, an example in which the color filters of the imaging pixels are arranged in a Bayer array is shown, but the configuration and arrangement of the color filters of the imaging pixels are not limited to this. For example, a complementary color filter (green: G, yellow: Ye, magenta: Mg, cyan: Cy) as shown in FIG. 4 may be employed.

  In the above-described imaging elements 212 to 212E, an example in which a color filter is not provided for the AF pixel has been shown. However, in the case where one filter (for example, a green filter) is provided among the color filters of the same color as the imaging pixel. However, the present invention can be applied. In this way, the interpolation accuracy of the image data of the AF pixel arranged at the position of the green pixel is improved by the interpolation calculation of equation (11).

  The imaging elements 212 to 212E described above can be formed as a CCD image sensor or a CMOS image sensor.

  In the above-described embodiment, an example in which the microlens of the imaging pixel and the microlens of the focus detection pixel have the same optical characteristics has been described. However, different optical characteristics may be used. Thereby, the exit pupil distance of the photographing optical system can be set separately to the optimum distance for imaging and the optimum distance for focus detection.

<< Imaging device >>
In the above-described embodiment, the digital still camera in which the interchangeable lens is mounted on the camera body has been described as an example. However, the imaging apparatus is not limited to the digital still camera including the interchangeable lens and the camera body. It can also be applied to an integrated digital still camera or video camera. The present invention can also be applied to a small camera module, a surveillance camera, or the like built in a mobile phone.

  As described above, according to the embodiment, when interpolating the image data at the position of the focus detection pixel, the image data of the imaging pixel in the arrangement direction of the focus detection pixel can be used. Accuracy can be improved and deterioration of image quality can be prevented. Further, even when a fine line pattern or an edge pattern is superimposed on the array of focus detection pixels, these patterns are not buried in the background.

The figure which shows the structure of one embodiment Diagram showing focus detection position on the shooting screen The figure which shows the Bayer arrangement | sequence of the pixel for imaging of an image pick-up element The figure which shows the complementary color arrangement | sequence of the pixel for imaging of an image pick-up element Front view showing detailed configuration of image sensor Front view of imaging pixels Front view of focus detection pixels The figure which shows the spectral characteristic of the green pixel for imaging of an image pick-up element, the red pixel for imaging, and the blue pixel for imaging The figure which shows the spectral characteristic of the pixel for AF Diagram showing spectral characteristics of green pixels Cross-sectional view of imaging pixels Cross-sectional view of AF pixel The figure explaining the focus detection principle by the pupil division type phase difference detection method Front view showing projection relationship on exit pupil plane The figure which shows the output ratio coefficient (pixel data value for imaging / pixel data value for focus detection) with respect to the aperture F value The flowchart which shows operation | movement of the digital still camera (imaging device) of one embodiment The figure explaining image shift detection calculation processing (correlation algorithm) The figure which shows the typical structure of the image pick-up element of one embodiment The figure explaining the image data interpolation process with respect to the image pick-up element of one embodiment The figure explaining the image data interpolation process with respect to the image pick-up element of one embodiment Front view showing a detailed configuration of an image sensor of a modification Schematic configuration diagram of image sensor Front view showing a detailed configuration of an image sensor of another modification The figure which shows the typical structure of an image sensor The figure explaining the image data interpolation process with respect to an image sensor Front view showing a detailed configuration of an image sensor of another modification The figure which shows the structure of the pixel for AF shown in FIG. Front view showing a detailed configuration of an image sensor of another modification The figure which shows the detailed structure of the image pick-up element of another modification. The figure which shows the interpolation calculation of the conventional image pick-up element.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Micro lens 11, 12, 13, 16, 17 Photoelectric conversion part 202 Interchangeable lens 212, 212A-212E Image pick-up element 214 Body drive control device 310a, 310b, 310c Imaging pixel 311a, 311b, 313, 314 Focus detection pixel

Claims (16)

An imaging device that regularly arranges a plurality of types of imaging pixels having different spectral sensitivity characteristics in a two-dimensional manner and images an image formed by an optical system,
A part of the imaging pixel along a predetermined direction is configured as a focus detection pixel, and has the same spectral sensitivity characteristic as the imaging pixel located at the position of the focus detection pixel in the regular array. The imaging pixels and the focus detection pixels are mixedly arranged on the pixel array in the predetermined direction so that the imaging pixels are arranged between the plurality of focus detection pixels. An image sensor. An imaging device that regularly arranges a plurality of types of imaging pixels having different spectral sensitivity characteristics in a two-dimensional manner and images an image formed by an optical system,
A part of the imaging pixels along a predetermined direction is configured as a focus detection pixel, and all types of the imaging pixels on the pixel array in the predetermined direction in the regular array have a plurality of the focal points. An imaging device, wherein the imaging pixels and the focus detection pixels are mixedly arranged on the pixel array in the predetermined direction so as to be arranged between the detection pixels. The imaging device according to claim 1 or 2,
The imaging device, wherein the plurality of types of imaging pixels are arranged in a square, and the predetermined direction is a direction of each side or diagonal line of the square arrangement. The imaging device according to claim 1 or 2,
The focus detection pixel receives a pair of light fluxes passing through different pupil portions along the predetermined direction in the exit pupil of the optical system. The imaging device according to claim 1 or 2,
The focus detection pixel includes a microlens and a photoelectric conversion unit, and the photoelectric conversion unit and the exit pupil of the optical system are conjugate with respect to the microlens. The imaging device according to claim 5,
The focus detection pixel includes a pair of photoelectric conversion units, and each photoelectric conversion unit receives a pair of light fluxes passing through different pupil portions of the optical system. The imaging device according to claim 5,
The focus detection pixel receives one of a pair of light beams passing through different pupil portions of the optical system by the photoelectric conversion unit. The image pickup device according to claim 7,
An image pickup device comprising a plurality of pairs of focus detection pixels that respectively receive a pair of light beams passing through different pupil portions of the optical system. In the image sensor according to any one of claims 1 to 8,
The imaging element, wherein the focus detection pixel has a spectral sensitivity characteristic different from that of the imaging pixel. In the image sensor according to any one of claims 1 to 8,
The imaging element, wherein the focus detection pixels have the same spectral sensitivity characteristics as the imaging pixels located at the position of the focus detection pixels in the regular array. In the imaging device according to any one of claims 1 to 10,
An imaging device comprising three types of imaging pixels that receive RGB light. The imaging device according to claim 11,
An image pickup device, wherein the three types of image pickup pixels that receive RGB are Bayer-arranged. In the imaging device according to any one of claims 1 to 12,
The imaging pixel includes a microlens and a photoelectric conversion unit, and the photoelectric conversion unit and the exit pupil of the optical system are conjugate with respect to the microlens. The image sensor according to claim 13, wherein
An image sensor, wherein the optical characteristics of the microlens of the imaging pixel are different from the optical characteristics of the microlens of the focus detection pixel. The imaging device according to any one of claims 1 to 10,
A focus detection device comprising: focus detection means for detecting a focus adjustment state of the optical system based on output data of the focus detection pixels of the image sensor. The image sensor according to any one of claims 1 to 14,
An imaging apparatus comprising: an interpolation calculation means for performing interpolation calculation on image data at the position of the focus detection pixel based on output data of the imaging pixel in the vicinity of the focus detection pixel of the imaging element. .

Images