JP5133533B2 - Imaging device - Google Patents

Description

The present invention relates to an imaging apparatus.

  A color imaging apparatus that uses an imaging device having focus detection pixels (hereinafter referred to as AF pixels) and interpolates image data of AF pixels from image data of imaging pixels around the AF pixels. Is known (see, for example, Patent Document 1).

Prior art documents related to the invention of this application include the following.
JP 2000-305010 A

However, the above-described conventional color imaging device has the following problems.
When a thin line pattern 402 (shown by a broken line in the drawing) in which the green component or the blue component is different from the background green component or the blue component is superimposed on the AF pixel row of the image sensor shown in FIG. In the interpolation of the image data according to the equations (2) and (2), the image data on the AF pixel column has the same color component as the background, and the fine line pattern 402 cannot be reproduced as the image data.
Further, when not only the fine line pattern but also the edge pattern is superimposed on the AF pixel column, the edge pattern cannot be faithfully reproduced by the conventional image data interpolation, and noise such as false color may occur.

(1) The imaging device according to the invention of claim 1 has a plurality of first imaging pixels having a first spectral sensitivity characteristic, a plurality of second imaging pixels having a second spectral sensitivity characteristic, and a third spectral sensitivity characteristic. Three types of imaging pixels including a plurality of third imaging pixels are arranged according to a two-dimensional regular arrangement, and the first imaging pixel and the second imaging pixel are arranged alternately. In some cases, a focus detection pixel is arranged by replacing at least the first imaging pixel, and among the plurality of focus detection pixels, a first detection pixel is disposed between the focus detection pixels and the focus detection pixels adjacent to each other. One imaging pixel and a second imaging pixel are arranged, an image sensor that captures an image formed by the optical system, and image data at a pixel position of the focus detection pixel are converted into the focus detection pixel and the focus It is arranged between the focus detection pixels adjacent to the detection pixels. A plurality of first imaging pixels adjacent to each other in the diagonal direction, the horizontal direction, and the vertical direction with respect to the focus detection pixel including the first imaging pixel of the first imaging pixel and the second imaging pixel. Interpolating means for interpolating based on the pixel data of the pixel, and each of the three types of imaging pixels includes a red filter, a green filter and a blue filter, and the three types of imaging pixels are arranged on the imaging device according to the Bayer array. It is characterized by that.
(2) In the imaging device according to the invention of claim 2, the focus detection pixels are arranged in the imaging device in place of the first imaging pixels and the second imaging pixels in a part of the pixel row, and the interpolation means The image data of the focus detection pixels arranged by replacing one image pickup pixel is obtained from the first image pickup pixels arranged between the focus detection pixels and the focus detection pixels adjacent to the focus detection pixels. Interpolation is performed based on the pixel data, and the image data of the focus detection pixels arranged by replacing the second imaging pixels is transferred between the focus detection pixels and the focus detection pixels adjacent to the focus detection pixels. The interpolation is performed based on the pixel data of the second imaging pixels arranged in the area.
(3) In the imaging device according to the invention of claim 3, the first imaging pixel and the second imaging pixel are squarely arranged on the imaging element, and the extending direction of the pixel column is the direction of each side or diagonal line of the square array. It is characterized by doing.
(4) In the image pickup device according to the invention of claim 4, the focus detection pixels receive a pair of light fluxes passing through different pupil portions along the extending direction of the pixel row at the exit pupil of the optical system. .
(5) In the imaging device according to the invention of claim 5, the focus detection pixel includes a microlens and a photoelectric conversion unit, and the photoelectric conversion unit and the exit pupil of the optical system are conjugate with respect to the microlens. .
(6) In the imaging device according to the invention of claim 6, the focus detection pixel includes a pair of photoelectric conversion units, and each of the pair of photoelectric conversion units receives a pair of light beams passing through different pupil portions of the optical system. It is characterized by that.
(7) The image pickup apparatus according to the invention of claim 7 is characterized in that one of a pair of light beams passing through different pupil portions of the optical system is received by the photoelectric conversion unit provided in the focus detection pixel. (8) The image pickup apparatus according to the invention of claim 8 is characterized in that a plurality of pairs of focus detection pixels that respectively receive a pair of light beams passing through different pupil portions of the optical system are arranged.
(9) In the imaging device according to the invention of claim 9, the focus detection pixel has spectral sensitivity characteristics different from those of the first imaging pixel and the second imaging pixel.
(10) In the imaging device according to the invention of claim 10, the focus detection pixels arranged by replacing the first imaging pixels constituting a regular array have the same spectral sensitivity as the replaced first imaging pixels. It has the characteristics.
(11) In the imaging device according to the invention of claim 11, each of the first imaging pixel and the second imaging pixel includes a microlens and a photoelectric conversion unit, and the photoelectric conversion unit and the exit pupil of the optical system with respect to the microlens. Is conjugated.
(12) In the imaging device according to the invention of claim 12, the optical characteristics of the microlens provided in each of the first imaging pixel and the second imaging pixel are different from the optical characteristics of the microlens provided in the focus detection pixel. It is characterized by that.
(13) The image pickup apparatus according to the thirteenth aspect of the invention is further characterized by further comprising focus detection means for detecting the focus adjustment state of the optical system based on the output data of the focus detection pixels of the image pickup device.
(14) In the imaging device according to the fourteenth aspect of the invention, the regular array is a square array of the first imaging pixels and the second imaging pixels, and replaces the first imaging pixels constituting the square array. The focus detection pixels adjacent to the focus detection pixels are not arranged at the pixel positions closest to the pixel positions of the focus detection pixels arranged in the square direction with respect to the pixel positions of the focus detection pixels. To do.
(15) An image pickup apparatus according to a fifteenth aspect of the invention has two first imaging pixels having green spectral characteristics, one second imaging pixel having blue spectral characteristics, and red spectral characteristics. A unit cell of 2 rows and 2 columns composed of 3 imaging pixels is two-dimensionally arranged as a Bayer array, and any one of the horizontal direction, the vertical direction, and the diagonal direction of the array of imaging pixels A plurality of focus detection pixels are arranged in place of each of the plurality of first imaging pixels among the imaging pixels arranged on the straight line, and the focus detection pixels adjacent to each other among the plurality of focus detection pixels. And at least one first imaging pixel is arranged between the focus detection pixels and four first detection pixels in the diagonal direction around each focus detection pixel. Imaging pixels next to each focus detection pixel Two first image-capturing pixels are arranged in a horizontal direction with each focus detection pixel being placed one by one from each focus detection pixel, and each focus detection pixel is placed in a vertical direction. An image sensor in which two first imaging pixels are arranged one pixel at a time from the focus detection pixels, and 4 arranged adjacent to each focus detection pixel diagonally across each focus detection pixel. Pixel data of two first imaging pixels, pixel data of two first imaging pixels arranged one pixel from each focus detection pixel in the horizontal direction across each focus detection pixel, and each focus detection Interpolation calculation for interpolating the image data at the position of each focus detection pixel based on the pixel data of two first imaging pixels arranged one pixel from each focus detection pixel in the vertical direction across the pixel Means .
(16) In the imaging device according to the sixteenth aspect, there are two first imaging pixels having green spectral characteristics, one second imaging pixel having blue spectral characteristics, and red spectral characteristics. A unit cell of 2 rows and 2 columns composed of 3 imaging pixels is arranged two-dimensionally as a Bayer array, and a plurality of focus detection pixels are arranged by replacing each of the plurality of second imaging pixels. The focus detection pixels of the plurality of focus detection pixels are positioned closest to each focus detection pixel in the horizontal direction, and are arranged one pixel from each focus detection pixel in the horizontal direction with the focus detection pixels interposed therebetween. Two second imaging pixels and each focus detection pixel are positioned closest to each other in the vertical direction, and are arranged one pixel from each focus detection pixel in the vertical direction across each focus detection pixel. Two second shots The image pixel and the focus detection pixel are positioned closest to each other in the diagonal direction, and are arranged one pixel from each focus detection pixel in the diagonal direction across each focus detection pixel. For each of the two second imaging pixels, the imaging element that is not replaced with a plurality of focus detection pixels and the position closest to each focus detection pixel in the horizontal direction and sandwiching each focus detection pixel The pixel data of the two second imaging pixels arranged one pixel from each focus detection pixel in the horizontal direction and the position closest to each focus detection pixel in the vertical direction, and for each focus detection The pixel data of two second imaging pixels arranged one pixel from each focus detection pixel in the vertical direction across the pixel, and located closest to each focus detection pixel in the diagonal direction For each focus detection Interpolation calculation is performed on the image data at the position of each focus detection pixel based on the pixel data of the four second imaging pixels arranged one pixel from each focus detection pixel in a diagonal direction across the element. And an interpolation calculation means .
(17) The imaging device according to the invention of claim 17 is a plurality of first imaging pixels having green spectral characteristics, a plurality of second imaging pixels having blue spectral characteristics, and a plurality having red spectral characteristics. The third imaging pixels are arranged in accordance with a two-dimensional regular arrangement, and are replaced with a plurality of first imaging pixels to form a pair of first focus detection pixels and second focus detection pixels. A plurality of pairs of image pickup elements that pick up an image formed by the optical system and have a plurality of second image pickup pixels equal to a plurality of third image pickup pixels. The number of pixels of the first imaging pixels is twice the number of pixels of the plurality of second imaging pixels, and the first focus detection pixels and the second focus detection pixels are the same as the plurality of first imaging pixels. First focus detection pixel and second focus detection pixel having a green spectral characteristic Each includes one microlens and one photoelectric conversion unit, and the photoelectric conversion unit and the exit pupil of the optical system are conjugate with respect to the microlens, and the optical conversion unit of the first focus detection pixel includes the optical system. One of the pair of light fluxes passing through different pupil portions is received, and the other of the pair of light fluxes is received by the photoelectric conversion unit included in the second focus detection pixel, and adjacent to each other in the arrangement direction of the pair of light fluxes. Image sensor in which first imaging pixels are arranged between the first focus detection pixels arranged and between the second focus detection pixels arranged adjacent to each other in the arrangement direction of the pair of light beams. And pixel data of the first imaging pixels arranged between the first focus detection pixels arranged adjacent to each other in the arrangement direction of the pair of light fluxes, and pixel data of the first focus detection pixels Including at least two Based on the pixel data, the image data at the position of the first focus detection pixel is interpolated and the first focus pixel arranged between the second focus detection pixels arranged adjacent to each other in the arrangement direction of the pair of light beams. Interpolation calculation means for interpolating image data at the position of the second focus detection pixel based on at least two pieces of pixel data including pixel data of the imaging pixel and pixel data of the second focus detection pixel It is characterized by.

  According to the present invention, it is possible to accurately reproduce the image data at the position of the focus detection pixel and prevent deterioration of the image quality.

  An embodiment in which an image sensor according to an embodiment of the present invention is applied to a digital still camera will be described. FIG. 1 shows the configuration of an embodiment. A digital still camera 201 according to an embodiment includes an interchangeable lens 202 and a camera body 203, and the interchangeable lens 202 is attached to the camera body 203 by a mount unit 204.

  The interchangeable lens 202 includes a lens drive control device 206, a zooming lens 208, a lens 209, a focusing lens 210, a diaphragm 211, and the like. The lens drive control device 206 includes peripheral components such as a microcomputer and a memory. The lens drive control device 206 controls driving of the focusing lens 210 and the aperture 211, detects the state of the aperture 211, the zooming lens 208 and the focusing lens 210, and body drive control described later. Transmission of lens information to the device 214 and reception of camera information are performed.

  The camera body 203 includes an imaging element 212, a body drive control device 214, a liquid crystal display element drive circuit 215, a liquid crystal display element 216, an eyepiece lens 217, a memory card 219, and the like. Pixels, which will be described later, are arranged in a two-dimensional manner on the imaging element 212, and are arranged on the planned imaging plane of the interchangeable lens 202 to capture a subject image formed by the interchangeable lens 202. Although details will be described later, focus detection pixels (hereinafter referred to as AF pixels) are arranged at predetermined focus detection positions of the image sensor 212.

  The body drive control device 214 includes a microcomputer and peripheral components such as a memory. The body drive control device 214 reads the image signal from the image sensor 212, corrects the image signal, detects the focus adjustment state of the interchangeable lens 202, and outputs from the lens drive control device 206. It receives lens information, transmits camera information (defocus amount), and controls the operation of the entire digital still camera. The body drive control device 214 and the lens drive control device 206 communicate via the electrical contact portion 213 of the mount portion 204 to exchange various information.

  The liquid crystal display element driving circuit 215 drives a liquid crystal display element 216 of a liquid crystal viewfinder (EVF: electrical viewfinder). The photographer can observe an image displayed on the liquid crystal display element 216 via the eyepiece lens 217. The memory card 219 is removable from the camera body 203 and is a portable storage medium that stores and stores image signals.

  The subject image formed on the image sensor 212 through the interchangeable lens 202 is photoelectrically converted by the image sensor 212 and the output is sent to the body drive controller 214. The body drive control device 214 calculates the defocus amount at a predetermined focus detection position based on the output data of the AF pixels on the image sensor 212 and sends this defocus amount to the lens drive control device 206. The body drive control device 214 stores an image signal generated based on the output of the image sensor 212 in the memory card 219 and sends the image signal to the liquid crystal display element drive circuit 215 to display an image on the liquid crystal display element 216. Let

  The camera body 203 is provided with operation members (not shown) (shutter buttons, focus detection position setting members, etc.), and the body drive control device 214 detects operation state signals from these operation members. The corresponding operations (imaging operation, focus detection position setting operation, image processing operation) are controlled.

  The lens drive control device 206 changes the lens information according to the focusing state, zooming state, aperture setting state, aperture opening F value, and the like. Specifically, the lens drive control device 206 monitors the positions of the lenses 208 and 210 and the diaphragm position of the diaphragm 211, calculates lens information according to the monitor information, or monitors from a lookup table prepared in advance. Select lens information according to the information. The lens drive control device 206 calculates a lens drive amount based on the received defocus amount, and drives the focusing lens 210 to a focal point by a drive source such as a motor (not shown) based on the lens drive amount.

  FIG. 2 shows a focus detection position on the photographing screen. Focus detection areas 101 to 105 are arranged at five locations on the photographing screen 100. These focus detection areas 101 to 105 indicate areas in which AF pixels of the image sensor 212 (to be described later) sample subject images on the photographing screen 100, and are used for AF in the longitudinal direction of the focus detection areas 101 to 105 indicated by rectangles. Pixels are arranged linearly. The photographer manually selects a focus detection area from a plurality of focus detection areas 101 to 105 by a focus detection area selection operation member (not shown) according to the shooting composition.

  FIG. 3 shows an array of imaging pixels of the image sensor 212. The imaging pixels that are two-dimensionally arranged on the substrate of the imaging element 212 are provided with a Bayer array color filter as shown in FIG. 3 shows an arrangement of color filters for four pixels (2 × 2) of image pickup pixels. On the image pickup element 212, there are four pixel image pickup pixel units having the Bayer arrangement shown in FIG. Arranged in two dimensions. In the Bayer array, two pixels of the green filter (G) are arranged on a diagonal line, and a blue filter (B) and a red filter (R) are arranged on the diagonal line one pixel at a time. In the Bayer array, the density of green pixels is higher than the density of red pixels and blue pixels.

  FIG. 8 shows spectral characteristics of the imaging green pixel, imaging red pixel, and imaging blue pixel of the imaging device 212. This spectral characteristic is a spectral characteristic that combines the spectral sensitivity of each color filter shown in FIG. 3, the spectral sensitivity of a photodiode (not shown) that performs photoelectric conversion, and the spectral characteristic of an infrared cut filter (not shown). The pixels, red pixels, and blue pixels have different light wavelength regions (having different spectral sensitivities) in order to perform color separation. In particular, the spectral characteristic of the green pixel has a sensitivity close to the peak of relative luminous sensitivity as shown in FIG.

  Note that a color filter is not disposed in the AF pixel to increase the amount of light, and the spectral characteristics thereof are the spectral sensitivity of a photodiode (not shown) that performs photoelectric conversion and the spectral characteristic of an infrared cut filter (not shown). The spectral characteristics shown in FIG. The spectral characteristic of the AF pixel is a spectral characteristic obtained by adding the spectral characteristics of the green pixel, red pixel, and blue pixel shown in FIG. 8, and the light wavelength region of the sensitivity is the sensitivity of the green pixel, red pixel, and blue pixel. It covers the optical wavelength region.

  FIG. 5 is a front view showing a detailed configuration of the image sensor 212. In FIG. 5, the vicinity of one focus detection area on the image sensor 212 is shown enlarged. The image sensor 212 includes imaging pixels (green pixel 310a, blue pixel 310b, red pixel 310c) and AF pixels (AF pixel 311a replacing the green pixel and AF pixel 311b replacing the blue pixel). The As shown in FIG. 6, the imaging pixels 310a, 310b, and 310c include a microlens 10, a photoelectric conversion unit 11, and a color filter (not shown). As shown in FIG. 7, the AF pixels 311 a and 311 b include a microlens 10 and a pair of photoelectric conversion units 12 and 13.

  The photoelectric conversion unit 11 of the imaging pixels 310a, 310b, and 310c is designed so as to receive all the light beams that pass through the exit pupil (for example, F1.0) of the bright interchangeable lens by the microlens 10. The pair of photoelectric conversion units 12 and 13 of the AF pixels 311a and 311b are designed in such a shape that the microlens 10 receives all the light beams that pass through a specific exit pupil (for example, F2.8) of the interchangeable lens.

  As shown in FIG. 5, the imaging pixels 310a, 310b, and 310c arranged two-dimensionally are provided with RGB Bayer color filters. The AF pixels 311a and 311b are arranged on a straight line in a row (column) where the green pixels 310a and the blue pixels 310b of the imaging pixels are to be arranged, and the AF pixels 311a and 311b are separated by two pixels. , One green pixel 310a and one blue pixel 310b are arranged there.

  By arranging the AF pixels 311a and 311b in the row (column) where the green pixels 310a and the blue pixels 310b of the imaging pixels are to be arranged, the pixel signal at the position of the AF pixel 311 is interpolated by pixel interpolation described later. Even if some errors occur, it can be made inconspicuous to the human eye. This is because the human eye is more sensitive to red than blue and because the density of green pixels is higher than that of blue and red pixels, the contribution to image degradation for a single pixel defect of green pixels is small.

  The AF pixels 311a and 311b are arranged in the focus detection area. By arranging the green pixel 310a and the blue pixel 310b between the AF pixels 311a and 311b, the image data of the positions of the AF pixels 311a and 311b is interpolated between the image data of the green pixel 310a and the blue pixel 310b. The image data can be favorably interpolated even when an image pattern such as an edge or a thin line is superimposed on the array of AF pixel columns.

  FIG. 11 is a cross-sectional view of the imaging pixels 310a, 310b, and 310c. In the imaging pixels 310 a, 310 b, and 310 c, the microlens 10 is disposed in front of the imaging photoelectric conversion unit 11, and the photoelectric conversion unit 11 is projected forward by the microlens 10. The photoelectric conversion unit 11 is formed on the semiconductor circuit substrate 29. The color filter (not shown) described above is disposed between the microlens 10 and the photoelectric conversion unit 11.

  FIG. 12 is a cross-sectional view of the AF pixels 311a and 311b. In the AF pixels 311a and 311b, the microlens 10 is disposed in front of the focus detection photoelectric conversion units 12 and 13, and the photoelectric conversion units 12 and 13 are projected forward by the microlens 10. The photoelectric conversion units 12 and 13 are formed on the semiconductor circuit substrate 29.

  Here, the principle of focus detection by the pupil division type phase difference detection method will be described with reference to FIG. In the figure, reference numeral 90 denotes an exit pupil set at a distance d4 in front of the microlens arranged on the planned imaging plane of the interchangeable lens. The distance d4 is a distance determined according to the curvature and refractive index of the microlens and the distance between the microlens and the photoelectric conversion unit, and is hereinafter referred to as a distance measuring pupil distance. 91 is an optical axis of the interchangeable lens, 50 and 60 are microlenses, 52 and 53 are a pair of photoelectric conversion units for focus detection pixels, 61 is a photoelectric conversion unit for imaging pixels, 72 and 73 are focus detection light fluxes, and 74 is This is the imaging light flux. Reference numeral 92 denotes a region of the photoelectric conversion unit 52 projected by the microlens 50 (hereinafter referred to as a ranging pupil), and reference numeral 93 denotes a region of the photoelectric conversion unit 53 projected by the microlens 50 (hereinafter referred to as a ranging pupil). , 94 is a region (hereinafter referred to as a distance measuring pupil) of the photoelectric conversion unit 61 projected by the micro lens 60.

  In FIG. 13, an AF pixel (having a microlens 50 and a pair of photoelectric conversion units 52 and 53) on an optical axis 91 and an adjacent imaging pixel (having a microlens 60 and a photoelectric conversion unit 61). In other AF pixels as well, the pair of photoelectric conversion units receive light beams coming from the pair of distance measuring pupils 92 and 93 to the microlenses, and in other imaging pixels. In addition, the photoelectric conversion units receive the light beams coming from the pair of regions 94 to the respective microlenses. The arrangement direction of the AF pixels is matched with the arrangement direction of the pair of distance measuring pupils.

  The microlens 50 is disposed in the vicinity of the planned imaging plane of the optical system, and the shape of the pair of photoelectric conversion units 52 and 53 disposed behind the microlens 50 disposed on the optical axis 91 is the microlens 50. Is projected onto the exit pupil 90 separated by the projection distance d4, and the projection shape forms distance measuring pupils 92 and 93. That is, the projection direction of each pixel is determined so that the projection shapes (ranging pupils 92 and 93) of the photoelectric conversion units of the focus detection pixels coincide on the exit pupil 90 at the projection distance d4. In the AF pixel, with respect to the microlens, the photoelectric conversion unit and the exit pupil of the interchangeable lens (imaging optical system) are conjugate.

  The photoelectric conversion unit 52 outputs a signal corresponding to the intensity of the image formed on the microlens 50 by the focus detection light beam 72 that passes through the distance measuring pupil 92 and faces the microlens 50. On the other hand, the photoelectric conversion unit 53 outputs a signal corresponding to the intensity of the image formed on the microlens 50 by the focus detection light beam 73 that passes through the distance measuring pupil 93 and faces the microlens 50. A large number of such AF pixels are arranged in a straight line, and the output of the pair of photoelectric conversion units of each pixel is collected into an output group corresponding to the distance measurement pupil 92 and the distance measurement pupil 93, thereby measuring the distance measurement pupil 92 and the measurement pupil 92. Information on the intensity distribution of a pair of images formed on the AF pixel row by the focus detection light beams that pass through the distance pupils 93 is obtained.

  By applying an image shift detection calculation process (correlation process, phase difference detection process) to be described later to this information, the image shift amount of a pair of images is detected by a so-called pupil division method. Further, by multiplying the image shift amount by a predetermined conversion coefficient, the deviation of the current imaging plane (the imaging plane at the focus detection position corresponding to the position of the microlens array on the planned imaging plane) with respect to the planned imaging plane (Defocus amount) can be calculated.

  The microlens 60 is disposed in the vicinity of the planned imaging plane of the optical system, and the shape of the photoelectric conversion unit 61 disposed behind the microlens 60 is on the exit pupil 90 separated from the microlens 60 by the projection distance d4. The projected shape forms a region 94. That is, the projection direction of each pixel is determined so that the projection shape (region 94) of the photoelectric conversion unit of each imaging pixel matches on the exit pupil 90 at the projection distance d4. The photoelectric conversion unit 61 outputs a signal corresponding to the intensity of the image formed on the microlens 60 by the imaging light flux 74 that passes through the region 94 and travels toward the microlens 60.

  By arranging a large number of such imaging pixels two-dimensionally and detecting the output of the photoelectric conversion unit of each pixel, information (image data) relating to the two-dimensional light intensity distribution of the image on the screen can be obtained. .

  FIG. 14 is a front view showing a projection relationship on the exit pupil plane. The circumscribed circle of the distance measuring pupils 92 and 93 obtained by projecting a pair of photoelectric conversion units of the AF pixels onto the exit pupil plane 90 by the microlens is a predetermined aperture F value (ranging pupil F value) when viewed from the imaging plane. Here, it is F2.8). In addition, a region 94 in which the photoelectric conversion unit of the imaging pixel is projected onto the exit pupil plane 90 by the microlens is a wide region including the distance measuring pupils 92 and 93. The area on the distance measuring pupil plane 90 corresponding to the brightest aperture (F1) is 95, and the area on the distance measuring pupil plane 90 corresponding to the dark aperture (F5.6) is 96.

  Since the area 94 is wider than the areas 95 and 96, the light flux received by the imaging pixels is limited by the aperture, and the data of the imaging pixels changes according to the aperture value. The distance measuring pupils 92 and 93 are narrower than the region 95 and wider than the region 96. Therefore, if the aperture value is brighter than F2.8, the AF pixel data does not change even if the aperture value changes. If the aperture value is darker than F2.8, the AF pixel data does not change. Varies depending on the aperture value.

  When the imaging pixel data value / focus detection pixel data value is graphed as an output ratio coefficient (normalized as 1 when the aperture value is F2.8), the aperture value is F2.8 as shown in FIG. When the aperture value is darker, the constant value is 1 regardless of the aperture value. When the aperture value is brighter than F2.8, the aperture value increases as the aperture value becomes brighter.

  FIG. 16 is a flowchart illustrating the operation of the digital still camera (imaging device) according to the embodiment. The body drive control device 214 repeatedly executes this operation while the camera is powered on. When the power of the camera is turned on in step 100, the process proceeds to step 110, where the data of the imaging pixels are read out and displayed on the electronic viewfinder. When thinning out and reading out the data of the imaging pixels, the display quality can be improved by performing the thinning out reading with a setting that does not include AF pixels as much as possible.

  In step 120, data is read from the AF pixel column. Here, it is assumed that the focus detection area is selected by the user using a selection operation member (not shown). Next, in step 130, an image shift detection calculation process is performed based on a pair of image data corresponding to the AF pixel row, an image shift amount is calculated, and a defocus amount is further calculated.

Here, the image shift detection calculation process (correlation algorithm) will be described with reference to FIG. Assuming that a pair of data corresponding to a certain AF pixel column is ei, fi (where i = 1 to m), first, a correlation amount C (L) is obtained by a differential correlation algorithm expressed by the following equation.
C (L) = Σ | e (i + L) −f (i) | (1)
In the equation (1), Σ represents a total operation of i = p to q. L is an integer, and is a relative shift amount with the detection pitch of a pair of data as a unit. Further, the range of L is Lmin to Lmax (-5 to +5 in the figure). Further, the range of the parameter i is from p to q, and is determined so as to satisfy the condition of 1 ≦ p <q ≦ m.

As shown in FIG. 17A, the calculation result of the equation (4) shows that the correlation amount C (L) is obtained when the shift amount L = kj (kj = 2 in FIG. 17A) with a high correlation between a pair of data. Be minimized. Next, a shift amount x that gives a minimum value C (L) min = C (x) with respect to a continuous correlation amount is obtained using a three-point interpolation method according to equations (2) to (5).
x = kj + D / SLOP (2),
C (x) = C (kj) − | D | (3),
D = {C (kj-1) -C (kj + 1)} / 2 (4),
SLOP = MAX {C (kj + 1) -C (kj), C (kj-1) -C (kj)} (5)
In addition, the defocus amount DEF of the subject image plane with respect to the planned image formation plane can be obtained from the following equation using the shift amount x obtained by the equation (1).
DEF = KX · PY · x (6)
In Equation (6), PY is a detection pitch, and KX is a conversion coefficient determined by the size of the opening angle of the center of gravity of the pair of distance measuring pupils.

  Whether or not the calculated defocus amount DEF is reliable is determined as follows. As shown in FIG. 17B, when the degree of correlation between a pair of data is low, the value of the minimum value C (X) of the interpolated correlation amount is large. Therefore, when C (X) is equal to or greater than a predetermined value, it is determined that the reliability is low. Alternatively, in order to normalize C (X) with the contrast of data, it is determined that the reliability is low when the value obtained by dividing C (X) by SLOP which is a value proportional to the contrast is equal to or greater than a predetermined value. Alternatively, if SLOP that is proportional to the contrast is equal to or less than a predetermined value, it is determined that the subject has low contrast and the reliability of the calculated defocus amount DEF is low.

  As shown in FIG. 17C, when the correlation between the pair of data is low and there is no drop in the correlation amount C (L) between the shift ranges Lmin to Lmax, the minimum value C (X) is obtained. In such a case, it is determined that the focus cannot be detected. When focus detection is possible, the defocus amount is calculated by multiplying the calculated image shift amount by a predetermined conversion coefficient.

  In step 140, it is checked whether or not the focus is close, that is, whether the calculated absolute value of the defocus amount is within a predetermined value. If it is determined that the lens is not in focus, the process proceeds to step 150, the defocus amount is transmitted to the lens drive control device 216, and the focusing lens 210 of the interchangeable lens 202 is driven to the focus position based on the defocus amount. Returning to 110, the above-described operation is repeated. If the focus cannot be detected, the process branches to step 150, a scan drive command is transmitted to the lens drive control device 216, and the focusing lens 210 of the interchangeable lens 202 is driven to scan from infinity to close, and the process returns to step 110. Repeat the above operation.

  On the other hand, if it is determined that the focus is close, the process proceeds to step 160, where it is determined whether or not a shutter release has been performed by operating a shutter button (not shown), and if it is determined that the shutter release has not been performed, the process returns to step 110. The above operation is repeated. If it is determined that the shutter release has been performed, the process proceeds to step 170, an aperture adjustment command is transmitted to the lens drive control device 216, and the aperture value of the interchangeable lens 202 is set to the control F value (the F value set by the user or automatically). Set. When the aperture control is completed, the image sensor 212 performs an image capturing operation, and image data is read from the image capturing pixels of the image sensor 212 and all the AF pixels.

  In step 180, the pixel data at each pixel position in the AF pixel column is interpolated based on the AF pixel data and the surrounding imaging pixel data. This interpolation process will be described later. In the next step 190, image data composed of the pixel data for imaging and the interpolated data is stored in the memory card, and the process returns to step 110 to repeat the above-described operation.

  Next, image data interpolation processing will be described. FIG. 18 is a diagram illustrating a schematic configuration of the image sensor 212 illustrated in FIG. 5 according to an embodiment. A green pixel G310a, a blue pixel B310b, and a red pixel R310c are arranged in a Bayer array. In a row corresponding to a part of the focus detection area, AF pixels A are arranged in a row, and two imaging pixels G and B in between. It is arranged across An AF pixel A311a is arranged at a position where the green pixel G should be, and an AF pixel A311b is arranged at a position where the blue pixel B should be.

FIG. 19 is a diagram illustrating image data interpolation processing for the image sensor 212 according to the embodiment. In the figure, the green pixel G data of the imaging pixel is Gn, the red pixel R data is Rn, the blue pixel B data is Bn, and the focus detection pixel A data is An (n = 1, 2,... ). Note that although the interpolation process will be described by paying attention to a specific pixel, the same process is performed on all the AF pixels arranged at the position of the green pixel. The image data I (A1) at the position of the AF pixel (data A1) arranged in the green pixel is interpolated by the following equation.
I (A1) = (G2 + G4 + G5 + G6 + G7 + G9 + G11 + G12) / 8 (7)
The image data G11 and G12 reflect the image pattern 400 even when the image pattern 400 such as an edge or a thin line is superimposed on the AF pixel array by performing the interpolation calculation on the image data as in the equation (7). Therefore, a good image interpolation result can be obtained.

  As a method for calculating the image data I (A1) at the position of the AF pixel by performing statistical processing, not only a simple average calculation as shown in the equation (7) but also a direction perpendicular to the AF pixel. Pixel data at a plurality of peripheral positions may be obtained by linear interpolation, interpolation by a second-order or higher-order multi-order expression, or median processing.

Further, the focus detection pixel data A1 may be used for the interpolation calculation of the image data.
I (A1) = {k1 · (G2 + G4 + G5 + G6 + G7 + G9 + G11 + G12) / 8} + k2 · ks · A1 (8)
In the equation (8), the coefficient ks is an output ratio coefficient shown in FIG. 15, and an output ratio coefficient corresponding to a control F value (aperture F value) at the time of imaging is selected. The coefficients k1 and k2 are weighting coefficients when the data interpolated from the surrounding pixel data and the AF pixel data are mixed, and satisfy k1 + k2 = 1. Since the AF pixel data A1 is data corresponding to the luminance at the position of the AF pixel, the high-frequency component of the image is obtained by using the AF pixel data A1 for the interpolation calculation of the image data as shown in the equation (8). Reproducibility can be improved.

FIG. 20 is a diagram illustrating image data interpolation processing for the image sensor 212 according to one embodiment. In the figure, the green pixel data of the imaging pixel is Gn, the red pixel data is Rn, the blue pixel data is Bn, and the AF pixel data is An (n = 1, 2,...). In the following description, the interpolation process will be described by focusing on a specific pixel, but the same process is performed on all AF pixels arranged at the positions of the blue pixels. The image data I (A21) at the position of the AF pixel (data A21) arranged at the blue pixel position is interpolated by the following equation.
I (A21) = (B21 + B22 + B23 + B24 + B25 + B26 + B27 + B28) / 8 (9)
By interpolating the image data with the equation (9), even when the image pattern 400 such as an edge or a thin line is superimposed on the AF pixel array, the image data B27 and B28 become data reflecting the image pattern 400. A good image interpolation result can be obtained.

Similarly to the equation (9), the AF pixel data A21 may be used for interpolation of image data.
I (A21) = {k1 · (B21 + B22 + B23 + B24 + B25 + B26 + B27 + B28) / 8} + k2 · ks · A21 (10)

<< Modification of Embodiment >>
FIG. 21 is a front view showing a detailed configuration of an imaging element 212A according to a modification. FIG. 22 is a schematic configuration diagram of the image sensor 212A. In the imaging device 212 shown in FIG. 5, two imaging pixels (green pixel and blue pixel) are arranged between the AF pixels 311, whereas in FIGS. 21 and 22, 3 between the AF pixels 311. Two imaging pixels (two green pixels and one blue pixel) are arranged. The AF pixel 311a is disposed at the position of the green pixel 310a.

  Also in the configuration of the image sensor 212A shown in FIGS. 21 and 22, the image data of the AF pixel arranged at the position of the green pixel can be interpolated by the equations (10) and (11). In this image sensor 212A, since the AF pixel is arranged by replacing only the green pixel, the blue component and the red component of the image can be detected without being affected by the arrangement of the AF pixel, and the color reproducibility of the image is improved. .

  Although the quality of the image data is further improved by further increasing the AF pixel interval, it is desirable that the AF pixel interval is narrow from the viewpoint of focus detection accuracy. However, in any case, it is a minimum essential requirement that an imaging pixel used for interpolation of image data is arranged between the AF pixel and the AF pixel.

  FIG. 23 is a front view showing a detailed configuration of an image sensor 212B according to another modification. FIG. 24 is a diagram showing a schematic configuration of the image sensor 212B. In the image sensor 212 shown in FIG. 5, the AF pixels 311 are arranged in the horizontal direction (rows composed of green pixels and blue pixels). However, in the image sensor 212B shown in FIGS. 23 and 24, the AF pixels 311c are diagonally left. Arranged in the 45-degree direction. The AF pixels 311 are arranged in place of green pixels, and are arranged every other pixel in the arrangement direction. The AF pixel 311c has the same configuration of the photoelectric conversion unit as the AF pixel 311a shown in FIG. 7 is rotated 45 degrees clockwise.

FIG. 25 is a diagram illustrating image data interpolation processing for the image sensor 212B illustrated in FIGS. In the figure, the green pixel G data of the imaging pixel is Gn, the red pixel R data is Rn, the blue pixel B data is Bn, and the focus detection pixel A data is An (n = 1, 2,... ). In the following description, the interpolation process will be described by focusing on a specific pixel, but the same process is performed on all AF pixels arranged at the position of the green pixel. The image data I (A2) at the position of the AF pixel (data A2) arranged at the position of the green pixel is interpolated by the following equation.
I (A2) = (G1 + G3 + G4 + G5 + G6 + G7 + G8 + G10) / 8 (11)
By interpolating the image data with the equation (11), even when the image pattern 401 such as an edge or a thin line is superimposed on the AF pixel array, the image data G3 and G8 become data reflecting the image pattern 400. A good image interpolation result can be obtained.

Alternatively, the image data may be interpolated using the AF pixel data A2 by the following equation.
I (A2) = {k1 · (G1 + G3 + G4 + G5 + G6 + G7 + G8 + G10) / 8} + k2 · ks · A2 (12)

  In the AF pixel arrangement of the image sensor 212B shown in FIGS. 23 and 24, the AF pixel interval can be made narrower than the AF pixel arrangement shown in FIG. 5, so that improvement in image quality and focus detection accuracy are expected. it can.

  FIG. 26 is a front view illustrating a detailed configuration of an imaging element 212C according to another modification. In the image sensor 212 shown in FIG. 5, the image sensor 212A of the modification shown in FIG. 21, and the image sensor 212B of the other modification shown in FIG. 23, the AF pixel 311 has a pair of photoelectric conversion units in one pixel. In the imaging device 212C shown in FIG. 26, the AF pixels 313 and 314 include one photoelectric conversion unit in one pixel. In FIG. 26, an AF pixel 313 and an AF pixel 314 are paired and correspond to the AF pixel 311a of the imaging elements 211, 211A, and 211B described above.

  FIG. 27 is a diagram showing a configuration of the AF pixel shown in FIG. As shown in (a), the AF pixel 313 includes the microlens 10 and the photoelectric conversion unit 16. Further, as shown in (b), the focus detection pixel 314 includes the microlens 10 and the photoelectric conversion unit 17. These photoelectric conversion units 16 and 17 are projected onto the exit pupil of the interchangeable lens by the microlens 10 to form distance measuring pupils 92 and 93 shown in FIG. Therefore, a pair of image data used for focus detection can be obtained by the AF pixels 313 and 314. By providing one photoelectric conversion unit in the AF pixel, it is possible to prevent complication of the readout circuit configuration of the image sensor.

  FIG. 28 is a front view showing a detailed configuration of an imaging element 212D of another modification. This image sensor 212D is obtained by replacing the AF pixel 311a of the image sensor 212A shown in FIG. 21 with an AF pixel 313 and an AF pixel 314.

  FIG. 29 is a diagram illustrating a detailed configuration of an image sensor 212E according to another modification. In the imaging elements 212 to 212D described above, an example in which the array of imaging pixels is a Bayer array is shown. However, in this imaging element 212E, green pixels G, blue pixels B, and red pixels R are arranged in the order of GBGR in the horizontal direction. It has an array. The AF pixel A is disposed at the position of the green pixel G, and three pixels of the green pixel G, the blue pixel B, and the red pixel R are disposed between the AF pixel and the AF pixel. The present invention can also be applied to such a pixel arrangement.

  In the imaging elements 212 to 212D described above, an example in which the color filters of the imaging pixels are arranged in a Bayer array is shown, but the configuration and arrangement of the color filters of the imaging pixels are not limited to this. For example, a complementary color filter (green: G, yellow: Ye, magenta: Mg, cyan: Cy) as shown in FIG. 4 may be employed.

  In the above-described imaging elements 212 to 212E, an example in which a color filter is not provided for the AF pixel has been shown. However, in the case where one filter (for example, a green filter) is provided among the color filters of the same color as the imaging pixel. However, the present invention can be applied. In this way, the interpolation accuracy of the image data of the AF pixel arranged at the position of the green pixel is improved by the interpolation calculation of equation (11).

  The imaging elements 212 to 212E described above can be formed as a CCD image sensor or a CMOS image sensor.

  In the above-described embodiment, an example in which the microlens of the imaging pixel and the microlens of the focus detection pixel have the same optical characteristics has been described. However, different optical characteristics may be used. Thereby, the exit pupil distance of the photographing optical system can be set separately to the optimum distance for imaging and the optimum distance for focus detection.

<< Imaging device >>
In the above-described embodiment, the digital still camera in which the interchangeable lens is mounted on the camera body has been described as an example. However, the imaging apparatus is not limited to the digital still camera including the interchangeable lens and the camera body. It can also be applied to an integrated digital still camera or video camera. The present invention can also be applied to a small camera module, a surveillance camera, or the like built in a mobile phone.

  As described above, according to the embodiment, when interpolating the image data at the position of the focus detection pixel, the image data of the imaging pixel in the arrangement direction of the focus detection pixel can be used. Accuracy can be improved and deterioration of image quality can be prevented. Further, even when a fine line pattern or an edge pattern is superimposed on the array of focus detection pixels, these patterns are not buried in the background.

The figure which shows the structure of one embodiment Diagram showing focus detection position on the shooting screen The figure which shows the Bayer arrangement | sequence of the pixel for imaging of an image pick-up element The figure which shows the complementary color arrangement | sequence of the pixel for imaging of an image pick-up element Front view showing detailed configuration of image sensor Front view of imaging pixels Front view of focus detection pixels The figure which shows the spectral characteristic of the green pixel for imaging of an image pick-up element, the red pixel for imaging, and the blue pixel for imaging The figure which shows the spectral characteristic of the pixel for AF Diagram showing spectral characteristics of green pixels Cross-sectional view of imaging pixels Cross section of AF pixel The figure explaining the focus detection principle by the pupil division type phase difference detection method Front view showing projection relationship on exit pupil plane The figure which shows the output ratio coefficient (pixel data value for imaging / pixel data value for focus detection) with respect to the aperture F value The flowchart which shows operation | movement of the digital still camera (imaging device) of one embodiment The figure explaining image shift detection calculation processing (correlation algorithm) The figure which shows the typical structure of the image pick-up element of one embodiment The figure explaining the image data interpolation process with respect to the image pick-up element of one embodiment The figure explaining the image data interpolation process with respect to the image pick-up element of one embodiment Front view showing a detailed configuration of an image sensor of a modification Schematic configuration diagram of image sensor Front view showing a detailed configuration of an image sensor of another modification The figure which shows the typical structure of an image sensor The figure explaining the image data interpolation process with respect to an image sensor Front view showing a detailed configuration of an image sensor of another modification The figure which shows the structure of the pixel for AF shown in FIG. Front view showing a detailed configuration of an image sensor of another modification The figure which shows the detailed structure of the image pick-up element of another modification. The figure which shows the interpolation calculation of the conventional image pick-up element.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Microlens 11, 12, 13, 16, 17 Photoelectric conversion part 202 Interchangeable lens 212, 212A-212E Image sensor 214 Body drive control apparatus 310a, 310b, 310c Imaging pixel 311a, 311b, 313, 314 Focus detection pixel

Claims (17)

Three types including a plurality of first imaging pixels having a first spectral sensitivity characteristic, a plurality of second imaging pixels having a second spectral sensitivity characteristic, and a plurality of third imaging pixels having a third spectral sensitivity characteristic The imaging pixels are arranged according to a two-dimensional regular arrangement,
In a part of the pixel row in which the first imaging pixels and the second imaging pixels are alternately arranged, at least the first imaging pixels are replaced and focus detection pixels are arranged,
Among the plurality of focus detection pixels, the first imaging pixel and the second imaging pixel are arranged between the focus detection pixel and the focus detection pixel that are adjacent to each other. An image sensor for capturing an image to be formed;
The first imaging pixel and the second imaging pixel that are arranged between the focus detection pixel and the focus detection pixel adjacent to the focus detection pixel are image data of the pixel position of the focus detection pixel. Based on pixel data of the plurality of first imaging pixels adjacent to each other in the diagonal direction, the horizontal direction, and the vertical direction with respect to the focus detection pixel including the first imaging pixel among the pixels. Interpolating means for interpolating,
Each of the three types of imaging pixels includes a red filter, a green filter, and a blue filter,
The image pickup apparatus, wherein the three kinds of image pickup pixels are arranged on the image pickup device according to a Bayer array. The imaging device according to claim 1,
The focus detection pixel is disposed in the imaging element in place of the first imaging pixel and the second imaging pixel in a part of the pixel row,
The interpolation means arranges the image data of the focus detection pixels arranged by replacing the first imaging pixels between the focus detection pixels and the focus detection pixels adjacent to the focus detection pixels. And interpolating based on the pixel data of the first imaging pixels, and replacing the second imaging pixels with the image data of the focus detection pixels arranged to replace the focus detection pixels and the focus detection pixels. An image pickup apparatus that performs interpolation based on pixel data of the second image pickup pixel disposed between focus detection pixels adjacent to the pixel. In the imaging device according to claim 1 or 2,
The first imaging pixel and the second imaging pixel are squarely arranged on the imaging element, and the extending direction of the pixel row is a direction of each side or diagonal line of the square array. . In the imaging device according to claim 1 or 2,
The focus detection pixel receives a pair of light fluxes passing through different pupil portions along the extending direction of the pixel row in the exit pupil of the optical system. In the imaging device according to claim 1 or 2,
The focus detection pixel includes a microlens and a photoelectric conversion unit,
The imaging apparatus, wherein the photoelectric conversion unit and the exit pupil of the optical system are conjugate with respect to the microlens. The imaging apparatus according to claim 5,
The focus detection pixel includes a pair of photoelectric conversion units,
An image pickup apparatus, wherein a pair of light fluxes passing through different pupil portions of the optical system are received by each of the pair of photoelectric conversion units. The imaging apparatus according to claim 5,
An imaging apparatus, wherein one of a pair of light beams passing through different pupil portions of the optical system is received by the photoelectric conversion unit provided in the focus detection pixel. The imaging apparatus according to claim 7,
An image pickup apparatus comprising a plurality of pairs of focus detection pixels that respectively receive a pair of light beams passing through different pupil portions of the optical system. In the imaging device according to any one of claims 1 to 8,
The focus detection pixel has a spectral sensitivity characteristic different from that of the first imaging pixel and the second imaging pixel. In the imaging device according to any one of claims 1 to 8,
The focus detection pixels arranged by replacing the first imaging pixels constituting the regular array have the same spectral sensitivity characteristics as the replaced first imaging pixels. . In the imaging device according to any one of claims 1 to 10,
The first imaging pixel and the second imaging pixel each include a microlens and a photoelectric conversion unit,
The imaging apparatus, wherein the photoelectric conversion unit and the exit pupil of the optical system are conjugate with respect to the microlens. The imaging device according to claim 11,
An imaging apparatus, wherein optical characteristics of a microlens provided in each of the first imaging pixel and the second imaging pixel are different from optical characteristics of a microlens provided in the focus detection pixel. In the imaging device according to any one of claims 1 to 10,
An image pickup apparatus, further comprising: a focus detection unit that detects a focus adjustment state of the optical system based on output data of the focus detection pixels of the image pickup element. The imaging device according to claim 1,
The pixel position closest in the direction of each side and diagonal of the square array to the pixel position of the focus detection pixel arranged by replacing the first imaging pixel included in the square array constituting the Bayer array The imaging apparatus is characterized in that no focus detection pixels adjacent to the focus detection pixels are arranged. 2 rows and 2 columns comprising two first imaging pixels having green spectral characteristics, one second imaging pixel having blue spectral characteristics, and one third imaging pixel having red spectral characteristics Are arranged two-dimensionally as a Bayer array,
A plurality of the first imaging pixels are replaced by a plurality of the imaging pixels arranged on a straight line in any one of the horizontal direction, the vertical direction, and the diagonal direction of the arrangement of the imaging pixels. Focus detection pixels are arranged,
Among the plurality of focus detection pixels, at least one first imaging pixel is arranged between the focus detection pixels and the focus detection pixels adjacent to each other , and around each focus detection pixel. The four first imaging pixels are arranged adjacent to the focus detection pixels in the diagonal direction across the focus detection pixels, and in the horizontal direction across the focus detection pixels. Two first imaging pixels are arranged one pixel from each focus detection pixel, and two one pixel from each focus detection pixel in the vertical direction across the focus detection pixels. An image sensor on which the first imaging pixels are disposed;
Pixel data of the four first imaging pixels arranged adjacent to each focus detection pixel in a diagonal direction across each focus detection pixel, and in a horizontal direction across each focus detection pixel Pixel data of two first imaging pixels arranged one pixel from each focus detection pixel and one pixel from each focus detection pixel in the vertical direction across each focus detection pixel. And an interpolation calculation means for performing an interpolation calculation on the image data at the position of each focus detection pixel based on the pixel data of the two first imaging pixels arranged in this manner. 2 rows and 2 columns comprising two first imaging pixels having green spectral characteristics, one second imaging pixel having blue spectral characteristics, and one third imaging pixel having red spectral characteristics Are arranged two-dimensionally as a Bayer array,
A plurality of focus detection pixels are arranged in place of each of the plurality of second imaging pixels, and
The focus detection pixels of the plurality of focus detection pixels are positioned closest to each focus detection pixel in the horizontal direction, and are arranged one pixel from each focus detection pixel in the horizontal direction across the focus detection pixels. The two second imaging pixels and the focus detection pixels that are positioned closest to each other in the vertical direction, and each of the focus detection pixels is 1 from the focus detection pixels in the vertical direction. The two second imaging pixels arranged separately from each other and the closest position in the diagonal direction with respect to each focus detection pixel, and in the diagonal direction across the focus detection pixels For each of the four second imaging pixels arranged one pixel from each focus detection pixel, an imaging device that is not replaced with the plurality of focus detection pixels,
The two second pixels are located closest to the focus detection pixels in the horizontal direction, and are arranged one pixel from the focus detection pixels in the horizontal direction with the focus detection pixels interposed therebetween. One pixel from each focus detection pixel is positioned in the vertical direction with respect to the pixel data of the imaging pixel and the focus detection pixel, and is positioned closest to the focus detection pixel in the vertical direction. The pixel data of the two second imaging pixels that are arranged, and each of the focus detection pixels are positioned closest to each other in the diagonal direction. Interpolation calculation means for interpolating the image data at the position of each focus detection pixel based on the pixel data of the four second imaging pixels arranged one by one from the focus detection pixel. Special Imaging device according to. A plurality of first imaging pixels having green spectral characteristics, a plurality of second imaging pixels having blue spectral characteristics, and a plurality of third imaging pixels having red spectral characteristics are two-dimensional rules. Arranged according to a typical sequence,
In place of the plurality of first imaging pixels, a plurality of pairs of first focus detection pixels and second focus detection pixels that are paired with each other are arranged, and
An image sensor that captures an image formed by an optical system,
The number of pixels of the plurality of second imaging pixels is equal to the number of pixels of the plurality of third imaging pixels,
The number of pixels of the plurality of first imaging pixels is twice the number of pixels of the plurality of second imaging pixels,
The first focus detection pixel and the second focus detection pixel have the same green spectral characteristics as the plurality of first imaging pixels,
Each of the first focus detection pixel and the second focus detection pixel includes one microlens and one photoelectric conversion unit,
The photoelectric conversion unit and the exit pupil of the optical system are conjugate with respect to the microlens,
One of a pair of light beams passing through different pupil portions of the optical system is received by the photoelectric conversion unit included in the first focus detection pixel, and the pair of light beams is included in the second focus detection pixel. The other of the luminous flux is received,
Between the first focus detection pixels arranged adjacent to each other in the arrangement direction of the pair of light beams and between the second focus detection pixels arranged adjacent to each other in the arrangement direction of the pair of light beams. An image sensor in which the first imaging pixels are respectively arranged between
Pixel data of the first imaging pixels arranged between the first focus detection pixels arranged adjacent to each other in the arrangement direction of the pair of light fluxes, and pixel data of the first focus detection pixels The second focus detection pixels arranged adjacent to each other in the arrangement direction of the pair of light fluxes while performing interpolation calculation on the image data at the position of the first focus detection pixels based on at least two pixel data including An image of the position of the second focus detection pixel based on at least two pieces of pixel data including pixel data of the first imaging pixel and pixel data of the second focus detection pixel arranged between each other An imaging apparatus comprising: an interpolation calculation means for performing interpolation calculation of data.

Images