JP5476810B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus.

  Imaging pixels with red, green, or blue color filters are two-dimensionally arranged, and image sensors and pupil division types are used in the two-dimensional array with focus detection pixels that do not have color filters An imaging apparatus that performs focus detection by a phase difference detection method is known (see, for example, Patent Document 1).

JP 2000-156823 A

  However, in such an imaging apparatus, when an image with high color purity is formed at the focus detection pixel position, the focus detection result may be inappropriate. For example, when a high-purity red image close to infrared is formed at the focus detection position (focus detection area), the focus detection result is a focus position corresponding to the wavelength of the high-purity red near infrared. And the optical system is focused on the focus position. However, since human visibility has a peak at the green wavelength, the focus position is also determined for the green image structure, and the chromatic aberration of the optical system is large. If the focus positions corresponding to the wavelength of are greatly separated, it is determined that the image is out of focus.

An image pickup apparatus according to a first aspect of the present invention receives a pick-up light beam that has passed through a photographic optical system and outputs an image pickup pixel signal, and passes through the photographic optical system and a plurality of types of image pickup pixels having different spectral sensitivity characteristics. An image sensor in which a plurality of focus detection pixels having spectral sensitivity characteristics different from those of the imaging pixels are arranged in accordance with a two-dimensional array, and the focus detection pixel signal Based on the focus detection means for detecting the focus adjustment state of the imaging optical system based on the imaging pixel signals output from the plurality of types of imaging pixels arranged in the vicinity of the plurality of focus detection pixels Interpolation means for calculating the imaging pixel signal at the position of the focus detection pixel by interpolation calculation, and the positions at the positions of the plurality of focus detection pixels calculated by the interpolation means. Based on an image pixel signal and chromatic aberration information of the photographing optical system, a correction unit that corrects a physical quantity representing the focus adjustment state detected by the focus detection unit, and the plurality of focal points calculated by the interpolation unit Prohibition of performing prohibition control for prohibiting correction by the correction means of the physical quantity representing the focus adjustment state detected by the focus detection means when the hue indicated by the imaging pixel signal at the position of the detection pixel is biased And means.
An imaging device according to a second aspect of the invention receives a plurality of types of imaging pixels having different spectral sensitivity characteristics, which receives an imaging light beam that has passed through the imaging optical system and outputs an imaging pixel signal, and passes through the imaging optical system. An image sensor in which a plurality of focus detection pixels having spectral sensitivity characteristics different from those of the imaging pixels are arranged in accordance with a two-dimensional array, and the focus detection pixel signal Based on the focus detection means for detecting the focus adjustment state of the imaging optical system based on the imaging pixel signals output from the plurality of types of imaging pixels arranged in the vicinity of the plurality of focus detection pixels Interpolation means for calculating the imaging pixel signal at the position of the focus detection pixel by interpolation calculation, and the positions at the positions of the plurality of focus detection pixels calculated by the interpolation means. Based on an image pixel signal and chromatic aberration information of the photographing optical system, a correction unit that corrects a physical quantity representing the focus adjustment state detected by the focus detection unit, and the plurality of focal points calculated by the interpolation unit The hue indicated by the imaging pixel signal at the position of the detection pixel, and the area disposed in a wide area larger than the area surrounding the area where the plurality of types of imaging pixels used for the interpolation processing by the interpolation unit are disposed. When the difference between the hues indicated by the imaging pixel signals of a plurality of types of imaging pixels is small, prohibition control is performed to prohibit the correction of the physical quantity representing the focus adjustment state detected by the focus detection unit by the correction unit. And a prohibiting means.
An image pickup apparatus according to an eighth aspect of the invention receives a plurality of types of image pickup pixels having different spectral sensitivity characteristics that receive an image pickup light beam that has passed through the image pickup optical system and outputs an image pickup pixel signal, and passes through the image pickup optical system. An image sensor in which a plurality of focus detection pixels having spectral sensitivity characteristics different from those of the imaging pixels are arranged in accordance with a two-dimensional array, and the focus detection pixel signal Based on the focus detection means for detecting a focus adjustment state of the photographing optical system and calculating a defocus amount, and the imaging pixels output from the plurality of types of imaging pixels arranged in the vicinity of the plurality of focus detection pixels Based on the signal, an interpolation means for calculating the imaging pixel signals at the positions of the plurality of focus detection pixels by interpolation calculation, and the plurality of focus detection calculated by the interpolation means. With the image-capturing pixel signal at the position of the pixel, on the basis of the chromatic information of the photographing optical system, and a correction means for correcting the defocus amount, the image pickup pixel, first, second, and third The first, second, and third imaging pixels each having a spectral sensitivity of, and the focus detection pixel has a spectral sensitivity that includes the first, second, and third spectral sensitivities, and the imaging The chromatic aberration information of the optical system is based on the first chromatic aberration information related to the first wavelength corresponding to the first spectral sensitivity, the second chromatic aberration related to the second wavelength corresponding to the second spectral sensitivity, and Each of the first, second, and third interpolation information includes third chromatic aberration information relating to a third wavelength corresponding to the third spectral sensitivity, and the interpolation means is disposed in the vicinity of the plurality of focus detection pixels. The imaging pixel signal output by the imaging pixel Based on the first, second, and third imaging pixel signals at the positions of the plurality of focus detection pixels, the correction means calculates the first, second, and the correction means calculated by the interpolation means. A first correction value obtained by multiplying the second chromatic aberration information by a value obtained by dividing the second imaging pixel signal calculated by the interpolation unit by the total value of the second and third imaging pixel signals; A second correction value obtained by multiplying the third chromatic aberration information by a value obtained by dividing the third imaging pixel signal calculated by the interpolation means by a total value is subtracted from the defocus amount. And

  According to the present invention, accurate focus detection can be performed even when an optical system with large chromatic aberration is used.

It is a cross-sectional view which shows the structure of the digital still camera of 1st Embodiment. It is a figure which shows the focus detection position on an imaging | photography screen. It is a front view which shows the detailed structure of an image pick-up element. It is a front view which shows the detailed structure of an image pick-up element. It is a figure which shows the structure of an imaging pixel. It is a figure which shows the structure of a focus detection pixel. It is a figure which shows the spectral characteristic of an imaging pixel. It is a figure which shows the spectral characteristic of a focus detection pixel. It is sectional drawing of an imaging pixel. It is sectional drawing of a focus detection pixel. It is a figure which shows the structure of the focus detection optical system of the pupil division type phase difference detection system using a micro lens. It is a flowchart which shows operation | movement of a digital still camera (imaging device). It is a figure for demonstrating the determination method of the reliability of a focus detection result. It is a flowchart which shows the detailed operation | movement of a pixel interpolation process. It is a figure which shows the data of each pixel by Gij, Bij, Rij when a focus detection pixel is arrange | positioned at one line among the imaging pixels arranged in a two-dimensional Bayer array. It is a figure which shows the direction of the continuity of an image. FIG. 4 is a diagram illustrating pixel positions in a direction (column direction) orthogonal to the focus detection pixel array on a horizontal axis, and pixel data on the pixel positions on a vertical axis. It is a front view of the image sensor of a modification. It is a front view of the focus detection pixel used for the image sensor of the modification shown in FIG. It is a figure which shows the relationship between a chromatic aberration and a wavelength. It is a figure which shows a visibility curve. It is a figure which shows the relationship between the area | region of the focus detection pixel row | line | column vicinity containing a focus detection pixel row | line | column, and the still larger area | region of the outer periphery of this area | region.

--- First embodiment ---
A lens interchangeable digital still camera will be described as an example of the imaging apparatus according to the first embodiment. FIG. 1 is a cross-sectional view showing the configuration of the digital still camera according to the first embodiment. The digital still camera 201 according to the first embodiment includes an interchangeable lens 202 and a camera body 203, and the interchangeable lens 202 is attached to the camera body 203 via a mount unit 204.

  The interchangeable lens 202 includes a lens 209, a zooming lens 208, a focusing lens 210, an aperture 211, a lens drive control device 206, and the like. The lens drive control device 206 includes a microcomputer (not shown), a memory, a drive control circuit, and the like. The lens drive control device 206 controls the driving of the focusing lens 210 and the aperture 211, detects the state of the zooming lens 208, the focusing lens 210, and the aperture 211, and the like. I do. In addition, lens information is transmitted and camera information is received by communication with a body drive control device 214 described later.

  The camera body 203 includes an imaging element 212, a body drive control device 214, a liquid crystal display element drive circuit 215, a liquid crystal display element 216, an eyepiece lens 217, a memory card 219, and the like. In the imaging element 212, imaging pixels are two-dimensionally arranged, and focus detection pixels are incorporated in portions corresponding to focus detection positions.

  The body drive control device 214 includes a microcomputer, a memory, a drive control circuit, and the like, and performs drive control of the image sensor 212 and reading of image signals and focus detection signals, image signal processing and recording, and focus detection calculation based on the focus detection signals. The focus adjustment of the interchangeable lens 202 and the operation control of the camera are performed. The body drive control device 214 communicates with the lens drive control device 206 via the electrical contact 213 to receive lens information and send camera information (defocus amount, aperture value, etc.).

  The liquid crystal display element 216 functions as a liquid crystal viewfinder (EVF: electrical viewfinder). The liquid crystal display element driving circuit 215 displays a through image by the imaging element 212 on the liquid crystal display element 216, and the photographer can observe the through image through the eyepiece lens 217. The memory card 219 is an image storage that stores an image captured by the image sensor 212.

  A subject image is formed on the light receiving surface of the image sensor 212 by the light flux that has passed through the interchangeable lens 202. This subject image is photoelectrically converted by the image sensor 212, and an image signal and a focus detection signal are sent to the body drive control device 214.

  The body drive control device 214 calculates the defocus amount based on the focus detection signal from the focus detection pixel of the image sensor 212 and sends the defocus amount to the lens drive control device 206. Further, the body drive control device 214 inputs an image signal and a focus detection signal from the image sensor 212, and picks up pixel data for imaging at the position of the focus detection pixel, that is, an imaging pixel that should be originally at the position of the focus detection pixel. When arranged, pixel data output from the imaging pixel is interpolated using pixel data of imaging pixels around the focus detection pixel. Then, an image signal composed of pixel data of the image pickup pixel and pixel data interpolated at the focus detection pixel position is stored in the memory card 219. Further, the through image signal input from the image sensor 212 is sent to the liquid crystal display element driving circuit 215, and the through image is displayed on the liquid crystal display element 216. Furthermore, the body drive control device 214 sends aperture control information to the lens drive control device 206 to control the aperture of the aperture 211.

  The lens drive control device 206 changes the lens information according to the focusing state, zooming state, aperture setting state, aperture opening F value, and the like. Specifically, the positions of the zooming lens 208 and the focusing lens 210 and the aperture value of the aperture 211 are detected, and lens information is calculated according to these lens positions and aperture values, or a lookup prepared in advance. Lens information corresponding to the lens position and aperture value is selected from the table. Also, chromatic aberration information that changes depending on the focusing state, zooming state, and full aperture F value is transmitted to the body drive control unit 214. Chromatic aberration information is determined for each of a plurality of focus detection areas described later.

  The lens drive control device 206 calculates a lens drive amount based on the received defocus amount, and drives the focusing lens 210 to a focal point according to the lens drive amount. Further, the lens drive control device 206 drives the diaphragm 211 in accordance with the received diaphragm value.

  An interchangeable lens 202 having various imaging optical systems can be attached to the camera body 203 via a mount unit 204, and the camera body 203 is based on an output of a focus detection pixel incorporated in the image sensor 212. Detect the focus adjustment state.

  FIG. 2 shows a focus detection area on which an image is sampled on the screen when focus detection is performed using a focus detection position on the photographing screen, that is, a focus detection pixel array to be described later. Focus detection areas 101 to 105 are arranged at five locations on the center, left and right, and top and bottom of the rectangular shooting screen 100. Focus detection pixels are linearly arranged in the longitudinal direction of the focus detection areas 101 to 105 indicated by rectangles.

  FIG. 3 is a front view showing a detailed configuration of the image sensor 212, and shows an enlarged view of the vicinity of the focus detection area 101 (the same applies to the focus detection areas 104 and 105) on the image sensor 212. The imaging element 212 includes an imaging pixel 310 and focus detection pixels 313 and 314. The imaging pixels 310 are arranged in a two-dimensional square lattice in the horizontal and vertical directions, and the focus detection pixels 313 and 314 are arranged in the horizontal direction.

  As shown in FIG. 5, the imaging pixel 310 includes the microlens 10, the photoelectric conversion unit 11, and a color filter (not shown). There are three types of color filters, red (R), green (G), and blue (B), and the respective spectral sensitivities have the characteristics shown in FIG. An imaging pixel 310 having each color filter is arranged in a Bayer array. In FIG. 7, λR, λG, and λB indicate the peak wavelengths (center wavelengths) of the red, green, and blue spectral sensitivity curves.

  As illustrated in FIG. 6A, the focus detection pixel 313 includes the microlens 10 and the photoelectric conversion unit 16. The shape of the photoelectric conversion unit 16 is a left semicircle in contact with the vertical bisector of the microlens 10. Further, as shown in FIG. 6B, the focus detection pixel 314 includes the microlens 10 and the photoelectric conversion unit 17. The shape of the photoelectric conversion unit 17 is a right semicircle in contact with the vertical bisector of the microlens 10. The front views of the photoelectric conversion units 16 and 17 are arranged side by side in the horizontal direction when they are displayed superimposed on the microlens 10, and have a symmetrical shape with respect to the vertical bisector of the microlens 10. The focus detection pixels 313 and the focus detection pixels 314 are alternately arranged in the horizontal direction, that is, the arrangement direction of the photoelectric conversion units 16 and 17.

  The focus detection pixels 313 and 314 are not provided with a color filter in order to increase the amount of light, and the spectral characteristics of the focus detection pixels 313 and 314 include the spectral sensitivity of a photodiode that performs photoelectric conversion and the spectral sensitivity of an infrared cut filter (not shown). Spectral characteristics (see FIG. 8). In other words, the spectral characteristics are obtained by adding the spectral characteristics of the green pixel, red pixel, and blue pixel shown in FIG. 7, and the light wavelength region of the sensitivity includes the light wavelength regions of the sensitivity of the green pixel, red pixel, and blue pixel. ing. The focus detection pixels 313 and 314 are arranged in a row where B and G of the imaging pixel 310 are to be arranged.

  FIG. 4 is a front view showing a detailed configuration of the image sensor 212, and shows an enlarged view of the vicinity of the focus detection area 102 (the same applies to the focus detection area 103) on the image sensor 212. The imaging element 212 includes an imaging pixel 310 and focus detection pixels 315 and 316. The focus detection pixels 315 and 316 have a structure in which the focus detection pixels 313 and 314 are rotated by 90 degrees, and are arranged in the vertical direction on the imaging surface. The focus detection pixels 315 and 316 are arranged in a column in which B and G of the imaging pixel 310 are to be arranged.

  The focus detection pixels 313, 314, 315, and 316 are arranged in the row or column where the B and G of the imaging pixel 310 are to be arranged when an interpolation error occurs in the pixel interpolation processing described later. This is because the blue pixel interpolation error is less conspicuous than the red pixel interpolation error.

  FIG. 9 is a cross-sectional view of the imaging pixel 310. In the imaging pixel 310, the microlens 10 is disposed in front of the photoelectric conversion unit 11 for imaging, and the photoelectric conversion unit 11 is projected forward by the microlens 10. The photoelectric conversion unit 11 is formed on the semiconductor circuit substrate 29. A color filter (not shown) is disposed between the microlens 10 and the photoelectric conversion unit 11.

  FIG. 10A is a cross-sectional view of the focus detection pixel 313. In the focus detection pixel 313, the microlens 10 is disposed in front of the photoelectric conversion unit 16, and the photoelectric conversion unit 16 is projected forward by the microlens 10. The photoelectric conversion unit 16 is formed on the semiconductor circuit substrate 29, and the microlens 10 is integrally and fixedly formed thereon by the manufacturing process of the semiconductor image sensor. The photoelectric conversion unit 16 is disposed on one side of the optical axis of the microlens 10.

  FIG. 10B is a cross-sectional view of the focus detection pixel 314. In the focus detection pixel 314, the microlens 10 is disposed in front of the photoelectric conversion unit 17, and the photoelectric conversion unit 17 is projected forward by the microlens 10. The photoelectric conversion unit 17 is formed on the semiconductor circuit substrate 29, and the microlens 10 is integrally and fixedly formed thereon by the manufacturing process of the semiconductor image sensor. The photoelectric conversion unit 17 is disposed on one side of the optical axis of the microlens 10 and on the opposite side of the photoelectric conversion unit 16.

  FIG. 11 shows a configuration of a focus detection optical system of the pupil division type phase difference detection method using the microlens 10. In FIG. 11, the exit pupil 90 is disposed on the planned imaging plane of the interchangeable lens and is set at a distance d in front of the microlens 10. The distance d is a distance determined according to the curvature and refractive index of the microlens 10, the distance between the microlens 10 and the photoelectric conversion units 13 and 14, and is referred to as a distance measuring pupil distance in this specification. FIG. 11 shows an optical axis 91 of the interchangeable lens, microlenses 10a to 10d, photoelectric conversion units (pixels) 313a, 313b, 314a, 314b, and light beams 73, 74, 83, 84.

  The region 93 is a region where the photoelectric conversion units 13a and 13b are projected by the micro lenses 10a and 10c, and is referred to as a distance measuring pupil in this specification. Similarly, the region 94 is also a distance measuring pupil, and the photoelectric conversion units 14a and 14b are projected by the micro lenses 10b and 10d. In FIG. 11, the range-finding pupil is shown as an elliptical area for easy understanding of the description. However, the shape of the photoelectric conversion unit is actually an enlarged projection.

  Although FIG. 11 schematically illustrates four adjacent pixels (pixels 313a, 313b, 314a, and 314b), in other pixels, the photoelectric conversion units arrive at the respective microlenses from the corresponding distance measurement pupils. Receives light flux. The arrangement direction of the focus detection pixels is made to coincide with the arrangement direction of the pair of distance measuring pupils, that is, the arrangement direction of the pair of photoelectric conversion units.

  The microlenses 10a to 10d are arranged in the vicinity of the planned imaging plane of the interchangeable lens. The shapes of the photoelectric conversion units 13a, 13b, 14a, and 14b arranged behind the microlenses 10a to 10d are projected onto the exit pupil 90 that is separated from the microlenses 10a to 10d by the distance measurement pupil distance d, and the projected shape thereof. Forms distance measuring pupils 93 and 94. That is, on the exit pupil 90 at the projection distance d, the projection direction of the photoelectric conversion unit in each pixel is determined so that the projection shapes (ranging pupils 93 and 94) of the photoelectric conversion unit of each pixel match.

  The photoelectric conversion unit 13a passes through the distance measuring pupil 93 and outputs a signal corresponding to the intensity of the image formed on the microlens 10a by the light beam 73 directed to the microlens 10a. The photoelectric conversion unit 13b outputs a signal corresponding to the intensity of the image formed on the microlens 10c by the light beam 83 passing through the distance measuring pupil 93 and directed to the microlens 10c. The photoelectric conversion unit 14a outputs a signal corresponding to the intensity of the image formed on the microlens 10b by the light beam 74 passing through the distance measuring pupil 94 and directed to the microlens 10b. The photoelectric conversion unit 14b passes through the distance measuring pupil 94 and outputs a signal corresponding to the intensity of the image formed on the microlens 10d by the light beam 84 directed to the microlens 10d.

  A large number of the two types of focus detection pixels as described above are arranged in a straight line, and the output of the photoelectric conversion unit of each pixel is grouped into an output group corresponding to the distance measurement pupil 93 and the distance measurement pupil 94, thereby the distance measurement pupil 93. And information on the intensity distribution of the pair of images formed on the pixel array by the focus detection light fluxes passing through the distance measuring pupil 94, respectively. By applying an image shift detection calculation process (correlation calculation process, phase difference detection process), which will be described later, to this information, the image shift amount of a pair of images is detected by a so-called pupil division type phase difference detection method. Furthermore, by performing a conversion operation according to the center of gravity distance of the pair of distance measuring pupils to the image shift amount, it corresponds to the current imaging plane with respect to the planned imaging plane, that is, the position of the microlens array on the planned imaging plane. The deviation (defocus amount) of the imaging plane at the focus detection position is calculated.

  FIG. 12 is a flowchart illustrating the operation of the digital still camera (imaging device) according to the first embodiment. The body drive control device 214 starts its operation from step S110 when the camera is turned on in step S100. In step S110, the image pickup pixel data is read out from the image pickup device 212 and displayed on the electronic viewfinder. In subsequent step S120, a pair of image data corresponding to the pair of images and data of imaging pixels near the periphery of the focus detection pixel array are read from the focus detection pixel array. Note that the focus detection area is selected by the photographer by operating a focus detection area selection switch (not shown).

  In step S130, an image shift detection calculation process (correlation calculation process) is performed based on the read pair of image data of the imaging pixels, and the image shift amount is calculated and converted into a defocus amount. In step S131, virtual imaging pixel data at the position of the focus detection pixel is obtained by interpolation processing to be described later, based on the imaging pixel data in the vicinity of the focus detection pixel. In step S132, it is determined whether defocus amount correction is prohibited. The case in which an affirmative determination is made will be described later. If the determination is affirmative, the next step S133 is skipped, and if the determination is negative, the process proceeds to step S133. In step S133, chromatic aberration information corresponding to the focus detection area position is read from the interchangeable lens, and the defocus amount is corrected based on the chromatic aberration information and the data of the virtual imaging pixel obtained in step S131. Details of the defocus amount correction processing will be described later.

  In step S140, it is determined whether or not the focus is close to focus, and whether or not the calculated absolute value of the corrected defocus amount is within a predetermined value. If it is determined that the lens is not in focus, the process proceeds to step S150, where the corrected defocus amount is transmitted to the lens drive controller 206, and the focusing lens 210 of the interchangeable lens 202 shown in FIG. 1 is driven to the focus position. Then, it returns to step S110 and repeats the operation | movement mentioned above.

  Even when focus detection is impossible, the process branches to step S150, a scan drive command is transmitted to the lens drive control device 206, and the focusing lens 210 of the interchangeable lens 202 is scan-driven from the infinity position to the closest position, and the process proceeds to step S110. Return and repeat the above operation.

  On the other hand, if it is determined in step S140 that the focus is close to the in-focus state, the process proceeds to step S160, where it is determined whether or not a shutter release has been performed by operating a shutter button (not shown). Returning to S110, the above-described operation is repeated. If it is determined that the shutter release has been made, an aperture adjustment command is transmitted to the lens drive control unit 206, and the aperture value of the interchangeable lens 202 is set to the control F value (F value set by the user or automatically). When the aperture control is finished, the image sensor 212 is caused to perform an imaging operation, and image data is read from the imaging pixels of the image sensor 212 and all focus detection pixels.

  In step S180, pixel interpolation of pixel data at each pixel position in the focus detection pixel row is performed based on data of imaging pixels around the focus detection pixel. Details of this pixel interpolation will be described later. In subsequent step S190, the image data composed of the imaged pixel data and the interpolated data is stored in the memory card 219, and the process returns to step S110 to repeat the above-described operation.

Next, details of the image shift detection calculation process (correlation calculation process) in step S130 of FIG. 12 will be described. The pair of images detected by the focus detection pixels has a possibility that the distance measurement pupil is displaced by the aperture of the lens and the balance of the light quantity is lost. Perform the operation. The correlation calculation of the following equation (1) is performed on the pair of data strings A1 _{1 to} A1 _M and A2 _{1 to} A2 _M (M is the number of data) read from the focus detection pixel array, and the correlation amount C (k) Is calculated.
C (k) = Σ | A1 _n · A2 _{n + 1 + k−} A2 _{n + k} · A1 _{n + 1} | (1)

In equation (1), the Σ operation is accumulated for n, but the range of values of _n is limited to the range in which data of A1 _n , A1 _{n + 1} , A2 _{n + k} , A2 _{n + 1 + k} exists according to the image shift amount k. Is done. The image shift amount k is an integer and is a relative shift amount with the data interval of the data string as a unit. As shown in FIG. 13A, the calculation result of the expression (1) indicates that the correlation amount C (k) is obtained when the pair of data has a high correlation amount (k = k _j = 2 in FIG. 13A). Minimal (the smaller the value, the higher the degree of correlation). The correlation calculation formula is not limited to the formula (1).

Next, the shift amount x that gives the minimum value C (x) with respect to the continuous correlation amount is obtained using the three-point interpolation method according to the equations (2) to (5).
x = k _j + D / SLOP (2)
C (x) = C (k _j ) − | D | (3)
D = {C (k _j −1) −C (k _j +1)} / 2 (4)
SLOP = MAX {C (k _{j + 1} ) −C (k _j ), C (k _j−1 ) −C (k _j )} (5)

  Whether or not the shift amount x calculated by the equation (2) is reliable is determined as follows. As shown in FIG. 13B, when the degree of correlation between a pair of data is low, the value of the interpolated minimum value C (x) of the correlation amount increases. Therefore, when C (x) is equal to or greater than a predetermined threshold value, it is determined that the calculated shift amount has low reliability, and the calculated shift amount x is canceled. Alternatively, in order to normalize C (x) with the contrast of data, if the value obtained by dividing C (x) by SLOP that is proportional to the contrast is equal to or greater than a predetermined value, the reliability of the calculated shift amount Is determined to be low, and the calculated shift amount x is canceled. Alternatively, when SLOP that is a value proportional to the contrast is equal to or less than a predetermined value, it is determined that the subject has low contrast and the reliability of the calculated shift amount is low, and the calculated shift amount x is canceled.

As shown in FIG. 13 (c), low level of correlation between the pair of data when there is no drop in correlation quantity C (k) between the shift range k _min to k _max is for a local minimum C (x) In such a case, it is determined that the focus cannot be detected.

When it is determined that the calculated shift amount x is reliable, it is converted into the image shift amount shft by the equation (6).
shft = PY · x (6)

In Expression (6), PY is a detection pitch, that is, a pitch of focus detection pixels. Next, the image shift amount calculated by the equation (6) is multiplied by a predetermined conversion coefficient _Kd to be converted into a defocus amount def.
def = K _d · shft (7)

  FIG. 14 is a flowchart showing a detailed operation of the pixel interpolation process in step S180 of FIG. In this pixel interpolation processing, data of virtual imaging pixels at the position of the focus detection pixel is interpolated based on data of imaging pixels in the vicinity of the focus detection pixel. The body drive control device 214 starts pixel interpolation processing from step 300. FIG. 15 shows the data of each pixel as Gij, Bij, and Rij when the focus detection pixel is arranged in one row (rows indicated by diagonal lines) in the two-dimensional Bayer array imaging pixels. Show. In the pixel interpolation process, pixel data (G03, B13, G23, B33, G43, and B53) in the hatched rows are interpolated from the data of surrounding imaging pixels.

  In step S310, among the focus detection pixels arranged in the GB row (the row in which Gij and Bij are arranged) or the GB column (the column in which Gij and Bij are arranged), the green pixel is located at the position where the green pixel is to be arranged. One of the arranged focus detection pixels is sequentially designated. Here, the description will proceed assuming that the focus detection pixel at the position where G23 is to be output is designated. In step S320, assuming that there is no direction in which the image continuity is high at the focus detection pixel position, a parameter DCG representing the direction in which the image continuity is high is initialized to zero. Here, the direction in which the continuity of the image is high is, for example, the direction in which the edge pattern or the line pattern extends when the image is an edge pattern or a line pattern.

In step S330, it is determined whether or not there is image uniformity (uniformity) in the vicinity of the focus detection pixel position. The uniformity of the image is determined as follows, for example. Originally, using the image data of the green pixels around the focus detection pixel arranged at the position where the green pixel G23 should be, the average value Gav of the green pixels around the focus detection pixel and the image data in the vicinity of the focus detection pixel position Gsd representing the deviation is calculated.
Gav = (G10 + G30 + G21 + G12 + G32 + G14 + G34 + G25 + G16 + G36) / 10 (8)
Gsd = (| G10-Gav | + | G30-Gav | + | G21-Gav | + | G32-Gav | + | G14-Gav | + | G34-Gav | + | G25-Gav | + | G16-Gav | + | G36-Gav |)) / (10 · Gav) (9)
When Gsd representing the deviation of pixel data in the vicinity of the focus detection pixel position is within a predetermined threshold value, it is determined that there is image uniformity. If it is determined that there is uniformity, the process proceeds to step S420, and if it is determined that there is no uniformity, the process proceeds to step S340.

In step S340, whether or not there is image continuity in the 45 ° upward diagonal direction (DCG = 4 direction shown in FIG. 16) with respect to the direction of the focus detection pixel array (left and right direction in the example shown in FIG. 15). Determine whether. The presence / absence of continuity of the image in the direction of 45 degrees diagonally upward is determined as follows, for example.
Gav1 = (G12 + G21) / 2 (10)
Gsd1 = | G12−G21 | / Gav1 (11)
Gav2 = (G14 + G32) / 2 (12)
Gsd2 = | G14−G32 | / Gav2 (13)
Gav3 = (G25 + G34) / 2 (14)
Gsd3 = | G25−G34 | / Gav3 (15)
Gdf12 = | Gav1-Gav2 | (16)
Gdf13 = | Gav1-Gav3 | (17)
Gdf23 = | Gav2-Gav3 | (18)

  Gsd1, Gsd2, and Gsd3 representing deviations of pixel data in the 45 ° upward diagonal direction are all within a predetermined threshold value, and pixel data deviations Gdf12, Gdf13 in a direction orthogonal to the 45 ° upward diagonal direction If any of Gdf23 exceeds a predetermined threshold value, it is determined that there is continuity of the image in the 45 ° upward diagonal direction. If it is determined that there is image continuity in the 45 ° upward diagonal direction (DCG = 4 direction shown in FIG. 16), the process proceeds to step S350, and parameter DCG indicating the direction with high image continuity is set to 4. After that, the process proceeds to step S360. On the other hand, if it is determined that there is no continuity, the process proceeds directly to step S360.

In step S360, it is determined whether or not there is image continuity in a 45 ° upward diagonal direction (DCG = 3 direction shown in FIG. 16) with respect to the direction of the focus detection pixel array. The presence / absence of image continuity in the 45 ° upward slanting direction is determined, for example, as follows.
Gav1 = (G14 + G25) / 2 (19)
Gsd1 = | G14−G25 | / Gav1 (20)
Gav2 = (G12 + G34) / 2 (21)
Gsd2 = | G12−G34 | / Gav2 (22)
Gav3 = (G21 + G32) / 2 (23)
Gsd3 = | G21−G32 | / Gav3 (24)
Gdf12 = | Gav1-Gav2 | (25)
Gdf13 = | Gav1-Gav3 | (26)
Gdf23 = | Gav2-Gav3 | (27)

  Gsd1, Gsd2, and Gsd3 representing deviations of pixel data in the 45 ° upward diagonal direction are all within a predetermined threshold value, and deviations Gdf12, Gdf13 of pixel data in a direction orthogonal to the 45 ° upward diagonal direction If any of Gdf23 exceeds a predetermined threshold value, it is determined that there is continuity of the image in the 45 ° upward diagonal direction. If it is determined that there is image continuity in the 45 ° upward and diagonal direction, the process proceeds to step S370, and a parameter DCG indicating a direction with high image continuity is set to 3, and the process proceeds to step S380. On the other hand, if it is determined that there is no continuity, the process directly proceeds to step S380.

In step S380, it is determined whether there is image continuity in a direction orthogonal to the direction of the focus detection pixel array (the direction of DCG = 2 shown in FIG. 16). The presence / absence of image continuity in a direction orthogonal to the direction of the focus detection pixel array is determined, for example, as follows.
Gav1 = (G12 + G14) / 2 (28)
Gsd1 = | G12−G14 | / Gav1 (29)
Gav2 = (G21 + G25) / 2 (30)
Gsd2 = | G21−G25 | / Gav2 (31)
Gav3 = (G32 + G34) / 2 (32)
Gsd3 = | G32−G34 | / Gav3 (33)
Gdf12 = | Gav1-Gav2 | (34)
Gdf13 = | Gav1-Gav3 | (35)
Gdf23 = | Gav2-Gav3 | (36)

  Gsd1, Gsd2, and Gsd3 representing deviations of pixel data in a direction orthogonal to the direction of the focus detection pixel array are all within a predetermined threshold, and pixel data deviations Gdf12, Gdf13, When any one of Gdf23 exceeds a predetermined threshold value, it is determined that there is image continuity in a direction orthogonal to the direction of the focus detection pixel array. If it is determined that there is image continuity in a direction orthogonal to the direction of the focus detection pixel array, the process proceeds to step S390, and a parameter DCG indicating a direction with high image continuity is set to 2, and the process proceeds to step S400. . On the other hand, if it is determined that there is no continuity, the process directly proceeds to step S400.

In step S400, it is determined whether or not there is image continuity in the direction of the focus detection pixel array (the direction of DCG = 1 shown in FIG. 16). The presence or absence of image continuity in the direction of the focus detection pixel array is determined, for example, as follows.
Gav1 = (G01 + G21 + G41) / 3 (37)
Gsd1 = (| G01−Gav1 | + | G21−Gav1 | + | G41−Gav1 | / (3 · Gav1) (38)
Gav2 = (G12 + G32) / 2 (39)
Gsd2 = | G12−G32 | / Gav2 (40)
Gav3 = (G14 + G34) / 2 (41)
Gsd3 = | G14−G34 | / Gav3 (42)
Gav4 = (G05 + G25 + G45) / 2 (43)
Gsd4 = (| G05-Gav4 | + | G25-Gav4 | + | G45-Gav4 |) / Gav4 (44)
Gdf12 = | Gav1-Gav2 | (45)
Gdf23 = | Gav2-Gav3 | (46)
Gdf34 = | Gav3-Gav4 | (47)

  Gsd1, Gsd2, Gsd3, and Gsd4 representing deviations of pixel data in the direction of the focus detection pixel array are all within a predetermined threshold value, and pixel data deviations Gdf12 in a direction orthogonal to the direction of the focus detection pixel array If any of Gdf23 and Gdf34 exceeds a predetermined threshold value, it is determined that there is image continuity in the direction of the focus detection pixel array. If it is determined that there is image continuity in the direction of the focus detection pixel array, the process proceeds to step S410, 1 is set to a parameter DCG indicating a direction in which the image continuity is high, and the process proceeds to step S420. On the other hand, if it is determined that there is no continuity, the process directly proceeds to step S420.

  In step S420, whether or not there is an image directionality in the vicinity of the focus detection pixel position is confirmed by the parameter DCG. If there is no directionality of the image in the vicinity of the focus detection pixel position (DCG = 0), that is, if the image is uniform or random in the vicinity of the focus detection pixel position, the process proceeds to step S440. Otherwise, step S430 is performed. Proceed to In step S430, whether or not the image has continuity in the direction of the focus detection pixel array in the vicinity of the focus detection pixel position is confirmed by the parameter DCG. If there is continuity in the direction of the focus detection pixel array (DCG = 1), the process proceeds to step S460. Otherwise, the process proceeds to step S450.

If there is no directionality of the image in the vicinity of the focus detection pixel position (DCG = 0), that is, if the image is uniform or random in the vicinity of the focus detection pixel position, in step S440, the imaging in the vicinity of the focus detection pixel is performed. The pixel data of the pixels are averaged without depending on the directionality and subjected to pixel interpolation to obtain pixel data at the focus detection pixel position. For example, for the focus detection pixel G23 shown in FIGS. 15 and 16, interpolation pixel data is calculated as follows.
G23 = (G12 + G32 + G14 + G34) / 4 (48)

  If there is continuity in a direction other than the direction of the focus detection pixel array, that is, if there is continuity in any one of DCG = 2, 3, and 4 shown in FIG. The data of the image pickup pixels in are averaged according to the directionality of the image and subjected to pixel interpolation to obtain pixel data at the focus detection pixel position. For example, for the focus detection pixel G23 shown in FIGS. 15 and 16, interpolation pixel data is calculated as follows.

When there is continuity of the image in the 45 ° upward diagonal direction (DCG = 4), the average of the pixel data of green pixels in the vicinity of the focus detection pixel position in the direction of DCG = 4 shown in FIG. The pixel data is the focus detection pixel position.
G23 = (G14 + G32) / 2 (49)

When there is continuity of the image in the 45 ° upward diagonal direction (DCG = 3), the average of the pixel data of the green pixels in the vicinity of the focus detection pixel in the direction of DCG = 3 shown in FIG. , Pixel data at the focus detection pixel position.
G23 = (G12 + G34) / 2 (50)

Further, when there is image continuity in a direction orthogonal to the direction of the focus detection pixel array (DCG = 2), pixel data of green pixels in the vicinity of the focus detection pixel in the direction of DCG = 2 shown in FIG. Is taken as pixel data at the focus detection pixel position.
G23 = (G21 + G25) / 2 (51)

When the image has continuity in the direction of the focus detection pixel array in the vicinity of the focus detection pixel position (DCG = 1 shown in FIG. 16), the imaging pixels are in the direction of the focus detection pixel array with reference to the focus detection pixel position. Does not exist. Therefore, pixel data at the focus detection pixel position cannot be calculated by averaging. Therefore, in the direction orthogonal to the focus detection pixel array (the direction of DCG = 2 shown in FIG. 16), the pixel data at the focus detection pixel position is interpolated and obtained based on the imaging pixel data sandwiching the focus detection pixel array. For example, using Gav1, Gav2, Gav3, and Gav4 obtained by Expressions (37), (39), (41), and (43), interpolation pixel data Gip0 is interpolated as follows.
Gip0 = Gav2 + Gav3- (Gav1 + Gav4) / 2 (52)

  In FIG. 17, the horizontal axis indicates the pixel position in the direction (column direction) orthogonal to the focus detection pixel array, and the vertical axis indicates the pixel data at the pixel position. The above equation (52) means that pixel data obtained by extending a straight line connecting two pixel data Gav1 and Gav2 on one side of the focus detection pixel to the focus detection pixel position, and the other side of the focus detection pixel. That is, the average of the pixel data obtained by extending the straight line connecting the two pixel data Gav3 and Gav4 to the focus detection pixel position is the interpolation pixel data Gip0. By such interpolation processing, the influence of the high frequency component near the focus detection pixel can be reflected in the value of the interpolation pixel data. Therefore, compared with the case where the pixel data Gip1 obtained by simply averaging the two pixel data Gav2 and Gav3 adjacent to the focus detection pixel position is used as the interpolated pixel data, an excellent result is obtained for an image containing a high frequency component. can get.

  However, when the image includes only a very high spatial frequency component (spatial frequency component close to the Nyquist frequency corresponding to the pixel data pitch), the interpolation pixel data using the equation (52) has a value of the interpolated pixel data. A large error may occur with respect to the ideal pixel data value. In FIG. 17, the dotted line is the waveform of an image containing a large amount of Nyquist frequency components, and the pixel data Gip0 obtained by the interpolation process deviates from the actual value of the pixel data. In order to prevent such an inconvenience, the pixel data of the focus detection pixel in which the value that the actual pixel data should be is reflected to some extent is referred to.

Here, the value of the pixel data of the focus detection pixel is A23. The focus detection pixel has the spectral characteristics shown in FIG. 8. This spectral characteristic includes the spectral characteristics of the green pixel, the red pixel, and the blue pixel. Therefore, the pixel data A23 of the focus detection pixel is expressed as follows. Is done.
A23 = Ka · (Ga + Ba + Ra) (53)

  In Expression (53), Ka is a coefficient obtained by dividing the area of the photoelectric conversion unit of the focus detection pixel by the area of the photoelectric conversion unit of the imaging pixel, and Ga, Ba, and Ra are green at the position of the focus detection pixel, respectively. This is pixel data when there are pixels, blue pixels, and red pixels.

Here, the spectral distribution of incident light at the focus detection pixel position (output ratio of green pixels, red pixels, and blue pixels) and the spectral distribution of incident light near the focus detection pixel position (green pixels, red pixels, and blue pixels) Assuming that the output ratio is equal, it is expressed as:
Ba / Ga = (B11 + B31 + B15 + B35) / (2 · (G21 + G25)) (54)
Ra / Ga = 2. (R22 + R24) / (G12 + G32 + G14 + G34) (55)

Substituting Equations (54) and (55) into Equation (53) to organize Ga, and assuming this as conversion data Gip2 for the pixel data of the green pixel at the focus detection pixel position based on the pixel data of the focus detection pixel, become that way.
Gip2 = Ga = A23 / (Ka · (1+ (B11 + B31 + B15 + B35) / (2 · (G21 + G25)) + 2 · (R22 + R24) / (G12 + G32 + G14 + G34)) (56)

Final interpolation pixel data G23 is determined as follows. If both Gip0 and Gip2 are larger or smaller than Gip1 (average value), equation (57) is used, and otherwise equation (58) is used.
G23 = Gip0 (57)
G23 = Gip2 (58)

  By performing the above processing, it is possible to prevent the interpolation pixel data from being greatly out of the image including a large amount of high-frequency components near the Nyquist frequency.

  In step S470, it is confirmed whether or not all the designations of the focus detection pixels arranged at the positions where the green pixels are to be arranged among the focus detection pixels arranged in the GB row or the GB column are completed. If not completed, the process returns to step S310 to specify the next focus detection pixel. If completed, step S480 is performed to perform pixel interpolation processing of the focus detection pixel arranged at the position where the blue pixel is to be arranged. Proceed to

  In step S480, one of the focus detection pixels arranged at the position where the blue pixel is to be arranged among the focus detection pixels arranged in the GB row or GB column is sequentially designated. Here, it is assumed that the focus detection pixel at the position where B33 is to be output in FIG. In step S490, the pixel data of the focus detection pixel position is calculated based on the ratio of the pixel data of the blue pixel and the pixel data of the green pixel in the vicinity of the focus detection pixel, and the process proceeds to step S500.

  The blue pixels have a lower density than the green pixels, and the pixel pitch is twice that of the green pixels. Although it is possible to interpolate the pixel data at the focus detection pixel position by the same method as that for the green pixel, the Nyquist frequency of the blue pixel becomes ½ of the Nyquist frequency of the green pixel. For this reason, when the image contains a lot of high frequency components, the interpolation accuracy of the blue pixel is lower than the interpolation accuracy of the green pixel, and the ratio with the surrounding green pixel data is out of order and the color changes. There is a case. In terms of human visual characteristics, if the color (ratio of blue pixel data to green pixel data) is consistent with the surroundings, it is difficult to feel uncomfortable, so when interpolating blue pixel data, it is interpolated to align with the surrounding colors. I do.

For example, assuming that the ratio of the blue pixel data at the focus detection pixel position and the pixel data of the surrounding green pixels is equal to the ratio of the blue pixel data near the focus detection pixel position and the pixel data of the surrounding green pixels. Interpolated pixel data B33 is calculated as follows.
B33 / (G23 + G32 + G34 + G43) = ((B31 / (G21 + G30 + G32 + G41) + (B35 / (G25 + G34 + G36 + G45)) / 2 (59)
B33 = (G23 + G32 + G34 + G43). ((B31 / (G21 + G30 + G32 + G41) + (B35 / (G25 + G34 + G36 + G45)) / 2 (60)

  In step S500, it is checked whether all of the focus detection pixels arranged at the positions where the blue pixels are to be arranged among the focus detection pixels arranged in the GB row or the GB column are finished. If not completed, the process returns to step S480 to designate the next focus detection pixel. If completed, the pixel interpolation process is terminated in step S501.

  In the above-described embodiment, since the pixel interpolation process is changed according to the direction of the continuity of the image, the directionality of the edge pattern, the line pattern, and the like is compared with the pixel interpolation process based on the simple average process. Pixel interpolation processing can be performed satisfactorily even when the image having the image is superimposed on the focus detection pixel row.

  In addition, when interpolating green pixel data having a high pixel density, high-accuracy pixel interpolation can be achieved by detecting the continuity of the image and performing pixel interpolation processing according to the detected direction. At the same time, when interpolating the blue pixel data with a low pixel density, the blue pixel data is interpolated so as to make the color of the green pixel data with a high pixel density uniform, so that a color change due to an interpolation error can be prevented.

  Furthermore, pixel interpolation processing when the focus detection pixel array direction matches the direction of image continuity (average processing according to the direction), and pixel interpolation processing when the image continuity is in another direction (direction Different from the averaging processing according to Therefore, highly accurate pixel interpolation data can be obtained regardless of the relationship between the focus detection pixel array direction and the image continuity direction.

  In the pixel interpolation process when the focus detection pixel array direction and the image continuity direction match, the interpolation data obtained by the interpolation process and the pixel of the focus detection pixel are based on the average data of the pixel data in the vicinity of the focus detection pixel. The final interpolation data is determined by comparing with the conversion data obtained based on the data. Therefore, even when the image includes a high-frequency component near the Nyquist frequency, an interpolation error due to the interpolation process can be prevented.

  In addition, when the image around the focus detection pixel position is uniform or random, pixel interpolation is performed by a simple averaging process, which simplifies the calculation process and shortens the calculation time. .

  In the operation of the embodiment shown in FIG. 14, the detection of the image continuity in the direction of the focus detection pixel array (step 400) in contrast to the detection of the image continuity in the direction orthogonal to the focus detection pixel array (step 380). ) Is later. Accordingly, priority is given to detection of image continuity in the focus detection pixel array direction. Therefore, even when an edge pattern or a line pattern is superimposed on the focus detection pixel array, pixel interpolation processing in the focus detection pixel array direction can be reliably performed.

  A detailed operation of the defocus amount correction process in steps S131 and S132 of FIG. 12 will be described. In step S131, the same processing as in step S180 described above is performed. That is, with respect to the focus detection pixel originally arranged at the position where the blue pixel and the green pixel should be arranged, assuming that the imaging pixel is arranged at the position, the data of the imaging pixel (virtual imaging pixel) is obtained. Interpolation is performed from data of imaging pixels and focus detection pixels arranged in the vicinity of the focus detection pixels.

  Of the data of the virtual imaging pixels on the focus detection pixel column calculated by the above interpolation processing, the average value of the green imaging pixel data is Gm, and the average value of the blue imaging pixel data is Bm. Further, assuming that a red imaging pixel is virtually arranged at the position of the focus detection pixel, the average of the red imaging pixels adjacent to the focus detection pixel column is calculated, and the average value is defined as Rm.

  In step S132, chromatic aberration information is first read from the lens drive control device. As shown in FIG. 20, the chromatic aberration information is represented by the amounts of chromatic aberrations ΔR, ΔG, and ΔB with respect to the peak wavelengths λR, λG, and λB of red, green, and blue imaging pixels, with the chromatic aberration ΔG of the wavelength λG as a reference (ΔG = 0). It is a thing.

The defocus amount def obtained by Expression (7) is considered to be a weighted average of the defocus amounts defR, defG, and defB for the peak wavelengths λR, λG, and λB. Therefore, the defocus amount def is expressed by the following equation (61) using the ratio of the average values of the virtual imaging pixel data obtained in step 131 as a weighting factor. However, in the formula (61), Tm = Rm + Gm + Bm.
def = (Rm / Tm) .defR + (Gm / Tm) .defG + (Bm / Tm) .defB (61)

Assuming that the defocus amount in the green wavelength region that humans use for focus determination is the defocus amount defG at the peak wavelength λG of the visibility characteristic, Equations (62) and (63) are substituted into Equation (61). By organizing defG, equation (64) is obtained. DefG obtained by correcting the defocus amount def thus obtained is used as the final defocus amount.
defR = defG + ΔR (62)
defB = defG + ΔB (63)
defG = def− (Rm / Tm) · ΔR− (Bm / Tm) · ΔB (64)

  As described above, in the present invention, the focus detection pixels of the pupil division type phase difference detection method having different spectral sensitivity characteristics are mixed on the image sensor, and the image pickup pixels adjacent to the focus detection pixel array are mixed. The color of the image on the focus detection pixel array is detected based on the data, and the chromatic aberration correction of the defocus amount obtained based on the focus detection pixel data is performed based on the detected color. Therefore, the color of the image on the focus detection pixel row can be detected with very high accuracy, and accurate chromatic aberration correction can be performed.

  In addition, when detecting the color of the image on the focus detection pixel array based on the data of the imaging pixels close to the focus detection pixel row, the same processing as the pixel interpolation processing when generating image information is used. Can be simplified.

--- Other embodiments ---
In steps S330, S340, S360, S380, and S400 of the flowchart about the operation of the digital still camera 201 of the first embodiment shown in FIG. 14, it is used in an expression for determining image uniformity and continuity. The range of the pixel data of the imaging pixels is not limited to the range used in the above-described formula, and can be arbitrarily changed according to the situation.

  In step S460 shown in FIG. 14, when the image includes an extremely high spatial frequency component (spatial frequency component close to the Nyquist frequency corresponding to the pixel data pitch), interpolation is performed by using the pixel data Gip0 obtained by the interpolation process. The error increases. Therefore, the conversion data Gip2 obtained based on the pixel data of the focus detection pixel and the pixel data Gip0 are compared with the average data Gip1, and when both Gip0 and Gip2 are larger or smaller than Gip1 (average value), G23 = Gip0. In other cases, G23 = Gip2. However, in other cases, G23 = Gip1 or G23 = (Gip2 + Gip0) / 2 can be set.

  In addition, in order to detect that the image contains an extremely high spatial frequency component (a spatial frequency component close to the Nyquist frequency corresponding to the pixel data pitch), the pixel data string is directly Fourier-transformed to directly detect the image space. The frequency distribution may be detected, and G23 = Gip2 may be set when there are many high frequency components according to the result, and G23 = Gip0 may be set otherwise.

  In step S460 shown in FIG. 14, whether or not to adopt the pixel data Gip0 obtained by the interpolation processing is determined according to the conversion data Gip2 obtained based on the pixel data of the focus detection pixel. However, the upper and lower limits of the interpolated pixel data obtained by the averaging process and the interpolated pixel data obtained by the interpolation process may be limited according to data obtained by multiplying the converted pixel data Gip2 by a predetermined coefficient. For example, when the value of the interpolation pixel data exceeds twice the converted pixel data, the value of the interpolation pixel data is clipped to a value twice that of the converted pixel data. By doing in this way, it can prevent that the error of interpolation pixel data becomes abnormally large.

  In step S460 of the flowchart of the operation of the digital still camera 201 of the first embodiment shown in FIG. 14, in order to obtain the conversion data Gip2 based on the pixel data of the focus detection pixel, the pixel of the focus detection pixel at that position Data value A23 is used. However, when the defocus amount is large, an image shift occurs, and an error occurs from the conversion data when there is an imaging pixel without a color filter at that position. A23 / 2 + (A13 + A33) / 4 may be used in place of A23 in order to cancel the influence of image shift due to defocusing. Here, A13 and A33 are pixel data of focus detection pixels adjacent to the designated focus detection pixel (reference focus detection pixel). In this way, even if there is an image shift due to defocusing, the data that has been shifted is averaged, so that the above errors can be reduced. Furthermore, it is possible to change and average the range of focus detection pixels used for averaging in accordance with the size of the defocus amount.

  In step S460 of the flowchart of the operation of the digital still camera 201 of the first embodiment shown in FIG. 14, in order to obtain the conversion data Gip2 based on the pixel data of the focus detection pixel, the pixel of the focus detection pixel at that position Data value A23 is used. However, since the ratio between the pixel data of the focus detection pixel and the pixel data of the imaging pixel that should originally be at the position also changes depending on the aperture value of the optical system, the pixel data of the focus detection pixel is converted and used according to the aperture. It may be.

  In the flowchart of the operation of the digital still camera 201 according to the first embodiment shown in FIG. 14, the focus detection pixel array is used for the detection of image continuity in the direction orthogonal to the focus detection pixel array (step S380). The detection of the continuity of the image in the direction (step S400) is later. Accordingly, priority is given to detection of image continuity in the focus detection pixel array direction. However, in the state where the continuity of the image in the direction orthogonal to the focus detection pixel array is detected in step S380 (DCG = 2), if the continuity of the image in the focus detection pixel array direction is also detected, the image is displayed. It may be recognized that the pattern is a lattice pattern or a random pattern, and the process proceeds to step S420 with no directionality (DCG = 0).

  In the first embodiment described above, the averaging processing method according to the surrounding pixel data is changed according to the uniformity of the image in the vicinity of the focus detection pixel array and the direction of image continuity. However, the spatial frequency component distribution of the image in the vicinity of the focus detection pixel array may be calculated in a plurality of directions, and the averaging method may be changed according to the calculation result. For example, when there are few high frequency components in all directions, simple averaging processing is performed, and when there are many high frequency components in a specific direction, averaging processing is performed to average pixel data in a direction orthogonal to that direction. The range of pixel data used for the averaging process can be changed according to the frequency component. For example, when there are many low frequency components, averaging processing of pixel data in a relatively wide range is performed, and when there are many high frequency components, averaging processing of pixel data in a relatively narrow range is performed.

  In the first embodiment described above, the defocus amount calculated by Expression (7) is corrected so that the wavelength used for the focus determination by the human becomes the defocus amount defG at the peak wavelength λG of the spectral sensitivity characteristic of the green pixel. doing. However, you may correct | amend so that it may become a defocus amount of wavelengths other than this. For example, it is assumed that the in-focus plane is a weighted average of the defocus amounts defR, defG, and defB at the respective peak wavelengths λR, λG, and λB of the spectral characteristics of the imaging pixel. Then, the defocus amount calculated by Expression (7) can be corrected by assuming that the weight corresponds to the values R0, G0, and B0 at the peak wavelengths λR, λG, and λB of the visibility curve as shown in FIG. . The defocus amounts defR, defG, and defB at the respective peak wavelengths λR, λG, and λB are obtained by the above-described equations (62) to (64).

The finally corrected defocus amount defT is obtained by the following equation (65). However, in Expression (65), T0 = R0 + G0 + B0.
defT
= (R0 / T0) .defR + (G0 / T0) .defG + (B0 / T0) .defB
= DefG + (R0 / T0) · ΔR + (B0 / T0) · ΔB
= Def + (R0 / T0−Rm / Tm) · ΔR + (B0 / T0−Bm / Tm) · ΔB (65)

  In the above-described first embodiment, the defocus amount calculated by Expression (7) is always corrected. However, if the color purity of the image received by the virtual imaging pixel at the focus detection pixel position is high, human focus determination is performed on the image structure of that color. The amount correction may be prohibited. For example, when any of Rm / Tm, Gm / Tm, and Bm / Tm is equal to or greater than a predetermined value, it is assumed that a monochromatic light image with high color purity is formed on the focus detection pixel array, and the defocus amount is Prohibit correction.

In addition, when there is little difference between the color of the area in the vicinity of the focus detection pixel array including the focus detection pixel array and the color of the larger area in the outer periphery of the area, the same color in a relatively large area on the screen It is thought that the image of is formed. Therefore, it is possible to prohibit the correction of the defocus amount on the assumption that the defocus amount calculated by Expression (7) represents the defocus amount of the image plane of the color. For example, in FIG. 22, an average color normalized with respect to brightness in a region 120 in the vicinity of the focus detection pixel array 110 (a region where an imaging pixel used to calculate virtual imaging pixel data exists). Are R1, G1, and B1 in terms of RGB values. In addition, average colors normalized with respect to brightness in a relatively large area 130 surrounding the area 120 are R2, G2, and B2. In this case, if (R1, G1, B1) and (R2, G2, B2) are considered as three-dimensional vectors, it can be said that the smaller the size of the outer product, the smaller the difference between the two colors. Therefore, when (G1, B2-B1, G2) ² + (B1, R2-R1, B2) ² + (R1, G2-G1, R2) ² becomes a predetermined value or less, correction of the defocus amount is prohibited. To do.

In addition, when there is a large difference between the color in the vicinity of the focus detection pixel array including the focus detection pixel array and the color of the larger area outside the area, it is formed in a relatively large area on the screen. The defocus amount may be corrected with respect to the image plane of the color of the current image. For example, when (G1 · B2−B1 · G2) ² + (B1 · R2−R1 · B2) ² + (R1 · G2−G1 · R2) ² becomes a predetermined value or less, the target colors R2 and G2 , B2 is calculated as in the following equation to obtain a corrected defocus amount defC. However, in Formula (66), T2 = R2 + G2 + B2.
defC
= (R2 / T2) .defR + (G2 / T2) .defG + (B2 / T2) .defB
= DefG + (R2 / T2) · ΔR + (B2 / T2) · ΔB
= Def + (R2 / T2-Rm / Tm) · ΔR + (B2 / T2-Bm / Tm) · ΔB (66)

  In the first embodiment described above, the defocus amount calculated by Expression (7) is corrected. Based on the defocus amount, the lens drive amount calculated by the lens drive control device 206 is calculated using chromatic aberration. You may make it correct | amend according to information.

  In the above-described first embodiment, the description has been given on the assumption that the array of focus detection pixels is linear (one-dimensional) in the two-dimensional array of imaging pixels. It is not limited. The present invention is widely applicable when an array of pixels (for example, defective pixels) that cannot be directly used for imaging in a two-dimensional array of imaging pixels is linear (one-dimensional).

  The arrangement of the focus detection areas in the image sensor is not limited to the arrangement of the embodiment shown in FIG. 2, and the focus detection areas can be arranged in the horizontal direction and the vertical direction in the diagonal direction and other positions.

  In the image sensor 212 illustrated in FIG. 3, the focus detection pixels 313, 314, 315, and 316 are provided with one photoelectric conversion unit in one pixel, but the image sensor 212 </ b> A of the modified example illustrated in FIG. As described above, a focus detection pixel including a pair of photoelectric conversion units in one pixel may be used. In FIG. 18, the focus detection pixel 311 includes a pair of photoelectric conversion units in one pixel. The focus detection pixel 311 performs a function corresponding to a pair of the focus detection pixel 313 and the focus detection pixel 314 of the image sensor 212 shown in FIG.

  As shown in FIG. 19, the focus detection pixel 311 includes a microlens 10 and a pair of photoelectric conversion units 12 and 13. The focus detection pixel 311 is not provided with a color filter in order to increase the amount of light, and its spectral characteristics are spectral characteristics that combine the spectral sensitivity of a photodiode that performs photoelectric conversion and the spectral characteristics of an infrared cut filter (not shown). (FIG. 8). That is, the spectral characteristics are the sum of the spectral characteristics of the green pixel, red pixel, and blue pixel shown in FIG. 7, and the optical wavelength range of the sensitivity includes the optical wavelength range of the sensitivity of the green pixel, red pixel, and blue pixel. doing.

  In the image sensor 212 shown in FIG. 3, the example in which the imaging pixel includes a Bayer color filter is shown, but the configuration and arrangement of the color filter are not limited to this, and complementary color filters (green: G, yellow: Ye, An arrangement of magenta: Mg, cyan: Cy) may be employed.

  In the image pickup device 212 shown in FIG. 3, the focus detection pixel does not include a color filter, but the present invention can be applied even when one filter (for example, a green filter) of the same color filter as the image pickup pixel is provided. Can be applied.

  FIGS. 5 and 18 show an example in which the shape of the photoelectric conversion unit of the focus detection pixel is a semicircular shape. However, the shape of the photoelectric conversion unit is not limited to the shape of the embodiment, and may be another shape. . For example, the shape of the photoelectric conversion unit of the focus detection pixel may be an ellipse, a rectangle, or a polygon.

  In the imaging element 212 shown in FIG. 3, the example in which the imaging pixels and the focus detection pixels are arranged in a dense square lattice arrangement is shown, but a dense hexagonal lattice arrangement may be used.

  In the image pickup device 212 shown in FIG. 3, the focus detection pixel does not include a color filter, but the present invention can be applied even when one filter (for example, a green filter) of the same color filter as the image pickup pixel is provided. Can be applied.

  The image pickup apparatus of the present invention is not limited to a digital still camera or a film still camera including an interchangeable lens and a camera body, and can also be applied to a lens-integrated digital still camera, a film still camera, and a video camera. Further, the present invention can be applied to a small camera module, a surveillance camera, a visual recognition device for a robot, etc. incorporated in a mobile phone. Alternatively, the present invention can also be applied to a focus detection device other than a camera, a distance measuring device, or a stereo distance measuring device.

10, 10a, 10b, 10c, 10d; micro lens, 11, 12, 13, 13a, 14a, 14b, 16, 17; photoelectric conversion unit, 29; semiconductor circuit board, 73, 74, 83, 84; for focus detection Luminous flux, 90; exit pupil, 91; optical axis of interchangeable lens, 93, 94; distance measurement pupil, 100; photographing screen, 101 to 103; focus detection area, 110; focus detection pixel row, 120; Neighboring area 130; A relatively large area surrounding the vicinity of the focus detection pixel array; 201; Digital still camera 202; Interchangeable lens 203; Camera body 204; Mount portion 206; Lens drive control device 208 Lens for zooming, 209; lens, 210; lens for focusing, 211; diaphragm, 212, 212A; image sensor, 213; 214; body drive control device, 215; liquid crystal display element drive circuit, 216; liquid crystal display element, 217; eyepiece, 219; memory card, 310; imaging pixels, 311, 313, 313a, 313b, 314, 314a, 314b, 315, 316; focus detection pixels

Claims (8)

A plurality of types of imaging pixels having different spectral sensitivity characteristics that receive an imaging light beam that has passed through the imaging optical system and output an imaging pixel signal, and a focus detection pixel signal that receives the focus detection light beam that has passed through the imaging optical system An image sensor in which a plurality of focus detection pixels having different spectral sensitivity characteristics from the imaging pixels are arranged according to a two-dimensional array;
Focus detection means for detecting a focus adjustment state of the photographing optical system based on the focus detection pixel signal;
Interpolating means for calculating the imaging pixel signals at the positions of the plurality of focus detection pixels based on the imaging pixel signals output from the plurality of types of imaging pixels arranged in the vicinity of the plurality of focus detection pixels by interpolation calculation. When,
Based on the imaging pixel signal at the position of the plurality of focus detection pixels calculated by the interpolation unit and chromatic aberration information of the photographing optical system, a physical quantity representing the focus adjustment state detected by the focus detection unit is obtained. Correction means for correcting;
When the hue indicated by the imaging pixel signal at the position of the plurality of focus detection pixels calculated by the interpolation means is biased, a physical quantity representing the focus adjustment state detected by the focus detection means is obtained by the correction means. An imaging apparatus comprising: prohibiting means for performing prohibition control for prohibiting correction. A plurality of types of imaging pixels having different spectral sensitivity characteristics that receive an imaging light beam that has passed through the imaging optical system and output an imaging pixel signal, and a focus detection pixel signal that receives the focus detection light beam that has passed through the imaging optical system An image sensor in which a plurality of focus detection pixels having different spectral sensitivity characteristics from the imaging pixels are arranged according to a two-dimensional array;
Focus detection means for detecting a focus adjustment state of the photographing optical system based on the focus detection pixel signal;
Interpolating means for calculating the imaging pixel signals at the positions of the plurality of focus detection pixels based on the imaging pixel signals output from the plurality of types of imaging pixels arranged in the vicinity of the plurality of focus detection pixels by interpolation calculation. When,
Based on the imaging pixel signal at the position of the plurality of focus detection pixels calculated by the interpolation unit and chromatic aberration information of the photographing optical system, a physical quantity representing the focus adjustment state detected by the focus detection unit is obtained. Correction means for correcting;
Surrounding the area where the hues indicated by the imaging pixel signals at the positions of the plurality of focus detection pixels calculated by the interpolation unit and the plurality of types of imaging pixels used for the interpolation processing by the interpolation unit are arranged, When the difference between the hues indicated by the imaging pixel signals of the plurality of types of imaging pixels arranged in a wide area larger than the area is small, the physical quantity representing the focus adjustment state detected by the focus detection means is the correction means An imaging device comprising: prohibiting means for performing prohibition control for prohibiting correction by the control unit. The imaging device according to claim 1 or 2,
The imaging pixels include first, second, and third imaging pixels having first, second, and third spectral sensitivities, respectively.
The focus detection pixel has a spectral sensitivity including the first, second, and third spectral sensitivities,
The chromatic aberration information of the photographing optical system is a second related to the second wavelength corresponding to the second spectral sensitivity with reference to the first chromatic aberration information related to the first wavelength corresponding to the first spectral sensitivity. Chromatic aberration information and third chromatic aberration information related to the third wavelength corresponding to the third spectral sensitivity, respectively,
The imaging device corrects a physical quantity representing the focus adjustment state detected by the focus detection unit based on the second chromatic aberration information and the third chromatic aberration information. The imaging device according to claim 3.
The two-dimensional array is a Bayer array;
The first imaging pixel is a green pixel having high spectral sensitivity in a wavelength band of green light,
The second imaging pixel is a red pixel having high spectral sensitivity in a wavelength band of red light,
The third image pickup pixel is a blue pixel having high spectral sensitivity in a wavelength band of light of blue color. In the imaging device according to any one of claims 1 to 4,
The physical quantity representing the focus adjustment state detected by the focus detection means is a defocus amount. In the imaging device according to any one of claims 1 to 4,
The focus detection means calculates a defocus amount based on the focus detection pixel signal,
The physical quantity representing the focus adjustment state detected by the focus detection means is a moving distance of the photographing optical system according to the defocus amount. In the imaging device according to any one of claims 1 to 4,
The image pickup apparatus, wherein the focus detection unit detects a focus adjustment state of the photographing optical system by a pupil division type phase difference detection method based on the focus detection pixel signal. A plurality of types of imaging pixels having different spectral sensitivity characteristics that receive an imaging light beam that has passed through the imaging optical system and output an imaging pixel signal, and a focus detection pixel signal that receives the focus detection light beam that has passed through the imaging optical system An image sensor in which a plurality of focus detection pixels having different spectral sensitivity characteristics from the imaging pixels are arranged according to a two-dimensional array;
A focus detection unit that detects a focus adjustment state of the photographing optical system based on the focus detection pixel signal and calculates a defocus amount;
Interpolating means for calculating the imaging pixel signals at the positions of the plurality of focus detection pixels based on the imaging pixel signals output from the plurality of types of imaging pixels arranged in the vicinity of the plurality of focus detection pixels by interpolation calculation. When,
Correction means for correcting the defocus amount based on the imaging pixel signals at the positions of the plurality of focus detection pixels calculated by the interpolation means and chromatic aberration information of the photographing optical system ,
The imaging pixels include first, second, and third imaging pixels having first, second, and third spectral sensitivities, respectively.
The focus detection pixel has a spectral sensitivity including the first, second, and third spectral sensitivities,
The chromatic aberration information of the photographing optical system is a second related to the second wavelength corresponding to the second spectral sensitivity with reference to the first chromatic aberration information related to the first wavelength corresponding to the first spectral sensitivity. Chromatic aberration and third chromatic aberration information related to the third wavelength corresponding to the third spectral sensitivity, respectively,
The interpolation unit is configured to detect the position of the plurality of focus detection pixels based on the imaging pixel signals output from the first, second, and third imaging pixels arranged in the vicinity of the plurality of focus detection pixels. Calculating the first, second, and third imaging pixel signals by interpolation;
The correction means is a value obtained by dividing the second imaging pixel signal calculated by the interpolation means by the total value of the first, second, and third imaging pixel signals calculated by the interpolation means, A value obtained by multiplying the third chromatic aberration information by the first correction value multiplied by the second chromatic aberration information and a value obtained by dividing the third imaging pixel signal calculated by the interpolation means by the total value. An image pickup apparatus that subtracts a correction value of 2 from the defocus amount .

Images