JP6234094B2 - Focus detection apparatus and imaging apparatus - Google Patents

Description

The present invention relates to a focus detection apparatus, an imaging apparatus, and the like that perform focus detection that detects a defocus amount of a subject by a phase difference detection method.

Conventionally, a so-called phase difference detection type focus detection apparatus is known as a focus detection apparatus for a camera that provides an AF (automatic focus adjustment) function. In such a focus detection apparatus, a light beam from a subject that has passed through different exit pupil regions of the photographing lens is imaged on a pair of focus detection sensors. Then, the correlation calculation is performed while relatively shifting the pair of pixel signal sequences, the relative phase difference between the pair of pixel signal sequences is detected, and the defocus amount of the photographing lens is obtained based on the phase difference. .

Patent Document 1 proposes a focus detection device that detects an abnormal pixel in a pixel of a focus detection sensor that generates an image signal and performs a correlation operation on a signal waveform of a pixel excluding the abnormal pixel.

JP 2012-42597 A

However, in the conventional technique disclosed in Patent Document 1 described above, when an abnormal pixel is detected, the abnormal pixel is always removed, and an operation for detecting a phase difference of image signals from other pixels is performed. However, if computation is performed by removing signals from abnormal pixels, the accuracy of focus detection may deteriorate if many abnormal pixels such as saturation occur at the time of ultra-high luminance. For example, when the luminance level of the subject is very bright, the number of pixels to be excluded increases and the number of calculation pixels decreases. Since a sufficient contrast cannot be obtained from a small number of calculation pixels, the amount of correlation when detecting the phase difference is also reduced. Therefore, it is easily affected by noise components generated in the pixels, and the focus detection accuracy may deteriorate.

Therefore, an object of the present invention is to provide a focus detection device that can reduce deterioration of focus detection accuracy even when a large number of abnormal pixels occur due to saturation or the like in photographing a super-bright subject, an imaging device using the focus detection device, and the like. That is.

In order to achieve the above object, a focus detection apparatus according to the present invention includes a plurality of pixels that receive and output an image signal of a pair of subjects formed by a light beam that has passed through a pair of pupil regions of a photographing optical system. Means, an abnormal pixel detecting means for detecting an abnormal pixel that generates an abnormal signal included in a pair of pixel signal sequences formed by the image signal output from the light receiving means, and an abnormality detected by the abnormal pixel detecting means A first computing means for detecting a phase difference between a pair of pixel signal sequences including a signal from a pixel; a second computing means for detecting a phase difference between a pair of pixel signal sequences excluding a signal from an abnormal pixel; And a determination unit that selects one of the results from the second calculation unit, and the focus state is detected based on the result selected by the determination unit.

According to the present invention, it is possible to perform focus detection with relatively high accuracy even when an abnormal pixel is generated by a subject such as an ultra-high brightness light source when a pair of image signals are received.

It is a figure which shows the structure of the digital still camera of embodiment. It is a figure which shows schematic structure of the focus detection apparatus containing the focus detection unit of FIG. 2 is a flowchart illustrating a shooting operation flow when a focus detection sensor is used in the camera of FIG. 1. It is a flowchart which shows the focus detection process in Example 1 of this invention. It is a figure which shows the image signal which has not reached the saturation level. It is a figure which shows the shift waveform in a 1st calculation. It is a figure which shows the correlation amount for every shift amount in a 1st calculation. It is a flowchart which shows the waveform generation of saturation pixel exclusion. It is a figure which shows the exclusion waveform and calculation pixel range which show the determination by the number of calculation pixels. It is a figure which shows the shift waveform in a 2nd calculation. It is a figure which shows the saturation pixel position in a shift waveform. It is a flowchart which shows the focus detection process in Example 2 of this invention. It is a figure which shows the relationship between the phase difference of an image signal, and AF error. It is a figure which shows the shift waveform of the image signal which has a phase difference of -0.5 bit. It is a figure which shows the shift waveform with the pixel which has reached the saturation level in the image signal of FIG. It is a figure which shows the correlation amount for every shift waveform of FIG. 14, FIG. It is a figure which shows the shift waveform of the image signal which has a -1 bit phase difference. It is a figure which shows the shift waveform with the pixel which has reached the saturation level in the image signal of FIG. It is a figure which shows the correlation amount for every shift waveform of FIG. 17, FIG. It is a flowchart which shows the focus detection process in Example 3 of this invention. It is the top view and sectional drawing of the pixel for imaging of an image sensor. It is the top view and sectional drawing of the AF pixel for horizontal directions of an image sensor. It is pixel array explanatory drawing of the minimum unit of an image pick-up element. It is explanatory drawing of the pixel grouping method at the time of a horizontal shift focus detection. 2 is a flowchart showing a shooting operation flow when focus detection pixels in an imaging surface of the camera of FIG. 1 are used. It is a flowchart of the imaging | photography subroutine of FIG. It is a flowchart which shows the focus detection process in Example 4 of this invention.

In the present invention, a first calculation for detecting a pixel that generates an abnormal signal in a pair of pixel signal sequences based on an image signal from the light receiving means, and detecting a phase difference between the pair of pixel signal sequences including the abnormal pixel signal; A second operation for detecting a phase difference between a pair of pixel signal sequences excluding abnormal pixel signals is executed. And even when an abnormal pixel occurs, select one of the results of the first calculation and the second calculation based on a predetermined criterion so that relatively accurate focus detection can be performed, The focus state is detected based on the selected result.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is an image pickup apparatus including a focus detection apparatus according to an embodiment of the present invention.
Example 1
In FIG. 1, the imaging apparatus according to the present embodiment is a lens interchangeable single-lens reflex camera (hereinafter simply referred to as “camera”) 100. The camera 100 includes a camera body 30 and a photographing lens unit 20 configured to be detachable from the camera body 30. The photographic lens unit 20 and the camera body 30 are connected via a mount indicated by a vertical dotted line in the center of the figure. The photographic lens unit 20 includes a photographic lens 21, a diaphragm 22, a lens side MPU (microprocessing unit) 1, a lens driving unit 2, a diaphragm driving unit 3, a repetition position detection unit 4, and an optical information table 5. Is provided.

The lens side MPU 1 performs all calculations and controls related to the operation of the photographic lens unit 20. The lens driving unit 2 is a driving unit that drives the photographing lens 21 in accordance with control by the lens side MPU 1. The aperture drive unit 3 is a drive unit that drives the aperture 22 according to control by the lens side MPU 1. The repeat position detection unit 4 is a detection unit that detects the repeat position of the lens barrel. The optical information table 5 is optical information necessary for automatic focus adjustment, and is stored in a memory (not shown).

The camera body 30 includes a camera-side MPU 6, a focus detection unit 7, a shutter drive unit 8, a dial unit 10, a photometry unit 11, and a memory 23. The camera body 30 includes a main mirror 12, a sub mirror 13, a focus plate 14, a penta mirror 15, a finder 16, an image sensor (image sensor) 101, a switch SW1_18, and a switch SW2_19. The camera side MPU 6 performs all calculations and controls related to the operation of the camera body 30. The camera-side MPU 6 is connected to the lens-side MPU 1 via a mount signal line, and acquires lens position information from the lens-side MPU 1 or acquires unique optical information for each lens driving and interchangeable lens.

The camera-side MPU 6 includes a ROM (not shown) that stores a program for controlling the operation of the camera body 30, a RAM (not shown) that stores variables, and an EEPROM (electrical erasure, A writable memory (not shown) is incorporated. A focus detection process, which will be described later, is executed by a program stored in the ROM. Reference numeral 17 denotes a display device such as an LCD, which displays information related to the shooting mode of the camera, a preview image before shooting, a confirmation image after shooting, and the like.

The focus detection unit 7 includes a focus detection sensor, which will be described later, and performs focus detection processing and also performs focus detection by a phase difference detection method. The focus detection unit 7 notifies the camera-side MPU 6 of the completion of signal readout from the focus detection sensor. The shutter drive unit 8 is a drive unit for driving a shutter (not shown). The dial unit 10 is an operation unit for changing various settings of the camera 100 and can switch, for example, continuous shooting speed (continuous shooting speed), shutter speed, aperture value, shooting mode, and the like. The photometric unit 11 includes a photometric sensor, and performs photometric processing via a photometric sensor (not shown) based on the light flux from the pentamirror 15 in response to a half-press operation on a release button (not shown). These are all connected to the camera-side MPU 6. The photometric sensor comprises a photoelectric conversion element such as a photodiode and its signal processing circuit, and outputs a signal related to the luminance level of the subject. The output signal is input to the camera-side MPU 6.

The main mirror 12 has a function of folding most of the light beam incident through the photographing lens unit 20 upward and forming a subject image on the focus plate 14. The subject image on the focus plate 14 is converted and reflected into an erect image by the pentamirror 15 and guided to the viewfinder 16. Thereby, it functions as an optical viewfinder. Part of the light transmitted through the pentamirror 15 is guided to the photometry unit 11. The main mirror 12 transmits a part of the incident light beam, and the transmitted light is guided to the focus detection unit 7 through the sub mirror 13. When the camera 100 is in a shooting state, the main mirror 12 and the sub mirror 13 are retracted, and a light flux from a subject incident through the shooting lens unit 20 is imaged on the image sensor 101. The switch SW1_18 is a switch that is turned on by a first stroke operation (half press) of the release button. The switch SW2_19 is a switch that is turned on by the second stroke operation (full press) of the release button.

Next, the configuration of the focus detection unit 7 will be described with reference to FIG. FIG. 2 shows a schematic configuration of the focus detection unit 7 of FIG. Part of incident light from the photographic lens unit 20 is transmitted through the main mirror 12, bent downward by the rear sub-mirror 13, and guided to the focus detection unit 7. The focus detection unit 7 includes a field mask 102, an infrared cut filter 103, a field lens 104, a diaphragm 105, a secondary imaging lens 106, a focus detection sensor 107, and a plurality of reflection mirrors 108a to 108c. The focus detection sensor 107 for focus detection includes a pair of line sensors composed of photoelectric conversion elements such as photodiodes, signal processing circuits thereof, and the like. Then, a pixel signal (image signal) relating to the distance to the subject or the defocus amount is output, and the image signal is input to the camera side MPU 6. In this specification, image signals from a pair of line sensors are referred to as an A image and a B image. The camera-side MPU 6 includes an abnormal pixel detection unit (abnormal pixel detection unit) 111, a correlation calculation unit (calculation unit) 112, and a calculation switching determination unit (determination unit) 113 as blocks related to the focus detection process.

Next, the basic operation of the camera 100 of FIG. 1 will be described using the flowchart of FIG. When the power of the camera 100 in FIG. 1 is turned on, the memory contents and the initialization state of the execution program are detected, and the shooting preparation operation is executed, and the flowchart in FIG. 3 is started. In step S201, the camera side MPU 6 recognizes that the user has performed various settings of the camera by operating various buttons (not shown). In step S202, the camera side MPU 6 determines whether or not the switch SW1_18 is turned on. When the switch SW1_18 is turned on, the process proceeds to step S204. On the other hand, if it is not turned on, the process proceeds to step S203 to determine whether the power is on or off. When the power is off, a series of shooting operations are terminated. If the power is on, the process returns to step S201.

In step S204, the camera side MPU 6 performs a focus detection process to be described later in response to the switch SW1_18 being turned on. In step S205, the camera-side MPU 6 determines whether or not the switch SW2_19 has been turned on. If it is determined that the switch SW2_19 has been turned on, the camera-side MPU 6 performs a photographing process in step S206. On the other hand, if the camera-side MPU 6 determines that the switch SW2_19 is not turned on, the process returns to step S201.

FIG. 4 is a flowchart showing the focus detection process in step S204 of FIG. 3 in the present embodiment. In step S301, the camera-side MPU 6 outputs a signal accumulation instruction to the focus detection sensor. Based on the instruction, the focus detection sensor starts a signal accumulation operation. When the accumulation operation ends, the focus detection sensor outputs the image signals obtained by the pair of line sensors to the camera side MPU 6 as an A image and a B image, respectively. An example of the A and B images output in step S301 is shown in FIG. In this example, the A image and the B image have a phase difference of −2 bits to −3 bits. Here, the case where the A image exists on the right side of the B image is defined as a negative phase difference. The signal levels of the A and B images in FIG. 5 do not exceed the saturation level. For the A and B images, if there is a pixel that exceeds the saturation level, the pixel signal that exceeds the saturation level is clipped to the saturation level and output.

Next, in step S302, the camera-side MPU 6 determines whether or not the contrast information obtained from the image signal output from the focus detection sensor in step S301 is equal to or less than a predetermined amount. If the contrast information is less than or equal to the predetermined amount, the process proceeds to step S303, and the user is warned that focus detection is impossible without performing focus detection. On the other hand, if the contrast information is greater than the predetermined amount, the process proceeds to the first calculation in step S304. The contrast information here is the total sum of values obtained by squaring the difference between pixel signals of pixels adjacent to each other in the pixel signal sequence in the calculation pixel range of the image signal by the camera side MPU 6. When the contrast information is small, the correlation amount described later is also small when detecting the phase difference. Therefore, it is easily affected by noise components generated in the pixels, and the accuracy of focus detection deteriorates. For the above reasons, focus detection is not performed when the contrast information is small and the reliability of focus detection is not sufficient.

Next, in step S304, the camera side MPU 6 performs a first calculation in the correlation calculation unit 112 of FIG. 2 based on the A image and the B image. The first calculation will be described in detail. A method for calculating the shift waveforms and correlation amounts of the A and B images in the first calculation will be described with reference to FIG. A waveform obtained by shifting the image signal acquired from the focus detection sensor by a predetermined pixel (bit) is defined as a shift waveform. FIG. 6A shows the image signal itself acquired from the sensor, which is a shift waveform with a shift amount of zero. FIG. 6B shows a shift waveform with a shift amount of −1 bit, and FIG. 6C shows a shift waveform with a shift amount of −2 bits. Here, the shift amount is controlled by fixing the B image and shifting the A image left and right. The correlation amount C (k) at the shift amount kbit is calculated using the following correlation calculation formula (1).

In this example, the maximum shift amount is ± 4 bits, and nine shift waveforms are stored in the RAM. The calculation range is set as a calculation range in consideration of the maximum shift amount and a range in which the first 4 bits and the last 4 bits of the total number 25 of the output A and B images are excluded from the calculation. Accordingly, the difference area between the A image and B image of each shift waveform (the area of the shaded portion in FIGS. 6A to 6C) is the correlation amount C, and the shift amount with the smallest correlation amount C of the shift waveform is A. This corresponds to the phase difference between the image and the B image.

FIG. 7 shows the correlation amount C when the two pixel signals in FIG. 6A are thus shifted within a relative range of ± 4 bits. Although the correlation amount is the smallest value when the shift amount k = -2 bits, the minimum value is in k = -2 bits to -3 bits as indicated by the dotted line. Therefore, the phase difference detection resolution is increased by performing an interpolation calculation from the shift amount k = −2 and the correlation amount information in the vicinity thereof. Here, linear interpolation is performed as shown by a broken line by using four correlation amounts C of shift amounts k = −1 bit to −4 bits. The four shift amounts used for linear interpolation are determined as follows. First, the shift amount k indicating the smallest correlation amount is determined, and the vicinity thereof is set as shift amounts k−2, k−1, and k + 1. When there are two items having the smallest correlation amount, the larger shift amount is adopted as the shift amount k. In this way, the first calculation means shifts the pair of pixel signal sequences relatively one pixel at a time, and performs an interpolation calculation from the correlation amounts at a plurality of shift positions, whereby the phase difference between the pair of pixel signal sequences of the image signal. Can be detected in units smaller than one pixel. The calculation principle is the same in the second calculation means described later.

Next, in step S305, the camera-side MPU 6 uses the abnormal pixel detection unit 111 of FIG. 2 having means for detecting an abnormal signal, and a pixel signal sequence of A and B images that are a pair of image signals acquired in step S301. Abnormal pixels (saturated pixels, defective pixels) are detected. Here, a saturated pixel is detected as an abnormal pixel. If there is no saturated pixel, the process proceeds to focus determination in step S310. On the other hand, if a saturated pixel exists, the process proceeds to step S306. In step 306, a waveform excluding the saturated pixels is generated from the four shift waveforms used in the first interpolation operation in step 304 and the saturated pixel information detected in step 305. Here, a waveform is generated by excluding the saturated pixel signal and the neighboring pixel signals from the four shift waveforms. Details of the waveform generation method excluding the saturated pixels will be described later.

In steps S307 to S308, the calculation switching determination unit 113 in FIG. 2 calculates the result of the first calculation in step S304 and the result of the second calculation calculated by excluding a saturated pixel and pixels near the saturated pixel, which will be described later. Perform reliability judgment. The reliability determination is made up of two items, the number of pixels and the contrast.

In step S307, it is determined whether or not the number of pixels in the calculation range is equal to or less than a predetermined number in the four excluded waveforms subjected to the waveform generation process excluding the saturated pixels. When the number of calculation pixels is equal to or less than the predetermined number, it is determined that the first calculation including the saturated pixel and the neighboring pixels is more accurate, and the process proceeds to the focus determination in step S310. On the other hand, when the number of operation pixels is larger than the predetermined number, the process proceeds to the determination based on the contrast information in step S308.

Here, step S307 will be described in detail with reference to FIG. FIG. 9 shows one of the four excluded waveforms that have been subjected to the processing of generating the excluded waveform. The horizontal axis indicates the pixel position, and the vertical axis indicates the level of the image signal. The exclusion range illustrated in FIG. 9 indicates the position of the pixel to be excluded, and it is assumed that the central signal exceeds the saturation level as compared to the waveform illustrated in FIG. The calculation pixel ranges are the 4th to 8th pixels and the 14th to 21st pixels.

In the case of an image signal with a small number of calculation pixels, the correlation amount is also small. Therefore, the ratio of noise increases with respect to the pixel signal obtained from the brightness of the subject, and it can be determined that the reliability is low. Therefore, when the number of pixels to be excluded becomes less than a predetermined number due to an increase in the number of pixels to be excluded, focus detection is performed based on the result of the first calculation. The reason for using the result of the first calculation is as follows. First, sufficient contrast is obtained by the determination in step S302, and is not easily affected by saturation. Furthermore, more accurate focus detection is possible when the calculation is performed using normal pixel signals to be excluded in the vicinity of the saturated pixel than the result of focus detection obtained from a waveform with a small number of calculation pixels. Because.

Next, in step S308, it is determined whether or not the contrast information of the excluded waveform generated in step S306 is equal to or less than a predetermined value. If the contrast information is less than or equal to the predetermined value, it is determined that the first calculation is more accurate, and the process proceeds to the focus determination in step S310. On the other hand, if the contrast is greater than the predetermined value, it is determined that the second calculation is more accurate than the first calculation, and the process proceeds to the second calculation in step S309.

Here, step S308 will be described in detail with reference to FIG. Contrast information is the sum of values obtained by squaring differences between adjacent pixels in the calculation pixel range, and is expressed by the following equation (2). CNT indicates contrast information.

Here, k is 0 when any pixel of A (i) and A (i + 1) is a pixel to be excluded, and both the pixels of A (i) and A (i + 1) are pixels in the calculation pixel range. If there is one, the contrast information is obtained from the pixels in the calculation pixel range. In FIG. 9, the 9th to 13th pixels are pixels to be excluded, and the contrast information based on the pixel signal during this period is not added to the CNT in Expression (2). The contrast information is obtained for each of the A image and the B image using Equation (2), and the contrast information of the smaller one of the A image and the B image is taken. The comparison is performed on the four excluded waveforms by comparing the magnitude relationship between the contrast information obtained from the equation (2) and the predetermined determination value stored in the calculation switching determination unit 113. Contrast information and AF accuracy are also in the same relationship as the number of calculated pixels in step S307, and the correlation amount obtained from an image signal with small contrast information is susceptible to noise. Therefore, when the contrast information obtained from the waveform subjected to the exclusion process is less than or equal to a predetermined amount, focus detection is performed based on the result of the first calculation.

If the prescribed value is exceeded for the two reliability items in steps S307 and S308, it is determined that the second calculation excluding the saturated pixel and its neighboring pixels is more accurate, and the second calculation in step S309 is performed. Move on to computation.

Next, in step S309, the correlation amount C is obtained from the four shift waveforms generated by the waveform generation excluding the saturated pixels in step S306 using the above equation (1) in the same manner as the first calculation in step S304. A shift amount that is a minimum value is obtained from each correlation amount, and is used as the second calculation result. The phase difference is calculated by setting the intersection of the straight line connecting the correlation amounts C (F-1) and C (F) and the straight line connecting the correlation amounts C (F + 1) and C (F + 2) as the minimum value of the correlation amount.

In step S310, it is determined whether or not the phase difference calculated in step S304 or step S309 is within a predetermined range. In order to convert to a defocus amount indicating the lens drive amount, the defocus amount is obtained by multiplying the calculated phase difference by a coefficient. If the calculated defocus amount is within a desired range, for example, within 1/4 Fδ (F: lens aperture value, δ: constant: 20 μm, and therefore 10 μm for the open aperture of F2.0 lens), it is determined to be in focus. Then, a series of focus detection processing ends. On the other hand, if it is greater than 1 / 4Fδ, the process proceeds to lens driving in step S311. In step S311, a lens driving amount is calculated based on the phase difference information calculated in step S304 or S309, and is transmitted to the lens side MPU1. The lens-side MPU 1 performs lens driving control based on the transmitted lens driving amount, returns to step S301, and repeats the above-described operation until it is in focus.

Here, a waveform generation method for excluding saturated pixels will be described in detail with reference to the flowchart of FIG. In step S401, a shift amount range used in the second calculation is determined based on the phase difference P detected from the first calculation. First, the shift amount k = F−1 of the shift waveform to be processed first is set. F is a value rounded to an integer smaller than the detected phase difference P. Taking FIG. 7 as an example, the phase difference P = −2 bits to −3 bits and F = −3. In step S402, the shift waveform of the shift amount (k) acquired by the first calculation is read from the RAM.

In step S403, it is determined whether one of the A image and the B image is a saturated pixel from the shift waveform read in step S402, and the position of the saturated pixel is detected. Saturation is determined by comparing the pixel signal with a saturation level value stored in advance in the RAM. In step S404, the position of the pixel determined as the saturated pixel in step S403 is stored in the RAM. In step S405, it is determined whether or not k is F + 2. If F + 2, the process proceeds to step S407. That is, here, when the saturated pixel detection process is performed for the shift waveforms of the four shift amounts, the process proceeds to the next step S407. On the other hand, if it is smaller than F + 2, the shift waveform for detecting the saturated pixel remains, so in step S406, the shift amount k = k + 1 is set, and the detection processing is performed for the shift waveform of the next shift amount.

In step S407, since the saturated pixel position in each shift waveform is stored in step S404, the pixel position stored as the saturated pixel position in step S404 is read from the RAM. Then, a pixel position stored as a saturated pixel position at any one of the four shift waveforms is set as a pixel position to be excluded.

Here, step S407 will be described in detail with reference to FIGS. FIG. 10 shows four shift waveforms used in the second calculation. The horizontal axis indicates the pixel position, and the vertical axis indicates the level of the image signal. The shift waveform in FIG. 10 is the shift amount of F-1, F, F + 1, F + 2 stored in the RAM as a result of the first calculation in step S304 in FIG. , −3, −2, and −1. FIG. 10A shows a shift waveform with a shift amount k = −1, a saturated pixel exists in either the A image or the B image at pixel positions 9 to 12, and the camera side MPU 6 sets the pixel position 9 as a saturated pixel position. Four pixels of 10, 11, and 12 are stored in the RAM. FIG. 10B shows a shift waveform with a shift amount k = −2, and three pixels at pixel positions 9, 10, and 11 are stored in the RAM by the same determination as in FIG. FIG. 10C shows a shift waveform with a shift amount k = -3, and three pixels at pixel positions 9, 10, and 11 are stored in the RAM by the same determination as described above. FIG. 10D shows a shift waveform with a shift amount k = −4, and four pixels at pixel positions 8, 9, 10, and 11 are stored in the RAM by the same determination as described above.

The saturated pixel positions in the shift waveforms of FIGS. 10A to 10D are collectively shown in the table of FIG. Pixel positions corresponding to saturated pixels and excluded pixels are represented by “x”. Thus, the number of pixels in which either the A image or the B image is saturated differs depending on the shift waveform. The correlation amount is a value obtained by summing up the differences between the image signals paired with each pixel, as shown in the correlation calculation expression (1). If only saturated pixels or only saturated pixels and their paired pixels are excluded, the number of operation pixels differs for each shift waveform, and the total number of pixels differs. It is different, and the correct calculation result cannot be obtained.

Therefore, when any pixel position (i) is stored as a saturated pixel position in any one of the shift waveforms in FIGS. 10A to 10D, the pixel position (i) is excluded from other shift waveforms. Let it be a pixel. In this case, the excluded pixel may be a normal pixel. That is, in the four shift waveforms used for the second calculation, five pixels of 8, 9, 10, 11, and 12 are excluded pixel positions. Accordingly, in the shift waveform of FIG. 10A, the pixel positions 9, 10, 11, and 12 are saturated pixels, but the pixel position 8 that is a normal pixel is also handled in the same manner as the saturated pixel. For this reason, a pixel that is a saturated pixel with any shift waveform is removed from the four shift waveforms. Accordingly, when the correlation amount C of the four shift waveforms is calculated from the equation (1), the calculation range and the number of calculation pixels become equal, and when calculating from the excluded waveform, a highly accurate phase difference can be detected. .

In step S408, in the four shift waveforms, the signal of the pixel that was excluded in step S407 is set to 0, thereby generating an excluded waveform that excludes the saturated pixel and its neighboring pixels.

According to the first embodiment, when a saturated pixel signal is present in the image signal, the AF error caused by the saturation can be reduced by performing the correlation calculation by excluding the saturated pixel signal and the neighboring pixel signal. However, depending on the subject situation, the reliability of the number of calculation pixels and contrast information is lost by excluding the saturated pixel and its neighboring pixels. Thus, reliability determination is performed on the excluded waveform based on the number of calculation pixels and contrast information (S307, S308), and correlation calculation is selected based on the determination result, thereby performing high-precision focus detection. I can do it.

(Example 2)
In the first embodiment, the first and second operations are switched by determining the reliability from the waveform after the exclusion process. In the second embodiment, a focus detection apparatus or method that uses the result of the first calculation for calculation switching determination will be described. Hereinafter, Example 2 will be described in detail with reference to FIG. About the part similar to the said Example 1, the description is abbreviate | omitted using the same code | symbol. Further, the configuration (FIG. 1) and the photographing operation (FIG. 3) of the camera 100 in the second embodiment are the same as those in the first embodiment, and the description thereof is omitted.

FIG. 12 is a flowchart showing the focus detection process in the present embodiment. In the flowchart of FIG. 12, steps other than step S507 are the same step numbers as those of the flowchart of FIG. 4 described in the first embodiment, and the description thereof is omitted. In step S507 of FIG. 12, it is determined whether or not the result of the first calculation (including the phase difference between the A and B images) including the saturated pixel and its neighboring pixels in step S304 is in the vicinity of the integer value. If the result of the first calculation is close to the integer value, it is determined that the AF error due to saturation is acceptable, and the process proceeds to step S311. On the other hand, when the result of the first calculation is not in the vicinity of the integer value, in order to perform the second calculation excluding the saturated pixels and the neighboring pixels, the process proceeds to waveform generation excluding the saturated pixels in step S306.

Here, the determination threshold value based on the first calculation result (phase difference) in step S304 will be described with reference to FIG. FIG. 13 is a diagram illustrating the AF error of the first calculation result. The horizontal axis represents the first calculation result, and the vertical axis represents the actual phase difference between the A and B images. When there is no error, the relationship between the first calculation result and the phase difference between the A and B images is a straight line with a slope of 1. However, in practice, an AF error occurs in the area of the broken line due to the influence of the saturated pixel. As a feature of AF error generated by saturation, AF error does not occur in 1 bit and 2 bit in which the phase difference is an integer, and a correct calculation result can be obtained. On the other hand, as the phase difference approaches 0.5 bits, the AF error increases. As the determination value here, the AF error range is set to 1/4 Fδ which is the focus detection focus range. When the AF error is converted into a defocus amount and within 1 / 4Fδ, the first calculation result is selected as it is. On the other hand, if the AF error is larger than 1 / 4Fδ, the second calculation result described above is used.

The principle that the magnitude of the AF error changes depending on the phase difference will be described with reference to FIGS. FIG. 14 shows four shift waveforms generated when correlation calculation is performed on an image signal in which the phase difference between the A image and the B image is −0.5 bit. The horizontal axis indicates the pixel position, and the vertical axis indicates the level of the image signal. In FIG. 14, the waveform shift amounts are (a): shift amount k = 1, (b): shift amount k = 0, (c): shift amount k = −1, and (d): shift amount k = −. 2 shows four shift waveforms. In each waveform, ● represents a pixel signal of the A image, ○ represents a pixel signal of the B image, and ▲ represents a pixel signal where the A image and the B image overlap. The shift waveform in FIG. 14 is a waveform in which both the A image and the B image do not exceed the saturation level. For each shift waveform, the correlation amount is obtained from equation (1).

In FIG. 15, it is assumed that a part of the pixel signals of the A image and the B image exceeds the saturation level by lowering the saturation level while keeping the image signal of FIG. 14 as it is. Of course, the same phenomenon occurs even if the signal level is raised without changing the saturation level. All pixels having a signal amount exceeding the saturation level are assumed to be saturated pixels, and the signal amount of a saturated pixel having a signal amount higher than the saturation level is clipped to the saturation level. For this reason, the signal amount is reduced from the original signal amount without saturation.

FIG. 16 is a diagram illustrating a correlation amount without saturated pixels and a correlation amount with saturated pixels. In FIG. 16, a value obtained by calculating the correlation amount of the shift waveform of the shift amount −2 from the equation (1) is C (−2), a straight line connecting C (−2) and C (−1), and C ( The phase difference is calculated with the intersection between the straight line connecting 0) and C (1) as the minimum value. It can be seen from FIG. 16 that the phase difference calculated from the shift waveform with saturated pixels is shifted from the phase difference calculated from the shift waveform without saturated pixels. This deviation is an AF error.

As a cause of this AF error, when attention is paid to the 10th pixel and the 12th pixel in FIGS. 14 (a) and 15 (a), there is a clip due to saturation compared to FIG. 14 (a) where there is no clip due to saturation. In 15 (a), the difference in signal amount between the A and B images may be reduced. Since the difference between the signal amounts of the A image and the B image is a correlation amount, the correlation amount of the image signal with saturation is smaller than when there is no saturation. As shown in FIG. 16, when the shift amount k = 0, the amount of decrease in the correlation amount due to saturation is smaller than when k = −2, −1, and 1. As can be seen from FIGS. 14B and 15B, which are shift waveforms with a shift amount of 0 bits, the 10th and 11th pixels are clipped to saturation level due to saturation, but the correlation amount The effect of this effect is small compared to other shift waveforms, and the amount of decrease in correlation due to the influence of saturation differs for each shift waveform in this way. If a minimum value is found in the vicinity of 0.5 bit, which is farthest from the integer bit, interpolation results are most affected by the fluctuation of the correlation amount. As described above, when an image signal having a fractional phase difference of 0.5 bits is calculated, the AF error becomes the largest.

On the other hand, the case where the phase difference is an integer bit will be described with reference to FIGS. FIG. 17 shows four shift waveforms of an image signal in which the phase difference between the A image and the B image is −1 bit. The amplitude and image signal other than the phase difference have the same waveforms as those in FIG. As in FIG. 17, the saturation level is set to a value larger than the maximum value of the image signal so that no saturated pixel is generated. FIG. 18 assumes a case where a part of pixel signals of the A and B images exceeds the saturation level by lowering the saturation level while keeping the image signal of FIG. 17 as it is.

FIG. 19 shows the correlation amount obtained from the shift waveforms in FIGS. 17 and 18. Also in FIG. 19, the minimum value of the correlation amount is obtained in the same manner as in FIG. 16, and the phase difference is calculated. In FIG. 19, which shows the correlation amount of image signals in which the phase difference between the A and B images is an integer, the same phase difference is shown regardless of the presence or absence of saturated pixels, and no AF error occurs. This is because, as can be seen from FIG. 18B, the phase difference between the A image and the B image is an integer, so the A image and the B image overlap regardless of the saturation pixel. If the image signals of the A and B images are completely the same, the correlation amount of the shift waveform in which the A and B images overlap is zero. Since the correlation amount does not take a negative value, the minimum value is a shift amount in which images match. Therefore, as described above, the shift amount with which the image signals match is not related to the saturated pixel, so that a correct phase difference can be calculated regardless of the presence of the saturated pixel.

In actual calculation, linear interpolation is performed from the correlation amount for each shift waveform. As shown in FIGS. 18 (a) and 18 (c), the waveforms of (a) and (c) shifted to the left and right with respect to the waveform of FIG. The statue was just replaced. Therefore, the correlation amounts in the shift waveforms of (a) and (c) are the same value. Therefore, in FIG. 19, when the minimum value is calculated as described above, it has an extreme value with the shift amount k = −1 regardless of the presence or absence of saturated pixels. Based on the above principle, when the phase difference between the A and B images is an integer value, no AF error occurs, and the first calculation result as the phase difference approaches a value between the integers and the fractional part is 0.5 bits. The AF error of increases.

According to the first embodiment, when a saturated pixel exists in the image signal, a saturated pixel excluded waveform is generated from the first calculation result, and the second calculation is performed based on the generated waveform. That is, when performing the second calculation, the correlation calculation needs to be performed twice. Therefore, in the second embodiment, when the phase difference is close to the integer value from the first calculation result, it is determined that the AF error is small (S507), and the second calculation is not performed, and the focus detection is performed from the first calculation result. Do. As a result, the time required for the focus detection process can be shortened and the focus detection can be performed with high accuracy.

(Example 3)
The third embodiment employs a calculation switching determination method that combines the concept of the first embodiment and the concept of the second embodiment. Since the configuration (FIG. 1) and the shooting operation (FIG. 3) of the camera 100 in the third embodiment are the same as those in the first embodiment, description thereof is omitted. FIG. 20 is a flowchart showing focus detection processing in the present embodiment. In the flowchart of FIG. 20, steps other than steps S601 to S607 of the calculation switching determination unit are the same step numbers as those in the flowchart of FIG. 4, and description thereof is omitted.

In step S601 in FIG. 20, it is determined whether or not the result of the first calculation (phase difference calculation based on an image signal including a saturated pixel) calculated before is an integer value. The determination here is the same as that described in the second embodiment. If the result of the first calculation is near the integer value and the AF error is within 1 / 4Fδ, the process proceeds to step S602. On the other hand, if the result of the first calculation is not near the integer value and the AF error is greater than 1 / 4Fδ, the process proceeds to step S605. In step S602, the same waveform generation as that in step S306 of the first embodiment is performed except for the saturated pixels. Saturated pixel exclusion waveform generation generates a waveform excluding saturated pixels from the four shift waveforms used in the first interpolation operation and the saturated pixel information detected in step S305. Here, the saturated pixel and its neighboring pixels are excluded from the four shift waveforms.

In step S603, it is determined whether or not the number of calculation pixels of the excluded waveform generated in step S602 is equal to or less than threshold value 1 (first threshold value). When the number of calculation pixels is larger than the threshold value 1, it is determined that the second calculation is valid in the determination based on the number of calculation pixels, and the process proceeds to determination based on contrast in step S604. On the other hand, when the number of calculation pixels is equal to or less than the threshold value 1, it is determined that the second calculation is not effective, and the process proceeds to the focus determination in step S310 using the result of the first calculation. The method for determining the threshold value 1 will be described later.

In step S604, it is determined whether or not the contrast information of the excluded waveform is equal to or less than threshold value 2 (second threshold value). As described in the first embodiment, the contrast information is obtained from the equation (2). If the contrast information is greater than the threshold value 2, it is determined that the second calculation is valid, and the process proceeds to the second calculation in step S309. On the other hand, if the contrast information is less than or equal to the threshold value 2, it is determined that the second calculation is not valid, and the process proceeds to the focus determination in step S310 using the result of the first calculation.

Here, the threshold value determination method in steps S603 and S604 will be described. From the determination of the phase difference in step S601, the AF error of the first calculation result is within 1 / 4Fδ from FIG. Therefore, in order to perform calculation with higher accuracy, in the second calculation, determination is performed by setting the number of calculation pixels and a threshold value of contrast information so that the AF error is within 1/8 Fδ. If both the number of calculation pixels and the contrast information are larger than the determination threshold and it is determined that the AF error of the second calculation result is within 1/8 Fδ, the second calculation is selected. On the other hand, if either the number of calculated pixels or the contrast information is equal to or smaller than the determination threshold value and the AF error in the second calculation is greater than 1 / 8Fδ, the first calculation is selected. Here, when the phase difference detected by the first calculation unit is in the vicinity of the integer value, the determination unit can change at least one threshold among the determination items of the number of calculation pixels and the contrast information. Thereby, the target within 1/8 Fδ of the AF error can be changed.

Next, a case will be described in which it is determined in step S601 that the result of the first calculation is not near the integer value and the AF error is greater than 1 / 4Fδ. In step S605, similarly to step S602, a saturated pixel excluded waveform is generated to exclude the saturated pixel and its neighboring pixels. In step S606, it is determined whether or not the number of calculation pixels of the excluded waveform generated in step S605 is equal to or less than threshold value 3 (third threshold value). When the number of calculation pixels is larger than the threshold 3, it is determined that the second calculation is valid in the determination based on the number of calculation pixels, and the process proceeds to determination based on contrast in step S607. On the other hand, when the number of calculation pixels is equal to or less than the threshold value 3, it is determined that the second calculation is not effective, and the process proceeds to step S303. In step S303, the focus detection is not performed and the user is warned that focus detection is impossible. A method for determining the threshold here will also be described later.

In step S607, it is determined whether or not the contrast information of the excluded waveform is equal to or less than threshold value 4 (fourth threshold value). If the contrast information is greater than the threshold value 4, it is determined that the second calculation is valid, and the process proceeds to the second calculation in step S309. On the other hand, if the contrast information is less than or equal to the threshold value 4, it is determined that the second calculation is not valid, and the process proceeds to step S303. In step S303, the user is warned that focus detection is not possible without performing focus detection.

Here, the threshold value determination method in steps S606 and S607 will be described. From the determination of the phase difference in step S601, the AF error of the first calculation result is larger than 1 / 4Fδ from FIG. Therefore, in the second calculation, the number of calculation pixels and the threshold value of contrast information are set so that the AF error is within 1 / 4Fδ, and determination is performed. When both the number of calculation pixels and the contrast information exceed the determination threshold and it is determined that the AF error of the second calculation result is within 1 / 4Fδ, the second calculation is selected. On the other hand, when it is determined that either the number of calculated pixels or the contrast information is equal to or less than the determination threshold and the AF error in the second calculation is greater than 1 / 4Fδ, focus detection is not performed.

According to the third embodiment, when a saturated pixel exists in the image signal, the AF error of the first calculation is determined based on the phase difference of the first calculation result (S601). If the determined AF error of the first calculation is within 1 / 4Fδ, it is determined whether higher-precision calculation can be performed in the second calculation based on the reliability determination thresholds 1 and 2 ( S603, S604). On the other hand, when the determined AF error of the first calculation is larger than 1 / 4Fδ and focus detection cannot be performed, the calculation is performed with accuracy within AF error 1 / 4Fδ by the second calculation by the reliability determination thresholds 3 and 4. It is determined whether or not it is possible (S606, S607). This determination makes it possible to determine whether or not focus detection by the second calculation is possible. As a result of the above determination, it is possible to select an optimal calculation corresponding to various subject situations and perform focus detection with high accuracy.

Example 4
In Example 4, focus detection processing in an imaging apparatus in which focus detection pixels are discretely arranged on an imaging element will be described. In the first embodiment, the main mirror 12 is disposed at the down position, and the sub-mirror 13 causes the light beam to enter the focus detection unit 7. The light beam to the image sensor 101 is generated by a so-called secondary optical system. The focus detection of the phase difference detection method was performed. However, the present invention is not limited to this, and some pixels of the image sensor 101 are diverted for focus detection, and a mask that divides pupils on the imaging surface is arranged at the focus detection pixels, thereby detecting a phase difference on the imaging surface. Focus detection may be performed. That is, in Example 4, the light receiving means is an image sensor in which photoelectric conversion elements of pixels are two-dimensionally arranged, and at least a part of the image sensor is used as a focus detection pixel, and an image signal from the focus detection pixel is received. The focus state is detected by detecting the phase difference based on the base.

The focus detection pixel includes, for example, a plurality of photoelectric conversion units, and is configured such that light beams that have passed through different regions on the pupil of the photographing lens are guided to different photoelectric conversion units, respectively. Optical images (A image and B image) generated by the light fluxes that have passed through different pupil regions are acquired by the image signal obtained by the photoelectric conversion unit included in each focus detection pixel. Then, the relative phase difference between the A image and the B image is calculated using the correlation calculation formula. On the other hand, the imaging pixel is, for example, a pixel having one photoelectric conversion unit in the pixel. In such a case, it is possible to perform focus detection using a signal acquired from each focus detection pixel and generate an image using a signal acquired from the imaging pixel. In addition, an image signal of a focus detection pixel that is close may be generated using a signal acquired by the imaging pixel. With such a configuration, the ratio of focus detection pixels in the image sensor can be reduced, and a high-quality image can be captured simultaneously with focus detection.

FIG. 21 to FIG. 23 are diagrams for explaining the structures of the imaging pixels and the focus detection pixels. In the fourth embodiment, out of 4 pixels of 2 rows × 2 columns, pixels having G (green) spectral sensitivity are arranged in the diagonal 2 pixels, and R (red) and B (blue) are arranged in the other 2 pixels. A Bayer arrangement in which one pixel each having a spectral sensitivity of 1 is arranged is employed. In addition, focus detection pixels having a structure to be described later are distributed and arranged in a predetermined rule between the Bayer arrays. FIG. 21 shows the arrangement and structure of the imaging pixels. FIG. 2A is a plan view of 2 × 2 imaging pixels. In the Bayer array, G pixels are arranged diagonally, and R and B pixels are arranged in the other two pixels. This 2-row × 2-column structure is repeatedly arranged. A cross section AA in FIG. 4A is shown in FIG. ML denotes an on-chip microlens arranged in front of each pixel, the CF _R color filters R, is CF _G is a color filter of G. PD schematically shows a photoelectric conversion unit of the C-MOS sensor, and CL is a wiring layer for forming signal lines for transmitting various signals in the C-MOS sensor. TL schematically represents the photographing optical system, and EP represents the exit pupil. The on-chip microlens ML and the photoelectric conversion unit PD of the imaging pixel are configured to capture the light beam that has passed through the photographing optical system TL as effectively as possible. In other words, the exit pupil EP of the photographing optical system TL and the photoelectric conversion unit PD are conjugated with each other by the microlens ML, and the effective area of the photoelectric conversion unit is designed to be large. In FIG. 5B, the incident light beam of the R pixel has been described, but the G pixel and the B pixel have the same structure. Accordingly, the exit pupil EP corresponding to each pixel for imaging RGB has a large diameter, and the S / N of the image signal is improved by efficiently capturing the light flux from the subject.

FIG. 22 shows the arrangement and structure of focus detection pixels for performing pupil division in the horizontal direction (lateral direction) of the photographing optical system. Here, the definition of the horizontal direction or the horizontal direction indicates a direction along a straight line that is orthogonal to the optical axis and extends in the horizontal direction when the camera is held so that the optical axis of the photographing optical system is horizontal. FIG. 5A is a plan view of 2 × 2 pixels including focus detection pixels. When obtaining an image signal for recording or viewing, the main component of luminance information is acquired with G pixels. Since human image recognition characteristics are sensitive to luminance information, image quality degradation is easily recognized when G pixels are lost. On the other hand, the R or B pixel is a pixel that acquires color information (color difference information). However, since human visual characteristics are insensitive to color information, the pixel that acquires color information has some defects. However, image quality degradation is difficult to recognize. Therefore, in this embodiment, among the pixels of 2 rows × 2 columns, the G pixel is left as an imaging pixel, and the R and B pixels are replaced with focus detection pixels. This is indicated by S _HA and S _HB in FIG. A cross section AA in FIG. 4A is shown in FIG. The microlens ML and the photoelectric conversion unit PD have the same structure as the imaging pixel shown in FIG. In the present embodiment, since the signal from the focus detection pixel is not used for image creation, transparent film CF _W instead of a color separation color filter is disposed. Moreover, since pupil division is performed by the image sensor, the opening of the wiring layer CL is biased in one direction with respect to the center line of the microlens ML. Specifically, since the opening OP _HA of the pixel S _HA and the opening OP _HA are biased to the right side, the light beam that has passed through the left exit pupil EP _HA of the photographing optical system TL is received. Similarly, since the opening OP _HB of the pixel S _HB is biased to the left side, the light beam that has passed through the right exit pupil EP _HB of the imaging optical system TL is received. Therefore, the pixels _SHA are regularly arranged in the horizontal direction, and the subject image acquired by these pixel groups is defined as an A image. Further, the pixels SH _B are also regularly arranged in the horizontal direction, the subject image acquired by these pixel groups is set as a B image, and the defocus amount can be detected by detecting the phase difference between the A image and the B image. In the pixels S _HA and S _HB , focus detection is possible for an object having a luminance distribution in the horizontal direction of the photographing screen, for example, a vertical line, but focus detection is not possible for a horizontal line having a luminance distribution in the vertical direction. . Therefore, in the present embodiment, pixels that perform pupil division are also provided in the vertical direction (longitudinal direction) of the photographing optical system so that the focus can be detected for the latter.

FIG. 23 and FIG. 24 are diagrams for explaining the arrangement rules of the imaging pixels and focus detection pixels described in FIG. 21 and FIG. FIG. 23 is a diagram for explaining a minimum unit arrangement rule when focus detection pixels are discretely arranged between imaging pixels. In the figure, 10 rows × 10 columns = 100 pixels are defined as one block. In the upper left block BLK (1, 1), the lower left R pixel and B pixel are replaced with a set of focus detection pixels S _HA and S _HB that perform pupil division in the horizontal direction. In the block BLK (1, 2) on the right side, the lower left R pixel and B pixel are similarly replaced with a set of focus detection pixels S _VC and S _VD that perform pupil division in the vertical direction. The pixel arrangement of the block BLK (2, 1) adjacent below the first block BLK (1, 1) is the same as that of the block BLK (1, 2). The pixel arrangement of the block BLK (2, 2) on the right side is the same as that of the leading block BLK (1, 1). Expressing this arrangement rule universally, in block BLK (i, j), if i + j is an even number, horizontal pupil division focus detection pixels are arranged, and if i + j is an odd number, vertical pupil division focus detection pixels are arranged. Will be placed. Then, a 2 × 2 = 4 block of FIG. 23, that is, an area of 20 rows × 20 columns = 400 pixels is defined as a cluster as an upper array unit of the block.

In this way, arrangement is performed in units of clusters, and arrangement is performed in units of fields in which clusters are gathered as units. FIG. 24 is a diagram for explaining a pixel grouping method in the case of performing focus detection in the lateral shift direction of the subject image formed by the photographing optical system. The focus detection in the lateral shift direction refers to performing a phase difference calculation using focus detection pixels for dividing the exit pupil of the photographing optical system in the horizontal direction. The pixel array shown in FIG. 24 shows the arrangement of clusters in which blocks are gathered. At the time of focus detection, a total of 10 blocks of 1 block in the horizontal direction and 10 blocks in the vertical direction are grouped into one group. In this embodiment, one focus detection area is configured by 30 sections arranged in the horizontal direction. That is, an area of 100 rows × 300 columns = 30,000 pixels is one focus detection area. This one focus detection area is defined as an AF area. Here, in one section, five pixels S _HA that perform one pupil division in the horizontal direction and five pixels S _HB that perform the other pupil division are included. Therefore, in this embodiment, the outputs of the five _SHAs are added to form one AF pixel of one image signal (A image) for phase difference calculation. Similarly, the outputs of the five _SHBs are added to form 1AF pixel of the other image signal (B image) for phase difference calculation. The principle of the pixel grouping method in the case of performing focus detection in the direction of longitudinal shift of the subject image formed by the photographing optical system is the same.

FIGS. 25 to 27 are flowcharts for explaining the photographing operation and the focus detection process according to the present embodiment. FIG. 25 is a flowchart of the photographing operation of the camera of this embodiment. When the photographer turns on the power switch of the camera, the target exposure value is set to an appropriate exposure value in step S701. In addition, the memory contents and the execution program are initialized and the shooting preparation operation is executed. In step S702, the imaging operation of the image sensor is started. Here, the exposure time is controlled so that the target exposure value set in step S701 or step S705 described later is obtained, and a low-pixel moving image for preview is output. In step S703, the read moving image is displayed on the display device 17 provided on the back of the camera, and the photographer visually determines the composition at the time of photographing by viewing the preview image. In step S704, it is determined whether or not SW1_18 has been turned on. If not turned on, the process returns to step S702. Then, the preview image display in step S703 is repeatedly executed from the image sensor driving. When SW1_18 is turned on in step S704, the process proceeds to step S705, and focus detection processing is executed. The focus detection process performed in step S705 will be described with reference to the flowchart of FIG.

FIG. 27 is a flowchart showing focus detection processing in the present embodiment. In step S901, the MPU 6 outputs a signal accumulation instruction to the 101 C-MOS sensor. Based on the instruction, the C-MOS sensor starts a signal accumulation operation. When the accumulation operation ends, the focus detection pixels are read out. In step S902, it is determined whether or not the contrast information obtained from the image signal output from the focus detection pixel in step S901 is equal to or less than a predetermined amount. If the contrast information is less than or equal to the predetermined amount, the process moves to step S916, and the user is warned that focus detection is not possible without performing focus detection. On the other hand, if the contrast information is larger than the predetermined amount, the process proceeds to the first calculation in step S903. Here, as in the first embodiment, the contrast information is the sum of values obtained by squaring the difference between the pixel signals of the AF pixels adjacent to the AF pixel in the calculation pixel range of the image signal. When the contrast information is small, the correlation amount is small when detecting the phase difference. Therefore, it is easily affected by noise components generated in the pixels, and the accuracy of focus detection deteriorates. For the above reasons, when the contrast information is small and the reliability of focus detection is not sufficient, focus detection is not performed. In step S903, the MPU 6 performs the first calculation similar to that in the first embodiment based on the A and B images. A shifted waveform of the image signal acquired from the AF pixel of the C-MOS sensor is defined as a shift waveform. Using the correlation calculation formula (1), the correlation amount C (k) is calculated from each shift waveform, and the phase difference of the image signal is calculated.

Next, in step S904, the MPU 6 detects abnormal pixels (saturated pixels, defective pixels) in the pair of image signals acquired in step S901. Here, a saturated pixel is detected as an abnormal pixel. In the fourth embodiment, the AF pixel is configured to add the outputs of a plurality of focus detection pixels. The saturated pixel defined here is treated as a saturated pixel when at least one saturated pixel is included in the focus detection pixels (SCTV / SCTH) included in the AF pixel unit. Next, it is determined whether or not a saturated pixel exists based on the detected information. If there is no saturated pixel, the process proceeds to focus determination in step S913. On the other hand, if there is a saturated pixel, the process proceeds to determination in step S905 based on the phase difference between the A and B images. The determination based on the phase difference in step S905 is as described in the second and third embodiments, and it is determined whether or not the result of the first calculation calculated earlier is in the vicinity of the integer value. In this determination, if the result of the first calculation is in the vicinity of the integer value and the AF error is within 1 / 4Fδ, the process proceeds to step S906. On the other hand, if the result of the first calculation is not near the integer value and the AF error is larger than 1 / 4Fδ, the process proceeds to step S909. In step S906, a saturated pixel excluded waveform generation similar to step S306 in the first embodiment is performed. The saturated pixel exclusion waveform generation generates a saturated pixel exclusion waveform from the four shift waveforms used in the first interpolation operation and the saturated pixel information detected in step S904. Here, a saturated pixel signal and signals of neighboring pixels are excluded from the four shift waveforms.

In step S907, it is determined whether or not the number of calculation pixels of the excluded waveform is equal to or less than the threshold value 1. When the number of calculation pixels is larger than the threshold 1, it is determined that the second calculation described in the first embodiment is effective in the determination based on the number of calculation pixels, and the process proceeds to the determination based on the contrast in step S908. On the other hand, when the number of calculation pixels is equal to or less than the threshold value 1, it is determined that the second calculation is not effective, and the process proceeds to the focus determination in step S913 using the result of the first calculation. In step S908, it is determined whether or not the contrast information of the excluded waveform is equal to or less than the threshold value 2. As described in the first embodiment, the contrast information is obtained from the above formula (2). If the contrast information is larger than the threshold value 2, it is determined that the second calculation is valid, and the process proceeds to the second calculation in step S912. On the other hand, if the contrast information is less than or equal to the threshold value 2, it is determined that the second calculation is not effective, and the process proceeds to the focus determination in step S913 using the result of the first calculation. Here, the threshold value determination method in steps S907 and S908 is the same as steps S603 and S604 in the third embodiment. In step S909, similarly to step S906, a saturated pixel excluded waveform is generated to exclude the saturated pixel and its neighboring pixels.

In step S910, it is determined whether or not the number of calculation pixels of the excluded waveform is equal to or less than the threshold value 3. If the number of calculation pixels is greater than the threshold 3, it is determined that the second calculation is valid in the determination based on the number of calculation pixels, and the process proceeds to determination based on contrast in step S911. On the other hand, when the number of calculation pixels is equal to or less than the threshold 3, it is determined that the second calculation is not effective, and the process proceeds to step S915. In step S915, focus detection is not performed, and in step S702 in FIG. 25, the target exposure value related to driving of the image sensor is changed to under by 1 EV. In step S911, it is determined whether or not the contrast information of the excluded waveform is a threshold value 4 or less. If the contrast information is larger than the threshold value 4, it is determined that the second calculation is valid, and the process proceeds to the second calculation in step S912. On the other hand, if the contrast information is equal to or less than the threshold value 4, it is determined that the second calculation is not valid, and the process proceeds to step S915. If it is determined in step S908 or S911 that the contrast information is greater than or equal to the threshold value determined in each case, in step S912, a second calculation is performed to calculate from the excluded waveform excluding the saturated pixel and its neighboring pixels. In step S913, it is determined whether or not the phase difference calculated in step S903 or step S912 is within a predetermined range. In step S914, the lens drive amount is calculated based on the phase difference information, lens drive control is performed, the process returns to step S901, and the above-described operation is repeated until the in-focus state is achieved. Here, the threshold value determination method in steps S910 and S911 is the same as steps S606 and S607 in the third embodiment. In step S915, the target exposure value is set under 1 EV. Therefore, the saturation amount of the focus detection pixels can be improved in the next focus detection process.

Returning to the description of the shooting operation flow of FIG. In step S706, it is determined whether or not the target exposure value has been changed. If the target exposure value has not been changed, the process proceeds to the determination of SW2_19 in step S707. On the other hand, when the target exposure value is changed, the image sensor is driven based on the changed target exposure value. In step S707, it is determined whether or not the SW2_19 is turned on. If the SW2_19 is not turned on, the process returns to the image sensor driving in step S702. On the other hand, when SW2_19 is turned on in step S708, the process proceeds to step S708, and an imaging subroutine is executed.

FIG. 26 is a flowchart of the photographing subroutine. When the shooting start switch is operated, it is determined in step S801 whether the exposure value is appropriate. If the target exposure value is changed in step S705 of the main flow and the preview image is output with an exposure value different from the appropriate exposure value set in step S701, the process proceeds to exposure value change in step S802. On the other hand, if it is output with a proper exposure value, the process proceeds to aperture / shutter drive in step S803. In step S802, the target exposure value changed to perform focus detection is changed to an appropriate exposure value to prepare for the operation in step S803. In step S803, the light quantity adjusting diaphragm is driven to perform opening control of the mechanical shutter that defines the exposure time. In step S804, image reading for high-pixel still image shooting, that is, reading of all pixels is performed. In step S805, defective pixel interpolation of the read image signal is performed. That is, the output of the focus detection pixel does not have RGB color information for imaging, and corresponds to a defective pixel in obtaining an image. Therefore, an image signal is created by interpolation from information on surrounding imaging pixels. To do. In step S806, image processing such as γ correction and edge enhancement of the image is performed, and in step S807, the captured image is recorded in the flash memory 23. In step S808, the photographed image is displayed on the display device 17, and the process returns to the photographing operation flow of FIG. Returning to the shooting operation flow of FIG. 25, the series of shooting operations is terminated.

As described above, since the exposure value is determined from the entire imaging surface at the time of live view shooting, if a bright subject exists in a part of the focus detection range, the focus detection pixels are saturated, and focus detection is performed. Accuracy will deteriorate. On the other hand, if the target exposure value is darkened so that saturated pixels do not occur in the focus detection range, the preview screen becomes dark, and the photographer becomes an obstacle in determining the composition. Therefore, even if a saturated pixel occurs, the exposure value is not changed as much as possible, and the focus detection process is performed by selecting an optimal calculation from the phase difference, the number of calculation pixels, and the contrast information as in the third embodiment (S903-S3). S915).

(Other embodiments)
The object of the present invention can also be achieved by the following embodiments. That is, a storage medium storing software program codes for realizing the functions of the above-described embodiments (functions of control means such as the camera side MPU) is supplied to the focus detection apparatus. Then, the computer (or CPU, MPU, etc.) of the camera controller reads the program code stored in the storage medium and executes the above function. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and stores a program for performing focus detection based on parallax image signals acquired from different viewpoints with respect to the subject. This storage medium constitutes the present invention. Specifically, a focus detection method that performs focus detection by a phase difference detection method using a sensor composed of at least one pair of a plurality of pixels includes the following steps. An output step of receiving and outputting an image signal of a pair of subjects formed by a light beam that has passed through a pair of pupil regions of the photographing optical system. An abnormal pixel detection step of detecting an abnormal pixel that generates an abnormal signal included in a pair of pixel signal sequences formed by the image signal output in the output step. First Starring Sanz step for detecting a phase difference between a pair of pixel signal sequence including the signal according to the detected abnormal pixel by said abnormal pixel detecting step. A second calculation step of detecting a phase difference between a pair of pixel signal sequences excluding signals due to abnormal pixels. A determination step of selecting one of the results of the first calculation step and the second calculation step based on a predetermined determination criterion. A focus detection step of detecting a focus state based on the result selected in the determination step;

The above-described focus detection apparatus and method according to the present invention can be used for an imaging apparatus such as a digital camera that requires a focus detection apparatus. Although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

1 ·· Lens side MPU, 6 ·· Camera side MPU, 7 ·· Focus detection unit (focus detection device), 20 ·· Lensing lens unit, 30 ·· Camera body, 100 ·· Camera (imaging device), 101 · · Image sensor (light receiving means) 107 · · Focus detection sensor (light receiving means) 111 · · Abnormal pixel detection unit (abnormal pixel detection unit) 112 · · Correlation calculation unit (first calculation unit, second calculation unit) , 113 .. Operation switching determination unit (determination means)

Claims (12)

A light receiving means including a plurality of pixels that receive and output an image signal of a pair of subjects formed by a light beam that has passed through a pair of pupil regions of the photographing optical system;
An abnormal pixel detecting means for detecting an abnormal pixel that generates an abnormal signal included in a pair of pixel signal sequences formed by an image signal output from the light receiving means;
First calculation means for detecting a phase difference between a pair of pixel signal sequences including a signal from an abnormal pixel detected by the abnormal pixel detection means;
Second computing means for detecting a phase difference between a pair of pixel signal sequences excluding signals due to abnormal pixels;
A determination means for selecting one of the results from the first calculation means and the second calculation means;
With
A focus detection apparatus for detecting a focus state based on a result selected by the determination means. 2. The focus detection apparatus according to claim 1, wherein the second calculation unit sets an abnormal pixel and a normal pixel in the vicinity thereof as an exclusion target pixel. The determination means includes
2. One of the results from the first calculation means and the second calculation means is selected from information obtained from the pair of pixel signal sequences output from the light receiving means. Or the focus detection apparatus of 2. The determination means includes
A range obtained by excluding the signal of the exclusion target pixel from the pair of pixel signal sequences is a calculation pixel range, and when the number of pixels included in the calculation pixel range is equal to or less than a predetermined threshold, the result from the first calculation unit is The focus detection apparatus according to claim 3, wherein the focus detection apparatus is selected. The determination means includes
A range obtained by excluding the signal of the exclusion target pixel from the pair of pixel signal sequences is set as a calculation pixel range, and when the contrast information of the calculation pixel range is equal to or less than a predetermined threshold, the result from the first calculation means is selected. The focus detection device according to claim 3, wherein The first calculation means and the second calculation means are:
The phase difference between the pair of pixel signal sequences of the image signal is detected in units smaller than one pixel by relatively shifting the pair of pixel signal sequences one pixel at a time and performing interpolation calculation from the correlation amounts at a plurality of shift positions. Is possible,
The determination means includes
6. The focus detection device according to claim 3, wherein when the phase difference detected by the first calculation means is in the vicinity of an integer value, the result from the first calculation means is selected. . The determination means includes
7. The focus detection according to claim 6, wherein when the phase difference detected by the first calculation means is in the vicinity of an integer value, at least one threshold is changed among the determination items of the number of calculation pixels and contrast information. apparatus. The light receiving means is an image sensor in which photoelectric conversion elements are two-dimensionally arranged,
At least a part of the image sensor is a focus detection pixel,
8. The focus detection apparatus according to claim 1, wherein the focus state is detected by detecting the phase difference based on an image signal from the focus detection pixel. 9. The determination means includes
9. The focus according to claim 8, wherein when an abnormal pixel is detected by the abnormal pixel detection unit, a target exposure value in the image sensor is changed according to a determination result of either the number of calculated pixels or contrast information. Detection device. An imaging device for imaging a subject,
An imaging apparatus comprising the focus detection apparatus according to claim 1. A focus detection method for performing focus detection by a phase difference detection method using a sensor composed of at least a pair of pixels,
An output step of receiving and outputting an image signal of a pair of subjects formed by a light beam that has passed through a pair of pupil regions of the photographing optical system;
An abnormal pixel detecting step for detecting an abnormal pixel that generates an abnormal signal included in a pair of pixel signal sequences formed by the image signal output in the output step;
A first Starring Sanz step for detecting a phase difference between a pair of pixel signal sequence including the signal according to the detected abnormal pixel by said abnormal pixel detecting step,
A second calculation step of detecting a phase difference between a pair of pixel signal sequences excluding signals due to abnormal pixels;
A determination step of selecting one of the results of the first calculation step and the second calculation step;
A focus detection step of detecting a focus state based on the result selected in the determination step;
A focus detection method characterized by comprising: A program for performing focus detection based on parallax image signals acquired from different viewpoints with respect to a subject,
A program for causing a computer to execute the focus detection method according to claim 11.

Images