JP2017158039A - Imaging apparatus and control method of the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of improving S/N without lowering focus detection accuracy.SOLUTION: An imaging apparatus 100 includes: a plurality of unit pixel cells 200 each having a plurality of divided pixel portions; and an imaging device capable of outputting a parallax image signal. The imaging apparatus 100 has a second synthesis mode in which signals from the divided pixel portions having the same positional relationship are combined in the plurality of pixel portions to generate a second parallax image signal, and a first synthesis mode in which signals from the divided pixel portions in the pixel portions are combined to generate a first parallax image signal.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging apparatus and a control method thereof.

  Patent Document 1 discloses a phase difference method using an imaging element that two-dimensionally arranges microlenses and includes a plurality of photoelectric conversion units (divided pixel units) so as to divide an exit pupil under each microlens. An imaging device that performs focus detection is disclosed. This imaging device realizes vertical pupil division (vertical eyes) and horizontal pupil division (horizontal eyes) by adding the divided pixel portions. Further, Patent Document 2 discloses an imaging device that improves S / N by adding a plurality of pixels to an image obtained by pupil division.

Japanese Patent No. 5157128 International Publication No. 2011/136031

  When the S / N is improved by addition between the divided pixel portions under the microlens as in the imaging device disclosed in Patent Document 1, a signal with a shallow depth of field is generated with respect to the signal before addition. Become. Therefore, the generated signal is not suitable for a signal used for focus detection related to an image region having a large defocus amount. This is because the edge component of the subject is mixed with the low frequency component due to blur, and it is difficult to extract with a linear filter such as a bandpass filter, so even if correlation calculation is performed, extreme values cannot be obtained. is there.

  Moreover, in the imaging apparatus disclosed in Patent Document 2, the spatial frequency of the image that is divided into pupils is lowered, so that the focus detection accuracy is lowered. An object of the present invention is to provide an imaging apparatus capable of improving S / N without reducing focus detection accuracy.

  An imaging device according to an embodiment of the present invention includes an imaging element that includes a plurality of pixel units each having a plurality of divided pixel units and that can output a parallax image signal, and divided pixels that have the same positional relationship in the plurality of pixel units. A second parallax image signal that generates a second parallax image signal by combining signals from the unit and a signal from the divided pixel unit in the pixel unit are combined to generate a first parallax image signal And a synthesis means having a first synthesis mode.

  According to the imaging apparatus of the present invention, it is possible to improve the S / N without reducing the focus detection accuracy.

It is a figure which shows the structural example of the imaging device of this embodiment. It is a figure explaining an example of composition of a unit pixel cell. It is a figure explaining the depth of field of the image obtained by the division | segmentation pixel addition in ML. 10 is a flowchart illustrating an example of a defocus amount detection process. 6 is a diagram illustrating a configuration of an image sensor used in Example 2. FIG. It is a figure explaining the correlation calculation which a focus detection part performs. It is a figure explaining the division pixel addition between ML regarding a diagonal line. It is a figure which shows the example of the output value of a diagonal detection filter.

Example 1
FIG. 1 is a diagram illustrating a configuration example of an imaging apparatus according to the present embodiment.
The imaging apparatus 100 includes an imaging optical system 101 to an optical unit drive control unit 110. The imaging optical system 101 receives subject light and guides a light beam to the imaging element 102 via a plurality of lens groups and a diaphragm (not shown). The light beam that has passed through the imaging optical system 101 forms an image on the image sensor 102 to form an optical image. The imaging optical system 101 includes a focus lens. The focus lens is driven in the optical axis direction by a drive control command from the optical unit drive control unit 110 or by manually adjusting a focus ring (not shown) configured in the imaging optical system.

  The image sensor 102 can output a parallax image signal. A plurality of unit pixel cells (pixel portions) are arranged in a two-dimensional matrix on the image sensor 102, and the exposure amount is controlled by a shutter included in the image pickup optical system 101. The image formed by the imaging optical system 101 is photoelectrically converted, and the charges accumulated in the divided pixel portions formed in the unit pixel are sequentially output to the A / D conversion portion 103 at the time of readout control. The imaging device 100 includes a CPU (Central Processing Unit) that functions as a system control unit (not shown). The system control unit controls each processing unit included in the imaging apparatus 100, thereby realizing the function of the imaging apparatus 100.

FIG. 2 is a diagram illustrating an example of the configuration of the unit pixel cell.
Each of the plurality of unit pixel cells 200 has a plurality of divided pixel portions under one microlens 201 that collects incident light. Specifically, the unit pixel cell 200 has at least four (2 × 2) divided pixel portions. The plurality of divided pixel portions are pupil-divided so as to receive light beams respectively passing through different pupil regions of the imaging optical system 101. The light incident through the imaging optical system 101 passes through the microlens 201 and is accumulated in each divided pixel unit. In this example, the unit pixel cell 200 includes R, G, and B color filters. The color filters are arranged on the solid-state imaging device in a two-dimensional Bayer array.

  A two-dimensional image composed only of pixels existing at the same pixel position under each microlens has a parallax with respect to a two-dimensional image composed only of pixels existing at the same pixel position. That is, an image composed of 2A, 3A, and 4A pixels corresponding to 1A in FIG. 2 and an image composed of 2B, 3B, and 4B pixels corresponding to 1B have different parallaxes. From the 2 × 2 divided pixel unit illustrated in FIG. 2, an image signal (parallax image signal) having a total of four parallaxes is output. By performing the correlation calculation processing of these parallax image signals, the amount of parallax can be detected.

  Returning to the description of FIG. The A / D conversion unit 103 converts an analog electric signal output from the image sensor 102 by analog signal processing in an analog signal processing unit (not shown) into a digital electric signal (pixel signal) and outputs the digital electric signal to the capture unit 104. . The analog signal processing unit is a CDS circuit, a nonlinear amplifier circuit, or the like that removes noise on the transmission path.

  The capture unit 104 gives an attribute for determining the relative position of the divided pixel unit with respect to the microlens 201, and outputs a signal to the image composition unit 105 and the divided pixel addition unit 108. The image synthesizing unit 105 adds all the signals of the pixels in the input unit pixel cell to generate an imaging image (main image), and outputs it to the digital signal processing unit 106. Specifically, the image composition unit 105 adds signals from the divided pixel units corresponding to the positions 1A, 1B, 1C, and 1D in FIG.

  The digital signal processing unit 106 performs digital signal processing typified by synchronization processing, gamma processing, and noise reduction processing on an image input in a Bayer array, and outputs the result to the external recording device 107. Digital signal processing such as synchronization processing, gamma processing, and noise reduction is well known and will not be described.

  The system control unit and the divided pixel adding unit 108 function as a combining unit that generates a parallax image signal by combining signals by adding signals from the divided pixel unit. The divided pixel addition unit 108 has a second synthesis mode and a first synthesis mode. The second synthesis mode is a synthesis mode in which a plurality of pixel units combine signals from the divided pixel units that have the same positional relationship to generate a second parallax image signal. The first synthesis mode is a synthesis mode in which signals from the divided pixel units in the pixel unit are synthesized to generate a first parallax image signal. The divided pixel addition unit 108 operates in the second synthesis mode or the first synthesis mode depending on the type of focus detection processing. For example, in the first combination mode, signals from the divided pixel units 1A and 1C are added, and signals from the divided pixel units 1B and 1D are added to generate the horizontal eye. This addition is the first addition, that is, ML (Micro Lens) divided pixel addition. Of course, the intra-ML divided pixel addition may be applied to the vertical eye generation. At the time of generating the vertical eye, for example, the divided pixel units 1A and 1B are added, and the divided pixel units 1C and 1D are added. That is, in this example, adding the signal from the divided pixel unit in the direction orthogonal to the pupil division direction in the pixel unit, that is, in the unit pixel cell 200, is the intra-ML divided pixel addition.

  Further, for example, in the second synthesis mode, signals from the divided pixel units 1A and 3A are added, and signals from the divided pixel units 1B and 3B are added to generate the horizontal eye. This addition is referred to as a second addition, that is, an ML divided pixel addition. Of course, the inter-ML divided pixel addition may be applied to the vertical eye generation. When generating the vertical eye, for example, the signals from the divided pixel units 1A and 2A are added, and the signals from the divided pixel units 1C and 2C are added. That is, in this example, adding the signals from the divided pixel portions at the same position in the direction orthogonal to the pupil division direction between the unit pixel cells 200 is referred to as intra-ML divided pixel addition. The divided pixel addition unit 108 operates in the second synthesis mode or the first synthesis mode, for example, according to the magnitude of the defocus amount detected in the focus detection process.

FIG. 3 is a diagram for explaining the depth of field of an image obtained by adding the divided pixels in ML.
Since the intra-ML divided pixel addition generates an image with a shallow depth of field, there is a feature that a distorted image can be obtained compared to the case of the inter-ML divided pixel addition.

  In FIG. 3, when the allowable circle of confusion is δ and the aperture value of the imaging optical system is F, the depth of field at the aperture value F is ± Fδ. A surface indicated by a dotted line is an image surface on which the image sensor is present. Usually, the pixel pitch of the unit pixel cell is set according to the size of the allowable circle of confusion δ.

  Light rays emitted from the position 300 are imaged on the image sensor through the pupil division region 301 that is divided by 2 × 2 and becomes narrower. At this time, the effective aperture value F01 in the horizontal and vertical directions of the light obtained through the pupil division region 301 becomes dark as F01 = 2F (2 is the number of divisions). The effective depth of field of each parallax image signal is twice as deep as ± 2Fδ, and the focusing range is doubled. An object image focused on each parallax image signal is acquired within the range of effective depth of field ± 2Fδ. That is, the horizontal depth generated by adding the divided pixel portions 1A and 1C and the divided pixel portions 1B and 1D is shallower than the horizontal eye generated only at the positions 1A and 1B. Therefore, since the image obtained by the intra-ML divided pixel addition hardly undergoes a change in the degree of coincidence when the correlation calculation process is performed, the detection accuracy is low for an image having a large defocus amount.

  On the other hand, the image subjected to the ML divided pixel addition remains ± 2Fδ, and a sharp image is obtained. Therefore, the degree of coincidence is likely to change, and the detection accuracy is increased for an image having a large defocus amount.

  In the case of an image sensor with 2 × 2 divisions, the division pixel addition direction in the ML is addition only in the direction orthogonal to the pupil division direction. The effect of depth of field is reduced. However, as the depth of field in the vertical eye direction becomes shallow, an increase in low frequency components occurs, and the range in which defocus detection is possible is narrowed.

  Further, the horizontal and vertical directions as the pixel array of the image sensor and the pupil division direction determined by the center of gravity of the aperture of each divided pixel portion do not necessarily match due to the influence of vignetting. Therefore, even if vertical division of ML divided pixels is performed, the depth of field in the horizontal direction becomes shallow, and the same problem occurs. Furthermore, in the image sensor divided into 4 × 4, when the horizontal eye is generated, when the pupil divided pixel is generated by adding the horizontal 2 pixels and the vertical 4 pixels, there is an effect that the depth of field becomes shallow. growing.

  Since the divided pixel addition between MLs lowers the resolution in the image height direction, even if a focus lens exists in the vicinity of the in-focus state, the detection accuracy of high-frequency components typified by thin lines is reduced. is there. On the other hand, the intra-ML divided pixel addition has a feature that the fine line detection accuracy is improved because the resolution in the image height direction is not lowered.

  In addition, since the ML divided pixel addition performs correlation calculation processing using an image that has passed through a part of the exit pupil, it has a feature that it is easily affected by shading at a portion where the image height is high. Specifically, the incident light decreases in the B and D images as it goes to the left of the image height, and the incident light decreases in the A and C images as it goes to the right. Further, the incident light decreases in the A and B images as it goes upward, and the incident light decreases in the C and D images as it goes downward. In order to solve such a problem, an ML divided pixel addition is performed after selecting a divided pixel portion that is not easily affected by shading as an output source divided pixel portion according to the image height. May be. Further, when performing the ML divided pixel addition, it is desirable to perform the pixel addition in the direction orthogonal to the shift direction when performing the correlation calculation, that is, the direction orthogonal to the pupil division direction.

  From the above-described features, the divided pixel addition unit 108 performs, for example, the inter-ML divided pixel addition or the intra-ML divided pixel addition according to the defocus amount detection mode. Specifically, the divided pixel adding unit 108 selects the divided pixel addition between ML in the detection mode for detecting a large defocus amount, and in the detection mode for detecting the defocus amount near the in-focus state, the divided pixel in the ML. The addition is selected, and an image obtained by the addition is output to the focus detection unit 109. The focus detection unit 109 performs correlation calculation to calculate an image shift amount N between the divided pixel units having a phase difference, and based on a fixed value K that is uniquely determined according to the state of the imaging optical system 101. The focus amount is detected and output to the optical unit drive control unit 110.

ＹＡｎおよびＹＢｎは、水平のマイクロレンズのｎ個の画素数を含んだ数列である。ｉは、各画素位置を表す。焦点検出部１０９が画素をずらしながら差の絶対値を算出するときのずらし量をｍとする。最も値の小さなＣを取るｍの位置が、合焦位置に対応するずれ量Ｎを示す。 FIG. 6 is a diagram for explaining the correlation calculation executed by the focus detection unit.
In FIG. 6, “a” and “b” are codes indicating pseudo pixel outputs generated by the inter-ML divided pixel addition or the intra-ML divided pixel addition. The focus detection unit 109 combines the pixel outputs a and b in the column direction or the row direction, and generates and datas A and B images as the output of the same color unit pixel cell group. Calculate by calculation. The result of the SAD calculation is obtained by equation (1).

YAn and YBn are a sequence of numbers including the number of n pixels of the horizontal microlens. i represents each pixel position. Let m be the shift amount when the focus detection unit 109 calculates the absolute value of the difference while shifting the pixel. The position of m taking the smallest value C indicates the shift amount N corresponding to the in-focus position.

(1) At the time of focusing, the position where the imaging optical system forms an image is the PD under the ML of P7, so that the A image pixel group and the B image pixel group substantially coincide. At this time, the image deviation amount N (1) between the A image pixel group and the B image pixel group obtained by the correlation calculation represents that the image deviation amount N (1) approximates 0.
(2) In the case of the rear pin, the A image pixel is a pixel below the ML of P5 and the B image pixel is a pixel below the ML of P9 as the position where the imaging optical system forms an image. At this time, an image shift amount N (2) between the A image pixel group and the B image pixel group obtained by the correlation calculation occurs.
(3) In the case of the front pin, the A image pixel is a pixel below the ML of P9 and the B image pixel is a pixel below the ML of P5 as a position where an image is formed with the photographing optical system. At this time, the image shift amount N (3) between the A image pixel group and the B image pixel group obtained by the correlation calculation indicates an image shift amount in the direction opposite to the rear pin. This means that at the time of focusing, the A image pixel group and the B image pixel group look at the same subject, but at the rear pin and the front pin, the A image pixel group and the B image pixel group are This means that the subject is shifted by the image shift amount N.

The defocus amount d can be obtained by using a known technique, for example, based on the relationship between the image shift amount N and K that is uniquely determined by the optical state up to the light receiving element, using Equation (2). .
d = N × K (2)

  Note that it is desirable to limit the band of the subject image using a bandpass filter before performing the correlation calculation by the focus detection unit 109. Therefore, the imaging apparatus 100 may include a bandpass filter that limits the band of the parallax image signal output from the divided pixel unit. Specifically, in the case of detecting a small defocus amount, the imaging apparatus 100 executes intra-ML divided pixel addition after applying a bandpass filter used for detecting a high-band image of the subject. In other words, the imaging apparatus 100 performs intra-ML divided pixel addition or inter-ML divided addition according to the band size obtained by band limitation. When the imaging apparatus 100 detects the final in-focus position, the intra-ML divided pixel addition may be executed after applying a bandpass filter used for detecting a high-band image of the subject. The optical unit drive control unit 110 illustrated in FIG. 1 drives the imaging optical system 101 to focus on the subject based on the defocus amount obtained by the focus detection unit 109.

FIG. 4 is a flowchart illustrating an example of a defocus amount detection process performed by the imaging apparatus.
In the following description, a detection mode for detecting a large defocus amount is described as a large defocus detection mode. A detection mode for detecting a small defocus amount is described as a small defocus detection mode. A system control unit (not shown) starts an autofocus operation while SW1 is being pressed. In step S400, the system control unit sets the detection mode to the large defocus detection mode.

  In step S401, the image sensor 102 reads out the A, B, C, and D images output from the divided pixel portions 1A, 1B, 1C, and 1D. Next, in step S402, the system control unit determines whether the detection mode is the large defocus detection mode. If the detection mode is the large defocus detection mode, the process proceeds to step S404. If the detection mode is not the large defocus detection mode (for example, the small defocus detection mode), the process proceeds to step S403.

  In step S404, the divided pixel adding unit 108 performs ML divided pixel addition. Thereby, two images having a phase difference are generated in a pseudo manner. Then, the process proceeds to step S405. Since the image obtained by the ML divided pixel addition has a deep depth of field, there is a feature that the defocus amount detection performance is high even when a large blur is generated.

  In step S403, the divided pixel addition unit 108 performs in-ML divided pixel addition. Then, the process proceeds to step S405. The image obtained by adding the divided pixels in ML has a feature that the detection performance of the high frequency component represented by the thin line is high because the spatial frequency component is not attenuated. Therefore, the intra-ML divided pixel addition is effective when the defocus amount for performing the final focus determination is set to be detected.

  In step S405, the focus detection unit 109 performs a correlation calculation (SAD calculation) on the image generated in step S403 or S404. Subsequently, in step S406, the focus detection unit 109 performs extreme value detection processing. Subsequently, in step S407, the system control unit determines whether an extreme value has been detected. If the extreme value can be detected, the system control unit outputs the defocus amount obtained by multiplying the image shift amount d by K to the optical unit drive control unit 110, and the process proceeds to S409. If the extreme value cannot be detected, the process proceeds to S408.

  In step S408, the system control unit sets the detection mode to the large defocus detection mode. In step S409, the system control unit sets the detection mode to the small defocus detection mode in order to detect the final focus position in the next frame. By changing to the small defocus detection mode in this way, it is possible to suppress the influence of the spatial frequency that is attenuated by the addition of the divided pixels between MLs in the large defocus detection mode.

  Next, in step S410, the system control unit determines whether SW1 is pressed. If SW1 is pressed, the process returns to step S402. If SW1 is not pressed, the autofocus operation is terminated. As described above, in this embodiment, the S / N ratio is reduced without reducing the accuracy of the focus detection process by performing the inter-ML divided pixel addition or the intra-ML divided pixel addition according to the defocus amount detection mode. Can be improved.

  The imaging apparatus may apply the following method. In other words, the imaging apparatus further performs inter-ML divided pixel addition or intra-ML division based on the defocus amount detection mode for an image obtained by performing a certain number of ML divided pixel additions for the purpose of converting the divided pixel portion into a luminance signal. Perform pixel addition. Specifically, an ML divided pixel addition is performed by adding two horizontal pixels and two vertical pixels for the purpose of performing a luminance signal to generate a pupil-divided image, and further, an ML divided pixel addition or ML for the purpose of improving S / N. The inner divided pixel addition is executed. In addition, the inter-ML divided pixel addition and the intra-ML divided pixel addition may be executed exclusively.

  Further, in this embodiment, the ML divided pixel addition is selected in the large defocus amount mode, and the ML divided pixel addition is selected in the small defocus amount mode. However, such control is not necessarily required. Absent. A correlation image obtained by addition of divided pixels between ML and a correlation image obtained by addition of divided pixels within ML is calculated at the same time, and a final focused position is determined using a correlation image obtained by addition of divided pixels between ML for the purpose of detecting a large defocus amount. For the purpose, a correlation image obtained by addition of divided pixels in ML may be used. Further, the correlation image obtained by one addition may be used for the purpose of evaluating the reliability of the other correlation image. Specifically, in the case where a plurality of extreme values exist in the correlation image obtained by the intra-ML divided pixel addition, the poles present in the vicinity of the image shift amount of the extreme value of the correlation image obtained by the inter-ML divided pixel addition. The value is valid.

  Further, the divided pixel addition unit 108 generates a parallax image signal that is further synthesized from the first parallax image signal obtained by the intra-ML divided pixel addition by addition of the divided pixels between ML as the second parallax image signal. May be.

(Example 2)
If all the divided pixel portions are read out from the image sensor 102 and the signals from the divided pixel portions are added by digital processing after A / D conversion, a transfer time and power consumption may be increased. Therefore, in the imaging apparatus according to the second embodiment, the system control unit performs signal synthesis in the second synthesis mode or the first synthesis mode in advance in the imaging device before A / D conversion.

FIG. 5 is a diagram illustrating a configuration of an image sensor used in the second embodiment.
In FIG. 5, PD1A, PD1B, PD1C, PD1D, PD3A, PD3B, PD3C, and PD3D are photodiodes corresponding to the divided pixel portions 1A, 1B, 1C, 1D, 3A, 3B, 3C, and 3D, respectively. The output of the photodiode is transferred onto the floating diffusion (FD) as charges by the read transistors Rd1A, Rd1B, Rd1C, Rd1D, Rd3A, Rd3B, Rd3C, and Rd3D. VDD is a power source of the image sensor, and the electric charge on the FD is cleared by Res. Sel is a row readout transistor, and a pixel value is output via a pixel source follower.

  In order to perform the same processing as the ML divided pixel addition described in the first embodiment, the system control unit resets the pixel value by Res, and then controls the transistors of Rd1A and Rd3A, and PD1A and PD1A on the FD The charge of PD3A is transferred. Next, the system control unit controls Sel and outputs the pixel value to the A / D conversion unit 103. Similarly, the system control unit controls Rd1B and Rd3B, so that ML divided pixel addition in the horizontal direction can be performed.

  On the other hand, when performing intra-ML divided pixel addition, Rd1A and Rd1C are accumulated in the FD and output as pixel values, and then Rd1B and Rd1D are similarly controlled and output. Thus, by sharing the FD in units for performing addition between the divided pixel portions, it is possible to add signals from the divided pixel portions in the image sensor 102.

  In the second embodiment, the system control unit determines whether to perform inter-ML divided pixel addition or intra-ML divided pixel addition, and controls a register that switches the order of transfer of transistors based on the determination result. By controlling in this way, transfer time can be reduced and power can be saved.

(Example 3)
The image pickup apparatus according to the third embodiment includes a diagonal line detection filter as a direction detection unit that detects a direction component of an image. Then, the imaging apparatus 100 operates in the second synthesis mode or the first synthesis mode based on the detected direction component of the image.

FIG. 7 is a diagram for explaining an image generated by ML divided pixel addition with respect to an oblique thin line.
L is a range of an autofocus frame in which correlation calculation is performed. N is an ML divided pixel addition number. P indicates a pixel pitch. θ1 and θ2 indicate the angles of the thin lines. FIG. 7A and FIG. 7C show images obtained by projecting thin lines with different angles onto the pixels. FIG. 7B shows the result of adding the ML divided pixels in the vertical direction to the image shown in FIG. FIG. 7D shows the result of adding the ML divided pixels in the vertical direction to the image shown in FIG.

  When an ML divided pixel addition of vertical N pixels is performed on the image shown in FIG. 7A, a uniform luminance value is obtained in the horizontal direction as shown in FIG. No longer exists. Therefore, even if the correlation calculation is performed based on the addition result of the divided pixel unit, since there is no pole, it is difficult to perform the focus detection process.

On the other hand, if ML-interleaved pixel addition of vertical N pixels is performed on the image shown in FIG. 7C, a pattern exists at the edge in the horizontal direction as shown in FIG. Makes it possible to detect the pole. That is, the thin line of the angle θ shown in the following formula (3) is a thin line that can detect the pole when performing the division addition between ML of N pixels.

  A range indicated by θ is an angle of a thin line capable of detecting a pole when ML divided pixel addition is performed. Therefore, in the second embodiment, the imaging apparatus 100 includes an oblique line detection filter (not shown), and this detection filter is applied to an image before performing the ML divided pixel addition. The imaging apparatus 100 compares the output value of the detection filter with a threshold value, and performs division addition between MLs when the output value is equal to or greater than the threshold value. When the output value is less than the threshold value, the imaging apparatus 100 performs in-ML divided pixel addition. As described above, the imaging apparatus 100 according to the second embodiment performs the inter-ML divided pixel addition or the intra-ML divided pixel addition according to the output of the oblique line detection filter. Of course, it is also possible to select the intra-ML divided pixel addition in accordance with the output of the diagonal line detection filter after selecting the ML divided pixel addition for the purpose of converting the divided pixel portion into a luminance signal.

FIG. 8 is a diagram illustrating an example of output values of the oblique line detection filter.
The diagonal line detection filters shown in FIGS. 8A to 8D are 3 × 3 tap digital filters. The digital filter can detect a diagonal line by applying a coefficient present at each tap position for each pixel. When the pole can be detected for the fine line up to an angle of 22.5 ° by adding the divided pixels between ML, the output value of the digital filter in FIG. 8B and the output value of the digital filter in FIG. Integrate into. If the result of integration is higher than the threshold value, the pole can be detected. As described above, it is possible to obtain a correlation image with a high S / N ratio by adding signals in the synthesis mode corresponding to the detection result of the oblique line by the oblique line detection filter.

  In this embodiment, the image sensor 102 divided into 2 × 2 divided pixel portions has been described. However, the scope of application of the present invention is limited to the image sensor divided into 2 × 2 divided pixel portions. Not. Using the image sensor divided into 4 × 4 divided pixel portions, the angle of the oblique line capable of detecting the pole by adding the divided pixels between MLs is calculated by the method described in the third embodiment, and then the intra-ML division is performed. Pixel addition may also be used together. Thereby, it is possible to ensure the accuracy of the detection angle of the fine line and the defocus amount that can be detected. The present invention can also be applied to a focus detection apparatus that performs intra-ML divided pixel addition or inter-ML divided pixel addition according to the type of focus detection processing.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 100 Imaging device 108 Divided pixel addition part 109 Focus detection part

Claims (15)

An image sensor that includes a plurality of pixel units each having a plurality of divided pixel units and is capable of outputting a parallax image signal;
A second synthesis mode for generating a second parallax image signal by synthesizing signals from the divided pixel units having the same positional relationship in the plurality of pixel units, and from the divided pixel units in the pixel unit An image pickup apparatus comprising: a combining unit having a first combining mode for combining signals and generating a first parallax image signal. The imaging according to claim 1, wherein the synthesizing unit generates a parallax image signal further synthesized from the first parallax image signal in the second synthesis mode as the second parallax image signal. apparatus. The imaging apparatus according to claim 1, further comprising a focus detection unit that performs focus detection using the second parallax image signal. The imaging apparatus according to any one of claims 1 to 3, wherein the synthesizing unit operates in the second synthesis mode or the first synthesis mode according to a type of focus detection processing. . The imaging apparatus according to claim 4, wherein the combining unit operates in the second combining mode or the first combining mode according to a defocus amount detection mode in the focus detection process. . The imaging according to claim 5, wherein the synthesizing unit operates in the second synthesis mode or the first synthesis mode according to a defocus amount detected in the focus detection process. apparatus. The imaging device according to claim 5 or 6, wherein the synthesizing unit operates in the first synthesis mode when detecting a defocus amount in the vicinity of in-focus in the focus detection process. . Comprising limiting means for limiting the band of the subject image used for detecting the defocus amount;
The said synthesis | combination means operate | moves in the said 2nd synthetic | combination mode or the said 1st synthetic | combination mode according to the magnitude | size of the zone | band obtained by the restriction | limiting of the said band. The imaging device according to item. 9. The imaging according to claim 1, wherein the synthesizing unit selects a divided pixel unit that is an output source of a signal to be synthesized in the second synthesis mode according to an image height. apparatus. One microlens for condensing incident light is provided for each of the pixel portions,
The imaging device according to any one of claims 1 to 9, wherein the plurality of divided pixel units are pupil-divided so as to receive light beams respectively passing through different pupil regions of the imaging optical system. . The signal adding direction when the synthesizing unit synthesizes the signals from the divided pixel units having the same positional relationship in the second synthesis mode is a direction orthogonal to the pupil division direction. The imaging apparatus according to claim 10, wherein the imaging apparatus is characterized. 12. The imaging according to claim 1, wherein the synthesizing unit operates in the second synthesis mode or the first synthesis mode in the imaging device to synthesize a signal. apparatus. The imaging device according to any one of claims 1 to 12, wherein each of the pixel units includes at least four or more divided pixel units. A direction detecting means for detecting a direction component of the image;
2. The imaging according to claim 1, wherein the synthesizing unit operates in the second synthesis mode or the first synthesis mode based on a detection result of a direction component of the image by the direction detection unit. apparatus. A method for controlling an imaging apparatus having an imaging device that includes a plurality of pixel units each having a plurality of divided pixel units and is capable of outputting a parallax image signal,
A second synthesis mode for generating a second parallax image signal by synthesizing signals from the divided pixel units having the same positional relationship in the plurality of pixel units, and from the divided pixel units in the pixel unit And a first synthesizing mode for synthesizing signals to generate a first parallax image signal.

Images