JP6467289B2 - Imaging apparatus and control method thereof - Google Patents

Description

  The present invention relates to an imaging apparatus and a control method thereof, and more particularly to an automatic focus detection (AF) technique.

  Although recent imaging apparatuses are equipped with an automatic focus detection (AF) function, there are scenes in which the AF function is not good. Therefore, when a mode for shooting a scene that the AF function is not good at, for example, a starry sky scene, is set, an imaging apparatus that drives the focus lens to a predetermined infinity position without using the AF function. Known (Patent Document 1).

JP-A-4-15629 JP 2014-2197 A

  However, when the focus lens is driven to a preset infinity position, it cannot cope with the shift of the infinity position due to secular change, environmental temperature, or camera posture, so an image with high focus cannot be obtained. There is.

  The present invention has been made in order to improve one or more of the problems of the prior art, and aims to provide an imaging apparatus improved in focus detection accuracy in a scene with low brightness and contrast and a control method thereof. To do.

  The above-described object is to divide the pupil region of the photographing optical system in the pupil division direction, and to acquire an image signal from the image pickup device capable of generating a plurality of image signals based on the light flux that has passed through different pupil partial regions. An acquisition unit; a generation unit configured to generate a pair of focus detection signals based on an image signal; a calculation unit configured to calculate a defocus amount of the photographing optical system based on a correlation amount between the pair of focus detection signals; Driving means for adjusting the focus of the photographic optical system based on the imaging device, wherein the generation means has a resolution at least equal to the spatial resolution of the image sensor in the pupil division direction and a direction different from the pupil division direction In the first focus detection mode for generating a pair of focus detection signals based on the image signals that have not been thinned in, and the generation means, the space of the image sensor in the pupil division direction and different directions A second focus detection mode for generating a focus detection signal of the pair on the basis of an image signal having a lower resolution than resolution is achieved by an imaging apparatus characterized by having a.

  With such a configuration, according to the present invention, it is possible to provide an imaging apparatus with improved focus detection accuracy in a scene with low brightness and contrast, and a control method therefor.

1 is a diagram illustrating an example of a functional configuration of a digital camera as an example of an imaging apparatus according to an embodiment of the present invention. The figure which shows typically the example of arrangement | positioning of the imaging pixel and focus detection pixel in the image pick-up element 107 of FIG. FIG. 2 is a plan view and a cross-sectional view schematically showing the configuration of the image pickup pixel shown in FIG. Schematic explanatory diagram showing the correspondence between the pixel structure shown in FIG. 3 and pupil division (A) is the figure which showed the correspondence of the image pick-up element of embodiment, and pupil division, (b) is the relationship of the image shift | offset | difference amount between a defocus amount, a 1st focus detection signal, and a 2nd focus detection signal in embodiment. Figure showing The flowchart of the focus detection process in 1st Embodiment of this invention. Flowchart for explaining defocus amount calculation processing in an embodiment of the present invention The figure which shows typically the reading of the 1st reading mode in embodiment, and the conversion method to a luminance signal Diagram showing the effect of the difference between the exit pupil distance of the imaging optical system and the set pupil distance of the image sensor on the pupil division for pixels with a large image height The figure which shows the example of the frequency characteristic of the band pass filter applied by the filter process applied in embodiment Flowchart of threshold processing in the second embodiment of the present invention Flowchart of focus detection processing in the third embodiment of the present invention

● (First embodiment)
Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, an example in which the present invention is applied to a digital camera as an example of an imaging apparatus will be described. However, the present invention can be applied to any device or device having an imaging function. Examples of such a device and device include, but are not limited to, a mobile phone, a personal computer, a tablet terminal, and a game machine.

[Digital camera configuration]
FIG. 1 is a diagram schematically illustrating an example of main components related to a focus detection operation in a digital camera 100 according to an embodiment of the present invention. In the focus detection operation, one or more constituent elements of a general digital camera that are not shown in FIG. 1 may be used. The photographing optical system 101 includes a variable power lens for changing the angle of view and a driving mechanism thereof, a focus lens 102 as a focusing optical system, a driving mechanism thereof, a diaphragm, and a driving mechanism thereof. The photographing optical system 101 may be configured to be detachable from the digital camera 100. Incident light from the photographing optical system 101 is converted into an electric signal (image signal) by an image sensor 103 which is a CCD or CMOS image sensor. As will be described later, the image sensor 103 has a configuration capable of performing image plane phase difference AF.

  In the present embodiment, it is assumed that the image sensor 103 is provided with a color filter having a specific repetitive pattern, specifically, a primary color Bayer pattern color filter. Each pixel of the image sensor 103 is assigned a color filter corresponding to one of the color components (red, green, blue) constituting the pattern. Hereinafter, the color of the assigned color filter will be described as the pixel color, and the pixel to which the same color filter is assigned may be referred to as the same color pixel or the same color component pixel.

  The image signal is subjected to A / D conversion and the like and then read out by the signal reading unit 105. The signal reading unit 105 generates a timing signal to be given to the horizontal and vertical reading circuits of the image sensor 103 according to the read mode set by the control unit 110 and reads an image signal from the image sensor 103. The signal reading unit 105 and the control unit 110 constitute acquisition means for acquiring an image signal from the image sensor 103.

  The AF signal generation unit 106 generates first and second focus detection signals that are a pair of parallax signals from the image signal read by the signal reading unit 105 from the image sensor 103. The signal reading unit 105 and the AF signal generation unit 106 constitute generation means for generating a pair of focus detection signals based on the image signal. The correcting unit 107 applies shading correction processing and filter processing caused by the optical characteristics of the photographing optical system 101 to the first and second focus detection signals. The focus detection unit 108 calculates the correlation amount between the first and second focus detection signals, and converts the correlation amount into a defocus amount that represents the amount and direction of focus deviation of the photographing optical system 101.

  The drive control unit 109 drives the focus lens 102 based on the defocus amount calculated by the focus detection unit 108.

  For example, when an instruction to start a shooting preparation operation is input, for example, when the release button is pressed halfway, the AF signal generation unit 106 starts from the image signal read from the pixels in the set focus detection area, for example. And start generating the second focus detection signal. Further, the control unit 110 performs AE processing based on the image signal read from the pixels in the focus detection area.

  The operation unit 104 is an input device group for a user to give settings and instructions to the digital camera 100, and includes a release button, a menu button, a direction key, a determination button, a power switch, a touch panel, and the like. The state and operation of the operation unit 104 are input to the control unit 110, and the control unit 110 performs control to realize an operation according to the state and operation of the operation unit 104.

  In the present embodiment, the operation unit 104 includes a mode dial 1041 for setting a shooting mode. As will be described later, the control unit 110 changes the processing of the signal reading unit 105 and the AF signal generation unit 106 according to the set shooting mode. Further, the digital camera 100 of the present embodiment has a “starry sky shooting mode” as a shooting mode.

  After the driving control of the focus lens 102 is completed, when a shooting operation start instruction is input, such as when the release button is fully pressed, the control unit 110 controls the charge accumulation time and diaphragm operation by the image sensor 103. Perform the shooting operation. The photographed image is recorded on a nonvolatile recording medium such as a memory card, for example, by applying a recording process by the control unit 110.

  The signal reading unit 105, the AF signal generation unit 106, the correction unit 107, the focus detection unit 108, the drive control unit 109, and the control unit 110 are, for example, a programmable processor (hereinafter referred to as a CPU) 120 such as a CPU or MPU. It may be realized by executing software. Alternatively, one or more of the signal readout unit 105, the AF signal generation unit 106, the correction unit 107, the focus detection unit 108, the drive control unit 109, and the control unit 110 are configured by hardware circuits such as ASIC, FPGA, and CPLD. Also good. In the following, it is assumed that at least the control unit 110 that realizes the functions of the digital camera 100 by controlling each unit is realized by the CPU 120 executing software.

[Image sensor]
FIG. 2 is a diagram schematically illustrating an arrangement example of imaging pixels and focus detection pixels in the imaging element 103, and representatively illustrates an area in which the imaging pixels are arranged in the horizontal 4 pixels × vertical 4 pixels. The imaging element 103 is configured to be able to generate a plurality of image signals based on a light beam that has passed through different pupil partial regions by dividing the pupil region of the photographing optical system 101 in the pupil division direction. Specifically, the photoelectric conversion area of each imaging pixel is divided into two in the horizontal direction (pupil division direction), and each photoelectric conversion area functions as a focus detection pixel. Therefore, in FIG. 2, it can be said that the focus detection pixels are regions in which 8 horizontal pixels × 4 vertical pixels are arranged.

  In the present embodiment, the 2 × 2 pixel group 200 in the upper left of FIG. 2 corresponds to a repeating unit of a primary color Bayer array color filter provided in the image sensor 103. Accordingly, the pixel 200R having R (red) spectral sensitivity is arranged at the upper left, the pixel 200G having G (green) spectral sensitivity is arranged at the upper right and lower left, and the pixel 200B having B (blue) spectral sensitivity is arranged at the lower right. Has been. Further, as representatively shown in the upper right pixel in FIG. 2, each imaging pixel has a photoelectric conversion unit equally divided into 2 × 1 in the horizontal direction, and the photoelectric conversion unit on the left half is the first focus. The detection pixel 201 and the right half photoelectric conversion unit can be used as the second focus detection pixel 202. When used as an imaging pixel, a signal obtained by adding signals obtained by two photoelectric conversion units is used as an imaging signal.

By arranging a number of 4 × 4 image pickup pixels (8 × 4 focus detection pixels) shown in FIG. 2 on the image pickup surface of the image sensor 103, focus detection is performed at various positions on the screen while acquiring a picked-up image. It is possible to perform focus detection using the imaging surface phase difference detection method used as the region. In the present embodiment, it is assumed that the pitch (period) P of the imaging pixels is 4 μm both vertically and horizontally, and the number of pixels N is 5575 × vertical 3725 = approximately 20.75 million pixels. Further, the vertical pitch P of the focus detection pixels is the same as that of the imaging pixels, but the horizontal pitch _PAF is 2 μm. Therefore, the focus detection pixel number N _AF is 11150 × horizontal 3725 = about 41.5 million pixels. It shall be.

  FIG. 3A shows a plan view of one imaging pixel (here, 200G) shown in FIG. 2 as viewed from the light receiving surface side (+ z side) of the imaging element, and a- in FIG. FIG. 3B shows a cross-sectional view of the a cross section viewed from the −y side.

As shown in FIG. 3, in the pixel 200G of this embodiment, a microlens 305 for condensing incident light is formed on the light receiving side of each pixel, and is divided into _NH (two divisions) in the x direction and in the y direction. N _V division (first division) by photoelectric conversion unit 301 and the photoelectric conversion portion 302 is formed. The photoelectric conversion unit 301 and the photoelectric conversion unit 302 correspond to the first focus detection pixel 201 and the second focus detection pixel 202, respectively.

  The photoelectric conversion unit 301 and the photoelectric conversion unit 302 may be a pin structure photodiode in which an intrinsic layer is sandwiched between a p-type layer and an n-type layer, or an intrinsic layer is omitted as necessary, and a pn junction is formed. A photodiode may be used.

  In each pixel, a color filter 306 is formed between the microlens 305 and the photoelectric conversion unit 301 and the photoelectric conversion unit 302. Further, as necessary, the spectral transmittance of the color filter may be changed between the first focus detection pixel 201 and the second focus detection pixel 202, or the color filter may be omitted.

  Light incident on the pixel 200 </ b> G illustrated in FIG. 3 is collected by the microlens 305, dispersed by the color filter 306, and then received by the photoelectric conversion unit 301 and the photoelectric conversion unit 302.

  In the photoelectric conversion unit 301 and the photoelectric conversion unit 302, a pair of electrons and holes are generated according to the amount of received light and separated by the depletion layer, and then negatively charged electrons are accumulated in the n-type layer. It is discharged to the outside of the image sensor 103 through the p-type layer 300 connected to the not shown.

  The electrons accumulated in the n-type layers of the photoelectric conversion unit 301 and the photoelectric conversion unit 302 are transferred to the electrostatic capacitance unit (FD) through the transfer gate and converted into a voltage signal.

  FIG. 4 is a schematic explanatory diagram showing the correspondence between the pixel structure of this embodiment shown in FIG. 3 and pupil division. In FIG. 4, in order to correspond to the coordinate axis of the exit pupil plane, the x-axis and y-axis of the cross-sectional view are inverted with respect to FIG.

  In FIG. 4, the first pupil partial region 501 of the first focus detection pixel 201 is substantially conjugate with the light receiving surface of the photoelectric conversion unit 301 whose center of gravity is decentered in the −x direction and the microlens 305. A pupil region that can be received by the first focus detection pixel 201 is shown. The first pupil partial region 501 of the first focus detection pixel 201 has an eccentric center of gravity on the + X side on the pupil plane.

  In FIG. 4, the second pupil partial region 502 of the second focus detection pixel 202 is substantially conjugate with the light receiving surface of the photoelectric conversion unit 302 whose center of gravity is decentered in the + x direction and the microlens 305. A pupil region that can be received by the bifocal detection pixel 202 is shown. The second pupil partial region 502 of the second focus detection pixel 202 has an eccentric center of gravity on the −X side on the pupil plane.

  In FIG. 4, a pupil region 500 is a pupil region that can receive light in the entire pixel 200 </ b> G including the photoelectric conversion unit 301 and the photoelectric conversion unit 302 (the first focus detection pixel 201 and the second focus detection pixel 202). .

  FIG. 5A is a schematic diagram showing a correspondence relationship between the image sensor of this embodiment and pupil division. The light beams that have passed through different pupil partial areas of the first pupil partial area 501 and the second pupil partial area 502 are incident on each (imaging) pixel of the imaging element from the imaging plane 800 at different angles and are divided by 2 × 1. The photoelectric conversion units 301 and 302 receive the light. In this embodiment, the pupil region is divided into two pupils in the horizontal direction, but pupil division may be performed in the vertical direction as necessary.

  The image sensor 103 includes a first focus detection pixel 201 that receives a light beam passing through the first pupil partial region 501 of the imaging optical system, and a second pupil partial region 502 of the imaging optical system different from the first pupil partial region. An image pickup pixel having a second focus detection pixel 202 that receives a light beam passing through is arranged. Therefore, the imaging pixel receives a light beam passing through the pupil region 500 that is a combination of the first pupil partial region 501 and the second pupil partial region 502 of the imaging optical system.

  Note that not all the pixels included in the image sensor 103 have a plurality of photoelectric conversion units, but the image pickup pixels, the first focus detection pixels, and the second focus detection pixels may have individual pixel configurations. Alternatively, an imaging pixel having one photoelectric conversion unit and a focus detection pixel having two photoelectric conversion units (which can also be used as an imaging pixel) may be arranged.

  In the present embodiment, the AF signal generation unit 106 generates a first focus detection signal from signals obtained from the plurality of first focus detection pixels 201, and the second focus from signals obtained from the plurality of second focus detection pixels 202. A detection signal is generated. Then, the focus detection unit 108 performs focus detection using the first focus detection signal and the second focus detection signal. Further, by adding the signals obtained by the first focus detection pixel 201 and the second focus detection pixel 202 for each image pickup pixel of the image pickup device, an image pickup signal (captured image) having a resolution of N effective pixels is generated.

[Relationship between defocus amount and image shift amount]
Hereinafter, the relationship between the defocus amount and the image shift amount of the first focus detection signal and the second focus detection signal acquired by the image sensor of the present embodiment will be described.
FIG. 5B shows a schematic relationship diagram of the defocus amount and the image shift amount between the first focus detection signal and the second focus detection signal. The imaging element 103 (not shown) is arranged on the imaging surface 800, and the exit pupil of the imaging optical system is placed in the first pupil partial region 501 and the second pupil partial region 502, as in FIGS. 4 and 5A. Divided into two.

  The magnitude | d | of the defocus amount d is the distance from the imaging position of the subject to the imaging surface 800. Further, when the defocus amount d is negative (d <0), the imaging position of the subject is a front pin state that is closer to the subject than the imaging surface 800, and when it is positive (d> 0), the imaging position of the subject. Means a rear pin state in which the image pickup surface 800 is on the opposite side of the subject. The defocus amount d is zero when the subject is focused on the imaging surface 800. In FIG. 5A, the subject 801 is in an in-focus state (d = 0), and the subject 802 shows an example of a front pin state (d <0). The front pin state (d <0) and the rear pin state (d> 0) are collectively referred to as a defocus state (| d |> 0).

  In the front pin state (d <0), the luminous flux that has passed through the first pupil partial area 501 (second pupil partial area 502) out of the luminous flux from the subject 802 is once condensed at a position on the subject side from the imaging plane 800. To do. Then, after that, the image expands in the width Γ1 (Γ2) with the center of gravity position G1 (G2) of the light beam as the center, resulting in a blurred image on the imaging surface 800. The blurred image is converted into an electrical signal by the first focus detection pixel 201 (second focus detection pixel 202) in each of the plurality of pixels that receive the blurred image. Then, the AF signal generator 106 generates a first focus detection signal (second focus detection signal) from the signals of the first focus detection pixel 201 group (second focus detection pixel 202 group). Therefore, the first focus detection signal (second focus detection signal) is recorded as a subject image in which the subject 802 is blurred by the width Γ1 (Γ2) at the gravity center position G1 (G2) on the imaging surface 800.

  The blur width Γ1 (Γ2) of the subject image increases approximately in proportion to the increase of the magnitude | d | of the defocus amount d. Similarly, the magnitude | p | of the object image displacement amount p (= difference G1-G2 in the center of gravity of the light beam) between the first focus detection signal and the second focus detection signal is also the size of the defocus amount d. It increases almost in proportion to the increase of | d |. In the rear pin state (d> 0), the magnitude of the defocus amount | d except that the image shift direction of the subject image between the first focus detection signal and the second focus detection signal is opposite to the front pin state. The relationship between |, the blur width of the subject image, and the image shift amount p is the same.

  Accordingly, the first focus detection signal and the second focus detection signal, or the first focus detection signal and the first focus detection signal are increased in accordance with an increase in the defocus amount of the imaging signal obtained by adding the first focus detection signal and the second focus detection signal. The amount of image shift between the second focus detection signals increases.

[Focus detection]
Hereinafter, the imaging surface phase difference AF in the present embodiment will be described.
In the imaging surface phase difference AF of the present embodiment, the first focus detection signal and the second focus detection signal are relatively shifted to calculate the amount of correlation indicating the degree of coincidence of signals, and the correlation (the degree of coincidence of signals) is good. The image shift amount is detected from the shift amount. Since the image shift amount between the first focus detection signal and the second focus detection signal increases as the defocus amount of the imaging signal increases, the focus detection is performed by converting the image shift amount into the defocus amount.

FIG. 6 is a flowchart for explaining an outline of the AF processing in the present embodiment. As described above, the AF operation is started in response to the input of the start instruction for the shooting preparation operation.
In S <b> 10, the control unit 110 determines whether or not the “starry sky shooting mode” is selected by the mode dial 1041. Here, for simplicity, it is assumed that the mode dial 1041 can select “starry sky shooting mode” or “normal mode”, but other shooting modes may be selectable.

  In the present embodiment, a case where the shooting mode is set by operating the mode dial 1041 will be described. However, a switch, button, touch panel, voice command, or the like may be used to set the shooting mode. Further, the configuration is not limited to the setting by the photographer, and for example, the digital camera 100 may be set according to the scene determination result using a known method. For example, it may be configured to automatically determine that a region in which point light sources (high-luminance small subjects) are scattered on a dark background is dominant, such as a starry sky scene, using a luminance histogram. Good.

  A starry sky scene is a typical example of a scene with low brightness and contrast, and the present invention is not necessarily limited to shooting a starry sky. Basically, the present invention is applied to a scene in which a region in which dotted (small area) high-brightness objects are scattered on a low-brightness background occupies a predetermined ratio or more (for example, half or more) of the screen or the focus detection region. The effect is fully obtained. As an example other than the starry sky scene, there is a night scene.

  If it is determined in S10 that the “starry sky shooting mode” is selected, the control unit 110 (selecting means) advances the process to S200. In S200, as will be described later, a defocus calculation process (first defocus calculation process) for the starry sky shooting mode is executed. On the other hand, if it is determined that “starry sky shooting mode” is not selected, that is, “normal mode” is selected, control unit 110 advances the process to S100. In S200, as described later, a defocus calculation process (second defocus calculation process) for the normal shooting mode is executed.

  In S300, the drive control unit 109 drives the focus lens 102 based on the defocus amount calculated in S100 or S200 to adjust the focus of the photographing optical system 101, and the AF process is ended. Note that a focus detection mode in which the focus of the photographing optical system 101 is adjusted based on the defocus amount calculated in the first defocus calculation process is referred to as a first focus detection mode. The focus detection mode in which the focus of the photographing optical system 101 is adjusted based on the defocus amount calculated in the second defocus calculation process is referred to as a second focus detection mode.

FIG. 7A is a flowchart for explaining details of the first defocus calculation process executed in S100 of FIG.
In S <b> 102, the control unit 110 sets the signal reading unit 105 to read the image sensor 103 in the first read mode in which an image signal having a resolution lower than the spatial resolution of the image sensor 103 is acquired. In the first readout mode, an addition process for adding three pixels of the same color adjacent in the readout (horizontal) direction of the image sensor 103 and readout every three rows in the (vertical) direction orthogonal to the readout direction (one row) Read out and skip two lines). In the first readout mode, the number of readout pixels is 1/3 in each of the horizontal and vertical directions, so that the readout time can be shortened. The signal readout unit 105 generates a timing signal for realizing the first readout mode, supplies the timing signal to the image sensor 103, and reads out the image signal. Note that horizontal addition and vertical thinning may be performed by the signal reading unit 105. In this case, the number of readout pixels does not decrease, but the processing load at the subsequent stage can be reduced.

  FIG. 8A schematically shows the relationship between 4 rows and 20 columns in the pixel array of the image sensor 103 and pixels read in the first readout mode. In the first read mode, line thinning is performed in which one line is read and two lines are skipped. Here, it is assumed that the 3 × M (M = 0 or greater) row is the read target row. Therefore, in FIG. 3, the 0th and 3rd rows are read target rows, and the 1st and 2nd rows are thinning target rows.

  In addition, addition reading of three pixels of the same color is performed in the horizontal direction. Taking the 0th row as an example, signals of G (green) pixels in the 0, 2, and 4 columns are added (averaged) in the image sensor 103. And read. Similarly, signals of three B (blue) pixels in the third, fifth, and seventh columns are added and read. The reason why the B (blue) pixel in the first column is not an addition target is that the positions of the central pixels of the pixel group to be added are equally spaced. The readout of the R (red) pixel and the G (green) pixel in the third row is the same as in the 0th row, and the G pixel in the first column is not an addition target.

  FIG. 8B schematically shows conversion processing from a signal obtained by horizontal addition and vertical thinning readout to a luminance (Y) signal. As shown in FIG. 8A, the image signal obtained in the first readout mode maintains the Bayer array. Therefore, one luminance (Y) signal is obtained from four pixels of 2 horizontal pixels × 2 vertical pixels, which is a repeating unit of the Bayer array. There is no limitation on the method for obtaining the luminance from the values of R, G (G1 and G2), and B, and a known method can be used. Simply, an average value of four pixels may be used as the luminance signal value.

  In FIG. 8, for simplicity of understanding and description, each pixel is described as an imaging pixel. However, in actuality, the output (A) of the first focus detection pixel 201 and the second focus detection pixel 202 In addition to the output (B), addition and conversion into a luminance signal are performed. From each pixel, A and B may be read out separately, or one of A and B and the sum (A + B) may be read out. In the latter case, one of A and B that has not been read (for example, A) can be obtained by subtracting one read (B) from the sum (A + B).

  In S <b> 120, the control unit 110 sets a focus detection area for the effective pixel area of the image sensor 103. The focus detection area may be set in advance as a fixed area, or may be dynamically set according to a face detection result or the like. Although a plurality of values can be set, it is assumed here that the number of focus detection areas is one for ease of explanation and understanding.

  The AF signal generation unit 106 generates a first focus detection signal (A image) from the output of the first focus detection pixel 201 of a plurality of pixels in the focus detection area, and outputs a second focus detection signal (A from the output of the second focus detection pixel). B image) is generated. Since the pixels of the image sensor 103 of the present embodiment are pupil-divided in the horizontal direction, first and second focus detection signals are generated from a plurality of pixels arranged in the horizontal direction. The first and second focus detection signals are supplied to the correction unit 107.

In S130, the correction unit 107 applies shading correction processing to the first focus detection signal and the second focus detection signal.
Here, the shading by the pupil shift of the first focus detection signal and the second focus detection signal will be described. FIG. 9 schematically shows the influence of the difference between the exit pupil distance Dl of the imaging optical system and the set pupil distance Ds of the image sensor on the pupil division in a pixel having a large image height (positioned away from the optical axis). It is a figure.

  FIG. 9A shows a case where the exit pupil distance Dl of the imaging optical system and the set pupil distance Ds of the image sensor 103 are the same. In this case, the first pupil partial region 501 and the second pupil partial region 502 are used for both a pixel having a small image height (positioned near the optical axis) and a pixel having a large image height (positioned away from the optical axis). The exit pupil 400 of the imaging optical system is divided into pupils almost uniformly.

  FIG. 9B shows a case where the exit pupil distance Dl of the imaging optical system is shorter than the set pupil distance Ds of the image sensor (D1 <Ds). In this case, in the peripheral pixels of the image sensor, a deviation occurs between the exit pupil 400 of the image forming optical system and the entrance pupil of the image sensor 103, and the exit pupil 400 of the image forming optical system is not as shown in the plan view. It is divided evenly.

  FIG. 9C shows a case where the exit pupil distance Dl of the imaging optical system is longer than the set pupil distance Ds of the image sensor (D1> Ds). Also in this case, as in the case of D1 <Ds, in the peripheral pixels of the imaging device, a deviation occurs between the exit pupil 400 of the imaging optical system and the entrance pupil of the imaging device 103, and the exit pupil 400 of the imaging optical system. Will be divided unevenly.

  If pupil division becomes non-uniform, there will be a difference in signal intensity obtained between the first focus detection pixel and the second focus detection pixel, so the intensity of the first focus detection signal and the second focus detection signal will be non-uniform (one side) ) Increases in intensity and decreases in intensity on the other side).

  In S130, the correction unit 107 performs the first shading correction of the first focus detection signal according to the image height of the focus detection region, the aperture value (F value) of the imaging lens (imaging optical system), and the exit pupil distance. A coefficient and a second shading correction coefficient of the second focus detection signal are respectively generated. Then, the control unit 110 multiplies the first focus detection signal by the first shading correction coefficient, multiplies the second focus detection signal by the second shading correction coefficient, and calculates the first focus detection signal and the second focus detection signal. Perform shading correction processing. The image height of the focus detection area may be a representative image height of a pixel position included in the focus detection area, for example, an image height of the center position.

  When calculating the defocus amount based on the correlation (signal coincidence) between the first focus detection signal and the second focus detection signal, if the above-described shading occurs, the accuracy of the defocus amount may decrease. . Therefore, in this embodiment, accurate calculation of the defocus amount is realized by performing shading correction on the focus detection signal.

  Note that although the setting pupil distance Ds of the image sensor 103 does not change and the exit pupil distance D1 of the imaging optical system changes as a cause of shading, the exit pupil distance D1 of the imaging optical system changes. The same applies to the case where the set pupil distance Ds of the image sensor 103 changes. In the imaging plane phase difference AF, the light flux received by the focus detection pixels (the first focus detection pixel and the second focus detection pixel) and the light flux received by the imaging pixels change as the set pupil distance Ds of the image sensor 103 changes. To do.

  In S140, the correction unit 107 applies filter processing to the first and second focus detection signals after shading correction. FIG. 10 shows an example of the frequency characteristics of the band pass filter applied in the filter processing applied in the present embodiment. When the focus detection in the large defocus state is performed by the phase difference detection method and the focus detection in the small defocus state is performed by the contrast method, a filter mainly including a low frequency band as a pass band as shown by a solid line. Can be applied. In the case of covering from the large defocus state to the small defocus state, a filter having a wider pass band as shown by a one-dot chain line in FIG. 10 can be applied. The correction unit 107 supplies the first and second focus detection signals after the filter processing to the focus detection unit 108.

  In S150, the focus detection unit 108 performs a shift process of relatively shifting the first focus detection signal and the second focus detection signal after the filter processing in the pupil division direction, and calculates a correlation amount representing the degree of coincidence of the signals.

The kth signals constituting the first focus detection signal and the second focus detection signal after the filter processing constitute A (k), B (k), and the first focus detection signal (and the second focus detection signal), respectively. Let W be the number of signals. Therefore, the range of the number k is 1 to W. The correlation amount COR is calculated by the equation (1), where s is the shift amount by the shift process and Γ is the range of the shift amount s.

  By the shift process of the shift amount s, the focus detection unit 108 performs the kth signal A (k) constituting the first focus detection signal and the (k−s) th signal B (k) constituting the second focus detection signal. -S) is subtracted to generate a shift subtraction signal. Then, the focus detection unit 108 calculates the absolute value of the generated shift subtraction signal, accumulates the value of k within the range W corresponding to the focus detection region while sequentially changing, and calculates the correlation amount COR (s) with respect to the shift amount s. calculate. The correlation amount COR (s) may be calculated by adding a plurality of correlation amounts calculated for a plurality of different pixel rows with the same shift amount s.

  In S <b> 160, the focus detection unit 108 calculates a real value shift amount at which the correlation amount is the minimum value from the correlation amount COR by sub-pixel calculation, and sets it as the image shift amount p. The defocus amount (Def) is calculated by multiplying the image shift amount p by the conversion factor K corresponding to the image height of the focus detection region, the aperture value of the imaging lens (imaging optical system), and the exit pupil distance. To do.

  FIG. 7B is a flowchart for explaining the details of the second defocus calculation process executed in S200 of FIG. 6, and the same reference is made to the operation steps similar to those of the first defocus calculation process. Numbers are attached.

  In S210, the control unit 110 sets the signal reading unit 105 to read the image sensor 103 in the second reading mode. In the second readout mode, pixel addition processing in the pupil division (horizontal) direction of the image sensor 103 is not performed, and a plurality of pixel rows adjacent in the (vertical) direction orthogonal to the pupil division direction (here, four rows as an example) )) Is performed. The signal readout unit 105 generates a timing signal for realizing the second readout mode, supplies the timing signal to the image sensor 103, and reads out the image signal. Note that the addition in the vertical direction may be executed by the signal reading unit 105. In this case, the number of readout pixels does not decrease, but the processing load at the subsequent stage can be reduced.

  When pixel addition processing (averaging processing) is performed on a scene where the image of the main subject (star), such as the starry sky, is extremely small and the background (sky) is low in contrast, the rise of the luminance of the main subject portion is moderated. Become. Therefore, if the addition process is applied in the pupil division direction, the phase difference detection accuracy, that is, the focus detection accuracy is reduced. Therefore, when the starry sky shooting mode is set, pixel addition processing is not performed in the pupil division direction. Also, if vertical thinning is performed as in the first defocus calculation process, the image of the main subject may be lost due to thinning.

  On the other hand, pixel addition in the vertical direction has a small influence on focus detection accuracy because of addition in a direction orthogonal to the pupil division direction. Also, unlike thinning, images are not lost. In addition, there is an effect of reducing random noise by the addition processing. Random noise is conspicuous in dark areas and high-sensitivity shooting, so it has a great effect in shooting scenes in the starry sky, where dark backgrounds occupy many and high-sensitivity shooting is likely. In this way, by performing addition processing in a direction different from the pupil division direction without performing addition processing in the pupil division direction, it is possible to reduce processing load and noise while suppressing influence on focus detection accuracy. it can.

  In S120 to S160, the same process as the first defocus calculation process described with reference to FIG. However, since the pixel addition process is not performed in the pupil division direction in the second defocus calculation process, a higher frequency component than the first defocus calculation process is included in the first and second focus detection signals. Yes. Therefore, the pass frequency characteristics of the filter applied in S140 may be different from the first defocus calculation process so that a more appropriate band can be extracted. Assume that the image shift amount obtained by the second defocus calculation processing is p1, and the defocus amount (Def) is p1K.

  As described above, the digital camera of the present embodiment has an operation mode in which automatic focus detection is performed based on an image signal that is added in a direction different from the pupil division direction without performing addition or thinning in the pupil division direction. . By performing automatic focus detection using the phase difference detection method in such an operation mode, it is possible to improve focus detection accuracy when shooting a subject with low brightness and contrast and to reduce the influence of noise.

  In the present embodiment, the configuration has been described in which the defocus calculation processing for the normal mode performs thinning in a direction different from the pupil division direction and does not perform addition. However, in the normal mode defocus calculation process, an addition process in a direction different from the pupil division direction may be executed instead of or in addition to the decimation. In this case, the defocus calculation process for the starry sky shooting mode can be configured to have a larger number of additions than the defocus calculation process for the normal mode.

  In addition, in the starry sky shooting mode, the number of additions in a direction different from the pupil division direction may be dynamically changed. For example, the control unit 110 counts high-luminance pixels (pixels exceeding a predetermined luminance value) in the focus detection area. And the control part 110 is standard when the total number of high-intensity pixels (= star image area | region) is more than a 1st threshold value and less than a 2nd threshold value, and when it is more than a 2nd threshold value, it is standard. When the number of additions is larger than the first threshold value, the number of additions smaller than the standard can be determined. If there are many star images in the focus detection area, this indicates that there is a large amount of information in the focus detection area. Therefore, positive image addition in a direction different from the pupil division direction has an effect on focus detection accuracy. This is because there are few.

  In addition, in the starry sky photographing mode, the number of additions in a direction different from the pupil division direction may be dynamically changed according to the aperture value of the photographing optical system 101. Specifically, as the aperture value becomes larger (the aperture becomes smaller), a dark star image is no longer detected, so the number of additions can be increased.

● (Second Embodiment)
Next, a second embodiment of the present invention will be described. Since this embodiment corresponds to another example of the defocus calculation process (second defocus calculation process) for the starry sky shooting mode in the first embodiment, the same configuration as the first embodiment, such as the configuration of a digital camera, etc. And the explanation about the processing is omitted. Specifically, this corresponds to another example of addition processing in a direction different from the pupil division direction (here, the vertical direction), which is executed in S210 of FIG. 7B.

  In the first embodiment, a preset number of additions (four rows) is unconditionally added. However, in this embodiment, a plurality of preset number of addition rows that exist in the same column. The difference is that the addition (average) processing is performed only when the pixel value of the above satisfies the condition. A specific addition process will be described with reference to the flowchart shown in FIG. This addition processing is executed by the signal reading unit 105. As described above, the set number of additions is not necessarily fixed.

The signal reading unit 105 executes the following processing for each column.
In S310, the signal readout unit 105 initializes the readout row number N to 1, and the intra-column peak value _SP to 0.
In S320, the signal reading unit 105 acquires _SN , which is a signal in the Nth row. This signal may be read from the image sensor 103 or may be stored in a buffer memory in the signal reading unit 105.

S330 the signal reading unit 105, the value of the acquired signal _{S N} in S320, it is determined whether or not _{S th} or more is a predetermined threshold value, to S360 in the case of more than the threshold value, if less than the threshold value The process proceeds to S340. Here, the threshold value S _th is set as an arbitrary value capable of discriminating a high-luminance subject such as a point light source, and may be, for example, an average signal intensity in the focus detection region.

S340 the signal reading unit 105 determines whether _{S P} is greater than 0. This process, among the plurality of rows of addition target, it is judged whether or not the pixel having the threshold S _th or more values were present in the lines up to this, the signal reading unit 105, determines that S _P is greater than 0 If so, the process proceeds to S380; otherwise, the process proceeds to S345.

In S345, the signal reading unit 105 determines whether N = 4. If N = 4, the process proceeds to S350, and if N <4, the process proceeds to S390.
In S350, the signal reading unit 105 executes an addition process (addition averaging process) of S _N (1 ≦ N ≦ 4), and ends the process for this column. The addition result is ΣS _N / 4. S350 is executed only when there is no high-luminance pixel in any of the addition target rows in this column, and the signals of the dark sky portions for four rows are added (averaged). On the other hand, if there is a high luminance pixel in any row, it is used as a result of addition of the columns in the peak value S _P.

S360 the signal reading section 105, _{S N} is determined whether the current is greater than the _{S P,} to S370 if it is determined to be larger, the process proceeds to S380 to be determined to be larger.
S370 the signal reading section 105 replaces the current _{S P} at _{S N} (updates).

In S380, the signal reading unit 105 determines whether or not N = 4. If N = 4, the process for this column ends, and if N <4, the process proceeds to S390.
In S390, the signal reading unit 105 adds 1 to N and returns the process to S320.

  The above addition process is performed for each column. In the present embodiment, the pixel signal obtained after the addition processing is configured such that a column (horizontal coordinate) in which a high luminance pixel exists is a pixel having a peak value, and a column in which no high luminance pixel exists is a pixel having an addition averaged value. Is done. Therefore, noise components existing in other rows are not superimposed on pixels corresponding to the star image, and noise is reduced by addition processing for pixels not corresponding to the star image (empty), and the accuracy of correlation calculation, that is, focus Detection accuracy can be improved. In this embodiment, the actual processing is performed separately for the outputs of the first and second focus detection pixels.

● (Third embodiment)
Next, a third embodiment of the present invention will be described. Since this embodiment corresponds to another example of the defocus calculation process (second defocus calculation process) for the starry sky shooting mode in the first embodiment, the same configuration as the first embodiment, such as the configuration of a digital camera, etc. And the explanation about the processing is omitted.

  The second defocus calculation process in the present embodiment will be described with reference to the flowchart of FIG. In FIG. 12A, the same reference numerals are assigned to the operation steps similar to those in the first defocus calculation process (FIG. 7A).

  In step S410, the control unit 110 sets the signal reading unit 105 to read the image sensor 103 in the third reading mode. In the third readout mode, addition and decimation are not performed in the pupil division (horizontal) direction of the image sensor 103 and in the (vertical) direction orthogonal to the pupil division direction. The signal readout unit 105 generates a timing signal that realizes the third readout mode, supplies the timing signal to the image sensor 103, and reads out the image signal.

  Unlike the first embodiment, the addition process is not performed in the vertical direction because the accuracy of the calculated correlation amount is given the highest priority, but on the other hand, it becomes a factor that increases the processing load. Therefore, in this embodiment, as will be described later, the processing load is reduced by a method different from the thinning process.

  S120 to S150 may be performed in the same manner as the first defocus calculation process described with reference to FIG. However, as described in the first embodiment, since the pixel addition process is not performed in the pupil division direction in the second defocus calculation process, the higher frequency components than the first defocus calculation process are the first. And included in the second focus detection signal. Therefore, the pass frequency characteristics of the filter applied in S140 may be different from the first defocus calculation process so that a more appropriate band can be extracted.

  In S460, the focus detection unit 108 determines a pixel row including a subject image (star image) based on the correlation amount calculated for each pixel row (or pixel line) in S450. Specifically, the focus detection unit 108 determines that a subject image is included in a pixel row in which the difference between the calculated maximum value and minimum value of the correlation amount is equal to or greater than a predetermined threshold, and calculates the defocus amount. Used for. As described above, this determination is a determination of a pixel row having a high correlation amount reliability. Thus, by calculating the defocus amount using only the correlation amount determined to have high reliability, a highly accurate defocus amount can be obtained. Further, by reducing the correlation amount used for calculating the defocus amount, the data amount and the processing amount relating to the defocus amount calculation can be reduced.

  In S160, the defocus amount is calculated in the same manner as in the first embodiment, based on the correlation amount determined to have high reliability in S460. Assume that the image shift amount obtained by the second defocus calculation processing is p1, and the defocus amount (Def) is p1K.

  In S460, the presence or absence of the subject image (whether the reliability of the correlation amount is high) is determined based on the correlation amount, but the same determination can be performed based on the first and second focus detection signals. . For example, it can be determined whether or not the difference between the maximum value and the minimum value of the first and second focus detection signals is greater than or equal to a predetermined threshold value. A region including a subject image (for example, a star image) has a very strong signal compared to the background (sky) portion. Therefore, if the difference between the maximum value and the minimum value of the first and second focus detection signals is equal to or greater than a predetermined threshold, it can be determined that the subject image is included. Note that, instead of the difference between the maximum value and the minimum value, it may be determined whether the maximum value is equal to or greater than a predetermined threshold.

  In this case, as shown in FIG. 12B, the determination process may be performed at any timing including immediately after (S530) if the first and second focus detection signals are generated in S120. it can. In a processing step after the determination process, the process can be performed using only data of pixel rows determined to have high reliability.

  The digital camera according to the present embodiment has an operation mode in which automatic focus detection is performed based on an image signal having a resolution equal to the spatial resolution of the image sensor without performing addition or thinning. In this operation mode, the defocus amount is calculated by excluding the area where no subject exists. By performing automatic focus detection in such an operation mode, it is possible to reduce the amount of processing while improving focus detection accuracy when shooting a subject with low brightness and contrast.

(Other embodiments)
In the case of a low-luminance scene such as a starry sky scene, it is also assumed that a highly reliable correlation amount cannot be calculated by a single defocus calculation process. Therefore, for example, in S150 of the above-described embodiment, the focus detection unit 108 may be configured to determine the reliability based on the difference between the maximum value and the minimum value of the correlation amount. If the reliability is low (for example, the above difference is equal to or less than the threshold value), the correlation amount obtained by the defocus calculation process performed on the next frame may be accumulated until it is determined that the reliability is high. it can. When correlation amounts are added over multiple frames, the accuracy of the correlation amount may decrease due to the movement of the subject between frames in general scenes, but when the movement of the subject is gentle like a starry sky scene. Provides sufficient accuracy.

  Although exemplary embodiments of the present invention have been described, the present invention is not limited to the configurations of the described embodiments, and various modifications and changes can be made. Modifications and changes included in the scope of the claims are included in the present invention.

  The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 100 ... Digital camera, 101 ... Shooting optical system, 102 ... Focus lens, 103 ... Imaging element, 105 ... Signal reading part, 106 ... AF signal generation part, 107 ... Focus detection part

Claims (14)

An imaging element capable of generating a plurality of image signals based on a light flux that has passed through different pupil partial areas by dividing the pupil area of the imaging optical system in the pupil division direction;
Obtaining means for obtaining an image signal from the image sensor;
Generating means for generating a pair of focus detection signals based on the image signal;
Calculating means for calculating a defocus amount of the photographing optical system based on a correlation amount of the pair of focus detection signals;
A drive unit that adjusts the focus of the imaging optical system based on the defocus amount,
The generating means has the resolution at least equal to the spatial resolution of the image sensor in the pupil division direction, and the pair of focus detection signals based on an image signal that has not been thinned out in a direction different from the pupil division direction A first focus detection mode for generating
A second focus detection mode in which the generating means generates the pair of focus detection signals based on an image signal having a resolution lower than a spatial resolution of the image sensor in the pupil division direction and the different direction;
An imaging device comprising:   2. The pair of focus detection signals according to claim 1, wherein in the second focus detection mode, the generation unit generates the pair of focus detection signals based on image signals that have been subjected to addition processing or thinning processing in the different directions. Imaging device.   In the first focus detection mode, the generation means generates the pair of focus detection signals based on image signals obtained by adding more signals in the different directions than in the second focus detection mode. The imaging device according to claim 1 or 2, wherein   The addition process in the first focus detection mode is applied to a plurality of pixel signals that do not have a peak value that is equal to or greater than a threshold value. The imaging apparatus according to claim 3, wherein the imaging apparatus is a signal.   3. The pair of focus detection signals according to claim 1, wherein in the first focus detection mode, the generation unit generates the pair of focus detection signals based on image signals not subjected to addition processing in the different directions. The imaging device described.   In the first focus detection mode, the calculation means calculates the defocus amount using the pair of focus detection signals or the correlation amount determined to have high reliability. The imaging device according to claim 5.   The imaging apparatus according to claim 6, wherein the reliability is determined based on the pair of focus detection signals or the maximum value of the correlation amount.   8. The apparatus according to claim 1, further comprising a selection unit that selects one of the first focus detection mode and the second focus detection mode in accordance with a set photographing mode. The imaging apparatus according to item 1.   The imaging apparatus according to claim 8, wherein the shooting mode is set by a user.   The imaging apparatus according to claim 8, wherein the shooting mode is set according to a scene discrimination result.   The photographing mode is a photographing mode for photographing a scene in which a region where dotted high-luminance subjects are scattered on a low-luminance background occupies a predetermined ratio or more of a screen or a focus detection region. The imaging device according to any one of claims 8 to 10.   12. The imaging apparatus according to claim 8, wherein the shooting mode is a shooting mode for shooting either a starry sky scene or a night scene. A method for controlling an imaging apparatus having an imaging element capable of generating a plurality of image signals based on a light flux that has passed through different pupil partial areas by dividing a pupil area of an imaging optical system in a pupil division direction,
An acquisition step of acquiring an image signal from the image sensor;
A generating step for generating a pair of focus detection signals based on the image signal;
A calculating step of calculating a defocus amount of the photographing optical system based on a correlation amount of the pair of focus detection signals;
A driving step for adjusting the focus of the photographing optical system based on the defocus amount;
The generating step includes
In the first focus detection mode, the pair of pairs is based on an image signal having a resolution at least equal to the spatial resolution of the image sensor in the pupil division direction and not thinned out in a direction different from the pupil division direction. Generating a focus detection signal;
Generating the pair of focus detection signals based on an image signal having a resolution lower than a spatial resolution of the image sensor in the pupil division direction and the different direction in the second focus detection mode;
A method for controlling an imaging apparatus, comprising:   The program for functioning the computer which an imaging device has as each means of the imaging device of any one of Claims 1-12.

Images