JP6222278B2 - Focus detection apparatus and imaging apparatus - Google Patents

Description

  The present invention relates to a focus detection apparatus and an imaging apparatus.

  Focus detection pixels including a microlens and a pair of photoelectric conversion units arranged behind the microlens are arranged on a planned focal plane of the photographing lens. Thus, a pair of image signals corresponding to the pair of images formed by the pair of focus detection light beams passing through the optical system is generated. By detecting the image shift amount (phase difference) between the pair of image signals, the focus adjustment state (defocus amount) of the photographing lens is detected. A so-called pupil division type phase difference detection type focus detection apparatus that performs such an operation is known.

  The signals of the focus detection pixels and the imaging pixels are read out at a fixed frame time interval from the imaging element (image sensor) in which the focus detection pixels and the imaging pixels are mixed, and live view display and focus detection are performed in parallel. At the same time, the focus detection pixel signal for each frame is stored over a plurality of past frames. If focus detection is impossible due to insufficient output level with only the focus detection pixel signal of the latest frame, the stored focus detection pixel signals over a plurality of past frames are added and time integration is performed. By doing so, there is known a focus detection device that performs focus detection using a signal of a focus detection pixel whose output level has increased (see, for example, Patent Document 1).

JP 2008-85738 A

  The focus detection apparatus as described above has a problem that an error may occur in the focus detection result for a moving subject.

(1) According to the first aspect of the present invention, the focus detection device detects light transmitted through the optical system, and outputs the first focus detection signal and the second focus detection signal in the first direction. In the imaging device in which a first number of focus detection units having a plurality of arranged focus detection pixels are provided in a second direction different from the first direction, and the first number of focus detection units . a first addition value obtained by adding the first focus detection signals respectively outputted from the arranged focus detection pixels in the second direction, the second direction in the focus detection unit of the first number an adder for calculating a second sum value obtained by adding the outputted second focus detection signals with each other from the placed the focus detection pixel, by the first addition value and the second sum value and a focus state detection unit for detecting a focus state of the optical system, the addition unit Until the first integrated value based on the first addition value and the second addition value becomes larger than a threshold value, the second number of the first focus detection signals with the first number as an upper limit, and you adding the second focus detection signal between said second number.
(2) According to the second aspect of the present invention, the imaging apparatus focuses the optical system based on the in-focus state detected by the in-focus state detection unit according to any one of the first to fifteenth aspects. A driving unit that drives the lens to a position; and an acquisition unit that acquires image data based on a photographic light beam from a subject passing through the optical system when the optical system is positioned at the in-focus position.

  According to the present invention, the focus detection apparatus can perform focus detection with high accuracy.

It is a cross-sectional view which shows the structure of a digital still camera. It is a block diagram which shows the relationship between an image pick-up element and a body drive control apparatus in detail. It is a figure which shows the focus detection position on an imaging | photography screen. It is a front view which shows the detailed structure of an image pick-up element. It is a front view which shows the detailed structure of an image pick-up element. It is a figure which shows the structure of the focus detection optical system of the pupil division type phase difference detection system using a micro lens. It is a figure for demonstrating the mode of the imaging | photography light beam which the imaging pixel of an image pick-up element light-receives. It is a flowchart which shows the imaging operation including the focus detection operation | movement of a digital still camera. It is a process flowchart of the production | generation of the data for focus detection. It is the figure which showed the mode of the space integration process. It is the figure which showed the mode of the time integration process. It is a timing chart of processing operation of generating data for focus detection. It is a process flowchart of the production | generation of the data for focus detection. It is the figure which showed the mode of the time integration process. It is a process flowchart of the production | generation of the data for focus detection. It is a graph for demonstrating the detail of the process which determines the frequency | count of space integration, and the frequency | count of time integration. It is a front view which shows the detailed structure of an image pick-up element.

<First Embodiment>
FIG. 1 is a cross-sectional view illustrating a configuration of a lens interchangeable digital still camera 201 as an imaging apparatus including a focus detection apparatus according to the first embodiment of the present invention. The digital still camera 201 includes an interchangeable lens 202 and a camera body 203. Various interchangeable lenses 202 are attached to the camera body 203 via the mount unit 204.

  The interchangeable lens 202 includes a lens 209, a zooming lens 208, a focusing lens 210, a diaphragm 211, a lens drive control device 206, and the like. The lens drive control device 206 includes a microcomputer (not shown), a memory, a drive control circuit, and the like. The lens drive control device 206 performs drive control for adjusting the focus of the focusing lens 210 and adjusting the aperture diameter of the aperture 211, and detects the states of the zooming lens 208, the focusing lens 210, and the aperture 211, and the like. In addition, the lens drive control device 206 transmits lens information and receives camera information through communication with a body drive control device 214 described later. The aperture 211 forms an aperture having a variable aperture diameter at the center of the optical axis in order to adjust the amount of light and the amount of blur.

  The camera body 203 includes an image sensor (image sensor) 212, a body drive control device 214, a liquid crystal display element drive circuit 215, a liquid crystal display element 216, an eyepiece lens 217, a memory card 219, and the like. In the image sensor 212, a plurality of image pickup pixels are two-dimensionally arranged, and a plurality of focus detection pixels are incorporated in a portion corresponding to the focus detection position. Details of the image sensor 212 will be described later.

  The body drive control device 214 includes a microcomputer, a memory, a drive control circuit, and the like. The body drive control device 214 repeatedly performs drive control of the image sensor 212, readout of the image signal and focus detection signal, focus detection calculation based on the focus detection signal, and focus adjustment of the interchangeable lens 202, and processing of the image signal. Recording and camera operation control are performed. The body drive control device 214 communicates with the lens drive control device 206 via the electrical contact 213 to receive lens information and transmit camera information (defocus amount, aperture value, etc.).

  The liquid crystal display element 216 functions as an electronic view finder (EVF). The liquid crystal display element driving circuit 215 displays a through image on the liquid crystal display element 216 based on the image signal obtained by the imaging element 212. The photographer can observe the through image through the eyepiece lens 217. The memory card 219 is an image storage that stores image data generated based on an image signal captured by the image sensor 212.

  A subject image is formed on the light receiving surface of the image sensor 212 by the light beam that has passed through the interchangeable lens 202. This subject image is photoelectrically converted by the image sensor 212, and an image signal and a focus detection signal are sent to the body drive control device 214.

  The body drive control device 214 calculates the defocus amount based on the focus detection signal from the focus detection pixel of the image sensor 212 and sends the defocus amount to the lens drive control device 206. The body drive control device 214 processes image signals from the image sensor 212 to generate image data, and stores the image data in the memory card 219. At the same time, the body drive control device 214 sends a through image signal from the image sensor 212 to the liquid crystal display element drive circuit 215 and causes the liquid crystal display element 216 to display the through image. Further, the body drive control device 214 sends aperture control information to the lens drive control device 206 to control the aperture of the aperture 211.

  The lens drive controller 206 updates the lens information according to the focusing state, zooming state, aperture setting state, aperture opening F value, and the like. Specifically, the positions of the zooming lens 208 and the focusing lens 210 and the aperture value of the aperture 211 are detected, and lens information is calculated according to these lens positions and aperture values, or prepared in advance. Lens information corresponding to the lens position and aperture value is selected from the look-up table.

  The lens drive control device 206 calculates a lens drive amount based on the received defocus amount, and drives the focusing lens 210 to the in-focus position according to the lens drive amount. Further, the lens drive control device 206 drives the diaphragm 211 in accordance with the received diaphragm value.

FIG. 2 is a block diagram showing in detail the relationship between the image sensor 212 and the body drive controller 214 related to the present invention. As shown in FIG. 2, an image sensor control unit 220, a buffer memory 221, a CPU (microcomputer) 222, and an internal memory 223 are housed in the body drive control device 214. The image sensor 212 performs charge accumulation control (charge accumulation time and charge accumulation timing) of the image pickup pixel and the focus detection pixel and output control of the image signal and the focus detection signal according to the control of the image sensor control unit 220. An image signal corresponding to a subject image read from the image sensor 212 by the image sensor control unit 220 and a focus detection signal corresponding to a pair of images described later are subjected to signal amplification and AD conversion preprocessing, and then buffer memory 221 is temporarily stored as data for one frame. The CPU 222 performs well-known image processing on the image signal of the data for one frame stored in the buffer memory 221 to perform image display and image recording, and data for one frame stored in the buffer memory 221. The focus detection process described later is performed on the focus detection signal. The internal memory 223 is a memory for storing focus detection signals for a plurality of past frames. The CPU 222 refers to the focus detection signal of the past frame stored in the internal memory 223 in the focus detection process. When the focus detection processing for the focus detection signal of the latest frame is completed, the focus detection signal of the latest frame data temporarily stored in the buffer memory 221 is transferred to the internal memory 223. The internal memory 223 has a FILO (first in last out) stack structure, and in the internal memory 223, the focus detection signals for the most recent predetermined frames in the past are sequentially updated and recorded.

  FIG. 3 is a diagram showing a focus detection position on the shooting screen, and an area (focus detection area, focus detection) for sampling an image on the shooting screen at the time of focus detection by a focus detection pixel array on the image sensor 212 described later. An example of (position) is shown. In this example, the focus detection area 101 is arranged at the center on the rectangular shooting screen 100. Focus detection pixels are arranged corresponding to the focus detection area 101 indicated by a rectangle.

  4 and 5 are front views showing the detailed configuration of the image sensor 212, and show an enlarged view of the focus detection area 101 on the image sensor 212. FIG. FIG. 4 shows a layout of the imaging pixel 310 and the focus detection pixel 311. In FIG. 4, on the image sensor 212, the imaging pixels 310 and the focus detection pixels 311 are mixed and densely arranged in a two-dimensional square lattice. Focus detection pixel rows L1 to L8 are formed by arranging focus detection pixels 311 every other pixel row in a plurality of pixel rows extending in the horizontal direction. FIG. 5 shows an arrangement of color filters arranged in the imaging pixel 310 and the focus detection pixel 311 shown in FIG. In the imaging pixel 310 and the focus detection pixel 311, a color filter, a red filter R, a green filter G, and a blue filter B are arranged according to the rules of the Bayer arrangement. The red filter R, the green filter G, and the blue filter B each exhibit high spectral sensitivity at different wavelengths. The focus detection pixel 311 includes a green filter G, and a horizontal phase difference according to data (focus detection signals) of a plurality of focus detection pixels 311 arranged in each of the horizontal focus detection pixel rows L1 to L8. Detection is performed. Data of a plurality of focus detection pixels 311 arranged in the focus detection pixel row L1 is data used for normal focus detection. Depending on the situation, the data of the plurality of focus detection pixels 311 arranged in each of the focus detection pixel rows L2 to L8 becomes the data of the plurality of focus detection pixels 311 arranged in the focus detection pixel row L1 corresponding to them. The accumulated value is used for focus detection. Since such integration is an integration of the data of the focus detection pixels 311 arranged at spatially different positions, such integration is hereinafter referred to as spatial integration.

  The imaging pixel 310 includes a rectangular microlens 10 and a photoelectric conversion unit 11 whose light receiving area is limited by a light shielding mask (not shown). The focus detection pixel 311 includes a rectangular microlens 10 and a pair of photoelectric conversion units 13 and 14 obtained by dividing the photoelectric conversion unit 11 of the imaging pixel 310 into two by an element isolation region 15 extending in the vertical direction. Is done. For simplicity, the color filter is not shown in FIG.

  The imaging pixel 310 is designed in such a shape that the photoelectric conversion unit 11 receives all the imaging light flux that passes through the exit pupil diameter (for example, F1.0) of the brightest interchangeable lens 202 by the microlens 10. Further, the focus detection pixel 311 includes a pair of focus detection light beams that pass through a pair of regions arranged in parallel with the arrangement direction of the pair of photoelectric conversion units 13 and 14 in the exit pupil of the interchangeable lens 202 by the microlens 10. The photoelectric conversion units 13 and 14 are designed to receive light.

  FIG. 6 shows a configuration of a focus detection optical system of the pupil division type phase difference detection method using the microlens 10. FIG. 6 schematically expands three adjacent focus detection pixels 311 and two imaging pixels 310 in the vicinity of the photographing optical axis 91 arranged in the horizontal focus detection pixel row L1 in the focus detection area 101. Show. In FIG. 6, the exit pupil 90 is set at a position of a distance d forward from the microlens 10 disposed on the planned imaging plane of the interchangeable lens 202 (see FIG. 1). This distance d is a distance determined according to the curvature and refractive index of the microlens 10, the distance between the microlens 10 and the photoelectric conversion units 13 and 14, and is referred to as a distance measuring pupil distance in this specification. In addition, FIG. 6 shows an optical axis 91 of the interchangeable lens, the microlens 10, the photoelectric conversion units 13 and 14, the focus detection pixel 311, the imaging pixel 310, and the focus detection light beams 73 and 74.

  The distance measuring pupil 93 is formed by projecting the photoelectric conversion unit 13 whose light receiving area is limited by the opening of the light shielding mask by the microlens 10. Similarly, the distance measuring pupil 94 is formed by projecting the photoelectric conversion unit 14 whose light receiving area is limited by the opening of the light shielding mask by the microlens 10. The pair of distance measuring pupils 93 and 94 are symmetrical with respect to a vertical line passing through the optical axis 91. The pair of distance measuring pupils 93 and 94 correspond to the pair of regions described above. By the microlens 10, the pair of photoelectric conversion units 13 and 14 and the above-described pair of regions, that is, the pair of distance measuring pupils 93 and 94 are in a conjugate relationship with each other.

  In all the focus detection pixels 311 arranged in the focus detection pixel rows L1 to L8 in the horizontal direction of the focus detection area 101, the pair of photoelectric conversion units 13 and 14 has the same horizontal as the focus detection pixels constituting the focus detection pixel row. Lined up in the direction. The pair of photoelectric conversion units 13 and 14 correspond to each other, and a pair of focal points arriving at each microlens from a pair of distance measuring pupils 93 and 94 arranged in the same direction as the arrangement direction of the pair of photoelectric conversion units 13 and 14. Detection light beams 73 and 74 are received. When the pair of photoelectric conversion units 13 and 14 included in each of the plurality of focus detection pixels 311 constituting each focus detection pixel row receive the pair of focus detection light beams 73 and 74, a pair of focus detection light beams is obtained by photoelectric conversion. A pair of image signals corresponding to the pair of images 73 and 74 are repeatedly output at predetermined frame intervals.

  With the configuration described above, the photoelectric conversion unit 13 corresponds to the intensity of the image formed on the microlens 10 by the focus detection light beam 73 that passes through the distance measuring pupil 93 and travels toward the microlens 10 of the focus detection pixel 311. Output a signal. Further, the photoelectric conversion unit 14 outputs a signal corresponding to the intensity of the image formed on the microlens 10 by the focus detection light beam 74 that passes through the distance measuring pupil 94 and faces the microlens 10 of the focus detection pixel 311.

  In the focus detection pixels 311 arranged in the horizontal focus detection pixel row L1, the outputs of the photoelectric conversion units 13 and 14 of each focus detection pixel 311 are output groups corresponding to the distance measurement pupil 93 and the distance measurement pupil 94, respectively. To summarize. Accordingly, a pair of focus detection light beams 73 and 74 that pass through the distance measurement pupil 93 and the distance measurement pupil 94, respectively, are formed on the array of the plurality of focus detection pixels 311 included in the horizontal focus detection pixel row L1. Information about the intensity distribution of the image is obtained. By applying an image shift detection calculation process (correlation calculation process, phase difference detection process) to this information, a pair of horizontal images in the horizontal focus detection pixel array L1 in a so-called pupil division type phase difference detection method is obtained. An image shift amount is detected.

  Similarly, using the outputs of the photoelectric conversion units 13 and 14 of the focus detection pixels 311 arranged in the horizontal focus detection pixel rows L2 to L8, an image shift amount of a pair of images in the horizontal direction in each focus detection pixel row is detected. can do.

  Further, by performing a conversion calculation according to the proportional relationship between the distance between the center of gravity of the pair of distance measurement pupils 93 and 94 and the distance measurement pupil distance with respect to the image shift amount, the current image plane relative to the planned image plane is determined. A deviation (defocus amount) is calculated. Specifically, by multiplying an image shift amount in a plane perpendicular to the optical axis 91 by a predetermined conversion coefficient, a defocus amount, that is, a deviation between the imaging plane and the planned imaging plane in the direction of the optical axis 91 is obtained. Will be calculated. The predetermined conversion coefficient is obtained as a value obtained by dividing the distance measuring pupil distance d by the center of gravity distance between the distance measuring pupils 93 and 94.

  FIG. 7 is a diagram for explaining the state of the imaging light flux received by the imaging pixel 310 of the imaging device 212 shown in FIG. 4 in comparison with FIG. FIG. 7 schematically shows an enlarged view of five adjacent imaging pixels 310 in the vicinity of the imaging optical axis 91 arranged in the horizontal imaging pixel column adjacent to the horizontal focus detection pixel column L1. Note that description of portions overlapping those in FIG. 6 is omitted.

  The imaging pixel 310 includes the microlens 10 and the photoelectric conversion unit 11 disposed behind the microlens 10. The shape of the opening of the light shielding mask arranged in the vicinity of the photoelectric conversion unit 11 is projected onto the exit pupil 90 separated from the microlens 10 by the distance measurement pupil distance d. The projection shape forms a region 95 that is substantially circumscribed by the distance measuring pupils 93 and 94. The photoelectric conversion unit 11 outputs a signal corresponding to the intensity of the image formed on the microlens 10 by the imaging light beam 71 that passes through the region 95 and travels toward the microlens 10. That is, the plurality of imaging pixels 310 receive the imaging light beam 71 from the subject passing through the interchangeable lens 202 and output subject image signals corresponding to the subject image by photoelectric conversion at predetermined frame intervals.

  FIG. 8 is a flowchart showing an imaging operation including a focus detection operation of the digital still camera (imaging apparatus) 201 including the focus detection apparatus of the present embodiment. When the power of the digital still camera 201 is turned on in step S100, the body drive control device 214 starts an imaging operation after step S110. In step S110, the image sensor control unit 220 of the body drive controller 214 reads out pixel data of all pixels, and the CPU 222 of the body drive controller 214 causes the liquid crystal display element 216 to display the pixel data of the image pixel 310. In the subsequent step S120, the CPU 222 of the body drive control device 214 generates focus detection data used for focus detection based on the pixel data of the focus detection pixel 311. The focus detection data generation process will be described in detail later.

  In step S130, the CPU 222 of the body drive control device 214 performs a phase difference detection calculation (image shift detection calculation) in the focus detection area 101 based on the focus detection data. The CPU 222 of the body drive control device 214 calculates the defocus amount based on the phase difference (image shift amount) detected by the phase difference detection calculation.

  In step S140, the CPU 222 of the body drive control device 214 checks whether or not it is close to focusing, that is, whether or not the calculated absolute value of the defocus amount is within a predetermined value. If it is determined that the focus is not close, the process proceeds to step S150, and the CPU 222 of the body drive control device 214 transmits the defocus amount to the lens drive control device 206, and the focusing lens 210 of the interchangeable lens 202 is adjusted. Drive to the focal position. Thereafter, the process returns to step S110, and the above-described operation is repeated.

  Even when focus detection is impossible, the process branches to step S150, and the CPU 222 of the body drive control device 214 transmits a scan drive command to the lens drive control device 206, and the focusing lens 210 of the interchangeable lens 202 is infinite. Scan driving is performed from a far position to a close position. Thereafter, the process returns to step S110 and the above-described operation is repeated.

  If it is determined in step S140 that the focus is close to the focus, the process proceeds to step S160. In step S160, the CPU 222 of the body drive control device 214 determines whether or not a shutter release has been performed by operating a shutter button (not shown). If it is determined that the shutter release has not been performed, the process returns to step S110, and the above-described operation is repeated. On the other hand, if it is determined that the shutter release has been performed, the process proceeds to step S170. In step S170, the CPU 222 of the body drive control device 214 transmits an aperture adjustment command to the lens drive control device 206, and sets the aperture value of the interchangeable lens 202 to the control F value (the F value set by the photographer or automatically set). F value). When the aperture control is completed, the image sensor control unit 220 of the body drive control device 214 causes the image sensor 212 to perform an imaging operation, and reads out pixel data from the imaging pixels 310 of the image sensor 212 and all the focus detection pixels 311. .

  In step S180, the CPU 222 of the body drive control device 214 arranges pixel data as image data at the position of each focus detection pixel 311 based on the pixel data of each focus detection pixel 311, that is, in each focus detection pixel 311. Calculation is performed by adding the output data of the pair of photoelectric conversion units 13 and 14. In subsequent step S190, the CPU 222 of the body drive control device 214 acquires pixel data used as image data of the imaging pixel 310 and image data of the focus detection pixel position, and stores them in the memory card 219. This image data is obtained based on the photographing light beam 71 from the subject passing through the interchangeable lens 202 when the focusing lens 210 of the interchangeable lens 202 is located at the in-focus position. This process returns to step S110 and the above-described operation is repeated.

  Note that the repetitive operations in steps S110 to S160 are performed in conjunction with a frame reading operation in which pixel data for one frame is periodically read from the image sensor 212 at predetermined frame intervals.

Details of the image shift detection calculation process (correlation calculation process, phase difference detection process) in step S130 of FIG. 8 will be described below. In the pair of images detected by the focus detection pixel 311, there is a possibility that the distance measurement pupils 93 and 94 are displaced by the aperture of the lens and the light quantity balance is lost. Therefore, in step S130, the CPU 222 of the body drive control device 214 performs a type of correlation calculation that can maintain the image shift detection accuracy with respect to the light amount balance. A pair of image signals _A1 n for focus detection _{(A1 1, ···, A1 M} : M is the number of _{signals), A2 n (A2 1,} ···, A2 M) relative to, for example, JP 2007-333720 The correlation amount C (k) is calculated using the following equation (1) which is a well-known correlation equation disclosed in the publication. In equation (1), the Σ operation is accumulated for variable n. The variable n is limited to a range in which data of A1 _n , A1 _{n + 1} , A2 _{n + k} , A2 _{n + 1 + k} exists according to the image shift amount k. The image shift amount k is an integer, and is a relative shift amount in units of the data interval of the signal sequence constituting the pair of image signals.
C (k) = Σ | A1 _n · A2 _{n + 1 + k} −A2 _{n + k} · A1 _{n + 1} | (1)

When the minimum value C (X) of the continuous correlation amount C (x) corresponding to the correlation amount C (k), which is a discrete value calculated by the above-described equation (1), is obtained, the following (2 ), The shift amount X that gives the minimum value C (X) of the continuous correlation amount C (x) is converted into the image shift amount shft. In the equation (2), the coefficient PY is a value twice the pixel pitch of the focus detection pixels 311 constituting the focus detection pixel rows L1 to L8, that is, the pixel pitch of the pixels arranged in the image sensor 212.
shft = PY · X (2)

  Details of generation of focus detection data based on the pixel data of the focus detection pixel 311 in step S120 of FIG. 8 will be described using the processing flowchart of FIG. N focus detection pixels 311 are arranged in each focus detection pixel column of the focus detection pixel columns L1 to L8 in FIG. Symbols n (n = 1 to N) are associated with the focus detection pixels 311 constituting each focus detection pixel row Lp (p = 1, 2,..., 8) in the horizontal arrangement order. Of the pair of photoelectric conversion units 13 and 14 included in the nth focus detection pixel 311 of the focus detection pixel row Lp, the s (s = 1, 2) th photoelectric conversion unit includes data B (s, n, p). Is output. That is, the pair of photoelectric conversion units 13 and 14 output a pair of data B (1, n, p) and B (2, n, p), respectively.

In step S200, the CPU 222 of the body drive control device 214 determines whether or not the maximum value of the pixel data of the focus detection pixels 311 constituting the focus detection pixel row L1 shown in FIG. 4 exceeds a predetermined threshold T1, that is, (3) Check whether the expression is satisfied. When the maximum value of the pixel data of the focus detection pixels 311 constituting the focus detection pixel row L1 exceeds the predetermined threshold value T1, the output level of the focus detection pixels 311 reaches a level sufficient for focus detection. In the expression (3), Max () is a function for obtaining the maximum value.
Max (B (s, n, 1))> T1 (3)

If the expression (3) is satisfied in step S200, the process proceeds to step S210. In step S210, the CPU 222 of the body drive control device 214 converts the pixel data of the focus detection pixels 311 arranged in the focus detection pixel row L1 (p = 1) into focus detection data B0 ( s, n). The process proceeds to step S310.
B0 (s, n) = B (s, n, 1) s = 1, 2 n = 1 to N (4)

On the other hand, if the expression (3) is not satisfied in step S200, the process proceeds to step S220. In step S220, the CPU 222 of the body drive control device 214 performs pixel data of the focus detection pixel 311 of the focus detection pixel column L1, pixel data of the focus detection pixel 311 of the focus detection pixel column L2 adjacent to the focus detection pixel column L1, and Is spatially integrated. As shown in the equation (5), by the spatial integration performed on the data output by the photoelectric conversion units of the same type among the focus detection pixels in which the variables n assigned in the horizontal order are arranged at the same position. , Data B1 (s, n, 2) is calculated.
B1 (s, n, 2) = B1 (s, n, 1) + B (s, n, 2) s = 1, 2 n = 1 to N
... (5)

  That is, the pixel data of the focus detection pixels 311 adjacent in the vertical direction is spatially integrated (spatial integration) according to the equation (5), and thus the focus detection pixels 311 spatially integrated from the focus detection pixel rows L1 to Lp. Data B1 (s, n, p) is obtained. Note that the data B1 (s, n, 1) is equal to the data B (s, n, 1).

In step S230, the CPU 222 of the body drive control device 214 causes the maximum value of the spatially integrated data B1 (s, n, p) (p = 2 when first coming to this step) to exceed the predetermined threshold T1. It is checked whether or not, that is, whether or not the expression (6) is satisfied.
Max (B1 (s, n, p))> T1 s = 1, 2 n = 1 to N (6)

If the expression (6) is satisfied in step S230, the process proceeds to step S240. In step S240, the CPU 222 of the body drive control device 214 uses the data B1 (s, n, p) of the focus detection pixel 311 that is spatially integrated up to the focus detection pixel row Lp as shown in equation (7) as focus detection data. It is determined as B0 (s, n). The process proceeds to step S310.
B0 (s, n) = B1 (s, n, p) s = 1, 2 n = 1 to N (7)

Meanwhile step S230, the case where (6) is not satisfied, the procedure proceeds to step S 250. In step S 250, CPU 222 of the body drive control device 214, whether space integration is performed to the focus detection pixel row L8, i.e. spatial integration number is checked whether or not reached the maximum accumulation number 7 times. If the spatial integration has not reached the focus detection pixel row L8, the process returns to step S220. In step S220, the CPU 222 of the body drive control device 214 uses the data B (s, n, p + 1) of the focus detection pixels 311 arranged in the next focus detection pixel row L (p + 1) as shown in equation (8), The spatial integration is performed on the data B1 (s, n, p) of the focus detection pixel 311 that has been spatially integrated so far.
B1 (s, n, p + 1) = B1 (s, n, p) + B (s, n, p + 1)
s = 1, 2, n = 1 to N (8)

This process step S220, step S230, the around the loop of steps S 250, Finally, in step S 250, if the spatial integration count is determined to have reached the maximum accumulation number, there is no i.e. more focus detection pixel row Therefore, if space integration cannot be performed, the process proceeds to step S260. At this time, spatial integration data B2 (s, n, 0) of the data of the focus detection pixels 311 that have been spatially integrated from the focus detection pixel rows L1 to L8 in the latest frame is obtained. In step S260, the CPU 222 of the body drive control device 214 reads the pixel data of the focus detection pixels 311 one frame before stored in the internal memory 223, and the focus detection pixels 311 arranged in the focus detection pixel rows L1 to L8. The data is spatially integrated as shown in equation (9). At this time, spatial integration data B2 (s, n, v) of the data of the focus detection pixel 311 before v frames with respect to the latest frame (v = 1 when entering step S260 for the first time) is obtained. In addition, it is assumed that focus detection pixel data up to 10 frames before (v = 10) is stored in the internal memory 223.
B2 (s, n, v) = ΣB (s, n, p) Σ operation is p = 1-8 (9)

In step S270, the CPU 222 of the body drive control device 214, as shown in equation (10), calculates the spatial integration data B2 (s, n, v) one frame before calculated in step S260 and the previous frames. Time integration data B3 (s, n, v-1) is integrated with time.
B3 (s, n, v) = B3 (s, n, v-1) + B2 (s, n, v)
s = 1, 2 n = 1 to N (10)

  That is, spatial integration data of temporally adjacent frames is integrated temporally (time integration) over the past according to the equation (10), and time integration data B3 (s, n, v) is obtained. The time integration data B3 (s, n, v) represents the focus detection pixel data obtained by time integration of the space integration data B2 (s, n, v) from the latest frame to the previous v frame. However, the time integration data B3 (s, n, 0) is equal to the space integration data B2 (s, n, 0) and data B1 (s, n, 8).

In step S280, the CPU 222 of the body drive control device 214 determines whether or not the maximum value of the time-integrated data B3 (s, n, v) exceeds a predetermined threshold value T1, that is, satisfies the expression (11). Check whether or not. When the determination process in step S280 is performed for the first time, v = 1.
Max (B3 (s, n, v))> T1 (11)

If the expression (11) is satisfied in step S280, the process proceeds to step S290. In step S290, the CPU 222 of the body drive control device 214 performs focus detection on the time integration data B3 (s, n, v) of the focus detection pixel that has been time integrated from the latest frame to the previous v frame as shown in equation (12). Data B0 (s, n) is determined. The process proceeds to step S310.
B0 (s, n) = B3 (s, n, v) s = 1, 2 n = 1 to N (12)

  On the other hand, if the expression (11) is not satisfied in step S280, the process proceeds to step S300. In step S300, the CPU 222 of the body drive control device 214 checks whether or not the time integration has been performed up to 10 frames before, that is, whether or not the time integration count has reached the maximum integration count of 10. If the time integration has not reached 10 frames before, the process returns to step S260, and the CPU 222 of the body drive control device 214 performs space for the previous frame one frame before the time integration has been performed so far. Accumulate. At the same time, the loop of step S260, step S270, step S280, and step S300 is repeated.

  If it is determined in step S300 that the number of time integrations has reached the maximum number of integrations 10, the CPU 222 of the body drive control device 214 determines the focus detection pixels that have been time integrated from the latest frame to 10 frames before in step S290. The time integration data B3 (s, n, 10) is determined as focus detection data B0 (s, n). The process proceeds to step S310.

In step S310, the pixel data of the focus detection pixel 311 of the latest frame is stored in the internal memory 223 in preparation for the focus detection calculation process in the next frame. This process goes to step S130 in the flow of FIG. At this time, the focus detection data B0 (s, n) is a pair of focus detection image signals A1 _n (A1 ₁ ,..., A1 _M : M is the number of data) of the formula (1), A2 _n (A2 ₁ ,..., A2 _M ).

  FIGS. 10 and 11 show the operation of the processing flowchart shown in FIG. 9 from the viewpoint of data processing. In the graphs shown in FIGS. 10 and 11, the vertical axis represents the data value, and the horizontal axis represents the data position (horizontal position). In FIG. 10, spatial integration data is generated by sequentially integrating the focus detection pixel data from the focus detection pixel row L1. In the figure, for simplicity, only one (s = 1) of the pair of image signals is shown as a representative. FIG. 10 is a diagram showing a state of spatial integration processing for the data of the focus detection pixel 311 of the latest frame. In FIG. 10, when the maximum value of the pixel data of the focus detection pixel 311 of the focus detection pixel row L1 does not exceed the predetermined threshold T1, the data of the focus detection pixels 311 of the focus detection pixel rows L2, L3. If the maximum value of the spatial integration data does not exceed the predetermined threshold value T1 even when the spatial integration is performed and the pixel data of the focus detection pixel 311 is spatially integrated up to the focus detection pixel row L8, the processing is the time integration processing shown in FIG. move on. In the spatial integration process between a plurality of focus detection pixel columns, nth focus detection pixels having the same pixel position in the horizontal direction of each focus detection pixel column are integrated. The integrated focus detection pixels are arranged at positions adjacent to each other in the vertical direction.

  As shown in FIG. 11, when the maximum value of the spatial integration data of the latest frame does not exceed the predetermined threshold value T1, the spatial integration data of one frame before, two frames before, etc. sequentially increases from the latest frame to the past. The time is accumulated up to 10 frames before. Time integration ends when the maximum value of the time integration data obtained by time integration of the space integration data exceeds a predetermined threshold T1.

  FIG. 12 is a timing chart showing the operation of the operation flowchart shown in FIG. In FIG. 12, pixel data of the imaging pixels and focus detection pixels are read from the imaging element 212 at a constant frame rate (for example, 1/60 seconds). FIG. 12 illustrates four frames from the (N−1) th frame to the (N + 2) th frame. The operation of the Nth frame will be described as a representative. First, image data (pixel data of the imaging pixel 310 and pixel data of the focus detection pixel 311) that has been stored for the Nth frame in the (N−1) th frame is read from the imaging device 212. At the same time, charge accumulation for the (N + 1) th frame starts in the image sensor 212. When the reading of the image data is completed, the live view display image is updated based on the read pixel data of the imaging pixel 310. The focus is based on the pixel data of the focus detection pixel 311 in the Nth frame and the pixel data (signal) of the focus detection pixel 311 before the (N-1) th frame stored at the time of generation of the (N-1) th frame. Detection is performed, and a defocus amount at the time of generating the Nth frame is calculated. When the defocus amount is calculated, focus adjustment is performed according to the defocus amount, and pixel data (signal) of the focus detection pixel 311 of the Nth frame is stored in the internal memory 223. The above operation is repeated every frame.

In the above description, as shown in FIG. 4, a plurality of focus detection pixels 311 are arranged in the horizontal direction in eight rows of focus detection pixel rows L1 to L8, and the pixel data of the focus detection pixels 311 in the vertical direction is the maximum in spatial integration. Although integration is performed up to seven times, the focus detection pixel rows Lp can be arranged in eight or more rows without being limited to this. In this case, the upper limit (maximum number of integrations) of the number of spatial integrations can be determined experimentally so as not to cause a decrease in focus detection accuracy due to a reduction in high-frequency components of the image due to vertical spatial integration. For example, as shown in FIG. 4, when the focus detection pixels 311 are arranged every other row and the pixel pitch is Pa, the upper limit Nmax of the spatial integration count N can be determined so as to satisfy the following equation (13). .
N ≦ Nmax = Ca / (2 · Pa) (13)

  Here, the constant Ca is experimentally determined and is determined by the lowest spatial frequency that can maintain the focus detection accuracy on the planned focal plane. Since the focus detection accuracy increases as the spatial frequency increases, the constant Ca is determined, for example, as the reciprocal of the lowest spatial frequency. For example, when the minimum spatial frequency is 10 lines / mm, the constant Ca is 100 μm. If the pixel pitch Pa is 5 μm, the distance between the focus detection pixels is 10 μm, which is twice that, so that Nmax = 100 μm / 10 μm = 10 times according to the equation (13). Also, the shooting factor that affects the focus detection accuracy with the lowest spatial frequency, such as the amount of image movement, the amount of camera shake, the aperture value of the taking lens, the brightness of the field, the shooting mode of the camera (still subject shooting mode / moving subject) By changing according to the shooting mode, etc., it becomes possible to set the upper limit Nmax of the spatial integration number N suitable for focus detection.

  In the above description, the focus detection pixel data of each frame is stored as it is in the internal memory. However, since the data of the focus detection pixels other than the latest frame are used for the generation of focus detection data after being spatially integrated, the data space integration (eight columns) of the focus detection pixels is performed before the step of storing in the internal memory. After the operation, the spatial integration data may be stored in the internal memory. For example, in FIG. 12, the data of the focus detection pixels of each frame is spatially integrated (eight columns) during the storage signal update processing of each frame, and the spatially integrated data (spatial integration of the Nth frame in the Nth frame) Data) is stored in the internal memory. In the next frame, the spatially accumulated data of the past frame stored in the internal memory is used for focus detection. In this way, it is not necessary to spatially integrate the focus detection pixel data of the past frame every time in the focus detection process, so that the calculation time can be shortened and the capacity of the internal memory can be saved.

  In the above description, in the spatial integration process, the data of the focus detection pixels are added one by one in the vertical direction, and in the time integration process, the past spatial integration data is integrated frame by frame toward the past. Then, it is checked whether or not the maximum value of the data after integration exceeds a predetermined threshold value, and it is determined whether or not to continue the integration process. Such processing has an advantage that it can be applied when a subject pattern different for each focus detection pixel column is formed on the focus detection pixel, or when the brightness of the object scene changes suddenly between frames. In addition, in the control of integration processing as described above, evaluation values representing data characteristics such as the average value of the integration data and the contrast value (difference between the maximum value and the minimum value) can be used in addition to the maximum value of the integration data. .

  In addition, when the spatial distance between focus detection pixels that perform spatial integration is small (in the case of FIG. 4, even when spatial integration is performed from focus detection pixel rows L1 to L8, only 14 pixels are separated in the vertical direction spatially. Therefore, even if it is assumed that the focus detection pixels for spatial integration receive substantially the same subject pattern, no large error occurs. Further, if the maximum number of times of integration is set to about 10 as in step S300 of FIG. 9, the time integration is 1/6 second even if 1/60 seconds per frame or 10 times of time integration. Even if it is assumed that there is no brightness fluctuation between them and it is uniform, no big error will occur.

  Assuming such spatial uniformity and temporal uniformity, the process of FIG. 9 can be further simplified as shown in FIG. The internal memory stores spatial integration data obtained by spatially integrating the data of the focus detection pixels of each frame during the storage signal update processing of each frame, that is, spatial integration data B2 (s, n, v) of the vth frame. Suppose that

  In step S400, the CPU 222 of the body drive control device 214 checks whether or not the maximum value of the focus detection pixel data exceeds a predetermined threshold T1 (whether the expression (3) is satisfied).

  If the expression (3) is satisfied in step S400, the process proceeds to step S410, and the CPU 222 of the body drive control device 214 determines the focus detection pixels arranged in the focus detection pixel row L1 as in expression (4). The data is determined as focus detection data. The process proceeds to step S450.

On the other hand, if the expression ( 3 ) is not satisfied in step S400, the process proceeds to step S420, and the CPU 222 of the body drive control device 214 sets the predetermined threshold T1 to the maximum value Max (B (s, n) of the focus detection pixel data. , 1)), it is checked whether or not the integer part Ns of the value is equal to or less than the maximum space integration number 8 (whether or not the expression (14) is satisfied).
Ns <8 (14)

If the expression (14) is satisfied in step S420, the process proceeds to step S430, and the CPU 222 of the body drive control device 214 converts the data of the focus detection pixel 311 to the (Ns + 1) th focus as shown in the expression (15). We calculate the spatial volume to detection pixel row L (Ns + 1). In step S440, the CPU 222 of the body drive control device 214 determines the spatial integration data B1 (s, n, Ns + 1) as focus detection data B0 (s, n). The process proceeds to step S450. In the equation (15), the Σ operation is performed with variables m = 1 to Ns + 1.
B0 (s, n) = B1 (s, n, Ns + 1) = ΣB (s, n, m)
s = 1, 2 n = 1 to N (15)

In step S450, the CPU 222 of the body drive control device 214 calculates the spatial integration data B4 (s, n) to be stored in the internal memory using the equation (16). The process proceeds to step S510. In equation (16), the Σ operation is performed with variables m = 1-8.
B4 (s, n) = B1 (s, n, 8) = ΣB (s, n, m)
s = 1, 2 n = 1 to N (16)

On the other hand, if the expression (14) is not satisfied in step S420, the process proceeds to step S460, and the CPU 222 of the body drive control device 214 sets the predetermined threshold T1 to the maximum value Max (B (s, n) of the focus detection pixel data. 1)), it is checked whether the integer part Nt of the value divided by 8 is 10 or more (whether or not the expression (17) is satisfied).
Nt> 10 (17)

  If the expression (17) is satisfied in step S460, the process proceeds to step S470, and the CPU 222 of the body drive control device 214 determines Nt as 10. The process proceeds to step S480. If the expression (17) is not satisfied in step S460, the process proceeds to step S480 as it is.

  In step S480, the CPU 222 of the body drive control device 214 spatially integrates the data of the focus detection pixel of the latest frame from the focus detection pixel rows L1 to L8 by the equation (16) as in step S450, and spatially integrates the latest frame. Data B2 (s, n, 0) is calculated.

In step S490, the CPU 222 of the body drive control device 214 performs spatial integration up to Nt frames before the spatial integration data B2 (s, n, 0) of the latest frame (assumed to be the Nath frame) and the internal memory. Using the data B2 (s, n, 1) to B2 (s, n, Nt), the time integration data B3 (s, n, Nt) is calculated as in equation (18). In step S500, the CPU 222 of the body drive control device 214 determines the time integration data B3 (s, n, Nt) as focus detection data B0 (s, n). The process proceeds to step S510. In the equation (18), the Σ operation is performed with variables m = 0 to Nt.
B0 (s, n) = B3 (s, n, Nt) = ΣB2 (s, n, m)
s = 1, 2 n = 1 to N (18)

  In step S510, the CPU 222 of the body drive control device 214 stores the spatial integration data of the latest frame in the internal memory in preparation for the focus detection calculation process in the next frame. This process goes to step S130 in the flow of FIG.

  In the processing flow of FIG. 13, the number of times of spatial integration and the number of time integration are determined at the initial stage of processing, and compared with the flow of FIG. 9, there is no determination processing for each loop processing, so the processing time can be shortened. In addition, since the spatial integration data is stored for each frame, the internal memory capacity can be reduced.

  Further, instead of storing the spatial integration data of each frame in the internal memory for each frame, the time integration data up to 10 frames before may be calculated for each frame, and the time integration data may be stored in the internal memory.

  That is, at the time of the storage signal update processing of each frame in FIG. 12, the spatial integration data of the latest frame is time-integrated and updated and stored with respect to the time integration data of 10 frames stored in the previous frame. Thereby, the time integration data stored in the internal memory becomes the time integration data from the latest frame to the previous v frame. FIG. 14 shows a state in which the time integration data stored in the internal memory in this way is used as focus detection data in correspondence with FIG. In the latest frame, the internal memory stores the time integration data one frame before (space integration data), the time integration data from one frame before to two frames before, the integration data from one frame before to three frames before (B2). (S, n, 1), B2 (s, n, 1) + B2 (s, n, 2), B2 (s, n, 1) + B2 (s, n, 2) + B2 (s, n, 3), ...) is recorded. When time integration data B3 (s, n, Nt) obtained by integrating the time from the latest frame to Nt frames before is required, spatial integration data B2 (s, n, 0) of the latest frame and Nt frames before 1 frame before Time integration data (B2 (s, n, 1) + B2 (s, n, 2) + B2 (s, n, 3) +... + B2 (s, n, Nt)) up to Good.

  In this way, when calculating the time integration data used for focus detection, it is only necessary to read out the time integration data for which the time integration from the previous frame to the previous Nt frame has been completed from the internal memory. Increases speed. Further, the processing speed can be further improved by calculating the time integration data and updating and storing the internal memory using a dedicated arithmetic circuit provided separately from the CPU.

  A digital still camera 201 as an imaging device including the focus detection device in the first embodiment described above includes an imaging device (image sensor) 212 and a body drive control device 214.

A plurality of focus detection pixel rows L1 to L8 are juxtaposed on the image sensor 212. The plurality of focus detection pixel rows L1 to L8 are configured by each focus detection pixel row Lp. Each focus detection pixel column Lp includes a plurality of focus detection pixels 311 arranged in the horizontal direction. The plurality of focus detection pixels 311 constituting each focus detection pixel row Lp are arranged in the horizontal direction, and are arranged in parallel with the horizontal direction that is the arrangement direction of the plurality of focus detection pixels 311 in the exit pupil 90 of the interchangeable lens 202. A pair of focus detection light beams 73 and 74 passing through the pair of distance measuring pupils 93 and 94 are received. The imaging device 212 repeatedly outputs a pair of image signals A1 _n and A2 _n corresponding to a pair of images by the pair of focus detection light beams 73 and 74 by photoelectric conversion by each focus detection pixel row Lp at predetermined frame intervals.

The body drive control device 214 causes the focus detection pixels of the focus detection pixel rows L1 to L8 each time the focus detection pixel rows Lp repeatedly output a pair of image signals A1 _n and A2 _n at predetermined frame intervals. Spatial integration data B2 (s, n, v) is calculated by integrating the pair of image signals A1 _n and A2 _n output from the column L1 and one or more focus detection pixel columns adjacent thereto.

The body drive control device 214 calculates the spatial integration data B2 (s, n, v) each time each focus detection pixel column Lp repeatedly outputs a pair of image signals A1 _n and A2 _n at predetermined frame intervals. The time integration data B3 (s, n, v) is calculated by integrating the spatial integration data B2 (s, n, v) from the latest frame to the previous v frame.

  The body drive control device 214 detects the focus state of the interchangeable lens 202 based on one of the space integration data B2 (s, n, v) and the time integration data B3 (s, n, v).

  In the digital still camera 201 as an imaging apparatus including the focus detection apparatus according to the first embodiment of the present invention having the above-described configuration, in the generation of focus detection data, spatial integration (of a plurality of adjacent focus detection pixels) is performed. Data integration) is prioritized, and time integration (past past pixels of the same focus detection pixel) is possible when the focus detection data validity (maximum value, average value, or contrast value is greater than or equal to a predetermined threshold) cannot be ensured by spatial integration alone. Integration with the frame data). For this reason, problems associated with time integration, for example, deterioration in focus detection accuracy due to temporal changes and movement of the subject image can be suppressed to a minimum level, and comfortable automatic focus adjustment can be performed even at low luminance or low contrast.

<Second Embodiment>
In the first embodiment, when integrating focus detection pixel data, spatial integration is always prioritized over time integration. In the second embodiment, the effects of the present invention are achieved by suitably combining space integration and time integration according to the situation of movement of the subject, camera shake, and the like.

  The configuration of the second embodiment is the same as that of the first embodiment, the general operation flow is the same (FIG. 8), and the focus detection data generation process is different. It is assumed that the focus detection pixel data (focus detection pixel rows L1 to L8) of each frame for 10 frames from the latest frame to 10 frames before is updated and stored in the internal memory.

  Details of generation of focus detection data based on focus detection pixel data in step S120 of FIG. 8 according to the second embodiment will be described with reference to a processing flowchart of FIG.

In step S600, the CPU 222 of the body drive control device 214 determines whether or not the maximum value of focus detection pixel data (data of the focus detection pixels 311 arranged in the focus detection pixel row L1 in FIG. 4 ) exceeds a predetermined threshold T1. (Whether (3) is satisfied) is checked.

  If the expression (3) is satisfied in step S600, the process proceeds to step S610, and the CPU 222 of the body drive control device 214 determines the focus detection pixels arranged in the focus detection pixel row L1 as in expression (4). The data is determined as focus detection data. The process proceeds to step S680.

  On the other hand, if the expression (3) is not satisfied in step S600, the process proceeds to step S620, and the CPU 222 of the body drive control device 214 sets the predetermined threshold T1 to the maximum value Max (B (s, n) of the focus detection pixel data. , 1)), the integral part +1 of the value divided by 1)) is calculated as the total number of integrations P.

  In step S630, the CPU 222 of the body drive control device 214 obtains a motion vector between frames based on the image data of the latest frame (imaging pixel data) and the image data of the previous frame by a known method, and the absolute value of the motion vector. And the amount of motion Mv between frames is calculated. It is assumed that the image data of the previous frame is left in the buffer memory or the internal memory.

  In step S640, the CPU 222 of the body drive control device 214 determines the space integration count Ps and the time integration count Pt based on the integration count P and the motion amount Mv. Details of this processing will be described later.

  In step S650, the CPU 222 of the body drive control device 214 determines the focus detection pixel data (focus points from the focus detection pixel row L1 to L (Ps) from the latest frame to the previous (Pt-1) frame based on the spatial integration number Ps. Detection pixel data) is spatially integrated to calculate spatial integration data for Pt frames.

  In step S660, the CPU 222 of the body drive control device 214 further time-integrates the spatial integration data for Pt frames to calculate time integration data from the latest frame to (Pt-1) frames before.

  In step 670, the CPU 222 of the body drive control device 214 determines the time integration data as focus detection data. The process proceeds to step S680.

  In step S680, the CPU 222 of the body drive control device 214 stores the focus detection pixel data of the latest frame in the internal memory in preparation for the focus detection calculation process in the next frame. This process goes to step S130 in the flow of FIG.

  FIG. 16 is a graph for explaining the details of the process of determining the spatial integration number Ps and the time integration number Pt in step S640 of FIG.

In FIG. 16, the horizontal axis represents a variable Px representing the number of times of spatial integration, and the vertical axis represents a variable Py representing the number of times of time integration. The variable Px representing the number of times of spatial integration is limited to 1 ≦ Px ≦ 8 because the focus detection pixel columns L1 to L8 are 8 columns. In addition, since the data up to 10 frames before is updated and stored in the internal memory, the time integration count Pt can be integrated up to a maximum of 11 frames together with the latest frame. That is, it is limited to 1 ≦ Py ≦ 11. Therefore, the combination of the spatial integration number Px and the time integration number Py is limited to a region (allowable range) surrounded by the shaded region in FIG.

  The product of the spatial integration count Px and the time integration count Py is the total integration count P. That is, P = Px · Py. In this way, Px and Py having a function relationship with each other can be determined according to the value of the number of integrations P. FIG. 16 shows an example in the case of P = 88, P0, 1 (1 <P0 <88).

  On the other hand, the function Py = K · Px is determined based on the motion amount Mv. Here, the coefficient K = D / Mv, and the ratio between the pitch D between the focus detection pixel columns (for example, the vertical interval between the focus detection pixel column L1 and the adjacent focus detection pixel column L2) and the motion amount Mv. It is. That is, when the motion amount Mv is large with respect to the pitch D, the amount of blurring of the image due to time integration increases (contrast decreases), and exceeds the influence of the increase of the amount of image blurring due to spatial integration (decrease in contrast). The coefficient K is reduced so as to suppress the time integration number Py.

  On the other hand, when the motion amount Mv is smaller than the pitch D, the amount of blurring of the image due to space integration increases (contrast decreases), and exceeds the influence of the increase of the amount of image blurring due to time integration (decrease in contrast). The coefficient K is increased so as to suppress the number of times of spatial integration Px. FIG. 16 shows the case where the coefficient K is large (K = K0) and the case where it is small (K = K1).

For example, when the total number of times of integration is P0 and the coefficient corresponding to the motion amount Mv is K0, the intersection of the function Px · Py = P0 and the function Py = K0 · Px is obtained. When the intersection coordinates (Px0, Py0) are within the allowable range as shown in FIG. 16, integer intersection coordinates (Pxs, Pyt) that are closest to the intersection coordinates (Px0, Py0) and are within the allowable range are set. The space integration count is Ps = Pxs, and the time integration count is Pt = Pyt .

  When the total number of integrations is P0 and the coefficient corresponding to the motion amount Mv is K1, the function Px · Py = P0 and the intersection coordinates (Px1, Py1) of the function and Py = K1 · Px are allowed. Out of range. In such a case, when the coordinates are moved from the intersection coordinates (Px1, Py1) along the function Px · Py = P0 in the direction of the allowable range, the coordinates when entering the allowable range (in the case of FIG. 16, (8 , P0 / 8)) and the intersection coordinates (Pxs, Pyt) of integers that are closest to each other and within the allowable range are obtained, the number of spatial integrations is Ps = Pxs, and the number of time integrations is Pt = Pyt.

  In summary, under the condition that the product of the number of times of space integration and the number of times of time integration is constant (P0), the value of the coefficient K that is inversely proportional to the amount of motion is such that the value of the number of times of time integration and the number of times of space integration The number of integrations and the number of time integrations are determined. As the constant value P0 that is the product of the number of times of spatial integration and the number of time integrations, an evaluation value (maximum value, average value, contrast value, etc.) that represents the data characteristics of the focus detection data is used in the first embodiment. The predetermined threshold value T1 is set. The space integration number Px and the time integration number Py are set so that the time integration number Py is smaller than the space integration number Px, and the product of the space integration number Px and the time integration number Py is substantially equal to the constant value P0. It is determined to be.

  In FIG. 16, the coefficient K is defined as the ratio between the pitch D and the motion amount Mv. However, the average value of the motion amount Mv over a plurality of frames is Mv1, and the ratio between the pitch D and the motion amount Mv1 (K = D / Mv1). Furthermore, the motion amount in the vertical direction of the motion vector may be defined as Mv2, and the coefficient K may be defined by the ratio between the pitch D and the motion amount Mv2 (K = D / Mv2). Further, the horizontal motion amount of the motion vector may be Mv3, and the coefficient K may be defined by the ratio of the pitch D and the motion amount Mv3 (K = D / Mv3).

  Also, the closer the direction of the motion vector is to the horizontal, the greater the influence of time integration on the focus detection accuracy compared to the effect of spatial integration on the focus detection accuracy, so the coefficient K is set to the vertical motion amount Mv2 and the horizontal direction. May be defined by the ratio of the movement amount Mv3 (K = Mv2 / Mv3).

  The detection of the motion amount is not limited to the detection of the motion vector of the image between frames, and a dedicated motion detection device may be provided. For example, an acceleration sensor may be provided in the body, and the movement (image blur) of the camera in one frame may be detected by time-integrating the output of the acceleration sensor according to the frame interval.

  Furthermore, the coefficient K is not limited to a value according to the amount of motion, and may be a value according to some factor that affects the focus detection accuracy in relation to time integration and space integration. For example, the high-frequency component in the horizontal direction of the image is likely to be reduced by time integration, and the focus detection accuracy is also reduced accordingly. Therefore, the amount of the high-frequency component in the horizontal direction of the image is obtained by image processing, The above-described coefficient K may be defined as a value inversely proportional to the amount of frequency components.

  When the defocus amount is large, the high frequency component is reduced due to the blur. Therefore, the coefficient K described above may be defined as a value proportional to the defocus amount at the time of focus detection.

  Other factors K are used for shooting factors that affect the focus detection accuracy, such as lens driving speed, shooting lens aperture value, brightness of the field, camera shooting mode (separate shooting mode or moving subject shooting mode), etc. By changing accordingly, it is possible to set the number of spatial integration times and the number of time integration times that are suitable for focus detection.

  As described above, in the second embodiment of the present invention, in the generation of focus detection data, the number of times of spatial integration (the number of integration of data of a plurality of adjacent focus detection pixels) and the number of time integration (data of past frames of the same focus detection pixel). The total number of integrations determined according to the evaluation value (maximum value, average value, contrast value, etc.) representing the data characteristics, and the image generated by spatial integration and time integration. Is determined based on the amount of motion of the image so that the blur (contrast reduction) is minimized. Therefore, it is possible to suppress problems associated with data integration (deterioration of focus detection accuracy due to contrast reduction) to a minimum level, and comfortable automatic focus adjustment can be performed even at low luminance or low contrast.

  In the imaging device 212 illustrated in FIG. 4, focus detection pixels 311 each having a pair of photoelectric conversion units are arranged in one pixel, but one photoelectric conversion unit (one of a pair of photoelectric conversion units) is provided in one pixel. The configuration may be such that the first focus detection pixels provided and the second focus detection pixels provided with one photoelectric conversion unit (the other of the pair of photoelectric conversion units) are alternately arranged.

  In the embodiment described above, the pixel data of the focus detection pixel 311 of the focus detection pixel column L1 arranged in the image sensor 212 shown in FIG. 4 and one or more focus detection pixels in the order of proximity to the focus detection pixel column L1. The pixel data of the focus detection pixels 311 in the columns L2 to Lp is spatially integrated. In the image sensor 212 shown in FIG. 4, the focus detection pixel rows L2 to L8 are all located below the focus detection pixel row L1. However, as shown in FIG. 17, focus detection pixel rows L2, L4, L6, and L8 are arranged in the order of approaching the lower side of the focus detection pixel row L1, and the focus detection pixels in the order of approaching the upper side of the focus detection pixel row L1. The imaging device 212 in which the rows L3, L5, and L7 are arranged may be used. Even in this case, spatial integration is performed in the order of proximity to the focus detection pixel column L1, that is, in the order of the focus detection pixel columns L2 to L8.

  In the embodiment described above, the pixel data of the focus detection pixel 311 of the focus detection pixel column L1 arranged in the image sensor 212 shown in FIG. 4 and one or more focus detection pixels in the order of proximity to the focus detection pixel column L1. The pixel data of the focus detection pixels 311 in the columns L2 to Lp is spatially integrated. At this time, as shown in Expression (5), the same type of focus detection pixels 311 adjacent to each other in the vertical direction, that is, the variables n assigned in the horizontal order are arranged at the same position. Data B1 (s, n, p) is calculated by spatial integration performed on the data output by the photoelectric conversion units.

However, as shown in the equation (19), as the focus detection pixel column Lp spatially integrated in the order of proximity to the focus detection pixel column L1 moves away from the focus detection pixel column L1, that is, as the variable p increases, Spatial integration of the data output by the focus detection pixels arranged at positions where the variables n assigned in the arrangement order are shifted one by one, that is, between the photoelectric conversion units of the same type between the focus detection pixels 311 adjacent in the oblique direction. May be performed.
B1 (s, n, p) = B1 (s, n, p-1) + B (s, n + p-1, p)
s = 1, 2 n = 1 to N p = 1 to 8 (19)

Alternatively, as shown in the equations (20) and (21), when the focus detection pixel column Lp is an odd-numbered focus detection pixel column, that is, when the variable p is an odd number, the focus detection pixel column Lp is an even number. The focus detection pixels arranged at the position where the variable n assigned in the order of arrangement in the horizontal direction is shifted by one when it is the first focus detection pixel row, that is, when the variable p is an even number, that is, staggered Spatial integration may be performed on data output by photoelectric conversion units of the same type between focus detection pixels 311 adjacent to each other.
B1 (s, n, p) = B1 (s, n, p-1) + B (s, n, p)
s = 1, 2 n = 1 to N p = 1, 3, 5, 7 (20)
B1 (s, n, p) = B1 (s, n, p-1) + B (s, n + 1, p)
s = 1, 2 n = 1 to N p = 2, 4, 6, 8 (21)

  In the imaging element 212 in the above-described embodiment, an example in which the imaging pixel includes a Bayer color filter is shown, but the configuration and arrangement of the color filter are not limited to this, and a complementary color filter (green: G, yellow). : Ye, magenta: Mg, cyan: Cy) and other arrangements other than the Bayer arrangement.

  In the above-described embodiment, the imaging pixels and the focus detection pixels are mixed on the imaging element. The present invention is applied to the focus detection element as long as the configuration is such that a mirror is arranged in the optical path and the light flux is separated into an image sensor consisting of only the imaging pixels and a focus detection element (image sensor) consisting of only the focus detection pixels. can do.

  Note that the imaging apparatus is not limited to a digital still camera having a configuration in which an interchangeable lens is attached to the camera body as described above. For example, the present invention can be applied to a lens-integrated digital still camera, film still camera, or video camera. Furthermore, the present invention can be applied to a small camera module built in a mobile phone, a surveillance camera, a visual recognition device for a robot, an in-vehicle camera, and the like.

10 microlens, 11, 13, 14 photoelectric conversion unit, 15 element isolation region,
71 shooting light beam, 73, 74 focus detection light beam, 90 exit pupil, 91 optical axis,
93, 94 Distance pupil, 95 area, 101 focus detection position,
201 digital still camera, 202 interchangeable lens, 203 camera body,
204 mount unit, 206 lens drive control device,
208 zooming lens, 209 lens, 210 focusing lens,
211 diaphragm 212 imaging device 213 electrical contact 214 body drive control device
215 liquid crystal display element driving circuit, 216 liquid crystal display element, 217 eyepiece,
219 Memory card, 220 Image sensor control unit, 221 Buffer memory,
222 CPU, 223 internal memory,
310 imaging pixels, 311 focus detection pixels

Claims (16)

A focus detection unit having a plurality of focus detection pixels arranged in a first direction for detecting light transmitted through the optical system and outputting a first focus detection signal and a second focus detection signal . A plurality of first imaging elements provided in a second direction different from the first direction ;
A first addition value obtained by adding the first of the first focus detection signals respectively outputted from the arranged focus detection pixels in the second direction in the focus detection unit number, the first number an adder for calculating a second sum value obtained by adding the second focus detection signals respectively output from the focus detection pixels arranged in the second direction in the focus detection unit,
An in-focus state detection unit that detects an in-focus state of the optical system based on the first addition value and the second addition value ;
The adding unit has a second number up to the first number until the first integrated value based on the first added value and the second added value is greater than a threshold value. focus detection signal with each other and the second number of the second focus detection signal between the focus detecting device you adds. The focus detection apparatus according to claim 1,
The image sensor has a focus detection area,
The focus detection unit of the first number of the focus detection device are found disposed in the focus detection area. The focus detection apparatus according to claim 2,
A time integration unit for calculating a time integration value for the first addition value and the second addition value;
Each of the focus detecting portion of the first number, and wherein the first focus detection signal the second focus detection signal Ri returns output Repetitive,
Before SL addition unit, the first number focus detection unit is the first focus detection signal and the second focus detection signal and the by integrating each of the first sum value and said second output from the Calculate the added value repeatedly,
The time integration unit integrates a third number of the first addition value and the second addition value obtained by repeatedly calculating the first addition value and the second addition value by the addition unit. To calculate the time integrated value,
The focus state detection unit is based on one of the first addition value, the second addition value, and the time integration value for the first addition value and the second addition value. A focus detection device for detecting the in-focus state. The focus detection apparatus according to claim 3,
Each focus detection unit is configured by arranging the plurality of focus detection pixels in the first direction, and is based on the first and second images by the first and second focus detection light beams by photoelectric conversion . first focus detection signal and the second focus detection signal and the focus detection device that outputs, repeatedly a. The focus detection apparatus according to claim 4,
When the focusing state detection unit detects the focus state based on the time integration value, evaluation value based on the said first focus detection signal the second focus detection signal is greater than said threshold value The focus detection apparatus in which the second number and the third number are determined as follows. The focus detection apparatus according to claim 5,
When the in-focus state detection unit detects the in-focus state based on the first addition value and the second addition value, the evaluation value becomes the first addition value and the second addition value. a first integration value based, said third number with defined range, wherein the first integrated value is the second number to be greater than before Ki閾 value does not exceed the number of the first When the in-focus state detection unit detects the in-focus state based on the time integration value, the evaluation value is a second integration value based on the time integration value, and the second integration value is determined. value before Ki閾 value the third focus detection device number is determined together with determined number the second number the first to be greater than. The focus detection apparatus according to claim 6,
Further comprising a storage device for storing a first focus detection signal and the second focus detection signals each focus detecting unit repeatedly outputs,
The adding unit adds the first focus detection signal and the second focus detection signal stored in the storage device to calculate the first addition value and the second addition value, respectively. Focus detection device. The focus detection apparatus according to claim 6,
The first addition value and the second addition value that are repeatedly calculated by the addition unit based on the first focus detection signal and the second focus detection signal output by each of the focus detection units are stored. Further comprising a storage device,
The focus detection device, wherein the time integration unit calculates the time integration value by adding the third addition value and the second addition value of the third number stored in the storage device. The focus detection apparatus according to claim 6,
Further comprising a storage device for storing said time integrated value calculated by the previous SL-time integrator,
The focus detection device detects the focus state based on the time integrated value stored in the storage device when the focus state detection unit detects the focus state based on the time integrated value. The focus detection apparatus according to claim 5,
A motion amount detection unit for detecting a motion amount between frames of an image formed on the image sensor;
When the in-focus state detection unit detects the in-focus state based on the time integrated value, the third number is smaller than the second number as the amount of movement is larger. and the like product of the second number and said third number is substantially equal to a constant value corresponding to said evaluation value and the previous Ki閾 value, said second number of said third number Fixed focus detection device. In the focus detection apparatus according to any one of claims 6 to 9,
The first integrated value and the second integrated value are an average value of the first added value and the second added value, an average value of the time integrated value, the first added value, and the second added value. The maximum value of the addition value and the maximum value of the time integration value, the difference between the maximum value and the minimum value of the first addition value and the second addition value, and the difference between the maximum value and the minimum value of the time integration value A focus detection device corresponding to any one of the above. The focus detection apparatus according to claim 10, wherein
The evaluation value includes an average value of the first focus detection signal and the second focus detection signal, a maximum value of the first focus detection signal and the second focus detection signal, and the first value. The focus detection apparatus corresponding to any one of the difference between the maximum value and the minimum value of the focus detection signal of the second focus detection signal and the second focus detection signal. In the focus detection apparatus according to any one of claims 4 to 12,
Each of the plurality of focus detection pixel includes a microlens transmitted through the first and second focus detection light fluxes, aligned in the first direction, by receiving the first and second focus detection light fluxes A first and a second photoelectric conversion unit for performing the photoelectric conversion;
The first and second photoelectric conversion units, and first and second regions arranged in parallel with the first direction in an exit pupil of the optical system through which the first and second focus detection light beams pass. However, the focus detection device is conjugated with each other by the microlens. The focus detection apparatus according to claim 13.
The plurality of imaging pixels that output the object image signals based on the subject image by photoelectric conversion by receiving photographing light flux from a subject passing through the optical system, disposed in the imaging element mixed with the plurality of focus detection pixels Focus detection device. The focus detection apparatus according to claim 4,
Wherein the plurality of focus detection pixels constituting the respective focus detection unit, the first direction and the first and second through the first and second areas arranged in parallel of the exit pupil of the optical system Receives the focus detection beam,
The focus detection unit of the first number of the focus detection device are juxtaposed to each other. The focus detection apparatus according to any one of claims 1 to 15,
A drive unit that drives the optical system to a focus position based on the focus state detected by the focus state detection unit;
An imaging apparatus comprising: an acquisition unit that acquires image data based on a photographing light beam from a subject passing through the optical system when the optical system is located at the in-focus position.

Images