JP2011252955A - Imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent increase of a focus detection error or impossibility of focus detection which is oppositely caused by accumulated focus detection pixel data of a current frame and a past frame for achieving a level suitable for a focus detection arithmetic processing.SOLUTION: A body drive control device in a digital still camera determines whether an absolute value of difference between a focal length data at an imaging time of the current frame and a focal length data at an imaging time of the past frame is less than or equal to a threshold value or not (S170). If the absolute value is less than or equal to the threshold value, then the focus detection pixel data of the current frame or the already accumulated focus detection pixel data is added to the focus detection pixel data of the past frame, and if the absolute value exceeds the threshold value, then such addition is prohibited (S180).

Description

  The present invention relates to an imaging apparatus capable of performing focus detection based on an output of an imaging element.

  An image pickup apparatus having an image pickup element including an image pickup pixel and a pupil detection type phase difference detection type focus detection pixel is disclosed (for example, see Patent Document 1). In this imaging apparatus, the focus detection pixel data read together with the imaging pixel data periodically is stored for each readout, and the focus detection pixel data stored for each readout is integrated, and the integrated focus detection pixel data is stored. A focus detection is performed by performing a phase difference detection calculation based on the data. By integrating the data of the focus detection pixels in this way, it is possible to prevent the focus detection pixel data level obtained at a predetermined frame rate from being lowered and becoming incapable of focus detection at low luminance.

JP 2008-85738 A

  However, in the imaging apparatus described above, since integration is performed under the condition that the data level of the focus detection pixel exceeds a predetermined level by integration, the data of the integrated focus detection pixel is inappropriate for focus detection depending on the imaging state. Data. For example, the case where the optical conditions change or the manner in which the user holds the imaging apparatus changes corresponds to the case where the imaging state changes. In such a case, there is a problem that the focus detection error increases or the focus detection becomes impossible due to the integration.

  The imaging device according to claim 1, wherein a plurality of pixels are arranged, an image of a subject is captured by the plurality of pixels, and pixel data corresponding to each of a plurality of continuous frames in time series is generated. An imaging optical system that forms an image of the subject on the imaging device, imaging state information acquisition means for acquiring imaging state information relating to the imaging state at the time of imaging the subject image, and at least a part of the pixel data Storage means for storing data and the imaging state information in association with each other, and at least one portion stored by the storage means in the partial data included in the latest pixel data among the pixel data generated by the imaging device Based on the imaging state information stored in the storage unit, an adding unit that performs addition processing for adding data to calculate the added partial data, And inhibiting means for performing inhibit control to prohibit the addition of the partial data to be communicated, based on the addition partial data, characterized by comprising a focus detection means for detecting a focusing state of the photographic optical system.

  According to the imaging apparatus of the present invention, since the integration control is performed so that the integrated focus detection pixel data becomes data suitable for focus detection, highly accurate focus detection is possible.

It is a figure which shows the structure of the digital still camera of one embodiment. It is a figure which shows the focus detection position on an imaging | photography screen. It is a front view which shows the detailed structure of an image pick-up element. It is a figure which shows the relationship between the defocus in a pupil division type phase difference detection system, and image shift. It is a figure which shows the relationship between an imaging pixel and an exit pupil. It is a flowchart which shows the imaging operation of a digital still camera. It is a figure for demonstrating the relationship of the focus detection pixel data and focal distance data which are obtained synchronizing with the output of a flame | frame and each flame | frame, and the memory | storage operation | movement to a memory | storage device. It is a figure showing the effect by data addition. It is the figure which showed typically the process in which the past focus detection pixel data are integrated | accumulated sequentially based on the newest focus detection pixel data. It is the table | surface which classified the imaging state. It is a front view which shows the detailed structure of an image pick-up element.

  As an imaging apparatus according to an embodiment, an interchangeable lens digital still camera will be described as an example. FIG. 1 is a cross-sectional view showing a configuration of a digital still camera according to an embodiment. A digital camera 201 according to the present embodiment includes an interchangeable lens 202 and a camera body 203, and the interchangeable lens 202 is attached to the camera body 203 via a mount unit 204. An interchangeable lens 202 having various photographing optical systems can be attached to the camera body 203 via a mount unit 204.

  The interchangeable lens 202 includes a lens 209, a zooming lens 208, a focusing lens 210, a diaphragm 211, a lens drive control device 206, and the like. The lens drive control device 206 includes a microcomputer (not shown), a memory, a drive control circuit, and the like. The lens drive control unit 206 performs drive control for adjusting the focus of the focusing lens 210 and adjusting the aperture diameter of the aperture 211, and detecting the states of the zooming lens 208, the focusing lens 210, and the aperture 211. Further, transmission of lens information and reception of camera information (defocus amount, aperture value, etc.) are performed by communication with a body drive control device 214 described later. The aperture 211 forms an aperture having a variable aperture diameter at the center of the optical axis in order to adjust the amount of light and the amount of blur.

  The camera body 203 includes an image sensor 212, a body drive control device 214, a liquid crystal display element drive circuit 215, a liquid crystal display element 216, an eyepiece lens 217, a memory card 219, and the like. In the image sensor 212, image pickup pixels are two-dimensionally arranged (rows and columns), and focus detection pixels are incorporated in portions corresponding to focus detection positions (focus detection areas). Details of the image sensor 212 will be described later.

  The body drive control device 214 includes a microcomputer, a memory, a drive control circuit, and the like. The body drive control device 214 repeatedly performs exposure control of the image sensor 212 and readout of the pixel signal from the image sensor 212, focus detection calculation based on the pixel signal of the focus detection pixel, and focus adjustment of the interchangeable lens 202, Processing and recording of image signals, camera operation control, etc. The body drive control device 214 communicates with the lens drive control device 206 through the electrical contact 213 to receive lens information and transmit camera information.

  The liquid crystal display element 216 functions as an electric view finder (EVF). The liquid crystal display element driving circuit 215 displays a through image on the liquid crystal display element 216 based on the image data read from the image sensor 212, and the photographer can observe the through image through the eyepiece 217. The memory card 219 is an image storage that stores image data captured by the image sensor 212.

  The AD conversion device 221 performs AD conversion on the pixel signal output from the image sensor 212 and sends it to the body drive control device 214. The imaging device 212 may have a configuration in which the AD conversion device 221 is incorporated.

  A subject image is formed on the image sensor 212 by the light beam that has passed through the interchangeable lens 202. This subject image is photoelectrically converted by the image sensor 212, and the pixel signals of the imaging pixels and focus detection pixels are sent to the body drive control device 214.

  The body drive control device 214 calculates the defocus amount based on the pixel signal (focus detection signal) from the focus detection pixel of the image sensor 212 and sends this defocus amount to the lens drive control device 206. In addition, the body drive control device 214 processes the pixel signal (imaging signal) of the imaging pixel of the imaging element 212 to generate image data, stores the image data in the memory card 219, and reads the through image read from the imaging element 212. A signal is sent to the liquid crystal display element driving circuit 215 to display a through image on the liquid crystal display element 216. Further, the body drive control device 214 sends aperture control information to the lens drive control device 206 to control the aperture of the aperture 211.

  The lens drive controller 206 updates the lens information according to the focusing state, zooming state, aperture setting state, aperture opening F value, and the like. Specifically, the positions of the zooming lens 208 and the focusing lens 210 and the aperture value of the aperture 211 are detected, and lens information is calculated according to these lens positions and aperture values, or a lookup prepared in advance. Lens information corresponding to the lens position and aperture value is selected from the table.

  The lens drive control device 206 calculates a lens drive amount based on the received defocus amount, and drives the focusing lens 210 to the in-focus position according to the lens drive amount. Further, the lens drive control device 206 drives the diaphragm 211 in accordance with the received diaphragm value.

  The storage device 220 stores focus detection pixel data, which will be described later, for a plurality of past frames. The body drive control device 214 stores the pixel data of the focus detection pixels in each frame read from the image sensor 212 in the storage device 220. Further, when the focus detection calculation process is performed using the pixel data of the focus detection pixel, if the output level of the pixel data of the focus detection pixel of the latest frame is insufficient, the past data stored in the storage device 220 is stored. Add with pixel data. In this way, the output level of the pixel data of the focus detection pixel is made appropriate, and the focus detection process described later is performed using the pixel data of the focus detection pixel after the addition.

  FIG. 2 shows a focus detection position on the photographing screen according to an embodiment, that is, a region (focus detection area) where a subject image is sampled on the photographing screen when a focus detection pixel column described later performs focus detection. In this embodiment, a focus detection area 101 is arranged at the center of the shooting screen 100. The focus detection pixels are linearly arranged in the longitudinal direction of the focus detection area 101 indicated by a rectangle.

  FIG. 3 is a front view showing a detailed configuration of the image sensor 212, and is an enlarged view of the area corresponding to the vicinity of the above-described focus detection area 101 on the image sensor 212. FIG. In FIG. 3, the vertical and horizontal directions (pixel rows and columns) correspond to the vertical and horizontal directions of the photographing screen 100 in FIG. The imaging element 212 includes an imaging pixel 310 for imaging and focus detection pixels 312 and 313 for focus detection. In a region corresponding to the focus detection area 101, focus detection pixels 312 and 313 are alternately arranged in the horizontal direction. The focus detection pixels 312 and 313 are linearly arranged in the row where B and G of the imaging pixel 310 are to be arranged, and form a focus detection pixel column.

  The imaging pixel 310 includes the microlens 10, the photoelectric conversion unit 11, and a color filter (not shown). The color filters include three types of red (R), green (G), and blue (B), and have spectral sensitivity characteristics corresponding to the respective colors. In the image pickup element 212, the image pickup pixels 310 having such color filters are arranged in a Bayer array.

  The focus detection pixel 312 includes the microlens 10 and the photoelectric conversion unit 12. The photoelectric conversion unit 12 has a rectangular shape in which the left side is substantially in contact with the vertical bisector of the microlens 10. The focus detection pixel 313 includes the microlens 10 and the photoelectric conversion unit 13. The photoelectric conversion unit 13 has a rectangular shape in which the right side is substantially in contact with the vertical bisector of the microlens 10. When the photoelectric conversion units 12 and 13 are displayed with being overlapped with the microlens 10 as a reference, the photoelectric conversion units 12 and 13 are arranged in the left-right horizontal direction and have a symmetrical shape with respect to the vertical bisector of the microlens 10. The focus detection pixels 312 and the focus detection pixels 313 are alternately arranged in the horizontal direction, that is, the arrangement direction of the photoelectric conversion units 12 and 13.

  The focus detection pixels 312 and 313 are not provided with a color filter in order to increase the amount of light. The focus detection pixels 312 and 313 include a spectral sensitivity characteristic of a photodiode that performs photoelectric conversion and an infrared cut filter (not shown). It has spectral sensitivity characteristics that combine the spectral sensitivity characteristics. That is, the spectral sensitivity characteristics of the focus detection pixels 312, 313 are spectral sensitivity characteristics obtained by adding the spectral sensitivity characteristics of the green pixel, the red pixel, and the blue pixel, and the light wavelength region in which the focus detection pixels 312, 313 exhibit high sensitivity. Includes a light wavelength region in which each color filter exhibits high sensitivity in each of a green pixel, a red pixel, and a blue pixel.

  The photoelectric conversion unit 11 of the imaging pixel 310 is designed to receive all the light flux that passes through the exit pupil (for example, F1.0) of the brightest interchangeable lens by the microlens 10. On the other hand, the photoelectric conversion units 12 and 13 of the focus detection pixels 312 and 313 are designed to receive a light beam that passes through a predetermined region (for example, F2.8) of the exit pupil of the interchangeable lens by the microlens 10. . By using the focus detection pixel 311, focus detection of the pupil division method disclosed in Japanese Patent Application Laid-Open No. 2008-85738 can be performed.

  A large number of two types of focus detection pixels as described above are arranged in a straight line, and the output of the photoelectric conversion unit of each pixel is output to an output group corresponding to a pair of distance measuring pupils disclosed in Japanese Patent Application Laid-Open No. 2008-85738. In summary, information on the intensity distribution of the pair of images formed on the pixel array by the pair of focus detection light beams passing through the pair of distance measuring pupils is obtained. By applying an image shift detection calculation process (correlation calculation process, phase difference detection process), which will be described later, to this information, an image shift amount of a pair of images is detected by a so-called pupil division type phase difference detection method. Note that the direction in which the pair of distance measuring pupils is aligned is the same as the direction in which the pair of photoelectric conversion units 12 and 13 are aligned, that is, the direction in which the focus detection pixels 312 and 313 are aligned.

  Further, by performing a conversion operation according to the center of gravity distance between the pair of distance measuring pupils on the image shift amount, the current imaging plane (the focus detection on the imaging screen 100) with respect to the planned imaging plane (the position of the microlens array). The deviation (defocus amount) of the actual image plane at the position is calculated. In practice, the distance measuring pupil has a shape and size limited by the aperture opening.

  FIG. 4 is a diagram showing the relationship between defocus and image shift in the pupil division type phase difference detection method. In FIG. 4A, light beams that are divided into distance pupils 92 and 93 on the exit pupil plane 90 of the optical system and form an image are a light beam 72 that passes through the distance pupil 92 and a light beam that passes through the distance pupil 93. 73. With such a configuration, for example, when a line pattern (a white line on a black background) on the optical axis 91 and perpendicular to the paper surface of FIG. 4 is imaged by the optical system, the distance measuring pupil 92 is formed on the focusing plane P0. The light beam 72 passing through and the light beam 73 passing through the distance measuring pupil 93 form a high-contrast line image pattern at the same position on the optical axis 91 as shown in FIG.

  On the surface P1 ahead of the in-focus plane P0, the light beam 72 passing through the distance measuring pupil 92 and the light beam 73 passing through the distance measuring pupil 93 are blurred line image patterns as shown in FIG. 4B. Form. On the surface P2 behind the focusing plane P0, the light beam 72 passing through the distance measuring pupil 92 and the light beam 73 passing through the distance measuring pupil 93 are as shown in FIG. 4 (b) as shown in FIG. Forms blurred line image patterns at different positions in the opposite direction. Two images formed by the light beam 72 passing through the distance measuring pupil 92 and the light beam 73 passing through the distance measuring pupil 93 are separated and detected. By calculating the relative positional relationship (image shift amount) between the two images, it is possible to detect the focus adjustment state (defocus amount) of the optical system on the surface where the two images are detected.

  FIG. 5 is a diagram illustrating the relationship between the imaging pixels and the exit pupil. The exit pupil plane 90 of the optical system is at a position separated from the microlens 70 disposed in the vicinity of the planned imaging plane of the optical system by the distance measurement pupil distance d. The photoelectric conversion unit 71 of the imaging pixel behind the micro lens 70 is formed on the semiconductor circuit substrate 29 and receives the imaging light beam 81. The region 94 is a region formed by a projection shape in which the shape of the photoelectric conversion unit 71 is projected by the microlens 70. 5 schematically illustrates the imaging pixels (having the microlens 70 and the photoelectric conversion unit 71) on the optical axis 91, but in other imaging pixels, the photoelectric conversion units are respectively connected from the region 94 to the respective micros. Receives the light flux coming into the lens.

  The photoelectric conversion unit 71 outputs a signal corresponding to the intensity of an image formed on the microlens 70 by the imaging light beam 81 that passes through the region 94 and travels toward the microlens 70. Many image pickup pixels as described above are arranged two-dimensionally, and image data is obtained based on signals output from the photoelectric conversion units of the respective pixels. In the above description, the region 94 is described as being not limited by the aperture opening. However, the region 94 actually has a shape and size limited by the aperture opening.

  FIG. 6 is a flowchart illustrating an imaging operation of the digital still camera 201 according to the embodiment. When the power of the digital still camera 201 is turned on in step S100, the body drive control device 214 starts an imaging operation after step S110. In step S <b> 100, the image sensor 212 is set to an operation mode in which the body drive control device 214 repeats the imaging operation at a constant period. In this operation mode, the image sensor 212 outputs, for example, 60 frames per second.

  In step S110, the body drive control device 214 communicates with the lens drive control device 206 in synchronization with the output of the frame, acquires the focal length data of the interchangeable lens 202, and stores the focal length data in the storage device 220. .

  In step S120, the body drive control device 214 reads out all pixel data for one frame from the image sensor 212 in synchronization with the output of the frame.

  In step S <b> 130, pixel interpolation is performed on virtual imaging pixel data (imaging signal) at each pixel position in the focus detection pixel row based on imaging pixel data around the focus detection pixel.

  For example, for a focus detection pixel that is originally arranged at a position where a green pixel should be arranged, the data of four green imaging pixels adjacent to four positions in the diagonal direction of the focus detection pixel are averaged, and the focus detection pixel is averaged. Data of the green imaging pixel at the position of the detection pixel is used. For the focus detection pixel that is originally arranged at the position where the blue pixel should be arranged, the data of the two blue imaging pixels adjacent to the two vertical positions of the focus detection pixel are averaged to detect the focus detection pixel. It is assumed that the data of the blue imaging pixel at the pixel position.

  Thus, the image data corresponding to the current frame is generated by synthesizing the imaging pixel data obtained by the interpolation at the position of the focus detection pixel and the original imaging pixel data at the position of each imaging pixel. For example, the body drive control device 214 is configured to display a through image based on image data of frames generated continuously in time series at a frame rate of 60 frames per second on the liquid crystal display element 216. The drive circuit 215 is controlled.

  In step S140, the body drive control device 214 displays image data corresponding to the current frame on the electronic viewfinder (live view display).

  In step S150, the body drive control device 214 causes the storage device 220 to store the focus detection pixel data in the image data read this time in association with the focal length data.

  Here, the relationship between the frame, the focus detection pixel data and the focal length data obtained in synchronization with the output of each frame, and the storage operation in the storage device 220 (ring buffer) will be described with reference to FIG.

  The upper half of FIG. 7 shows focus detection pixel data I (N) to I (N-7) from the latest frame N to the past seven frames (up to frame (N-7)) when time is taken on the horizontal axis. And the focal length data F (N) to F (N-7). As shown in the lower half of FIG. 7, the correspondence relationship is stored in association with each memory area of the storage device 220 constituting the ring buffer. The pointer indicates the position (address) of the memory area in which the latest focus detection pixel data and focal length data are stored.

  The past frame is traced back to the previous 7 frames. As described above, when the frame rate is 60 frames / second, when going back to the past 8 frames, the past frame is traced back by 120 ms or more. This is because it can be ignored. Therefore, for example, as long as the influence of dark current is negligible, or depending on the frame rate, it is not necessarily limited to going back to the past seven frames.

  In step S160 to step S180, when the data level of the latest focus detection pixel data is insufficient, the data is adjusted so that the data level is appropriate by performing addition with the past focus detection pixel data. The focus detection calculation is performed using the focus detection pixel data after the addition at the data level.

  In step S160, if the focus detection pixel data satisfies a predetermined condition, that is, if the focus detection pixel data has reached a level suitable for an image shift detection calculation process described later, or the integration in step S180 is performed for the past seven frames. If it has been performed, the integration process is skipped and the process proceeds to step S190. Here, the focus detection pixel data is initially the focus detection pixel data of the latest frame, but is updated to the focus detection pixel data after integration after the integration processing. The final data in the figure indicates the oldest focus detection pixel data I (N-7).

  FIG. 8 is a diagram showing the effect of data addition with the horizontal axis as the focus detection pixel position and the vertical axis as the data level of the focus detection pixel data. First, the maximum value of the focus detection pixel data of the current frame is detected. If the maximum value does not exceed the threshold value, the previous focus detection pixel data is read from the storage device 220 and added. If the maximum value of the added focus detection pixel data (addition data) does not exceed the threshold, the previous focus detection pixel data is further added. By repeating such an addition operation until the maximum value of the addition data exceeds the threshold value, the addition data reaches a data level suitable for the focus detection calculation process.

  The threshold value is, for example, ½ of the maximum value of the dynamic range of AD conversion. However, even if it is an average value, for example, the maximum value of the added data does not exceed the maximum value of the dynamic range of AD conversion, and It may be sufficient if the saturation value of the pixel signal value is not reached, or may be another condition value. If this threshold value is increased too much, the addition time becomes longer, and changes in the imaging state such as a change in the subject position within the frame are likely to occur during the addition time, and high-precision focus detection by integration processing can be hindered. If the threshold value is too small, the added data may not reach a data level suitable for the focus detection calculation process. Therefore, it is preferable to determine a condition value considering them as a threshold value by experiment.

  In the digital still camera 201, since the frame rate is fixed to a fixed value (60 frames / second) as described above, the exposure time of the image sensor 212 cannot be 1/60 seconds or more. If the frame rate is reduced in order to increase the data level of the focus detection pixel data, the readout interval of the imaging pixel data becomes the same frame rate in the imaging element 212 in which the focus detection pixels and the imaging pixels are mixedly arranged. It is not preferable. Accordingly, when the frame rate is suitable for the readout interval of the imaging pixel data, the focus detection pixel data for one frame may be at a low level at low luminance and the focus detection may not be performed. The processing is effective for highly accurate focus detection.

  If the integration has not reached up to the past seven frames in step S160 and the focus detection pixel data has not reached a level suitable for image shift detection calculation processing described later, the process proceeds to step S170.

  In step S170, the body drive control device 214 stores the focal length data when the current (latest) focus detection pixel data is obtained and the focal length data when the past focus detection pixel data is obtained. Read from device 220 and compare. That is, it is determined whether or not the absolute value of the difference between the latest focal length data and the past focal length data exceeds a threshold value. If the absolute value exceeds the threshold value, the process returns to step S160. Advances to step S180. The case where the absolute value of the difference between the latest focal length data and the past focal length data does not exceed the threshold value is when the change in the focal length between the latest frame and the past frame is small and the change in the imaging state is small. include.

  If this threshold value is too large, it may occur that the past frames to be added include a plurality of past frames that are imaged under different conditions with a large focal length. That is, a change in the imaging state is likely to occur between the plurality of past frames, and high-precision focus detection by integration processing can be hindered. If the threshold value is too small, the added data may not reach a data level suitable for the focus detection calculation process. Therefore, it is preferable to determine a condition value considering them as a threshold value by experiment.

  In step S180, the focus detection pixel data related to the focal length data satisfying the condition of step S170 is sequentially integrated with the latest focus detection pixel data, and the process returns to step S160.

  FIG. 9 is a diagram schematically showing the above-described processing. The focus detection pixel data I (N) to I (N-7) and the focal length data F from the latest frame N to the last frame (N-7) are shown. (N) -F (N-7) correspondence, and past focus detection pixel data I (N-1) -I (N-7) integration allowance (◯), prohibition (×) correspondence relationship. Show. As shown in FIG. 9, focus detection pixels corresponding to focal length data F (N-6) and F (N-7) whose absolute value of the difference from the focal length data F (N) of the latest frame exceeds a threshold ΔF. Data I (N-6) and I (N-7) are not used for integration.

  In the processing loop of step S160 to step S180, the past data (focus detection pixel data, focal length data) is determined according to the pointer indicating the address of the storage device 220, the previous frame (N-1) → the previous frame (N-2) → ... Read and process in order of last frame (N-7). For example, in the case shown in FIG. 9, since the focus detection pixel data I (N-6) and I (N-7) are not used for integration as described above, the focus detection pixel data finally obtained. Becomes I (N) + I (N-1) + I (N-2) + I (N-3) + I (N-4) + I (N-5).

  In step S170, when the absolute value of the difference between the latest focal length data and the past focal length data exceeds the threshold value, the process returns to step S160. However, the process loops out and the process proceeds to step S190. Good.

  In step S190, focus detection calculation processing described later is performed based on the focus detection pixel data obtained in the integration processing loop of steps S160 to S180 (no integration when the latest focus detection pixel data has reached an appropriate level). The defocus amount in the focus detection area is calculated.

  In step S200, it is determined whether or not the focus is in focus (the absolute value of the defocus amount is equal to or less than a predetermined threshold). If the focus is in focus, the process proceeds to step S220. If the focus is not in focus, the process proceeds to step S210.

  In step S210, the body drive control device 214 transmits the defocus amount to the lens drive control device 206, and drives the focusing lens 210 of the interchangeable lens 202 to the in-focus position. When the defocus amount reliability is low or focus detection is not possible, the body drive control device 214 transmits that fact to the lens drive control device 206 and does not update the drive control of the focusing lens 210 of the interchangeable lens 202. . Thereafter, the process proceeds to step S220.

  In step S220, it is determined whether or not a shooting instruction has been given by an operating means (not shown). If no shooting instruction has been given, the process returns to step S180. If a shooting instruction has been given, the image data corresponding to the current frame is stored in the memory card 219 in step S230, and the process returns to step S110. .

  As a general image shift detection calculation process (correlation calculation process) used in step S190 of FIG. 6, a correlation calculation process disclosed in Japanese Patent Application Laid-Open No. 2007-333720 is used. The pair of images detected by the focus detection pixels is a type that can maintain the image displacement detection accuracy with respect to the light amount balance because the distance measurement pupil may be vignetted by the aperture of the lens and the light amount balance may be lost. The correlation calculation is performed.

For a pair of data strings (A1 _{1 to} A1 _M , A2 _{1 to} A2 _M : M is the number of data) read out from the focus detection pixel string, a correlation calculation formula (1) disclosed in Japanese Patent Laid-Open No. 2007-333720 ) To calculate the correlation amount C (k). Note that the pair of data strings (A1 _{1 to} A1 _M , A2 _{1 to} A2 _M ) includes the case where the focus detection pixel data used for the calculation is the focus detection pixel data after integration as described above.
C (k) = Σ | A1 n × A2 n + 1 + k -A2 n + k × A1 n + 1 | (1)

The Σ operation is accumulated for n. The range taken by n is limited to the range in which the data of A1 _n , A1 _{n + 1} , A2 _{n + k} , A2 _{n + 1 + k} exists according to the image shift amount k. The image shift amount k is an integer and is a relative shift amount with the data interval of the data string as a unit.

According to the calculation method disclosed in Japanese Patent Application Laid-Open No. 2009-141791, the image shift amount shft is calculated by Expression (2) using the image shift amount ks that gives the minimum value C (ks) of the correlation amount C (k). Can be calculated. In Expression (2), PY is twice the pixel pitch of the focus detection pixels (detection pitch).
shft = PY × ks (2)

The image shift amount calculated by the equation (2) is multiplied by a predetermined conversion coefficient Kd to be converted into a defocus amount def. The conversion coefficient Kd corresponds to the opening angle of the pair of light beams received by the focus detection pixel, and corresponds to a value obtained by dividing the distance measurement pupil distance d by the center of gravity distance between the pair of distance measurement pupils.
def = Kd × shft (3)

  As described above, in the present invention, when the focal length data when the focus detection pixel data of the past frame is acquired is different from the focal length data when the focus detection pixel data of the latest frame is acquired, It is prohibited to integrate the data of the focus detection pixels of the frame. As a result, it is possible to avoid problems such as an increase in focus detection error or focus detection failure caused by focus detection using focus detection pixel data obtained by superimposing images with different magnifications. The focus detection pixel data of the past frame corresponding to the focal length data in a range not far from the focal length data when the focus detection pixel data of the latest frame is acquired are integrated. As a result, highly accurate focus detection is possible.

  In the embodiment described above, focus detection is performed using focus detection pixel data in one focus detection area at the center of the screen, but the present invention is not limited to this. A plurality of focus detection areas may be provided on the screen, and focus detection may be performed by selecting one of the plurality of focus detection areas, or focus detection may be performed simultaneously in the plurality of focus detection areas.

  In the above-described embodiment, the change in focal length is cited as a condition for prohibiting the integration of focus detection pixel data of past frames. However, the present invention is not limited to this, and various imaging states that affect focus detection. (Imaging conditions) can be applied.

  FIG. 10 is a table in which such imaging states are classified. As the optical characteristics of the photographing optical system included in the interchangeable lens 202, in addition to the focal length, a diaphragm aperture F value (diaphragm aperture diameter) and a position of the focusing lens 210 are shown. (Distance from the planned imaging plane to the focusing lens 210), PO value (exit pupil distance), and the like.

  When the aperture value F changes, the conversion coefficient Kd in Equation (3) changes, so the image shift amount changes even with the same defocus amount, and past focus detection pixel data is integrated. It is not preferable to do so.

  It is conceivable that the user changes the position of the focusing lens 210 during the focus detection pixel data addition process. When the position of the focusing lens 210 is different, since the defocus amount has changed in the first place, it is not preferable to integrate past focus detection pixel data.

  The body drive control device 214 can acquire the PO value from the lens drive control device 206. When the focal length of the photographing optical system, the stop aperture F value, or the position of the focusing lens 210 changes, the exit pupil distance, that is, the PO value can also change. When the PO values are different, the conversion coefficient Kd corresponding to the opening angle of the pair of light beams received by one or a plurality of focus detection pixels changes, so that past focus detection pixel data is integrated. Is not preferred.

  The body drive control device 214 receives data of the aperture aperture F value (diaphragm aperture diameter), the position of the focusing lens 210, and the PO value (exit pupil distance) from the lens drive control device 206 through communication.

  The body drive control device 214 may prohibit the integration of the focus detection pixel data of the past frame according to the posture information of the digital still camera 201.

  The body drive control device 214 detects posture change (composition change) between the latest frame and the past frame by using posture data detected by a posture sensor (not shown). In many cases, a posture change such as a composition change is caused by a change in the orientation and orientation of the digital still camera 210 by the user, that is, a movement of the digital still camera 210 intended by the user. The posture sensor is, for example, a vertical sensor, a direction sensor, or an angular velocity sensor. If it is determined that the composition has changed significantly as a result of the posture change detection, it is determined that the focus detection pixels receive different images in different frames, and the integration of past focus detection pixel data is prohibited.

  The posture information of the digital still camera 201 includes shake data of a shake detection sensor (not shown) in addition to the posture data detected by the posture sensor. The blur is caused by the movement of the digital still camera 210 not intended by the user. The body drive control device 214 uses the blur data detected by the blur detection sensor, and prohibits the integration of past focus detection pixel data when the blur amount of the past frame exceeds the threshold. This is because the focus detection accuracy decreases even if the blurred images are integrated.

  The body drive control device 214 may prohibit the integration of the focus detection pixel data of the past frame according to the information on the subject to be imaged.

  When the posture data or the blur data changes as described above, in many cases, a luminance change occurs between the latest frame and the past frame. The body drive control device 214, for example, causes the photometric sensor (not shown) to detect the subject brightness, and prohibits the integration of the focus detection pixel data of the past frame when the brightness change between the latest frame and the past frame is large. . This is because the photographing situation has changed greatly and the focus detection accuracy is lowered even if past focus detection pixel data is integrated.

  The body drive control device 214 extracts a partial image (such as a face image) of a specific pattern from the frame image by image processing, detects the position of the partial image in the frame image, and when the position change is large, that is, When the position change amount exceeds the threshold value, the integration of the focus detection pixel data of the past frame is prohibited. This is because the focus detection accuracy is lowered even if past focus detection pixel data is integrated.

  Note that different threshold values may be used for the horizontal position change amount and the vertical position change amount. For example, the horizontal position change amount threshold is set to 1 pixel, and the vertical position change amount threshold is set to 0 pixel.

  The body drive control device 214 calculates the image plane moving speed for each frame by dividing the difference between the defocus amount calculated in the previous frame and the defocus amount calculated in the current frame by the frame interval. When the image plane moving speed of the past frame exceeds the threshold, the integration of the focus detection pixel data of the past frame is prohibited. This is because in a frame in which the image plane moving speed exceeds the threshold value, the amount of movement of the subject in the optical axis direction is large, and the focus detection accuracy decreases even if past focus detection pixel data is integrated.

  In addition, the digital still camera 201 includes an illumination unit (not shown) that illuminates AF (autofocus) auxiliary light at low luminance, and the body drive control device 214 determines whether or not the latest frame is illuminated and whether or not the previous frame is illuminated. If they do not match, the integration of the focus detection pixel data of the past frame is prohibited. This is because the focus detection accuracy is lowered even if past focus detection pixel data is integrated.

  The body drive control device 214 may prohibit integration of focus detection pixel data of past frames in accordance with control information (operation mode) of the image sensor 212.

  When the image sensor 212 has a plurality of pixel signal readout modes, the detection pitch and addition state of the focus detection pixel data differ depending on the readout mode. The pixel signal readout mode is, for example, an all-pixel readout mode and a thinning readout mode disclosed in Japanese Unexamined Patent Application Publication No. 2009-128892, and an addition readout mode disclosed in Japanese Unexamined Patent Application Publication No. 2009-86424. When the reading mode of the latest frame and the reading mode of the past frame do not match, the body drive control device 214 prohibits the integration of the focus detection pixel data of the past frame. This is because the focus detection accuracy is lowered even if past focus detection pixel data is integrated.

  When the amplification levels of the pixel signals output from the image sensor 212 are different, the state of noise added to the focus detection pixel data is different. When the difference between the amplification degree of the latest frame and the amplification degree of the past frame is large, the body drive control device 214 prohibits the integration of the focus detection pixel data of the past frame. This is because the focus detection accuracy is lowered even if past focus detection pixel data is integrated.

  The operation mode of the image sensor 212 may be manually changed by the user. For example, the amplification degree of the pixel signal is changed according to the change in sensitivity setting of the image sensor 212.

  As described above, when the integration of the focus detection pixel data of the past frame is prohibited according to the change of the imaging state, the deviation between the imaging state data of the latest frame and the past imaging state data sets the threshold value as shown in FIG. The change in various imaging states is determined by exceeding, but the threshold value is determined by experiment.

  Based on a plurality of combinations of the various imaging states shown in FIG. 10, the threshold determination in step S170 of FIG. 6 may be performed. For example, the body drive control device 214 has exceeded either one of the absolute value of the deviation of the focal length data of the current frame and the past frame and the absolute value of the deviation of the attitude data of the current frame and the past frame exceeding the threshold value. When it is determined that the imaging state has changed. The threshold value Tf regarding the absolute value of the deviation of the focal distance data and the threshold value Ta of the absolute value of the deviation of the posture data may be changed in association with each other. For example, when only one of the focal length data and the posture data changes, Tf = T1 and Ta = T2, and when both the focal length data and the posture data change, both threshold values are set to 50%. Weighting may be performed so that Tf = (T1) / 2 and Ta = (T2) / 2.

  When prohibiting the integration of focus detection pixel data of past frames according to changes in various imaging states, the body drive control device 214, as shown in FIG. 10, captures the latest frame imaging state data and past imaging state data. When the deviation from the above exceeds a threshold, changes in various imaging states are determined. The body drive control device 214 may determine changes in various imaging states based on differences between the imaging state data of the latest frame and the past imaging state data. For example, when the diaphragm aperture F value changes discretely for each stage (F1.4, F2.0, F2.8,...), The body drive control device 214 has exceeded the threshold with a single stage change. judge. That is, a change in imaging state is determined based on different aperture F values.

  In the embodiment described above, the body drive control device 214 stores the focus detection pixel data and the imaging state data in the past seven frames in pairs in the storage device 220, and integrates the focus detection pixel data in the past frames. Is determined based on imaging state data corresponding to the focus detection pixel data, but is not limited to such a method.

  The body drive control device 214 can also prevent the storage device 220 from storing focus detection pixel data that cannot be used for integration processing based on the imaging state data. For example, if the focus detection pixel data of a frame determined to have a large blur is not stored in the storage device 220, the memory area of the storage device 220 can be used effectively.

  The body drive control device 214 converts the focus detection pixel data corresponding to the imaging state data whose deviation from the imaging state data in the latest frame exceeds a threshold value, and the past focus detection pixel data older than the focus detection pixel data into a new focus. Each time detection pixel data is acquired, it may be excluded from the target of integration processing. At the time of such exclusion, the body drive control device 214 updates the effective frame number corresponding to the oldest focus detection pixel data indicating which past frame focus detection pixel data can be used for integration processing. It is good. By performing the integration process on the focus detection pixel data of the past frames up to the effective frame number, the loop process (steps S160 to 180) in FIG. 6 can be simplified and the processing time can be shortened. In the example shown in FIG. 9, the frame number (N-5) of the focus detection pixel data I (N-5) is used as the effective frame number and used in the integration process for the next focus detection pixel data I (N + 1). Thus, the focus detection pixel data I (N-6) is excluded from the integration processing target.

  In the embodiment described above, the body drive control device 214 stores the focus detection pixel data output from the image sensor 212 in the storage device 220 as it is, and performs an image shift detection process using the focus detection pixel data. However, it is not limited to this.

  For example, the body drive control device 214 performs digital filter processing (addition averaging filter) for removing high-frequency components of the image on the focus detection pixel data and digital filter (difference filter) for removing low-frequency components of the image. You may remove the frequency component which has a bad influence on deviation detection. The body drive control device 214 removes the frequency component that adversely affects the image shift detection as described above, stores the filtered focus detection pixel data in the storage device 220, and uses the filtered focus detection pixel data. An image shift detection process may be performed.

  The body drive control device 214 performs so-called sensitivity variation correction processing and dark current variation correction processing on the focus detection pixel data, stores focus detection pixel data after the correction processing, and performs focus detection pixel data after the correction processing. The image shift detection process may be performed using

  The body drive control device 214 may change the contents of the image processing such as the filtering processing performed on the focus detection pixel data and the characteristic variation correction processing for each pixel according to the shooting situation. In such a change, the focus detection pixel data after the image processing and the content of the image processing are stored in the storage device 220 as a pair. When the content of the image processing of the latest frame is different from the content of the image processing of the past frame, the integration of the focus detection pixel data of the past frame may be prohibited. This is because a focus detection error may occur due to integration of focus detection pixel data of past frames. For example, if the dark current variation correction processing for the latest frame differs from the dark current variation correction processing for the past frame under the condition that the dark current variation correction processing is performed only at low luminance, the focus of the past frame The integration of detected pixel data is prohibited.

  The present invention is not limited to an image sensor provided with a focus detection pixel for pupil division type phase difference detection, and can also be applied to an image sensor provided with a focus detection pixel used for so-called contrast detection.

<< Modification of Embodiment >>
FIG. 11 is a front view showing a detailed configuration of an imaging element 212A according to a modification. The image sensor 212 illustrated in FIG. 3 has a pair of focus detection pixels 312 and 313. In contrast, in the imaging device 212A illustrated in FIG. 11, the focus detection pixel 311 has a pixel structure including a pair of photoelectric conversion units under one microlens. FIG. 11 is an enlarged view of the vicinity of an area corresponding to one focus detection area on the image sensor 212A. The vertical and horizontal directions correspond to the vertical and horizontal directions of the imaging screen shown in FIG. An imaging element 212A of this modification includes an imaging pixel 310 for imaging and a focus detection pixel 311 for focus detection.

  The focus detection pixel 311 includes a microlens 10 and a pair of photoelectric conversion units 12 and 13. The basic principle of the focus detection method of the pupil division method using the focus detection pixel 311 is, as disclosed in Japanese Patent Laid-Open No. 2008-85738, pupil division using the pair of focus detection pixels 312 and 313 shown in FIG. This is the same as the focus detection method. The pair of photoelectric conversion units respectively receive light beams that arrive at the microlenses from the pair of distance measurement pupils. The arrangement direction of the focus detection pixels 311 is made to coincide with the arrangement direction of the pair of distance measuring pupils, that is, the arrangement direction of the pair of photoelectric conversion units.

  A large number of focus detection pixels 311 as described above are arranged in a straight line, and the output of the pair of photoelectric conversion units 12 and 13 of each pixel is collected into an output group corresponding to the pair of distance measurement pupils, thereby making a pair of distance measurement pupils. Information on the intensity distribution of the pair of images formed on the focus detection pixel array by the pair of focus detection light fluxes passing through is obtained.

  Although the example in which the focus detection pixel includes the white filter is shown in the image sensor in the above-described embodiment, the present invention can be applied to a case where the same color filter (for example, a green filter) as that of the image capture pixel is provided. .

  For example, only the focus detection pixels 311 shown in FIG. 11 may be two-dimensionally arranged to form an image sensor, and the two-dimensionally arranged focus detection pixels may have a color filter arranged in a Bayer array. In such a configuration, at the time of imaging, it is possible to calculate data equivalent to the imaging pixel by adding the data of the pair of photoelectric conversion units of the focus detection pixel. At the time of focus detection, by performing correlation calculation between focus detection pixels of the same color, it is possible to obtain a focus detection result for each color, and it is possible to detect a focus even when contrast is not obtained only by luminance.

  In the above-described embodiment, a CCD image sensor or a CMOS image sensor can be used as the image sensor.

  In the image pickup device according to the above-described embodiment, an example in which the image pickup pixel includes a color filter with a Bayer array is shown. However, the configuration and the array of the color filter are not limited to this. For example, the present invention can be applied to an arrangement other than the arrangement of complementary color filters (green: G, yellow: Ye, magenta: Mg, cyan: Cy) or a Bayer arrangement. Further, the present invention can also be applied to a monochrome image sensor that does not include a color filter.

  Note that the imaging apparatus is not limited to the digital still camera configured as described above in which the interchangeable lens is mounted on the camera body. For example, the present invention can be applied to a lens-integrated digital still camera or video camera. Furthermore, the present invention can be applied to a small camera module built in a mobile phone, a surveillance camera, a visual recognition device for a robot, an in-vehicle camera, and the like.

10, 70 micro lens, 11, 12, 13, 71 photoelectric conversion unit,
29 Semiconductor circuit board, 72, 73 Focus detection beam, 81 Imaging beam,
90 Exit pupil plane, 91 optical axis, 92, 93 Distance pupil, 94 area,
100 shooting screen, 101 focus detection area,
201 digital still camera, 202 interchangeable lens, 203 camera body,
204 mount unit, 206 lens drive control device,
208 zooming lens, 209 lens, 210 focusing lens,
211 Aperture, 212 Image sensor, 213 Electrical contact,
214 body drive control device,
215 liquid crystal display element driving circuit, 216 liquid crystal display element, 217 eyepiece,
219 memory card, 220 storage device, 221 AD converter,
310 imaging pixels, 311, 312, 313 focus detection pixels

Claims (20)

An image sensor in which a plurality of pixels are arranged, and an image of a subject is captured by the plurality of pixels to generate pixel data corresponding to each of a plurality of frames that are continuous in time series;
A photographing optical system for forming an image of the subject on the image sensor;
Imaging state information acquisition means for acquiring imaging state information relating to an imaging state at the time of imaging the subject image;
Storage means for storing at least a partial data of the pixel data and the imaging state information in association with each other;
Of the pixel data generated by the image sensor, an addition process for adding at least one of the partial data stored in the storage unit to the partial data included in the latest pixel data is performed to obtain the added partial data. Adding means for calculating;
Prohibiting means for performing prohibition control for prohibiting addition of the partial data related to the imaging state information based on the imaging state information stored by the storage unit;
An imaging apparatus comprising: a focus detection unit configured to detect a focus adjustment state of the photographing optical system based on the addition partial data. The imaging device according to claim 1,
The prohibiting unit is based on a comparison result between the imaging state information associated with the partial data included in the latest pixel data and the imaging state information associated with the partial data stored by the storage unit. An image pickup apparatus that performs the prohibition control. The imaging device according to claim 2,
The prohibiting unit has a predetermined degree of difference between the imaging state information associated with the partial data included in the latest pixel data and the imaging state information associated with the partial data stored by the storage unit. The image pickup apparatus performs the prohibition control when the threshold value is larger than the threshold value. The imaging device according to any one of claims 1 to 3,
The imaging apparatus is characterized in that the imaging state is expressed by optical characteristics of the photographing optical system. The imaging apparatus according to claim 4,
The image pickup apparatus according to claim 1, wherein the optical characteristic is expressed by a diaphragm aperture F value of a photographing optical system. The imaging apparatus according to claim 4,
The optical apparatus is characterized in that the optical characteristic is expressed by a focal length of a photographing optical system. The imaging apparatus according to claim 4,
The image pickup apparatus, wherein the optical characteristic is expressed by a position of a focusing lens of a photographing optical system. The imaging device according to any one of claims 1 to 3,
The imaging apparatus is characterized in that the imaging state is represented by attitude information regarding the attitude of the imaging apparatus. The imaging device according to claim 8,
The posture information is represented by a composition of the frame. The imaging device according to claim 8,
The posture information is represented by a blur amount of the image pickup device. The imaging device according to any one of claims 1 to 3,
The imaging apparatus, wherein the imaging state is represented by subject information related to the subject. The imaging device according to claim 11,
The imaging apparatus, wherein the subject information is a luminance of the subject. The imaging device according to claim 11,
The imaging apparatus, wherein the subject information is a position of the subject in the frame. The imaging device according to any one of claims 1 to 3,
The imaging apparatus, wherein the imaging state is an operating state of the imaging device. The imaging device according to claim 14, wherein
The image pickup apparatus, wherein the operation state is a read mode of the pixel data generated by the image pickup element. The imaging device according to claim 14, wherein
The operation state is represented by an output gain of the pixel data generated by the image sensor. In the imaging device according to any one of claims 1 to 16,
The storage means sequentially stores the partial data of the pixel data generated in time series from the imaging element,
The adding means sequentially reads out the partial data from the storage means in chronological order from the old time, performs the addition processing on the read partial data, and the added partial data satisfies a predetermined condition Then, the image pickup apparatus ends the addition process. The imaging device according to any one of claims 1 to 17,
Image processing means for performing image processing on the partial data;
The storage means sequentially stores the partial data of the pixel data generated in a time series from the imaging element, and stores the partial data subjected to the image processing when storing the partial data. An imaging device characterized by storing. The imaging device according to claim 18, wherein
The image processing means can perform the image processing on the partial data by a plurality of types of image processing methods,
The imaging apparatus, wherein the imaging state is expressed by the image processing method used for the image processing performed by the image processing means. In the imaging device according to any one of claims 1 to 19,
The plurality of pixels include imaging pixels that output imaging signal data relating to the image of the subject, and pupil division that receives a pair of focus detection light beams passing through the exit pupil and outputs focus detection signal data relating to the focus adjustment state. A focus detection pixel of the type,
The image pickup apparatus, wherein the partial data is the focus detection pixel signal data.

Images