JP2015025960A - Focus detection system, focus detection method, program, and image capturing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize high-precision, high-speed contrast AF without driving lenses.SOLUTION: A focus detection system detects focus by acquiring pixel data representing object light intensity distribution formed by an image capturing optical system, which includes an image capturing lens, and angle information and includes: image generation means which generates reconfigured images at image reconfiguration positions from the pixel data; addition control means which updates unit pixels of the reconfigured images in accordance with a minimum value of the number of pixels added to form a unit pixel of the reconfigured images; contrast evaluation means which generates contrast evaluation values of the reconfigured images using the updated unit pixels; and focus evaluation means which detects a focal position of the object light based on the contrast evaluation values of the reconfigured images generated at different image reconfiguration positions.

Description

  The present invention relates to a focus detection system applicable to an imaging apparatus typified by a digital camera, and more particularly to a focus detection system and focus detection method for autofocus (hereinafter referred to as AF) using light field data.

  Conventionally, as an autofocus device for focus adjustment, a digital single-lens reflex camera generally has a so-called phase difference AF, and a digital compact camera has a contrast AF. As a feature of these AFs, contrast AF enables precise focusing, but has a drawback that AF processing is slower than phase difference AF because lens driving is required for image acquisition.

  In recent years, an imaging device (Light Field Camera) using a technique called “Light Field Photography” has been proposed. In this imaging apparatus, a microlens array is provided in addition to the conventional imaging optical system, so that data obtained from the imaging element (light field data) includes not only the light intensity distribution on the light receiving surface but also the angle information of the light beam. . Based on this information, an image of an arbitrary viewpoint and direction can be reconstructed.

  For example, Patent Document 1 discloses a method of performing contrast AF by image reconstruction without driving a lens using information obtained from an image sensor by pupil division.

JP 2009-258610 A

  However, in the prior art disclosed in the above-mentioned patent document, a contrast AF method by image reconstruction is described for one example of the optical configuration of a plurality of Light Field Cameras. In some cases, the image reconstruction and the focusing by the image reconstruction cannot be performed with sufficiently high accuracy.

Therefore, an object of the present invention is to solve the problem of accuracy reduction that occurs in contrast AF due to reconstruction using light field data obtained by using the optical configuration of FIGS. 7B and 7C. In particular, an object of the present invention is to provide a focus detection system that can realize high-speed and high-precision contrast AF by solving the above problem.

  To achieve the above object, according to the present invention, there is provided a focus detection system that performs focus detection by acquiring pixel data that gives intensity distribution and angle information of an optical image of a subject formed by a shooting optical system including a shooting lens. Image generating means for generating a reconstructed image at the image reconstruction position from the pixel data, holding means for detecting and holding the number of pixel data added to generate unit pixels of the reconstructed image, and holding A contrast evaluation unit that detects a minimum value of the added number and updates a unit pixel of the reconstructed image according to the minimum value, and generates a contrast evaluation value of the reconstructed image using the updated unit pixel Means and focus evaluation means for detecting the focal position of the optical image of the subject based on the contrast evaluation values of the reconstructed images generated at different image reconstruction positions.

  According to the present invention, it is possible to provide an imaging apparatus capable of performing AF with high accuracy and high speed without driving a lens.

The figure which shows the system configuration | structure of the imaging device to which the focus detection system based on the Example of this invention is applied. The figure which shows pupil division typically in the optical system of the imaging device of the Example of this invention The figure which shows the flowchart of the focus detection operation | movement of the focus detection system which concerns on the Example of this invention. Diagram for explaining image reconstruction method The figure for demonstrating the reconstruction method of the image in several positions The figure for demonstrating addition area | region control in the focus detection system based on the Example of this invention. The figure which shows typically the structure of the optical system of Light Field Camera

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  Hereinafter, a focus detection system according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 7 by taking an imaging apparatus to which the focus detection system is applied as an example.

  FIG. 1 is a block diagram showing an electrical configuration of a digital camera and a lens that are the photographing apparatus 100. The imaging apparatus 100 is a camera system including a camera 101 and a lens 102, and includes an imaging system, an image processing system, a recording / reproducing system, and a control system. The imaging system includes a photographing optical system 103 and an imaging element 106, and the image processing system includes an image processing unit 107. The recording / reproducing system includes a memory unit 108 and a display unit 109, and the control system includes a camera system control unit 105, an operation detection unit 110, a lens system control unit 112, and a lens driving unit 113. The lens driving unit 113 can drive a focus lens, a shake correction lens, a diaphragm, and the like for focus adjustment.

  The imaging system is an optical processing system that forms an image of light from an object (an optical image of a subject) on the imaging surface of the imaging element 106 via a photographic optical system 103 having a photographic lens. Microlenses are arranged in a lattice pattern on the surface of the image sensor 106 (light receiving surface), forming a so-called microlens array (hereinafter referred to as MLA). The MLA constitutes pupil dividing means in this embodiment. Details of the functions and arrangement of the MLA will be described later with reference to FIG. As will be described later, since the focus evaluation amount / appropriate exposure amount is obtained from the output signal of the image sensor 106, the photographing optical system 103 is appropriately adjusted based on this signal, so that an appropriate amount of object light can be obtained. While the image sensor 106 is exposed, a subject image is formed in the vicinity of the image sensor 106.

  The image processing unit 7 includes an A / D converter, a white balance unit, a gamma correction unit, an interpolation calculation unit, and the like, and can generate a recording image. In addition, a developing unit, a contrast evaluation unit, a correlation calculation unit, and the like, which are main parts of the present invention, can be included. In this embodiment, these elements are described assuming that they are arranged as a focus detection system in the camera system control.

  The memory unit 108 includes a processing unit necessary for recording in addition to an actual storage unit. The memory unit outputs to the recording unit and generates and stores an image to be output to the display unit 109. In addition, the memory unit 108 compresses images, moving images, sounds, and the like using a predetermined method.

  The camera system control unit 105 generates and outputs a timing signal at the time of imaging, and controls the imaging system, the image processing system, and the recording / reproducing system in response to an external operation. For example, the operation detection unit 110 detects pressing of a shutter release button (not shown), and controls driving of the image sensor 106, operation of the image processing unit 107, compression processing of the memory unit 108, and the like. Further, the state of each segment of the information display device that displays information on a liquid crystal monitor or the like is controlled by the display unit 109.

  The adjustment operation of the taking optical system of the control system will be described. An image processing unit 107 is connected to the camera system control unit 105, and an appropriate focal position and aperture position for photographing are obtained based on a signal from the image sensor 106. The camera system control unit 105 issues a command to the lens system control unit 112 via the electrical contact 111, and the lens system control unit 112 appropriately controls the lens driving unit 113. Furthermore, a camera shake detection sensor (not shown) is connected to the lens system control unit 112, and in the camera shake correction mode, the camera shake correction sensor is appropriately controlled based on the signal of the camera shake detection sensor. To do.

  Here, a method for arranging a plurality of microlenses proposed by the Light Field Camera will be described with reference to FIG. In FIG. 7, 106 is an image sensor, 220 is MLA, 231 to 235 are pupil regions, 751 is a subject plane, 751a and 751b are appropriate points on the subject, and 752 is a pupil plane of the photographing optical system. Show. In FIGS. 7B and 7C, 106a indicates a virtual image sensor, and 220a indicates a virtual MLA. These are shown as a reference for clarifying the correspondence with FIG. In the example of FIG. 7A, the MLA 220 is disposed in the vicinity of the imaging plane of the photographing optical system 103 so that the imaging element 106 and the pupil plane 752 of the photographing optical system are in a conjugate relationship. In the example of FIG. 7B, the MLA 220 forms an image of the light beam from the photographing optical system 103, and the image sensor 106 is provided on the image formation surface. By arranging in this way, the object plane 751 and the image sensor 106 are in a conjugate relationship. The points on the subject are imaged at different positions of the image sensor by the passing area on the pupil plane. In the example of FIG. 7C, the MLA 220 re-images the light beam from the photographing optical system 103 (this is called re-imaging because the light beam once imaged is diffused). An image sensor 106 is provided on the image plane. By arranging in this way, the object plane 751 and the image sensor 106 are in a conjugate relationship.

  FIG. 7A is an example of the optical configuration of the Light Field Camera, and FIGS. 7B and 7C are examples of the optical configuration of the other Light Field Camera. Light field information can be obtained in any optical configuration as in FIG. 7A, but the image reconstruction method is different.

  That is, Patent Document 1 discloses a contrast AF method by reconstructing an image of the Light Field optical system in FIG. 7C, but contrast AF in the optical configuration in FIGS. 7B and 7C is disclosed. Is not adequately stated. In the optical configurations of FIGS. 7B and 7C, when an image in a certain focal plane is generated by refocusing, refocusing calculation is performed as usual, and pixels are shifted and added. Since there is a difference in the number of added pixels in the plane, a different image such as a blurred state is generated after the addition, and there is a possibility of erroneous determination when taking an evaluation value such as AF. Therefore, in this embodiment, in contrast AF by reconstruction using light field data obtained by using the optical configurations of FIGS. 7B and 7C, the addition pixel for each unit pixel in one reconstructed image Contrast AF is performed using a reconstructed image obtained by performing refocusing so that the numbers are substantially equal.

  FIG. 2 is a diagram for explaining a main part of the photographing optical system in the present embodiment. In order to apply the focus detection system of the present invention, it is necessary to acquire pixel data including angle information in addition to the position (intensity distribution) of the light beam, which is so-called light space information (light field data). In the present embodiment, the MLA 200 is disposed in the vicinity of the imaging plane of the photographing optical system 103 for obtaining angle information, and a plurality of pixels are associated with one lens constituting the MLA 200.

  FIG. 2A is a diagram schematically illustrating the relationship between the image sensor 106 and the MLA 200. FIG. 2B is a diagram schematically showing the correspondence between the pixels of the image sensor and the MLA. FIG. 2C is a diagram showing that pixels provided under the MLA by the MLA are associated with a specific pupil region.

  As shown in FIG. 2A, the MLA 200 is provided on the image sensor 106, and the front principal point of the MLA 200 is arranged in the vicinity of the imaging plane of the photographing optical system 103. FIG. 2A is a view of the MLA as viewed from the side and from the front of the photographing apparatus. The lens of the MLA 200 is disposed so as to cover the pixels on the image sensor 106 when viewed from the front of the photographing apparatus. In FIG. 2A, the micro lenses constituting the MLA 200 are illustrated in order to make the micro lenses easy to see. However, each micro lens 220 is actually only about several times as large as a pixel (about the actual size). Will be described with reference to FIG.

  FIG. 2B is a partially enlarged view from the front of the apparatus of FIG. A grid-like frame shown in FIG. 2B indicates each pixel of the image sensor 106. On the other hand, each microlens constituting the MLA 200 is indicated by thick circles 220a, 220b, 220c, and 220d. As apparent from FIG. 2B, a plurality of pixels are assigned to one microlens. In the example of FIG. 2B, 12 rows × 12 columns = 144 pixels are assigned to one microlens. It is provided for. That is, the size of each microlens is 12 times x 12 times the size of the pixel.

  FIG. 2C is a diagram in which the image sensor 106 is cut so that the longitudinal direction of the sensor includes the optical axis of the microlens 220 and the lateral direction of the sensor is in the horizontal direction of the drawing. Rectangular portions such as 221, 222, 223, 224, and 225 in FIG. 2C indicate pixels (one photoelectric conversion unit) of the image sensor 106. In the figure, Δx represents the pixel pitch. On the other hand, the figure shown above FIG. 2C shows the exit pupil plane of the photographing optical system 103. Actually, when the direction is aligned with the sensor diagram shown in the lower part of FIG. 2 (c), the exit pupil plane is in the direction perpendicular to the plane of FIG. 2 (c) (y direction). Is changing. In FIG. 2C, one-dimensional projection / signal processing will be described to simplify the description. In an actual device, this can be easily extended to two dimensions.

  The pixels 221, 222, 223, 224, and 225 in FIG. 2C are in a positional relationship corresponding to the pixels 221a, 222a, 223a, 224a, and 225a in FIG. In the example of FIG. 2C, the pixel 221 and the region 231 correspond to the pixel 222 and the region 232, the pixel 223 and the region 233, the pixel 224 and the region 234, and the pixel 225 and the region 235, respectively. That is, information on the light beam that has passed through the region 231 on the exit pupil plane of the photographing optical system 103 is incident on the pixel 221. The same applies to the other pixels. As a result, it is possible to obtain angle information from the positional relationship between the light ray passage area on the pupil plane and the light receiving area on the image sensor 106. Each pixel (unit pixel) of the image output at the time of development is expressed by integrating this angle information. In the example of FIG. 2C, since 12 × 12 = 144 image sensors are arranged below one microlens, information on 12 × 12 = 144 sensors is added on the imaging surface, and unit pixels at the time of output Form. In the present text, one pixel at the time of output in which the angle information is integrated is called an output unit pixel.

  Processing for obtaining a focus evaluation value (focus position) from a signal from the image sensor 106 using the imaging optical system of the present embodiment will be described with reference to FIG.

  FIG. 3 is a diagram illustrating a flowchart of operations executed by the focus detection system according to the present embodiment to obtain a focus evaluation value. The focus detection operation according to this flowchart is realized by the CPU of the camera system control unit 105 executing a program stored in a memory (not shown) to control each unit of the imaging apparatus.

  3A shows a flowchart of the entire operation for obtaining the focus evaluation value, FIG. 3B shows a flowchart of the image generation operation, and FIG. 3C shows a flowchart of the addition control operation. FIG. 3D shows a flowchart of the contrast evaluation value calculation operation, and FIG. 3E shows a flowchart of the focus evaluation operation. These operations will be described in the order of each step from FIG.

  Step S301 indicates the start of the focus evaluation value acquisition operation. For example, this is the case when the operation detection unit 110 in FIG. 1 detects a specific action (for example, pressing of the release button) by the photographer.

  In step S <b> 302, the camera system control unit 105 controls the image sensor 106 and the image processing unit 107, performs an imaging operation once, and acquires pixel data (light field data). By generating a reconstructed image from this image data, the focus position can be obtained by AF based on the contrast evaluation value without driving the lens.

  Steps S303 to S307 form a loop. In this loop, image generation, addition control, contrast, and evaluation value calculation are performed at each evaluation position while shifting the evaluation position from a predetermined initial value to an end value at an appropriately set interval. The initial value can be set to be different for coarse / fine adjustment. In step S303, the coarse adjustment interval and adjustment range are set coarsely and broadly, and fine adjustment is finely fined. The operation of performing coarse adjustment and fine focus detection is called fine adjustment.

In step S304, an image generation operation is performed. Details of the image generation operation will be described later with reference to the flowchart of FIG.
In step S305, an addition control operation is performed. In this operation, the number of added pixels of each output unit pixel is unified. Details of the addition control operation will be described later with reference to the flowchart of FIG.
In step S306, a contrast evaluation operation is performed. Details of the contrast evaluation operation will be described later with reference to the flowchart of FIG.
In step S307, the image reconstruction and evaluation value acquisition loop operations are terminated.

  In step S308, a focus evaluation operation is performed based on the result obtained in step S307 (in step S307, since a loop is formed, a plurality of evaluation values are obtained while changing the focus position), the focus evaluation operation is performed. Determine the position. Details of the focus evaluation operation will be described later with reference to the flowchart of FIG.

  In step S309, the operation of the focus detection system is terminated. By this series of operations, it is possible to detect the focus surface at high speed and with high accuracy without reciprocating the vicinity of the focus surface of the lens as in conventional contrast detection.

  Next, details of the image generation operation will be described with reference to FIGS. In this embodiment, calculation of the image on the development surface (= reconstruction surface) is realized by shifting (moving in the XY direction) the image according to the pupil region. This developing method is an example, and another developing algorithm may be used.

  Step S321 indicates the start of the developing operation. Steps S322 to S326 form a loop. In step S322, the loop calculation is executed by the number corresponding to the number of pupil divisions. Considering the reconstruction of the image, the amount of shifting the image is different if the incident angle is different even if the reconstruction plane is the same (when the exit pupil is sufficiently far away, it is almost synonymous with the fact that the passing pupil region is different). This is a loop for appropriately reflecting this in development calculation.

In step S324, the shift amount in each pupil region corresponding to the evaluation position is calculated based on the data from step S323. In step S323, the correspondence between each pixel and the MLA is stored, and information indicating which pupil region of each pixel is receiving light is stored.
In step S325, pixels that have obtained light rays with the same incident angle (obtain light rays from the same pupil region) are shifted based on the information in step S323.
In step S326, a series of image shift operations is terminated.

  In step S327, the area data for addition in step S331 is initialized (filled with 0).

  Steps S328 to S334 form a loop. In step S328, loop calculation is executed according to the number of microlenses constituting the MLA. For example, in the example shown in FIG. 2, the number of pixels of the original image sensor ÷ 144 (number of pupil divisions) is the number of microlenses, that is, the number of output unit pixels generated by this loop processing.

  Steps S329 to S332 further form a loop. In step S329, the loop calculation is executed by the number corresponding to the number of pupil divisions, and one output unit pixel is generated. For example, in the example shown in FIG. 2, since the light is divided into 25, there are light fluxes from 25 pupil positions.

  In step S330, a determination is made as to whether or not it is an area where addition is performed for the light flux from each pupil area. As will be described later with reference to FIGS. 4 and 5, in the image generation operation, a pixel to be added to the output unit pixel is designated. When the pixel corresponds to the pixel to be added, the process proceeds to step S331 to perform addition. In other cases, the process proceeds to step S332. If the shift amount determined in step S325 is not an integral multiple of the pixel, the addition in step S331 is performed while appropriately dividing the amount (addition may be made as appropriate depending on the overlapping area).

In step S332, a series of operations of addition processing performed to generate an output unit pixel ends.
In step S333, the number of pixels added in each output unit pixel is recorded. Based on the information of the added pixels recorded here, an addition control operation described later is performed.
In step S334, the series of operations is terminated, and in step S335, the process returns to the caller step S304.

  Here, the generation method of the output unit pixel at the image reconstruction position performed in the image generation operation of FIG. 3B described above will be specifically described with reference to FIGS.

  First, an image reconstruction method in the case where the configuration of the photographing optical system is the configuration of FIG. 7B will be described with reference to FIG. In FIG. 4, an alternate long and short dash line indicates a light beam that passes through the main lens 400 but does not enter the microlens 220. As shown in the figure, the microlens 220 is placed at a position other than the image plane of the main lens 400. The main lens surface corresponds to the principal point indicated by the photographing optical system 103 in FIG. A method for obtaining the output unit pixel Ia on the reconstruction surface I in the case of this photographing optical system will be described.

  First, the output unit pixel and the apex of each microlens 220 are connected by a line. At this time, a pixel whose straight line is included in the main lens 400 is selected. In FIG. 4, five straight lines are drawn from the output unit pixel toward the apex of the microlens 220. Of these, the upper and lower two shown by dotted lines do not pass through the main lens and are not selected. Next, the output unit pixel on the reconstruction plane Ia is generated by adding the information of the pixels through which the three straight lines selected in FIG. 4 pass (pixels shown in black in FIG. 4). . In this way, an output unit pixel on an arbitrary reconstruction plane can be generated.

  Next, the difference in the number of pixel additions in the output unit pixel depending on the position in the reconstruction plane and the defocus amount will be described with reference to FIG. FIG. 5 shows the addition pixels in the output unit pixels Ia and Ib on the reconstruction plane I and the output unit pixel II-a on the reconstruction plane II (with different defocus amounts). When the number of added pixels in each output unit pixel is obtained in accordance with the rules described above, the output unit pixel I-a has been added by 3 pixels, but the output unit pixel I-b has 4 pixels added, and the output unit pixel II. At -a, 6 pixels are added. Thus, the number of pixels to be added differs depending on the position in the reconstruction plane and the defocus amount. The difference in the number of added pixels in the output unit pixel in this way means that the F number is different for each output unit pixel and the amount of blur changes, which has an adverse effect during AF.

  In the present embodiment, the above adverse effect is suppressed by generating the reconstructed image by unifying the same number of output unit pixels by addition control described later.

  Further, as apparent from FIG. 5, the change in the number of added pixels increases as the defocus amount from the imaging surface increases. Accordingly, during rough adjustment in which the focus position is roughly searched during AF, the change in the defocus amount becomes large, and the addition control is operated. However, during fine adjustment in which a plurality of images are reconstructed in order to increase the focusing accuracy near the focusing surface, the defocusing change is small, and the addition control operation may be stopped. That is, the addition control operation may be switched on and off according to the size of the evaluation position shift interval.

  Next, details of the operation of the addition control will be described with reference to FIGS.

  Step S341 indicates the start of the addition control operation.

Steps S342 to S345 form a loop. In step S342, loop calculation is executed according to the number of microlenses constituting the MLA. For example, in the example illustrated in FIG. 2, the number of microlenses is the number of pixels of the original image sensor ÷ 144 (number of pupil divisions).
In step S343, the number of added pixels of each output unit pixel held in step S333 is called.
In step S344, the minimum value (minimum addition number) of the read addition pixel number is detected and recorded. For example, in FIG. 5, three pixels in the output unit pixel Ia are the minimum addition number. In step S345, the loop processing ends.

  Steps S346 to S350 form a loop. Image generation calculation is performed according to the recorded minimum addition number while shifting the image reconstruction position by an appropriately set step within a range from a predetermined initial value to an end value. In step S346, the initial value can be set so as to be different depending on whether the adjustment is rough or fine, and is set to be coarse and wide at the time of coarse adjustment, and finely and narrowly set at the time of fine adjustment.

  In step S347, it is determined whether or not the added pixel number of the output unit pixel is the minimum value recorded in step S344. When the number of added pixels is the minimum value, the data of the output unit pixel obtained in FIG. 3B is not updated. If the number of added pixels is not the minimum value, the process proceeds to step S348.

  In step S348, the operation of the image generation unit shown in FIG. 3B is performed to generate the value of the output unit pixel. However, the number of added pixels is the minimum added value recorded in step S344, and the pixels to be used are selected as described later with reference to FIG.

  In step S349, the data of the output unit pixel is updated with the value generated in step S348. In step S350, the loop is terminated, and in step S351, the process returns to the caller step S305.

  Here, a preferable limiting method when the number of added pixels of the output unit pixel is limited by the addition control operation will be described with reference to FIG. FIG. 6 is a diagram schematically showing a case where the pixels added in the output unit pixels Ia and Ib are projected on the pupil plane of the main lens 400. The projection of the output unit pixel Ia is a black area Ia-1 to Ia-3, and the projection of the output unit pixel output unit pixel Ib is a spot area Ib-1 to I- b-4. Since it is desirable that the pupil information of the output unit pixel be at the same position as much as possible, it is possible to obtain close pupil information from Ia-1 to Ia-3 and Ib-1 to Ib-4 It is good to choose. In other words, in FIG. 6, when the output unit pixel Ib is changed from 4 pixel addition to 3 pixel addition, Ib-1 is omitted and output by Ib-2 + Ib-3 + Ib-4. The unit pixel Ib may be formed. At this time, the output unit pixel may be generated by changing the weight of each pixel output as ((I−b−1 + I−b−2) ÷ 2) + I−b−3 + I−b−4. Further, the addition control may be performed with the minimum area when pixels are projected on the pupil plane in consideration of the vignetting on the pupil, not the limitation of the minimum addition number.

  Further, the number of additions taken by each unit pixel is not limited to the above-described minimum addition number in the entire unit pixels constituting the reconstructed image, but may be any value if the same addition number is taken in the unit pixels corresponding to the contrast evaluation area. May be set. Even when other numbers of additions are set, detection of the minimum number of pixels is effective for determining whether a predetermined number can be used as a unified number of additions. However, the number of additions clearly below the minimum number of pixels is effective. In order to make them equal, it is not necessary to detect the minimum number of pixels, and the number of additions may be set. Further, in this embodiment, the reconstructed image used for the contrast evaluation value calculation is temporarily added to the pixel data according to the reconstructed position to generate each unit pixel, and then the pixel data of the unit pixel that is not the minimum addition number, A method of selecting pixel data and updating it to unit pixel data having the minimum number of additions was used. However, since the addition number of each unit pixel can be obtained by calculation for specifying addition pixel data for generating each unit pixel without generating a reconstructed image, steps S331 and S349 are omitted, and the reconstructed image is obtained. After the number of additions to be equalized is determined by calculation, each unit pixel may be generated according to the number of additions determined in step S348.

Next, details of the operation of contrast evaluation value calculation will be described with reference to FIG.

  Step S361 indicates the start of the contrast evaluation operation.

  In step S362, the number of evaluation points for contrast evaluation and the size of the evaluation frame are set. If the number of evaluation points is increased, the entire screen can be covered, but there is a problem that the evaluation takes time. This is set as appropriate based on user settings detected by the operation detection unit 110. On the other hand, when the size of the evaluation frame is increased, focus detection is possible even for textures that do not have a pattern locally, but when the texture is too large, images of subjects with different distances are simultaneously evaluated. Perspective conflict will occur. The size of the evaluation frame is appropriately set so that these problems can be solved.

  In step S363, appropriate filtering is performed on the signal of each unit pixel of the reconstructed image. Appropriate filtering may be performed by focusing on a lower frequency during coarse adjustment, and by focusing on a higher frequency than during coarse adjustment during fine adjustment. By changing the filtering characteristics in this way, it is possible to improve the focus detection accuracy while avoiding the local minimum.

Steps S364 to S367 form a loop. In step S364, the set evaluation points are repeatedly calculated so as to obtain evaluation values corresponding to the evaluation points determined in step S362.
In step S365, the maximum value of one line in the filtered evaluation frame is peak-held.

  In step S366, the evaluation value is calculated by adding the maximum value of one line peak-held in S365 for the rows in the evaluation frame.

  In this embodiment, the contrast evaluation value is obtained by the method as described above. However, other calculation methods may be used as long as the contrast evaluation value is associated with the focus fluctuation. For example, a method of using an evaluation value obtained by making the secondary contrast the square of the primary contrast and making the interrupt luminance value dimensionless can be considered. Next, in step S368, the process returns to the caller step S306.

Next, details of the focus evaluation operation will be described with reference to FIG.
Step S371 indicates the start of the operation of the focus evaluation unit.

  Steps S372 to S376 form a loop. In step S372, the maximum value, the second largest value, and the third largest value are obtained by calculation from the plurality of evaluation values obtained by the contrast evaluation value calculation in the loop from step S303 to S307 in FIG.

In step S373, the maximum value of the obtained evaluation values is peak-held as EVMax.
In step S374, the second largest value among the obtained evaluation values is peak-held as EVMax2.
In step S375, the third largest value among the obtained evaluation values is peak-held as EVMax3. In step S376, the loop is terminated.

  In step S377, an equation L1 representing a straight line passing through EVMax obtained in step S373 and EVMax3 obtained in step S375 is obtained.

  In step S378, an equation L2 representing a straight line passing through EVMax2 obtained in step S374 is obtained, which has an inclination opposite to that of L1 obtained in step S377.

  In step S379, the x coordinate (def coordinate) of the intersection of the straight line equation L1 obtained in step S377 and the straight line equation L2 obtained in step S378 is obtained as the focus position. The imaging apparatus performs focus adjustment by driving the photographing optical system based on the obtained focus position information.

  In this embodiment, the focus evaluation value is obtained by the method as described above. However, any other calculation method can be used as long as the peak position can be calculated from the obtained plurality of contrast evaluation values. For example, a method may be used in which quadratic function approximation is performed from the obtained four contrast evaluation values, and the value of the vertex is set as the peak position. In step S380, the process returns to the caller step S308.

  As described above, according to the present embodiment, even when AF is performed without driving the lens, it is possible to perform high-speed AF with high focus accuracy by limiting the number of added pixels in the output unit pixel. An image pickup apparatus can be provided.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

  In the above-described embodiment, each process in FIG. 3 is realized by reading a program for realizing the function of each process from the memory and executing it by the CPU of the camera system control unit 105.

  Note that the present invention is not limited to the above-described configuration, and all or some of the functions shown in FIG. 3 may be realized by dedicated hardware. The memory described above may be a non-volatile memory such as a magneto-optical disk device or a flash memory, a storage medium that can only be read such as a CD-ROM, or a volatile memory other than a RAM. Moreover, you may comprise from the computer-readable / writable storage medium by those combination.

  3 is recorded in a computer-readable storage medium, and the program recorded in the storage medium is read into the computer system and executed. May be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices. Specifically, the program read from the storage medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. After that, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instructions of the program, and the functions of the above-described embodiments are realized by the processing.

  The “computer-readable storage medium” refers to a storage device such as a flexible disk, a magneto-optical disk, a portable medium such as a ROM or a CD-ROM, and a hard disk built in a computer system. Furthermore, the “computer-readable storage medium” includes a medium that holds a program for a certain period of time. For example, it includes a volatile memory (RAM) inside a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.

  The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

  A program product such as a computer-readable recording medium in which the above program is recorded can also be applied as an embodiment of the present invention. The above program, recording medium, transmission medium, and program product are included in the scope of the present invention.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

Claims (15)

In a focus detection system for performing focus detection by acquiring pixel data that gives intensity distribution and angle information of an optical image of a subject formed by a shooting optical system including a shooting lens,
Image generating means for generating a reconstructed image at an image reconstruction position from the pixel data;
Holding means for detecting and holding the number of pixel data to be added to generate each unit pixel of the reconstructed image;
An addition control means for detecting a minimum value of the held addition number and updating a unit pixel of the reconstructed image according to the minimum value;
Contrast evaluation means for generating a contrast evaluation value of the reconstructed image using the updated unit pixel;
Focus evaluation means for detecting a focal position of the optical image of the subject based on the contrast evaluation value of the reconstructed image generated at a different image reconstruction position;
A focus detection system comprising:   In the unit pixel in which the number of pixel data added to generate the unit pixel is larger than the minimum value when the addition control unit adds the pixel data for each unit pixel according to the image reconstruction position, The unit pixel is updated by selecting and adding pixel data of a pupil region corresponding to the angle information of the pixel data added by the unit pixel in which the minimum value is detected. Focus detection system.   The image generation means includes setting means for setting a range and interval of the image reconstruction position, and the addition control means switches on / off of addition according to the interval of the image reconstruction position. The focus detection system according to claim 1 or 2.   The focus detection system according to claim 3, wherein the contrast evaluation unit includes a reconstructed image filtering unit, and changes a characteristic of the filtering unit according to an interval between the image reconstruction positions.   The image generation unit determines the shift amount of the pixel data according to the image reconstruction position, and generates the output unit pixel by weighting and adding the pixel data according to the shift amount. The focus detection system according to any one of 1 to 4.   The focus detection system according to claim 1, wherein the contrast evaluation unit further includes a setting unit that sets an evaluation position for generating the contrast evaluation value in a reconstructed image. In a focus detection system for performing focus detection by acquiring pixel data that gives intensity distribution and angle information of an optical image of a subject formed by a shooting optical system including a shooting lens,
Image generating means for generating a reconstructed image at an image reconstruction position from the pixel data so that the number of pixel data added to generate each unit pixel of the reconstructed image is equal;
Calculating means for calculating a contrast evaluation value of the reconstructed image;
Focus evaluation means for detecting a focus position of the subject based on the contrast evaluation value of the reconstructed image generated at a different image reconstruction position;
A focus detection system comprising: A taking optical system including a taking lens;
Pupil dividing means for dividing the pupil of the photographing lens;
Imaging means for photoelectrically converting an optical image of a subject formed by the photographing optical system and pupil dividing means to generate pixel data;
Image generating means for generating a reconstructed image at an image reconstruction position from the pixel data;
Holding means for detecting and holding the number of pixel data to be added to generate unit pixels of the reconstructed image;
An addition control means for detecting a minimum value of the held addition number and updating a unit pixel of the reconstructed image according to the minimum value;
Contrast evaluation means for generating a contrast evaluation value of the reconstructed image using the updated unit pixel;
Focus evaluation means for detecting a focal position of the optical image of the subject based on the contrast evaluation value of the reconstructed image generated at a different image reconstruction position;
An imaging apparatus comprising: a focus adjusting unit that controls the photographing optical system according to the detected focal position. In a focus detection method for performing focus detection by acquiring pixel data that gives intensity distribution and angle information of an optical image of a subject formed by a shooting optical system including a shooting lens,
An image generation step of generating a reconstructed image at an image reconstruction position from the pixel data;
A holding step for detecting and holding the number of pixel data to be added to generate the unit pixel of the reconstructed image;
An addition control step of detecting a minimum value of the held addition number and updating a unit pixel of the reconstructed image according to the minimum value;
A contrast evaluation step for generating a contrast evaluation value of the reconstructed image using the updated unit pixel;
A focus evaluation step of detecting a focus position of the optical image of the subject based on the contrast evaluation value of the reconstructed image generated at a different image reconstruction position;
A focus detection method comprising: In a focus detection method for performing focus detection by acquiring pixel data that gives intensity distribution and angle information of an optical image of a subject formed by a shooting optical system including a shooting lens,
An image generation step of generating a reconstructed image at an image reconstruction position from the pixel data, so that the number of pixel data added to generate each unit pixel of the reconstructed image is equal;
A calculation step of calculating a contrast evaluation value of the reconstructed image;
A focus evaluation step of detecting a focus position of the subject based on the contrast evaluation value of the reconstructed image generated at a different image reconstruction position;
A focus detection method comprising: A program for controlling a focus detection system that performs focus detection by acquiring pixel data that gives intensity distribution and angle information of an optical image of a subject formed by a shooting optical system including a shooting lens,
Computer
Image generation means for generating a reconstructed image at an image reconstruction position from the pixel data;
Holding means for detecting and holding the number of pixel data to be added to generate the unit pixel of the reconstructed image;
Addition control means for detecting a minimum value of the held addition number and updating a unit pixel of the reconstructed image according to the minimum value;
Contrast evaluation means for generating a contrast evaluation value of the reconstructed image using the updated unit pixel,
Focus evaluation means for detecting a focus position of the optical image of the subject based on the contrast evaluation value of the reconstructed image generated at a different image reconstruction position;
Program to function as. A program for controlling a focus detection system that performs focus detection by acquiring pixel data that gives intensity distribution and angle information of an optical image of a subject formed by a shooting optical system including a shooting lens,
Computer
Image generating means for generating a reconstructed image at an image reconstruction position from the pixel data so that the number of pixel data added to generate each unit pixel of the reconstructed image is equal;
Calculating means for calculating a contrast evaluation value of the reconstructed image;
Focus evaluation means for detecting a focus position of the subject based on the contrast evaluation values of the reconstructed images generated at different image reconstruction positions;
Program to function as.   A computer-readable storage medium recording the program according to claim 11 or 12.   The program which functions a computer as each means of the focus detection system as described in any one of Claims 1 thru | or 7.   A storage medium storing a program that causes a computer to function as each unit of the focus detection system according to any one of claims 1 to 7.

Images