JP5615756B2 - Imaging apparatus and imaging program - Google Patents

Description

  The present invention relates to an imaging apparatus and an imaging program, and more particularly, to an imaging apparatus and an imaging program that detect a distance to a subject and control a focal position of an imaging lens.

  Contrast methods and phase difference autofocus (hereinafter referred to as phase difference AF) methods are known for focus position detection for detecting the distance to a main subject. The phase difference AF method is often used in single-lens reflex cameras because it can detect the in-focus position at a higher speed and with higher accuracy than the contrast method.

  Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique for determining a shooting scene from the luminance distribution of a shot image, performing filter processing according to the determined scene, and then calculating an evaluation value of phase difference AF.

  Patent Document 2 discloses a technique for switching between a phase difference AF method and a contrast method according to the frequency distribution of a captured image.

  Further, in Patent Document 3, a subject is detected from a captured image by face feature amount extraction processing, an AF evaluation value calculation inhibition region is determined according to the type of subject detected in advance before the focusing operation, and the like. A technique is disclosed in which an AF evaluation value is calculated using a remaining area after the area is excluded from each AF area of AF.

JP 2003-215434 A JP 2009-63922 A JP 2010-160297 A

  However, in order to detect the in-focus position with respect to the subject, a certain area (view angle) is required, and not only the subject but also the area including the background may be the detection target area. For example, when it is desired to focus on a main subject such as a face, when a certain region including the background of the face is a focus detection target, the background may be focused.

  The present invention has been made in view of the above facts, and an object thereof is to provide an imaging apparatus and an imaging program capable of faithfully focusing on a subject when the subject is focused by phase difference AF. And

In order to achieve the above object, an imaging apparatus according to a first aspect of the present invention includes a first phase difference detection pixel and a second phase division pixel divided into pupils in a phase difference detection area provided on a light receiving surface for capturing an image of a subject. An image sensor in which paired pixels composed of phase difference detection pixels are arrayed and a focus position that is placed in front of the image sensor and forms a focused optical image of the subject on the light receiving surface It includes an imaging lens which is to identify the object to be focused, and encompasses the area including the area and the background area of the subject to be focused該合, Ri Do from part of the internal region of the subject to be focused 該合, the background area An area setting means for setting each of the non- included areas , a spatial frequency distribution of each of the set inclusion area and the included area, and a frequency corresponding to the subject area based on each of the calculated spatial frequency distributions Area sky Wherein a filter creation unit for creating a filter for removing frequency, the first detection information of the first phase difference detection pixels corresponding to the spatial frequency of the frequency range corresponding to the region of the subject using a filter that the created A generating unit that generates second detection information of the second phase difference detection pixel, a correlation calculation curve of a subject indicating a correlation between the generated first detection information and the second detection information, and the correlation calculation curve comprises said imaging lens phase difference computation means for obtaining the phase difference to control the focus position, and position control means for controlling the imaging lens based on the phase difference the calculated to the case Asei location, from .

  According to the present invention, an inclusion area that includes all of the subjects to be focused and an inclusion area that is a partial area of the subject are set, and the spatial frequency of the subject is determined based on the spatial frequency distribution of these inclusion areas and inclusion areas. Create a filter that passes. Using this filter, first detection information and second detection information corresponding to the region of the subject are generated, and a phase difference for controlling the imaging lens to the in-focus position is obtained from these correlations, and the imaging lens is focused. . This makes it possible to create a filter that obtains a spatial frequency (band) based only on the subject. By filtering the detection information of the phase difference detection pixels using a filter that passes the spatial frequency band based only on the subject, the background is included. Even in the inclusion area, the subject can be focused with high accuracy.

  Note that the imaging element can include a plurality of photoelectric conversion elements arranged in a horizontal direction and a vertical direction. In addition, the imaging device includes a solid-state imaging device that includes a plurality of pixels arranged in a two-dimensional array on a semiconductor substrate and in which some of the plurality of pixels are formed as phase difference detection pixels. Can do.

In the invention of claim 2, wherein the filter creation means, the frequency range of the inclusion area corresponding to the frequency band of the spatial frequency distribution similar to one of the spatial frequency distribution of the inclusion area and enclosed areas, areas of the subject Create a filter as With the created filter, for example, a filter that passes a low frequency band in the spatial frequency distribution, calculation based on the spatial frequency of only the subject becomes possible, and focusing accuracy is improved.

In the invention of claim 3, wherein the filter creation means, based on the spatial frequency distribution of the inclusion area and enclosed areas, frequency corresponding to each of said object regions and background regions on the spatial frequency distribution of the inclusion area separating the band, the frequency band corresponding to a region separate the object, to create a filter as an area of the subject. For example, by using the inflection point as the separation point on the low frequency band side of the spatial frequency, it is possible to separate the subject and the background into the spatial frequency, and the focusing accuracy is improved.

In the invention of claim 4, wherein the area setting means sets a circumscribed area circumscribing the periphery of at least contain and region of the subject around the area of the subject to the inclusion area, the subject comprises only region of the subject setting the inscribed area inscribing the outer periphery of the area containing the area. In this way, by setting the inscribed area and the circumscribed area, the number of pixels for performing the in-focus calculation can be increased, and the in-focus accuracy is improved.

In the invention of claim 5, wherein the area setting means, the inclusion area and enclosed area is a square, enclosed area inscribed in the outer peripheral region of the gradually expanding to the subject a rectangle from the center of the object to focus the alloy it sets a set inclusion area gradually reduced rectangle from ambient circumscribes the outer peripheral region of the subject region of the subject. Thus, the area setting is facilitated by a quadrangle, for example, a rectangle.

Enlarge the invention of claim 6, wherein the area setting means, when setting the contained area gradually expanding the square, only larger set amount determined in advance from the inscribed enclosed area on the outer peripheral region of the subject Set the enlarged area as the inclusion area. For example, when the area is extremely small, the area can be enlarged, so that the number of pixels for performing the focusing calculation can be increased, and the focusing accuracy is improved.

  According to a seventh aspect of the present invention, the imaging lens includes an aperture, and the area setting unit obtains a focal depth based on a focal length and an aperture value of the imaging lens, and the enlargement setting based on the obtained focal depth. Set the amount. The area can be enlarged according to the photographing conditions such as the depth of focus, and the focusing accuracy is improved.

  In the invention of claim 8, when the area setting means sets the inclusion area and the inclusion area to a rectangle and gradually sets the rectangle to set the inclusion area, the area setting unit and the area according to the number of pixels included in the rectangle For the rectangular aspect ratio, an area exceeding a predetermined threshold and an enlarged area with an aspect ratio are set as an inclusion area. Therefore, since the area can be sine by the pixel area and the aspect ratio (aspect ratio), the number of pixels necessary for the focusing process can be ensured.

In the invention of claim 9, wherein the area setting means determines a center of gravity of area of the object, it sets the encapsulated area gradually expanded heavy heart square was determined. Thereby, a starting point for setting an area faithful to the subject can be set.

According to a tenth aspect of the present invention, a pair pixel composed of a first phase difference detection pixel and a second phase difference detection pixel obtained by dividing a pupil in a phase difference detection area provided on a light receiving surface for picking up an image of a subject. And an imaging lens that is placed in front of the imaging element and controlled to a focus position that forms an optical image focused on the subject on the light receiving surface. an imaging program to be executed in, to identify the object to be focused, and encompasses the area including the area and the background area of the subject to be focused該合, Ri Do from part of the internal region of the subject to be focused該合background an area setting step of setting each of the enclosed area without the area, calculates the spatial frequency distribution of each of the inclusion area and enclosed area for which the setting of the object based on each computed spatial frequency distribution A filter creation step of creating a filter for removing spatial frequency of the frequency band corresponding to frequency, the first phase difference detection corresponding to the spatial frequency of the frequency range corresponding to the region of the subject using a filter that the created A generation step of generating first detection information of a pixel and second detection information of the second phase difference detection pixel, and a subject correlation calculation indicating a correlation between the generated first detection information and second detection information calculated curves, a phase difference calculating step for obtaining the phase difference controlling the correlation operation curve the imaging lens to the focus position, a position for controlling the imaging lens based on the phase difference the calculated on the alloy Asei location The control steps are executed by a computer.

  According to the present invention, by filtering the detection information of the phase difference detection pixels using a filter that obtains a spatial frequency based only on the subject, it is possible to focus on the subject with high accuracy even in the inclusion area including the background. Has the effect.

It is a schematic block diagram of an imaging device. FIG. 4 is a schematic enlarged view of a part of a surface within a phase difference detection area 40. It is a schematic diagram of the phase difference detection pixels 1x and 1y. It is a flowchart which shows the flow of the imaging process which concerns on 1st Embodiment. It is a flowchart which shows the flow of a focusing process. It is a schematic diagram of an area, (A) shows an in-subject area and (B) shows a subject inclusion area. It is a diagram which shows the spatial frequency distribution of a to-be-photographed area. It is a flowchart which shows the flow of the circumscribed area setting process which concerns on 2nd Embodiment. It is a flowchart which shows the flow of the inscribed area setting process which concerns on 2nd Embodiment. It is explanatory drawing about the flow of a setting process of an inscribed area. It is explanatory drawing about the flow of a setting process of circumscribed area. It is a flowchart which shows the flow of the inscribed area setting process which concerns on 3rd Embodiment. It is explanatory drawing about the flow of a setting process of an inscribed area. It is a diagram which shows the relationship of the expansion setting value with respect to the focal depth determined by imaging | photography conditions. It is a flowchart which shows the flow of the shape determination process which concerns on 4th Embodiment. It is explanatory drawing of the threshold value setting process regarding the shape of a to-be-photographed area. It is a flowchart which shows the flow of the gravity center setting process which concerns on 5th Embodiment. It is explanatory drawing about the flow of a gravity center setting process. It is a flowchart which shows the flow of the setting process of the to-be-photographed area based on 6th Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 shows a schematic block diagram of an imaging apparatus 10 according to an embodiment of the present invention. The imaging device 10 has a function of capturing a still image or a moving image of a subject and digitally processing the captured image signal in the imaging device 10, and an imaging lens 20 including a telephoto lens and a focus lens, and a back of the imaging lens 20. The solid-state image sensor 21 placed on the imaging plane and analog image data output from each pixel of the solid-state image sensor 21 are subjected to analog processing such as automatic gain adjustment (AGC) and correlated double sampling processing. In response to an instruction from the signal processing unit 22, an analog / digital conversion unit (A / D) 23 that converts analog image data output from the analog signal processing unit 22 into digital image data, and a system control unit (CPU) 29 described later. A drive control unit 24 that performs drive control of the A / D 23, the analog signal processing unit 22, the solid-state imaging device 21, and the imaging lens 20; It is configured to include a flash 25 that emits light in response to an instruction from the CPU 29.

  In addition, the imaging apparatus 10 includes a digital signal processing unit 26 that takes in digital image data output from the A / D 23 and performs interpolation processing, white balance correction, RGB / YC conversion processing, and the like. A compression / decompression processing unit 27 that compresses or decompresses the image in a reverse direction, a display unit 28 that displays a menu or the like, displays a through image or a captured image, and a system control unit (CPU) that performs overall control of the entire imaging apparatus. 29, an internal memory 30 such as a frame memory, and a media interface (I / F) unit 31 that performs an interface process between a recording medium 32 that stores JPEG image data and the like, and a bus 40 that interconnects them. The system control unit 29 is connected to an operation unit 33 for inputting instructions from the user.

  The system control unit 29 analyzes the captured image data (through image) output from the solid-state imaging device 21 in a moving image state and processed by the digital signal processing unit 26 using the subordinate digital signal processing unit 26 and the like as described later. To obtain an evaluation curve (correlation calculation curve) and detect the distance to the main subject. Then, the system control unit 29 controls the position of the focus lens of the imaging lens 20 via the driving unit 24 so that the optical image focused on the subject is formed on the light receiving surface of the solid-state imaging device 21. .

  In this embodiment, the solid-state imaging device 21 is a CMOS (Complementary Metal Oxide Semiconductor) type, and an output signal of the solid-state imaging device 21 is processed by an analog signal processing unit (AFE: analog front end) 22. A circuit that performs correlated double sampling processing and clamping, a signal amplifier circuit that performs gain control, and the like) are generally provided as a peripheral circuit on a solid-state imaging device chip. In addition, a horizontal scanning circuit, a vertical scanning circuit, a noise reduction circuit, a synchronization signal generation circuit, and the like are also formed as peripheral circuits around the light receiving unit on the chip of the solid-state imaging device 21, and the A / D conversion unit 23. May also be formed. The solid-state imaging device 21 may be a CCD (Charge Coupled Device) type and can be applied to the following embodiments.

  On the light receiving surface of the solid-state image sensor 21, a large number of pixels (light receiving elements: photodiodes) (not shown) are arranged in a two-dimensional array. In the present embodiment, a plurality of pixels are arranged in a square lattice pattern. Note that the pixel arrangement is not limited to a square lattice arrangement, and may be a so-called honeycomb pixel arrangement in which even-numbered pixel rows are shifted by 1/2 pixel pitch with respect to odd-numbered pixel rows.

  A rectangular phase difference detection area 40 is provided at an arbitrary partial region position (for example, the center position) of the light receiving surface. In this example, the phase difference detection area 40 is provided only at one location with respect to the light receiving surface, but may be provided at a plurality of locations so that it can be focused anywhere in the imaging screen. The entire area of the light receiving surface may be used as a phase difference detection area. The phase difference detection area 40 detects the in-focus position up to the subject in the phase difference detection direction (in this example, the left-right direction, that is, the x direction is the phase difference detection direction).

  FIG. 2 is a schematic enlarged view of a portion of the surface within the phase difference detection area 40. A large number of pixels are arranged in a square lattice on the light receiving surface of the solid-state imaging device 21, and the same applies to the phase difference detection area 40.

  In the illustrated example, each pixel is indicated by R (red), G (green), and B (blue). R, G, and B represent colors of the color filters stacked on each pixel, and the color filter array is a Bayer array in this example, but is not limited to the Bayer array, and other color filters such as stripes. An array may be used.

  The pixel array and the color filter array in the phase difference detection area 40 are the same as the pixel array and the color filter array on the light receiving surface outside the phase difference detection area 40. Pixels of the same color adjacent to are designated as phase difference detection pixels 1x and 1y. In the present embodiment, paired pixels for phase difference detection are provided at discrete and periodic positions in the area 40, in the illustrated example, at checkered positions.

  In the illustrated example, the same color pixels are diagonally adjacent because the color filter array is a Bayer array, and in the case of a horizontal stripe array, the same color pixels are arranged in the horizontal direction. Will be adjacent to. Alternatively, instead of providing two pixels constituting a pair in the same color filter row in the horizontal stripe arrangement, each pixel constituting the pair may be provided separately in each filter row of the same color closest in the vertical direction. The same applies to the vertical stripe arrangement.

  In the present embodiment, the phase difference detection pixels 1x and 1y are provided in the most G filter mounted pixels among R, G, and B, and 8 pixels are arranged in the horizontal direction (x direction) and in the vertical direction (y direction). It is arranged so that every eight pixels and the entire checkered position. Accordingly, when viewed in the phase difference detection direction (left-right direction), the phase difference detection pixels 1x are arranged every four pixels.

  FIG. 3 is a diagram schematically showing only the phase difference detection pixels 1x and 1y in FIG. For example, the phase difference detection pixels 1x and 1y constituting the pair pixel are formed such that the light shielding film openings 2x and 2y are smaller than the other pixels (pixels other than the phase difference detection pixels), and the light shielding film opening 2x of the pixel 1x. Is provided eccentrically in the left direction, and the light shielding film opening 2y of the pixel 1y is provided eccentrically in the right direction (phase difference detection direction).

  A curve X shown in the lower part of FIG. 3 is a graph in which detection signal amounts of the phase difference detection pixels 1x arranged in a horizontal row are plotted, and a curve Y is a detection signal of the phase difference detection pixel 1y that forms a pair with these pixels 1x. It is the graph which plotted quantity.

  Since the pair pixels 1x and 1y are adjacent pixels and are very close to each other, it is considered that light from the same subject is received. For this reason, it is considered that the curve X and the curve Y have the same shape, and the shift in the left-right direction (phase difference detection direction) is caused by the difference between the image seen by one pixel 1x of the paired pixel divided by the pupil and the other pixel 1y. This is the amount of phase difference from the image seen in.

By performing the correlation calculation between the curve X and the curve Y, the phase difference amount (lateral deviation amount) can be obtained, and the distance to the subject can be calculated from the phase difference amount. As a method for obtaining the evaluation value of the correlation amount between the curve X and the curve Y, a known method (for example, see Japanese Patent Application Laid-Open Nos. 2010-8443 and 2010-91991) can be used. For example, the integrated value of the absolute value of the difference between each point X (i) constituting the curve X and each point Y (i + j) constituting the curve Y is used as the evaluation value, and the j value giving the maximum evaluation value is expressed as the phase difference. The amount (lateral deviation).

  By the way, when the light receiving area of each pixel is small, the amount of each signal is small and the ratio of noise is increased. Therefore, it is difficult to detect the phase difference with high accuracy even if correlation calculation is performed. Therefore, by increasing the number of pixels in the phase difference detection area 40 or enlarging the phase difference detection area 40 itself, the influence of noise is reduced and the detection accuracy (AF accuracy) of the in-focus position is increased. It becomes possible to improve.

  However, when the number of pixels in the phase difference detection area 40 is increased or the phase difference detection area 40 itself is enlarged, the arrangement area of the phase difference detection pixels is changed in the vertical direction (vertical direction) or the horizontal direction (horizontal direction). It will extend to. As a result, the phase difference detection area (actual phase difference detection area) for the subject that should be focused will include the surrounding pattern such as the background in addition to the subject pattern that should be focused. The evaluation value for obtaining may decrease. For example, there may be a so-called rear pin state that is focused on a surrounding pattern such as the background.

  Therefore, in the present embodiment, with respect to the region for detecting the phase difference with respect to the subject to be focused, a partial area (inclusive area) of the subject and an entire area (inclusive area) including the periphery of the subject are defined, By obtaining a spatial frequency distribution for the area and focusing with a detection signal that is rarely related to the subject to be focused, for example, a detection signal with the background removed, the detection accuracy (AF accuracy) of the focus position is improved. .

  That is, the spatial frequency distributions of the subject and the background of the subject are generally different. For this reason, the spatial frequency distribution in which the spatial frequency of the subject appears prominently in the inclusion area, and in the inclusion area, the spatial frequency includes the spatial frequency of both the subject and an article that is rarely related to the subject such as the background. Distribution. By removing the spatial frequency of an article that is rarely related to the subject such as the background from the spatial frequency of the inclusion area, a spatial frequency distribution faithful to the subject can be adopted as the actual phase difference detection area.

  Next, the operation of the imaging device 10 according to the present embodiment will be described. First, the overall operation of the imaging apparatus 10 during shooting will be briefly described.

  First, imaging is performed via the imaging lens 20 by the solid-state imaging device 21, and R (red), G (green), and B (blue) signals indicating the subject image are sequentially output to the analog signal processing unit 22. The analog signal processing unit 22 performs analog signal processing such as correlated double sampling processing on the signal input from the solid-state imaging device 21 and then sequentially outputs the signal to the A / D conversion unit 23. The A / D conversion unit 23 converts the R, G, and B signals input from the analog signal processing unit 22 into, for example, 12-bit R, G, and B signals (digital image data), respectively. 26 are sequentially output. The digital signal processing unit 26 accumulates digital image data sequentially output from the A / D conversion unit 23 in a built-in line buffer and temporarily stores the digital image data in a predetermined area of the internal memory 30.

  The digital image data stored in a predetermined area of the internal memory 30 is read by the digital signal processing unit 26 under the control of the system control unit 29, and white balance adjustment is performed by applying a digital gain corresponding to a predetermined physical quantity to these. In addition, gamma processing and sharpness processing are performed to generate, for example, 8-bit digital image data, and further YC signal processing is performed to generate a luminance signal Y and chroma signals Cr and Cb (hereinafter referred to as “YC signal”). Then, the YC signal is stored in an area different from the predetermined area of the internal memory 30.

  The display unit 28 is configured to display a moving image (through image) obtained by continuous imaging by the solid-state imaging device 21 and to be used as a finder. In this case, the generated YC signals are sequentially output to the display unit 28. As a result, a through image is displayed on the display unit 28.

  Next, the outline of the process of the system control unit 29 will be described with reference to the imaging process routine shown in FIG.

  The photographer determines the composition while confirming the subject on the through image displayed on the display unit 28. At this time, when the release button of the operation unit 33 is pressed halfway by the photographer, the system control unit 29 exposes the exposure state (AE (Automatic Exposure) function) to such an extent that a through image can be confirmed. Setting of the shutter speed and aperture state) and focusing control using an AF (Auto Focus) function are performed, and the subject is recognized with the through image (step 100). Thereafter, when the release button is fully depressed S2 (step 102), after the AE function is activated and the final exposure state is set (step 104), the AF function, which will be described in detail later, is activated and the focus control is performed. Then, the solid-state image sensor 21 is continuously exposed by the imaging lens 20 (step 108), and an analog signal is read from the solid-state image sensor 21 (step 110). Next, the analog signal is converted into digital image data (step 112), image processing such as white balance adjustment, gamma processing, and sharpness processing is performed (step 114), and the compression / decompression processing unit 27 performs a predetermined compression format (for example, After compression in the JPEG format (step 116), recording is performed on the recording medium 32 via the media interface unit 31 (step 118).

  Next, the focusing process in step 106 of FIG. 4 will be described in detail with reference to the flowchart shown in FIG.

  When the AF process is started, an object recognition process is executed in step 120. This object recognition process is a process for specifying a subject that has been determined as a photographing target as a result of the process of recognizing a subject with a through image performed in step 100. An example of this object is a person's face, and the process here can execute a face detection process. In the next step 122, it is determined whether or not an object such as a face has been detected. Note that the object recognition process can use a known method (for example, see JP-A-2006-270137, JP-A-2007-142525, etc.).

  When the object is not detected, the process proceeds to step 138, and the phase difference detection area 40 is obtained from the curve X obtained from the phase difference detection pixel 1x and the phase difference detection pixel 1y in the same manner as in the conventional phase difference AF. Correlation calculation with the curve Y is performed. Then, based on the area evaluation curve obtained as a result of the correlation calculation, a defocus amount is calculated (step 140), and AF determination is performed in the next step 142. This AF determination is determined by determining whether or not the result of the correlation calculation is within an allowable range, determining whether or not the calculated defocus amount is within an allowable range, and the like. If an affirmative determination is made in step 142, in step 144, focusing control of the imaging lens is performed, and this process is terminated. If a negative determination is made in step 142, a predetermined error process is executed in step 146.

  On the other hand, if an object is detected (Yes in step 122), the process proceeds to step 124, and the number of subject areas for AF processing (two in this embodiment, large and small) is determined. The number of subject areas is not limited to two, but may be three large, medium, and small, or more.

  In the next step 126, a subject inclusion area (large area) including at least all the subjects of the object recognized in step 120 is set as one of the subject areas determined in step 124. The spatial frequency distribution of the inclusion area is calculated. In the next step 130, an in-subject area (small area) that is an area in the subject that is the object is set as the other subject area, and in the next step 132, the spatial frequency distribution of the in-subject area is calculated.

  FIG. 6 schematically shows the large and small areas set in steps 126 and 130, where (A) shows the in-subject area and (B) shows the subject inclusion area. FIG. 6 shows a case where a rectangular shape is adopted for the subject area. The subject area is not limited to a rectangular shape. As shown in FIG. 6A, the in-subject area 60 is a closed area inside the face 50 that is the subject, and is within the subject-only area 64. As shown in FIG. 6B, the subject inclusion area 62 is set to a closed region that includes the entire region of the face 50 that is the subject, and is composed of a subject-only area 64 and a background area 66. . When the subject area is rectangular, it is preferable to set an inscribed area that is inscribed in the contour of the face 50 from the viewpoint of securing the area. The subject inclusion area 62 is preferably set as a circumscribed area circumscribing the outline of the face 50.

  FIG. 7 shows the spatial frequency distribution of the subject area. A curve 70 shows the spatial frequency distribution of the in-subject area 60, and a curve 72 shows the spatial frequency distribution of the subject inclusion area 62. The curve 70 is a characteristic having a peak 74 on the low frequency side, and the curve 72 is a characteristic having peaks 76 and 78 on both the low frequency side and the high frequency side. On the other hand, the subject inclusion area 62 is composed of a subject-only area 64 and a background area 66, in which both spatial frequency distributions are superimposed. On the other hand, the in-subject area 60 is a spatial frequency distribution of the subject. From this, it can be estimated that the frequency region including the peak 74 and the peak 76 on the low frequency side corresponds to the subject, and the frequency region including the peak 78 is due to the background area 66. Therefore, if a certain frequency range including the peak 78 is removed from the curve 72, a spatial frequency (band) based only on the subject can be obtained. Thus, if the phase difference detection signal is subjected to filter calculation using a filter that passes through the spatial frequency band of only the subject, the subject can be accurately focused even if the area including the background is determined as the subject area.

  In step 134, a filter is created from the spatial frequencies calculated in steps 128 and 132. Here, in order to determine a certain frequency range including the peak 78 for the curve 72, the inflection point of the curve 72, that is, the point 80 having the lowest frequency distribution from the peak 76 to the peak 78 is obtained, and the spatial frequency fx is obtained. Set to the boundary spatial frequency. Then, a filter that band-passes the frequency band F up to the spatial frequency fx including the spatial frequency of the peak 76 is created.

  In the next step 136, the phase difference detection signal is signal-processed using the filter created in step 134. This signal-processed signal has the background area removed. Then, as described above, the correlation calculation is performed, the defocus amount is calculated based on the obtained area evaluation curve, and the focusing control of the imaging lens is performed (steps 138 to 144), so that the area including the background is determined. Even if the subject area is determined, the subject can be focused accurately.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. In addition, the same code | symbol is attached | subjected to the same part as the said embodiment, and the detailed description is abbreviate | omitted.

  The present embodiment shows an example of the setting of the subject area, and the others are the same as in the first embodiment.

  In the present embodiment, the circumscribed area setting process shown in FIG. 8 is executed for the process of step 126 in FIG. In the process of step 130 in FIG. 5, the inscribed area setting process shown in FIG. 9 is executed. Hereinafter, a detailed description will be given with reference to the flowcharts of FIGS.

  When the subject inclusion area setting process is executed (step 126 in FIG. 5), the process proceeds to step 150 in FIG. 8, and a boundary line (contour) 54 of the subject 52 of the recognized object is obtained. Next, the maximum value Xmax and the minimum value Xmin of the abscissa are determined for the boundary line (contour) 54 (step 152), and the maximum value Ymax and the minimum value Ymin of the ordinate are determined (step 154). From these coordinates, in the next step 156, the circumscribed area 63 is set as the subject inclusion area (large area). For setting the circumscribed area 63, first, a diagonal coordinate point 63A (Xmin, Ymin) and a diagonal coordinate point 63B (Xmax, Ymax) are obtained. A quadrangle that is a diagonal line of the diagonal coordinate points 63A and 63B is set in the circumscribed area 63 (see FIG. 11).

  When the subject area setting process is executed (step 130 in FIG. 5), the process proceeds to step 158 in FIG. 9 to obtain the center point 61P of the subject 52 of the recognized object. The center point 61P may be the center obtained at the time of object recognition, or may be the intersection of the diagonal lines of the circumscribed area 63 obtained in step 156. In the next step 160, the boundary line (outline) 54 of the subject 52 is set in the same manner as in step 150. Next, a predetermined size (for example, surrounding 8 pixels) around the center point 61P is set as the extension area 61A. Then, the expansion area is expanded until the side or corner of the expansion area touches the boundary line (contour) 54 (see FIG. 10, the order of the expansion areas 61A, 61B, and 61C). The extended area (rectangle) where the sides or corners are in contact is set as the inscribed area 63 (see FIG. 10).

  For the expansion of the expansion area, for example, surrounding pixels may be increased one by one, or pixels in a direction in which sides or corners are not in contact may be increased.

  In the present embodiment, even if the subject has a complicated outline, each of the circumscribed area and the inscribed area can be set to be quick and maximum area.

(Third embodiment)
Next, a third embodiment of the present invention will be described. In addition, the same code | symbol is attached | subjected to the same part as the said embodiment, and the detailed description is abbreviate | omitted.

  In the present embodiment, an allowable range is provided for setting the in-subject area, and the configuration is the same as that of the above-described embodiment.

  When an in-subject area is set, if the set subject area, for example, the inscribed area is extremely small, the minimum inscribed area may not have a sufficient number of pixels for focusing. Therefore, in this embodiment, when the inscribed area is minimal, the inscribed area can be set to be enlarged to a predetermined allowable range. The maximum value of this allowable range is the set circumscribed area. That is, the enlarged area 65 that is enlarged from the inscribed area 61 to the circumscribed area 63 can be set as the inscribed area 61 (see FIG. 13).

  In the present embodiment, when the inscribed area set in step 168 in FIG. 9 is minimal, the inscribed area setting process shown in FIG. 12 is executed. Hereinafter, this will be described in detail with reference to the flowchart of FIG.

  When the setting of the in-subject area (inscribed area) is completed (FIG. 9), the process proceeds to step 170 in FIG. 12 to determine whether or not the set inscribed area is minimal. Since the area can secure a sufficient number of pixels for focusing, the processing is terminated as it is. As a criterion for determining whether or not the inscribed area is extremely small, a predetermined number of pixels as the number of pixels sufficient for focusing or an area based on the number of pixels can be used.

  If the determination in step 170 is affirmative, the process proceeds to step 172, where an allowable range (enlargement setting value) of the inscribed area that can be enlarged is set. In the next step 174, the expansion area is expanded, for example, by one pixel, and is repeatedly executed until the expansion set value is reached (until affirmative in step 176). Then, the final extended area (Yes in step 176) is set as the in-subject area (step 178).

  As described above, the enlargement set value can be a circumscribed area that has been set as the maximum value of the allowable range (enlargement set value) (see FIG. 13), but is a predetermined pixel required for focusing. It will be enough if it becomes a number area. Further, if the extended area is enlarged to the vicinity of the circumscribed area, it will include a lot of background, which may hinder the focusing accuracy. Therefore, as the enlargement setting value, a value up to the circumscribed area that has already been set (for example, the number of pixels or the area) may be set, but a predetermined specified value (for example, a preliminarily required for focusing) It is preferable to set a predetermined number of pixels.

  By the way, in actual shooting, shooting conditions differ depending on the shooting scene. For example, at the time of short-distance shooting such as macro shooting, the depth of focus (depth of field) becomes shallow depending on the shooting conditions such as the focal length and aperture value. On the other hand, the depth of focus is relatively deep under shooting conditions such as landscapes and group photos at close range shooting. As described above, since the depth of focus varies depending on the shooting conditions, if the enlargement setting value is uniformly determined and the expansion area can be enlarged, the subject is not focused but the background is focused (so-called rear pin). Focusing accuracy deteriorates.

  Therefore, in step 172, it is preferable to set an allowable range (enlargement setting value) of the inscribed area that can be enlarged according to the photographing conditions.

  FIG. 14 shows the relationship between the enlargement setting value and the focal depth (depth of field) determined by the photographing condition based on the focal length and the aperture value. There is a correspondence between the depth of focus (depth of field) and the enlargement setting value. That is, if the depth of focus is deep, the focusing distance in the photographing axis direction becomes long. Therefore, even if the expansion area includes some background, the influence on focusing is reduced, and the enlargement setting value can be increased. On the other hand, if the depth of focus is shallow, the focusing distance in the photographing axis direction is shortened, so the enlargement setting value needs to be small. Therefore, if the depth of focus (depth of field) at the time of shooting is obtained, the permissible range (enlargement setting value) of the inscribed area that can be expanded can be set according to the shooting conditions. In FIG. 14, as an example, a characteristic curve h that is provided in the imaging apparatus 10 and that can set an allowable range (enlargement setting value) of an inscribed area that can be enlarged according to a zoom position that directly affects the focal length is shown. Indicated. That is, the focal length becomes shorter on the tele side, and the focal length becomes longer on the wide side. An imaging range such as an aperture may be further added to the characteristic curve h to set an allowable range (enlarged setting value).

  In addition, the above-described spatial frequency distribution can be used to determine the allowable range of the inscribed area that can be expanded according to the imaging conditions. As shown in FIG. 7, in the curve 72 of the spatial frequency distribution of the subject area including the background, the frequency region including the peak 78 can be estimated as the background area 66 by the above-described processing. On the other hand, enlarging the extended area includes more of the background area 66. That is, the spatial frequency fx shifts to the high frequency side. Therefore, the extent of the influence of the background area 66 when the spatial frequency fx is shifted to the high frequency side by enlarging the expansion area is obtained by determining the variation ratio of the spatial frequency region including the peak 78 from the spatial frequency fx. The allowable range (enlargement setting value) may be set based on a predetermined relationship with the focal length (imaging condition).

  In this embodiment, even a very small subject area (inscribed area) can be expanded by the value (for example, the number of pixels and area) determined by the shooting conditions such as the focal length and aperture value. The number of pixels for the focus calculation can be increased, and the focusing accuracy can be improved.

  In addition, although the said process was performed after step 168, you may make it change a threshold value. That is, in FIG. 9, the threshold is set when the extended area contacts the boundary line (outline) of the subject. However, the minimum of the area may be determined for the threshold determination. Specifically, the processing of FIG. 12 may be executed after the above-described determination of whether or not contact is possible in step 164.

(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described. In addition, the same code | symbol is attached | subjected to the same part as the said embodiment, and the detailed description is abbreviate | omitted.

  In the present embodiment, in order to maintain focus calculation accuracy, a subject area is set by providing a threshold value for the shape of the subject area. That is, in the present embodiment, the shape determination process shown in FIG. 15 is executed after the process of step 166 of FIG. Hereinafter, this will be described in detail with reference to the flowchart of FIG.

  When the extended area is expanded (step 166 in FIG. 9), the process proceeds to step 180 in FIG. 15, and a threshold relating to the shape of the subject area for maintaining the focus calculation accuracy is set. This threshold value is a threshold value of the area based on the number of pixels and a threshold value of an aspect ratio (aspect ratio) of the area. In the next step 182, the area and aspect ratio for the extended area set in step 166 are calculated, and it is determined whether or not the set threshold value is exceeded (steps 184 and 186).

  If the result in step 184 or 186 is negative, the focusing calculation accuracy is insufficient and the process is continued. That is, the process returns to step 166 to reset the extended area. Here, when the area of the extension area is less than the threshold, a process for extending the extension area is instructed. Further, when the aspect ratio is less than the threshold value, an instruction to extend the aspect ratio of the extension area to be equal to or greater than the threshold value is instructed.

  On the other hand, if the result is affirmative in steps 184 and 186, a sufficient area (number of pixels) and area shape for focusing can be secured, and the shape determination process is terminated and the process returns to step 164. In this case, the process is not limited to returning to step 164. For example, since it is an extended area that can secure a sufficient area and area shape for focusing, the process may proceed to step 168 as it is.

  Thereby, in the present embodiment, a sufficient area and area shape for focusing can be ensured regardless of the size of the subject 52 (see FIG. 16).

(Fifth embodiment)
Next, a fifth embodiment of the present invention will be described. In addition, the same code | symbol is attached | subjected to the same part as the said embodiment, and the detailed description is abbreviate | omitted.

  The present embodiment shows an example in which the center of gravity of a subject is obtained as an example of obtaining a base point when setting an in-subject area, and the rest is the same as the above embodiment.

  In the present embodiment, the center-of-gravity setting process shown in FIG. 17 is executed instead of the process of step 158 in FIG. Hereinafter, this will be described in detail with reference to the flowchart of FIG.

  When the in-subject area setting process is executed (step 130 in FIG. 5), the process proceeds to step 190 in FIG. 17 instead of the process in step 158 (FIG. 9), and the area (number of pixels) of the subject area is calculated. In this process, the number of pixels in the closed region by the boundary line is obtained using the boundary line of the subject. In the next step 192, pixels are spread from one end in the horizontal direction of the subject, and the vertical straight line passing through the pixel position where the number of pixels is half the number of pixels obtained in step 190 is defined as the vertical dividing line 52V. (See FIG. 18). In the next step 194, pixels are laid out from one end of the subject in the vertical direction, and a horizontal straight line passing through the pixel position where the number of pixels obtained in step 190 is half the number of pixels is defined as a horizontal dividing line 52H. (See FIG. 18). Then, in the next step 196, the intersection 52G between the obtained vertical dividing line 52V and horizontal dividing line 52H is set as the center of gravity instead of the above-mentioned center point (see FIG. 18).

  In the present embodiment, since the center of gravity of the subject is obtained, it is possible to prevent the setting of the inscribed area from being biased by shifting the point serving as the base point of the in-subject area even if the subject has a complex outline.

(Sixth embodiment)
Next, a sixth embodiment of the present invention will be described. In addition, the same code | symbol is attached | subjected to the same part as the said embodiment, and the detailed description is abbreviate | omitted.

  The present embodiment is a combination of the second embodiment and the fifth embodiment. In the present embodiment, the process shown in FIG. 19 is executed instead of the process in step 158 of FIG. Hereinafter, this will be described in detail with reference to the flowchart of FIG.

  First, when the subject area setting process is executed (step 130 in FIG. 5), the process proceeds to step 200 in FIG. 19 instead of the process in step 158 (FIG. 9), and in the same manner as in step 158 in FIG. The center of is calculated. In the next step 202, the center of gravity of the subject is calculated (FIG. 17). In the next step 204, an inscribed area based on the subject center is calculated, and in step 206, an inscribed area based on the subject center of gravity is calculated. In the next step 208, it is determined whether or not the area of the inscribed area based on the determined subject center of gravity is larger than the area of the inscribed area based on the subject center. If the determination in step 208 is affirmative, the inscribed area based on the subject center of gravity is adopted, and if the result is negative, the inscribed area based on the subject center is adopted.

  In the present embodiment, an area having a large area is employed as the inscribed area determined by the center of gravity or the center of the subject, so that the focusing accuracy can be improved by increasing the number of pixels.

  As described above, according to the present embodiment, at the time of focusing, the spatial frequency distribution of a part of the subject and the spatial frequency distribution of the area including all the background and including the background are obtained, and the spatial frequency band only by the subject is obtained. By filtering the phase difference detection signal using a filter that passes through, the subject can be accurately focused even if the area including the background is determined as the subject area.

  In each of the above embodiments, the pupil-divided pair pixels constituting the phase difference detection pixel have been described using an example in which the reduced light shielding film openings are shifted in opposite directions. The method of configuring is not limited to this, and for example, one microlens may be mounted on a pair of pixels to divide the pupil.

  In each of the above embodiments, the example in which the pair pixels for detecting the phase difference are provided at discrete and periodic positions in the phase difference detection area has been described. However, the pixel need not necessarily be provided at the periodic and discrete positions. The position may be random, or all pixels may be phase difference detection pixels.

  In each of the above embodiments, the color filter arrangement of the three primary color filters of RGB has been described, but the type of color filter is not limited to this.

  Further, in each of the above embodiments, when creating a filter, the spatial frequencies of the in-subject area and the subject-inclusive area are compared. When the comparison is made, it is determined that the spatial frequency of the subject and the background are the same. There is. In this case, the correlation calculation may be performed using the in-subject area.

DESCRIPTION OF SYMBOLS 10 Imaging device 20 Imaging lens 21 Solid-state image sensor 28 Display part 29 System control part 40 Phase difference detection area 60 In-subject area 62 Subject inclusion area 70 Curve 72 Curve

Claims (10)

An imaging element in which a pair of pixels composed of a first phase difference detection pixel and a second phase difference detection pixel divided into pupils are arrayed in a phase difference detection area provided on a light receiving surface for picking up an image of a subject. When,
An imaging lens that is placed in front of the imaging device and controlled to a focus position that forms an optical image of the subject focused on the light receiving surface;
Identify the object to be focused, each of the inclusion area including the area and the background area of the subject to be focused該合, Ri Do from part of the internal region of the subject to be focused 該合, the encapsulated area that does not include the background area Area setting means for setting
Filter creation means for computing a spatial frequency distribution of each of the set inclusion area and inclusion area and creating a filter that removes the spatial frequency of the frequency range corresponding to the subject area based on each of the calculated spatial frequency distributions When,
First detection information of the first phase difference detection pixel and second detection information of the second phase difference detection pixel corresponding to a spatial frequency in a frequency range corresponding to the region of the subject using the created filter, Generating means for generating
Phase difference calculation means for obtaining a correlation calculation curve of a subject indicating a correlation between the generated first detection information and second detection information, and calculating a phase difference for controlling the imaging lens to the in-focus position from the correlation calculation curve When,
And position control means for controlling the imaging lens based on the calculated phase difference in the case Asei location,
An imaging apparatus comprising: The filter creation means, the frequency range of the inclusion area corresponding to the frequency band of the spatial frequency distribution similar to one of the spatial frequency distribution of the inclusion area and enclosed areas, to create a filter as a region of the subject The imaging apparatus according to claim 1, wherein the imaging apparatus is characterized. It said filter producing means, the inclusion area and on the basis of the spatial frequency distribution of the encapsulated area, separated into frequency range corresponding to each of the region and the background region of the subject on a spatial frequency distribution of the inclusion area and separated the frequency range corresponding to a region of the subject, the imaging apparatus according to claim 1 or claim 2, characterized in that to create a filter as a region of the subject. It said area setting means sets a circumscribed area circumscribing at least contain and the outer peripheral region of the subject around the area of the subject to the inclusion area, inscribing the outer periphery region of the object comprises only region of the subject The imaging apparatus according to claim 1, wherein the inscribed area is set as an inclusion area. Said area setting means, together with the a quadrangle inclusion area and enclosed areas, gradually expanding the square from the center of the object to focus the case to set the enclosed area that is inscribed in the outer peripheral region of the subject, the subject the imaging apparatus according to claim 4 from the surrounding area by reducing gradually the rectangle and sets the inclusion area circumscribes the outer peripheral region of the subject. Said area setting means, when setting the contained area gradually expanding the square, the enlarged area enlarged by enlarging the set amount a predetermined from enclosed areas inscribed in the outer peripheral region of the subject as contained area The imaging device according to claim 5, wherein the imaging device is set. The imaging lens includes a diaphragm,
The imaging apparatus according to claim 6, wherein the area setting unit obtains a focal depth based on a focal length and an aperture value of the imaging lens, and sets the enlargement setting amount based on the obtained focal depth. . In the case where the area setting means sets the inclusion area and the inclusion area as a rectangle and sets the inclusion area by gradually expanding the rectangle, the area according to the number of pixels included in the rectangle and the aspect ratio of the rectangle are set in advance. The imaging device according to any one of claims 5 to 7, wherein an area that exceeds a predetermined threshold and an enlarged area with an aspect ratio are set as an inclusion area. It said area setting means determines a center of gravity of area of the subject, according to any one of claims 5 to 8, characterized in that to set the enclosed area gradually expanded heavy heart square was determined Imaging device. An imaging element in which a pair of pixels composed of a first phase difference detection pixel and a second phase difference detection pixel divided into pupils are arrayed in a phase difference detection area provided on a light receiving surface for picking up an image of a subject. And an imaging lens that is placed in front of the imaging device and controlled to a focusing position that focuses the focused optical image of the subject on the light receiving surface. And
Identify the object to be focused, each of the inclusion area including the area and the background area of the subject to be focused該合, Ri Do from part of the internal region of the subject to be focused 該合, the encapsulated area that does not include the background area An area setting step to set
A filter creation step of calculating a spatial frequency distribution of each of the set inclusion area and inclusion area and creating a filter that removes a spatial frequency of a frequency range corresponding to the subject area based on each of the calculated spatial frequency distributions When,
First detection information of the first phase difference detection pixel and second detection information of the second phase difference detection pixel corresponding to a spatial frequency in a frequency range corresponding to the region of the subject using the created filter, A generation step for generating
A phase difference calculation step of obtaining a correlation calculation curve of a subject indicating a correlation between the generated first detection information and second detection information, and obtaining a phase difference for controlling the imaging lens to the in-focus position from the correlation calculation curve. When,
A position control step of controlling the imaging lens based on the calculated phase difference in the case Asei location,
An imaging program for causing a computer to execute the steps.