JP5853369B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program, and more particularly, to a technique for focusing on a subject of interest even when a fence or the like is present in front.

  2. Description of the Related Art Conventionally, there is an imaging apparatus that can execute AF (Automatic Focus) processing. As a typical AF evaluation processing method, the frequency component (excluding the DC component) included in the image signal is maximized at the focus lens position (focus position), and the frequency included in the image signal is used. There is one that detects the in-focus state using the amount of the component as an evaluation value. The evaluation value detected by this method is detected from one area of the captured image. The detected evaluation value is hereinafter referred to as “AF evaluation value”, and the one area where the AF evaluation value is detected is hereinafter referred to as “AF evaluation area”.

  As a conventional setting method for such an AF evaluation area, a focus lens position is calculated for each of a plurality of AF evaluation areas, and the closest focus lens position is calculated among the calculated focus lens positions. A method of selecting a corresponding AF evaluation area is described in Patent Document 1.

JP 2004-361484 A

  However, in the method described in Patent Document 1, when a fence or the like is present in front of a subject such as an animal as a special case, the position of the fence or the like may be calculated as the focus lens position. In other words, since the fence or the like is in front of the subject and is often located closest to the imaging device, the position of the fence or the like is calculated as the focusing lens position, the AF evaluation area is selected, and the fence or the like is focused. Resulting in. In other words, if there is a fence or the like in front of the subject, this may be regarded as a subject to be focused on and may not be focused on the subject to be truly focused.

  The present invention has been made in view of such a situation. In an environment in which a plurality of targets exist in the perspective direction, image processing focusing on a target at a far position is performed. The object is to focus on the subject of interest even if there is a fence or the like in front.

In order to achieve the above object, an image processing apparatus according to an aspect of the present invention includes an acquisition unit that acquires an image, and all edges whose gradient direction is substantially constant based on the luminance gradient of the image. And an AF evaluation value calculating means for calculating an AF evaluation value from an image area excluding an area including all edges having a substantially constant gradient direction specified by the specifying means. Features.

  According to the present invention, even when a fence or the like is present in front of the subject of interest, the subject of interest can be focused.

1 is a block diagram illustrating a hardware configuration of an imaging apparatus that is an embodiment of an image processing apparatus according to the present invention. It is a functional block diagram which shows the functional structure for performing AF evaluation process among the functional structures of the imaging device of FIG. It is a schematic diagram which shows a through image. It is a schematic diagram which shows the luminance image produced | generated by extracting a luminance value from the through image of FIG. FIG. 5 is a schematic diagram showing a part of the luminance image of FIG. 4 enlarged to a pixel level. It is a figure which shows preparation of a gradient histogram. It is a flowchart explaining the flow of AF evaluation processing. It is a figure which shows a modification.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a hardware configuration of an imaging apparatus 1 which is an embodiment of an image processing apparatus according to the present invention.

  The imaging device 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, an image processing unit 14, a bus 15, an input / output interface 16, and an imaging unit. 17, an operation unit 18, a display unit 19, a storage unit 20, a communication unit 21, and a drive 22.

  The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 20 to the RAM 13.

  The RAM 13 appropriately stores data necessary for the CPU 11 to execute various processes.

The image processing unit 14 is configured by a DSP (Digital Signal Processor), a VRAM (Video Random Access Memory), and the like, and performs various image processing on image data in cooperation with the CPU 11.
For example, the image processing unit 14 performs image processing such as noise reduction, white balance, camera shake correction, and the like on captured image data output from the imaging unit 17 described later.

  The CPU 11, ROM 12, RAM 13, and image processing unit 14 are connected to each other via a bus 15. An input / output interface 16 is also connected to the bus 15. An imaging unit 17, an operation unit 18, a display unit 19, a storage unit 20, a communication unit 21, and a drive 22 are connected to the input / output interface 16.

  The imaging unit 17 includes an optical system 31, an image sensor 32, and a lens driving unit 33.

The optical system 31 includes a lens that collects light for imaging a subject, such as a focus lens and a zoom lens.
The focus lens is a lens that forms a subject image on the light receiving surface of the image sensor 32, and is a lens that focuses the subject by adjusting a position (focusing position) by a lens driving unit 33 described later. .
The zoom lens is a lens that freely changes the focal length within a certain range.

The image sensor 32 includes a photoelectric conversion element, an AFE (Analog Front End), and the like.
The photoelectric conversion element is composed of, for example, a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element or the like. The subject image is incident on the photoelectric conversion element from the optical system 31. Therefore, the photoelectric conversion element photoelectrically converts (captures) the subject image, accumulates the image signal for a predetermined time, and sequentially supplies the accumulated image signal as an analog signal to the AFE.
The AFE performs various signal processing such as A / D (Analog / Digital) conversion processing on the analog image signal. A digital signal is generated by various signal processing in the AFE and output as an output signal of the imaging unit 17.
Hereinafter, the output signal of the imaging unit 17 is referred to as “captured image data”. Accordingly, captured image data is output from the imaging unit 17 and is appropriately supplied to the CPU 11, the image processing unit 14, the storage unit 20, and the like.

  The lens driving unit 33 adjusts each position by driving a focus lens, a zoom lens, and the like constituting the optical system 31 based on the control of the CPU 11 and the like. For example, the lens driving unit 33 drives the zoom lens and adjusts the position based on the result of AF position determination processing by the CPU 11 described later.

  The operation unit 18 includes, for example, various buttons such as a relay button (not shown), and accepts a user instruction operation.

  The display unit 19 includes a display that can display various images.

  The storage unit 20 is configured by a DRAM (Dynamic Random Access Memory) or the like, and stores data of various images such as a through image described later, an original image to be displayed, and an image that can be combined with the original image.

  The communication unit 21 controls communication with other devices (not shown) via a network including the Internet.

  A removable medium 23 composed of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately attached to the drive 22. The program read from the removable medium 23 by the drive 22 is installed in the storage unit 20 as necessary. The removable medium 23 can also store various data such as image data stored in the storage unit 20 in the same manner as the storage unit 20.

  With the hardware configuration described above, the imaging apparatus 1 can execute AF processing that can focus on a subject of interest even when a fence or the like is present in front.

  As AF processing, processing for calculating an AF evaluation value at each position of the focus lens (hereinafter referred to as AF evaluation processing) and processing for determining an in-focus position by comparing the calculated AF evaluation values (hereinafter referred to as AF). Called position determination processing).

In the present embodiment, as an AF evaluation process among such AF processes, an AF evaluation value is considered without considering the shielding object in order to focus on the subject of interest even when a shielding object such as a fence exists in front. AF evaluation processing that enables calculation of the above is adopted.
Therefore, the AF evaluation process adopted in the present embodiment will be mainly described below.

  FIG. 2 is a functional block diagram showing a functional configuration for executing the AF evaluation process among the functional configurations of the imaging apparatus 1 of FIG.

  In the present embodiment, the image processing unit 14 performs a AF evaluation process, and a through image acquisition unit 41, a luminance image generation unit 42, a noise removal unit 43, a gradient histogram creation unit 44, and a gradient peak selection unit. 45, an excluded pixel selection unit 46, an AF evaluation image generation unit 47, and an AF evaluation unit 48.

  2 is merely an example, and it is naturally possible to transfer at least a part of the functions of the image processing unit 14 to other components (CPU 11 or the like) other than the image processing unit 14.

The through image acquisition unit 41 acquires the captured image data output from the imaging unit 17 as through image data.
That is, the CPU 11 or the like displays the through image on the display unit 19 by executing the through imaging process and the through display process during or before the AF evaluation process.
Specifically, for example, when the release button of the operation unit 18 is half-pressed, the CPU 11 or the like continues the imaging operation by the imaging unit 17. The CPU 11 and the like temporarily store captured image data sequentially output from the imaging unit 17 in the memory (the storage unit 20 in the present embodiment) while the imaging operation by the imaging unit 17 is continued. . Such a series of control processes is the “through imaging process” referred to herein.
In addition, the CPU 11 and the like sequentially read out data of each captured image temporarily recorded in the memory (the storage unit 20 in the present embodiment) during the through imaging process, and sequentially display the captured image on the display unit 19. Let Such a series of control processes is the “through display process” here, and the captured image displayed on the display unit 19 by the through display process is hereinafter referred to as a “through image”.

Here, in order to facilitate understanding of the present embodiment, it will be described below that the through image acquisition unit 41 has acquired through image data as shown in FIG.
FIG. 3 is a schematic diagram illustrating an example of a through image.
Specifically, in FIG. 3, a captured image obtained when the imaging unit 17 images a subject PO (for example, a tree) that is present behind the fence F is displayed on the display unit 19 as a through image. The situation is shown.
Here, the fence F corresponds to a shield existing in front of the subject PO, and is formed to have a parallelogram mesh shape by combining a plurality of column lines bent at a predetermined angle. A fence, a so-called diamond wire mesh (hereinafter referred to as “fence”).
In the through image, since the fence F is in front of the subject and is closest to the imaging apparatus, the position of the fence F is calculated as the focus lens position and is in focus on the fence F.

Returning to FIG. 2, the luminance image generation unit 42 extracts luminance components in pixel units from the data of the through image acquired by the through image acquisition unit 41, and images each having the extracted luminance components as pixel values (hereinafter referred to as pixel values). , Called “luminance image”).
For example, the luminance image generation unit 42 generates luminance image data as illustrated in FIG. 4 from the through image data as illustrated in FIG. 3.
FIG. 4 is a schematic diagram illustrating an example of a luminance image generated from the through image of FIG.
In the luminance image in the example of FIG. 4, the luminance decreases as the color becomes black, and the luminance increases as the color becomes white. For example, the fence F is clearly shown in its shape and is displayed in a color close to white due to its high brightness.

  Returning to FIG. 2, the noise removing unit 43 performs a process of removing noise on the luminance image data generated by the luminance image generating unit 42. A noise removal method in the noise removal unit 43 suffices as long as the method mainly smoothes a change in luminance in an image, that is, a so-called smoothing process. In the present embodiment, a Gaussian filter is sufficient. A method using a (Gaussian filter) is employed.

  Then, the intensity of the edge is calculated by calculating the luminance gradient in the image, the local direction of the edge is predicted from the direction of the gradient, and a part where the gradient in the direction is locally maximum is searched. In the present embodiment, the gradient of the image brightness is estimated by creating a gradient histogram of the luminance in the luminance image data generated by the luminance image generating unit 42 by the gradient histogram generating unit 44.

Here, an example of a gradient histogram creation method will be described.
The gradient histogram is a diagram showing a distribution for each gradient direction θ with respect to the gradient intensity M that can be acquired in pixel units from the data of the luminance image.
First, the gradient histogram creation unit 44 calculates the gradient direction θ in each pixel from the luminance image data.
The gradient direction θ in each pixel is calculated by the following equation (1), for example.
Here, (x, y) on the left side of Equation (1) is a coordinate indicating the position of a pixel to be noted as a processing target (hereinafter referred to as “target pixel”) in the luminance image. In the luminance image, the coordinates are represented by an orthogonal coordinate system constructed with the horizontal right direction as the x axis and the vertical downward direction as the y axis. That is, θ (x, y) on the left side indicates the gradient direction θ of the pixel located at the coordinate (x, y) in the luminance image, that is, the target pixel.
Mod (α, β) on the right side of Equation (1) is a MODULO function that outputs a remainder when the value α is divided by the value β.
H (x, y) indicates the degree of contrast in the horizontal direction (hereinafter referred to as “horizontal contrast length”) at the pixel located at the coordinates (x, y) in the luminance image, that is, the target pixel. And is defined by the following equation (2).
V (x, y) is the degree of contrast in the vertical direction (hereinafter referred to as “vertical contrast length”) at the pixel located at the coordinate (x, y) in the luminance image, that is, the target pixel. And is defined by the following equation (3).
In the equations (2) and (3), P (xα, xb) represents the luminance value (pixel value) with respect to the coordinates (xα, xb) in the luminance image.

Here, with reference to FIG. 5, the horizontal contrast length of Expression (2) and the vertical contrast length of Expression (3) will be described in more detail.
FIG. 5 is a schematic diagram showing a part of the luminance image of FIG. 4 enlarged to the pixel level.
In FIG. 5, one black circle indicates one pixel. The figure shows a total of 25 (= 5 × 5) pixels, with 5 pixels in the x direction (horizontal direction) and 5 pixels in the y direction (vertical direction). Yes.
Here, if a pixel adjacent to the target pixel is hereinafter referred to as an “adjacent pixel”, the horizontal contrast length H (x, y) of the target pixel in Expression (2) is the luminance value P of the right adjacent pixel. It is obtained by the difference between (x + 1, y) and the luminance value P (x-1, y) of the left adjacent pixel. In addition, the vertical contrast length V (x, y) of the target pixel in Expression (3) is the luminance value P (x, y + 1) of the lower adjacent pixel and the luminance value P (x, y−1) of the upper adjacent pixel. ) And the difference.
Specifically, for example, the luminance value P (x + 1, y) of the right adjacent pixel is “1” indicating white, and the luminance value P (x−1, y) of the left adjacent pixel is black. It is assumed that it is “0” indicating In this case, the horizontal contrast length H (x, y) of the target pixel in Expression (2) is “1” (= 1-0).
Further, for example, the luminance value P (x, y + 1) of the lower adjacent pixel is “1” indicating white, and the luminance value P (x, y−1) of the upper adjacent pixel is “0” indicating black. Suppose that In this case, the vertical contrast length V (x, y) of the target pixel in Expression (3) is “1” (= 1-0).
Therefore, in this example, tan ⁻¹ {V (x, y) / H (x, y)} in equation (1) is 45 degrees, so the output of the MODULO function on the right side of equation (1) is , 45 which is the remainder of (45/180). That is, the gradient direction θ of the target pixel is 45 degrees.
Note that the MODULO function is used to divide the angle specified in all directions (0 degrees to 360 degrees) by 180 degrees, and the remainder is output as the gradient direction θ of the target pixel. The gradient direction θ of the pixel is specified within a range of 0 degrees to 180 degrees.

Next, the gradient intensity M in each pixel is calculated from the luminance image data.
The gradient intensity M is specified by calculating a luminance difference between pixels in all pixels.
The gradient intensity M in each pixel is calculated by the following equation (4), for example.
As shown in Expression (4), the gradient intensity M of the pixel of interest takes the square root of the sum of the square of the horizontal contrast length H (x, y) and the square of the vertical contrast length V (x, y). Is calculated by
Specifically, for example, in the above-described example described with reference to FIG. 5, the horizontal contrast length H (x, y) of the target pixel in Expression (2) is “1”, and the vertical direction of the target pixel in Expression (3) is Since the contrast length V (x, y) is “1”, the gradient intensity M of the target pixel is “√2”.

  In the present embodiment, the gradient strength M takes into account the actual distance from the imaging apparatus 1 regarding the subject indicated by the target pixel. Specifically, for example, when the target pixel is a pixel indicating a part of a subject at a distance close to the imaging device 1, the degree of weighting with respect to the gradient intensity M is increased. On the other hand, when the target pixel is a pixel indicating a part of a subject at a distance far from the imaging device 1, the degree of weighting with respect to the gradient intensity M is low. In this way, by changing the weighting of the gradient intensity M according to the distance from the imaging device 1, the gradient intensity M of the pixel indicating the shielding object at a short distance is increased, while the gradient intensity of the pixel indicating the distant shielding object is increased. M becomes smaller. For this reason, for example, it becomes possible to positively remove a subject such as a wire mesh in front.

  In this way, when the gradient histogram creating unit 44 calculates the gradient direction θ and the gradient intensity M for all the pixels constituting the luminance image, based on the calculation results, for example, a gradient histogram as shown in FIG. Create If the gradient intensity M of the pixel is a gradient angle of the pixel, it can be said that the gradient histogram represents the distribution of gradient angles included in the image.

FIG. 6 shows a gradient histogram created from the luminance image of FIG.
In FIG. 6, the vertical axis represents the gradient strength M, and the horizontal axis represents the gradient direction θ.
Here, the gradient intensity M on the vertical axis does not mean the gradient intensity M (x, y) itself, which is the calculated value of each pixel in Equation (4), and the same gradient direction θ for each gradient direction θ. The sum of the gradient intensities M (x, y) (calculation results of Expression (4)) of the pixel group is shown.
However, in this embodiment, among the gradient intensities M (x, y) of each pixel in Expression (4), those that are equal to or smaller than the predetermined threshold ThM are assumed to be affected by noise. Do not reflect on the histogram. That is, the gradient histogram creation unit 44 creates a gradient histogram after removing noise.
Specifically, for example, in this embodiment, a gradient histogram is created according to the following equation (5).
In Expression (5), “hist (θ) + = M (x, y)” means gradient strength M (vertical axis height) for a predetermined gradient direction θ (predetermined value on the horizontal axis) in the gradient histogram. ) Is added to the gradient intensity M (x, y), which is the calculation result of the expression (4) of the target pixel. That is, of the pixel group having a predetermined gradient direction θ (predetermined value on the horizontal axis), only those whose gradient intensity M (x, y) exceeds the threshold ThM are to be calculated, The sum of the gradient intensities M (x, y) of the pixels is plotted in the gradient histogram as the gradient intensity M (the height of the vertical axis) for the predetermined gradient direction θ. In other words, a pixel group having a predetermined gradient direction θ (predetermined value on the horizontal axis) whose gradient intensity M (x, y) is equal to or less than the threshold value ThM is not calculated and is removed as noise. .

  Heretofore, an example of a gradient histogram creation method has been described. The gradient histogram created by the gradient histogram creation unit 44 of FIG. 2 according to such a creation method is supplied to the gradient peak selection unit 45.

  The gradient peak selection unit 45 selects a gradient peak in the gradient histogram created by the gradient histogram creation unit 44.

The gradient peak is a portion corresponding to the maximum point of the gradient intensity M in the predetermined gradient direction θ in the gradient histogram. That is, the portion where the gradient strength M is the highest in the gradient direction θ (the portion where the differential value changes from positive to negative when the gradient strength M is differentiated in the gradient direction θ direction) It is a peak.
The gradient direction θ that is the peak of the gradient is the actual angle of the edge of the focused object among the objects included in the through image (original image) (the real object corresponding to the object in the real world) The angle of the contour) is high. Since the contrast of the object becomes higher as the object is focused (focused), the gradient intensity M (x, y) (calculation result of Expression (4)) of each pixel constituting the object is higher. Because it becomes. That is, as a result, the gradient strength M (the height of the vertical axis of the gradient histogram), which is the sum of them, increases.
However, when the through image is focused on a subject such as an animal, there are few regions (pixel groups) having the same angle at the edge of the subject. That is, the number of pixels having the same gradient direction θ is reduced for the edge of a subject such as an animal. For this reason, even if the gradient strength M (x, y) (calculation result of Expression (4)) of each pixel constituting the edge of a subject such as an animal has a certain high value, the total number of pixels is small. As described above, the gradient intensity M (the height of the vertical axis of the gradient histogram) that is the sum of them is not so high.
On the other hand, when the through image is focused on the front fence, the fence is almost entirely composed of a collection of edges (lines) of substantially the same angle, usually a collection of edges of two angles. Has been. That is, the number of pixels having one of the two types of angles as the gradient direction θ is very large. For this reason, when focusing on the fence, the gradient strength M (x, y) of each pixel constituting the fence (the calculation result of Expression (4)) has a certain high value, and the total number of pixels Therefore, in the gradient direction θ corresponding to each of the two types of angles, the gradient intensity M (the height of the vertical axis of the gradient histogram) that is the sum of them is very high.
Here, among the plurality of gradient peaks in the gradient histogram, the gradient peak having the highest gradient intensity M (vertical height) is hereinafter referred to as a “first gradient peak” and is the second highest gradient intensity. Hereinafter, the gradient peak having M (the height of the vertical axis) is referred to as a “second gradient peak”. In this case, the gradient direction θ at the first gradient peak and the gradient direction θ at the second gradient peak can be determined to be the gradient direction θ of each pixel constituting the fence based on the above-described contents. is there.
Therefore, the imaging apparatus 1 can identify the focused object such as the fence F in focus in the through image by determining the first and second gradient peaks.

Specifically, for example, the gradient histogram of FIG. 6 is created based on the luminance image of FIG. 4 created from the through image of FIG. As shown in FIG. 4, this fence image is composed of an aggregate of an edge of approximately 40 degrees and an edge of approximately 130 degrees. For this reason, in the gradient histogram of FIG. 6, the gradient peaks appear when the gradient direction θ is approximately 40 degrees and approximately 130 degrees, respectively.
That is, the gradient peak when the gradient direction θ is approximately 40 degrees is the first gradient peak. In addition, the gradient peak when the gradient direction θ is approximately 130 degrees is the second gradient peak.
In this case, the gradient peak selection unit 45 selects the second first gradient peak and the second gradient peak from the gradient histogram.
In this way, the gradient peak selection unit 45 can identify the fence F, which is a continuous shield arranged in the through image, by selecting the first and second gradient peaks. it can.

The excluded pixel selection unit 46 selects pixels corresponding to the gradient peak selected by the gradient peak selection unit 45 (in this embodiment, the first gradient peak and the second gradient peak).
The pixel corresponding to the gradient peak is a pixel having the gradient direction θ (approximately 40 degrees and approximately 130 degrees in the above example) that is the gradient peak, and is a continuous-like arrangement of pixels in the through image. It is a pixel that is supposed to show a part of a shield (a fence F in the above example). Although details will be described later, in this embodiment, such pixels are excluded and AF evaluation is performed. Therefore, hereinafter, the pixel corresponding to the gradient peak is referred to as “excluded pixel”.
That is, the excluded pixel selecting unit 46 selects a pixel corresponding to the gradient peak as an excluded pixel from the pixels constituting the luminance image, and notifies the AF evaluation image generating unit 47 of the selected pixel.

The AF evaluation image generation unit 47 extracts the frequency component (excluding the DC component) from the luminance image data generated by the luminance image generation unit 42 in units of pixels, and uses the extracted values as the pixel values. Generate data.
Here, as described above, each pixel value constituting the image data generated by the AF evaluation image generation unit 47 is a frequency component (excluding a DC component) extracted from the luminance image data, which will be described later. It is used as an AF detection value in the calculation of the AF evaluation value. Therefore, hereinafter, the image data generated by the AF evaluation image generation unit 47 in this way is hereinafter referred to as “AF evaluation image data”.
Here, the AF evaluation image generation unit 47 is notified of the exclusion pixel selected by the exclusion pixel selection unit 46, that is, the exclusion pixel assumed to indicate a part of a shield such as the fence F in FIG. 4. ing. Therefore, the AF evaluation image generation unit 47 generates AF evaluation image data with the excluded pixels masked. Masking an excluded pixel means that the pixel value of the excluded pixel in the AF evaluation image cannot be used as an AF detection value, that is, the pixel value is set to 0 or a low value.

The AF evaluation unit 48 integrates the data (pixel value as the AF detection value) of each pixel in the AF evaluation area among the data of the AF evaluation image generated by the AF evaluation image generation unit 47, An AF evaluation value is obtained.
In this case, if there is an excluded pixel in each pixel in the AF evaluation area, the data of the excluded pixel (pixel value as the AF detection value) is 0 or a low value, so the AF evaluation value as the integration result is Is not reflected. In other words, processing that is equivalent to obtaining the AF evaluation value by masking the excluded pixels is performed.

  Therefore, the CPU 11 or the like of the imaging apparatus 1 can execute the AF position determination process with the excluded pixels masked by using the AF evaluation value generated by the AF evaluation unit 48 described above. That is, an excluded pixel is a pixel that shows a part of a shield that is regularly covered in a predetermined direction, such as the fence F in FIG. Focusing only on the subject to be focused, the in-focus position can be automatically determined. For example, when it is desired to photograph a landscape behind the fence F through the fence F as shown in FIG. 3, the in-focus position can be automatically adjusted to a tree (subject PO) behind the fence F.

  The functional configuration of the imaging device 1 according to the present embodiment has been described above with reference to FIGS.

Next, the flow of the AF evaluation process among the processes related to the AF function executed by the imaging apparatus 1 having such a functional configuration will be described.
FIG. 7 is a diagram for explaining the flow of AF evaluation processing executed by the imaging apparatus 1 of FIG. 1 having the functional configuration of FIG.

In the present embodiment, the AF evaluation process is performed, for example, when the user performs a half-press operation on the relays button of the operation unit 18 so that the through image acquisition unit 41 passes over the fence F as illustrated in FIG. The process is started when a through image on which a landscape is projected is acquired. As a result, the lens driving unit 33 moves the focus lens to a predetermined position (initial position).
Note that after the AF evaluation process is completed and the in-focus position is determined, the user performs a full-press operation on the relays button of the operation unit 18 so that imaging at the determined in-focus position is performed. Is called.

  In step S1, the luminance image generation unit 42 generates a luminance image. More specifically, the luminance image generation unit 42 extracts luminance components from the through image data acquired by the through image acquisition unit 41 in units of pixels, and luminance image data each having each extracted luminance component as a pixel value. Is generated. In the present embodiment, the luminance image generation unit 42 generates luminance image data as shown in FIG. 4 from the through image data as shown in FIG. 3.

  In step S2, the noise removing unit 43 removes noise. More specifically, the luminance image data generated by the luminance image generation unit 42 is subjected to Gaussian filter processing for removing noise.

  In step S3, the gradient histogram creating unit 44 creates a gradient histogram. Specifically, a gradient histogram is created from the luminance image data generated by the luminance image generation unit 42. In the present embodiment, the gradient histogram creation unit 44 calculates the above-described equation (1) from each pixel of the luminance image from which noise has been removed by the noise removal unit 43 to calculate the gradient direction θ, and the above-described equation (4) ) To calculate the gradient strength M. Further, the gradient histogram creation unit 44 performs noise removal according to the above-described equation (5). Finally, the gradient histogram creating unit 44 creates a gradient histogram as shown in FIG. 6 based on the calculated gradient direction θ and gradient intensity M of each pixel.

  In step S4, the gradient peak selection unit 45 selects a gradient peak. Specifically, the gradient peak selection unit 45 selects a gradient peak in the gradient histogram created by the gradient histogram creation unit 44. In addition, the gradient peak selection unit 45 selects the first gradient peak and the second gradient peak in the gradient histogram created by the gradient histogram creation unit 44. In the present embodiment, as shown in FIG. 6, the gradient direction θ is selected to be approximately 40 degrees as the first gradient peak, and the gradient direction θ is selected as approximately 130 degrees as the second gradient peak.

  In step S5, the excluded pixel selecting unit 46 selects an excluded pixel. Specifically, the excluded pixel selection unit 46 selects pixels corresponding to the gradient peaks selected by the gradient peak selection unit 45 (first gradient peak and second gradient peak in the present embodiment).

  In step S6, the AF evaluation image generation unit 47 generates an AF evaluation image. Specifically, the AF evaluation image generation unit 47 extracts a frequency component (excluding a DC component) from the luminance image data generated by the luminance image generation unit 42 in units of pixels, and extracts the extracted value for each pixel value. The image data for AF evaluation is generated.

  In step S7, the AF evaluation unit 48 performs AF evaluation. Specifically, by integrating the data of each pixel in the AF evaluation area (pixel value as the AF detection value) out of the AF evaluation image data generated by the AF evaluation image generation unit 47, the AF evaluation is performed. Find the value.

  In the imaging apparatus 1, when the processing of steps S1 to S7 is executed and the AF evaluation process is completed, the CPU 11 of the imaging apparatus 1 performs the AF evaluation process again at other in-focus positions. Thereafter, the CPU 11 or the like of the imaging apparatus 1 uses the AF evaluation value generated by the AF evaluation unit 48 described above to execute the AF position determination process that masks the excluded pixels, and automatically adjusts the in-focus position. . Thereafter, when the user fully presses the relays button of the operation unit 18, imaging at the determined in-focus position is performed.

  As described above, the imaging apparatus 1 according to the present embodiment includes the through image acquisition unit 41 that acquires a through image based on the captured image data output from the imaging unit 17, and the gradient based on the gradient of the luminance of the image. An excluded pixel selection unit 46 that specifies an edge having a substantially constant direction, and an AF evaluation unit 48 that calculates an AF evaluation value from an image region that excludes an area that includes the edge specified by the excluded pixel selection unit 46.

With such a configuration, the imaging apparatus 1 calculates an AF evaluation value based on an image obtained by identifying and removing the shielding object from the acquired through image even if there is a shielding object such as a fence before and after the subject. can do.
Therefore, the imaging apparatus 1 can focus on the subject of interest based on the calculated AF evaluation value even if there is a shield such as a fence before and after the subject.

In addition, the imaging apparatus 1 further includes a first gradient peak (gradient angle at which the gradient strength becomes the maximum value) and a second gradient angle at which the gradient strength is the second highest value, unlike the gradient angle. Is provided with a gradient peak selection unit 45 that identifies the edges of
As a result, the imaging apparatus 1 can be selected as a shielding object even if a region having a certain gradient angle, for example, a continuous fence is present in the through image. As a result, the imaging apparatus 1 can focus on the subject of interest even if there is a shield such as a fence in front of it.

Further, the gradient peak selection unit 45 further selects a region having a second gradient peak (gradient angle other than the first gradient peak where the gradient intensity is the second largest value), and has the first gradient peak. The region and the region having the second gradient peak are selected as shielding objects.
Thereby, the imaging device 1 can be selected as a shielding object even if there are two regions having a predetermined gradient angle in the through image, for example, a lattice-like fence. As a result, the imaging apparatus 1 can focus on the subject of interest even if there is a shield such as a fence in front of it.

In addition, the gradient peak selection unit 45 selects a shield in consideration of only the gradient histogram having a gradient strength equal to or greater than a predetermined value among the gradient histograms calculated by the gradient histogram creation unit 44.
Therefore, the imaging apparatus 1 can improve the selection accuracy of the shielding object to be excluded when calculating the AF evaluation value.

The imaging apparatus 1 further includes an AF evaluation image generation unit 47 that generates an AF evaluation image that masks the area of the shielding object specified by the excluded pixel selection unit 46, and the AF evaluation unit 48 includes an AF evaluation image. An AF evaluation value is calculated from the AF evaluation image generated by the generation unit 47.
As a result, the imaging apparatus 1 uses the through image to focus on the subject of interest without acquiring data or the like related to special focusing even if there is a shield such as a fence in front. Can do.

In addition, this invention is not limited to the above-mentioned embodiment and numerical formula, The deformation | transformation, improvement, etc. in the range which can achieve the objective of this invention are included in this invention.
That is, in the above-described embodiment, a gradient peak is calculated by calculating a gradient histogram, and an edge having a fixed gradient direction is specified. However, the present invention is not limited to this, and other methods such as a Prewitt operator and Roberts cross are used. It may be used to estimate the gradient of the luminance of the image. That is, any method that can identify an edge having a constant gradient direction may be used.

Further, in the above-described embodiment, as shown in FIG. 3, the shielding direction having a constant gradient direction θ (in this embodiment, an edge of about 40 degrees and an edge of about 130 degrees) is specified. However, the present invention is not limited to this.
For example, a case is assumed in which a gradient histogram is created for an image of a shield whose gradient direction changes depending on the position in the image, as shown in FIG. There are two gradient peaks formed. However, in the image shown in FIG. 8, since the direction of the fence changes gently, the gradient peak of the gradient histogram obtained is more than the gradient peak of the gradient histogram obtained from the image shown in FIG. 3 (see FIG. 6). The width becomes wider.
Therefore, as shown in FIG. 8, a shield whose gradient direction changes gently according to the position in the image, or a shield whose gradient direction changes gently in the image due to the state of reflection in the image such as perspective. When selecting an object, for example, the range for specifying the pixels having the gradient direction θ selected by the gradient peak selection unit 45 may be expanded by θ ± γ. That is, the peak width of the gradient histogram selected by the gradient peak selection unit 45 may be widened by a change in the assumed gradient direction, for example, γ.

  Further, in the above-described embodiment, it has been described that noise removal is performed by performing a smoothing process on a luminance image, and noise removal is performed by removing a predetermined gradient strength even when creating a gradient histogram. However, the removal of these noises only increases the accuracy for specifying the shielding object, and is not an essential configuration, but may be appropriately adopted depending on various purposes of use, etc. in order to achieve the object of the present invention. it can.

  Further, in the above-described embodiment, the description has been made assuming that the identification of the shielding object in the image is determined by adding the perspective weight from the imaging device to the gradient direction θ, but the present invention is not limited to this. The weighting of perspective from the imaging device is assumed to be taken through the fence, and the obstacle in front has a high tendency for the edge (contrast) of the outline to be strong, and it is easily reflected in the AF evaluation value. This is based on the idea of positively removing the shields. For this reason, it is not necessarily an indispensable structure, and can be adopted as appropriate depending on various purposes.

  In the above-described embodiment, the electronic apparatus to which the present invention is applied has been described by taking the imaging apparatus 1 such as a digital camera as an example, but is not particularly limited thereto. The present invention can be applied to general electronic devices capable of executing the above-described AF evaluation processing. Specifically, for example, the present invention is applicable to a notebook personal computer, a video camera, a portable navigation device, a mobile phone device, a portable game machine, a WEB camera, and the like.

  The series of processes described above can be executed by hardware or can be executed by software. For example, the series of processes described above can be executed by hardware or can be executed by software. In other words, the functional configuration of FIG. 2 is merely an example and is not particularly limited. That is, it is sufficient that the imaging apparatus 1 has a function capable of executing the above-described series of processing as a whole, and what functional blocks are used to realize this function is not particularly limited to the example of FIG. In addition, one functional block may be constituted by hardware alone, software alone, or a combination thereof.

  When a series of processing is executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium. The computer may be a computer incorporated in dedicated hardware. The computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

  The recording medium including such a program is not only configured by the removable medium 23 of FIG. 1 distributed separately from the apparatus main body in order to provide the program to the user, but also in a state of being incorporated in the apparatus main body in advance. The recording medium etc. provided in The external memory 26 is configured by, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like. The optical disk is composed of, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disk), or the like. The magneto-optical disk is configured by an MD (Mini-Disk) or the like. In addition, the recording medium provided to the user in a state of being preliminarily incorporated in the apparatus main body includes, for example, the ROM 12 in FIG. 1 in which a program is recorded, the hard disk included in the storage unit 20 in FIG.

In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series along the order, but is not necessarily performed in time series, either in parallel or individually. The process to be executed is also included.
Further, in the present specification, the term “system” means an overall device configured by a plurality of devices, a plurality of means, and the like.

  As mentioned above, although several embodiment of this invention was described, these embodiment is only an illustration and does not limit the technical scope of this invention. The present invention can take other various embodiments, and various modifications such as omission and replacement can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope and gist of the invention described in this specification and the like, and are included in the invention described in the claims and the equivalents thereof.

The invention described in the scope of claims at the beginning of the filing of the present application will be appended.
[Appendix 1]
An acquisition means for acquiring an image;
A specifying means for specifying an edge having a substantially constant gradient direction based on a luminance gradient of the image;
AF evaluation value calculating means for calculating an AF evaluation value from an image area excluding an area including an edge specified by the specifying means;
An image processing apparatus comprising:
[Appendix 2]
The specifying means includes a first edge having a gradient angle at which the gradient strength becomes a maximum value, and a second edge having a gradient angle at which the gradient strength has a second highest value, unlike the gradient angle. Identify,
The image processing apparatus according to appendix 1, wherein the AF evaluation value calculation means calculates an AF evaluation value from an image area excluding an area including the first edge and the second edge.
[Appendix 3]
A histogram calculating means for calculating a gradient histogram representing a gradient direction distribution of the gradient intensity based on the luminance gradient of the image;
The image processing apparatus according to appendix 1 or 2, wherein the specifying unit specifies an edge having a substantially constant gradient direction based on the gradient histogram calculated by the histogram calculation unit.
[Appendix 4]
The specifying unit specifies an edge having a substantially constant gradient direction in consideration of only a gradient histogram having a gradient strength equal to or greater than a predetermined value among gradient histograms calculated by the histogram calculation unit. An image processing apparatus according to 1.
[Appendix 5]
The AF evaluation value calculating means includes a generating means for generating a mask image in which an area including an edge specified by the specifying means is masked from an image acquired by the acquiring means,
The image processing apparatus according to any one of appendices 1 to 4, wherein the AF evaluation value calculation unit calculates an AF evaluation value from the mask image generated by the generation unit.
[Appendix 6]
An input step for inputting an image;
A specifying step of specifying an edge having a substantially constant gradient direction based on the gradient of brightness of the input image;
An AF evaluation value calculating step for calculating an AF evaluation value from the image area excluding the area including the specified edge;
An image processing method comprising:
[Appendix 7]
To the computer that controls the process,
An input function for inputting images,
A specific function for identifying an edge having a substantially constant gradient direction based on a gradient of brightness of the image;
An AF evaluation value calculation function for calculating an AF evaluation value from an image area excluding an area including an edge specified by performing the specific function;
A program to realize

  DESCRIPTION OF SYMBOLS 1 ... Imaging device, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... Image processing part, 15 ... Bus, 16 ... Input / output interface, 17 ... Image pick-up unit, 18 ... operation unit, 19 ... display unit, 20 ... storage unit, 21 ... communication unit, 22 ... drive, 23 ... removable media, 41 ... through Image acquisition unit 42 ... Luminance image generation unit 42 ... Noise removal unit 44 ... Gradient histogram creation unit 45 ... Gradient peak selection unit 46 ... Excluded pixel selection unit 47 ... AF evaluation image generation unit, 48... AF evaluation unit

Claims (7)

An acquisition means for acquiring an image;
A specifying means for specifying all edges having a substantially constant gradient direction included in the image based on the luminance gradient of the image;
AF evaluation value calculating means for calculating an AF evaluation value from an image area excluding an area including all edges whose gradient direction specified by the specifying means is substantially constant ;
An image processing apparatus comprising: The specifying means includes all first edges having a gradient angle at which the gradient strength is a maximum value, and all second edges having a gradient angle at which the gradient strength is the second highest value, unlike the gradient angle. Identify the edges,
2. The image according to claim 1, wherein the AF evaluation value calculation unit calculates an AF evaluation value from an image area excluding an area including all of the first edges and all of the second edges. Processing equipment. A histogram calculating means for calculating a gradient histogram representing a gradient direction distribution of the gradient intensity based on the luminance gradient of the image;
The image processing apparatus according to claim 1, wherein the specifying unit specifies all edges having a substantially constant gradient direction based on the gradient histogram calculated by the histogram calculation unit.   The specifying unit specifies all edges having a substantially constant gradient direction in consideration of only a gradient histogram having a gradient strength equal to or greater than a predetermined value among the gradient histograms calculated by the histogram calculation unit. The image processing apparatus according to claim 3. The AF evaluation value calculating means includes a generating means for generating a mask image that masks an area including all edges whose gradient directions specified by the specifying means are substantially constant from the image acquired by the acquiring means,
The image processing apparatus according to claim 1, wherein the AF evaluation value calculation unit calculates an AF evaluation value from the mask image generated by the generation unit. An input step for inputting an image;
Based on the brightness gradient of the input image, a specifying step for specifying all edges having a substantially constant gradient direction included in the image;
An AF evaluation value calculating step for calculating an AF evaluation value from an image area excluding an area including all the edges in which the specified gradient direction is substantially constant ;
An image processing method comprising: To the computer that controls the process,
An input function for inputting images,
Based on the brightness gradient of the image, a specifying function for specifying all edges having a substantially constant gradient direction included in the image;
An AF evaluation value calculation function for calculating an AF evaluation value from an image area excluding an area including all edges whose gradient direction specified by the specific function is substantially constant ;
A program to realize