JP6514504B2 - IMAGE PROCESSING APPARATUS, CONTROL METHOD THEREOF, AND PROGRAM - Google Patents

Description

The present invention relates to an image processing apparatus and a control method thereof, and relates to a program.

  Processing for performing tone correction and tone compression at the stage of outputting a signal with an expanded dynamic range (D range) of input, such as High Dynamic Range (HDR) shooting and dodging to partially brighten an image, is known There is.

  In tone correction and tone compression techniques, in addition to generating an image having high contrast and wide dynamic range characteristics, noise reduction processing (also referred to as NR processing or noise removal processing) according to the amount of tone correction or A technique for performing edge enhancement processing to generate an image with a good SN ratio is desired. Furthermore, when photographing a plurality of subjects, a technique for performing noise removal and edge enhancement processing for each subject area to generate an impressive image is also desired.

  For example, in the area A in which the gradation correction is performed using a large amount of gradation correction and in the area B in which the gradation correction is performed using a small amount of gradation correction equal to almost nothing Suppose that the signal values are equal. The noise in the area A is amplified by the large amount of gradation correction, so the amount of noise is larger than that in the area B. On the other hand, if processing is performed so that the effect of noise removal becomes stronger in accordance with the region A (that is, the effect of edge enhancement processing becomes weaker), the region B hardly subjected to gradation correction loses a sense of resolution more than necessary. It will

  With regard to such a problem, Patent Document 1 outputs an image obtained by combining an image subjected to noise removal processing and an image prior to noise removal using a combining ratio calculated based on the tone correction amount. Proposed technology. Further, Patent Document 2 proposes a technique for changing the noise removal strength according to the tone correction amount.

JP 2011-15277 A Patent No. 04069943

  However, while the technique proposed in Patent Document 1 calculates the synthesis ratio of the image according to the gradation correction amount, changing the noise removal intensity according to the gradation correction amount is not considered. That is, since only the synthesis ratio of the image before noise removal and the image after noise removal is adjusted according to the tone correction amount, it is optimal for an image in which the tone correction amount changes extremely for each area. Results may not be obtained.

  Further, the technique proposed in Patent Document 2 is a technique for changing the noise removal intensity according to the tone correction amount according to the local light / dark difference in the image, so noise removal can be performed according to the subject region. It does not consider about the noise removal process which changed intensity appropriately.

The present invention has been made in view of the problems of prior art described above, the gradation correction amount and intensity controllable image processing apparatus and a control method thereof to noise removal taking into account the subject area, and program Intended to provide.

In order to solve this problem, for example, an image processing apparatus according to the present invention has the following configuration. That is, a determination unit that determines a plurality of subject areas from an image, a determination unit that determines a gradation correction amount for performing gradation correction for each predetermined area of the image, and an image that is gradation-corrected by the gradation correction amount Noise removal means for changing the noise removal strength applied to the image according to the gradation correction amount and the subject area determined, and the edge enhancement strength applied to the image from which noise is removed by the noise removal means. And an edge emphasizing unit that changes according to the subject area determined as the tonality correction amount .

  According to the present invention, it is possible to control the noise removal strength in consideration of the gradation correction amount and the subject area.

A block diagram showing an example of a functional configuration of a digital camera as an example of an image processing apparatus according to an embodiment of the present invention Block diagram showing an example of the functional configuration of the image processing unit 104 according to the first embodiment (A) block diagram showing a functional configuration example of the NR processing unit 205 according to the first embodiment, (b) block diagram showing a functional configuration example of the threshold adjustment unit 302 (A) block diagram showing an example of functional configuration of the edge emphasizing processing unit 206 according to the first embodiment; (b) block diagram showing an example of functional configuration of the edge correcting unit 403 Flowchart showing a series of operations of gradation correction processing according to the first embodiment A flowchart showing a series of operations of the NR process according to the first embodiment The flowchart which shows a series of operation | movement of (a) threshold value calculation processing which concerns on Embodiment 1, (b) The flowchart which shows a series of operation | movement of threshold value adjustment processing A flowchart showing a series of operations of edge enhancement processing according to the first embodiment Flowchart showing a series of operations of edge correction processing according to the first embodiment (A) A diagram showing the correspondence between the threshold used in the NR process and the luminance signal, (b) A diagram showing the correspondence between the weighted addition coefficient used in the NR process and the luminance signal A diagram showing an example of the area determined in the subject determination process Diagram for explaining the gradation characteristic according to the subject area Diagram for explaining (a) gain table and (b) gain MAP used in tone correction processing (A) A diagram showing the correspondence between the coring amount and the luminance signal used in edge emphasis processing, (b) A diagram showing the correspondence between edge gain and luminance signal used in edge emphasis processing (A) block diagram showing a functional configuration example of the image processing unit 104 according to the second embodiment, (b) block diagram showing a functional configuration example of the NR processing unit 1501 (A) block diagram showing an example of functional configuration of the edge emphasizing processing unit 1502 according to the second embodiment; (b) block diagram showing an example of functional configuration of the edge correcting unit 1601 A diagram for explaining a method of calculating a gradation correction amount according to the second embodiment (A) A diagram for explaining input / output characteristics to be applied according to the third embodiment, (b) A diagram for explaining the relation between a subject area and the input / output characteristics to be applied Block diagram showing an example of the functional configuration of the image processing unit 104 according to the third embodiment (A) block diagram showing a functional configuration example of the NR processing unit 2101 according to the third embodiment, (b) block diagram showing a functional configuration example of the threshold adjustment unit 2203 (A) block diagram showing an example of functional configuration of the edge emphasizing processing unit 2102 according to the third embodiment; (b) block diagram showing an example of functional configuration of the edge correcting unit 2301 A diagram for explaining a method of calculating a gradation correction amount for each subject area according to the third embodiment

(Embodiment 1)
Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. In the following, as an example of the image processing apparatus, an example in which the present invention is applied to an arbitrary digital camera capable of performing noise removal processing on a captured image will be described. However, the present invention is not limited to a digital camera having a photographing function, and can be applied to any device to which a separately photographed image can be acquired and noise removal processing can be applied. These devices may include, for example, a mobile phone, a game console, a tablet terminal, a personal computer, and a watch-type or glasses-type information terminal.

  Note that tone correction processing according to the present invention, which will be described in detail later, refers to the entire processing including tone characteristic determination processing, gain processing, NR processing, and edge enhancement processing, which will be described later. And the processing other than the edge emphasis processing will be described. Further, in the present embodiment, assuming a person scene in a back light where gradation correction is most required, gradation correction processing in which a person is the darkest and the sky is the brightest is described as an example. However, the applicable scene of the present invention is not limited to this.

(Configuration of digital camera 100)
FIG. 1 is a block diagram showing an example of the functional configuration of a digital camera 100 as an example of an image processing apparatus according to this embodiment. Note that one or more of the functional blocks shown in FIG. 1 may be realized by hardware such as an ASIC or programmable logic array (PLA), or realized by a programmable processor such as a CPU or MPU executing software. May be Also, it may be realized by a combination of software and hardware. Therefore, in the following description, even when different functional blocks are described as an action subject, the same hardware can be realized as the subject.

  The optical system 101 is a lens group including a zoom lens and a focus lens, and includes an aperture adjustment device and a shutter device. The optical system 101 adjusts the magnification, the in-focus position, or the light amount of the subject image formed on the imaging device included in the imaging unit 102. The imaging unit 102 includes an imaging element such as a CCD or a CMOS sensor, and photoelectrically converts the light flux of the subject having passed through the optical system 101, and outputs the obtained electric signal. The A / D conversion unit 103 converts the input analog signal into a digital signal, and outputs an image of the digital signal.

  The image processing unit 104 performs various image processing such as white balance adjustment and γ correction on the image output from the A / D conversion unit 103 or the image read out from the recording unit 110. Further, the image processing unit 104 performs processing such as face detection, depth detection, shift amount calculation of an object image, and the like on the image to calculate a predetermined calculation value, and outputs the calculated value to the system control unit 107, for example. The system control unit 107 performs, for example, an AF (Auto Focus) process based on the calculation value obtained from the image processing unit 104. Further, the image processing unit 104 performs tone correction processing according to the present invention described later.

  The exposure amount calculation unit 105 is a portion that calculates an exposure amount at the time of shooting in order to obtain an optimal input image for performing the tone correction processing according to the present invention. The processing result of the image processing unit 104 is input, and the calculated exposure amount is output to the exposure amount control unit 106.

  The exposure amount control unit 106 controls the optical system 101 and the imaging unit 102 in order to realize the exposure amount calculated by the exposure amount calculation unit 105, and controls the aperture, the shutter speed, and the analog gain of the sensor.

  The system control unit 107 includes, for example, a CPU or an MPU, and controls the entire digital camera 100 by expanding a program stored in a ROM (not shown) into a work area of a RAM (not shown) and executing the program. Further, the system control unit 107 performs drive control of the optical system 101 and the imaging unit 102 based on a user instruction transmitted from the operation unit 108 in addition to the operation value input from the image processing unit 104.

  The display unit 109 includes a liquid crystal display and an organic EL (Electro Luminescence) display, and displays an image captured by the imaging unit 102 and an image read out from the recording unit 110. The recording unit 110 includes a recording medium including an information recording medium using a package containing a rotating recording medium such as a memory card or a magneto-optical disk mounted with a semiconductor memory, and performs image processing by writing and reading an image. Output to the unit 104 or the display unit 109. The recording unit 110 may be configured to be removable.

  The bus 111 is a transmission path of digital signals, and performs data transmission and reception among the image processing unit 104, the system control unit 107, the display unit 109, and the recording unit 110 by time division, for example.

(Configuration of the image processing unit 104)
Next, a functional configuration example of the image processing unit 104 will be described with reference to FIG. The flow of processing indicated by a solid line in FIG. 2 represents a flow (also referred to as processing of a main line system) in which direct processing such as changing of a pixel value is performed on the input image. On the other hand, an arrow shown by a dotted line represents a flow of processing for calculating an evaluation value based on an input image (also referred to as processing of an evaluation value system). The same notation applies to other block diagrams.

  The image output from the imaging unit 102 is input to the image processing unit 104 via the A / D conversion unit 103. The input image is input to a subject discrimination unit 201 inside the image processing unit 104, and the subject discrimination unit 201 discriminates a subject area from the image.

  The gradation characteristic determination unit 202 determines the gradation characteristic based on the subject area determined by the subject determination unit 201. The gradation characteristic to be determined is, for example, the characteristic (also referred to as a gain table) of the gain value with respect to the value of the input signal as shown in FIG.

  The gain MAP generation unit 203 generates a gain MAP using the gradation characteristic determined by the gradation characteristic determination unit 202. For example, as shown in FIG. 13B, the gain MAP is image information in which the pixel value at each coordinate position indicates the gain with the upper left coordinate set as (0, 0).

  The gain processing unit 204 performs gain processing according to the gain value obtained from the gain MAP, using the gain MAP generated by the gain MAP generation unit 203.

  The NR processing unit 205 performs noise removal processing (also referred to as NR processing) on the image subjected to gain processing by the gain processing unit 204. The NR processing unit 205 implements the NR processing according to the gradation correction amount for each pixel (region) by referring to the gain MAP in the execution of the NR processing. Also, the edge enhancement processing unit 206 performs edge enhancement processing on the image on which the NR processing has been performed. The edge emphasis processing unit 206 realizes edge emphasis processing according to the amount of gradation correction for each pixel (area) by referring to the gain MAP in the edge emphasis processing as in the processing by the NR processing unit 205.

(A series of operations related to gradation correction processing by the image processing unit 104)
Next, with reference to FIG. 5, a series of operations relating to the tone correction processing by the image processing unit 104 will be described. The series of processes related to the tone correction process execute processes corresponding to the respective functional blocks shown in FIG. Further, in the series of operations shown in FIG. 5, for example, the image pickup element is exposed by a photographing instruction from the user to the operation unit 108, and an image read from the image pickup unit 102 via the A / D conversion unit 103 is an image processing unit It starts from the time when it is input to 104.

  In step S501, the subject determining unit 201 determines a subject area in the input image. For example, with respect to an image captured with the configuration of the subject as shown in FIG. 11A, the subject discrimination unit makes the face region 1101 of a person, the body region 1102 of a person, a cloud and so on as shown in FIG. The sky area 1103 including the sun and the background area 1104 other than that are respectively determined. For the determination processing of the subject region, it is possible to use a known method using feature amounts of edge or color information disclosed in Japanese Patent Application Laid-Open No. 2006-039666, for example, and learning data by a neural network. When the discrimination process is completed, the subject discrimination unit 201 outputs the subject discrimination result to the gradation characteristic determination unit 202. The subject discrimination result includes, as described later, numerical values representing the degree of being a person area, a sky area, and a background area for each pixel position (x, y). These degrees may be set, for example, in accordance with the feature value or the evaluation value when learning data is used.

  In step S502, the gradation characteristic determination unit 202 inputs the input image and the subject region determined in step S501, and determines gradation characteristics. The gradation characteristics will be described in detail with reference to FIG. 12 separately, but the horizontal axis indicates the value of the input signal, and the vertical axis indicates the gain table representing the gain value.

  Note that, at the time of exposure control before main shooting, in order to obtain an optimal exposure amount for performing tone correction processing with the calculated tone characteristics, the tone characteristics calculated at the time of calculating the exposure amount are required. Therefore, while performing the exposure control, the gradation characteristic determination unit 202 outputs the calculated gradation characteristic to the exposure amount calculation unit 105. Further, at the time of actual shooting, the gradation characteristic determination unit 202 outputs the determined gradation characteristic to the gain MAP generation unit 203, and the process proceeds to S503.

  In step S503, the gain MAP generation unit 203 generates a gain MAP based on the image input to the image processing unit 104 and the gradation characteristic determined in step S502. In this embodiment, the method of generating the gain MAP is, for example, a known method using hierarchical images of different reduction sizes generated based on an input image as disclosed in, for example, Japanese Patent Laid-Open No. 2014-154108. MAP can be generated.

  In step S504, the gain processing unit 204 performs gain processing on the input image input to the image processing unit 104. The gain processing unit 204 receives the gain MAP generated in S503, and applies the gain value of the corresponding coordinate of the gain MAP to the pixel value of each coordinate of the input image. When the gain processing is completed, the gain processing unit 204 outputs the input image subjected to the gain processing to the NR processing unit 205.

  In step S505, the NR processing unit 205 performs NR processing on the image subjected to gain processing in step S504. The NR processing unit 205 also performs NR processing with reference to the gain MAP generated in S503. Details of the NR process will be described later.

  In step S506, the edge enhancement processing unit 206 performs edge enhancement processing on the image subjected to the NR processing in step S505. The edge enhancement processing unit 206 also performs edge enhancement processing with reference to the gain MAP generated in S503. Details of the edge emphasis processing will be described later. When the edge enhancement processing is completed, the edge enhancement processing unit 206 outputs the input image subjected to the edge enhancement processing, and the image processing unit 104 ends the series of operations of this processing.

(Gradation characteristic determination processing)
The tone characteristic determination process in step S502 will be described below with reference to FIG.

  In the present embodiment, gain processing is performed by determining one tone characteristic (gain table) common to the entire image without having tone characteristics for each subject region. The one gradation characteristic is first determined based on these gradation characteristics after the gradation characteristics are determined for each subject region. By using such one gradation characteristic, it is possible to determine the gradation characteristic in consideration of the balance of brightness among the object areas.

  FIG. 12A shows the gradation characteristics for each subject area, with input / output characteristics in which the horizontal axis represents the value of the input signal and the vertical axis represents the value of the output signal. The characteristic shown by the dotted line shows the input / output characteristic when the input signal is not changed (ie, gradation correction is not performed), and the characteristic shown by the solid line is the input / output characteristic (HUMAN_GAIN, for person area, background area and sky area) It shows BACK_GAIN and SKY_GAIN). Since the well-known method which is disclosed by Unexamined-Japanese-Patent No. 2014-155001 can be used for calculation of the input-output characteristic with respect to each to-be-photographed object area | region which concerns on this embodiment, for example Description of is omitted.

  FIG. 12 (b) shows two types of input / output characteristics (D range priority and priority) for determining the gradation characteristic common to the whole image based on the gradation correction given to each subject area shown in FIG. 12 (a). Represents the input / output characteristics of contrast priority. These characteristics are generated based on each input / output characteristic shown in FIG. 12A and a luminance range in which each subject region is distributed in large numbers. The input / output characteristics of the person area are applied to the low luminance side section where many human areas are distributed from 0 of the input signal, and the sky side is the empty section where many empty areas are distributed from the maximum value of the input signal The input and output characteristics of the region apply. The contrast priority input / output characteristic shown in FIG. 12 (b) is an input / output characteristic for emphasizing the contrast, and its feature is the background area in addition to the input / output characteristic of the person area shown in FIG. 12 (a). , The slope of the input and output characteristics of the empty region. That is, the person area can realize brightness and contrast, and the background area and the sky area can realize contrast.

  On the other hand, the input / output characteristic of D range priority shown in FIG. 12B is an input / output characteristic for securing a dynamic range in the entire image. The characteristic of this input / output characteristic is that the input / output characteristic of the person area shown in FIG. 12A and the input / output characteristic of the sky area are taken into consideration. The brightness and the contrast of the person area and the sky area Can be realized. The input / output characteristics of the person area are used in the low luminance side section where many human areas are distributed from 0 of the input signal, while the high luminance section where many empty areas are distributed from the maximum value is the empty area It is an output characteristic. A section of intermediate luminance, which is between these sections, is a characteristic that connects the input / output characteristic of the face area of the person on the low luminance side and the input / output characteristic of the sky area on the high luminance side.

  FIG. 12C is an input / output characteristic calculated by weighted addition of the two gradation characteristics shown in FIG. 12B, which is one gradation characteristic commonly applied to the entire image. For the calculation of the input / output characteristics shown in FIGS. 12 (b) and 12 (c), known gradation calculation processing as disclosed in, for example, JP-A-2014-153959 can be used. .

  FIG. 12 (d) shows a gain table generated based on the input / output characteristics of FIG. 12 (c), where the vertical axis shows the gain value (tone correction amount) and the horizontal axis shows the input signal. . The gradation characteristic determination unit 202 substitutes the input / output characteristic shown in FIG. 12C into the equation (1) to calculate the gain value Gain (X), and generates a gain table based on the gain value Gain (X). . When generation of the gain table is completed, the gradation characteristic determination unit 202 outputs the result to the gain MAP generation unit 203 and ends the gradation characteristic determination processing.

(Outline of NR processing)
Next, the NR process in S505 will be described in more detail.

  The NR process is composed of two processes: a thresholded average value process for an input image, and an image combining process for combining the image after the thresholded average value process and the input image. The thresholded average value process is an average value calculation process in which peripheral pixels having a strong correlation with the processing target pixel are used, and noise can be reduced while maintaining a sense of resolution. In the present embodiment, as the thresholded average value processing, for example, an MTM (Modified Trimmed Mean) filter represented by Expression (2) is applied to the input image. Note that TH is a threshold, N is a range of reference pixels, W is a filter coefficient, pix is a pixel value before filtering, and out_pix is a pixel to be processed after filtering. Further, the coordinates of the processing target pixel are represented by (i, j).

  On the other hand, when an image is generated by applying only the average value processing shown in Equation (2), since the effect of noise reduction differs depending on the region, unnatural unevenness may remain in the image. In order to prevent the occurrence of such unevenness in the image, an image combining process is performed to combine the image after the average value processing shown in Equation (2) with the image before the average value processing. The NR processing unit 205 performs weighted addition of pixel values at the same coordinates of each image according to Equation (3), and generates a natural NR-processed image. Note that prepix is a pixel value before average value processing, postpix is a pixel value after average value processing, and z is a weighted addition coefficient to the image after average value processing. Further, the coordinates of the processing target pixel are represented by (i, j).

  The threshold value TH of the thresholded average value process shown in the equation (2) and the value of the weighted addition coefficient z of the image combining process shown in the equation (3) are shown, for example, in FIGS. 10 (a) and 10 (b). You may make it have a characteristic. The horizontal axes of the respective drawings each represent a pixel value (here, a luminance signal) of the pixel position (i, j) to be processed. The vertical axis in FIG. 10A represents the value of the threshold TH, and the vertical axis in FIG. 10B represents the value of the weighted addition coefficient z of the image combining process. For example, the threshold TH indicates a characteristic represented by a logarithmic function curve with respect to a pixel value (here, luminance signal), and the weighted addition coefficient z indicates a characteristic represented by a curve with an inverse gamma characteristic with respect to the pixel value. There is.

(A series of operations related to NR processing)
As described above, when the gradation correction amount is large, the influence of amplification of noise by the gradation correction processing appears. Therefore, it is necessary to appropriately adjust the threshold value TH of the threshold value-based average value processing and the value of the weighted addition coefficient z of the image combining processing according to the gradation correction amount. Hereinafter, the process of calculating the threshold TH and the process of calculating the value of the weighted addition coefficient z will be described while describing a series of operations of the NR process with reference to FIGS. 6 and 7. Note that a series of operations of the NR process shown in FIG. 6 are realized by executing processes corresponding to the respective functional blocks shown in FIGS. 3A and 3B. Therefore, in the description of each step shown in FIG. Further, arrows shown by solid lines and dotted lines shown in FIG. 3A indicate flows of processing of the main line system and processing of the evaluation value system, as in FIG.

  In addition, the NR processing unit 205 sequentially changes the position (x, y) of the processing target pixel and performs the processing described below on all the pixels.

  In step S601, the NR processing unit 205 initializes the pixel position (x, y) to be processed to the start point (0, 0).

  In S602, a threshold TH (x, y) for performing thresholded average value processing is calculated based on Expression (2). The calculation process (also referred to as a threshold calculation process) of the threshold TH (x, y) according to this step will be further described with reference to FIG. 7 (a). The threshold calculation process according to this step is realized by executing processes respectively corresponding to the brightness-based threshold calculation unit 301 and the threshold adjustment unit 302 shown in FIG. 3B.

  In S701, the by-luminance threshold calculation unit 301 calculates by-luminance threshold TH_y (x, y) based on an input signal of the pixel position (x, y) to be processed. In the present embodiment, the luminance-based threshold calculation unit 301 calculates the luminance-based threshold TH_y (x, y) with reference to the characteristics shown in FIG. That is, the threshold calculation unit for each luminance 301 obtains corresponding TH_y for luminance signals Y at all pixel positions (x, y).

  In step S702, the threshold adjustment unit 302 receives the threshold TH_y (x, y) for luminance calculated in step S701, the gradation correction amount determined in step S502, and the subject discrimination result, and reflects the gradation correction amount and the subject area. The calculated threshold value TH (x, y) is calculated. This process is also referred to as threshold adjustment process.

  The threshold adjustment processing by the threshold adjustment unit 302 will be described in more detail with reference to FIG. 7 (b). As shown in FIG. 3B, the threshold adjustment unit 302 includes a reliability calculation unit 310, a gradation correction amount adjustment unit 311, and a subject-by-subject adjustment unit 312. The inside of the tone correction amount adjustment unit 311 and the subject-by-subject adjustment unit 312 is further illustrated.

  In S703, the gradation correction amount adjustment unit 311 acquires the gradation correction amount Gain (x, y) of the pixel position (x, y) to be processed. In the present embodiment, the gradation correction amount Gain (x, y) acquires the gradation correction amount with reference to the pixel position (x, y) of the corresponding gain MAP as shown in the equation (4). Here, GainMap (x, y) represents the value of the pixel position (x, y) for which the gain MAP is to be processed.

  In S704, the gradation correction amount adjustment unit 311 multiplies the input threshold value TH_y (x, y) by the gradation correction amount Gain (x, y) and the fixed parameter k shown in Expression (5). Thus, the threshold TH_g (x, y) is calculated. Here, the fixed parameter k is a parameter for adjusting the amount of reflecting the gradation correction amount Gain (x, y), and if an optimum value is used for the gradation correction amount previously obtained by experiment etc. Good.

  As described above, the threshold value is calculated such that the threshold value increases as the gradation correction amount increases. If such a threshold value is calculated, pixels are selected so as to include pixels having a large luminance difference in the averaging shown in equation (2) as the gradation correction amount is larger. That is, the width or the number of pixels of the pixel value to be averaged can be changed according to the magnitude of the gradation correction amount, and the noise removal intensity can be changed according to the magnitude of the gradation correction amount .

  In step S705, the reliability calculation unit 310 obtains the subject determination result dis (x, y) calculated in step S501, and calculates the reliability indicating the possibility of each subject region. In the present embodiment, the subject determination result dis (x, y) indicates the sky area degree, the background area degree, and the person area degree, and the degree is represented by a value from 0 to 1023. The reliability calculation unit 310 normalizes the subject discrimination result dis (x, y) to obtain the reliability of the human region satisfying equation (6) for each image position (x, y) α, and the reliability of the background region. Each degree of reliability is calculated, where the degree is β and the reliability of the empty area is γ.

  Although the reliability of the three subject areas is calculated in the present embodiment, the human area may be divided into the face area and the body area, and the reliability of the face area and the body area may be calculated.

  In S706, the by-subject adjustment unit 312 reflects the adjustment parameter (that is, weighting) for each subject area on the threshold TH_g (x, y) calculated at S704, and calculates the threshold for each subject area. In the present embodiment, the threshold TH_human (x, y) of the person area, the threshold TH_back (x, y) of the background area, and the threshold TH_sky (x, y) of the sky area are calculated as in equation (7). Note that human_adj, back_adj, and sky_adj are adjustment parameters for a person area, a background area, and a sky area, respectively.

  As described above, by reflecting the adjustment parameter for each subject area, it is possible to adjust the threshold applied to each subject area even with the same gradation correction amount. Specifically, in the person area and the sky area, noise is reduced in the person area and the sky area by making adjustments so as to weaken the intensity of the edge enhancement while increasing the strength of the NR process compared to the background area. In this case, it is possible to generate an impressive image with high resolution. In the present embodiment, the threshold is calculated using the adjustment parameters of the three subject areas, but the human area may be divided into the face area and the body area, and the threshold may be calculated using four adjustment parameters.

  In step S707, the adjustment unit for each subject 312 calculates a final threshold TH (x, y) based on the reliability of each subject area calculated in step S705 and the threshold value of each subject area calculated in step S706. In the present embodiment, the threshold TH (x, y) is calculated by taking a weighted average as shown in equation (8). Here, TH_human (x, y), TH_back (x, y), and TH_sky (x, y) are the person area, background area, and sky area thresholds, respectively, and α, β, and γ are person area and background, respectively. It is the reliability of the area and the empty area.

  As described above, by adjusting the threshold according to the gradation correction amount according to the reliability, the width or the number of pixels of the pixel value averaged in equation (2) is changed according to the subject area. It is possible to change the noise removal intensity according to the subject area.

  When the threshold adjustment process is completed, the threshold adjustment unit 302 returns the process to S602 that is the read source.

  In S603, the average value processing unit 303 executes thresholded average value processing based on Expression (2) using the threshold value TH (x, y) calculated in S602, and combines the processed input image with the combination processing unit 306. Output to

  In S604, the luminance-based weighted addition coefficient calculation unit 304 and the weighted addition coefficient adjustment unit 305 calculate a weighted addition coefficient z for performing the image combining process based on Expression (3). The method of calculating the weighted addition coefficient z can be calculated using the gradation correction amount and the object discrimination result as in the method of calculating the threshold value TH (x, y) of the NR process, and thus detailed description will be omitted. . In addition, in the calculation of the weighted addition coefficient, the threshold value part in the above calculation method of the threshold value is replaced with the weighted addition coefficient, and the weighted addition coefficient according to the pixel position (x, y) is similar to the threshold value TH (x, y) Calculate z (x, y). First, the luminance-based weighted addition coefficient calculation unit 304 calculates the luminance-based weighted addition coefficient with reference to the weighted addition coefficient shown in FIG. The weighted addition coefficient adjustment unit 305 further uses the gradation correction amount Gain (x, y), the adjustment parameter for the subject region, and the reliability of each subject region (or the result of the subject determination) to add the weighted addition coefficient z (x, y) should be calculated.

  In step S605, the composition processing unit 306 uses the weighted addition coefficient z (x, y) (also simply referred to as z) calculated in step S604 to calculate the pixel value and the average value before the average value processing based on equation (3). Image composition processing is performed by combining the pixel values after processing. The composition processing unit 306 outputs the pixel value after composition as a result of the NR process.

  In step S606, the NR processing unit 205 determines whether processing has been performed on all pixels in order to perform the processing of steps S602 to S605 described above on all pixels of the image. If the NR processing unit 205 has not completed the processing for all the pixels, it proceeds the processing to S607 to return the processing to S602 again, and if the processing has been completed for all the pixels, A series of operations related to this process are completed.

  In S607, the NR processing unit 205 updates the pixel position (x, y) to be processed, and executes the series of operations from S602 again.

(Edge emphasis processing)
Next, the edge emphasizing process shown in S507 will be described in more detail. In the edge enhancement processing according to the present embodiment, first, an AC signal obtained by extracting a high frequency component from an input image and a DC signal obtained by removing the high frequency component are generated, and separate signal processing is performed on the generated AC signal and DC signal. In addition, an image is generated by adding these signals.

  In the following, generation processing of each signal and edge correction processing will be described while describing a series of operations of the edge enhancement processing with reference to FIG. 8 and FIG. Note that a series of operations of the edge enhancement process shown in FIG. 8 are realized by executing processes corresponding to the respective functional blocks shown in FIGS. 4A and 4B. Therefore, in the description of each step shown in FIG. Further, arrows shown by solid lines and dotted lines shown in FIG. 4A indicate flows of processing of the main line system and processing of the evaluation value system, as in FIG.

  Also, the edge emphasis processing unit 206 sequentially changes the position (x, y) of the processing target pixel and performs the processing described below on all the pixels.

  In step S801, the edge emphasis processing unit 206 initializes the pixel position (x, y) to be processed to (0, 0), for example, in order to set the coordinates of the upper left pixel of the input image as the start point.

  In step S802, the AC / DC signal generation unit 401 extracts an AC signal AC_in (x, y) from which the high frequency component is extracted based on the signal in (x, y) of the input image, and a DC signal DC_in (x) from which the high frequency component is removed. , y). The AC / DC signal generation unit 401 can use, for example, a method using a known band pass filter as disclosed in JP 2012-39603 A as a signal generation method. Therefore, the detailed description of the signal generation method is omitted.

  In step S803, the edge correction unit 403 receives the AC signal AC_in (x, y) generated in step S802 and calculates a signal AC_out (x, y) subjected to the edge correction process. The edge correction processing will be described later.

  In step S804, the γ processing unit 402 receives the DC signal DC_in (x, y) generated in step S802 and calculates a signal DC_out (x, y) subjected to the γ processing. The γ processing can use a known method for matching the signal value to the characteristics of the display system, and the γ processing unit 402 performs processing for compressing m + n bits to m bits according to, for example, a curve of γ characteristics (m, Do n ≧ 0).

  In step S805, the signal addition unit 404 adds the AC_out (x, y) calculated in step S803 and the DC_out (x, y) calculated in step S804, and outputs the signal out (x, y) after edge enhancement processing. Generate

  In step S806, the edge emphasis processing unit 206 determines whether the processing up to step S805 has been completed for all the pixels of the image. If the processing for all the pixels is completed, the series of operations related to the edge enhancement processing is ended, while if the processing for all the pixels is not completed, the processing proceeds to S807.

  In step S807, the edge emphasis processing unit 206 updates the pixel position (x, y) to be processed, and the process proceeds to step S802 again.

(Outline of edge correction processing)
Next, the edge correction processing in step S803 described above will be described. In the edge correction processing in the present embodiment, edge coring processing and edge gain processing are performed. In addition, in this embodiment, when applying the coring process which reduces the noise component generate | occur | produced by edge emphasis, the edge coring process which is disclosed by Unexamined-Japanese-Patent No. 2011-49965, for example is performed. More specifically, in order to cut off a minute noise component included in the high frequency signal, the coring amount calculated from the input signal using Expression (9) is subtracted. Here, BC is a coring amount, AC_in is an input signal, and AC_BC is a signal after coring processing.

  Further, in the edge gain processing, the high frequency signal after the coring processing is multiplied by the gain using the equation (10) to emphasize the high frequency signal. AC_BC is a signal after coring processing, AC_out is a signal after edge gain processing, and APCGain is an edge gain amount.

  The values of the coring amount BC of the edge coring processing of equation (9) and the gain amount APC_Gain of the edge gain processing of equation (10) are represented by FIGS. 14 (a) and (b), respectively. The horizontal axis in FIG. 14 indicates the value of the input signal (luminance signal Y generated from the CD signal), and the vertical axis indicates the coring amount and the edge gain value, respectively. The coring amount BC of the edge coring processing is represented by, for example, a curve of a logarithmic function as shown in FIG. 14 (a), and the weighted addition coefficient APC_Gain of the image combining processing has an inverse gamma characteristic as shown in FIG. 14 (b). Represented by a curve.

(A series of operations related to edge correction processing)
Furthermore, a series of operations of the edge correction process according to the present embodiment will be described with reference to FIG. In the present embodiment, since the noise is affected by the tone correction processing, the coring amount BC of the edge coring processing of equation (9) and the edge gain processing of equation (10) according to the tone correction amount. Control the value of the gain amount APC_Gain of

  Hereinafter, the process of calculating the amount of coring and the process of calculating the amount of gain in the edge gain process will be described while describing a series of operations of the edge correction process. The series of operations of the edge correction process shown in FIG. 9 are realized by executing processes corresponding to the respective functional blocks shown in FIGS. 4A and 4B. Therefore, in the description of each step shown in FIG. 9, the description will be made by appropriately referring to each of these operation subjects. Further, arrows shown by solid lines and dotted lines shown in FIGS. 4A and 4B indicate the flows of the processing of the main line system and the processing of the evaluation value system, as in FIG.

  In S901, the luminance-based coring amount calculation unit 411 and the coring amount adjustment unit 412 calculate a coring amount BC (x, y) used for the edge coring processing of Expression (9). The calculation of the coring amount BC (x, y) can be performed in the same manner as the calculation method of the threshold value TH (x, y) of the NR processing described above. For example, the coring amount by luminance calculating unit 411 calculates the coring amount by luminance with reference to the coring amount illustrated in FIG. Then, the coring amount adjustment unit 412 performs weighted averaging using the gradation correction amount Gain (x, y), the adjustment parameter for the subject area, and the reliability of each subject area (or the result of the subject determination). It suffices to obtain BC (x, y).

  In S902, the coring processing unit 413 performs edge coring processing on the input AC signal AC_in (x, y), and outputs a signal AC_BC (x, y) after the coring processing. Specifically, edge coring processing according to equation (9) is performed using an AC signal and the coring amount BC (x, y) calculated in step S901 to obtain AC_BC (x, y) as a gain multiplication unit 416. Output to

  In step S 903, the luminance edge gain calculator 414 and the edge gain adjuster 415 calculate a gain amount APCGain (x, y) used for edge gain processing (that is, used in equation (10)). The calculation of the gain amount APCGain (x, y) of the edge gain processing can also be performed in the same manner as the calculation method of the threshold value TH (x, y) of the NR processing. For example, first, the luminance-based edge gain calculation unit 414 calculates the luminance-based edge gain with reference to the edge gain illustrated in FIG. Further, the edge gain adjustment unit 415 performs weighted averaging of APCGain (x, y) using the gradation correction amount Gain (x, y), the adjustment parameter for the subject area, and the reliability of each subject area (or the result of the subject determination). ) Should be calculated.

  Thus, the gain amount of the processing is calculated so that the gain amount of the edge gain processing becomes larger as the gradation correction amount is larger. If such a gain amount is calculated, the larger the gradation correction amount is, the larger the gain is given in the sharpening shown in the equation (10). That is, the edge correction amount can be varied according to the magnitude of the gradation correction amount, and the strength of the edge enhancement can be changed according to the magnitude of the gradation correction amount. Furthermore, by adjusting the edge correction amount according to the gradation correction amount according to the reliability, the edge correction amount can be further varied according to the subject area, and edge enhancement is performed according to the subject area. The strength can be changed.

  In step S904, the gain multiplication unit 416 performs edge gain processing according to equation (10) using AC_BC (x, y) calculated in step S902 and the gain amount APCGain (x, y) calculated in step S903. The gain multiplication unit 416 outputs the signal AC_out (x, y) after edge gain processing (after correction) as a signal after edge correction processing. When the AC signal after the correction is output, the edge correction unit 403 ends the series of processes related to the edge correction process.

  As described above, in the present embodiment, in the NR process, the threshold value and the weighted addition coefficient in the thresholding averaging process are adjusted in accordance with the gradation correction amount and the subject area. In addition, also in the edge emphasizing process, the coring amount and the edge gain are adjusted in accordance with the gradation correction amount and the subject region. By doing this, it is possible to control the noise removal and the edge emphasis in consideration of the gradation correction amount and the subject area. Therefore, it is possible to generate an image with high contrast, wide dynamic range, and high SN ratio according to the characteristics of the subject area.

Second Embodiment
Next, the second embodiment will be described. In the second embodiment, as in the first embodiment, NR and edge enhancement processing are performed according to the tone correction amount and the subject area. However, in the present embodiment, the gradation correction amount used for the NR process and the edge enhancement process is not from the gain MAP but different from the point of calculation from the gain table. That is, the NR processing unit 1501 and the edge enhancement processing unit 1502 shown in FIG. 15A receive not the gain MAP generated by the gain MAP generation unit 203 but the gradation characteristic determined by the gradation characteristic determination unit 202. The point of performing each processing is different. By doing this, it is not necessary to record the gain MAP having the same size as that of the input image in the memory until NR and edge enhancement processing, and there is an advantage that the memory usage can be made more efficient.

  The configuration other than the NR processing unit 1501 and the edge enhancement processing unit 1502 is the same as that of the first embodiment, and therefore the same components are denoted by the same reference numerals and redundant description will be omitted. explain.

  As shown in FIG. 15A, the NR processing unit 1501 and the edge enhancement processing unit 1502 according to the present embodiment are configured to receive the gradation characteristic output from the gradation characteristic determining unit 202. .

  An exemplary functional configuration of the NR processing unit 1501 according to the present embodiment will be described with reference to FIG. In the present embodiment, the NR processing unit 1501 inputs the gradation characteristic determined by the gradation characteristic determination unit 202, and performs new processing in the gradation correction amount estimation unit 1503. In the first embodiment, although the tone correction amount is obtained directly from the gain MAP according to the equation (4), in the present embodiment, the equation using the tone characteristics determined by the tone characteristic determination unit 202 is used. Gradation correction amount is acquired according to (11). Here, Gain is a tone correction amount, in (x, y) is an input signal for reference, and GainTbl_after is a gain table for the input signal after gain. Also, the input signal for reference is an image (image subjected to gradation correction by the gain processing unit 204) input to the NR processing unit 1501.

  The gain table GainTbl_after for the input signal after the gain of Expression (11) will be described with reference to FIG. FIG. 17A shows the gradation characteristic determined by the gradation characteristic determination unit 202, and is a gain table for the input signal X before gain. This gain table is represented by the relationship between GainTbl_pre and the gradation correction amount Gain as represented by equation (12).

  On the other hand, FIG. 17B is a gain table for the input signal X 'after gain. Assuming that this gain table is GainTbl_after, the relationship with the gain table GainTbl_pre for the input signal X before gaining is as shown in Expression (13).

  As described above, the gradation correction amount estimation unit 1503 receives the gain table GainTbl_pre, creates the gain table GainTbl_after for the input signal after gaining, and performs the gradation correction based on the reference input signal. Calculate the amount Gain.

  The gradation correction amount estimation unit 1503 outputs the calculated gradation correction amount to the threshold value adjustment unit 302 and the weighted addition coefficient adjustment unit 305, and these functional blocks use the input gradation correction amount as described above in the first embodiment. Perform each process you

  Furthermore, a functional configuration example of the edge enhancement processing unit 1502 will be described with reference to FIGS. 16 (a) and 16 (b). As described above, the edge emphasis processing unit 1502 inputs the gradation characteristic determined by the gradation characteristic determination unit 202 in the same manner as the NR processing unit 1501. The input gradation characteristic is configured to be input to the edge correction unit 1601. Also, the gradation correction amount estimation unit 1602 shown in FIG. 16B calculates the gradation correction amount based on the input gradation characteristic. The tone correction amount estimation unit 1602 performs the same processing as the tone correction amount estimation unit 1503 provided in the NR processing unit 1501. The configuration other than the tone correction amount estimation unit 1602 is the same as that of the first embodiment.

  As described above, in the present embodiment, NR processing and edge enhancement processing are performed using the gradation characteristic determined by the gradation characteristic determination unit 202. In this manner, by calculating the gradation correction amount from the gain table, it is possible to further improve the efficiency of memory usage.

(Embodiment 3)
The third embodiment will be further described. In the third embodiment of the present invention, tone processing using a plurality of tone characteristics is performed for each subject region, and NR processing and edge enhancement processing according to the tone processing are performed. For this reason, the second embodiment differs from the first and second embodiments in that the plurality of gradation characteristics are mainly handled in the gradation characteristic determination processing, the NR processing, and the edge enhancement processing. Since the other configuration is the same as that of the second embodiment, the same reference numerals are given to the same configuration, and the redundant description will be omitted, and differences will be mainly described.

  First, region-specific tone mapping processing, which is gradation processing of the present embodiment, will be described with reference to FIG.

  FIG. 18A shows, in the same manner as FIG. 12A showing the gradation characteristics, the gradation characteristics in the case of giving each object region as input / output characteristics. In the embodiment described above, when the gradation characteristic is applied to the image photographed with the configuration of the object as shown in FIG. 12A, the gradation characteristic common to the whole screen is based on the gradation characteristic corresponding to each object region. (Gain table) was generated and gradation processing was performed. On the other hand, in the present embodiment, as shown in FIG. 18B, tone processing is performed by using different tone characteristics (gain table) for each subject region. For example, the input / output characteristics 2001 for the person area is applied to the person area, and the input / output characteristics 2002 and 2003 are applied to the background area and the sky area, respectively. That is, gain tables are generated for the person area, the background area, and the sky area, respectively, and gradation processing is performed. As described above, by applying different gradation characteristics to each subject area, it is possible to apply a constant input / output characteristic regardless of the range of the input signal within the same subject area.

  Further, the present embodiment is the same as the second embodiment in that the gradation correction amount is calculated from the gradation characteristic, but since the gradation characteristic exists for each object region, the gradation correction amount is calculated from these Is different.

  Next, a functional configuration example of the image processing unit 104 in the present embodiment will be described with reference to FIG. In the present embodiment, the gradation characteristic determination unit 2103 outputs different gradation characteristics for each subject region shown in FIG. 18A, and inputs them to the gain MAP generation unit 203, the NR processing unit 2101, and the edge emphasis processing unit 2102. Be done. The NR processing unit 2101 and the edge enhancement processing unit 2102 generate a gradation correction amount from the gradation characteristic for each subject area input as described above.

  The gain MAP generation unit 203 receives an object determination result which is an output of the object determination unit 201. The gain MAP generation unit specifies the subject area using the input subject discrimination result, and generates the gain MAP by applying the corresponding gradation characteristic.

  Further, the NR processing unit 2101 will be described with reference to FIG. The subject-specific tone correction amount estimation unit 2201 inputs the tone characteristics for each subject area output from the tone characteristic determination unit 2103 and calculates the tone correction amount for each subject area. The gradation correction amount estimation unit for each subject 2201 outputs the calculated gradation correction amount to the weighted addition coefficient adjustment unit 2202 and the threshold adjustment unit 2203 for each object region. The second embodiment is the same as the second embodiment except that the gradation correction amount is calculated for each subject region.

  A method of calculating the gradation correction amount for each subject based on the gradation characteristic for each subject will be described with reference to FIG.

  FIG. 22A shows the tone characteristics determined by the tone characteristics determination unit 2103, and shows a gain table for each subject region with respect to the input signal X before gain. The gain tables of the human region, background region, and sky region are GainTbl_pre_human, GainTbl_pre_back, and GainTbl_pre_sky, respectively, and the tone correction amounts of the human region, background region, and sky region are Gain_human, Gain_back, and Gain_sky, respectively. The relationship between the gain table and the gradation correction amount is expressed by equation (14).

  On the other hand, FIG. 22 (b) shows a gain table for the input signal X 'after gain. Assuming that gain tables for the human region, background region, and empty region are GainTbl_after_human, GainTbl_after_back, and GainTbl_after_sky, and gain tables GainTbl_pre_human, GainTbl_pre_back, and GainTbl_pre_sky for the input signal X before gain, these relationships are given by It will be expressed.

  Although Equation (15) represents the relationship between the gain tables of the person area, the same applies to the gain tables of the background area and the sky area.

  A subject-specific tone correction amount estimation unit 2201 receives gain tables GainTbl_pre_human, GainTbl_pre_back, and GainTbl_pre_sky for each subject region from the tone characteristic determination unit 2103, and gain tables GainTbl_after_human for each subject region for the input signal after gaining. , GainTbl_after_back, GainTbl_after_sky are created. Then, the tone correction amounts Gain_human, Gain_back, and Gain_sky for each subject area are calculated based on the input signal for reference.

  Next, a functional configuration example of the threshold value adjustment unit 2203 that inputs the gradation correction amount for each subject region will be described with reference to FIG. The threshold adjustment unit 2203 according to the present embodiment inputs the threshold TH for luminance and the gradation correction amount for each subject region. Then, the gradation correction amount adjustment unit 2205 multiplies the gradation correction amounts Gain_human, Gain_back, and Gain_sky for each object region by the fixed values k and l, m with the threshold value TH for each luminance as in Expression (16). Note that TH_human, TH_back, and TH_sky are thresholds adjusted by the gradation correction amount of each subject area.

  The processing after the thresholds TH_human, TH_back, and TH_sky are output from the subject-specific tone correction amount estimation unit 2201 is the same as in the first embodiment and the second embodiment. Also, although the processing of the threshold adjustment unit 2203 has been described as an example, the processing of the weighted addition coefficient adjustment unit 2202 similarly performs processing in which the amount of gradation correction for each subject region is input.

  Furthermore, a functional configuration example of the edge emphasis processing unit 2102 according to the present embodiment will be described with reference to FIG. The edge enhancement processing unit 2102 according to the present embodiment performs edge enhancement processing by inputting the tone characteristic for each subject determined by the tone characteristic determination unit 2103 as in the NR processing unit 2101.

  Further, the edge correction unit 2301 included in the edge enhancement processing unit 2102 will be described with reference to FIG. The edge correction unit 2301 inputs the gradation characteristic for each object area, the object discrimination result, and the DC / AC signal determined by the gradation characteristic determination unit 2103. A subject-specific tone correction amount estimation unit 2302 outputs a tone correction amount for each subject region according to the input tone characteristic of each subject region. Then, the edge gain adjustment unit 415 and the coring amount adjustment unit 412 input the gradation correction amount for each subject region and output the gain amount and the coring amount.

  As described above, in the present embodiment, the tone correction amount is calculated for each subject region using the tone characteristics for each subject region, and NR processing and edge enhancement processing according to each tone correction amount are performed. did. By doing this, it is possible to control the noise removal and the edge emphasis in accordance with the gradation correction amount applied to each object region, that is, in consideration of the gradation correction amount and the object region. Therefore, it is possible to generate an image with high contrast, wide dynamic range, and high SN ratio according to the characteristics of the subject area.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. Can also be realized. It can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

104: image processing unit, 201: subject determination unit, 202: gradation characteristic determination unit, 204: gain processing unit, 205: NR processing unit, 206: edge enhancement processing unit

Claims (13)

Determining means for determining a plurality of subject areas from the image;
Determining means for determining a gradation correction amount for performing gradation correction for each predetermined area of the image;
Noise removing means for changing the noise removal intensity to be applied to the image subjected to tone correction by the tone correction amount according to the tone correction amount and the determined subject area;
Edge emphasizing means for changing the intensity of edge emphasis to be applied to the image from which noise has been removed by the noise removing means in accordance with the gradation correction amount and the subject area determined as described above;
An image processing apparatus comprising: The noise removal means varies the width of the pixel value used for the averaging according to the gradation correction amount in averaging the pixel values for removing noise, and for each of the determined subject regions. The image processing apparatus according to claim 1, wherein the noise removal intensity is changed by adjusting the magnitude of the variation. The noise removal means performs the averaging by selecting pixels used for the averaging so that the width of the pixel value included in the averaging becomes larger as the gradation correction amount becomes larger. The image processing apparatus according to claim 2 . The noise removal unit performs weighting on a predetermined subject for the magnitude of the change according to the gradation correction amount, thereby removing noise by changing the width of the pixel value used for the averaging according to the subject. The image processing apparatus according to claim 3 , wherein the intensity of the image is changed. The subject area determined by the determining means includes at least two areas of a person area, an empty area, and a background area in the image;
The noise removing unit performs weighted averaging on the width of the weighted pixel value according to the reliability indicating the possibility that the plurality of determined subject regions are the respective subject regions, and the weighted averaged pixel The image processing apparatus according to claim 4 , wherein the averaging is performed using a range of values.   The edge emphasizing means varies the correction amount for emphasizing the high frequency signal according to the gradation correction amount, and adjusts the magnitude of the edge emphasis by adjusting the magnitude of the fluctuation for each of the determined subject regions. The image processing apparatus according to claim 1, wherein 7. The image processing according to claim 6 , wherein the edge emphasizing means changes the strength of the edge emphasis by increasing the correction amount for emphasizing the high frequency signal as the gradation correction amount is larger. apparatus. The edge emphasizing means performs weighting for each predetermined subject on the magnitude of the change according to the gradation correction amount, thereby changing the amount of correction for emphasizing the high frequency signal according to the subject. The image processing apparatus according to claim 7 , wherein the strength of the emphasis is changed. The subject area determined by the determining means includes at least two areas of a person area, a sky area, and a background area,
The edge emphasizing means performs weighted averaging of the correction amount for emphasizing the weighted high-frequency signal according to the reliability indicating the possibility that the plurality of determined subject regions are the respective regions, and the weighting is performed. 9. The image processing apparatus according to claim 8 , wherein the intensity of the edge enhancement is changed using a correction amount for enhancing the averaged high frequency signal. The noise removal means inputs the image subjected to the tone correction, estimates the tone correction amount based on the image, and changes the noise removal intensity according to the estimated tone correction amount. The image processing apparatus according to any one of claims 1 to 9 , characterized in that: The determination means determines the gradation correction amount for each of the determined subject areas.
11. The apparatus according to any one of claims 1 to 10 , wherein the noise removal unit changes the noise removal strength according to the gradation correction amount for each of the subject regions and the determined subject region. Image processing apparatus as described. A determining step of determining a plurality of subject areas from an image;
A determining step of determining a tone correction amount for tone correction for each predetermined area of the image;
A noise removing step of changing noise removal intensity applied to the image tone-corrected by the tone correction amount according to the tone correction amount and the subject region determined by the noise removing means;
An edge emphasizing step of changing an edge emphasizing intensity to be applied to the image from which noise has been removed in the noise removing step in accordance with the gradation correction amount and the subject region determined in the noise removing step;
And a control method of an image processing apparatus. A program for causing a computer to function as each means of the image processing apparatus according to any one of claims 1 to 11 .

Images