JP6525700B2 - Image processing apparatus, image processing method and program - Google Patents

Description

  The present invention relates to an image processing technique for reducing color noise.

  It is generally known that color image data captured by an imaging device such as a digital camera is subjected to image processing by being separated into a luminance signal representing brightness and a color difference signal representing the color difference of each color component. In recent years, high-quality images are desired for imaging devices such as digital cameras. In recent years, in particular, there is a high demand for high sensitivity imaging, and it is desired that a high-quality image with low noise be obtained even in a dark place or at night. However, in a dark place or an environment where a sufficient S / N ratio can not be obtained as in the nighttime, noise (color noise) of the color difference signal appears as low frequency random noise, and the image quality is degraded.

  In order to suppress this color noise, a smoothing process using an averaging filter, a Gaussian filter or the like, and a color noise reduction process using an order statistical filter such as a median filter have been conventionally performed. However, in the case of using a smoothing process or an order statistical filter, in order to sufficiently reduce a large range of noise (low frequency noise), it is necessary to design a large number of filter taps, causing an increase in circuit scale. It will In this respect, a method has been proposed in which the same effect is obtained without increasing the number of taps by performing filter processing after reducing the input image (see Patent Document 1).

JP, 2010-157163, A

  By the way, when performing color noise reduction of an input image, color bleeding may occur in the vicinity of an edge. For example, although it is possible to reduce color noise in a large range by reducing the input image and performing filter processing, color bleeding occurs at the boundary of the color region due to the enlargement of the reduced image.

  Further, even in the case of reducing color noise without reducing the input image, color bleeding is likely to occur at the boundary due to the large range of color noise reduction. Therefore, if filter processing is performed in a range that does not generate color bleeding as much as possible, color noise will remain near the edge this time. In the vicinity of an edge, color noise may not be sufficiently reduced even by filtering in a large range. The improvement is particularly important because high-contrast edge portions are particularly noticeable, and color noise present in achromatic regions is particularly noticeable visually.

An image processing apparatus according to the present invention performs reduction processing on input image data to generate reduced image data of a plurality of layers having a reduced resolution, reduction of the input image data and the plurality of layers. the pixel value of the pixel of interest in the image data, and correcting means for correcting a value from which noise has been reduced, of the reduced image data of the input image data and the plurality of hierarchies, from the image data to be the focused and the focused image data and deriving means also derives the synthesis ratio for synthesizing the image data of low resolution is lower level, based on the pixel value of a local region including the pixel of interest definitive to each image data corrected by said correction means an evaluation value deriving means for deriving an evaluation value corresponding to the pixel of interest, based on the derived synthesis ratio and the evaluation value, the correction is performed image Of over data, and image data to be the interest has, synthesizing means for synthesizing the image data of the under layer, the combining means, when the evaluation value satisfies a predetermined condition, the lower The image data is composited using a predetermined composition ratio having a large ratio to the image data of layer, and when the evaluation value does not satisfy the predetermined condition, at least one of the image data of interest and the image data of the lower layer It is characterized by combining using a combining ratio derived according to one .

  According to the present invention, it is possible to effectively reduce color noise near the edge of the achromatic region.

It is a figure showing an example of the hardware constitutions of an image processing device. (A) is a block diagram which shows an example of the logic structure which concerns on the noise reduction process of an image processing apparatus, (b) is a block diagram which shows the internal structure of a color noise reduction process part. It is the sequence diagram which showed the flow of the whole process implement | achieved by the logic configuration | structure shown in FIG.2 (b). It is a figure which shows the specific example of the filter used for a low pass filter process. It is a figure which shows the specific example of how to low-pass-filter with respect to an attention pixel. It is a figure explaining how a pixel value is corrected. (A) is a figure which shows an example of a two-dimensional area | region centering on an attention pixel, (b)-(e) shows an example of a one-dimensional area | region. It is a figure explaining a mode that a color difference evaluation value is derived | led-out with respect to the one-dimensional area | region centering on an attention pixel. It is a figure which shows the specific example of a priority use determination process. It is a flowchart which shows the flow of each process in a color noise reduction process. It is a flowchart which shows the flow of reduction processing. It is a flowchart which shows the flow of expansion synthetic | combination processing.

  Hereinafter, embodiments of the present invention will be described with reference to the attached drawings. The configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

Example 1
FIG. 1 is a diagram illustrating an example of a hardware configuration of an image processing apparatus according to the present embodiment. The image processing apparatus 100 is, for example, a PC, and includes a CPU 101, a RAM 102, an HDD 103, a general-purpose interface (I / F) 104, a monitor 108, and a main bus 109. Then, an imaging device 105 such as a camera, an input device 106 such as a mouse and a keyboard, and an external memory 107 such as a memory card are connected to the main bus 109 by the general-purpose I / F 104.

  The CPU 101 realizes various processing as described below by operating various software (computer programs) stored in the HDD 103.

  First, the CPU 101 starts an image processing application stored in the HDD 103 and expands the image processing application on the RAM 102 and displays a user interface (UI) on the monitor 108. Subsequently, various data stored in the HDD 103 or the external memory 107, image data acquired by the imaging device 105, a user instruction from the input device 106, and the like are transferred to the RAM 102. Further, the data stored in the RAM 102 is arithmetically processed based on an instruction from the CPU 101 in accordance with processing in the image processing application. The result of the arithmetic processing is displayed on the monitor 108 or stored in the HDD 103 or the external memory 107. The image data stored in the HDD 103 or the external memory 107 may be transferred to the RAM 102. Further, image data transmitted from the server via a network (not shown) may be transferred to the RAM 102.

  In this embodiment, in the image processing apparatus 100 having the configuration as described above, an aspect of inputting image data to an image processing application and outputting image data with reduced color noise based on an instruction from the CPU 101 will be described. It shall be.

(Logical configuration of image processing apparatus)
Subsequently, a logical configuration relating to noise reduction processing in the image processing apparatus 100 will be described.

  FIG. 2A is a block diagram showing an example of the logical configuration relating to the noise reduction processing of the image processing apparatus 100. As shown in FIG. The image processing apparatus 100 includes a signal conversion processing unit 201, a color noise reduction processing unit 202, a luminance noise reduction processing unit 203, and a signal integration processing unit 204.

  A color signal (color signal represented in RGB color space) of color image data input from the imaging device 105 or the HDD 103 or the external memory 107 is input to the signal conversion processing unit 201 first. Although the color image data in the present embodiment is described on the premise that it is an RGB color space, the present invention is not limited to this. For example, the color image data may be converted to an L * a * b * color space.

  The signal conversion processing unit 201 generates a luminance signal representing the luminance component (Y) and a color difference signal representing the color difference components (Cr and Cb) from the RGB color signal of the input color image data by a known conversion formula. Do. As a result, it is possible to obtain a color signal composed of a luminance signal representing luminance and a color difference signal representing a color difference (hue vector). Here, the spatial frequency of noise of the luminance signal (hereinafter, luminance noise) and the spatial frequency of noise of the color difference signal (hereinafter, color noise) are different from each other, and the spatial frequency of the noise of the color difference signal is lower. In this processing, YCrCb color signals are generated from RGB color signals related to input color image data, thereby performing optimal noise reduction processing on each of the luminance signal representing the luminance component and the color difference signal representing the color difference component. be able to. The input RGB color signal and the YCrCb color signal generated by the signal conversion process are sent to the color noise reduction processing unit 202 and the luminance noise reduction processing unit 203.

  The color noise reduction processing unit 202 performs processing to reduce noise (color noise) of the color difference signal based on the RGB color signal and the YCrCb color signal. Details of the color noise reduction processing will be described later.

  The luminance noise reduction processing unit 203 performs processing to reduce noise of the luminance signal. In the luminance noise reduction processing unit 203, general noise reduction processing may be used.

  The signal integration processing unit 204 integrates the color signal whose color noise has been reduced by the color noise reduction processing unit 202 and the color signal whose luminance noise has been reduced by the luminance noise reduction processing unit 203. The signal integration processing unit 204 outputs color image data in which both color noise and luminance noise are reduced, which are generated as a result of the signal integration processing. The color image data after integration is output to the monitor 108, the HDD 103, or the like. In addition, for example, the data may be output to the external memory 107 connected to the general-purpose I / F 104, an external server (not shown), a printer, or the like.

(Details of color noise reduction processing unit)
Subsequently, the color noise reduction processing unit 202 will be described in detail. FIG. 2B is a block diagram showing an internal configuration of the color noise reduction processing unit 202. As shown in FIG. As shown in FIG. 2B, the color noise reduction processing unit 202 includes a reduction processing unit 210, a color difference correction unit 220, a combination ratio derivation unit 230, a color difference evaluation value derivation unit 240, and an enlargement combination processing unit 250. Ru. And FIG. 3 is a sequence diagram showing the flow of the entire processing realized by the logical configuration shown in FIG. 2 (b). Hereinafter, each part of the color noise reduction processing unit 202 will be described with reference to FIG. 2 (b) and FIG. 3.

<Reduction processing unit>
The reduction processing unit 210 performs processing for reducing the resolution of the input image data (reduction processing 302 a to 302 c in FIG. 3) to generate reduced image data. In the example shown in FIG. 3, the resolution of the input image is reduced by half, reduced by one-half, reduced by one-half 302b, and reduced by one-eighth by one-half 302c. Generate three kinds of reduced image data. In general, when reducing image data, aliasing noise occurs if the low-pass filter processing is not performed, and a pattern of the aliasing noise appears in the image data to be finally output. Therefore, when the reduction processing is performed, for example, an algorithm including an average pixel method or other low pass filter processing is used, or a bi-linear method is applied after low pass filter processing is performed in advance. The filter used for the low-pass filter processing is determined based on the reduction factor. For example, when the reduction factor is 1/2, a filter as shown in FIG. 4 is used. The filter size and coefficients are not limited to those shown in FIG. Hereinafter, the input image data and the plurality of reduced image data generated from the input image data will be collectively referred to as “multiresolution image data”. The reduced image data generated by the reduction process is stored in the RAM 102 for use in the color difference correction process described later.

<Color difference correction unit>
The color difference correction unit 230 corrects the value (color difference signal) indicating the color difference of the pixel of interest so that the color noise is reduced based on the correction parameter set in advance for the multi-resolution image data described above (FIG. 3). Color difference correction processing 303a to 303d). Here, the correction parameter means each threshold value for determining whether or not the target pixel is averaged with a predetermined reference area. It is desirable to change these to appropriate values according to the amount of noise. Therefore, for example, it is determined in advance in accordance with the imaging sensitivity, or adaptively determined based on the information of the pixel of interest and its peripheral pixels. In the color difference correction process, the values of the color difference signals Cr and Cb of the pixel of interest of the input color image data are corrected using, for example, the following equations (1) and (2).

In the above equations (1) and (2), Cr [i] and Cb [i] represent pixels satisfying the following equation (3) within a predetermined reference area for the pixel of interest. M is the number of pixels that satisfy the following formula (3) in the reference area for the pixel of interest. Note that M 注目 1 because the pixel of interest itself is always included.

In the above equation (3), ∧ represents a logical product (AND) (hereinafter the same), and Y represents the luminance of the pixel of interest. Further, Th _Y represents a threshold for luminance, Th _Cr represents a threshold for chrominance Cr, and Th _Cb represents a threshold for chrominance _Cb . Further, Y _diff , Cr _diff , and Cb _diff are differences between pixel values in the reference area and pixel values to be compared, and are represented by the following formulas (4) to (6), respectively.

The pixel values of the target pixel may be used as the pixel values Y _comp , Cr _comp , and Cb _comp to be compared in the above formulas (4) to (6). Alternatively, the result of applying a low pass filter to the pixel value of the target pixel may be used as the pixel value of the target pixel. FIG. 5 is a diagram showing a specific example of how to apply a low pass filter to a pixel of interest. In the example of FIG. 5, the pixel value of the “target pixel” is low-pass filtered as the “pixel values to be compared Y _comp , Cr _comp , Cb _comp ” using the target pixel and four pixels adjacent in the upper, lower, left, and right directions The result is derived. Note that if there is an effect similar to that of the low pass filter, the weighting and the number of adjacent pixels to be used may be freely determined. In addition, since human eyes have relative luminosity characteristics that green light is felt most brightly, G may be treated as simplified luminance instead of luminance Y. In this case, the value Cr (Cb) indicating the color difference is determined by R-G (B-G).

FIG. 6 is a diagram for explaining how the pixel value is corrected by the above-mentioned equations (1) to (3). Here, image data having pixel values (14 bits: 0 to 16383) indicated by reference numerals 601 to 603 are correction targets, and the reference area is an area indicated by a bold frame. In this case, pixel values to be compared indicated by reference numerals 604 to 606 = pixel values of target pixels (Y _Comp : 3394, Cr _Comp : -33, Cb _Comp : -150), and indicated by reference numerals 607 to 609. Y _diff , Cr _diff and Cb _diff are first derived. Then, fitting to the above equation (3) is made using the derived Y _diff , Cr _diff , and Cb _diff, and all seven pixels having a value “1”, which is indicated by the reference numeral 610, have the above equation (1). And the pixel used for averaging in equation (2). Finally, correction values (Cr _result : −27, Cb _result : −48) of values Cr and Cb indicating the color difference at the pixel of interest indicated by reference numeral 611 are derived.

  Note that the color difference correction process shown here is an example, and any other method may be used as long as it is a method of correcting a value indicating a color difference so as to reduce color noise. The multi-resolution image data whose color noise has been reduced by the color difference correction process is stored in the RAM 102 for use in the combination ratio derivation process and the color difference evaluation value derivation process described later.

<Composition ratio derivation unit>
A process of deriving a combining ratio with lower layer image data with respect to each image data included in multi-resolution image data in which color noise is reduced by the color difference correction process (composition ratio deriving process in FIG. 3) 304a to 304c). However, as is clear from the sequence diagram of FIG. 3, the image data of the lowermost layer (the reduced image data reduced to 1/8 in this embodiment) has no image data lower than itself. It is not the subject of this processing. Here, the combination ratio is determined in accordance with the degree of abrupt change of the color difference signal, and can be derived, for example, by edge detection with the color difference signals Cr and Cb of the pixel of interest as inputs. Specifically, among the results of applying the filter in the horizontal direction and the vertical direction using a spatial filter having a filter coefficient [-1, 2, -1], one that is considered to have a high degree of color edge is previously determined. Map to a coefficient K between 0 and 1 by the defined function. A coefficient K having a value closer to 1 is output as the filter application result considered to have a higher degree of color edge, and a coefficient K having a value closer to 0 is output as the filter application result considered to have a lower color edge degree. Then, this coefficient K is the combining ratio. Note that a luminance signal may be used for edge detection. Also, the filter coefficient and the determination method are not limited to those described above. It may be derived in any way as long as it is possible to obtain a large value combination ratio when the degree of color edge is considered to be high and a small value combination ratio when the degree of color edge is considered low. The derived data of the combining ratio is stored in the RAM 102 for use in the enlargement and combining process described later.

<Color difference evaluation value derivation unit>
A process of deriving an evaluation value based on color difference for each image data included in the multi-resolution image data processed by the color difference correction unit 220 (color difference evaluation value derivation process 305 a to 305 d in FIG. 3) Do). Specifically, for the two-dimensional area and the one-dimensional area centered on the pixel of interest, Y _avg and Cr _avg are used as color difference evaluation values using Formulas (7) to (9) below, respectively. , Cb _avg are derived respectively.

The equation (7) to formula _{(9), Y i, Cr} i and Cb _i represent pixels respectively in each region, N is the represents the number of pixels constituting the region. FIG. 7A shows an example of a two-dimensional area centered on the pixel of interest, and FIGS. 7B to 7E show an example of a one-dimensional area. Then, in the case of a one-dimensional area, evaluation values in each direction are respectively derived.

FIG. 8 is a diagram for explaining how the color difference evaluation value is derived for the area shown in FIG. 7B in the one-dimensional area centered on the pixel of interest. In this case, for each of the Y component, the Cr component, and the Cb component, fitting to the above equations (7) to (9) is performed using pixel values of thick frame portions indicated by reference numerals 801 to 803. As a result, color difference evaluation values ( _Yavg : 3028, _Cravg : 68, _Cbavg : 106) as indicated by reference numeral 804 are derived.

  The size of the area and the number of directions of the one-dimensional area are not limited to this. Moreover, although the average value is used as the evaluation value in the above-mentioned example, an addition value may be used. The information of the derived evaluation value of the color difference is stored in the RAM 102 for use in the priority use determination processing described later.

<Magnification processing unit>
The enlargement / composition processing unit 250 combines two types of image data having different resolutions among the multi-resolution image data processed by the color difference correction unit 220 with the combination ratio derived by the combination ratio derivation unit 230 and the color difference evaluation value derivation unit 240. A combining process is performed based on the derived color difference evaluation value. The enlargement composition processing unit 250 further includes an enlargement processing unit 251, a priority usage determination unit 252, and a pixel value combining unit 253. Hereinafter, each part which comprises the expansion compositing process part 250 is demonstrated.

«Expansion processing unit»
The enlargement processing unit 251 doubles the image data of each layer excluding the image data of the highest layer (the input image data itself; the input image 301 in FIG. 3) in the multi-resolution image data by, for example The enlargement processes 306a to 306c in FIG. 3 are performed. At this time, the image data of the lowest layer is the image data processed by the color difference correction unit 220 and the image data of the other lower layers is the object after the composition by the pixel value composition process described later. It is image data. In the example of the sequence diagram of FIG. 3, the target of the enlargement processing 306 a is the image data of the lowest layer subjected to the color difference correction processing 303 d. Further, the target of the enlargement processing 306 b is the image data after combination subjected to pixel value combination processing 308 a described later. The target of the enlargement processing 306 c is the image data after combination subjected to pixel value combination processing 308 b described later. Note that the method of enlargement processing is not limited to the bilinear method, and, for example, the nearest neighbor method, the bicubic method, the Lanczos method, or the like may be used.

«Priority use judgment unit»
A process of determining whether or not to use image data of a lower hierarchy preferentially when combining the image data of interest and the image data of the next lower hierarchy (FIG. 3). The priority usage determination processing 307a to 307c) is performed. This determination is made based on the color difference evaluation value derived by the color difference evaluation value deriving unit 240. Here, “image data of one lower hierarchy” is image data whose resolution is lower by one step than the image data of interest, and is enlarged by the enlargement processing unit 251 to the same resolution as the image data of interest. Point image data. This will be described using the example of the sequence diagram of FIG. First, in the preferential use determination processing 307a, the image data reduced to the resolution of 1/4 of the input image data and the resolution to 1/8 of the input image data are reduced (the enlargement processing makes the resolution the same) Determination is made when combining with the image data. Then, in the priority use determination processing 307b, the image data reduced to a resolution of 1⁄2 of the input image data and the resolution to a resolution of 1⁄4 of the input image data are processed (the enlargement processing causes the same resolution Determination is made when combining with the image data. Then, in the preferential use determination processing 307c, the determination when combining the input image data itself and the image data that has been reduced to half the resolution of the input image data (which has become the same resolution by the enlargement processing) Is done.

  Then, the priority use determination process is performed by fitting the result of the lower layer side among the color difference evaluation values derived by the above-mentioned equations (7) to (9) to the following equation (10) in pixel units. .

In the above equation (10), a and b are coefficients set according to the hierarchy and the imaging sensitivity, and the lower the hierarchy or the smaller the imaging sensitivity, the smaller the value, the higher the hierarchy, or the higher the imaging sensitivity Set to a value as large as possible. The above equation (10) corresponds to FIG. 7 and is a determination equation when one direction is set in four directions of (b) to (e) for a one-dimensional area. That is, it is determined whether or not the pixel of interest and its surrounding pixels are close to an achromatic color by the evaluation value of the two-dimensional area (FIG. 7A), and the one-dimensional area (FIGS. The evaluation value of d) determines that there is no color edge. Although the direction of the one-dimensional area is not necessarily limited to four directions, it is desirable to set one-dimensional areas in four or more directions as much as possible. Actually, even if there is a color edge, it is determined that the area is achromatic, thereby preventing the occurrence of an event that the color edge which is not desired and eventually disappears disappears. When the condition of the above equation (10) is satisfied, the determination is further performed using the following equation (11).

The above equation (11) compares the color difference of the pixel of interest between the lower layer side (low resolution side: Cr _down , Cb _down ) and the upper layer side (high resolution side: Cr _up , Cb _up ). It is determined whether the color difference is smaller. When the condition of equation (11) is further satisfied after satisfying the condition of equation (10), the composition ratio is changed such that image data of the lower layer is adopted, for example, 100% in pixel value composition processing It will be.

FIG. 9 is a diagram showing a specific example of the preferential use determination process. Now, reference is made to two-dimensional or one-dimensional areas (FIGS. 7A to 7E) in image data having pixel values (14 bits: 0 to 16383) indicated by reference numerals 601 to 603, respectively. The color difference evaluation values indicated by reference numerals 901 to 905 are derived. If these are applied to the above-mentioned equation (10), the determination result of “the condition is not satisfied” indicated by reference numeral 906 is obtained. Temporarily, when the conditions of the judgment formula of said Formula (10) are satisfy | filled, it will progress to the fitting to said Formula (11). In the example of | Cr _down | = 56, | Cb _down | = 62, | Cr _up | = 69, | Cb _up | = 79 shown in FIG. It becomes the judgment result of When such a determination result is obtained, for example, the combination ratio u (0 ≦ u ≦ 1) representing the ratio on the upper layer side is changed to a value close to “0”. The data of the combined ratio changed in this manner is stored in the RAM 102.

«Pixel value synthesis unit»
The pixel value composition unit 253 performs the pixel value of the image data of interest and the pixel value of the image data (same resolution as the image data of interest) after enlargement processing of the image data of the next lower hierarchy to each stage. A combining process is performed based on the combining ratio derived by the combining ratio deriving process. In FIG. 3, pixel value combining processing 308 a to 308 c correspond to this. Specifically, assuming that the pixel value on the lower layer side is I _down , the pixel value on the upper layer side is I _up , and the composition ratio is u, the pixel value I _post after composition is calculated using Equation (12) below. Is required.

For example, when the composition ratio u = 0.1, the lower layer side pixel value I _down = 3405, and the upper layer side pixel value I _up = 3621, the pixel value I _post after composition is “3427” according to the above equation (12). Become. Note that the pixel values here may be in any format, may be YcrCb values, or may be RGB values.

  As described above, the composition ratio u is a coefficient indicating how much image data on the upper layer side is to be used, and is basically derived by the composition ratio deriving unit 230 and appropriately changed according to the result of the priority use determination process. It will be. In the case of the present embodiment, the color image data after combining, which is the result of the pixel value combining process 308a / 308b, is temporarily stored in the RAM 102 and provided to the enlargement process 306b / 306c. On the other hand, the image data after combining, which is the result of the pixel value combining process 308c, is output as it is (see the sequence diagram of FIG. 3). The output mode at this time includes, for example, display on the monitor 108, storage on the HDD 103, storage on the external memory 107 via the general-purpose I / F 104 or an external server (not shown), printing on a printer (not shown) There is something.

(Flow of color noise reduction processing)
Next, a rough flow of each process in the color noise reduction processing unit 202 will be described. FIG. 10 is a flowchart showing the flow of each process in the color noise reduction process. This series of processing is realized by the CPU 101 loading a program stored in the HDD 130 into the RAM 102 and executing it.

  When image data is input to the color noise reduction processing unit 202, in step 1001, the reduction processing unit 210 performs the above-described reduction processing on the acquired input image data to generate multi-resolution image data. Details of the reduction process will be described later.

  In step 1002, the color difference correction processing of the color difference correction unit 220 corrects a value (color difference signal) indicating the color difference of the target pixel so as to reduce color noise with respect to the image data of each layer in the generated multiresolution image data. Do.

  In step 1003, the composition ratio deriving unit 230 derives a composition ratio of the multi-resolution image data of which color noise has been reduced by the color difference correction process to the image data of the lower layer with respect to each image data excluding the lowest layer. Perform the process.

  In step 1004, the color difference evaluation value deriving unit 240 performs processing to derive a color difference evaluation value for each piece of image data included in the multi-resolution image data whose color noise has been reduced by the color difference correction processing.

  In step 1005, the enlargement / composition processing unit 250 applies the combination ratio derived in step 1003 and the color difference evaluation value derived in step 1004 to the multi-resolution image data whose color noise has been reduced by the color difference correction processing. , Perform the above-mentioned enlargement synthesis process. Details of the enlargement synthesis process will be described later.

  The above is the rough flow of each processing in the color noise reduction processing unit 202.

(Reduction processing flow)
Next, details of the reduction process (step 1001) in the flow of FIG. 10 described above will be described. FIG. 11 is a flowchart showing the flow of the reduction process.

  In step 1101, the reduction processing unit 210 acquires a reduction processing parameter. The reduction processing parameters include the depth of the hierarchy of multiple resolutions and information as to what fraction (a multiple of two) of the minimum resolution with respect to the input image data. Then, such reduction processing parameters are set in advance and stored in the HDD 103 or the like. In the case of this embodiment, the reduction processing parameter of the content that the minimum resolution is 1/8 of the input image data in all four layers (1 ×, 1/2 ×, 1/4 ×, 1/8 ×) Are set and stored in advance, and are acquired by reading them from the HDD 103 or the like.

  In step 1102, the reduction processing unit 210 determines a scaling factor to be applied to the reduction processing based on the reduction processing parameter acquired in step 1101. In the case of the present embodiment in which the contents of the reduction processing parameter are all four layers and the minimum resolution is one eighth of the input image data, three types of magnifications of 1/2, 1/4, and 1/8 are sequentially It will be decided.

  In step 1103, the reduction processing unit 210 performs processing of reducing the image data to be processed to the scaling factor determined in step 1102.

  In step 1104, the reduction processing unit 210 determines whether all the reduction processing based on the reduction processing parameter acquired in step 1101 is completed. In the case of the present embodiment, it is determined whether or not the reduction processing for all of the three types of magnifications of 1/2, 1/4, and 1/8 has been completed. If all the reduction processes corresponding to the acquired reduction process parameter have been completed as a result of the determination, this process ends. On the other hand, if there is an unprocessed magnification, the process returns to step 1102, the next magnification is determined, and the reduction processing is continued.

(Magnification processing flow)
Next, details of the enlargement and combining process (step 1005) in the flow of FIG. 10 described above will be described. FIG. 12 is a flowchart showing the flow of the enlargement and combining process.

  In step 1201, the enlargement / composition processing unit 250 acquires all image data (multi-resolution image data) whose color noise has been reduced by the color difference correction processing in step 1002 described above.

  In step 1202, the enlargement / combination processing unit 250 acquires the reduction processing parameter and the combination processing parameter described above. Here, the combination processing parameter means the combination ratio derived in step 1003 described above and the color difference evaluation value derived in step 1004 described above.

  In step 1203, the enlargement processing unit 251 of the enlargement / combination processing unit 250 performs the above-mentioned enlargement processing on the image data of the lowest layer (small resolution) among the plurality of image data acquired in step 1201 (this embodiment) Then do 2).

  In step 1204, the enlargement / composition processing unit 250 sets the image data of the minimum resolution among unprocessed image data among the plurality of image data acquired in step 1201 as one image data to be subjected to the pixel value synthesis process. select. In this case, the image data of the lowest hierarchy is not included in the unprocessed image data.

  In step 1205, the pixel value combining unit 253 of the enlargement combining processing unit 250 selects a pixel to be subjected to the pixel value combining process. Specifically, in the case of the first routine, the target of pixel value composition processing on the image data subjected to the enlargement processing in step 1203 and the image data selected in step 1204 having the same resolution as the image data subjected to the enlargement processing. Pixels are selected. In the case of the second and subsequent routines, pixel value composition processing is performed on the image data subjected to enlargement processing in step 1212 described later and the image data selected in step 1204 having the same resolution as the image data subjected to the enlargement processing. The target pixel of is selected.

  In the following steps 1206 to 1208, the priority use determination unit 252 of the enlargement / composition processing unit 250 performs the above-described priority determination process for the pixel selected in step 1205. That is, it is determined whether to preferentially adopt the image data (image data on the lower layer side) of the two image data to be subjected to the pixel value composition processing which has been subjected to the enlargement process.

  First, at step 1206, it is determined whether or not preferential adoption of image data on the lower layer side is possible (whether or not the above equation (10) is satisfied). As a result of the determination, if the image data on the lower layer side can be preferentially adopted, the process proceeds to step 1207. On the other hand, when the preferential adoption of the image data on the lower layer side is not possible, the process proceeds to step 1209.

  In step 1207, the color difference evaluation values of the two image data to be processed are compared, and the color difference evaluation value (E_low) of the image data on the lower layer side is the color difference evaluation value of the image data selected in step 1204 ( It is determined whether it is smaller than E_high). If it is determined that the color difference evaluation value (E_low) of the image data on the lower layer side is smaller, the process proceeds to step 1208. On the other hand, if the color difference evaluation value (E_low) of the image data on the lower layer side is not smaller, the process proceeds to step 1209.

  In step 1208, the combination ratio obtained in the combination ratio derivation process (step 1003 described above) is changed to a combination ratio that employs, for example, 100% of the pixel values of the image data on the lower layer side.

In step 1209, the pixel value combining unit 253 of the enlargement combining processing unit 250 performs the above-described pixel value combining process based on the currently set combining ratio. Thereby, a pixel value I _post is obtained by combining selected pixel values in two image data.

  At step 1210, the enlargement / composition processing unit 250 determines whether or not the combination processing for all pixels is completed. If the processing for all the pixels is completed, the process proceeds to step 1211. On the other hand, if there is an unprocessed pixel, the process returns to step 1205 to select the next pixel as the pixel to be processed and continue the process.

  In step 1211, the enlargement / combination processing unit 250 has performed enlargement processing on all image data (that is, all reduced image data excluding input image data) except multi-resolution image data other than image data of the highest hierarchy. Determine if. If it is determined that there is reduced image data for which enlargement processing has not been performed, the process proceeds to step 1212. On the other hand, if enlargement processing has been performed on all reduced image data, this processing ends.

  In step 1212, the enlargement processing unit 251 of the enlargement and combining processing unit 250 performs the above-mentioned enlargement processing (2 × in this embodiment) on the image data after combination processing obtained in step 1209. After the enlargement process is performed, the process proceeds to step 1204.

  The above is the contents of the enlargement synthesis process.

  As described above, according to this embodiment, color noise in the vicinity of the edge of the achromatic region in the input image data can be effectively reduced.

<Other Embodiments>
In the first embodiment, an example in which processing is performed by an image processing application has been described. However, these may be methods of processing image data captured by an imaging device on image processing hardware in the imaging device. Further, the image data may be transmitted from the client device to the image processing application on the server device, and the image data may be processed on the server device.

  The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. Can also be realized. It can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

Claims (19)

Reduction processing means for performing reduction processing on input image data to generate reduced image data of a plurality of layers having a reduced resolution;
A correction unit configured to correct the pixel value of the target pixel in the input image data and the reduced image data of the plurality of layers to a value with reduced noise;
Deriving means for deriving a combining ratio for combining the image data of interest and the image data of the lower layer having a lower resolution than the image data of interest among the input image data and the reduced image data of the plurality of layers When,
Based on the pixel value of a local region including the pixel of interest definitive to each image data corrected by said correction means, and the evaluation value deriving means for deriving an evaluation value corresponding to the pixel of interest,
Combining means for combining the image data of interest among the image data subjected to the correction and the image data of the lower layer based on the derived combining ratio and the evaluation value;
Have
The combining means is
When the evaluation value satisfies a predetermined condition, combining is performed using a predetermined combining ratio having a large ratio to the image data of the lower layer,
When the evaluation value does not satisfy the predetermined condition, combining is performed using a combining ratio derived according to at least one of the image data of interest and the image data of the lower layer. Image processing device. Image data of said next lower layer, the image processing apparatus according to claim 1, wherein an image data of one of a hierarchy of the focused image data. The combining means is
Enlargement processing means for enlarging image data of the next lower layer ;
Determining when synthesizing the image data of the one of a hierarchy between the target image data, whether it is possible to use the image data of preferentially the one lower hierarchy, on the basis of the evaluation value Priority use determination means to
Wherein the pixel value of the focused image data and a pixel value of the image data enlarged by the process of the expansion. The image data of the one of a hierarchy to the same resolution as the image data to be the focus, the synthesis ratio 3. The image processing apparatus according to claim 2, further comprising: pixel value combining means for combining based on W. The pixel value composition unit obtains a pixel value after composition using the following equation:
In the above equation, I _post represents the pixel value after composition, I _down represents the pixel value of the lower layer image data, I _up represents the upper layer pixel value, and u represents the upper layer image data. Represents the factor as the composite ratio which represents how much to use
The image processing apparatus according to claim 3, characterized in that:   The evaluation value deriving means is characterized in that the evaluation value is derived for a two-dimensional area centered on the target pixel and a one-dimensional area on a plurality of different directions centered on the target pixel. The image processing apparatus according to any one of claims 3 or 4, wherein the image processing is performed. The preferential use determining unit determines whether the target pixel and its surrounding pixels are close to an achromatic color by the evaluation value of the two-dimensional area, and further, determines the color by the evaluation value of the one-dimensional area. The image processing apparatus according to claim 5, wherein it is determined whether it is possible to use the image data of the next lower layer preferentially by determining that there is no edge.   The evaluation value deriving means calculates the evaluation value based on a luminance signal representing a luminance component (Y) of the input image data and a color difference signal representing a color difference component (Cr, Cb). Or the image processing apparatus as described in any one of 6. The evaluation value deriving means calculates the evaluation value for each of the luminance signal and the color difference signal using the following equation:
In each of the above formulas, Y _i , C r _i and Cb _i each represent a pixel in the two-dimensional area or the one-dimensional area, and N represents the number of pixels constituting each area. The image processing apparatus according to claim 7. The correction means corrects the value of the color difference signal representing the color difference component (Cr, Cb) of the pixel of interest according to the following equation:
Cr [i] and Cb [i] in the above formulas each represent a pixel in a predetermined area including the target pixel, and M represents the number of pixels in the predetermined area.
The image processing apparatus according to any one of claims 1 to 8, characterized in that: The pixels in the predetermined area are pixels that satisfy the following equation for the target pixel,
In the above equation, Y _diff , Cr _diff and Cb _diff represent the difference between the value of each pixel in the predetermined area and the predetermined pixel value to be compared, and Th _Y , Th _Cr and Th _Cb respectively represent luminance components ( Represents a threshold for Y) and a threshold for chrominance components (Cr, Cb),
The image processing apparatus according to claim 9, characterized in that:   The image processing apparatus according to claim 10, wherein the predetermined pixel value to be compared is a value obtained by applying a low-pass filter to the value of the target pixel or the value of the target pixel. 2. The image processing apparatus according to claim 1, wherein, when the evaluation value satisfies a predetermined condition, 100% of image data of the lower layer is used as the predetermined combination ratio. 2. The apparatus according to claim 1, wherein the deriving means derives a combining ratio corresponding to the degree of edge for each pixel based on at least one of the image data to which attention is paid and the image data of the lower layer. Image processing apparatus as described. The composition ratio according to the degree of the edge is
If the evaluation value satisfies a predetermined condition, the predetermined combination ratio is overwritten,
If the evaluation value does not satisfy the predetermined condition, the predetermined combination ratio is not overwritten.
The image processing apparatus according to claim 13, characterized in that: The image processing apparatus according to any one of claims 1 to 14, wherein the combining unit combines the image data to which attention is paid and the image data of the lower layer for each pixel. The priority use determining means preferably determines whether it is possible to use the image data of the next lower layer by priority depending on whether the evaluation value satisfies the predetermined condition. The image processing apparatus according to claim 3, 5 or 6. The preferential use determination unit performs the determination using the following equation as the predetermined condition,
In the above equation, a and b are coefficients set according to the hierarchy and the imaging sensitivity, and are set to smaller values as the hierarchy is lower or the imaging sensitivity is smaller.
The image processing apparatus according to claim 16, characterized in that: Performing a reduction process on the input image data to generate reduced image data of a plurality of layers with reduced resolution;
Correcting the pixel value of the pixel of interest in the input image data and the reduced image data of the plurality of layers to a value with reduced noise;
Deriving a combining ratio for combining the image data of interest and the image data of the lower layer having a lower resolution than the image data of interest among the input image data and the reduced image data of the plurality of layers; ,
Deriving an evaluation value based on pixel values of a local region including the pixel of interest definitive to each image data corrected in the step of correcting, corresponding to the pixel of interest,
Combining the image data of interest among the image data subjected to the correction and the image data of the lower layer based on the derived combination ratio and the evaluation value ;
Including
In the combining step,
When the evaluation value satisfies a predetermined condition, combining is performed using a predetermined combining ratio having a large ratio to the image data of the lower layer,
When the evaluation value does not satisfy the predetermined condition, combining is performed using a combining ratio derived according to at least one of the image data of interest and the image data of the lower layer.
An image processing method characterized in that. A program for causing a computer to function as the image processing apparatus according to any one of claims 1 to 17 .