JP6757407B2 - Image processing equipment, image processing method and image processing program - Google Patents

Description

The present invention relates to an image processing apparatus, an image processing method, and an image processing program.

An image processing method that reduces noise based on the edge strength of the acquired image is known (see, for example, Patent Document 1 and Patent Document 2). This method uses a nonlocal averaging process (Non Local Means: NLM process) that utilizes weights based on the similarity between the pixel region of interest and its neighboring pixel region as spatial information. Further, a technique for reducing noise with a frequency component based on edge strength as frequency information is also known (see, for example, Patent Document 3).

Japanese Unexamined Patent Publication No. 2013-150778 Japanese Unexamined Patent Publication No. 2015-225665 Japanese Unexamined Patent Publication No. 2014-81764

In image noise reduction technology, it is required to reduce noise and at the same time maintain the contrast that the subject in the image can originally have. The NLM treatment has a great effect of achieving both noise reduction and contrast maintenance in a portion having a relatively large contrast other than a flat portion where a texture pattern does not exist (that is, reducing noise does not affect the contrast).
However, for parts with low contrast, the difference between the amount of noise and the magnitude of contrast is small when viewed in an image, so the trade is that the contrast decreases when trying to reduce the noise, and the noise remains when trying to maintain the contrast. There is a disadvantage that the image quality cannot be improved due to turning off.

On the other hand, when noise reduction is performed by a frequency component, it is assumed that noise reduction is performed according to the magnitude of the frequency. Even for a portion with low contrast, noise reduction is performed for the corresponding frequency, which has the effect of achieving both noise reduction and contrast maintenance.
However, since the frequency component refers only to the pixel region of interest, the above effect is low if the difference between the contrast frequency component and the noise frequency component within the region is small. For example, random noise has a relatively large number of high-frequency components, but the high-contrast portion also has a relatively large number of high-frequency components. There is an inconvenience.

The present invention has been made in view of the above circumstances, and provides an image processing apparatus, an image processing method, and an image processing program capable of generating an image with high image quality by reducing noise without reducing contrast. The purpose is to do.

One aspect of the present invention is an edge strength detection unit that detects edge strength information indicating edge strength in an acquired image, and a first noise reduction unit that performs noise reduction processing using spatial information on the image. A second noise reduction unit that performs noise reduction processing using frequency information on the image, a first processed image that has been noise-reduced in the first noise reduction unit, and a second noise reduction unit. The image synthesizing unit is provided with an image synthesizing unit for synthesizing the noise-reduced second-processed image, and the image synthesizing unit has the edge strength represented by the edge strength information detected by the edge strength detecting unit. In a region larger than a predetermined threshold, the composite ratio of the first processed image is higher than the composite ratio of the second post-processed image, and in a region where the edge intensity is smaller than the threshold value, the composite of the second post-processed image is performed. This is an image processing apparatus that synthesizes images using weights whose ratio is higher than the composition ratio of the first processed image.

According to this aspect, the image acquired by the image acquisition device is input to the edge strength detection unit, the first noise reduction unit, and the second noise reduction unit. The edge strength detection unit detects edge strength information indicating the strength of the edge in the input image, and the first noise reduction unit performs noise reduction processing using spatial information to perform the first post-processing image. Is generated, and in the second noise reduction unit, noise reduction processing using frequency information is performed to generate an image after the second processing.

Then, in the image synthesizing unit, the image after the first processing and the image after the second processing are combined by the weight based on the information representing the intensity of the edge detected by the edge strength detecting unit. As a result, when the edge strength is larger than the predetermined threshold value, the image after the first processing is synthesized at a higher composition ratio than the image after the second processing, and in the region where the edge strength is smaller than the predetermined threshold value, after the second processing. The image is synthesized at a higher composition ratio than the image after the first processing.

That is, in the region where the edge strength is small, the composition ratio of the second processed image subjected to the noise reduction processing using the frequency information is increased, and in the region where the edge strength is large, the noise reduction processing is performed using the spatial information. By increasing the composition ratio of the image after the first processing, it is possible to generate a high-quality image in which noise is reduced and the contrast is maintained regardless of the magnitude of the contrast of the image.

In the above aspect, in the first threshold value, the image synthesizing unit has a higher composition ratio of the second processed image than the composition ratio of the first processed image, and in a second threshold value larger than the first threshold value. The composite ratio of the first processed image is higher than the composite ratio of the second processed image, and the strength of the edge represented by the edge strength information is between the first threshold value and the second threshold value. The composite ratio of the first-processed image may gradually increase as the size increases, and the composite ratio of the second-processed image may gradually decrease.
By doing so, the influence of the noise reduction processing on the contrast can be smoothly changed according to the magnitude of the edge intensity.

Further, in the above aspect, the image synthesizing unit may synthesize using a weight at which the composition ratio of the first processed image and the composition ratio of the second processed image are switched at the threshold value.
By doing so, it is possible to reduce noise and generate a high-quality image without reducing the contrast by a simple configuration.

Further, in the above aspect, the information representing the intensity of the edge may be the luminance dispersion in the pixel block composed of a plurality of pixels.
Further, in the above aspect, the edge strength detecting unit may detect the amount of change in brightness by using the edge detecting filter.

Further, in the above aspect, the information representing the strength of the edge may be the texture degree in the pixel block composed of a plurality of pixels.
Further, in the above aspect, the information representing the intensity of the edge may be the distribution of frequency components in a pixel block composed of a plurality of pixels.

Further, in the above aspect, the information representing the strength of the edge may be a contrast value in a pixel block composed of a plurality of pixels.
Further, in the above aspect, the information representing the strength of the edge may be the amount of blur in the pixel block composed of a plurality of pixels.
Further, in the above aspect, the information representing the strength of the edge may be the flatness of the pattern in the pixel block composed of a plurality of pixels.

In addition, another aspect of the present invention includes an edge strength detection step of detecting edge strength information representing the edge strength in the acquired image, and a first method of performing noise reduction processing using spatial information on the image. A noise reduction step, a second noise reduction step of performing noise reduction processing using frequency information on the image, a first processed image subjected to noise reduction processing in the first noise reduction step, and the second image. The edge including the image synthesis step of synthesizing the noise-reduced second post-process image in the noise reduction step, and the image synthesis step represented by the edge strength information detected by the edge strength detection step. In a region where the intensity of the image is greater than a predetermined threshold, the composite ratio of the image after the first treatment is higher than the composite ratio of the image after the second treatment, and in a region where the intensity of the edge is smaller than the threshold, after the second treatment. This is an image processing method in which images are combined using a weight at which the image composition ratio is higher than the image composition ratio of the first processed image.

In addition, another aspect of the present invention includes an edge strength detection step of detecting edge strength information representing the edge strength in the acquired image, and a first method of performing noise reduction processing using spatial information on the image. A noise reduction step, a second noise reduction step of performing noise reduction processing using frequency information on the image, a first processed image subjected to noise reduction processing in the first noise reduction step, and the second image. The second post-processed image that has been noise-reduced in the noise reduction step is combined with the first image in a region where the edge intensity represented by the edge intensity information detected by the edge intensity detection step is larger than a predetermined threshold value. In a region where the composition ratio of the processed image is higher than the composition ratio of the second processed image and the edge intensity is smaller than the threshold value, the composition ratio of the second processed image is the composition ratio of the first processed image. It is an image processing program that causes a computer to perform an image composition step of synthesizing using a higher weight.

According to the present invention, there is an effect that noise can be reduced and an image with high image quality can be generated without reducing the contrast.

It is a block diagram which shows the camera provided with the image processing apparatus which concerns on one Embodiment of this invention. It is a block diagram which shows the edge strength calculation part of the image processing apparatus of FIG. It is a figure which shows an example of the noise model which is subtracted in the noise model subtraction part provided in the edge strength calculation part of FIG. It is a figure which shows an example of the edge strength conversion characteristic stored in the edge strength conversion table provided in the edge strength calculation part of FIG. It is a block diagram which shows the 1st noise reduction part of the image processing apparatus of FIG. It is a block diagram which shows the 2nd noise reduction part of the image processing apparatus of FIG. It is a figure which shows an example of the relationship between the input frequency component and the output frequency component before the process explaining the coring process of the 2nd noise reduction part of FIG. It is a figure which shows an example of the relationship between the input frequency component and the output frequency component after the process explaining the coring process of the 2nd noise reduction part of FIG. It is a block diagram which shows the image synthesis part of the image processing apparatus of FIG. It is a figure which shows an example of the relationship between the edge strength and the weighting coefficient calculated by the weight calculation unit provided in the image composition part of FIG. It is a flowchart explaining the image processing method which concerns on one Embodiment of this invention. It is a figure which shows another example of the relationship between the edge strength and the weighting coefficient calculated by the weight calculation unit provided in the image composition part of FIG. It is a block diagram which shows the modification of the image synthesis part of the image processing apparatus of FIG.

Hereinafter, the image processing apparatus 1, the image processing method, and the image processing program according to the embodiment of the present invention will be described below with reference to the drawings.
As shown in FIG. 1, the image processing device 1 according to the present embodiment is provided in, for example, the camera 100.

The camera 100 captures a photographing lens 110 that collects light from a subject, an image sensor 120 that photographs the light collected by the photographing lens 110, and an image signal acquired by the image pickup element 120 in a RAW data state. It includes an image pickup unit 130 that outputs the image, a pre-processing unit 140 that preprocesses the RAW data output from the image pickup unit 130, an image processing device 1 according to the present embodiment, and a post-processing unit 150.

The pre-processing unit 140 uses OB (Optical Black) correction processing, WB (White Balance) correction processing, and color information interpolation (using conditions at the time of shooting, for example, F value, focal length, ISO sensitivity, shutter speed, etc.). The RGB image after development is output after performing demosiking) processing and the like.
The post-processing unit 150 performs distortion correction, data compression processing, and the like on the image output from the image processing device 1 to generate a final output image.

The image processing device 1 according to the present embodiment includes an edge strength calculation unit (edge strength detection unit) 2, a first noise reduction unit 3, and a second noise in which a developed RGB image output from the preprocessing unit 140 is input. It includes a reduction unit 4, an image composition unit 5 that synthesizes a first post-processing image output from the first noise reduction unit 3, and a second post-processing image output from the second noise reduction unit 4. ..

As shown in FIG. 2, the edge strength calculation unit 2 converts the input developed RGB image into a plurality of blocks composed of a plurality of pixels, and the block division unit 6 and information on contrast for each block. The contrast calculation unit 7 that calculates and generates a contrast image, the conversion characteristic extraction unit 8 that reads out the edge intensity conversion characteristics based on the contrast image generated by the contrast calculation unit 7, and the conversion characteristic extraction unit 8 reads out. It is provided with an edge strength conversion unit 9 that generates an edge strength image from a contrast image based on the edge strength conversion characteristics.

The information regarding the contrast calculated by the contrast calculation unit 7 includes, for example, the calculation result of a filter (edge detection filter) related to edge detection. Specifically, a known filter such as a Laplacian filter, a Prewitt filter, a Sobel filter, or a Canny filter may be used. These filters detect an edge, that is, a change in brightness, and utilize the intensity of the edge as the amount of change in brightness. In this case, the larger the amount of change in brightness, the larger the edge strength.

In addition, other information on contrast includes information that evaluates contrast independently of edges. Specifically, the luminance dispersion in the block, the texture degree of the pattern in the block, the distribution of the frequency components in the block, the contrast value of the subject in the block, the amount of blur, and the like can be mentioned. Most of this information is based on the fluctuation of the brightness in the block. For example, it can be considered that the larger the brightness dispersion, the larger the fluctuation of the brightness and the larger the edge strength. Similarly, it can be considered that the larger the texture degree, frequency distribution and contrast value are, the larger the edge strength is, and the smaller the amount of blur is, the larger the edge strength is.

As yet other information regarding contrast, as an evaluation value indicating the absence of edges, one indicating the flatness of the pattern can be mentioned. The large flatness in the block means that there is no pattern in which the brightness changes in the block, and it can be considered that the edge strength is small.

The contrast calculation unit 7 uses one or more of the above-mentioned information on contrast to generate a contrast image in which each pixel includes information on contrast.
The conversion characteristic extraction unit 8 includes a noise model subtraction unit 10 and an edge strength conversion table 11.
The noise model subtraction unit 10 subtracts the noise component according to the noise model obtained by measuring the amount of noise with respect to the brightness of the image in advance from the contrast image calculated by the contrast calculation unit 7.

An example of the noise model is shown in FIG. In this noise model, the larger the ISO sensitivity, the more noise there is. In addition, the brightness of the noise model after being generated as an RGB image may be converted from the RAW data by the gradation characteristic (γ characteristic), and if the RGB pixel value is large, the amount of noise is not necessarily large. In the example shown in FIG. 3, the amount of noise is maximum with respect to the intermediate gradation or the slightly darker luminance.

Then, the contrast image information from which the noise component has been subtracted by the noise model subtraction unit 10 refers to the characteristic for converting the contrast image information into the edge intensity information in the edge intensity conversion table 11. FIG. 4 shows an example of conversion characteristics stored in the table.

The conversion characteristic of FIG. 4 represents the edge strength with respect to the contrast image information (pixel value). The edge strength is set to zero as the minimum. When the contrast image information is smaller than the lower threshold value, it is considered that there is no edge and the edge intensity is set to zero (minimum). Further, when the contrast image information is larger than the upper threshold value, the edge strength is maximized assuming that there is sufficient edge strength.

The characteristics of FIG. 4 assume that the luminance distribution is used as the contrast image information. Instead of this, a plurality of characteristics are stored in the edge intensity conversion table 11 according to the characteristics defined as the contrast image information, and the characteristics are selected and referred to according to the type or configuration of the contrast image information. You may decide.

The edge strength conversion unit 9 converts the contrast image generated by the contrast calculation unit 7 into an edge strength by using the edge strength conversion characteristic output from the edge strength conversion table 11. In the present embodiment, the edge strength increases as the contrast image information increases due to the edge strength conversion characteristic of FIG.

As shown in FIG. 5, the first noise reducing unit 3 separates the developed RGB image into a first luminance color difference separating section 12 and a first luminance color difference separating section 12 that separates the developed RGB image into luminance information and color difference information. The first luminance band separator 13 that separates the separated luminance information into bands, the first color difference band separator 14 that separates the separated color difference information into bands, and the noise for each band separated by the first luminance band separator 13. Band-specific luminance NLM processing unit 15 that performs NLM processing, which is reduction processing, band-specific color difference NLM processing unit 16 that performs NLM processing for each band separated by the first color difference band separation unit 14, and NLM-processed band-specific It includes a first coupling unit 17 that combines luminance information and color difference information to generate a noise reduction processed image after the first processing.

Separating into bands means thinning out the images. For example, the first band has the same size as the RGB image, and the second band has the width and height of the RGB image reduced by half. However, the third band corresponds to providing a plurality of band images such that the width and height of the RGB image are each reduced to 1/4 in size.

The NLM process is a process that reduces noise by evaluating the similarity of the area existing in the periphery to the area containing the pixel of interest, and in particular, it provides excellent noise reduction performance in areas with large contrast such as edges. Have.
The first coupling unit 17 does not convert the brightness information and the color difference information into an RGB image, but has a format for reference in the subsequent synthesis processing (for example, the brightness NLM processed image in the first band, the first band). The information is converted into information according to a predetermined order format such as a color difference NLM processed image and a brightness NLM processed image in the second band. This combined information is used as the image after the first processing.

As shown in FIG. 6, the second noise reduction unit 4 separates the developed RGB image into the brightness information and the color difference information in the second luminance color difference separation unit 18 and the second luminance color difference separation unit 18. It includes a second luminance band separating section 19 that separates the separated luminance information into bands, and a second color difference band separating section 20 that separates the separated color difference information into bands.
Further, the second noise reduction unit 4 is separated into bands and a band-specific brightness frequency conversion unit 21 that performs discrete cosine conversion (DCT) for each band to convert the brightness information separated into bands into brightness information of frequency components. It is provided with a band-specific color difference frequency conversion unit 22 that performs discrete cosine conversion for each band to obtain color difference information of frequency components.

Further, the second noise reduction unit 4 is a band-specific brightness DCT processing unit 23 that performs DCT-Coring processing, which is a noise reduction processing, on the brightness information converted into frequency components by the band-specific brightness frequency conversion unit 21, and a band-specific color difference. Using the band-specific color difference DCT processing unit 24 that performs DCT-Coring processing on the color difference information converted into frequency components by the frequency conversion unit 22, and the brightness information that has been DCT-Coring processed, the frequency by the band-specific brightness frequency conversion unit 21. It is provided with a band-specific brightness WF processing unit 25 that performs Wiener Filtering processing on the brightness information converted into components.

Further, the second noise reduction unit 4 reverse-converts the luminance frequency by frequency inverse conversion of the DCT-Coring processed band-specific luminance information, the DCT-Corning-processed color difference information, and the Wiener Filtering-processed luminance information for each band. A unit 26, 27, a color difference frequency inverse conversion unit 28, and a second coupling unit 29 that combines the images output from these inverse conversion units to generate a noise-reducing processed second-processed image. ing.

The DCT-Coring process in the band-specific luminance DCT processing unit 23 and the band-specific color difference DCT processing unit 24 is a process of reducing the frequency component in a predetermined range shown in FIG. 7A as shown in FIG. 7B. The coring process is performed so that the output frequency component becomes zero between Th1 and Th1. However, the output frequency component does not necessarily have to be zero, and for example, the absolute value of the output frequency component may be small between the threshold value -Th1 and Th1 and may be a characteristic that gradually changes according to the input frequency component.
In the present embodiment, paying attention to the fact that the noise component is composed of a relatively high frequency component, the high frequency component is subjected to a coring process to remove the noise.

The band-specific brightness WF processing unit 25 performs Wiener Filtering processing on the brightness information converted into frequency components by the band-specific brightness frequency conversion unit 21 using the DCT-Corning processed brightness information as a guide for a specific band. Is supposed to do. Here, the specific band is a band having a relatively high frequency component. Here, a Wiener filter based on the idea of the least squares error is applied with the aim of removing only the noise component while leaving the pattern that the image can originally have.

The luminance frequency inverse converters 26 and 27 and the color difference frequency inverse converter 28 include the luminance frequency information of the band subjected to the DCT-Coring process and the Wiener Filtering process, and the luminance and color difference frequency of the band subjected to the DCT-Coring process only. Inverse frequency conversion is performed on the information to return it to the information in the real space. Since the luminance frequency conversion unit 21 for each band and the color difference frequency conversion unit 22 for each band have undergone DCT processing as frequency conversion processing, the luminance frequency inverse conversion units 26 and 27 and the color difference frequency inverse conversion unit 28 perform inverse DCT processing. It is supposed to be done.

The second coupling unit 29 follows the same format as that of the first coupling unit 17, and the luminance frequency noise reduction image for each band output from the luminance frequency inverse converters 26 and 27 and the band output from the color difference frequency inverse converter 28. An image with different color difference frequency noise reduction is combined as a single piece of information and output as an image after the second processing.

Next, the image synthesizing unit 5 will be described.
As shown in FIG. 8, the image synthesizing unit 5 calculates a weight representing a synthesizing ratio for synthesizing brightness information based on the edge intensity information output from the edge intensity calculating unit 2. And the color difference composition weight calculation unit 31 that calculates the weight representing the composition ratio for synthesizing the color difference information based on the edge intensity information, and the luminance information and the second in the image after the first processing based on the calculated weight. The band-specific luminance synthesizer 32 that synthesizes the luminance information in the processed image, and the band that synthesizes the color difference information in the first processed image and the color difference information in the second processed image based on the calculated weight. Band synthesis is performed by the different color difference synthesizing unit 33, the luminance band synthesizing unit 34 that band-synthesizes the synthesized luminance composite image for each band, and the luminance band synthesizer 35 that band-synthesizes the synthesized luminance composite image for each band. It is provided with a luminance color difference compositing unit 36 that generates an RGB image from the luminance composite image and the luminance composite image.

The luminance composite weight calculation unit 30 and the color difference composite weight calculation unit 31 use the edge strength information calculated by the edge strength calculation unit 2 to calculate the weight coefficient as shown in FIG.
FIG. 9 shows the relationship between the edge strength and the weight. When the edge strength is smaller than the first threshold value, the weighting coefficient Ws = 0 with the composite ratio of the first processed image subjected to the noise reduction processing using spatial information as 0%, and the noise reduction processing using the frequency information. When the weighting coefficient Wf = 1 with the composite ratio of the second processed image as 100% and the edge strength is larger than the second threshold value larger than the first threshold value, the composite ratio of the first processed image is 100. The weighting coefficient Ws = 1 is set to%, and the weighting coefficient Wf = 0 is set to 0% of the composite ratio of the second processed images.

Further, between the first threshold value and the second threshold value, the weighting coefficient Ws of the image after the first processing is gradually increased, the weighting coefficient Wf of the image after the second processing is gradually decreased, and the sum of both weighting coefficients Ws and WF. Is set to 1.
That is, the composition of the image after the first processing and the image after the second processing can be expressed by the following equation.
NRI = Wf x NRf + Ws x NRs
Wf + Ws = 1

Here, NRs indicates an image after the first processing, NRf indicates an image after the second processing, and NRI indicates a composite image.

An image processing method using the image processing apparatus 1 according to the present embodiment configured in this way will be described below.
In order to perform the noise reduction processing included in the image using the image processing device 1 according to the present embodiment, as shown in FIG. 10, the preprocessing is performed by the preprocessing unit 140 acquired by the imaging unit 130. Edge strength information is calculated from the RGB image developed by (edge strength detection step S1), and the first noise reduction unit 3 performs NLM processing to generate an image after the first processing (first noise reduction). In step S2) and the second noise reduction unit 4, the DCT-Coring process and the Wiener Filtering process are performed to generate an image after the second process (second noise reduction step S3).

Then, the weight representing the compositing ratio is determined based on the edge strength information calculated in step S1, and the first processed image and the second post-processed image are combined according to the determined weight (image compositing step S4). ).
In this case, a noise reduction image using spatial information and a noise reduction image using frequency information based on the edge strength (large or small contrast) included in the image can be adaptively used, and each image can be used. Information with a large noise reduction effect can be used. As a result, there is an advantage that it is possible to acquire a high-quality image in which noise is reduced and the contrast is maintained without depending on the magnitude of the contrast that the subject originally has.

That is, when the edge strength is smaller than the first threshold value, there are almost no edges, so the frequency information is superior to the spatial information calculated from the image features such as pattern matching in terms of noise reduction performance. There is. In this case, noise can be effectively reduced by maximizing the weighting coefficient Wf related to frequency information, minimizing the weighting coefficient Ws related to spatial information, and using only frequency information.

On the other hand, when the edge strength is larger than the second threshold value, since strong edges exist, the spatial information calculated from the image features such as pattern matching is superior in noise reduction performance to the frequency information. In this case, noise can be effectively reduced by minimizing the weighting coefficient Wf related to frequency information, maximizing the weighting coefficient Ws related to spatial information, and using only spatial information.

Further, when the edge strength is between the first threshold value and the second threshold value, the weighting coefficients Wf, Ws so that the composition ratio of the first processed image using the spatial information increases as the edge strength increases. By setting, the noise reduction image using spatial information and the noise reduction image using frequency information can be used adaptively.

In the present embodiment, as shown in FIG. 9, the weight with respect to the edge strength is specified to be changed by two threshold values, but as shown in FIG. 11, it may be switched by a single threshold value. Good.

Further, as shown in FIG. 12, in the band-specific luminance synthesis unit 32 in the image synthesis unit 5, in addition to the weight coefficient calculated by the band-specific luminance synthesis weight calculation unit 30, the color difference composition weight calculation unit 31 calculates. It may be synthesized by using the weighting coefficient.

There may be colored edges / high contrast areas in the image. Even if the color difference information is large (the primary colors of red and blue are strong), the luminance information may be relatively small. In such a case, in the image synthesizing unit 5 of FIG. 12, when the luminance information is small, the edge strength is small, so that the image after the second processing using the frequency information is combined with a large weight. However, the color edge / contrast can be small because the weight of the information about the high contrast structure is small.

By as shown in FIG. 12, in the portion where the color edge / contrast is large, the weight of the image after the first processing using the spatial information that maintains the structure can be increased even if the edge information is weak. The luminance composite image according to FIG. 12 is as follows.
NRI = (Wf (y) -Wf (y) x Ws (c)) x NRf
+ (Ws (y) + Wf (y) x Ws (c)) x NRs
Wf (y) + Ws (y) = 1
0 ≦ Ws (c) ≦ 1

Here, Wf (y) and Ws (y) are the weighting coefficients of the first processed image and the second processed image in the luminance information, respectively, and Ws (c) is the weighting coefficient of the first processed image in the color difference information. Is. According to this equation, the weighting coefficient of the image after the first processing of the color difference information is more important than the first weighting coefficient of the luminance information. Table 1 shows the composite images when the weighting factors are the minimum and maximum.

According to this, even if the weighting coefficient of the first processed image in the luminance information is small, if the weighting coefficient of the first processed image in the color difference information is large, the weighting coefficient of the first processed image is also large in the luminance information. Become. Therefore, in an image in which the luminance information is small but the color difference information is large and the color edge / contrast is large, the weight in the composition of the luminance information can be corrected by using the weight of the color difference information having a large contrast, and appropriate noise reduction can be achieved. Can be planned.

Further, in the present embodiment, the configuration in which the image processing method is realized by hardware has been described, but it may be realized by an image processing program that can be executed by a computer. In this case, the computer includes a main storage device such as a CPU and RAM, and a computer-readable recording medium, and an image processing program for realizing the above-described processing is recorded on the recording medium. Then, the CPU can read the image processing program recorded on the recording medium and realize the same processing as that of the image processing device 1.

Further, in the present embodiment, the case where the image processing device 1 is applied to the camera 100 has been described, but instead, it may be applied to any imaging device such as a microscope and an endoscope. ..

1 Image processing device 2 Edge strength calculation unit (edge strength detection unit)
3 1st noise reduction unit 4 2nd noise reduction unit 5 Image composition unit S1 Edge strength detection step S2 1st noise reduction step S3 2nd noise reduction step S4 Image composition step

Claims (12)

An edge strength detector that detects edge strength information that represents the strength of the edge in the acquired image,
A first noise reduction unit that performs noise reduction processing using spatial information on the image,
A second noise reduction unit that performs noise reduction processing using frequency information on the image,
An image synthesizing unit for synthesizing a noise-reduced first-processed image in the first noise-reducing unit and a second-processed image noise-reduced in the second noise-reducing unit
In the region where the edge strength represented by the edge strength information detected by the edge strength detection unit is larger than a predetermined threshold value, the image synthesizing unit sets the composition ratio of the image after the first processing to the second processing. Image processing in which the composite ratio of the second post-processed image is higher than the composite ratio of the first post-processed image in a region higher than the composite ratio of the post-image and the intensity of the edge is smaller than the threshold value. apparatus. In the first threshold value, the image synthesizing unit has a higher synthesizing ratio of the second processed image than the synthesizing ratio of the first processed image, and in a second threshold value larger than the first threshold value, after the second processing. The composition ratio of the image after the first processing is higher than the composition ratio of the image, and as the intensity of the edge represented by the edge strength information increases between the first threshold value and the second threshold value, the first The image processing apparatus according to claim 1, wherein the composite ratio of the 1-processed image gradually increases, and the composite ratio of the second post-processed image gradually decreases. The image processing apparatus according to claim 1, wherein the image synthesizing unit synthesizes images using a weight at which the synthesizing ratio of the first processed image and the synthesizing ratio of the second processed image are switched at the threshold value. The image processing apparatus according to any one of claims 1 to 3, wherein the information representing the intensity of the edge is brightness dispersion in a pixel block composed of a plurality of pixels. The image processing apparatus according to any one of claims 1 to 3, wherein the edge strength detection unit detects a change in brightness using an edge detection filter. The image processing apparatus according to any one of claims 1 to 3, wherein the information representing the strength of the edge is the degree of texture in a pixel block composed of a plurality of pixels. The image processing apparatus according to any one of claims 1 to 3, wherein the information representing the intensity of the edge is a distribution of frequency components in a pixel block composed of a plurality of pixels. The image processing apparatus according to any one of claims 1 to 3, wherein the information representing the strength of the edge is a contrast value in a pixel block composed of a plurality of pixels. The image processing apparatus according to any one of claims 1 to 3, wherein the information representing the strength of the edge is the amount of blur in a pixel block composed of a plurality of pixels. The image processing apparatus according to any one of claims 1 to 3, wherein the information representing the strength of the edge is the flatness of a pattern in a pixel block composed of a plurality of pixels. An edge strength detection step that detects edge strength information representing the edge strength in the acquired image, and
The first noise reduction step of performing noise reduction processing using spatial information on the image, and
A second noise reduction step of performing noise reduction processing using frequency information on the image, and
An image composition step of synthesizing a noise-reduced first post-process image in the first noise reduction step and a noise-reducing second-processed image in the second noise reduction step is included.
In the region where the edge intensity represented by the edge intensity information detected by the edge intensity detection step is larger than a predetermined threshold value in the image composition step, the composition ratio of the image after the first processing is the second processing. Image processing in which the composite ratio of the second post-processed image is higher than the composite ratio of the first post-processed image in a region higher than the composite ratio of the post-image and the intensity of the edge is smaller than the threshold value. Method. An edge strength detection step that detects edge strength information representing the edge strength in the acquired image, and
The first noise reduction step of performing noise reduction processing using spatial information on the image, and
A second noise reduction step of performing noise reduction processing using frequency information on the image, and
The edge detected by the edge strength detection step of the image after the first processing that has been noise-reduced in the first noise reduction step and the image after the second processing that has been noise-reduced in the second noise reduction step. In the region where the strength of the edge represented by the intensity information is larger than a predetermined threshold, the composite ratio of the first processed image is higher than the composite ratio of the second processed image, and the strength of the edge is smaller than the threshold. An image processing program that causes a computer to perform an image composition step of synthesizing images using a weight that makes the composition ratio of the second processed image higher than the composition ratio of the first processed image in the region.

Images