JP6357980B2 - Image processing apparatus, digital camera, and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus, a digital camera, and an image processing program.

  Generates a low-frequency noise-removed image obtained by removing low-frequency noise from the input image and a high-frequency noise-removed image obtained by removing high-frequency noise from the input image, and combines the low-frequency noise-removed image and the high-frequency noise-removed image to remove noise. There is known an imaging apparatus that generates a captured image (see Patent Document 1). In this imaging apparatus, a low-frequency noise-removed image and a high-frequency noise-removed image are adopted so that a high-frequency noise-removed image is mainly used at a portion where the edge strength is high, and a low-frequency noise-removed image is mainly used at a portion where the edge strength is low. Is synthesized.

JP 2012-105091 A

  In the method described in Patent Document 1, low-frequency noise remains in the high-frequency noise-removed image, and high-frequency components disappear in the low-frequency noise-removed image. Therefore, according to the method described in Patent Document 1, low-frequency noise remains because a high-frequency noise-removed image is mainly used in a portion where the edge strength is large, and low-frequency noise removal is mainly performed in a portion where the edge strength is small. Since the image is used, the high frequency component disappears. Thus, the method described in Patent Document 1 cannot achieve both low-frequency noise removal and high-frequency component retention.

(1) An image processing apparatus according to claim 1, wherein the input image data is reduced to generate reduced image data, and noise reduction is performed on the reduced image data to generate low-frequency noise removal data. Low frequency noise removing means, difference calculating means for calculating difference data between reduced image data and low frequency noise removed data, and high frequency noise removal for performing noise removal on input image data to generate high frequency noise removed data And output image generation means for generating output image data based on the high-frequency noise removal data and the difference data.
(2) A digital camera according to a sixth aspect includes the image processing apparatus according to any one of the first to fifth aspects.
(3) An image processing program according to claim 7 reduces the input image data to generate reduced image data, and performs noise removal on the reduced image data to generate low-frequency noise removal data. Low-frequency noise removal processing, difference calculation processing for calculating difference data between reduced image data and low-frequency noise removal data, and high-frequency noise removal that performs noise removal on input image data to generate high-frequency noise removal data It is characterized by causing a computer to execute processing and output image generation processing for generating output image data based on high-frequency noise removal data and difference data.

  According to the present invention, both low frequency noise removal and high frequency component retention can be achieved.

It is a block diagram which shows the structure of the digital camera in one embodiment of this invention. It is a block diagram which shows the structure of a noise removal part. It is a figure explaining the range referred to the production | generation of reduced image data. It is a figure explaining a smoothing process. It is a figure explaining the range referred to high frequency noise removal. (A) is a graph illustrating image data without noise, (b) is a graph illustrating input image data with noise, and (c) is a graph illustrating high-frequency noise removal data. (A) is a graph illustrating reduced image data, (b) is a graph illustrating low-frequency noise removal data, and (c) is a graph illustrating difference data. (A) is a graph which shows provisional noise removal data, (b) is a graph which illustrates output image data. It is a block diagram which shows the structure of the noise removal part in the modification 1. It is a figure explaining the program supply to a computer.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration of a digital camera 100 according to the present embodiment. The digital camera 100 includes an operation member 101, a lens 102, an image sensor 103, a control device 104, a memory card slot 105, and a monitor 106.

  The operation member 101 includes various input members operated by the user, such as a power button, a release button, a zoom button, a cross key, an enter button, a play button, a delete button, a help button, and the like.

  The lens 102 is composed of a plurality of optical lenses, but is representatively represented by one lens in FIG. The image sensor 103 is, for example, a CCD or a CMOS, and captures a subject image formed by the lens 102 to acquire an image. Then, the acquired image data is output to the control device 104.

  The control device 104 includes a CPU, a memory, and other peripheral circuits, and functionally includes a noise removal unit 104a. The memory constituting the control device 104 includes a ROM (for example, a flash memory) that stores programs and the like, a RAM (for example, SDRAM) that is used as a buffer memory, and the like.

  The control device 104 performs a known Bayer interpolation process on the image acquired by the image sensor 103, and then inputs the image to the noise removal unit 104a to perform the noise removal process. This noise removal processing may be performed only for the color difference component of the image, may be performed only for the luminance component, or may be performed for both. Moreover, you may perform with respect to another color component. In addition to the noise removal process described in the present embodiment, a known process for removing noise in a lower frequency band may be added.

  Then, the control device 104 performs known gradation conversion, saturation enhancement, contour enhancement, and the like on the image after the noise removal processing by the noise removal unit 104a, and performs known compression processing as necessary to obtain an image file. And output to the memory card slot 105. However, the order of the noise removal process and other processes is not limited to the above.

  The memory card slot 105 is a slot for inserting a memory card as a storage medium, and the image file output from the control device 104 is written and recorded on the memory card. Further, based on an instruction from the control device 104, the image file stored in the memory card is read and reproduced, or the recorded image file is deleted.

  The monitor 106 is a liquid crystal monitor (rear monitor) mounted on the back of the digital camera 100, and the monitor 106 is set to set an image (reproduced image) stored in the memory card and the digital camera 100. Various information such as a menu is displayed. In addition, the control device 104 displays a through image on the monitor 106 by sequentially outputting frames input in time series from the image sensor 103 to the monitor 106.

  FIG. 2 is a block diagram illustrating a configuration of the noise removing unit 104a. The noise removing unit 104a functionally includes a reducing unit 201, a low frequency noise removing unit 202, a subtracting unit 203, a high frequency noise removing unit 204, an adding unit 205, and a range limiting unit 206. Hereinafter, the noise removal process executed by the noise removing unit 104a will be described with reference to FIG. The noise removing unit 104a selects a target pixel in the input image data, performs a noise removal process described below on the selected target pixel, and outputs a pixel value after noise removal. The noise removal unit 104a performs this noise removal processing on all the pixels of the input image data to remove noise from the entire input image data.

1. Generation of Reduced Image Data Hereinafter, the coordinates of the pixel of interest are assumed to be (x, y). The reduction unit 201 generates reduced image data corresponding to the target pixel (x, y) from the input image data. In the reduction unit 201, for example, as shown in FIG. 3, a range of 15 × 15 pixels (x−7, y−7) to (x + 7, y + 7) centered on the pixel of interest (x, y) is referred to. 7 × 7 pixel reduced image data is generated. The reduced image data of 7 × 7 pixels is smoothed in each of the pixels at the positions marked with “◯” in FIG. 3, that is, 7 × 7 pixels arranged every other pixel with the target pixel (x, y) as the center. Generated by arranging the output values. That is, the output value of the smoothing process at the pixel at the position where the circle is marked becomes the pixel value of one pixel of the reduced image data. Further, the smoothing process is performed as follows, for example, with respect to the pixel value of a 3 × 3 pixel centered on the pixel at the position where the mark “◯” is described. As shown in FIG. 4, the pixel value of the center pixel (the pixel at the position where the circle is marked) is 4/16, and the pixel value of four pixels adjacent to the center pixel vertically and horizontally is The position where a circle is written by applying a weight of 1/16 to the pixel values of 2/16 pixels at the four corners (four pixels adjacent to the center pixel in an oblique direction). The pixel value after the smoothing process in is calculated.

2. Calculation of Low-frequency Noise Removal Value The low-frequency noise removal unit 202 performs a known noise removal filter process on the center pixel of the 7 × 7 pixel reduced image data generated by the reduction unit 201, thereby reducing the low-frequency noise. A pixel value (hereinafter, referred to as a low-frequency noise removal value) from which is removed is calculated. As the noise removal filter process, for example, an ε filter process described below may be used.

  When ε filter processing is used, arithmetic processing is performed as follows using reduced image data of 7 × 7 pixels. In the following description, the pixel value of the center pixel in the reduced image data of 7 × 7 pixels is referred to as the center pixel value. In addition, pixel values of peripheral pixels around the center pixel (that is, 48 pixels other than the central pixel among 7 × 7 pixels) are referred to as peripheral pixel values.

[Ε filter processing]
(1) The integrated value is initialized to 0.
(2) For each of the 48 peripheral pixels, | peripheral pixel value-center pixel value | <adds (peripheral pixel value-center pixel value) to the integrated value if it is a predetermined threshold value. Performs an arithmetic operation that does not change the value. The predetermined threshold value is set to a value that is about three times the standard deviation of noise, for example.
(3) The low-frequency noise removal value is calculated by calculating the low-frequency noise removal value = center pixel value + integrated value / 49.

  Note that the noise removal filter processing is not limited to the ε filter processing described above. For example, integration may be performed with a weight according to the peripheral pixel position, or may be integrated with a weight according to | peripheral pixel value−center pixel value |. Moreover, you may combine with well-known noise removal processes, such as direction smoothing and protrusion removal. Alternatively, a weighted average with a value before noise removal may be applied by applying a predetermined noise removal rate. Other known techniques may be applied. Further, the reference range is not limited to the 7 × 7 pixels described above, and a range of about 3 × 3 pixels to 11 × 11 pixels may be referred to, for example.

3. Calculation of Difference Value The subtraction unit 203 subtracts the central pixel value (pixel value before low frequency noise removal) of the reduced image from the low frequency noise removal value calculated by the low frequency noise removal unit 202 (that is, low frequency noise). The difference value between the low-frequency noise removal value and the center pixel value of the reduced image is calculated. This difference value is a value added to the original pixel value by the low frequency noise removing unit 202 in order to remove the low frequency noise.

4). High-frequency noise removal The high-frequency noise removal unit 204 performs a known noise removal filter process on the target pixel (x, y) of the input image data, thereby removing a pixel value from which high-frequency noise has been removed (hereinafter referred to as a high-frequency noise removal value). Is calculated). The high-frequency noise removal value is a 7 × 7 pixel range (x−3, y−3) to (x + 3, y + 3) centered on the pixel of interest (x, y), as indicated by a dashed frame in FIG. Calculated by reference. About the noise removal filter process, it is the same as that of the case of the low frequency noise removal part 202 mentioned above.

5). Calculation of provisional noise removal value The addition unit 205 removes low frequency noise and high frequency noise by adding the difference value calculated by the subtraction unit 203 to the high frequency noise removal value calculated by the high frequency noise removal unit 204. The obtained pixel value (hereinafter referred to as a provisional noise removal value) is calculated.

6). Range Restriction of Output Pixel Value The range restriction unit 206 clips the provisional noise removal value calculated by the addition unit 205 within the range from the low frequency noise removal value to the high frequency noise removal value, and outputs the target pixel. The output pixel value at (x, y) is output. Specifically, the processing is performed as follows.

  First, the case where low frequency noise removal value> high frequency noise removal value will be described. In this case, when the high frequency noise removal value ≦ the provisional noise removal value ≦ the low frequency noise removal value, the provisional noise removal value is set as the output pixel value. When the provisional noise removal value <the high frequency noise removal value, the high frequency noise removal value is set as the output pixel value. When the provisional noise removal value> the low frequency noise removal value, the low frequency noise removal value is set as the output pixel value.

  Next, a case where the low frequency noise elimination value ≦ the high frequency noise elimination value will be described. In this case, when the low frequency noise removal value ≦ the provisional noise removal value ≦ the high frequency noise removal value, the provisional noise removal value is set as the output pixel value. When the provisional noise removal value <the low frequency noise removal value, the low frequency noise removal value is set as the output pixel value. When the provisional noise removal value> the high frequency noise removal value, the high frequency noise removal value is set as the output pixel value.

  The noise removal unit 104a repeats the above noise removal processing until all the pixels of the input image data are selected as the target pixel, obtains output pixel values for all the pixels, and noise is removed from the input image data. Output image data is generated.

  According to the above-described noise removal processing, low-frequency noise can be removed while leaving a high-frequency component even in a portion where the input image data has a high-frequency texture structure. Hereinafter, the effects of the above-described noise removal processing will be described using the graphs illustrated in FIGS. In the graphs illustrated in FIGS. 6 to 8, the vertical axis represents the pixel value, and the horizontal axis represents the pixel coordinate.

  FIG. 6A is a graph illustrating image data including high-frequency texture components. The graph of FIG. 6A shows an original texture pattern without noise. FIG. 6B is a graph illustrating image data (input image data) when the image shown in FIG. 6A is taken with high sensitivity. In this case, as shown in FIG. 6B, image data mixed with noise is captured. FIG. 6C is a graph illustrating high-frequency noise removal data (high-frequency noise removal value) after high-frequency noise removal is performed on the input image data shown in FIG. 6B by the high-frequency noise removal unit 204. . In general, since it is difficult to remove high-frequency noise at a place where there is a high-frequency texture structure, the noise is not reduced much in FIG.

  FIG. 7A is a graph illustrating reduced image data (low frequency image) generated by reducing the image shown in FIG. 6B by the reduction unit 201. As shown in FIG. 7A, in the reduced image data, high frequency components are removed and low frequency noise is extracted. FIG. 7B illustrates low-frequency noise removal data (low-frequency noise removal value) after low-frequency noise removal is performed by the low-frequency noise removal unit 202 on the reduced image data illustrated in FIG. It is a graph to do. In FIG. 7B, low frequency noise is removed. FIG. 7C is a graph illustrating difference data (difference value) obtained by subtracting the reduced image data shown in FIG. 7A from the low-frequency noise removal data shown in FIG. It is.

  FIG. 8A is a graph showing provisional noise removal data (provisional noise removal value) obtained by adding the difference data of FIG. 7C to the high-frequency noise removal data of FIG. It is. In the provisional noise removal data shown in FIG. 8A, noise is reduced by removing low frequency noise. FIG. 8B shows output image data (output pixel value) obtained by clipping the provisional noise removal data in FIG. 8A by the range restriction unit 206 into a range from low frequency noise removal data to high frequency noise removal data. It is a graph to illustrate. In the provisional noise removal data in FIG. 8A, the pixel data enclosed by the broken line is lower than the high frequency noise removal data in FIG. 6C, and is outside the range from the low frequency noise removal data to the high frequency noise removal data. For this reason, in the output image data of FIG. 8B, high-frequency noise removal data is adopted in the pixel data surrounded by the broken line instead of the provisional noise removal data of FIG.

  Even if the noise removal processing disclosed in Patent Document 1 is performed on the image structure as shown in FIG. 6B, low-frequency noise removal is not applied at a place where there is a high-frequency texture structure. A result of remaining noise corresponding to FIG. 6C is output. Alternatively, by adjusting the setting of the noise removal process disclosed in Patent Document 1 and adjusting so that the low-frequency noise removal is applied even to a place where there is a high-frequency texture structure as shown in FIG. The image data from which the high-frequency component corresponding to FIG. Thus, in the noise removal process disclosed in Patent Document 1, it is difficult to achieve both low-frequency noise removal and high-frequency component retention.

  Further, in the noise removal processing of the present embodiment, the strength of high frequency noise removal and the strength of low frequency noise removal can be controlled independently. This is because the high-frequency component is excluded from the reduced image data, and the difference data calculated based on this does not include the high-frequency component. Even if the difference data is added to the high-frequency noise removal data, the high-frequency component is not affected. Because. That is, the high frequency component is retained even if the low frequency noise is strongly removed. In general, in order to sufficiently remove low-frequency noise that is easily noticeable while leaving a delicate texture, it is desirable that the high-frequency noise removal is somewhat weak and the low-frequency noise removal is applied strongly. According to the noise removal processing of the present embodiment, the above-described effect (delivery texture) can be obtained by setting the threshold for high frequency noise removal and the noise removal rate to a small value and setting the threshold for low frequency noise removal and the noise removal rate to a large value. It is possible to sufficiently remove low-frequency noise that is easily noticeable).

  On the other hand, in Patent Document 1, if low frequency noise removal is to be applied strongly, the low frequency noise removal value must be set to be applied strongly in the synthesis of the low frequency noise removal value and the high frequency noise removal value. I don't get it. In this case, since the high frequency component is not included in the low frequency noise removal value, the high frequency component is easily lost in the output image. In this embodiment, the problem is solved.

  However, in the present embodiment, the value obtained by adding the difference value to the high-frequency noise removal value (provisional noise removal value) may generate noise with an opposite sign due to excessive subtraction of noise. In this embodiment, in order to solve the problem, the range limiting unit 206 clips the provisional noise removal value between the high frequency noise removal value and the low frequency noise removal value to obtain an output pixel value.

According to the embodiment described above, the following operational effects can be obtained.
(1) In the digital camera 100, the reduction unit 201 reduces the input image data and generates reduced image data. The low frequency noise removal unit 202 performs noise removal on the reduced image data to generate low frequency noise removal data (low frequency noise removal value). The subtraction unit 203 calculates difference data (difference value) between the reduced image data and the low-frequency noise removal data. The high frequency noise removal unit 204 performs noise removal on the input image data to generate high frequency noise removal data (high frequency noise removal value). The adding unit 205 generates provisional noise removal data (provisional noise removal value) based on the high frequency noise removal data and the difference data. Thereby, both low frequency noise removal and high frequency component retention can be achieved.

(2) In the digital camera 100, the range limiting unit 206 limits the provisional noise removal data to a predetermined level range (a range between the low-frequency noise removal data and the high-frequency noise removal data) and outputs it as output image data. Output. Thereby, when the noise of the reverse code | symbol by having subtracted the noise excessively by the process to the addition part 205 generate | occur | produces, the noise can be reduced.

  The following modifications are also within the scope of the present invention, and one or a plurality of modifications can be combined with the above-described embodiment.

(Modification 1)
In the above-described embodiment, the example of calculating the low-frequency noise removal value for each pixel of interest (that is, the total number of pixels) has been described. A noise removal value may be calculated. In the case of the first modification, a difference value is calculated for discrete pixel positions, and this is up-sampled to obtain difference values for the total number of pixels.

  FIG. 9 is a block diagram illustrating a configuration of the noise removing unit 104a in the first modification. The noise removing unit 104a according to the first modification includes a functional upsampling unit 207 in addition to the same configuration as that of the above-described embodiment (FIG. 2).

  In the above-described embodiment, the example in which the reduction unit 201 generates reduced image data for each pixel of interest has been described. However, in Modification 1, one reduced image data is generated for input image data. . For example, as shown in FIG. 3, a reduced image having half the number of pixels of the input image is obtained by performing smoothing processing on each of the pixels arranged every other pixel in the input image data and arranging the output values. Generate data.

  The low frequency noise removal unit 202 calculates a low frequency noise removal value for each pixel in the reduced image data generated by the reduction unit 201. That is, a low frequency noise removal value is calculated for discrete pixel positions in the input image data. The subtraction unit 203 subtracts the original pixel value (pixel value before low-frequency noise removal) of the reduced image data from the low-frequency noise removal value calculated by the low-frequency noise removal unit 202 for each pixel of the reduced image data. By doing so, the difference value is calculated. That is, a difference value is calculated for discrete pixel positions in the input image.

  The up-sampling unit 207 up-samples the difference values for the discrete pixel positions calculated by the subtraction unit 203 (that is, performs an enlargement process) to obtain the difference values for the total number of pixels and outputs the difference values to the addition unit 205 To do. Thereafter, the same processing as in the above-described embodiment is performed.

  According to the modified example 1, by calculating the low-frequency noise removal value at discrete pixel positions, the number of low-frequency noise removal processes can be reduced as compared with the above-described embodiment. Cost reduction effect can also be obtained.

(Modification 2)
In the embodiment described above, the example in which the range limiting unit 206 clips and outputs the provisional noise removal value within the range from the low frequency noise removal value to the high frequency noise removal value has been described. It does not have to be limited. For example, the range limiting unit 206 may clip the provisional noise removal value within a range from the low frequency noise removal value to the pixel value of the input image (that is, the original pixel value). Further, noise removal processing may be performed separately on the input image, and the range restriction unit 206 may clip the provisional noise removal value within a range based on the result of the noise removal processing.

(Modification 3)
The strength of high-frequency noise removal by the high-frequency noise removal unit 204 and the strength of low-frequency noise removal by the low-frequency noise removal unit 202 may be controlled independently for each screen portion. If the high frequency noise removal is strengthened, the sharpness is often lost. For example, the high frequency noise removal may be weakened for a portion having a large high frequency edge.

(Modification 4)
In the above-described embodiment, the example in which the subtraction unit 203 obtains the difference value by subtracting the center pixel value of the reduced image from the low frequency noise removal value has been described. However, the low frequency noise removal value from the center pixel value of the reduced image is described. May be subtracted to obtain the difference value. In this case, the provisional noise removal value may be obtained by subtracting the difference value from the high frequency noise removal value.

(Modification 5)
In the embodiment described above, an example in which the digital camera 100 performs the noise removal processing has been described. However, an image processing apparatus that performs the noise removal processing may be configured by a personal computer. A personal computer may be caused to execute an image processing program for performing the noise removal processing.

  As shown in FIG. 10, the loading of the image processing program to the personal computer 300 may be performed by setting a storage medium 301 such as a CD-ROM storing the program in the personal computer 300, or a communication line such as a network. It may be loaded into the personal computer 300 by a method via 302. When passing through the communication line 302, the image processing program is stored in the hard disk device 304 of the server (computer) 303 connected to the communication line 302. The image processing program can be supplied as various types of computer program products such as provision via the storage medium 301 or the communication line 302.

  Although various embodiments and modifications have been described above, the present invention is not limited to these contents. Other embodiments conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention.

DESCRIPTION OF SYMBOLS 100 ... Digital camera, 101 ... Operation member, 102 ... Lens, 103 ... Image sensor, 104 ... Control apparatus, 104a ... Noise removal part, 105 ... Memory card slot, 106 ... Monitor, 201 ... Reduction part, 202 ... Low frequency noise Removal unit, 203 ... subtraction unit, 204 ... high frequency noise removal unit, 205 ... addition unit, 206 ... range limiting unit, 207 ... upsampling unit

Claims (7)

Image reduction means for reducing the input image data to generate reduced image data;
Low-frequency noise removal means for performing noise removal on the reduced image data to generate low-frequency noise removal data;
Difference calculating means for calculating difference data between the reduced image data and the low-frequency noise removal data;
High-frequency noise removal means for performing high-frequency noise removal data by performing noise removal on the input image data;
Output image generation means for generating output image data based on the high-frequency noise removal data and the difference data;
An image processing apparatus comprising: The image processing apparatus according to claim 1.
An image processing apparatus, further comprising range limiting means for limiting the output image data to a predetermined level range. The image processing apparatus according to claim 2,
The image processing apparatus according to claim 1, wherein the predetermined level range is a range between the low-frequency noise removal data and the high-frequency noise removal data. The image processing apparatus according to claim 2,
The image processing apparatus according to claim 1, wherein the predetermined level range is a range between the low-frequency noise removal data and the input image data. In the image processing device according to any one of claims 1 to 4,
Further comprising upsampling means for upsampling the difference data;
The low-frequency noise removal means generates low-frequency noise removal data for discrete pixel positions in the input image data,
The difference calculation means calculates the difference data for the discrete pixel positions,
The image processing apparatus, wherein the up-sampling means calculates the difference data for each pixel position of the input image data by up-sampling the difference data for the discrete pixel positions.   A digital camera comprising the image processing apparatus according to claim 1. Image reduction processing for reducing the input image data to generate reduced image data;
Low-frequency noise removal processing for performing noise removal on the reduced image data to generate low-frequency noise removal data;
Difference calculation processing for calculating difference data between the reduced image data and the low-frequency noise removal data;
High-frequency noise removal processing for performing noise removal on the input image data to generate high-frequency noise removal data;
Output image generation processing for generating output image data based on the high-frequency noise removal data and the difference data;
An image processing program for causing a computer to execute.

Images