JP2008293424A - Noise removal device, program, and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a noise removal method and a program and a device for establishing direction discrimination corresponding to a fine structure and direction discrimination which is not affected by any noise, and for reducing an arithmetic scale as for a noise removal method for combining filter processing having multiplex resolution conversion and directionality. <P>SOLUTION: This noise removal device for preparing a plurality of band image signals having different frequency bands, and for operating noise removal processing to each band image signal by performing the multiplex resolution conversion of an input image signal is provided with a direction discrimination means for discriminating the direction of the edge components of the band image signal by using a first image signal including the information of the frequency band of the band image signal as the object of noise removal and a frequency at the low band side of the frequency band and a second image signal including the information of the frequency at the low band side of the frequency band of the image signal; and a noise removal means for removing the noise of the band image signal according to the direction of the edge components. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to image processing, and more particularly to a noise removal apparatus, program, and method for removing image noise.

  In a noise removal apparatus that removes a noise signal included in an image signal, smoothing processing using a low-pass filter is widely performed. However, when noise is reduced by a smoothing process using a low-pass filter, the edge component included in the image signal is also smoothed, resulting in a problem that the sharpness of the image is lowered.

  As a method for dealing with this problem, Patent Document 1 detects the direction of edge components included in an image, and applies a smoothing process along the direction to reduce noise without reducing the sharpness of the image. The method to do is shown. Further, in Patent Document 2, the image to be referred to when detecting the direction of the edge component is not an original image but an image obtained by performing low-pass filter processing on the original image, thereby stabilizing the direction determination result. The scheme is shown.

  Further, Non-Patent Document 1 discloses a noise removal method using multi-resolution conversion. In this method, an image signal is divided into a plurality of frequency band signals using a filter bank or Laplacian pyramid method, and after some noise removal processing is performed on each band signal, these are re-synthesized. There is an advantage that noise removal processing having an intensity suitable for each of the divided bands can be performed.

  FIG. 14 shows an example of multi-resolution conversion using a Laplacian pyramid and noise removal using the same. In FIG. 14, the input image 100 is reduced after the low-pass filter processing by the filter processing / reduction processing unit 101. By this processing, the image size is halved in both the horizontal direction and the vertical direction. The reduced image is supplied to the filter processing / reduction processing unit 110 in the next stage and further reduced further. The reduced image is enlarged to the original image size by the enlargement processing unit 102, and then a difference signal is created by the subtractor 103.

  This difference signal corresponds to a high-frequency component blocked by the low-pass filter characteristic used in the filter processing / reduction processing unit 101. The NR processing unit 500 performs noise removal processing on the differential signal containing a lot of high frequency components. Similarly, in the next stage process, the filter processing / reduction processing unit 110, the enlargement processing unit 111, and the subtractor 112 generate a signal in a band corresponding to the next stage, and the NR processing unit 501 performs noise removal processing on the signal. Apply. In this way, noise removal processing is applied to the sequentially band-divided signal.

  The output of the NR processing unit 502 in the lowest frequency band is added to the low-frequency side signal by the adder 124, enlarged by the enlargement processing unit 125, and then sent to the high-frequency side processing. In this way, the signals are sequentially recombined from the low frequency side, and finally the highest frequency signal is added by the adder 109 to form an output image 503.

Patent Document 3 discloses a method in which the NR processing units 500, 501, and 502 in FIG. 14 are configured using a filter process having a directionality similar to that of Patent Document 1. In addition, for the image used for direction determination, in order to obtain an image subjected to low-pass filter processing as shown in Patent Document 2, a method for determining direction using an image on the low frequency side in multi-resolution conversion, or A method is shown that combines the result of direction determination using an image on the low frequency side and the result of direction determination using an image including a band of interest.
JP 55-133179 A JP-A-8-202870 JP 2001-57677 A Surendra Ranganath, “Image Filtering Using Multiresolution Representations”, IEEE Trans. On Pattern Anal. MachineIntell., Vol. PAMI-13 No.5, May, 1991

  However, in a noise removal apparatus that combines multi-resolution conversion and directional filter processing, if the image used for direction determination is a low-frequency image, the resolution of the low-frequency image is too low. There is a problem that the direction discrimination result corresponding to the structure cannot be obtained, and the fine structure is crushed as a result of noise removal. On the other hand, if the image used for direction determination is an image including the band of interest, the problem is that the direction determination result is strongly influenced by noise because it is an image before noise removal, and the desired noise removal result cannot be obtained. There is.

  In Patent Document 2, a reference image used for direction determination is obtained by applying a low-pass filter to an original image. By adjusting the strength of the low-pass filter, the above-mentioned problem of trade-off is dealt with. Is possible. However, in multi-resolution conversion, images of each band have already been generated, and it is desirable to use these images from the viewpoint of reducing the operation scale, and it is not desirable to separately perform low-pass filter processing with desired characteristics.

  Patent Document 3 discloses a method for dealing with the above-mentioned trade-off relationship by combining a direction determination result using a low-frequency image and a direction determination result using an image including a target band. However, it is necessary to calculate the direction determination process twice, which is not desirable from the viewpoint of reducing the operation scale.

  The present invention has been made in view of the above problems, and in a noise removal apparatus that combines multi-resolution conversion and directional filter processing, both direction determination corresponding to a fine structure and direction determination not affected by noise are compatible. In addition, an object of the present invention is to provide a noise removing device, a program, and a method capable of reducing the operation scale.

In order to solve the above problems, the present invention employs the following means.
The noise removal apparatus according to the present invention generates a plurality of band image signals having different frequency bands by performing multi-resolution conversion on an input image signal, and performs noise removal processing on each band image signal. A first image signal including information on a frequency band of a band image signal to be subjected to noise removal and a frequency on a lower frequency side than the frequency band, and a frequency band lower than the frequency band of the band image signal Direction discriminating means for discriminating the direction of the edge component of the band image signal using the second image signal including the frequency information of the side, and noise of the band image signal according to the direction of the edge component Noise removing means for removing.

Since the first image signal including the frequency band of the band image signal to be denoised and the frequency on the lower side of the frequency band includes a high frequency component, there is an edge with a fine structure. However, it is possible to determine the direction with high accuracy by using this to determine the direction, but there is a drawback that it is strongly influenced by noise included in the high frequency component. On the other hand, the second image signal including a frequency lower than the frequency band of the band image signal to be subjected to noise removal does not have a fine structure edge, but has less noise, so that stable direction determination is possible. Become.
Therefore, according to the noise removal device of the present invention, the first image signal and the second image signal are used to determine the direction of the edge component of the band image signal to be noise-removed, thereby providing a fine structure. It is possible to realize a stable direction determination that is not affected by noise while also responding to the change of.

  The noise removal apparatus according to the present invention generates a plurality of band image signals having different frequency bands by performing multi-resolution conversion on an input image signal, and performs noise removal processing on each band image signal. A removal apparatus, which uses a band image signal to be subjected to noise removal and a second image signal including information on a frequency lower than the frequency band of the band image signal. It is characterized by comprising direction discriminating means for discriminating the direction of the edge component, and noise removing means for removing the noise of the band image signal in accordance with the direction of the edge component.

The band image signal that is the target of noise removal contains high-frequency components, so there are edges with fine structure, and it is possible to determine the direction with high accuracy by using this to determine the direction, There is a drawback that it is strongly influenced by noise contained in the high frequency component. On the other hand, the second image signal including a frequency lower than the frequency band of the band image signal to be subjected to noise removal does not have a fine structure edge, but has less noise, so that stable direction determination is possible. Become.
Therefore, according to the noise removing device of the present invention, the direction of the edge component of the band image signal to be denoised is determined using the band image signal to be denominated and the second image signal. Thus, it is possible to realize a stable direction determination that is not affected by noise while also responding to changes in the fine structure.
In addition, since the first image signal is generated using the band image signal to be subjected to noise removal and the second image signal, the frequency band of the band image signal to be subjected to noise removal and the frequency band lower than the frequency band. There is no need to hold the first image signal including the frequency on the band side. Thereby, the amount of memory required for image holding can be reduced.

  A noise removal program according to the present invention includes a first process for creating a plurality of band image signals having different frequency bands by performing multi-resolution conversion on an input image signal, and noise removal for each band image signal. A noise removal program for causing a computer to execute a second process for performing a process, wherein the second process includes a frequency band of a band image signal to be subjected to noise removal and a frequency on a lower frequency side than the frequency band. The direction of the edge component of the band image signal is discriminated using the first image signal including the information of the band image and the second image signal including the information of the frequency lower than the frequency band of the band image signal. The noise of the band image signal is removed according to the direction of the edge component.

  A noise removal program according to the present invention includes a first process for creating a plurality of band image signals having different frequency bands by performing multi-resolution conversion on an input image signal, and noise removal for each band image signal. A noise removal program for causing a computer to execute a second process for performing processing, wherein the second process includes a band image signal to be subjected to noise removal, and a lower frequency side than the frequency band of the band image signal. Determining the direction of the edge component of the band image signal using the second image signal including the frequency information of the frequency, and removing the noise of the band image signal according to the direction of the edge component. Features.

  An imaging system according to the present invention includes the above-described noise removing device.

  According to the imaging system of the present invention, it is possible to obtain an image suitable for a noise level of a captured image and a user's preference.

  The noise removal method according to the present invention includes a first step of creating a plurality of band image signals having different frequency bands by performing multi-resolution conversion on an input image signal, and noise removal for each band image signal. A first image including information on a frequency band of a band image signal to be subjected to noise removal and a frequency on a lower frequency side than the frequency band. Using the signal and the second image signal including information on a frequency lower than the frequency band of the band image signal, the direction of the edge component of the band image signal is determined, and the direction of the edge component is determined. Accordingly, the noise of the band image signal is removed.

  The noise removal method according to the present invention includes a first step of creating a plurality of band image signals having different frequency bands by performing multi-resolution conversion on an input image signal, and noise removal for each band image signal. A second step of performing processing, wherein the second step includes a band image signal to be subjected to noise removal, and a second information including information on a frequency lower than the frequency band of the band image signal. The direction of the edge component of the band image signal is discriminated using the image signal, and noise of the band image signal is removed according to the direction of the edge component.

  According to the present invention, it is possible to achieve both direction determination corresponding to a fine structure and direction determination not affected by noise, and to reduce the calculation scale.

[First Embodiment]
Hereinafter, a noise removal apparatus, a program, and a method according to a first embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a functional block diagram showing the functions provided in the noise removing apparatus according to the present embodiment.
The noise removal apparatus according to the present embodiment includes a multiresolution conversion unit that performs multiresolution conversion using a Laplacian pyramid, and NR processing units 105, 113, and 121 that perform noise removal on signals in each band.

The multi-resolution conversion unit includes a plurality of stages of filter processing / reduction processing units 101, 110, 118, and enlargement processing units 102, 111, 119 provided corresponding to the filter processing / reduction processing units 101, 110, 118, and And a synthesizing unit that re-synthesizes the signal using the image after noise removal.
In such a configuration, the input image 100 is reduced by the filter processing / reduction processing unit 101 after the low pass filter processing. By this processing, the image size is halved in both the horizontal direction and the vertical direction. The reduced image is supplied to the filter processing / reduction processing unit 110 in the next stage and further reduced further. The reduced image is given to the NR processing unit 105, and noise removal processing is performed by the NR processing unit 105. In the present embodiment, the NR processing unit 105 that performs NR processing as the noise removing unit will be described as an example, but the present invention is not limited to this example.

  The image reduced by the filter processing / reduction processing unit 101 is enlarged to the original image size by the enlargement processing unit 102 and then output to the subtracter 103. In the subtracter 103, the band image signal 104 is created by subtracting the input image 100 and the image given from the enlargement processing unit 102. This band image signal 104 corresponds to a signal in a high frequency part cut off by the low-pass filter characteristic used in the filter processing / reduction processing unit 101.

  The NR processing unit 105 performs noise removal processing on the band image signal 104 including a lot of high frequency components. Similarly, in the next stage processing, a band image signal corresponding to the next stage is generated by the filter processing / reduction processing unit 110, the enlargement processing unit 111, and the subtractor 112, and the NR processing unit 113 performs noise removal processing on the band image signal. Apply. In this way, noise removal processing is applied to the sequentially band-divided signal.

  The output of the NR processing unit 121 is added to the low-frequency side signal by the adder 124 of the synthesis unit, enlarged by the enlargement processing unit 125 of the synthesis unit, and then sent to the high-frequency side processing. In this way, the signals are sequentially recombined from the low frequency side, and finally the highest frequency signal is added by the adder 109 to form an output image 126.

Next, details of the NR processing unit 105 will be described.
In the NR processing unit 105, the band image signal 104 is input, and the NR processing result 106 is output. Here, it is assumed that the NR process is performed based on a filter process having directionality, and the first image signal 107 and the second image signal 108 are used to determine the direction. Create a reference image for. The first image signal 107 includes the band image signal 104 as its frequency component, and also includes a signal on the lower frequency side. The second image signal 108 is a low-frequency signal after noise removal processing. Similarly, in the NR processing units 113 and 121, which are NR processing units in other bands, first reference images are created using the first image signals 114 and 122 and the second image signals 115 and 123, respectively. .

FIG. 2 shows the detailed contents of the NR processing unit 105.
The direction determination processing unit 200 performs direction determination using the first image signal 107 and the second image signal 108, and outputs a direction determination result 201 to the directional filter processing unit 202. The directional filter processing unit 202 performs filter processing on the band image signal 104 by determining a filter coefficient or filter processing based on the edge direction indicated by the direction determination result 201. The coring processing unit 203 performs a coring process for making the minute signal zero with respect to the filter processing result to obtain the NR processing result 106. Here, the coring process is typically such that when the absolute value of the input signal is less than or equal to the threshold value, the signal is uniformly zero, and when the input signal is greater than the threshold value, the signal is obtained by subtracting the absolute value by the threshold value. It is processing.
Note that the NR processing unit 113 and the NR processing unit 121 in other bands perform the same processing as the NR processing unit 105, and thus description thereof is omitted.

Next, details of the operation of the direction determination processing unit 200 will be described with reference to FIG.
Here, a case where a 5 × 5 pixel block around the target pixel is used when determining the direction of the edge of the target pixel will be described as an example. Note that this block size can be set to other sizes such as 7 × 7 as necessary.

A00 to A44 in FIG. 3 indicate pixels of the pixel block in the first image signal 107, and the center pixel A22 corresponds to the target pixel position. Similarly, B00 to B44 indicate pixels of the pixel block in the second image signal 108, and the center pixel B22 corresponds to the target pixel position.
In the direction discrimination processing in the present embodiment, only the central target pixel is the pixel A22 of the first image signal 107 as the pixel block of the reference image combining these pixels, and the other pixels are the second image signal 108. These pixels are used.

FIG. 4 is a diagram showing how the direction is determined from the pixel block of the reference image generated by combining as described above.
In this embodiment, the directions of the edges to be determined are four directions as shown in FIG. The number of directions can be increased to 8 directions as required. In the direction discrimination, an evaluation value is calculated for each direction in order to determine which of the directions 1 to 4 is the direction along the edge most. Although various things can be considered as an evaluation value, in the present embodiment, the following evaluation values are used.
E1 = | A22−B23 | + | A22−B24 | + | A22−B21 | + | A22−B20 |
E2 = | A22−B13 | + | A22−B04 | + | A22−B31 | + | A22−B40 |
E3 = | A22−B12 | + | A22−B02 | + | A22−B32 | + | A22−B42 |
E4 = | A22−B11 | + | A22−B00 | + | A22−B33 | + | A22−B44 |

  E1 to E4 are evaluation values corresponding to directions 1 to 4, respectively. The evaluation value increases when an edge exists in the direction to be evaluated, and decreases when there is no edge. Therefore, by detecting the minimum value from E1 to E4 and determining that there is no edge in the corresponding direction, the direction is determined as the direction along the edge.

Since the first image signal 107 includes a high-frequency component, there is an edge with a fine structure. By using this to determine the direction, it is possible to determine the direction with high accuracy. There is a drawback that it is strongly affected by the included noise. Since the second image signal 108 is an image composed of only low-frequency components from which noise has been removed, there is no edge with a fine structure, but there is little noise so that stable direction determination is possible.
In the present embodiment, only the center pixel of interest is used as the pixel of the first image signal 107, so that the direction discrimination corresponding to the change in the fine structure is realized, and the other pixels are low in noise. By obtaining from the second image signal 108, it is possible to realize stable direction determination that is not affected by noise.
In the present embodiment, only the central pixel is replaced with the pixel of the first image signal 107. However, for example, the central 3 × 3 pixel can be used as a replacement target. By adjusting this range, it is possible to adjust the trade-off relationship between highly accurate direction determination and stable direction determination resistant to noise.

As described above, according to the noise removal device according to the present embodiment, the direction of the edge component of the band image signal to be subjected to noise removal using the first image signal 107 and the second image signal 108. Can be adjusted to adjust the trade-off relationship between the first image signal 107 and the second image signal 108. As a result, it is possible to realize stable direction determination that is not affected by noise while also responding to changes in the fine structure of the input image.
Further, since the first reference image is created by an easy process of partially replacing the pixels of the first image signal 107 and the pixels of the second image signal 108, noise removal with a small amount of computation and a circuit scale is possible. Is possible. Further, since the images already generated in the multi-resolution conversion are used as the first image signal 107 and the second image signal 108, the first reference image can be generated more easily.
In addition, since the second image signal 108 is an image signal composed only of low-frequency components from which noise has been removed, more stable direction determination is possible.

In the present embodiment, the first image signal 107 is described as an image having a frequency band of a band image signal to be subjected to noise removal and a frequency on a lower frequency side than the frequency band. It may be processed by adding some filter processing.
The second image signal 108 has been described as an image having a frequency lower than the frequency band of the band image signal subjected to noise removal processing and subjected to noise removal. It is also possible to use a signal before the operation is performed. For example, instead of the second image signal 108 described above, the output of the enlargement processing unit 102 in FIG. 1 may be employed. Further, the second image signal 108 does not necessarily have to be adjacent to the band image signal to be subjected to noise removal. For example, instead of the second image signal 108 described above, the output image 123 from the enlargement processing unit 119 may be used after being enlarged to an appropriate image size.

[Second Embodiment]
Next, a second embodiment of the present invention will be described with reference to FIG.
The noise removal apparatus, program, and method according to the present embodiment are different from the first embodiment in that the first image signal and the second image signal are weighted and averaged at a predetermined ratio. This is a point for generating a reference image. Hereinafter, regarding the noise removal apparatus according to the present embodiment, description of points that are the same as those of the first embodiment will be omitted, and differences will be mainly described.

In the second embodiment, the operation of the direction determination processing unit 200 in the first embodiment is changed. This point will be described with reference to FIG.
In the first embodiment, by selecting one of the pixel block of the first image signal 107 and the pixel block of the second image signal 108 to configure a pixel block used for direction determination, The trade-off relationship between the highly accurate direction determination and the stable direction determination resistant to noise was adjusted. On the other hand, in the second embodiment, a new pixel value is calculated from these pixels, and a pixel block of a reference image used for direction determination is configured.

C00 to C44 in FIG. 5 are pixels of the reference image used for direction determination. These are the weighted average values of the pixel value of the first image signal 107 and the pixel value of the second image signal 108 as follows. (1).
Cij = wij * Aij + (1−wij) * Bij (i, j = 0,..., 4) (1)
Here, wij is a weight. When the value is close to 1, the first image signal 107 is prioritized. Therefore, the direction determination with high accuracy is prioritized. When the value is close to 0, the second image signal 108 is prioritized. Therefore, priority is given to stable direction determination resistant to noise. The weight wij can be a different value depending on the position of the pixel block. For example, by setting wij to a value close to 1 for the portion close to the central portion and close to 0 as going to the peripheral portion, it is possible to approach the configuration as in the first embodiment.

  In the present embodiment, since it is necessary to calculate a new pixel block from the first image signal 107 and the second image signal 108, there is a problem that the amount of calculation or the circuit scale increases. However, a substantial increase in the amount of calculation can be suppressed by setting the weight wij to have a value only in the portion close to the central portion and zero in the peripheral portion. Further, assuming that there are a large number of directions for direction determination, etc., a large amount of calculation is required for calculation of the evaluation value, so that the pixel block including the first image signal 107 and the second image signal 108 are included. Compared with the case where direction determination is performed for each pixel block, it is considered that the amount of calculation and the increase in circuit scale are small.

  As described above, according to the noise removal device according to the present embodiment, the weighted averaging process of the first image signal 107 and the second image signal 108 is performed, so that the first is in a trade-off relationship. The adjustment of the characteristics of the image signal 107 and the second image signal 108 can be controlled more finely. In addition, the first reference image can be more easily generated by using images already generated in the multi-resolution conversion as the first image signal 107 and the second image signal 108. Furthermore, by changing the weighting of the first image signal 107 and the second image signal 108, it is possible to perform more detailed noise removal according to the input image.

[Third Embodiment]
Next, a third embodiment of the present invention will be described with reference to FIG.
The noise removal apparatus, the program, and the method according to the present embodiment are different from the above-described embodiments in the image signal used when determining the direction of the edge component. Hereinafter, regarding the noise removal apparatus of the present embodiment, description of points that are common to the above-described embodiments will be omitted, and different points will be mainly described.

FIG. 6 is a functional block diagram showing the functions provided in the noise removing apparatus according to the present embodiment.
In the first embodiment, an image used by the NR processing unit for direction determination is a first image signal 107 including a high-frequency component and a second image signal 108 that is an image including a low-frequency component. there were. On the other hand, in the third embodiment, this is a band image signal 104 and a second image signal 108 composed of low-frequency components. Hereinafter, the purpose of such a configuration will be described.

  In the first embodiment, the band component obtained by subtracting from the first image signal 107 through the filter processing / reduction processing unit 101 and the enlargement processing unit 102 by the subtractor 103 is the band image signal 104. If the band image signal 104 is added to the low-frequency image, the first image signal 107 is obtained. The low-frequency image here is strictly different from the second image signal 108 which is a low-frequency image that has been subjected to noise removal processing by the NR processing unit in the subsequent stage, but can be approximated. it is conceivable that. Therefore, by appropriately adding the second image signal 108 and the band image signal 104, an image including a high frequency component can be generated. If direction determination is performed using this, direction determination along a fine structure can be performed. It becomes possible. Since the first image signal 107 is not required as an image signal to be input to the NR processing unit 300, there is an advantage that the amount of memory required for holding the image can be reduced.

FIG. 7 shows the detailed contents of the NR processing unit 300. Unlike the first embodiment, the direction determination processing unit 400 performs direction determination using the band image signal 104 and the second image signal 108 composed of low-frequency components. Details of this operation will be described with reference to FIG.
D00 to D44 in FIG. 8 are values of the band image signal 104, and the center value D22 corresponds to the target pixel position. Similarly, B00 to B44 indicate pixels of the pixel block in the second image signal 108, and the center pixel B22 corresponds to the target pixel position. In the direction determination process in the present embodiment, as a pixel block combining these, only the center pixel of interest is set to E22 which is a pixel value obtained by adding the band image signal 104 and the second image signal 108, and the other pixels are the first pixel block. It is assumed that pixels of the second image signal 108 are used.

As described above, since the high frequency component is included in E22, only the central target pixel is replaced with this, thereby realizing direction determination corresponding to a fine structural change, and other pixels. By obtaining from the second image signal 108 with less noise, it is possible to realize stable direction discrimination that is not affected by noise.
In the present embodiment, only the central pixel is replaced. However, for example, it is also possible to set the replacement range after performing an addition operation on the central 3 × 3 pixels. In this way, the replacement range is adjusted. Thus, it is possible to adjust the trade-off relationship in achieving both highly accurate direction determination and stable direction determination resistant to noise.

  As described above, according to the noise removal device according to the present embodiment, the noise removal processing is performed using the band image signal 104 and the second image signal 108 that are noise removal targets. Thus, there is no need to hold the first image signal 107. Thereby, the amount of memory required for image holding can be reduced.

[Fourth Embodiment]
Next, a fourth embodiment of the present invention will be described with reference to FIG.
The noise removal apparatus, the program, and the method according to the present embodiment are different from the third embodiment in that an image obtained by multiplying a band image signal to be subjected to noise removal by a predetermined ratio and the second image signal The second reference image is generated using an image obtained by adding. Hereinafter, regarding the noise removal device of the present embodiment, description of points that are common to the third embodiment will be omitted, and differences will be mainly described.

In the present embodiment, the operation of the direction determination processing unit 400 in the third embodiment is changed. This point will be described with reference to FIG.
In the present embodiment, a weighted addition of the band image signal 104 to be subjected to noise removal and the second image signal 108 is performed to constitute a pixel block used for direction determination. F00 to F44 in FIG. 9 are pixels used for direction determination. These are the following (2) as the sum of the pixel value of the second image signal 108 and the signal obtained by multiplying the band image signal 104 by a predetermined gain. ).
Fij = Bij + wij * Dij (i, j = 0, ..., 4) (2)
Here, wij is a weight, and when wij is 1, simple addition is performed as in the third embodiment, and an image including a high frequency component is used for direction determination. In the case of 0, the high frequency component is not included, and the direction is determined only from the low frequency component. When the value is larger than 1, direction determination is performed based on an image in which a high frequency component is emphasized, and direction determination is performed in consideration of a finer structure, but stability against noise is reduced. The weight wij can be a different value depending on the position in the pixel block. For example, the portion close to the central portion takes a value close to 1, and the value close to 0 as it goes to the peripheral portion makes it possible to approach the configuration as in the third embodiment.

  As described above, according to the noise removal device according to the present embodiment, it is not necessary to hold the first image signal 107 as in the third embodiment, and the band image signal 104 is in a trade-off relationship. Adjustment of the characteristics of the second image signal 108 can be controlled more finely. In addition, the second reference image can be generated more easily by using images already generated in the multi-resolution conversion as the band image signal 104 and the second image signal 108. Furthermore, by changing the weighting, it is possible to perform more detailed noise removal according to the input image.

[Fifth Embodiment]
Next, a fifth embodiment of the present invention will be described with reference to FIG.
In the present embodiment, the noise removal device described in each of the above-described embodiments is incorporated in an imaging system such as a digital camera. Hereinafter, the imaging system according to the present embodiment will be described mainly with reference to FIG. In addition, description is abbreviate | omitted about the point which is common in each above-mentioned embodiment, and a different point is mainly demonstrated.

  The imaging system according to the present embodiment includes a lens system 500, a CCD 501, an image processing unit 502, a noise removal unit 503, an image compression unit 504, a recording medium 505, a noise removal condition setting unit 506, and noise removal. A parameter setting unit 507 is provided as a main component.

  The image signal captured by the CCD 501 through the lens system 500 is subjected to white balance processing, edge enhancement processing, color signal processing, and the like in the image processing unit 502, and then noise is removed in the noise removal unit 503. The image signal from which noise has been removed is compressed into a JPEG format or the like by the image compression unit 504 and then stored in a recording medium 505 such as a memory card. The noise removing unit 503 applies the noise removing device described in each of the above embodiments.

  In removing noise, the image signal output from the image processing unit 502 may be an image composed of RGB signals, and the noise removing unit 503 may individually perform noise removal processing on the RGB image composed of three surfaces. The signal output from the processing unit 502 may be YCbCr, and the noise removal unit 503 may individually perform noise removal processing on the three-sided YCbCr image. Further, the image processing unit 502 outputs RGB signals, and after converting the RGB signals into YCbCr signals or the like in the noise removal unit 503, the noise removal processing may be individually performed on the YCbCr image composed of three surfaces. . Further, it is not necessary to separate the noise removing unit 503 and the image processing unit 502 as in the present embodiment. For example, the edge enhancement processing of the image processing unit 502 and the processing order of the noise removing unit 503 can be switched. Moreover, the noise removal part 503 has a function with which the noise removal apparatus which concerns on any one of embodiment mentioned above is equipped.

  The operation of the noise removal unit 503 is controlled by the noise removal parameter setting unit 507. In the noise removal parameter setting unit 507, a plurality of parameter candidates corresponding to the required noise removal strength are stored as noise removal parameters at the time of factory shipment. The noise removal parameter setting unit 507 determines the noise removal strength based on a setting value related to the noise removal strength of the user, ISO sensitivity information from the noise removal condition setting unit 506, and the like, and selects an appropriate parameter from the plurality of parameter candidates. And the noise removal operation of the noise removal unit 503 is controlled.

The parameters related to noise removal may include selection of a configuration related to the present invention. For example, when the noise removal method corresponding to the first embodiment is applied to the noise removal unit 503, the following parameters may be stored in the noise removal parameter setting unit 507 as candidates.
In the first embodiment, the reference image used for direction determination is generated by a combination of the first image signal 107 including a high frequency component and the second image signal 108 including a low frequency component. As a selection regarding this combination, only the position of the target pixel is the pixel of the first image signal 107, the surrounding 3 × 3 pixels of the target pixel, or all the first image signal 107 or the second image. There may be a choice as to whether the signal 108 is used. By this selection, a balance between direction determination corresponding to fine structural changes and stable direction determination not affected by noise is determined. Accordingly, as shown in FIG. 11, a plurality of reference image generation methods as described above may be set as parameter candidates, and the first reference image may be generated based on preset conditions. The noise removal parameter setting unit 507 selects an appropriate configuration from user settings, ISO sensitivity information, and the like, and controls the noise removal unit 503.

Note that the settings in each band of the multi-resolution need not be the same, and the settings in the high band and the settings in the low band may be different. For example, the condition shown in FIG. 11 may be set in advance for each band image signal, and the first reference image may be generated based on the condition.
Alternatively, the number of reductions in multi-resolution conversion may be set in advance according to the noise level, and the number of reductions in multi-resolution conversion may be determined according to the noise level of the input image signal.
Furthermore, the noise removal unit 503 may select a noise removal method according to any embodiment according to a parameter instead of selecting any one of the above-described embodiments. For example, the number of times of resolution conversion in multi-resolution conversion is changed according to imaging conditions (ISO sensitivity, etc.).

  As described above, by adopting the noise removal method according to at least one of the embodiments and configuring the imaging system, stable direction determination that is not affected by noise while also supporting changes in the fine structure of the input image. Can be realized. In addition, by appropriately changing this configuration according to user settings, ISO sensitivity information, and the like, an imaging system capable of obtaining an image suitable for the noise level of the image to be captured and the user's preference can be obtained.

  In each of the above-described embodiments, hardware processing is assumed as the noise removal device, but it is not necessary to be limited to such a configuration. For example, a configuration in which processing is performed separately by software is also possible. In this case, the noise removing device includes a main storage device such as a CPU and a RAM, and a computer-readable recording medium on which a program for realizing all or part of the above processing is recorded. Then, the CPU reads out the program recorded in the storage medium and executes information processing / calculation processing, thereby realizing the same processing as the above-described noise removing apparatus. Here, the computer-readable recording medium means a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like.

Specific processing by the above software will be described below based on the flowchart shown in FIG.
First, the input image signal is sequentially reduced by performing multi-resolution conversion, and a plurality of reduced images are generated (S1).
Next, each reduced image is enlarged to the image size before the reduction (S2), and subtraction with each image before the reduction is performed, so that each band image signal to be subjected to noise removal is created ( S3).
Next, noise removal processing is performed on each created band image signal (S4).
Then, each band image signal after noise removal is recombined, and the recombined image signal is output as an output image signal (S5).

Next, details of the noise removal processing (S4) will be described mainly with reference to FIG. Here, as an example, the noise removal processing of the band image signal 104 shown in FIG. 1 among the plurality of band image signals created in the above-described processing will be described.
First, in the noise removal process, the direction of the edge is specified by the direction determination process (S11). In this process, a first reference image is created using the first image signal 107 and the second image signal 108, and the band image signal 104 to be subjected to noise removal is generated based on the first reference image. Direction discrimination processing is performed.
Next, filter coefficients and the like are determined based on the edge direction indicated by the result of the direction determination process, and the filter process is performed on the band image signal 104 to be subjected to noise removal (S12).
Next, a coring process is performed on the band image signal that has been subjected to the filter process so that the minute signal becomes zero, and is output as a band image signal 106 from which noise has been removed (S13).

In the above example, the band image signal 104 created in the first stage of multi-resolution conversion has been described. However, the same applies to each band image signal created in the second and subsequent stages of multi-resolution conversion. Processing is performed.
Furthermore, in the above, the noise removal method corresponding to the first embodiment has been described, but other embodiments can be similarly processed by software.

It is the functional block diagram which expanded and showed the function with which the noise removal apparatus which concerns on the 1st Embodiment of this invention is provided. FIG. 2 is a functional block diagram of an NR processing unit of the noise removal device shown in FIG. 1. It is a figure explaining operation | movement of the direction discrimination | determination process part shown in FIG. It is a figure explaining operation | movement of the direction discrimination | determination process part shown in FIG. It is a figure explaining operation | movement of the direction discrimination | determination process part which concerns on the 2nd Embodiment of this invention. It is the functional block diagram which expanded and showed the function with which the noise removal apparatus which concerns on the 3rd Embodiment of this invention is provided. FIG. 7 is a functional block diagram of an NR processing unit of the noise removal device shown in FIG. 6. It is a figure explaining operation | movement of the direction discrimination | determination process part shown in FIG. It is a figure explaining operation | movement of the direction discrimination | determination process part which concerns on the 4th Embodiment of this invention. It is a schematic block diagram of the imaging system which concerns on the 5th Embodiment of this invention. It is a condition setting table | surface which shows the conditions of the production | generation method of a 1st reference image. It is a flowchart of the noise removal method which concerns on this invention. It is a flowchart of the noise removal process in FIG. It is a functional block diagram which shows the conventional noise removal apparatus.

Explanation of symbols

100 Input image 101, 110, 118 Filter processing / reduction processing unit 104 Band image signal 105, 113, 121, 300, 301, 302 NR processing unit 107, 114, 122 First image signal 108, 115, 123 Second Image signal 200, 400 Direction discrimination processing unit 202 Directional filter processing unit

Claims (15)

A noise removing device that creates a plurality of band image signals having different frequency bands by performing multi-resolution conversion on an input image signal, and performs noise removal processing on each band image signal,
A first image signal including information on a frequency band of a band image signal to be subjected to noise removal and a frequency on a lower frequency side than the frequency band, and information on a frequency on a lower frequency side than the frequency band of the band image signal Direction discriminating means for discriminating the direction of the edge component of the band image signal using the second image signal including
Noise removing means for removing noise of the band image signal according to the direction of the edge component;
A noise removing apparatus comprising: The direction discriminating means includes
The direction of the edge component of the band image signal is determined using the pixel value of the target pixel position acquired from the first image signal and the pixel value of other pixel positions acquired from the second image signal. The noise removing device according to claim 1. The direction discriminating means includes
The band image signal using a pixel value of a pixel position of interest acquired from the first image signal and a pixel position in a predetermined range around the target pixel position and a pixel value of other pixel positions acquired from the second image signal The noise removal apparatus according to claim 1, wherein the direction of the edge component of the first is determined. The direction discriminating means includes
2. The direction of an edge component of the band image signal is determined using an image signal calculated by performing a weighted average of the first image signal and the second image signal. Noise removal device. A noise removing device that creates a plurality of band image signals having different frequency bands by performing multi-resolution conversion on an input image signal, and performs noise removal processing on each band image signal,
The direction of the edge component of the band image signal is discriminated using the band image signal to be subjected to noise removal and the second image signal including information on a frequency lower than the frequency band of the band image signal. Direction discriminating means,
Noise removing means for removing noise of the band image signal according to the direction of the edge component;
A noise removing apparatus comprising: The direction discriminating means includes
The direction of the edge component of the band image signal is determined using the pixel value of the target pixel position acquired from the band image signal to be subjected to noise removal and the pixel value of other pixel positions acquired from the second image signal. The noise removing device according to claim 5, wherein the noise removing device is discriminated. The direction discriminating means includes
The pixel value of the target pixel position acquired from the band image signal that is the target of noise removal and the pixel value of the pixel position in the predetermined range around it, and the pixel value of the other pixel position acquired from the second image signal 6. The noise removing apparatus according to claim 5, wherein the direction of the edge component of the band image signal is discriminated. The direction discriminating means includes
The direction of an edge component of the band image signal is determined using an image signal calculated by performing a weighted average of the band image signal to be subjected to noise removal and the second image signal. 5. The noise removing device according to 5.   The noise removal apparatus according to claim 4 or 8, wherein the weighted average ratio is changeable.   The second image signal is an image signal generated based on a signal obtained by performing noise removal processing on a band image signal in a band lower than the frequency band of the band image signal to be subjected to noise removal. The noise removing device according to claim 1, wherein A first process of creating a plurality of band image signals having different frequency bands by performing multi-resolution conversion on the input image signal;
A noise removal program for causing a computer to execute a second process for performing a noise removal process on each band image signal,
The second processing includes a first image signal including information on a frequency band of a band image signal to be subjected to noise removal and a frequency lower than the frequency band, and a frequency band of the band image signal. Using the second image signal including the low frequency information, the direction of the edge component of the band image signal is determined, and the noise of the band image signal is removed according to the direction of the edge component. Noise removal program to perform. A first process of creating a plurality of band image signals having different frequency bands by performing multi-resolution conversion on the input image signal;
A noise removal program for causing a computer to execute a second process for performing a noise removal process on each band image signal,
The second processing uses the band image signal to be subjected to noise removal and the second image signal including information on a frequency lower than the frequency band of the band image signal. A noise removal program that discriminates the direction of the edge component and removes noise from the band image signal in accordance with the direction of the edge component.   An imaging system comprising the noise removal device according to claim 1. A first step of creating a plurality of band image signals having different frequency bands by performing multi-resolution conversion on the input image signal;
A second step of performing noise removal processing on each band image signal,
The second step includes a first image signal including information on a frequency band of a band image signal to be subjected to noise removal and a frequency on a lower frequency side than the frequency band, and a frequency band of the band image signal. The direction of the edge component of the band image signal is discriminated using the second image signal including the low frequency information, and the noise of the band image signal is removed according to the direction of the edge component. Noise removal method. A first step of creating a plurality of band image signals having different frequency bands by performing multi-resolution conversion on the input image signal;
A second step of performing noise removal processing on each band image signal,
The second step uses the band image signal to be subjected to noise removal and the second image signal including information on a frequency lower than the frequency band of the band image signal. A noise removal method for discriminating the direction of the edge component and removing the noise of the band image signal in accordance with the direction of the edge component.

Images