JP5027757B2 - Moving image denoising device, method and program thereof - Google Patents

Description

  The present invention relates to a moving image noise removing apparatus, a method thereof, and a program thereof for removing noise from a moving image signal.

In general, as a noise removal method for removing noise from a still image, a method using a nonlinear process such as a median filter, a TV (Total Variation) method, a wavelet shrinkage method, or the like is known. Yes.
For example, in the wavelet degeneration method, a noise signal level is set to “0” by a threshold function (Threshold Function) for a DWT expansion coefficient obtained by decomposing (wavelet decomposition) an image signal by a discrete wavelet transform (DWT). This is a non-linear method that performs inverse discrete wavelet transform (IDWT) after attenuation toward (see Non-Patent Document 1). When this method is used, when the image signal is concentrated on a specific DWT expansion coefficient and the absolute value of the expansion coefficient of the image signal is larger than the absolute value of the noise expansion coefficient, the noise is almost eliminated without impairing the characteristics of the original image signal. Removal can be performed.

On the other hand, as a noise removal method for moving images, a method of removing noise by motion compensation and filter processing is generally used (see Patent Documents 1 and 2).
For example, Patent Document 1 discloses a technique for removing noise from a moving image by obtaining a value for subtracting noise from the current image based on a difference between the current image and an image one frame before. Further, in Patent Document 1, when obtaining a value for subtracting noise, image shape determination is performed from the front and back pixels of the current image and the image one frame before, and when there is a correlation between the shapes, A technique for controlling to reduce the ratio of noise removal with respect to the current image is disclosed.

In addition, for example, in Patent Document 2, when a motion correction is performed on a previously input frame based on a local motion vector between frames in a moving image, a coefficient of a low-pass filter between frames is set. Techniques for controlling and reducing noise by minimizing blur in moving images are disclosed.
JP 2008-113352 A JP-A-7-288719 DL Donoho, “De-noising by soft-thresholding,” IEEE Trans. Inform.Theory, vol.41, pp.613-627, May 1995.

However, the conventional noise removal method for moving images has a problem in that, when the signal-to-noise ratio is low, the motion detection accuracy is lowered, so that the noise removal performance is naturally lowered.
In recent years, moving images have become high definition like Super Hi-Vision (SHV), and such high-definition moving images have a small light receiving area per pixel of the image sensor, so that the signal-to-noise ratio depends on the shooting conditions. It will decline. Therefore, the conventional noise removal method has a problem that noise cannot be accurately removed from a moving image. Further, such unnecessary noise components not only deteriorate the subjective image quality of the moving image but also reduce the coding efficiency.

  The present invention has been made to solve the above problems, and a moving image noise removing device capable of accurately removing noise even with a moving image signal having a low signal-to-noise ratio, An object is to provide the method and the program.

  The present invention has been developed to achieve the above object. First, the moving image noise removing apparatus according to claim 1 is a moving image noise removing apparatus for removing noise included in a moving image signal. Thus, the time direction resolving means, the space direction resolving means, the degeneration means, the space direction reconfiguring means, and the time direction reconfiguring means are provided.

  In such a configuration, the moving image noise removing apparatus converts the moving image signal into a time high frequency expansion coefficient that is a high frequency component in the time direction and a time low frequency expansion coefficient that is a low frequency component in the time direction by the time direction decomposition means. Decompose. Each expansion coefficient can be generated by performing a one-dimensional first-order discrete wavelet transform for each of a plurality of frames of the moving image signal.

  In addition, the moving image noise removing device wavelet decomposes the temporal high frequency expansion coefficient and temporal low frequency expansion coefficient into frequency components in the spatial direction by the spatial direction decomposition means. Each expansion coefficient can be generated by performing a two-dimensional first-order discrete wavelet transform on the time high-frequency expansion coefficient and the time low-frequency expansion coefficient.

  As a result, the moving image signal is decomposed into frequency components in the time direction and the spatial direction. The moving image noise removing apparatus can individually remove noise according to the noise characteristics for each frequency component.

  That is, the moving image noise removing device uses the degeneration means to perform temporal high frequency expansion coefficient wavelet decomposition in the spatial direction, temporal high frequency / spatial high frequency expansion coefficient including a high frequency component in at least one of the horizontal and vertical directions, and temporal low frequency expansion coefficient. In the time low frequency / space expansion coefficient obtained by wavelet decomposition in the spatial direction, a larger threshold is set for the time low frequency / space expansion coefficient than the time high frequency / space high frequency expansion coefficient, and the threshold is calculated by the wavelet degeneration method. The signal levels of the following frequency components are attenuated. As a result, the moving image noise removing device can remove noise by increasing the threshold value of the time low frequency component having a large signal-to-noise ratio.

  Then, the moving image noise removing apparatus is adapted to develop a temporal high frequency / spatial high frequency expansion coefficient and a temporal low frequency / spatial expansion coefficient whose signal level is attenuated by the degeneration means by the spatial direction reconstruction means, and temporal high frequency expansion by the spatial direction decomposition means. The time high frequency and spatial low frequency expansion coefficients, which are low frequency components in the spatial direction obtained by wavelet decomposition of the coefficients, are wavelet reconstructed in the spatial direction. This wavelet reconstruction in the spatial direction can be reconfigured into frequency components of a time high frequency expansion coefficient and a time low frequency expansion coefficient by performing a two-dimensional first-order discrete wavelet inverse transform on each expansion coefficient.

  Then, the moving image noise removal apparatus generates a moving image signal by reconstructing the frequency components reconstructed by the spatial direction reconstructing means in the time direction by the time direction reconstructing means. In this time-direction wavelet reconstruction, a frequency component can be reconstructed into a moving image signal by performing one-dimensional first-order discrete wavelet inverse transformation on each expansion coefficient.

  According to a second aspect of the present invention, there is provided a moving image noise removing apparatus according to the first aspect, wherein the moving image noise removing apparatus includes temporal high frequency / spatial high frequency isolated point removing means.

  In such a configuration, the moving image noise removing device binarizes the temporal high frequency / spatial high frequency expansion coefficient by the temporal high frequency / spatial high frequency isolated point removing means. In this way, by binarizing the temporal high frequency / spatial high frequency expansion coefficient, the coefficient can be handled as a binarized image. Then, the moving image noise removing device performs processing such as replacing the value of an independent pixel different from the value of the surrounding pixel in the binarized image with the value of the surrounding pixel by the temporal high frequency / spatial high frequency isolated point removing means. Then, the isolated points are removed. As a result, the moving image noise removing apparatus removes noise in the temporal high-frequency / spatial high-frequency expansion coefficient before the reduction means performs wavelet reduction.

  Furthermore, the moving image noise removing device according to claim 3 is the moving image noise removing device according to claim 1 or 2, wherein the moving image noise removing device includes temporal high frequency / spatial low frequency isolated point removing means.

  In such a configuration, the moving image noise removing apparatus binarizes the temporal high frequency / spatial low frequency expansion coefficient by the temporal high frequency / spatial low frequency isolated point removing means. In this way, by performing binarization processing on the temporal high frequency / space low frequency expansion coefficient, the coefficient can be handled as a binarized image. Then, the moving image noise removal device performs processing such as replacing the independent pixel value different from the peripheral pixel value in the binarized image with the peripheral pixel value by the temporal high frequency / spatial low frequency isolated point removing means. Thus, the isolated points are removed. Thereby, the moving image noise removing apparatus removes noise in the temporal high frequency / spatial low frequency expansion coefficient before the spatial direction reconstruction means performs wavelet reconstruction.

  According to a fourth aspect of the present invention, there is provided a moving image noise removing apparatus according to any one of the first to third aspects, comprising a maximum noise characteristic storage unit.

  In such a configuration, the moving image noise removing device has each expansion obtained by performing wavelet decomposition on the dark portion moving image signal obtained by photographing the dark portion by the camera that has photographed the original moving image of the moving image signal in the maximum noise characteristic storage unit in the time direction and the spatial direction. The maximum value of the coefficient is stored in advance as the noise characteristic of the camera. Then, the moving image noise removing device uses the threshold value calculated by the difference between the maximum value of each expansion coefficient stored in the maximum noise characteristic storage means and the absolute value of the temporal low frequency / space expansion coefficient by the degeneration means. The signal level of the temporal low frequency / space expansion coefficient is attenuated. Thereby, the moving image noise removing apparatus removes noise in the temporal low frequency / space expansion coefficient.

  Furthermore, the moving image noise removing device according to claim 5 is the moving image noise removing device according to any one of claims 1 to 4, wherein the moving image noise removing device includes a central noise characteristic storage unit.

  In such a configuration, the moving image noise removing device has a central noise characteristic storage unit that performs wavelet decomposition on a dark portion moving image signal obtained by capturing a dark portion with a camera that has captured the original moving image of the moving image signal in a time direction and a spatial direction. The median value of the coefficients is stored in advance as camera noise characteristics. Then, the moving image noise removing apparatus uses the degeneration means as a reference based on the threshold value calculated by the difference between the median of each expansion coefficient stored in the central noise characteristic storage means and the absolute value of the time high frequency / space high frequency expansion coefficient. As described above, the signal level of the time high frequency / space high frequency expansion coefficient is attenuated. As a result, the moving image noise removing apparatus removes noise in the temporal high frequency / spatial high frequency expansion coefficient.

  The moving image noise removing method according to claim 6 is a moving image noise removing method for removing noise included in a moving image signal, wherein a time direction decomposition step, a spatial direction decomposition step, a degeneration step, The procedure includes a spatial direction reconstruction step and a time direction reconstruction step.

  In this procedure, the moving image noise removing method is a time direction decomposition step in which the time direction decomposition means converts the moving image signal into a time high frequency expansion coefficient that is a high frequency component in the time direction and a time low frequency component that is a low frequency component in the time direction. Wavelet decomposition into frequency expansion coefficients. Subsequently, in the moving image noise removal method, in the spatial direction decomposition step, the temporal high frequency expansion coefficient and the temporal low frequency expansion coefficient are wavelet decomposed into frequency components in the spatial direction by the spatial direction decomposition means.

  Then, the moving image noise removing method includes a temporal high-frequency / spatial high-frequency expansion coefficient including a high-frequency component in at least one of the horizontal and vertical directions obtained by wavelet decomposition of the temporal high-frequency expansion coefficient in the spatial direction by the degeneration means in the reduction step, Wavelet degeneration method by setting a larger threshold for temporal low frequency and spatial expansion coefficient than temporal high frequency and spatial high frequency expansion coefficient in temporal low frequency and spatial expansion coefficient obtained by wavelet decomposition of low frequency expansion coefficient in spatial direction. Thus, the signal level of the frequency component below the threshold is attenuated.

  Thereafter, the moving image noise removing method includes a spatial high frequency / spatial high frequency expansion coefficient and a temporal low frequency / spatial expansion coefficient in which the signal level is attenuated by the degeneration means by the spatial direction reconstruction means in the spatial direction reconstruction step, The time high frequency / spatial low frequency expansion coefficient, which is a low frequency component in the spatial direction obtained by wavelet decomposition of the temporal high frequency expansion coefficient by the direction decomposition means, is reconstructed in the spatial direction. Subsequently, in the moving image noise removing method, in the time direction reconstruction step, the frequency component reconstructed by the spatial direction reconstruction unit is reconstructed by the time direction reconstruction unit in the time direction, thereby moving the moving image signal. Is generated.

  Furthermore, the moving image noise removal program according to claim 7 is a computer program comprising: a time direction decomposing unit, a spatial direction decomposing unit, a degenerating unit, a spatial direction reconfiguring unit, in order to remove noise included in the moving image signal; It was set as the structure which functions as a time direction reconstruction means.

  In this configuration, the moving image noise removal program converts the moving image signal into a time high frequency expansion coefficient that is a high frequency component in the time direction and a time low frequency expansion coefficient that is a low frequency component in the time direction by the time direction decomposition means. Decompose. Also, the moving image noise removal program wavelet decomposes the temporal high frequency expansion coefficient and the temporal low frequency expansion coefficient into frequency components in the spatial direction by the spatial direction decomposition means.

  Then, the moving image noise removal program uses the degeneration means to perform temporal high frequency expansion coefficient wavelet decomposition in the spatial direction, temporal high frequency / spatial high frequency expansion coefficient including a high frequency component in at least one of the horizontal and vertical directions, and temporal low frequency expansion coefficient. In the time low frequency / space expansion coefficient obtained by wavelet decomposition in the spatial direction, a larger threshold is set for the time low frequency / space expansion coefficient than the time high frequency / space high frequency expansion coefficient, and the threshold is calculated by the wavelet degeneration method. The signal levels of the following frequency components are attenuated.

  The moving image denoising program uses the temporal high frequency / spatial high frequency expansion coefficient and the temporal low frequency / spatial expansion coefficient attenuated by the degeneration means by the spatial direction reconstruction means, and the temporal high frequency expansion by the spatial direction decomposition means. The time high frequency and spatial low frequency expansion coefficients, which are low frequency components in the spatial direction obtained by wavelet decomposition of the coefficients, are wavelet reconstructed in the spatial direction. Further, the moving image noise removal program generates a moving image signal by reconstructing the frequency components reconstructed by the spatial direction reconstructing means in the time direction by the time direction reconstructing means.

The present invention has the following excellent effects.
According to the first, sixth, and seventh aspects of the present invention, noise can be removed according to the characteristics of each frequency component obtained by wavelet decomposition of the moving image signal in the time direction and the spatial direction. Even a moving image with a low ratio can accurately remove noise. Further, according to the first, sixth, and seventh aspects of the present invention, the noise component in the frequency component of the temporal high frequency and the spatial high frequency of the moving image signal is further removed, so that the frequency component is not degraded without degrading the original signal. It is possible to remove the thermal noise of the camera and the minute camera shake at the time of shooting.

  According to the second and third aspects of the present invention, it is possible to remove noise components from temporal high frequency / spatial high frequency components and temporal high frequency / spatial low frequency components of moving image signals. Further, according to the second aspect of the present invention, since the noise component of the isolated point is removed before the wavelet degeneration in the degeneration means is performed, the degeneration means can efficiently perform noise removal.

  According to the fourth and fifth aspects of the present invention, the temporal low frequency component has a higher signal-to-noise ratio than the temporal high-frequency component, depending on the characteristic that the signal-to-noise ratio of the temporal moving signal is biased in a direction larger than the signal-to-noise ratio of the original moving image signal. In addition, since a larger threshold is set for the time low-frequency component and wavelet degeneration is performed, noise can be accurately removed.

[Outline of video noise removal method]
First, an outline of a moving image noise removal method (hereinafter, this method) in the present invention will be described. This method extends the wavelet reduction method in the time direction.

  When the moving image signal is subjected to discrete wavelet decomposition in the time direction and the spatial direction, the temporal low frequency component has a high signal correlation in the time direction, and therefore, the signal-to-noise ratio is biased in a direction larger than that of the original moving image signal.

  On the other hand, the temporal and spatial high-frequency components are considered to be almost noise components from the three-dimensional spectrum. Here, in the frame constituting the moving image signal, when the original image at time t in the xy coordinate system is s (x, y, t) and the noise component is n (x, y, t), noise is included. The moving image signal F (x, y, t) can be expressed by the following equation (1).

  Since the noise component n (x, y, t) is a thermal noise component of the camera that has captured the moving image, it has whiteness and Gaussianity. That is, the noise component n (x, y, t) has the characteristics that the occurrence probability follows a normal distribution, has substantially the same energy distribution in all frequency bands, and has a low correlation in the time direction and the spatial direction. .

Therefore, this method increases the threshold in the wavelet reduction method for temporal low-frequency components obtained by discrete wavelet decomposition of moving image signals in the time direction and the spatial direction, and the threshold for wavelet reduction methods for temporal high-frequency and spatial high-frequency components. The noise of the moving image signal is removed by reducing.
Hereinafter, the best embodiment for realizing the present technique will be described with reference to the drawings.

[Configuration of moving image noise elimination device]
First, with reference to FIG. 1, the structure of the moving image noise removal apparatus which concerns on embodiment of this invention is demonstrated. FIG. 1 is a block configuration diagram showing the configuration of a moving image noise removing apparatus according to an embodiment of the present invention.

  This moving image noise removing device 1 applies discrete wavelet decomposition to a moving image signal in the time direction and the spatial direction, and performs wavelet degeneration in the time direction and the spatial direction, thereby removing noise included in the moving image signal. It is. Here, the moving image noise removing apparatus 1 includes a time / space wavelet decomposition unit 10, a noise characteristic storage unit 20, a time / space wavelet degeneration unit 30, and a time / space wavelet reconstruction unit 40. .

  The time / space wavelet decomposition means 10 inputs a moving image signal from the outside and performs wavelet decomposition in the time direction and the space direction. This moving image signal can be a high-definition moving image signal such as Super Hi-Vision (SHV). Here, the time / space wavelet decomposition means 10 includes a time direction decomposition means 11 and a space direction decomposition means 12.

The time direction decomposition unit 11 performs wavelet decomposition on the moving image signal in the time direction. This time direction decomposition means 11 inputs a moving image signal in units of frames corresponding to the length of the wavelet coefficient in synchronization with a clock output from a clock circuit (not shown), and 1 in all horizontal and vertical coordinates of the frame. By performing the first-order discrete wavelet transform (DWT), a temporal low-frequency expansion coefficient (temporal low-frequency component) L ¹ _{T that} is a low-frequency component in the time direction and a temporal high-frequency expansion coefficient that is a high-frequency component in the time direction ( Wavelet decomposition into temporal high-frequency components) H ¹ _T.

That is, as shown in FIG. 2, the time direction decomposing means 11 converts all the frame images F (t), F (t + 1), F (t + 2), F (t + 3),. By performing wavelet transform on the pixel value of (x, y), the wavelet expansion coefficient is decomposed (wavelet decomposition) into the time low frequency expansion coefficient L ¹ _T and the time high frequency expansion coefficient H ¹ _T.

The wavelet coefficient and its length vary depending on the wavelet used. For example, when a Daubechies secondary wavelet is used, four coefficients (wavelet coefficient length = “4”) of [−0.1294 0.2241 0.8365 0.4830] can be used as wavelet coefficients. This wavelet coefficient length is the number of frames in the time direction.
The time direction decomposition means 11 outputs the time low frequency expansion coefficient L ¹ _T and the time high frequency expansion coefficient H ¹ _T subjected to wavelet decomposition to the spatial direction decomposition means 12.

The spatial direction decomposition means 12 performs wavelet decomposition on the temporal low frequency expansion coefficient L ¹ _T and temporal high frequency expansion coefficient H ¹ _T decomposed by the time direction decomposition means 11 into frequency components in the spatial direction. Here, the spatial direction decomposition unit 12 includes a temporal low frequency decomposition unit 12a and a temporal high frequency decomposition unit 12b.

The temporal low frequency decomposition means 12a performs wavelet decomposition on the temporal low frequency expansion coefficient L ¹ _T decomposed by the time direction decomposition means 11 in the spatial direction. This time low frequency decomposition means 12a performs two-dimensional first-order DWT on the time low frequency expansion coefficient L ¹ _T , thereby further performing wavelet decomposition on the time low frequency component in the spatial direction (L ¹ _{T ^{LL 1 S, L 1 T LH}} 1 S, L 1 T HL 1 S, generates the L ₁ T ^HH 1 _S). The generated wavelet expansion coefficients (L ¹ _T LL ¹ _S , L ¹ _T LH ¹ _S , L ¹ _T HL ¹ _S , L ¹ _T HH ¹ _S ) are output to the time / space wavelet degeneration means 30.

The temporal high-frequency decomposition means 12b performs wavelet decomposition on the temporal high-frequency expansion coefficient H ¹ _T decomposed by the time-direction decomposition means 11 in the spatial direction. This time high frequency decomposition means 12b performs a two-dimensional first-order DWT on the time high frequency expansion coefficient H ¹ _T , thereby further performing wavelet decomposition on the time high frequency component in the spatial direction (H ¹ _T LL ¹ _{S ^{, H 1 T LH 1 S,}} H 1 T HL 1 S, generates the H ₁ T ^HH 1 _S). The generated wavelet expansion coefficients (H ¹ _T LL ¹ _S , H ¹ _T LH ¹ _S , H ¹ _T HL ¹ _S , H ¹ _T HH ¹ _S ) are output to the time / space wavelet degeneration means 30.

That is, as shown in FIG. 3 (a), the spatial direction decomposition means 12 uses a temporal low frequency expansion coefficient L ¹ _T obtained by wavelet decomposition of a moving image signal in the time direction as a two-dimensional image, 4 divided wavelet expansion coefficients to the frequency components in the vertical direction _{^{(L 1 T LL 1 S,}} L 1 T LH 1 S, L 1 T HL 1 S, L 1 T HH 1 S) for generating a. Further, as shown in FIG. 3 (b), the spatial direction decomposition means 12 takes the time and high frequency expansion coefficient H ¹ _T obtained by wavelet decomposition of the moving image signal in the time direction as a two-dimensional image, and horizontally and vertically displays the image. Wavelet expansion coefficients (H ¹ _T LL ¹ _S , H ¹ _T LH ¹ _S , H ¹ _T HL ¹ _S , H ¹ _T HH ¹ _S ) divided into four frequency components in the direction are generated.

The wavelet expansion coefficient generated by the time / space wavelet decomposition means 10 can be represented as a region diagram of a three-dimensional spectrum as shown in FIG. FIG. 4 is a time-space first-order DWT region diagram of a moving image signal, in which a horizontal frequency μ (horizontal maximum frequency μ _max ) in the spatial direction, a vertical frequency ν (vertical maximum frequency ν _max ), and a time frequency in the time direction. f (time maximum frequency f _max ) is taken as three axes.

Here, with reference to FIG. 5, the characteristics of the wavelet expansion coefficient subjected to wavelet decomposition in the time direction and the space direction will be described. FIG. 5 is a diagram in which the spectrum plane of the moving speed of the moving object in the moving image signal is associated with the time-space first floor DWT region diagram of FIG. Note that FIG. 5 (a) shows a case in which x _v = 1, y _v = at the inter-frame horizontal / vertical moving speed (x _v , y _v ) ∈m _v of the moving object of the pixel F (x, y) in the frame. In FIG. 5 (b), the spectrum plane at x _v = 2 and y _v = 2 is associated with the time-space first-order DWT region diagram.

[Characteristics of temporal low frequency component (temporal low frequency expansion coefficient L ¹ _T )]
First, the characteristics of the temporal low frequency expansion coefficient L ¹ _T , which is the temporal low frequency component of the moving image signal, will be described.

Time low frequency expansion coefficients ^L ₁ wavelet expansion coefficient wavelet decomposed into spatial direction _{^{T L 1 T HH 1 S,}} L 1 T LH 1 S, L 1 T HL 1 S is the _{x v ≦ 1, y v ≦} 1 Of the original image s component (see the above formula (1)), it is a temporal low-frequency / spatial high-frequency component that is greater than μ _max / 2 and v _max / 2 with _respect to the maximum horizontal and vertical frequencies μ _max and v _max . The wavelet expansion coefficient L ¹ _T LL ¹ _S is the maximum horizontal and vertical frequency μ _max , ν _max among the original image s components of x _v > 1, y _v > 1 (see the above equation (1)). , Μ _max / 2, less than ν _max / 2, a temporal low frequency / spatial low frequency component.

The time low frequency / spatial high frequency components (L ¹ _T HH ¹ _S , L ¹ _T LH ¹ _S , L ¹ _T HL ¹ _S ), and the time low frequency / spatial low frequency component (L ¹ _T LL ¹ _S ) are The correlation of the original image s component with respect to the time direction has a very high characteristic as compared with the noise component n (see the above equation (1)).

  That is, in the moving image signal F, the absolute value difference between the original image s component and the noise component n of the wavelet expansion coefficient of the temporal low frequency / spatial high frequency component and temporal low frequency / spatial low frequency component is the original image s component and noise. It is larger than the absolute value difference with the component n. That is, the signal-to-noise ratio of the temporal low-frequency component has a characteristic that it is biased in a direction larger than the signal-to-noise ratio of the original moving image signal.

  In order to remove noise from a moving image signal, it is possible to remove noise more accurately by setting a larger threshold value for temporal low frequency components than for temporal high frequency components and applying the wavelet degeneration method. It means you can do it.

[Characteristics of time high frequency component (time high frequency expansion coefficient H ¹ _T )]
Next, the characteristics of the temporal high-frequency expansion coefficient H ¹ _T , which is the temporal high-frequency component of the moving image signal, will be described.

Wavelet expansion coefficients H ¹ _T HH ¹ _S , H ¹ _T LH ¹ _S , and H ¹ _T HL ¹ _{S obtained} by wavelet decomposition of the time high-frequency expansion coefficient H ¹ _T in the spatial direction are elements of x _v ≦ 2, y _v ≦ 2. Of the image s component (see the above formula (1)), it is a time high-frequency / spatial high-frequency component of μ _max / 2 or v _max / 2 or more with _respect to the horizontal and vertical maximum frequencies μ _max and v _max .
These temporal high-frequency and spatial high-frequency components (H ¹ _T HH ¹ _S , H ¹ _T LH ¹ _S , and H ¹ _T HL ¹ _S ) are small in thermal noise and photographing at the time of photographing except for the minute moving component of each direction edge component. Possible camera shake.

The wavelet expansion coefficient H ¹ _T LL ¹ _S is the maximum horizontal and vertical frequency μ _max and ν _max among the original image s components of x _v > 2 and y _v > 2 (see the above equation (1)). , Μ _max / 2, less than ν _max / 2, the time high frequency / spatial low frequency component.

This temporal high-frequency / spatial low-frequency component (H ¹ _T LL ¹ _S ) is considered to be a moving region in which the foreground and background textures are greatly different, and has a characteristic that the correlation of the original image s component with respect to the time direction is low. In particular, when the luminance of the moving area of the original image s component is low and the luminance difference between the moving area and the background is large, in the moving image signal F, the original image s component and noise component of the wavelet expansion coefficient of the temporal high frequency / spatial low frequency component The absolute value difference from n is small. That is, since the temporal high frequency component is almost a noise component other than the minute motion component, it means that the noise can be accurately removed by the isolated point noise removal method.

  Based on the characteristics of the temporal low-frequency component and temporal high-frequency component as described above, the moving image noise removing apparatus 1 removes noise from the moving image signal. Returning to FIG. 1, a configuration in which the moving image noise removing apparatus 1 removes noise will be described.

  The noise characteristic storage means (maximum noise characteristic storage means, central noise characteristic storage means) 20 wavelet-decomposes the dark part moving image signal obtained by photographing the dark part by the camera that has photographed the original moving picture of the moving picture signal in the time direction and the spatial direction. The maximum value and median value of each expansion coefficient are stored in advance as camera noise characteristics, and are general storage devices such as hard disks.

In this noise characteristic storage means 20, as shown in FIG. 6, wavelet expansion in which the moving image signal N (x, y, t) photographed by the camera C is subjected to wavelet decomposition (time-space DWT) in the time direction and the space direction. Coefficients L ¹ _T LL ¹ _S , L ¹ _T LH ¹ _S , L ¹ _T HL ¹ _S , L ¹ _T HH ¹ _S , H ¹ _T LL ¹ _S , H ¹ _T LH ¹ _S , H ¹ _T HL ¹ _S The maximum value and the median value for each of H ¹ _T HH ¹ _S are stored.

The time-space DWT performed when obtaining the noise characteristics of the camera stored in the noise characteristic storage means 20 may be performed by the time / space wavelet decomposition means 10 or decomposed by the time / space wavelet decomposition means 10. Alternatively, the maximum value and the median value may be written into the noise characteristic storage unit 20 via a noise characteristic generation unit (not shown) that obtains the maximum value and the median value from each wavelet expansion coefficient.
The noise characteristic (maximum value, median value) stored in the noise characteristic storage unit 20 is referred to by the time / space wavelet degeneration unit 30.

  The time / space wavelet reduction means 30 attenuates each wavelet expansion coefficient decomposed by the time / space wavelet decomposition means 10 toward the signal level of “0” by the wavelet reduction method. Here, the time / space wavelet reduction means 30 includes a temporal high frequency isolated point removal means 31 and a reduction means 32.

The temporal high frequency isolated point removing means 31 is a wavelet expansion coefficient (H ¹ _T LL ¹ _S , H ¹ _T LH ¹ _S) which is a high frequency component in the time direction decomposed by the time high frequency decomposition means 12 b of the time / space wavelet decomposition means 10. ^{, H} ₁ T ^HL ₁ S, for _{H 1} T ^HH 1 _S), is to remove the isolated point. Here, the temporal high frequency isolated point removing unit 31 includes a spatial high frequency isolated point removing unit 31a and a spatial low frequency isolated point removing unit 31b.

Spatial high frequency isolated point removing means (temporal high frequency / spatial high frequency isolated point removing means) 31a is arranged in at least one of the horizontal and vertical directions among the wavelet expansion coefficients decomposed by the temporal high frequency decomposition means 12b of the spatial direction decomposition means 12. The isolated point is removed from the temporal high frequency / spatial high frequency expansion coefficient H ¹ _T HH ¹ _S , H ¹ _T LH ¹ _S , and H ¹ _T HL ¹ _S including high frequency components.

Here, the spatial high-frequency isolated point removing unit 31a converts the temporal high-frequency / spatial high-frequency expansion coefficients H ¹ _T HH ¹ _S , H ¹ _T LH ¹ _S , and H ¹ _T HL ¹ _S into a two-dimensional image (for example, | H ¹ _It is assumed that _T HH ¹ _S (x, y) |), and binarization processing is performed using a predetermined threshold value, and isolated points on the binarized image B (x, y) are removed.

For example, as shown in FIG. 7, the spatial high frequency isolated point removing means 31a generates a binarized image B (x, y) from H ¹ _T HH ¹ _S of the temporal high frequency and spatial high frequency expansion coefficients. Then, the spatial high-frequency isolated point removing unit 31a determines that the total pixel value B _{3 × 3} (x, y) of a predetermined region (for example, 3 × 3 pixel region) including the peripheral pixels of the target pixel (x, y) is in advance. If it is equal to or less than the predetermined threshold value θ _{H **} , the target pixel is regarded as isolated point noise, and | H ¹ _T HH ¹ _S (x, y) | = 0. This threshold value θ _{H **} is set to, for example, θ _{H **} = 3 in order to prevent disappearance of the edge component.

Incidentally, the spatial high-frequency isolated point removing unit 31a are other times high frequency and high spatial frequency expansion coefficient ^H ₁ T ^LH ₁ ^S, for even _{H 1} T ^HL 1 ^_S, by the same processing as _{H 1} T ^HH 1 _S isolated points Remove.
The time isolated point has been removed high-frequency and high spatial frequency expansion coefficient _{^{H 1 T HH 1 S, H}} 1 T LH 1 S, H 1 T HL 1 S is output to the degeneration means 32.

The spatial low frequency isolated point removing means (temporal high frequency / spatial low frequency isolated point removing means) 31b is a temporal high frequency / spatial low frequency among the wavelet expansion coefficients decomposed by the temporal high frequency decomposition means 12b of the spatial direction decomposition means 12. An isolated point is removed from the expansion coefficient H ¹ _T LL ¹ _S.

This spatial low-frequency isolated point removing means 31b binarizes the time high-frequency / spatial low-frequency expansion coefficient H ¹ _T LL ¹ _S as described in FIG. 7 in the same manner as the spatial high-frequency isolated point removing means 31a. Then, isolated points on the binarized image B (x, y) are removed.

That is, the spatial low-frequency isolated point removing unit 31b generates a binary image B (x, y) from the temporal high-frequency / spatial low-frequency expansion coefficient H ¹ _T LL ¹ _S. Then, the spatial low-frequency isolated point removing unit 31b calculates that the total pixel value B _{3 × 3} (x, y) of a predetermined region (for example, 3 × 3 pixel region) including the peripheral pixels of the target pixel (x, y) is If it is less than or equal to a predetermined threshold value θ _HLL , the target pixel is regarded as isolated point noise, and | H ¹ _T LL ¹ _S (x, y) | = 0. The threshold θ _HLL is set to a value larger than θ _{H **} , for example, θ _HLL = 4, because H ¹ _T LL ¹ _S is a spatial low frequency component.
The temporal high-frequency / spatial low-frequency expansion coefficient H ¹ _T LL ¹ _S from which the isolated points are removed is output to the temporal / spatial wavelet reconstruction unit 40.

Degeneration means 32, the time low-frequency time wavelet decomposed expansion coefficients in the spatial direction low-frequency and spatial expansion coefficients ^{_{L 1 T LL 1 S, L}} 1 T LH 1 S, L 1 T HL 1 S, L 1 T HH 1 _S and, on at least one time high-frequency and high spatial frequency expansion coefficient ^H ₁ T ^LH ₁ S containing high frequency components in the horizontal or vertical direction which is wavelet decomposition in the spatial direction of the time frequency expansion ^{_{coefficient, H 1 T HL 1 S,}} H 1 T In HH ¹ _S , a larger threshold is set for the time low frequency / space expansion coefficient than the time high frequency / space high frequency expansion coefficient by the wavelet reduction method, and the signal level of the frequency component below the threshold is attenuated. It is. Here, the degeneration means 32 includes a time low frequency degeneration means 32a and a time high frequency degeneration means 32b.

The temporal low frequency degeneration means 32a is a wavelet expansion coefficient decomposed by the temporal low frequency decomposition means 12a of the spatial direction decomposition means 12 (temporal low frequency / space expansion coefficients L ¹ _T LL ¹ _S , L ¹ _T LH ¹ _S , L ¹ _T HL ¹ _S, for ^L ₁ T ^HH _{1 S),} sets the threshold value function, in which attenuates the "0" to the following frequency components threshold calculated in the threshold function.

This time-low frequency degeneration means 32a performs wavelet degeneration using the maximum value of the wavelet expansion coefficient of the corresponding frequency component among the noise characteristics stored in the noise characteristic storage means 20. That is, as shown in FIG. 8, the temporal low frequency degeneration means 32a performs temporal low frequency / space expansion coefficients L ¹ _T LL ¹ _S , L ¹ _T LH ¹ _S , L ^{1 obtained} by wavelet decomposition of the moving image signal captured in the dark part. _{^{T HL 1 S, L 1 T}} HH 1 respective maximum value _n n of _{^{S (L 1 T LL 1 S}} ), n n (L 1 T LH 1 S), n n (L 1 T HL 1 S), n Wavelet degeneration is performed using _N (L ¹ _T HH ¹ _S ).

Specifically, the temporal low frequency degeneration means 32a sets the maximum value of the wavelet expansion coefficient stored in the noise characteristic storage means 20 to n _N (*), and sets the wavelet coefficient length when the discrete wavelet decomposition is performed to w _l. Then, the threshold function shown in the following equation (2) is set, and the frequency component is expressed by “Soft-Threshold” processing using the threshold Th (i, n _N (*)) calculated by the function. Perform wavelet degeneration to attenuate to 0 ".

  Here, sgn (i) is a function that returns the sign of i.

  As described above, the temporal low-frequency degeneration means 32a has the characteristics of the above-described temporal low-frequency component, that is, the wavelet expansion coefficient of the temporal low-frequency / spatial high-frequency component and temporal low-frequency / spatial low-frequency component in the moving image signal F. Since the absolute value difference between the original image s component and the noise component n is larger than the absolute value difference between the original image s component and the noise component n, the noise component is determined by setting the maximum value of the noise component as a threshold value. It can be removed with high accuracy.

The time high frequency degeneration means 32b is subjected to wavelet decomposition by the spatial direction decomposition means 12, and further, a wavelet expansion coefficient (time) including a high frequency component in at least one of the horizontal and vertical directions from which the isolated points are removed by the time high frequency isolated point removal means 31. high-frequency and high spatial frequency expansion coefficient ^{_{H 1 T LH 1 S, H}} 1 T HL 1 S, for H ₁ T ^HH 1 _S), sets the threshold function, a threshold value below the frequency components calculated in the threshold function "0" Is to be attenuated.

Similar to the time low frequency degeneration means 32a, the time high frequency degeneration means 32b sets the threshold function shown in the equation (2), and sets the frequency component to “0” by the threshold Th (i, n _N (*)). Perform wavelet degeneration to attenuate. However, the time high-frequency degeneration means 32b uses, as n _N (*), the maximum value of the wavelet expansion coefficient obtained by performing discrete wavelet transform on the moving image N (x, y, t) captured by the camera in the time direction and the space direction. Instead, the median value of the wavelet expansion coefficients stored in the noise characteristic storage means 20 is used. That is, as shown in FIG. 9, the temporal high frequency degeneration means 32b performs temporal high frequency / spatial high frequency expansion coefficients H ¹ _T LH ¹ _S , H ¹ _T HL ¹ _S , H ¹ _{T obtained} by wavelet decomposition of the moving image signal captured in the dark part. HH ¹ _S of the respective median ^{_{n n (H 1 T LH 1}} S), n n (H 1 T HL 1 S), using ^{_{n n (H 1 T HH 1}} S), performs the wavelet shrinkage.

  As described above, the temporal high-frequency degeneration means 32b has the characteristics of the temporal high-frequency component, that is, the absolute value of the original image s component and the noise component n of the wavelet expansion coefficient of the temporal high-frequency and spatial low-frequency components in the moving image signal F. Because of the small value difference, noise is eliminated by setting a threshold value (median value) smaller than the threshold value (maximum value) set for the time low frequency component. Can be removed.

In the degeneration means 32, the wavelet degeneration is not performed for the time high frequency / space low frequency expansion coefficient H ¹ _T LL ¹ _S. This is because if the degeneration processing is performed on the temporal high-frequency and spatial low-frequency components, image disturbance (artifact) is likely to occur near the boundary between the moving region and the background.

The degenerate means 32 in wavelet shrinkage wavelet expansion coefficients ^{_{L 1 T LL 1 S, L}} 1 T LH 1 S, L 1 T HL 1 S, L 1 T HH 1 S, H 1 T LH 1 S, H 1 T HL ¹ _S and H ¹ _T HH ¹ _S are output to the time / space wavelet reconstruction means 40.

  The temporal / spatial wavelet reconstruction unit 40 generates a moving image signal by reconstructing the wavelet expansion coefficient obtained by attenuating the noise signal level by the temporal / spatial wavelet degeneration unit 30 in the spatial direction and the temporal direction, respectively. Is. Here, the time / space wavelet reconstruction unit 40 includes a space direction reconstruction unit 41 and a time direction reconstruction unit 42.

The spatial direction reconstruction means 41 includes wavelet expansion coefficients L ¹ _T LL ¹ _S , L ¹ _T LH ¹ _S , L ¹ _T HL ¹ _S , L ¹ _T HH ¹ _S , H ¹ _T LL ¹ _S , H ¹ _T LH _{^{1 S, H 1 T HL 1}} S, the _{H 1} T ^HH 1 _S, is intended to wavelet reconstruction the spatial direction.

Here, the spatial direction reconstruction means 41 includes the time high frequency / spatial low frequency expansion coefficient H ¹ _T LL ¹ _{S from} which the isolated points have been removed by the time high frequency isolated point removal means 31, and the time that the wavelet has been degenerated by the reduction means 32. high-frequency and high spatial frequency expansion coefficient _{^{H 1 T LH 1 S, H}} 1 T HL 1 S, H 1 T HH 1 S, and the time the low-frequency and spatial expansion coefficients _{^{L 1 T LL 1 S, L}} 1 T LH 1 S, Two-dimensional first-order discrete wavelet inverse transform (IDWT) is performed on L ¹ _T HL ¹ _S and L ¹ _T HH ¹ _S. Accordingly, the wavelet expansion coefficients L ¹ _T LL ¹ _S , L ¹ _T LH ¹ _S , L ¹ _T HL ¹ _S , L ¹ _T HH ¹ _S , H ¹ _T LL ¹ _S , H ¹ _T LH ¹ _S , H ¹ _T HL ¹ _S and H ¹ _T HH ¹ _S are reconfigured into a temporal high frequency expansion coefficient H ¹ _T and a temporal low frequency expansion coefficient L ¹ _T.

The time direction reconstruction unit 42 reconstructs the frequency component (wavelet expansion coefficient) reconstructed by the spatial direction reconstruction unit 41 in the time direction. Here, the time direction reconstruction unit 42 calculates the one-dimensional first floor IDWT for the time high frequency expansion coefficient H ¹ _T and the time low frequency expansion coefficient L ¹ _T reconstructed by the spatial direction reconstruction unit 41. By doing so, a moving image signal is generated.

  By configuring the moving image noise removing apparatus 1 as described above, the moving image noise removing apparatus 1 is configured such that the signal to noise ratio of the temporal low frequency component of the moving image signal is equal to the signal to noise ratio of the original moving image signal. It is possible to remove noise by wavelet degeneracy using the characteristic of being biased in a larger direction. Furthermore, the moving image noise removing apparatus 1 removes noise by applying isolated point noise using a characteristic that the temporal high-frequency component of the moving image signal is almost a noise component other than a minute moving component. be able to.

Although the best embodiment of the present invention has been described above, the moving image noise removal apparatus 1 may have a simple configuration in which the time high frequency isolated point removal unit 31 is omitted from the configuration. In this case, the time frequency decomposition unit 12b is degraded time high-frequency and high spatial frequency expansion coefficient ^{_{H 1 T LH 1 S, H}} 1 T HL 1 S, H 1 T the ^HH _{1 S,} high temporal frequency degeneracy unit 32b of the degeneration means 32 Will be output. Further, the temporal high frequency decomposition means 12 b outputs the decomposed temporal high frequency / spatial low frequency expansion coefficient H ¹ _T LL ¹ _S to the spatial direction reconstruction means 41 of the temporal / space wavelet reconstruction means 40. As a result, the moving image noise removing apparatus 1 can effectively remove the noise of the moving image signal by applying the wavelet degeneration in the time direction.

  The moving image noise removing apparatus 1 can be realized by a general computer having a CPU and a memory (not shown). At this time, the moving image noise removing apparatus 1 can operate the computer by a moving image noise removing program that causes the computer to function as each of the above-described means. This moving image noise elimination program can be distributed via a network, or can be written and distributed on a recording medium such as a CD-ROM.

[Operation of moving image noise elimination device]
Next, the operation (moving image noise removing method) of the moving image noise removing apparatus according to the embodiment of the present invention will be described with reference to FIG. FIG. 10 is a flowchart showing the operation of the moving image noise removing apparatus according to the embodiment of the present invention.

[Time direction decomposition step]
First, the moving image noise removal apparatus 1 inputs a moving image signal in units of frames in synchronization with a clock (CLK) by the time direction decomposition unit 11 of the time / space wavelet decomposition unit 10 and performs one-dimensional first-order discrete wavelet transform. (DWT) is performed (step S1). Thus, the moving image signal is subjected to wavelet decomposition in the time direction, and is decomposed into a temporal high frequency expansion coefficient H ¹ _T and a temporal low frequency expansion coefficient L ¹ _T.

[Space direction decomposition step]
Then, the moving image noise removing apparatus 1 performs a two-dimensional first-order wavelet transform on the temporal high-frequency expansion coefficient H ¹ _T and temporal low-frequency expansion coefficient L ¹ _T decomposed in step S 1 by the spatial direction decomposition means 12. Do.

That is, the moving image noise removing apparatus 1 performs two-dimensional first-order DWT on the time high frequency expansion coefficient H ¹ _T decomposed in step S1 by the time high frequency decomposition means 12b, thereby further reducing the time high frequency components. Wavelet expansion coefficients (H ¹ _T LL ¹ _S , H ¹ _T LH ¹ _S , H ¹ _T HL ¹ _S , H ¹ _T HH ¹ _S ) generated by wavelet decomposition in the direction are generated (step S2).

Further, the moving image noise removing apparatus 1 performs a two-dimensional first-order DWT on the temporal low frequency expansion coefficient L ¹ _T decomposed in step S1 by the temporal low frequency decomposition means 12a, thereby generating a temporal low frequency component. further wavelet expansion coefficients wavelet decomposition in the spatial direction _{^{(L 1 T LL 1 S,}} L 1 T LH 1 S, L 1 T HL 1 S, L 1 T HH 1 S) generates a (step S3).
Note that steps S2 and S3 may be operated in the reverse order or may be operated in parallel.

[Degeneration step]
Then, the moving image noise removal apparatus 1 performs a reduction process by the wavelet reduction method on the wavelet expansion coefficients wavelet decomposed in step S2 and step S3 by the time / space wavelet reduction means 30.

Here, first, the moving image noise removing apparatus 1 uses at least one of the horizontal and vertical directions among the wavelet expansion coefficients decomposed in step S2 by the spatial high frequency isolated point removing unit 31a of the temporal high frequency isolated point removing unit 31. The isolated points are removed from the temporal high frequency / spatial high frequency expansion coefficients H ¹ _T HH ¹ _S , H ¹ _T LH ¹ _S , and H ¹ _T HL ¹ _S including the high frequency component in (step S4). Further, the moving image noise removing apparatus 1 includes the temporal high frequency / spatial low frequency expansion coefficient H among the wavelet expansion coefficients decomposed in step S2 by the spatial low frequency isolated point removing means 31b of the temporal high frequency isolated point removing means 31. ^An isolated point is removed from ¹ _T LL ¹ _S (step S5). Note that steps S4 and S5 may be operated in the reverse order or may be operated in parallel.

The moving image noise removal device 1, by the time the high-frequency degeneration means 32b degeneration means 32, the time high-frequency and high spatial frequency expansion coefficients isolated point has been removed in step _{^{S4 H 1 T HH 1 S,}} H 1 T LH 1 S for ^H ₁ T ^HL _{1 S,} sets the threshold value function (the (2) reference expression), the wavelet shrinkage attenuating to "0" the following frequency components threshold calculated in the threshold function (step S6). Note that in the wavelet degeneration performed in step S6, a moving image signal obtained by capturing a dark portion with a camera that has captured a moving image is set to n _N (*) in the threshold value Th (i, n _N (*)) of the equation (2). coefficient wavelet decomposed high temporal frequency and spatial frequency deployed _{^{H 1 T LH 1 S, H}} 1 T HL 1 S, H 1 T HH 1 each median _{^{S n n (H 1 T LH}} 1 S), n n ( H ¹ _T HL ¹ _S), and the use of _{^{n n (H 1 T HH 1}} S).

In addition, the moving image noise removing apparatus 1 uses the wavelet expansion coefficients (temporal low frequency spatial expansion coefficients L ¹ _T LL ¹ _S , L ¹ _T LH ¹⁾ decomposed in step S 3 by the temporal low frequency degeneration means 32 a of the degeneration means 32. _{^{S, L 1 T HL 1 S}} , for _{L 1} T ^HH 1 _S), sets the threshold value function (the (2) reference expression), attenuates the "0" to the following frequency components threshold calculated in the threshold function Wavelet degeneration is performed (step S7). In the wavelet degeneration performed in step S7, a moving image signal obtained by photographing a dark portion with a camera that has photographed a moving image is set to n _N (*) in the threshold Th (i, n _N (*)) of the equation (2). Wavelet-decomposed time low frequency / space expansion coefficients L ¹ _T LL ¹ _S , L ¹ _T LH ¹ _S , L ¹ _T HL ¹ _S , L ¹ _T HH ¹ _S maximum values n _N (L ¹ _T LL ¹ _S ), n _N (L ¹ _T LH ¹ _S ), n _N (L ¹ _T HL ¹ _S ), n _N (L ¹ _T HH ¹ _S ) are used.
It should be noted that the order of step S6 and step S7 may be reversed, or may be operated in parallel.

[Spatial direction reconstruction step]
Then, the moving image noise removal apparatus 1 has the temporal high-frequency / spatial low-frequency expansion coefficient H ¹ _T LL ¹ _{S from} which the isolated points are removed in step S5 by the spatial direction reconstruction unit 41 of the temporal / space wavelet reconstruction unit 40. If, wavelet shrinkage time high-frequency and high spatial frequency expansion coefficient _{^{H 1 T LH 1 S, H}} 1 T HL 1 S, H 1 T HH 1 S and the low frequency-space-time that is wavelet Shrinkage in step S7 in step S6 Two-dimensional first-order discrete wavelet inverse transform (IDWT) is performed on the expansion coefficients L ¹ _T LL ¹ _S , L ¹ _T LH ¹ _S , L ¹ _T HL ¹ _S , and L ¹ _T HH ¹ _S (step S8). ). Thereby, each wavelet expansion coefficient is reconstructed into a temporal high frequency expansion coefficient H ¹ _T and a temporal low frequency expansion coefficient L ¹ _T.

[Time direction reconstruction step]
Then, the moving image noise removal apparatus 1 includes the temporal high-frequency expansion coefficient H ¹ _T and the temporal low-frequency expansion coefficient L ¹ reconstructed in step S8 by the time-direction reconstruction means 42 of the temporal / space wavelet reconstruction means 40. One-dimensional first-order discrete wavelet inverse transform (IDWT) is performed on _T (step S9). As a result, a noise-removed image from which noise has been removed is generated.

  The moving image noise removing apparatus 1 sequentially performs the operations of steps S1 to S9 each time a frame of a moving image signal is input, thereby generating a noise-removed image in time series and moving images from which noise has been removed. A signal will be generated.

  As described above, the moving image noise removing apparatus 1 performs wavelet decomposition on the moving image signal in the time direction and the spatial direction, and then removes noise according to the characteristics of each wavelet expansion coefficient. Noise can be removed with high accuracy.

It is a block block diagram which shows the structure of the moving image noise removal apparatus which concerns on embodiment of this invention. It is explanatory drawing for demonstrating the discrete wavelet transform (DWT) of a time direction. It is explanatory drawing for demonstrating the discrete wavelet transform (DWT) of a spatial direction. It is a time-space 1st-order discrete wavelet transform (DWT) area | region figure of a moving image signal. It is the figure which matched the spectrum plane of the moving speed of the moving body in a moving image signal with the time-space 1st floor DWT area | region figure. It is explanatory drawing for demonstrating the method of measuring the thermal noise characteristic of a camera. It is explanatory drawing for demonstrating the method of removing an isolated point noise. It is explanatory drawing for demonstrating the relationship between the wavelet degeneration of a time low frequency component, and a thermal noise characteristic. It is explanatory drawing for demonstrating the relationship between the wavelet degeneration of a time high frequency / space high frequency component, and a thermal noise characteristic. It is a flowchart which shows operation | movement of the moving image noise removal apparatus which concerns on embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Moving image noise removal apparatus 10 Time and space wavelet decomposition means 11 Time direction decomposition means 12 Spatial direction decomposition means 12a Time low frequency decomposition means 12b Time high frequency decomposition means 20 Noise characteristic storage means (maximum noise characteristic storage means, central noise characteristic storage means)
30 time / space wavelet degeneration means 31 time high frequency isolated point removal means 31a spatial high frequency isolated point removal means (time high frequency / spatial high frequency isolated point removal means)
31b Spatial low frequency isolated point removal means (temporal high frequency / spatial low frequency isolated point removal means)
32 degeneration means 32a time low frequency degeneration means 32b time high frequency degeneration means 40 time / space wavelet reconstruction means 41 spatial direction reconstruction means 42 time direction reconstruction means

Claims (7)

A moving image noise removing apparatus for removing noise included in a moving image signal,
Time direction decomposition means for wavelet decomposition of the moving image signal into a time high frequency expansion coefficient that is a high frequency component in the time direction and a time low frequency expansion coefficient that is a low frequency component in the time direction;
Spatial direction decomposition means for wavelet decomposition of the temporal high frequency expansion coefficient and the temporal low frequency expansion coefficient into frequency components in the spatial direction,
A time high frequency / space high frequency expansion coefficient containing a high frequency component in at least one of the horizontal or vertical directions obtained by wavelet decomposition of the time high frequency expansion coefficient in the spatial direction, and a time low frequency / wavelength decomposition in which the time low frequency expansion coefficient is wavelet decomposed in the spatial direction. In the spatial expansion coefficient, a larger threshold is set for the temporal low frequency / spatial expansion coefficient than the temporal high frequency / spatial high frequency expansion coefficient, and the signal level of frequency components below the threshold is attenuated by the wavelet degeneration method. Degeneration means to cause,
The time high frequency / space high frequency expansion coefficient and the time low frequency / space expansion coefficient whose signal level is attenuated by the degeneration means, and the space direction low frequency component obtained by wavelet decomposition of the time high frequency expansion coefficient by the space direction decomposition means. Spatial direction reconstruction means for reconstructing wavelet in the spatial direction with temporal high frequency and spatial low frequency expansion coefficients,
Time direction reconstruction means for generating a moving image signal by reconstructing the frequency components reconstructed by the spatial direction reconstruction means in the time direction,
A moving image noise removing apparatus comprising: A temporal high frequency / spatial high frequency isolated point removing means for binarizing the temporal high frequency / spatial high frequency expansion coefficient and removing isolated points;
2. The degeneration means attenuates the signal level of the frequency component for the time high frequency / spatial high frequency expansion coefficient from which the isolated points have been removed by the time high frequency / spatial high frequency isolated point removal means. Moving image denoising device. A temporal high frequency / spatial low frequency isolated point removing means for binarizing the temporal high frequency / spatial low frequency expansion coefficient and removing isolated points;
The spatial direction reconstructing means performs wavelet reconstruction using a temporal high frequency / spatial low frequency expansion coefficient from which isolated points have been removed by the temporal high frequency / spatial low frequency isolated point removing means. The moving image noise removing apparatus according to claim 1 or 2. Maximum noise characteristic for storing in advance the maximum value of each expansion coefficient obtained by wavelet decomposition of a dark part moving image signal captured in the dark part by a camera that has captured the original moving image of the moving image signal as a noise characteristic of the camera A storage means,
The degeneration means uses the threshold value calculated by the difference between the maximum value of each expansion coefficient stored in the maximum noise characteristic storage means and the absolute value of the time low frequency / space expansion coefficient, to reduce the time low frequency / space. 4. The moving image noise removing apparatus according to claim 1, wherein the expansion coefficient signal level is attenuated. A central noise characteristic in which a median value of each expansion coefficient obtained by wavelet decomposition of a dark part moving image signal obtained by capturing a dark part by a camera that has captured the original moving image of the moving image signal in time direction and spatial direction is stored in advance as a noise characteristic of the camera. A storage means,
The degeneration means uses the threshold value calculated by the difference between the median value of each expansion coefficient stored in the central noise characteristic storage means and the absolute value of the time high frequency / spatial high frequency expansion coefficient to determine the time high frequency / spatial high frequency. 5. The moving image noise removing apparatus according to claim 1, wherein the expansion coefficient signal level is attenuated. A moving image denoising method for removing noise contained in a moving image signal,
Time for wavelet decomposition of the moving image signal into a time high frequency expansion coefficient which is a high frequency component in the time direction and a time low frequency expansion coefficient which is a low frequency component in the time direction by time direction decomposition means for performing wavelet decomposition in the time direction. A direction decomposition step;
A spatial direction decomposition step of performing wavelet decomposition of the temporal high frequency expansion coefficient and the temporal low frequency expansion coefficient into frequency components in the spatial direction by a spatial direction decomposition means for performing wavelet decomposition in the spatial direction;
A time high frequency / space high frequency expansion coefficient containing a high frequency component in at least one of the horizontal or vertical directions obtained by wavelet decomposition of the time high frequency expansion coefficient in the spatial direction, and a time low frequency / wavelength decomposition in which the time low frequency expansion coefficient is wavelet decomposed in the spatial direction. In the spatial expansion coefficient, a degeneration means for attenuating noise by the wavelet degeneration method sets a larger threshold value for the temporal low frequency / spatial expansion coefficient than the temporal high frequency / spatial high frequency expansion coefficient. A degeneration step for attenuating the signal level of the frequency component;
A time high frequency / space high frequency expansion coefficient and a time low frequency / space expansion coefficient whose signal level is attenuated by the degeneration means by the space direction reconstruction means for performing wavelet reconstruction in the spatial direction, and the time by the space direction decomposition means A spatial direction reconstruction step for reconstructing the time high frequency / spatial low frequency expansion coefficient, which is a low frequency component in the spatial direction obtained by wavelet decomposition of the high frequency expansion coefficient, into a wavelet in the spatial direction;
Time direction reconstruction step of generating a moving image signal by reconstructing the frequency components reconstructed by the spatial direction reconstruction unit in the time direction by the time direction reconstruction unit performing wavelet reconstruction in the time direction. When,
A moving image denoising method comprising: In order to remove the noise contained in the video signal,
Time direction decomposition means for wavelet decomposition of the moving image signal into a time high frequency expansion coefficient that is a high frequency component in the time direction and a time low frequency expansion coefficient that is a low frequency component in the time direction;
Spatial direction decomposition means for wavelet decomposition of the temporal high frequency expansion coefficient and the temporal low frequency expansion coefficient into frequency components in the spatial direction,
A time high frequency / space high frequency expansion coefficient containing a high frequency component in at least one of the horizontal or vertical directions obtained by wavelet decomposition of the time high frequency expansion coefficient in the spatial direction, and a time low frequency / wavelength decomposition in which the time low frequency expansion coefficient is wavelet decomposed in the spatial direction. In the spatial expansion coefficient, a larger threshold is set for the temporal low frequency / spatial expansion coefficient than the temporal high frequency / spatial high frequency expansion coefficient, and the signal level of frequency components below the threshold is attenuated by the wavelet degeneration method. Degeneration means,
The time high frequency / space high frequency expansion coefficient and the time low frequency / space expansion coefficient whose signal level is attenuated by the degeneration means, and the space direction low frequency component obtained by wavelet decomposition of the time high frequency expansion coefficient by the space direction decomposition means. Spatial direction reconstruction means for reconstructing wavelet in the spatial direction with temporal high frequency and spatial low frequency expansion coefficients,
Time direction reconstruction means for generating a moving image signal by reconstructing the frequency components reconstructed by the spatial direction reconstruction means in the time direction,
A moving image denoising program characterized by functioning as

Images