JP5022400B2 - Moving image noise removing device, moving region image noise removing device, moving image noise removing program, and moving region image noise removing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving image noise cancellation apparatus, moving regional image cancellation apparatus, moving image noise cancellation program and moving regional image noise cancellation program, in which even from a moving image signal of a low S/N, noise can be canceled with accuracy. <P>SOLUTION: A moving image noise cancellation apparatus 1 is provided for canceling noise included in a moving image signal constituted of a plurality of source frame images continued time-serially and is mainly constituted of a time-base direction-frequency component decomposing means 11; a frame image stream moving region extracting means 12; a frame image inside moving region extracting means 13; a frame image inside static region extracting means 14; a moving regional image generating means 21; a static regional image generating means 31; a spatial frequency direction smoothing means 22; a time base direction smoothing means 32; and a combination means 40. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a moving image noise removing device, a moving region image noise removing device, a moving image noise removing program, and a moving region image noise removing program for removing noise from a moving image signal.

In general, as a noise removal method for removing noise from a still image, a method using nonlinear processing such as a median filter, a TV (Total Variation) method, a wavelet shrinkage method, or the like is known. Yes.
For example, in the wavelet degeneration method, a noise signal level is set to “0” by a threshold function (Threshold Function) for a DWT expansion coefficient obtained by decomposing (wavelet decomposition) an image signal by a discrete wavelet transform (DWT). This is a non-linear method that performs inverse discrete wavelet transform (IDWT) after attenuation toward (see Non-Patent Document 1). When this method is used, when the image signal is concentrated on a specific DWT expansion coefficient and the absolute value of the expansion coefficient of the image signal is larger than the absolute value of the noise expansion coefficient, the noise is almost eliminated without impairing the characteristics of the original image signal. Removal can be performed.

On the other hand, as a noise removal method for moving images, a method of removing noise by motion compensation and filter processing is generally used (see Patent Documents 1 and 2).
For example, Patent Document 1 discloses a technique for removing noise from a moving image by obtaining a value for subtracting noise from the current image based on a difference between the current image and an image one frame before. In Patent Document 1, when obtaining a value to which noise is subtracted, the shape of an image is determined from pixels before and after the current image and the image one frame before, and there is a correlation between the shapes of the frame images. Discloses a technique for controlling so as to reduce the ratio of noise removal with respect to the current image.

  Further, for example, in Patent Document 2, when a motion correction is performed on a previously input frame image using a local motion vector between the frame images in a moving image, a low-pass filter between the frame images is disclosed. A technique for controlling noise and reducing noise by minimizing blur in a moving image is disclosed.

JP 2008-113352 A JP-A-7-288719

D.L.Donoho, “De-noising by soft-thresholding,” IEEE Trans. Inform.Theory, Vol.41, pp.613-627, May 1995.

Such unnecessary noise components not only deteriorate the subjective image quality of moving images but also reduce the coding efficiency. Therefore, it is desired to establish a technique that can be removed with high accuracy.
However, the conventional noise removal method for moving images has a problem in that, when the signal-to-noise ratio is low, the motion detection accuracy is lowered, so that the noise removal performance is naturally lowered.
In recent years, moving images have become high definition like Super Hi-Vision (SHV), and such high-definition moving images have a small light receiving area per pixel of the image sensor, so that the signal-to-noise ratio depends on the shooting conditions. It will decline. Therefore, the conventional noise removal method has a problem that noise cannot be accurately removed from a moving image.

  The present invention has been made to solve the above problems, and a moving image noise removing apparatus capable of accurately removing noise even with a moving image signal having a low signal-to-noise ratio, It is an object of the present invention to provide a moving area image noise removing apparatus, a moving image noise removing program, and a moving area image noise removing program.

  The present invention was devised to achieve the above object. First, the moving image denoising device according to claim 1 is a moving image signal composed of a plurality of original frame images continuous in time series. Is a moving image noise removing device for removing noise contained in a time axis direction frequency component decomposing means, a frame image sequence moving area extracting means, a frame image moving area extracting means, and a frame image still area extracting means. A moving area image generating means, a static area image generating means, a spatial frequency direction smoothing means, a time axis direction smoothing means, and a synthesizing means. The spatial frequency direction smoothing means is a spatial frequency decomposition means. And a motion vector detecting means, a spatial high-frequency component reducing means, and a wavelet reconstruction means.

  In such a configuration, the moving image noise removing device shifts the moving image signal in the time direction by sequentially shifting the moving image signal by one frame image for each predetermined number of original frame image sequences composed of the original frame images by the time axis direction decomposing means. The moving image signal is decomposed into a high frequency component in the time axis direction and a low frequency component in the time axis direction for each predetermined number of original frame image sequences. This can be decomposed into temporal high-frequency components and temporal low-frequency components by performing one-dimensional first-order discrete wavelet transform for each frame image sequence.

  Further, the moving image noise removing device binarizes a high frequency component in the time axis direction with a predetermined threshold by the frame image sequence moving region extraction unit, and performs frame image sequence moving as a moving region in a predetermined number of original frame image sequences. A frame image sequence moving region binarized image indicating the region is generated. Since the temporal high-frequency component is considered to appear in a portion where there is movement in the frame image sequence, the temporal high-frequency component is further binarized with a threshold value so that the portion of the temporal high-frequency component that is larger than the threshold value is moved in the frame image sequence. It can be extracted as a region.

  Further, the moving image noise removing apparatus performs, for each predetermined number of consecutive frame images, the frame image sequence moving region binarized image generated by the frame image sequence moving region extracting unit by the frame image moving region extracting unit. Then, a common area of the frame image sequence moving area is extracted, and a moving area binarized image indicating a moving area within the frame image which is a moving area for one frame image is generated. Thus, a moving area for one frame image can be extracted.

Then, the moving image noise removing apparatus inverts the pixel value of the moving region binarized image by the frame image inner region extracting means, thereby indicating a still region in the frame image that is a still region for one frame image. A region binarized image is generated. Thereby, a static area for one frame image can be extracted.
In this way, by extracting the static region and the moving region separately from the moving image signal, even when the signal-to-noise ratio is low, the characteristics of noise included in the static region and the moving region can be accurately extracted. In addition, noise can be individually removed according to the relationship between the noise in the static region and the dynamic region, so that the noise removal performance can be improved.

  In addition, the moving image noise removing apparatus corresponds to the moving region binarized image and the time axis direction based on the moving region binarized image generated by the moving region image generating unit by the moving region image generating unit. A moving area is extracted from the original frame image to generate a moving area image.

  Still further, the moving image noise removing device is configured to generate a static region binarized image and a temporal axis direction based on the static region binarized image generated by the static region image generating unit by the static region image generating unit. A static region image is generated by extracting a static region from the corresponding original frame image.

The moving image noise removing apparatus removes noise from the moving region image by performing a smoothing process in the spatial frequency direction on the moving region image generated by the moving region image generating unit by the spatial frequency direction smoothing unit. A moving area noise-removed image is generated.
Here, the spatial frequency direction smoothing means includes spatial frequency decomposition means, motion vector detection means, spatial high frequency component reduction means, and wavelet reconstruction means.

  The moving image noise removing device performs wavelet decomposition on the moving region image generated by the moving region image generating unit in the spatial direction by the spatial frequency decomposing unit of the spatial frequency direction smoothing unit, thereby converting the moving region image into a spatial low frequency. Breaks down into components and spatial high frequency components. In the spatial wavelet decomposition, the moving area image can be decomposed into a spatial low frequency component and a spatial high frequency component by performing a two-dimensional first-order discrete wavelet transform on the moving area image.

  In addition, the moving image noise removing device includes a spatial low frequency component of one moving region image obtained by the spatial frequency decomposing unit, a single moving region image and a time axis by the motion vector detecting unit of the spatial frequency direction smoothing unit. Block matching is performed between the spatial low-frequency components of moving region images adjacent in the direction, and a motion vector of one moving region image is detected. Each spatial low-frequency component is reduced in noise because a low-pass filter process and a thinning process that are ½ in the horizontal and vertical directions are added to each moving area image. Since noise is reduced in this way, motion detection accuracy can be improved, and therefore, motion vectors can be detected well by performing block matching between spatial low-frequency components.

  Furthermore, the moving image noise removing apparatus converts the motion vector of one moving area image detected by the motion vector detecting unit by the spatial high frequency component reducing unit of the spatial frequency direction smoothing unit in the horizontal direction, the vertical direction, and the diagonal direction. The three directions are classified, and the spatial high-frequency component is reduced according to the classified directions. Since the image taken with the camera has accumulated blur in the direction in which the subject moved, noise can be accurately removed by reducing the spatial high-frequency component individually according to the direction of movement by the spatial high-frequency component reducing means. it can.

Still further, the moving image noise removing apparatus reconstructs the one moving region image in which the spatial high frequency component is reduced by the spatial high frequency component reducing unit in the spatial frequency direction by the wavelet reconstructing unit of the spatial frequency direction smoothing unit. As a result, a moving image noise-removed moving image signal is generated. As the wavelet reconstruction in the spatial direction, two-dimensional first-order discrete wavelet inverse transformation is performed on the spatial high-frequency component and the spatial low-frequency component, so that each frequency component of the spatial high-frequency component and the spatial low-frequency component is converted into a video signal. Can be reconfigured.
Since the moving region has low signal correlation in the time direction along with noise, noise is removed by performing smoothing processing in the spatial direction in this way.

  Furthermore, the moving image noise removing apparatus removes noise from the static region image by performing the time axis direction smoothing process on the static region image generated by the static region image generating unit by the time axis direction smoothing unit. A static area noise-removed image is generated. The static region has a high signal correlation in the time axis direction, and the noise has a low signal correlation in the time axis direction. Therefore, the noise is removed by performing the time axis direction smoothing process by the time axis direction smoothing means.

  Then, the moving image noise removing device generates a noise-removed moving image signal for one frame image by combining the moving region noise-removed image and the static region noise-removed image by the combining unit. Thus, it is possible to obtain a noise-removed moving image signal from which noise is individually removed according to the relationship between the noise in the moving area and the static area.

  According to a second aspect of the present invention, there is provided the moving area image noise removing device based on moving area information indicating a moving area extracted in advance from a moving image signal composed of a plurality of original frame images continuous in time series. A moving area noise removing apparatus for generating a noise-removed moving image signal from which noise of the moving image signal has been removed, comprising moving area image generating means and spatial frequency direction smoothing means, and spatial frequency direction smoothing The means includes a spatial frequency resolving means, a motion vector detecting means, a spatial high frequency component reducing means, and a wavelet reconstruction means.

The moving area image noise removing device generates a moving area image by extracting a moving area from an original frame image based on moving area information by moving area image generation means.
In addition, the moving region noise removing device removes noise from the moving region image by performing a smoothing process in the spatial frequency direction on the moving region image generated by the moving region image generating unit by the spatial frequency direction smoothing unit. A moving area noise-removed image is generated.

  That is, the moving area image noise removing device performs wavelet decomposition on the moving area image generated by the moving area image generating means in the spatial direction by the spatial frequency decomposing means, thereby converting the moving area image into a spatial low frequency component and a spatial high frequency. Decomposes into ingredients. In addition, the moving area noise removing device includes a moving low frequency component of one moving area image obtained by the spatial frequency direction decomposing means and a moving area adjacent to the one moving area image in the time axis direction by the motion vector detecting means. Block matching is performed with the spatial low-frequency component of the image, and a motion vector of one moving area image is detected. Furthermore, the moving area noise removing device classifies the motion vector of one moving area image detected by the motion vector detecting means into three directions of a horizontal direction, a vertical direction, and an oblique direction by the spatial high frequency component reducing means, The spatial high frequency component is reduced according to the classified direction. Then, the moving area noise removing device performs wavelet reconstruction in the spatial frequency direction of one moving area image in which the spatial high frequency component is reduced by the spatial high frequency component reducing means by the wavelet reconstruction means, thereby removing noise in the moving area. A moving image signal is generated.

  According to a third aspect of the present invention, there is provided a moving image noise removal program that performs a time-axis direction frequency component in order to remove noise included in a moving image signal composed of a plurality of time-series continuous original frame images. Decomposing means, frame image sequence moving area extracting means, frame image moving area extracting means, frame image still area extracting means, moving area image generating means, still area image generating means, spatial frequency direction smoothing means, time axis direction smoothing And functioning as a synthesizing means and a synthesizing means.

  In such a configuration, the moving image noise removal program shifts the moving image signal by one frame image sequentially for each of a predetermined number of original frame image sequences composed of original frame images by the time axis direction frequency component decomposition means. By performing wavelet decomposition in the direction, the frame image sequence is decomposed into a temporal high frequency component and a temporal low frequency component.

  Further, the moving image noise removal program binarizes the time high-frequency component of the frame image sequence obtained by the time axis direction decomposition unit by the frame image sequence moving region extraction unit with a predetermined threshold value, thereby obtaining a frame image sequence. A frame image sequence moving region binarized image indicating a frame image sequence moving region which is a moving region in is generated.

  Still further, the moving image noise removal program performs the predetermined continuous number of frame image sequence moving region binarized images sequentially generated by the frame image sequence moving region extracting unit by the frame image moving region extracting unit. By extracting a common area of the frame image sequence moving area, a moving area binarized image indicating a moving area within the frame image which is a moving area for one frame image is generated.

  Still further, the moving image noise removal program reverses the pixel value of the moving region binarized image generated by the frame image moving region extracting unit by the frame image still region extracting unit, thereby obtaining one frame image worth. A static region binarized image indicating a static region in the frame image that is a static region is generated.

  In addition, the moving image denoising program, in accordance with the moving region binarized image generated by the moving region image generating unit and the moving region binarized image generated by the in-frame moving region extracting unit, in the time axis direction. A moving area is extracted from a corresponding original frame image to generate a moving area image.

  In addition, the moving image noise removal program is further configured to generate a static region binarized image and a time axis direction based on the static region binarized image generated by the static region image generating unit by the static region image generating unit. A static region image is generated by extracting a static region from the original frame image corresponding to.

Further, the moving image noise removing program performs wavelet decomposition on the moving region image generated by the moving region image generating unit in the spatial direction by the spatial frequency decomposing unit, thereby converting the moving region image into a spatial low frequency component and a spatial high frequency component. And decompose.
In addition, the moving image noise removal program uses the motion vector detection means to obtain a spatial low frequency component of one moving area image obtained by the spatial frequency decomposition means, and a moving area image adjacent to the one moving area image in the time axis direction. Block matching is performed with the spatial low-frequency component of, and a motion vector of one moving area image is detected.
Further, the moving image noise removal program classifies the motion vector of one moving area image detected by the motion vector detecting means into three directions of a horizontal direction, a vertical direction, and an oblique direction by the spatial high frequency component reducing means. The spatial high frequency component is reduced according to the direction.
Still further, the moving image noise removal program reconstructs the moving region of the moving region by reconstructing the one moving region image in which the spatial high frequency component is reduced by the spatial high frequency component reducing unit by the wavelet reconstructing unit in the spatial frequency direction. A noise-removed moving image signal is generated.

Furthermore, the moving image noise removal program removes noise from the static region image by performing the smoothing process in the time axis direction on the static region image generated by the static region image generating unit by the temporal axis direction smoothing unit. A static area noise-removed image is generated.
Then, the moving image noise removal program generates a noise-removed moving image signal for one frame image by combining the moving region noise-removed image and the static region noise-removed image by the combining unit.

  According to a fourth aspect of the present invention, there is provided a moving region image denoising program based on moving region information indicating a region having a motion extracted in advance from a moving image signal composed of a plurality of original frame images continuous in time series. In order to generate a noise-removed moving image signal from which the noise of the moving image signal has been removed, a computer is converted into a moving region image generating means, a spatial frequency decomposing means, a motion vector detecting means, a spatial high frequency component reducing means, and a wavelet reconstruction means. It was set as the structure made to function as.

The moving area image noise removal program generates a moving area image by extracting a moving area from an original frame image based on moving area information by moving area image generation means.
Further, the moving image noise removal image generating program performs wavelet decomposition on the moving region image generated by the moving region image generating unit in the spatial direction by the spatial frequency decomposing unit, thereby converting the moving region image into a spatial low-frequency component and a space. Decomposes into high frequency components.

  Further, the moving area image denoising image generation program is adjacent to the spatial low frequency component of one moving area image obtained by the spatial frequency decomposition means and the one moving area image in the time axis direction by the motion vector detecting means. Block matching is performed with the spatial low-frequency component of the moving area image, and a motion vector of one moving area image is detected.

Still further, the moving area image denoising image generation program generates a motion vector of one moving area image detected by the motion vector detecting means in three directions, ie, a horizontal direction, a vertical direction, and an oblique direction, by the spatial high frequency component reducing means. Classify and reduce spatial high frequency components according to the classified direction.
Then, the moving region image noise removal image generation program performs wavelet reconstruction in the spatial frequency direction by using the wavelet reconstruction unit to reconstruct one moving region image in which the spatial high frequency component is reduced by the spatial high frequency component reduction unit. The denoising video signal is generated.

The present invention has the following excellent effects.
According to the inventions described in claims 1 to 4, in order to extract a moving area and a static area from a moving image signal, and to remove noise individually according to the relationship between each noise of the moving area and the static area, Even a moving image signal having a low signal-to-noise ratio can accurately remove noise. Therefore, for example, noise can be accurately removed even for a high-definition moving image such as Super Hi-Vision (SHV).
In addition, according to the first to fourth aspects of the present invention, since the spatial high-frequency component is individually reduced according to the direction of the motion vector of the moving area image, noise can be accurately removed.

It is a block diagram which shows the structure of the moving image noise removal apparatus of this invention. In this invention, it is a conceptual diagram for demonstrating the process which carries out the wavelet decomposition | disassembly of a frame image sequence in a time direction. In this invention, it is a conceptual diagram for demonstrating the process which extracts a moving region from the frame image row | line | column by which wavelet decomposition was carried out. (A) is a conceptual diagram for explaining processing for extracting a moving region for one frame image in the present invention, and (b) is a one-frame image from the moving region for one frame image extracted in (a). It is a conceptual diagram for demonstrating the process which extracts the still area of a minute. In this invention, it is a conceptual diagram for demonstrating the process which carries out wavelet decomposition | disassembly of a moving area image to a spatial direction. In this invention, it is a conceptual diagram which shows the process which detects the motion vector of a moving region image. In this invention, it is a conceptual diagram which shows the process which reduces the spatial high frequency component of a moving region image. In this invention, it is a conceptual diagram for demonstrating the process which removes the noise of a static area image. It is a flowchart which shows operation | movement of the moving image noise removal apparatus which concerns on embodiment of this invention. It is a flowchart which shows operation | movement of the moving image noise removal apparatus which concerns on embodiment of this invention.

Hereinafter, a moving image noise removing apparatus according to the present invention will be described with reference to the drawings. First, the outline of the processing of the moving image noise removal apparatus will be described, and then the configuration will be described.
[Overview of video noise removal processing]
First, an outline of a moving image noise removal process performed by the moving image noise removal device of the present invention will be described.
In a frame image constituting a moving image signal, when the original image at time t in the xy coordinate system is s (x, y, t) and the noise component is n (x, y, t), a moving image including noise The signal F (x, y, t) can be expressed by the following equation (1).

(Equation 1)
F (x, y, t) = s (x, y, t) + n (x, y, t) (1)

  Since the noise component n (x, y, t) is a thermal noise component of the camera that has captured the moving image, it has whiteness and Gaussianity. That is, the noise component n (x, y, t) has the characteristics that the occurrence probability follows a normal distribution, has substantially the same energy distribution in all frequency bands, and has a low correlation in the time direction and the spatial direction. .

  Therefore, in the moving image noise removal apparatus of the present invention, after moving image signal wavelet decomposition in the time direction to perform static / moving region determination, moving image noise removal is performed by applying effective spatio-temporal filtering processing to each region. Is to do.

[Configuration of moving image noise elimination device]
With reference to FIG. 1, the structure of the moving image noise removal apparatus which concerns on embodiment of this invention is demonstrated. Here, the moving image noise removing apparatus 1 includes a still image / moving region component extracting unit 10 in a frame image, a moving region image noise removing unit 20, a static region image noise removing unit 30, and a synthesizing unit 40. Yes.

  The frame image still / moving region component extraction means 10 extracts moving region components and still region components for one frame image from the moving image signal. The frame image still / moving area component extracting means 10 includes a time-axis direction frequency component decomposing means 11, a frame image sequence moving area extracting means 12, a frame image inner moving area extracting means 13, and a frame image inner still area extracting means. 14, a moving area storage memory 15, and a static area storage memory 16.

  The time axis direction frequency component decomposition means 11 performs wavelet decomposition on the moving image signal in the time direction. This time-axis direction frequency component decomposition means 11 inputs a moving image signal in units of frame images (frame image sequence) corresponding to the length of the wavelet coefficient in synchronization with a clock output from a clock circuit (not shown). By performing one-dimensional first-order discrete wavelet transform (DWT) on all horizontal and vertical coordinates of the column, a time-low frequency expansion coefficient (temporal low-frequency component) Lt, which is a low-frequency component in the time-axis direction, and a time axis Wavelet decomposition is performed on a time high frequency expansion coefficient (time high frequency component) Ht which is a high frequency component in the direction.

  That is, as shown in FIG. 2, the time-axis direction frequency component decomposition means 11 sets an arbitrary reference time position (t = 0) to t, and its time position frame image F (t), and from F (t) to the time direction. Wavelet transform of the frame images F (t), F (t + 1), F (t + 2), F (t + 3),... For the wavelet coefficient lengths of all (x, y) pixel values in the time direction. Thus, the time low frequency component Lt and the time high frequency component Ht are decomposed (wavelet decomposition).

Further, the time-axis direction frequency component decomposition means 11 performs wavelet decomposition in the time direction by sequentially shifting one frame image for each frame image sequence.
For example, the temporal high-frequency component OddH _t (t) next to the temporal high-frequency component EvenH _t (t) is F (t + 1) shifted by one frame image in the time axis direction from the frame image F (t), as shown in FIG. To F (t + 4) is obtained by performing the wavelet decomposition in the time direction by the time-axis direction frequency component decomposition means 11. In FIG. 3, for convenience of explanation, a moving image is used as an example in which the ball moves in the time direction from the left side to the right side of the frame image.
Similarly, temporal high-frequency components OddH _t (t + 1), temporal high-frequency components EvenH _t (t + 1)... Can be obtained by performing wavelet decomposition in the time direction by sequentially shifting one frame image for each frame image sequence.

The wavelet coefficient length and its length vary depending on the wavelet used. For example, when a Daubechies secondary wavelet is used, four coefficients (wavelet coefficient length = “4”) of [−0.1294 0.2241 0.8365 0.4830] can be used as wavelet coefficients. This wavelet coefficient length is the number of frame images in the time direction. That is, the time high-frequency components EvenH _t (t), OddH _t (t)... Obtained by the time-axis direction frequency component decomposing means 11 each include a change in the moving area for four frame images.
The time axis direction frequency component decomposition means 11 outputs the time high frequency components EvenH _t (t), OddH _t (t)... Of the frame image sequence subjected to wavelet decomposition to the frame image sequence moving region extraction means 12.

The frame image sequence moving region extracting unit 12 extracts a frame image sequence moving region which is a moving region in the frame image sequence from the time high frequency component of the frame image sequence decomposed by the time axis direction frequency component decomposing unit 11.
That is, the frame image sequence moving region extraction unit 12 binarizes the time high-frequency components EvenH _t (t), OddH _t (t)... Of the frame image sequence with a predetermined threshold, and a portion larger than the threshold is frame image sequence. Extract as a moving area. That is, the frame image sequence moving region extraction unit 12 generates frame image sequence moving region binarized images B1, B2,... As shown on the upper side of the drawing in FIG. The generated frame image sequence moving region binarized images B1, B2,... Are output to the frame image moving region extracting means 13.

The frame image moving area extraction unit 13 extracts a common area for each predetermined continuous number of the frame image sequence moving area binarized images generated by the frame image sequence moving area extraction unit 12 to obtain 1 A moving area binarized image is generated that indicates a moving area within a frame image that is a moving area for a frame image. Here, the predetermined consecutive number is determined by the wavelet coefficient length.
That is, the frame image moving area extracting means 13 calculates the frame image sequence by calculating a common set B1∩B2∩B3∩B4 of the frame image sequence moving region binarized images B1, B2, B3, and B4. A common area of F (t + 3) moving areas included in each of the moving area binarized images B1, B2, B3, and B4 is extracted, and a moving area binary for one frame image as shown in FIG. MovArea (t + 3) that is a converted image can be generated.
The generated moving area binarized images MovArea (t + 3), MovArea (t + 4),... Are output to the frame image still area extracting means 14 and the moving area storage memory 15, respectively.

  The frame image still area extracting unit 14 generates a one-frame image of the original image from a binarized moving image indicating a moving area within the frame image, which is a moving area for one frame image generated by the frame-image moving area extracting unit 13. Minute static region binarized image is generated. That is, the frame image still area extracting means 14 inverts the pixel values of the moving area binarized images MovArea (t + 3), MovArea (t + 4)... 1 as shown in FIG. It is possible to generate StaArea (t + 3), StaArea (t + 4)... That are static region binarized images for the frame image. The generated static area binarized images, StaArea (t + 3), StaArea (t + 4)... Are output to the static area storage memory 16.

  The moving area storage memory 15 stores the moving area binarized image generated by the moving area extracting means 13 in the frame image. The moving area binarized images MovArea (t + 3), MovArea (t + 4)... Stored in the moving area storage memory 15 are appropriately read out by the moving area image generating means 21 of the moving area image noise removing means 20 described later.

The static area storage memory 16 stores the static area binarized image generated by the static area extracting means 14 in the frame image. The static area binarized images StaArea (t + 3), StaArea (t + 4)... Stored in the static area storage memory 16 are appropriately read out by the static area image generating means 31 of the static area image noise removing means 30 described later.
In FIG. 1, the moving area storage memory 15 and the static area storage memory 16 are illustrated separately for the sake of explanation, but they may be configured by one storage means (not shown).

The moving area image noise removing means 20 (moving area image noise removing apparatus) removes noise of moving area images for one frame image, and includes moving area image generating means 21, spatial frequency direction smoothing means 22, and so on. It is equipped with.
Based on the moving area binarized image for one frame image stored in the moving area storage memory 15, the moving area image generating means 21 moves from the moving area binarized image and the original frame image corresponding to the time direction. A region is extracted to generate a moving region image.

  That is, the moving area image generation means 21 includes, for example, a moving area binarized image MovArea (t + 3), and the moving area binarized image MovArea (t + 3) and the original frame image F (t + 3) corresponding to the time direction. By performing an AND operation (AND), a moving area image MovF (t + 3) is generated by extracting only the moving area in the original frame image F (t + 3). The moving region images MovF (t + 3), MovF (t + 4)... Generated in this way are output to the spatial frequency direction smoothing means 22. The moving area image generation means 21 performs the same processing for moving area binarized images MovArea (t + 4) and later.

  The spatial frequency direction smoothing unit 22 performs a spatial direction smoothing process on the moving region image generated by the moving region image generating unit 21 to generate a moving region noise-removed image obtained by removing noise from the moving region image. To do. The spatial frequency direction smoothing unit 22 includes a spatial frequency decomposition unit 221, a motion vector detection unit 222, a spatial high frequency component reduction unit 223, and a wavelet reconstruction unit 224.

The spatial frequency decomposition means 221 performs wavelet decomposition on the moving area image generated by the moving area image generation means 21 into frequency components in the spatial direction. That is, as shown in FIG. 5, the spatial frequency decomposition means 221 performs, for example, a spatial low-frequency expansion that is a low-frequency component in the spatial direction by performing a two-dimensional first-order discrete wavelet transform on the moving region image MovF (t + 3). Wavelet decomposition is performed into a coefficient (spatial low-frequency component) LL_F (t + 3) and a spatial high-frequency expansion coefficient (spatial high-frequency component) LH_F (t + 3), HL_F (t + 3), and HH_F (t + 3) that are high-frequency components in the spatial direction. At this time, the Haar wavelet is used and applied to the horizontal direction and the vertical direction, respectively. Here, the reason for using the Haar wavelet is that it is the smallest filter that satisfies the linear phase characteristics.
The spatial low frequency expansion coefficient (spatial low frequency component) and the spatial high frequency expansion coefficient (spatial high frequency component) generated in this manner are sequentially output to the motion vector detection means 222.

  The motion vector detection means 222 is between the spatial low frequency component of one moving area image obtained by the spatial frequency decomposition means 221 and the spatial low frequency component of one moving area image and a moving area image adjacent in the time direction. Block matching is performed to detect a motion vector of one moving area image. In the following description, it is assumed that one moving area image is MovF (t + 3), and a moving area image adjacent to one moving area image is MovF (t + 4).

  As illustrated in FIG. 6, the motion vector detection unit 222 includes a spatial low-frequency component LL_F (t + 3) of the moving region image MovF (t + 3) and a moving region image MovF (t + 4) adjacent to the moving region image MovF (t + 3). By acquiring the spatial low frequency component LL_F (t + 4) and performing block matching using the n × n size block between the spatial low frequency component LL_F (t + 3) and the spatial low frequency component LL_F (t + 4), The motion vector of each block is detected.

  Here, block matching is performed between LL_F (t + 3) and LL_F (t + 4), which are spatial low-frequency components that have inferior motion detection accuracy compared to the moving region images MovF (t + 3) and Mov (t + 4). This is because it is advantageous in detecting the direction of the motion vector of the moving region image MovF (t + 3) including That is, the spatial low-frequency components LL_F (t + 3) and LL_F (t + 4) are halved by a low-pass filter process and a thinning process that are ½ in the horizontal and vertical directions compared to the moving area images MovF (t + 3) and MovF (t + 4) Is added, noise is reduced. Therefore, the motion detection accuracy can be increased by performing block matching between the spatial low frequency component LL_F (t + 3) and the spatial low frequency component LL_F (t + 4).

The motion vector information MovMem (i, j) detected in this way is output to the spatial high-frequency component reduction means 223. The spatial low frequency component LL_F (t + 3) is output to the wavelet reconstruction unit 224.
The motion vector information may be stored as motion vector information MovMem (i, j) in a motion vector detection memory (not shown). The maximum horizontal and vertical sampling frequencies I and J of the motion vector information MovMem (i, j) are I = (1/2) · X, J = when the resolution of the original image is horizontal X pixels and vertical Y lines. (1/2) · Y. When a motion vector detection memory is provided, the motion vector information MovMem (i, j) is appropriately read out from the motion vector detection memory by the spatial high frequency component reduction means 223.

The spatial high-frequency component reduction unit 223 classifies the motion vector information of one moving area image detected by the motion vector detection unit 222 into three directions of a horizontal direction, a vertical direction, and an oblique direction, and a space is generated according to the classified direction. The high frequency component is reduced.
First, the spatial high-frequency component reduction means 223 calculates the motion direction of an arbitrary ∀ (i, j) of MovMem (i, j) that is motion vector information. This is because accumulation blur in camera imaging occurs in the direction of movement.

  Next, the spatial high-frequency component reducing means 223 classifies the direction of the motion vector into three directions of 0 degree (horizontal direction), 45 degrees (oblique direction), and 90 degrees (vertical direction) when viewed in the first quadrant. In this classification, for example, 0 to 20 degrees is a horizontal direction, 21 to 70 degrees is an oblique direction, and 71 to 90 degrees is a vertical direction. The reason why the diagonal direction is wide is that in human vision, a decrease in resolution in the diagonal direction is less perceivable than in the horizontal and vertical directions.

  As shown in FIG. 7, the spatial high frequency component reducing unit 223 further performs the spatial high frequency component LL_F (t + 3) when the direction of the motion vector at the pixel position (i, j) is, for example, 0 degrees (horizontal direction). The frequency component at the pixel position (i, j) in the spatial high-frequency component LH_F (t + 3) located in the horizontal direction is reduced to ½. Here, since the Haar wavelet is used when performing the two-dimensional first-order discrete wavelet decomposition of MovF (t + 3) in the spatial direction, the process of reducing the frequency component at the pixel position (i, j) of LH_F (t + 3) to ½ Corresponds to a process of reducing the difference (corresponding to differentiation) between the frequency component of the pixel position (x, y) of the original image and the frequency component of the pixel position (x + 1, y) adjacent in the horizontal direction to ½.

Further, when the direction of the motion vector at the pixel position (i, j) is 45 degrees (diagonal direction), for example, the pixel position of the spatial high frequency component HH_F (t + 3) located in an oblique direction with respect to the spatial low frequency component LL_F (t + 3) The frequency component of (i, j) is reduced to ½. That is, the difference between the frequency component at the pixel position (x, y) of the original image and the frequency component at the pixel position (x + 1, y + 1) located obliquely with respect to the pixel position (x, y) of the original image is 1 Reduce to / 2.
Further, when the direction of the motion vector at the pixel position (i, j) is 90 degrees (vertical direction), for example, the pixel position of the spatial high-frequency component HL_F (t + 3) positioned in the vertical direction with respect to the spatial low-frequency component LL_F (t + 3). The frequency component of (i, j) is reduced to ½. That is, the difference between the frequency component of the pixel position (x, y) of the original image and the frequency component of the pixel position (x, y + 1) positioned in the direction perpendicular to the pixel position (x, y) of the original image is halved. To reduce.

  In this way, the spatial high-frequency component reducing unit 223 performs processing to reduce the spatial high-frequency components at all pixel positions of the motion vector information MovMem (i, j), and the result is output to the wavelet reconstruction unit 224. Is done.

The wavelet reconstruction unit 224 generates a moving-region noise-removed moving image signal by reconstructing the spatial high-frequency component obtained by reducing the noise signal level by the spatial high-frequency component reduction unit 223 in the spatial direction and the time direction, respectively. To do.
That is, the wavelet reconstruction unit 224 first performs two-dimensional first-order discrete wavelet inverse transform on the spatial low frequency component LL_F (t + 3) and the spatial high frequency components LH_F (t + 3), HH_F (t + 3), and HL_F (t + 3) (IDWT) is performed. As a result, each frequency component is reconstructed into a temporal high frequency component and a temporal low frequency component.
The generated noise-removed image of the moving area is output to the synthesizing unit 40.

The static region image noise removing unit 30 removes noise of the static region image for one frame image, and includes a static region image generating unit 31 and a time axis direction smoothing unit 32.
Based on the static region binarized image stored in the static region storage memory 16, the static region image generation means 31 extracts a static region from the static region binarized image and the original frame image corresponding to the time direction. A static region image is generated.

  That is, the static region image generation unit 31 includes, for example, a static region binarized image StaArea (t + 3), and the static frame binarized image StaArea (t + 3) and an original frame image F (t + 3) corresponding to the time direction. The static region image StaF (t + 3) is generated by extracting only the static region in the original frame image F (t + 3). The static region images StaF (t + 3), StaF (t + 4)... Generated in this way are output to the time axis direction smoothing means 32.

  The time axis direction smoothing unit 32 performs a smoothing process in the time direction on the static region image generated by the static region image generating unit 31, thereby generating a static region noise-removed image from which noise has been removed from the static region image. Is. As described above, the thermal noise has the characteristics that it has white Gaussian characteristics and the signal correlation in the time direction is low. Therefore, the time axis direction smoothing means 32 uses this characteristic to make a static region image StaF (t + 3). , StaF (t + 4)...

  That is, as shown in FIG. 8, the time axis direction smoothing means 32 obtains StaF (t + 3) and STAF (t + 2) and StaF (t + 4) which are frame images before and after the time direction of this StaF (t + 3). By using a low-pass filter of [1/4 1/2 1/4] to the pixel position (x, y), noise at the pixel position (x, y) is removed. In this way, the low-pass filter is applied to all pixel positions, and a noise-removed image in the static region is generated. The static area noise-removed image generated in this way is output to the synthesis means 40.

The synthesizing unit 40 synthesizes the noise-removed image of the moving region generated by the moving-region image noise removing unit 20 and the noise-removed image of the static region generated by the still-region image noise removing unit 30 to obtain a noise-removed moving image. A signal is generated.
That is, the synthesizing unit 40 generates a noise-removed image NR (t + 3) of the original image by calculating a logical OR operation between the noise-removed image of the moving area and the noise-removed image of the static area.

  By configuring the moving image noise removing apparatus 1 as described above, it is possible to individually remove noise according to the characteristics of the frequency components of the static region and the moving region. That is, since thermal noise has a characteristic that signal correlation in the time direction is low, noise in the static region can be removed by applying a low-pass filter in the time direction. In addition, since the moving region has a characteristic that the signal correlation in the time direction is low, noise in the moving region can be removed by applying a low-pass filter in the spatial direction. Furthermore, noise in the moving area can be removed in consideration of the accumulated blur in the moving direction.

  The moving image noise removing apparatus 1 can be realized by a general computer having a CPU and a memory (not shown). At this time, the moving image noise removing apparatus 1 can operate the computer by a moving image noise removing program that causes the computer to function as each of the above-described means. This moving image noise elimination program can be distributed via a network, or can be written and distributed on a recording medium such as a CD-ROM. The same applies to the still image / moving region component extracting device and the moving image noise removing device in the frame image.

[Operation of moving image noise elimination device]
Next, referring to FIGS. 9 and 10 (refer to FIG. 1 as appropriate for the configuration), the operation of the moving image noise removal apparatus according to the embodiment of the present invention will be described.

[Time-axis direction frequency component decomposition processing]
As shown in FIG. 9, first, the moving image noise removal apparatus 1 is synchronized with a clock (CLK) by a time axis direction frequency component decomposing unit 11 of a frame image static / moving region component extracting unit 10. Are input in units of frames, and one-dimensional first-order discrete wavelet transform (DWT) is performed for each frame image sequence (step S1). Thus, the moving image signal is subjected to wavelet decomposition in the time direction, and is decomposed into a temporal high frequency component and a temporal low frequency component for each frame image sequence.

[Frame image sequence moving area extraction processing]
Next, the moving image noise removal apparatus 1 determines in advance the time high-frequency components EvenH _t (t), OddH _t (t)... Of the frame image sequence decomposed in step S1 by the frame image sequence moving region extraction unit 12. By binarizing using the threshold value, frame image sequence moving region binarized images B1, B2,... Indicating frame image sequence moving regions that are moving regions in the frame image sequence are generated (step S2).

[Frame image moving area extraction processing]
Furthermore, the moving image noise removing apparatus 1 extracts the common area of the frame image sequence moving area binarized images B1, B2,... A moving area binarized image MovArea (t + 3)... Indicating a moving area in a frame image which is a moving area for one frame image is generated (step S3).

[Static region extraction processing in frame image]
Still further, the moving image noise removing apparatus 1 uses the still image extraction unit 14 in the frame image to invert the pixel values of the moving region binarized image generated in step S3 so that the moving image noise removing device 1 can obtain a still region corresponding to one frame image. A static area binarized image StaArea (t + 3)... Showing a static area in a frame image is generated (step S4).

[Dynamic image noise removal processing]
Subsequently, the moving image noise removing apparatus 1 generates a moving region image from the moving region binarized image MovArea (t + 3)... Generated by the moving region image noise removing unit 20 in step S3. Remove image noise.
As shown in FIG. 10, first, the moving image noise removing apparatus 1 uses the moving region binarized image MovArea (t + 3)... Generated in step S3 by the moving region image generating unit 21 and the time direction. Is obtained by performing AND operation (AND) with the original frame image F (t + 3)... Corresponding to the moving frame image MovF (t + 3). ... (Step S5).

  Next, the moving image noise removing apparatus 1 performs two-dimensional first-order discrete wavelet decomposition on the moving region image MovF (t + 3)... Generated in step S5 by the spatial frequency decomposing unit 221 of the spatial frequency direction smoothing unit 22. (DWT), the moving region image MovF (t + 3)... Is converted into a spatial low frequency expansion coefficient (spatial low frequency component) LL_F (t + 3) which is a low frequency component in the spatial direction and a high frequency component in the spatial direction. Wavelet decomposition is performed into certain spatial high-frequency expansion coefficients (spatial high-frequency components) LH_F (t + 3), HL_F (t + 3), and HH_F (t + 3) (step S6).

  Further, the moving image noise removing apparatus 1 includes a spatial low frequency component LL_F (t + 3) of the moving region image MovF (t + 3) wavelet decomposed in step S6 by the motion vector detecting unit 222 of the spatial frequency direction smoothing unit 22. Block matching is performed between the moving region image MovF (t + 3) and the spatial low frequency component LL_F (t + 4) of the adjacent moving region image Mov (t + 4), and the pixel position (i, The motion vector in j) is detected, and the spatial low-frequency component LL_F (t + 3) is output to the wavelet reconstruction unit 224 (step S7).

  Furthermore, the moving image noise removing apparatus 1 converts the motion vector information detected in step S7 by the spatial high-frequency component reducing unit 223 of the spatial frequency direction smoothing unit 22 into three directions, horizontal, vertical, and diagonal directions. The spatial high frequency components LH_F (t + 3), HH (t + 3), and HL (t + 3) are reduced according to the classified movement direction (step S8).

[Time / space reconstruction]
Then, the moving image noise removal apparatus 1 uses the wavelet reconstruction unit 224 of the spatial frequency direction smoothing unit 22 to reduce the high-frequency components LH_F (t + 3), HH_F (t + 3), and HL_F, in which the noise signal level is reduced in step S8. Two-dimensional first-order discrete wavelet inverse transform (IDWT) is performed on (t + 3) and the spatial low-frequency component LL_F (t + 3) (step S9).

[Static region image generation processing]
Next, in the moving image noise removal apparatus 1, the static region binarized image StaArea (t + 3) and the static region binarized image StaArea (t + 3) are generated by the static region image generation unit 31 of the static region image noise removal unit 30. A static region image StaF (t + 3) obtained by extracting only a static region in the original frame image F (t + 3) by performing a logical product operation (AND) with the original frame image F (t + 3) corresponding to the time direction. Generate (step S10).

[Time axis direction smoothing]
Further, the moving image noise removing apparatus 1 uses the time-axis direction smoothing unit 32 of the static region image noise removing unit 30 to perform frames before and after the still region image StaF (t + 3) and the still region image StaF (t + 3) in the time direction. By applying a low-pass filter of [1/4 1/2 1/4] to all pixel positions using the static region images StaF (t + 2) and StaF (t + 4) which are images, all A noise-removed image of a static region is generated by removing noise at the pixel position (step S11). Note that steps S10 and S11 may be performed continuously with step S4, or may be operated in parallel with steps S5 to S9.

(Composition process)
Then, the moving image noise removing apparatus 1 calculates a logical OR operation between the noise-removed image in the moving area and the noise-removed image in the static area by the synthesizing unit 40, so that the noise-removed image NR (t + 3) of the original image ) Is generated (step S12).

  Then, each time a frame image of a moving image signal is input, the moving image noise removing apparatus 1 sequentially performs the operations of steps S1 to S12 to generate a noise-removed image in time series, and noise removal with noise removed. A moving image signal is generated.

  As described above, the moving image noise removing apparatus 1 is effective in accordance with the relationship between each static / moving region and noise after performing wavelet decomposition of the moving image signal in the time direction and performing static / moving region determination. By performing a spatio-temporal filter process, noise can be accurately removed from the moving image signal.

Although the embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment.
For example, in the above-described embodiment, the spatial high frequency component reducing unit 223 classifies the direction of the motion vector as 0 to 20 degrees as a horizontal direction, 21 to 70 degrees as an oblique direction, and 71 to 90 degrees as a vertical direction. However, the present invention is not limited to this, and the numerical range can be changed as appropriate. Considering that a decrease in resolution in the oblique direction is less perceptible in human vision than in the horizontal and vertical directions, the oblique direction is made wide even when changing the numerical value range.
According to this, since the numerical range of the classification of the direction of the motion vector can be adjusted for each moving image to be processed, the accuracy of noise removal can be further improved.

  Further, for example, in the above-described embodiment, when the spatial high-frequency component is reduced by the spatial high-frequency component reduction unit 223, the pixel position (i, j) in each spatial high-frequency component corresponding to the motion vector of the pixel position (i, j). ) Is reduced to ½, but the present invention is not limited to this. The reduction value may be changed depending on the magnitude of the motion vector. For example, if the value of the motion vector is large, the reduction value is reduced accordingly. On the other hand, if the value of the motion vector is small, the reduction value is set large accordingly. The relationship between the motion vector value and the reduction value may satisfy, for example, a linear relationship. According to this, since the reduction value can be adjusted in correspondence with the value of the motion vector for each moving image to be processed, the accuracy of noise removal can be further improved.

DESCRIPTION OF SYMBOLS 1 Moving image noise removal apparatus 10 Frame image static / moving area extraction means 11 Time axis direction frequency component decomposition means 12 Frame image sequence moving area extraction means 13 Frame image moving area extraction means 14 Frame image sequence static area extraction means 15 Motion Area storage memory 16 Static area storage memory 20 Moving area image noise removing means 21 Moving area image generating means 22 Spatial frequency direction smoothing means 221 Spatial frequency decomposing means 222 Motion vector detecting means 223 Spatial high frequency component reducing means 224 Wavelet reconstruction means 30 Static area image noise removing means 31 Moving area image generating means 32 Time axis direction smoothing means 40 Synthesis means

Claims (4)

A moving image noise removing apparatus for removing noise included in a moving image signal composed of a plurality of original frame images continuous in time series,
The moving image signal is wavelet-decomposed in the time direction by sequentially shifting one frame image for each of a predetermined number of the original frame image sequences constituted by the original frame images, and the moving image signal is converted into the predetermined number of original frame images. Time axis direction frequency component decomposition means for decomposing into a high frequency component in the time axis direction and a low frequency component in the time axis direction for each column;
A frame for binarizing the high-frequency component in the time axis direction with a predetermined threshold value and generating a frame image sequence moving region binarized image indicating a frame image sequence moving region that is a moving region in a predetermined number of the original frame image sequences Image sequence moving region extraction means;
For the frame image sequence moving region binarized image generated by the frame image sequence moving region extracting means, a common region of the frame image sequence moving region is extracted for each predetermined number of consecutive frame images, and one frame A frame image in-motion area extracting means for generating a motion area binarized image indicating an in-frame area in the frame image, which is a motion area for the image;
A frame image static area extraction means for generating a static area binary image indicating a static area in a frame image that is a static area for one frame image by inverting the pixel value of the moving area binarized image;
A moving area image generating means for generating a moving area image by extracting a moving area from the original frame image corresponding to the moving area binarized image and the time axis direction based on the moving area binarized image;
Based on the static region binarized image, a static region image generation means for generating a static region image by extracting a static region from the original frame image corresponding to the static region binarized image and the time axis direction;
Spatial frequency direction smoothing means for generating a moving area noise-removed image obtained by removing noise from the moving area image by performing a smoothing process in the spatial direction on the moving area image;
Time axis direction smoothing means for generating a static area noise-removed image obtained by removing noise from the static area image by performing a smoothing process in the time axis direction on the static area image;
A synthesis means for generating a noise-removed moving image signal for one frame image by synthesizing the moving-region noise-removed image and the static-region noise-removed image;
The spatial frequency direction smoothing means includes:
Spatial frequency decomposition means for decomposing the dynamic region image into a spatial low frequency component and a spatial high frequency component by performing wavelet decomposition in the spatial frequency direction on the dynamic region image generated by the dynamic region image generation unit;
Block matching is performed between the spatial low-frequency component of one moving region image obtained by the spatial frequency decomposition means and the spatial low-frequency component of the moving region image adjacent to the one moving region image in the time axis direction. Motion vector detecting means for detecting a motion vector of the one moving area image;
Spatial high-frequency component reduction that classifies the motion vector of one moving area image detected by this motion vector detection means into three directions of horizontal, vertical, and diagonal directions, and reduces the spatial high-frequency component according to the classified direction Means,
Wavelet reconstruction means for generating a noise-removed moving image signal in a moving area by reconstructing the one moving area image in which the spatial high frequency component is reduced by the spatial high frequency component reducing means in the spatial frequency direction. A moving image noise removing apparatus comprising: A noise-removed moving image signal obtained by removing noise from the moving image signal based on moving region information indicating a moving region extracted in advance from a moving image signal composed of a plurality of original frame images continuous in time series. A moving region image denoising device to generate,
Based on the moving area information, moving area image generating means for generating a moving area image by extracting a moving area from the original frame image;
A spatial frequency direction smoothing unit that generates a moving region noise-removed image obtained by removing noise from the moving region image by performing a spatial direction smoothing process on the moving region image generated by the moving region image generating unit; With
The spatial frequency direction smoothing means includes:
A spatial frequency decomposition means for decomposing the dynamic area image into a spatial low frequency component and a spatial high frequency component by performing wavelet decomposition on the dynamic area image generated by the dynamic area image generation means in a spatial direction;
Block matching is performed between the spatial low-frequency component of one moving region image obtained by the spatial frequency decomposition means and the spatial low-frequency component of the moving region image adjacent to the one moving region image in the time axis direction. Motion vector detecting means for detecting a motion vector of the one moving area image;
Spatial high-frequency component reduction that classifies the motion vector of one moving area image detected by this motion vector detection means into three directions of horizontal, vertical, and diagonal directions, and reduces the spatial high-frequency component according to the classified direction Means,
Wavelet reconstruction means for generating a noise-removed moving image signal of a moving area by reconstructing the one moving area image in which the spatial high frequency component is reduced by the spatial high frequency component reducing means in a spatial frequency direction; A moving area image denoising device comprising: In order to remove noise included in a moving image signal composed of a plurality of original frame images continuous in time series, a computer is provided.
The moving image signal is wavelet-decomposed in the time direction by sequentially shifting one frame image for each of a predetermined number of the original frame image sequences constituted by the original frame images, and the moving image signal is converted into the predetermined number of original frame images. Time axis direction frequency component decomposition means for decomposing into a high frequency component in the time axis direction and a low frequency component in the time axis direction for each column,
A frame for binarizing the high-frequency component in the time axis direction with a predetermined threshold value and generating a frame image sequence moving region binarized image indicating a frame image sequence moving region that is a moving region in a predetermined number of the original frame image sequences Image sequence moving region extraction means,
For the frame image sequence moving region binarized image generated by the frame image sequence moving region extracting means, a common region of the frame image sequence moving region is extracted for each predetermined number of consecutive frame images, and one frame A frame image in-motion area extracting means for generating a motion area binarized image indicating an in-frame area in the frame image, which is a motion area for the image;
A frame image static region extraction means for generating a static region binary image indicating a static region in a frame image that is a static region for one frame image by inverting the pixel value of the moving region binary image;
A moving area image generating means for generating a moving area image by extracting a moving area from the moving frame binarized image and the original frame image corresponding to the time axis direction based on the moving area binarized image;
A static region image generating means for generating a static region image by extracting a static region from the original frame image corresponding to the static region binarized image and the time axis direction based on the static region binarized image;
Spatial frequency decomposition means for decomposing the dynamic region image into a spatial low frequency component and a spatial high frequency component by performing wavelet decomposition in the spatial direction on the dynamic region image generated by the dynamic region image generation unit;
Block matching is performed between the spatial low-frequency component of one moving region image obtained by the spatial frequency decomposition means and the spatial low-frequency component of the moving region image adjacent to the one moving region image in the time axis direction. , Motion vector detection means for detecting a motion vector of the one moving area image,
Spatial high-frequency component reduction that classifies the motion vector of one moving area image detected by this motion vector detection means into three directions of horizontal, vertical, and diagonal directions, and reduces the spatial high-frequency component according to the classified direction means,
Wavelet reconstruction means for generating a noise-removed moving image signal of a moving area by reconstructing the one moving area image in which the spatial high frequency component is reduced by the spatial high frequency component reducing means in the spatial frequency direction;
Time axis direction smoothing means for generating a static area noise-removed image obtained by removing noise from the static area image by performing a smoothing process in the time axis direction on the static area image,
A moving image denoising program that functions as a synthesizing unit that generates a denoised moving image signal for one frame image by synthesizing the moving region noise-removed image and the static region noise-removed image. A noise-removed moving image signal obtained by removing noise from the moving image signal based on moving region information indicating a moving region extracted in advance from a moving image signal composed of a plurality of original frame images continuous in time series. To generate a computer,
A moving area image generating means for generating a moving area image by extracting a moving area from the original frame image based on the moving area information;
A spatial frequency direction decomposition unit that decomposes the dynamic region image into a spatial low frequency component and a spatial high frequency component by performing wavelet decomposition on the dynamic region image generated by the dynamic region image generation unit in the spatial direction;
Block matching is performed between the spatial low-frequency component of one moving region image obtained by the spatial frequency direction decomposition means and the spatial low-frequency component of the moving region image adjacent to the one moving region image in the time axis direction. Performing motion vector detection means for detecting a motion vector of the one moving area image;
Spatial high-frequency component reduction that classifies the motion vector of one moving area image detected by this motion vector detection means into three directions of horizontal, vertical, and diagonal directions, and reduces the spatial high-frequency component according to the classified direction means,
The one moving region image in which the spatial high-frequency component is reduced by the spatial high-frequency component reducing unit is reconstructed into a wavelet in the spatial frequency direction, thereby functioning as a wavelet reconstructing unit that generates a moving-region noise-removed moving image signal. A moving area image denoising program characterized by the above.

Images