JP2002223374A - Device and method for removing noise - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a noise removing method for excellently removing a noise even concerning a mobile part while making the best use of a merit in a noise removal ability with respect to a still part in a motion adaptive recursive filter. SOLUTION: The motion adaptive recursive filter 11 is used as a first noise removing part. As a second noise removing part, a class classification adaptive removing circuit 12 is used, which extracts the pixel of each frame at the same position between multiple frames, classifies the noise components of the pixels into classes based on the inter-frame change of the pixels and generates an output picture signal obtained by removing the noise component from an input picture signal by an arithmetic processing which is previously set in accordance with the classified classes. An output selection part 13 decides the movement of the picture by prescribed number-pixel unit, selects one of the output picture signals of the first and the second noise removing parts in accordance with the decision result and outputs it.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a noise elimination device and a noise elimination method for removing noise from an image signal.

[0002]

2. Description of the Related Art In order to remove noise from an image signal,
Conventionally, a motion adaptive recursive filter has been used. FIG. 7 shows an example of the configuration of this motion adaptive recursive filter.

An input image signal is supplied to an adding circuit 2 through an amplifier 1 for adjusting the amplitude for each pixel. The frame memory 2 stores an output image signal of a frame (hereinafter, referred to as a previous frame) immediately before a current frame (a current frame (hereinafter, referred to as a current frame) of an output image signal). The image signal stored in the frame memory 2 is sequentially read out for each pixel corresponding to each pixel position of the input image signal, and supplied to the addition circuit 2 through the amplifier 4 that performs amplitude adjustment.

[0004] The addition circuit 2 adds the pixels of the current frame and the previous frame through the amplifier 2 and the amplifier 4, outputs the added output as an output image signal, and supplies it to the frame memory 3. In the frame memory 3, the stored image signal is rewritten to the output image signal of the added output.

The input image signal of the current frame is supplied to a subtraction circuit 5 for each pixel. Further, the image signal of the previous frame stored in the frame memory 3 is sequentially read out for each pixel corresponding to each pixel position of the input image signal and supplied to the subtraction circuit 5. Therefore, the subtraction circuit 5
From, the difference between the pixel value of the current frame at the same pixel position on the image and the pixel value of the previous frame is obtained.

[0006] The difference output from the subtraction circuit 5 is supplied to an absolute value conversion circuit 6 and converted into an absolute value, and then supplied to a threshold value processing circuit 7. In the threshold processing circuit 7,
The absolute value of the pixel difference supplied thereto is compared with a predetermined threshold value, and a determination is made for each pixel as to whether it is a moving part or a stationary part. That is, in the threshold value processing circuit 7, when the absolute value of the pixel difference is smaller than the threshold value, the input pixel is determined to be a stationary portion, and when the absolute value of the pixel difference is larger than the threshold value, the input pixel is Judge as a moving part.

[0007] The result of the static motion determination in the threshold processing circuit 7 is
It is supplied to the weight coefficient generation circuit 8. Weight coefficient generation circuit 8
Sets the value of the weight coefficient k (0 ≦ k ≦ 1) in accordance with the result of the static motion determination in the threshold value processing circuit 7, supplies the coefficient k to the amplifier 1, and sets the coefficient 1-k to the amplifier 4 The amplifier 1 multiplies its input signal by k, and the amplifier 4
The input signal is multiplied by 1-k.

In this case, when the threshold processing circuit 7 determines that the pixel of the current frame is stationary, a fixed value between k = 0 to 0.5 is set as the value of the coefficient k. Therefore, the output of the adding circuit 2 is a value obtained by weighting and adding the pixel value of the current frame and the pixel value of the previous frame from the frame memory 3.

On the other hand, when the threshold value processing circuit 7 determines that the pixel of the current frame is a moving part, k = 1 is set as the value of the coefficient k. Therefore, the pixel value of the current frame (the pixel value of the input image signal) is output from the adding circuit 2 as it is.

The signal stored in the frame memory 3 is rewritten every frame by the output image signal from the adder circuit 2. Therefore, in the stationary portion in the image signal stored in the frame memory 3, the pixel values of a plurality of frames are integrated. Will be done. Therefore, assuming that the noise changes randomly for each frame, the noise is gradually reduced and removed by the weighted addition, and the static portion of the image signal (same as the output image signal) stored in the frame memory 3 is removed. Is
The noise has been removed.

[0011]

However, in the above-described noise removal by the motion adaptive recursive filter, it is necessary to: For example, when the noise level is high, the moving part may be mistaken for the stationary part, in which case,
Image quality degradation such as blurring may be observed. 2. Moving parts cannot be denoised. There is a problem.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a noise elimination apparatus and method which can overcome the above-mentioned problems while taking advantage of the noise elimination ability of a stationary portion of the above-mentioned motion adaptive recursive filter. is there.

[0013]

In order to solve the above-mentioned problems, a noise elimination apparatus according to the present invention has a frame memory for storing an image signal, the image signal stored in the frame memory, and an input image. And adding the signals by weighting according to the stillness and motion of the image based on the input image signal, and rewriting the image signal of the frame memory by the added output, thereby removing the first noise-free signal as the added output. A first noise elimination unit that generates an output image signal of a plurality of frames, and extracts corresponding pixels on an image between a plurality of frames, and classifies noise components of the pixels based on a change between the frames of the pixels. Generating a second output image signal from which a noise component has been removed from the input image signal by an arithmetic process set in advance corresponding to the classified class; A noise elimination unit, and determines the stillness of the image in units of a predetermined number of pixels at image positions corresponding to each other between the first output image signal and the second output image signal, and according to the determination result, An output selection unit that selects and outputs one of the first output image signal and the second output image signal in units of the predetermined number of pixels.

According to the present invention having the above-described configuration, the first noise elimination unit performs the weighted addition of the current frame and the previous frame to perform the weighting addition of the current frame and the previous frame in the same manner as the above-described motion adaptive recursive filter. Good noise removal is achieved.

On the other hand, the second noise removing unit extracts pixels of each frame located at the same position among a plurality of frames, classifies the noise components of the pixels based on a change between the frames of the pixels, Since the noise component is removed from the input image signal by the arithmetic processing set in advance corresponding to the classified class, the noise is removed regardless of the moving part and the stationary part. However, for a completely stationary portion, the first noise elimination unit capable of storing information of a long frame has a larger noise elimination effect in the second noise elimination unit.

The output selection unit determines the stillness of the image in units of a predetermined number of pixels, and according to the result of the determination, in the unit of the predetermined number of pixels, in the stationary part, from the first noise removal unit. By selecting the first output image signal and selecting the second output image signal from the second noise removal unit in the moving part, good noise removal was performed in both the stationary part and the moving part. An output image signal is obtained.

[0017]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a noise elimination device according to the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of the noise elimination device of this embodiment. As shown in FIG. 1, an input image signal is supplied to a motion adaptive recursive filter 11 constituting an example of a first noise removing unit for each pixel, and a class constituting an example of a second noise removing unit. It is supplied to the classification adaptive noise elimination circuit 12.

The structure of the motion adaptive recursive filter 11 is exactly the same as the example shown in FIG. The output image signal from the motion adaptive recursive filter 11 is supplied to an output selection circuit 13.

The classification adaptive noise elimination circuit 12
Extracts pixels of each frame at the same position among a plurality of frames, classifies the noise components of the pixels based on a change between the frames of the pixels, and presets the noise components corresponding to the classified classes. The output image signal from which the noise component has been removed is generated from the input image signal by the arithmetic processing described above, and its detailed configuration will be described later. The output image signal from the classification adaptive noise elimination circuit 12 is also supplied to the output selection circuit 13.

The output selection circuit 13 includes a static / motion determination circuit 14
A delay circuit 15 for adjusting timing, and a selection circuit 16
The output image signal from the motion adaptive recursive filter 11 is supplied to the selection circuit 16 through the delay circuit 15, and the output image signal from the classification adaptive noise removal circuit 12 is directly supplied to the selection circuit 16. .

The motion adaptive recursive filter 11
And the output image signal from the classification adaptive noise elimination circuit 12 are supplied to a still / moving judgment circuit 14. In this example, the static / movement determining circuit 14 determines whether the pixel is a stationary portion or a moving portion for each pixel from the two output image signals, and uses the determination output as a selection control signal.
It is supplied to the selection circuit 16.

As described above, in the output image signal from the motion adaptive recursive filter 11, pixels in the still portion of the image are noise-removed, but pixels in the moving portion of the image are
It is output as it is without noise removal. On the other hand, noise is removed from the output image signal from the classification adaptive noise elimination circuit 12 irrespective of the still and moving parts of the image.

For this reason, when the output image signal from the motion adaptive recursive filter 11 and the output image signal from the class adaptive noise removal circuit 12 are compared with each other, since both of the static portions are noise-removed, Although the pixel values are substantially equal, in the moving part, noise remains in the output image signal of the motion adaptive recursive filter 11, whereas in the output image signal from the adaptive noise reduction circuit 12, noise remains. Since they have been removed, the pixel values of the two differ by the amount of noise.

In this example, the static / movement determining circuit 14 determines whether each pixel is a static portion or a moving portion of an image by using the above-described properties. That is, the static / movement determination circuit 14 is configured to control the motion adaptive recursive filter 11
Value calculation circuit 141 for calculating the difference between the pixel value of the output image signal from the image signal and the pixel value of the output image signal from the classification adaptive noise elimination circuit 12, and the difference value calculation circuit 141
An absolute value conversion circuit 142 for converting the difference value from the absolute value into an absolute value, and a comparison determination circuit 143.

In the comparison determination circuit 143, the absolute value conversion circuit 1
If the absolute value of the difference value from the absolute value conversion circuit 142 is larger than a predetermined value, it is determined to be a moving portion. If the absolute value of the difference value from the absolute value conversion circuit 142 is smaller than the predetermined value, the stationary portion is determined. Is determined. The selection circuit 16 is controlled so as to select an output image signal from the motion adaptive recursive filter 11 for a pixel determined to be a still part of the image. The selection circuit 16 is controlled so as to select the output image signal from the class adaptive noise removal circuit 12.

Therefore, from the selection circuit 16, that is, from the output selection circuit 13, the information of the long frame can be accumulated for the stationary portion, and the output from the motion-adaptive recursive filter from which the noise is well removed can be obtained. An image signal is output, and for a moving part, an output image signal from the classification adaptive noise elimination circuit 12 is output instead of an output image signal from a motion adaptive recursive filter from which noise is not removed. Therefore, from the output selection circuit 13, an output image signal from which noise has been removed is obtained over all of the stationary portion and the moving portion.

[Explanation of Classification Adaptive Noise Removal Circuit]
Next, the classification adaptive noise elimination circuit used in this embodiment will be described in detail. In the example described below, as the classification adaptive processing, the classification is performed according to the three-dimensional (spatio-temporal) distribution of the signal level of the input image signal, and the prediction coefficients obtained by learning in advance for each class are stored in the memory. Then, an optimal estimation value (ie,
A process of outputting a pixel value after noise removal is adopted.

In this example, noise is removed by performing a classification adaptive process in consideration of the motion of an image. That is, in accordance with the motion estimated from the input image signal, a pixel region to be referred to for detecting a noise component and a pixel region to be used for an arithmetic process for removing noise are cut out. And outputs an image from which noise has been removed by the classification adaptive processing based on.

FIG. 2 shows the overall configuration of a class adaptive noise elimination circuit used in this embodiment.

An input image signal to be processed is supplied to a frame memory 21. The frame memory 21 stores the supplied image of the current frame and supplies the image of the previous frame to the frame memory 22. The frame memory 22 stores the supplied image of one frame and supplies the image of the previous frame to the frame memory 23. Thus, the frame memories 21, 2
The images of newer frames are stored in the order 2 and 23 in this order.

In the following description, the frame memory 22 stores the current frame, and the frame memories 21 and 23
Are respectively stored in the case where the frames after and before the current frame are stored.

The contents stored in the frame memories 21, 22, and 23 are not limited to these. For example, images at a time interval of two frames may be stored. Also,
Not limited to three consecutive frames, five frame memories may be provided to store images of five consecutive frames. Furthermore, a field memory can be used instead of the frame memory.

The image data of the subsequent frame, the current frame, and the previous frame stored in the frame memories 21, 22, and 23 are respectively stored in the motion vector detecting section 24, the motion vector detecting section 25, the first area extracting section 26, and the second It is supplied to the area cutout unit 27.

The motion vector detecting section 24 detects a motion vector for a target pixel between the image of the current frame stored in the frame memory 22 and the image of the previous frame stored in the frame memory 23. In addition, the motion vector detection unit 25 detects a motion vector for a pixel of interest between the image of the current frame stored in the frame memory 22 and the image of the subsequent frame stored in the frame memory 21.

The motion vector (movement direction and motion amount) for the target pixel detected by each of the motion vector detection units 24 and 25 is supplied to a first area cutout unit 26 and a second area cutout unit 27. As a method of detecting a motion vector, a block matching method, estimation using a correlation coefficient, a gradient method, or the like can be used.

The first area cutout unit 24 refers to the motion vectors detected by the motion vector detection units 24 and 25 from the image data of each frame supplied thereto,
A pixel at a position as described later is extracted, and the extracted pixel value is supplied to the noise component detection unit 28.

The noise component detection unit 28 generates a class code representing information on the noise component based on the output of the first area cutout unit 24 and supplies the generated class code to the coefficient ROM 29, as described later. I do. As described above, the pixels extracted by the first area cutout unit 24 are used for generating a class code, and thus are called class taps.

The coefficient ROM 29 stores in advance prediction coefficients determined by learning as will be described later for each class, more specifically, along an address associated with a class code. Then, the coefficient ROM 29 receives the class code supplied from the noise component detection unit 28 as an address, and outputs a prediction coefficient corresponding to the address.

On the other hand, the second area cutout unit 27 extracts prediction pixels from the data of three consecutive frames stored in the frame memories 21, 22, and 23, and estimates the values of the extracted pixels. It is supplied to the arithmetic unit 30. The estimation calculation unit 30 calculates a value based on the output of the second area cutout unit 27 and the prediction coefficient read from the coefficient ROM 29.
A weighting operation as shown in the following equation (1) is performed to generate a predicted image signal from which noise has been removed. As described above, the pixel values extracted by the second area cutout unit 27 are used in the weighted addition for generating the predicted image signal, and thus are referred to as prediction taps.

Y = w ₁ × x ₁ + w ₂ × x ₂ +... + W _n × x _n (1) Here, x ₁ , ‥‥, x _n are each prediction tap,
w ₁ , ‥‥, and w _n are respective prediction coefficients.

Next, with reference to FIG. 3, the processing performed by the first area cutout unit 26 will be described in more detail. First
The area cutout unit 26 extracts a pixel at a pixel position specified by a tap structure as shown in FIG. Here, pixels indicated by black squares are extracted as class taps. That is, only the pixel of interest is extracted as a class tap from the current frame fr0, and the previous frame fr-1 and
From the subsequent frame fr1, one pixel corresponding to the pixel of interest is extracted as a class tap.

That is, this example has a tap structure in which only one pixel is extracted in each of the previous frame fr-1, the current frame fr0, and the subsequent frame frl. In the first area cutout unit 26, if the motion vector of the pixel of interest detected by the motion vector detection units 24 and 25 is sufficiently small and determined to be a still part, the previous frame fr-1, the current frame fr0, Pixels at the same pixel position in each frame of the frame frl are extracted as class taps for noise detection. Therefore, the pixel position of the class tap in each frame to be processed is constant, and there is no change in the tap structure.

On the other hand, if it is determined that the movement of the pixel of interest is larger than a certain level and it is a moving part, the first area cutout unit 26 sets each of the previous frame fr-1, the current frame fr0, and the subsequent frame frl. In order to extract the pixel at the corresponding position on the image from the frame as a class tap, the pixel position extracted corresponding to the motion vector is corrected. The position of the pixel extracted from the image data of the subsequent frame fr1 is corrected by the motion vector detected by the motion vector detection unit 24, and the position of the pixel extracted from the image data of the previous frame fr-1 is determined by the motion vector detection unit 25. Is corrected by the motion vector detected in.

In this example, the same tap structure as the above-described class tap is used for the prediction tap cut out by the second area cut-out unit 27. Then, in the second area cutout unit 27, the motion correction for the pixel extracted as the prediction tap is performed in the same manner as described above.

As a result of such a motion correction, the class tap extracted by the first area cutout unit 26 becomes a corresponding pixel on the image between a plurality of frames. The prediction tap extracted by the second region cutout unit 27 also becomes a corresponding pixel on an image between a plurality of frames due to the motion correction.

The number of frame memories is increased to, for example, five instead of three. For example, the current frame and two frames before and after the current frame are recorded, and only the target pixel is extracted from the current frame. A class tap structure may be used in which a pixel corresponding to the target pixel is extracted from every two subsequent frames. In such a case, the pixel region to be extracted is temporally expanded, so that more effective noise removal can be performed.

As will be described later, the noise component detecting section 28 detects the level fluctuation of the noise component for the pixel of interest from the fluctuation of the pixel values of the pixels of the three frames cut out as class taps by the first area cutting section 26. Detected and class code corresponding to the level change of the noise component
Output to M29. That is, the noise component detection unit 28
The noise component of the target pixel is classified into classes according to the level fluctuation of the corresponding pixel of the target pixel in a plurality of frames, and a class code indicating which of the classified classes is output.

In this embodiment, the noise component detector 28 determines the output of the first region
ADRC (Adaptive Dynamic Ran)
(geCodlng), and generates a class code composed of ADRC output based on the level fluctuation of the pixel corresponding to the target pixel over a plurality of frames.

FIG. 4 shows an example of the noise component detector 28. FIG. 4 shows a case where a class code is generated by 1-bit ADRC.

The dynamic range detection circuit 281 includes:
As described above, a total of three pixels are supplied from each of the frame memories 21, 22, and 23, that is, the pixel of interest in the current frame and two pixels corresponding to the pixel of interest in frames before and after the current frame. You. The value of each pixel is, for example, 8
Expressed in bits. The dynamic range detection circuit 281 calculates the maximum value MAX and the minimum value M of the three pixels.
IN is detected, and a dynamic range DR is calculated by an operation of MAX-MIN = DR.

The dynamic range detection circuit 28
1 is the calculated dynamic range D as its output
R, the minimum value MIN, and the pixel values Px of the three input pixels are output.

The pixel values Px of the three pixels from the dynamic range detection circuit 281 are sequentially supplied to a subtraction circuit 282, and the minimum value MIN is subtracted from each pixel value Px. By removing the minimum value MIN from each pixel value Px, the normalized pixel value is supplied to the comparison circuit 283.

The output (D) of the bit shift circuit 284 for reducing the dynamic range DR to １ ／ is supplied to the comparison circuit 283.
R / 2) is supplied, and the magnitude relationship between the pixel value Px and DR / 2 is detected. When the pixel value Px is larger than DR / 2, the 1-bit comparison output of the comparison circuit 283 is set to "1", otherwise, the comparison output is set to "0". Then, the comparison circuit 283 generates a 3-bit ADRC output by parallelizing the comparison outputs of the sequentially obtained three pixels.

The dynamic range DR is supplied to a bit number conversion circuit 285, and the number of bits is converted from 8 bits to, for example, 5 bits by quantization. Then, the dynamic range converted into the number of bits and the 3-bit ADRC output are used as a class code as a coefficient ROM.
29.

Under the above-described class tap structure, the pixel value should not fluctuate or be small between the target pixel of the current frame and the corresponding pixels of the preceding and succeeding frames. Therefore, when a change in pixel value is detected, it can be determined that the change is caused by noise.

To explain one example, in the case of the example shown in FIG. 5, the pixel value of the class tap extracted from each of the temporally successive frames of t-1, t, t + 1 is 1 bit ADR.
By receiving the process of C, an ADRC output of 3 bits [010] is generated. Then, the dynamic range DR converted into 5 bits is output. The 3-bit ADRC output represents a change in the noise level of the target pixel.

In this case, instead of one bit, multiple bits A
If the DRC is performed, the noise level fluctuation can be more accurately expressed. Further, the magnitude of the noise level is expressed by a code obtained by converting the dynamic range DR into 5 bits. The reason why 8 bits are converted to 5 bits is to clip so that the number of classes does not increase so much.

As described above, the class code generated by the noise component detecting unit 28 in this case includes a code consisting of, for example, 3 bits relating to a temporal noise level variation obtained as a result of ADRC, and a dynamic range DR.
And a code consisting of, for example, 5 bits related to the noise level obtained as a result of By using the dynamic range DR for class classification, motion and noise can be distinguished, and a difference in noise level can be distinguished.

Next, learning, that is, processing for obtaining a prediction coefficient stored in the coefficient ROM 29 will be described with reference to FIG. Here, the same components as those in FIG. 2 are denoted by the same reference numerals.

A noise-free input image signal (referred to as a teacher signal) used for learning is supplied to a noise adding section 31 and a normal equation adding section 32.
The noise adding unit 31 adds a noise component to the input image signal to generate a noise added image (referred to as a student signal), and supplies the generated student signal to the frame memory 21. FIG.
As described with reference to FIG.
In images 2 and 23, images of three consecutive frames of the student signal are stored, respectively.

The following description is made on the assumption that the frame memory 22 stores the image of the current frame, and the frame memories 21 and 23 store the images of the frames after and before the current frame, respectively. However, as described above, the storage contents of the frame memories 21, 22, and 23 are not limited to this.

In the subsequent stage of the frame memories 21, 22, and 23, substantially the same processing as the processing described above with reference to FIG. 2 is performed. However, the class code generated by the noise component detection unit 28 and the prediction tap extracted by the second region extraction unit 27 are supplied to the normal equation addition unit 32. The teacher signal is further supplied to the normal equation adding unit 32. The normal equation adding unit 32 performs calculation processing for solving a normal equation based on these three types of inputs, and the prediction coefficient determination unit 33 determines a prediction coefficient for each class code from the calculation processing result. Then, the prediction coefficient determination unit 33
Supplies the determined prediction coefficient to the memory 34. The memory 34 stores the supplied prediction coefficients. The prediction coefficient stored in the memory 34 and the prediction coefficient stored in the coefficient ROM 29 (FIG. 2) are the same.

Next, the normal equation will be described. In the above equation (1), before learning, the prediction coefficients w ₁ ,.
_n is an undetermined coefficient. Learning is performed by inputting a plurality of teacher signals for each class. When the number of types of the teacher signal for each class is represented by m, the following equation (2) is set from the equation (1).

Y _k = w ₁ × x _k1 + w ₂ × x _k2 +... + W _n × x _kn (2) (k = 1, 2,..., M) When m> n, the prediction coefficient w ₁ , ‥‥, because w _n are not uniquely determined, elements e _k of an error vector e, it is defined by the following equation (3).

E _k = y _k − {w ₁ × x _k1 + w ₂ × x _k2 +... + W _n × x _kn } (3) (k = 1, 2,..., M) The prediction coefficient is determined so as to minimize the error vector e defined by (4).
That is, the prediction coefficient is uniquely determined by the so-called least square method.

[0067]

(Equation 1)

As a practical calculation method for obtaining the prediction coefficient minimizing e ² in equation (4), partial differentiation of e ² with the prediction coefficient w _i (i = 1, 2 ‥‥) (hereinafter referred to as Equation (5)), each prediction coefficient w _i may be determined so that the partial differential value becomes 0 for each value of _i .

[0069]

(Equation 2)

A specific procedure for determining each prediction coefficient w _i from equation (5) will be described. When X _ji and Y _i are defined as in the equations (6) and (7), the equation (5) becomes the following equation (8)
Can be written in the form of the determinant

[0071]

(Equation 3)

Equation (8) is generally called a normal equation. The prediction coefficient determination unit 33 calculates each parameter in the normal equation (8) based on the above three types of inputs, and further solves the normal equation (8) according to a general matrix solution such as a sweeping-out method. _Is calculated to calculate the prediction coefficient w _i .

Next, in order to add noise in the noise adding unit 31, the following methods can be used, for example.

As in the case of the computer simulation, random noise is generated and added to the input image signal.

Noise is added to an input image signal via an RF system.

A flat image signal with little level change;
A noise component is extracted as a difference between the image signal and a signal obtained by performing a process through an RF system,
The extracted noise component is added to the input image signal.

A noise component is extracted as a difference between a signal obtained by performing a process using an RF system on a flat image signal and an image signal component from which noise has been removed by frame addition of the signal. The extracted noise component is added to the input image signal.

The noise elimination circuit 12 using the above-described class classification adaptive processing extracts, for example, a pixel of interest and a pixel corresponding to the pixel of interest as a class tap when performing the class classification adaptive processing for removing noise from the image signal. Then, a change in the noise level between frames is detected based on the data of the class tap, and a class code is generated in accordance with the detected change in the noise level.

Then, pixels to be used for noise component detection processing (class taps) and pixels to be used for prediction calculation processing (prediction taps) are estimated so as to estimate motion between frames and correct the estimated motion. Is extracted. Then, an image signal from which noise has been removed is calculated by linear linear combination of a prediction tap and a prediction coefficient for each class information reflecting the noise component.

Therefore, it is possible to appropriately select a prediction coefficient corresponding to the inter-frame variation of the noise component. By performing an estimation operation using such a prediction coefficient, the noise component can be removed satisfactorily. be able to.

Then, even when there is a motion, the noise level can be correctly detected, and the noise can be removed. In particular, it is possible to prevent the image from being blurred due to erroneous determination of a moving part as a stationary part as in the case of the motion adaptive recursive filter described with reference to FIG.

Further, a class tap structure having no spatial spread in a frame, for example, only the pixel of interest is extracted from the current frame,
When a tap structure in which a pixel corresponding to a pixel of interest is extracted from a subsequent frame is used as a class tap and / or a prediction tap, it is possible to prevent a blurring factor in the spatial direction from affecting processing. . That is, it is possible to avoid blurring in the output image signal due to the influence of, for example, edges.

As described above, in the classification adaptive noise elimination circuit 12, the noise is removed without depending on the stillness or the motion of the image, but the information of the long frame can be accumulated for the complete still portion. It is inferior to a possible motion adaptive recursive filter.

In the present invention, as described above, the output of the motion adaptive recursive filter is selectively output in the stationary portion, and the output of the class adaptive noise removal circuit is selectively output in the motion portion. In any of the stationary portions, an image signal output from which noise has been successfully removed can be obtained.

Note that the class taps and prediction taps in the first area cutout section 26 and the second area cutout section 27 in the description of the class classification adaptive elimination circuit are merely examples, and it goes without saying that they are not limited to these. In the above description, the structure of the class tap and the structure of the prediction tap are the same, but they need not be the same structure.

In the above description, the noise component detector 28 uses a one-bit ADRC encoding circuit. However, as described above, a multi-bit ADRC encoding circuit may be used. A circuit may be used.

Further, in the above description, the selection of the output of the motion adaptive recursive filter 11 and the output of the classification adaptive noise elimination circuit 12 is performed on a pixel-by-pixel basis. The selection may be performed in units of a pixel block or an object including a predetermined number of pixels, or in units of frames. In these cases, the static / movement determination circuit performs the static / movement determination in units of selection.

In the above example, the output of one motion adaptive recursive filter and the output of one class classification adaptive elimination circuit are selected as alternatives. However, the motion adaptive recursive filter and / or the class It is also possible to provide a plurality of noise removing circuits based on the classification adaptive processing and select an output image signal from them.

[0089]

As described above, according to the present invention, in the stationary portion, the output of the first noise elimination circuit having a large noise elimination effect for the stationary portion such as a motion adaptive recursive filter is selectively output, and In the part, the output of the second noise elimination circuit capable of removing noise in the moving part such as the class classification adaptive noise elimination circuit is selectively output, so that the noise can be removed well in both the moving part and the still part of the image. Is obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of a noise elimination device according to the present invention.

FIG. 2 is a diagram showing a configuration example of a class classification adaptive noise elimination circuit used in the embodiment.

FIG. 3 is a diagram for explaining a class classification adaptive noise elimination circuit.

FIG. 4 is a diagram illustrating a configuration example of a noise component detection circuit forming a part of a class classification adaptive noise removal circuit.

FIG. 5 is a diagram for explaining a configuration example of a noise component detection circuit in FIG. 3;

FIG. 6 is a diagram for explaining a method of creating coefficient data used in a class classification adaptive noise elimination circuit.

FIG. 7 is a diagram illustrating a configuration example of a motion adaptive recursive filter.

[Explanation of symbols]

1, 4 ... Amplifier for amplitude adjustment, 2 ... Addition circuit, 3 ... Frame memory, 5 ... Subtraction circuit, 6 ... Absolute value conversion circuit, 7 ... Threshold processing circuit, 8 ... Weight coefficient generation circuit, 11 ... Motion Adaptive recursive filter, 12: Classification adaptive noise elimination circuit, 13: Output selection circuit, 14: Static / dynamic judgment circuit

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Yasushi Node 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Katsunao Shinmei 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo No. Sony Corporation F term (reference) 5C021 PA57 PA66 PA67 PA79 RA01 RB06 YA01 5C059 KK01 LA00 MA28 NN01 NN23 PP04 TA80 TB04 TC02 TC05 TC13 TD02 TD05 TD11 TD13

Claims (16)

[Claims] A frame memory for storing an image signal; an image signal stored in the frame memory;
The input image signal is added by performing weighting according to the static movement of the image based on the input image signal, and rewriting the image signal of the frame memory by the added output, thereby removing the noise as the added output. A first noise removing unit for generating one output image signal; extracting a corresponding pixel on an image between a plurality of frames; and classifying a noise component of the pixel based on a change between the frames of the pixel. A second noise removing unit configured to generate a second output image signal from which a noise component has been removed from the input image signal by an arithmetic process set in advance corresponding to the classified class; A static motion of the image is determined in pixel units, and one of the first output image signal and the second output image signal is determined in accordance with the determination result in the predetermined number of pixel units. Noise removal device, characterized in that it comprises an output selection unit which selects and outputs. 2. The image processing apparatus according to claim 1, wherein the output selection unit includes: a determination unit configured to determine whether the predetermined number of pixels is a static portion or a moving portion of the image; 2. The noise according to claim 1, further comprising: a selector configured to select and output the first output image signal, and to select and output the second output image signal for a pixel in a moving portion. Removal device. 3. The method according to claim 1, wherein the determining unit calculates, for each of the predetermined number of pixels, a difference value between the first output image signal and the second output image signal. When the absolute value of the difference value is equal to or larger than the threshold value, a determination value indicating that the pixel is the moving portion is output based on a comparison result between the absolute value and a preset threshold value. 3. The noise elimination apparatus according to claim 2, further comprising: a comparison unit that outputs a determination value indicating that the pixel is a still part when the absolute value of the difference value is smaller than the threshold value. apparatus. 4. A motion judging section for judging a still or static state of an image based on the input image signal, a first noise removing section, and the input image signal and the A weighting unit that weights the image signal stored in the frame memory; and an adding unit that adds the weighted input image signal and the image signal from the frame memory. The noise eliminator according to claim 1, wherein は is rewritten into an image signal from the adder. 5. A motion information deriving unit for deriving motion information for a pixel of interest in an image based on the input image signal, wherein the second noise removing unit includes: A class tap extracting unit that extracts, as a class tap, a plurality of pixels at positions corresponding to the target pixel, for a plurality of frames, based on a feature of the class tap extracted by the class tap extracting unit. A noise class for the pixel, a class classifying unit that classifies the class, and, based on the class classified by the class classifying unit, a calculation process corresponding to the class is determined. The noise removing apparatus according to claim 1, further comprising: an arithmetic processing unit configured to generate an image signal from which a noise component has been removed. 6. The apparatus according to claim 5, wherein the feature of the class tap used in the class classification unit is a noise component distribution of the plurality of pixels as the class tap. 7. The arithmetic processing unit, comprising: pixel values of a plurality of pixels at positions corresponding to the target pixel;
By performing an operation with an operation coefficient for the plurality of pixels set in advance according to the class classified by the class classification unit, an image signal from which the noise component for the pixel of interest has been removed is generated. The noise removing device according to claim 5, wherein 8. The arithmetic coefficient used in the arithmetic processing unit according to each of a plurality of classes classified by the class classification unit, wherein a pixel of interest is extracted from teacher image data having less noise than the input image signal. And, from student image data having noise equivalent to the input image signal, deriving motion information about the pixel of interest, according to the motion information derived for the pixel of interest, Extracting, from the student image data, a plurality of pixels at positions corresponding to the pixel of interest as class taps; classifying a noise component of the pixel of interest based on characteristics of the class tap; Corresponding to the class, extracted at least as class taps from the student image data as prediction taps, Deriving a prediction coefficient for generating an output image signal of the same quality as the teacher image data from the student image data for a pixel including a plurality of pixels at a position corresponding to the target pixel extracted by The noise removal apparatus according to claim 7, wherein the noise reduction apparatus calculates the prediction coefficient. 9. An image signal stored in a frame memory and an input image signal are added by performing weighting in accordance with static movement of an image based on the input image signal, and the added image is output from the frame memory based on the added output. By rewriting the signal, a first noise removing step of generating a first output image signal from which noise has been removed as the addition output, and as a process parallel to the first noise removing step, a plurality of frames, The corresponding pixels on the image are extracted, the noise components of the pixels are classified into classes based on the change between the frames of the pixels, and the input processing is performed by a preset arithmetic processing corresponding to the classified class. A second noise removing step of generating a second output image signal from which a noise component has been removed from the image signal; and determining a static motion of the image in units of a predetermined number of pixels; And an output selection step of selecting and outputting one of the first output image signal and the second output image signal in units of the predetermined number of pixels. Noise removal method. 10. The output selecting step includes: a determining step of determining whether the predetermined number of pixels is a stationary part or a moving part of an image; and, based on a determination result in the determining step, determining a pixel of a stationary part. Selecting and outputting the first output image signal;
10. The noise removing method according to claim 9, further comprising a selecting step of selecting and outputting the second output image signal for a pixel in a moving portion. 11. A difference value calculating step of calculating a difference value between the first output image signal and the second output image signal for each of the predetermined number of pixels; Based on a comparison result between the absolute value of the difference value calculated in the step and a preset threshold value, if the absolute value of the difference value is equal to or larger than the threshold value, And a comparing step of outputting a determination value indicating that there is a pixel, and when the absolute value of the difference value is smaller than the threshold value, outputting a determination value indicating that the pixel is a still part. The noise removal method according to claim 9. 12. The first noise removing step includes: a motion judging step of judging a still / moving state of an image based on the input image signal; A weighting step of weighting the image signal stored in the frame memory; and an adding step of adding the weighted input image signal and the image signal from the frame memory. 10. The method according to claim 9, wherein? Is replaced with a pixel signal from the adding step. 13. A motion information deriving step for deriving motion information for a target pixel in an image based on the input image signal, wherein the second noise removing step includes: A class tap extracting step of extracting a plurality of pixels at positions corresponding to the target pixel as class taps for a plurality of frames, based on the feature of the class tap extracted in the class tap extracting step. A noise class for the pixel, a class classification step of classifying, and, based on the class classified by the class classification step, a calculation process corresponding to the class is determined. 10. An arithmetic processing step of generating an image signal from which a noise component has been removed. The method of noise removal. 14. A feature of the class tap used in the class classification step is a noise component distribution of the plurality of pixels as the class tap.
3. The noise removing method according to 3. 15. In the arithmetic processing step, pixel values of a plurality of pixels at a position corresponding to the target pixel,
By performing an operation with an operation coefficient for the plurality of pixels set in advance according to the class classified in the class classification step, an image signal from which the noise component for the pixel of interest has been removed is generated. The method according to claim 13, wherein the noise is reduced. 16. The method according to claim 16, wherein the operation coefficient corresponding to each of the plurality of classes classified in the class classification step includes: extracting a pixel of interest from teacher image data having less noise than the input image signal; Deriving motion information about the pixel of interest from student image data having noise equivalent to a signal; and responding to the motion information derived for the pixel of interest, the pixel of interest from the student image data for a plurality of frames. Extracting a plurality of pixels at positions corresponding to the class tap as class taps; and classifying a noise component of the pixel of interest based on characteristics of the class taps. The note extracted as at least a class tap extracted from the student image data as a prediction tap. Deriving a prediction coefficient for generating an output image signal of the same quality as the teacher image data from the student image data for a pixel including a plurality of pixels at positions corresponding to the pixel, thereby calculating the prediction coefficient. The method according to claim 15, wherein: