WO1998011718A1

WO1998011718A1 - Noise reduction process for image signals

Info

Publication number: WO1998011718A1
Application number: PCT/DE1997/001859
Authority: WO
Inventors: Aishy Amer; Hartmut Schröder; Udo Bremer
Original assignee: Siemens Aktiengesellschaft
Priority date: 1996-09-11
Filing date: 1997-08-27
Publication date: 1998-03-19
Also published as: DE19636952C2; DE19636952A1

Abstract

In a noise reduction process for image signals with a recursive filter (3, 4, 5), the fedback image signals (G*) are converted on the basis of the motion vector into the motion phase of the input image signals (S). In the event of conversion errors, recursion is appropriately reduced (k, 1-k). Local noise reduction (29) is provided for revealed areas and for a release mode.

Description

description

_V out for noise reduction of an image signal

The invention relates to a method for noise reduction of an image signal according to the preamble of patent claim 1.

In conventional noise reduction methods for video signals, as described, for example, in EP-A-0 581 059, temporal, recursive filtering of the image signal is carried out. For this purpose, the noise-reduced image signal on the output side is delayed in an image memory and, after weighting with a factor 1-k, is additionally superimposed on the input-side image signal, the latter being weighted with the factor k. The filter has low pass characteristics. The filtering affects not only the noise component contained in the video signal, but also the pixel information. The low-pass filtering over time leads to a smoothing of motion. To remedy this, it is proposed to influence the weighting factor k depending on the movement. This is usually done in such a way that the feedback component is reduced and the proportion of the input signal in the output signal is thereby increased. However, this also reduces the noise reduction with increasing movement. If the movement is sufficiently large and slow so that the eye can follow the moving object, the noise disturbance present within the moving surface remains perceptible.

The object of the invention is to provide a method for noise reduction of an image signal, which provides better results when there is movement in the image signal.

This object is achieved by a method according to the features of patent claim 1. _V orteilhafte embodiments and further developments of the OF INVENTION _d ungsgemäßen method are specified in the dependent claims.

_In the method according to the invention, the image content of images of the feedback image signal is converted to the movement phase of the images of the input image signal using motion vectors. The detection areas created by the movement are replaced in the feedback path by image portions of the input image, which are subjected to a local noise reduction. To change the weighting actuator k or 1-k, an incorrect movement determination is taken into account. Advantageously, smaller errors lead to a disproportionately larger weighting factor k, that is to say a disproportionately higher weighting of the input path compared to the feedback path. For large errors in the motion-compensated feedback image, a fallback mode is provided in which only the image on the input side is locally reduced in noise. Overall, the method works in a motion vector error-tolerant manner with non-linear filter control and a non-linear fallback mode which is influenced by the threshold value. To correct the motion vector field, it is advantageous to carry out a homogenization of the motion vectors within the moving object. If the original and homogenized motion vector is used in parallel, two vectors are optionally available, of which the one that delivers the lower error is selected for motion compensation.

The invention is described in more detail below with reference to the figures shown in the drawing. Show it:

FIG. 1 shows a circuit diagram for carrying out the method according to the invention,

Figure 2 characteristic curves for the reduction factor k and _F igur ₃ is a diagram considering a homogenization ^¬ overbased motion vector.

_The basic circuit diagram of FIG. 1 has an input connection 1, to which an image signal S is fed, the

Is to reduce oise _R. _D as noise-reduced image signal G is present at an output terminal. 2 The image signals S, G comprise a sequence of images, for example zeilenver- _k ammten fields or frames for noise reduction, a recursive filtering is performed. This means that the output signal G - as described in more detail below - is delayed frame-wise and is linked to the input channel by means of an adder 3. The fed-back image signal is weighted in a multiplier 4 with the factor 1-k, the signal S in the input channel is weighted in a multiplier 5 with the factor k. By changing k, the relation of the input path to the feedback path is set. At the edge of the value range of k at k = 1, the feedback is prevented, so that only the image signal S on the input side is forwarded. A memory device 20 is provided for the image-wise delay of the noise-reduced image signal G on the output side within the feedback loop. The embodiment shown relates to image signals S, G with interlaced fields. The device 20 then has two field memories 21, 22 which are connected in series n rows. The image memory 22 then contains a field delayed twice by one field duration, which is supplied as the image signal G * via a connection 23 to the multiplier 4 of the feedback path. The fields fed back to adder 3 for linking have the same raster position in comparison to the field S on the input side.

The fed back fields are motion-compensated depending on the existing movement. This means that the image content of the respective fields is converted to the _j enige movement phase containing the corresponding field _d it ingangssignalε _E S comprises the time of the link in the adder. ₃ For this purpose, the motion information is onnection at a _A 7 fed. A signal D is present at connection 7, _{which contains} a motion vector field for the input field. _T his means that for each pixel the displacement rela _t iv to the position of the corresponding Bildpu kts is indicated in the previous field. Motion vector fields can be obtained in various conventional ways. For example, a motion vector field can be determined on the receiver side by forming the spatial difference of the pixels of two fields. The motion vector field can, however, also be transmitted on the transmitter side, which is the case in particular in the case of motion-based coding methods by means of prediction, for example according to the MPEG standard. The motion vector field can contain a motion vector for each pixel or for image blocks composed in blocks. The former variant is used in the exemplary embodiment.

In addition, detection areas which result from the different movement of objects in an image are taken into account in the method according to the invention. If there is a detection area, this is indicated by a corresponding identifier for each pixel or for each block by a signal F at connection 6.

The motion compensation, i.e. the conversion of the feedback signal G to the movement phase on the input side by means of the motion vectors present in the motion vector field is carried out by means of the image memories 21, 22. These enable motion-compensated image storage of

Pixels m as a function of the displacement vectors in the vector field D. The image memory is written nonlinearly with the pixels of the noise-reduced output signal G to be compensated as a function of the respective displacement vectors of the vector field D. The pixels are written to the position in the image memory that is assigned to them by the motion vector. The motion compensated _t en _image points are read out linearly. The image memory therefore has a partial image memory 21a which is designed as an image memory with random access. The large _beta e of this section is determined by the maximum possible _V ektorverschiebung. On the output side, a memory part 21b is provided which can be linearly addressed. The image memory 22 is organized accordingly.

_A multiplexer 24 or 25 is provided on the output side of the image memories 21, 22. The output of the respectively assigned image memory and correspondingly prepared pixels of the input image signal S are fed to the multiplexer. The switching control of the multiplexers 24, 25 takes place by means of the signal F m as a function of the detection areas. If there is a detection area, the pixels of this detection area are filled with pixels from the preferably locally noise-reduced input image S. The pixels fed to the multiplexer 25 at the connection 26 are already in the correct grid. The pixels supplied to the multiplexer 24 at the connection 27 are to be converted by a linear raster interpolation 28 to the raster of the field read out from the image memory 21. For raster interpolation, averaging of the image lines immediately adjacent to the image line to be generated is expediently suitable.

The pixel signals fed to the raster interpolation 28 or the connection 26 are preferably generated from the input signal S by a local noise reduction 29. In contrast to a temporal feedback, this noise reduction only works locally within a single field of the input signal S. It is expedient to calculate a centrally weighted average value with pixels of a neighborhood including the pixel currently being viewed that are homogeneous with the input pixel. The neighborhood pixels are those in horizontal, vertical and in two diagonal directions ver _t ei t _l are. If the absolute difference of a neighboring point _b i _ld of a neighborhood to the central center pixel within a predetermined Wertbereichε is, this means that this neighbor pixel not beyond an edge _l IEGT It is called homogeneous. That is determined from the four directions, the two Nachbarbild- points are homogeneous and that of the homogeneity is optimum for all four directions with respect to this direction ge _f reasons, so ng the neighboring sheep m that direction ex- pandiert until either a maximum defined size of the

Neighbor sheep or an inhomogeneity is achieved. Those pixels that lie within the expansion range in the direction found are center-weighted If no direction is found to be homogeneous, the neighborhood is searched for as large a number of similar points as possible and these are averaged with the weighted current point If no homogeneous neighboring point is found, the current input image point is output unchanged from the device 29 so that the sharpness of the image is not reduced. The central weighting of the image points is expediently carried out as a function of an estimated noise power described below.

The problem with the displacement vectors is the accuracy. Practically, the motion vectors are with one

Provide errors in the amount and / or direction A device 30 takes account of incorrect motion vectors in order to reduce their negative effects on the image quality. An error measurement is carried out by determining the difference between the field on the input side and the feedback, motion-compensated field G * at the output 23 of the memory device 20 in a subtractor 31. The difference signal is rectified in a device 32 and smoothed in a low-pass filter 33, on which the error signal V to be processed is present on the output side. Since the input image signal as the reference signal is subject to the noise disturbance to be reduced, the deviation becomes zwec preferably not _k separately for each pixel, son ^¬ _d ren picture _b ereichsweise determined. The area is central to those pixel to be arranged, the noise is reduced just we _d. The measurement of the deviation takes place as a linear development center-he _d values of the differences between the Referenzbildpun _kt s and the collocated compensated pixels. The error signal V is evaluated in a device 34 with a characteristic curve. The weighting factors _k and 1-k result from this. Since the measured deviation only depends on the image signals themselves, motion vector errors are only corrected if there is a pixel error. Motion vector errors that do not produce a pixel error have no visible effect.

The characteristic curve formed in block 34 is expediently also made dependent on the noise power contained in the input image sequence S. In order to be able to process the largest possible noise power range from, for example, 25 dB to 50 dB signal-to-noise ratio, the amplitude of the evaluation characteristic curve of block 34 must be adjusted. A device 35 therefore carries out a noise level estimation with respect to the input image signal S. Homogeneity considerations are carried out, the pixels lying within a neighborhood being regarded as the more homogeneous the less the amplitude of the neighboring pixels deviate from the center pixel. The homogeneity measures determined in this way for a large number of smaller neighborhoods are then summed up and standardized within a larger neighborhood. The larger neighborhood that was determined to be the most homogeneous is used to calculate the noise power contained. The characteristic curve is made dependent on the noise level estimate determined in this way.

The characteristic curve basically has a negative quadratic course according to the mapping rule k (V) = 2V-V ² . To take into account the noise power, both the amplitude as well as the error signal V depending on the noise power determined. In addition to the basic curve 34a, FIG. 2 shows two curve curves 34b and 34c which are different depending on the noise level estimate. The block 34 executes the characteristic curve shown in FIG. 2, a reduction factor k being taken directly from an input value of the error! Signal V by means of the noise-dependent characteristic curves 34a, b, c.

A fallback mode is advantageously provided. This is activated when the error signal V exceeds a threshold value. Then, instead of the temporally noise-filtered signal output by the adder 3, the locally noise-filtered signal output by the device 29 is switched through to the noise-reduced image signal G at the output 2. For this purpose, a multiplexer 40 is provided, to which the output of the adder 3 or the output of the device 29 is alternatively fed and which is connected on the output side to the connection 2. The multiplexer 40 is controlled by a threshold detector 41, to which the error signal V is fed.

The threshold value detector 41 is preferably also controlled by the detection signal F. If the error signal V exceeds the threshold value within a detection range, the temporally noise-reduced signal is nevertheless preferred and the output of the adder 3 is switched through in the multiplexer 40. This takes into account that the high error indicated by the error signal V is essentially due to the local noise reduction in the detection area.

The motion-compensated image storage device 20 can also be designed for the temporary storage and motion compensation of progressive images. For this purpose, the image memories 21, 22 are connected in parallel, in that the image memory 21 receives the upper half of the progressive image, the image memory 22 the lower half of the progressive image. A raster interpolation 28 is not required.

Since erroneous motion vectors generate compensation errors m of the image storage device 20 and thus a high error signal V and a noise reduction correspondingly reduced by the reduction factor k, it is advantageous to carry out a homogenization of the motion vectors within objects. For this purpose, the image of the input image signal S just processed is bmarized, that is to say the brightness pixel values of the corresponding input image are compared with a threshold value, so that a binar image is generated. The binarization is not sensitive to noise. Edges are determined from the binary image by a shape-preserving edge detection. The edges are then linked together to form closed contours. Objects can be reconstructed from this. An output signal generated by the reconstruction indicates the object affiliation of the pixels of a neighborhood of the current pixel to the object to which this current pixel belongs. The

Whether ect membership also applies to the corresponding displacement vectors.

Since the majority of motion vectors within an object can be assumed to be correct and it can be expected that only some of the motion vectors are defective, especially in the peripheral areas of objects, the minority of incorrect vectors can be corrected without error if they refer to the majority motion vector within the Object boundaries adjusted, eg replaced. As the size of an object decreases, the reliability of determining a correct majority motion vector decreases. Faulty vectors can be determined by object-based vector homogenization because, for example, a faulty majority vector is calculated on the basis of a too small object size. It is therefore advisable to use an error signal both for a motion vector that is actually fed with the motion compensated image and for an image motion compensated with the majority motion vector. The motion-compensated image signal that supplies the minimum error signal is used for the time filtering for feedback.

A circuit section taking into account the use of a homogenized motion vector is shown in FIG. 3. The motion-compensated memory 40, which corresponds to the memory 20 in FIG. 1, generates two motion-compensated output image signals G * 1, G * 2, one of which is motion-compensated according to FIG. 1 by the motion vector field D, the other by the homogenized motion vector DH. In devices 41, 42, which correspond to elements 31, 32, 33 of FIG. 1, an error signal VI or V2 is calculated in each case by forming the difference with the image signal S on the input side. The minimum error signal which corresponds to the error signal V of FIG. 1 and is supplied to the movement device 34 is determined in a device 43 from the error signals VI, V2. Depending on the minimum error signal, a switch 44 is controlled, which forwards that of the image signals G * l, G * 2 as signal G * to the recursive filter 3, 4, 5, which has provided the minimum of the error signals VI and V2.

Claims

claims

1. Method for noise reduction of an image signal, in which the input-side image signal (S) comprises a sequence of images, a noise-reduced output-side image signal (G) is delayed by the duration of at least one image and the delayed noise-reduced image signal (G *) with the input-side image signal (S) is linked (3), characterized in that the images of the delayed image signal (G *) are corrected as a function of movement existing between images of the input-side image signal (S).

2. The method according to claim l, characterized in that the image content of images of the delayed noise-reduced image signal (G *) on the movement phase of those images of the input image signal (S) with which the images of the delayed image signal (G *) are linked, depending - Speed of motion information (D) present between images of the input image signal.

3. The method according to any one of claims 1 or 2, characterized in that pixels in detection areas (F) are determined, which are generated by the movement between the images, that these pixels for motion-dependent correction in the delayed image signal (G *) by a local noise reduction (29) carried out on the images of the input image signal.

4. The method according to any one of claims 1 to 3, characterized in that before the link (3) a weighting of the input-side image signal (S) with a first factor (k) and a weighting of the delayed image signal (G *) with one of the first linear dependent second factor (l - k) is carried out that an error signal (V) is generated by forming the difference (31) between the input-side image signal (S) and the delayed image signal (G *), depending on which the factors <k, 1 - k) are controlled.

5. The method of claim 4, d a d u r c h g e k e n n z e i c h n e t that the first factor (k) n depending on the error signal (V) for smaller values of the error signal (V) is controlled disproportionately larger than for larger values of the error signal (V)

6 Method according to one of claims 4 or 5, so that the noise power (35) is determined within one of the images of the input image signal (S) and the factors (k, 1 - k) are controlled as a function thereof.

7. The method according to any one of claims 4 to 6, d a d u r c h g e k e n n z e i c h n e t that depending on a comparison of the error signal (V) with a

Threshold (41) the linkage of the input-side image signal (S) with the delayed image signal (G *) is switched off and that the noise-reduced output signal (G) is formed by a signal consisting of only one of the images of the input-side image signal performed local noise reduction (29) is obtained

8. The method according to any one of claims l to 7, characterized in that a detection of objects in the images of the input image signal (S) is carried out, that a motion motion vector is determined for motion vectors of pixels within one of the objects and that this motion - vectors are replaced depending on the Ma oritäts movement vector by a homogenized movement vector (DH).

9. The method according to claim 8, characterized in that for object recognition a binarization of the respective image is carried out by comparing the uminance values of each pixel with a threshold value, that brightness edges are determined in the binarized image and that objects are determined by concatenating the edges.

10. The method according to claim 8 or 9, if referred to one of claims 4 to 7, characterized in that a further delayed image signal (G * 2) is generated which is movement-compensated by means of the homogenized motion vector (DH) that the error signal (V) is determined as a minimum from further error signals (VI, V2) which are formed by forming the difference between the input-side image signal (S) and the delayed image signal (G * l) or the further delayed image signal (G * 2) , and that the delayed image signal (G * l or G * 2) providing the minimum error signal (V) is linked to the input-side image signal (S).

CHANGED REQUIREMENTS

[Received at the International Bureau on January 16, 1998 (January 16, 1998); original claims 1-10 replaced by amended claims 1-7 (3 pages)]

1. A method for noise reduction of an image signal, in which the input-side image signal (S) comprises a sequence of images, with the following features:

a noise-reduced output-side image signal (G) is delayed by the duration of at least one image,

- By forming the difference (31) between the input-side image signal (S) and the delayed image signal (G *), an υ error signal (V) is generated,

the noise power (35) is determined within one of the images of the input image signal (Ξ),

- A weighting of the input image signal (S) with a first factor (k) and a weighting of the delayed

15 noise-reduced image signal (G *) with a second factor (1 - k) which is dependent on the first linear is carried out, the factors (k, 1 - k) being controlled as a function of the error signal (V) and as a function of the determined noise power (35) 0 - the delayed noise-reduced image signal (G *) weighted by the second factor (1-k) is linked to the input-side image signal (S) weighted by the first factor (k) (3),

- The image content of images of the delayed noise-reduced image signal (G *) weighted by the second factor (1-k) 5 is based on the movement phase of those images of the input-side image signal (S) with which the images of the image factor (1-k ) weighted delayed image signal (G *) are linked, depending on the movement information (D) present between images 30 of the input-side image signal.

35 2. The method according to claim 1, characterized in that the first factor (k) m as a function of the error signal (V) is controlled disproportionately larger for smaller values of the error signal (V) than for larger values of the error signal (V).

5 3. Method according to one of claims 1 or 2, characterized in that pixels m detection areas (F) are determined, which are generated by the movement between the images, that these pixels for movement-dependent correction U ture in the delayed image signal (G * ) are replaced by a local noise reduction (29) carried out in the images of the input-side image signal.

4. The method according to any one of claims 1 to 3, 5 characterized in that depending on a comparison of the error signal (V) with a threshold (41), the linkage of the input image signal (S) with the delayed image signal (G *) is switched off and that the noise-reduced output signal (G) is formed by a signal which is obtained by local noise reduction (29) carried out in only one of the images of the input-side image signal.

5. The method according to any one of claims 1 to 4, characterized in that a detection of objects in the images of the input-side image signal (S) is carried out, that for motion vectors of pixels within one of the objects a Ma oritatsungsvectorvector is determined and that this motion - Vectors depending on the Ma πtatsbewegungvektor be replaced by a homogenized motion vector (DH).

6. The method according to claim 5, characterized in that for object detection a bmarization of the respective image is carried out by comparing the luminance values of each pixel with a threshold value that Hel- in the binarized image Liability edges are determined and that objects are determined by concatenating the edges.

7. The method according to any one of claims 1 to 6, characterized in that a further delayed image signal (G * 2) is generated, which is motion-compensated by means of the homogenized motion vector (DH) that the error signal (V) as a minimum of further error signals (VI, V2) is determined, which are formed by forming the difference between the input-side image signal (S) and the delayed image signal (G * l) or the further delayed image signal (G * 2), and that this is the minimum error signal (V) delivering delayed image signal (G ^ l or G * 2) is linked to the input image signal (S).