JP4572606B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus that detects a motion vector from an image signal and performs image processing using the motion vector.

  For example, transmitted or reproduced image data or audio data generally includes temporally varying noise, but there are various methods for removing noise included in the data.

  J. et al. S. As described in Lim's book, “Two Dimensional Signal Image Processing” (Prentice Hall, 1990, page 568 et seq.) (See Non-Patent Document 1), temporal filtering is most simply frame averaging. Is done by Frame averaging is very effective for processing continuous video frames that are contaminated by random noise, although there is not much change in video information between frames.

  As is known, there are various methods for frame averaging. Frame averaging is very simple and effective, but accurate recording of signals between frames is essential. In actual applications such as video and TV, the video can change from frame to frame. A part of the image may move due to parallel movement, rotation, change in size, or a combination thereof. In some prior art systems, frame averaging is applied only to static parts in any video, i.e., where there is no motion between frames. Another prior art system attempts to compensate for this motion by estimating the motion of the video from one frame to the next and performing frame averaging. In order to perform this motion compensated video restoration, video frames are averaged along the approximate motion trajectory.

  J. et al. M.M. Boyce's paper, "Noise Reduction of Image Sequences Using Adaptive Motion Compensated Frame Averaging", IEEE ICAS SP-92, III-461 to III-464 (non-patent document 2) In addition, a method for reducing noise in a video sequence has been proposed by appropriately switching between motion compensation type frame averaging performed in units of blocks. In particular, the displacement frame averaging method (motion compensation type averaging) and the non-displacement averaging method (simple frame averaging) are appropriately switched based on relative differences between two blocks caused by noise and motion, respectively. A method is disclosed. Displacement frame averaging is performed on blocks containing moving objects, and no displacement occurs in blocks where the difference between frames is due solely to noise.

  Another way to achieve noise reduction is T.W. J. et al. Dennis's paper, “Nonlinear Temporal Filter For Television Picture Noise Reduction”, IEEE Proceedings, Vol. 127, Pt. G, No. 2, April 1980, page 52 onwards (see Non-Patent Document 3). More specifically, a conventional inter-frame recursive low-pass filter for a 5.5 MHz monochrome television with 625 scanning lines is modified so that the attenuation of the frame difference immediately depends on its amplitude. This reduces the large area spatial interference such as streaking under the condition that the filter does not include zero frequency or frame frequency components. However, with this method, spatial degradation may occur in a portion of the video that includes motion.

  Of these, motion detection as used in the motion compensation type averaging method is used not only for noise removal but also for interlace-progressive conversion and frame rate conversion.

  Motion detection is generally performed by taking an inter-frame difference, and there is a method of detecting a motion vector as a method of detecting finer information with respect to the motion.

  As a method of detecting this motion vector, the sum of absolute differences (or sum of squares of differences) between corresponding pixels is calculated for the processing block of the current frame and the prediction block of the reference frame. There is a way to estimate. As this procedure, a full search procedure for performing the sum of absolute differences for all block patterns in the search area of the reference frame, a three-step search procedure for simply detecting a motion vector in three steps, and the like are common. It is.

Patent Document 1 and the like also describe a method for detecting this motion vector.
JP-A-7-123416 Prentice Hall, 1990, p. 568- IEEE ICASSP-92, III-p. 461-III-p. 464 IEEE Proceedings, Vol. 127, Pt. G, No. 2, April 1980, p. 52-

  However, in motion detection and motion vector detection, since motion is detected for noise components, motion vectors are detected even for noise that does not exist in the original image. However, there is a problem that the accuracy of the motion vector is lowered as a whole.

  The present invention provides a method of comparing a plurality of detected detection results using a plurality of image data from which noise has been removed by a plurality of noise removal circuits of different methods and outputting a highly accurate motion vector. Objective.

  In order to solve the problem, the image processing apparatus of the present invention uses a plurality of image data from which noise is removed by a plurality of noise removal circuits of different methods, and compares a plurality of detected detection results to detect the detected result. By weighting motion vectors or using only motion vectors common to each detection result, only motion vectors with reliability data or highly accurate motion vectors are output. .

  As a result, the priority of the detected motion vector with respect to the noise component is lowered or the erroneously detected motion vector is reduced. Therefore, the adverse effect of noise is reduced in image processing using this motion vector.

  As described above, according to the present invention, when detecting a motion vector in image processing, it is possible to obtain an advantageous effect of reducing erroneous detection of a motion vector due to noise and improving the accuracy of the motion vector.

  According to a first aspect of the present invention, in an image processing apparatus that processes an input image signal and outputs an output video as a result of the processing, a plurality of means having different methods for removing noise from the input image signal, and the noise A means for detecting a motion vector for each of the image signals from the plurality of means for removing, and a means for comparing and selecting a plurality of detection results output from the means for detecting the motion vector and outputting a motion vector. And has the effect of outputting a highly accurate motion vector with reduced results of erroneous detection of noise components.

  According to a second aspect of the present invention, in an image processing apparatus that processes an input image signal and outputs an output video as a result of the processing, a plurality of means having different methods for removing noise from the input image signal, and the noise A plurality of detection results output from the means for detecting motion vectors and the means for detecting the motion vectors are compared with respect to each of the image signals from the plurality of means to be removed, and only the vectors existing in any of the detection results are compared. And a means for outputting a motion vector, and has a function of outputting a motion vector in which a false detection result with respect to a noise component is reduced.

  According to a third aspect of the present invention, in an image processing apparatus that processes an input image signal and outputs an output video as a result of the processing, a plurality of means having different methods for removing noise from the input image signal; A means for detecting a motion vector for each of the image signals from a plurality of means for removal is compared with a plurality of detection results output from the means for detecting the motion vector, and the motion vector of the detection result is compared. , Including means for weighting and outputting a motion vector included in the detection result and information indicating the reliability of the motion vector, and a motion vector accompanied by reliability data is output. Therefore, it has the effect of reducing the influence of the erroneously detected motion vector on the noise component.

  A fourth aspect of the present invention is an image processing apparatus having means for removing noise using a high-pass filter and means for removing noise by a frame averaging method as means for removing noise from the input image signal. .

  The fifth aspect of the present invention performs frame interpolation using the motion vector output by the above image processing apparatus, and has the effect of reducing frame interpolation errors due to noise.

  According to a sixth aspect of the present invention, motion compensation type noise removal is performed using the motion vector output by the image processing apparatus described above, and the compensation type is used by using a motion vector with less false detection due to noise. It has the effect of removing noise.

(Embodiment 1)
Hereinafter, embodiments of the present invention will be described with reference to FIGS.

  FIG. 1 is a block diagram showing a schematic configuration of an image processing apparatus according to an embodiment of the present invention.

  The image processing apparatus in FIG. 1 uses a high-pass filter type noise removal circuit 1 that removes noise using a high-pass filter, and an input image signal and an image signal stored in the frame memory 6 using a frame averaging method. A frame average noise removing circuit 2 for removing noise, a motion vector detecting circuit 3 and 4 for detecting a motion vector, an AND circuit 5 for outputting only a vector at either input at two inputs, and image processing Frame memories 7 and 8 for storing image signals, an interpolation frame generation circuit 9 for generating an interpolation frame from the image signals and motion vectors, and an interpolation frame for inserting the interpolation frame into the original signal and performing frame rate conversion An insertion circuit 10 is provided.

  In the above description, the example in which only the common part of the motion vectors is output from the AND circuit 5 has been described. However, all other motion vectors detected by changing the other parts of the AND circuit 5 to the motion vector comparison circuit are used. Can be similarly applied to a configuration that can output reliability data of the motion vector.

  In the present embodiment, noise is removed from the input image signal S1 by the high-pass filter type noise removal circuit 1, and the motion vector 11 is detected from the obtained image signal by using the motion vector detection circuit 3. On the other hand, noise is removed from the input image signal S1 by the frame average noise removal circuit 2, and the motion vector 12 is detected from the obtained image signal by using the motion vector detection circuit 4. The motion vector 11 and the motion vector 12 are input to the AND circuit 5 to output a motion vector 13 in which only the motion vectors existing in both motion vector information are selected. An interpolation frame generation circuit 9 generates an interpolation frame from the motion vector 13 and the original image signal stored in the frame memory 7, and an interpolation frame insertion circuit 10 is generated from the interpolation frame and the original image signal stored in the frame memory 8. , Frame insertion is performed and frame conversion is performed.

  In this embodiment, as shown in FIG. 2, the image in which the object A is moving slightly downward includes a plurality of noise components, and is stored in the input image signal S1 for two frames and the frame memory 6. The N-2 frame image signal is used as the input image signal S1. FIG. 3 shows the image output when the image signal as shown in FIG. 2 is input to the high-pass filter type noise removal circuit 1 of FIG. 1 and the output of the motion vector detection circuit 3 of FIG. FIG. 4 shows the image output when the image signal as shown in FIG. 2 is input to the frame average type noise removal circuit 2 of FIG. 1 and the output of the motion vector detection circuit 4 of FIG.

  When an image signal as shown in FIG. 2 is input to the image processing apparatus shown in FIG. 1, noise having a small luminance change such as noise B in FIG. 2 is cut in the high-pass filter type noise removal circuit 1 shown in FIG. Noise such as noise C having a large change is not removed. Therefore, an image signal as shown in FIG. 3 is output. Then, since the motion vector is detected from the image signal as shown in FIG. 3, the motion vector A for the object A and the motion vector C for the noise C in FIG. 3 are detected.

  On the other hand, in the frame average type noise elimination circuit of FIG. 1, in order to extract only signals that exist in common in the previous and subsequent frames, noise that does not exist at the same position, such as noise C, is removed. Of these, the common part between the two frames remains. Therefore, an image signal as shown in FIG. 4 is output.

  The N-1 frame in FIG. 4 is generated from the N-2 frame and the N-1 frame in FIG. 2, and the N frame in FIG. 4 is generated from the N frame and the N-1 frame in FIG. Then, since the motion vector is detected from the image signal as shown in FIG. 4, the motion vector A for the object A and the motion vector B for the noise B in FIG. 4 are detected. As a result, since only the motion vector A in FIG. 3 is output as the motion vector detection result in the AND circuit 5 in FIG. 1, the effects of the noises 2 and 3 seen in FIG. 2 appear in the motion vector detection result. Absent.

  Since an interpolation frame is generated using this motion vector, no interpolation error due to noise is observed in the interpolation frame.

  Since the present invention is characterized by improved accuracy of motion detection, it is useful as a display device that performs image processing that requires motion detection, such as frame rate conversion and noise reduction.

1 is a block diagram showing a schematic configuration of an image processing apparatus according to an embodiment of the present invention. In the image processing apparatus, an image in which the object A is moving slightly downward includes a plurality of noise components (B, C), and is stored in the input image signal S1 for two frames and the frame memory 6. Schematic of the N-2 frame image signal 2 is a diagram showing an image output when an image signal as shown in FIG. 2 is input to the high-pass filter type noise removal circuit 1 of FIG. 1 and an output of the motion vector detection circuit 3 of FIG. 2 is a diagram showing an image output when an image signal as shown in FIG. 2 is input to the frame average noise removal circuit 2 of FIG. 1 and an output of the motion vector detection circuit 4 of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 High-pass filter type noise removal circuit 2 Frame averaging type noise removal circuit 3, 4 Motion vector detection circuit 5 AND circuit 6, 7, 8 Frame memory 9 Interpolation frame generation circuit 10 Interpolation frame insertion circuit 11, 12, 13 Motion vector

Claims (3)

A high-pass filter type noise removing means for removing noise from the input image signal using a high-pass filter;
Noise is removed from the input image signal by a frame averaging method, and a frame average type noise removing unit that extracts only signals that are commonly present in the preceding and following frames;
First motion vector detecting means for detecting a first motion vector with respect to the output of the high-pass filter type noise removing means;
Second motion vector detection means for detecting a second motion vector with respect to the output of the frame average type noise removal means;
A third motion obtained by selecting only motion vectors that exist in common with the first motion vector output from the first motion vector detection means and the second motion vector output from the second motion vector detection means. AND means for outputting a vector;
An image processing apparatus. The image processing apparatus according to claim 1, wherein frame interpolation is performed using the third motion vector. The image processing apparatus according to claim 1, wherein motion compensation noise removal is performed using the third motion vector.

Images