JPH07307925A - Motion estimating device for image - Google Patents

Abstract

PURPOSE:To provide motion correcting field interpolation so as not to generate any distortion in the image of a character or the like in the foreground even in the flowing background. CONSTITUTION:Low-pass filters 3 and 4 are respectively low-pass filters in x-axis (horizontal) and y-axis (vertical) directions in which pass band width is different from that of low-pass filters 1 and 2. A two-dimensional band filter is formed by these low-pass filters 1-4 and a subtracter 5. When an input image provided with a time display area superimposed on the flowing background is inputted to the two-dimensional band filter, for example, a signal in a size corresponding to the size of an edge degree is extracted. Concerning the signal, a threshold value less than previously decided one is replaced with '0' and only the threshold value more than it is inputted to a motion estimating part 7. The motion estimating part 7 generates a motion vector from the signal higher than the threshold value.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion estimation apparatus for an image, and more particularly to a motion estimation using only a portion having a relatively large edge degree in an image, which is suitable for a field interpolation method using a motion vector. The present invention relates to an image motion estimation device capable of performing.

[0002]

2. Description of the Related Art A general moving image including a television image is not given as a temporally continuous signal on the basis of displaying still images one after another at a constant frame rate per second.

For example, looking at the broadcast television system,
A frame rate of 30 frames / second is adopted in Japan, while a frame rate of 25 frames / second is adopted in European countries. Therefore, when exchanging broadcast programs between countries that adopt television systems having different frame rates, a frame rate conversion technique is indispensable.

FIG. 7 shows a principle diagram of the frame rate conversion. In the figure, the frame rate is 25 frames / second (a25, b25, c25, ...)
It shows conversion to images (a30, b30, c30, ...) Of 30 frames / sec. Now, assuming that a25 and a30 frames are synchronized, the frame rate ratio is 2
Since 5: 30 = 5: 6, both frames are synchronized every 6 frames.

As the frame to be synchronized, the frame of the original image can be used as it is, but the frame not to be synchronized is created by interpolating two adjacent frames of the original image.
For example, a b30 frame is an a25 frame that is adjacent to a b25 frame.
The c30 frame is created by interpolating frames, and the c30 frame is created by interpolating adjacent b25 and c25 frames. When generating this interpolation frame, that is, the interpolation field, it is necessary to correct the movement of the original image.

In various motion estimations when performing field interpolation by performing this motion correction, conventionally, assuming the uniformity of motion in an image, one field image is divided into a number of blocks, The motion estimation is performed every time. Specifically, a motion is detected for each block between two adjacent fields, and an interpolation field is created based on the vector by referring to the two fields.

The field interpolation method using the motion vector is described in detail in Japanese Patent Publication No. 3-25119, "Motion picture frame rate conversion method using motion amount", which is a patent of the present applicant. Explained.

[0008]

However, in a real image, an image without motion and an image with motion often coexist in the same block. For example,
When the background of the superimposed time display appearing in the video of the sports broadcast is flowing, when this is divided into blocks, a part of the time display and the background are mixed in the same block.

FIG. 6 shows an example of this, and (a)
The figure shows the pixel value of any one raster 11c in the previous field 11, and the figure (b) shows the pixel value of the raster at the same position in the current field. It is assumed that the field 11 is composed of a superimposed character area 11a (blank portion) and a background 11b (hatched portion).
In the example shown in the figure, as is clear from comparison between the (a) diagram and the (b) diagram, it can be seen that the superposed character region does not move, but the background moves greatly.

In the above case, the character area 1 is included in the same block.
When a part of 1a and a part of the background 11b are mixed, a motion vector other than 0 may be assigned to a character region that should be stationary. Then, the interpolated image becomes
In the case where distortion occurs in the character area and the result is as shown in FIG. 6, the background 1
1b has a problem in that the converted image looks like the time display is distorted.

An object of the present invention is to eliminate the above-mentioned problems of the prior art, and to perform motion correction so that an image such as a character in the foreground does not generate distortion as much as possible in an interpolated image even under a flowing background. Field interpolation can be done,
An object is to provide an image motion estimation device.

[0012]

In order to achieve the above-mentioned object, the present invention relates to an image motion estimating apparatus for estimating a motion from two images, and a band characteristic input to each of the two images. And a data conversion unit that replaces, in the output of the filter unit, a value equal to or less than a threshold value of a predetermined size with 0, and motion estimation based on the output from the data conversion unit. It is characterized in that it is provided with a motion estimation unit for performing.

[0013]

According to the present invention, first, the first image is input to the filter means having the band characteristic, and out of the output values thereof, the value below the threshold value is replaced with zero. Then
Similarly to the above, the second image is input to the filter means having the band characteristic, and among the output values thereof, the value equal to or less than the threshold value is replaced with 0. By this processing, only a portion having a high edge degree such as a contour portion of a character area has a value, and a portion having a low edge degree such as a background portion is suppressed.
Then, the motion estimation unit performs motion estimation based on these data. As a result, it is possible to obtain a motion-compensated interpolated image in which a region having a clear edge is reproduced without distortion, without the contour portion of the character region being dragged by the background having a large flow.

[0014]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to the drawings. First, the principle of the present invention will be described. The present invention takes advantage of the human visual properties. Visual processing in the human retina looks at the image as a whole and sensitively captures the edges of the image. For this reason, vision is particularly sensitive to changes in the contours of objects in the image. Conversely, it is relatively insensitive to slowly changing parts of the image and very small random changes. Therefore, in the present invention, a two-dimensional bandpass filter is first applied to the two images for which the motion vector is calculated, and the low frequency component (slow change portion) and the high frequency component (very fine change portion) are suppressed. . Then, the output image has a value proportional to the edge degree of each part in the image, that is, the clarity of the edge.

For example, when the background of the superimposed time display is flowing, the outline of the time display is
Not surprisingly, it has a clear edge. That is, the edge degree is large. On the other hand, the flowing background is less likely to have a clear edge due to the accumulation effect of the camera. That is, the edge degree is small.

Therefore, as described above, by applying a two-dimensional band filter having a predetermined bandwidth to the two images (blocks) for which the motion vector is calculated, the outline of the character area is large. The output value is obtained, and a small output value is obtained in the background portion. After that, by applying an appropriate threshold value to the small output value and replacing it with 0, the large output value, that is, only the outline portion of the character area has a value and the background portion is suppressed. Become.

When motion estimation is performed after the above processing, the character area is not dragged by the change in the background value, and a motion-corrected interpolated image in which a character area with a clear edge is reproduced without distortion is obtained. be able to.

Next, one embodiment of the present invention will be described with reference to FIG. In the figure, 1 is a low-pass filter in the x-axis direction (horizontal direction), 2 is a low-pass filter in the y-axis direction (vertical direction) having the same pass band width as the low-pass filter 1, and 3 is the above A low-pass filter in the x-axis direction (horizontal direction) having a different pass band width from the low-pass filter 1 and a low-pass filter 4 in the y-axis direction (vertical direction) having the same pass band width as the low-pass filter 3. It is a filter. Here, the low-pass filters 1, 2, and 3 and 4 are cascaded types, respectively.
It realizes a low-pass filter of dimension. The subtractor 5 is
The outputs of the low pass filters 3 and 4 are subtracted from the outputs of the low pass filters 1 and 2. The low pass filters 1 to 4 and the subtractor 5 form a two-dimensional band filter.

That is, as shown in FIG. 2, the filter characteristics formed by the low pass filters 1 and 2 are as shown by a curve a1 in the figure, and formed by the low pass filters 3 and 4. The filter characteristics that
2 becomes, the characteristic of the subtracter 5 is as the characteristic a3 obtained by subtracting a2 from the curve a1.
It has a two-dimensional bandpass filter characteristic.

Next, 6 is a ROM which uses the output value of the two-dimensional bandpass filter as an address and outputs the data corresponding to the address. In the ROM 6, for example, as shown in FIG. 3, the absolute value of the two-dimensional bandpass filter output value whose absolute value is equal to or less than d is replaced with 0, and the absolute value is changed to d.
For a larger output value, data for adding and subtracting d in the direction of approaching 0 is stored.

In this data conversion, the input of the ROM 6 is a,
When the output of the ROM 6 is b, and d is a threshold value having an appropriate size, it can be expressed by the following equation.

[0022]

[Equation 1] From the above, the output of the ROM 6 has an output of 0 in a portion with a small edge degree and a non-zero output in a portion of a large edge degree.

The output of the ROM 6 enters the motion estimation section 7. The motion estimation unit 7 detects a motion vector suitable for field interpolation based on the output of the ROM 6.
The motion vector detected by the motion estimator 7 is, for example, the motion of the image signal shown in FIG. 6 of Japanese Patent Publication No. 3-25119, “Motion picture frame rate conversion system using motion amount”. The quantities are vx and vy.

As described above, according to the present embodiment, in a block composed of an image in which a motionless image and a motionless image are mixed, a part of the image that changes gently and a very fine random The changed portion is removed by the two-dimensional bandpass filter, and the edge having a small edge degree included in an image such as a flowing background is removed by the data conversion by the ROM 6. Therefore, the ROM
From 6, an edge having a large degree of edge such as a character region of a superimposed time display or the like is extracted, and this is input to the motion estimation unit 7. Therefore, the motion estimation unit 7 can allocate the motion vector of 0 to the stationary character area.

Next, a second embodiment of the present invention will be described with reference to FIG. In the figure, the circuit before the subtractor 5 is
The illustration is omitted because it is the same as or equivalent to FIG. 1. This embodiment is characterized in that a low-pass filter 8 is provided between the ROM 6 and the motion estimation unit 7 in FIG.

According to this embodiment, the waveform of the output of the ROM 6 as shown in FIG. 3 can be smoothed as shown in FIG. 5, and the processing in the motion estimating section 7 can be simplified. And there is an advantage that it can be stabilized.

[0027]

As is apparent from the above description, according to the present invention, it is possible to extract only a portion having a relatively large edge degree, and to perform motion estimation using the data of the portion. become. As a result, for example, in a motion compensation frame rate converter or the like, it is possible to expect an effect that it is possible to easily prevent distortion of a converted image in which superimposed characters or the like are dragged by a flowing background. it can. Further, therefore, it becomes possible to generate a high quality interpolation frame or interpolation field.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an exemplary embodiment of the present invention.

FIG. 2 is an explanatory diagram of a bandpass filter in FIG.

FIG. 3 is an explanatory diagram of data conversion of the ROM of FIG.

FIG. 4 is a block diagram of an essential part of a second embodiment of the present invention.

FIG. 5 is a waveform diagram of the output of the LPF of the second embodiment.

FIG. 6 is a diagram showing an example of an image composed of a superimposed superimposed text area and a moving background.

FIG. 7 is a diagram showing the principle of frame rate conversion.

[Explanation of symbols]

1 to 4 ... Low-pass filter, 5 ... Subtractor, 6 ... ROM,
7 ... Motion estimation unit, 8 ... Low-pass filter.

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Office reference number FI technical display location H04N 11/04 B 9185-5C

Claims (3)

[Claims] 1. An image motion estimation apparatus for estimating motion from two images, wherein filter means having band characteristics inputted to each of the two images and output of the filter means are predetermined. A motion of an image, which comprises a data conversion means for replacing a value equal to or smaller than a threshold of a certain magnitude with 0, and a motion estimation part for estimating a motion based on an output from the data conversion means. Estimator. 2. The image motion estimation apparatus according to claim 1, further comprising a low-pass filter connected to the next stage of the data conversion means. 3. The image motion estimation device according to claim 1, wherein the filter means having the band characteristic has a low-pass filter in the x-axis and y-axis directions having the first filter characteristic, and a second low-pass filter having the first filter characteristic. An image motion estimation apparatus comprising: an x-axis and a y-axis direction low-pass filter having filter characteristics; and a subtractor for subtracting outputs of the low-pass filters.