JPS63115479A - Interpolating system for movement correction subsample - Google Patents

Abstract

PURPOSE:To suppress deterioration of the image resolution degree in a correction mode by giving approximate estimation the movement vector between fields with a space equal to a single field secured from the movement vector between fields having a space equal to a single frame and correcting movements of all signals existing between the 1st and 4th fields. CONSTITUTION:A divider 5 and a 2nd movement correcting field memory 6 are added and the divider 5 divides the movement vector by 2. The memory 6 stores the signals equivalent to a single frame having a sample rate of 16.2MHz and passing through a switch 2 and performs the movement correction by the inter-field estimated movement vector received from the divider 5. An inter-field interpolating filter 7 interpolates an omitted sample point based on the signal passing through the switch 2 as well as the output signal of the memory 6. An intra-field interpolating filter 8 interpolates the omitted sample point based on only the signal passing through the switch 2. A switch 9 usually transmits the output signal of the filter 7 and then the output signal of the filter 8 when the movement is detected. Thus the interpolation is stabilized sepite occurrence of panning and furthermore the interpolation is secured with a reduced error.

Description

[Detailed Description of the Invention] [Industrial Application Field] This invention is applied to image processing in a MUSE type high-definition television receiver, and calculates the amount of motion correction between each field from the transmitted motion information. The present invention relates to a motion compensation subsample interpolation method that calculates and interpolates missing points between fields.

[Prior art] This type of motion compensation subsample interpolation method that has been proposed in the past was presented at the foundation commemorative lecture held in June 1980 by the NHK Research Institute of Technology and the N1-IK Broadcast Chemistry Research Institute. ``New transmission method for high-definition television (MUSE) J
There is something mentioned in the preliminary paper. FIG. 8 shows the configuration of the motion compensation subsample interpolation method.

In FIG. 8, (1) is a video input terminal that inputs a video signal transmitted at a sample rate of 15.2 MHz;
(2) is a switch which switches between the video signal input to the input terminal (1) and the output signal of a motion correction field memory (4), which will be described later, at a subsample timing of 32.4 M)Iz. (3) is uncorrected field memory,
16.21) The signal for one frame at the sample rate of Iz passing through the switch (2) is stored. Reference numeral (4) denotes a motion correction field memory which stores signals for one frame at a sample rate of 16.2 1z output from the non-correction field memory (3) and performs motion correction using a motion vector.

(7) is an interfield interpolation filter that interpolates missing sample points based on the output signal of the non-corrected field memory (3) and the signal passed through the switch (2). (8) is an intra-field interpolation filter that interpolates missing sample points based only on the signal that has passed through the switch (2). (
9) is a switch, which is connected to the upper contact (9x) when motion compensation is not performed, and is connected to the interfield interpolation filter (7x).
) is connected to the lower contact (9y) to pass the output signal of the in-field sleeve filter (8) when motion compensation is performed or motion detection is performed.

(10) is 64.8 MH after the missing sample points are interpolated.
This is a video output terminal that outputs a signal having a sample rate of z and passing through the switch (9).

Next, the operation of the above configuration will be explained.

The transmission method for high-definition television is sub-Nyquist sampling, which goes around in four fields, and the required bandwidth is 8.
1 MHz.

When the first field signal is input to the video output terminal (1), the non-motion compensation field memory (3) stores the second and fourth field signals before the -cycle.
On the other hand, in the motion correction field memory (4),
Field and third field signals are stored.

When a motion vector is present due to panning of the camera, the contents of the motion correction field memory (4) move two-dimensionally according to the amount of the motion vector. At this time, if the transmitted motion vector is the amount of motion correction between fields separated by one frame, the motion correction field memory (
The content of the motion correction in 4) is performed based on the first field signal input to the video output terminal (1).

The switch (2) switches at the timing of 32.4 MHz sub-samples, and the phase is reversed for each field.
It is also reversed depending on the motion vector. Therefore, in the above case, the signal of the first field input from the video output terminal (1) and the signal of the third field from one cycle before motion correction are passed through the switch (2).

The signal passed through the switch (2) and the output signal of the non-motion compensated field memory (3) are input to an interfield interpolation filter (7) to perform interfield interpolation.

When not performing motion compensation, that is, when connected to the switch (9) or the upper contact (9x), the video output terminal (
10) outputs a video signal with a sample rate of 64.8 MHz that has been interpolated between fields.

In addition, the field signal that has passed through the switch (2) is
It is input to the field inner sleeve filter (8), performs the field inner sleeve filter, and is sent to the switch (9) or the lower contact (9y).
When connected to the video output terminal (lO), a video signal with a sample rate of 64.8 Mllz, which is inter-field spaced, is output from the video output terminal (lO).

When the ax field is input to the video input terminal (1) without motion correction, the states of the signals 9a to 9c in FIG. 8 are as shown in FIG. 9.

However, if the signal is expressed as a field...a+
, b+ , c, , d+ , ax , bx , (!
It flows in the order of +b... A in FIG. 9 is an interpolation function fA, which is interfield interpolation, so that b+, C
+, d+, and interpolated values interpolated from at.

When motion correction is performed and 82 fields are input to the video input terminal (1), the states of the signals 10a to 10e in FIG. 8 are as shown in FIG. 1O, and the symbols in FIG. The horizontal line above indicates that motion compensation has been performed,
B is ε1 by the interpolation function f11 which is intra-field interpolation.
．． Indicates the interpolated value interpolated from al.

[Problems to be Solved by the Invention] When using the conventional motion compensation subsample interpolation method as described above, the relationship between adjacent fields is not taken into consideration at all when performing motion compensation, resulting in missing samples. Interpolation of points must be performed within the field by means of an intrafield spacing filter. Therefore, there was a problem in that the resolution during motion correction was reduced.

This invention was made to solve the above-mentioned problems, and it is possible to calculate the motion vector between each field from the transmitted motion vector and perform motion compensation in the same way as when motion compensation is not performed. Another object of the present invention is to provide a motion compensation subsample interpolation method that can suppress a decrease in resolution by using an interfield interpolation filter.

[Means for Solving the Problems] The motion compensation subsample interpolation method according to the present invention predicts by approximation a motion vector between fields separated by one field from a motion vector between transmitted fields separated by one frame. By motion-compensating all the signals from the first field to the fourth field,
It is characterized by being able to perform inter-field interpolation.

[Operation] According to the present invention, by utilizing the fact that there is no sudden change in the transmitted motion vector, the motion vector between fields separated by one field is approximated to 1/2 of the motion vector on the transmission side. By making predictions using this method, it is possible to always perform interpolation using an interfield interpolation filter, thereby suppressing a decrease in resolution during motion correction.

[Embodiment J Hereinafter, one embodiment of the present invention will be described based on the drawings.

FIG. 1 is a block diagram showing a motion compensation subsample interpolation method according to an embodiment of the present invention.
1) is a video input terminal that inputs a video signal transmitted at a sample rate of 18.2 MHz, and (2) is a switch that connects the video signal input to the input terminal (1) with the first motion correction described below. The output signal of the field memory (4) is switched at the timing of 32.4 MHz sub-samples.

(g) is a non-motion compensated field memory with a sample rate of 16.2 MHz passing through the switch (2) above.
Stores one frame's worth of signals. (4) is a first motion compensation field memory that stores one frame worth of signals at a sample rate of 16.2 MHz output from the non-motion compensation field memory (3), and performs motion compensation using a motion vector.

(5) is a divider that divides the transmitted motion vector by two. (6) is a second motion compensation field memory that stores the signal for one frame at a sample rate of 16.2 MHz that passes through the switch (2), and stores the signal for one frame at a sample rate of 16.2 MHz that passes through the switch (2).
Motion correction is performed using the predicted motion vector between fields output from 5). (7) is an interfield interpolation filter that interpolates missing sample points based on the signal passing through the switch (2) and the output signal of the second motion correction field memory (6). (8) is an intra-field interpolation filter that interpolates missing sample points based only on the signal passing through the switch (2). (9) is a switch which is normally connected to the upper contact (9x) and passes the output signal of the interfield interpolation filter (7), and is connected to the lower contact (9y) in pixel units when motion is detected. The output signal of the above-mentioned intra-field sleeve filter (8) is passed therethrough. (lO) is the missing sample point or interpolated and is 54
．． The sample rate is 8MHz, and the above switch (
9) is a video output terminal that outputs a signal passing through.

Next, the operation of the above configuration will be explained.

In order to specifically explain the operation shown in Fig. 1, Fig. 2 shows motion vectors when panning is occurring.
represents the horizontal axis of the screen, and y represents the vertical axis of the screen.

Each field of the video signal is expressed as a1 to dz, and when the signal is expressed as a field, it is "86.bo, Ca, da.
, a+ , C++ d + + ax l bx +
It flows in the order of Ca-*dz...-. The inter-field motion vectors separated by one frame are
m＊＊＊＊Al, B+, C+, Dr, Az,
It is represented by Bz, ***...

In addition, the inter-field motion vectors separated by l fields that are written over the plot are +e* +1111 blo, ClCl
, ago +8g6 river"". The following relational expression holds between the inter-field motion vectors separated by l frames and the inter-field motion vectors separated by one field.

ago + boo = AIboo
+C+o fable 81 coo + ago m Cl Here, only the relationship between the three fields 81 to C1 will be considered. However, the inter-field motion vectors separated by l frames are transmitted one per field as a digital signal with a horizontal component of 5 bits and a vertical component of 3 bits.

In other words, if we describe the magnitude of the motion vector between fields separated by one frame in terms of pixel length, the horizontal component is in the range -15 to +16, the vertical component is in the range -3 to +4, and the horizontal component, vertical component Both direction components have discrete components that are integers. To illustrate this, as shown in FIG. 3, the inter-field motion vectors separated by one frame are within the range surrounded by broken lines. However, the origin in FIG. 3 is an arbitrary pixel in the a1 field.

If the parallel movement of the image due to panning is smooth, a
The inter-field motion vectors separated by one field between the 1 field and the b1 field are within the range surrounded by the chain line in FIG. Here, the relative relationships of the three fields a+ to (+) are changed in three ways and are shown in FIGS. 4 to 6.

First, FIG. 4 shows a case where the magnitude of the inter-field motion vector A1 transmitted one frame apart is maximum. At this time, ■ inter-field motion vector at
o must also be the maximum, and the inter-field motion vector boo that is separated by one field must also be within the range of the two-dot chain line or must have a maximum value. Therefore, vector b+o corresponds to A1/2.

Next, FIG. 5 shows a case where the components of the inter-field motion vector A1 separated by one frame are (X, y)=(10, 2). As in Fig. 4, the inter-field motion vector a+・ separated by one field exists within the range surrounded by '1Asi, and when the b+ field is at the point shown in the figure, the inter-field motion vector lBo separated by l field is It exists within the range surrounded by the two-dot chain line. In fact, such movement by panning is possible since the Cl field is within the dash-dotted line as shown.

From the above, when the relative positions of the al field and the Cl field are determined by the inter-field motion vector A1 transmitted one frame apart, the position of the b1 field with respect to those fields is within the shaded range. . Furthermore, if the parallel movement of the image due to panning is smooth, the position of the b1 field will be 3x centered on the midpoint of the positions of the a1 field and the Cl field, which are indicated by the black circles in Fig. 5, even within the diagonally shaded range. It has high flexibility (2) existing on the square lattice of 3. vector b
It is reasonable to substitute A+/2 for lo, and the error is about one pixel.

Next, FIG. 6 shows a case where the components of the inter-field motion vector A1 separated by one frame are (x, y) = (3, 1). As in Fig. 4, the inter-field motion vector au) separated by one field exists within the range surrounded by the chain line, and if the b+ field is at the point shown in the figure, the inter-field motion vector lH separated by one field exists.
o exists within the range surrounded by the two-dot chain line. fact, C
Since one field is within the dash-dotted line as shown, such movement by panning is possible. Since there is no panning that exceeds the perpendicular angle to the direction change of the motion vector between fields separated by adjacent l fields, the motion vector A between fields separated by one frame is transmitted.
When the relative positions of the 81 field and the Cl field are defined by I, the position of the b1 field with respect to those fields is within the shaded range. moreover,
If the parallel movement of the image due to panning is smooth, then b1
Even within the diagonally shaded range, the field positions are likely to exist on a 2×2 square lattice centered on the midpoint between the positions of the a1 field and the Cl field indicated by the black circles in FIG. Therefore, is it possible to substitute vector baa with A1/2? Since the component of A1/2 in this case is not an integer value, the vector connecting one of the black circles in Figure 6 and the position of field 01 can be used as vector bIO. Use as a substitute. In this case as well, the error is about one pixel.

As described above, the inter-field motion vector b+a separated by l fields can be predicted by approximating the integer part of A+/2 using the inter-field motion vector At separated by l frames, and the error is about 1 pixel or less. It is as follows. Similarly.

It can be predicted as follows: B+ Aya [Do/27 b+ ”= [Al/2] C i sacrifice [B+/2 ]. The tatami [ ] represents the integer part.

Moreover, the error in prediction of inter-field motion vectors separated by l fields does not affect subsequent prediction, so -
Even if there is a large temporal change in the vector and the error becomes large, if the subsequent vector change becomes gradual, it is possible to make a prediction with a smaller error again.

Motion correction performed by prediction as described above will be explained with reference to FIG.

First, when a d1 field signal is input to the video input terminal (1) in Fig. 1, the 81 field and c1 field signals are input to the non-motion compensation field memory (co), and the c1 field signal is also input to the first motion compensation field memory (co). ) to d, field,
The signal of the bl field is stored. At this time, when the motion vector B1 is input, the first motion correction field memory (4) moves the stored d, field, and b+ field two-dimensionally according to the motion vector B+, and the video input terminal ( 1) Motion correction is performed based on the d1 field input in step 1). This will cause the switch (
The signal passing through 2) is a signal of the d1 field and the b1 field corrected by the motion vector B+.

On the other hand, the input motion vector B1 is divided by 2 by a divider (5), and the integer part of the output becomes the predicted value of the inter-field motion vector CIO separated by one field.

The second motion compensation field memory (6) includes the aI field, as well as the non-motion compensation field memory (3).
The C+ field signal is stored.

The contents of the second motion correction field memory (6) are moved two-dimensionally by the predicted vector [B+/2] of the inter-field motion vector CI separated by l fields, and motion correction is performed based on the dl field.

In this way, signals for four fields, the al field, the b1 field, the cl field, and the d1 field, subjected to motion compensation based on the d1 field, enter the interfield interpolation filter (7) to enable interfield interpolation. . Normally the switch (9) is the upper contact (9x
), and the inter-field interpolated signal passes through.However, for video signals, inter-field interpolation can only be used to interpolate missing sample points, so when motion is detected, the switch (9) Lower contact (9y)
The intra-field interpolation filter (8) performs intra-field interpolation only from the signal that has passed through the switch (2).

The sample rate of the signal whose missing sample points are interpolated by the interfield interpolation filter (7) or the intrafield interpolation filter (8) is 64.8 kl Hz, and is output from the video output terminal (10).

The motion compensation subsample interpolation method that calculates inter-field motion vectors separated by l fields by such prediction is not only stable at the start and end of panning, but also eliminates temporary errors in the transmitted motion vectors. Even if it happens, it will not leave any negative effects afterwards.

It is also stable even in special cases where the scene changes during panning or the image changes during panning. Furthermore, since the motion vector changes gradually due to panning, this method is very good.

FIG. 7 shows the states of the signals 7a to 7g in FIG. 1 when a dI field signal is input to the video input terminal (1). In Fig. 7, the horizontal line above the symbol indicates that the movement has been corrected, and A is M+, 'F+, B,
B indicates an interpolated value interpolated from d+, and B indicates an interpolated value interpolated from -b+, dl by an interpolation function fB.

[Effects of the Invention] As described above, according to the present invention, the motion vector for one field is obtained by approximation from the transmitted motion vector, and motion correction is performed for each field. Even if panning occurs, interpolation by the interfield interpolation filter can be stabilized, and interpolation can be performed with little error, thereby suppressing a decrease in resolution during panning.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing an example of a schematic configuration of a motion compensation subsample interpolation method according to an embodiment of the present invention, and FIG. 2 is an example of motion vectors during panning to explain the operation of FIG. 1. 3 is a vector diagram showing the existence range of inter-field motion vectors separated by l frames and inter-field motion vectors separated by 1 field, Figs. 4 and 5.
6 is a vector diagram showing the relationship between inter-field motion vectors separated by one frame and inter-field motion vectors separated by one field, respectively. FIG. 7 is a signal flow of the motion compensation sub-sample interpolation method according to the present invention. 8 is a block diagram showing an example of a schematic configuration of the conventional motion compensation subsample interpolation method. FIG. 9 is a signal diagram of the conventional motion compensation subsample interpolation method when no motion compensation is performed. FIG. 10 is a timing chart showing the flow of signals when performing motion correction using the conventional motion correction sub-sample interpolation method. (1)...Video input terminal, (2)...Switch, (
3)...Non-motion compensation field memory, (4)...
First motion compensation field memory, (5)...Divider, (6)...Second motion compensation field memory, (
7) - inter-field interpolation filter, (8)...intra-field interpolation filter, (8)...switch, (10
)...Video output terminal. In addition, in the figures, the same reference numerals indicate the same or corresponding parts.

Claims (1)

[Claims] (1) Based on the sample values that are intermittently sub-sampled and transmitted in a cycle of 4 fields while maintaining a predetermined sample position, the receiving side interpolates the missing sample values received during the 4-field period. This is a sub-sample interpolation method that reproduces a video signal by delaying and interpolating sample values on the receiving side based on motion information between fields separated by one frame of the video signal detected on the transmitting side. In the motion compensation subsample interpolation method that corrects the interpolation position, the field of interest is determined on the receiving side based on the motion information detected on the transmitting side and the motion information between the field of interest and a past field that is one frame apart. The motion information between the past field separated by one field from the motion information detected on the transmitting side is
/2, the interpolation position of the sample values of the past three fields is corrected for the field of interest together with the detected motion information on the transmitting side, and inter-field interpolation is performed. Subsample interpolation method.