JPS63115481A - Interpolating system for movement correction subsample - Google Patents

Abstract

PURPOSE:To suppress deterioration of image resolution in a correction mode by giving the approximate estimation to the inter-field vector having a single field space from the received inter-field movement vector having a single frame space and correcting movements of all signals existing between the 1st and 4th fields. CONSTITUTION:A 1-field delay means 5 delays the received movement vector by a single field. An adder 6 adds the output of the means 5 with said movement vector. A divider 7 divides the output of the adder 6 by 4. A subtractor 8 subtracts the output of the divider 7 from the received movement vector. A 2nd movement correcting field memory 9 stores the signal equivalent to a single frame and passing through a switch 2 and performs movement correction by the inter-field estimated movement vector, i.e., the output of the subtractor 8. The means following the inter-field interpolating filters 11 and 12 have operations as conventional. Thus it is possible to always perform interpolation by means of an inter-field interpolating filter and to suppress deterioration of image resolution in a correction mode owing to the estimation carried out with approximation to 1/4 movement vector of the transmission side.

Description

[Detailed Description of the Invention] [Industrial Application Field] This invention is applied to image processing in a USE system high-definition television receiver, and the amount of motion correction between each field is determined from transmitted motion information. The present invention relates to a motion compensation subsample interpolation method that calculates missing points and interpolates between fields.

[Conventional technology].

This type of motion-compensated subsample interpolation method that has been proposed in the past was proposed by the NHK Research Institute of Technology and the NIIK Broadcasting Chemistry Research Institute in June 1980 at their founding commemorative lecture. There is a transmission method (MUSE) described in the preliminary document called J. Fig. 11 shows the configuration of the motion compensation subsample interpolation method.

In Fig. 11, (1) is a video input terminal that inputs a video signal transmitted at a sample rate of 16.2 MHz, and (2) is a switch that connects the video signal input to the input terminal (1) and the video signal that will be described later. Motion correction field memory (4)
The output signal is switched at the timing of the 32.4 MHz sub-sample. (3) is uncorrected field memory,
A signal for one frame at a sample rate of 16.2 MHz passing through the switch (2) is stored. Reference numeral (4) denotes a motion correction field memory, which stores signals for one frame at a sample rate of 16.2 MHz output from the non-correction field memory (3), and performs motion correction using a motion vector.

(7) is an interfield interpolation filter that interpolates missing sample points based on the output signal of the non-corrected field memory (3) and the signal passed through the switch (2). (8) is an intra-field interpolation filter that interpolates missing sample points based only on the signal that has passed through the switch (2). (
9) is a switch that is connected to the upper contact when motion compensation is not performed and passes the output signal of the interfield interpolation filter (7), and connected to the lower contact when motion compensation is performed or when motion is detected. and passes the output signal of the intra-field interpolation filter (8). (lO) is a video output terminal that outputs a signal that has a sample rate of 64.8 Ml (z) by interpolating the missing sample points and passes through the switch (9).

Next, the operation of the above configuration will be explained.

The transmission method for high-definition television is sub-Nyquist sampling, which goes around in four fields, and the required bandwidth is 8.
1 MHz.

When the first field signal is input to the video input terminal (1), the non-motion compensation field memory (3) stores the second and fourth field signals from one round before.
On the other hand, the motion correction field memory (4) contains the first data from the previous round.
Field and third field signals are stored.

When a motion vector exists due to camera panning, the contents of the motion correction field memory (4) move two-dimensionally in accordance with the amount of the vector. At this time, if the transmitted motion vector is the amount of motion correction between fields separated by one frame, the motion correction field memory (
The motion correction content in step 4) is performed based on the first field signal input to the video input terminal (1).

The switch (2) is switched at the timing of 32.4 Mllz sub-samples, and the phase is inverted for each field and also depending on the motion vector. therefore,
In the above case, the signal of the if field input from the video input terminal (1) and the signal of the third field from one cycle before motion correction are passed through the switch (2).

The signal passed through the switch (2) and the output signal of the non-motion compensated field memory (3) are input to an interfield interpolation filter (7) to perform interfield interpolation.

When motion correction is not performed, that is, when the switch (9) is connected to the upper contact, a video signal with a sample rate of 54.8 MHz interpolated between fields is output from the video output terminal (lO).

In addition, the field signal that has passed through the switch (2) is
It is input to the field inner sleeve filter (8), performs the field inner sleeve filter, and the switch (9) connects the lower contact (9b).
), the video output terminal (10) outputs a video signal with a sampling rate of 64.8 M)lz within the field.

When 82 fields are input to the video input terminal (1) without motion correction, the states of the 111th signals 12a to 12e are as shown in FIG. 12.However, when the signals are expressed in fields... ...a+, b+, c
+, d+, as, bz, CL, da = ・
It flows in the order of... A in Fig. 12 is an interpolation function r^ between fields, so that b+, C+, d+
, az.

Furthermore, when motion correction is performed and the a2 field is input to the video input terminal (1), the states of the signals 10a to 10e in FIG. 8 are as shown in FIG. 13, and in FIG.
A horizontal line above the symbol indicates that motion compensation has been performed, and B
is τ+, a by the interpolation function f8, which is intra-field interpolation.
Indicates an interpolated value interpolated from L.

[Problems to be Solved by the Invention] When using the conventional motion compensation subsample interpolation method as described above, since the relationship between adjacent fields is not taken into account at all when performing motion compensation, missing sample points interpolation must be performed within the field by an intrafield filter. Therefore, there was a problem in that the resolution during motion correction was reduced.

This invention was made to solve the above-mentioned problems, and it is possible to calculate the motion vector between each field from the transmitted motion vector and perform motion compensation in the same way as when motion compensation is not performed. Another object of the present invention is to provide a motion compensation subsample interpolation method that can suppress a decrease in resolution by using an interfield interpolation filter.

[Means for Solving the Problems] The motion compensation smooth sample interpolation method according to the present invention predicts, by approximation, a motion vector between fields separated by one field from a motion vector between transmitted fields separated by one frame. By motion-compensating all the signals from the first field to the fourth field,
It is characterized by being able to perform inter-field interpolation.

[Operation] According to the present invention, by utilizing the fact that there is no sudden change in the transmitted motion vector, the motion vector between fields separated by l fields is estimated by approximating it to 174 of the motion vector on the transmission side. By doing so, interpolation can always be performed using the interfield interpolation filter, thereby suppressing a decrease in resolution during motion correction.

[Example] Hereinafter, an example of the present invention will be described based on the drawings.

FIG. 1 is a block diagram showing the configuration of a motion compensation subsample interpolation method according to an embodiment of the present invention. The video input terminal (2) is a switch that connects the video signal input to the input terminal (1) and the output signal of the first motion correction field memory (4), which will be described later, at a subsample timing of 32.4M1lz. Switch. (3) is a non-motion compensation field memory which stores the signal for one frame at a sample rate of 16.211 tlz that passes through the switch (2), and performs motion compensation using a motion vector.

(5) is an l-field delay device that delays the transmitted motion vector by one field. (6) is an adder,
The transmitted motion vector is processed by a 1-field delay device (5
) is added to the motion vector l field before. (7) is a divider which divides the output of the adder (6) by 4. (8) is a subtracter that subtracts the output of the divider (7) from the transmitted motion vector.

(9) is a second motion correction field memory that stores the signal for one frame at a sample rate of 16.2 MHz that passes through the switch (2), and stores the field that is the output of the subtracter (8). Motion correction is performed using the predicted motion vector between the two. (10) is an interfield interpolation filter that interpolates missing sample points based on the signal passing through the switch (2) and the output signal of the second motion correction field memory (9). (11) is an intra-field interpolation filter that interpolates missing sample points based only on the signal passing through the switch (2). (12) is a switch,
Usually connected to the upper contact to pass the output signal of the interfield interpolation filter (lO), and when motion is detected, connected to the lower contact in pixel units to signal the output of the intrafield interpolation filter (11). pass. (13
) is a video output terminal that outputs a signal that interpolates the missing sample points to obtain a sample rate of 64.8 MHz and passes through the switch (12).

Next, the operation of the above configuration will be explained.

Figure 2 shows the motion vectors generated in the non-motion compensation field memory (3) for the video input from the video input terminal (1) on the transmitting side, where X is the screen horizontal axis;
y represents the vertical axis of the screen.

Each field of the video signal is old to d2. When the signal is expressed as a field, it is ao, bo, co,
do, aI, b1+CI, di +82 +b2 +
It flows in the order of CZ and dz """. Each 1
Motion vectors between fields separated by frames...
・＾+ , B+ , C+ , D+ , At, Bλ,...
It is represented by... Also, the inter-field motion vectors separated by l fields that are overwritten on the diagram...+1+1
@ E-7,, C70, Z◎, ag.

It is expressed as... The following relational expression holds between the inter-field motion vectors separated by l frames and the inter-field motion vectors separated by one field.

Pao + T'; Om rt baa + C10881 Here, for example, b+o can be rewritten as in the following equation.

If the second term on the right side of the above equation is sufficiently small, 0 can be predicted as follows.

Here, for example, only the relationship between the four fields a1 to d1 will be considered. However, the motion vector between fields separated by l frames has a horizontal component of 5 bits and a vertical component of 3 bits.
One bit per field is transmitted as a digital signal. In other words, if we express the magnitude of the motion vector between fields separated by l frames in terms of pixel length, the horizontal component is -1
5 to +16, the vertical component is within the range of -3 to +4, and both the horizontal and vertical components! ! !
The number gI has a dispersive component. To illustrate this, the third
As shown in the figure, the inter-field motion vectors separated by l frames are within the range surrounded by the dashed line.

However, the origin in FIG. 3 is any painting in the C1 field.

If the parallel movement of the image due to panning is smooth, a
The inter-field motion vectors separated by one field between the l field and the b1 field are within the range surrounded by the chain line in FIG. Here, the relative relationship between the three fields at to C1 is changed in three ways as shown in FIGS. 4, 6, and 8, and the position of the d+ field and the prediction accuracy of the motion vector are examined for each.

First, FIG. 4 shows the case where the magnitude of the inter-field motion vector A1 transmitted one frame apart is at its maximum. At this time, the fields rtn motion vectors 5 and no that are separated by one field also become maximum, and the relative positions of fields 81 to CI are determined. At this time, the C1 field exists within the range indicated by the three-dot chain line in the figure, but the d+ field exists within the range indicated by the diagonal line if there is no sudden change in motion. Furthermore, if the parallel movement of the image, such as panning, is smooth, there is a high possibility that the C1 field will exist at the position indicated by the black circle in the figure. - As an example, the predicted vector (
A+ + 8+ )/4 and the vector boo as the fifth
As shown in the figure. The prediction error at this time is one pixel length, and if there is a dim yield at the position indicated by the black circle, the error will be less than that, so such prediction is appropriate.

Next, FIG. 6 shows a case where the components of the inter-field motion vector A1 separated by one frame are (x, y)=(10, 2). As in Fig. 4, the inter-field motion vectors separated by one field exist within the range surrounded by the dashed line, and if the b1 field is at the point shown in the figure,
The inter-field motion vector io that is separated by l fields exists within the range surrounded by the two-dot chain line. In fact, such movement by panning is possible because the C1 field is within the range enclosed by the dashed-double line as shown. At this time, the d+ field exists within the range indicated by the three-dot chain line in the figure, but as in the explanation in FIG. 4, it is within the diagonally shaded range, and there is a high possibility that it exists at the position indicated by the black circle. - As an example, the predicted vector (AI+Bl)/4 when the d1 field is at the position shown in the diagram.
and vector boo are shown in FIG.

The prediction error at this time is one pixel length on the diagonal, and if the d1 field is located at the position indicated by the black circle, the error will be less than that, so the prediction is valid in the case of FIG. 6 as well.

Next, FIG. 8 shows a case where the components of the inter-field motion vector At separated by one frame are (x, y) = (3, 1). Similarly to FIG. 4, the inter-field motion vector a + a that is separated by one field exists within the range surrounded by the chain line, and if the b+ field is at the point shown in the figure, the inter-field motion vector m O that is separated by one field is
exists within the range surrounded by the two-dot chain line. In fact, since the CI field is within the range enclosed by the dash-dotted line as shown, such movement by panning is usable. At this time, the d1 field exists within the range indicated by the three-dot chain line in the figure, or within the diagonally shaded range as explained in Figure 4, and there is a very high probability that it exists at the position indicated by the black circle. . - As an example, the predicted vector (r+ +
fl )/4 and the vector rHs are shown in FIG. The prediction error at this time is one pixel long on the diagonal, and if the d1 field is located at the position indicated by the black circle, the error will be less than that, so the prediction is reasonable in the case of FIG. 8 as well.

As described above, the inter-field motion vector l) Ill separated by l fields can be predicted by approximation to the integer part of (T+)/4 using the inter-field motion vector rotation separated by l frames and the cod. The error is 1 diagonal
It is about the size of a pixel or smaller. Similarly, it can be predicted as follows: Tsume # [(Uryuyuan) / 4] "0 kon [(T + Jumei) / 41 Otonan [(Jau +? +) / 4]". Represents the integer part.

Moreover, the error in prediction of inter-field motion vectors separated by l fields does not affect subsequent prediction, so -
Even if the error becomes large due to a large temporal change in the vector, if the subsequent vector change becomes gradual, predictions can be made again with a small error.

Motion correction performed by prediction as described above will be explained with reference to FIG.

First, when a dt field signal is input to the video input terminal (1) in figure ff11, the non-motion compensation field memory (3
) stores the signals of the L field and the C1 field, and the first motion correction field memory (4) stores the signals of the do field and the b1 field. At this time, if the motion vector 81 is input, the first motion correction field memory (4) two-dimensionally moves the d0 field and b1 field stored in accordance with the motion vector B1, and the video input terminal ( 1) Motion correction is performed based on the d1 field input in step 1). As a result, the signal passing through the switch (2) is a signal of the d1 field and the b1 field corrected by the motion vector B1.

On the other hand, the input motion vector B1 is input to the l-field delay device (5), and the output of this l-field delay device (5) is the motion vector of the previous l-field. and further divided by 4 by the divider (7).

The integer part of the output becomes the predicted value of the inter-field motion vector b10 separated by one field.

The output of the divider (7) is subtracted from the motion vector 8I by the subtractor (8) to obtain the predicted value of the inter-field motion vector c7o separated by l fields.

The contents of the second motion correction field memory (9) are moved two-dimensionally by () the f-measured vector of the inter-field motion vectors separated by fields - [('1+Iku+)/4]' ( 11 fields) is subjected to motion compensation on a quasi-X basis.

In this way, the signals of the four fields of the a1 field, the b1 field, the CI field, and the d+ field, which have been subjected to motion compensation based on the X quasi, enter the interfield interpolation filter (10) to enable interfield interpolation. do. Normally, the switch (12) is connected to the upper contact, and the inter-field interpolated signal passes through, or inter-field interpolation is not performed for video signals, so when motion is detected, the switch (12) or the pixel Intra-field interpolation is performed only from the signal connected to the lower contact at the middle level and passed through the switch (2) by the intra-field interpolation filter (11).

The sample rate of the signal whose missing sample points are interpolated using the interfield interpolation filter (10) or the intrafield interpolation filter (11) is 64.8 MHz.
It is output from the video output terminal (13).

The motion-corrected smooth sample interpolation method that calculates inter-field motion vectors separated by l fields by such prediction is not only stable at the start and end of panning, but also eliminates temporary errors in the transmitted motion vectors. Even if it does, it will not leave any negative effects.

It is also stable even in special cases where the scene changes during panning or the image changes during panning. Furthermore, since the change in motion vector due to panning is gradual, this method can be said to be very good.

The state of the signals 10a to 10g in Figure 1 when the az field signal is input to the video input terminal (1) is shown in the first diagram.
It is shown in figure O. In Fig. 10, the horizontal line above the symbol indicates that the motion has been corrected, and A is b+, C1 by the interpolation function fA.
, d+, az, and B indicates an interpolated value interpolated from E+, az using the interpolation function fs.

[Effects of the Invention] As described above, according to the present invention, a motion vector for one field is predicted from a transmitted motion vector by approximating it to 1/4 of the motion vector on the transmission side, and based on this approximate prediction. Since motion compensation is performed for each field, the interpolation by the interfield interpolation filter can be stabilized even when panning occurs, just like in a still image state. Furthermore, interpolation with fewer errors can be performed, making it possible to stabilize the interpolation during panning. The reduction in resolution can be sufficiently suppressed.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing an example of a schematic configuration of a motion compensation subsample interpolation method according to an embodiment of the present invention, and FIG. 2 shows an example of motion vectors during panning to explain the operation of FIG. 1. 3 is a diagram showing the existence range of inter-field motion vectors separated by 1 frame and inter-field motion vectors separated by 1 field. FIGS. 4, 6, and 8 are vector diagrams showing fields separated by 1 frame, respectively. Vector diagrams showing the relationship between inter-field motion vectors and inter-field motion vectors separated by one field. FIGS. 5, 7, and 9 show fields separated by one field predicted from inter-field motion vectors separated by l frame, respectively. A vector diagram showing an example of an inter-field motion vector and an actual inter-field motion vector separated by one field. Figure 1θ is a timing chart showing the signal flow of the motion compensation sub-sample interpolation method according to the present invention. Block diagram illustrating an example of a schematic configuration of a motion compensation subsample interpolation method, 12th block diagram
The figure is a timing chart showing the signal flow when motion compensation is not performed in the conventional motion compensation smooth sample interpolation method, and Figure 13 is the signal flow when motion compensation is performed in the conventional motion compensation smooth sample interpolation method. FIG. (1)--Video input terminal, (2)...Switch, (
3)...Motion compensation field memory, (4)...
First motion correction field memory, (5)-1 field delayer, (6)--adder, (7)--divider,
(8) --- Subtractor, (9) --- Second motion compensation field memory, (10) --- Interfield interpolation filter, (11) --- Intrafield interpolation filter, (1
2) -...Switch, (13)...Video output terminal. In addition, in the figures, the same reference numerals indicate the same or corresponding parts. Agent Masu Oiwa Yudaigm y Figure 10 A: fA (bl, +, d+, clJ B = f, (c+, az) Order 2 Figure A: fa (QbT)l, 2. d+) 13th Figure B-fa (b+, d+) Procedural amendment (voluntary)

Claims (1)

[Claims] (1) Based on the sample values that are intermittently sub-sampled and transmitted in a cycle of 4 fields while maintaining a predetermined sample position, the receiving side interpolates the missing sample values received during the 4-field period. This is a sub-sample interpolation method that reproduces a video signal by delaying and interpolating sample values on the receiving side based on motion information between fields separated by one frame of the video signal detected on the transmitting side. In the motion compensation subsample interpolation method that corrects the interpolation position, motion information detected on the transmitting side is the motion information between the field of interest and a past field that is one frame apart, and the field of interest is one field from the field of interest. The motion information between the previous field and a past field that is one frame apart from the previous field, and the motion information between the field of interest and a past field that is one frame apart on the receiving side, and the above that is detected on the transmitting side. 2
Approximate prediction is made by subtracting 1/4 of the sum of the types of motion information from the former motion information, and the interpolated position of the sample values of the past three fields for the field of interest together with the former motion information detected on the transmitting side. A motion-compensated sub-sample interpolation method, characterized in that it corrects and performs inter-field interpolation.