JPS63245188A - Sub-sample interpolating system for movement correction - Google Patents

Abstract

PURPOSE:To constantly carry out an interfield interpolation and to suppress the deterioration in a resolution at the time of correcting a movement by approximating a movement vector between the fields spaced by one field to 1/4 of the sum of the two types of the movement vectors of the adjacent fields of a transmission side and predicting. CONSTITUTION:A switch 2 switches a video signal inputted to a video input terminal 1 and the output signal of a movement correcting field memory 8 in the timing of a sub-sample. An adder 4 adds a transmitted movement vector and a movement vector before one field transmitted from a one field delay device 3. A divider 5 divides the output of the adder 4 by four. A movement correcting field memory 7 stores a signal of the one frame of a sample rate to correct the movement by a predictive movement vector between the fields outputted from a subtracter 6. The memory 8 stores the signal of the one frame of the sample rate outputted from the memory 7 to correct the movement according to an output from the subtracter 6. A interfield interpolating filter 9 interpolates a defective sample point based on a signal passing through the switch 2 and the output signal of the memory 7.

Description

[Detailed Description of the Invention] [Industrial Application Field] This invention is applied to image processing in a MUSE system high-definition television receiver, and calculates the amount of motion correction between each field from the transmitted fh & information. The present invention relates to a motion compensation subsample interpolation method that performs calculations to interpolate missing points between fields.

[Prior art] This type of motion compensation sub-sample interpolation method that has been proposed in the past is based on the ``New transmission method for high-definition television'' announced by the NHK Broadcasting Technology Research Institute at its founding commemorative lecture in June 1980. There is something mentioned in the preliminary materials called (MUSE)J.

FIG. 11 is a block circuit diagram of a configuration example that performs this motion compensation subsample interpolation method.

In the figure, (1) is a video input terminal that inputs 1/4 video signal transmitted at a sample rate of 18.2M HI;
(2) is a switch that switches between the video signal input to the input terminal (1) and the output signal of a motion correction field memory (8), which will be described later, at the timing of 32.4 MH2 sub-samples. (13) is a non-motion compensation field memory that stores 1/4 of the signal for one frame of the 32.4 MH2 sample rate that has passed through the switch (2). Reference numeral (8) denotes a motion correction field memory, which stores 1/4 of one frame of the sample rate of 32.4 MHz output from the non-correction field memory (13), and performs motion correction using a motion vector. (9) is an interfield interpolation filter that interpolates missing sample points based on the output signal of the non-motion compensation field memory (13) and the signal passed through the switch (2). (io) is an intra-field filter that interpolates missing sample points based only on the signal that has passed through switch (2). (11) is a switch, and when not performing motion compensation, the upper contact (
llx) connected to the interfield interpolation filter (3)
When performing motion correction or when motion is detected, it is connected to the lower contact (fly) and output signal 1 of the field inner sleeve filter (10) is passed through.
/4 is passed. (12) is a video output terminal for interpolating missing sample points and outputting 1/4 of the signal that has passed through switch (11) at a sample rate of 84.8MH7.

Next, the operation of the above configuration will be explained.

The transmission method for high-definition television is sub-Nyquist sampling, which consists of four fields in one cycle, and the required bandwidth is 8.
It is 1MHz.

Now, 1/4 of the video signal input to the video input terminal (1) is expressed as a l + b l + with 4 fields as one unit.
Expressed as CI + dl, -... ao, bo + co
+dotl+tz+c+.

d+, a2. bl, c2. dz..., and if the a2 field signal is now input to the video input terminal (1), the non-motion compensation field memory (13) contains the signal from the previous round. 2 field b1
and the signal of the fourth field d are stored,
On the other hand, the motion correction field memory (8) contains the first field a1 from the previous round and the third field C! signals are stored.

When a motion vector is present due to panning of the camera, the contents of the motion correction field memory (8) move two-dimensionally according to the amount of the vector. At this time, if the transmitted motion vector is the amount of motion correction between fields separated by one frame, the motion correction field memory (8
) is performed based on the signal 1/4 of the first field a2 input to the video input terminal (L).

The switch (2) is switched at the timing of 32.4 MH2 sub-samples, and the phase is inverted for each field and also depending on the motion vector. Therefore, in the above case, the signal of the first field a2 input from the video input terminal (1) and the signal of the third field a2 from the previous round where motion correction was performed
The signal in field 31 passes through switch (2).

The signal passed through the switch (2) and the output signal of the non-motion compensated field memory (13) are input to an interfield interpolation filter (9) to perform interfield interpolation.

f! When not performing h correction, that is, switch (11
) is connected to the upper contact (llx), 84.8M interpolated between fields is output from the video output terminal (12).
Outputs 1/4 video signal with a sample rate of Hz.

In addition, the field signal that has passed through the switch (2) is
It is input to the field inner sleeve filter (10), performs the field inner sleeve filter, and the switch (11) switches the lower contact (
fly), the video output terminal (12) outputs 1/4 video signal with a sample rate of 134.8 MHz, which is inter-field spaced.

The signals (12a) to (12e) of each part in FIG. 11 when the a2 field is input to the video input terminal (1) without motion correction are shown in FIGS. 12(a) to (II).
), and A in FIG. 12(e) is bl, C1, d by the interpolation function f^, which is interpolation between fields.
It shows the interpolated value interpolated from lla2.

In addition, motion compensation is performed, and a2 is input to the video input terminal (1).
The signals (13a) to (13e) of each part in FIG. 11 when the field is input are as shown in FIG. 13(a) to (13e).
e), the horizontal line above the symbol in Fig. 13 indicates that motion compensation has been performed, and B in Fig. 13 is C++ az
Indicates the interpolated value interpolated from .

[Problem to be Solved by the Invention 1] When using the conventional motion compensation subsample interpolation method as described above, the relationship between adjacent fields is not taken into consideration at all when performing motion compensation, so Interpolation of the sample points must be performed within the field by means of an intrafield filter (10), so that
There is a problem in that the resolution decreases when motion compensation is performed.

The present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to provide a motion compensation subsample interpolation method that can suppress a decrease in resolution even when motion compensation is performed.

[Means for Solving the Problems] The motion compensation subsample interpolation method according to the present invention calculates motion vectors between fields separated by one field from motion vectors transmitted between fields separated by one frame by approximation. The present invention is characterized in that it is possible to perform inter-field interpolation by correcting all signals from the first field to the fourth field by ih based on a predetermined value Δ11.

[Operation] According to the present invention, by taking advantage of the fact that there is no sudden change in the transmitted motion vector, the motion vector between fields separated by one field is determined by two types of motion of adjacent fields on the transmission side. By making predictions by approximating the sum of vectors, interpolation can always be performed using an interfield interpolation filter, thereby suppressing a decrease in resolution during motion correction.

[Embodiment of the Invention] Hereinafter, an embodiment of the present invention will be described based on the drawings.

FIG. 1 is a block circuit diagram of this embodiment, in which the same symbols as in FIG. 11 indicate the same parts, respectively.
The switch (2) connects the video signal input to the video input terminal (1) and the second motion compensation field memory (described later).
5) is switched at the timing of the 32.4 MHz sub-sample. (3) is a 1-field delay device,
The transmitted motion vector is delayed by one field.

(4) is an adder that adds the transmitted motion vector and l
The motion vector of one field before outputted from the field delay device (3) is added. (5) is a divider which divides the output of adder (4) by 4. (6) is a subtracter that subtracts the output of the divider (5) from the transmitted motion vector. (7) is the first motion compensation field memory;
1/4 of the signal for one frame of the 32.4 MHz sample rate passed through the switch (2) is stored, and the subtracter (θ)
Motion correction is performed using the predicted motion vector between fields output from the first motion correction field memory (8), which performs motion correction using the predicted motion vector between fields output from the first motion correction field memory (7). The interfield interpolation filter (8) stores 1/4 signal for one frame and performs motion correction using the predicted interfield motion vector output from the subtracter (6).
2) and the output signal of the first motion correction field memory (7), the missing sample points are interpolated. The operations of the intra-field sleeve filter (lO) and the switch (11) are the same as in the conventional example shown in FIG.

Next, the operation of this embodiment will be explained.

In order to specifically explain the operation shown in Figure 1, Figure 2 shows the motion vector when banging occurs, and in the figure, the X axis represents the horizontal axis of the screen, and the y axis represents the vertical axis of the screen. .

If each field of the video signal is expressed as ao-d2, °-a
o, bo+co+do+a++tz+c++d+, a2
．． b2+c2. In the 0 diagram flowing in the order of d2-..., the inter-field motion vectors separated by one frame are...A I + B l * CI + DI +
It is expressed as A 2 + B 2 *..., and the inter-field movement □---n-)-n-) written on the diagram is expressed as 1°-b+o, CItl+cllO,
Expressed as a+o.-, the following relational expression holds between the inter-field motion vectors separated by one frame and the inter-field motion vectors separated by (1) field.

Here, for example, blG can be rewritten as in the following equation.

” - If the second term on the right side of the equation is sufficiently small, blO can be predicted as follows.

→ Al +331 blG=□ Here, for example, only the relationship between the four fields a1 to dl will be considered. However, the inter-field motion vectors separated by one frame are digital signals of 5 bits in the horizontal direction and 3 bits in the vertical direction, and are transmitted once per field. In other words, if we describe the magnitude of the motion vector between fields separated by l frames in terms of pixel length, the horizontal component is in the range -15 to +16, the vertical component is in the range -3 to +4, and the horizontal component, vertical component It has a discrete component whose direction component is also an integer. To illustrate this, the third
As shown in the figure, the inter-field motion vectors separated by one frame are within the range surrounded by broken lines. However, the origin in FIG. 3 is an arbitrary pixel in the al field.

If the parallel movement of the image due to panning is smooth, a
The inter-field motion vectors separated by one field between the 1 field and the BL field are within the range surrounded by the dashed line in FIG. Here, the relative relationship between the three fields of at-C4 is changed in three ways, and the results shown in FIGS. 4, 6,
The prediction accuracy of the position of the dl field and the motion vector will be investigated for each case as shown in FIG.

First, FIG. 4 shows the case where the magnitude of the inter-field motion vector A1 transmitted one frame apart is maximum.

At this time, the interfield motion vectors &Le and bIσ separated by l fields also become maximum, and the relative positions of fields a1 to C1 are determined. At this time, the d1 field exists within the range indicated by the three-dot chain line in the figure, but if there is no sudden change in motion, the dl field exists within the shaded range. Furthermore, if the parallel movement of the image, such as panning, is smooth, there is a high possibility that the dt field will exist at the position indicated by the black circle in the figure.

- As an example, the predicted vector (Az+Bt) /4 when the dl field is in the position shown, and the pect 1
This is the pixel length, and if the di field is located at the position indicated by the black circle, the error will be less than that, so such prediction is appropriate.

Next, in FIG. 6, the components of the inter-field motion vector Az separated by one frame are (x, y) = (10, 2
) is shown. Similar to FIG. 4, the inter-field motion vector atO, which is separated by one field, exists within the range surrounded by the dashed line.

When the b1 field is at the point shown in the figure, the inter-field motion vector blG separated by the ■ field exists within the range surrounded by the two-dot chain line. In fact, such movement by panning is possible because the C1 field is within the range enclosed by the dashed-double line as shown.

At this time, the dl field exists within the range surrounded by the three-dot chain line in the figure, but as in the explanation in Figure 4,
The predicted vector (Al+8
+) /4 and the vector blo are shown in Figure 7. The prediction error at this time is one pixel length on the diagonal, and if there is a d1 field at the position indicated by the black circle, the error will be less than that, so Figure 6 The prediction is also valid.

Next, FIG. 8 shows a case where the components of the inter-field motion vector A1 separated by one frame are (x, y)=(3°l). Similarly to FIG. 4, the inter-field motion vector alo that is separated by one field exists within the range surrounded by the dashed line, and when the bl field is at the point shown in the figure, the inter-field motion vector bo that is separated by one field is
o exists within the range surrounded by the two-dot chain line. Fact, C.I.
Such movement by panning is possible because the field is within the range enclosed by the dashed-two dotted line as shown. At this time, the di field exists within the range indicated by the three-dot chain line in the figure, but as in the explanation in Figure 4, it is also within the diagonally shaded range, and there is a very high possibility that it exists at the position indicated by the black circle. , - As an example, the dl field is /4 and the vector blQ is shown in Fig. 9. In this case, the prediction error is one pixel length on the diagonal, and if the dl field is at the position indicated by the black circle, the error is Since it is less than that, the prediction is valid also in the case of FIG.

The prediction is also valid in the case of the figure.

As described above, the inter-field motion vector blO separated by 1 field is the motion vector blO separated by 1 frame (A+ +
Prediction can be made by approximating the integer part of B+)/4, and the error is about one pixel on the diagonal or less. Similarly, prediction can be made as c+o'=[(B++C+)/4]. However, [ ] represents the integer part.

However, in the motion compensation subsample interpolation method that has been proposed in the past, if we consider the timing of the transmitted video signal and motion vector, the interfield motion vector separated by l fields is the interfield motion vector separated by one frame. Using the vector, we have to predict as follows.

Moreover, the error in prediction of inter-field motion vectors separated by l fields does not affect subsequent predictions, so even if there is a temporary large vector change and the error becomes large, the subsequent vector change will be gentle. Once again, predictions with small errors can be made.

Motion correction performed by prediction as described above will be explained using timing charts shown in FIGS. 1 and 10.

First, when a d1 field signal is input to the video input terminal (1), the first motion correction field memory (7)
The signals of the a1 field and the C1 field are
In addition, the second motion correction field memory (8)
field, and b1 field signals are stored. At this time, motion vector B! is input, the output of the l-field delay device (3) is outputted as the motion vector of one field before, which is further divided by 4 in the divider (5), and subtracted from B1 in the subtracter (8) to obtain the motion vector of the previous field. Output B1-
The integer part of %(A1+Bt) becomes the predicted value of the inter-field motion vector CIO separated by one field. The contents of the first motion correction field memory (7) are moved to the source of the inter-field motion pect separated by l fields, and motion correction is performed based on the dl field input to the 1 video input terminal (1).

On the other hand, the signals of the dO field and bl field stored in the second field memory (8) were stored in the first motion compensation field memory (7) one field before, and at this time Input terminal (1
), based on 1/4 of the C1 field input in
The predicted vectors of the inter-field motion vectors blo separated by fields are the same as the first motion correction field memory (7) based on the dl field. The al field and bl are thus motion-corrected based on the dl field. The signals for the four fields of field, C1 field, and d1 field enter the interfield interpolation filter (9), which enables interfield interpolation.Normally, the switch (11) is connected to the upper contact (llx). The inter-field interpolated signal passes through the video signal, but when motion is detected, the switch (11) connects to the lower contact (lly) in pixel units. The intra-field interpolation is performed only from the signal that has been passed through the switch (2) by the intra-field interpolation filter (lO).

The sample rate of the signal whose missing sample points are interpolated by the interfield interpolation filter (8) or the intrafield interpolation filter (10) is 64.8MH1, and is output from the video output terminal (12).

In an example in which motion vectors between fields separated by one field are determined by such prediction, not only is it stable at the start and end of panning, but even if there is a temporary error in the transmitted motion vector, it is Leaves no negative impact.

It is also stable even in special cases where the scene changes during panning or the image changes during panning. Furthermore, changes in motion vectors due to panning become more linear.

Figure 10 shows the states of the signals (10a) to (10f) in Figure 1 when the C2 field signal is input to the video input terminal (1). The horizontal line indicates that the motion has been corrected, and A is the interpolated value interpolated from bl, c+, dl, a2 by the interpolation function fA, and B
indicates an interpolated value interpolated from dl and a2 by the interpolation function fB.

[Effects of the Invention] As described above, according to the present invention, a motion vector for one field is approximately predicted from a transmitted motion vector,
Based on this approximate prediction, we configured the structure to perform motion compensation for each field, so as in the still image state,
Even if panning occurs, interpolation by the interfield interpolation filter can be stabilized, and interpolation with less error can be performed, so that it is possible to sufficiently suppress a decrease in resolution during panning.

[Brief explanation of the drawing]

Fig. 1 is a block circuit diagram of an embodiment of the present invention, Fig. 2 is a vector diagram showing an example of motion vectors during panning to explain the operation of this embodiment, and Fig. 3 is a field separated by one frame. A diagram showing the existence range of inter-field motion vectors and inter-field motion vectors separated by one field, FIG.
6 and 8 are vector diagrams showing the relationship between inter-field motion vectors separated by l frames and inter-field motion vectors separated by 1 field, respectively;
9 and 9 are vector diagrams showing examples of inter-field motion vectors separated by one field predicted from inter-field motion vectors separated by one frame and actual inter-field motion vectors separated by one field, respectively, and FIG. A timing chart showing the signal flow of the embodiment,
Fig. 11 is a block circuit diagram of a configuration example in which the conventional motion correction sub-sample interpolation method is applied, Fig. 12 is a timing chart showing the signal flow when motion correction is not performed in this conventional example, and Fig. 13. is a timing chart showing the flow of signals when motion correction is similarly performed. (2)...switch, (3)...1 field delayer, (0...adder, (5)...divider, (6)...
...Subtractor, (7)...First motion compensation field memory, (8)...! 82 motion correction field memory, (9)...interfield interpolation filter, (lO).
. . . field inner sleeve filter, (11) . . . switch, (12) . . . video input terminal. In each figure, the same reference numerals indicate the same or corresponding parts.

Claims (1)

[Claims] (1) The sample values that are intermittently subsampled and transmitted at predetermined sample positions that go around in four fields are interpolated on the receiving side to reproduce the video signal, and then transmitted. In the motion compensation subsample interpolation method, which corrects the interpolation position of sample values to be interpolated by delaying the video signal based on motion information between fields separated by one frame of the video signal detected on the side, the field of interest and this First motion information between a past field that is one frame apart from the field, and second motion information between a field that is one field before the field of interest and a past field that is one frame apart from this field. Find 1/4 of the sum of , subtract this value from the first motion information to approximately predict the motion information of the field of interest, and based only on this approximately predicted motion information, calculate the motion information of the field of interest. A motion compensation subsample interpolation method characterized in that interpolation positions of sample values of the past three fields are corrected to interpolate missing sample values.