JPH07114500B2 - Motion-corrected subsample interpolation method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial field of application] The present invention is applied to image processing in a MUSE-type high-definition television receiver, and calculates the amount of motion correction between fields from transmitted motion information. The present invention relates to a motion interpolation sub-sample interpolation method in which a missing point is calculated and interpolated between fields.

[Prior Art] This type of motion-compensated subsample interpolation method that has been proposed in the past was announced by NHK Broadcasting Technology Laboratories at the "Foundation Commemorative Lecture" in June 1984. (MUSE) ”is mentioned in the draft document. FIG. 8 shows the configuration of the motion compensation sub-sample interpolation method.

In FIG. 8, (1) is a video input terminal for inputting a video signal transmitted at a sample rate of 16.2 MHz, (2) is a switch, and the video signal input to the above-mentioned input terminal (1) and the movement described later. The output signal of the correction field memory (5) is switched at the timing of 32.4 MHz sub-sampling. (1
Reference numeral 0) is a non-motion correction field memory, which stores a signal for one frame having a sample rate of 32.4 MHz passing through the switch (2). Reference numeral (5) is a motion correction field memory, which stores a signal for one frame at a sample rate of 32.4 MHz output from the non-motion correction field memory (10), and performs motion correction using a motion vector.

(6) is an inter-field interpolation filter, which interpolates the missing sample points based on the output signal of the non-motion correction field memory (10) and the signal passed through the switch (2).
(7) is an intra-field interpolation filter that interpolates the missing sample points based on only the signal that has passed through the switch (2). (8) is a switch, which is connected to the upper contact (8x) when motion compensation is not performed and allows the output signal of the inter-field interpolation filter (6) to pass therethrough. It is connected to the side contact (8y) and passes the output signal of the inter-field interpolation filter (7). In (9), the missing sample points are interpolated and 6
With the sample rate of 4.8MHz, the above switch (8)
Is a video output terminal for outputting a signal passing through.

Next, the operation of the above configuration will be described.

The transmission method for high-definition television is sub-Nyquist sampling, which makes a cycle of four fields, and its required bandwidth is 8.1M.
Hz.

When the first field signal is input to the video input terminal (1), the non-motion compensation field memory (10) stores the second field signal and the fourth field signal one cycle before,
On the other hand, in the motion compensation field memory (5), the first
The signals of the field and the third field are stored.
When a motion vector exists due to panning of the camera, the contents of the motion correction field memory (5) move two-dimensionally according to the vector amount. At this time, if the transmitted motion vector is a motion correction amount between fields separated by one frame, the motion correction content of the motion correction field memory (5) is input to the video input terminal (1) as the first field. The signal is used as a reference.

The switch (2) is switched at the timing of 32.4 MHz sub-sampling, and the phase is inverted for each field and also by the motion vector. Therefore, in the above case, the signal of the first field input from the video input terminal (1) and the signal of the third field one cycle before the motion correction are passed through the switch (2).

The signal that has passed through the switch (2) and the output signal of the non-motion correction field memory (10) are input to the inter-field interpolation filter (6), and inter-field interpolation is performed.

When motion compensation is not performed, that is, when the switch (8) is connected to the upper contact (8x), a video signal with a sample rate of 64.8 MHz interpolated between fields is output from the video output terminal (9).

Also, the signal of the field that passed through switch (2) is
It is input to the inter-field interpolation filter (7) to perform inter-field interpolation, and the switch (8) contacts the lower contact (8y).
When connected to, the video output terminal (9) outputs a video signal with a sample rate of 64.8 MHz interpolated within the field.

When the a ₂ field is input to the video input terminal (1) without motion compensation, the states of the signals 9a to 9e in FIG. 8 are as shown in FIG.

However, when the signal is represented by a field, a ₁ , b ₁ , c ₁ , d ₁ ,
It flows in the order of a ₂ , b ₂ , c ₂ , d ₂ . A in FIG. 9 indicates an interpolated value interpolated from b ₁ , c ₁ , d ₁ , a ₂ by the interpolating function fA which is the inter-field processing.

Also, when motion compensation is performed and the a ₂ field is input to the video input terminal (1), the signal states of 10a to 10e in FIG. 8 are as shown in FIG. 10, and in FIG. The horizontal line above indicates that motion compensation has been performed, and B indicates the interpolated value interpolated from ₁ and a ₂ by the interpolating function fB, which is in-field processing.

[Problems to be Solved by the Invention] In the conventional motion compensation sub-sample interpolation method as described above, since the relationship between adjacent fields is not taken into consideration at the time of performing motion compensation, the missing sample points are not taken into consideration. Must be done in the field by an in-field interpolation filter. Therefore, there is a problem that the resolution at the time of performing the motion correction is lowered.

The present invention has been made in order to solve the above problems, and also when the motion vector between the fields is calculated from the transmitted motion vector and the motion correction is performed in the same manner as when the motion correction is not performed. It is an object of the present invention to provide a motion-compensated sub-sample interpolation method that can reduce the resolution by using an inter-field interpolation filter.

[Means for Solving the Problems] In the motion compensation sub-sample interpolation method according to the present invention, sample values transmitted by intermittently sub-sampling so as to make a cycle of four fields while maintaining a predetermined sample position. Is a sub-sample interpolation method for reproducing a video signal by interpolating the missing points from the sample values received in the four-field period on the receiving side based on the above. On the basis of the inter-field motion vector, the position of the sample value one frame before is corrected on the receiving side, and the missing point is interpolated to reproduce the video signal. The inter-field motion vector separated by 1 field from the inter-field motion vector Of vector
Relative position of the sample values of the past 3 fields based on the field of interest, using only the inter-field motion vector generated by this prediction on the receiving side and generated by approximating to 1/2. Is corrected, and the missing point is interpolated from the sample value received in the four-field period in the moving image as well as the interpolation in the still image.

[Operation] According to the present invention, the fact that there is no abrupt change in the transmitted motion information is used to convert the motion vector between fields separated by one field on the reception side into the field separated by one frame on the transmission side. It is generated by approximating prediction to 1/2 of the inter-motion vector, and using the inter-field motion vector generated above 1 frame apart, always using the inter-field interpolation filter By performing inter-field interpolation in the same manner as described above, it is possible to improve the resolution in the case of a moving image.

[Embodiment] An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing a motion compensation sub-sample interpolation method according to an embodiment of the present invention. In FIG.
(1) is a video input terminal for inputting a video signal transmitted at a sample rate of 16.2 MHz, (2) is a switch, and the video signal input to the video input terminal (1) and a second motion correction described later. The output signal of the field memory (5) is set to 32.4
Switching at the timing of the subsample of MHz (3) is a divider, which reduces the transmitted motion vector to 1/2
Divide by. (4) is a first motion compensation field memory, which stores a signal for one frame having a sample rate of 32.4 MHz passing through the switch (2) and stores it in the divider (3).
The motion compensation is performed by using the motion vector predictor between the fields output from.

(5) is a second motion compensation field memory,
32.4 output from the motion compensation field memory (4)
A signal for one frame at the sample rate of MHZ is stored, and motion correction is performed by using the predicted motion vector between the fields, which is the output of the divider (3).

(6) is an inter-field interpolation filter, which interpolates the missing sample points based on the signal passing through the switch (2) and the output signal of the first motion correction field memory (4). (7) is an intra-field interpolation filter which interpolates the missing sample points based on only the signal passing through the switch (2). (8) is a switch, which is normally connected to the upper contact (8x) to allow the output signal of the inter-field interpolation filter (6) to pass through, and is connected to the lower contact (8y) pixel by pixel when motion is detected. The output signal of the inter-field interpolation filter (7) is passed. In (9), the missing sample points are interpolated, resulting in a sample rate of 64.8 MHz,
A video output terminal for outputting a signal passing through the switch (8).

Next, the operation of the above configuration will be described.

FIG. 2 shows a motion vector when panning occurs in order to specifically explain the operation of FIG. 1, where x represents a screen horizontal axis and y represents a pixel vertical axis.

Each field of the video signal is represented by a _{1 to} d ₂ , and when the signal is represented by a field ... a ₀ , b ₀ , c ₀ , d ₀ , a ₁ , c ₁ , d ₁ , a ₂ , b ₂ , c ₂ ,
d ₂ ……. The inter-field motion vectors separated by one frame are ... A ₁ , B ₁ , C ₁ , D ₁ , A ₂ , B ₂ ,.
It is indicated by…. In addition, the motion vectors between the fields that are separated by one field that are obtained in the drawing are ... b ₁₀ , c ₁₀ , d ₁₀ , a ₂₀
Represented by ……. The following relational expression holds between the inter-field motion vector separated by one frame and the inter-field motion vector separated by one field.

_{a 10 + b 10 = A 1} b 10 + c 10 = B 1 c 10 + d 10 = C 1 Here, for example consider only the relationship 3 field of a ₁ to c _1. However, the inter-field motion vector separated by one frame has a horizontal component of 5 bits and a vertical component of 3 bits.
One bit digital signal is transmitted per field. In other words, when describing the magnitude of the inter-field motion vector separated by one frame in terms of pixel length, the horizontal component is within the range of −15 to +16, the vertical component is within −3 to +4, and the horizontal component and the vertical component are The direction component also has a discrete component that is an integer. When this is illustrated, as shown in FIG. 3, the inter-field motion vectors separated by one frame are within the range surrounded by the broken line. However, the origin in FIG. 3 is an arbitrary pixel of a ₁ field.

If the parallel movement of the image due to panning is smooth, a ₁
The inter-field motion vector separated by one field between the field and b ₁ field is within the range surrounded by the chain line in FIG. Here, the relative relationship of the three fields a _{1 to} c ₁ is shown in FIGS.

First, FIG. 4 shows the case where the inter-field motion vector A ₁ transmitted one frame apart has the maximum magnitude. At this time, a motion vector a between fields separated by one field
₁₀ must also be the maximum. The inter-field motion vector b ₁₀ separated by one field is also within the range of the chain double-dashed line, but it must also be the maximum value. Therefore, the vector b ₁₀ coincides with A _1/2 .

Next, FIG. 5 shows a case where the component of the inter-field motion vector A ₁ separated by one frame is (x, y) = (10, 2). As in FIG. 4, the inter-field motion vector a ₁₀ separated by one field exists within the range surrounded by the chain line, and when the b ₁ field is at the point shown in the figure, the inter-field motion vector b ₁₀ separated by _one field is shown. Exists within a range surrounded by a chain double-dashed line. In fact, such movement by panning is possible because the c ₁ field is within the range enclosed by the chain double-dashed line as shown.

From the above, when the relative positions of the a ₁ field and the c ₁ field are determined by the inter-field motion vector A ₁ transmitted one frame apart, the position of the b ₁ field with respect to those fields is It is within the shaded area. Moreover, if the parallel movement of the image due to panning is smooth, the b ₁ field position is centered on the midpoint between the positions of the a ₁ field and c ₁ field indicated by the black circles in FIG. It is highly possible that they exist on a 3 × 3 square lattice. Therefore, it is reasonable to substitute the vector b ₁₀ with A _1/2 , and the error is about 1 pixel.

Next, FIG. 6 shows a case where the component of the inter-field motion vector A separated by one frame is (x, y) = (3, 1). As in FIG. 4, the inter-field motion vector a ₁₀ separated by one field exists within the range surrounded by the chain line, and when the b ₁ field is at the point shown in the figure, the inter-field motion vector b ₁₀ separated by _one field is shown. Exists within a range surrounded by a chain double-dashed line. In fact, since the c ₁ field is within the chain double-dashed line as shown, such movement by panning is possible. Since there is no panning in which the direction change of the motion vector between the fields separated by one field adjacent to each other exceeds a right angle, the motion vector A ₁ between the fields separated by one frame is transmitted as a ₁ field.
When the relative positions of the c ₁ fields are defined, the positions of the b ₁ fields with respect to those fields are within the shaded area. Furthermore, if the image is translated smoothly by panning, the position of the b ₁ field is centered on the midpoint between the positions of the a ₁ field and c ₁ field shown by the black circles in FIG. It is highly possible that they exist on the 2 × 2 square lattice. Was but connexion, the vector b ₁₀ A _1/2
However, since the component of A _1/2 in this case does not take an integer value, the vector connecting any one of the black circles in FIG. 6 and the position of c ₁ field is substituted for the vector b ₁₀ . Even in this case, the error is about one pixel.

As described above, the inter-field motion vector b ₁₀ separated by one field can be predicted by approximation using the inter-field motion vector A ₁ separated by one frame and the integer part of A _1/2 , and the error is about 1 pixel. Or less. Similarly, it is _{a 10 ≒ [D 0/2} ] b 10 ≒ prediction as _{[A 1/2] c 10} ≒ [B 1/2]. However, [] represents an integer part.

Moreover, in the prediction of the inter-field motion vector separated by one field, the error does not affect the subsequent prediction, so even if there is a temporary large vector change and the error becomes large, the subsequent vector changes will not become gradual. For example, the prediction with small error can be made again.

The motion correction performed by the above prediction will be described with reference to FIG.

First, when a signal of d ₁ field is input to the video input terminal (1) of FIG. 1, the first motion compensation field memory (4)
The signals of a ₁ field and c ₁ field are stored in, and the signals of d ₀ field and b ₁ field are stored in the second motion compensation field memory (5). At this time, the motion vector
When B ₁ is input, the motion vector B _{1 is} calculated by the divider (3).
Is divided by B _1/2 , and the integer part of the output is the predicted value of the inter-field motion vector c ₁₀ separated by one field.
The contents of the first motion compensation field memory (4) is moved by the prediction vector of the field between the motion vector c ₁₀ that are separated one field [B _1/2] two-dimensionally, the input to the video input terminal (1) Performs motion compensation based on the d ₁ field that is set.

On the other hand, it is stored in the second field memory (5).
The signals of d ₀ field and b ₁ field are stored in the first motion compensation field memory (4) one field before that, and the signal of c ₁ field input to the video input terminal (1) at this time is prediction vector [a _1/2] by because the motion compensation has been made, d ₁ first motion relative to the field correction field memory (4) Like the prediction vector of the field between the motion vector b ₁₀ that are separated one field to the reference it may be carried out motion compensation in [B _1/2].

In this way, motion compensation was performed based on the d ₁ field.
Signals for 4 fields of a ₁ field, b ₁ field, c ₁ field, d ₁ field itself enter inter-field interpolation filter (6) to enable inter-field interpolation.
Normally, the switch (8) is connected to the upper contact (8x), and the signal interpolated between fields passes through, but for the video signal, interpolation of the missing sample points cannot be performed by inter-field interpolation, so motion detection At this time, the switch (8) is connected to the lower contact (8y) on a pixel-by-pixel basis, and the intra-field interpolation is performed only from the signal that has passed through the switch (2) by the intra-field interpolation filter (7).

The sample rate of the signal in which the missing sample points are interpolated by the inter-field interpolation filter (6) or the intra-field interpolation filter (7) is 64.8 MHz and is output from the video output terminal (9).

In the motion-compensated sub-sample interpolation method that obtains inter-field motion vectors separated by one field by such prediction, not only is it stable at the beginning and end of panning, but also the transmitted motion vector has a temporary error. Even if it does, it will not have any adverse effects.

It is also stable in special cases where the scene changes during panning and the image becomes a panning image. Furthermore, this method is very good because the change of the motion vector due to panning is gradual.

FIG. 7 shows the states of the signals from 7a to 7f in FIG. 1 when the d ₁ field signal is input to the video input terminal (1).
In FIG. 7, the horizontal line above the symbol indicates that motion compensation has been performed, A is the interpolation value interpolated from ₁ , ₁ , ₁ , ₁ , d ₁ by the interpolation function fA, and B is ₁ by the interpolation function fB. Indicates the interpolated value interpolated from d ₁ .

[Effects of the Invention] As described above, according to the motion compensation sub-sample interpolation method of the present invention, samples are transmitted by intermittently sub-sampling so as to make a cycle of four fields while maintaining a predetermined sample position. A sub-sample interpolation method for reproducing a video signal by interpolating the missing points from the sample value received in the four-field period on the reception side based on the value, and one frame of the video signal detected on the transmission side. In the motion correction sub-sample interpolation method in which the position of the sample value one frame before is corrected on the receiving side based on the separated inter-field motion vector, and the missing point is interpolated to reproduce the video signal, From the inter-field motion vector separated, the inter-field motion vector separated by one field on the receiving side is converted into the inter-field motion vector separated by one frame. Generated by approximating to 1/2 of the vector, and using only the inter-field motion vector generated by this receiving side and separated by one field, based on the field of interest, the sample values of the past three fields Since the relative position is corrected and the missing point is interpolated from the sample value received in the 4-field period in the moving image as well as the interpolation in the still image, in the moving image also in the 4-field period as in the still image. The missing point can be interpolated from the received sample value, and as a result, an image with excellent resolution can be obtained even in the case of a moving image in which panning has occurred.

[Brief description of drawings]

FIG. 1 is a block diagram showing an example of a schematic configuration of a motion correction sub-sample interpolation method according to an embodiment of the present invention, and FIG. 2 is an example of a motion vector at the time of panning for explaining the operation of FIG. The vector diagram shown in FIG. 3 is a diagram showing existence ranges of inter-field motion vectors separated by one frame and inter-field motion vectors separated by one field, FIG. 4, FIG.
FIG. 6 and FIG. 6 are vector diagrams showing the relationship between the inter-field motion vector separated by one frame and the inter-field motion vector separated by one field, and FIG. 7 is a signal flow of the motion compensation sub-sample interpolation method according to the present invention. FIG. 8 is a block diagram showing an example of a schematic configuration of a conventional motion compensation sub-sample interpolation method, and FIG. 9 is a signal when motion compensation of the conventional motion compensation sub-sample interpolation method is not performed. Timing chart showing the flow of
FIG. 10 is a timing chart showing a signal flow in the case of performing the motion compensation of the conventional motion compensation sub-sample interpolation method. (1) …… Video input terminal, (2) …… Switch, (3)
…… Divider, (4) …… First motion compensation field memory, (5) …… Second motion compensation field memory,
(6) ... Inter-field interpolation filter, (7) ... Inter-field interpolation filter, (8) ... Switch, (9) ...
… Video output terminal, (10) …… Non-motion compensation field memory. In the drawings, the same reference numerals indicate the same or corresponding parts.

Claims (1)

[Claims] 1. Based on a sample value transmitted by intermittently sub-sampling so as to make a cycle of four fields while maintaining a predetermined sample position, the sample value is received from the sample value received in a four-field period on the receiving side. A sub-sample interpolation method for reproducing a video signal by interpolating a missing point, and based on inter-field motion vectors separated by one frame of the video signal detected on the transmission side, the above-mentioned sample value one frame before on the reception side In the motion-compensated sub-sample interpolation method in which the position of is corrected and the missing point is interpolated to reproduce the video signal, from the inter-field motion vector separated by one frame,
The inter-field motion vector separated by one field on the receiving side is generated by approximating and predicting to 1/2 of the inter-field motion vector separated by one frame, and the inter-field motion separated by one field generated on the receiving side. Using only the vector, the relative positions of the sample values of the past 3 fields are corrected based on the field of interest, and the missing points are extracted from the sample values received in the 4 field period in the moving image as well as the interpolation in the still image. A motion-compensated sub-sample interpolation method characterized by interpolating.