JPH07114497B2 - Motion-corrected subsample interpolation method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial field of application] The present invention is applied to image processing in a MUSE-type high-definition television receiver, and calculates the amount of motion correction between fields from transmitted motion information. The present invention relates to a motion-corrected sub-sample interpolation method in which a missing point is calculated and interpolated between fields.

[Prior Art] NHK Research Institute of Technology and NHK Science and Technology Research Laboratories announced at the foundation commemorative lecture in June 1984 to the motion compensation subsample interpolation method of this kind that has been proposed so far. There is one mentioned in a preliminary document called "New Transmission System (MUSE) of High Quality Television". FIG. 8 shows the configuration of the motion compensation sub-sample interpolation method.

In FIG. 8, (1) is a video input terminal for inputting a video signal transmitted at a sampling rate of 16.2 MHz, and (2) is a switch, which is a video signal input to the input terminal (1) and a motion described later. The output signal of the correction field memory (4) is switched at the timing of 32.4 MHz sub-sampling.
(3) is a non-motion correction field memory, which stores a signal for one frame having a sample rate of 16.2 MHz passing through the switch (2). Reference numeral (4) is a motion correction field memory, which stores a signal for one frame at a sample rate of 32.4 MHz output from the non-correction field memory (3) and performs motion correction using a motion vector.

(7) is an inter-field interpolation filter, which interpolates the missing sample points based on the output signal of the non-motion correction field memory (3) and the signal passed through the switch (2).
(8) is an intra-field interpolation filter, which interpolates the missing sample points based on only the signal that has passed through the switch (2). (9) is a switch, which is connected to the upper contact (9x) when motion compensation is not performed and allows the output signal of the inter-field interpolation filter (7) to pass therethrough. When motion compensation is performed or motion detection is performed, the lower side is used. It is connected to the contact (9y) and allows the output signal of the in-field interpolation filter (8) to pass through. In (10), the missing sample points are interpolated to 64.8MH
This is a video output terminal for outputting a signal that passes through the switch (9) at the sample rate of z.

Next, the operation of the above configuration will be described.

The transmission method for high-definition television is sub-Nyquist sampling, which makes a cycle of four fields, and its required bandwidth is 8.1M.
Hz.

When the signal of the first field of the video output terminal (1) is input, the non-motion compensation field memory (3) stores the signals of the second field and the fourth field of one cycle before,
On the other hand, in the motion compensation field memory (4), the first
The signals of the field and the third field are stored.
When a motion vector exists due to panning of the camera, the contents of the motion correction field memory (4) move two-dimensionally according to the amount of the motion vector. At this time, if the transmitted motion vector is a motion correction amount between fields separated by one frame, the content of the motion correction of the motion correction field memory (4) is input to the video output terminal (1). It is performed based on the field signal.

The switch (2) is switched at the timing of 32.4 MHz sub-sampling, and the phase is inverted for each field and also by the motion vector. Therefore, in the above case, the signal of the first field input from the video output terminal (1) and the signal of the third field one cycle before the motion correction are passed through the switch (2).

The signal that has passed through the switch (2) and the output signal of the non-motion correction field memory (3) are input to the inter-field interpolation filter (7) to perform inter-field interpolation.

When motion compensation is not performed, that is, when the switch (9) is connected to the upper contact (9x), the video output terminal (1
The video signal with the sample rate of 64.8 MHz interpolated between fields is output from 0).

Also, the signal of the field that passed through switch (2) is
In-field interpolation filter (8) is used to perform in-field interpolation, and when the switch (9) is connected to the lower contact (9y), the 64.8MHz inter-field interpolation from the video output terminal (10) is performed. Output a video signal at the sample rate.

When the a ₂ field is input to the video input terminal (1) without motion compensation, the states of the signals 9a to 9e in FIG. 8 are as shown in FIG.

However, if the signal is expressed as a field ... a ₁ , b ₁ , c ₁ , d ₁ ,
It flows in the order of a ₂ , b ₂ , c, d ₂ . A in FIG. 9 indicates an interpolated value interpolated from b ₁ , c ₁ , d ₁ , a ₂ by the interpolating function fA which is the inter-field processing.

When motion compensation is performed and the a ₂ field is input to the video input terminal (1), the states of signals 10a to 10e in FIG. 8 are as shown in FIG. 10, and in FIG. The horizontal line above the symbol indicates that the motion correction has been performed, and B indicates the interpolated value interpolated from ₁ and a ₂ by the interpolating relation fB which is in-field processing.

[Problems to be Solved by the Invention] In the conventional motion compensation sub-sample interpolation method as described above, since the relationship between adjacent fields is not taken into consideration at the time of performing motion compensation, the missing sample points are not taken into consideration. Must be done in the field by an in-field interpolation filter. Therefore, there is a problem that the resolution at the time of performing the motion correction is lowered.

The present invention has been made in order to solve the above problems, and also when the motion vector between the fields is calculated from the transmitted motion vector and the motion correction is performed in the same manner as when the motion correction is not performed. It is an object of the present invention to provide a motion-compensated sub-sample interpolation method that can reduce the resolution by using an inter-field interpolation filter.

[Means for Solving the Problems] In the motion compensation sub-sample interpolation method according to the present invention, sample values transmitted by intermittently sub-sampling so as to make a cycle of four fields while maintaining a predetermined sample position. Is a sub-sample interpolation method for reproducing a video signal by interpolating the missing points from the sample values received in the four-field period on the receiving side based on the above. On the basis of the inter-field motion vector, the position of the sample value one frame before is corrected on the receiving side, and the missing point is interpolated to reproduce the video signal. The inter-field motion vector separated by 1 field from the inter-field motion vector Of vector
Generate by performing an approximate prediction to 1/2 and use the inter-field motion vector separated by one frame detected on the transmission side and the inter-field motion vector separated by one field generated on the reception side, and pay attention. The relative positions of the sample values in the past three fields are corrected based on the field, and the missing points are interpolated from the sample values received in the four field period in the moving image as well as the interpolation in the still image. .

[Operation] According to the present invention, the fact that there is no abrupt change in the transmitted motion information is used to convert the motion vector between fields separated by one field on the reception side into the field separated by one frame on the transmission side. The inter-field motion vector generated by approximating to 1/2 of the inter-motion vector and detected by the transmitting side and using the inter-field motion vector generated by the receiving side and using the inter-field motion vector generated by the receiving side. Thus, even in the case of a moving image by always using the inter-field interpolation filter, it is possible to improve the resolution in the case of a moving image by performing the inter-field interpolation similarly to the still image.

[Embodiment] An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing a motion correction sub-sample interpolation method according to an embodiment of the present invention. In FIG.
(1) is a video input terminal for inputting a video signal transmitted at a sample rate of 16.2 MHz, (2) is a switch, which is a video signal input to the input terminal (1) and a first motion compensation field described later. Output signal of memory (4) is 32.4MHz
Switch at the timing of the sub-sample of.

(3) is a non-motion compensation field memory, which stores a signal for one frame having a sample rate of 32.4 MHz passing through the switch (2). (4) is a first motion compensation field memory, which stores a signal for one frame at a sample rate of 16.2 MHz output from the non-motion compensation field memory (3) and performs motion compensation using a motion vector.

(5) is a divider that divides the transmitted motion vector by two. (6) is a second motion compensation field memory, which stores a signal for one frame having a sample rate of 32.4 MHz passing through the switch (2) and stores it in the divider (5).
The motion correction is performed by using the motion vector predictor between the fields output from. (7) is an inter-field interpolation filter, which interpolates a missing sample point based on the signal passing through the switch (2) and the output signal of the second motion correction field memory (6). (8) is an intra-field interpolation filter, which interpolates the missing sample points based on only the signal passing through the switch (2). (9) is a switch,
Normally, it is connected to the upper contact (9x) to allow the output signal of the inter-field interpolation filter (7) to pass through, and when motion is detected, it is connected to the lower contact (9y) pixel by pixel to connect the inter-field interpolation filter ( Pass the output signal of 8). Reference numeral (10) is a video output terminal for outputting a signal which passes through the switch (9) at a sample rate of 64.8 MHz by interpolating the missing sample points.

Next, the operation of the above configuration will be described.

FIG. 2 shows motion vectors during panning in order to specifically explain the operation of FIG.
Represents the horizontal axis of the screen and y represents the vertical axis of the screen.

Each field of the video signal is represented by a _{1 to} d ₂ , and when the signal is represented by a field ... a ₀ , b ₀ , c ₀ , d ₀ , a ₁ , c ₁ , d ₁ , a ₂ , b ₂ , c ₂ ,
d ₂ ……. The inter-field motion vectors separated by one frame are ... A ₁ , B ₁ , C ₁ , D ₁ , A ₂ , B ₂ ,.
It is indicated by…. In addition, the motion vectors between the fields that are separated by one field that are obtained in the drawing are ... b ₁₀ , c ₁₀ , d ₁₀ , a ₂₀
Represented by ……. The following relational expression holds between the inter-field motion vector separated by one frame and the inter-field motion vector separated by one field.

_{a 10 + b 10 = A 1} b 10 + c 10 = B 1 c 10 + d 10 = C 1 Here, for example consider only the relationship 3 field of a ₁ to c _1. However, the inter-field motion vector separated by one frame has a horizontal component of 5 bits and a vertical component of 3 bits.
One bit digital signal is transmitted per field. In other words, when describing the magnitude of the inter-field motion vector separated by one frame in terms of pixel length, the horizontal component is within the range of −15 to +16, the vertical component is within −3 to +4, and the horizontal component and the vertical component are The direction component also has a discrete component that is an integer. When this is illustrated, as shown in FIG. 3, the inter-field motion vectors separated by one frame are within the range surrounded by the broken line. However, the origin in FIG. 3 is an arbitrary pixel of a ₁ field.

If the parallel movement of the image due to panning is smooth, a ₁
The inter-field motion vector separated by one field between the field and b ₁ field is within the range surrounded by the chain line in FIG. Here, the relative relationship of the three fields a _{1 to} c ₁ is shown in FIGS.

First, FIG. 4 shows a case where the inter-field motion vector A ₁ transmitted one frame apart has the maximum magnitude. At this time, a motion vector a ₁₀ between fields separated by one field
Must also be maximum. The inter-field motion vector b ₁₀ separated by one field is also within the range of the chain double-dashed line, but it must also be the maximum value. Therefore, the vector b ₁₀ coincides with A _1/2 .

Next, FIG. 5 shows a case where the component of the inter-field motion vector A ₁ separated by one frame is (x, y) = (10, 2). As in FIG. 4, the inter-field motion vector a ₁₀ separated by one field exists within the range surrounded by the chain line, and when the b ₁ field is at the point shown in the figure, the inter-field motion vector b ₁₀ separated by _one field is shown. Exists within a range surrounded by a chain double-dashed line. In fact, since the c ₁ field is within the chain double-dashed line as shown, such movement by panning is possible.

From the above, when the relative positions of the a ₁ field and the c ₁ field are determined by the inter-field motion vector A ₁ transmitted one frame apart, the position of the b ₁ field with respect to those fields is It is within the shaded area. Moreover, if the parallel movement of the image due to panning is smooth, the position of the b ₁ field is within the range of the diagonal line,
It is highly possible that they exist in a 3 × 3 square lattice centered on the midpoint of the positions of the a ₁ field and c ₁ field shown by the black circles in FIG. Therefore, it is reasonable to substitute the vector b ₁₀ with A _1/2 , and this error is about one pixel.

Next, FIG. 6 shows a case where the component of the inter-field motion vector A ₁ separated by one frame is (x, y) = (3, 1). As in FIG. 4, the inter-field motion vector a ₁₀ separated by one field exists within the range surrounded by the chain line, and when the b ₁ field is at the point shown in the figure, the inter-field motion vector b ₁₀ separated by _one field is shown. Exists within a range surrounded by a chain double-dashed line. In fact, since the c ₁ field is within the chain double-dashed line as shown, such movement by panning is possible. Since there is no panning in which the change in direction of the motion vector between adjacent fields separated by one field exceeds a right angle, the transmitted motion vector A ₁ between fields separated by one frame leads to a ₁ field.
When the relative positions of the c ₁ fields are defined, the positions of the b ₁ fields with respect to those fields are within the range of the chain line. Furthermore, if the image is smoothly translated by panning, the position of the b ₁ field will be centered on the midpoint between the positions of the a ₁ field and c ₁ field indicated by the black circle in FIG. It is highly possible that they exist on the 2 × 2 square lattice. Was but connexion, the vector b ₁₀ A _1/2
However, since the component of A _1/2 in this case does not take an integer value, the vector connecting any one of the black circles in FIG. 6 and the position of c ₁ field is substituted for the vector b ₁₀ . Even in this case, the error is about one pixel.

As described above, the inter-field motion vector b ₁₀ separated by one field can be predicted by approximating the integer part of A _1/2 using the inter-field motion vector A ₁ separated by one frame, and the error is 1 pixel. Degree or less. Similarly, it is _{a 10 ≒ [D 0/2} ] b 10 ≒ prediction as _{[A 1/2] c 10} ≒ [B 1/2]. However, [] represents an integer part.

Moreover, in the prediction of the inter-field motion vector separated by one field, the error does not affect the subsequent prediction, so even if there is a temporary large vector change and the error becomes large, the subsequent vector changes will not become gradual. For example, the prediction with small error can be made again.

The motion correction performed by the above prediction will be described with reference to FIG.

First, a ₁ when the signal d ₁ field has entered the video input terminal of FIG. 1 (1), the non-motion compensated field memory (3)
The signals of the field and c ₁ field are stored, and the signals of the d ₀ field and b ₁ field are stored in the first motion compensation field memory (4). At this time, the motion vector B ₁
Is input, the d ₀ field and the b ₁ field stored by the motion vector B ₁ are two-dimensionally moved in the first motion compensation field memory (4), and the video input terminal (1) The motion is corrected based on the d ₁ field input to. As a result, the signal passing through the switch (2) is the signal of d ₁ field and b ₁ field corrected by the motion vector B ₁ .

On the other hand, the input motion vector B ₁ is divided by 2 by the divider (5), and the integer part of the output becomes the predicted value of the inter-field motion vector c ₁₀ separated by one field.

The second motion compensation field memory (6) stores signals of a ₁ field and c ₁ field as in the non-motion compensation field memory (3). The contents of the second motion compensation field memory (6) is one field prediction vector of fields between the motion vector c ₁ that are separated [B _1/2] by motion compensation based on the d ₁ field moves two-dimensionally Done.

In this way, motion compensation was performed based on the d ₁ field.
Signals for 4 fields of a ₁ field, b ₁ field, c ₁ field, d ₁ field itself enter inter-field interpolation filter (7) to enable inter-field interpolation.
Normally, the switch (9) is connected to the upper contact (9x), and the signal interpolated between fields passes through, but for the video signal, interpolation of the missing sample points cannot be performed by inter-field interpolation, so motion detection At this time, the switch (9) is connected to the lower contact (9y) on a pixel-by-pixel basis, and the intra-field interpolation is performed only from the signal that has passed through the switch (2) by the intra-field interpolation filter (8).

The sample rate of the signal in which the missing sample points are interpolated by the inter-field interpolation filter (7) or the intra-field interpolation filter (8) is 64.8 MHz and is output from the video output terminal (10).

In the motion-correction sub-sample interpolation method that obtains inter-field motion vectors separated by one field by such prediction, not only is it stable at the beginning and end of panning, but also a temporary error in the transmitted motion vector. Even if it happens, there will be no adverse effect.

It is also stable in special cases where the scene changes during panning and the image becomes a panning image. Furthermore, this method is very good because the change of the motion vector due to panning is gradual.

FIG. 7 shows the states of the signals 7a to 7g in FIG. 1 when the d ₁ field signal is input to the video input terminal (1).
In FIG. 7, the horizontal line above the symbol indicates that motion compensation has been performed, A is the interpolation value interpolated from ₁ , ₁ , ₁ , ₁ , d ₁ by the interpolation function f _A , and B is the interpolation function f _B. Indicates the interpolated value interpolated from ₁ and d ₁ .

[Effects of the Invention] As described above, according to the motion compensation sub-sample interpolation method of the present invention, samples are transmitted by intermittently sub-sampling so as to make a cycle of four fields while maintaining a predetermined sample position. A sub-sample interpolation method for reproducing a video signal by interpolating the missing points from the sample value received in the four-field period on the reception side based on the value, and one frame of the video signal detected on the transmission side. In the motion correction sub-sample interpolation method in which the position of the sample value one frame before is corrected on the receiving side based on the separated inter-field motion vector, and the missing point is interpolated to reproduce the video signal, From the inter-field motion vector separated by one frame, the inter-field motion vector separated by one field on the receiving side is converted into the inter-field motion vector separated by one frame. Generated by making an approximate prediction to 1/2 of the
Using the inter-field motion vector separated by one frame detected on the transmission side and the inter-field motion vector generated by the one field separated on the reception side, the sample values of the past three fields are compared based on the field of interest. Since the relative position is corrected and the missing point is interpolated from the sample value received in the four-field period in the moving image as well as the interpolation in the still image, the moving image is also
Like a still image, the missing points can be interpolated from the sample values received in the four-field period, and as a result, an image with excellent resolution can be obtained even in the case of a moving image with panning. effective.

[Brief description of drawings]

FIG. 1 is a block diagram showing an example of a schematic configuration of a motion correction sub-sample interpolation method according to an embodiment of the present invention, and FIG. 2 is an example of a motion vector at the time of panning for explaining the operation of FIG. The vector diagram shown in FIG. 3 is a diagram showing existence ranges of inter-field motion vectors separated by one frame and inter-field motion vectors separated by one field, FIG. 4, FIG.
FIG. 6 and FIG. 6 are vector diagrams showing the relationship between the inter-field motion vector separated by one frame and the inter-field motion vector separated by one field, and FIG. 7 is a signal flow of the motion compensation sub-sample interpolation method according to the present invention. 8 is a block diagram showing an example of a schematic configuration of a conventional motion compensation sub-sample interpolation method, and FIG. 9 is a signal in the case where motion compensation of the conventional motion compensation sub-sample interpolation method is not performed. Timing chart showing the flow of
FIG. 10 is a timing chart showing a signal flow in the case of performing the motion correction of the conventional motion correction sub-sample interpolation method. (1) …… Video input terminal, (2) …… Switch, (3)
...... Non-motion compensation field memory, (4) …… First motion compensation field memory, (5) …… Divider, (6)…
… Second motion compensation field memory, (7) …… Inter-field interpolation filter, (8) …… In-field interpolation filter, (9) …… Switch, (10) …… Video output terminal. In the drawings, the same reference numerals indicate the same or corresponding parts.

Claims (1)

[Claims] 1. Based on a sample value transmitted by intermittently sub-sampling so as to make a cycle of four fields while maintaining a predetermined sample position, the sample value is received from the sample value received in a four-field period on the receiving side. A sub-sample interpolation method for reproducing a video signal by interpolating a missing point, and based on inter-field motion vectors separated by one frame of the video signal detected on the transmission side, the above-mentioned sample value one frame before on the reception side In the motion-compensated sub-sample interpolation method in which the position of is corrected and the missing point is interpolated to reproduce the video signal, from the inter-field motion vector separated by one frame,
An inter-field motion vector separated by one field on the receiving side is generated by approximate prediction of 1/2 of the inter-field motion vector separated by the one frame, and the inter-field motion vector detected by the one frame separated by the transmitting side. Using the inter-field motion vectors separated by one field generated on the receiving side and the receiving side, the relative positions of the sample values of the past three fields are corrected with respect to the field of interest, and the moving image is moved in the same manner as the interpolation in the still image. Also in the above, the motion compensation sub-sample interpolation method is characterized in that the missing points are interpolated from the sample values received in the 4-field period.