JP2519450B2 - Motion compensated sub-sample transmission system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial field of application] The present invention relates to a MUSE high-definition television signal,
The present invention relates to a motion correction sub-sample transmission system applied when a receiving side predicts a motion correction amount between fields based on the transmitted motion information by approximation and interpolates a missing point between fields.

[Conventional technology]

This is a motion compensation sub-sample transmission method that has been proposed in the past, and the NHK Broadcasting Technology Research Laboratories announced a new transmission method (MUSE) for high-definition television announced at the foundation commemorative lecture in June 1984. Some are mentioned in the material. FIG. 11 shows the configuration of the motion compensation sub-sample transmission system.

In FIG. 11, 1 is a video input terminal for inputting a video signal having a sample rate of 64.8 MHz, 2 is a sub-sample switch, and the video signal input to the video input terminal 1 is kept at a predetermined sample position, 16.2. Perform sub-sampling in four fields at a sampling rate of Hz.

Further, 3 is a motion vector detection circuit, which is a video input terminal 1
The motion vector between the fields separated by one frame is detected from the video signal input to the frame 1 to generate one vector in one field.

The switch 5 is the sub-sample switch 2 described above,
The video signal that is subsampled and transmitted and the output signal of the motion compensation field memory 10, which will be described later, are switched at a subsample timing of 32.4 MHz.

The non-motion correction field memory 15 stores a signal for one frame having a sample rate of 32.4 MHz that passes through the switch 5.

The motion correction field memory 10 stores the signal for one frame at the sample rate of 32.4 MHz output from the non-motion correction field memory 15 and performs the motion correction by the motion vector.

The inter-field interpolation filter 12 interpolates the missing sample points based on the output signal of the non-motion correction field memory 15 and the signal passed through the switch 5.

The intra-field interpolation filter 11 interpolates the missing sample points based on the signal passed through the switch 5.

The switch 13 is connected to the lower contact 13q to pass the output signal of the inter-field interpolation filter 12 when motion compensation is not performed, and is connected to the upper contact 13p when motion compensation is performed or when motion is detected. Then, the output signal of the intra-field interpolation filter 11 is passed.

Reference numeral 14 is a video output terminal for outputting a signal passing through the switch 13 at a sample rate of 64.8 MHz by interpolating the missing sample points.

Next, the operation of the above configuration will be described. The transmission method for high-definition television signals is sub-Nyquist sampling, which makes a cycle of four fields, and the required bandwidth is 8.
It is 1MHz.

In the video signal transmission system having the configuration shown in FIG. 11, the video signal input from the video input terminal 1 is subsampled by switching the switch 2, and the motion vector detection circuit 3 detects one field. A motion vector between fields separated by one frame is detected and transmitted separately.

On the other hand, on the receiving side, when the signal of the first field is transmitted to the upper contact 5a of the switch 5, the non-motion correction field memory 15 stores the signals of the second field and the fourth field before one cycle, On the other hand, the motion correction field memory 10 stores the signals of the first and third fields of the filter.

When the motion vector is transmitted, the contents of the motion compensation field memory 10 move two-dimensionally according to the vector amount. Since the motion vector represents the amount of motion correction, the motion correction is performed on the basis of the transmitted signal of the first field having the contents of the motion correction field memory 10.

The switch 5 is switched at the timing of 32.4 MHz sub-sampling, and the phase is inverted for each field and also by the motion vector. Therefore, in the above-described case, the signal of the first field transmitted and the signal of the third field before one cycle in which the motion correction is performed by the motion vector pass through the switch 5.

The signal that has passed through the switch 5 is input to the intra-field interpolation filter 11. The intra-field interpolation filter 11 performs intra-field interpolation, and when the switch 13 is continuous with the upper contact 13p, outputs the video signal of the sample rate of 64.8 MHz interpolated in the field from the video output terminal 14.

The switch 13 is connected to the upper contact 13p when motion compensation is performed and when motion detection is performed.
The former is a field unit, and the latter is a pixel unit.
Switches.

The signal that has passed through the switch 5 and the output signal of the non-motion correction field memory 15 are input to the inter-field interpolation filter 12 to perform inter-field interpolation. When motion compensation is not performed, that is, switch 13 is the lower contact.
When connected to 13q, the video signal with the sample rate of 64.8 MHz inter-field interpolated is output from the video output terminal 14.

FIGS. 12 (a) to 12 (f) show the states of the signals 12a to 12f in FIG. 11 when the d ₁ field is input to the video input terminal 1 without motion compensation. However, if the signal is expressed in fields ... a ₀ , b ₀ , c ₀ , d ₀ , a ₁ , b ₁ , c ₁ ,
The sequence is d ₁ …….

In the figure, A is an interpolation function f _A that is an inter-field processing.
shows the _{a 1, b 1, c 1} , the interpolation value interpolated by d _1.

Further, FIGS. 13 (a) to 13 (e) show the states of the signals 13a to 13f in FIG. 11 when the motion correction is performed and the d ₁ field is input to the video input terminal 1. In the figure, the horizontal line above the symbol indicates that motion correction has been performed, and B indicates the interpolated value interpolated by ▲ ▼ and d ₁ by the interpolating function f _B which is the in-field processing.

Another Japanese Patent Laid-Open No. 61-201580 discloses a field interpolation device. This field interpolating device includes a motion vector detecting means for detecting an inter-frame motion vector indicating a parallel movement amount of the entire image between temporally continuous frames, and a detected inter-frame motion vector between temporally consecutive fields. A motion vector conversion means for converting an inter-field motion vector indicating the motion of the entire image, and the converted inter-field motion vector are used to match the relative position of the image information of the current field with the image information of the current field. And an interpolating means for superimposing.

[Problems to be solved by the invention]

In the conventional motion compensation sub-sample transmission method as described above, since the relationship between adjacent fields is not taken into consideration at the time of performing the motion compensation, the interpolation of the missing sample points depends on the intra-field interpolation by the intra-field interpolation filter. are doing. Therefore, there is a problem that the resolution is lowered when the motion correction is performed.

Further, in the inter-field interpolation device disclosed in Japanese Patent Laid-Open No. 61-201580, similarly to the above, there is a problem that the resolution is lowered when the motion correction is performed.

The present invention has been made in order to solve such a problem, and an object of the present invention is to obtain a motion correction sub-sample transmission system capable of suppressing a decrease in resolution when performing motion correction.

[Means for solving problems]

The motion-corrected sub-sampling transmission method according to the present invention maintains a predetermined sample position and intermittently sub-samples a video signal so as to make a cycle in a plurality of fields, and separates one frame in one field unit. A motion compensation sub-sampling transmission method for transmitting motion information (I) between fields, in which a sample value of the video signal is delayed from the motion information (I) by one field and transmitted. At the receiving side, the motion information (II) between fields separated by one field from the field of interest on the receiving side is defined as the motion information (I) for the field of interest and the motion information (I) transmitted one field before. By using the prediction, the motion values in the past multiple fields were corrected from only the sample value in the field of interest and the motion information (II). It is characterized in that the missing point is interpolated using the sample value.

[Action]

In the present invention, the fact that there is no sudden change in the transmitted motion vector is used to convert the motion information (II) between fields separated by one field on the receiving side into the motion information (I) for the field of interest. By performing prediction using the motion information (I) transmitted one field before, interpolation is performed by the inter-field interpolation filter also at the time of motion correction.

〔Example〕

An embodiment of the motion compensation sub-sample transmission system of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of the present invention. In FIG.
A video input terminal for inputting a video signal with a sample rate of 64.8 MHz, 2 is a sub-sample switch, and the video signal input to the video input terminal 1 is maintained in a predetermined sample position and at a sample rate of 16.2 MHz in 4 fields. Perform a round of sub-samples.

The motion vector detection circuit 3 detects a motion vector between fields separated by one frame from the video signal input to the video input terminal 1 according to the motion, and generates one vector in one field.

The first 1-field delay device 4 delays the video signal subsampled by the sample switch 2 by 1 field.

The switch 5 switches between the fixed terminals 5a and 5b at the timing of the sub-sample of 32.4 MHz between the video signal output from the first field delay unit 4 and transmitted and the output signal of the second motion compensation field memory 10 described later.

The second one-field delay device 6 delays the motion vector output from the motion vector detection circuit 3 by one field, and the motion vector output from the motion vector detection circuit 3 and transmitted by the second one-field delay circuit. The output from the adder 6 and the motion vector one field before are added to the adder 7
I'm trying to add.

The divider 8 divides the motion vector added by the adder 7 by “4” and outputs the division result to the first motion correction field memory 9 and the second motion correction field memory 10. ing.

The first motion compensation field memory 9 is a switch 5
A signal for one frame having a sample rate of 32.4 MHz passing through is stored, and motion compensation is performed by the predicted motion vector between fields output from the divider 8.

The second motion compensation field memory 10 is output from the first motion compensation field memory 9. A signal for one frame at a sample rate of 32.4 MHz is stored, and motion compensation is performed by the inter-field predicted motion vector output from the divider 8. This second motion compensation field memory 10
The output of is added to the lower contact 5b of the switch 5.

The in-field interpolation filter 11 interpolates a missing sample point based on the signal passed through the switch 5, and its output is applied to the upper contact 13p of the switch 13.

The inter-field interpolation filter 12 interpolates the missing sample points based on the signal passed through the switch 5 and the signal output from the first motion correction field memory 9.

The switch 13 is normally connected to the lower contact 13q to allow the output signal of the inter-field interpolation filter 12 to pass, and when motion is detected, it is connected to the upper contact 13p on a pixel-by-pixel basis and output from the motion field interpolation filter 11 Pass the signal.

Video output terminal 14 is 64.8MHz with missing sample points interpolated
At a sample rate of 1 to output a signal passing through the switch 13.

Next, the operation of the above configuration will be described. FIG. 2 shows a motion vector generated in the motion vector detection circuit 3 according to a video input from the video input terminal 1 on the transmission side. In the figure, x represents the horizontal axis of the screen and y represents the vertical axis of the screen.

Also, each field of the video signal is represented by a _{1 to} d ₂ , and when the signal is represented by fields, ... a ₀ , b ₀ , c ₀ , d ₀ , a ₁ ,
The flow is in the order of b ₁ , c ₁ , d ₁ , a ₂ , b ₂ , c ₂ , d ₂ . The inter-field motion vectors separated by 1 frame each
It represents at _{A 1, B 1, C 1} , D 1, A 2, B 2 .......

Also, it represents the inter-field motion vectors spaced one field obtained on drawing in _{...... b 10, c 10, d} 10, a 20 ....... The following relationship is established between the inter-field motion vector separated by one frame and the inter-field motion vector separated by one field.

_{a 10 + b 10 = A 1} b 10 + c 10 = B 1 c 10 + d 10 = C 1 where, for example b ₁₀ can be rewritten as follows.

If the second term on the right side of the above equation is sufficiently small, b ₁₀ can be predicted as follows.

Here, for example, consider only the relation of the four fields a _{1 to} d ₁ . However, the inter-field motion vector separated by one frame is transmitted in one field as a digital signal having a horizontal direction component of 5 bits and a vertical direction component of 3 bits.

That is, when describing the inter-field motion vector and the size separated by one frame in terms of pixel length, it is within the range of horizontal component −15 to +16 and vertical component −3 to +4. Both have discrete components that are integers. This is illustrated in FIG. 3. As shown in FIG. 3, the inter-field motion vector separated by one frame is within the range surrounded by the broken line. However, the origin in FIG. 3 is an arbitrary pixel in the a ₁ field.

If the parallel movement of the image due to panning is smooth,
The inter-field motion vector separated by one field from the a ₁ field and the b ₁ field is within the range surrounded by the chain line in FIG. Here, the relative relationships of the three fields a _{1 to} c ₁ are changed in three ways as shown in FIGS. 4, 6, and 8, and
The position of the d ₁ field and the prediction accuracy of the motion vector are examined for each.

First, FIG. 4 shows the case where the magnitude of the inter-field motion vector A ₁ transmitted one frame apart is the maximum. At this time, a motion vector between fields separated by one field
a ₁₀ and b ₁₀ are also the maximum, and the relative positions of the a _{1 to} c ₁ fields are determined. At this time, the d ₁ field exists within the range of the three-dot chain line in the figure, but when there is no sudden change in motion, the d ₁ field exists within the range of the diagonal lines.

Furthermore, if the parallel movement of the image such as panning is smooth, it is highly possible that the d ₁ field exists at the position indicated by the black circle in the figure. As an example, FIG. 5 shows the prediction vector (A ₁ + B ₁ ) / 4 and the vector b ₁₀ when the d ₁ field is at the position shown.

The prediction error at this time is one pixel length, and if there is a d ₁ field at the position represented by the black circle, the error will be less than that, so such prediction is appropriate.

Next, FIG. 6 shows a case where the component of the inter-field motion vector A ₁ separated by one frame is (x, y) = (10, 2). As in FIG. 4, the inter-field motion vector a ₁₀ separated by one field exists within the range surrounded by the chain line, and b ₁
When the fields are at the points shown in the figure, the inter-field motion vector b ₁₀ separated by one field exists within the range surrounded by the chain double-dashed line. In fact, such movement by panning is possible because the c ₁ field lies within the chain double-dashed line as shown.

At this time, although the d ₁ field exists within the range surrounded by the three-dot chain line in the figure, it exists within the range of the diagonal lines as in the description in FIG. 4, and may exist at the position indicated by the black circle among them. Is high. As an example, FIG. 7 shows the prediction vector (A ₁ + B ₁ ) / 4 and the vector b ₁ when the d ₁ field is at the position shown.

The prediction error at this time is one pixel diagonal, and if there is a d ₁ field at the position represented by the black circle, the error will be less than that. Therefore, the prediction is valid also in the case of FIG.

Next, FIG. 8 shows the case where the component of the inter-field motion vector A ₁ separated by one field is (x, y) = (3, 1). As in FIG. 4, the inter-field motion vector a ₁₀ separated by one field exists within the range surrounded by the chain line, and when the b ₁ field is at the point shown, the inter-field motion vector b ₁₀ separated by one field. Exists within a range surrounded by a chain double-dashed line. In fact, such a movement by panning is possible because the c ₁ field is within the range enclosed by the chain double-dashed line as shown.

At this time, although the d ₁ field exists within the range surrounded by the three-dot chain line in the figure, it exists within the range of the diagonal lines as in the description in FIG. 4, and may exist at the position indicated by the black circle among them. Is high. As an example, FIG. 9 shows the prediction vector (A ₁ + B ₁ ) / 4 and the vector b ₁₀ when the d ₁ field is at the position shown.

The prediction error at this time is one pixel length diagonally, and if there is a d ₁ field at the position represented by a black circle, the error will be less than that. Therefore, the prediction is also valid in the case of FIG.

In view of the above, the inter-field motion vector b ₁₀ separated by one field is predicted by approximation with the integer part of (A ₁ + B ₁ ) / 4 using the inter-frame motion vector A ₁ + B ₁ separated by 1 frame. The error is about one pixel diagonal or less. In the same way, a ₁₀ ≈ [(D _O , A ₁ ) / 4] b ₁₀ ≈ [(A ₁ , B ₁ ) / 4] c ₁₀ ≈ [(B ₁ , C ₁ ) / 4] Can be predicted. However, [] represents an integer part.

Moreover, in the prediction of the inter-field motion vector separated by one field, the error does not affect the subsequent prediction.
Even if the error increases temporarily due to a large vector change, if the vector change thereafter becomes gentle,
It is possible to predict again with a small error.

The motion correction performed by the above prediction will be described with reference to FIG. First, when the signal of the a ₂ field is input to the video input terminal 1 of FIG. 1, the motion vector detection circuit 3 generates the motion vector C ₁ .

On the other hand, the signal input to the video input terminal 1 is subsampled by the subsample switch 2 and further delayed by one field by the first one-field delay unit 4.

Therefore, at this time, the signal transmitted from the transmission side through the transmission path is the output of the first 1-field delay unit 4 d ₁
The field signal and the motion vector C ₁ output from the motion vector detection circuit 3.

On the other hand, on the receiving side, when the signal of the d ₁ field is input to the switch 5, the first motion correction field memory 9
In the second motion compensation field memory 10, the signals of the a ₁ field and the c ₁ field are stored, and in the second motion compensation field memory 10, the signals of the d ₀ field and the b ₁ field are stored.

At this time, the transmitted motion vector C ₁ and the second 1
The motion vector B ₁ delayed by one field by the field delay unit 6 is input to the adder 7. The motion vectors B ₁ and C ₁ added by the adder 7 are divided by 4 by the divider 8 to obtain the predicted value of the inter-field motion vector c ₁₀ .

The content of the first motion compensation field memory 9 is the prediction vector of the inter-field motion vector c ₁₀ separated by one field. Then, it moves two-dimensionally, and the motion is corrected based on the d ₁ field transmitted from the transmitting side and entering the switch 5.

On the other hand, the signals of the d ₀ field and the b ₁ field stored in the second motion compensation field memory 10 are stored in the first motion compensation field memory 9 one field before, and are transmitted at this time. c ₁ field 1 field spaced field between values motion vector signal relative to the that
b ₁₀ prediction vector Since the motion compensation is performed by, the prediction vector is the same as that of the first motion compensation field memory 9 based on the d ₁ field. Motion correction can be performed with.

In this way, signals of four fields of the a ₁ field, b ₁ field c ₁ field, and b ₁ field which have been motion-corrected on the basis of the d ₁ field enter the inter-field interpolation filter 12 to perform inter-field interpolation. It is possible.

Normally, the switch 13 is connected to the lower contact 13q and passes the inter-field interpolated signal, but since the inter-field interpolation cannot be performed for the moving image signal, when the motion is detected, the switch 13 is in pixel units. Upper contact 13
The intra-field interpolation is performed only from the signal connected to p and passed through the switch 5 by the intra-field interpolation filter 11.

The signal in which the missing sample points are interpolated by the inter-field interpolation filter 12 or the intra-field interpolation filter 11 has a sample rate of 64.8 MHz and is output from the video output terminal 14.

In such a motion-corrected sub-sample transmission method that obtains an inter-field motion vector separated by one field based on prediction on the receiving side, not only is the image resolution stable at the beginning and end of panning, but also the transmitted motion. Even if there is a temporary error in the vector, it will not have a bad effect later.

It is also stable in special cases where the scene changes during panning and the image becomes a panning image.

Furthermore, since the change of the motion vector due to panning is gradual, it can be said that this method is very good.

In this method, the structure of predicting the inter-field motion vector on the receiving side is simplified by delaying the video signal transmitted by one field with respect to the motion vector transmitted by the first one-field delay device 4 on the transmitting side. ing.

FIG. 10 (a) shows the states of the signals from 10a to 10h in FIG. 1 when the a ₂ field signal is input to the video input terminal 1.
~ Fig. 10 (h). In FIG. 10, the horizontal line above the signal indicates that motion compensation has been performed, and A is ₁ by the interpolation function f _A ,
Interpolated values interpolated from ₁ , ₁ , d ₁ and B an interpolated value interpolated from ₁ , d ₁ by the interpolation function f _B.

〔The invention's effect〕

As described above, according to the present invention, motion information (I) between fields separated by one field from the field of interest on the receiving side can be obtained without significantly changing the configuration on the receiving side.
I) is the motion information for the field of interest (I)
And the motion information (I) transmitted one field before are used for the prediction, and the motion correction is performed for each field. The interpolation by can be stabilized. Moreover, the present invention is excellent in that it is possible to perform interpolation with a small error, and therefore, it is possible to sufficiently control the resolution reduction during panning, the resolution does not decline during motion correction, and a high-resolution image can be displayed. Produce the effect.

[Brief description of drawings]

FIG. 1 is a block diagram showing an example of a schematic configuration of a motion compensation sub-sample transmission system of the present invention, and FIG. 2 is a vector diagram showing an example of a motion vector at the time of panning for explaining the operation of FIG. FIG. 3 is a diagram showing the existence range of the inter-field motion vector separated by one frame and the inter-field motion vector separated by one field. FIGS. 4, 6, and 8 are the inter-field motion vector separated by one frame and the inter-field motion vector separated by 1 frame, respectively. Vector diagrams showing the relationship between inter-field motion vectors separated by fields, and FIGS. 5, 7, and 9 are the inter-field motion vectors predicted by one field and the actual motion vectors predicted from the inter-field motion vectors separated by one frame, respectively. Is a vector diagram showing an example of inter-field motion vectors separated by one field, and FIG. Timing chart showing a record, 11
FIG. 13 is a block diagram showing an example of a schematic configuration of a conventional motion compensation sub-sample transmission system, FIG. 12 is a timing chart showing a signal flow when motion compensation of the conventional motion compensation sub-sample transmission system is not performed, and FIG. The figure is a timing chart showing the flow of signals in the case of performing the motion compensation of the conventional motion compensation sub-sample transmission system. 1 ... video input terminal, 2, 5, 13 ... switch, 3 ... motion vector detection circuit, 4 ... first 1-field delay device,
6 ... Second 1-field delay device, 9 ... First motion compensation field memory, 10 ... Second motion compensation field memory, 11 ... In-field interpolation filter, 12 ... Inter-field interpolation filter, 14 …… Video output terminal. In the drawings, the same reference numerals indicate the same or corresponding parts.

Claims (4)

(57) [Claims] 1. A sample value of a video signal intermittently subsampled so as to make a cycle in a plurality of fields while maintaining a predetermined sample position, and motion information (I) between fields separated by one frame per field. In the motion compensation sub-sample transmission method for transmitting the following, the transmission side delays the sample value of the video signal with respect to the motion information (I) by one field, and By predicting the motion information (II) between fields separated by 1 field from the target field using the motion information (I) for the field of interest and the motion information (I) transmitted one field before, Interpolation of missing points using sample values in the field of interest and motion-compensated sample values in past multiple fields from motion information (II) only Motion compensation subsample transmission method according to claim Rukoto. 2. The motion compensation sub-sample transmission system according to claim 1, wherein the motion information (II) between fields separated by one field from the field of interest is converted into motion information (I) for the field of interest. ) And the motion information (I) transmitted one field before the field of interest, and approximately 1/4 of the sum of the motion information (I) is transmitted. A motion-compensated sub-sample transmission method characterized in that a missing point is interpolated by using motion-compensated sample values in a plurality of fields. 3. The motion-corrected sub-sample transmission method according to claim 1, wherein a sample value intermittently sub-sampled from the transmitting side so as to make a cycle of four fields while maintaining a predetermined sample position. By predicting the motion information (II) for the field of interest transmitted and received by the motion information (I) for the field of interest and the motion information (I) transmitted one field before, A motion-compensated sub-sample transmission method characterized in that a missing point is interpolated using sample values in a field of interest and sample values in the past three fields that are motion-compensated only from motion information (II). 4. The motion compensation sub-sample transmission system according to claim 3, wherein the motion information (II) for the field of interest on the receiving side.
The motion information (I) for the field of interest and 1
1/4 of the sum of motion information (I) transmitted before the field
Corrected sub-sample transmission characterized by interpolating a missing point using the sample value in the field of interest and the sample values of the past three fields that are motion-corrected only from the motion information (II) by performing approximate prediction to method.