JPS59128881A - Encoder of inter-field and interframe with dynamic compensation - Google Patents

Abstract

PURPOSE:To code efficiently both static and dynamic pictures by detecting each dynamic vector between adjacent frames and fields of a TV picture and selecting one of two kinds of dynamic vectors to generate a forecast signal by using a selected dynamic vector. CONSTITUTION:A TV signal A/D-converted is inputted to a delay circuit 3 and the 1st and 2nd vector detectors 31, 35 from a terminal 1, a difference between an output of the circuit 3 and a forecast error signal from a selector 29 is obtained by a subtractor circuit 5 and inputted to a quantizer 7. Further, the detector 31 detects a dynamic vector with a preceding field outputted from a frame memory 18, the detector 35 detects the dynamic vector of the interframe, and the result of detection is inputted respectively to a discriminating circuit 37 and a selector 33. The circuit 37 supervises the forecast of interfield and interframe, decides those for the next block according to the switching rules and the result is applied to the selector 33. Further, the selector 33 selects any one dynamic vector and oututs a forecast signal.

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to an encoding apparatus for television signals. In digital transmission of television signals, adjacent 7
By using interframe coding, which encodes and transmits the difference signal between frames, normal pulse code modulation (
The number of transmission bits can be significantly reduced compared to when using f'CM), and the compression rate is high (the ratio of transmission bits reduced compared to PCM), especially for still images and images with small movement. You can get it. However, for images containing large movements, the difference signal between adjacent frames becomes large, so the above-mentioned compression ratio decreases. However, in order to maintain a high compression ratio even for images containing large motions, "motion compensated interframe coding" has been considered.

This method detects a motion vector V representing the motion of the image shown in FIG. 1, shifts the previous frame signal by the motion vector, and uses it as a predicted signal.

In other words, in FIG. 1, the predicted signal 1g is expressed as Y''(7+
) is used as a prediction signal. Prediction error Y(7) Y'(r
+7) is the prediction value u'l by simple interframe coding
'(7) Y' Indicates a much smaller value than (7). By encoding and transmitting the generated pre-cursor difference peak signal and motion vector V in this way, efficient encoding can be achieved even for pictures with large movements.

An example of a method for detecting this motion vector is ``Motion correction in interframe coding'' by Ninomiya, Institute of Electronics and Communication Engineers, Image Engineering Research Group, document number LE78-6, 1978.5.
The method reported under the title 26th May) can be used.

In this method, the television screen is divided into a plurality of blocks, and for each block, the signal of the block of the previous frame at a position shifted by an extreme amount of displacement (shift vector) from the same position on the television screen is used as a reference. The degree of similarity with the block signal of the current frame is checked, and the shift vector for the previous seven frame block having the highest degree of similarity is detected as a motion vector. However, since television signals often contain ephemeral and large movements, it is necessary to be able to generate prediction signals for a large number of ephemeral motion vectors in order to improve the encoding performance. For example, it compensates for movements up to 8 lines/frame on the top and bottom, and 8 pixels/frame on the right.
)X(2X8+1)=289 motion vectors, which requires a function to generate prediction signals, and has the disadvantage that the apparatus becomes large-scale.

In addition, as a way to compensate for a wide range of motion without increasing the scale of the motion-compensated inter-frame coding device, attempts have been made to perform motion-compensated inter-field prediction.

In other words, in a normal television signal, there are 3
Although a 0-frame picture signal is transmitted, one frame of the television signal is composed of two pictures called fields, which are line-interlaced.

If we consider one field as one picture, there are 60 per picture.
It can be seen that Kaede Maki is being transmitted, and for example, 211 AI + elements moving between one frame (2 pixels/frame) is 1 pixel moving between one field (1 pixel/field). 151 God bless you.

Therefore, inter-field motion (inter-field motion vector) fc is detected, and compensation inter-field prediction is performed to generate a prediction signal that compensates for the inter-field motion. By using (2X4+1)X(2X4+1)=81 inter-field motion vectors for line/field, left and right up to 41 elements/field, the above-mentioned 8 vertical
It is possible to compensate for movements of up to a line/frame and 8 pixels/frame left and right, and the scale of the device can be made smaller than inter-frame motion compensation. Conversely, the motion compensation range is 8 lines/7 frames above and below and 8 pixels/frame left and right using the same device scale as a motion compensated interframe coding device that can compensate for up to 4 pixels/frame left and right of the upper F4 line/frame. can be expanded to.

However, in television signals, adjacent fields are line interlaced as described above, so the horizontal scanning distances of adjacent fields are shifted vertically by the distance i horizontal scanning f distances on the screen of the television, so there is no movement. Compensated inter-field prediction has the disadvantage that prediction efficiency decreases for stationary parts. Also, the minimum unit of motion vector detected between fields is 1 pixel/field.
The above-mentioned tight movement becomes 2 pixels/frame when viewed between adjacent frames.

In this way, motion compensated inter-field prediction has lower motion vector detection accuracy than motion compensated inter-frame prediction, but it is easier to widen the motion compensation range. On the other hand, motion compensated interframe prediction has better prediction efficiency for still images or images with small movement.

Therefore, in parts of the image where the movement is small, the motion compensation frame inter-frame prediction side is operated, and in the large-motion parts, the motion compensation field interval prediction side is operated. Both images and moving images can be encoded efficiently.

The present invention provides motion compensation that performs efficient encoding of both still images and moving images by adaptively providing motion compensation inter-frame prediction and wJ compensation 7' inter-field prediction. An object of the present invention is to provide an interframe encoding device.

The present invention provides a motion vector detection means for detecting motion in adjacent frames of a television image (interframe motion vector) and motion in adjacent fields (interfield vector), and a system for generating a prediction signal for the television image. Select either an inter-frame motion vector or an inter-field motion vector using the magnitude of at least one of the two detected motion vectors, and compensate the motion based on the selected motion vector. The predicted signal is generated and the input TV screen image 1#! and a means for predictively encoding the information.

Next, the principle of the present invention will be explained using an example in which motion compensation is performed on a block-by-block basis. In order to drastically change whether to perform motion compensated inter-frame prediction or wJ-compensated inter-field prediction for the current block, for example, the following kikaikon rule is used.

[An example of a coded block surrounding a desk block is that when the excitation compensation frame interprediction side is performed on the left-hand neighbor block (reference block), the prediction number 1g is applied to the reference block. Sea bream pect I used to generate
If the norm of L/ (ie, the interframe motion vector) is large, zbntd performs the current blotter block compensated interfield prediction, and if it is small, n performs the motion compensated interframe prediction. Similarly, when motion-compensated inter-field prediction is performed on a reference block, if the norm of the inter-field motion vector of the reference block is large, motion-compensated inter-field prediction:
f; otherwise, switch between motion-compensated inter-claim prediction and motion-compensated inter-field prediction;

Here, interframe motion vector V'= (Vx, Vy
) is defined as follows, for example.

+ i + = Jvx2+ v,” Also, the inter-field motion vector V'l = (VX',
The magnitude 1v°1 of V,') is defined as follows, since as mentioned above, when the same movement is measured between fields, it is half of the value measured between frames.

1v"1-2\Riko Nodoro If you switch between motion-compensated inter-frame prediction and motion-compensated inter-field prediction in this way, the motion compensation range will be narrow for small parts of the image, but , motion compensation frame prediction 61 with high motion detection accuracy and high coding efficiency.
1", and in parts of the image with large movement,
Although the motion detection accuracy is somewhat rough, it is possible to operate motion-compensated inter-field prediction, which can widen the motion compensation range with small-scale hardware, and can efficiently encode both still and video images. can do.

Note that by using this switching rule, it is possible to distinguish between motion-compensated inter-frame prediction and motion-compensated inter-field prediction without adding information indicating switching between them. There is also the advantage that there is no need to provide separate codes for both, and the number of bits required to transmit all vectors can be reduced.

Various modifications of this switching rule are possible, and this point will be described later.

Next, embodiments of the present invention will be described with reference to the drawings.

Note that the following description will be made assuming that the current block and the block located one position to the left on the television screen are used as reference blocks.

FIG. 2 is a block diagram showing the configuration of an encoding section of a motion compensated interframe/interfield encoding apparatus according to an embodiment of the present invention.

In Fig. 2, analog/digital conversion (A/D conversion) television signal (hereinafter referred to as rTV signal)
) is manually applied to terminal 1. Delay circuit 3 via TV multiplier signal 1g line 2 input to terminal 1;
First vector detector 31 and second vector detector 3
5 is input.

The delay circuit 3 uses the subtracter 5 to synchronize the timing of the input TV signal which is input multiple times with the predicted signal outputted to the signal line 15, that is, the time required for motion vector detection and predicted signal generation. Used for correction.

The 'rv signal outputted from the delay circuit 3 to the signal line 4 is subtracted by the subtracter 5 from the predicted signal manually input from the multi-breather 29 to the signal line 15, and this difference signal (prediction error signal) is converted into a signal. The signal is inputted to a quantizer 7 through a line 6, quantized, and inputted to a first encoder 9 and an adder 16 via a signal line 8. Here, the first encoder 9 performs variable length encoding on a quantized prediction error signal, similar to that used in a conventional interframe encoding device.

The quantized prediction error signal input to the adder 16 via the signal line 8 is added to the prediction signal inputted to the signal line 15 by the adder 16, and locally decoded.

This locally decoded signal is written to the frame memory 18 via signal IrM17 and is used to generate a prediction signal in the next frame or field.

The first vector detector 31 outputs approximately one field before the input TV flaw signal inputted from the signal line 2 from the frame memory 18 to the signal line 19.

The second vector detector 35 detects an interframe motion vector (interframe motion vector) from the input TV flaw signal frame memory 18 inputted from the signal line 2 to No. 13 approximately one frame before inputted from the signal line 20. is detected and output to the signal line 36.

Note that the signal output from the frame memory 18 about one field before and the signal about l frame before, that is, the signal on the signal line 19.20, scans the vertical detection range of the motion vector up and down ±M horizontally. Line/frame (or ±M horizontal scanning line/field) f, L-7, assuming the vertical block size is B horizontal scanning lines (2M+8)
Only the horizontal scanning lines are output in parallel to each signal line.

Note that the configuration of the vector detector 35 is detailed in the above-mentioned document 1, so a description thereof will be omitted.

Further, the vector detector 31 can also be configured in exactly the same manner as shown in Document 1.

The inter-field motion vector output to the signal line 32 is the machihen delay time y! 525, is input to the selector 33 and the determination circuit 37. The freight shift vector outputted to the signal line 36 is inputted to the variable delay circuit 26, the selector 33, and the determination circuit 37.

On the other hand, the signal line 19. which is output from the frame memory 18.
20 signals, i.e. approximately l fields before and approximately l
The TV multiplied signals before the frame are each input to delay circuits 45 and 46, and output as signals yIIA 47 and 48 ljj. Here, the delay circuits 45 and 46 are used to compensate for the time required by the first vector detector 31 and the second vector detector 35 to detect the inter-field motion vector and the inter-frame motion vector, respectively.

The variable delay circuit 25 receives approximately l input from the signal line 47.
The Tl' signal before the field is shifted according to the field opening motion vector signal manually inputted via the signal line 32 and outputted to the signal line 27 as a motion compensated interfield prediction signal.

The variable delay circuit 26 inputs a signal from approximately one frame before the signal line 48 to the signal line 28 as a motion compensated interframe prediction J41J in accordance with the interframe motion vector inputted via the signal line 36. Output.

The variable delay circuits 25 and 26 are configured so that the preview name can be recorded two-dimensionally using the randomly accessible memory 2. Signals are sequentially read out pixel by pixel.

The determination circuit 37 determines which of the motion-compensated Huberd inter-prediction and the motion-compensated frame 14 prediction (hereinafter each abbreviated as "MC inter-field prediction", rMC inter-frame prediction dilJJ) for the current block signal.・Depending on the size of the field co-movement vector input by 1g No.32.36 and the inter-memovement vector, that is, based on the above-mentioned switching rule, the missing block is It is determined whether to perform MC inter-field prediction or MC inter-frame prediction. That is, if Me inter-field prediction is to be performed in the next block, the selector 29 outputs the MC inter-field prediction signal, signal 1ts15, and the selector 33 outputs the signal g (inter-field prediction) on the signal line 32. A control signal for outputting the motion vector) to the signal line 34 is output to the signal line 39, passed through the decision delay circuit 38, and is applied to the selector 29.33 via the signal line 52.

Note that the configuration of the determination circuit 37 will be described later.

The determination delay circuit 38 delays the output of the determination circuit 37 by the time required for signal processing of one block (one block time), outputs the delayed output, and inputs the delayed output to the selector 29.33. Therefore, the determination result of the determination circuit 37 is determined by the selector 2 after one block time.
9.33, and the predicted signal for television number 1δ input from the delay circuit 3 to the subtracter 5 is output to the signal line 15. - The motion vector output from the force selector 33 to the signal line 34 is input to the second encoder 30 and encoded, for example, variable length encoded, and output to the signal line 51.The motion vector is input to the multiplexer 11 via the signal line lO. The signal is multiplexed with the encoded prediction error signal and sent to the signal line 12. The multiplexed signal on the signal @12 is written to the buffer memory IJ13 for speed matching with the transmission speed of the transmission line 14, and is read out at the transmission speed of the transmission line and sent to the transmission line 14. .

Next, the decoding section will be explained with reference to FIG. The signal outputted from the encoder to the '-1 transmission line 14 is written to the receiving side buffer memory I7101 at the transmission speed of the transmission line 14. The contents of the receiving side buffer memory 101 are read onto a signal line 102, and a demultiplexer 103 separates seven codes indicating a prediction error signal and a code indicating a motion vector, and outputs them to signal lines 104 and 118, respectively. The code indicating the prediction error signal is returned to the prediction error signal by the first decoder 105,
A prediction error signal is input to adder 107 via signal line 106.

Also, the selector 11 is connected to the adder 107 via a signal line 117.
A prediction signal is inputted from 6, added to the prediction error signal, predictively decoded as a TV flaw signal, and output to the signal line 108.

The decoded TV flaw signal is also written into the frame memory 109 and used for decoding the TV flaw signal of the next frame.

From the frame memory 109 to the signal line 110, approximately 1
TV scratch signal in front of the field, signal [111 has approximately 1
The signals of the previous frame are output each with variable delay 1gl path 11
2. Input at 11'3.

On the other hand, the code indicating the motion vector outputted from the demultiplexer 103 to the signal line 118 is returned to a motion vector signal by the second decoder 119, and then transmitted to the signal line 120 by the delay circuits 112 and 113 and the receiving side determination circuit. 121.

The variable delay circuits 112 and 113 are connected to the signal lines 110 and 110, respectively.
11 is shifted in accordance with the motion vector signal applied to signal line 120, and outputted to signal lines 114 and 115 as MC inter-field prealignment signal and MC inter-frame prediction signal, respectively. and are respectively input to the selector 116. The receiving side determination circuit 121 monitors whether the predictive decoding of the signal of the current block is performed by MC inter-field prediction or Me inter-frame prediction.
Similar to the determination circuit 37 on the encoder side, the motion vector signal manually inputted by 0 is determined based on the above-mentioned switching rule, and the determination result is provided to the determination delay circuit 123 through the signal line 122. The determination delay circuit 123 delays the determination result by one block time and inputs it to the selector 116 via the signal line 124. That is, the judgment ml field is executed by the selector 116 after l block time.

The selector 116 selects either one of the signals on the signal line 114 or 115, that is, the MC inter-field pre-multiplied signal and the MC inter-frame pre-side signal, according to the control signal inputted through the signal line 124, and outputs the selected signal as a prediction signal. Output to signal line 117.

As described above, the 'rV signal can be predictively decoded.

Next, referring to FIG. 4, the determination circuit 37, the receiving side determination circuit 1
21 will be explained. In FIG. 4, (al is a block diagram showing the configuration of the judgment circuit w!&37, (bl is a block diagram showing the configuration of the receiving side judgment circuit 121, but the only difference is the connection of the input signal, and the components used in both are exactly the same. It is.

150.160 is the lth size determination circuit, component 152.162 is the second size determination circuit, component 15
4.164 is a selector, JR element 156.166 is a 7-lip frog, and element 157.167 is a delay circuit.

First, the determination circuit 37 will be explained. In the discrimination circuit 37, as described above, the inter-field motion vector signal is sent to the signal line 32, and the input motion vector signal is sent to the signal line 36.
Input by The first magnitude determination circuit 150 determines the magnitude of the input motion vector between fields, and if the magnitude is small, it outputs a signal "1", for example, and if it is a dog, it outputs a signal "0" to the signal line 151. The paternal signal is sent to the selector 154 via the selector 154.

In addition, the second magnitude determination circuit 152 adds the magnitudes of the motion vectors between the input frames, determines the thickness,
If the magnitude is large, the signal is "1", and if the magnitude is small, the signal is "1".
0'' is manually input to the selector 154 via the signal line 153.

The selector 154 responds to the control signal V input from the delay circuit 157 via the signal line 158 to 51% on line 1g.
153, make sure to select either one of ``N'' or ``1G'', and connect the signal line 155 to the 9th frig 70 knob 156. Here, if No. 16 or Ill on the signal line 158 is found, it indicates that MC inter-field predictive coding was performed in the reference block (in this case, the block before l block time), and 9, rOJ. This indicates that MC interframe predictive coding has been performed in the reference block. In other words, the output of the slow positive circuit 157 becomes "1".
”, the selector 154 selects the 15th signal #11151, and when the output of the delay circuit 157 is “()”, the selector 154 selects the signal a
153 signal is selected. Note that the delay amount M157 is the same as that of the determination delay circuit 38.123.

7 Rig 70 Pug 156 inverts the contents when the internal force signal from the first side line 155 is "1" and predicts the predicted signal between the reference block and the current block by Me inter-field prediction → MC
Interframe prediction or MC interframe prediction → MC74
- Corresponding to the case of changing to inter-ruby prediction) If it is "0", the contents are not changed. The contents of the 7-lip flop are output by signal #39 and are also output by delay circuit 157.
and is used to control the selection operation of the selector 154 after l block time.

In this way, switching between MC field prealignment and Me frame prealignment is performed based on the switching rules described above.
Ru.

Next, the reception side determination circuit 121 will be explained.

However, since the operation of the reception side determination circuit 121 is almost the same as that of the determination circuit 37, only the differences from the determination circuit 37 will be explained. In the reception side determination circuit 121, the motion vectors decoded as described above are exactly the same as those in the determination circuit 37. In this case, when the output signal of the delay circuit 167 is "1" (indicating that the MC inter-field prediction signal is selected as the prediction signal for the current block),
First judgment circuit 160 @ foot polishing C' + FI No. 161
the upper signal) is selected by the selector 164.

At this time, the second judgment circuit 1/62 outputs an incorrect value, that is, even though the input motion vector is an inter-field motion vector, it is regarded as an inter-frame excitation vector. Although the box power of the magnitude determination is all output, there is no problem because the cancer selector 164 does not select the output signal of the second magnitude determination circuit, that is, the signal on the signal line 163. The output signal of the delay circuit 167 is “0”
The same applies to case 1), in which the output signal of the first determination circuit, that is, the signal on the signal line 161 is not selected by the selector 164.

In this way, in the receiving side determination circuit, even though the signal indicating the division of the inter-field motion vector and the 7-frame motion vector is not sent over the transmission path, the input motion vector It is possible to correctly determine whether the motion vector is a motion vector or an interframe motion vector.

In the above explanation, in switching between motion compensated interframe prediction and IIJ compensated interfield prediction,
The block one block to the left on the television screen was used as a reference block, and switching was done by looking at the size of the frame opening motion vector or field opening motion vector used in the reference block. A temporally and spatially adjacent block, such as a block one frame before, can also be used as a reference block.

Also, by using multiple blocks including these blocks as reference blocks, and taking into account the size of the inter-frame motion vector or inter-field motion vector that has already been selected in these multiple reference blocks, It is also possible to decide whether to use the motion compensated interframe prediction or the motion compensated interfield prediction σi++.

Further, in the above, only the case where the above-mentioned switching rule is applied to a motion compensation prediction method that detects the motion of an image on a block-by-block basis has been explained in detail. For example, N.N. Netravali, J.D. Davis (A, N, N
et(aval i, J-L)-&bbins) by Sibel System Technical Journal/l/(Be1l
System Technical Journal
) Vol 58. A3 (March 1979) 631-66
Page 9 “Motion-Compen
satedTelevision Coding:
The above-mentioned switching rules can be applied in exactly the same way to the motion compensation prediction method announced under the title Part I). That is, the above-mentioned switching rule is applied while making it possible to estimate both the inter-frame motion vector and the field opening motion vector at the same time.

Further, even if the above-mentioned switching rule is modified as follows, it is possible to operate the motion compensated frame memory in parts of the image where the motion is small, and to operate the motion compensated inter-field prediction in the parts where the motion is large.

“If the norm of the interframe motion vector detected in the current block is small, the motion compensation 7
Inter-frame prediction is performed, and if the norm is large, motion-compensated inter-field prediction is performed. ” (Modified switching rule 1) In this case, the decision delay circuit 38.123 becomes unstable.

In addition, in the publisher circuits 37 and 121, the second size judgment circuit 152, 162, selector], 54, 164, and delay circuit 157, 167 are not recommended. .7 lip 70 tube 156 is simply connected directly. The same applies to the judgment time @121.

Of course, it is also possible to make the same determination as in the modified switching rule l regarding the magnitude of the norm of the inter-field motion vector detected in the tfc and JJ blocks.

[Brief explanation of the drawing]

FIG. 1 is a diagram for explaining motion-compensated inter-frame coding, and FIGS. 2 and 3 show the encoding unit and decoding unit of the motion-compensated inter-frame inter-field coding/decoding device of the present invention, respectively. The block diagram of the section is 3.38.45.46.123,...delay circuit, 5.
・・Subtraction circuit, 7・Doji converter, 9.3o・
...Signal generator, 11...Multiplexer, 13
,101...Buffer memo! j, 16,107・
...Adder, 18.109...Frame memory,
25.26.112.11゛3... variable delay circuit,
29.33.116...Selector, 31.35.
...vector detector, 37.121...judgment circuit,
103... Demultiplexer, 105.119
. . . each shows a decoder. FIG. 4 is a block diagram showing the configuration of the determination circuit.
21 is shown. In the M4 diagram, 150.152.160,162-...size determination! 2I
w5.154.164...Selector, 156.166.
...7 block flops, 157.167...delay circuits, respectively. f'tl"i人ji1-SAT 1ノ゛、H2・、f名'
1., -- Beni j 1.11 ゝ-8-ノ' lzo (a) (b)

Claims (1)

[Claims] Motion-compensated frame-to-frame interfield coding that performs predictive coding of the television image by detecting all motion of the television image and using a prediction signal generated based on the detected motion and compensating for the motion of the image. An apparatus comprising: the television program II! motion vector detection means for detecting motion in adjacent frames of the Il image (frame opening motion vector A/) and motion in adjacent fields (interfield motion vector); Select one of the inter-frame motion vector, inter-field motion vector, and inter-field vector using the magnitude of at least one of the two types of motion vectors, and calculate the motion based on the selected motion vector. a motion-compensated interframe interfield encoding device, comprising means for generating a compensated slave signal and predictively encoding the input television image.