JP2530217B2 - Interframe predictive coding device and decoding device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION (Industrial field of application) The present invention is effective in various devices such as recording devices, transmission devices, and other display devices that perform signal processing of digital signals, so that moving image signals can be efficiently processed with a smaller code amount. Particularly, it relates to an inter-frame predictive coding method and a decoding method among the high-efficiency coding methods for coding.

(Prior Art) Among the high-efficiency coding methods for coding moving image signals with a smaller code amount, there is inter-frame predictive coding as a coding method that utilizes the correlation between frames of image signals.

This is because normal moving images are quite similar between each frame, so the signal of the frame to be encoded is predicted from the signal of the previous (past) frame that has been encoded, and the prediction error (residual error) It only encodes.

A typical conventional configuration of interframe predictive coding and decoding is shown in FIG. 4 for a coding device and in FIG. 5 for a decoding device.

In FIG. 4, a prediction signal (prediction value) is subtracted from the prediction signal subtractor 2 for moving image signals continuously input from the image signal input terminal 1, and the prediction error (residual error) is encoded. . The method of forming the prediction signal will be described later.

Here, the prediction error (residual error) may be quantized as it is, but it is generally quantized by the quantizer 4 after being orthogonally transformed by the orthogonal transformer 3 in order to obtain higher coding efficiency. It has become. Since the distribution of the quantized signal is concentrated around 0 (zero), the variable length encoder 5
Is converted into a variable length code such as a Huffman code, and is output from the data output terminal 6 as variable length digital data for recording or transmission.

On the other hand, on the decoding device side, as shown in FIG.
The variable-length digital data input from the data input terminal 21 is converted into the original fixed-length data by the variable-length decoder 22, replaced with the representative value by the inverse quantizer 23 (representative value setting), and the orthogonal inverse The converter 24 performs inverse transform processing of orthogonal transform.

Since the signal thus obtained is a prediction error (residual error), the same signal as the prediction signal in the encoder is added by the adder.
Playback image signal output terminal by adding in 25 and playing back as a playback image signal
It is output from 26.

The predicted signal is obtained by delaying the reproduced image signal by one frame by the frame memory 27 and passing the delayed image signal through the same spatial low-pass filter (LPF) 28 as the encoder.

On the other hand, the prediction signal in the encoding device needs to obtain the same signal as in the decoding device side, and needs to be generated from the quantized signal. Otherwise, in the frame cyclic prediction processing as shown in FIG. 4, the difference between the prediction signals in the encoding device and the decoding device is accumulated for each frame, resulting in a large error.

Therefore, in the encoding device shown in FIG. 4, the quantized signal is replaced with a representative value by the inverse quantizer 7 (representative value setting) as in the decoding device shown in FIG. Inverse processing of orthogonal transformation is performed by.

Since the signal thus obtained corresponds to the decoded prediction error (residual error), the predicted signal of the preceding frame is added by the adder 9 to be a decoded image signal.
Further, this signal is delayed by one frame by the frame memory 10, and the prediction signal is obtained by multiplying the spatial LPF 11 by a coefficient different depending on the spatial frequency.

The spatial LPF11 is used here to reduce the residual of the quantization error in the prediction error (residual error). However, in the encoding process, the quantization error is mostly in the high frequency region of the spatial frequency, while the noise As a result, the inter-frame correlation also decreases in the high frequency range, which is effective.

Further, in the cyclic interframe predictive coding as shown in FIG. 4, since a code error generated in the transmission path is propagated and a calculation error is accumulated due to a mismatch between the transmitting side and the receiving side of the orthogonal inverse converter 8, Inter-frame prediction is reset once in an appropriate section (30 to 100 frames), and the prediction signal of that frame is fixed and is substantially intra-frame encoded (independent frame). This operation is performed by periodically changing the changeover switch 12. In this case, it is better to perform the reset in a short section from the viewpoint of error countermeasures, but since the intra-frame coding is performed at the time of reset, the coding efficiency decreases in image transmission with high inter-frame correlation such as video conference. To do.

(Problems to be Solved by the Invention) Such a cyclic interframe predictive coding system using a previous frame, when trying to decode a certain frame,
All the data in the past is needed because the data is accumulated in the past. Therefore, there is no great inconvenience when sending images continuously such as in a video conference, but in storage media such as information recording disks and information recording tapes, random access or search is used to place data in any location in the media. It is necessary to reset the inter-frame prediction in small units so that decoding can be performed from. Especially when trying to perform a visual search (fast forward playback),
Since it is necessary to decode every few frames, the prediction is reset each time, which leads to a one-way decrease in coding efficiency.

On the other hand, in the case of the reverse reproduction in which the reproduction is performed in the reverse order in time with respect to the normal reproduction, the conventional prediction with the previous frame cannot obtain the prediction signal for the decoding and thus cannot perform the decoding.

In addition, the prediction from the previous frame is a prediction from one side on the time axis, which is not sufficient from the viewpoint of prediction efficiency. In particular, when the image changes significantly such as a scene change, the appropriate prediction cannot be performed.

Furthermore, since the prediction signal must be obtained by the same decoder as that on the decoding device side, it is necessary to perform decoding processing on the encoding device side without fail, and the device scale becomes large.

Further, in the conventional cyclic prediction, when there is a difference in the calculation accuracy of the decoding process, there is a problem that the prediction signal shifts and accumulates.

The present invention has been made in view of the above-mentioned problems of the prior art, and in high-efficiency coding of moving image information, a highly accurate prediction signal adapted to a change in image is formed to improve coding efficiency. It is an object of the present invention to provide an interframe predictive coding device as described above and a decoding device for moving image information coded by the coding device.

(Means for Solving the Problem) An interframe predictive coding apparatus according to the present invention cyclically transmits a specific frame every predetermined number of frames (three frames or more) among consecutive frames of image signals continuously input. First encoding means that sets and encodes the specific frame using image information of only the specific frame, and a prediction signal of each frame for a non-specific frame between the specific frames. On the time axis of the frame, the signal of the specific frame in the front (past) direction and the signal of the specific frame in the rear (future) direction are directly formed from a mixed signal, and all the frames of the non-specific frame Prediction signal forming means for forming a prediction signal in the order of input, and a second coding means for coding a prediction error between the signal of the non-specific frame and the prediction signal corresponding thereto. And an interframe predictive coding apparatus including a stage.

Further, the inter-frame predictive decoding device of the present invention is an inter-frame predictive decoding device that decodes image information coded using inter-frame prediction. And a non-specific frame between the specific frames, first decoding means for decoding the specific frame based on the image code of only the specific frame, and a prediction signal for each of the continuous non-specific frames. A prediction signal forming means for forming the prediction signals of all the non-specific frames in the order of input by forming the prediction signals of the non-specific frames in a single prediction from a signal obtained by mixing the signals of the specific frames before and after each frame. An inter-frame prediction having a second decoding means for decoding the non-specific frame by adding the output of the prediction signal forming means and the prediction error corresponding to the output. A decoding device.

(Operation) In the inter-frame predictive coding device having the above-described configuration, the first coding means sets a specific frame for each predetermined frame among the continuous frames of the image signals continuously input, and The particular frame is encoded within the frame or using prediction from the particular frame in the forward direction. Two or more non-specific frames are continuously set between the specific frames.

In the prediction signal forming means, when obtaining the prediction signal for the non-specific frame, instead of forming the prediction signal from the frames immediately before and immediately after the encoding target frame,
When the frame adjacent to the encoding target frame is a non-specific frame, the non-specific frame is skipped, and a prediction signal is formed from the specific frames preceding and succeeding (old and new) in time.

In this case, in the prediction signals of a plurality of non-specific frames between two specific frames, all the frames are directly formed from the preceding and following specific frames in the input order.

The second encoding means encodes the difference between the prediction signal output from the prediction signal forming means and the input non-specific frame signal.

This is shown in FIG. FIG. 6 is a diagram illustrating an interframe prediction method according to the present invention and a conventional interframe prediction method.

In the figure, A is a conventional interframe prediction method (cyclic interframe prediction), and B is an interframe prediction method (bidirectional interframe prediction) according to the present invention.

In the figure, a square is a continuous frame of a moving image signal which is continuously input, in which a shaded part is a frame which is independently encoded in the frame, and is the first in A (or at the time of reset). Only an independent frame, but B
Then there are regular independent frames. Arrows indicate the directional relationship of inter-frame prediction. In A, prediction is performed only from the previous frame as in each frame, but in B, prediction is performed from the two preceding and following independent frames. (In the explanation below,
A frame in which the prediction signal is temporally formed from both the frame before and after the frame to be encoded is referred to as a bidirectional prediction frame. In addition, prediction is performed based on only a specific frame, and a frame predicted from both directions is not used for another prediction.

As the specific frame, it is optimal to use an independent frame in which an error during prediction signal formation does not accumulate, but a part of the specific frame may be a cyclic prediction frame obtained by conventional cyclic interframe prediction, If the frame has a small number of cyclic frequencies, there will be no problem because the accumulated error is small.

(Example) An example of the inter-frame predictive coding method according to the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing a configuration of a first embodiment of an interframe predictive coding apparatus according to the present invention, and FIG. 2 is a block showing a configuration of a second embodiment of an interframe predictive coding apparatus according to the present invention. FIG. 3 is a block diagram showing the configuration of an embodiment of the interframe predictive decoding device according to the present invention.

The basic configuration of these encoding device and decoding device is
It is based on the conventional example, and the same components as those shown in FIG. 4 or FIG.

In FIG. 1 and FIG. 2, there is a (N-1) frame memory 31 [N is an integer of 3 or more] for encoding a non-specific frame after encoding of a specific frame used for prediction. .

Further, in order to form a prediction signal (prediction value) of a non-specific frame based on two specific frames before and after, two frame memories 32 and 33 and two coefficient multipliers {× α, × (1-α)} 34,35,
There are those adders 36. [However, 0 <α <1] Further, the changeover switch 37 is provided between the image signal input terminal 1 and the (N-1) frame memory 31.
38 is between the prediction signal subtractor 2 and the orthogonal transformer 3, the changeover switch 39 is between the quantizer 4 and the dequantizer 7, and the changeover switch 40 is between the two frame memories 32 and 33. Provide each.

As will be described later in detail, the one in FIG. 1 obtains the same prediction signal as in the decoding device side as in the conventional example, but the one in FIG. 2 extracts the prediction signal from the original image signal to be encoded. That is, the prediction signal is different between the encoding device side and the decoding device side.

The configuration of FIG. 2 does not require a decoding process on the encoder side, but this is particularly suitable when the specific frame is an independent frame that is independently encoded within the frame.

FIG. 3 is a diagram showing an embodiment of the interframe predictive decoding apparatus of the present invention, and there is a predictive signal adder 41 for adding the predictive signal to the signal obtained from the orthogonal inverse transformer 24.

In addition, in order to form a prediction signal of a non-specific frame based on two specific frames before and after, two frame memories are used.
There are 42, 43, two coefficient multipliers {xα, x (1-α)} 44, 45 for weighting the respective signals, and an adder 46 for them.

Further, a changeover switch 47 is provided between the orthogonal inverse converter 24 and the prediction signal adder 41, a changeover switch 48 is provided between the prediction signal adder 41 and the reproduced image signal output terminal 26, and a changeover switch 49 is provided between the two frames. It is provided between the memories 42 and 43, respectively.

In the configuration of the first embodiment shown in FIG. 1, the changeover switches 37, 38 are changed over to the a side for every predetermined number of frames of 3 or more, and the signal of the moving image input from the image signal input terminal 1 (continuous frame ), Set a specific frame. The signal of the specific frame is guided to the orthogonal transformer 3 without passing through the (N-1) frame memory 31 or the prediction signal subtractor 2, and is independently encoded in the frame.

The operations of the orthogonal transformer 3, the quantizer 4, and the variable length encoder 5 are basically the same as those of the conventional example.

On the other hand, the non-independent frame between the specific frames is a bidirectional prediction frame for obtaining a prediction signal by bidirectional interframe prediction, and the prediction signal is subtracted from the input signal, but the independent frame needs to be encoded first. Therefore, the non-independent frame is delayed by that amount.

Here, if the independent frame that is the specific frame is one frame for N frames [N is an integer of 3 or more], the delay amount is (N-1) frames. That is, when there are non-independent frames between specific frames, the changeover switches 37, 3
8 is connected to the b side, and the signal is (N-1) frame memory 31
Is delayed by (N-1) frames, and the prediction signal is subtracted by the prediction signal subtractor 2 and then guided to the orthogonal transformer 3.
The prediction error (residual error) is encoded.

Here, the changeover switches 37 and 38 are periodically connected to the a side for one N frame, and are connected to the b side otherwise. Subsequent orthogonal transformer 3, quantizer 4,
The operation of the variable length encoder 5 is the same as that for the independent frame. The above operation is the same in the case of FIG. 2 which is the second embodiment of the encoding device.

In the case of the encoding device having the configuration shown in FIG. 2, the prediction signal is obtained not from the encoded reproduction image signal but from the original image signal before encoding. The operation for forming the prediction signal is the same as that in FIG. 1 except that the original image signal is input instead of the encoded reproduction image signal.

The configuration of FIG. 2 eliminates the need for the inverse quantizer 7 and the orthogonal inverse transformer 8 of FIG. In this case, the prediction signal differs between the transmitting side and the receiving side, but in the present embodiment, the error is not accumulated for each frame.
Rather, the quantization error does not remain in the prediction error (residual error), eliminating the need for the spatial LPF in the conventional example,
Prediction efficiency is improved.

Next, how to make a prediction signal in the method of the present invention will be described. First, similar to the conventional example, a case where the same prediction signal is obtained on the encoding device side and the decoding device side is the first example.
It is the first embodiment shown in the figure. Here, the difference from the conventional example is that in the conventional example, all the frames are used to obtain a prediction signal, but in the present example, only the independently encoded specific frame (independent frame) is used. Since the prediction signal is generated, the changeover switch 39 is connected to the side a only for the independent frame, and the subsequent processing is performed.

The quantized signal is replaced with the representative value by the inverse quantizer 7 (representative value setting) as in the conventional example, and the orthogonal inverse transformer 8 performs the inverse transformation process of the orthogonal transformation.

Since the signals obtained in this way are encoded independently, they are used in the prediction of the predicted signal without being added to the previous frame, etc.
Is written to. At this time, the changeover switch 40 is connected to the a side, and the signal of the previous independent frame held in the frame memory 32 until then is replaced in the frame memory 33. By such an operation, the reproduction frame signal used for prediction is prepared in the frame memories 32 and 33 at the same time as the independent frame encoding process.

This reproduction frame signal is held until the signal of the next independent frame is supplied, and is (N-1) for prediction processing.
It is output repeatedly.

The prediction signal is obtained by multiplying the two reproduction frame signals by the weighting factors α and (1−α) by the coefficient multipliers 34 and 35 and adding them by the adder 36.

Here, the weighting coefficient is determined by the time relationship between the coding target frame input to the prediction signal subtractor 2 for coding and the specific frame used for prediction. The most general method is a method based on linear prediction, which is given by the following equation.

α = (m−mp) / N where m is the frame number (1,2,3,
…), Mp is the past (old) specific frame number (0, N, 2N,
, And m> mp, and N is an integer of 3 or more.

An example of the prediction signal (predicted value) created in this way is N
FIG. 7 shows the case of = 4. This gives greater weight to the frame that is closer in time, and gives a more appropriate prediction value when the signal changes in a nearly linear manner for each frame.

In FIG. 1 and FIG. 2 described above, the input image signal is made into independent frames every several frames by the switches 37 and 38, in which the interframe correlation of the encoded data is cut off. Therefore, visual access can be performed by random access in that unit or by decoding only independent frame data.

On the other hand, the signal of the non-independent frame is delayed by the (N-1) frame memory 31, and the image signal of the independent frame used for the prediction is delayed by two frames by the frame memories 32 and 33 before performing the prediction processing of the non-independent frame. By being stored, a prediction signal can be directly obtained from independent frames before and after (old and new) in time without using other prediction.

Also, in order to multiply an appropriate coefficient according to the time relationship between the frame predicted by the coefficient multipliers 34 and 35 and the independent frame,
It is possible to make predictions that adapt to changes in the image, and the S /
Since N is also improved, higher prediction efficiency can be obtained.

Further, since the encoded data obtained in this way has a symmetrical structure on the time axis, reverse reproduction can be easily realized.

Next, in the processing of the decoding apparatus example, in the configuration of the embodiment shown in FIG. 3, first, the variable length digital data input from the data input terminal 21 in the configuration of the embodiment shown in FIG. The quantizer 23 and the orthogonal inverse transformer 24 obtain a reproduced image signal in an independent frame which is a specific frame and a prediction error (residual error) signal in a bidirectional prediction frame which is a non-specific frame. Since the signal of the independent frame is used for prediction, connect the changeover switch 47 to the a side to
Written on 42. At this time, the changeover switches 48 and 49 are also connected to the side a, the signal of the preceding independent frame is exchanged in the frame memory 43, and at the same time, the reproduced image output terminal 26
Will be output. In this way, the reproduction frame signal used for prediction is prepared in the frame memories 42 and 43 at the same time as the independent frame decoding process.

On the other hand, in the case of bidirectional prediction frame, the changeover switch 4
7, 48, and 49 are connected to the b side, the same prediction signal (prediction value) as on the encoding device side is added by the prediction signal adder 41, and the result is output from the reproduced image signal output terminal 26. Also, coefficient multiplier 4
The method of forming the prediction signal by 4,45 and the adder 46 is the same as that on the encoder side.

Note that the data transmitted from the encoding device side comes from the independent frame first, so the decoding device side corrects it. Is output from the frame memory 42. That is, the frame memory 42 also serves as time correction.

In the embodiment of the present invention, an independent frame in which no error is accumulated is described as an example of the specific frame, but even if a part of the specific frame is a cyclic prediction frame obtained by conventional cyclic interframe prediction, If the frame has a small number of cyclic frequencies, error accumulation is small and substantially the same effect can be obtained. For example, in FIG. 6B, the central independent frame may be replaced with a prediction frame predicted from the leftmost independent frame.

(Effects of the Invention) As described above, according to the method of the present invention, a specific frame that does not use bidirectional prediction is periodically set every N (N is an integer of 3 or more) frames among consecutive frames of image signals that are continuously input. Are mostly set within a frame, and non-specific frames between them are directly predicted and coded from specific frames before and after (new and old). Therefore, independent frames are set at almost constant intervals (several frames). Exists, and the correlation between data frames is cut off at that point, so random access can be performed in that unit in storage media, and a decoded image can be immediately obtained. On the other hand, when trying to perform a visual search, since independent frames exist at almost regular intervals (several frames), only the data of the independent frames are decoded, so that there is no data waste and a smooth search image is obtained. Is obtained. Furthermore, since the bidirectional prediction frame is basically symmetrically encoded with respect to the time axis, the reverse reproduction can be performed by decoding in the reverse order. However, since the independent frame data is recorded prior to other data, it is necessary to correct it during reverse playback, but it is sufficient to correct the encoded and compressed data in time, so small-scale data Delay is good.

On the other hand, since coding is performed independently at approximately constant intervals (several frames), there is a side in which the coding efficiency is reduced for images with high inter-frame correlation, but such images with high inter-frame correlation are It is easy to obtain a good reproduced image quality, and there is not much problem. On the contrary, when the change of the image is large, the prediction error (residual difference) between the frames becomes large, so that the image quality is easily deteriorated, and the effect of the present invention in which the prediction accuracy is improved is remarkable.

Further, in the method of the present invention, since the non-specific frames between the specific frames are predicted by the preceding and following (new and old) specific frames, it is possible to perform prediction in accordance with changes in the image, and the coding efficiency is improved.

Further, since the prediction signal is formed by adding a plurality of frames, the S / N of the prediction signal is improved and the prediction accuracy is improved.

Further, even if there is an error in the prediction signal between the encoding device and the decoding device side, the error is not accumulated, and the original image signal may be predicted by the encoding device instead of the encoded reproduction image signal. It is possible, in which case no signal processing is required at the encoder.

As described above, according to the method of the present invention, as compared with the case of only the conventional cyclic interframe prediction, a highly accurate prediction signal adapted to a change in an image is formed by bidirectional prediction. Can be increased and the coding efficiency is improved. In addition, in storage media, random access, visual search, reverse playback, etc. are possible, and it is possible to encode images with scene changes and motions with high efficiency.

[Brief description of drawings]

FIG. 1 is a first diagram of an interframe predictive coding apparatus according to the present invention.
2 is a block diagram showing the configuration of the embodiment, FIG. 2 is a block diagram showing the configuration of the second embodiment of the interframe predictive coding apparatus according to the present invention, and FIG. 3 is an implementation of the interframe predictive decoding apparatus according to the present invention. FIG. 4 is a block diagram showing a configuration of an example, FIG. 4 is a block diagram showing a configuration of an interframe predictive coding device in a conventional example, FIG. 5 is a block diagram showing a configuration of an interframe predictive decoding device in a conventional example, and FIG. Is a diagram illustrating an interframe prediction method according to the present invention and a conventional interframe prediction method, and FIG. 7 is a diagram illustrating an example of an interframe prediction value according to the present invention. 1 ... Image signal input terminal, 2 ... Prediction signal subtractor, 3 ... Orthogonal transformer, 4 ... Quantizer, 5 ... Variable length encoder, 6 ... Data output terminal, 7,23 ... … Inverse quantizer, 8,24 …… Orthogonal inverse transformer, 21 …… Data input terminal, 22 …… Variable length decoder, 26 …… Playback image signal output terminal, 27,32,33,42,43… … Frame memory, 28 …… Spatial LPF, 31 …… (N-1) frame memory, 34,35,44,45 …… Coefficient multiplier, 36,46 …… Adder, 37,38,39,40, 47,48,49 …… Selection switch, 41 …… Prediction signal adder.

Claims (2)

(57) [Claims] 1. A specific frame is set in consecutively input image signals, a prediction signal of a non-specific frame between the specific frames is formed from the specific frame, and the non-specific frame is encoded. In the inter-frame predictive coding device, the specific frame is set periodically for each N (N is an integer of 3 or more) frames, and the specific frame is encoded based on information of only the specific frame. Coding means, and a prediction signal of each frame of the non-specific frame, the signal of the specific frame in the front (past) direction and the signal of the specific frame in the rear (future) direction on the time axis of each frame A prediction signal forming means for forming the prediction signals of all the frames of the non-specific frame in the order of input by directly forming the mixed signals of Inter-frame predictive encoding apparatus is characterized in that a second coding means for coding the prediction error between the prediction signal corresponding to the. 2. An inter-frame predictive decoding apparatus for decoding image information encoded by using inter-frame prediction, comprising: Within a specific frame,
A first decoding means for decoding the specific frame based on information of only the specific frame; and a prediction signal of each continuous frame of the non-specific frame before (past) on the time axis of each frame. Prediction signal formation for forming the prediction signals of all the non-specific frames in the order of input by directly forming the signals of the specific frame in the direction and the signals of the specific frame in the rear (future) direction An inter-frame predictive decoding apparatus comprising: means for decoding; and a second decoding means for decoding the non-specific frame by adding an output of the predictive signal forming means and a corresponding prediction error.