JP2530217C - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION (Industrial Application Field) The present invention is an apparatus for efficiently processing a moving image signal with a smaller code amount in various devices such as recording and transmission devices and other display devices for performing signal processing of digital signals. In particular, the present invention relates to an inter-frame predictive coding method and a decoding method among the high-efficiency coding methods for encoding data. (Prior Art) Among high-efficiency coding methods for coding a moving image signal with a smaller code amount, there is an inter-frame predictive coding as a coding method using correlation between frames of an image signal. This is because a normal moving image is quite similar between the frames, so that the signal of the frame to be encoded is predicted from the signal of the previous (past) frame that has been encoded,
It encodes only the prediction error (residual error). A typical conventional configuration of inter-frame predictive encoding and decoding is described as follows.
FIG. 5 shows the decoding device in FIG. In FIG. 4, the moving image signal continuously input from the image signal input terminal 1 is
The prediction signal (prediction value) is subtracted in the prediction signal subtractor 2, and the prediction error (residual) is encoded. The method of forming the prediction signal will be described later. Here, the prediction error (residual) may be quantized as it is, but is generally quantized by the quantizer 4 after being orthogonally transformed by the orthogonal transformer 3 in order to obtain higher coding efficiency. It has become. Since the distribution of the quantized signal is concentrated near 0 (zero), the signal is converted into a variable length code such as a Huffman code by the variable length encoder 5 and output from the data output terminal 6 as variable length digital data. , Recorded or transmitted. On the other hand, on the decoding device side, the variable length digital data input from the data input terminal 21 is converted into the original fixed length data by the variable length decoder 22 as shown in FIG. The data is replaced with a representative value by 23 (representative value setting), and the orthogonal inverse transformer 24 performs an inverse transformation process of the orthogonal transformation. Since the signal obtained in this manner is a prediction error (residual error), the same signal as the prediction signal in the encoding device is added by the adder 25 and output from the reproduction image signal output terminal 26 as a reproduction image signal. Is done. The prediction signal is obtained by delaying the reproduced image signal by one frame by the frame memory 27 and passing the delayed image signal through the same spatial low-pass filter (LPF) 28 as the encoder. On the other hand, the prediction signal in the encoding device needs to obtain the same signal as that of the decoding device,
It needs to be made from the quantized signal. Otherwise, in the frame cyclic prediction processing as shown in FIG. 4, the difference between the prediction signals of the encoding device and the decoding device is accumulated for each frame, resulting in a large error. For this purpose, in the encoding device shown in FIG. 4, the quantized signal is replaced with a representative value (representative value setting) by an inverse quantizer 7 similarly to the decoding device shown in FIG. To perform the inverse transformation of the orthogonal transformation. The signal obtained in this manner corresponds to a decoded prediction error (residual error), and the prediction signal one frame before is added to this by the adder 9 to be a decoded image signal. Further, this signal is delayed by one frame by the frame memory 10,
The prediction signal is obtained by multiplying the spatial LPF 11 by a coefficient different depending on the spatial frequency. Here, the spatial LPF 11 is used in order to reduce the quantization error remaining in the prediction error (residual error). However, in the encoding process, the quantization error is large in a high frequency range in the spatial frequency, while the noise is noise. This is effective because the inter-frame correlation also decreases in the high frequency range. In addition, cyclic inter-frame predictive coding as shown in FIG. 4 causes accumulation of calculation errors due to propagation of code errors occurring in the transmission path and mismatch between the transmission side and the reception side of the orthogonal inverse transformer 8. The inter-frame prediction is reset once in an appropriate section (30 to 100 frames), and the prediction signal is fixed, and the frame is substantially intra-frame coded (independent frame). This operation is performed by periodically switching the changeover switch 12. In this case, it is better to perform the reset in a short section from the viewpoint of countermeasures against errors, but since the encoding is performed within the frame at the time of the reset, the encoding efficiency decreases in image transmission with high inter-frame correlation such as a video conference I do. (Problems to be Solved by the Invention) In such a cyclic inter-frame predictive coding method using a previous frame, when a certain frame is to be decoded, since data is accumulated in the past, all past data are accumulated. Data is needed. For this reason, there is no major inconvenience when transmitting images continuously, such as in a video conference, but in storage media such as information recording disks and information recording tapes, random access and search can be performed at any location in the media. It is necessary to reset the inter-frame prediction in small units so that decoding can be performed from. In particular, when a visual search (fast-forward playback) is to be performed, it is necessary to decode every several frames, so that the prediction is reset each time, and the encoding efficiency is reduced unilaterally. On the other hand, in the case of reverse reproduction, in which reproduction is performed in a temporally reverse order with respect to normal reproduction, decoding cannot be performed in the conventional prediction based on the previous frame because a prediction signal for decoding cannot be obtained. Further, the prediction from the previous frame is performed in one direction on the time axis, and is not sufficient in terms of prediction efficiency. In particular, when the image changes greatly due to a scene change or the like, appropriate prediction cannot be performed. Further, since the prediction signal must be obtained by the same decoder as that of the decoding device side, it is necessary to always perform the decoding process on the encoding device side, which increases the device scale. Also, in the conventional cyclic prediction, if there is a difference in the calculation accuracy of the decoding process,
There is also a problem that the predicted signal is shifted and accumulated. SUMMARY OF THE INVENTION The present invention has been made in view of the above-described problems of the related art, and in a high-efficiency encoding of moving image information, forms a highly accurate prediction signal adapted to a change in an image to improve encoding efficiency. It is an object of the present invention to provide an inter-frame predictive encoding device as described above, and to provide a decoding device for moving image information encoded by the encoding device. (Means for Solving the Problems) The inter-frame predictive encoding apparatus of the present invention sets an independent frame in a continuously input image signal, and generates a prediction signal of a non-independent frame between the independent frames. An inter-frame predictive coding apparatus that forms an independent frame and encodes the non-independent frame, wherein the independent frame is periodically set for every N (N is an integer of 3 or more) frames, and the code of the independent frame is set. First encoding means for performing the conversion based on the information of only the independent frame, a first frame memory for storing a signal of an independent frame in the most recent past of the non-independent frame, and a signal of an independent frame in the most recent future. A second frame memory to be stored; an output signal of the first frame memory;
Means for forming prediction signals of all non-independent frames sandwiched between two independent frames from only signals obtained by mixing the output signals of the frame memories with the input signals of the non-independent frames. And a second encoding unit that encodes a prediction error between the signal of the non-independent frame and the corresponding prediction signal. Further, an inter-frame prediction decoding apparatus according to the present invention is an inter-frame prediction decoding apparatus for decoding image information encoded using inter-frame prediction, wherein N (N is an integer of 3 or more) A first decoding unit that decodes the independent frame based on information of only the independent frame among an independent frame periodically set for each frame and a non-independent frame between the independent frames; A first frame memory for storing a signal of an independent frame in the most recent past of an independent frame, a second frame memory for storing a signal of an independent frame in the most recent future, an output signal of the first frame memory, and a second The prediction signals of all the non-independent frames sandwiched between the two independent frames are calculated from only the signals obtained by mixing the output signals of the frame memories of the non-independent frames. A prediction signal generating means having means for forming the input order,
An inter-frame predictive decoding device comprising: a second decoding unit that decodes the non-independent frame by adding a prediction signal of the non-independent frame output from the prediction signal forming unit and a prediction error corresponding thereto. . (Operation) In the inter-frame predictive coding apparatus having the above-described configuration, the first coding unit includes:
An independent frame is set for each predetermined frame in the continuous frames of the image signal continuously input, and the independent frame is encoded in the frame. Two or more non-independent frames are continuously set between the independent frames. When obtaining a prediction signal for the non-independent frame, the prediction signal forming unit does not form a prediction signal from a frame immediately before and immediately after the encoding target frame, but generates a non-independent frame adjacent to the encoding target frame. In the case of a frame, this non-independent frame is skipped, and a prediction signal is formed from temporally earlier and later (old and new) independent frames. In this case, the prediction signals of a plurality of non-independent frames between two independent frames are all formed directly from the preceding and succeeding independent frames in the input order of the non-independent frames. The second encoding unit encodes a difference between the prediction signal output from the prediction signal forming unit and the input signal of the non-independent frame. This is shown in FIG. FIG. 6 is a diagram illustrating an inter-frame prediction method according to the present invention and a conventional inter-frame prediction method. In the figure, A is a conventional inter-frame prediction method (cyclic inter-frame prediction), and B is an inter-frame prediction method (bidirectional inter-frame prediction) according to the present invention. In the same figure, a square is a continuous frame of a moving image signal that is continuously input, and shaded frames are frames that are independently encoded within a frame.
Only (or at the time of reset) is an independent frame, but B has an independent frame periodically. Arrows indicate the directional relationship of inter-frame prediction. In A, prediction is performed only from the previous frame as in each frame, but in B, prediction is performed from two preceding and succeeding independent frames. (In the following description, a frame in which a prediction signal is temporally formed from both frames before and after the current frame is referred to as a bidirectional prediction frame.
In addition, the prediction is performed only based on the independent frame, and the frame predicted in both directions is not used for another prediction. (Embodiment) An embodiment of the interframe predictive coding system according to the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of a first embodiment of an inter-frame prediction encoding apparatus according to the present invention, and FIG. 2 is a block diagram showing the configuration of a second embodiment of the inter-frame prediction encoding apparatus according to the present invention. FIG. 3 is a block diagram showing the configuration of an embodiment of the inter-frame prediction decoding apparatus according to the present invention. The basic configuration of these encoding apparatus and decoding apparatus is similar to the conventional example, and the same reference numerals are given to the same components as those in the above-mentioned FIG. 4 or FIG. Attach. In FIGS. 1 and 2, there is provided an (N-1) frame memory 31 [N is an integer of 3 or more] for encoding a non-independent frame after encoding of an independent frame used for prediction. . Further, in order to form a prediction signal (prediction value) of a non-independent frame based on two preceding and succeeding independent frames, two frame memories 32 and 33 and two coefficient multipliers ｛× which weight each signal are used. α, × (1−α)｝ 34,35 and their adders 36
There is. [However, 0 <α <1] Further, the changeover switch 37 is connected to the image signal input terminal 1 and the (N-1) frame memory.
31, a switch 38 between the predictive signal subtractor 2 and the orthogonal transformer 3, a switch 39 between the quantizer 4 and the inverse quantizer 7, and a switch 40 between the two. It is provided between the frame memories 32 and 33, respectively. As will be described in detail later, the one shown in FIG. 1 obtains the same prediction signal as that of the decoding device side as in the conventional example, while the one shown in FIG. 2 converts the prediction signal from the original image signal to be encoded. The prediction signal is different between the encoding device side and the decoding device side. The configuration shown in FIG. 2 does not require decoding processing on the encoding device side, but is possible because the method of the present invention is not a cyclic processing as in the conventional example. FIG. 3 is a diagram showing an embodiment of the inter-frame prediction decoding apparatus according to the present invention, and includes a prediction signal adder 41 for adding a prediction signal to a signal obtained from the orthogonal inverse transformer 24. Further, in order to form a prediction signal of a non-independent frame based on two preceding and succeeding independent frames, two frame memories 42 and 43 and two coefficient multipliers ｛× α, × ( 1-α)｝ 44, 45 and their adders 46. Further, a changeover switch 47 is provided between the orthogonal inverse transformer 24 and the prediction signal adder 41, a changeover switch 48 is provided between the prediction signal adder 41 and the reproduced image signal output terminal 26, and a changeover switch 49 is provided for two frames. It is provided between the memories 42 and 43, respectively. In the configuration of the first embodiment shown in FIG. 1, the changeover switches 37 and 38 are switched to the "a" side every predetermined number of three or more frames, so that a moving image signal (continuous frame) input from the image signal input terminal 1 is input. ), Set an independent frame. The signal of this independent frame is output to the orthogonal transformer 3 without passing through the (N-1) frame memory 31 or the prediction signal subtractor 2.
And are encoded independently within the frame. The operations of the orthogonal transformer 3, the quantizer 4, and the variable-length encoder 5 are basically the same as those of the conventional example. On the other hand, a non-independent frame between the independent frames is a bidirectional predicted frame for obtaining a predicted signal by bidirectional inter-frame prediction, and the predicted signal is subtracted from an input signal, but the independent frame needs to be encoded first. Therefore, the non-independent frame is delayed by that amount. Here, if one independent frame is set to N frames [N is an integer of 3 or more], the delay amount is (N-1) frames. That is, in the case of a non-independent frame between independent frames, the changeover switches 37 and 38 are connected to the b side, and the signal is (N-1)
The frame signal is delayed by (N-1) frames in the frame memory 31, and the predicted signal is subtracted by the predicted signal subtractor 2, and then guided to the orthogonal transformer 3, where the prediction error (residual) is encoded. Here, the changeover switches 37 and 38 are periodically connected to the a side only for one frame in N frames, and are connected to the b side in other cases. The subsequent operations of the orthogonal transformer 3, the quantizer 4, and the variable-length encoder 5 are the same as in the case of the independent frame. The above operation is the same as in the case of FIG. 2 which is the second embodiment of the encoding device. In the case of the coding apparatus having the configuration shown in FIG. 2, the prediction signal is not obtained from the coded reproduced image signal, but is obtained from the original image signal before coding. The operation at the time of forming the prediction signal is the same as the first operation except that the original image signal is input instead of the encoded reproduced image signal.
It is the same as the case of the figure. 2, the inverse quantizer 7 and the inverse orthogonal transformer 8 in FIG. 1 are not required. In this case, the prediction signal differs between the transmission side and the reception side, but in the present embodiment, the error is not accumulated for each frame. Rather, since the quantization error does not remain in the prediction error (residual), the necessity of the spatial LPF in the conventional example is eliminated, and the prediction efficiency is improved. Next, how to create a prediction signal in the method of the present invention will be described. First, as in the conventional example, the same prediction signal is obtained on the encoding device side and the decoding device side, and this is the first embodiment shown in FIG. Here, the difference from the conventional example is that in the conventional example, all the frames are used to obtain the predicted signal, whereas in the present embodiment, the predicted signal is formed by information of only the independently encoded independent frames. Therefore, the changeover switch 39 is connected to the a side only for the independent frame, and the subsequent processing is performed. The quantized signal is replaced with a representative value by the inverse quantizer 7 (representative value setting) as in the conventional example, and the orthogonal inverse transformer 8 performs an inverse transform process of the orthogonal transform. Since the signal obtained in this manner is independently encoded, it is written to the frame memory 32 without being added to the previous frame or the like and used as it is for generating a prediction signal. At this time, the changeover switch 40 is connected to the “a” side, and the signal of the immediately preceding independent frame held in the frame memory 32 is replaced by the frame memory 33. By such an operation, a reproduction frame signal used for prediction simultaneously with the encoding processing of the independent frame is prepared in the frame memories 32 and 33. This reproduced frame signal is held until the signal of the next independent frame is supplied, and is repeatedly output (N-1) times for prediction processing. The prediction signal is obtained by multiplying the two reproduced frame signals by the weight multipliers α and (1−α) by the coefficient multipliers 34 and 35 and adding the result by the adder 36. Here, the weighting coefficient is determined by the time relationship between the encoding target frame input to the prediction signal subtractor 2 for encoding and the independent frame used for prediction. The most common approach is the one with linear prediction, given by: α = (m−mp) / N where m is the frame number to be encoded (1, 2, 3,...) and mp is the past (old)
Independent frame numbers (0, N, 2N,...), M> mp, and N is an integer of 3 or more. An example of a prediction signal (prediction value) generated in this manner will be described in the case of N = 4.
Shown in the figure. As a result, a frame that is closer in time is heavily weighted, and a more appropriate prediction value is given when the signal changes in a nearly linear manner for each frame. In FIGS. 1 and 2 described above, the input image signal is made into an independent frame every few frames by the switches 37 and 38, where the inter-frame correlation of the encoded data is cut off. Therefore, a visual search can be performed by random access in that unit or decoding only data of an independent frame. On the other hand, the signal of the non-independent frame is delayed by the (N-1) frame memory 31,
Before performing the non-independent frame prediction processing, the image signals of the independent frames used for the prediction are stored for two frames by the frame memories 32 and 33, so that the independent frames before and after (old and new) are temporally separated. A prediction signal is directly obtained without any other prediction. In addition, since an appropriate coefficient is multiplied by the time relationship between the frame predicted by the coefficient multipliers 34 and 35 and the independent frame, prediction suitable for a change in the image can be performed, and the S / N of the prediction signal can be improved. High prediction efficiency is obtained. In addition, since the encoded data obtained in this manner has a symmetrical structure on the time axis, reverse reproduction can be easily realized. Next, on the decoding device side, in the configuration of the embodiment shown in FIG. 3, first, similarly to the conventional example, the variable length digital data input from the data input terminal 21 is converted into a variable length coder 22, By the quantizer 23 and the orthogonal inverse transformer 24, a reproduced image signal is obtained in an independent frame that is an independent frame, and a prediction error (residual) signal is obtained in a bidirectional predicted frame that is a non-independent frame. Since the signal of the independent frame is used for prediction, the changeover switch 47 is connected to the side a and written into the frame memory 42. At this time, the changeover switches 48 and 49 are also connected to the a side, and the signal of the immediately preceding independent frame is replaced in the frame memory 43, and is simultaneously output from the reproduced image output terminal 26. In this way, the reproduction frame signals used in the prediction simultaneously with the decoding processing of the independent frame are prepared in the frame memories 42 and 43. On the other hand, in the case of a bidirectional prediction frame, the changeover switches 47, 48, and 49 are connected to the b side, and the prediction signal adder 41 adds the same prediction signal (prediction value) as that on the encoding apparatus side to obtain a reproduced image signal. Output from the output terminal 26. The method of forming a prediction signal by the coefficient multipliers 44 and 45 and the adder 46 is the same as that of the encoder. Since the data transmitted from the encoding device is transmitted in advance of the independent frame, the decoding device corrects the data. Is output from the frame memory 42. That is, the frame memory 42 also serves as time correction. (Effects of the Invention) As described above, in the method of the present invention, an independent frame not using bidirectional prediction is periodically set for every N (N is an integer of 3 or more) frames in a continuous frame of a continuously input image signal. In order to predict and encode non-independent frames between them before and after (new and old) independent frames, independent frames exist at regular intervals (several frames). Then, since the inter-frame correlation of the data is broken, random access can be made in units of the storage medium and a decoded image can be obtained immediately. On the other hand, if you try to do a visual search,
Since the independent frames exist at regular intervals (several frames), by decoding only the data of the independent frames, a smooth search image can be obtained without wasting data. Furthermore, since bidirectional prediction frames are basically symmetrically encoded on the time axis in reverse reproduction, they can be decoded in reverse order. However, since the independent frame data is recorded before other data, it is necessary to correct it during reverse playback.However, it is only necessary to perform time correction on the encoded and compressed data. A delay is fine. On the other hand, since encoding is performed independently at regular intervals (several frames), the coding efficiency is reduced in an image having a high inter-frame correlation. Good reproduction image quality can be easily obtained, and there is not much problem. Conversely, if the change in the image is large, the prediction error between frames (residual)
, The image quality is likely to deteriorate, and the effect of the present invention in which the prediction accuracy is improved is remarkable. In addition, in the method of the present invention, since a non-independent frame between the independent frames is predicted by preceding and succeeding (new and old) independent frames, prediction suitable for a change in an image becomes possible, and coding efficiency is improved. Further, since a prediction signal is formed by adding a plurality of frames, the S / N of the prediction signal is improved, and the prediction accuracy is improved. Furthermore, even if there is an error in the prediction signal between the encoding device and the decoding device, the error is not accumulated, and the encoding device may perform prediction on the original image signal instead of the encoded reproduction image signal. It is possible, in which case no signal processing is required in the encoding device. As described above, according to the method of the present invention, a high-precision prediction signal adapted to a change in an image is formed by bidirectional prediction as compared with the conventional case of only cyclic inter-frame prediction. Can be increased, and the coding efficiency is improved. In addition, random access, visual search, reverse playback, and the like can be performed on storage media, and encoding can be performed with high efficiency even for images that include scene changes or motion.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing the configuration of a first embodiment of an inter-frame predictive coding apparatus according to the present invention, and FIG. FIG. 3 is a block diagram showing a configuration of an embodiment of an inter-frame prediction decoding apparatus according to the present invention, and FIG. 4 is a block diagram showing a configuration of an inter-frame prediction encoding apparatus in a conventional example. FIG. 5, FIG. 5 is a block diagram showing a configuration of a conventional inter-frame prediction decoding apparatus, FIG. 6 is a diagram illustrating an inter-frame prediction method according to the present invention and a conventional inter-frame prediction method, and FIG. 5 is a diagram showing an example of an inter-frame prediction value in FIG. DESCRIPTION OF SYMBOLS 1 ... Image signal input terminal, 2 ... Prediction signal subtractor, 3 ... Orthogonal transformer, 4 ... Quantizer, 5 ... Variable length encoder, 6 ... Data output terminal, 7, 23 ... Inverse quantizer, 8 , 24 ... orthogonal inverse transformer, 21 ... data input terminal, 22 ... variable length decoder, 26 ... reproduced image signal output terminal, 27, 32, 33, 42, 43 ... frame memory, 28 ... spatial LPF, 31 ... ( N-1) Frame memory, 34, 35, 44, 45 ... coefficient multiplier, 36, 46 ... adder, 37, 38, 39, 40, 47, 48, 49 ... switch, 41 ... predicted signal adder.

Claims (1)

All Claims (1) successively setting the independent frame in the image signal input to form a prediction signal of the non-independent frame between the independent frame <br/> over beam from said independent frame An inter-frame predictive coding apparatus for coding the non- independent frame, wherein the independent frame is periodically set for every N (N is an integer of 3 or more) frames;
First frame storing the first coding means for performing, based coding of the independent frame to the information of only the independent frame, the last signal of the previous independent frame of the non-independent frame
And a second frame memory for storing signals of an independent frame in the immediate future.
, An output signal of the first frame memory and an output signal of the second frame memory
And all non-independent frames between the two independent frames.
A prediction signal of the frame, the encoding for the prediction error between the prediction signal forming means for chromatic and means for forming the input order, and the predicted signal and the corresponding signal of the non-independent frame of the non-independent frame 2. An inter-frame predictive coding apparatus comprising: (2) In an inter-frame predictive decoding device for decoding image information encoded using inter-frame prediction, the continuous image information is periodically set in advance every N (N is an integer of 3 or more) frames. of the independent frame am with a non-independent frame between the independent frame, the German
First decoding means for decoding a standing frame based on information of only the independent frame; and a first frame for storing a signal of an independent frame in the most recent past of the non-independent frame.
And a second frame memory for storing signals of an independent frame in the immediate future.
, An output signal of the first frame memory and an output signal of the second frame memory
And all non-independent frames between the two independent frames.
A prediction signal of les over arm, the prediction signal forming means for chromatic and means for forming the input order of the non-independent frame, the prediction error prediction signal and the corresponding said non-independent frames output from the prediction signal generating means And a second decoding means for decoding the non- independent frame by adding