JP2007060688A - Moving picture decoding method and apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce computational complexity and an increase in overhead of coded data as to a fade image, and to considerably improve the predictive efficiency. <P>SOLUTION: In carrying out motion compensation predictive inter-frame decoding to a decoding object block in a decoding object frame of a moving picture by using a plurality of reference frames with a prescribed combination and a motion vector between the decoding object block and at least one reference frame, the moving picture decoding apparatus decodes coded data including respective information items such as a predictive error signal between a predicted block and the decoding object block, the reference frames, weighting factors, a DC offset value and the motion vector, computes a linear sum by using the weighting factors as to a plurality of reference blocks extracted from the reference frames and adds the DC offset value to the linear sum to produce the predictive block and generates a reproduction moving picture signal by using the predictive error signal and a signal of the predictive block. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a moving picture decoding method and apparatus using motion compensated prediction interframe decoding.

  MPEG1 (ISO / IEC11172-2), MPEG2 (ISO / IEC13818-2), MPEG4 (ISO / IEC14496-2), and the like have been widely put into practical use as moving image compression coding techniques. In these moving image encoding systems, encoding is performed by a combination of intra-frame encoding (intra encoding), forward prediction inter-frame encoding, and bidirectional prediction inter-frame encoding, and encoding is performed in these encoding modes. Each frame is called an I picture, a P picture, and a B picture. The P picture is encoded using the immediately preceding P or I picture as a reference frame, and the B picture is encoded using the immediately preceding and immediately following P or I picture as a reference frame. Forward prediction interframe coding and bidirectional prediction interframe coding are called motion compensated prediction interframe coding.

  In MPEG video encoding, a predicted image can be selectively generated for each macroblock from one or a plurality of video frames. In the P picture, normally, a predicted image is generated in units of macroblocks from one reference frame. In B picture, when predictive image is generated from either forward or backward reference frame, reference macroblock is extracted from forward and backward reference frames, respectively, and predicted image is generated from the average value of those reference blocks There is a case to do. Information on these prediction modes is embedded in encoded data for each macroblock.

  In such motion-compensated prediction interframe coding, a good prediction result can be obtained when the same video is temporally translated between frames in a macroblock size or larger area. However, high prediction efficiency cannot always be obtained with respect to temporal fluctuations in signal amplitude such as temporal enlargement / reduction and rotation of video, or fade-in / fade-out. In encoding at a fixed bit rate, if a video that cannot achieve such a high prediction efficiency is input, the image quality may be greatly degraded. On the other hand, in variable bit rate encoding, a large amount of code is consumed for video with poor prediction efficiency in order to suppress image quality deterioration, and the total code amount is increased.

  Since temporal enlargement / reduction, rotation, and fade-in / fade-out of video can be approximated by affine transformation of moving image signals, prediction using affine transformation greatly improves the prediction efficiency. However, in order to estimate the affine transformation parameters, enormous parameter estimation calculations are required at the time of encoding. Specifically, it is necessary to convert the reference image with a plurality of conversion parameters and determine a parameter that minimizes the prediction residual, which increases the amount of conversion calculation. As a result, the amount of calculation for encoding or the cost of hardware becomes enormous. Further, it is necessary to encode not only the prediction residual signal but also the transformation parameter itself, and the overhead of encoded data becomes enormous. Furthermore, inverse affine transformation is required at the time of decoding, resulting in an enormous amount of decoding calculation or hardware cost.

  As described above, the conventional moving picture coding method such as MPEG has a problem that sufficient prediction efficiency cannot be obtained with respect to a time change of a moving picture other than parallel movement, and a moving picture using affine transformation is used. In the image encoding and decoding method, although the prediction efficiency itself is improved, there is a problem that the overhead of encoded data is increased and the encoding and decoding costs are greatly increased.

  The present invention reduces the amount of calculation and the increase in the overhead of encoded data, and predicts, particularly for faded images, which are not good at the conventional moving image encoding methods such as MPEG, in encoding and decoding of moving images. The objective is to greatly improve efficiency.

  In order to solve the above-described problem, in the first aspect of the present invention, a plurality of reference frames in a predetermined combination and a plurality of reference frames to be encoded and at least one reference frame, In the moving picture coding in which motion compensation prediction interframe coding is performed using a motion vector between each of the plurality of reference frames, at least one reference macroblock is extracted from each of the plurality of reference frames. A prediction macroblock is generated by calculating a linear sum using a set of weight coefficients, a prediction error signal between the prediction macroblock and the encoding target macroblock is generated, the prediction error signal, and the plurality of reference frames A first index indicating a combination of weights, a second index indicating the set of weighting factors, and the To encode information can vector.

In the second aspect of the present invention, a plurality of reference frames in a predetermined combination and a motion vector between the encoding target macroblock and at least one reference frame are used for the encoding target macroblock of the moving image. In the moving picture coding method or apparatus for performing motion compensated prediction inter-frame coding, a first reference macroblock corresponding to the motion vector candidate is extracted from a first reference frame, and the motion vector candidate is at least one Scaling according to the inter-frame distance between the second reference frame and the encoding target frame, extracting at least one second reference macroblock corresponding to the scaled motion vector candidate from the second reference frame; A linear sum is calculated using a predetermined set of weighting factors for the first and second reference macroblocks. To generate a prediction macroblock, generate a prediction error signal between the prediction macroblock and the encoding target macroblock, and generate a prediction error between the linear sum of the first and second reference macroblocks and the encoding target macroblock. The motion vector is determined based on a signal magnitude, the prediction error signal, a first index indicating the first and second reference frames, a second index indicating the set of weighting factors, and the determined motion In the third aspect of the present invention for encoding vector information, at least one reference frame in the past in time and the encoding target macroblock and the reference frame are encoded with respect to the encoding target macroblock of the moving image. In the moving picture coding method or apparatus for performing motion compensation prediction interframe coding using a motion vector between A motion vector of a decoding target macroblock having the same intra-frame position as that of the encoding target macroblock in the encoded frame immediately before the encoding target frame including the black block is used, or the motion vector is newly determined. The motion compensated prediction interframe coding is performed by switching whether to encode each macroblock to be encoded.

  In the fourth aspect of the present invention, a motion compensated prediction frame is generated using at least one reference frame and a motion vector between the encoding target macroblock and the reference frame with respect to the encoding target macroblock of the moving image. In a video encoding method or apparatus for performing inter-coding, a first prediction mode in which at least one encoded frame in the past in time is the reference frame, and an encoded frame in the future in time is the reference frame A third prediction mode in which a linear sum of past and future encoded frames in time is used as the reference frame, and a linear sum of a plurality of temporally encoded frames in time. The motion compensation prediction interframe coding is performed by switching the fourth prediction mode as the reference frame for each of the encoding target macroblocks.

  In the fifth aspect of the present invention, a plurality of reference frames in a predetermined combination and a motion vector between the decoding target macroblock and at least one reference frame are used for the decoding target macroblock of the moving image. In the video decoding method or apparatus for performing motion compensated prediction inter-frame decoding, a prediction error signal for each of the decoding target macroblocks, a first index indicating a combination of the plurality of reference frames, and a linear sum with respect to the reference macroblock A plurality of reference macros from the plurality of reference frames according to the decoded motion vector and the information of the first index according to the decoded motion vector and the information of the first index. The block is extracted and indicated by the decoded second index information. A prediction macroblock is generated by calculating a linear sum of the extracted plurality of reference macroblocks using the set of weighting factors, and the prediction for each decoding target macroblock decoded with the prediction macroblock The moving image signal is decoded by adding the error signal.

  In the sixth aspect of the present invention, a plurality of reference frames in a predetermined combination and a motion vector between the decoding target macroblock and at least one reference frame are used for the decoding target macroblock of the moving image. In the moving picture decoding method or apparatus for performing motion compensated prediction inter-frame decoding, a prediction error signal for each decoding target macroblock, a first index indicating a combination of the plurality of reference frames, and a frame of an encoded frame Decode encoded data including information on a second index indicating a number and the motion vector, and extract a plurality of reference macroblocks from the plurality of reference frames according to the decoded information on the motion vector and the first index. , The plurality of reference frames according to the decoded information of the second index And the encoded frame, and a linear sum of the extracted plurality of reference macroblocks is calculated using a set of weighting factors determined according to the calculated interframe distance. Thus, a prediction macroblock is generated, and the motion picture signal is decoded by adding the prediction macroblock and the decoded prediction error signal.

  In the seventh aspect of the present invention, with respect to a decoding target macroblock of a moving image, at least one reference frame in the past in time and a motion vector between the decoding target macroblock and at least one reference frame In the moving picture decoding method or apparatus for performing motion compensated prediction inter-frame decoding using, the prediction error signal for each decoding target macroblock and the encoded first motion vector or the immediately preceding encoded frame Decoding target macroblock that receives and decodes encoded data including any information of a flag indicating that the second motion vector of the macroblock at the same position in the frame is used, and receives the information of the first motion vector For the decoded first motion vector, for the decoding target macroblock that received the flag, Serial second motion vector using each generate a predictive macroblock signal, decodes the moving image signal by adding the predictive macroblock and the decoded the prediction error signal.

  In the eighth aspect of the present invention, motion compensated prediction interframe decoding is performed on a decoding target macroblock of a moving image using a motion vector between the decoding target macroblock and at least one reference frame. In the moving picture decoding method or apparatus, the prediction error signal information for each of the decoding target macroblocks and the first prediction mode in which at least one encoding target frame in the past is the reference frame, temporally A second prediction mode in which a future encoding target frame is the reference frame, a third prediction mode in which a linear sum of the past and future encoding target frames is the reference frame, and the temporal past Prediction mode information indicating any of the fourth prediction modes using a linear sum of a plurality of encoding target frames as the reference frame, and information on the motion vector Is received and decoded, a prediction macroblock signal is generated using the prediction mode information and the motion vector information, and the prediction macroblock signal and the decoded prediction error signal are added Thus, the moving image signal is decoded.

  In a ninth aspect of the present invention, for a coding target macroblock of a moving image, at least one reference frame selected from a plurality of reference frames, the coding target macroblock and the at least one reference frame, In a moving picture coding method or apparatus for performing motion compensated prediction interframe coding using a motion vector between the motion vectors, motion vectors for a plurality of macro blocks adjacent to the coding target macro block of the moving picture And at least one reference frame selected for the encoding target macroblock matches a reference frame for the macroblock for which the prediction vector is selected, and the motion compensated prediction frame A code in which prediction error signals to be encoded are all 0 in inter coding The number of macroblocks for which the motion compensation prediction interframe coding is skipped for the target macroblock and the motion compensation prediction interframe coding is skipped during the motion compensation prediction interframe coding of the next coding target macroblock. Is encoded.

  In the tenth aspect of the present invention, for a coding target macroblock of a moving image, at least one first reference frame selected from a plurality of reference frames, and the coding target macroblock and the first reference frame In a moving picture coding method or apparatus for performing motion compensated prediction interframe coding using a motion vector between a prediction error signal obtained by the motion compensated prediction interframe coding, and the motion compensated prediction interframe coding A difference vector between a motion vector used for the prediction vector, a prediction vector selected from motion vectors between a plurality of macroblocks adjacent to the encoding target macroblock and the second reference frame, and an index indicating the first reference frame; A difference value from an index indicating the second reference frame is encoded.

  In an eleventh aspect of the present invention, for a decoding target macroblock of a moving image, a motion vector between the decoding target macroblock and at least one reference frame selected from a plurality of reference frames is calculated. In the moving picture decoding method or apparatus for performing motion compensation prediction interframe decoding using the prediction error signal for each decoding target macroblock obtained by motion compensation prediction interframe coding, the macroblock skipped immediately before Received and decoded encoded data including information on the number and an index indicating the selected at least one reference frame, and from motion vectors of a plurality of macroblocks adjacent to the skipped macroblock in the skipped macroblock A prediction vector is selected, and the prediction vector is selected. It generates a predictive macroblock in accordance with at least one reference frame and the predicted vector for the block, and outputs the prediction macroblock as the decoded image signal of the skipped macroblock.

  In the twelfth aspect of the present invention, for a decoding target macroblock of a moving image, motion between the decoding target macroblock and at least one first reference frame selected from a plurality of reference frames In a video decoding method or apparatus for performing motion compensation prediction interframe decoding using a vector, a prediction error signal obtained by motion compensation prediction interframe coding, and a motion vector used for the motion compensation prediction interframe coding And a difference vector between a prediction vector selected from motion vectors between a plurality of macroblocks adjacent to the decoding target macroblock and a second reference frame, a first index indicating the first reference frame, and the first index Receiving and decoding encoded data including a difference value from a second index indicating two reference frames; The prediction vector is selected from a plurality of macroblocks adjacent to the block, the motion vector is reproduced by adding the selected prediction vector and the decoded difference vector, and the macro from which the prediction vector is selected The first index is reproduced by adding the index of the reference frame in the block and the decoded difference value, and a predicted macroblock is generated according to the reproduced motion vector and the reproduced first index. The decoded macroblock and the decoded prediction error signal are added to generate a decoded reproduced image signal of the decoding target macroblock.

  In the thirteenth aspect of the present invention, with respect to the encoding target block in the encoding target frame of the moving image, a plurality of reference frames in a predetermined combination and between the encoding target block and at least one reference frame are included. In the moving picture coding method for performing motion compensation prediction interframe coding using a motion vector, at least one weighting factor is determined from a ratio between an AC component value in the encoding target frame and an AC component value in the reference frame. Determining at least one DC offset value from a difference between a DC component value in the encoding target frame and a DC component value of the reference frame, and a plurality of reference blocks extracted from the reference frame Generate a prediction block by calculating a linear sum using a weighting factor and adding the DC offset value A step of generating a prediction error signal between the prediction block and the encoding target block, and encoding each information of the prediction error signal, the reference frame, the weighting factor, the DC offset value, and the motion vector. Comprising the steps of:

  On the other hand, for a decoding target block in a decoding target frame of a moving image, motion compensation is performed using a plurality of reference frames in a predetermined combination and a motion vector between the decoding target block and at least one reference frame. In a video decoding method for performing inter-prediction frame decoding, encoding including information on each of a prediction error signal between a prediction block and the decoding target block, the reference frame, a weighting factor, a DC offset value, and the motion vector Decoding the data; generating a prediction block by calculating a linear sum using the weighting factor for a plurality of reference blocks extracted from the reference frame and adding the DC offset value; A step of generating a playback video signal using the prediction error signal and the prediction block signal. Comprising the door.

  As described above, according to the present invention, the amount of computation and cost of encoding and decoding are reduced for video such as fade-in and fade-out, which has been unsatisfactory with conventional video encoding methods such as MPEG. The prediction efficiency can be greatly improved without requiring a large increase, and the overhead of encoded data is small, enabling high-quality and highly efficient video encoding and decoding. .

(First embodiment)
FIG. 1 shows the configuration of a video encoding apparatus according to the first embodiment of the present invention. The moving picture encoding apparatus shown in FIG. 1 may be realized by hardware, or may be executed by software using a computer. Some processing may be realized by hardware, and other processing may be performed by software. This also applies to moving picture coding apparatuses according to other embodiments described later.

  In FIG. 1, the encoded frame stored in the first reference frame memory 117 and the second reference frame memory 118 are stored for the moving image signal 100 (frame to be encoded) input for each frame. Prediction macroblock signals 130 to 133 are generated by the prediction macroblock generator 119 from the encoded frame. In the prediction macroblock selector 120, an optimal prediction macroblock signal is selected from the prediction macroblock signals 130 to 133, and the prediction image signal 106 is generated by the selected prediction macroblock signal.

  The prediction image signal 106 is input to the subtractor 110, and a prediction error signal 101 indicating an error of the prediction image signal 106 with respect to the input moving image signal 100 is generated. The prediction error signal 101 is subjected to discrete cosine transform by the DCT transformer 112, and the DCT coefficient data obtained thereby is quantized by the quantizer 113 to generate quantized DCT coefficient data 102. The quantized DCT coefficient data 102 is bifurcated while being encoded by the variable length encoder 114.

  On the other hand, the quantized DCT coefficient data 102 is reproduced as a prediction error signal via an inverse quantizer 115 and an inverse DCT transformer 116. The reproduced prediction error signal is added to the prediction image signal 106, thereby generating a locally decoded image signal 103. The locally decoded image signal 103 is input to the first reference frame memory 117. The first reference frame memory 117 stores a locally decoded image signal 103 of a frame that has been encoded immediately before the current encoding target frame, which is the input moving image signal 100, as a reference frame. In the second reference frame memory 118 connected to the output of the frame memory 117, the locally decoded image signal of the encoded frame encoded before that is stored as a reference frame.

  In the prediction macroblock generator 119, the prediction macroblock signal 130 generated only from the reference macroblock signal 104 extracted from the reference frame stored in the first reference frame memory 117 and the reference stored in the second reference frame memory 118. A prediction macroblock signal 131 generated from only the reference macroblock signal 105 extracted from the frame and a prediction macroblock signal 104 and 105 extracted from the first and second reference frame memories 117 and 118, respectively. It is generated by subtracting the reference macroblock signal 105 extracted from the second reference frame memory 118 from the signal obtained by doubling the amplitude of the macroblock signal 132 and the reference macroblock signal 104 extracted from the first reference frame memory 117. Predictive macrob The click signal 133 is generated.

  The prediction macroblock selector 120 calculates a difference between the plurality of prediction macroblock signals 130 to 133 generated by the prediction macroblock generator 119 and the encoding target macroblock signal extracted from the input moving image signal 100. The prediction macroblock signal that minimizes the error is selected for each encoding target macroblock.

  Further, the prediction macroblock selector 120 uses, for each encoding target macroblock, the relative position of the selected prediction macroblock signal viewed from the encoding target macroblock signal as motion vector information, and the selected prediction macroblock signal. Is generated as prediction mode information (a generation method of the prediction macroblock signals 130 to 133). Details of the prediction mode information will be described later in detail. By applying the motion vector and the prediction mode, the predicted image signal 106 is generated, and the prediction error signal 101 is generated based on the predicted image signal 106.

  When the signal component of the input moving image signal 100 includes a luminance signal and two color difference signals, the prediction macroblock selector 120 applies the same motion vector and prediction mode to each signal component of each macroblock. .

  The side information 107 including the quantized DCT coefficient data 102 obtained from the prediction error signal 101 through the DCT converter 112 and the quantizer 113, and the motion vector information and the prediction mode information output from the prediction macroblock selector 120, It is encoded by the variable length encoder 114 and output as encoded data 108. The encoded data 108 is sent to a storage system or transmission system (not shown).

  In the present embodiment, the prediction error signal 101 is encoded through the DCT converter 112, the quantizer 113, and the variable length encoder 114. For example, the DCT conversion is replaced with wavelet conversion, or the variable is variable. The long coding may be replaced with arithmetic coding.

(Second Embodiment)
FIG. 2 shows the configuration of a moving image encoding apparatus according to the second embodiment of the present invention. In the present embodiment, a fade detector 140 for the input moving image signal 100 is added to the moving image coding apparatus of the first embodiment shown in FIG. The fade detector 140 calculates an average luminance value for each frame of the input moving image signal 100, and determines that the image of the input moving image signal 100 is a fade image when there is a certain inclination in the luminance temporal change. Then, the determination result is notified to the prediction mode selector 120 as a fade detection signal 141.

  In the prediction mode selector 120, when the image of the input moving image signal 100 is determined to be a fade image by the fade detector 140, the prediction mode is predicted from one reference frame or linear extrapolation or linearity of a plurality of reference frames. The motion vector and the prediction mode that are optimal for each macroblock are determined by limiting to any one of the predictions by interpolation. A first flag indicating the determined motion vector and prediction mode is written in the header of the macroblock, and the prediction error signal 101 is encoded. On the other hand, the second flag indicating the set of prediction modes is written in the header data of the frame and output.

  Further, when the fade detector 140 determines that the image of the input moving image signal 100 is not a fade image, the prediction mode selector 120 sets the prediction mode based on prediction from one reference frame or an average value of a plurality of reference frames. Similarly to the prediction, the optimal motion vector and prediction mode are similarly determined, and the motion vector, the prediction mode, and the prediction error signal 101 are similarly encoded.

When receiving and decoding the encoded data 108 output from the moving picture encoding apparatus of FIG. 2, the prediction mode for each macroblock is determined from the first and second flags indicating the prediction mode, and A prediction macroblock signal is generated from the motion vector sent to and the determined prediction mode, and the encoded prediction error signal is decoded and added to the prediction signal for decoding.
With such a configuration, it is possible to reduce the coding overhead of prediction mode information.

  FIG. 3 shows the configuration of a moving picture decoding apparatus according to the first and second embodiments of the present invention. The moving picture decoding apparatus shown in FIG. 3 may be realized by hardware, or may be executed by software using a computer. Some processing may be realized by hardware, and other processing may be performed by software. This also applies to a moving picture decoding apparatus according to another embodiment described later.

  The moving picture decoding apparatus according to the present embodiment has a configuration corresponding to the moving picture encoding apparatus shown in FIG. 1 or FIG. That is, the encoded data 108 output from the moving image encoding apparatus shown in FIG. 1 or 2 is input as encoded data 200 to the moving image decoding apparatus of FIG. 3 via the transmission system or the storage system. The

  The encoded data 200 input to the moving picture decoding apparatus is first decoded into a variable length code by a variable length code decoder 214, and includes side data including quantized DCT coefficient data 201, motion vector information, and prediction mode information. Information 202 is extracted. The quantized DCT coefficient data 201 is decoded through an inverse quantizer 215 and an inverse DCT transformer 216, thereby reproducing a prediction error signal. The prediction error signal is added to the prediction image signal 206, and a decoded image signal 203 is generated.

  In the first reference frame memory 217, a decoded image signal 203 of a frame decoded immediately before the current moving image frame that is the encoded data 200 is stored as a reference frame, and is output to the output of the first reference frame memory 217. The connected second reference frame memory 218 further stores a decoded image signal of a previously decoded frame as a reference frame.

  In the prediction macroblock generator 219, the prediction macroblock signal generated only from the reference macroblock signal extracted from the reference frame stored in the first reference frame memory 217 and the reference frame stored in the second reference frame memory 218 are used. A predicted macroblock signal generated only from the extracted reference macroblock signal, a predicted macroblock signal generated by averaging the reference macroblock signals extracted from the first and second reference frame memories 217 and 218, and the first A prediction macroblock signal generated by subtracting the reference macroblock signal extracted from the second reference frame memory 218 from the signal obtained by doubling the amplitude of the reference macroblock signal extracted from the reference frame memory 217 is generated. These prediction macroblock signals are input to the prediction macroblock selector 220.

  The prediction macroblock selector 220 further receives the side information 202 from the variable length decoder 214. In the prediction macroblock selector 220, the prediction macro used at the time of encoding among the prediction macroblock signals output from the prediction macroblock generator 219 according to the motion vector information and the prediction mode information included in the side information 202. A predicted image signal 206 is generated by selecting the same signal as the block signal.

(About motion compensation prediction interframe coding)
FIG. 4 is an example schematically showing the interframe prediction relationship in the first and second embodiments. The encoding / decoding target frame 302, the immediately preceding frame 301, and the preceding frame 300 are shown. Show.

  When the frame 302 is encoded or decoded, the decoded image signal of the frame 301 is stored in the first reference memory 117 of FIGS. 1 and 2 or the first reference frame 217 of FIG. 1 and the second reference frame memory 118 of FIG. 2 or the second reference frame memory 218 of FIG. 3 stores the decoded image signal of the frame 300.

  For the encoding / decoding target macroblock 305, a prediction macroblock is generated using one or both of the reference macroblock 303 of the reference frame 300 and the reference macroblock 304 of the reference frame 301. Motion vectors 306 and 307 are vectors indicating the positions of the reference macroblocks 303 and 304, respectively.

  At the time of encoding, a search for an optimal motion vector and prediction mode for the encoding target macroblock 305 is performed. At the time of decoding, a prediction macroblock signal is generated for the decoding target macroblock 305 using the motion vector and prediction mode information included in the side information 202.

  5 and 6 show examples of prediction coefficient tables used in a prediction mode based on a linear sum of a plurality of reference frames among the prediction modes in the first and second embodiments. The prediction coefficient changes for each macroblock in the first embodiment and for each frame in the second embodiment, and there are two sets of coefficients, average and linear extrapolation.

  The index (Code_number) shown in FIG. 5 or 6 is encoded as header data for each macroblock or each frame. In the second embodiment, since the linear prediction coefficient is fixed for each frame, it may be encoded using only the header data of the frame. The numerical value of the coefficient is explicitly defined in the prediction coefficient table shown in FIG. 5, and the average or linear prediction (interpolation or extrapolation) is shown in the prediction coefficient table shown in FIG. By encoding such an index, the amount of information to be encoded can be reduced and the encoding overhead can be reduced as compared with the case where the linear prediction coefficient is directly encoded.

  FIG. 7 is a table showing combinations of reference frames (Reference_frame) related to various prediction modes in the first and second embodiments of the present invention. In FIG. 7, Code_numbe = 0 is a prediction mode from the immediately preceding frame (one frame before), Code_numbe = 1 is a prediction mode from two frames before, and Code_numbe = 3 is a prediction based on a linear sum from one frame and two frames before. Each reference frame combination in the mode is shown. When Code_number = 3, the prediction mode using the linear prediction coefficient described above is used.

  In the first and second embodiments, the combination of reference frames can be changed for each macroblock, and the index in the table of FIG. 7 is encoded for each macroblock.

(Third embodiment)
8 and 9 show the configurations of a moving picture encoding apparatus and a moving picture decoding apparatus according to the third embodiment of the present invention, respectively. In the first and second embodiments, prediction is performed based on a linear sum from a maximum of two reference frames. In the third embodiment, three or more reference frames are used to specify each macroblock. Can be predicted by either selecting one frame or by linear sum of a plurality of reference frames.

  8 includes reference frame memories 117, 118, and 152 corresponding to the maximum number of reference frames (n), and the moving image decoding apparatus in FIG. 9 similarly corresponds to the maximum number of reference frames (n). Reference frame memories 217, 218 and 252 are provided. In the present embodiment, when performing prediction by linear sum, the prediction macroblock generators 151 and 251 perform a sum-of-products operation between the prediction coefficients W1 to Wn and the reference macroblock extracted from each reference frame, and the result is the Wd bit. Only a right shift is performed to generate a predicted image signal. The selection of the reference frame can be changed for each macroblock, and the linear prediction coefficient can be changed for each frame. Therefore, a set of linear prediction coefficients is encoded as frame header data, and a reference frame is selected as header data for each macroblock.

FIG. 10 shows a data syntax for encoding the linear prediction coefficient according to the present embodiment as a frame header. In the encoding of the linear prediction coefficient, first, the maximum number of reference frames is encoded as Number_Of_Max_References, then WeightingFactorDenominatorExponent (Wd in FIGS. 8 and 9) indicating the calculation accuracy of the linear prediction coefficient is encoded, and the number of Number_Of_Max_References is further encoded. Coefficients WeightingFactorNumerator [i] (W1 to Wn in FIGS. 8 and 9) for each reference frame are encoded. The linear prediction coefficient for the i-th reference frame is expressed by the following equation.

  FIG. 11 shows a reference frame combination table encoded for each macroblock according to the present embodiment. Code_number = 0 indicates prediction based on a linear sum of all reference frames, and after Code_number = 1, the reference frame is a specific one frame, and indicates how many frames before the reference frame. When performing prediction by linear sum of all reference frames, prediction using the prediction coefficient shown in FIG. 10 is performed. Here, by setting a part of the prediction coefficients to 0, in the linear prediction mode, it is possible to switch linear prediction based on a combination of arbitrary reference frames in units of frames.

  12 and 13 show the relationship of inter-frame prediction using three or more reference frames according to this embodiment. FIG. 12 shows an example using a plurality of past reference frames, and FIG. 13 shows an example using a plurality of past and future reference frames. In FIG. 12, reference frames 800 to 803 are used for the encoding target frame 804.

  At the time of encoding, reference macroblocks 809 to 812 are extracted from each reference frame according to motion vectors 805 to 808 for each reference frame for each encoding target macroblock 813, and linearity from the extracted reference macroblocks 809 to 812 is extracted. A prediction macroblock is generated by prediction. Next, a set of a motion vector and a prediction mode that minimizes the prediction error is selected in one of a plurality of reference macroblocks 809 to 812 or a prediction mode of a prediction macroblock based on linear prediction. One set of linear prediction coefficients is determined for each encoding target frame, for example, from a temporal change in average luminance between frames. The determined set of prediction coefficients is encoded as header data of the encoding target frame, and the motion vector, prediction mode, and prediction error signal of each macroblock are encoded for each macroblock.

  At the time of decoding, using a set of linear prediction coefficients received for each frame, a prediction macroblock is generated from a plurality of reference frames from motion vector and prediction mode information for each macroblock, and added to a prediction error signal. To perform decryption.

  In FIG. 13, reference frames 900, 901, 903, and 904 are used for the encoding target frame 902. In the case of FIG. 13, the frames are rearranged so that the encoding target frame 902 and the reference frames 900, 901, 903, and 904 are in the order of 900, 901, 903, 904, and 902 at the time of encoding and decoding. In the case of encoding, a plurality of locally decoded image frames are used as reference frames, and in the case of decoding, a plurality of encoded frames are used as reference frames.

  For the encoding target macroblock 911, one of the reference macroblocks 909, 910, 912, and 913 or a prediction signal based on linear prediction therefrom is selected for each macroblock as in the example of FIG. Encoded.

  FIG. 14 is a diagram illustrating an encoding method and a decoding method of motion vector information according to the embodiment of the present invention. As in the example of FIG. 4, in interframe coding using a plurality of reference frames, when generating a prediction macroblock signal using a plurality of reference macroblock signals for each encoding target macroblock, for each macroblock It is necessary to encode a plurality of motion vector information. Therefore, as the number of macroblocks to be referred to increases, the overhead of motion vector information to be encoded increases, which causes a decrease in encoding efficiency.

  In the example of FIG. 14, when a reference macroblock signal is extracted from each of two reference frames to generate a predicted macroblock signal, one motion vector and a motion vector obtained by scaling the motion vector according to the interframe distance Is used. Reference frames 401 and 400 are used for the encoding target frame 402, and motion vectors 411 and 410 are detected. A black point indicates a pixel position in the vertical direction, and a white point indicates an interpolation point with 1/4 pixel accuracy.

  FIG. 14 shows an example in which motion compensation prediction interframe coding is performed with 1/4 pixel accuracy. The pixel accuracy of motion compensation is defined for each encoding method, such as 1 pixel, 1/2 pixel, and 1/8 pixel. In general, a motion vector is generally expressed with motion compensation accuracy, and a reference image is generated by interpolation from image data of a reference frame.

  In FIG. 14, focusing on the pixel 405 to be encoded, it is assumed that a point 403 that is 2.5 pixels away from the reference frame 400 is referred to, and a motion vector 410 indicating a deviation of 2.5 pixels is encoded. Is done. On the other hand, the prediction from the reference frame 401 for the pixel 405 is generated by scaling the coded motion vector 410 described above according to the interframe distance. Here, the motion vector for the frame 401 is 2.5 / 2 = 1.25 pixels in consideration of the inter-frame distance, and the pixel 404 in the reference frame 401 is used as the reference pixel of the pixel 405 in the encoding target frame 402.

  By performing motion vector scaling with the same accuracy during encoding and decoding, the motion vector to be encoded for each macroblock is one motion vector even when the encoding target macroblock refers to multiple frames. Thus, it is possible to prevent an increase in coding overhead. Here, if the scaling result of the motion vector is not on the sample point of the accuracy of motion compensation, the scaled motion vector is rounded off by rounding off to the nearest whole number.

  FIG. 15 is a diagram illustrating an encoding method and a decoding method of motion vector information different from those in FIG. 14 according to the embodiment of the present invention. In the example of FIG. 14, it is possible to efficiently reduce the overhead of the motion vector in the encoded data when the temporal motion speed of the moving image is constant. On the other hand, if the temporal motion of a moving image is monotonous but the speed of motion is not constant, using a simply scaled motion vector will cause a decrease in prediction efficiency and cause a decrease in coding efficiency. It may become.

  In FIG. 15, as in FIG. 14, a prediction pixel is generated from the reference pixels of the two frames of the reference frames 500 and 501 as the reference pixel of the pixel 506. Here, the pixel 503 of the frame 500 and the pixel 505 of the frame 501 are referred to. Similar to the example of FIG. 14, the motion vector 510 for the frame 500 is encoded, and in addition, the motion vector 511 for the frame 501 is encoded as a difference vector 520 with a vector obtained by scaling the motion vector 510. By scaling the motion vector 510 to ½, the position of the pixel 504 in the frame 501 is indicated, and a difference vector 520 indicating the difference amount between the original predicted pixel 505 and the pixel 504 is encoded.

  Normally, the size of the above-mentioned difference vector becomes small with respect to monotonous motion in time, so even if the speed of motion is not constant, the increase in motion vector overhead is suppressed without reducing prediction efficiency. Thus, efficient encoding can be performed.

  FIG. 16 is a diagram illustrating still another motion vector information encoding method and decoding method according to the embodiment of the present invention. In the example of FIG. 16, the frame 603 is an encoding target frame, the frame 602 is skipped, and the frames 601 and 600 are reference frames. Furthermore, with respect to the pixel 606, the pixel 604 of the reference frame 600 and the pixel 605 of the reference frame 601 are reference pixels for generating a prediction pixel. Similarly to the example of FIG. 14 or FIG. 15, it is possible to encode the motion vector 611 for the reference frame 600 and generate a motion vector for the reference frame 601 using a motion vector obtained by scaling the motion vector 611. In the case of FIG. 16, the motion vector 611 needs to be scaled 2/3 times from the relationship between the frame distances of the reference frame and the encoding target frame.

  In order to perform arbitrary scaling, not limited to the example of FIG. 16, the denominator becomes an arbitrary integer that is not a power of 2, and division is required. Motion vector scaling is required for both encoding and decoding. In particular, division requires a lot of cost and calculation time in both hardware and software. It will increase. In FIG. 16, a motion vector 610 obtained by normalizing the motion vector 611 to be encoded is encoded, and the normalized motion vector 610 is converted according to the interframe distance between the encoding target frame and each reference frame. A difference vector between the scaled motion vector and the original motion vector is encoded. That is, the reference pixel 604 is generated from the motion vector obtained by multiplying the normalized motion vector 610 by three times and the difference vector 620, and the reference pixel 605 is obtained by subtracting the motion vector obtained by doubling the normalized motion vector 610 from the difference. Generated from vector 621.

  In this way, the configuration shown in FIG. 16 prevents an increase in the coding overhead of the motion vector without reducing the prediction efficiency, and further, the scaling of the motion vector can be realized only by multiplication. It is possible to reduce the calculation cost.

In the encoding of the motion vector or the difference vector in the embodiment of the present invention, the motion vector code amount is further reduced by using the spatial correlation or the temporal correlation of the motion vector as follows.
First, an example of a motion vector compression method using spatial correlation will be described with reference to FIG. A, B, C, D, and E shown in FIG. 17 indicate adjacent macroblocks in one frame. When the motion vector or difference vector of the macroblock A is encoded, a prediction vector is generated from the motion vectors of the adjacent macroblocks B, C, D, and E, and the prediction vector and the motion vector of the macroblock A to be encoded are Only the error is encoded, and on the decoding side, a prediction vector is calculated in the same manner as the encoding, and is added to the encoded miscalculation signal, thereby generating a motion vector or a difference vector of the macroblock A.

  The motion vector error can be encoded with high efficiency by using a variable length code or an arithmetic code. The prediction of the motion vector can be realized by using, for example, the median value or the average value of the motion vectors of the macroblocks B, C, D, and E as the prediction vector.

  Next, an example of a motion vector compression method using time correlation will be described with reference to FIG. FIG. 18 shows two consecutive frames (F0, F1). In the figure, A, B, C, D, and E are adjacent macroblocks in the frame F1, and a, b, c, d, and e are the same positions as A, B, C, D, and E in the frame F0. Each macroblock is shown. Here, when the motion vector or the difference vector in the macroblock A is encoded, the motion vector of the macroblock a at the same position in the immediately preceding frame F0 is set as the prediction vector, and the prediction vector and the vector to be encoded in the macroblock A are It is possible to compress the motion vector information by encoding only the error.

  Furthermore, using the space-time correlation, the motion vector of the macroblock A is calculated using the motion vectors of the macroblocks B, C, D, E of the frame F1 and the macroblocks a, b, c, d, e of the frame F0. By performing three-dimensional prediction and encoding only the error between the prediction vector and the vector to be encoded, the motion vector can be further highly efficiently compressed.

  For example, three-dimensional prediction of a motion vector can be realized by generating a prediction vector from a median value or an average value of a plurality of motion vectors adjacent to the space-time.

  Next, a macroblock skip embodiment according to the present invention will be described. In the macro block in which the prediction error signal to be encoded becomes 0 by DCT and quantization in the motion compensated predictive encoding, in order to reduce the encoding overhead, in the macro block that satisfies the predetermined condition, Encoding only the number of macroblocks skipped continuously in the header of the macroblock to be encoded next without encoding any data, including the macroblock header data such as the prediction mode and motion vector, It is assumed that the macroblock skipped at the time of decoding is decoded in a predetermined mode defined in advance.

  In the first aspect of the macroblock skip according to the embodiment of the present invention, all reference frames used for prediction are predetermined, all motion vector elements are 0, and all prediction error signals to be encoded are 0. A condition that satisfies the condition is defined as a macroblock skip, and when decoding, a predicted macroblock is generated from a predetermined reference frame in the same manner as when the motion vector is 0, and the generated predicted macroblock is decoded. As a macroblock signal.

  Here, by setting the linear sum of the previous two frames as a reference frame as a reference frame skip condition, macroblock skip is generated even in a video whose signal intensity changes with time, such as a fade image. Therefore, it is possible to improve the encoding efficiency. Alternatively, a configuration may be adopted in which an index of a reference frame serving as a skip condition is sent as header data of each frame, and the skip condition is variable for each frame. By making the frame skip condition variable for each frame, it is possible to set an optimal skip condition in accordance with the nature of the input image and to reduce the coding overhead.

  In the second mode of the macroblock skip according to the embodiment of the present invention, the motion vector is assumed to be predictive-coded, and the case where the motion vector error signal is 0 is set as the macroblock skip condition. Other conditions are the same as in the above-described first mode of macroblock skip. In the second aspect, when a skipped macroblock is decoded, a prediction motion vector is first generated, a prediction image is generated from a predetermined reference frame using the generated prediction motion vector, and decoding of the macroblock is performed. Signal.

  In the third aspect of the macroblock skip according to the embodiment of the present invention, the skip condition is that the motion vector information to be encoded is the same as the motion vector information encoded in the immediately preceding macroblock. Here, the motion vector information to be encoded is a prediction error vector when performing motion vector predictive encoding, and is a motion vector itself when not performing predictive encoding. Other conditions are the same as in the first aspect described above.

  In this third mode, when a skipped macroblock is decoded, first, motion vector information is reproduced assuming that the motion vector information to be encoded is 0, and a predetermined reference frame is reproduced according to the reproduced motion vector. A predicted image is generated from the above and used as a decoded signal of a macroblock.

  In the fourth aspect of macroblock skip according to the embodiment of the present invention, the combination of reference frames used for prediction is the same as the macroblock encoded immediately before, and other skip conditions are the above-described first It is set as the same structure as an aspect.

  In the fifth aspect of the macroblock skip according to the embodiment of the present invention, the combination of reference frames used for prediction is the same as that of the macroblock encoded immediately before, and other skip conditions are the above-described second conditions. It is set as the same structure as an aspect.

  In the sixth aspect of the macroblock skip according to the present invention, the combination of reference frames used for prediction is the same as that of the macroblock encoded immediately before, and other skip conditions are the same as those of the third aspect described above. The configuration.

  In any of the skip conditions of the first to sixth aspects, the macroblock skip is efficiently generated and encoded using the property that the correlation between the motions of adjacent macroblocks and the temporal change in signal strength is high. It is possible to reduce overhead and realize highly efficient encoding.

(Fourth embodiment)
FIG. 19 shows the configuration of a moving picture encoding apparatus according to the fourth embodiment of the present invention. In this embodiment, a linear prediction coefficient estimator 701 is added to the moving picture coding apparatus according to the third embodiment shown in FIG. The linear prediction coefficient estimator 701 determines a prediction coefficient in linear prediction from a plurality of reference frames according to the interframe distance between the reference frame and the encoding target frame, the temporal change of the intra-frame DC component of the input frame, and the like. Do. Hereinafter, a plurality of embodiments regarding specific prediction coefficient determination will be described.

FIG. 20 shows a case where prediction is performed using a linear sum from the past two frames, and reference frames F0 and F1 are used for the encoding target frame F2. Ra and Rb indicate interframe distances between the reference frames F0 and F1 and the encoding target frame F2. The linear prediction coefficients for the reference frames F0 and F1 are W0 and W1, respectively. A set of first linear prediction coefficients is (0.5, 0.5), that is, a simple average of two reference frames. The second linear prediction coefficient is determined by linear extrapolation according to the interframe distance. In the example of FIG. 20, it becomes like Formula (2). For example, when the frame interval is constant, Rb = 2 * Ra, and the linear prediction coefficient is (W0, W1) = (− 1, 2) from Equation (2).

  From equation (2), even when the interframe distance between each reference frame and the encoding target frame is arbitrarily changed, appropriate linear prediction is possible. For example, high prediction efficiency can be maintained even when variable frame rate encoding using frame skip or the like, or when two arbitrary past frames are selected as reference frames. At the time of encoding, either the first prediction coefficient or the second prediction coefficient may be used in a fixed manner, or may be selected adaptively. A specific method for adaptively selecting the prediction coefficient can be determined using, for example, an average luminance value (DC value) in each frame.

The average luminance values in the frames F0, F1, and F2 are DC (F0), DC (F1), and DC (F2), respectively, and the prediction error in the case where each linear prediction coefficient is used for the DC component in the frame. The size is calculated as in equations (3) and (4).

If the value of Expression (3) is smaller than the value of Expression (4), the first prediction coefficient is selected, and if the value of Expression (4) is smaller than the value of Expression (3), the second prediction coefficient is selected. By changing these prediction coefficients for each frame to be encoded, optimal linear prediction can be performed according to the characteristics of the video signal. Even if the third and fourth prediction coefficients are determined for each frame as shown in the equation (5) or (6) using the ratio of the DC value in the frame, efficient linear prediction is possible.

  The third linear prediction coefficient shown in Expression (5) is a weighted average that takes into account the ratio of the DC values within the frame, and the fourth linear prediction coefficient shown in Expression (6) is the ratio between the DC value within the frame and the inter-frame DC value. It is a linear prediction that takes distance into account. In the above second to sixth linear prediction coefficients, division is required for linear prediction, but linear prediction by multiplication and bit shift is performed without using division by matching the calculation accuracy at the time of encoding and decoding. Is possible.

As a specific syntax, each linear prediction coefficient may be expressed by a denominator of power of 2 and an integer numerator as in the example shown in FIG. FIG. 21 shows a case where prediction is performed by a linear sum from two frames preceding and following in time, F1 in the figure indicates an encoding target frame, and F0 and F2 indicate reference frames, respectively. Ra and Rb indicate the inter-frame distance between each reference frame and the encoding target frame. Also, the linear prediction coefficients for the reference frames F0 and F2 are W0 and W2, respectively. In addition, the average value of the luminance value of each frame is assumed to be DC (F0), DC (F1), and DC (F2). Expressions (7) to (10) show examples of sets of four types of prediction coefficients similar to the case of FIG.

  Equation (7) is a simple average prediction, Equation (8) is a weighted average prediction of interframe distance, Equation (9) is a weighted prediction based on a ratio of DC values, and Equation (10) is a ratio of DC values to an interframe. It is a weighted prediction based on distance.

  FIG. 22 shows a case of performing prediction by linear sum from the past three frames, where F0, F1, and F2 are reference frames, and F3 is a frame to be encoded. The interframe distances between the reference frames F0, F1, and F2 and the encoding target frame F3 are indicated by Rc, Rb, and Ra, respectively. Also in FIG. 22, a set of a plurality of linear prediction coefficients can be considered. Specific examples are shown below. Here, it is assumed that the linear prediction coefficients for each reference frame are W0, W1, and W2.

A set of first prediction coefficients is shown in Equation (11). The first prediction coefficient is a simple average prediction from three reference frames. A predicted image aF ₃ ⁰¹² based on the first set of prediction coefficients is expressed by Expression (12).

The second, third, and fourth prediction coefficients are coefficients for selecting two frames from three reference frames and performing extrapolation prediction by linear extrapolation in the same manner as Expression (2). The prediction image of the encoding target frame F3 predicted from F1 is set to eF ₃ ^12, and the prediction image of the encoding target frame F3 predicted from the reference frames F2 and F0 is set to eF ₃ ⁰² and the encoding target predicted from the reference frames F1 and F0. Assuming that the predicted image of the frame F3 is eF ₃ ⁰¹ , it is represented by the equations (13), (14) and (15), respectively.

Also, assuming that a predicted value obtained by averaging equations (13) to (15) is eF ₃ ⁰¹² , eF ₃ ⁰¹² is expressed by equation (16), and this is set as a fifth prediction coefficient.

  Any one of the first to fifth linear prediction coefficients may be used, and the average luminance values DC (F0), DC (F1), and DC in the frames F0, F1, F2, and F3, respectively. (F2) and DC (F3) are calculated, the intraframe average luminance value of the encoding target frame F3 is predicted by the above five types of linear prediction, and the linear prediction coefficient with the smallest prediction error is calculated for each encoding target frame. The structure used by switching may be used. If the latter configuration is used, an optimal linear prediction is automatically selected for each frame in accordance with the properties of the input image, and highly efficient encoding can be realized.

Moreover, the structure using the prediction coefficient which multiplied the ratio of the average brightness | luminance of each flame | frame to each said 1st-5th linear prediction coefficient may be sufficient. For example, when the first prediction coefficient is multiplied by the ratio of the average luminance value, the prediction coefficient is expressed by Expression (17). The same applies to the other prediction coefficients.

FIG. 23 shows a case where prediction by linear sum from the past two frames and the future one frame is performed, where F0, F1, and F3 are reference frames, and F2 is an encoding target frame. The interframe distance between each reference frame and the encoding target frame is indicated by Rc, Rb, and Ra, respectively. Also in this case, similarly to the example of FIG. 22, it is possible to determine a set of a plurality of prediction coefficients by using the ratio of the inter-frame distance and the ratio of the intra-frame DC value. From the error, it is possible to determine the optimum set of prediction coefficients for each frame.
Expressions (18) to (23) show linear prediction expressions or prediction coefficients corresponding to the expressions (12) to (17) in the prediction structure of FIG.

  FIG. 24 shows a first example of motion vector search in moving picture coding according to the embodiment of the present invention. FIG. 24 shows a motion vector search method in the case where two consecutive frames are predicted as reference frames as shown in FIG. 14 and one representative motion vector is encoded. F2 in the figure is the encoding target frame, and F0 and F1 indicate reference frames. Reference numeral 10 denotes a macroblock to be encoded, and 12, 14, 16, and 18 denote a part of reference macroblock candidates in a reference frame.

  In order to obtain an optimal motion vector for the macroblock 10, motion vector candidates for the reference frame F1 (in FIG. 24, motion vector candidates 11 and 15) within the motion vector search range, and scaled according to the interframe distance The motion vectors (in FIG. 24, the motion vector 13 obtained by scaling the motion vector candidate 11 and the motion vector 17 obtained by scaling the motion vector candidate 15) are used as motion vectors for the reference frame F0 and extracted from the two reference frames F0 and F1. A prediction macroblock is generated from the linear sum of the reference macroblocks 14 and 12 or 16 and 18, a difference value between the prediction macroblock and the encoding target macroblock 10 is calculated, and a motion vector having the smallest difference value is moved. Macros as vector search results Tsu determined for each click, using the determined motion vector, motion compensation predictive encoding for each macroblock.

  As an evaluation method for determining a motion vector, in addition to the above, the determination may be made in consideration of not only the difference value but also the encoding overhead of the motion vector itself, or the difference signal and the motion vector are actually encoded. It is also possible to select a motion vector that minimizes the amount of code at that time. By searching for a motion vector as described above, it is possible to obtain an accurate motion vector with a smaller amount of computation than when searching optimal motion vectors for F0 and F1 individually.

  FIG. 25 shows a second example of motion vector search in moving picture coding according to the embodiment of the present invention. FIG. 25 is similar to the example of FIG. 24, as shown in FIG. 14, when two consecutive frames are predicted as reference frames and one representative motion vector is encoded, or one representative motion vector A motion vector search method when a difference vector is encoded will be described. F2 in the figure is the encoding target frame, and F0 and F1 indicate reference frames. Reference numeral 10 denotes a macroblock to be encoded, and 12, 14, 16, and 18 denote a part of reference macroblock candidates in a reference frame.

  In the second motion vector search, first, similarly to the first motion vector, one motion vector is searched for two reference frames. In FIG. 25, it is assumed that the motion vector 11 and the motion vector 13 obtained by scaling the motion vector 11 are selected as optimal motion vectors. Next, the motion vector is re-searched for the reference macroblock from the frame F0 in the vicinity region of the motion vector 13. In the re-search, the reference macroblock 12 extracted from the frame F1 using the motion vector 11 is fixed, and the linear sum of the reference macroblock 12 and the reference macroblock 14 extracted from the vicinity of the motion vector 13 of the frame F0 is fixed. To generate a prediction macroblock, and re-search the motion vector for the frame F0 so that the difference between the prediction macroblock and the encoding target macroblock is minimized.

  In the case where the moving image signal has a fixed frame rate and the frames F2 and F1 and the intervals between F1 and F0 are equal, for example, in order to search for a certain motion, the search range for the reference frame F0 is: It is necessary to search an area that is four times the area ratio with respect to the search range for the reference frame F1, and if a motion vector search with the same accuracy is performed for both reference frames F0 and F1, prediction from only F1 is performed. Compared to the motion vector search in FIG.

  On the other hand, according to the second motion vector search method, first, a motion vector search with full accuracy with respect to the reference frame F1, a search with respect to the reference frame F0 by a motion vector obtained by scaling the motion vector twice, and then, By adopting a two-stage configuration for performing full-accuracy re-searching with respect to the reference frame F0, it is possible to reduce the amount of motion vector search computation to almost ¼.

  In the second motion vector search method, motion vectors for the reference frames F0 and F1 are obtained separately. These motion vectors are encoded by first encoding the motion vector 11 for the reference frame F1, then scaling the motion vector 11 and the motion vector obtained as a result of the re-search for the reference frame F0. By encoding each difference vector, the motion vector encoding overhead can be reduced.

  Further, since the re-search range is ± 1, that is, the motion vector 13 obtained by scaling the motion vector 11 has a coarse accuracy of ½, the re-search is performed when only the re-search is performed to make it full accuracy. By scaling the motion vector for the reference frame F0 to ½, the motion vector 11 for the reference frame F1 can be reproduced uniquely regardless of the re-search result, so only the motion vector for the reference frame F0 is obtained. May be encoded. At the time of decoding, if the received motion vector is scaled to ½, the motion vector 11 for the reference frame F1 can be obtained.

  FIG. 26 shows a third example of motion vector search in moving picture coding according to the embodiment of the present invention. As in the example of FIG. 24, FIG. 26 shows a case where two consecutive frames are predicted as reference frames and one representative motion vector is encoded, or one representative motion vector and The motion vector search method in the case of encoding a difference vector is shown. F2 in the figure is the encoding target frame, and F0 and F1 indicate reference frames. Reference numeral 10 denotes a macroblock to be encoded, and 12, 14, 16, and 18 denote a part of reference macroblock candidates in a reference frame.

  In the third motion vector search, similarly to the first example or the second example, the motion vector for the reference frames F0 and F1 is searched, and the motion vector for the reference frame F1 is searched again. In general, a moving image utilizes the property that the correlation between temporally close frames is strong, and in the third motion vector search, the motion vector for the reference frame F1 closest in time to the encoding target frame F2 is more accurately determined. By obtaining, it becomes possible to improve the prediction efficiency.

  FIG. 27 shows a motion vector encoding method according to the embodiment of the present invention. In the figure, F2 indicates a frame to be encoded, F1 indicates a frame encoded immediately before, 30 and 31 are macroblocks to be encoded, and 32 and 33 are macroblocks 30 and 31 in the frame F1 and in the frame, respectively. Macroblocks at the same position are shown. Reference numerals 34 and 35 denote motion vectors to be encoded of the macro blocks 30 and 31, respectively. Reference numerals 36 and 37 denote encoded motion vectors of the macro blocks 32 and 33, respectively.

  In this embodiment, when the motion vector to be encoded is the same as the motion vector in the macroblock at the same position in the immediately preceding encoding target frame, the motion vector is not encoded and the motion vector is immediately before the encoding target frame. A flag indicating that it is the same as the macroblock at the same position in is encoded as a prediction mode. If the motion vector is not the same as the macro block at the same position in the immediately preceding encoding target frame, the motion vector information is encoded. In the example of FIG. 27, since the motion vectors 34 and 36 are the same, the motion vector 34 is not encoded, and the motion vector 35 and the motion vector 37 are different, so the information of the motion vector 35 is encoded.

  By encoding motion vector information as described above, redundancy of motion vector information can be reduced and encoding efficiency can be improved when still images or temporally uniform motion is performed. Is possible.

  FIG. 28 shows another motion vector encoding method according to the embodiment of the present invention. In the example of FIG. 28, when the motion vector of the macroblock at the same position and the motion vector of the macroblock to be encoded are the same in the immediately preceding encoding target frame as in the example of FIG. 27, the motion vector is encoded. Although it is not, the determination of the identity of the motion vectors is to determine whether or not the angles of motion are the same. In FIG. 28, F3 is an encoding target frame, and the macroblocks 40 and 41 in the frame F3 perform motion compensation prediction using the motion vectors 44 and 45 with the immediately preceding encoding target frame F2 as a reference frame. In the macroblock 42 in the same position as the macroblock 40 in the encoding target frame F2 immediately before the frame F1, the frame F0 two frames before the frame F2 is used as a reference frame, and motion compensation prediction using the motion vector 46 is performed. To do.

  The motion vector 46 and the motion vector 44 indicate the same angle, and the distance from the reference frame is twice different, so the size of the vector is doubled. Therefore, since the motion vector 44 can be reproduced by scaling the motion vector 46 in accordance with the distance between frames, the motion vector 44 is a mode that uses the motion vector of the immediately preceding frame without encoding. The prediction mode information to be shown is set.

  The motion vector 45 in the macroblock 41 and the motion vector 47 in the macroblock 43 at the same position in the immediately preceding frame have the same angle, and the motion vector 45 is encoded in the same way as the macroblock 40. Not. For macroblocks for which no motion vector is encoded as described above, a motion vector obtained by scaling the motion vector at the same position in the immediately previous encoding target frame according to the interframe distance between the encoding target frame and the reference frame is used. The motion compensated prediction interframe encoding and decoding are performed.

  FIG. 29 is a diagram illustrating predictive coding of a macroblock skip and an index indicating a reference frame according to the embodiment of the present invention. In the figure, F3 is an encoding target frame, A is an encoding target macroblock, and B, C, D, and E are adjacent macroblocks that have already been encoded. Further, F0, F1, and F2 are reference frames, and one or a plurality of reference frames are selected for each macroblock, and motion compensation prediction encoding is performed. In macroblock A, prediction is performed using motion vector 50 using frame F1 as a reference frame, and in macroblocks B, C, and E, prediction using motion vectors 51, 52, and 55 is performed using frames F2, F1, and F0 as reference frames, respectively. Each is done. For the macroblock D, prediction using two reference frames F1 and F2 is performed. For encoding the motion vector 50 of the macroblock A, a prediction vector is selected from the motion vectors of the adjacent macroblocks B, C, D, and E, and a difference vector between the prediction vector and the motion vector 50 of the macroblock A is encoded. The

  The prediction vector is determined by, for example, a method of selecting a motion vector that is a median of motion vectors of adjacent macroblocks B, C, and E, or a residual signal in adjacent macroblocks B, C, D, and E The motion vector of the macroblock with the smallest size is used as a prediction vector.

  Here, the difference between the prediction vector and the motion vector of the macroblock to be encoded is 0, the reference frame of the macroblock for which the prediction vector is selected matches the reference frame of the macroblock to be encoded, and When all the prediction error signals to be encoded are 0, the macro block is skipped without encoding, and the number of macro blocks skipped successively is encoded next without skipping. Encode as header information. For example, when the prediction vector for the macroblock A becomes the motion vector 52 of the macroblock C, the reference frames in the macroblock A and the macroblock C match, and the motion vector 50 and the motion vector 52 match. If the prediction error signals of the macroblock A are all zero, the macroblock A is skipped without being encoded. At the time of decoding, a prediction vector is selected in the same manner as at the time of encoding, a prediction image is generated using the reference frame of the macroblock for which the prediction vector is selected, and the generated prediction image is skipped. Let it be a decoded image.

  When at least one condition of the macroblock skip is not satisfied, a difference vector between a prediction vector and a motion vector of a macrobook to be encoded, a prediction error signal, and an index indicating a reference frame are encoded.

  The index indicating the reference frame encodes a difference value between the reference frame index of the adjacent macroblock for which the prediction vector is selected and the reference frame index of the encoding target frame.

  When the motion vector 52 of the macroblock C is selected as the prediction vector of the macroblock A as in the above example, the difference vector between the motion vector 50 and the motion vector 52 and the prediction error signal of the macroblock A are encoded. Further, for example, according to the table of FIG. 11, the reference frame is expressed by an index (Code_number), and the difference value between the index 2 indicating the reference frame two frames before the macroblock C and the index 2 of the macroblock A, that is, 0 is encoded as a reference frame index difference value.

  FIG. 30 shows another motion vector encoding method according to the embodiment of the present invention. In FIG. 30, a frame F2 indicates an encoding target frame, and the frame F2 is a B picture that performs motion compensation prediction from temporally preceding and succeeding frames. In the macroblock 61 in the frame F2, the frame F3 is subjected to backward prediction. A reference frame is used, and a frame F1 is used as a reference frame for forward prediction. Therefore, the encoding or decoding of the frame F3 is performed prior to the encoding or decoding of F2.

  Here, in the reference frame F3 for backward prediction in the encoding target macroblock 61, attention is focused on the macroblock 60 at the same position in the frame as the encoding target macroblock 61. When motion compensated prediction based on a linear sum from two frames F0 and F1 is used for the macroblock 60, the motion vector of the macroblock 60 with respect to the reference frame F1 for forward prediction of the encoding target macroblock 61 (see FIG. Medium 62) is scaled according to the inter-frame distance and used as a motion vector for forward prediction and backward prediction of the encoding target macroblock 61.

  That is, if the interframe distance from the frame F1 to the frame F2 is R1, and the interframe distance from the frame F2 to the frame F3 is R2, the motion vector 62 multiplied by R1 / (R1 + R2) is the front of the macroblock 61. The motion vector 64 for prediction is multiplied by −R2 / (R1 + R2) to become the motion vector 65 for backward prediction of the macroblock 61.

  In the encoding target macroblock 61, the motion vector information is not encoded, and only the flag indicating that the prediction mode, that is, bidirectional prediction based on the scaling of the motion vector is performed, is encoded.

  At the time of decoding, the frame F3 is first decoded, and the motion vector of each decoded macroblock in the frame F3 is temporarily stored. Next, in the macro block in which the flag indicating the prediction mode is set in the frame F2, the motion vector of the macro block at the same position in the temporarily stored frame F3 is scaled, so that Motion vectors for prediction and backward prediction are calculated, and bidirectional prediction decoding is performed.

  FIG. 31 is another example of the bi-directional prediction shown in FIG. 30. In FIG. 31, the reference frame for forward prediction of the encoding target macroblock 71 of the frame F2 is F0, and the others are the same as FIG. . In this case, the forward and backward motion vectors of the encoding target macroblock 71 are scaled according to the interframe distance of the motion vector 73 for the frame F0 of the macroblock 70 at the same position as the encoding target macroblock 71 of the frame F3. And ask.

  That is, if the interframe distance from the frame F0 to the frame F2 is R1, the interframe distance from the frame F3 to the frame F2 is R2, and the interframe distance from the frame F0 to the frame F3 is R3, the motion vector 73 is R1 / R3. The multiplied motion vector is the forward motion vector 74 of the encoding target macroblock 71, and the motion vector 73 multiplied by -R2 / R3 is the backward motion vector 75 of the encoding target macroblock 71. 75, bi-directional predictive encoding and decoding of the encoding target macroblock 71 is performed.

  In the example of FIGS. 30 and 31, in the reference frame for backward prediction in the bidirectional prediction macroblock to be encoded, attention is paid to the macroblock at the same position in the frame as the encoding target macroblock, and the macroblock includes a plurality of forward blocks. When the reference frame is used, the forward and backward motion vectors of the encoding target macroblock are generated by scaling the motion vector for the same reference frame as the forward reference frame in the bidirectional prediction macroblock to be encoded.

  As described above, by generating a motion vector by scaling, it is possible to reduce the coding overhead for coding the motion vector and improve the coding efficiency. In addition, when there are multiple motion vectors that are the basis of scaling, it is possible to improve the prediction efficiency by selecting and scaling the motion vectors that match the forward reference frame, enabling highly efficient encoding. It can be realized.

  FIG. 32 is another example of the bidirectional prediction shown in FIGS. 30 and 31. In FIG. 32, the frame F3 is the encoding target frame, and the encoding target macroblock 81 sets the frame F4 as the backward reference frame. And bi-directional prediction using the frame F2 as a forward reference frame. Further, the macro block 80 at the same position as the macro block 81 in the frame F4 performs prediction based on the linear sum from the two front frames F0 and F1. Therefore, unlike FIGS. 30 and 31, the macroblock 80 and the encoding target macroblock 81 do not use the same forward reference frame.

  In this case, among the forward reference frames F0 and F1 of the macroblock 80, the motion vector for the frame temporally closest to the forward reference frame F2 of the encoding target macroblock 81 is scaled according to the interframe distance, thereby The forward and backward vectors of the conversion target macroblock 81 are generated. That is, if the interframe distance from the frame F2 to the frame F3 is R1, the interframe distance from the frame F4 to the frame F3 is R2, and the interframe distance from the frame F1 to the frame F4 is R3, the encoding target macroblock 81 The forward motion vector 84 is obtained by multiplying the motion vector 82 of the macroblock 80 for the frame F1 by R1 / R3, and the backward motion vector 85 of the encoding target macroblock 81 is the motion vector 82 -R2 / R3. It is obtained by doubling. The encoding target macroblock 81 performs bi-directional prediction using the motion vectors 84 and 85 obtained by scaling.

  As described above, by generating a motion vector by scaling, it is possible to reduce the coding overhead for coding the motion vector and improve the coding efficiency. Furthermore, when there are a plurality of motion vectors to be scaled and there is no motion vector that matches the forward reference frame, the motion vector for the reference frame that is temporally closest to the forward reference frame of the encoding target macroblock is determined. By selecting and scaling, it becomes possible to improve prediction efficiency and to realize highly efficient encoding.

  FIG. 33 is a diagram showing a flowchart of the moving picture coding method according to the embodiment of the present invention. FIG. 34 is a diagram for explaining weighted prediction according to the embodiment of the present invention. First, the weighted prediction regarding the embodiment will be described with reference to FIG. 34, and then the weight coefficient determination method according to FIG. 33 will be described.

  In FIG. 34, F0, F1, F2, and F3 indicate temporally continuous frames, and F3 indicates an encoding target frame. Frames F0, F1, and F2 are reference frames for the encoding target frame F3.

  Among the encoding target pixel blocks A, B, C, and D in the encoding target frame F3, in the blocks A, B, and C, reference pixel block signals with motion compensation are generated from the frames F1, F0, and F2, respectively. A predicted pixel block signal is generated by multiplying these reference pixel block signals by weighting coefficients and adding a DC offset value. Further, the difference between the prediction pixel block signal and the encoding target pixel block signal is calculated, and the difference signal is encoded together with the identification information and the motion vector information of the reference frame.

  In the block D, a reference block signal is generated from each of the frames F0 and F1 with motion compensation, and a prediction pixel block signal is generated by adding a DC offset value to a linear combination of these reference pixel block signals. . The difference signal between the encoding target pixel block signal and the predicted pixel block signal is encoded together with the identification information and motion vector information of the reference frame.

  On the other hand, at the time of decoding, the reference frame identification information and the motion vector information are decoded, and the reference pixel block signal is generated based on them. A prediction pixel block signal is generated by multiplying the generated reference pixel block signal by a weighting factor and adding a DC offset value. The encoded difference signal is decoded, and the decoded difference signal and the predicted pixel block signal are added to decode the moving image.

Generation of the prediction pixel block signal at the time of encoding and decoding is performed by the following calculation. For example, assuming that the prediction signal of the pixel block A is predA and the reference pixel block signal cut out from the frame F1 is ref [1], predA is calculated as follows.

Here, w [1] is a weighting factor for the reference pixel block, and d [1] is a DC offset value. These values are encoded as header data for each encoding frame or slice as a coefficient table. The weighting factor and the DC offset value are individually determined for a plurality of reference frames corresponding to each encoded frame. For example, in the pixel block B in FIG. 34, since the reference pixel block ref [0] is cut out from the frame F0, the prediction signal predB is expressed by the following equation.

In the pixel block D, a reference pixel block is cut out from each of the frames F0 and F1, and a signal obtained by multiplying each reference pixel block by a weighting coefficient and adding a DC offset value as shown below is predicted. A signal predD is generated.

  In this way, in this embodiment, the weighting factor and the DC offset value are determined for each reference frame.

  Next, a method for determining the weighting factor and the DC offset value at the time of encoding according to the present embodiment will be described with reference to FIG. The inter-frame prediction relationship shown in FIG. 34, that is, the case where the frame F3 is the encoding target frame and the frames F0, F1, and F2 are reference frames will be described as an example along the flowchart of FIG.

  The weighting factor and the DC offset value are independent values for a plurality of reference frames, and the table of the weighting factor and the DC offset value is encoded for each encoded frame or for each slice. For example, in the encoded frame F3 in FIG. 34, the weighting factors and DC offset values (w [0], d [0]), (w [1], d [1]), (w for the frames F0, F1, and F2. [2] and d [2]) are encoded respectively. These values may be changed for each slice in the encoded frame.

First, an average value DCcur (a magnitude of a direct current component, hereinafter referred to as a direct current component value) of pixel values for the entire frame of the encoding target frame F3 or for each slice in the frame is calculated as follows (step S10).

Here, F3 (x, y) represents the pixel value at the position of the coordinate (x, y) of the frame F3, and N represents the number of pixels in the frame or slice. Next, the size of the alternating current component (hereinafter referred to as an alternating current component value) for the entire frame of the encoding target frame F3 or for each slice in the frame is calculated by the following mathematical formula (step S11).

In the measurement of the AC component value, a standard deviation as shown below may be used, but in this case, the amount of calculation when obtaining the AC component value increases.

  As is apparent from the comparison between the equations (28) and (29), the AC component value measurement method according to the equation (28) has an effect of reducing the amount of calculation when obtaining the AC component value.

  Next, the index indicating the reference frame number is set to “ref_id”, and the ref_idx-th reference frame is the DC component value DCref [ref_idx] of the reference frame and the AC component value in the same manner as Equation (27) and Equation (28). ACref [ref_idx] is calculated (steps S13 to S14).

From the above calculation results, the DC offset value d [ref_idx] with respect to the ref_idx-th reference frame is determined as a DC component difference as follows (step S15).

The weighting factor w [ref_idx] is determined as the gain of the AC component as follows (step S16).

  The above calculation is performed for all reference frames (from ref_idx = 0 to MAX_REF_IDX) (steps S17 and S18). Here, MAX_REF_IDX indicates the number of reference frames. When all weighting factors and DC offset values are determined, they are encoded as table data for each encoding frame or slice, and weighted predictive encoding of each pixel block is performed according to the encoded weighting factors and DC offset values. The generation of the prediction pixel block signal in the encoding and decoding is calculated as in the above mathematical formulas (24) to (26).

  As described above, by generating a prediction signal using a different weighting factor and DC offset value for each reference frame and performing predictive encoding, the signal amplitude for each frame or slice varies with time, or the DC offset value It is possible to generate a prediction signal appropriately from a plurality of reference frames even for a moving image signal whose frequency fluctuates, and to realize high-efficiency and high-quality encoding with high prediction efficiency. it can.

  Next, a specific example of a method for encoding information on weighting factors and DC offset values will be described. FIG. 35, FIG. 36 and FIG. 37 show data structures related to the encoding of weight coefficient and DC offset value information.

  FIG. 35 shows a part of the header data structure of an encoded frame or slice. The maximum number of indexes “number_of_max_ref_idx” indicating a reference frame for the encoded frame or slice, information on a weight coefficient and a DC offset value are shown in FIG. The indicated data table “weighting_table ()” is encoded. “Number_of_max_ref_idx” is equivalent to MAX_REF_IDX in FIG.

  FIG. 36 shows a first example of the encoded data structure relating to the weighting factor and the DC offset data table. In this example, according to the maximum reference frame index number “number_of_max_ref_idx” sent as header data of a frame or a slice, the data of the weight coefficient and the DC offset value corresponding to each reference frame is encoded. The DC offset value d [i] for the i-th reference frame is directly encoded as an integer pixel value.

On the other hand, the weighting factor w [i] related to the i-th reference frame is generally not an integer. Therefore, the weighting factor w [i] is approximated by a rational number w ′ [i] whose denominator is a power of 2, as shown by the equation (32), and the numerator w_numerator [i] expressed as an integer and the denominator 2 Encoding is performed by dividing into a power number w_exponential_denominaotr.

The value of the numerator and the power of 2 in the denominator can be obtained, for example, by Expression (33).

In encoding and decoding, a predicted image is generated using the encoded approximate value w ′ [i]. According to the equations (32) and (33), there are the following advantages.
In the weighting factor expression of Equation (32), the denominator of the weighting factor is fixed for each encoded frame, and the numerator is different for each reference frame. This makes it possible to reduce the amount of weight coefficient data to be encoded, compared to the case where the weight coefficient for each reference frame is independently divided into a denominator and a numerator and encoded. Thus, it is possible to improve the coding efficiency.

  Further, when the denominator is a power of 2, multiplication of the reference pixel block signal by a weighting coefficient can be realized by integer multiplication and bit shift, so that processing such as floating point arithmetic and division is not necessary, and encoding and decoding hardware It is possible to reduce the scale and the amount of calculation.

The above calculation will be described more specifically. Formula (34) generalizes the prediction formulas shown in Formulas (24) and (25), and shows a prediction formula for generating a predicted image block signal for a pixel block whose reference frame number is i. Here, Pred _i is a prediction signal, ref [i] is a reference pixel block signal cut out from the i-th reference frame, and w [i] and d [i] are cut out from the i-th reference frame, respectively. The weighting factor and DC offset value for the reference pixel block are shown.

Formula (35) is a prediction formula when the weighting coefficient w [i] in Formula (34) is expressed as a rational number represented by Formula (32). Here, wn [i] indicates w_numerator [i] in Equation (32), and wed indicates w_exponential_denominator.

  In general, an effective weighting coefficient w [i] in an arbitrary fade image or the like is not an integer, and therefore, the floating-point multiplication is required in the equation (34). Furthermore, if w [i] is expressed as an arbitrary rational number, integer multiplication and division are required. On the other hand, by performing a rational number representation in which the denominator shown in Equation (32) is a power of 2, as shown in Equation (35), an integer multiplication using an integer coefficient wn [i] and an offset considering rounding off are performed. It is possible to perform a weighted prediction operation by addition, right bit shift of the wed bits, and integer addition of the DC offset value, eliminating the need for floating point multiplication.

  Moreover, a power of 2 indicating the size of the denominator is shared for each encoded frame or slice regardless of the reference frame number i, and there are a plurality of values that the reference frame number i can take for each encoded frame. Even in this case, it is possible to suppress an increase in the amount of code when the weighting coefficient is encoded.

Equation (36) shows a case where the weighting factor expression of Equation (32) is applied to the prediction based on the linear sum from the two reference frames indicated by the equation, similarly to Equation (35).

  Even in the prediction based on the linear sum from the two reference frames described above, since the weighting factor is not generally an integer, Equation (26) requires two floating-point multiplications, but Equation (36) requires integer multiplication. Only with bit shift and integer addition, it is possible to generate a prediction signal based on a linear sum from two reference frames. Also, information wed relating to the size of the denominator is shared, and it is possible to suppress an increase in the amount of code when the weighting coefficient is encoded.

  Further, according to Equation (36), the numerator of the weighting coefficient is expressed by 8 bits. For example, when the pixel signal value is expressed by 8 bits, encoding and decoding are performed with a calculation accuracy fixed to 16 bits. Can be performed.

  Furthermore, in the same encoded frame, the denominator, that is, the shift amount is fixed regardless of the reference frame, so that it is not necessary to change the shift amount even when the reference frame is switched for each pixel block in encoding or decoding. It is possible to reduce the calculation amount or the hardware scale.

On the other hand, the weighting factor for all reference frames is

When the above condition is satisfied, the denominator and numerator of the weighting coefficient to be encoded calculated by Expression (36) may be converted as follows.

  Equation (38) has the effect of dividing each weighting factor expressed as a rational number into an irreducible fraction. By converting and encoding in this way, it is possible to reduce the dynamic range of the encoded data of the weighting coefficient without reducing the accuracy of the weighting coefficient, and further reduce the amount of code for encoding the weighting coefficient. It becomes possible to do.

FIG. 37 shows a second example of the encoded data structure relating to the weighting factor and the DC offset data table. In the embodiment shown in FIG. 37, the encoding of the DC offset value is the same as that of the embodiment of FIG. 36, but the encoding of the weighting coefficient is different from the embodiment shown in FIG. Encoding of the powers of 2 is not performed, and only the numerator of the weight coefficient expressed as a rational number with the denominator as a fixed value is encoded. In the embodiment of FIG. 37, for example, a weighting factor is expressed as a rational number as follows, and only the numerator w_numerator [i] is encoded.

  In the present embodiment, since the power of 2 representing the denominator of the weighting coefficient is fixed, it is not necessary to encode information on the power of the denominator for each encoded frame, and the code for encoding the weighting coefficient table The amount can be further reduced.

  In addition, when a rational number is expressed under a fixed numerator (“16” in the above example), by clipping the value of the numerator to 8 bits, for example, when a pixel signal is expressed in 8 bits, It becomes possible to perform encoding and decoding with a calculation accuracy fixed to 16 bits.

  Furthermore, in this embodiment, since the shift amount relating to the multiplication of the weight coefficient is fixed, it is not necessary to implement a function for loading the shift amount for each frame in encoding and decoding, and an encoding or decoding device or software Hardware mounting cost and hardware scale can be reduced.

  FIG. 38 schematically shows the structure of the entire time series of moving image encoded data including the data structure shown in FIGS. At the beginning of the encoded data, information on a plurality of encoding parameters that are invariant throughout the entire encoding sequence, such as the image size, is encoded as a sequence header (SH). Next, each image frame or field is encoded as a picture, and each picture is sequentially encoded as a set of a picture header (PH) and picture data (Picture data).

  In the picture header (PH), the maximum number of indexes “number_of_max_ref_idx” indicating the reference frame shown in FIG. 35 and the table “weighting_table ()” of the weighting coefficient and DC offset data are encoded as MRI and WT, respectively. Yes. In “weighting_table ()” (WT), as shown in FIG. 36, a power of 2 indicating the common denominator of each weighting factor w_exponential_denominaotr is encoded as WED, followed by the magnitude of the numerator of each weighting factor. A set of w_numerator [i] and DC offset value d [i] indicating the above is encoded as WN and D, respectively.

  The combination of the numerator of the weighting factor and the direct current offset value is encoded based on the number of “number_of_max_ref_idx” (MRI) included in the picture header. Each picture data (Picture data) is divided into one or a plurality of slices (SLC) and sequentially encoded for each slice. In each slice, first, encoding parameters for each pixel block in the slice are encoded as a slice header (SH), and one or a plurality of macroblock data (MB) are sequentially encoded following the slice header.

  In the macroblock data, information relating to the encoding of each pixel in the macroblock such as prediction mode information (MBT) and motion vector information (MV) of the pixel block in the macroblock is encoded, and finally the pixel to be encoded An orthogonal transform coefficient (DCT) encoded by performing orthogonal transform (for example, discrete cosine transform) on the signal or the prediction error signal is included. Here, a configuration may be adopted in which “number_of_max_ref_idx” (MRI) and “weighting_table ()” (WT) included in the picture header are encoded in the slice header (SH).

  In the case of the configuration of the weighting coefficient table data shown in FIG. 37, the encoding of data indicating the size of the denominator of the weighting coefficient can be omitted, so that the WED encoding in FIG. 38 can be omitted.

  FIG. 39 is a flowchart showing a moving picture decoding procedure according to the embodiment of the present invention. Here, a description will be given of a procedure in which encoded data encoded by the moving image encoding apparatus according to the embodiment described with reference to FIG. 33 is input and decoding is performed.

  The encoded frame or slice header data including the weight coefficient and DC offset data tables described in FIGS. 35 to 37 is decoded from the input encoded data (step S30), and the reference is made for each encoded block. The header data of the encoded block including the reference frame index for identifying the frame is decoded (step S31).

  For each pixel block, a reference pixel block signal is cut out from the reference frame indicated by the reference frame index (step S32). Based on the reference frame index of the coding block, the weighting factor and the DC offset value are determined by referring to the table of the decoded weighting factor and DC offset data.

  A prediction pixel block signal is generated from the reference pixel block signal using the weighting coefficient and the DC offset value thus determined (step S33), the encoded prediction error signal is decoded, and the decoded prediction error is calculated. The decoded image is generated by adding the signal and the predicted pixel block signal (step S34).

  Each encoded pixel block is sequentially decoded, and when all the pixel blocks in the encoded frame or slice are decoded, the next picture header or slice header is continuously decoded.

  By the encoding method and the decoding method in the above-described procedure, an appropriate prediction image can be generated at the time of encoding and decoding even for a moving image signal having a time variation of the signal amplitude and a time variation of the DC offset value. It is possible to perform high-efficiency, high-quality video encoding and decoding with high prediction efficiency.

Next, preferred embodiments of the present invention disclosed in the above-described embodiment are listed below.
(1) A motion compensation prediction frame between a plurality of reference frames in a predetermined combination and a motion vector between the coding target macroblock and at least one reference frame with respect to a coding target macroblock of a moving image In the moving image encoding method for performing encoding, (a) at least one reference macroblock is extracted from each of the plurality of reference frames, and (b) a set of predetermined weight coefficients is used for the plurality of extracted reference macroblocks. (C) generating a prediction error signal between the prediction macroblock and the encoding target macroblock, and combining the prediction error signal and the plurality of reference frames. A first index indicating, a second index indicating the set of weighting factors, and information on the motion vector Encode.
<Effect> As described above, by predicting by linear sum of a plurality of reference frames and changing the coefficient of the linear sum to be variable, it is possible to perform appropriate prediction with respect to temporal signal intensity changes such as fading. Thus, it is possible to improve the prediction efficiency in encoding. In addition, by appropriately selecting a frame to be referred to, it is possible to improve prediction efficiency by selecting an appropriate reference frame in a portion where occlusion (visible / hidden) occurs temporally. By encoding the combination of the linear prediction coefficient and the reference frame as an index, the encoding overhead can be suppressed.

(2) In (1), an index indicating a set of weighting factors of the linear sum is encoded as header data for each frame or a plurality of frames, and the prediction error signal, an index indicating a combination of the plurality of reference frames, and A motion vector is encoded for each macroblock.
<Effect> In general, a temporal change in signal strength such as fading occurs uniformly over the entire frame, and occlusion occurs locally within the frame. According to (2), one set of linear prediction coefficients corresponding to temporal changes in signal strength is encoded for each frame, and the index indicating the combination of reference frames is variable for each macroblock, thereby improving the encoding efficiency. Thus, it is possible to achieve both improvement in encoding and reduction in encoding overhead, and it is possible to improve encoding efficiency including overhead.

(3) In (1) or (2), the encoded motion vector is a motion vector related to one specific reference frame among the plurality of reference frames.
<Effect> When motion compensation predictive coding is performed using a plurality of reference frames for each macroblock, if a motion vector for each reference frame is individually coded, an increase in coding overhead is caused. However, according to (3), a motion vector for a specific reference frame is sent, and a motion vector for another frame is scaled according to the interframe distance between the frame to be encoded and each reference frame. By using a motion vector, it is possible to prevent an increase in encoding overhead and improve encoding efficiency.

(4) In (3), the motion vector related to the one specific reference frame is a motion vector normalized according to an interframe distance between the reference frame and a frame to be encoded.
<Effect> In this way, the motion vector to be encoded is a motion vector normalized by the distance between unit frames, so that motion vector scaling for an arbitrary reference frame is generated at low cost by multiplication or shift operation and addition processing. It becomes possible to do. Assuming uniform motion in time, normalizing with the distance between unit frames minimizes the size of the motion vector to be encoded, reducing the amount of motion vector information. The effect of reducing the overhead can be obtained.

(5) In (3), the motion vector relating to the one specific reference frame is a motion vector for the reference frame having the longest interframe distance from the encoded frame among the plurality of reference frames.
<Effect> According to (3), it is possible to reduce the amount of motion vector coding and realize motion vector scaling at a low cost. On the other hand, as the interframe distance between the reference frame and the encoding target frame increases, the motion increases. A reduction in compensation accuracy occurs. On the other hand, according to (5), the motion vector for the reference frame with the longest interframe distance is encoded among the plurality of reference frames, and the motion vector for the other reference frame is the encoded motion vector between the frames. According to the distance, it can be generated by being divided internally, and a decrease in accuracy of motion compensation for each reference frame can be suppressed. As a result, prediction efficiency is further improved, and highly efficient encoding can be performed.

(6) In (1) or (2), the encoded motion vector includes a first motion vector related to one specific reference frame among the plurality of reference frames and one or more other references. A motion vector for a frame, and a motion vector for one or more other reference frames is scaled according to an interframe distance between the frame to be encoded and the one or more reference frames. The motion vector is encoded as a difference vector between the motion vector and the one or more motion vectors.
<Effect> When the temporal change of local video can be approximated by parallel movement, prediction from multiple reference frames can be performed with one motion vector and a motion vector scaled according to the distance between frames. It is. However, if it does not change at a constant speed in time, appropriate motion compensation is difficult only by scaling. According to (6), by encoding a motion vector for a plurality of reference frames as one representative vector and a difference vector between a scaled motion vector and an optimal motion vector for each reference frame, a plurality of motions are encoded. Compared with the case of encoding a vector, it is possible to reduce the amount of coding of a motion vector, and it is possible to achieve both reduction in encoding overhead and improvement in prediction efficiency.

  (7) In (6), the first motion vector is a motion vector normalized according to an interframe distance between the reference frame and a frame to be encoded.

  (8) In (6), the first motion vector is a motion vector for a reference frame having the longest interframe distance from a frame to be encoded among the plurality of reference frames.

(9) In any one of (1) to (8), an index indicating a combination of the plurality of reference frames is a predetermined value, and all elements of the motion vector to be encoded are 0. In the macroblock in which the prediction error signals to be encoded are all 0, encoding is skipped without outputting any encoded data, and the number of skipped macroblocks in the macroblock to be encoded next is Encode.
<Effect> As a condition for skipping a macroblock, if the above conditions are matched on the transmission / reception side, an index indicating a combination of reference frames, which is encoding information for each macroblock, a motion vector of size 0, It is possible to reproduce video on the receiving side without encoding and sending a zero residual signal, and it is possible to reduce the amount of these encoded data and improve the encoding efficiency. Furthermore, by encoding the prediction coefficient according to the time change of the signal strength for each frame, adaptive macroblock skip can be realized without increasing the coding overhead according to the nature of the video signal. It becomes possible.

(10) A macroblock in which an index indicating a combination of the plurality of reference frames in any one of (1) to (8) is a predetermined value, and the motion vector to be encoded is encoded immediately before In a macroblock that matches the motion vector of the above and the prediction error signal to be encoded is all 0, the encoding is skipped without outputting any encoded data, and the next macroblock to be encoded is skipped. The number of macroblocks that have been processed is encoded.
<Effect> When a region larger than the macroblock in the frame moves in parallel in time, it can be encoded as a skip macroblock without sending motion vector information, and the encoding overhead is reduced. Efficiency can be improved.

(11) In (9) or (10), the index indicating the predetermined combination of reference frames indicates that two frames encoded immediately before are used as reference frames.
<Effect> By using two frames encoded immediately before as a reference image as a macroblock skip condition, even when the signal strength changes with time due to fading, linear extrapolation, etc. Prediction can easily generate an accurate prediction image, and it is possible to skip encoding of macroblocks even though the signal strength changes with time. Due to the effect, the encoding efficiency can be further improved.

(12) In (9) or (10), the index indicating the predetermined reference frame combination can be changed for each encoding target frame, and the index indicating the predetermined reference frame combination is encoded. It is encoded as header data of the target frame.
<Effect> It is possible to flexibly change the macroblock skip condition according to the time change of the video signal. By changing the skip condition appropriately for each frame in accordance with the video so that macroblock skip is likely to occur at the time of encoding, it is possible to reduce encoding overhead and realize highly efficient encoding.

(13) In any one of (1) to (8), an index indicating a combination of the plurality of reference frames is the same as the macroblock encoded immediately before, and all elements of the motion vector to be encoded Is a macroblock in which the prediction error signal to be encoded is all 0, the encoding is skipped without outputting any encoded data, and the macroblock skipped in the next encoded macroblock Encode the number of blocks.
<Effect> By using the same reference frame combination as that of the immediately preceding macroblock as a macroblock skip condition, the macro can be efficiently used by utilizing the correlation of the spatio-temporal characteristics of neighboring regions in the moving image signal. It is possible to improve the coding efficiency by skipping blocks.

(14) In any one of (1) to (8), an index indicating a combination of the plurality of reference frames is the same as the macroblock encoded immediately before, and the motion vector to be encoded is In the macroblock that matches the motion vector of the encoded macroblock and the prediction error signal to be encoded is all 0, the encoding is skipped without outputting any encoded data, and then encoded. The number of skipped macroblocks is encoded.
<Effect> By having the configuration of (14) in addition to (13), it is possible to further reduce the coding overhead and improve the coding efficiency.

(15) In any one of (1) to (8), the motion vector to be encoded is predicted from the motion vectors of one or more macroblocks adjacent in a frame, and the motion vector to be encoded and the A difference vector from the predicted motion vector is encoded.
<Effect> In addition to improving the encoding efficiency by (1) to (8), focusing on the spatial correlation of motion vectors, the motion vector to be encoded is predicted from adjacent macroblocks in the frame, and the difference By encoding only the vector, it is possible to reduce the encoding overhead of the motion vector and further improve the encoding efficiency.

(16) In any one of (1) to (8), the motion vector to be encoded is predicted from the motion vector in the macroblock at the same position in the frame encoded immediately before, and the motion vector to be encoded And a difference vector between the predicted motion vector and the predicted motion vector.
<Effect> Focusing on temporal correlation of motion vectors, predicting the motion vector to be encoded from the motion vector of the macroblock at the same position in the frame encoded immediately before, and encoding only the difference vector Thus, it is possible to reduce the coding overhead of the motion vector and further improve the coding efficiency.

(17) In any one of (1) to (8), the motion vector to be encoded is the motion vector of one or more macroblocks adjacent in the frame and the same in the frame encoded immediately before. The motion vector in the macroblock at the position is predicted, and the difference vector between the motion vector to be encoded and the predicted motion vector is encoded.
<Effect> Focusing on the spatio-temporal correlation of motion vectors, by performing motion vector prediction within a frame and between frames, it becomes possible to combine both features (15) and (16), and more motion vectors The encoding efficiency can be improved.

(18) In any one of (15) to (17), an index indicating a combination of the plurality of reference frames is a predetermined value, and a difference vector of the motion vector to be encoded is 0, and In the macroblock in which the prediction error signals to be encoded are all 0, encoding is skipped without outputting any encoded data, and the number of skipped macroblocks is encoded in the next macroblock to be encoded To do.
<Effect> Due to a synergistic effect with any of the configurations (15) to (17), it is possible to further reduce the coding overhead and improve the coding efficiency.

(19) In any one of (15) to (17), an index indicating a combination of the plurality of reference frames is a predetermined value, and the difference vector of the motion vector to be encoded is encoded immediately before. In a macroblock that matches the difference vector of the generated macroblock and whose prediction error signal to be encoded is all 0, the encoding is skipped without outputting any encoded data, and the next encoded macro In the block, the number of skipped macroblocks is encoded.
<Effect> Due to the synergistic effect of any of the configurations (15) to (17) and the configuration (10), it is possible to further reduce the coding overhead and improve the coding efficiency.

(20) In any one of (18) to (19), it is indicated that an index indicating the predetermined reference frame combination is used as a reference frame immediately before two frames encoded.
<Effect> Due to the synergistic effect of the configurations (18) to (19) and the configuration (11), it is possible to further reduce the coding overhead and improve the coding efficiency.

(21) In any one of (18) to (19), an index indicating the predetermined combination of reference frames can be changed for each encoding target frame, and the index indicating the predetermined combination of reference frames Are encoded as header data of the encoding target frame.
<Effect> Due to the synergistic effect of the configurations (18) to (19) and the configuration (12), it is possible to further reduce the coding overhead and improve the coding efficiency.

(22) In any one of (15) to (17), the index indicating the combination of the plurality of reference frames is the same as the macroblock encoded immediately before, and the difference vector of the motion vector to be encoded In a macroblock in which all elements are 0 and the prediction error signal to be encoded is all 0, encoding is skipped without outputting any encoded data, and in a macroblock to be encoded next, The number of skipped macroblocks is encoded.
<Effect> The synergistic effect of any of the configurations (15) to (17) and the configuration (13) makes it possible to further reduce the coding overhead and improve the coding efficiency.

(23) In any one of (15) to (17), an index indicating a combination of the plurality of reference frames is the same as the macroblock encoded immediately before, and a difference vector of the motion vector to be encoded is In the macroblock that matches the difference vector of the macroblock coded immediately before and the prediction error signal to be coded is all 0, the coding is skipped without outputting any coded data, In the macroblock to be encoded, the number of skipped macroblocks is encoded.
<Effect> Due to the synergistic effect of any of the configurations (15) to (17) and the configuration (14), it is possible to further reduce the coding overhead and improve the coding efficiency.

(24) In any one of (1) to (2), the set of weighting factors of the linear sum is determined according to the interframe distance between the encoding target frame and the plurality of reference frames.
<Effect> For linear temporal fluctuations in signal strength such as fading, appropriate prediction images can be easily obtained by linear interpolation or linear extrapolation prediction according to the interframe distance between the frame to be encoded and multiple reference frames Therefore, it is possible to generate at a low cost and to realize highly efficient encoding with high prediction efficiency.

(25) In any one of (1) to (2), an average DC value in a frame or a field in an input moving image signal is calculated, and based on the DC values in a plurality of reference frames and encoding target frames Then, a set of weighting factors of the linear sum is determined.
<Effect> By calculating the linear prediction coefficient from the temporal change of the DC value in the encoding target frame and the frames of the plurality of reference frames, not only when the temporal change of the signal strength is constant, but also of arbitrary signal strength It is possible to generate an appropriate prediction image with respect to temporal variation, and it is possible to realize highly efficient encoding with higher prediction efficiency.

(26) The encoder according to any one of (1) to (2), wherein the input moving image signal has a variable frame rate, or an arbitrary frame of the input moving image signal is thinned to a variable frame rate. When the moving image signal having the variable frame rate is encoded, the set of weight coefficients of the linear sum is determined according to the change in the interframe distance between the encoding target frame and the plurality of reference frames.
<Effect> For encoding with variable frame rate in which the interframe distance between the encoding target frame and the plurality of reference frames dynamically changes, by using an appropriate linear prediction coefficient corresponding to the interframe distance, It is possible to perform highly efficient encoding while maintaining high prediction efficiency.

(27) A motion compensation prediction frame between a plurality of reference frames in a predetermined combination and a motion vector between the encoding target macroblock and at least one reference frame with respect to a coding target macroblock of a moving image. In the moving image encoding method for performing encoding, (a) extracting a first reference macroblock corresponding to the motion vector candidate from the first reference frame, and (b) extracting at least one second motion vector candidate from the first reference frame. (C) extracting at least one second reference macroblock corresponding to the scaled motion vector candidate from the second reference frame; and (c) extracting the second reference macroblock corresponding to the scaled motion vector candidate from the second reference frame. d) calculating a linear sum using a predetermined set of weighting factors for the first and second reference macroblocks; (E) generating a prediction error signal between the prediction macroblock and the encoding target macroblock, and (f) a linear sum of the first and second reference macroblocks and the encoding target. The motion vector is determined based on the magnitude of a prediction error signal with a macroblock, and (g) a first index indicating the prediction error signal, a first index indicating the first and second reference frames, and a set of the weighting coefficients. Two indexes and information of the determined motion vector are encoded.
<Effect> When a plurality of reference macroblocks are extracted from a plurality of reference frames for one encoding target macroblock and a prediction macroblock is generated from the linear sum, an optimal motion is individually performed for each reference frame. If the vector is determined, the calculation amount becomes enormous. According to the configuration of (27), the motion vector candidates for the first reference frame are scaled to become motion vector candidates for the other reference frames, thereby searching for an optimal plurality of motion vectors with a very small amount of calculation. Thus, the encoding cost can be greatly reduced.

(28) In (27), the determined motion vector is scaled according to the distance between each reference frame and the encoding target frame so that the prediction error signal is reduced in the vicinity of the scaled motion vector. The reference macroblock corresponding to at least one reference frame is individually re-searched, and motion compensation prediction is performed using the motion vector obtained as a result of the re-search.
<Effects> By re-searching motion vectors around the scaled motion vector candidates, it is possible to achieve a more accurate motion vector search with a small amount of computation, and accuracy with a slight increase in the amount of computation. It is possible to realize highly efficient motion compensation prediction and perform highly efficient encoding.

(29) With respect to a macro block to be encoded of a moving image, at least one reference frame in the past in time and a motion vector between the encoding target macro block and the reference frame are used to inter-motion compensated prediction frames. In the moving image encoding method for performing encoding, a decoding target macroblock having the same intra-frame position as the encoding target macroblock in the encoded frame immediately before the encoding target frame including the encoding target macroblock The motion compensated prediction interframe coding is performed by switching for each coding target macroblock whether the motion vector is used or whether the motion vector is newly determined and coded.
<Effect> As described above, in motion compensation predictive coding, overhead for motion vector coding affects coding efficiency. In particular, when encoding video with high prediction efficiency, or when encoding with a large macroblock size and a large number of motion vectors to be encoded, the amount of motion vector coding may be dominant. According to the configuration of (29), the motion vector of the macro block at the same position as the encoding target macro block of the immediately preceding frame is used using the temporal correlation of the motion of the video, and the motion vector is used for the macro block that can be used as it is. In addition, the motion vector coding overhead is reduced by encoding new motion vectors only for macroblocks whose prediction efficiency decreases when the motion vector of the previous frame is used. Encoding can be realized.

(30) A moving image in which motion compensation prediction interframe coding is performed on at least one reference frame and a motion vector between the coding target macroblock and the reference frame with respect to a coding target macroblock of a moving image. In an image encoding method, (a) a first prediction mode in which at least one encoded frame in the past in time is used as the reference frame, and (b) an encoded frame in the future in time is used as the reference frame. A second prediction mode, (c) a third prediction mode in which a linear sum of the temporally past and future encoded frames is used as the reference frame, and (d) a plurality of temporally past encoded frames. The motion compensation prediction interframe coding is performed by switching the fourth prediction mode using the linear sum of the reference frames as the reference frame for each of the encoding target macroblocks.
<Effect> In a B picture (bidirectional predictive coding) used in MPEG2 video coding, any one of prediction from one front frame, prediction from one rear frame, and average prediction from preceding and following frames is a macro clock unit. Can be switched to. In the average prediction, the averaging process functions as a loop filter, which has the effect of improving the prediction efficiency by removing the noise of the original picture and the coding noise in the reference image. However, bi-directional prediction is difficult before and after the scene change, and prediction is performed from one frame forward or backward, and the loop filter effect does not act, leading to a decrease in prediction efficiency. According to the configuration of (30), even in the prediction only in the forward direction, the prediction image is generated by the linear sum from the plurality of reference frames, so that it is possible to improve the prediction efficiency due to the loop filter effect.

(31) In (30), the prediction based on the linear sum is linear interpolation and linear extrapolation according to the interframe distance.
<Effect> Even when the signal intensity changes with time due to fade or the like, it is possible to easily generate an appropriate prediction image by linear interpolation or linear extrapolation prediction from a plurality of frames, and to obtain high prediction efficiency. Is possible.

(32) With respect to a decoding target macroblock of a moving image, a plurality of reference frames in a predetermined combination and a motion vector between the decoding target macroblock and at least one reference frame are used for motion compensation prediction frames In the moving picture decoding method for decoding, a set of (a) a prediction error signal for each decoding target macroblock, a first index indicating a combination of the plurality of reference frames, and a weight coefficient of a linear sum for the reference macroblock (B) decoding a plurality of reference macroblocks from the plurality of reference frames according to the decoded motion vector and the information of the first index. (C) the weight indicated by the decoded second index information. A prediction macroblock is generated by calculating a linear sum of the extracted plurality of reference macroblocks using a set of only coefficients, and (d) for each decoding target macroblock decoded with the prediction macroblock The video signal is decoded by adding the prediction error signals.
<Effect> It is possible to decode the encoded data encoded by (1), and has the same encoding efficiency improvement effect as (1).

(33) In (32), an index indicating the set of weighting factors of the linear sum is received as header data for each frame or a plurality of frames, and the index indicating a combination of the prediction error signal and the plurality of reference frames , And the motion vector are received and decoded for each macroblock.
<Effect> It is possible to decode the encoded data encoded by (2), and has the same encoding efficiency improvement effect as (2).

(34) In (32) or (33), the received motion vector is a motion vector related to a specific reference frame among the plurality of reference frames, and the received motion vector is referred to as a decoded frame. Scaling is performed according to the distance between frames, and a motion vector for one or more other reference frames is generated using the scaled motion vector.
<Effect> It is possible to decode the encoded data encoded by (3), and has the same encoding efficiency improvement effect as (3).

(35) In (34), the motion vector related to the one specific reference frame is a motion vector normalized according to an interframe distance between the reference frame and a frame to be encoded.
<Effect> It is possible to decode the encoded data encoded by (4), and has the same effect of improving the encoding efficiency as (4).

(36) In (34), the motion vector related to the one specific reference frame is a motion vector for the reference frame having the longest interframe distance from the encoded frame among the plurality of reference frames.
<Effect> It is possible to decode the encoded data encoded by (5), and has the same encoding efficiency improvement effect as (5).

(37) In (32) or (33), the received motion vector corresponds to a first motion vector related to a specific one reference frame and the other one or more reference frames among the plurality of reference frames. A difference vector, wherein the first motion vector is scaled according to an inter-frame distance between a frame to be encoded and the one or more reference frames, and the scaled motion vector and the received one or By adding difference vectors for a plurality of reference frames, motion vectors for the other one or more reference frames are generated.
<Effect> It is possible to decode the encoded data encoded by (6), and has the same effect of improving the encoding efficiency as (6).

(38) In (37), the received first motion vector is a motion vector normalized according to an interframe distance between the reference frame and a frame to be encoded.
<Effect> It is possible to decode the encoded data encoded by (7), and has the same encoding efficiency improvement effect as (7).

(39) In (37), the received first motion vector is a motion vector for a reference frame having the longest interframe distance from a frame to be encoded among the plurality of reference frames.
<Effect> It is possible to decode the encoded data encoded by (8), and has the same encoding efficiency improvement effect as (8).

(40) In any one of (32) to (39), when information on the number of macroblocks skipped for each macroblock is received and one or more macroblocks are skipped, each skipped macro Assuming that all motion vector elements necessary for decoding a macroblock are zero for a block, a reference macroblock is obtained from the plurality of reference frames using a predetermined combination of reference frames. Extraction is performed, a prediction macroblock is generated from the plurality of reference macroblocks by a linear sum based on an index indicating a set of received weight coefficients of the linear sum, and the prediction macroblock is used as a decoded image.
<Effect> It is possible to decode the encoded data encoded by (9), and has the same encoding efficiency improvement effect as (9).

(41) In any one of (32) to (39), when information on the number of skipped macroblocks is received for each macroblock, and one or more macroblocks are skipped, each skipped macro For a block, a reference macroblock is extracted from the plurality of reference frames using a motion vector of a macroblock encoded without skipping immediately before and a combination of a plurality of predetermined reference frames. A prediction macroblock is generated from a reference macroblock by a linear sum based on an index indicating the received set of linear sum weighting factors, and the prediction macroblock is used as a decoded image.
<Effect> It is possible to decode the encoded data encoded by (10), and has the same effect of improving the encoding efficiency as (10).

(42) In (40) or (41), the predetermined reference frame combination is two frames decoded immediately before.
<Effect> It is possible to decode the encoded data encoded by (11), and has the same encoding efficiency improvement effect as (11).

(43) In (40) or (41), an index indicating a predetermined combination of reference frames is received as header data of an encoded frame, and decoding of a skipped macroblock is performed according to the received index. Do.
<Effect> It is possible to decode the encoded data encoded by (12), and has the same encoding efficiency improvement effect as (12).

(44) In any one of (32) to (39), when information on the number of skipped macroblocks is received for each macroblock, and one or more macroblocks are skipped, each skipped macro For a block, assuming that all the motion vector elements necessary for decoding the macroblock are 0, use an index indicating a combination of a plurality of reference frames in the immediately preceding macroblock encoded without skipping. A reference macroblock is extracted from the plurality of reference frames, a prediction macroblock is generated from the plurality of reference macroblocks by a linear sum based on an index indicating a set of weight coefficients of the received linear sum, and the prediction macro A block is used as a decoded image.
<Effect> It is possible to decode the encoded data encoded by (13), and has the same effect of improving encoding efficiency as (13).

(45) In any one of (32) to (39), when information on the number of skipped macroblocks is received for each macroblock, and one or more macroblocks are skipped, each skipped macro For a block, the plurality of references using a motion vector of a macroblock encoded without skipping immediately before and an index indicating a combination of a plurality of reference frames in the macroblock encoded without skipping immediately before A reference macroblock is extracted from the frame, a prediction macroblock is generated from the plurality of reference macroblocks by a linear sum based on an index indicating a set of weighting factors of the received linear sum, and the prediction macroblock is decoded Used as an image.
<Effect> It is possible to decode the encoded data encoded by (14), and has the same effect of improving the encoding efficiency as (14).

(46) In any one of (32) to (39), the received motion vector is encoded as a difference vector with a motion vector predicted from motion vectors of one or a plurality of macroblocks adjacent in the frame. The motion vector of the corresponding macroblock is decoded by generating a motion vector predictor from the decoded motion vectors of the plurality of adjacent macroblocks and adding the motion vector to the received motion vector.
<Effect> It is possible to decode the encoded data encoded by (15), and has the same encoding efficiency improvement effect as (15).

(47) In any one of (32) to (39), the received motion vector is encoded as a difference motion vector with a motion vector predicted from a motion vector in a macroblock at the same position in the immediately preceding frame. In addition, the motion vector predicted from the decoded motion vector in the macroblock at the same position in the frame decoded immediately before is added to the received motion vector, thereby decoding the motion vector of the corresponding macroblock. This is a 47th feature.
<Effect> It is possible to decode the encoded data encoded by (16), and has the same effect of improving the encoding efficiency as (16).

(48) In any one of (32) to (39), the received motion vector is obtained from motion vectors in one or more adjacent macroblocks in a frame and macroblocks at the same position in the immediately preceding frame. It is encoded as a difference motion vector with the predicted motion vector, and the decoded motion vector in the adjacent macroblock and the decoded motion in the macroblock at the same position in the frame decoded immediately before A motion vector of a corresponding macroblock is decoded by generating a motion vector predictor from the vector and adding the motion vector predictor and the received motion vector.
<Effect> It is possible to decode the encoded data encoded by (17), and has the same encoding efficiency improvement effect as (17).

(49) In any one of (46) to (48), when information on the number of macroblocks skipped for each macroblock is received and one or more macroblocks are skipped, each skipped macro For a block, a reference macroblock is extracted from the plurality of reference frames using a combination of a plurality of predetermined reference frames as a motion vector of a macroblock in which the predicted motion vector is skipped, and the plurality of reference macroblocks Then, a predicted macroblock is generated by a linear sum based on the received index indicating the set of linear sum weighting factors, and the predicted macroblock is used as a decoded image.
<Effect> It is possible to decode the encoded data encoded by (18), and has the same encoding efficiency improvement effect as (18).

(50) In any one of (46) to (48), when information on the number of macroblocks skipped for each macroblock is received and one or more macroblocks are skipped, each skipped macro For a block, using a combination of a motion vector obtained by adding a motion vector of a macroblock encoded without skipping immediately before to the predicted motion vector and a plurality of reference frames determined in advance, from the plurality of reference frames A reference macroblock is extracted, a prediction macroblock is generated from the plurality of reference macroblocks by a linear sum based on an index indicating a set of the received linear sum weight coefficients, and the prediction macroblock is used as a decoded image. Use.
<Effect> It is possible to decode the encoded data encoded by (19), and has the same encoding efficiency improvement effect as (19).

(51) In (49) or (50), the predetermined reference frame combination is two frames decoded immediately before.
<Effect> It is possible to decode the encoded data encoded by (20), and has the same encoding efficiency improvement effect as (20).

(52) In (49) or (50), an index indicating the predetermined reference frame combination is received as header data of an encoded frame, and the skipped macroblock is decoded in accordance with the received index. Do.
<Effect> It is possible to decode the encoded data encoded by (21), and has the same encoding efficiency improvement effect as (21).

(53) In any one of (46) to (48), when information on the number of skipped macroblocks is received for each macroblock and one or more macroblocks are skipped, each skipped macro For a block, a reference macro from the plurality of reference frames is used by using an index indicating a combination of a plurality of reference frames in a macroblock coded without skipping as a motion vector of a macroblock in which the predicted motion vector is skipped. A block is extracted, a prediction macroblock is generated from the plurality of reference macroblocks by a linear sum based on an index indicating a set of weight coefficients of the received linear sum, and the prediction macroblock is used as a decoded image.
<Effect> It is possible to decode the encoded data encoded by (22), and has the same encoding efficiency improvement effect as (22).

(54) In any one of (46) to (48), when information on the number of macroblocks skipped for each macroblock is received and one or more macroblocks are skipped, each skipped macro For the block, a motion vector is generated by adding a difference motion vector of the macroblock encoded without skipping immediately before to the predicted motion vector, and a plurality of macroblocks encoded without skipping immediately before are encoded. A reference macroblock is extracted from the plurality of reference frames using an index indicating a combination of reference frames, and a linear sum based on an index indicating a set of received weight coefficients of the linear sum is extracted from the plurality of reference macroblocks. Generate a prediction macroblock and decode the prediction macroblock Used as a digitized image.
<Effect> It is possible to decode the encoded data encoded by (23), and has the same encoding efficiency improvement effect as (23).

(55) Between motion compensation prediction frames using a plurality of reference frames in a predetermined combination and a motion vector between the decoding target macroblock and at least one reference frame for a decoding target macroblock of a moving image In the moving picture decoding method for decoding, (a) a prediction error signal for each decoding target macroblock, a first index indicating a combination of the plurality of reference frames, and a second index indicating a frame number of the encoded frame Decoding encoded data including index and motion vector information; and (b) extracting a plurality of reference macroblocks from the plurality of reference frames according to the decoded motion vector and the first index information. c) The plurality of reference frames according to the decoded information of the second index and the Calculate the inter-frame distance from the encoded frame, and (d) calculate the linear sum of the extracted reference macroblocks using a set of weighting factors determined according to the calculated inter-frame distance. Thus, a prediction macroblock is generated, and (e) a moving picture signal is decoded by adding the prediction macroblock and the decoded prediction error signal.
<Effect> It is possible to decode the encoded data encoded by (24), and has the same effect of improving the encoding efficiency as (24).

(56) Motion-compensated prediction for a decoding target macroblock of a moving image using at least one reference frame in the past in time and a motion vector between the decoding target macroblock and at least one reference frame In the moving picture decoding method for performing inter-frame decoding, (a) the prediction error signal for each decoding target macroblock and the encoded first motion vector or the same position in the frame in the immediately preceding encoded frame Receiving and decoding encoded data including any information of a flag indicating that the second motion vector of the macroblock is used; and (b) for the decoding target macroblock having received the information of the first motion vector. For the decoded first motion vector and the decoding target macroblock that has received the flag, the second motion vector is used. A prediction macroblock signal is generated using each of the vectors, and (c) the moving picture signal is decoded by adding the prediction macroblock and the decoded prediction error signal.
<Effect> It is possible to decode the encoded data encoded by (29), and has the same encoding efficiency improvement effect as (29).

(57) In a moving picture decoding method for performing motion compensation prediction interframe decoding on a moving picture decoding target macroblock using a motion vector between the decoding target macroblock and at least one reference frame. (A) Prediction error signal information for each decoding target macroblock, a first prediction mode in which at least one encoding target frame in the past is the reference frame, and a future encoding target in the future A second prediction mode in which a frame is the reference frame, a third prediction mode in which a linear sum of the temporally and future encoding target frames is the reference frame, and the temporally multiple encoding targets Encoded data including prediction mode information indicating one of the fourth prediction modes using a linear sum of frames as the reference frame, and information on the motion vector And (b) generating a prediction macroblock signal using the prediction mode information and the motion vector information, and (c) adding the prediction macroblock signal and the decoded prediction error signal. Thus, the moving image signal is decoded.
<Effect> It is possible to decode the encoded data encoded by (30), and has the same encoding efficiency improvement effect as (30).

(58) In (57), the prediction based on the linear sum is linear interpolation and linear extrapolation according to the interframe distance.
<Effect> It is possible to decode the encoded data encoded by (31), and has the same encoding efficiency improvement effect as (31).

(59) At least one reference frame selected from a plurality of reference frames, and a motion vector between the encoding target macroblock and the at least one reference frame, with respect to the encoding target macroblock of the moving image In the moving picture coding method for performing motion compensation prediction interframe coding using the motion vector, the motion vector is identical to a prediction vector selected from motion vectors for a plurality of macroblocks adjacent to the coding target macroblock of the moving picture. And at least one reference frame selected for the encoding target macroblock matches a reference frame for the macroblock for which the prediction vector is selected, and the prediction to be encoded in the motion compensated prediction interframe coding For the macroblock to be encoded whose error signal is all zero Thus, the motion compensated prediction interframe coding is skipped, and the number of macroblocks for which the motion compensated prediction interframe coding is skipped at the time of motion compensated prediction interframe coding of the next encoding target macroblock is coded.
<Effect> Similar to (22), by using the correlation between the motion vector and the reference frame selection in the inter-frame prediction between adjacent macroblocks, the macroblock skip is efficiently generated and the coding overhead is reduced. It is possible to improve coding efficiency and to use a reference frame that is the same as the adjacent macroblock used in motion vector prediction as a skip condition. It is possible to generate a macro block skip more efficiently by using the correlation between adjacent macro blocks.

(60) At least one first reference frame selected from a plurality of reference frames for a coding target macroblock of a moving image, and a motion vector between the coding target macroblock and the first reference frame In the moving picture coding method for performing motion compensation prediction interframe coding using a motion compensation prediction interframe coding, a prediction error signal obtained by the motion compensation prediction interframe coding, a motion vector used for the motion compensation prediction interframe coding and the coding A difference vector between prediction vectors selected from motion vectors between a plurality of macroblocks adjacent to the target macroblock and the second reference frame, and an index indicating the first reference frame and an index indicating the second reference frame The difference value is encoded.
<Effect> Similar to (15) to (17), motion vector information is efficiently encoded using the correlation of motion vectors between adjacent macroblocks, and each macro is selected from a plurality of reference frames. A motion vector is encoded by encoding a difference value between an index indicating a reference frame in an adjacent macroblock for which a prediction vector is selected and an index indicating a reference frame in an encoding target macroblock with respect to an index related to a frame referred to by the block. It is possible to improve the coding efficiency of the index indicating the reference frame by using the correlation between adjacent macroblocks based on the combination of the frame and the reference frame, and perform highly efficient video coding by reducing the coding overhead. It becomes possible.

(61) For a decoding target macroblock of a moving image, a motion compensated prediction frame using a motion vector between the decoding target macroblock and at least one reference frame selected from a plurality of reference frames In the video decoding method for performing inter-decoding, (a) a prediction error signal for each decoding target macroblock obtained by motion compensated prediction interframe coding, the number of macroblocks skipped immediately before and the selected Encoded data including index information indicating at least one reference frame is received and decoded, and (b) one of motion vectors of a plurality of macroblocks adjacent to the skipped macroblock in the skipped macroblock. (C) a macroblock from which the prediction vector is selected. A prediction macroblock is generated according to at least one reference frame and the prediction vector, and (d) the prediction macroblock is output as a decoded image signal of the skipped macroblock.
<Effect> It becomes possible to decode the encoded data encoded by (59), and has the same encoding efficiency improvement effect as (59).

(62) For a decoding target macroblock of a moving image, motion compensation is performed using a motion vector between the decoding target macroblock and at least one first reference frame selected from a plurality of reference frames. In a video decoding method for performing inter-prediction frame decoding, (a) a prediction error signal obtained by motion-compensated prediction inter-frame coding, a motion vector used for the motion-compensated prediction inter-frame coding, and the decoding target A difference vector between a prediction vector selected from motion vectors between a plurality of macroblocks adjacent to the macroblock and the second reference frame, and a first index indicating the first reference frame and the second reference frame are indicated. Receiving and decoding encoded data including a difference value from the second index, and (b) adjacent to the decoding target macroblock Selecting the prediction vector from a plurality of macroblocks to be reproduced, (c) regenerating the motion vector by adding the selected prediction vector and the decoded difference vector, and (d) selecting the prediction vector Regenerating the first index by adding the index of the reference frame in the reconstructed macroblock and the decoded difference value, and (e) predicting the macroblock according to the reconstructed motion vector and the reconstructed first index (F) The decoded prediction image signal of the decoding target macroblock is generated by adding the generated prediction macroblock and the decoded prediction error signal.
<Effect> It becomes possible to decode the encoded data encoded by (60), and has the same encoding efficiency improvement effect as (60).

  As described above, the moving image encoding and decoding processing according to the present invention may be realized as hardware (apparatus) or may be executed by software using a computer. Some processing may be realized by hardware, and other processing may be performed by software. Therefore, according to the present invention, it is also possible to provide a program for causing a computer to perform the moving image encoding or decoding processing described in (1) to (62).

The block diagram which shows the structure of the moving image encoder which concerns on the 1st Embodiment of this invention. The block diagram which shows the structure of the moving image encoder which concerns on the 2nd Embodiment of this invention. The block diagram which shows the structure of the moving image decoding apparatus which concerns on the 1st and 2nd embodiment of this invention. The figure which shows the relationship of the inter-frame prediction which concerns on embodiment of this invention. The figure which shows the example of the linear prediction coefficient table which concerns on embodiment of this invention. The figure which shows the example of the linear prediction coefficient table which concerns on embodiment of this invention. The figure which shows the example of the table which shows the reference frame which concerns on embodiment of this invention The block diagram which shows the structure of the moving image encoder which concerns on the 3rd Embodiment of this invention. The block diagram which shows the structure of the moving image decoding apparatus which concerns on the 3rd Embodiment of this invention. The figure which shows the example of the syntax which shows the linear prediction coefficient which concerns on embodiment of this invention The figure which shows the example of the table which shows the reference frame which concerns on embodiment of this invention The figure which shows the relationship of the inter-frame prediction which concerns on embodiment of this invention. The figure which shows the relationship of the inter-frame prediction which concerns on embodiment of this invention. The figure which shows the example of the encoding method of the motion vector information which concerns on embodiment of this invention, and a decoding method The figure which shows the example of the encoding method of the motion vector information which concerns on embodiment of this invention, and a decoding method The figure which shows the example of the encoding method of the motion vector information which concerns on embodiment of this invention, and a decoding method The figure explaining the prediction encoding method of the motion vector information which concerns on embodiment of this invention The figure explaining the prediction encoding method of the motion vector information which concerns on embodiment of this invention The block diagram which shows the structure of the moving image encoder which concerns on the 4th Embodiment of this invention. The figure explaining the example of the determination method of the linear prediction coefficient which concerns on embodiment of this invention The figure explaining the example of the determination method of the linear prediction coefficient which concerns on embodiment of this invention The figure explaining the example of the determination method of the linear prediction coefficient which concerns on embodiment of this invention The figure explaining the example of the determination method of the linear prediction coefficient which concerns on embodiment of this invention The figure explaining the example of the determination method of the linear prediction coefficient which concerns on embodiment of this invention The figure explaining the motion vector search method which concerns on embodiment of this invention The figure explaining the motion vector search method which concerns on embodiment of this invention The figure explaining the motion vector encoding method which concerns on embodiment of this invention The figure explaining the motion vector encoding method which concerns on embodiment of this invention The figure which shows the relationship of the inter-frame prediction which concerns on embodiment of this invention. The figure explaining the motion vector encoding method which concerns on embodiment of this invention The figure explaining the motion vector encoding method which concerns on embodiment of this invention The figure explaining the motion vector encoding method which concerns on embodiment of this invention The flowchart which shows the procedure of the moving image encoding regarding embodiment of this invention. The figure explaining the weighted prediction regarding embodiment of this invention The figure which shows the data structure of the picture header or slice header regarding embodiment of this invention The figure which shows the 1st example of the data structure of the coefficient table of the weighted prediction regarding embodiment of this invention The figure which shows the 2nd example of the data structure of the coefficient table of the weighted prediction regarding embodiment of this invention. The figure which shows the data structure of the moving image encoded data which concerns on embodiment of this invention. The flowchart which shows the procedure of the moving image decoding regarding embodiment of this invention.

Explanation of symbols

100: Input video signal (frame to be encoded)
DESCRIPTION OF SYMBOLS 101 ... Prediction error signal 102 ... Quantization DCT coefficient data 103 ... Local decoded image signal 104, 105 ... Reference macroblock signal 106 ... Prediction image signal 107 ... Side information 108 ... Encoded data 111 ... Subtractor 112 ... DCT converter DESCRIPTION OF SYMBOLS 113 ... Quantizer 114 ... Variable length encoder 115 ... Inverse quantizer 116 ... Inverse DCT converter 117, 118, 152 ... Reference frame memory 119 ... Prediction macroblock generator 120 ... Prediction macroblock selector 140 ... Fade Detector 150 ... Prediction macroblock selector 151 ... Linear predictor 200 ... Encoded data 201 ... Quantized DCT coefficient data 202 ... Side information 203 ... Decoded image signal 204, 205 ... Reference macroblock signal 206 ... Predicted image signal 214 ... variable length decoder 215 ... inverse quantization 216 ... inverse DCT transformer 217,218,252 ... reference frame memory 219 ... prediction macroblock generator 220 ... prediction macroblock selector 250 ... prediction macroblock selector 251 ... linear predictor

Claims (3)

A motion compensated prediction frame using a plurality of reference frames in a predetermined combination and a motion vector between the decoding target block and at least one reference frame for a decoding target block in a decoding target frame of a moving image In a video decoding method for performing inter-decoding,
Decoding encoded data including information of a prediction error signal of the prediction block and the decoding target block, the reference frame, a weighting coefficient, a DC offset value, and the motion vector;
Generating a prediction block by calculating a linear sum using the weighting factor for a plurality of reference blocks extracted from the reference frame and adding the DC offset value; and
Generating a reproduced moving image signal using the prediction error signal and the signal of the prediction block. A motion compensated prediction frame using a plurality of reference frames in a predetermined combination and a motion vector between the decoding target block and at least one reference frame for a decoding target block in a decoding target frame of a moving image In a video decoding device that performs inter-decoding,
Means for decoding encoded data including information of a prediction error signal between the prediction block and the decoding target block, the reference frame, a weighting coefficient, a DC offset value, and the motion vector;
Means for generating the prediction block by calculating a linear sum using the weighting factor for a plurality of reference blocks extracted from the reference frame and adding the DC offset value;
Means for generating a reproduced moving image signal using the prediction error signal and the signal of the prediction block. A motion compensated prediction frame using a plurality of reference frames in a predetermined combination and a motion vector between the decoding target block and at least one reference frame for a decoding target block in a decoding target frame of a moving image A program for causing a computer to perform a video decoding process for performing inter-decoding,
A process of decoding encoded data including each information of a prediction error signal of the prediction block and the decoding target block, the reference frame, a weighting coefficient, a DC offset value, and the motion vector;
A process of generating the prediction block by calculating a linear sum using the weighting factor for a plurality of reference blocks extracted from the reference frame and adding the DC offset value;
A program for causing a computer to perform a moving picture decoding process including a process of generating a reproduced moving picture signal using the prediction error signal and the signal of the prediction block.

Images