JP2010206848A - Method and device for generating predictive image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently code moving pictures of which luminance changes with respect to time, such as, fading and dissolving images. <P>SOLUTION: A variable length decoder 303 decodes coding data 300. The data include a predictive error signal indicating an error of a predictive image signal with respect to a moving picture signal, including a luminance signal and two color difference signals; motion vector information 414; and index information 415 indicating a combination of at least one reference image number and a predictive parameter prepared for each of the luminance and two color difference signals, beforehand. A frame memory/predictive image generator 308 generates a predictive image signal 412, in accordance with a combination of the reference image number and the predictive parameter indicated by the decoded index information, and uses the predictive error signal and the predictive image signal, to generate a reproduced moving picture signal. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a prediction image generation method and apparatus suitable for decoding corresponding to moving picture coding that performs highly efficient coding on fade images and dissolve images.

  ITU-TH. 261, H.M. In motion picture coding standard systems such as H.263, ISO / IEC MPEG-2 and MPEG-4, motion compensated inter-frame coding is used as one of coding modes. As a prediction model in motion-compensated prediction interframe coding, a model that has the highest prediction efficiency when the brightness does not change in the time direction is employed. In the case of a fade image in which the brightness of an image changes, for example, when a normal image is faded in from a black image, a method for appropriately predicting a change in the brightness of the image is known. Absent. Accordingly, there is a problem that a large amount of code is required to maintain the image quality even in a fade image.

  For example, Japanese Patent No. 3166716 “Fade Image Corresponding Video Encoding Device and Encoding Method” (Patent Document 1) addresses this problem by detecting a fade image portion and changing the code amount allocation. Yes. Specifically, in the case of a fade-out image, a large amount of code is assigned to the beginning of the fade-out where the luminance changes. Since the last part of the fade-out usually becomes a monochromatic image and can be easily encoded, the allocation of the code amount is reduced. In this way, the overall image quality is improved without increasing the total code amount too much.

  On the other hand, Japanese Patent No. 2938412 “Method for compensating luminance change of moving image, moving image encoding device, moving image decoding device, recording medium recording moving image encoding or decoding program, and recording medium recording moving image encoded data (Patent Document 2) proposes an encoding method corresponding to a fade image by compensating a reference image according to two parameters of a luminance change amount and a contrast change amount.

  Thomas Wiegand and Berand Girod, “Multi-frame motion-compensated prediction for video transmission”, Kluwer Academic Publishers 2001 (Non-Patent Document 1) proposes an encoding method based on a plurality of frame buffers. In this method, the prediction efficiency is improved by selectively creating a prediction image from a plurality of reference frames held in the frame buffer.

Japanese Patent No. 3166716 Japanese Patent No. 2938412

Thomas Wiegand and Berand Girod, “Multi-frame motion-compensated prediction for video transmission”, Kluwer Academic Publishers 2001

  In Patent Document 1, in order to improve the image quality without increasing the total code amount in the coding of the fade image by detecting the fade image portion and changing the code amount allocation, the framework of the existing coding scheme is used. Although there is an advantage that can be realized, the prediction efficiency is not essentially increased, so that a large improvement in coding efficiency cannot be expected.

  On the other hand, in Patent Document 2, there is a merit that prediction efficiency for a fade image is improved, but for a so-called dissolve image (also called a cross-fade image) in which an image gradually changes from one image to another, Sufficient prediction efficiency cannot be obtained.

  In the method of Non-Patent Document 3, a fade image and a dissolve image are not sufficiently dealt with, and even if a plurality of reference frames are prepared, the prediction efficiency cannot be improved.

  As described above, according to the prior art, a large amount of code is required to encode a fade image or dissolve image while maintaining high image quality, and there is a problem in that improvement in encoding efficiency cannot be expected.

  Therefore, the present invention provides a predictive image generation method and apparatus capable of decoding corresponding to highly efficient encoding, particularly for moving images whose luminance changes with time such as fade images and dissolve images. For the purpose.

  In order to solve the above-described problem, in the first aspect of the present invention, on the moving image encoding side, a reference image and a motion vector between the input moving image signal and the reference image are set for the input moving image signal. When performing motion-compensated predictive encoding using one of the plurality of combinations of at least one reference image number and a prediction parameter prepared in advance, one combination is selected for each encoding target block of the input moving image signal. Generating a prediction image signal according to the reference image number and the prediction parameter of the selected combination, generating a prediction error signal representing an error of the prediction image signal with respect to the input moving image signal, and selecting the prediction error signal, the motion vector information and The index information indicating the combination is encoded.

  On the other hand, on the moving image decoding side, encoded data including a prediction error signal indicating an error of the predicted image signal with respect to the moving image signal, motion vector information, and index information indicating a combination of at least one reference image number and a prediction parameter. Is decoded, a predicted image signal is generated according to the reference image number and the prediction parameter of the combination indicated by the decoded index information, and a reproduced moving image signal is generated using the prediction error signal and the predicted image signal.

  In another aspect of the present invention, one combination is selected from a plurality of combinations of prediction parameters prepared in advance on the moving image encoding side for each encoding target block of the input moving image signal and designated. A prediction image signal is generated in accordance with a reference image of at least one reference image number and a selected combination of prediction parameters, a prediction error signal representing an error of the prediction image signal with respect to the input moving image signal is generated, and a prediction error signal and a motion vector Information, a designated reference image number, and index information indicating the selected combination are encoded.

  On the moving image decoding side, encoded data including a prediction error signal indicating an error of a predicted image signal with respect to a moving image signal, motion vector information, a designated reference image number, and index information indicating a combination of prediction parameters is included. Decoded, a predicted image signal is generated according to a combination of prediction parameters indicated by the decoded reference image number and decoded index information, and a playback video signal is generated using the prediction error signal and the predicted image signal.

  As described above, according to the present invention, a combination of a reference image number and a prediction parameter, or a plurality of prediction methods having different combinations of a plurality of prediction parameters corresponding to a designated reference image number are prepared, and a fade image or a dissolve image is prepared. An appropriate prediction image signal can be generated based on a prediction method with higher prediction efficiency even for a moving image signal in which an appropriate prediction image signal cannot be generated by a normal prediction method of moving image encoding.

  Further, the moving image signal is a signal in which a frame-wise image signal of a progressive signal, an image signal in a frame unit obtained by merging two fields of an interlaced signal, and an image signal in a field unit of an interlaced signal are mixed. In the case of an image signal in units of frames, the reference image number indicates the number of the reference image signal in units of frames. When the moving image signal is an image signal in units of fields, the reference image number indicates the reference image signal in units of fields. To do.

  As a result, the moving image signal is a moving image signal in which a frame structure and a field structure are mixed, and an appropriate predicted image signal cannot be created by a normal moving image encoding prediction method such as a fade image or a dissolve image. An appropriate predicted image signal can also be created for a moving image signal based on a prediction method with higher prediction efficiency.

  Further, instead of sending the reference picture number and prediction parameter information itself from the encoding side to the decoding side, index information indicating a combination of the reference picture number and the prediction parameter is sent, or the reference picture number is sent separately. In such a case, the encoding efficiency can be improved by sending index information indicating a combination of prediction parameters.

  According to the present invention, it is possible to perform appropriate prediction for a moving image whose luminance changes with time, such as a fade image and a dissolve image, and perform highly efficient moving image encoding.

The block diagram which shows the structure of the moving image encoder which concerns on the 1st Embodiment of this invention. FIG. 2 is a block diagram showing a detailed configuration of the frame memory / predictive image creator in FIG. The figure which shows the example of the combination table of the reference frame number and prediction parameter which are used in the embodiment The flowchart which shows an example of the procedure of selection of the prediction method (combination of a reference frame number and a prediction parameter) for every macroblock and encoding mode determination in the embodiment The block diagram which shows the structure of the moving image decoding apparatus which concerns on the same embodiment The block diagram which shows the detailed structure of the frame memory / predictive image generator in FIG. The figure which shows the example in the case of reference frame number 1 of the prediction parameter combination table in the case of sending the reference frame number which concerns on the 2nd Embodiment of this invention as mode information The figure which shows the example in the case of the reference frame number 2 of the prediction parameter combination table at the time of sending the reference frame number which concerns on the embodiment as mode information The figure which shows the example in the case of the reference frame number 1 of the reference image number and the prediction parameter combination table which concerns on the 3rd Embodiment of this invention. The figure which shows the example in the case of the table of only the luminance signal which concerns on the same embodiment The figure which shows the example of the syntax for every block in the case of encoding index information. The figure which shows the example of the concrete encoding bit stream in the case of producing | generating a predicted image using one reference image The figure which shows the example of the concrete encoding bit stream in the case of producing a predicted image using two reference images The figure which shows the example of the combination table of the reference frame number in case the encoding object which concerns on the 4th Embodiment of this invention is a top field, a reference field number, and a prediction parameter The figure which shows the example of the combination table of the reference frame number in case the encoding object which concerns on the embodiment is a bottom field, a reference field number, and a prediction parameter

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
(About encoding side)
FIG. 1 shows the configuration of a video encoding apparatus according to the first embodiment of the present invention. In this example, the moving image signal 100 is input to the moving image encoding apparatus in units of frames, for example. The moving image signal 100 is input to a subtractor 101, where a difference from the predicted image signal 212 is taken to generate a prediction error signal. Either the prediction error signal or the input moving image signal 100 is selected by the mode selection switch 102, and orthogonal transform, for example, discrete cosine transform (DCT) is performed by the orthogonal transformer 103. The orthogonal transformer 103 obtains orthogonal transform coefficient information, for example, DCT coefficient information. The orthogonal transform coefficient information is quantized by the quantizer 104 and then bifurcated. One of the bifurcated quantized orthogonal transform coefficient information 210 is guided to the variable length encoder 215.

  The other of the bifurcated quantized orthogonal transform coefficient information 210 is subjected to sequential processing reverse to the processing of the quantizer 104 and the orthogonal transformer 103 by the inverse quantizer 105 and the inverse orthogonal transformer 106 to obtain a prediction error signal. After the same signal is obtained, the adder 107 adds the predicted image signal 212 input via the switch 109 to generate a locally decoded image signal 211. The locally decoded image signal 211 is input to the frame memory / predicted image generator 108.

  The frame memory / predicted image generator 108 selects one combination from a plurality of combinations of reference frame numbers and prediction parameters prepared in advance. For the image signal (local decoded image signal 211) of the reference frame indicated by the reference frame number in the selected combination, a linear sum is calculated according to the prediction parameter in the selected combination, and further according to the prediction parameter By adding the offset, in this example, a reference image signal for each frame is generated. Thereafter, the frame memory / predicted image generator 108 performs motion compensation on the reference image signal using a motion vector, and generates a predicted image signal 212.

  In this process, the frame memory / predicted image generator 108 generates motion vector information 214 and index information 215 indicating the selected combination of the reference frame number and the prediction parameter, and further selects a coding mode in the mode selector 212. Send necessary information to. The motion vector information 214 and the index information 215 are input to the variable length encoder 111. The frame memory / predicted image generator 108 will be described in detail later.

  The mode selector 110 selects a coding mode in units of macroblocks based on the prediction information P from the frame memory / predictive image generator 108, that is, intraframe coding (hereinafter referred to as intra coding) and a motion compensated prediction frame. One of inter coding (hereinafter referred to as inter coding) is selected, and switch control signals M and S are output.

  In the intra coding mode, the switches 102 and 112 are switched to the A side by the switch control signals M and S, and the input moving image signal 100 is input to the orthogonal transformer 103. In the inter coding mode, the switches 102 and 112 are switched to the B side by the switch control signals M and S, the orthogonal transformer 103 generates a prediction error signal from the subtractor 102, and the adder 107 generates a frame memory / predicted image. The predicted image signals 212 from the unit 108 are respectively input. Mode information 213 is output from the mode selector 212 and input to the variable length encoder 111.

  In the variable length encoder 111, the orthogonal transform coefficient information 210, the mode information 213, the motion vector information 214, and the index information 215 are variable length encoded, and each variable length code generated thereby is multiplexed by the multiplexer 114. Then, the output buffer 115 smoothes the result. Thus, the encoded data 116 output from the output buffer 115 is sent to a transmission system or storage system (not shown).

  The encoding controller 113 controls the encoding unit 112 composed of elements from the subtracter 101 to the variable length encoder 111, specifically monitors the buffer amount of the output buffer 115, for example, and the buffer amount is constant. The encoding parameters such as the quantization step size of the quantizer 104 are controlled so that

(About frame memory / predicted image generator 108)
FIG. 2 shows a detailed configuration of the frame memory / predicted image creator 108 in FIG. In FIG. 2, the local decoded image signal 211 input from the adder 107 in FIG. 1 is stored in the frame memory set 202 under the control of the memory controller 201. The frame memory set 202 includes a plurality (N) of frame memories FM1 to FMN for temporarily storing the locally decoded image signal 211 as a reference frame.

  The prediction parameter controller 203 prepares a plurality of combinations of reference frame numbers and prediction parameters as a table in advance, and the reference frame number of the reference frame used for generating the predicted image signal 212 based on the input moving image signal 100 and the prediction A combination of parameters is selected, and index information 215 indicating the selected combination is output.

  The multi-frame motion evaluator 204 creates a reference image signal according to the combination of the reference frame number selected by the prediction parameter controller 203 and the index information, and the amount of motion and the prediction error are determined from the reference image signal and the input image signal 100. Evaluation is performed, and motion vector information 214 that minimizes the prediction error is output. The multi-frame motion compensator 205 generates a predicted image signal 212 by performing motion compensation on the reference image signal selected for each block by the multi-frame motion evaluator 204 according to the motion vector.

(About reference frame number and prediction parameter combination table)
FIG. 3 is an example of a reference frame number / prediction parameter combination table prepared by the prediction parameter controller 203. The index corresponds to a predicted image that can be selected for each block. In this example, it can be seen that there are eight types of predicted images. In other words, the reference frame number n is a number of locally decoded images used as a reference frame, and represents the number of locally decoded images for the past n frames.

When the predicted image signal 212 is generated using the image signals of a plurality of reference frames stored in the frame memory set 202, a plurality of reference frame numbers are designated, and the luminance signal (Y) and the color difference are also used for the prediction parameters. For each signal (Cb, Cr), (reference frame number + 1) coefficients are designated. Here, as shown in Equations (1) to (3), when the number of reference frames is n, the prediction parameter is n + 1 of Di (i =,..., N + 1) with respect to the luminance signal Y, and the color difference signal. N + 1 Ei (i =,..., N + 1) are prepared for Cb, and n + 1 Fi (i =,..., N + 1) are prepared for the color difference signal Cr.

  More specifically, using FIG. 3, the last number of the prediction parameter in FIG. 3 represents an offset, and the other numerical values including the first number of the prediction parameter represent a weighting coefficient (prediction coefficient). In index 0, the number of reference frames is n = 2, the reference frame number is 1, and the prediction parameter is 1, 0 for all of the luminance signal Y and the color difference signals Cr, Cb. That the prediction parameter is 1, 0 as in this example means that the local decoded image signal of reference frame number 1 is multiplied by 1 and offset 0 is added, in other words, local decoding of reference frame number 1 is performed. This is a case where the image signal is used as a reference image signal as it is.

  Index 1 uses two reference frames, which are locally decoded image signals of reference frame numbers 1 and 2, and according to prediction parameters 2, -1, 0 for luminance signal Y, reference frame number 1 for luminance signal Y The local decoded image signal is doubled, the local decoded image signal of reference frame number 2 is subtracted, and an offset 0 is added. That is, a reference image signal is generated by performing extrapolation prediction from a locally decoded image signal of two frames. For the color difference signals Cr and Cb, since the prediction parameters are 1, 0 and 0, the local decoded image signal of the reference frame number 1 is used as the reference image signal as it is. This prediction method corresponding to index 1 is particularly effective for dissolve images.

  In the index 2, the luminance signal Y of the locally decoded image signal of the reference frame number 1 is multiplied by 5/4 according to the prediction parameters 5/4 and 16, and an offset 16 is added. For the color difference signals Cr and Cb, the prediction parameter is 1, so that the reference image signal is used as it is. This prediction method is particularly effective for a fade-in image from a black screen.

  By enabling reference image signals to be selected based on a plurality of prediction methods having different combinations of reference frame numbers and prediction parameters to be used in this way, there is no appropriate prediction method so far, and image quality deteriorates. It is also possible to deal with faded images and dissolve images.

(For prediction method selection and coding mode determination procedure)
Next, an example of a specific procedure for selecting a prediction method (combination of a reference frame number and a prediction parameter) for each macroblock and determining a coding mode in this embodiment will be described with reference to FIG.
First, the maximum value that can be assumed is entered in the variable min_D (step S101). LOOP1 (step S102) indicates repetition for selection of a prediction method in inter coding, and variable i indicates the value of the index shown in FIG. Here, the code amount related to the motion vector information 214 (the variable length code output from the variable length encoder 111 corresponding to the motion vector information 214 is calculated so that the optimal motion vector for each prediction method can be obtained. The evaluation value D of each index (combination of the reference frame number and the prediction parameter) is calculated from the sum of the code amount) and the prediction error absolute value, and the motion vector that minimizes the evaluation value D is selected (step S103). The evaluation value D is compared with min_D (step S104). If the evaluation value D is smaller than min_D, the evaluation value D is set to min_D, and the index i is assigned to min_i (step S105).

  Next, an evaluation value D in the case of intra coding is calculated (step S106), and this evaluation value D is compared with min_D (step S107). As a result of this comparison, if min_D is smaller, mode MODE is determined to be inter-coding, and min_i is substituted into index information INDEX (step S108). If the evaluation value D is smaller, the mode MODE is determined to be intra coding (step S109). Here, the evaluation value D is an estimated amount of code for the same quantization step size.

(About decryption side)
Next, a video decoding device corresponding to the video encoding device shown in FIG. 1 will be described. FIG. 5 shows the configuration of the moving picture decoding apparatus according to this embodiment. The encoded data 300 sent from the moving picture encoding apparatus having the configuration shown in FIG. 1 and sent via the transmission system or the storage system is once stored in the input buffer 301 and is demultiplexed by the demultiplexer 302 every frame. Are separated based on the syntax and then input to the variable length decoder 303. The variable length decoder 303 decodes the variable length code of each syntax of the encoded data 300 and reproduces the quantized orthogonal transform coefficient, mode information 413, motion vector information 414, and index information 415.

  Of each reproduced information, the quantized orthogonal transform coefficient is inversely quantized by the inverse quantizer 304 and inversely orthogonally transformed by the inverse orthogonal transformer 305. Here, when the mode information 413 indicates the intra coding mode, a reproduced image signal is output from the inverse orthogonal transformer 305 and is output as a final reproduced image signal 310 via the adder 306.

  When the mode information 413 indicates the inter coding mode, the prediction error signal is output from the inverse orthogonal transformer 305, and the mode selection switch 309 is turned on. The prediction image signal 412 output from the prediction error signal and the frame memory / prediction image generator 308 is added by the adder 306, whereby the reproduced image signal 310 is output. The reproduced image signal 310 is stored in the frame memory / predicted image generator 308 as a reference image signal.

  The mode information 413, motion vector information 414, and index information 415 are input to the frame memory / predicted image creator 308. The mode information 413 is also input to the mode selection switch 309. The switch 309 is turned off in the intra coding mode and turned on in the inter coding mode.

  The frame memory / predictive image generator 308 prepares a plurality of combinations of reference frame numbers and prediction parameters prepared in advance as a table in the same manner as the encoding-side frame memory / predictive image generator 108 shown in FIG. From this, one combination indicated by the index information 415 is selected. For the image signal (reproduced image signal 210) of the reference frame indicated by the reference frame number in the selected combination, a linear sum is calculated according to the prediction parameter in the selected combination, and an offset according to the prediction parameter is further calculated. By adding, a reference image signal is generated. Thereafter, the predicted image signal 412 is generated by performing motion compensation on the generated reference image signal using the motion vector indicated by the motion vector information 414.

(About frame memory / predicted image generator 308)
FIG. 6 shows a detailed configuration of the frame memory / predicted image creator 308 in FIG. In FIG. 6, the reproduced image signal 310 output from the adder 306 in FIG. 5 is stored in the frame memory set 402 under the control of the memory controller 401. The frame memory set 402 includes a plurality (N) of frame memories FM1 to FMN for temporarily holding the reproduced image signal 310 as a reference frame.

  The prediction parameter controller 403 prepares combinations of reference frame numbers and prediction parameters in advance as a table similar to that shown in FIG. 3, and performs prediction based on the index information 415 from the variable length decoder 303 in FIG. A combination of a reference frame number of a reference frame used for generating the image signal 412 and a prediction parameter is selected. The multi-frame motion compensator 404 creates a reference image signal according to the combination of the reference frame number and the index information selected by the prediction parameter controller 403, and the variable length decoder 303 in FIG. The predicted image signal 412 is generated by performing motion compensation in units of blocks in accordance with the motion vector indicated by the motion vector information 414 from.

[Second Embodiment]
Next, a second embodiment of the present invention will be described with reference to FIGS. Since the overall configuration of the video encoding device and video decoding device in the present embodiment is substantially the same as that of the first embodiment, only differences from the first embodiment will be described.

  In the present embodiment, an example of how to represent a prediction parameter in a method in which a plurality of reference frame numbers can be specified by mode information in units of macroblocks is shown. In this case, the reference frame number is determined from the mode information for each macroblock. Therefore, instead of the reference frame number / prediction parameter combination table as in the first embodiment, the prediction parameter combination table is used as shown in FIGS. 7 and 8, and the index information does not specify the reference frame number. Only the combination of prediction parameters is specified.

  The table shown in FIG. 7 shows an example of combinations of prediction parameters when the number of reference frames is one. As prediction parameters, two parameters (one weight coefficient and one offset), which are (reference frame number + 1), are designated for each of the luminance signal (Y) and the color difference signals (Cb, Cr).

  The table shown in FIG. 8 is an example of combinations of prediction parameters when the number of reference frames is two. In this case, three parameters (two weighting factors and one offset) that are (reference frame number + 1) for each of the luminance signal (Y) and the color difference signals (Cb, Cr) are used as prediction parameters. Specify it. This table is prepared on the encoding side and the decoding side as in the first embodiment.

[Third Embodiment]
Next, a third embodiment of the present invention will be described with reference to FIGS. Since the overall configuration of the moving picture encoding apparatus and decoding apparatus in the present embodiment is substantially the same as in the first embodiment, only the differences from the first and second embodiments will be described below. .

  In the first and second embodiments, examples of managing images in units of frames have been described. However, in this embodiment, images are managed in units of images called pictures. When both a progressive signal and an interlace signal exist as input image signals, the unit of an image to be encoded is not necessarily a frame. Considering this, the picture here is (a) one frame image of a progressive signal, (b) one frame image generated by merging two fields of an interlace signal, and (c) an interlace signal. Any one of the images in one field is pointed out.

  When the encoding target image is an image having a frame structure as shown in (a) or (b), the reference image used in motion compensation prediction also has a frame structure or a field structure. Regardless of whether the frame is managed as a frame, a reference image number is assigned. Similarly, when the encoding target image is an image having a field structure as shown in (c), whether the encoded image as the reference image is a frame structure or a field structure for the reference image used in motion compensation prediction. Regardless, it is managed as a field and a reference image number is assigned.

Equations (4), (5), and (6) below show examples of prediction expressions for reference image numbers and prediction parameters prepared by the prediction parameter controller 203. The example shown here is a prediction formula for creating a predicted image signal by motion compensation prediction using one reference image (picture) signal.

  Here, Y is the predicted image signal of the luminance signal, Cb and Cr are the predicted image signals of the two color difference signals, and RY (i), RCb (i), RCr (i) are the luminances of the reference image signal of index i. The pixel values of the signal and the two color difference signals are shown. D1 (i) and D2 (i) are the prediction coefficient and offset of the luminance signal of index i, respectively. E1 (i) and E2 (i) are the prediction coefficient and offset of the color difference signal Cb of index i, respectively. F1 (i) and F2 (i) are the prediction coefficient and offset of the color difference signal Cr of index i, respectively. The index i takes a value from 0 to (the maximum number of reference images−1), is encoded for each encoding target block (for example, for each macroblock), and is transmitted to the video decoding device.

  Prediction parameters D1 (i), D2 (i), E1 (i), E2 (i), F1 (i), F2 (i) are values determined in advance between the moving picture coding apparatus and the decoding apparatus, Alternatively, it is a predetermined encoding unit such as a frame, a field, and a slice, and is encoded with the encoded data and transmitted from the moving picture encoding apparatus to the decoding apparatus, and is shared by both apparatuses.

  Equations (4), (5), and (6) avoid the division by selecting the denominator of the prediction coefficient to be multiplied by the reference image signal as a power of 2, that is, 2, 4, 8, 16,. It is a prediction formula that can be calculated by This avoids an increase in calculation cost due to division.

  That is, >> in Equations (4), (5), and (6) is an operator that arithmetically shifts the integer a to the right by b bits when a >> b is set. The function clip is a clipping function in which the value in () is 0 when it is smaller than 0 and is 255 when it is larger than 255.

  Here, LY is the shift amount of the luminance signal, and LC is the shift amount of the color difference signal. As these shift amounts LY and LC, values determined in advance by the moving image encoding device and the decoding device are used, or in a predetermined encoding unit such as a frame, a field, or a slice in the moving image encoding device. By being encoded together with the table and the encoded data and transmitted to the moving picture decoding apparatus, they are shared by both apparatuses.

  In this embodiment, the prediction parameter controller 203 in FIG. 2 prepares a combination table of reference image numbers and prediction parameters as shown in FIGS. 9 and 10. 9 and 10, the index i corresponds to a predicted image that can be selected for each block. In this example, there are four types of predicted images corresponding to 0 to 3 of the index i. In other words, the reference image number is a number of a locally decoded image used as a reference image.

  Flag is a flag indicating whether or not to apply a prediction formula using a prediction parameter to the reference image number indicated by the index i. If Flag is “0”, motion compensation prediction is performed using the local decoded image of the reference image number indicated by the index i without using the prediction parameter. If Flag is “1”, using the locally decoded image of the reference image number indicated by the index i and the prediction parameter, a predicted image is created according to Equations (4), (5), and (6) to perform motion compensation prediction. Also for the information of the Flag, a value determined in advance by the video encoding device and the decoding device is used, or in the video encoding device, a table and a predetermined encoding unit such as a frame, a field, or a slice are used. The data is encoded together with the encoded data and transmitted to the moving picture decoding apparatus, so that both apparatuses share the same.

  In these examples, for the reference image number 105, when the index i is i = 0, a prediction image is generated using the prediction parameter, and when i = 1, motion compensation prediction is performed without using the prediction parameter. . As described above, a plurality of prediction methods may exist for the same reference image number.

  The table shown in FIG. 9 shows the prediction parameters D1 (i), D2 (i), E1 (i), E2 assigned to the luminance signal and the two color-difference signals corresponding to the equations (4), (5), and (6). (i), F1 (i), and F2 (i). On the other hand, FIG. 10 is an example of a table in which prediction parameters are assigned only to luminance signals. In general, the code amount of the color difference signal is not large compared to the code amount of the luminance signal. Therefore, in order to reduce the amount of calculation and the transmission code amount of the table when creating the prediction image signal, a table in which the prediction parameter for the color difference signal is deleted and the prediction parameter is assigned only to the luminance signal as shown in FIG. prepare. At this time, only the formula (4) is used as the prediction formula.

The following formulas (7) to (12) are examples of prediction formulas when a plurality (two in this example) of reference images are used for prediction.

  Prediction parameters D1 (i), D2 (i), E1 (i), E2 (i), F1 (i), F2 (i), LY, LC, and Flag information are pre-decoded by the moving picture coding apparatus. A value determined between encoding devices, or a predetermined encoding unit such as a frame, a field, and a slice, encoded together with encoded data, and transmitted from the moving image encoding device to the decoding device , Shared by both devices.

  When the decoding target image is an image having a frame structure, the reference image used in motion compensation prediction is also managed as a frame regardless of whether the decoded image that is the reference image is a frame structure or a field structure. An image number is assigned. Similarly, when the program target image is an image having a field structure such as that described above, the reference image used in motion compensation prediction also has a field regardless of whether the decoded image that is the reference image is a frame structure or a field structure. And a reference image number is assigned.

(About index information syntax)
FIG. 11 shows an example of syntax when index information is encoded in each block. First, mode information MODE exists for each block. Whether or not to encode the index information IDi indicating the value of the index i and the index information IDj indicating the value of the index j is determined according to the mode information MODE. After the encoded index information, motion vector information MVi for motion compensated prediction of index i and motion vector information MVj for motion compensated prediction of index j are encoded as motion vector information of each block. .

(Data structure of encoded bit stream)
FIG. 12 shows an example of a specific encoded bit stream for each block when a predicted image is created using one reference image. Following the mode information MODE, index information IDi is arranged, and then motion vector information MVi is arranged. The motion vector information MVi is usually two-dimensional vector information. However, depending on the motion compensation method inside the block indicated by the mode information, a plurality of two-dimensional vectors may be sent.

FIG. 13 shows an example of a specific coded bitstream for each block when a predicted image is created using two reference images. Following the mode information MODE, index information IDi and index information IDj are arranged, followed by motion vector information MVi and motion vector information MVj. The motion vector information MVi and the motion vector information j are usually two-dimensional vector information, but a plurality of two-dimensional vectors may be sent depending on the motion compensation method inside the block indicated by the mode information. .
Note that the syntax and bitstream structure described above can be similarly applied to all the embodiments.

[Fourth Embodiment]
Next, a fourth embodiment of the present invention will be described with reference to FIGS. The overall configuration of the video encoding device and video decoding device in the present embodiment is substantially the same as in the first embodiment, and therefore only the differences from the first, second, and third embodiments. Will be explained. In the third embodiment, switching between frame-based encoding and field-based encoding is performed for each image (picture), whereas in the fourth embodiment, frame-based encoding is performed for each macroblock. This is an example of switching between field-based encoding.

  When switching between frame-based encoding and field-based encoding for each macroblock, the same reference image is used in both cases where the macroblock is encoded in the frame and in the field. It will refer to images with different numbers. Therefore, there is a possibility that an appropriate predicted image signal cannot be created with the tables of FIGS. 9 and 10 used in the third embodiment.

  In order to solve this point, in the present embodiment, the prediction parameter controller 203 in FIG. 2 prepares a combination table of reference image numbers and prediction parameters as shown in FIGS. When a macroblock is encoded in a field, as shown in the tables of FIGS. 14 and 15, a reference image number (reference frame index number) used when the macroblock is encoded in units of frames. The same prediction parameter as that corresponding to) is used.

  FIG. 14 is a table used when the macroblock is encoded in units of fields and when the encoding target image is a top field. The upper row of the field index column corresponds to the top field, and the lower row corresponds to the bottom field. As shown in FIG. 14, the frame index j and the field index k have a relationship of k = 2j in the top field and k = 2j + 1 in the bottom field. The relationship between the reference frame number m and the reference field number n is n = 2m in the top field and n = 2m + 1 in the bottom field.

  FIG. 15 is a table in which the macroblock is encoded in units of fields and the encoding target is a bottom field. Similarly to the table of FIG. 14, the upper row of the field index column corresponds to the top field, and the lower row corresponds to the bottom field. In the table of FIG. 15, the relationship between the frame index j and the field index k is k = 2j + 1 in the top field and k = 2j in the bottom field. By doing so, a small value is assigned as the field index k to the bottom field of the same phase. The relationship between the reference frame number m and the reference field number n is the same as that in the table of FIG.

  When the macroblock block is encoded in units of fields, the frame index and the field index are encoded as index information using the tables shown in FIGS. On the other hand, when the macroblock is encoded on a frame basis, only the frame index common to the tables of FIGS. 14 and 15 is index-encoded as index information.

  In this embodiment, the frame and field prediction parameters are assigned in one table, but a frame table and a field table may be prepared separately in one image or slice.

  Furthermore, in each of the above-described embodiments, the example of the moving image encoding / decoding method using the orthogonal transform in units of blocks has been described. However, for example, when other conversion methods such as a wavelet transform are used, The method of the present invention described in the above embodiment can be similarly applied.

  The moving image encoding and decoding processes according to the present invention may be realized as hardware (apparatus) or may be executed by software using a computer. Some processing may be realized by hardware, and other processing may be performed by software. Therefore, according to the present invention, it is possible to provide a program for causing a computer to perform the above-described moving image encoding or decoding process or a storage medium storing the program.

DESCRIPTION OF SYMBOLS 100 ... Input moving image signal 101 ... Subtractor 102, 109 ... Mode selection switch 103 ... Orthogonal transformer 104 ... Quantizer 105 ... Inverse quantizer 106 ... Inverse orthogonal transformer 107 ... Adder 108 ... Frame memory / predicted image Generator 110 ... Mode selector 111 ... Variable length encoder 112 ... Encoder 113 ... Encoding controller 114 ... Multiplexer 115 ... Output buffer 116 ... Encoded data 201 ... Memory controller 202 ... Multiple frame memory 203 ... prediction parameter controller 204 ... multiple frame motion estimator 205 ... multiple frame motion compensator 211 ... local decoded image signal 212 ... predicted image signal 213 ... mode information 214 ... motion vector information 215 ... index information 300 ... encoded data 301 ... Input buffer 302 ... Demultiplexer 303 ... Variable length Encoder 304 ... Inverse quantizer 305 ... Inverse orthogonal transformer 306 ... Adder 307 ... Frame memory / predicted image generator 308 ... Adder 309 ... Mode changeover switch 310 ... Reproduced image signal 401 ... Memory controller 402 ... Multiple Frame memory 403 ... Prediction parameter controller 404 ... Multiple frame motion compensator 412 ... Prediction image signal 413 ... Mode information 414 ... Motion vector information 415 ... Index information

Claims (2)

In a prediction image generation method for generating a prediction image from encoded data obtained by predictively encoding a moving image having luminance and two color differences,
A plurality of combinations including (1) a luminance and a weighting factor for each of two color differences, and (2) a luminance and an offset for each of two color differences are encoded for each unit including one or more decoding target blocks. A) motion vector information, (B) (a) one combination among the plurality of combinations, and (b) a plurality of predetermined numbers of index information respectively indicating reference images, Receiving input of encoded data encoded for each block;
Determining whether a decoding unit of the decoding target block is a frame or a field;
Obtaining the predetermined number of weighting factors and the predetermined number of offsets for each luminance and two color differences from the plurality of combinations and the predetermined number of index information;
According to the motion vector, a prediction image is generated by multiplying the predetermined number of the reference images by the weighting coefficient corresponding to each reference image and adding the predetermined number of the offsets for each luminance and two color differences. And comprising steps
When the decoding unit is a frame, each value that the index information can take indicates the different combination, and when the decoding unit is a field, each value that the index information can take A predicted image generation method in which two values indicating the different reference images are the same combination. In a predicted image generation device that generates a predicted image from encoded data obtained by predictively encoding a moving image having luminance and two color differences,
A plurality of combinations including (1) a luminance and a weighting factor for each of two color differences, and (2) a luminance and an offset for each of two color differences are encoded for each unit including one or more decoding target blocks. A) motion vector information, (B) (a) one combination among the plurality of combinations, and (b) a plurality of predetermined numbers of index information respectively indicating reference images, Means for receiving input of encoded data encoded for each block;
Means for determining whether a decoding unit of the decoding target block is a frame or a field;
Means for determining the predetermined number of the weighting factors and the predetermined number of the offsets for each luminance and two color differences from the plurality of combinations and the predetermined number of index information;
According to the motion vector, a prediction image is generated by multiplying the predetermined number of the reference images by the weighting coefficient corresponding to each reference image and adding the predetermined number of the offsets for each luminance and two color differences. Means,
When the decoding unit is a frame, each value that the index information can take indicates the different combination, and when the decoding unit is a field, each value that the index information can take A predicted image generation device in which two values indicating different reference images are the same combination.

Images