JP4227167B2 - Moving picture decoding method and apparatus - Google Patents

Description

  The present invention relates to a moving image encoding / decoding method and apparatus for performing highly efficient encoding / decoding particularly for fade images and dissolve images.

  ITU-TH. 261, H.M. In motion picture coding standard systems such as H.263, ISO / IEC MPEG-2, and MPEG-4, motion compensation prediction interframe coding is used as one of coding modes. As a prediction model in motion-compensated prediction interframe coding, a model that has the highest prediction efficiency when the brightness does not change in the time direction is employed. In the case of a fade image in which the brightness of an image changes, for example, when a normal image is faded in from a black image, a method for appropriately predicting a change in the brightness of the image is known. Absent. Accordingly, there is a problem that a large amount of code is required to maintain image quality even in a fade image.

  For example, Japanese Patent No. 3166716 “Fade Image Corresponding Video Encoding Device and Encoding Method” (Patent Document 1) addresses this problem by detecting a fade image portion and changing the code amount allocation. Yes. Specifically, in the case of a fade-out image, a large amount of code is assigned to the beginning of the fade-out where the luminance changes. Since the last part of the fade-out usually becomes a monochromatic image and can be easily encoded, the allocation of the code amount is reduced. In this way, the overall image quality is improved without increasing the total code amount too much.

  On the other hand, Japanese Patent No. 2938412 “Method for compensating luminance change of moving image, moving image encoding device, moving image decoding device, recording medium recording moving image encoding or decoding program, and recording medium recording moving image encoded data (Patent Document 2) proposes an encoding method corresponding to a fade image by compensating a reference image according to two parameters of a luminance change amount and a contrast change amount.

Thomas Wiegand and Berand Girod, “Multi-frame motion-compensated prediction for video transmission”, Kluwer Academic Publishers 2001 (Non-Patent Document 1) proposes an encoding method based on a plurality of frame buffers. In this method, the prediction efficiency is improved by selectively creating a prediction image from a plurality of reference frames held in the frame buffer.
Japanese Patent No. 3166716 Japanese Patent No. 2938412 Thomas Wiegand and Berand Girod, “Multi-frame motion-compensated prediction for video transmission”, Kluwer Academic Publishers 2001

  In Patent Document 1, in order to improve the image quality without increasing the total code amount in the coding of the fade image by detecting the fade image portion and changing the code amount allocation, the framework of the existing coding scheme is used. Although there is an advantage that can be realized, the prediction efficiency is not essentially increased, so that a large improvement in coding efficiency cannot be expected.

  On the other hand, in Patent Document 2, there is a merit that prediction efficiency for a fade image is improved, but for a so-called dissolve image (also called a cross-fade image) in which an image gradually changes from one image to another, Sufficient prediction efficiency cannot be obtained.

  In the method of Non-Patent Document 3, a fade image and a dissolve image are not sufficiently dealt with, and even if a plurality of reference frames are prepared, the prediction efficiency cannot be improved.

  As described above, according to the prior art, a large amount of code is required to encode a fade image or dissolve image while maintaining high image quality, and there is a problem in that improvement in encoding efficiency cannot be expected.

  Accordingly, the present invention enables high-efficiency encoding and a small amount of calculation of moving image encoding and decoding, especially for moving images whose luminance changes with time, such as fade images and dissolve images. It is an object of the present invention to provide a method and apparatus.

  In order to solve the above-described problem, in the first aspect of the present invention, at the moving image encoding side, at least one reference image signal for the input moving image signal, and the input moving image signal and the reference image signal When the number of reference images used for motion compensation prediction encoding is one when performing motion compensation prediction encoding using a motion vector between, at least one reference image number prepared in advance and a prediction parameter A first predicted image signal generation method is used that generates a predicted image signal according to a reference image number and a prediction parameter of one combination selected for each encoding target area of the input moving image signal from among a plurality of combinations.

  On the other hand, when there are a plurality of reference images used for motion compensation predictive coding, calculation is performed based on the reference image numbers of the plurality of reference images and the inter-image distances of the plurality of reference images for each encoding target region. A second prediction signal generation method for generating a prediction image signal according to a prediction parameter to be used is used.

  A prediction error signal representing an error of the prediction image signal with respect to the input moving image signal of the prediction image signal generated in this way is generated, information on the prediction error signal and the motion vector, and the selected combination and a plurality of reference images Index information indicating any one of the reference image numbers is encoded.

  In another aspect of the present invention, the first prediction signal generation method is used when the prediction type of the encoding target region of the input moving image signal is the first prediction type using one reference image for motion compensation prediction encoding. The second prediction signal generation method is used when the prediction type of the encoding target region is a bidirectional prediction type and the number of reference images used for motion compensation prediction encoding is plural.

  On the other hand, on the moving image decoding side, a prediction error signal indicating an error of a predicted image signal with respect to a moving image signal, motion vector information, a combination of one reference image number and a prediction parameter, and reference image numbers of a plurality of reference images The encoded data including the index information indicating any of the above is decoded. When the decoded index information indicates a combination, a predicted image signal is generated according to the reference image number and the prediction parameter of the combination. When the decoded index information indicates reference image numbers of a plurality of reference images, a prediction image signal is generated according to a prediction parameter calculated based on the reference image numbers and the inter-image distances of the plurality of reference images. To do. A reproduction moving image signal is generated using the prediction error signal and the prediction image signal thus generated.

  As described above, according to the present invention, the first predicted image generation method for generating the predicted image signal according to the combination of the reference image number and the prediction parameter, and the prediction calculated based on the interframe distances of the plurality of selected reference images. A second prediction image generation method for generating a prediction image signal using parameters is prepared, and one of the prediction image generation methods is selected according to the number of reference images and the prediction type used for motion compensation prediction encoding. Use.

  As a result, an input video signal that cannot be generated with a normal video coding prediction method such as a fade image or a dissolve image can be generated based on a prediction method with higher prediction efficiency. An appropriate predicted image signal can be created.

  In addition, since the number of multiplications per pixel can be set to one, both the encoding side and the decoding side can reduce the hardware scale and calculation cost.

  Further, instead of sending the reference picture number and prediction parameter information itself from the encoding side to the decoding side, index information indicating a combination of the reference picture number and the prediction parameter is sent, or a reference picture number is sent separately. In some cases, encoding efficiency can be improved by sending index information indicating a combination of prediction parameters.

  As described above, according to the present invention, a moving image with high efficiency and a small amount of calculation can be obtained by appropriately predicting a moving image whose luminance changes with time, such as a fade image and a dissolve image. Encoding / decoding can be performed.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
(About encoding side)
FIG. 1 shows the configuration of a video encoding apparatus according to the first embodiment of the present invention. In this example, the moving image signal 100 is input to the moving image encoding apparatus in units of frames, for example. This moving image signal 100 is input to a subtractor 101, where a difference from the predicted image signal 212 is taken to generate a prediction error signal. Either the prediction error signal or the input moving image signal 100 is selected by the mode selection switch 102, and orthogonal transform, for example, discrete cosine transform (DCT) is performed by the orthogonal transformer 103. The orthogonal transformer 103 obtains orthogonal transform coefficient information, for example, DCT coefficient information. The orthogonal transform coefficient information is quantized by the quantizer 104 and then bifurcated. One of the bifurcated quantized orthogonal transform coefficient information 210 is guided to the variable length encoder 215.

  The other of the bifurcated quantized orthogonal transform coefficient information 210 is subjected to sequential processing reverse to the processing of the quantizer 104 and the orthogonal transformer 103 by the inverse quantizer 105 and the inverse orthogonal transformer 106 to obtain a prediction error signal. After the same signal is obtained, the adder 107 adds the predicted image signal 212 input via the switch 109 to generate a locally decoded image signal 211. The locally decoded image signal 211 is input to the frame memory / predicted image generator 108.

  The frame memory / predicted image generator 108 selects one combination from a plurality of combinations of reference frame numbers and prediction parameters prepared in advance. For the image signal (local decoded image signal 211) of the reference frame indicated by the reference frame number in the selected combination, a linear sum is calculated according to the prediction parameter in the selected combination, and further according to the prediction parameter By adding the offset, in this example, a reference image signal for each frame is generated. Thereafter, the frame memory / predicted image generator 108 performs motion compensation on the reference image signal using a motion vector, and generates a predicted image signal 212.

  In this process, the frame memory / predicted image generator 108 generates motion vector information 214 and index information 215 indicating the selected combination of the reference frame number and the prediction parameter, and further selects a coding mode in the mode selector 212. Send necessary information to. The motion vector information 214 and the index information 215 are input to the variable length encoder 111. The frame memory / predicted image generator 108 will be described in detail later.

  The mode selector 110 selects a coding mode in units of macroblocks based on the prediction information P from the frame memory / predictive image generator 108, that is, intraframe coding (hereinafter referred to as intra coding) and a motion compensated prediction frame. One of inter coding (hereinafter referred to as inter coding) is selected, and switch control signals M and S are output.

  In the intra coding mode, the switches 102 and 112 are switched to the A side by the switch control signals M and S, and the input moving image signal 100 is input to the orthogonal transformer 103. In the inter coding mode, the switches 102 and 112 are switched to the B side by the switch control signals M and S, the orthogonal transformer 103 generates a prediction error signal from the subtractor 102, and the adder 107 generates a frame memory / predicted image. The predicted image signals 212 from the unit 108 are respectively input. Mode information 213 is output from the mode selector 212 and input to the variable length encoder 111.

  In the variable length encoder 111, the orthogonal transform coefficient information 210, the mode information 213, the motion vector information 214, and the index information 215 are variable length encoded, and each variable length code generated thereby is multiplexed by the multiplexer 114. Then, the output buffer 115 smoothes the result. Thus, the encoded data 116 output from the output buffer 115 is sent to a transmission system or storage system (not shown).

  The encoding controller 113 controls the encoding unit 112 composed of elements from the subtractor 101 to the variable length encoder 111, specifically monitors the buffer amount of the output buffer 115, for example, and the buffer amount is constant. The encoding parameters such as the quantization step size of the quantizer 104 are controlled so that

(About frame memory / predicted image generator 108)
FIG. 2 shows a detailed configuration of the frame memory / predicted image creator 108 in FIG. In FIG. 2, the local decoded image signal 211 input from the adder 107 in FIG. 1 is stored in the frame memory set 202 under the control of the memory controller 201. The frame memory set 202 includes a plurality (N) of frame memories FM1 to FMN for temporarily storing the locally decoded image signal 211 as a reference frame.

  The prediction parameter controller 203 prepares a plurality of combinations of reference frame numbers and prediction parameters as a table in advance, and the reference frame number of the reference frame used for generating the predicted image signal 212 based on the input moving image signal 100 and the prediction A combination of parameters is selected, and index information 215 indicating the selected combination is output.

  The multi-frame motion evaluator 204 creates a reference image signal according to the combination of the reference frame number selected by the prediction parameter controller 203 and the index information, and the amount of motion and the prediction error are determined from the reference image signal and the input image signal 100. Evaluation is performed, and motion vector information 214 that minimizes the prediction error is output. The multi-frame motion compensator 205 generates a predicted image signal 212 by performing motion compensation on the reference image signal selected for each block by the multi-frame motion evaluator 204 according to the motion vector.

(Prediction image generation)
The following formulas (1), (2), and (3) show examples of prediction formulas prepared by the prediction parameter controller 203 using the reference image number and the prediction parameters. In the example shown here, a case where a prediction image signal is generated by motion compensation prediction using one reference image (reference picture) with respect to an encoding target image called a so-called P picture, and a so-called B picture code 7 shows a prediction formula applied when a predicted image signal is generated by motion compensation prediction using only one of two reference images with respect to a target image.

Here, Y is a predicted image signal of a luminance signal, Cb and Cr are predicted image signals of two color difference signals, and R _Y (i), R _Cb (i), and R _Cr (i) are reference image signals of index i. The pixel values of the luminance signal and the two color difference signals are respectively shown. D ₁ (i) and D ₂ (i) are the prediction coefficient and offset of the luminance signal of index i, respectively. E ₁ (i) and E ₂ (i) are the prediction coefficient and offset of the color difference signal Cb of index i, respectively. F ₁ (i) and F ₂ (i) are the prediction coefficient and offset of the color difference signal Cr of index i, respectively. The index i takes a value from 0 to (the maximum number of reference images−1), is encoded for each encoding target block (for example, for each macroblock), and is transmitted to the video decoding device.

The prediction parameters D ₁ (i), D ₂ (i), E ₁ (i), E ₂ (i), F ₁ (i), and F ₂ (i) are preliminarily determined between the moving picture coding apparatus and the decoding apparatus. Or a predetermined encoding unit such as a frame, a field, and a slice, which are encoded together with encoded data and transmitted from the moving image encoding apparatus to the decoding apparatus. Shared on.

  Equations (1), (2), and (3) avoid the division by selecting the denominator of the prediction coefficient to be multiplied by the reference image signal as a power of 2, that is, 2, 4, 8, 16,. It is a prediction formula that can be calculated by This avoids an increase in calculation cost due to division.

  That is, >> in Equations (1), (2), and (3) is an operator that arithmetically shifts the integer a to the right by b bits when a >> b is set. The function clip () is a clipping function in which the value in () is 0 when it is smaller than 0, and 255 when it is larger than 255, and returns an integer value from 0 to 255.

Here, L _Y is the shift amount of the luminance signal, and L _C is the shift amount of the color difference signal. For these shift amounts L _Y and L _C , values determined in advance by the moving image encoding device and the decoding device are used, or in the moving image encoding device, predetermined encoding such as a frame, a field, or a slice is performed. By being encoded together with the table and encoded data in units and transmitted to the moving picture decoding apparatus, both apparatuses share the same.

In the present embodiment, the prediction parameter controller 203 in FIG. 2 prepares a reference image number / prediction parameter combination table as shown in FIG. This table is used when the number of reference images is one. In FIG. 3, the index i corresponds to a predicted image that can be selected for each block. In this example, there are four types of predicted images corresponding to 0 to 3 of the index i. In other words, the reference image number is a number of a locally decoded image used as a reference image. The table shown in FIG. 3 shows the prediction parameters D ₁ (i), D ₂ (i), E ₁ (i) assigned to the luminance signal and the two color difference signals corresponding to the equations (1), (2), and (3). ), E ₂ (i), F ₁ (i), and F ₂ (i).

  Flag is a flag indicating whether or not to apply a prediction formula using a prediction parameter to the reference image number indicated by the index i. If Flag is “0”, motion compensation prediction is performed using the local decoded image of the reference image number indicated by the index i without using the prediction parameter. If Flag is “1”, using the locally decoded image of the reference image number indicated by the index i and the prediction parameter, a prediction image is created according to Equations (1), (2), and (3) to perform motion compensation prediction. Also for the information of the Flag, a value determined in advance by the video encoding device and the decoding device is used, or in the video encoding device, a table and a predetermined encoding unit such as a frame, a field, or a slice are used. The data is encoded together with the encoded data and transmitted to the moving picture decoding apparatus, so that both apparatuses share the same.

  In these examples, for the reference image number 105, when the index i is i = 0, a prediction image is generated using the prediction parameter, and when i = 1, motion compensation prediction is performed without using the prediction parameter. . As described above, a plurality of prediction methods may exist for the same reference image number.

The following mathematical formulas (4), (5), and (6) are the prediction formulas of the reference image number and the prediction parameter prepared by the prediction parameter controller 203 when the prediction image signal is generated using the number of two reference images. An example is shown.

Here, since the relationship of Formula (5) is established, Formula (4) can be modified as follows.

  Here, an example of a prediction formula when bi-directional prediction is performed in the case of a so-called B picture is shown. At this time, there are two indexes i and j, and R (i) and R (j) are reference images corresponding to the respective indexes i and j. Accordingly, two pieces of information i and j are sent as index information. Here, W (j, j) is a prediction coefficient when the indexes are i and j. A function U used for calculation of the prediction coefficient W is a function representing a distance between images, and U (i, j) represents a distance between a reference image indicated by an index i and a reference image indicated by an index j. n is the position of the current image to be encoded.

In the present embodiment, it is assumed that the position information of a smaller value is stored in the past image in terms of time. Therefore, if the reference image indicated by index i is temporally later than the reference image indicated by index j, U (i, j)> 0, and index i and index j indicate the same reference image in time. U (i, j) = 0, and U (i, j) <0 if the reference image indicated by index i is temporally past the reference image indicated by index j. The value of the prediction coefficient W is 2 ^L-1 when U (i, j) is 0.

  Specifically, the temporal positional relationship between the current image to be encoded and the two reference images is expressed as shown in FIGS. 4 to 7 using index i and index j. FIG. 4 shows an example in which the encoding target image n is in a relationship between the reference image indicated by the index i and the reference image indicated by the index j.

Here, T _n , T _i , and T _j represent the position of the encoding target image, the reference image indicated by the index i, and the reference image indicated by the index j, respectively, and take a larger value toward the right here. It has become. Therefore, the relationship of T _i <T _n <T _j is established. Here, the function U used to calculate the prediction coefficient W is obtained by U (n, i) = T _n −T _i , U (j, i) = T _j −T _i , and U (n, i)> 0, U (j, i)> 0.

  FIG. 5 shows an example in which the reference image indicated by the index i and the reference image indicated by the index j are both temporally past in relation to the encoding target image n. Here, U (n, i)> 0 and U (j, i) ≦ 0.

  FIG. 6 shows another example in the case where both the reference image indicated by the index i and the reference image indicated by the index j are in a past position relationship with respect to the encoding target image n. . Here, U (n, i)> 0 and U (j, i) ≧ 0.

  FIG. 7 shows an example in which the reference image indicated by the index i and the reference image indicated by the index j are both in the future position relationship with respect to the encoding target image n. Here, U (n, i) <0 and U (j, i) ≧ 0.

  In Equations (4) to (8), L is a shift amount, and a value determined in advance between the moving image encoding device and the decoding device is used, or a predetermined encoding unit such as a frame, a field, and a slice is used. Thus, the data is encoded together with the table and the encoded data, transmitted from the encoding device to the decoding device, and shared by both devices. Furthermore, the function called clip2 in the equations (6) and (9) is a function that returns an integer by limiting the maximum value and the minimum value of the values in () of clip2 () (hereinafter simply referred to as values). Regarding this function clip2, a plurality of configuration examples are shown below.

The first configuration of the function clip2 is a clipping function to -2 ^M, when the value is greater than ^{(2 M -1) (2 M} -1) when the value is smaller than -2 ^M, -2 ^M Returns an integer value not less than (2 ^M −1). With this configuration, assuming that the pixel is 8 bits, 9 bits are required to express the value of (R (j) -R (i)), and (M + 1) bits are required to express the prediction coefficient W, so (M + 10) The predicted image value can be calculated with a bit calculation accuracy. Note that M is a non-negative integer greater than or equal to L.

The second configuration of the function clip2 has a rule of 2 ^L-1 when the value is smaller than −2 ^M and 2 ^L-1 when the value is larger than (2 ^M −1), and − A function that returns an integer value of 2 ^M or more and (2 ^M −1) or less. By adopting such a configuration, when the distance relationship between two reference images is exceptional, all can be average value prediction.

A third configuration of the function clip2, 1 when the value is less than 1, in the clipping function value to 2 ^M when larger 2 ^M, a function that returns an integer value of 1 or more 2 ^M. The difference from the first configuration of the function clip2 is that the value of the prediction coefficient W does not become negative, and the positional relationship of the reference image is more limited. Therefore, even with a combination of two identical reference images, it is possible to switch between prediction by the prediction coefficient W and average value prediction by reversing the indication of the index i and the index j as in the relationship of FIGS. It becomes.

The fourth configuration of the function clip2, 0 when the value is less than 0, with the clipping function value to 2 ^L when larger 2 ^L, a function that returns an integer value of 0 or 2 ^L. With this configuration, since the value of the prediction coefficient W is always a non-negative value of 2 ^L or less, extrapolation prediction is prohibited, and one of the two reference images is predicted in bi-directional prediction instead. Will be used.

The fifth structure of the function clip2, 2 ^L-1 when the value is less than 1, in the clipping function to 2 ^L-1 when the value is greater than 2 ^L, 1 or 2 ^L -1 or less integer A function that returns a numeric value. With such a configuration, since the value of the prediction coefficient W is always a non-negative value of 2 ^L −1 or less, extrapolation prediction is prohibited, and instead, it is used for the average value prediction of two reference images. become.

When the distance between two reference images is unknown or undefined, for example, when one or both of the reference images is a reference image for background or long-term storage, the prediction coefficient W is 2 ^L− A value of ¹ is assumed. Since the prediction coefficient W can be calculated in advance in units of encoding such as a frame, a field, and a slice, even when a predicted image signal is created with two reference images in calculation per pixel, 1 It only takes 1 multiplication.

  Expression (9) is another example obtained by modifying Expression (4). In the formula (7), an operation to shift R (i) to the left by L bits in advance is necessary, but in the formula (10), the arithmetic shift is omitted by putting it outside the parentheses. The amount of calculation can be reduced accordingly. Instead, since the rounding direction when shifting is different depending on the magnitude relationship between the values of R (i) and R (j), the result is not exactly the same as Expression (4).

Further, the following mathematical formulas (10) to (20) may be used instead of the mathematical formulas (4) to (8). This is the same method as the method of creating a predicted image in the case of one reference image. A predicted image of one reference image with index i and a predicted image of one reference image with index j are created and This is a method of creating a final predicted image by taking an average. Since it is possible to use the same processing routine as when the number of reference images is one halfway, there is an advantage that the amount of hardware and code can be reduced.

(For prediction method selection and coding mode determination procedure)
Next, an example of a specific procedure for selecting a prediction method (combination of a reference image number and a prediction parameter) for each macroblock and determining a coding mode in this embodiment will be described with reference to FIG.
First, the maximum value that can be assumed is entered in the variable min_D (step S101). LOOP1 (step S102) indicates repetition for selection of a prediction method in inter coding, and variable i indicates the value of the index shown in FIG. Here, the code amount related to the motion vector information 214 (the variable length code output from the variable length encoder 111 corresponding to the motion vector information 214 is calculated so that the optimal motion vector for each prediction method can be obtained. The evaluation value D of each index (combination of the reference frame number and the prediction parameter) is calculated from the sum of the code amount) and the prediction error absolute value, and the motion vector that minimizes the evaluation value D is selected (step S103). The evaluation value D is compared with min_D (step S104). If the evaluation value D is smaller than min_D, the evaluation value D is set to min_D, and the index i is assigned to min_i (step S105).

  Next, an evaluation value D in the case of intra coding is calculated (step S106), and this evaluation value D is compared with min_D (step S107). As a result of this comparison, if min_D is smaller, mode MODE is determined to be inter-coding, and min_i is substituted into index information INDEX (step S108). If the evaluation value D is smaller, the mode MODE is determined to be intra coding (step S109). Here, the evaluation value D is an estimated amount of code for the same quantization step size.

(About decryption side)
Next, a video decoding device corresponding to the video encoding device shown in FIG. 1 will be described. FIG. 9 shows the configuration of the video decoding apparatus according to this embodiment. The encoded data 300 sent from the moving picture encoding apparatus having the configuration shown in FIG. 1 and sent via the transmission system or the storage system is once stored in the input buffer 301 and is demultiplexed by the demultiplexer 302 every frame. Are separated based on the syntax and then input to the variable length decoder 303. The variable length decoder 303 decodes the variable length code of each syntax of the encoded data 300, and reproduces the quantized orthogonal transform coefficient, mode information 413, motion vector information 414, and index information 415.

  Of each reproduced information, the quantized orthogonal transform coefficient is inversely quantized by the inverse quantizer 304 and inversely orthogonally transformed by the inverse orthogonal transformer 305. Here, when the mode information 413 indicates the intra coding mode, a reproduced image signal is output from the inverse orthogonal transformer 305 and is output as a final reproduced image signal 310 via the adder 306. When the mode information 413 indicates the inter coding mode, the prediction error signal is output from the inverse orthogonal transformer 305, and the mode selection switch 308 is turned on. The prediction image signal 412 output from the prediction error signal and the frame memory / prediction image generator 308 is added by the adder 306, whereby the reproduced image signal 310 is output. The reproduced image signal 310 is stored in the frame memory / predicted image generator 308 as a reference image signal.

  The mode information 413, motion vector information 414, and index information 415 are input to the frame memory / predicted image creator 308. The mode information 413 is also input to the mode selection switch 309. The switch 309 is turned off in the intra coding mode and turned on in the inter coding mode.

  The frame memory / predicted image generator 308 prepares a plurality of combinations of reference image numbers and prediction parameters prepared in advance as a table, similar to the frame memory / predicted image generator 108 on the encoding side shown in FIG. From this, one combination indicated by the index information 415 is selected. For the image signal of the reference image (reproduced image signal 210) indicated by the reference image number in the selected combination, a linear sum is calculated according to the prediction parameter in the selected combination, and an offset according to the prediction parameter is further calculated. By adding, a reference image signal is generated. Thereafter, the predicted image signal 412 is generated by performing motion compensation on the generated reference image signal using the motion vector indicated by the motion vector information 414.

(About frame memory / predicted image generator 308)
FIG. 10 shows a detailed configuration of the frame memory / predicted image creator 308 in FIG. In FIG. 10, the reproduced image signal 310 output from the adder 306 in FIG. 9 is stored in the frame memory set 402 under the control of the memory controller 401. The frame memory set 402 includes a plurality (N) of frame memories FM1 to FMN for temporarily holding the reproduced image signal 310 as a reference image.

  The prediction parameter controller 403 prepares combinations of reference image numbers and prediction parameters in advance as a table similar to that shown in FIG. 3, and performs prediction based on the index information 415 from the variable length decoder 303 in FIG. A combination of a reference image number of a reference image and a prediction parameter used for generating the image signal 412 is selected. The multi-frame motion compensator 404 creates a reference image signal according to the combination of the reference image number and the index information selected by the prediction parameter controller 403, and the variable length decoder 303 in FIG. The predicted image signal 412 is generated by performing motion compensation in units of blocks in accordance with the motion vector indicated by the motion vector information 414 from.

(About index information syntax)
FIG. 11 shows an example of syntax when index information is encoded in each block. First, mode information MODE exists for each block. Whether or not to encode the index information IDi indicating the value of the index i and the index information IDj indicating the value of the index j is determined according to the mode information MODE. After the encoded index information, motion vector information MVi for motion compensated prediction of index i and motion vector information MVj for motion compensated prediction of index j are encoded as motion vector information of each block. .

(Data structure of encoded bit stream)
FIG. 12 illustrates an example of a specific encoded bit stream for each block when a predicted image is generated using one reference image. Following the mode information MODE, index information IDi is arranged, and then motion vector information MVi is arranged. The motion vector information MVi is usually two-dimensional vector information. However, depending on the motion compensation method inside the block indicated by the mode information, a plurality of two-dimensional vectors may be sent.

FIG. 13 shows an example of a specific encoded bit stream for each block when a predicted image is created using two reference images. Following the mode information MODE, index information IDi and index information IDj are arranged, followed by motion vector information MVi and motion vector information MVj. The motion vector information MVi and the motion vector information j are usually two-dimensional vector information, but a plurality of two-dimensional vectors may be sent depending on the motion compensation method inside the block indicated by the mode information. .
(Summary)
As described above, according to the present embodiment, when a predicted image is generated using one reference image, a predicted image is generated by performing linear prediction using a prediction coefficient and an offset as a prediction parameter. By this method, an appropriate predicted image can be created for a feed image that is a mixed video with a single color screen. In the method of simply selecting one combination for each coding target block from a plurality of combinations of reference picture numbers and prediction parameters, when there are a plurality of reference pictures, the number of multiplications per pixel is also required multiple times. This increases the amount of calculation, but in the present embodiment, only one multiplication is required per pixel.

  On the other hand, when a predicted image is created using two reference images, a prediction is performed by performing a weighted average of the two reference images using a weighting coefficient that can be obtained from the distance between the two reference images. Create an image. By this method, an appropriate predicted image can be created for a dissolve image in which two videos are mixed. At this time, if the mathematical formula used in the present embodiment is used, the necessary number of multiplications is one per pixel.

  As described above, according to the present embodiment, an appropriate predicted image can be created by multiplying once per pixel for both the feed video and the dissolve video. Since only one multiplication per pixel is required, it is possible to reduce the hardware scale and calculation cost on both the encoding side and the decoding side.

  In the above description, the method of creating a predicted image is switched depending on the number of reference images. However, the method of creating a predicted image may be switched in units of pictures or units of slices depending on a prediction type called a so-called picture type or slice type. . For example, when only one of the reference images is used in the case of a B picture, a motion compensated prediction using a normal local decoded image is performed without generating a predicted image using a prediction parameter.

  A predicted image creation procedure using a method of switching the predicted image creation method depending on the prediction type in addition to the number of reference images will be specifically described with reference to FIG. In this example, the prediction image creation method is switched in units of slices.

  First, a prediction type (referred to as a slice type) of an encoding target slice that is an encoding target region is determined, and an I (intraframe prediction) slice that performs intraframe encoding on the encoding target slice and one reference image are used. The P (one-way prediction) slice that performs prediction and the B (bidirectional prediction) slice that performs prediction using a maximum of two reference images are branched (step S201).

  If the result of determination in step S201 is that the current slice is an I slice, intraframe coding (intra coding) is performed (step S202). When the encoding target slice is a P slice, a prediction method based on a combination of one reference image and a prediction parameter described above is employed (step S203).

  When the encoding target slice is a B slice, the number of reference images is checked (step S204), and the prediction method is switched accordingly. That is, when the encoding target slice is a B slice and there is one reference image, normal motion compensation prediction is employed (step S205). When the encoding target slice is a B slice and there are two reference images, the prediction method according to the inter-image distance between the two selected reference images described above is employed (step S206).

[Second Embodiment]
Next, a second embodiment of the present invention will be described. Since the overall configuration of the video encoding device and video decoding device in the present embodiment is substantially the same as that of the first embodiment, only differences from the first embodiment will be described. In this embodiment, an example in which the first embodiment is combined with another method is shown.

The following formula (21) is a prediction formula of a prediction image in the case of so-called B picture bi-directional prediction using two reference images, and is a first equation that simply averages the motion compensated prediction images of two reference images. It is a technique.

  In this first method, the prediction formula shown in any one of the formulas (4) to (6), the formulas (7) to (8), the formula (9), or the formulas (10) to (20), and the formula ( The prediction formula and the switching information shown in 21) are encoded in a predetermined encoding unit such as a picture, a frame, a field, and a slice, and a switching flag is encoded together with the encoded data, and transmitted from the moving picture encoding apparatus to the decoding apparatus. And be shared by both devices. That is, the prediction formula shown in any one of Formulas (4) to (6), Formulas (7) to (8), Formula (10), or Formulas (10) to (20) and Formula (21) as necessary. ) Is switched to the prediction formula shown in FIG.

  According to this first method, it is possible to adaptively switch between the weighted average according to the inter-image distance and the simple average of the two reference images, and an improvement in prediction efficiency can be expected. Since Equation (21) does not include multiplication, the amount of calculation does not increase.

The following mathematical formulas (22) to (27) and mathematical formulas (28) to (33) show a method of creating a prediction parameter when there are two reference images using a prediction parameter when there is one reference image. In the present embodiment, an example in which the first embodiment and these methods are combined is shown. First, Equations (22) to (27) are a second method for obtaining a prediction value by averaging the values of the prediction equations when there is one reference image.

Here, P _Y (i), P _Cb (i), and P _Cr (i) are intermediate results of predicted values of the luminance signal Y, the color difference signal Cb, and the color difference signal Cr, respectively.
In the second method, the prediction formula shown in any one of Formulas (4) to (6), Formulas (7) to (8), Formula (9), or Formulas (10) to (20), and Formula ( The switching information with the prediction formulas shown in (22) to (27) is encoded in units of predetermined encoding such as a picture, a frame, a field, and a slice, and a switching flag is encoded together with the encoded data from the moving picture encoding apparatus. It is transmitted to the decryption device so that it can be shared by both devices. As described above, the prediction formula shown in any one of Formulas (4) to (6), Formulas (7) to (8), Formula (9), or Formulas (10) to (20), and Formula ( The prediction formulas shown in 22) to (27) are switched.

  By this second method, it is possible to adaptively switch the prediction image based on the linear prediction using the weighted average according to the inter-image distance and the two reference images, and an improvement in prediction efficiency can be expected. However, according to the prediction formulas shown in Equations (22) to (27), the number of multiplications per pixel is two, but there is an advantage that the degree of freedom of the prediction coefficient is increased, and the prediction efficiency is further improved. I can expect.

The following mathematical formulas (28) to (33) show examples of linear prediction formulas for two reference images created using two prediction parameters for one reference image as other prediction formulas.

  In the third method, the prediction formula shown in any one of Formulas (4) to (6), Formulas (7) to (8), Formula (9), or Formulas (10) to (20), and Formula ( 28) to (33) are used to encode switching information together with encoded data in a predetermined encoding unit such as a picture, a frame, a field, and a slice. Is transmitted to the decryption device and can be shared by both devices, as necessary, Formula (4) to (6), Formula (7) to (8), Formula (9), or Formula (10) to (10) 20) and the prediction formulas shown in the mathematical formulas (28) to (33) are switched.

  According to the third method, it is possible to adaptively switch the prediction image based on the linear prediction using the weighted average according to the inter-image distance and the two reference images, and an improvement in prediction efficiency can be expected. However, according to the prediction formulas of Equations (28) to (33), the number of multiplications per pixel is 2, but there is an advantage that the degree of freedom of the prediction coefficient is increased, and further improvement in prediction efficiency can be expected. .

  In the above-described embodiment, the example of the moving image encoding / decoding method using the orthogonal transform in units of blocks has been described. However, the above-described implementation is also performed when another transform method such as a wavelet transform is used. The method of the present invention described in the embodiment can be similarly applied.

  The moving image encoding and decoding processes according to the present invention may be realized as hardware (apparatus) or may be executed by software using a computer. Some processing may be realized by hardware, and other processing may be performed by software. Therefore, according to the present invention, it is possible to provide a program for causing a computer to perform the above-described moving image encoding or decoding process or a storage medium storing the program.

The block diagram which shows the structure of the moving image encoder which concerns on the 1st Embodiment of this invention. FIG. 2 is a block diagram showing a detailed configuration of the frame memory / predictive image creator in FIG. The figure which shows the example of the combination table of the reference image number and prediction parameter which are used in the embodiment The figure which shows the 1st positional relationship between two reference images and the encoding object image in the embodiment. The figure which shows the 2nd positional relationship between two reference images and the encoding object image in the embodiment. The figure which shows the 3rd positional relationship between two reference images and the encoding object image in the embodiment. The figure which shows the 4th positional relationship between two reference images and the encoding object image in the embodiment. The flowchart which shows an example of the procedure of selection of the prediction method (combination of a reference image number and a prediction parameter) for every macroblock and encoding mode determination in the embodiment The block diagram which shows the structure of the moving image decoding apparatus which concerns on the same embodiment The block diagram which shows the detailed structure of the frame memory / predictive image generator in FIG. The figure which shows the example of the syntax for every block in the case of encoding index information. The figure which shows the example of the concrete encoding bit stream in the case of producing a predicted image using one reference image The figure which shows the example of the concrete encoding bit stream in the case of producing a predicted image using two reference images The flowchart which shows the procedure which switches a prediction system by the kind of encoding object area | region which concerns on embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Input moving image signal 101 ... Subtractor 102, 109 ... Mode selection switch 103 ... Orthogonal transformer 104 ... Quantizer 105 ... Inverse quantizer 106 ... Inverse orthogonal transformer 107 ... Adder 108 ... Frame memory / predicted image Generator 110 ... Mode selector 111 ... Variable length encoder 112 ... Encoder 113 ... Encoding controller 114 ... Multiplexer 115 ... Output buffer 116 ... Encoded data 201 ... Memory controller 202 ... Multiple frame memory 203 ... prediction parameter controller 204 ... multiple frame motion estimator 205 ... multiple frame motion compensator 211 ... local decoded image signal 212 ... predicted image signal 213 ... mode information 214 ... motion vector information 215 ... index information 300 ... encoded data 301 ... Input buffer 302 ... Demultiplexer 303 ... Variable length Encoder 304 ... Inverse quantizer 305 ... Inverse orthogonal transformer 306 ... Adder 307 ... Frame memory / predicted image generator 308 ... Adder 309 ... Mode changeover switch 310 ... Reproduced image signal 401 ... Memory controller 402 ... Multiple Frame memory 403 ... Prediction parameter controller 404 ... Multiple frame motion compensator 412 ... Prediction image signal 413 ... Mode information 414 ... Motion vector information 415 ... Index information

Claims (2)

An image decoding method for performing motion compensation predictive decoding on a decoding target block of a decoding target image,
A first step of decoding encoded data of a decoding target image to obtain a quantized orthogonal transform coefficient, motion vector information, and index information;
A first distance between the first reference image indicated by the index information and the decoding target image and a second distance between the first reference image and the second reference image indicated by the index information are obtained. A second step;
A third step of determining a first weighting factor for the first reference image and a second weighting factor for the second reference image based on a ratio of the first distance to the second distance;
An image of an area specified by the motion vector information in the first reference image and an image of an area specified by the motion vector information in the second reference image according to the first weight coefficient and the second weight coefficient A fourth step of calculating a linear sum of and generating a predicted image;
A fifth step of obtaining a prediction residual by inverse quantization and inverse orthogonal transform of the quantized orthogonal transform coefficient information;
A sixth step of adding the prediction residual and the prediction image to generate a reproduced image signal;
Have
In the third step, when the second weighting factor is larger than the upper limit, or when the second weighting factor is smaller than the lower limit, both values of the first weighting factor and the second weighting factor are set. Even when the average value is set to a value for prediction and at least one of the first image and the second image is a long-term reference image, both values of the first weighting factor and the second weighting factor are used. Is set to the value for average value prediction, The moving picture decoding method characterized by the above-mentioned. An image decoding apparatus that performs motion compensation predictive decoding on a decoding target block of a decoding target image,
First means for decoding encoded data of an image to be decoded and obtaining quantized orthogonal transform coefficients, motion vector information, and index information;
A first distance between the first reference image indicated by the index information and the decoding target image and a second distance between the first reference image and the second reference image indicated by the index information are obtained. Two means,
Third means for determining a first weighting factor for the first reference image and a second weighting factor for the second reference image based on a ratio of the first distance to the second distance;
An image of an area specified by the motion vector information in the first reference image and an image of an area specified by the motion vector information in the second reference image according to the first weight coefficient and the second weight coefficient A fourth means for calculating a linear sum of and generating a predicted image;
A fifth means for dequantizing and inverse orthogonal transform the quantized orthogonal transform coefficient information to obtain a prediction error;
Sixth means for adding the prediction residual and the prediction image to generate a reproduced image signal;
Have
The third means averages the values of both the first weighting factor and the second weighting factor when the second weighting factor is larger than the upper limit or when the second weighting factor is smaller than the lower limit. Even when at least one of the first image and the second image is a long-term reference image, the values of both the first weight coefficient and the second weight coefficient are set to values for value prediction. A moving picture decoding apparatus characterized in that it is set to a value for average value prediction.

Images