JP2012124542A - Moving image encoding method and moving image decoding method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving image encoding technology and a moving image decoding technology for supplying a video of high image quality with a small code amount.SOLUTION: An encoding stream is inputted, and a prediction mode of a block being a decoding object, which is included in the encoding stream, is discriminated. An estimation index value of a prediction error of the block of the decoding object is calculated based on a previously decoded image by a calculation system corresponding to the discriminated prediction mode. A decoding table used for a variable length decoding processing is determined from a plurality of decoding tables based on the calculated estimation index value. A variable length decoding processing is performed on data of the encoding stream based on the determined decoding table. An inverse quantization processing and an inverse frequency conversion processing are performed on data on which the variable length decoding processing is performed, and a prediction difference is decoded. A decoding image is generated based on the decoded prediction difference and a predicted image generated by a prediction processing by the discriminated prediction mode.

Description

The present invention relates to a moving image encoding technique for encoding a moving image and a moving image decoding technique for decoding an encoded moving image.

  Encoding methods such as MPEG (Moving Picture Experts Group) method have been established as a method for recording and transmitting large-capacity moving image information as digital data, and MPEG-1 standard, MPEG-2 standard, MPEG-4 standard, It is an international standard encoding method such as H.264 / AVC (Advanced Video Coding) standard. These systems have been adopted as encoding systems for digital satellite broadcasting, DVDs, mobile phones, digital cameras, and the like, and the range of use is now expanding and becoming familiar.

  In these standards, the encoding amount is reduced by predicting the encoding target image in units of blocks using the image information for which the encoding process has been completed, and variable-length encoding the prediction difference from the original image.

  In particular, H.264 / AVC employs an intra-screen predictive coding scheme that uses the surrounding pixels of the encoding target block and an inter-screen predictive encoding scheme that uses the pixels of the screen before and after the encoding target block. Only the prediction difference is encoded.

  However, since the code table in the variable length coding is created on the assumption that the prediction difference is small, there is a problem that the prediction error increases and the code amount becomes enormous if the prediction accuracy is low. .

  As an attempt to solve this problem, paragraph 0007 of Patent Document 1 states that “based on the number of non-zero coefficients included in an encoded block located around the encoding target block, A prediction step for calculating a prediction value for the number of coefficients other than 0 included in the table, a table selection step for selecting a variable length coding table based on the prediction value calculated by the prediction step, and the table selection A variable length coding step for performing variable length coding on the number of non-zero coefficients included in the coding target block with reference to the variable length coding table selected in the step. It is described that it is constituted as follows.

JP2008-193725

  However, the technique disclosed in Patent Document 1 is intended to perform variable-length coding processing with high efficiency for “the number of coefficients other than 0 included in a block to which orthogonal transform is applied”. There was a problem that highly efficient variable-length encoding processing could not be performed.

  The present invention has been made in view of the above problems, and an object thereof is to provide a moving picture encoding technique and a moving picture decoding technique that provide high-quality video with a small code amount.

  In one embodiment of the present invention, for example, an encoded stream is input, a prediction mode of a decoding target block included in the encoded stream is determined, and already decoded by a calculation method corresponding to the determined prediction mode An estimated index value of a prediction error of a decoding target block is calculated based on the image of, and a decoding table used for variable length decoding processing is determined from a plurality of decoding tables based on the calculated estimated index value, Based on the determined decoding table, the encoded stream data is subjected to variable length decoding processing, the variable length decoding processing data is subjected to inverse quantization processing and inverse frequency transform processing to decode the prediction difference, A decoded image is generated based on the decoded prediction difference and the predicted image generated by the prediction process in the determined prediction mode.

  According to the present invention, it is possible to provide a moving picture encoding technique and a moving picture decoding technique that provide high-quality video with a small code amount.

Example of block diagram of image coding apparatus according to embodiment 1 Example of block diagram of variable-length encoding unit according to embodiment 1 Example of block diagram of image decoding apparatus according to embodiment 1 Example of block diagram of variable length decoding unit according to embodiment 1 Conceptual explanatory diagram of an example of inter-screen prediction according to the first embodiment. Conceptual explanatory diagram of an example of intra prediction according to the first embodiment. Conceptual illustration of H.264 standard variable length coding Conceptual explanatory diagram of an example of a prediction error estimation index value calculation process for inter-screen prediction according to the first embodiment Explanatory drawing of an example of the prediction error estimation index value calculation process for the inter-screen prediction which concerns on Example 1. FIG. Explanatory drawing of an example of the prediction error estimation index value calculation process for inter-screen prediction which concerns on Example 2. FIG. Explanatory drawing of an example of the prediction error estimation index value calculation process for the prediction in a screen which concerns on Example 1. FIG. Explanatory drawing of an example of the prediction error estimation index value calculation process for the prediction in a screen which concerns on Example 2. FIG. Explanatory drawing of an example of the determination method of the encoding / decoding table which concerns on Example 1. FIG. An example of a flowchart of an image encoding method according to the first embodiment Example of flowchart of image decoding method according to embodiment 1 Example of flowchart of variable length coding method according to embodiment 1 Example of flowchart of variable length decoding method according to embodiment 1 Explanatory drawing of an example of the encoding table which concerns on Example 1. FIG.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 shows an example of a moving image encoding apparatus according to the first embodiment. The moving image encoding apparatus performs an intra-screen prediction in units of blocks, an input image memory (102) that holds an input original image (101), a block division unit (103) that divides the input image into small regions. An intra-screen prediction unit (105) that generates a predicted image and an inter-screen prediction unit that calculates a motion vector by performing a motion search on the target block, and generates a predicted image by an inter-screen prediction process based on the motion vector (106), a mode determination unit (107) for determining a prediction mode (prediction method and block size) suitable for the nature of the image, and outputting a prediction image of the determined prediction mode, and a prediction mode determined from the input image A subtractor (108) that subtracts the predicted image to generate a prediction difference, a frequency conversion unit (109) and a quantization unit (110) that perform encoding on the prediction difference, and performs variable-length encoding processing Variable length encoder (111), frequency conversion and quantization An inverse quantization processing unit (112) and an inverse frequency transform unit (113) for decoding the predicted prediction difference, and an adding unit for generating a decoded image using the decoded prediction difference and the predicted image of the determined prediction mode (114) and a reference image memory (104) that holds a decoded image for use in later prediction processing. Hereinafter, detailed operations of each unit will be described.

  The input image memory (102) holds one image from the original image (101) as an encoding target image. The block dividing unit (103) divides the original image into a plurality of blocks, and outputs them to the intra-screen prediction unit (105) and the inter-screen prediction unit (106). The inter-screen prediction unit (106) calculates the motion vector of the target block by motion search using the decoded image stored in the reference image memory (117), and executes the inter-screen prediction process on a block basis .

  Here, details of the inter-screen prediction processing by the inter-screen prediction unit (106) will be described with reference to FIG. The inter-screen prediction unit (106) performs encoding in units of blocks on the encoding target image in the raster scan order. When performing inter-screen prediction, the decoded image of the encoded image included in the same video (501) as the encoding target image (503) is set as the reference image (502), and the target block (504) in the target image Motion search is performed to search a reference block (predicted image) (505) having a high correlation with the reference image from the reference image. A motion vector is calculated as a search result, and a prediction image (505) used for encoding the target block (504) is determined.

  In FIG. 1, the inter-screen prediction unit (106) outputs the motion vector to the variable length coding unit (111), and outputs the predicted image for the target block to the mode determination unit (107).

  The intra-screen prediction unit (105) performs intra-screen prediction processing in units of blocks. Details of the intra-screen prediction processing by the intra-screen prediction unit (105) will be described with reference to FIG. The intra prediction unit (105) performs prediction using the decoded images of the encoded blocks adjacent to the left, upper left, upper, and upper right of the encoding target block. In particular, for prediction, 13 pixels included in these blocks are used as reference pixels, and all pixels on the same straight line having a prediction direction vector as an inclination are predicted from the same pixel (601). Further, as shown in the description (602), the optimum prediction direction is selected in units of blocks from eight types of prediction direction candidates such as vertical, horizontal, and diagonal directions. That is, the prediction in the in-screen prediction process is to generate the prediction image of the encoding target block by copying the pixel value of the pixel of the adjacent encoded block to the pixel in the selected prediction direction. . In addition to the prediction along the above eight types of directions, the 13 pixels are encoded by the average value of the pixel values of the 8 pixels belonging to the block adjacent to the target block above or to the left of the reference pixels. “DC prediction” for predicting all pixels included in the conversion target block can also be used (602). In this case, all the pixel values of the predicted image are average values of the pixel values of the eight pixels.

  In FIG. 1, an intra-screen prediction unit (105) uses variable-length codes for information on the type of prediction method used in the intra-screen prediction process (for example, information indicating the type of intra-screen prediction mode such as prediction direction or DC prediction). And outputting the prediction image for the target block to the mode determining unit (107).

  Next, in FIG. 1, the mode determination unit (107) determines an optimal prediction mode based on the result of the intra-screen prediction process or the inter-screen prediction process. The subtraction unit (108) subtracts the prediction image in the prediction mode determined from the target block image of the original image to generate a prediction difference, and outputs the prediction difference to the frequency conversion unit (109). The frequency conversion unit (109) performs frequency conversion processing such as DCT (Discrete Cosine Transformation) on the acquired prediction difference. Subsequently, the quantization processing unit (110) performs quantization processing and outputs the result to the variable length coding processing unit (111) and the inverse quantization processing unit (112). Further, the variable length coding processing unit (111) calculates a prediction error estimation index value based on the pixel values of the pixels of the peripheral block of the target block, and determines a coding table based on the prediction error estimation index value. Next, using the determined coding table, variable length information necessary for decoding, such as quantized prediction difference information, prediction direction used in intra prediction processing, and motion vector used in inter prediction processing Encode to generate an encoded stream. Details of the processing of the variable length coding processing unit (111) will be described later.

  The inverse quantization processing unit (112) performs an inverse quantization process on the quantized frequency transform coefficient of the prediction difference information, and the inverse frequency transform unit (113) performs IDCT (Inverse DCT: Inverse DCT). ) And the like, and a prediction difference is generated and output to the adding unit (114). Subsequently, the adding unit (114) adds the prediction difference to the prediction image in the prediction mode in which the prediction difference is determined, and generates a decoded image. The generated decoded image is stored in the reference image memory (104).

  Next, details of the processing of the variable length coding processing unit (111) will be described in comparison with the conventional variable length coding processing of the H.264 standard.

  First, variable length coding processing (CAVLC) of the H.264 standard will be described with reference to FIG. In H.264 / AVC, a coefficient value obtained by performing frequency conversion processing (such as DCT conversion) and quantization processing on a prediction difference generated by inter-screen prediction processing or intra-screen prediction processing is encoded by a zigzag scan. Convert to a dimensional array. Next, the six types of information shown in (1) to (6) included in the one-dimensional array are subjected to variable length encoding using, for example, an encoding table having the illustrated characteristics.

  On the other hand, FIG. 2 shows an example of a detailed configuration of the variable length coding unit (111) of the moving picture coding apparatus according to the present embodiment. The variable length coding unit (111) includes, for example, a prediction error estimation index value calculation unit (201), a table determination unit (202), and a coding unit (203). A prediction error estimation index value calculation unit (201) obtains pixel values of encoded peripheral blocks adjacent to the encoding target block from the reference image memory (104), and estimates prediction errors for the encoding target block. Calculate the value. The prediction error estimation index value calculation unit (201) acquires information on the prediction mode determined from the mode determination unit (107), and predicts depending on whether the determined prediction mode is the intra prediction mode or the inter prediction mode. Switches the type of error estimation index value calculation processing. The prediction error estimation index value calculation processing is performed by acquiring information on the type of the prediction method described above from the intra-screen prediction unit (105) and acquiring the motion vector described above from the inter-screen prediction unit (106). Details of the prediction error estimation index value calculation process according to the prediction mode will be described later. Based on the prediction error estimation index value calculated by the prediction error estimation index value calculation unit (201), the table determination unit (202) is used for variable length encoding processing from among a plurality of encoding tables held in advance. Determine the encoding table. The encoding unit (203) performs variable length encoding processing on the quantized coefficients acquired from the quantization processing unit (110) using the determined encoding table. The encoding unit (203) acquires information on the type of the prediction method described above from the intra-screen prediction unit (105), and acquires the motion vector described above from the inter-screen prediction unit (106). The encoding may be performed using an encoding table other than the encoding table determined by the table determination unit (202), or other encoding methods.

  Here, an example of a coding table determination method by the above-described variable length coding unit (111) will be described with reference to FIGS.

  First, the variable length coding unit (111) holds, for example, a plurality of coding tables having different characteristics as shown in FIG. Here, the table A in FIG. 18 has a characteristic that the code length is relatively small when the coefficient value of the quantized prediction difference is small, but the code length is relatively large when the coefficient value is large. It is a table. In contrast, the table C is a table having a characteristic that the code length does not become relatively large even when the coefficient value is large, but the code length does not become relatively small even when the coefficient value is small. Table B is a table having characteristics between Table A and Table C.

  Here, for example, when the prediction error estimation index value calculated by the prediction error estimation index value calculation unit (201) is X and X is equal to or less than a predetermined threshold value S1, the table determination unit (202) performs table A in FIG. Select. In this case, by using the table A in the encoding process of the encoding unit (203), the code length can be made smaller than when the tables B and C are used, and the code amount can be reduced.

  Further, the table determination unit (202) selects the table C in FIG. 13 when X is larger than the predetermined threshold value S2. In this case, by using the table C in the encoding process of the encoding unit (203), the code length can be made smaller than when the tables A and B are used, and the code amount can be reduced.

  Similarly, the table determination unit (202) selects the table B in FIG. 13 when X is larger than the predetermined threshold value S1 and equal to or smaller than the threshold value S2. In this case, by using the table B in the encoding process of the encoding unit (203), the code length can be made smaller than when the tables A and C are used, and the code amount can be reduced.

  Note that the table selected in the above procedure can be applied to any information as long as it relates to the prediction error. For example, the six items shown in (1) to (6) of FIG. Of these, (2) when applied to DCT coefficient values, etc., it is particularly effective in reducing variable length and code amount.

  Next, a method for calculating a prediction error estimation index value used for determining the above coding table will be described.

  First, as described above, the prediction error estimation index value calculation unit (201) acquires information on the prediction mode determined from the mode determination unit (107), and whether the determined prediction mode is the intra prediction mode or the inter prediction mode. Depending on the type, the type of prediction error estimation index value calculation processing is switched.

  Here, when the determined prediction mode is inter-screen prediction, prediction error estimation index value calculation processing for inter-screen prediction shown in FIGS. 8 and 9 is performed. Hereinafter, FIGS. 8 and 9 will be described.

  As shown in FIG. 8, the prediction error estimation index value calculation process for inter-screen prediction belongs to the encoded pixel region belonging to the peripheral block of the target block (801) and the peripheral block of the predicted image (802). A difference value from the encoded pixel region is calculated as a prediction error estimation index value. A specific example of the calculation process is shown in FIG.

  FIG. 9 shows an example of prediction error estimation index value calculation processing for inter-screen prediction according to the present embodiment. In general, the spatial correlation of images is high. That is, the target block and the peripheral area of the target area are similar. Therefore, by comparing the surrounding area of the target block with the image of the surrounding area of the reference block obtained by motion search, the correlation between the target block and the reference block image is estimated, that is, the index value of the size of the prediction error is estimated. Is possible.

  For example, in the example of FIG. 9, the decoded image of the area in the encoded block adjacent to the left, upper left, upper, and upper right of the encoding target block, and the code adjacent to the left, upper left, upper, and upper right of the predicted image A difference value from the decoded image of the area in the segmented block is used as an estimation value of the prediction error.

  Here, the reason for using the encoded region encoded before the target block located on the left, upper left, upper, upper right in the adjacent block is that the decoding side performs the same prediction error estimation index value calculation process. This is because the pixel cannot be used unless it is decoded before the target block.

  Here, in the example of FIG. 9, in the calculation of the estimated value of the prediction error, the pixel located at the right end of the block adjacent to the left of the target block and the upper left, upper, and upper right of the target block in the target image Thirteen pixels including the pixel at the lower end of the block to be processed are set as peripheral region pixels. Further, in the reference image, a pixel at a position corresponding to the 13 peripheral region pixels is set as a peripheral region pixel. The value N obtained by adding the differences between the corresponding pixels of the peripheral region pixel in the target image and the peripheral region pixel in the reference image for all the pixels in the peripheral region pixel is divided by the quantization step value of the target block. Then, a value Error obtained by a formula (901) obtained by multiplying a predetermined coefficient α (α is larger than 0) and adding a predetermined value β is set as a prediction error estimation index value. The quantization step value may be acquired from the quantization processing unit (110).

  In the example of FIG. 9, when calculating the difference value of the peripheral region, only 13 pixels corresponding to one pixel width region are used as the peripheral region pixels, but pixels in a region having a plurality of pixel widths (for example, for two lines) 28 pixels) can be used as the peripheral region pixels.

  In the example of FIG. 9, some pixels in the encoded block adjacent to the target block on the left, upper left, upper, and upper right are set as peripheral region pixels. However, any pixel may be combined as long as the pixels constituting the peripheral region pixel are pixels that are encoded before the target block and belong to a block that touches the side or corner of the target block. .

  For example, pixels belonging to three blocks adjacent to the target block on the left, upper, and upper right, for example, 12 pixels excluding pixel A from 13 pixels in FIG. 9 may be used.

  According to the prediction error estimation index value calculation process for inter-screen prediction described above, the index value for estimating the magnitude of the prediction error can be calculated using the fact that the correlation in the spatial direction of the image is high.

  Next, when the prediction mode determined by the mode determination unit (107) is intra prediction, the prediction error estimation index value calculation unit (201) performs prediction error estimation index value calculation processing for intra prediction shown in FIG. Do. Hereinafter, FIG. 11 will be described.

  FIG. 11 shows an example of the operation of the prediction error estimation index value calculation process for intra prediction according to the present embodiment. In the example of FIG. 11, a prediction error estimation index is obtained using a pixel located at the right end of a block adjacent to the left of the target block and a pixel belonging to a block adjacent to the upper left, upper, and upper right of the target block in the target image. Calculate the value. At this time, the calculation method of the prediction error estimation index value is changed according to the intra-screen prediction mode (prediction direction) used in the intra-screen prediction process.

  For example, among the prediction directions shown in FIG. 6, in the case of the intra prediction mode 0 (prediction direction in the screen downward direction), pixels in the lower two columns of the upper adjacent block adjacent on the target block are used. Of the two columns of pixels, a pair of two pixels arranged in the same direction as the intra-screen prediction mode 0 (prediction direction in the lower direction of the screen) is used as a pair, and a difference between the paired pixels is calculated. Next, for example, a value N obtained by summing the differences of a plurality of pixel pairs arranged in the boundary direction between the target block and the upper adjacent block is divided by the quantization step value of the target block, and a predetermined coefficient α (α The value Error obtained by the formula (1101) of multiplying by a value greater than 0 and adding a predetermined value β is set as a prediction error estimation index value. Here, the pixel pairs used for the calculation may be, for example, three pairs as shown by AA ′, BB ′, and CC ′ in FIG. It doesn't matter.

  Further, for example, in the case of intra prediction mode 1 (prediction direction to the right side of the screen) among the prediction directions shown in FIG. 6, pixels in the rightmost two columns of the left adjacent block adjacent to the left of the target block are used. Of the two columns of pixels, a pair of two pixels arranged in the same direction as the intra-screen prediction mode 1 (prediction direction in the right direction of the screen) is used as a pair, and a difference between the paired pixels is calculated. Next, for example, a value N obtained by summing the differences of a plurality of pixel pairs arranged in the boundary direction between the target block and the left adjacent block is divided by the quantization step value of the target block, and a predetermined coefficient α (α The value Error obtained by the formula (1102) obtained by multiplying by 0 and greater than the predetermined value β is set as the prediction error estimation index value. Here, the pixel pairs used for the calculation may be, for example, three pairs as shown as DD ′, EE ′, and FF ′ in FIG. It doesn't matter.

  According to the prediction error estimation index values in the intra prediction mode 0 and the intra prediction mode 1 described above, based on the difference between the pixel values of a predetermined pixel pair according to the prediction direction of the peripheral area of the target block, An estimated index value of the block prediction error can be determined.

  Further, for example, in the case of the intra prediction modes 2 to 8 among the intra prediction modes shown in FIG. 6, as shown in FIG. 11, in the encoded block adjacent to the left and the upper side of the target block. Using the pixels in such two columns (the lower two columns in the upper adjacent block and the right two columns in the left adjacent block), among the pixels in the two columns, the sum of the pixel values of the pixels far from the target block and the target An expression (1103) for dividing the difference N from the sum of the pixels in the column close to the block by the quantization step value of the target block, multiplying by a predetermined coefficient α (α is greater than 0), and adding a predetermined value β The value Error obtained by the above is used as a prediction error estimation index value.

  According to the prediction error estimation index values of the prediction modes 2 to 8 described above, the estimation index value of the prediction error of the target block is determined based on the difference between the pixel values of a predetermined pixel pair in the peripheral area of the target block. be able to.

  That is, in the prediction error estimation index value calculation process for intra prediction described above, two pixels out of the pixels belonging to the block adjacent to the decoding target block according to the type of the intra prediction mode of the decoding target block A group is determined, a sum of pixel values belonging to each pixel group is calculated, and an estimation index value of a prediction error of the decoding target block is calculated based on a difference in sum of pixel values between the two pixel groups.

  In the above-described prediction error estimation index value calculation process for intra prediction, when the number of pixels in the surrounding area used in each prediction mode is different, the pixels in the surrounding area used in the calculation result of the prediction error estimation index value Error By dividing by the number of and normalizing, the influence of the difference in the number of pixels can be adjusted.

  According to the variable length coding unit (111) of the first embodiment described above and the moving picture coding apparatus having the same, the index for estimating the prediction error of the target block based on the information of the pixels in the peripheral area of the target block By calculating the value and switching the encoding table based on this value, the amount of code can be reduced. In particular, highly efficient variable-length encoding processing can be performed for items other than (3) the number of coefficients other than non-zero in FIG.

  Next, an example of a moving picture decoding apparatus according to the first embodiment will be described with reference to FIG. For example, the moving image decoding apparatus performs reverse processing of variable length coding on the encoded stream (301) generated by the moving image encoding apparatus shown in FIG. 1, motion vectors used for inter-screen prediction, and intra-screen prediction. A variable length decoding unit (302) that performs decoding processing of type information (intra prediction mode information) of the intra prediction method to be used, and an inverse that performs inverse quantization processing on the output from the variable length decoding unit (302) A quantization processing unit (303), an inverse frequency conversion unit (304) that performs inverse frequency conversion processing on the output from the inverse quantization processing unit (303) and decodes a prediction difference, and a variable length decoding unit (302) An inter-screen prediction unit (305) that performs inter-screen prediction to generate a predicted image using the motion vector output from the decoded image stored in the reference image memory (308), and a variable-length decoding unit (302) Screen prediction mode information output from the image and the decoded image stored in the reference image memory (308) In-screen prediction unit (306) that performs intra-screen prediction to generate a prediction image, prediction image generated by inter-screen prediction unit (305) or in-screen prediction unit (306), and output from inverse frequency conversion unit (304) And a reference image memory (308) for storing the generated decoded image.

  Here, details of the inter-screen prediction process by the inter-screen prediction unit (305) will be described with reference to FIG. The inter-screen prediction unit (305) performs decoding in units of blocks on the decoding target image according to the raster scan order. When performing inter-screen prediction, the motion vector for the target block (504) in the decoding target image (503) is obtained from the variable length decoding unit (302), and the motion vector has been decoded based on the motion vector. A reference block (predicted image) (505) in the reference image is specified, and a predicted image used for decoding is determined.

  Details of the intra prediction process by the intra prediction unit (306) will be described with reference to FIG. The intra prediction unit (306) acquires intra prediction mode information about the target block from the variable length decoding unit (302), and based on the intra prediction mode information, the type of inter prediction method (prediction direction) Prediction or DC prediction) is specified, and a prediction image is generated by performing prediction similar to that at the time of encoding by the specified prediction method.

  Next, details of the variable length decoding unit (302) will be described with reference to FIG. The variable length decoding unit (302) includes a prediction error estimation index value calculation unit (401), a table determination unit (402), and an encoding unit (403).

  The prediction error estimation index value calculation unit (401) determines whether the decoding target block of the encoded stream input to the variable length decoding unit (302) is an intra-screen prediction block or an inter-screen prediction block.

  Here, if the decoding target block is an inter-picture prediction block, the motion vector information of the decoding target block included in the encoded stream and the decoded reference image acquired from the reference image memory (308) are used. Then, the prediction error estimation index value calculation process described with reference to FIGS. 8 and 9 is performed in the same manner as the encoding side.

  Here, if the decoding target block is an intra prediction block, the type information (intra prediction mode information) of the intra prediction method of the decoding target block of the decoding target block included in the encoded stream, and the reference image The prediction error estimation index value calculation process described with reference to FIG. 11 is performed using the decoded peripheral region pixels acquired from the memory (308) in the same manner as the encoding side.

  The table determination unit (402) determines a decoding table to be used for variable length decoding processing based on the prediction error estimation index value calculated by the prediction error estimation index value calculation unit (401). The table determination unit (402) is the same as the processing on the encoding side already described with reference to FIG. 13 and FIG. The table in FIG. 18 is referred to as an encoding table at the time of encoding and a decoding table at the time of decoding, but both are the same table.

  The decoding unit (403) performs variable-length encoding processing on the encoded coefficient data included in the encoded stream, using the determined encoding table. The quantization coefficient generated by the variable length coding process is output to the inverse quantization unit (303). Further, the decoding unit (403) transmits the motion vector decoded from the encoded stream to the inter-screen prediction unit (306), and the information about the type of the intra-screen prediction method decoded from the encoded stream to the intra-screen prediction unit (306). Output to.

  According to the variable length decoding unit (302) of the first embodiment described above and the moving picture decoding apparatus having the variable length decoding unit (302), the index for estimating the prediction error of the target block based on the information of the pixels in the peripheral region of the target block By calculating the value and switching the decoding table based on this value, it becomes possible to perform variable length decoding processing corresponding to the encoding table selected on the encoding side, and it is preferable to use an encoded stream with a smaller code amount. Can be decoded. In particular, highly efficient variable-length encoding processing can be performed for items other than (3) the number of coefficients other than non-zero in FIG.

FIG. 14 shows a procedure for encoding one frame in the moving image encoding method according to the first embodiment. First, the following processing is performed for all blocks existing in a frame to be encoded (1401). That is, the prediction process is executed for all prediction modes (combination of prediction method and block size) for the corresponding block (1402). Next, the prediction mode for performing the prediction process is determined (1403). If the determination result is the intra prediction mode, the intra prediction process is performed (1404). An example of the in-screen prediction is as described with reference to FIG. If the determination result is an inter-screen prediction mode, an inter-screen prediction process is performed (1405). An example of the inter-screen prediction is as described in FIG. A predicted image is generated in both the intra-screen prediction process and the inter-screen prediction process, and the prediction difference between the target block image and the predicted image is calculated. Subsequently, frequency conversion processing (1410), quantization processing (1411), and variable length encoding processing (1412) are performed on the prediction difference, and image quality distortion and code amount of each prediction mode are calculated. If the above processing is completed for all prediction modes, the most efficient coding mode is determined based on the above results (1413), and the variable length encoded data of the determined prediction mode is encoded stream. Include in When selecting the one with the highest coding efficiency from among many prediction modes, for example, by using the RD-Optimization method that determines the optimal prediction mode from the relationship between image quality distortion and code amount, the efficiency is improved. Encode well. See Reference 1 below for details of the RD-Optimization method.
[Reference 1] G. Sullivan and T. Wiegand: “Rate-Distortion Optimization for Video Compression”, IEEE Signal Processing Magazine, vol.15, no.6, pp.74-90, 1998.
Subsequently, the quantized frequency transform coefficient of the determined prediction mode is subjected to inverse quantization processing (1414) and inverse frequency transform processing (1415) to decode the prediction difference, and the predicted image of the determined prediction mode And a decoded image is generated and stored in the reference image memory (1416). When the above processing is completed for all the blocks, the encoding for one frame of the image is completed (1417).

  Next, a detailed procedure of the variable length coding process (1412) shown in FIG. 14 will be described with reference to FIG. First, a prediction mode when performing the variable length coding process (1412) of FIG. 14 is determined (1601). If the determination result is intra prediction, prediction error estimation index value calculation processing for intra prediction processing is performed (1602). An example of the prediction error estimation index value calculation process for the intra-screen prediction process is as described with reference to FIG. If the determination result is inter-screen prediction, prediction error estimation index value calculation processing for inter-screen prediction processing is performed (1603). An example of the prediction error estimation index value calculation process for the inter-screen prediction process is as described with reference to FIGS. Next, an encoding table is determined by table determination processing (1604). The table determination process is as described in FIGS. Subsequently, coefficient encoding (1605) is performed using the determined encoding table, and the variable length encoding process is terminated (1606).

  According to the moving picture coding method of the first embodiment of the first embodiment described above, the index value for estimating the prediction error of the target block is calculated based on the information on the pixels in the peripheral area of the target block, and based on this. Thus, by switching the encoding table, it is possible to realize a moving image encoding method that reduces the code amount.

  FIG. 15 shows a decoding process procedure for one frame in the moving picture decoding method according to the first embodiment. First, the following processing is performed for all blocks in one frame (1501). That is, the variable length decoding process (1502) is performed on the input stream, the inverse quantization process (1503) and the inverse frequency transform process (1504) are performed, and the prediction difference is decoded. Subsequently, the prediction mode of the decoding target block is determined (1505). If the determination result is intra prediction, intra prediction processing is performed (1506). An example of the in-screen prediction is as described with reference to FIG. If the determination result is inter-screen prediction, inter-screen prediction processing is performed (1507). An example of the inter-screen prediction is as described in FIG. When the above processing is completed for all the blocks in the frame, decoding for one frame of the image is completed (1508).

  Next, the detailed procedure of the variable length decoding process (1502) shown in FIG. 15 will be described with reference to FIG. First, the prediction mode for performing the variable length decoding process (1502) of FIG. 14 is determined (1701). If the determination result is intra prediction, prediction error estimation index value calculation processing for intra prediction processing is performed (1702). An example of the prediction error estimation index value calculation process for the intra-screen prediction process is as described with reference to FIG. If the discrimination result is inter-screen prediction, prediction error estimation index value calculation processing for inter-screen prediction processing is performed (1703). An example of the prediction error estimation index value calculation process for the inter-screen prediction process is as described with reference to FIGS. Next, a decoding table is determined by table determination processing (1704). The table determination process is as described in FIGS. Subsequently, coefficient coding (1705) is performed using the determined decoding table, and the variable length coding process is terminated (1706).

  According to the moving picture decoding method of the first embodiment described above, the index value for estimating the prediction error of the target block is calculated based on the information of the pixels in the peripheral area of the target block, and the decoding table is based on this. Is switched, it is possible to perform variable length decoding processing corresponding to the encoding table selected on the encoding side, and it is possible to suitably decode an encoded stream with a smaller code amount.

  In the first embodiment described above, DCT is cited as an example of frequency transformation, but DST (Discrete Sine Transformation), WT (Wavelet Transformation), DFT (Discrete Fourier Transformation). Any transformation process may be used as long as it is an orthogonal transform used for inter-pixel correlation removal, such as KLT (Karhunen-Loeve Transformation). You may encode with respect to the prediction difference itself, without performing frequency conversion.

  Further, in the first embodiment, the method of estimating the prediction accuracy of the target block from the information on the peripheral area is shown by taking as an example the case of performing intra prediction and inter prediction. The present invention can be similarly used for other prediction methods such as the direct mode and the direct mode. That is, the present invention may be applied to any prediction method as long as the prediction error can be estimated by performing prediction on the peripheral region in the same procedure as that performed for the target block.

  According to the moving picture coding method, the moving picture decoding method, the moving picture coding apparatus, and the moving picture decoding apparatus according to the first embodiment described above, the prediction accuracy is estimated from the encoded peripheral area of the coding target block. Then, by switching the encoding table based on the size of the prediction error estimated value, the code amount at the time of encoding can be reduced and the compression efficiency can be improved. Thereby, it is possible to provide a high-quality video with a small code amount.

  Next, a second embodiment of the present invention will be described.

  The moving picture coding method, the moving picture decoding method, the moving picture coding apparatus, and the moving picture decoding apparatus according to the second embodiment of the present invention are the prediction error estimation index value calculation process for the inter-screen prediction process according to the first embodiment. And the prediction error estimation index value calculation process for the in-screen prediction process are changed to the processes shown in FIGS. 10 and 12, respectively. Since other operations and configurations are the same as those in the first embodiment, the description thereof is omitted.

First, the prediction error estimation index value calculation process for the inter-screen prediction process according to the second embodiment will be described with reference to FIG. In the second embodiment, the decoded image of any of the encoded / decoded blocks adjacent to the left, upper, upper left, and upper right of the encoding / decoding target block, and the left and upper of the prediction image of the reference image The difference value between the adjacent encoded block and the decoded image of the corresponding block is calculated, and a difference value block is generated. Next, a frequency transform such as DCT transform or Hadamard transform is performed on the difference value block. Formula (1001) for dividing the sum S ′ of coefficient values generated by the conversion process by the quantization step value of the target block, multiplying by a predetermined coefficient γ (γ is greater than 0), and adding a predetermined value δ The value Error obtained by the above is used as a prediction error estimation index value.

  In the calculation of S ′ described above, all coefficient values in the block may be used. However, for simplification of the calculation, the value Error calculated by the equation (1002) is predicted using only the low frequency component. An error estimation index value may be used.

  In the example of FIG. 10, an example is shown in which only the block adjacent to the encoding / decoding target block and the reference block in the upward direction is used. However, an adjacent block that is encoded and decoded before the target block ( Left, upper, upper left, upper right adjacent block). A value obtained by adding or averaging the results of S ′ of a plurality of blocks among these blocks may be set as S ′. In this case, as to which block to use, whether to perform addition or averaging, if the corresponding method is performed on the encoding side and the decoding side, the corresponding table on the encoding side and the decoding side is selected. Therefore, suitable encoding / decoding processing can be performed.

Next, prediction error estimation index value calculation processing for intra-screen prediction processing according to the second embodiment will be described with reference to FIG. In the second embodiment, in the decoded image of any of the encoded / decoded blocks adjacent to the left, upper, upper left, and upper right of the encoding / decoding target block, adjacent pixels (in FIG. 12, A difference value block S is generated by calculating a difference between the pixel a and the pixel a ′) and calculating the difference for each pixel in each position in the block. Next, a frequency transform such as DCT transform or Hadamard transform is performed on the difference value block. Formula (1201) for dividing the sum S ′ of coefficient values generated by the conversion processing by the quantization step value of the target block, multiplying by a predetermined coefficient γ (γ is greater than 0), and adding a predetermined value δ The value Error obtained by the above is used as a prediction error estimation index value.

  The difference value block S is a block smaller than the original adjacent block because the difference between pixel values of adjacent pixels in the adjacent block is used. For example, when the original adjacent block is 16 × 16 pixels, the difference value block S is 16 × 15 pixels at the maximum. However, when performing frequency conversion such as DCT conversion or Hadamard conversion, it is necessary to form a square block. Therefore, processing is performed after reducing one pixel column to a square block of 15 × 15 pixels. That is, the difference value block S is generated by generating a square difference value block using a difference in pixel values between some adjacent pixels of the adjacent block.

  In the example of FIG. 12, an example is shown in which only the block adjacent to the encoding / decoding target block and the reference block in the upward direction is used. However, the neighboring block (encoded and decoded before the target block) Left, upper, upper left, upper right adjacent block). A value obtained by adding or averaging the results of S ′ of a plurality of blocks among these blocks may be set as S ′. In this case, as to which block to use, whether to perform addition or averaging, if the corresponding method is performed on the encoding side and the decoding side, the corresponding table on the encoding side and the decoding side is selected. Therefore, suitable encoding / decoding processing can be performed.

  Further, similarly to the prediction error estimation index value calculation process for the intra-screen prediction process of the first embodiment illustrated in FIG. 11, the direction of the combination of pixels for calculating the difference (vertical direction in the example of FIG. 12) is set to the target block. You may change according to a prediction direction. For example, in the case of the intra prediction mode 0, a difference value between pixels adjacent in the vertical direction may be used using the upper adjacent block as in the example of FIG. In addition, for example, in the case of the intra prediction mode 1, it is only necessary to use a difference value between pixels adjacent in the left-right direction using the left adjacent block. Further, for example, in the case of the intra prediction modes 2 to 8, a plurality of blocks among the left, upper, upper left, and upper right adjacent blocks are used, and the difference value of the pixels adjacent in the left-right direction is used for the left adjacent block. For the upper, upper left, and upper right adjacent blocks, a difference value between pixels adjacent in the vertical direction may be used.

  In the calculation of S ′, all coefficient values in the block may be used. However, for simplification of calculation, only the low frequency component is used, and the value Error calculated by Expression (1202) is used as the prediction error estimation. It may be an index value.

  As described above, by including the frequency conversion process in the calculation process of the prediction error estimation index value, the accuracy suitable for switching of the encoding / decoding table is improved as compared with the prediction error estimation index value calculation process of the first embodiment. An index value can be calculated.

  According to the moving picture coding method, the moving picture decoding method, the moving picture coding apparatus, and the moving picture decoding apparatus according to the second embodiment described above, it is possible to use a prediction error estimation index value with higher accuracy. Thus, it is possible to provide a high-quality video with a smaller code amount than in the first embodiment.

DESCRIPTION OF SYMBOLS 102 ... Input image memory, 103 ... Block division part, 104 ... Reference image memory, 105 ... Inter-screen prediction part, 106 ... In-screen prediction part, 107 ... Mode determination part, 108 ... Subtraction part, 109 ... Frequency conversion part, 110 DESCRIPTION OF SYMBOLS ... Quantization part 111 ... Variable length encoding part 112 ... Inverse quantization part 113 ... Inverse frequency conversion part 114 ... Addition part 201 ... Prediction error estimation index value calculation part 202 ... Table determination part 203 ... Coding section 302 ... variable length decoding section 303 ... inverse quantization processing section 304 ... inverse frequency transform section 305 ... inter-screen prediction section 306 ... intra-screen prediction section 307 ... addition section 308 ... reference image Memory 401: Prediction error estimation index value calculation unit 402: Table determination unit 403: Decoding unit

Claims (10)

Inputting an encoded stream;
Determining a prediction mode of a decoding target block included in the encoded stream;
A prediction error estimation index value calculating step of calculating an estimation index value of a prediction error of the decoding target block based on an already decoded image by a calculation method corresponding to the determined prediction mode;
A decoding table determining step for determining a decoding table to be used for variable length decoding processing from a plurality of decoding tables based on the estimated index value calculated in the prediction error estimation index value calculating step;
Based on the decoding table determined in the decoding table determination step, the encoded stream data is subjected to variable length decoding processing, and the variable length decoding processing is subjected to inverse quantization processing and inverse frequency conversion processing. And a decoded image generating step of generating a decoded image based on the decoded prediction difference and the predicted image generated by the prediction process using the determined prediction mode. Method.   When the determined prediction mode is the inter-screen prediction mode, the prediction error estimation index value calculation step includes: a pixel belonging to a block adjacent to the decoding target block in the decoding target image; and the decoding target image Are different blocks, and based on a difference value of a pixel value from a pixel belonging to a block adjacent to a reference block in a reference image indicated by a motion vector for the decoding target block, a prediction error of the decoding target block The method of claim 1, further comprising: calculating an estimated index value.   The moving image according to claim 2, wherein an estimation index value of a prediction error of the decoding target block is calculated based on a value obtained by dividing the difference value by a quantization step value of the decoding target block. Decryption method.   When the determined prediction mode is the intra prediction mode, two pixels among the pixels belonging to the block adjacent to the decoding target block according to the type of the intra prediction mode of the decoding target block Determining a group, calculating a sum of pixel values belonging to each pixel group, and calculating an estimation index value of a prediction error of the decoding target block based on a difference in sum of pixel values between the two pixel groups. The moving picture decoding method according to claim 1, wherein:   5. The moving image according to claim 4, wherein an estimation index value of a prediction error of the decoding target block is calculated based on a value obtained by dividing the difference value by a quantization step value of the decoding target block. Decryption method. Prediction that generates a prediction image for an encoding target image block in an encoding target image in an intra prediction mode or an inter prediction mode, and generates a prediction difference between the image of the encoding target image block and the prediction image Processing steps;
A frequency conversion / quantization step for generating a quantized data by performing a frequency conversion process and a quantization process on the prediction difference generated in the prediction process step;
A prediction error estimation index value calculation step of calculating an estimation index value of a prediction error of the encoding target block based on an already encoded image by a calculation method corresponding to the prediction mode in the prediction processing step;
An encoding table determining step for determining an encoding table to be used for variable length encoding processing from a plurality of encoding tables based on the estimated index value calculated in the prediction error estimation index value calculating step;
A coded stream generation step that includes variable length coding of the quantized data generated in the frequency conversion / quantization step using the coding table determined in the coding table determination step and includes the encoded data in the encoded stream; and A moving image encoding method.   When the prediction mode in the prediction processing step is an inter-screen prediction mode, the prediction error estimation index value calculation step includes: a pixel belonging to a block adjacent to the encoding target block in the encoding target image; and the encoding target Based on a difference value of a pixel value of a block different from an image and a pixel belonging to a block adjacent to a reference block in a reference image indicated by a motion vector for the encoding target block, the encoding target block The moving picture coding method according to claim 6, wherein an estimated index value of a prediction error is calculated.   8. The moving image according to claim 7, wherein an estimation index value of a prediction error of the encoding target block is calculated based on a value obtained by dividing the difference value by a quantization step value of the encoding target block. Encoding method.   When the prediction mode in the prediction processing step is the intra prediction mode, two of the pixels belonging to the block adjacent to the encoding target block are selected according to the type of the intra prediction mode of the encoding target block. One pixel group is determined, the sum of pixel values belonging to each pixel group is calculated, and the estimation index value of the prediction error of the encoding target block is calculated based on the difference in the sum of pixel values between the two pixel groups The moving picture encoding method according to claim 6, wherein:   The moving image according to claim 9, wherein an estimation index value of a prediction error of the encoding target block is calculated based on a value obtained by dividing the difference value by a quantization step value of the encoding target block. Encoding method.

Images