JP6585776B2 - Processing method - Google Patents

Description

  The present invention relates to a moving picture decoding technique for decoding a moving picture.

  An MPEG (Moving Picture Experts Group) system and other encoding systems have been developed as techniques for recording and transmitting large-volume moving image information as digital data. These standards predict the encoding target image in units of blocks using image information that has been encoded, and encode the difference (prediction difference) from the original image, thereby providing redundancy of the moving image. The code amount is reduced except for.

  In particular, inter-screen prediction that refers to an image different from the target image enables high-precision prediction by searching for a block having a high correlation with the encoding target block from the reference image. In addition, the prediction difference is encoded by performing frequency conversion, for example, discrete cosine transform (DCT) once in order to increase the degree of numerical integration, and quantizing the converted coefficient values. Since the prediction difference also has a strong correlation with the local region, the frequency conversion is also performed in units of blocks obtained by finely dividing the image.

  However, because these methods set a fixed-size block (macroblock) as the basic unit of encoding processing, it is possible to set a block that exceeds the macroblock or a block that spans multiple macroblocks. This has hindered improvement in compression efficiency.

  On the other hand, for example, in Patent Document 1, as described in paragraphs 0003 to 0005, “enhancement of encoding efficiency” for “video material having a large area that is regarded as the same motion amount, such as a high-definition moving image”. To achieve this, “in the moving picture coding apparatus that performs motion prediction and codes a moving picture, the upper limit of the macroblock size of the picture to be coded is set to the picture immediately before the picture and / or the picture. A technique is disclosed that includes means for determining optimally based on the feature amount of a macroblock, and the upper limit of the macroblock size when performing motion prediction can be arbitrarily selected in units of pictures or macroblocks.

JP 2006-339774 A

  The technique disclosed in Patent Document 1 has a problem in that prediction accuracy is reduced because a block for performing prediction is enlarged. When the prediction accuracy is lowered, noise that is noticeable to human eyes is generated, and subjective image quality is lowered.

  The present invention has been made in view of the above problems, and an object thereof is to reduce the amount of code and improve the subjective image quality.

  The moving image decoding method according to an aspect of the present invention performs the following processing. Input an encoded stream. A variable length decoding process is performed on the input encoded stream. A prediction difference is generated by performing an inverse quantization process and an inverse frequency transform process on the data subjected to the variable length decoding process on a first block basis. Prediction processing is performed in units of second blocks. A decoded image is generated based on the generated prediction difference and the result of the prediction process. The first block unit is a larger block unit than the second block unit.

  According to the present invention, it is possible to more suitably reduce the code amount and improve the subjective image quality.

It is a block diagram of the image coding apparatus used by this embodiment. It is a block diagram of the image decoding apparatus used by this embodiment. H. 2 is a conceptual explanatory diagram relating to an H.264 / AVC encoding method. H. 2 is a conceptual explanatory diagram relating to an H.264 / AVC encoding method. H. 2 is a conceptual explanatory diagram relating to an H.264 / AVC encoding method. H. 2 is a conceptual explanatory diagram relating to an H.264 / AVC encoding method. H. 2 is a conceptual explanatory diagram relating to an H.264 / AVC encoding method. It is a conceptual explanatory drawing regarding the encoding method of this embodiment. It is a conceptual explanatory drawing regarding the encoding method of this embodiment. It is an example of the variable length code table utilized in this embodiment. It is a conceptual explanatory drawing regarding the encoding method of this embodiment. It is a conceptual explanatory drawing regarding the encoding method of this embodiment. It is a conceptual explanatory drawing regarding the encoding method of this embodiment. It is an example of the encoding stream produced | generated by this embodiment. It is a flowchart of the image coding apparatus used by this embodiment. It is a flowchart of the image decoding apparatus used by this embodiment. It is a conceptual explanatory drawing regarding the encoding method of this embodiment. It is an example of the variable length code table utilized in this embodiment. It is an example of the encoding stream produced | generated by this embodiment. It is a flowchart of the image coding apparatus used by this embodiment. It is a flowchart of the image decoding apparatus used by this embodiment.

Embodiment 1. FIG.
Hereinafter, the first embodiment of the present invention will be described in H.264. This will be described in comparison with the processing in H.264 / AVC. First, H. H.264 / AVC predicts an encoding target image using image information that has been encoded, and encodes a prediction difference from the original image, thereby reducing the redundancy of the moving image and increasing the code amount. Reduced. Here, in order to use the local property of a moving image, prediction is performed in units of blocks obtained by finely dividing an image.

  As shown in FIG. 3, the encoding process is performed on the target image 305 in units of macroblocks 302 configured by 16 × 16 pixels according to the raster scan order (arrow) 301. In FIG. 3, the target image 305 includes an already encoded area 306 and an uncoded area 307. Prediction is roughly classified into intra-screen prediction and inter-screen prediction.

  FIG. The operation | movement of the inter prediction process by H.264 / AVC is shown notionally. When performing inter-screen prediction, a decoded image of an encoded image included in the same video 401 as the encoding target image 403 is used as a reference image 402, and a block (predicted image) having a high correlation with the target block 404 in the target image is used. ) 405 is searched from the reference image 402.

  At this time, in addition to the prediction difference calculated as the difference between both blocks, the motion vector 406 represented as the difference between the coordinate values of both blocks is encoded as side information necessary for prediction. On the other hand, the reverse procedure described above may be performed at the time of decoding, and the decoded image can be acquired by adding the decoded prediction difference to the block (predicted image) 405 in the reference image.

  H. H.264 / AVC can perform the above prediction by dividing a macroblock into smaller blocks. FIG. 5 shows a macroblock division pattern allowed when inter-screen prediction is performed. That is, H.I. H.264 / AVC is optimal for predicting each macroblock 502 in the target image 501 from among a predetermined division pattern (macroblock division pattern) 503 from 4 × 4 pixel size to 16 × 16 pixel size. You can choose one. Information indicating which division pattern is used for each macroblock is encoded on a macroblock basis.

  On the other hand, the prediction difference generated by the prediction process is decomposed into frequency components by DCT (Discrete Cosine Transformation) which is one of frequency conversion methods, and the coefficient value is encoded. FIG. 6 conceptually shows how the prediction difference is decomposed into frequency components by DCT. DCT is a method of frequency conversion in which an input signal is expressed by a weighted sum of a base signal 603 and its coefficient value. By applying DCT to the prediction difference 601, the coefficient value 602 is often biased toward low frequency components, so variable length coding can be performed efficiently.

  H. In H.264 / AVC, DCT can be applied to a prediction difference by dividing a macroblock into smaller blocks, but the block size when performing DCT is fixed. In the Baseline profile of H.264 / AVC, as shown in FIG. 7, it is specified that the size is 4 × 4 pixels. In FIG. 7, the macro block 702 of the prediction difference 701 is divided into 4 × 4 small blocks (pixels) (703 in FIG. 7).

  As described above, H.P. H.264 / AVC achieves high performance by adaptively dividing and encoding an image into fine blocks. However, H. Since H.264 / AVC uses macroblocks as a basic unit of encoding processing, it has not been possible to handle blocks larger in size than macroblocks or blocks that straddle a plurality of macroblocks. This restriction on the block shape was one of the factors that hindered the improvement of compression efficiency.

  Generally, fine processing is possible when using a small-sized block, so that the accuracy of prediction and DCT is improved and the image quality is improved. On the other hand, however, there is a problem that the code amount increases when a small block is used. This is due to the increase in the number of blocks in the image. For example, when performing inter-screen prediction, it is necessary to encode a motion vector necessary for prediction processing for each block. Therefore, when the number of blocks increases, the number of motion vectors increases accordingly, and the code amount increases.

  In addition, when DCT is performed, the number of significant low-frequency components in the DCT coefficients increases with an increase in the number of blocks, so that the efficiency of VLC (Variable Length Coding) decreases and the code amount increases. To do. Therefore, it is necessary to consider such a trade-off between image quality and code amount when determining an appropriate block size.

  On the other hand, in recent years, demand for high-definition video exceeding high-definition such as digital cinema and super HD is increasing, and the appearance of a method for efficiently encoding such high-definition video is desired. In general, high-definition video with high resolution has high correlation in the screen, and it is known that there is little deterioration in image quality even when a large size block is used.

  Therefore, if the encoding target is narrowed down to high-resolution video, the compression ratio can be dramatically improved by increasing the size of the block that is the processing unit of encoding. For example, the technique of Patent Document 1 makes it possible to change the size of a macroblock and adaptively changes the upper limit value according to the feature amount of an already-encoded area. According to this method, it is possible to enlarge the macroblock according to the property of the image, and in particular, it is possible to increase the compression efficiency of high definition video.

  However, this method has a problem in that the prediction accuracy is reduced because the block used for prediction is enlarged. When the prediction accuracy is lowered, noise that is noticeable to human eyes is generated, and subjective image quality is lowered.

  The present embodiment improves the above problem and further reduces the code amount while maintaining the subjective image quality more suitably. Specifically, in the present embodiment, inter-screen prediction processing is performed in small blocks, for example, in units of macroblocks, while the application size of frequency conversion for the prediction difference (this embodiment uses DCT as an example). Make it expandable.

  For example, FIG. 8 shows a prediction difference 801. As shown in FIG. 8, even when the target image has a complex texture composed of a plurality of objects, if the prediction accuracy is high, the prediction difference has a gentle distribution with many low-frequency components, and the DCT has a large block unit. Even if it is applied, image quality degradation is reduced. Therefore, for the region 802 with high prediction accuracy, the prediction difference of a plurality of adjacent blocks is integrated, a large block is formed, and DCT is performed, so that the code amount of DCT coefficients can be greatly reduced.

  In addition, for a region 803 with low prediction accuracy, such as a part of an object with complex motion, the distribution of prediction differences becomes complicated, and high-frequency components increase. Therefore, when DCT is performed on a large block basis, image quality degradation is conspicuous. . Therefore, for such regions with low prediction accuracy, block integration is not performed, and image quality is maintained by applying DCT in units of blocks that are the same as or smaller than the block at the time of prediction. Can do. As described above, by compressing the prediction differences of a plurality of blocks and performing DCT, the compression rate can be increased and the code amount can be reduced.

  Details of this embodiment will be described below. Note that the processing described in the present embodiment will be described as being applied to a frame (P slice or B slice in H.264 / AVC) that can perform inter-frame coding. The processing described in the following embodiment may or may not be applied to a frame (I slice in H.264 / AVC) in which all areas in the screen are encoded. good.

  In the first embodiment, when all areas are inter-coded in a frame (P slice or B slice in H.264 / AVC) in which inter picture coding can be performed, that is, an inter macro is included in the picture. An example will be described in which there is only a block (macroblock that performs inter-screen coding) and no intra macroblock (macroblock that performs intra-screen coding).

  FIG. 9 shows an example of a block size used for DCT in the present embodiment. Here, the prediction difference 901 is generated by the following method, for example. This method is described in, for example, H.H. The same means as in H.264 / AVC (FIG. 5) performs block division in units of 16 × 16 pixel macroblocks, performs inter-screen prediction of each macroblock, and integrates those prediction differences for one screen. In this embodiment, when DCT is performed on this prediction difference, for example, a block group 902 (64 × 64 pixels) in which 16 adjacent macroblocks are integrated is formed, and block division is performed on a block group basis.

  However, the size of the block group 902 is not limited to the 64 × 64 pixel size, and may be any size such as 32 × 32 or 128 × 128 as long as a plurality of macroblocks 903 are integrated. One preferable method is to prepare a number of types such as 8 × 8 pixels, 16 × 16 pixels, 32 × 32 pixels, 64 × 64 pixels in advance as the division pattern 903 of the block group 902, and the optimum among them. DCT is performed by selecting a proper pattern.

  At this time, for example, a code table as shown in FIG. 10 is used, and information indicating which pattern is selected is encoded for each block group. Here, the overall code amount can be reduced by assigning a short code length to a frequently selected pattern. The selection of the block pattern is effective when, for example, the cost function shown in Equation 1 is used and it is determined that the division pattern for minimizing this is optimal.

  In Equation 1, Dist represents the sum of errors between the original image and the decoded image, Rate represents the sum of the code amount of the DCT coefficient and the code amount of the block division pattern, and Weight represents the weight coefficient. Here, the trade-off between image quality and code amount can be controlled by adjusting the value of Weight. For example, if the code amount is to be significantly reduced even if the image quality is somewhat deteriorated, the value of Weigh may be set higher so that the contribution rate of the code amount to the cost value increases.

  FIG. 11 shows a prediction difference encoding procedure for each block. In this encoding, first, DCT is performed on the prediction difference 1101 of the target block to obtain a DCT coefficient 1102. Subsequently, the DCT coefficient 1102 is quantized to reduce the number of elements to be encoded. At this time, if DCT is performed with a large block size as in the present embodiment, many DCT coefficients are generated in the high frequency components and the amount of codes increases. For example, a large quantization step is applied to the high frequency components of the DCT coefficients. By setting the quantization step weight 1103 in this way, it is possible to significantly reduce the high frequency components and perform the encoding efficiently. However, in this figure, the reference quantization step is represented as Q.

  Subsequently, the quantized DCT coefficient 1104 is subjected to one-dimensional expansion by scanning in a two-dimensional zigzag direction from the low frequency component to the high frequency component (1105), and VLC is applied to generate a code word ( 1106). The above processing is repeated for all blocks obtained by dividing the block group.

  Any processing order may be used for each block group, and FIG. 12 shows an example. Here, an example is shown in which a block group is processed in the order of raster scanning. First, the block group 1201 located at the upper left corner of the screen is processed, and then the block group 1202 adjacent to the right side of the block group 1201 is processed. Thereafter, the processing further proceeds on the block group 1203 and the block group 1204 adjacent to the right side. When the processing reaches the right end of the screen, the block group 1205 adjacent below the block group 1201 is processed. The above processing is performed until the lower right corner of the screen is reached. At this time, the processing order of the macroblocks included in the same block group may be any, but it is effective to process along the zigzag direction 1210, for example.

  Further, in the present embodiment, in order to perform DCT by integrating prediction differences due to a plurality of macroblocks, a memory for temporarily storing the prediction differences is required. The area stored in the memory at once is called an “access group”. At this time, prediction and DCT are performed for each access group. In this encoding method, for example, when the entire screen is set as one access group, prediction processing is performed on all macroblocks in the screen, and sequentially stored in the memory.

  Subsequently, the prediction difference for one screen stored in the memory is divided into blocks in units of blocks and DCT is performed. The access group may be set in any range. For example, as shown in FIG. 13, if one access group is constituted by a block group for one line, encoding can be performed efficiently.

  In this case, first, prediction and DCT are performed on the access group 1311 configured by the block group 1301 to block group 1304 located on the top line of the screen, and then the block group 1305 to block group located on the next line. Prediction and DCT are performed on the access group 1312 configured by 1308. If this is continued until the bottom line of the screen is reached, the encoding process for one frame is completed.

  FIG. 14 illustrates a configuration example (for one block group) of the encoded stream in the present embodiment. Here, a case will be described in which 16 macroblocks serving as basic units for prediction processing exist in the block group. Here, first, as a combination of a prediction method (forward inter-screen prediction, reverse inter-screen prediction, bidirectional inter-screen prediction, intra-screen prediction, etc.) and its block division pattern for the first macroblock (macroblock 1) The represented prediction mode 1401 is encoded, and then the motion vector in each block is encoded as side information 1402 necessary for prediction.

  Subsequently, the prediction mode 1403 for the second macroblock 2 and the motion vector 1404 in each block obtained by dividing the macroblock are encoded. This is repeated for all macroblocks included in the corresponding block group. Subsequently, the block division pattern 1405 when performing DCT on the prediction difference of the corresponding block group and the DCT coefficient 1406 of each block are encoded. At this time, the block size for performing DCT may be set to a fixed value such as 64 × 64. In this case, the encoding of the block group division pattern 1405 is not necessary.

  FIG. 1 shows an example of a moving image encoding apparatus according to this embodiment. The moving image encoding apparatus performs an intra-screen prediction in units of blocks, an input image memory 102 that holds an input original image 101, a block division unit 103 that performs block division on an image in the input image memory 102 An intra-screen prediction unit 104, an inter-screen prediction unit 106 that performs inter-screen prediction based on the motion vector detected by the motion search unit 105, and a prediction method and a prediction method that determine a block shape according to the nature of the image A block determination unit 107 is included.

  The moving image encoding apparatus further includes a subtraction unit 108 for generating a prediction difference, a DCT unit 110 that performs frequency conversion on the prediction difference, and a frequency that determines a block shape for frequency conversion that matches the nature of the prediction difference. A transform block determining unit 116; a quantization processing unit 111 that performs quantization on a coefficient value after frequency conversion; a variable-length coding processing unit 112 that performs coding according to a symbol generation probability; An inverse quantization processing unit 113 and an inverse DCT unit 114 for decoding the normalized prediction difference, an addition unit 115 for generating a decoded image using the decoded prediction difference, and holding the decoded image Thus, a reference image memory 117 for use in later prediction is provided.

  The input image memory 102 holds one image from the original images 101 as an encoding target image. The block dividing unit 103 divides the image data into blocks of an appropriate size and sends them to the intra-screen prediction unit 104, motion search unit 105, inter-screen prediction unit 106, and subtraction unit 108. The motion search unit 105 calculates the motion amount of the corresponding block using the decoded image stored in the reference image memory 117 and sends the motion vector to the inter-screen prediction unit 106. The intra-screen prediction unit 104 and the inter-screen prediction unit 106 execute the intra-screen prediction process and the inter-screen prediction process in units of several types of blocks. The prediction method / block determination unit 107 selects an optimal prediction method and block shape (macroblock division pattern).

  Subsequently, the subtraction unit 108 generates a prediction difference by an optimal prediction encoding unit using the original image and the prediction result, and sends the prediction difference to the prediction difference memory 109. The prediction difference memory 109 sends the prediction difference to the DCT unit 110 when the prediction difference for one access group is stored. The DCT unit 110 and the quantization processing unit 111 divide the blocks into several types of blocks and perform frequency conversion and quantization processing such as DCT, respectively. The variable length coding processing unit 112 and the inverse quantization processing unit 113. The inverse quantization processing unit 113 and the inverse DCT unit 114 perform inverse quantization and inverse frequency transform (for example, IDCT (Inverse DCT)) on the quantized frequency transform coefficient, respectively, and obtain a prediction difference And sent to the adder 115.

  Subsequently, the adding unit 115 generates a decoded image. The frequency transform block determination unit 116 and the reference image memory 117 store the decoded image. The frequency transform block determination unit 116 determines an optimal block shape (block group division pattern) for performing frequency conversion, and sends the information to the variable length coding processing unit 112. Furthermore, the variable-length encoding processing unit 112 has optimal block shape information (macroblock, division pattern of block group) when performing prediction / frequency conversion, frequency conversion coefficient (prediction difference information) based on the optimal block shape, and Side information necessary for prediction processing at the time of decoding (for example, a prediction direction when performing intra-screen prediction and a motion vector when performing inter-screen prediction) is variable-length-encoded based on the occurrence probability of a symbol and is encoded stream Is generated.

  FIG. 2 shows an example of a moving picture decoding apparatus according to this embodiment. For example, the moving picture decoding apparatus performs variable length decoding for decoding various information by performing the reverse procedure of variable length coding on the encoded stream 201 generated by the moving picture encoding apparatus shown in FIG. , An inverse quantization processing unit 203 and an inverse DCT unit 204 for decoding prediction difference information, a prediction difference memory 205 for storing a prediction difference for one access group, and inter-screen prediction An inter-screen prediction unit 206, an intra-screen prediction unit 207 that performs intra-screen prediction, an addition unit 208 for acquiring a decoded image, and a reference image memory 209 for temporarily storing the decoded image Have.

  The variable-length decoding unit 202 performs variable-length decoding on the encoded stream 201, and acquires block shape information when performing prediction and frequency conversion, prediction difference information, and side information necessary for prediction processing at the time of decoding. Among these, block shape information (block group division pattern) and prediction difference information when performing frequency conversion are sent to the inverse quantization processing unit 203, and block shape information (macroblock division pattern) when performing prediction. The side information necessary for the prediction process at the time of decoding is sent to the inter-screen prediction unit 206 or the intra-screen prediction unit 207.

  Subsequently, the inverse quantization processing unit 203 and the inverse DCT unit 204 respectively perform inverse frequency transforms such as inverse quantization and inverse DCT on the prediction difference information in the block shape (block group division pattern) specified in units of blocks. To be decoded and sent to the prediction difference memory 205. Subsequently, the inter-screen prediction unit 206 or the intra-screen prediction unit 207 refers to the block shape (macroblock division pattern) designated with reference to the reference image memory 209 based on the information sent from the variable length decoding unit 202. The prediction process is executed at. The adding unit 208 generates a decoded image from the prediction processing result and the prediction difference for one access group stored in the prediction difference memory 205, and stores the decoded image in the reference image memory 209.

  FIG. 15 shows an encoding processing procedure for one frame in the present embodiment. First, the following processing is performed for all regions existing in the frame to be encoded (1501). That is, for all macroblocks in the corresponding access group (1502), all available prediction methods (forward inter-screen prediction, backward inter-screen prediction, bidirectional inter-screen prediction, intra-screen prediction, etc.) and Prediction is executed with the block shape (macroblock division pattern) (1503), and the prediction difference is calculated.

  Then, an optimal combination is selected from the prediction results of all prediction methods and block shapes (1504), information on the combination is encoded, and the prediction difference is stored in the memory. The term “optimal” here means a case where both the prediction difference and the code amount are small. For selecting the combination of the prediction method and the block shape, for example, it is effective to calculate the coding cost (Cost) expressed by Equation 2 for all the combinations and select the combination having the smallest value.

  Here, SAD (Square Absolute Difference) represents the sum of the absolute values of the prediction differences, and R (Rate) represents an estimated value of the code amount when the prediction differences are encoded. Λ is a constant for weighting, and this value differs depending on the prediction method (intra-screen prediction / inter-screen prediction), quantization parameters, etc., so it is effective to use different values accordingly. Is. It is desirable to calculate the estimated code amount in consideration of not only the prediction difference information but also the code amount such as block shape information and motion vector.

  When the above processing is completed for all the macroblocks in the access group, the prediction difference corresponding to the corresponding access group stored in the memory (1505) is subsequently obtained for all the blocks available for each block group. DCT (1506), quantization (1507), and variable length coding (1508) are performed in the block shape (block group division pattern). Then, the quantized DCT coefficients are subjected to inverse quantization (1509) and inverse DCT (1510) to decode the prediction difference information, and further using Equation 1, an optimal block shape (block group division) Pattern) is selected (1511), and the shape information is encoded.

  In addition to Equation 1, the shape information can be selected using, for example, an RD-Optimization method that determines an optimal encoding mode from the relationship between image quality distortion and code amount. The RD-Optimization method is a well-known technique and will not be described in detail here. For details, see Reference 1, for example (Reference 1: G. Sullivan and T. Wiegand: “Rate-Distortion Optimization for Video Compression”, IEEE Signal Processing Magazine, vol. 15, no. 6, pp. .74-90, 1998.).

  Subsequently, a decoded image is obtained by adding the decoded prediction difference and the predicted image (1512), and stored in the reference image memory. When the above processing is completed for all the access groups, the encoding for one frame of the image is completed (1513).

  FIG. 16 shows a decoding processing procedure for one frame in the present embodiment. First, the following processing is performed for all access groups in one frame (1601). That is, all the block groups in the access group (1602) are subjected to variable length decoding processing (1603), and the inverse quantization processing (1604) and the designated block shape (block group division pattern) are performed. Inverse DCT (1605) is applied to decode the prediction difference and store it in the memory.

  When the above processing is completed for all the block groups in the access group, subsequently, for the same access group (1606), the variable length decoding prediction method and the block shape (macroblock Based on the division pattern), prediction (1607) is performed, and a decoded image is obtained by adding the prediction difference stored in the memory (1608). When the above processing is completed for all access groups in the frame, decoding for one frame of image is completed (1609).

  In this embodiment, DCT is cited as an example of frequency transformation, but DST (Discrete Sine Transformation), WT (Wavelet Transformation), DFT (Discrete Fourier Transformation), KLT (Karhunen). -Loeve Transformation (Carhunen-Leave transformation), etc., any orthogonal transformation can be used to remove correlation between pixels, and encoding can be performed on the prediction difference itself without frequency conversion. . Furthermore, variable length coding is not particularly required. Further, this embodiment may be used in combination with another method.

  According to the video encoding device and encoding method, the video decoding device, and the decoding method according to Embodiment 1 described above, prediction is performed in encoding and decoding of a frame configured by inter macroblocks. Moving picture encoding apparatus, encoding method, moving picture decoding apparatus, and decoding method for reducing code amount while maintaining subjective image quality more suitably by performing frequency conversion in units of blocks larger than the macroblock to be used Can be realized.

Embodiment 2. FIG.
In the first embodiment, when all areas are inter-coded in a frame (P slice or B slice in H.264 / AVC) in which inter-picture coding can be performed, that is, an inter macro is included in the picture. An example has been described in which only blocks (macroblocks that perform inter-frame encoding) exist and intra macroblocks (macroblocks that perform intra-frame encoding) do not exist.

  On the other hand, in the second embodiment, when intra-frame coding can be applied to a frame (P slice or B slice in H.264 / AVC) in which inter-frame coding can be performed, that is, an inter macroblock ( A case will be described in which encoding can be performed by mixing a macro block that performs inter-screen encoding) and an intra macro block (macro block that performs intra-screen encoding).

  In an intra macroblock, prediction is performed using a decoded image of a block that has already been encoded. Therefore, when predicting a target block, it is necessary to complete DCT processing in adjacent blocks. For this reason, it is impossible to perform batch prediction on macroblocks for one block group and perform DCT by integrating these prediction differences.

  Therefore, prediction and DCT are performed on a macroblock basis for a block group including even one intra macroblock. At this time, the DCT process is performed in units of blocks obtained by dividing the macroblock, and the size thereof is equal to or smaller than the macroblock. Therefore, 32 × 32, 64 × 64, etc. in FIG. 9 cannot be used as a block size (DCT macroblock division pattern) in DCT processing. At this time, for example, the division pattern is encoded using a code table shown in FIG.

  FIG. 17 conceptually shows an example of an encoding method for each block group. This example shows a case where the access group matches the block group. In the encoding process, first, prediction is performed for all macroblocks included in the block group 1701 located at the upper left end of the image, and if all macroblocks are inter macroblocks, the same means as in the first embodiment (see FIG. In 9), the division pattern of the block group is determined and DCT is performed. Subsequently, the same processing is performed for the block group 1702 adjacent to the right side of the block group 1 if no intra macroblock is included.

  Further, the same processing is advanced for the block group 1703 and the block group 1704, and when the right end of the screen is reached, the block group 1705 adjacent below the block group 1701 is processed. Here, for example, if one or more macroblocks included in the block group 1706 are intra macroblocks, prediction and DCT are performed on the block group in units of macroblocks. As described above, in the present embodiment, the unit for performing DCT is switched between the macroblock and the macroblock group depending on whether or not the corresponding macroblock group includes an intra macroblock. When the process reaches the lower right corner of the screen, the encoding for the position screen ends.

  FIG. 19 shows a configuration example of an encoded stream for a block group in which one or more intra macroblocks exist in the present embodiment. Here, a case where 16 macroblocks exist in the block group will be described. First, the prediction method (forward inter-screen prediction, reverse inter-screen prediction, bi-directional inter-screen prediction, intra-screen prediction, etc.) and its division pattern when predicting the first macroblock (macroblock 1) A prediction mode 1901 expressed as a combination is encoded. Subsequently, as side information 1902 necessary for prediction, a motion vector is encoded in the case of an inter macroblock, and information regarding a prediction direction is encoded in the case of an intra macroblock.

  Subsequently, the macroblock block division pattern 1903 when DCT is performed on the same macroblock and the DCT coefficient 1904 of each block are encoded. The above processing is performed for all macroblocks included in one block group. Also, the block size for performing DCT may be set to a fixed value such as 8 × 8. In this case, encoding of a division pattern for each macroblock is not necessary. Note that the configuration of the encoded stream for a block group having no intra macroblock is the same as that of the first embodiment (FIG. 14).

  FIG. 20 shows an encoding processing procedure in a block group in which one or more intra macroblocks exist in the present embodiment. This process is performed for all macroblocks included in the corresponding block group (2001), for all available prediction methods (forward inter-screen prediction, backward inter-screen prediction, bidirectional inter-screen prediction, intra-screen prediction). Etc.) and a block shape (macroblock division pattern) (2002), and a prediction difference is calculated.

  Then, a suitable combination is selected from the prediction results of all prediction methods / block shapes (2003), and information on the combination is encoded. The term “preferable” here means a case where both the prediction difference and the code amount are small, and it is effective to use a cost function expressed by Formula 2 or another formula for the evaluation.

  Subsequently, DCT (2004), quantization (2005), and variable length coding (2006) are performed on the prediction difference of the same macroblock in an available block shape (macroblock division pattern subjected to DCT). Do. Then, the quantized DCT coefficients are subjected to inverse quantization (2007) and inverse DCT (2008) to decode the prediction difference information, and further using Equation 1, an optimal block shape (for DCT) Macroblock division pattern) is selected (2009), and its shape information is encoded. The selection of the shape information may use another method of Reference Document 1 described above or another method.

  Subsequently, a decoded image is acquired by adding the decoded prediction difference and the predicted image (2010), and stored in the reference image memory. When the above processing is completed for all the macroblocks included in the corresponding block group, the encoding of the corresponding block group is completed. Note that the encoding processing procedure in a block group in which no intra macroblock exists is the same as that in the first embodiment (FIG. 15).

  FIG. 21 shows a decoding processing procedure in a block group in which one or more intra macroblocks exist in this embodiment. In this processing, all macroblocks included in the block group (2101) are subjected to variable length decoding processing (2102), and inverse quantization processing (2103) and inverse DCT (2104) are performed in the designated block shape. ) To decode the prediction difference and store it in the memory.

  Subsequently, prediction (2105) is performed based on the prediction method subjected to variable length decoding and the block shape (macroblock division pattern for DCT) when performing prediction, and the prediction difference stored in the memory is added. Thus, a decoded image is acquired (2106). When the above processing is completed for all the macroblocks included in the block group, the decoding of the block group is completed. Note that the decoding processing procedure in the block group in which no intra macroblock exists is the same as that in the first embodiment (FIG. 16).

  Note that the moving picture encoding apparatus and the moving picture decoding apparatus that perform the processing of the second embodiment are components of the moving picture encoding apparatus and the moving picture decoding apparatus according to the first embodiment shown in FIGS. 1 and 2. Therefore, the description of the configuration itself is omitted.

  In this embodiment, DCT is cited as an example of frequency transformation, but DST (Discrete Sine Transformation), WT (Wavelet Transformation), DFT (Discrete Fourier Transformation), KLT (Karhunen). -Loeve Transformation (Caroonen-Leave transformation), etc., any orthogonal transformation can be used to remove the correlation between pixels, and the prediction difference itself may be encoded without any frequency transformation. . Furthermore, variable length coding is not particularly required. Further, this embodiment may be used in combination with another method.

  According to the moving picture coding apparatus and coding method, the moving picture decoding apparatus, and the decoding method according to Embodiment 2 described above, coding and decoding of a frame having a configuration in which inter macro blocks and intra macro blocks are mixed. Video coding apparatus, coding method, and video decoding that further reduce code amount while maintaining subjective image quality more appropriately by performing frequency conversion in units of blocks larger than macroblocks used for prediction A decoding device and a decoding method can be realized.

  The present invention has been described in detail with reference to the accompanying drawings, but the present invention is not limited to such a specific configuration, and various modifications and equivalents within the spirit of the appended claims. The configuration is included.

  As a representative method from the viewpoint of the present invention other than those described in the scope of claims, the following treatment can be given.

  (1) Input an encoded stream. A variable length decoding process is performed on the input encoded stream. A prediction difference is generated by performing an inverse quantization process and an inverse frequency transform process on the data subjected to the variable length decoding process on a first block basis or on a second block basis. At this time, if the block group includes an intra-screen prediction block, the block group performs the inverse quantization process and the inverse frequency transform process in units of the second block, and the block group includes a screen. When an intra prediction block is not included, the block group performs the inverse quantization process and the inverse frequency transform process in units of the first block. Prediction processing is performed in units of the second block. A decoded image is generated based on the generated prediction difference and the prediction processing result. One block in the first block unit is one block obtained by integrating a block group composed of a plurality of blocks in the second unit.

  (2) Input an encoded stream. A variable length decoding process is performed on the input encoded stream. A prediction difference is generated by performing an inverse quantization process and an inverse frequency transform process on the data subjected to the variable length decoding process on a first block basis or on a second block basis. Prediction processing is performed in units of the second block. A decoded image is generated based on the generated prediction difference and the prediction processing result. One block in the first block unit is one block obtained by integrating a block group composed of a plurality of blocks in the second unit. When an intra-screen prediction block is included in the block group, the inverse quantization process and the inverse frequency transform process are performed in the block group in units of the second block. When the intra-frame prediction block is not included in the block group, the inverse quantization process and the inverse frequency transform process are performed in the block group in the first block unit. In the encoded stream to be input, the stream configuration of the block group including the intra prediction block includes the prediction mode information of the second block unit and the frequency transform coefficient of the second block unit, The stream configuration of the block group that does not include the intra-screen prediction block includes the prediction mode information of the second block unit and the frequency conversion coefficient of the first block unit.

  (3) Input an input image. A prediction difference is generated by performing a prediction process on the input image for each first block. The generated prediction difference is subjected to frequency conversion processing and quantization processing to generate quantized data. At this time, when an intra-screen prediction block is included in a block group including a plurality of blocks, the block group performs the frequency conversion process and the quantization process in the first block unit, and the block group includes When the intra prediction block is not included, in the block group, the frequency conversion process and the quantization process are performed in a second block unit having a size obtained by integrating a plurality of blocks in the first block unit. Do. Variable length coding is performed on the generated quantized data to generate an encoded stream.

  (4) Input an input image. A prediction difference is generated by performing a prediction process on the input image for each first block. The generated prediction difference is subjected to frequency conversion processing and quantization processing to generate quantized data. Variable length coding is performed on the generated quantized data to generate an encoded stream. When an intra-screen prediction block is included in a block group consisting of a plurality of blocks, the block group performs the frequency conversion process and the quantization process in units of the first block, and the block group includes an intra-screen prediction block. Is not included in the block group, the frequency conversion process and the quantization process are performed in a second block unit having a size obtained by integrating a plurality of blocks in the first block unit. In the stream configuration for the block group including the plurality of blocks of the generated encoded stream, when the intra-frame prediction block is included in the block group, the stream configuration of the block group includes the first block. When the prediction mode information of the unit and the frequency conversion coefficient of the first block unit are included, and the intra-prediction block is not included in the block group, the prediction mode information of the first block unit and And a frequency conversion coefficient of the second block unit.

  As described above, the present invention can be applied to encoding / decoding of moving images, and in particular, can be applied to encoding / decoding in units of blocks.

Claims (1)

A processing method in a system including a moving image encoding device and a moving image decoding device,
In the moving image encoding device, an encoded stream generating step of generating an encoded stream by encoding a moving image;
In the video decoding device, an input step of inputting the encoded stream;
In the video decoding device, a variable length decoding step for performing a variable length decoding process on the encoded stream input in the input step;
In the moving image decoding device, predicted the variable length decoding data subjected to variable length decoding in step, performing an inverse quantization process and an inverse frequency conversion processing in the first block or the second block An inverse quantization / inverse frequency conversion step for generating a difference;
In the moving picture decoding apparatus, a prediction step for performing a prediction process in units of the second block;
In the moving image decoding apparatus, the decoding image generation step of generating a decoded image based on the prediction difference generated in the inverse quantization / inverse frequency conversion step and the prediction processing result in the prediction step,
Wherein one block of the first block unit is one block that integrates block group including a plurality of blocks of said second block,
In the video decoding device, possible processing states in the inverse quantization / inverse frequency conversion step include:
When the block group includes an intra prediction block, the block group performs the inverse quantization process and the inverse frequency transform process in the second block unit in the order of the upper left block, the upper right, the lower left, and the lower right. A first processing state to proceed;
Wherein when the group of blocks does not include intra prediction block in the block group, there is a second processing state of performing the first inverse quantization process in units of blocks and inverse frequency conversion processing, the processing method.

Images