JP2010135864A - Image encoding method, device, image decoding method, and device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide the encoding device of a moving image, which reduces the operation cost and improves the encoding efficiency. <P>SOLUTION: An image encoding method includes the steps of: a predicted image for an encoding object pixel block is generated, on the basis of a selected predicted mode; an optimum predicted mode is determined, on the basis of a prediction error between an input image and the predicted image, and the code amount of the predicted mode; the order of the selection frequency of the predicted mode indicating the selection frequency of the predicted mode is rearranged by the determined predicted mode; the index of a rearranged frequency information table is generated; for the encoding object pixel block, predicted mode information is extracted from the index; a predicted image signal corresponding to the extracted predicted mode information is generated; the cost of the predicted mode is calculated; one encoding mode is selected from the cost; and prediction error signals, the table length of the frequency information table, and an index number indicating the selected encoding mode are encoded according to the selected encoding mode. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a prediction mode estimation, image encoding, and decoding method and apparatus for moving images or still images.

  A moving picture coding method with significantly improved coding efficiency compared to the prior art has been developed in collaboration with ITU-T Rec. H. H.264 and ISO / IEC 14496-10. (Hereinafter referred to as “H.264”). ISO / IEC MPEG-1, 2, 4, ITU-TH 261, H.H. A conventional intra-picture encoding method such as H.263 performs intra-frame prediction on the frequency domain (DCT coefficient) after orthogonal transform to reduce the code amount of the transform coefficient. H.264 uses direction prediction (Non-Patent Document 1) in the spatial region (pixel region), and compared with the intra-frame prediction of the conventional (ISO / IEC MPEG-1, 2, 4) video coding system. High prediction efficiency is achieved.

  H. In the H.264 high profile and the like, three types of intra-frame prediction methods are defined for luminance signals, and one of them can be selected in units of macroblocks (16 × 16 pixel blocks). The three types of intra-frame prediction methods are called 4 × 4 pixel prediction, 8 × 8 pixel prediction, and 16 × 16 pixel prediction, respectively.

  In 16 × 16 pixel prediction, four encoding modes are defined, which are called vertical prediction, horizontal prediction, DC prediction, and plane prediction. The pixel value of the surrounding macroblock before applying the deblocking filter after the decoding process is used as the reference pixel value, and used for the prediction process. The prediction mode information of 16 × 16 prediction is included in the macroblock type, and the amount of codes for transmitting the mode is significantly reduced compared to other predictions.

  On the other hand, in the 4 × 4 pixel / 8 × 8 pixel prediction, the luminance signal in the macroblock is divided into 16/4/4 × 4/8 × 8 pixel blocks, and any of the nine modes is divided into block units for each pixel block. Select Each of the nine modes has a prediction direction of 22.5 degrees except for DC prediction (mode 2) in which prediction is performed using an average value of available reference pixels, and extrapolates in the prediction direction using reference pixels. Interpolation is performed to generate a predicted value. Since 4x4 / 8x8 pixel prediction has a smaller unit of prediction processing than 16x16 pixel prediction, relatively efficient prediction can be performed even for images with complex textures, but simple interpolation with respect to the prediction direction There is a problem that the prediction error is increased as the distance from the reference pixel is increased.

  Thus, in recent video coding systems, the number of prediction modes that can be selected tends to increase as the performance of hardware increases, and the increase in code amount due to the encoding of mode information in the prediction mode is a major problem. It has become. H. Even in the H.264 high profile, there are 4 modes for 16x16 pixel prediction, 9 modes each for 4x4 pixel / 8x8 pixel prediction, and there are many prediction modes, and the prediction mode for small pixel blocks tends to be difficult to select when encoding at a low bit rate. . On the other hand, an increase in the number of prediction modes causes an increase in calculation cost, and there is a problem that the prediction mode of the small pixel block cannot be used at the time of encoding in a portable device or a power saving device.

To deal with such a problem, Non-Patent Document 2 introduces a direct prediction mode that changes to 16 × 16 pixel prediction with a low selection rate as a prediction mode. In particular, the planar prediction of 16 × 16 pixel prediction generates prediction pixel values for all 256 pixels, and thus the calculation cost is increased even when compared with other prediction modes. The direct prediction mode is a prediction mode in which the prediction of 4 × 4 pixel prediction is used as it is and the prediction mode information is not transmitted to the decoder. Therefore, H.H. The prediction mode is predicted from the prediction modes of the upper and lower pixel blocks adjacent to the pixel block to be encoded using the mode derivation method defined in H.264. Since the 4 × 4 pixel prediction is used, it is possible to reduce the prediction mode information while maintaining the encoding efficiency if the prediction mode is predicted. However, there is a problem that the encoding efficiency is reduced when the prediction mode is not predicted.
Greg Conklin, “New Intra Prediction Modes”, ITU-T Q.6 / SG16 VCEG, VCEG-N54, Sep. 2001. Lu Yu, Feng Yi, “Low complexity intra prediction”, ITU-T SG16 / Q.6 VCEG-Z14, April 2005

  As explained above, H.P. H.264 When the encoding mode is transmitted by the method defined in the high profile, since the code amount of the mode information cannot be ignored at a low bit rate, it is difficult to select a prediction mode with good prediction performance, and the encoding efficiency is lowered. Moreover, in the direct prediction mode, there is a problem that the encoding efficiency is lowered when the mode is not predicted.

  According to the embodiment of the present invention, the step of preparing a frequency information table indicating the selection frequency of the auxiliary information regarding the prediction mode, the step of dividing the input image into a plurality of pixel blocks, and the encoding target pixel block of the pixel block A step of selecting supplementary information related to the prediction mode, a step of generating a prediction image for the encoding target pixel block using a reference image based on the selected supplementary information, a prediction error between the input image and the prediction image, The optimal prediction mode is determined based on the code amount of the prediction mode, the selection frequency order of the prediction mode of the frequency information table is rearranged according to the determined prediction mode, and the index of the rearranged frequency information table is generated And 1 from the index for the pixel block to be encoded. Extracting the above supplementary information; generating a prediction signal corresponding to the extracted supplementary information; calculating a cost of the prediction mode; and selecting one encoding mode from the cost; Encoding the prediction error signal and the table length of the frequency information table according to the selected encoding mode, and encoding an index number in the frequency information table indicating the selected encoding mode. An image encoding method is provided.

  According to the present invention, it is possible to realize an image encoding / decoding method and apparatus with improved encoding efficiency while reducing hardware costs.

  Exemplary embodiments of a moving image encoding method, a moving image encoding device, a moving image decoding method, and a moving image decoding device according to the present invention will be explained below in detail with reference to the accompanying drawings.

  With reference to FIG. 1, the structure of the moving image encoding device concerning embodiment of this invention is demonstrated.

Configuration of moving picture encoding apparatus (encoding: first embodiment)
According to the moving image encoding apparatus shown in FIG. 1, the moving image signal is divided for each small pixel block and input to the encoding unit 100. In the encoding unit 100, a plurality of prediction modes having different block sizes and prediction image signal generation methods are prepared as prediction modes performed by the internal prediction and mode determination unit 102. In the present embodiment, it is assumed that the encoding process is performed from the upper left to the lower right as shown in FIG.

  The input image signal 110 input to the encoding unit 100 is divided by the screen dividing unit 101 into blocks of 16 × 16 pixels as shown in FIG. A pixel block of the input image signal 110 is input to the internal prediction / mode determination unit 102. The pixel block that has passed through the internal prediction / mode determination unit 102 is finally encoded by the encoding processing unit 105 via a mode determination unit 103 and a transform quantization unit 104 described later. The encoded pixel block is accumulated in the output buffer, and then output as encoded data 115 at an output timing managed by the encoding control unit 108.

  The 16 × 16 pixel block is called a macro block and has a basic processing block size for the following encoding process. The encoding unit 100 reads the input image signal 110 for each macroblock and performs an encoding process. Note that the macroblock may be a 32 × 32 pixel block unit or an 8 × 8 pixel block unit. An example of the macroblock is shown in FIG.

  The internal prediction / mode determination unit 102 uses the encoded reference pixels temporarily stored in the reference image memory 107 to generate the prediction image signal 111 in all prediction modes that can be selected in the macroblock. That is, the internal prediction / mode determination unit 102 generates all prediction image signals in encoding modes that can be taken by the encoding target pixel block. However, H. Like in H.264 intra-frame prediction (4 × 4 pixel prediction (see FIG. 5C)) or 8 × 8 pixel prediction (see FIG. 5D), the next prediction cannot be performed unless a locally decoded image is created in the macroblock. In such a case, the internal prediction / mode determination unit 102 may internally perform coefficient transformation and quantization, inverse quantization, and inverse transformation.

  The predicted image signal 111 generated by the internal prediction / mode determination unit 102 is input to the mode determination unit 103 together with the input image signal 110. The mode determination unit 103 inputs the predicted image signal 111 to the inverse quantization inverse transform unit 106, generates a prediction error signal 112 by subtracting the predicted image signal 111 from the input image signal 110, and outputs the prediction error signal 112 to the transform quantization unit 104. input. At the same time, the mode determination unit 103 performs mode determination based on the mode information predicted by the internal prediction / mode determination unit 102 and the generated prediction error signal 112. More specifically, in the present embodiment, the mode determination unit 103 performs mode determination using a cost such as the following equation.

K = SAD + λ × OH (1)
However, OH is mode information, and SAD is the absolute sum of prediction error signals. Λ is given as a constant and is determined based on the quantization width and the value of the quantization parameter. The mode is determined based on the cost thus obtained. In this case, the mode giving the value with the smallest cost K is selected as the optimum mode.

  In the present embodiment, the absolute sum of the mode information and the prediction error signal is used. However, as another embodiment, the mode may be determined using only the mode information or only the absolute sum of the prediction error signal. These may be Hadamard transformed, or values approximate to these may be used. Further, the cost may be created using the activity of the input image signal, or the cost function may be created using the quantization width and the quantization parameter.

  As another embodiment for calculating the cost, a provisional encoding unit is prepared, and the amount of code when the prediction error signal generated in the encoding mode of the provisional encoding unit is actually encoded, The mode may be determined using a square error between the locally decoded image 114 and the input image signal 110 obtained by locally decoding the digitized data. The mode judgment formula in this case is as follows.

J = D + λ × R (2)
Here, D is a coding distortion representing a square error between the input image signal 110 and the locally decoded image 114. On the other hand, R represents a code amount estimated by provisional encoding. When this cost is used, provisional coding and local decoding (inverse quantization processing and inverse transformation processing) are required for each coding mode, so the circuit scale increases, but accurate code amount and coding distortion are increased. Can be used, and the encoding efficiency can be kept high. As for this cost, the cost may be calculated using only the code amount or only the coding distortion, or the cost function may be created using a value approximate to these costs.

  The mode determination unit 103 is connected to the transform quantization unit 104 and the inverse quantization inverse transform unit 106, and the mode information selected by the mode determination unit 103 and the prediction error signal 112 are input to the transform quantization unit 104. The The transform quantization unit 104 transforms the input prediction error signal 112 into transform coefficients, and generates transform coefficient data. Here, the prediction error signal 112 is orthogonally transformed using, for example, discrete cosine transformation. As another embodiment, the transform coefficient may be created using a technique such as wavelet transform or independent component analysis. The transform coefficient data is quantized by the transform quantization unit 104, and a quantized transform coefficient 113 is generated. The quantization parameter required for the quantization is set in the encoding control unit 108.

  The quantized transform coefficient 113 is input to the encoding processing unit 105 together with information related to a prediction method such as mode information and a quantization parameter. The encoding processing unit 105 performs entropy encoding (for example, Huffman encoding or arithmetic encoding) on the quantized transform coefficient 113 together with the input mode information and the like. The encoded data 115 entropy-encoded by the encoding processing unit 105 is output to the outside of the encoding unit 100, multiplexed by a multiplexer or the like (not shown), and output buffer (not shown). ). In this case, the index length of the frequency table can be sent for each coding sequence, each picture, or each coding slice. The table length can be sent in sequence units, picture units or slice units, and the index can be sent in macroblock units or block units. The table length can be sent in the header data in sequence units or slice units, and / or the index can be sent in the header data in macro block units.

  The inverse quantization inverse transform unit 106 inversely quantizes the transform coefficient 113 quantized by the transform quantization unit 104 in accordance with a quantization parameter set in the encoding control unit 108, a quantization matrix, and the like. The inversely quantized transform coefficient is inversely transformed (for example, inverse discrete cosine transformation) and restored to the prediction error signal (112). The restored prediction error signal (112) obtained by the inverse transformation is added to the prediction image signal 111 of the prediction mode corresponding to the prediction mode of the prediction error signal supplied from the mode determination unit 103. The addition result signal becomes a local decoded signal 114 and is input to the reference image memory 107. The reference image memory 107 stores the reconstructed image. The reconstructed image stored in the reference image memory 107 in this manner is referred to when a predicted image signal or the like is generated by the internal prediction / mode determination unit 102.

  The encoding loop (the process that flows in the order of internal prediction / mode determination unit 102 → mode determination unit 103 → transform quantization unit 104 → inverse quantization inverse transform unit 106 → reference image memory 107 in FIG. 1) is an encoding target macro. When processing is performed for all modes selectable in the block, a loop is performed once. When the encoding loop is completed for this macroblock, the input image signal 110 of the next macroblock is input and encoding is performed.

  The encoding control unit 108 performs feedback control of the generated code amount, quantization characteristic control, mode determination control, etc., and performs rate control for controlling the generated code amount, control of the internal prediction / mode determination unit 102, external input parameter Control of overall control and encoding. At the same time, it has a function of controlling an output buffer (not shown) and outputting encoded data to the outside at an appropriate timing. The functions of these units can be realized by a program stored in a computer.

  The above is the configuration of the moving picture coding apparatus according to the present embodiment. Hereinafter, the moving picture coding method according to the present invention will be described with reference to FIGS. 2, 3, and 4 taking as an example a case where the moving picture coding apparatus implements.

  FIG. 2 is a block diagram illustrating a configuration of the internal prediction / mode determination unit 102 in the encoding unit 100 of FIG. In FIG. 2, the same components as those in FIG.

  The internal prediction / mode determination unit 102 receives a prediction control unit 501 that receives an index length 116 from the encoding control unit 108, and a mode control unit 502 that receives a local decoded signal (reference image) 114 from the input image signal 110 and the reference image memory 107. Have The prediction control unit 501 and the mode control unit 502 are connected as shown in FIG. That is, the predicted image signal 111 output from the prediction control unit 501 is input to the mode control unit 502 and also input to the inverse quantization inverse transform unit 106 via the mode determination unit 103 in FIG. The decoded signal 504 output from the mode control unit 502 is input to the prediction control unit 501. Further, the input image signal 110 is input to the subtractor 506, and the prediction image signal 111 output from the prediction control unit 501 is subtracted to generate the prediction error signal 112.

  The prediction control unit 501 will be described in detail with reference to FIG. The prediction control unit 501 includes a frequency information table extraction unit 201 and a frequency information table generation unit 202 that receive the frequency information table 116 that receives index length information from the encoding control unit 108 shown in FIG. The frequency information table generation unit 202 tabulates the frequency of the prediction information 209 of the pixel block encoded up to now. When coding a pixel block, the frequency information table of the frequency information table generating unit 202 is updated according to the prediction information 209 given from the control unit 210. The updated frequency information table is sent to the frequency information table extraction unit 201.

  The frequency information table will be specifically described. FIG. 7 shows the update of the frequency information table. The numbers shown in FIG. 7 indicate the prediction mode numbers. The frequency information table is updated every time the mode determination of one pixel block is completed according to the number of the selected prediction mode. First, rearrangement (sorting) is performed on prediction modes of pixel blocks adjacent to the upper and left sides of the encoding target pixel block. For example, the pixel block at the right end in the figure will be described. The prediction mode above this pixel block is 1, and the left prediction mode is 7. At this time, the prediction mode 7 located on the left side in the previous frequency information table is searched from the table and moved to the first place (table index 0). Next, 1 that is the upper prediction mode is searched from the table and moved to the second place (table index 1). Thus, by rearranging (sorting) the upper left prediction mode adjacent to each pixel block in the higher order of the frequency information table, it is possible to obtain prediction mode frequency information.

  Generally, captured images often have similar properties depending on the optical characteristics of the camera and the process of transformation / quantization, and tend to be selected easily in a region where a similar prediction method is used for encoding. is there. If this frequency information table is used, the prediction mode selected before the pixel block to be encoded exists in the upper part of the table, and the prediction mode not used exists in the lower part of the table.

  The index length set in the control unit 210 defines the length of the table index shown in FIG. For example, when the index length is 0, it means that only the prediction mode set to index 0 of the frequency information table is predicted and encoded. Similarly, when the index length is 1, prediction modes 0 to 3 in the table are predicted and encoded. Similarly, when the index length is 2, prediction modes 0 to 7 in the table are predicted and encoded. Similarly, when the index length is 3, prediction modes from 0 to 15 in the table are predicted and encoded. When the index length is N, the number of prediction modes in the table that can be used is determined according to the following equation.

L = 1 << N (3)
In the frequency information table, among the prediction modes used for encoding, a prediction mode having a high frequency exists at the top of the table, so that only a prediction mode that is more likely to be predicted in the prediction mode is generated. . Hereinafter, prediction using this method is referred to as flexible mode prediction.

  The frequency information table generated by the frequency information table generation unit 202 is output to the frequency information table extraction unit 201. The frequency information table extraction unit 201 extracts L prediction modes corresponding to the index length information 116 from the input frequency information table. The frequency information table extraction unit 201 can extract the prediction mode depending on whether the quantization scale value of the encoding target macroblock is large or small. Further, the frequency information table extraction unit 201 can extract the prediction mode depending on whether the resolution of the input image signal is high or low. The extracted prediction mode is output to the prediction mode setting unit 203. The prediction mode setting unit 203 selects one of the input extracted prediction modes and sets the selected prediction mode. This information is set in the control unit 210 as table information 211, and the prediction changeover switch 207 is switched according to the selected prediction mode. The end of the switched switch 207 is connected to one of the corresponding predictors (1, 2,... N) 204.

  The predictor (1, 2,... N) 204 represents a plurality of prediction methods. The prediction modes set by the prediction mode setting unit 203 correspond to the numbers 1 to N of the predictors 204 corresponding to the prediction modes, and prediction is performed by a predetermined prediction method. Here, as an example, The 4 × 4 pixel (direction) prediction defined in H.264 is performed.

  H. There are nine prediction modes of H.264, and as shown in FIG. 8 (a), except for mode 2, each has a different prediction direction by 22.5 degrees. Mode 0 to mode 8 are defined, and mode 2 is DC prediction. FIG. 8B shows the relationship between the prediction block of 4 × 4 pixel prediction and the reference pixel. Pixels from uppercase letters A to M are reference pixels, and pixels from lowercase letters a to p are target prediction pixels.

  A prediction method for the predictor 204 will be described. In the predictor 204, when mode 2 DC prediction is selected, a prediction pixel is calculated by the following equation.

H = (A + B + C + D), V = (I + J + K + L) (4)
a to p = (H + V + 4) >> 3
When the reference pixel cannot be used, the average value of the available reference pixels is predicted. When there is no reference pixel that can be used, a predicted value is calculated with a value that is half of the maximum luminance value of the encoding device (128 for 8 bits). When the other mode is selected, the predictor 204 uses a prediction method that copies the prediction value interpolated from the reference pixel with respect to the prediction direction shown in FIG. Specifically, a prediction value generation method when mode 0 (vertical prediction) is selected will be described with the following equation.

a, e, i, m = A
b, f, j, n = B

c, g, k, o = C
d, h, l, p = D (5)
This mode can be selected only when reference pixels A to D are available. Details of the prediction method are shown in FIG. The luminance values of the reference pixels A to D are copied as they are in the vertical direction and compensated as predicted values.

  A substantially similar framework is used for prediction methods other than prediction modes 0 and 2, and an interpolation value is generated from reference pixels that can be used in the prediction direction, and the prediction is made by copying the value according to the prediction direction. Do. The correspondence between the pixel block and the prediction mode is shown in FIG. N / A in the figure indicates that the corresponding prediction method is not defined.

  The predicted image signal 111 output from the predictor 204 is output to the mode control unit 502 (FIG. 2) of the internal prediction / mode determination unit 102 and also input to the mode determination unit 103 of the encoding unit 100.

  Here, the prediction image signal 111 is added to the prediction mode residual signal 305 generated by the local decoding processing in the mode control unit 502 shown in FIG. The decoded signal 306 is input to the internal reference image memory 205. The internal reference image memory 205 stores the input decoded signal 306. The decoded image stored here is read out as necessary at the time of subsequent prediction image generation, is output to the predictor 204, and is used as a reference image.

  The above is the configuration of the prediction control unit 501 according to the present embodiment. Next, the configuration of the mode control unit 502 will be described with reference to FIG. Here, the same reference numerals are given to the same components as those in FIGS. 1 and 2, and the description thereof is omitted.

  The mode control unit 502 also predicts a block size smaller than the macroblock size. The mode control unit 502 includes an internal mode determination unit 301, an internal transform quantization unit 302, a provisional encoding processing unit 303, an internal inverse quantization inverse transform unit 304, and an adder 305.

  The predicted image signal 111 output from the mode control unit 501 together with the input image signal 110 and the local decoded signal 114 is input to the internal mode determination unit 301 in the mode control unit 502. The internal mode determination unit 301 has a function of determining the prediction mode. The encoding cost of the prediction mode is calculated using Equation (1), Equation (2), etc., and the optimal prediction mode is determined. The predicted image signal 111 that has passed through the internal mode determination unit 301 is input to the internal transform quantization unit 302 and orthogonally transformed. Here, orthogonal transform is performed using, for example, discrete cosine transform. As another embodiment, the transform coefficient may be created using a technique such as wavelet transform or independent component analysis. The transform coefficient 308 is further quantized. The quantization parameter required for the quantization is set in the encoding control unit 108. The transform coefficient 308 is output to the provisional encoding processing unit 303 and also output to the internal inverse quantization inverse transform unit 304. The temporary encoding processing unit 303 performs temporary encoding for calculating the code amount 309 based on the obtained transform coefficient 308. The code amount 309 obtained here may be fed back to the internal mode determination unit 301 to calculate the coding cost. The transform coefficient 308 encoded by the temporary encoding processing unit 303 corresponds to the encoded data 115 of the encoding unit 100.

  On the other hand, the internal inverse quantization inverse transform unit 304 inversely quantizes the obtained transform coefficient 308. Here, processing is performed using the parameters relating to quantization used in the transform quantization unit 302. Further, the inversely quantized transform coefficient is subjected to inverse transform (for example, inverse discrete cosine transform) to generate a quantized prediction residual signal. This prediction residual signal is input to the adder 305 and added to the prediction image signal 111 supplied from the internal mode determination unit 301. An addition signal of the prediction residual signal and the prediction image signal 111 becomes a decoded signal 306. The mode control unit 502 outputs the decoded signal 306 to the prediction control unit 501. The above-described local decoding processing refers to processing corresponding to the internal mode determination unit 301 in the mode control unit 502 ⇒ the internal transform quantization unit 302 ⇒ the internal inverse quantization inverse transform unit 304 ⇒ the adder 305. .

  Internal prediction loop (prediction mode setting unit 203 in FIGS. 3 and 4 ⇒ prediction changeover switch 207 ⇒ predictor 204 ⇒ internal mode determination unit 301 ⇒ internal transform quantization unit 302 ⇒ internal inverse quantization inverse transform unit 304 ⇒ adder 305 ⇒ Processing that flows in the order of the internal reference image memory 205) is a loop when processing is performed for all prediction modes that can be selected in the small pixel block in the macroblock.

  For example, a total of 16 internal prediction loops are performed for 4 × 4 pixel prediction. In this case, the control unit 210 sets a prediction mode corresponding to the prediction mode selected by the frequency information table extraction unit 201 by the prediction mode setting unit 203, operates the prediction changeover switch 207, and executes 16 internal prediction loops. To determine the optimal mode combination. The prediction mode obtained here is sequentially input to the internal mode determination unit 301 of the mode control unit 502 together with the prediction image signal 111, and the optimal mode of the encoding target pixel block is determined.

  When the inner prediction loop is completed for the macroblock, the input image signal 110 of the next macroblock is input and encoding is performed.

  The above is the outline of the moving picture coding apparatus 100 in the present embodiment.

  In the present embodiment, as a prediction method of the predictor 204, H.264 is used. An example using H.264 intra-frame prediction is shown. However, since it does not depend on the prediction method, it is possible to apply a different prediction method. For example, a motion compensation block size may be predicted using a frequency information table during inter-frame prediction, or a motion vector may be predicted. Furthermore, a frequency information table may be created for a prediction mode of unidirectional prediction or bidirectional prediction.

  In the present embodiment, the prediction mode of the left and upper pixel blocks is referred to as the use pixel block position when updating the frequency information table of the prediction mode, but as the adjacent pixel block of the encoding target pixel block The table may be updated in a wider area. Specifically, the prediction mode of the pixel block at the same position that moves back and forth in time may be used, or the upper right pixel block, the upper left pixel block, the upper pixel block, and the upper pixel block, left The frequency information table may be updated using the prediction mode selected in the pixel block further to the left of the pixel block.

  Further, in this embodiment, as the update rule of the prediction mode frequency information table, the prediction mode of the left pixel block is inserted into the index 0, and the prediction mode of the upper pixel block is inserted into the index 1, and sorting is performed. However, the prediction mode of the upper pixel block may be inserted into the index 0, the prediction mode of the left pixel block may be inserted into the index 1, and sorting may be performed, or the adjacent pixel block may be expanded as described above, and the frequency information table You may sort. Further, a plurality of frequency information tables may be provided in accordance with the number of prediction modes, and different update rules may be applied for each table. In any case, the encoder and decoder need to have the same frequency information table.

  In this embodiment, a case has been described in which a processing target frame is divided into short blocks of 16 × 16 pixel size and the like, and encoding is performed sequentially from the upper left block to the lower right block. May be in other orders. For example, the processing may be performed from the lower right to the upper left, or the processing may be performed in a spiral shape from the center of the screen. The processing may be performed from the upper right to the lower left, or the processing may be performed from the periphery of the screen toward the center.

  In the embodiment, the transform quantization block size is divided as a macro block of 16 × 16 pixel units, and the case of an 8 × 8 pixel block or a 4 × 4 pixel block as an intra-frame prediction processing unit has been described. The target block does not need to have a uniform block shape, and can be applied to block sizes of 16 × 8 pixels, 8 × 16 pixels, 8 × 4 pixels, 4 × 8 pixels, and the like. For example, an 8 × 4 pixel block and a 2 × 2 pixel block can be realized with the same framework. Furthermore, it is not necessary to take a uniform block size in one macroblock, and different block sizes may be selected. For example, an 8 × 8 pixel block and a 4 × 4 pixel block may be mixed in a macro block. In this case, as the number of divisions increases, the amount of codes for encoding the division information increases, but more accurate prediction is possible and prediction errors can be reduced. Therefore, the block size may be selected in consideration of the balance between the code amount of the transform coefficient and the locally decoded image. That is, the size of the prediction pixel block corresponding to each coding mode may be switched within a specific pixel block size.

  In the embodiment, a transform quantization unit 104, an inverse quantization inverse transform unit 106, an internal transform quantization unit 302, and an internal inverse quantization inverse transform unit 304 are provided. However, it is not always necessary to perform transform quantization and inverse quantization inverse transform on all prediction error signals, and the prediction error signal may be directly encoded by the encoding processing unit 105 and the provisional encoding processing unit 303. The quantization and inverse quantization processes may be omitted. Similarly, the conversion process and the inverse conversion process may not be performed.

  The above is the configuration of the internal prediction and mode determination unit 102 according to the present embodiment. Hereinafter, the moving picture coding method according to the present invention will be described with reference to FIG.

  When the input image signal 110 for one frame is input to the encoding unit 100 (step S1), the image dividing unit 101 divides the input image signal 110 into a plurality of macroblocks and further divides it into a plurality of small pixel blocks. (Step S2). The input image signal 110 is input to the internal prediction and mode determination unit 102 in units of blocks. At this time, the mode determination unit 103 initializes the index indicating the mode and the cost (step S3).

  Using the input image signal 110, the internal prediction and mode determination unit 102 generates a prediction image signal in one prediction mode that can be selected in the encoding target block (step S4). The frequency table is temporarily updated according to the prediction mode used at this time (step S5).

  A difference between the predicted image signal 111 and the input image signal 110 is taken to generate a prediction error signal 112. The cost cost is calculated from the code amount OH in the prediction mode and the absolute value sum SAD of the prediction error signal 112. Alternatively, the encoding cost is calculated using the equation (2) from the encoding distortion D and the code amount R (step S6).

  The mode determination unit 103 determines whether or not the calculated cost cost is smaller than the minimum cost min_cost (step S7). If the calculated cost cost is smaller (YES), the minimum cost is updated with the cost and the code at that time is also updated. Is stored in the frequency information table as a best_mode index (step S8). At the same time, the predicted image signal 111 is temporarily stored in the internal memory (step S9). If the calculated cost cost is greater than the minimum cost min_cost, the index indicating the mode number is incremented, and it is determined whether the incremented index is the end of the mode (step S10).

  If the index is larger than MAX, which is the last number of the mode (YES), the frequency information table is updated with the determined best mode (step S11). The coding mode information of best_mode and the prediction error signal 112 are passed to the transform quantization unit 104, and transformation and quantization are performed (step S12). The quantized transform coefficient 113 is input to the encoding processing unit 105, and the prediction information 109 is entropy encoded by the encoding processing unit 105 (step S13).

  On the other hand, if the index is smaller than MAX, which is the last number of the mode (NO), the frequency information table is reset (step S14), the process returns to step S4, and the prediction of the encoding mode indicated by the next index is performed. An image signal 111 is generated.

  When encoding with the best_mode is performed, the quantized transform coefficient 113 is input to the inverse quantization inverse transform unit 106, and inverse quantization and inverse transform are performed (step S15). The decoded prediction error signal 112 and the predicted image signal 111 of the best_mode supplied from the mode determination unit 103 are added and stored in the reference image memory 107 as a decoded image signal 114 (step S16).

  Here, it is determined whether or not one frame has been encoded (step S17). When the process is completed (YES), the process returns to step S1, the input image signal of the next frame is input, and the encoding process is performed. On the other hand, if the encoding process for one frame is not completed (NO), the process returns to step S2, the input signal of the next small pixel block is input, and the encoding process is continued.

  In this embodiment, when encoding by multi-pass in units of frames, it is not necessary to change the index length of the frequency information table every time, and only the increase in the code amount is tabulated separately and accumulated, It is possible to calculate the coding cost and determine the optimal index length. Therefore, since the prediction error does not change without using re-encoding, the processing can be greatly reduced.

The above is the outline of the moving picture coding apparatus according to the present embodiment. Next, a syntax encoding method used in this prediction method will be described.
FIG. 10 shows an outline of the syntax structure used in this embodiment. The syntax mainly consists of three parts, and the high-level syntax (401) is packed with syntax information of higher layers above the slice. In the slice level syntax (402), information necessary for each slice is specified, and in the macro block level syntax (403), a change value of a quantization parameter required for each macro block, mode information, and the like are specified. ing.

  Each has a more detailed syntax. The high-level syntax (401) is composed of a sequence parameter sequence syntax (404) and a picture parameter set syntax (405), and a picture level syntax. The slice level syntax (402) includes a slice header syntax (406), a slice data syntax (407), and the like. Furthermore, the macroblock level syntax (403) is composed of a macroblock layer syntax (408), a macroblock prediction syntax (409), and the like.

  In the present embodiment, the required syntax information is a sequence parameter set syntax (404), a picture parameter set syntax (405), a slice header syntax (406), and a macroblock layer syntax (408). I will explain it.

  The seq_flexible_mode_prediction_flag shown in the sequence parameter set syntax of FIG. 11 is a flag indicating whether or not the availability of the flexible mode prediction is changed for each sequence, and when the flag is TRUE, whether to use the flexible mode prediction. It is possible to switch whether or not in sequence units. On the other hand, when the flag is FALSE, flexible mode prediction cannot be used in the sequence.

  The pic_flexible_mode_prediction_flag shown in the picture parameter set syntax of FIG. 12 is a flag indicating whether to use the flexible mode prediction for each picture. When this flag is TRUE, whether to use the flexible mode prediction. It is possible to switch whether or not for each picture. On the other hand, when the flag is FALSE, flexible mode prediction cannot be used in the picture. When seq_flexible_mode_prediction_flag is TRUE, pic_flexble_mode_prediction_flag is always transmitted. At this time, if pic_flexible_mode_prediction_flag is FALSE, table_index_length is transmitted. This syntax represents the index length of the frequency information table that can be used in flexible mode prediction.

  The slice_flexible_mode_prediction_flag shown in the slice header syntax of FIG. 13 is a flag indicating whether or not the availability of the flexible mode prediction is changed for each slice. When this flag is TRUE, whether or not the flexible mode prediction is used. Can be switched in units of slices. On the other hand, when the flag is FALSE, the flexible mode prediction cannot be used in the slice. When pic_flexible_mode_prediction_flag is TRUE, slice_flexible_mode_prediction_flag is always transmitted. At this time, if slice_flexible_mode_prediction_flag is FALSE, table_index_length is transmitted. This syntax represents the index length of the frequency information table that can be used in flexible mode prediction.

  Mb_flexible_mode_prediction_flag shown in the macroblock layer syntax of FIG. 14 is a flag indicating whether to use the availability of flexible mode prediction in the encoding target macroblock. When this flag is TRUE, flexible mode prediction is performed. Use. On the other hand, when the flag is FALSE, flexible mode prediction is not used. When this flag is TRUE, mode_index is always transmitted. This indicates the prediction mode index of the encoding target macroblock, and indicates what number of prediction modes in the frequency information table is selected. BlkSize in the syntax represents the number of encoding target pixel blocks, and 16 corresponds to 4 × 4 pixel blocks, 4 corresponds to 8 × 8 pixel blocks, and 1 corresponds to 16 × 16 pixel blocks. The code amount of mode_index is changed by table_index_length described in the upper syntax in the binarization process. For example, when table_index_length is 4, table indexes from 0 to 15 can be used according to the formula [3]. The mode_index represents the 16 table indexes. When an equal length code is given, it becomes a 4-bit syntax. In binarization, it is desirable that the code amount is designed to be minimized according to the frequency information of the corresponding syntax element. An example of binarization will be described with reference to FIG.

  FIG. 17 shows an example of binarization when the index length is L = 16 (N = 4). When L = 16, there are 16 selectable values of the table index (corresponding to the numbers in the first column of the table). If the occurrence probability of these table index numbers is not known, it is easiest to binarize using isometric codes (second column of the table). The bit string in the table represents mode_index. On the other hand, when the occurrence probability of the table index number is known in advance, it is possible to reduce the number of bits of the mode_index representing the table index by performing binarization using a Huffman code according to the occurrence probability. . An example of the Huffman code is shown in the second column of the table of FIG. Since the frequency information table is updated in accordance with the occurrence frequency in advance, it is possible to further reduce the code amount of the table index by giving a short code as shown in the table above the table index. The fourth column in the table indicates the occurrence probability when the Huffman code is generated.

  The table index number indicating the prediction mode selected by the mode determination unit 103 and the prediction mode setting unit 203 is binarized by a conversion table as shown in FIG. 17, and this binarized bit string is encoded. And the temporary encoding unit 303 performs entropy encoding (for example, arithmetic encoding). In addition to the above encoding, the index may be binarized using a method such as entropy code, Shannon code, or arithmetic code.

  As another embodiment of the present invention, the macroblock_data syntax shown in FIG. 14 may be changed to the syntax shown in FIG. The difference from FIG. 14 is that when the value of mode_index [iBlk] is a predetermined ESCAPE_CODE, ecs_mode_index [iBlk] is further sent.

  For example, in FIG. 17, when the code string 1111 (or Huffman code 11111) is ESCAPE_CODE, it indicates that the corresponding prediction mode is not included in the frequency information table indicated by the current table length table_index_length. ecs_mode_index [iBlk] further indicates an index number after the table length indicated by table_index_length. For example, FIG. 18 shows an example where the length of the frequency information table (indicating the total number of prediction modes) M = 15 and the index length L = 8. When the index of the selected frequency information table protrudes from within the index length L = 8, ESCAPE_CODE is set in mode_index [iBlk]. Further, ecs_mode_index [iBlk] corresponding to the protruding index is sent. For example, when the index of the frequency information table selected in the figure is 10, ESCAPE_CODE = 111 is set in mode_index [iBlk], and 011 is set in ecs_mode_index [iBlk] at the same time. In this way, since all prediction modes can be transmitted to the decoder, expansion such as addition or reduction of prediction modes is facilitated. In addition, syntax design corresponding to each prediction mode is not required.

  As another embodiment of the present invention, syntax as shown in FIG. 16 may be used. In this case, seq_flexible_mode_prediction_flag, pic_flexble_mode_prediction_flag, slice_flexble_mode_prediction_flag, and table_index_length added to the upper syntax are not required. In this case, since the index length is not transmitted, all values in the table can always be used. By updating the frequency information table, the most frequent information is always higher in the block, so by preparing the binarization table represented in the third column of FIG. It is possible to send the table index number by the code amount.

  As described above, in the present embodiment, by using the frequency information table of the selected prediction mode, the appearance existing at the top of the table among the prediction modes given from the table for the encoding target block. The prediction image signal is generated by extracting only the prediction mode with high frequency, and the index length of the extracted table is multiplexed into the syntax and transmitted to the decoder. It is possible to generate a predicted image with reduced hardware calculation costs while maintaining efficiency. That is, suitable encoding can be performed according to the contents of the pixel block.

(Encoding: Second Embodiment)
FIG. 19 is a block diagram showing a prediction control unit 600, which is a block different from the first embodiment, in the configuration of the video encoding apparatus according to the second embodiment. In the present embodiment, unlike the prediction control unit 501 described in the first embodiment, the prediction control unit 600 is provided with two different prediction units, an L0 prediction unit 604 and an L1 prediction unit 608. On the other hand, L0 and L1 frequency information table extraction units 601, 605, L0, L1 frequency information generation units 602, 606, L0, and L1 prediction mode setting units 603, 607 respectively corresponding to the prediction units 604, 608 are provided. Also, an adaptive filter unit 609 that performs filter processing on the prediction images output from these different L0 and L1 prediction units 604 and 608 is provided. The same reference numerals are given to those having the same functions as those already described, and the description thereof is omitted.

  In the internal prediction and mode determination unit 102, a prediction control unit 600 and a mode control unit 502 corresponding to the prediction control unit 501 shown in FIG. The prediction control unit 600 is connected to the mode control unit 502 similarly to the prediction control unit 501, and the prediction image signal 111 output from the prediction control unit 600 is input to the mode control unit 502 and output from the mode control unit 502. The decoded signal 306 is input to the prediction control unit 600. The input image signal 110 is input to the subtractor 506, and the prediction image signal 111 output from the prediction control unit 600 is subtracted to generate a prediction error signal 112. Since the mode control unit 502 also performs prediction of a block size smaller than the macroblock size, an internal mode determination unit 301, an internal transform quantization unit 302, a provisional encoding processing unit 303, an internal inverse unit, as shown in FIG. A quantization inverse transform unit 304 and an adder 305 are included.

  The L0 prediction unit 604 performs prediction using a reference image (local decoded image) that has been encoded and indicates the past in time. On the other hand, the L1 prediction unit 608 performs prediction using a reference image (local decoded image) indicating the future in time that has already been encoded. Each prediction requires a corresponding reference image number (hereinafter referred to as L0REF and L1REF), prediction block shape, prediction information 612 such as motion vector information, and the like.

  The L0 frequency information table generation unit 602 tabulates the frequency of the prediction information 612 related to the L0 prediction of the pixel block encoded so far. When the pixel block to be encoded is encoded, the L0 frequency information table is updated according to prediction information 612 described later given from the control unit 210. The updated L0 frequency information table is sent to the L0 frequency information table extraction unit 601. The L0 frequency information table generated by the L0 frequency information table generation unit 602 is output to the L0 frequency information table extraction unit 601. The L0 frequency information table extraction unit 601 extracts L pieces of L0 prediction information corresponding to the table index length set in the control unit 210 from the input L0 frequency information table. The extracted L0 prediction information is output to the L0 prediction mode setting unit 603. The L0 prediction mode setting unit 603 selects one from the input extracted L0 prediction information. The selected LO prediction information is set in the control unit 210 as L0 table information 613, and a reference image (L0REF) necessary for L0 prediction is called from the internal reference image memory 610. The called reference image (L0REF) is input to the L0 prediction unit 604, and L0 prediction is performed using the reference image (L0REF). The L0 prediction image signal 614 generated by the L0 prediction unit 604 is input to the adaptive filter unit 609.

  On the other hand, the L1 frequency information table generation unit 606 tabulates the frequency of the prediction information 612 related to the L1 prediction of the pixel block encoded up to now. When the pixel block to be encoded is encoded, the L1 frequency information table is updated according to prediction information 612 described later given from the control unit 210. The updated L1 frequency information table is sent to the L1 frequency information table extraction unit 605. The L1 frequency information table generated by the L1 frequency information table generation unit 606 is output to the L1 frequency information table extraction unit 605. The L1 frequency information table extraction unit 605 extracts L pieces of L1 prediction information corresponding to the table index length set in the control unit 210 from the input L1 frequency information table. The extracted L1 prediction information is output to the L1 prediction mode setting unit 607. The L1 prediction mode setting unit 607 selects one of the input extracted L1 prediction information. The selected L1 prediction information is set in the control unit 210 as L1 table information 617, and a reference image (L1REF) necessary for L1 prediction is called from the internal reference image memory 610. The called reference image (L1REF) is input to the L1 prediction unit 608, and L1 prediction is performed. The L1 prediction image signal 615 generated by the L1 prediction unit 608 is input to the adaptive filter unit 609.

  The adaptive filter unit 609 performs filtering on the two input signals using the following expression.

Pred = (L0Pred + L1Pred) >> 1 (6)
Here, Pred represents a predicted image signal obtained after filtering. L0Pred represents the L0 predicted image signal 614 corresponding to the pixel at the same position, and L1Pred represents the L1 predicted image signal 615 to be exercised to the pixel at the same position. Here, a filter other than the average value filter as shown in Expression (6) may be used. Specifically, as shown by the following equation, a filter that weights in the L0 and L1 directions may be used.

Pred = (WL0 × L0Pred + WL1 × L1Pred)
>> (BIT_SHIFT) (7)
WL0 and WL1 represent filter weight coefficients for the L0 predicted image signal 614 and the L1 predicted image signal 615, respectively. BIT_SHIFT is a shift factor introduced to avoid division. At this time, the following relationship holds between the weighting coefficient and the shift coefficient.
WL0 + WL1 = (1 << BIT_SHIFT) (8)
Further, a filter using the following offset may be used.
Pred = (WL0 × L0Pred + WL1 × L1Pred)
>> (BIT_SHIFT) + OFFSET (9)
By changing the OFFSET value, it is possible to effectively predict a temporally continuous change in luminance value.
The predicted image signal 111 generated by the adaptive filter unit 609 is output to the mode control unit 502.

  Next, the prediction information 612 of the control unit 210 will be described. When performing inter coding (interframe coding), prediction mode information indicating which prediction method is used, a reference image index indicating which reference image is used, and which prediction target pixel block is in the reference image Information relating to a motion vector indicating whether the pixel block is indicated and pixel block shape information indicating what shape the prediction target pixel block has are required. In the present embodiment, a frequency information table is generated for the prediction mode information and the reference image index. FIG. 20 shows the correspondence between prediction pixel blocks and modes in the L0 prediction mode and the L1 prediction mode. In FIG. 20, 16 × 16 pixel prediction, 8 × 8 pixel prediction, and 4 × 4 pixel prediction are shown for the L0 prediction modes 0 to 3 and the L1 prediction modes 0 to 3, respectively. The frequency information table will be specifically described. FIG. 7 shows the update of the frequency information table. The numbers shown in FIG. 7 indicate the prediction mode number or the corresponding reference image index number. The frequency information table is updated every time the mode determination of one pixel block is completed according to the selected prediction mode or reference image index number. Since the update of the frequency information table has already been described in the first embodiment, the description thereof is omitted here.

Next, the L0 / L1 prediction unit 604/608 will be described with reference to FIG. The L0 prediction unit 604 performs prediction using a reference image (local decoded image) that has been encoded and indicates the past in time. Specifically, an interpolation image with 1/4 pixel accuracy is created for the prediction target pixel block and the reference image L0REF, and block matching is performed. The numbers described in the area indicated by the L0 prediction reference image in the figure indicate the L0REF number. Here, the amount of positional deviation between the matched pixel block and the prediction target block is mainly measured as a motion vector. After that, the pixel block of the reference image matched with the pixel block for prediction is supplemented with the pixel block for prediction. In this way, predicted image generation is performed. Similarly, the L1 prediction unit 607 performs prediction using a reference image (local decoded image) that has already been encoded and indicates the future in time. Specifically, an interpolation image with 1/4 pixel accuracy is created for the encoding target pixel block and the reference image L1REF, and block matching is performed. In the figure, the numbers described in the region of the L1 prediction reference image indicate the L1REF number. Here, the amount of positional deviation between the matched pixel block and the prediction target block is mainly measured as a motion vector. Thereafter, the predicted pixel block is supplemented with the pixel block of the matched reference image. The generation of the interpolated image may be 1/2 pixel accuracy or 1/8 pixel accuracy.
The above is the configuration of the prediction control unit 600 according to the present embodiment.

  In the present embodiment of the present invention, in FIG. 19, the L0 prediction unit 604 and the L1 prediction unit 608 are block diagrams in which a prediction image is always generated by inputting the index length information 611. However, there are cases where a future reference image cannot be used in the actual encoded frame structure. At this time, in the prediction information 612 given from the control unit 210, the L1REF prohibition information of the reference image is added and input to the L1 frequency information table generation unit 605, and the available L1 prediction modes are limited. As a result, the L1 prediction mode setting unit 607 transmits the L1REF prohibition information to the L1 prediction unit 608. At this time, the control unit 210 transmits L1REF prohibition information to the adaptive filter unit 609 unit. When the L1REF prohibition information is input, the adaptive filter unit 609 switches the predicted image signal according to the following equation (10).

L1Pred = L0Pred (10)
Alternatively, L0Pred is directly output as a predicted image signal.

  In addition, when both the L0REF required by the L0 prediction unit 604 and the L1REF required by the L1 prediction unit 608 are available, when the one-side predicted image signal is output, the above-described L1REF prohibition information or L0REF Prohibition information is added to the prediction information 612 and is input to the frequency information table generation units 602 and 605, so that the L0 prediction image signal 614, the L1 prediction image signal 615, and the filtered prediction image signal are separately obtained. It is possible to output.

  In the present embodiment, the prediction mode of the left and upper pixel blocks is referred to as the use pixel block position when the prediction mode frequency information table is updated, but as the adjacent pixel block of the prediction target pixel block, The table may be updated in a wider area. Specifically, the prediction mode of the pixel block at the same position that moves back and forth in time may be used, and the upper right pixel block, the upper left pixel block, the upper upper pixel block, the left left pixel block, and the like that can be used. The frequency information table may be updated using the prediction mode selected in (1).

  Further, in the present embodiment, an example regarding inter prediction (inter-frame prediction) has been described in detail. However, intra prediction (intra-frame prediction) can also be performed with the same encoder structure. More specifically, the H.264 shown in FIGS. One direction prediction mode defined by H.264 (for example, vertical prediction in 4 × 4 pixel prediction) is L0 prediction, and the other direction prediction mode (for example, vertical left prediction in 4 × 4 pixel prediction) is L1 prediction. At this time, the respective L0 prediction image signals 614 and L1 prediction image signals 615 generated in the prediction control unit 600 are input to the adaptive filter unit 609, and prediction image signals obtained by newly filtering two prediction image signals are obtained. Generated. An L0 frequency information table is generated for the L0 prediction mode, and an L1 frequency information table is generated for the L1 prediction mode. In this manner, by generating a predicted image signal, a new predicted image signal can be generated from the two predicted image signals.

  Next, a syntax encoding method used in this prediction method will be described.

  FIG. 10 shows an outline of the syntax structure used in this embodiment. The syntax mainly consists of three parts, and the high-level syntax (401) is packed with syntax information of higher layers above the slice. Detailed descriptions of the sequence parameter set syntax, picture parameter set syntax, and slice header syntax have already been described above with reference to FIGS.

  Mb_flexible_mode_prediction_flag shown in the macroblock layer syntax of FIG. 22 is a flag indicating whether to use the flexible mode prediction in the prediction target macroblock. When this flag is TRUE, the flexible mode prediction is used. To do. On the other hand, when the flag is FALSE, flexible mode prediction is not used. When this flag is TRUE, mode_index_10 is always transmitted. This indicates the L0 prediction mode index of the macroblock, and indicates what number of prediction modes in the L0 frequency information table is selected. mode_index_l1 indicates an L1 prediction mode index, which indicates which prediction mode in the L1 frequency information table is selected.

  BlkSize in the macroblock layer syntax represents the number of prediction target pixel blocks, which corresponds to 16 for 4 × 4 pixel blocks, 4 for 8 × 8 pixel blocks, and 1 for 16 × 16 pixel prediction. L1PredAvailableFlag is a flag indicating whether or not L1 prediction can be selected in the prediction target pixel block. When this flag is TRUE, mode_index_l1 is transmitted. On the other hand, when it is FALSE, mode_index_l1 is not transmitted.

  The code amounts of mode_index_l0 and mode_index_l1 are changed by table_index_length described in the upper syntax in the binarization process. For example, when table_index_length is 4 (L = 16), table indexes from 0 to 15 can be used according to Equation 3. mode_index_l0 and mode_index_l1 represent the 16 table indexes. When an equal length code is given, a 4-bit syntax is provided. In binarization, it is desirable that the code amount is designed to be minimized according to the frequency information of the corresponding syntax element. An example of binarization has been described above with reference to FIG.

  As another example of the present embodiment, the macroblock_data syntax shown in FIG. 22 may be changed to the syntax shown in FIG. The difference from FIG. 22 is that when the value of mode_index [iBlk] is ESCAPE_CODE determined in advance, ecs_mode_index_10 [iBlk] and ecs_mode_index_l1 [iBlk] are sent.

  For example, when the code string 1111 (or Huffman code 11111) in FIG. 17 is ESCAPE_CODE, the frequency information table indicated by the current table length table_index_length does not include a prediction mode corresponding to the prediction target pixel block. Indicates. ecs_mode_index_l0 [iBlk] and ecs_mode_index_l1 [iBlk] indicate index numbers after the table length indicated by table_index_length. For example, FIG. 18 shows an example where the length of the frequency information table (indicating the total number of prediction modes) M = 15 and the index length L = 8. When the index of the selected frequency information table protrudes from the index length L = 8, ESCAPE_CODE = 111 is set in mode_index_l0 [iBlk] or mode_index_l1 [iBlk]. Further, ecs_mode_index_l0 [iBlk] or ecs_mode_index_l1 [iBlk] corresponding to the protruding index is sent. By doing in this way, since it becomes possible to transmit all the prediction modes of L0 and L1 to a decoder, expansion, such as addition and reduction of a prediction mode, becomes easy. In addition, syntax design corresponding to each prediction mode is not required.

  As another example of the present embodiment, syntax as shown in FIG. 24 may be used. In this case, seq_flexible_mode_prediction_flag, pic_flexble_mode_prediction_flag, slice_flexble_mode_prediction_flag, and table_index_length added to the upper syntax are not required. In this case, since the index length is not transmitted, all values in the table can always be used. By updating the frequency information table, since the most frequent information is always higher in the prediction target block, by preparing a binarization table as expressed in the third column of FIG. 16 described above, It is possible to send table index numbers corresponding to move_index_l0 and mode_index_l1 with a small code amount.

  As described above, it is not necessary to perform a heavy encoding process for all the modes, and it is only necessary to perform the encoding in the selected mode, so that an increase in calculation load can be suppressed. That is, in this embodiment, it is possible to realize high-speed and suitable mode selection and high-speed and high-efficiency video coding.

  Note that when encoding in the selected mode as described above, the generation of the decoded image signal need only be performed for the selected mode, and may not necessarily be executed in the loop for prediction mode determination. Good.

(2) Configuration of moving picture decoding apparatus (decoding: first embodiment)
FIG. 25 shows the configuration of the decoding unit 400 of the video encoding apparatus according to this embodiment. The encoded data sent from the encoding unit 100 and sent via the transmission system or storage system is once stored in the input buffer 401, and the multiplexed encoded data is demultiplexed by the demultiplexing unit 402. The The separated encoded data is input to the code string decoding unit 403, and a parsing process is performed for each frame based on the syntax. That is, the code sequence decoding unit 403 sequentially decodes the code sequence of each syntax of the encoded data for each of the high level syntax, the slice level syntax, and the macroblock level syntax according to the syntax structure shown in FIG. The quantized transform coefficient, quantization matrix, quantization parameter, prediction mode information, information table index length, and the like are restored. Here, the prediction mode information includes an index number to be described later.

  Of the data decoded by the code string decoding unit 403, the decoded transform coefficient is input to the inverse quantization inverse transform unit 404. In the inverse quantization inverse transform unit 404, the input transform coefficient 415 is inversely quantized. Here, the necessary parameters relating to quantization are set from the code string decoding unit 403 to the decoding control unit 409, and are read at the time of inverse quantization. Further, the inversely quantized transform coefficient is inversely transformed (for example, inverse discrete cosine transformation) and output as an error signal 413. Here, the inverse orthogonal transform has been described. However, when wavelet transform or independent component analysis is performed by the encoder, the inverse quantization inverse transform unit 404 executes the corresponding inverse wavelet transform or inverse independent component analysis. May be.

  The error signal 413 is input to the adder 405 and is added to the predicted image signal 411 output from the prediction unit 407 described later. When the error signal 413 and the predicted image signal 411 are added, a decoded signal 414 is obtained, and the decoded signal 414 is output to the reference image memory 406. The decoded signal 414 is further output to the outside of the moving image decoding unit 400 via the reference image memory 406, accumulated in the output buffer 408, etc., and then output at a timing managed by the decoding control unit 309. If the decoded signal is a reference image, the reference image memory 406 sends the decoded signal 414 to the output buffer and stores it in the internal memory. The stored decoded signal 414 is used for prediction as a reference signal 412. On the other hand, if the decoded signal is a non-reference image, the decoded signal 414 is not stored in the internal memory but sent to the output buffer. A signal indicating whether or not the signal is a reference image is multiplexed with the encoded data.

  On the other hand, the prediction mode information 409 and the information table index length 410 decoded by the code string decoding unit 403 are input to the prediction unit 407. The reference signal 412 that has already been decoded is supplied from the reference image memory 406 to the prediction unit 407. The prediction unit 407 generates a predicted image signal 411 based on the input mode information and the like, and supplies it to the adder 405.

  The decoding control unit 409 performs control of output timing for the input buffer 401 and output buffer 408, control of decoding timing, and the like.

  The above is the configuration of the moving picture decoding apparatus according to the present embodiment. Hereinafter, the example which the decoding part 400 implements about the moving image decoding method concerning this invention is demonstrated. In this video decoding, the prediction control unit 407 has the same configuration as that of the prediction control unit 501 in FIG. 3 used in the encoding unit 100 in FIG. 1 and will be described with reference to FIG.

  The frequency information table generation unit 206 tabulates the frequency of prediction mode information of pixel blocks decoded so far. When decoding the decoded pixel block, the frequency information table is updated according to the prediction information 209 given from the control unit 210. The updated frequency information table is sent to the frequency information table extraction unit 201.

  The frequency information table will be specifically described. FIG. 7 shows the update of the frequency information table. The numbers shown in FIG. 7 indicate the prediction mode numbers. The frequency information table is updated every time the mode determination of one pixel block is completed according to the number of the selected prediction mode. First, sorting is performed on prediction modes of pixel blocks adjacent on the left and above the decoding target pixel block. For example, the pixel block at the right end in the figure will be described. The prediction mode above this pixel block is 1, and the left prediction mode is 7. At this time, the prediction mode 7 located on the left side in the previous frequency information table is searched from the table and moved to the first place (table index 0). Next, 1 that is the upper prediction mode is searched from the table and moved to the second place (table index 1). Thus, by sorting the upper left prediction mode adjacent to each pixel block in the upper part of the frequency information table, it is possible to obtain the frequency information of the prediction mode. When this frequency information table is used, the prediction mode selected before the pixel block to be decoded exists in the upper part of the table, and the prediction mode not used exists in the lower part of the table.

  The index length set in the control unit 210 defines the length of the table index shown in FIG. When decoding the prediction mode, it is only necessary to decode bits corresponding to the table index length, so that redundant bits can be reduced.

Hereinafter, prediction using this method is referred to as flexible mode prediction.
The frequency information table generated by the frequency information table generation unit 202 is output to the frequency information table extraction unit 201. The frequency information table extraction unit 201 extracts a prediction mode number corresponding to the index number decoded by the code string decoding unit from the input frequency information table. The extracted prediction mode is output to the prediction mode setting unit 203. The prediction mode setting unit 203 sets the input extraction prediction mode in the control unit 210 and switches the prediction changeover switch 207 to the corresponding prediction mode. The prediction mode set by the prediction mode setting unit 203 corresponds to the prediction unit number corresponding to the prediction mode, and the prediction is performed by a predetermined prediction method. Here, as an example, The 4 × 4 pixel (direction) prediction defined in H.264 is performed.

  H. There are nine prediction modes of H.264, and as shown in FIG. 8 (a), except for mode 2, each has a different prediction direction by 22.5 degrees. Mode 0 to mode 8 are defined, and mode 2 is DC prediction. FIG. 8B shows the relationship between the prediction block of 4 × 4 pixel prediction and the reference pixel. Pixels from uppercase letters A to M are reference pixels, and pixels from lowercase letters a to p are decoding target prediction pixels.

  A prediction method for the predictor 204 will be described. In the predictor 204, when the mode 2 DC prediction is selected, a prediction pixel is calculated using Expression (3).

  When the reference pixel cannot be used, the average value of the available reference pixels is predicted. If there is no reference pixel that can be used, the predicted value is calculated with half the maximum luminance value of the decoding device (128 for 8 bits).

  When the other mode is selected, the predictor 204 uses a prediction method that copies the prediction value interpolated from the reference pixel with respect to the prediction direction shown in FIG. Specifically, a prediction value generation method when mode 0 (vertical prediction) is selected will be described using Equation (4) as an example.

  This mode can be selected only when reference pixels A to D are available. Details of the prediction method are shown in FIG. The luminance values of the reference pixels A to D are copied as they are in the vertical direction and compensated as predicted values.

  A substantially similar framework is used for prediction methods other than prediction modes 0 and 2, and an interpolation value is generated from reference pixels that can be used in the prediction direction, and the prediction is made by copying the value according to the prediction direction. Do. FIG. 9 shows the correspondence between the prediction mode and the prediction pixel block shape according to the present embodiment.

  The predicted image signal 411 output from the predictor 204 is output to the outside of the prediction control unit 407, and is added to the error signal output from the inverse quantization inverse transform unit 404 in the adder 405 described above, and the decoded signal. 414 is generated.

  The above is the configuration of decoding section 400 in the present embodiment of the present invention. Hereinafter, the example which the moving image decoding part 400 implements about the moving image decoding method concerning this invention is demonstrated. The syntax structure used for the video decoding, the respective syntaxes, and binarization are the same as the syntax structure of FIG. 10 used for video encoding and the syntaxes of FIGS. . Also, the binarization example is the same as the example described in the encoding with reference to FIG.

  The number of the table index indicating the prediction mode selected by the mode determination unit 103 and the internal mode setting unit 203 is binarized by a conversion table as shown in FIG. 17, and the binarized bit string is a code string. Entropy decoding (for example, arithmetic decoding) is performed by the decoding unit 403. In addition to the above decoding, binarization may be performed using a method such as entropy code, Shannon code, or arithmetic code. In any case, the same binarization method needs to be performed in the encoding unit and the decoding unit.

  As another example of the present embodiment, the macroblock_data syntax shown in FIG. 14 may be changed to the syntax shown in FIG. The difference from FIG. 14 is that when the value of mode_index [iBlk] is a predetermined ESCAPE_CODE, ecs_mode_index [iBlk] is further sent.

  For example, in FIG. 17, when the code string 1111 (or Huffman code 11111) is ESCAPE_CODE, the frequency information table indicated by the current table length table_index_length does not include a prediction mode corresponding to the decoding target block. Show. ecs_mode_index [iBlk] further indicates an index number after the table length indicated by table_index_length. For example, FIG. 18 shows an example where the length of the frequency information table (indicating the total number of prediction modes) M = 15 and the index length L = 8. When the index of the selected frequency information table protrudes from within the index length L = 8, ESCAPE_CODE is set in mode_index [iBlk]. Further, ecs_mode_index [iBlk] corresponding to the protruding index is set. For example, when the index of the frequency information table selected in the figure is 10, ESCAPE_CODE = 111 is set in mode_index [iBlk], and 011 is set in ecs_mode_index [iBlk] at the same time. By doing so, it becomes possible to receive all prediction modes, so that expansion such as addition or reduction of prediction modes is facilitated. In addition, syntax design corresponding to each prediction mode is not required.

  As another example of the present embodiment, syntax as shown in FIG. 16 may be used. In this case, seq_flexible_mode_prediction_flag, pic_flexble_mode_prediction_flag, slice_flexble_mode_prediction_flag, and table_index_length added to the upper syntax are not required. In this case, since the index length is not received, all values of the table can always be used. By updating the frequency information table, since the most frequent information is always higher in the decoding target block, by preparing a binarization table as expressed in the third column of FIG. 16 described above, It is possible to receive a table index number.

(Decoding: Second Embodiment)
In the present embodiment, the prediction control unit 407 provided in the decoding unit 400 of the decoding device in FIG. 25 is the same as the prediction control unit 501 shown in FIG. 19 provided in the encoding device in the second embodiment. It is configured. That is, unlike the first embodiment, an L0 predictor 604 and an L1 predictor 608, which are two different predictors, are provided. Furthermore, frequency information table extraction units 601 and 605, L0 frequency information table generation unit 602, L0 frequency information table extraction unit 601, L0 prediction mode setting unit 603, and L1 frequency information respectively corresponding to the L0 predictor 604 and the L1 predictor 608. A table generation unit 602, an L1 frequency information table extraction unit 605, and an L1 prediction mode information setting unit 607 are provided. In addition, an adaptive filter unit 609 that performs filter processing on the prediction images output from these different prediction units 604 and 608 is provided. Since the operation of the prediction control unit 407 is the same as the operation of the prediction control unit 501 of the encoding device, detailed description thereof is omitted.

  The predicted image signal generated by the adaptive filter unit 609 is output to the adder 405 of the decoding unit 400. Since the prediction information 612 and the frequency information table have the same configuration and function as those of the encoding device, description thereof will be omitted. Since the L0 / L1 prediction units 604/608 also have the same configuration and function as the encoding device, description thereof will be omitted.

  In the present embodiment, the example related to inter prediction (interframe prediction) has been described in detail. However, intra prediction (intraframe prediction) can also be implemented with the same decoder structure. More specifically, the H.264 shown in FIGS. One direction prediction mode (for example, vertical prediction in 4 × 4 pixel prediction) defined in H.264 is L0 prediction, and the other direction prediction mode (for example, vertical left prediction in 4 × 4 pixel prediction) is L1 prediction. At this time, the L0 prediction image signal 614 and the L1 prediction image signal 615 generated in the prediction control unit 600 are input to the adaptive filter unit 609, and a prediction image signal obtained by newly filtering the two prediction image signals. Is generated. An L0 frequency information table is generated for the L0 prediction mode, and an L1 frequency information table is generated for the L1 prediction mode. In this manner, by generating a predicted image signal, a new predicted image signal can be generated from the two predicted image signals.

  Since the syntax decoding method used in this prediction method is the same as that in the first embodiment, the description thereof will be omitted.

  In this way, when this method is used, the prediction mode used when performing prediction can be changed for each sequence, each slice, or each macroblock, so that a predicted image can be generated with high accuracy for each block. Further, although the present embodiment has been described by taking moving image coding as an example, the present invention can also be applied to still image coding.

  As described above, according to the present invention, by using the frequency information table of the selected prediction mode, the appearance frequency existing at the top of the table is selected from the prediction modes given from the table for the encoding target block. By extracting only the high prediction mode and generating the prediction image signal, the index length of the table at the time of extraction is multiplexed into the syntax and sent to the decoder, so that the encoding efficiency higher than the conventional prediction image generation method While maintaining the above, it is possible to generate a predicted image with reduced hardware calculation costs.

The block diagram which shows the structure of the moving image encoder according to one Embodiment of this invention. The block diagram which shows the internal prediction and mode determination part which are 1 part of the structure of the moving image encoder according to one Embodiment. The block diagram which shows the prediction control part which is 1 part of the structure of the internal prediction and mode determination part according to one Embodiment. The block diagram which shows the mode control part which is 1 part of the structure of the internal prediction and mode determination part according to one Embodiment. The figure which shows the encoding order and block size concerning one Embodiment. The flowchart which shows the flow of the encoding process concerning one Embodiment. The figure which shows the update method of the frequency information table concerning one Embodiment. The figure which shows the prediction direction of the reference image utilized for the direction prediction concerning one Embodiment. The table showing the name of the prediction method in a screen according to one Embodiment. Schematic of the syntax structure according to one embodiment. The figure which shows the data structure of the sequence parameter set syntax according to one Embodiment. The figure which shows the data structure of the picture parameter set syntax according to one Embodiment. The figure which shows the data structure of the slice header syntax according to one Embodiment. The figure which shows the data structure of the macroblock layer syntax according to one Embodiment. The figure which shows the data structure of the macroblock layer syntax according to one Embodiment. The figure which shows the data structure of the macroblock layer syntax according to one Embodiment. The table which shows the binarization of the frequency information table index according to one embodiment. The table which shows the outline of the binarization of the frequency information table index according to one Embodiment, and an escape code. The block diagram which shows the prediction control part which is 1 part of the structure of the internal prediction and mode determination part according to one Embodiment of this invention. The table showing the prediction name of L0 prediction mode and L1 prediction mode according to one Embodiment of this invention. Schematic showing the prediction method using L0 prediction mode and L1 prediction mode according to one Embodiment of this invention. The figure which shows the data structure of the macroblock layer syntax according to one Embodiment of this invention. The figure which shows the data structure of the macroblock layer syntax according to one Embodiment of this invention. The figure which shows the data structure of the macroblock layer syntax according to one Embodiment of this invention. The block diagram which shows the structure of the moving image decoding apparatus according to one Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Coding part, 101 ... Screen division | segmentation part, 102 ... Internal prediction / mode determination part,
DESCRIPTION OF SYMBOLS 103 ... Mode determination part, 104 ... Transformation quantization part, 105 ... Coding process part, 106 ... Dequantization inverse transformation part, 107 ... Reference image memory, 108 ... Encoding control part, 201 ... Frequency information table extraction part, 202 ... Frequency information table generation unit, 203 ... Prediction mode setting unit, 204 ... Predictor, 205 ... Internal reference image memory, 207 ... Prediction changeover switch, 210 ... Control unit,
301 ... Internal mode determination unit, 302 ... Internal transform quantization unit, 303 ... Temporary encoding processing unit,
304: internal dequantization inverse transform unit 401: input buffer 402: demultiplexing unit
403 ... Code sequence decoding unit, 404 ... Inverse quantization inverse transformation unit, 405 ... Adder, 406 ... Reference image memory, 407 ... Prediction control unit, 408 ... Output buffer, 409 ... Decoding control unit,
501 ... Prediction control unit, 502 ... Mode control unit, 506 ... Subtractor, 600 ... Prediction control unit,
601... L0 frequency information table extraction unit, 602... L0 frequency information table generation unit,
603 ... L0 prediction mode setting unit, 604 ... L0 prediction unit, 605 ... L1 frequency information table extraction unit, 606 ... L1 frequency information table generation unit, 607 ... L1 prediction mode setting unit,
608 ... L1 prediction unit, 609 ... adaptive filter unit, 610 ... internal reference image memory

Claims (27)

Preparing a frequency information table indicating the frequency of selection of incidental information related to the prediction mode;
Dividing the input image into a plurality of pixel blocks;
Selecting incidental information related to a prediction mode according to an encoding target pixel block of the pixel block;
Generating a prediction image for the encoding target pixel block using a reference image based on the selected supplementary information;
Determining an optimum prediction mode based on a prediction error between an input image and a prediction image and a code amount of the prediction mode, and rearranging a selection frequency order of prediction modes in the frequency information table according to the determined prediction mode;
Generating an index of the sorted frequency information table;
Extracting one or more additional information from the index for the encoding target pixel block;
Generating a prediction signal corresponding to the extracted incidental information;
Calculating the cost of the prediction mode and selecting one encoding mode from the cost;
Encoding the prediction error signal and the table length of the frequency information table according to the selected encoding mode, and encoding an index number in the frequency information table indicating the selected encoding mode;
An image encoding method comprising: Preparing a frequency information table indicating a selection frequency of a plurality of prediction modes;
Dividing the input image into a plurality of pixel blocks;
Selecting a prediction mode according to an encoding target pixel block of the pixel block;
Generating a prediction image for the encoding target pixel block using a reference image based on the selected prediction mode;
Determining an optimum prediction mode based on a prediction error between an input image and a prediction image and a code amount of the prediction mode, and rearranging a selection frequency order of prediction modes in the frequency information table according to the determined prediction mode;
Generating an index of the sorted frequency information table;
Extracting one or more prediction modes from prediction modes corresponding to the index for the encoding target pixel block;
Generating a prediction signal and prediction mode information corresponding to the extracted prediction mode;
Calculating a coding cost of the prediction mode, and selecting one coding mode from the coding cost;
Encoding a prediction error signal in the selected encoding mode, a table length of the frequency information table, and an index number in the frequency information table indicating the selected encoding mode;
An image encoding method comprising: Dividing the input image into a plurality of pixel blocks;
Selecting a prediction mode according to an encoding target pixel block of the pixel block;
Generating a first frequency information table by tabulating the selection frequency of the prediction mode for the selected first type prediction mode;
Generating a second frequency information table by tabulating the selection frequency of the prediction mode with respect to the selected second type prediction mode;
Generating an index of the first and second type frequency information table;
A prediction mode extraction step of extracting one or more prediction modes from the prediction modes given from the first and second frequency information tables for the encoding target pixel block of the pixel block;
Generating a first type prediction signal, a second type prediction signal, and prediction mode information corresponding to the prediction mode extracted from the first and second frequency information tables, respectively.
Performing a filtering process on the first type prediction signal and the second type prediction signal to generate one prediction signal;
Calculating a prediction error signal for the prediction mode and selecting one encoding mode;
The first type and the prediction error signal generated in the selected encoding mode, the table length of the first frequency information table, the table length of the second information table, and the selected encoding mode; An encoding step for encoding an index number corresponding to the second type;
An image encoding method comprising:   The encoding step includes a step of performing a conversion process on the prediction error signal, and a step of performing a quantization process on the converted coefficient to generate a quantized conversion coefficient, The image encoding method according to claim 1 or 2.   The image encoding method according to claim 1, further comprising a step of switching a size of a prediction pixel block corresponding to each encoding mode within a specific pixel block size.   3. The image encoding method according to claim 1, wherein the prediction mode extracting step includes a step of sending an index length of the frequency table for each encoding sequence, each picture, or each encoding slice.   The prediction mode extraction step includes a step of switching whether or not to extract the prediction mode depending on whether the quantization scale value of the encoding target macroblock is large or small. The image encoding method according to claim 1 or 2.   The prediction mode information extraction step switches whether to extract the mode information according to whether the resolution of the input image signal is high or low. The image encoding method described. The encoding mode selection step includes a code amount calculation step of calculating a code amount when a signal generated in the selected encoding mode is encoded;
A step of locally decoding a signal generated in the selected encoding mode to generate a locally decoded image; and an encoding distortion calculating step of calculating an encoding distortion representing a difference from the input image signal. The image encoding method according to claim 1, wherein the image encoding method is performed.   3. The image encoding method according to claim 1, further comprising a step of sending the table length in sequence units, picture units or slice units, and sending the index in macroblock units or block units.   3. The method according to claim 1, further comprising a step of sending the table length including header data in sequence units or slice units and / or sending the index included in header data in macro block units. Image coding method. Decoding the encoded signal for each pixel block according to the encoding mode of the encoded signal;
Decoding the table length of the frequency information table for tabulating the selection frequency of the additional information regarding the prediction mode of the decoded pixel block;
Generating the frequency information table based on the decrypted table length;
Decoding an index number of the frequency information table;
Extracting additional information corresponding to the decoded pixel block from the index;
Generating a prediction signal and a prediction mode corresponding to the extracted additional information;
Generating a prediction error signal based on the decoded signal;
Adding a prediction signal and a prediction error signal to generate a decoded image;
An image decoding method comprising: Decoding the encoded signal for each pixel block according to the encoding mode of the encoded signal;
Decoding a table length of a frequency information table for tabulating a selection frequency of a prediction mode of a decoded pixel block;
Generating the frequency information table based on the decrypted table length;
Decoding an index number of the frequency information table;
Extracting one or more prediction modes from prediction modes corresponding to the index;
Generating a prediction signal corresponding to the extracted prediction mode;
Generating a prediction error signal based on the decoded signal;
Adding a prediction signal and a prediction error signal to generate a decoded image;
An image decoding method comprising: Decoding the encoded signal for each pixel block according to the encoding mode of the encoded signal;
Decoding the table length of the frequency information table for tabulating the selection frequencies of the first type prediction mode and the second type prediction mode of the decoded pixel block;
Generating a first type frequency information table by tabulating the selection frequency of the prediction mode for the first type prediction mode;
Generating a second type frequency information table by tabulating the selection frequency of the prediction mode for the second type of prediction mode;
Respectively decoding the index numbers of the first and second type frequency information tables;
Extracting a prediction mode corresponding to each of the first type and second type prediction modes from the index;
Generating a first type prediction signal, a second type prediction signal, and prediction mode information corresponding to the extracted first type and second type prediction modes;
Performing a filtering process on the first type prediction signal and the second type prediction signal to generate one prediction signal;
Generating a prediction error signal based on the decoded signal;
Adding a prediction signal and a prediction error signal to generate a decoded image;
An image decoding method comprising:   14. The prediction error signal generation step includes a step of inversely quantizing a decoded coefficient and a step of generating a prediction error signal by inversely transforming the inverse quantization transform coefficient. Image decoding method.   The image decoding method according to claim 12 or 13, further comprising a step of switching a size of a predicted pixel block corresponding to each encoding mode within a specific pixel block size.   14. The image decoding method according to claim 12, wherein the prediction mode extraction step sends the index length of the frequency table for each sequence, each picture, or each slice when performing the extraction of the prediction mode.   The image decoding according to claim 12 or 13, wherein the prediction mode extraction step extracts the prediction mode depending on whether a quantization scale value of a decoding target macroblock is large or small. Method.   The image decoding method according to claim 10 or 11, wherein the prediction mode extracting step extracts the mode information according to whether the resolution of the decoding target input image signal is high or low. . A memory for storing a frequency information table indicating the selection frequency of the incidental information related to the prediction mode;
A dividing unit for dividing the input image into a plurality of pixel blocks;
A selection unit that selects incidental information related to a prediction mode according to a pixel block to be encoded of the pixel block;
A prediction unit that generates a prediction image for the encoding target pixel block using a reference image based on the selected supplementary information;
A table updating unit that determines an optimal prediction mode based on a prediction error between an input image and a prediction image and a code amount of the prediction mode, and rearranges a selection frequency order of prediction modes in the frequency information table according to the determined prediction mode; ,
An index generation unit for generating an index of the sorted frequency information table;
An extraction unit that extracts one or more auxiliary information from the index for the encoding target pixel block;
A prediction signal generation unit that generates a prediction signal corresponding to the extracted auxiliary information;
A selection unit that calculates a cost of the prediction mode and selects one encoding mode from the cost;
An encoding unit that encodes the prediction error signal, the table length of the frequency information table according to the selected encoding mode, and an index number in the frequency information table indicating the selected encoding mode;
An image encoding apparatus comprising: A decoding unit that decodes the encoded signal for each pixel block according to an encoding mode of the encoded signal;
Table decoding for decoding the table length of the frequency information table for tabulating the selection frequency of the additional information related to the prediction mode of the decoded pixel block;
A frequency information table generating unit that generates the frequency information table based on the decrypted table length;
An index decoding unit for decoding an index number of the frequency information table;
An additional information extraction unit that extracts additional information corresponding to the decoded pixel block from the index;
A prediction signal generation unit that generates a prediction signal and a prediction mode corresponding to the extracted additional information;
A prediction error signal generation unit that generates a prediction error signal based on the decoded signal;
A decoded image generating unit that generates a decoded image by adding the prediction signal and the prediction error signal;
An image decoding apparatus comprising: A procedure for preparing a frequency information table indicating the selection frequency of the incidental information related to the prediction mode;
Dividing the input image into a plurality of pixel blocks;
A procedure for selecting incidental information related to a prediction mode according to an encoding target pixel block of the pixel block;
Generating a prediction image for the encoding target pixel block using a reference image based on the selected supplementary information;
A step of determining an optimal prediction mode based on a prediction error between an input image and a prediction image and a code amount of the prediction mode, and rearranging a selection frequency order of prediction modes in the frequency information table according to the determined prediction mode;
A procedure for generating an index of the sorted frequency information table;
A procedure for extracting one or more additional information from the index for the encoding target pixel block;
Generating a prediction signal corresponding to the extracted incidental information;
Calculating a cost of the prediction mode and selecting one encoding mode from the cost;
Encoding the prediction error signal, the table length of the frequency information table according to the selected encoding mode, and the index number in the frequency information table indicating the selected encoding mode;
An image encoding program for causing a computer to execute. A procedure for decoding the encoded signal for each pixel block according to an encoding mode of the encoded signal;
A procedure for decoding the table length of the frequency information table for tabulating the selection frequency of the additional information related to the prediction mode of the decoded pixel block;
Generating the frequency information table based on the decrypted table length;
A procedure for decoding an index number of the frequency information table;
A procedure for extracting additional information corresponding to a decoded pixel block from the index;
Generating a prediction signal and a prediction mode corresponding to the extracted additional information;
Generating a prediction error signal based on the decoded signal;
A procedure for adding a prediction signal and a prediction error signal to generate a decoded image;
Decoding program for causing a computer to execute. Based on a selection history of a plurality of information related to prediction modes in blocks encoded before each of the plurality of blocks into which the input image is divided, in order of the high possibility of being selected in each block according to a predetermined rule A table generating step for generating a table in which the plurality of pieces of information are arranged;
A selection step of selecting selection information to be used for prediction of each block from the plurality of pieces of information;
A prediction step of generating a prediction residual signal of each block from the image signal of each block by performing prediction according to the selection information;
An encoding step of generating encoded data by encoding the prediction residual signal of each block, information indicating the length of the table, and an index number corresponding to the selection information in the selection table;
An image encoding method comprising:   25. The image encoding method according to claim 24, wherein, in the table generation step, the table is generated by using a part extracted in order from a higher possibility of being selected from the plurality of pieces of information. Based on a selection history of a plurality of information related to prediction modes in blocks encoded before each of the plurality of blocks into which the input image is divided, in order of the high possibility of being selected in each block according to a predetermined rule A table generator for generating a table in which the plurality of pieces of information are arranged;
A selection unit that selects selection information used for prediction of each block from the plurality of pieces of information;
A prediction unit that generates a prediction residual signal of each block from the image signal of each block by performing prediction according to the selection information;
An encoding unit that generates encoded data by encoding the prediction residual signal of each block, information indicating the length of the table, and an index number corresponding to the selection information in the selection table;
An image encoding device having:   27. The image encoding apparatus according to claim 26, wherein the table generation unit generates the table using a part extracted in order from a higher possibility of being selected from the plurality of pieces of information.

Images