JP2005086249A - Dynamic image coding method and dynamic image coding apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a dynamic image coding method for carrying out moving picture coding at a faster speed by suppressing increase in a computing volume without decreasing the coding efficiency. <P>SOLUTION: The dynamic image coding method for dividing an input image signal into a plurality of pixel blocks, selecting one mode among a plurality of coding modes by each pixel block; and carrying out coding by each pixel in the selected coding mode, generates an estimated parameter different from each pixel block and each coding mode, uses the estimated parameter to estimate a generated code amount of the pixel block of the input image signal and coding distortion, and determines an optimum coding mode on the basis of the estimated generated code amount and the estimated coding distortion. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a moving picture coding method capable of adaptively selecting one mode from a plurality of coding modes for each pixel block.

  A moving picture encoding method using predictive coding having a plurality of prediction modes and a plurality of block shapes is a combination of ITU-T and ISO / IEC. H. H.264 and ISO / IEC 14496-10 (hereinafter referred to as H.264).

  H. In the H.264 intra-frame predictive coding, the shape of a 4 × 4 prediction block and a 16 × 16 prediction block can be selected for each prediction block, and prediction can be performed from a plurality of prediction modes. Yes. In the intra-frame predictive encoding of MPEG-1, MPEG-2, and MPEG-4, which are conventional encoding methods, there are few selectable prediction modes. In H.264, the prediction block shape is divided into small blocks such as a 16 × 16 pixel block and a 4 × 4 pixel block, and it is possible to select an optimal encoding mode from abundant prediction modes according to image characteristics. It is possible.

  In inter-frame predictive coding, conventionally, prediction of 8 × 8 pixel block size or larger has been used. In H.264, prediction with a 4 × 4 pixel block size is possible, and more accurate prediction than in the conventional method is possible by motion compensation from a plurality of already encoded reference images. As described above, the number of prediction modes that can be selected for each block is increased, and an encoding mode with higher prediction efficiency is selected to improve the encoding efficiency.

  Also, a code amount-coding distortion optimization method has been proposed in which encoding control is performed using a Lagrange undetermined multiplier method with the generated code amount as a constraint. For this, the encoding mode with the highest encoding efficiency is selected from the generated code amount actually obtained by encoding and the encoding distortion (for example, the square error and the mean square error between the original image and the local decoded image). Is the method. However, as a problem of this method, when the number of encoding modes and the number of block shapes increase, it becomes necessary to repeatedly encode the number of combinations that the mode can take, so that the actual calculation time increases.

  In addition, a moving image encoding method using a Lagrange undetermined multiplier has been proposed (Reference 1). According to this method, in the inter-frame predictive encoding, the motion vector information is used, and the reference degree table is created by using the reference from the other frames with respect to the encoding target block. When determining the code amount allocation in this referenced level table, the referenced level and the Lagrange undetermined multiplier are made to correspond one-to-one. A Lagrangian cost is obtained from a given undetermined multiplier, and encoding is performed repeatedly. In this method, the undetermined multiplier is determined using the degree of reference, and the amount of calculation increases because the amount of generated code and encoding distortion are actually measured. As in the case of H.264, the speed is not increased with respect to the case where the prediction mode or the block shape increases.

In addition, in order to suppress degradation of image data caused by noise, a quantization step is determined for each block so that the code amount for one frame is constant, and the generated code amount and encoding distortion in the target block are immediately before. There has been proposed a moving picture coding method in which a prediction mode is estimated based on data encoded in (3) and a prediction mode having a small sum of the estimated generated code amount and coding distortion is selected (Reference 2). Since this method considers the mode determination of inter-frame predictive coding and intra-frame predictive coding, it does not consider the case where a plurality of prediction modes can be taken in a certain coding mode, and the estimated value for the coding mode However, since it does not vary depending on the prediction mode, the coding efficiency is not improved.
JP2002-218467 JP 2001-332031 A

  As described above, in the moving picture coding method in which various coding modes can be selected for each block, in order to select the optimum coding mode, the generated code is increased as the number of types of prediction modes increases. There is a problem that the amount of calculation for obtaining the amount and coding distortion increases. Further, there is a problem that as the target code amount becomes smaller, the difference in image quality characteristics between adjacent blocks becomes obvious, and the subjective image quality decreases.

  In the present invention, for each prediction mode corresponding to a plurality of pixel blocks, the generated code amount and encoding distortion when actually encoded are estimated using the estimation parameter, so that the calculation efficiency is not reduced. An object of the present invention is to provide a higher-speed moving picture encoding method while suppressing an increase in the amount. Furthermore, a function for reducing an error with an adjacent block is introduced into the cost function, image quality deterioration due to a difference in prediction characteristics between blocks is reduced, and subjective image quality is improved.

  According to a first aspect of the present invention, an input image signal is divided into a plurality of pixel blocks, one mode is selected from a plurality of encoding modes for each pixel block, and the pixel block is selected in the selected encoding mode. In the moving picture coding method in which coding is performed every time, different estimation parameters are generated for each pixel block and each coding mode, and the generated code amount and coding distortion of the pixel block of the input image signal are generated using the estimation parameters. Is provided, and an optimal encoding mode is determined based on the estimated generated code amount and encoding distortion.

  According to a second aspect of the present invention, means for generating different estimation parameters for each pixel block and for each coding mode, and means for estimating a generated code amount and coding distortion of a pixel block of an input image signal using the estimation parameters And means for determining an optimal encoding mode from a plurality of encoding modes based on the estimated generated code amount and encoding distortion; and means for encoding each pixel block in the optimal encoding mode; A moving picture encoding apparatus is provided.

  According to the present invention, the generated code amount R and the coding distortion D are calculated using different estimation parameters for each prediction mode, and the hierarchical prediction mode determination or the mode determination considering the encoding distortion of adjacent pixels By performing the above, it is possible to significantly reduce the amount of calculation required for mode determination while suppressing image quality deterioration.

  FIG. 1 is a block diagram showing a configuration of a moving picture coding apparatus according to an embodiment of the present invention.

  According to FIG. 1, the moving image signal is input to the encoding unit 114. The subtractor 101 of the encoding unit 114 performs variable length encoding via an orthogonal transform unit (for example, a discrete cosine transform) 102 that orthogonally transforms an input signal and a quantization unit 103 that quantizes an orthogonal transform coefficient (DCT coefficient). Connected to the unit 111. The output terminal of the quantization unit 103 is connected to the frame memory 107 via an inverse quantization unit 104, an inverse orthogonal transform unit 105, and an adder 106 that constitute a local decoder. An output terminal of the frame memory 107 is connected to input terminals of an inter-frame prediction unit 108 and an intra-frame prediction unit 109 described later. Output terminals of the inter-frame prediction unit 108 and the intra-frame prediction unit 109 are connected to an MB (macroblock) prediction mode selection unit 115 described later. The output terminals of the MB prediction mode selection unit 115 are connected to the subtracter 101 and the adder 106, respectively.

  The output terminal of the variable length coding unit 111 is connected to the output buffer 113 via the multiplexing unit 112. The encoding control unit 110 is provided to control the encoding unit 114.

  In the above configuration, the input moving image signal is divided into a plurality of pixel blocks, and is input to the intra-frame prediction unit 109 and the inter-frame prediction unit 108 for each pixel block. The intra-frame prediction unit 109 or the inter-frame prediction unit 108 uses the reference frame recorded in the frame memory 107 to select an optimal prediction mode from among a plurality of prediction modes, and performs prediction using the selected prediction mode. A pixel signal is generated. Based on the prediction pixel signal, the MB prediction mode selection unit 115 selects an optimal prediction mode. That is, the MB prediction mode selection unit 115 is based on the prediction pixel signal generated by the optimal prediction mode selected by the intra-frame prediction unit 109 and the prediction image signal generated by the optimal prediction mode selected by the inter-frame prediction unit 108. Selects an optimal prediction mode of one of the inter-frame prediction mode and the intra-frame prediction mode. A prediction pixel signal corresponding to the selected prediction mode is input to the subtractor 101. A subtracter 101 calculates a prediction residual signal between the prediction pixel signal and the input image signal. This prediction residual signal is input to the orthogonal transform unit 102 and subjected to orthogonal transform (for example, DCT transform).

  The orthogonal transform coefficients are quantized by the quantization unit 103, and the quantized orthogonal transform coefficients are variable length codes along with prediction mode information output from the MB prediction mode selection unit 115, information on prediction methods such as quantization coefficients, and the like. The encoding unit 111 performs variable length encoding. These encoded data are multiplexed by the multiplexing unit 112 and output as encoded data through the output buffer 113.

  Further, the quantized orthogonal transform coefficient is locally decoded through the inverse quantization unit 104 and the inverse orthogonal transform unit 105. The local decoded signal, that is, the decoded prediction residual signal is added to the prediction signal in the adder 106 and stored in the frame memory 107 as a reference frame.

  The encoding control unit 110 performs feedback control and quantization characteristic control of the generated code amount, and performs rate control for controlling the generated code amount, control of the prediction unit, and control of the entire encoding.

  A specific prediction mode will be described with reference to FIGS.

  In the predictive encoding of this embodiment, there are a plurality of block shapes for each macroblock, and each has a prediction mode. For example, H.C. For luminance signals in intra-frame prediction such as H.264, two types of macroblocks having 16 4 × 4 pixel blocks and macroblocks having one 16 × 16 pixel block have been proposed. A 4 × 4 pixel block has nine prediction modes, and a 16 × 16 pixel block has four prediction modes.

  FIG. A block shape of a macroblock used for H.264 intra-frame prediction is shown. H. In H.264, the encoding target frame is divided into 16 16 × 16 pixel macroblocks, and the macroblock is further divided into 16 4 × 4 pixel blocks in intra-frame prediction. In the case of a 4 × 4 pixel block, 4 × 4 prediction is sequentially performed 16 times in the frame.

  FIG. All prediction modes in 4 × 4 pixel block of H.264 intra-frame prediction, that is, vertical prediction mode, horizontal prediction mode, DC prediction mode, orthogonal lower left prediction mode, orthogonal lower right prediction mode, vertical right prediction mode, horizontal lower prediction mode, A vertical left prediction mode and a horizontal upper prediction mode are shown. Symbols A to M are reference pixel signals that have already been encoded. For example, in the vertical prediction mode, prediction is performed from reference pixels A, B, C, and D, respectively, along the vertical direction. In the DC prediction mode, an average value of reference pixels A to D and J to M is obtained, and all pixels of a 4 × 4 block are predicted based on the average value.

  FIG. 2 shows the configuration of the intra-frame prediction unit 109 of FIG. According to this, the reference image signal 205 obtained from the frame memory 107 in FIG. 1 is divided or aligned by the block shape control unit 201 into a shape used in each pixel block. According to the pixel block shape, the intra-frame prediction mode and intra-frame prediction in each pixel block shape are performed by the 4 × 4 block intra-frame prediction unit 202 and the 16 × 16 block intra-frame prediction unit 203 in the pixel block intra-frame prediction unit 211. A mode is selected.

  The intra-block prediction mode determination unit 204 performs mode determination (for example, selection of a prediction mode with a small square error between the decoded pixel signal and the input image signal) from the plurality of prediction modes obtained by the pixel block intra-frame prediction units 202 and 203. Do. That is, the intra-block prediction mode determination unit 204 determines the optimal prediction mode, and outputs the prediction pixel signal 207, the decoded pixel signal 208, the orthogonal transform coefficient 209, the quantization parameter 210, and the prediction mode information 212. The intra-block prediction mode determination unit 204 may use mode determination using an estimation parameter, which will be described later, instead of the above-described mode determination, or perform mode determination using a generated code amount and an actual measurement value of coding distortion. Also good.

  3 and 4 show a 4 × 4 block intra-frame prediction unit 202 and a 16 × 16 block intra-frame prediction unit 203. When the 4 × 4 pixel block is selected by the block shape control unit 201 in FIG. 2, the reference pixel signal 205 is input to the 4 × 4 prediction unit 301 in FIG. 3. When the 4 × 4 prediction unit 301 inputs the prediction pixel signal 309 to the prediction mode determination unit 304, the prediction mode determination unit 304 determines the optimal prediction mode of the selected pixel block.

  In other words, at this time, the prediction pixel signal 309 is subtracted from the input image signal 206 by the subtractor 310, and the prediction residual signal 311 is generated. The prediction residual signal 311 is sent to the code amount estimation unit 302 and the coding distortion estimation unit 303. The code amount estimation unit 302 uses the prediction mode 319, the quantization parameter 320, and the prediction residual signal 311 to calculate the generated code amount estimation value R ^ (312). Similarly, the coding distortion estimation unit 303 calculates the coding distortion estimated value D ^ (313) of the decoded pixel signal 318 and the input image signal 206 using the prediction mode 319 and the prediction pixel signal 309. The prediction mode determination unit 304 calculates the cost of each prediction mode according to the Lagrange undetermined multiplier method using the estimated generated code amount R ^ and the estimated coding distortion D ^, and selects the prediction mode with the minimum cost as the optimal prediction mode. The prediction mode information 319 is output. Further, the prediction mode determination unit 304 selects and outputs the prediction pixel signal 321 and the quantization parameter 320.

  The predicted pixel signal 321 is subtracted from the input image signal by the subtractor 314 to generate a predicted residual signal 315. The prediction residual signal 315 is orthogonally transformed and quantized by the orthogonal transformation unit 305 and the quantization unit 306. The quantized data is input to the adder 316 via the inverse quantization unit 307 and the inverse orthogonal transform unit 308, and is added to the predicted pixel signal 321. As a result, a decoded pixel signal 318 is generated. The decoded pixel signal 318 is used as a reference pixel signal for the next 4 × 4 block.

  Since the calculation of the prediction residual signal 315 is redundant, the prediction residual signal input to the estimation units 302 and 303 may be stored and directly input to the orthogonal transform unit 305. The above-described mode determination process is sequentially performed on each 4 × 4 pixel block in the macroblock, that is, 16 blocks. The prediction mode of the 4 × 4 pixel block becomes a 4 × 4 prediction combination candidate of the macroblock, and is input to the intra-block prediction mode determination unit 204.

  4, when a 16 × 16 pixel block is selected by the block shape control unit 201 in FIG. 2, the reference pixel signal 205 is input to the 16 × 16 block prediction unit 401. Is done. The 16 × 16 block prediction unit 401 generates a prediction pixel signal 409 with reference to the reference image signal 205. When the prediction pixel signal 409 is input to the 16 × 16 block prediction mode determination unit 404, the 16 × 16 block prediction mode determination unit 404 determines the optimal prediction mode of the shape of the selected pixel block.

  That is, at this time, the prediction pixel signal 409 is subtracted from the input image signal 206 by the subtractor 410 to generate the prediction residual signal 411. The prediction residual signal 411 is sent to the code amount estimation unit 402 and the coding distortion estimation unit 403. The code amount estimation unit 402 uses the prediction mode 419, the quantization parameter 420, and the prediction residual signal 411 to calculate the generated code amount estimated value R ^ (412). Similarly, the coding distortion estimation unit 403 calculates a coding distortion estimated value D ^ (413) of the decoded pixel signal 418 and the input image signal 206 using the prediction mode 419 and the prediction pixel signal 409. The prediction mode determination unit 404 calculates the cost of each mode based on the Lagrange undetermined multiplier method using the estimated code amount R ^ and the estimated coding distortion D ^, and selects the prediction mode that minimizes the cost as the optimum prediction mode. The prediction mode information 419 is output. Further, the 16 × 16 block prediction mode determination unit 404 outputs a quantization parameter 420 and a prediction pixel signal 421.

  The prediction pixel signal 421 is subtracted from the input image signal by the subtracter 414 to generate a prediction residual signal 415. The prediction residual signal 415 is orthogonally transformed and quantized through the orthogonal transformation unit 405 and the quantization unit 406. The quantized data is input to the adder 416 via the inverse quantization unit 407 and the inverse orthogonal transform unit 408, and is added to the predicted pixel signal 421. As a result, a decoded pixel signal 418 is generated. The decoded pixel signal 418 is used as a reference pixel signal for the next macroblock.

As described above, the prediction mode determination units 304 and 404 in FIGS. 3 and 4 use the Lagrange multiplier method. Here, the Lagrangian cost is J, the generated code amount is R, and the coding distortion is D. λ is a Lagrange multiplier and depends on the quantization parameter.

  Lagrange's undetermined multiplier method is a technique that reduces a maximization problem with a certain constraint condition to a maximization problem without a constraint condition. Thomas Wiegand and Berand Girod, “Multi-frame motion-compensated prediction for video transmission”, Kluwer Academic Publishers 2001 proposes a mode selection of a video encoding method.

  The encoding distortion D is an error between the local decoded image obtained by orthogonally transforming the prediction residual signal (315, 415), quantizing, inversely quantizing, inversely orthogonally transforming and adding the prediction signal and the original image. Calculated. Further, since the generated code amount R is obtained after encoding using the orthogonal transform coefficient immediately after quantization, the calculation load increases when it has a plurality of modes or block shapes.

According to the moving picture coding method of the embodiment of the present invention, by calculating the estimation parameter composed of the estimated coding distortion D ^ and the estimated generated code quantity R ^ from the variance σ ² of the prediction residual signal and the quantization parameter QP In addition, processing such as orthogonal transform, quantization, inverse quantization, and inverse orthogonal transform in the prediction mode loop is omitted. Further, without adding components such as a variable-length encoding unit and an encoding distortion calculation unit, it is possible to reduce the calculation amount and hardware cost, and to suppress image quality deterioration.

  The processing of the intraframe prediction unit 108 will be described with reference to the flowchart of FIG. First, a variable min_cost for updating the minimum cost is initialized. In the prediction mode loop, intra-frame prediction (step 2), code amount estimation (step 4), coding distortion estimation (step 5), and cost J evaluation (step 6) are sequentially performed. Thereafter, min_cost> J is determined (step 8). If this determination is YES, the minimum cost is updated (step 9), and the prediction mode is updated (step 10).

  After the prediction mode is updated and when the determination of min_cost> J is NO, the end of prediction is determined (step 11). When the prediction mode loop of steps 2 to 10 is repeated for the number of prediction modes in which the prediction mode loop exists, the prediction loop is terminated.

  When the prediction loop is completed and the prediction mode giving the minimum cost is determined, orthogonal transformation (step 12), quantization (step 13), inverse quantization (step 14), and inverse orthogonal transformation (step 15) are sequentially performed. The reference pixel is updated (step 16). Thereafter, the process ends.

  In the present embodiment, the above conversion process may be performed only in the prediction mode with the lowest cost. Further, since the code amount is estimated, there is no need to perform variable length coding. Therefore, the processing can be greatly speeded up as compared with the conventional method.

Next, code amount estimation and coding distortion estimation methods will be described. The coding distortion is approximated by the quantization parameter QP and the prediction mode I as follows:

The generated code amount depends on the prediction mode, the quantization parameter, the input image signal, and the like, but can be approximated by the following equation for intra-frame prediction.

Here, I is a prediction mode, a _Ι , b _,, c _,, and d _Ι are estimation parameters that depend on the prediction mode I, and σ ² represents the variance of the prediction residual signal. The estimated coding distortion is represented by D ^, the estimated code amount is represented by R ^, and the quantization parameter is represented by QP. By using different estimation parameters for each prediction mode for estimating the coding distortion and the generated code amount, it is possible to further improve the estimation accuracy.

The estimation parameters a _Ι , b _Ι , c _Ι , and d _{上 記} can be determined by learning in advance using a plurality of sample images as described below. First, the prediction mode is fixed for each pixel block of the sample image, encoding is performed while sequentially changing the quantization parameter for various sample images, and the relationship between the generated code amount and the encoding distortion in the image is obtained. Measure the variance of the predicted residual signal.

FIG. 7 shows an example of encoding distortion estimation. The horizontal axis represents the value of the quantization parameter QP, and the vertical axis represents the coding distortion D obtained from the actual measurement value. A point is an actual measurement value obtained by encoding, and a solid line represents an approximate curve. For estimation of coding distortion, for each quantization parameter value, exponential approximation is performed on the relationship between the measured value D of the coding distortion obtained in the fixed prediction mode and the quantization parameter, and the estimation parameter of the prediction mode is determined. . In the estimation of the generated code amount for each block, the estimation parameter of the prediction mode is determined by linearly or exponentially approximating the relationship between the code amount obtained in the fixed prediction mode and log (σ ² / D ^). . Using the estimation parameters determined in advance as described above, the encoding distortion estimation value and the generated code amount estimation value are calculated from the variance of the prediction residual signal and the quantization parameter QP. As a result, it is possible to provide a moving picture encoding method that eliminates complicated processing in the prediction mode loop, reduces the amount of calculation and hardware cost, and suppresses image quality deterioration.

In another embodiment in which mode determination is performed using estimated parameters, only a few frames are actually encoded using a plurality of reference quantization parameters at the time of encoding. The generated code amount R and encoding distortion D of the encoding target image are measured. The obtained code amount-coding distortion curve is approximated, and estimation parameters a _Ι , b _Ι , c _Ι , and d _の for each macroblock are calculated. Encoding is performed using the estimation parameters obtained here. In this way, by encoding several frames before encoding and determining estimation parameters, a set of estimation parameters suitable for the encoding target can be used, and image quality with high encoding efficiency can be obtained.

  Next, a second embodiment will be described. In the second embodiment, the mode determination method using the estimation parameter introduced in the first embodiment is hierarchized for each pixel block. That is, the input image signal is divided into a plurality of large pixel blocks, and the large pixel block is further divided into a plurality of small pixel blocks. For example, H.C. In intra-frame prediction such as H.264, as shown in FIG. 5, a macroblock is divided into a large pixel block of 16 × 16 pixels and a small pixel block of 4 × 4 pixels.

  In the 16 × 16 pixel block, the number of prediction modes is as small as four, but in the 4 × 4 pixel block, there are nine types. In a 4 × 4 pixel block, in order to obtain a prediction of one macroblock, enormous combinations are conceivable and the amount of calculation increases. Therefore, in the small pixel block having a large number of calculations, mode determination using the generated code amount estimated value and the encoded distortion estimated value introduced with the estimation parameter described above is used in order to suppress an increase in the amount of calculation. Since the number of prediction modes is small in the large pixel block, the generated code amount and the encoding distortion are encoded to improve the image quality. At this time, mode determination is performed by actually measuring the amount of generated code and encoding distortion. By using a hierarchical structure such as a large pixel block and a small pixel block, image quality deterioration can be prevented and the amount of calculation can be greatly reduced.

  In the small pixel block, an example of performing the mode determination by calculating the generated code amount estimated value and the encoded distortion estimated value using the estimation parameter is the mode shown in FIG. 3, and the processing process at that time is shown in the flowchart of FIG. Indicated. This method has high accuracy because estimation is performed using different estimation parameters for each prediction mode. Also, since no complicated processing is performed in the prediction mode loop, the encoding process is very fast.

  On the other hand, in the large pixel block, the embodiment in which the generated code amount and the coding distortion are actually measured and the mode is determined is shown in FIG. 8, and the processing process at this time is shown in the flowchart of FIG.

  FIG. 8 shows the pixel block intra-frame prediction unit 203 of FIG. According to this, when the reference pixel signal 205 is input to the 16 × 16 prediction unit 501, the 16 × 16 prediction unit 501 outputs a prediction pixel signal 512. The predicted pixel signal 512 is subtracted from the input image signal 206 by a subtracter 509 to generate a predicted residual signal 511. The prediction residual signal 511 is subjected to orthogonal transform (for example, DCT) by the orthogonal transform unit 502. The orthogonal transform coefficient obtained by the orthogonal transform is quantized by the quantization unit 503. The quantized transform coefficient is sent to the variable length coding unit 506, where it is subjected to variable length coding to obtain a generated code amount actual measurement value R (513).

  Further, the quantized orthogonal transform coefficient is inversely quantized by the inverse quantization unit 504 and further decoded by the inverse orthogonal transform unit 505 to generate a local decoded signal. This local decoded signal is added to the prediction pixel signal 512 obtained from the prediction unit 501. The addition result is input to the coding distortion calculation unit 507 as a decoded pixel signal 516. The coding distortion calculation unit 507 calculates a coding distortion actual value D (514) and a generated code amount actual value R (513) based on the input image signal 206 and the decoded pixel signal 516. The actual coding distortion value D (514) and the generated code amount actual value R (513) are input to the 16 × 16 pixel block prediction mode determination unit 508.

  The 16 × 16 pixel block prediction mode determination unit 508 calculates a Lagrangian cost from the actual measured coding distortion value D (514) and the actual generated code amount R (513) using a Lagrange undetermined multiplier method, and sets the prediction mode with the lowest cost. As the final prediction mode, it is output together with the decoded pixel signal 516, the orthogonal transform coefficient 515, the quantization parameter 519, and the like.

  According to the flowchart shown in FIG. 12, first, a variable min_cost for updating the minimum cost is initialized (step 01). When intra-frame prediction (step 02) is performed, in the prediction mode loop, orthogonal transform (step 12), quantization (step 13), inverse quantization (step 14), inverse orthogonal transform (step 15), variable length Encoding (step 17) and encoding distortion calculation (step 18) are sequentially performed. In this case, if the prediction mode increases, the amount of calculation increases accordingly. However, an accurate generated code amount and encoding distortion can be calculated, and a high-quality encoded image with high encoding efficiency can be obtained.

  Thereafter, cost evaluation is performed based on the coding distortion D (step 06), and min_cost> J is determined (step 8). If this determination is YES, the minimum cost is updated (step 9), and the prediction mode is updated (step 10).

  After the prediction mode is updated and when the determination of min_cost> J is NO, the end of prediction is determined (step 11). When the prediction mode loop of steps 2 to 10 is repeated by the number of prediction modes, the prediction loop is terminated. Thereafter, the reference pixel is updated (step 16), and the process ends.

  The mode decision method that actually performs coding and calculates the generated code amount R and coding distortion D causes an increase in the amount of computation for small pixel blocks with a large number of prediction modes, but for large pixel blocks. Since the number of modes is small, the amount of computation does not increase greatly. As described above, in the small pixel block in which the calculation amount may increase by utilizing the features of the large pixel block and the small pixel block, the generated code amount estimated value and the code described in the first embodiment in which the estimation parameter is introduced. Mode determination using the estimated distortion estimation value is performed. In the large pixel block, in order to improve the image quality, mode determination for obtaining an accurate Lagrangian cost is performed by calculating the generated code amount and the encoding distortion from the actually measured values. By performing mode determination using the hierarchical structure in this way, it is possible to significantly reduce the amount of calculation while suppressing a decrease in encoding efficiency, and it is possible to reduce hardware costs.

  In another embodiment for improving image quality, the combination of optimal prediction modes for small pixel blocks is not limited to only one, but mode determination using the above-described estimation parameters is performed, and a plurality of combination candidates in a small pixel block A plurality of prediction mode information selected from the above are sent to the large pixel block. In the large pixel block, from among the combination candidates of the prediction candidate predicted in the prediction mode of the large pixel block using the mode determination based on the actual measurement value described above and the prediction candidate of the small pixel block sent from the small pixel block, The optimum prediction mode or combination of prediction modes is determined. Thereby, it is possible to increase the encoding efficiency when the generated code amount calculated using the estimation parameter and the estimated value of the encoding distortion deviate.

  FIG. 2 illustrates an example of the intra-block prediction mode determination unit 204 in FIG. In the pixel block intra-frame prediction unit 211 of FIG. 2, a 4 × 4 pixel block combination candidate (a combination of 4 × 4 prediction modes) predicted by the 4 × 4 block intra-frame prediction unit 202 is an intra-block prediction mode determination unit. 204. That is, 4 × 4 pixel combination candidate 1 (601), 4 × 4 pixel combination candidate 2 (602), and 4 × 4 pixel combination candidate 3 (603) are input to the candidate data control unit 608.

  In the 16 × 16 block intra-frame prediction unit 203 in FIG. 4, the code amount estimation unit 402, the coding distortion estimation unit 408, the orthogonal transform unit 405, the quantization unit 406, the inverse quantization unit 407, and the inverse orthogonal transform unit 408 pass. First, the 16 × 16 prediction mode 401 (16 × 16 prediction mode 0 (604), 16 × 16 prediction mode 1 (605), 16 × 16 prediction mode 2 (606), 16 × 16 prediction mode 3 (607)) is transmitted. Prediction information is sent to the variable length coding unit 609 and the coding distortion calculation unit 610 via the candidate data control unit 608. The variable length encoding unit 609 and the encoding distortion calculation unit 610 measure the generated code amount R (614) and the encoding distortion D (615), calculate the Lagrangian cost using these values, and perform mode determination. The optimal prediction mode or combination of prediction modes is determined, and the decoded pixel signal 613, prediction mode information 616, orthogonal transform coefficient 617, and quantization parameter 618 are output. Here, the 16 × 16 block prediction mode determination unit 203 may select only one of the prediction modes or may output all prediction modes. In this way, by using a plurality of small pixel block combination candidates and a large pixel block prediction candidate to actually measure and calculate the generated code amount and coding distortion, and perform mode determination, the mode determination of the small pixel block is performed. Accuracy is improved and an image quality improvement effect is obtained.

  In the third embodiment, a new cost function is introduced to improve subjective image quality. The encoder may use the first embodiment or the second embodiment, and introduces an error of encoding distortion with an adjacent block in consideration of subjective image quality into the cost function in mode determination.

The probability of obtaining an optimal decoded pixel signal LD under the constraint that the code amount R is smaller than Rc is maximized.

Here, the probability P (LD | I, QP) assumes a Gibbs distribution and is expanded to the following equation using Bayes' theorem.

Here, C = P (I, QP), and the joint probability that both the prediction mode I and the quantization parameter QP are given is a constant. The likelihood function of the first term and the prior function of the second term each have a Gibbs distribution, and the energy function (cost) is defined by the following equation under the constraint condition of R <Rc.

The likelihood function represents the conventional Lagrangian cost, and conforms to the Lagrange undetermined multiplier method framework that minimizes the coding distortion under the constraint condition R <Rc. The prior probability function indicates the strength of the correlation between the current block and the adjacent block. By maintaining the coding distortion of the adjacent block and the current block within a certain range, the subjective image quality improvement effect of the entire image can be obtained. It is done. Here, D (I, QP) is encoding distortion, S is all pixels in the pixel block, and N is all pixels in the adjacent pixel block. FIG. 10 shows the relationship between adjacent pixel blocks. Coding distortion is calculated using the left block, the upper block, the upper left block, the upper right block of the coding target block, and the reference block at the same position as the coding target block of the coded reference frame as an adjacent block. The likelihood function η is a multiplier for changing the strength of the correlation, and is represented by the following equation, for example.

When the error between the adjacent blocks is smaller than the threshold value theta _th is lambda _alpha is selected, when large lambda _beta is chosen. η represents a parameter for introducing the strength of correlation with adjacent blocks into the Lagrangian cost encoding distortion. As a result, λ _α is set so that prediction modes with similar coding distortion can be easily obtained for blocks with similar image quality features, and small λ _{β for} blocks with significantly different image quality features so as not to have a significant impact on conventional Lagrangian costs. Set. By effectively changing these parameters, the coding distortion of the current pixel block and the adjacent pixel block is kept constant, and at the same time, the correlation between blocks having different characteristics is weakened. This is equivalent to extending the Lagrangian cost as a constraint for coding distortion to a maximization problem with two constraints. By substituting [Equation 6] and [Equation 7] into [Equation 5] and performing mean field approximation, the following equation is obtained.

Here, when partial differentiation is performed on both sides with R _s , the second term does not affect this equation due to mean field approximation,

In other words, it is consistent with the Lagrange multiplier method framework. As a result, without changing the framework of the Lagrange undetermined multiplier method described above, coding distortion is kept constant in blocks with similar image quality characteristics, and cost is not greatly affected in blocks with greatly different image quality characteristics. Therefore, it is possible to visually reduce block noise and improve the subjective image quality of the entire image frame.

  As described above, according to the present invention, there are a plurality of encoding modes, and different parameters are set for each block and each mode. The optimum encoding mode is estimated using this parameter according to the input image signal.

  In the present invention, the frame is divided into a hierarchical structure from a large block to a small block, and a different encoding mode is set for each layer. In this case, since the number of modes increases as going to the lower layer, the encoding mode is determined using the estimation parameter in the lower layer. That is, mode determination is performed by a high-speed estimation algorithm. Since the number of candidates is small in the upper hierarchy, mode determination is performed based on the generated code amount and encoding distortion generated by actual encoding. That is, mode determination is performed at high speed using the estimation parameter in a hierarchy with a large number of mode combinations, and highly accurate mode determination is performed through actual measurement in a hierarchy with a small number of modes.

  Furthermore, the present invention uses a method called Lagrange undetermined multiplier method for mode determination, and introduces a new term in Lagrange cost calculation without changing the framework of Lagrange undetermined multiplier method, so that distortion between blocks does not occur. Mode determination is performed.

  The moving image encoding apparatus according to the present invention as described above is suitable for image compression processing in a moving image transmission system and a moving image recording apparatus.

1 is a block circuit diagram of a video encoding apparatus according to an embodiment of the present invention. FIG. 2 is a block circuit diagram of an intra-frame prediction unit in FIG. 1. FIG. 3 is a block circuit diagram of a 4 × 4 pixel block intra-frame prediction unit using estimation parameters shown in FIG. 2. FIG. 3 is a block circuit diagram of a 16 × 16 pixel block intra-frame prediction unit using estimation parameters shown in FIG. 2. The figure which shows the example of a division | segmentation of pixel block and prediction mode concerning embodiment of this invention. H. according to an embodiment of the present invention. The figure which shows the example of a prediction mode of the 4x4 pixel block in H.264. The figure which shows the example which selects the estimation parameter concerning embodiment of this invention. The figure which shows the example which performs mode determination by actually measuring the code amount and encoding distortion concerning embodiment of this invention. The figure which shows the mode determination which performs mode determination from the several candidate concerning embodiment of this invention. The figure which shows the example of the adjacent pixel block of the cost function concerning embodiment of this invention. The flowchart figure which shows the prediction in a flame | frame using the estimation parameter concerning embodiment of this invention. FIG. 5 is a flowchart of intra-frame prediction in which mode determination is performed by actually measuring a code amount and coding distortion according to an embodiment of the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 ... Subtractor 102 ... Orthogonal transformation part 103 ... Quantization part 104 ... Inverse quantization part 105 ... Inverse orthogonal transformation part 106 ... Adder 107 ... Frame memory 108 ... Inter-frame prediction part 109 ... Intraframe prediction unit 110 ... Coding control unit 111 ... Variable length coding unit 112 112 Multiplexing unit 113 ... Output buffer 114 ... Coding unit 115 ... MB prediction mode selection unit 201 ... Block shape control unit 202 ... 4 × 4 block intra-frame prediction unit, 203 ... 16 × 16 block intra-frame prediction unit, 204 ... intra-block prediction mode determination unit, 205 ... reference image signal, 206 ... input image signal, 207 ... prediction pixel signal, 208: Decoded pixel signal, 209: Orthogonal transform coefficient, 210 ... Quantization parameter, 211 ... Intra-pixel block intra-frame prediction unit

Claims (6)

A moving image in which an input image signal is divided into a plurality of pixel blocks, one mode is selected from a plurality of encoding modes for each pixel block, and encoding is performed for each pixel block in the selected encoding mode. In the encoding method,
A different estimation parameter is generated for each pixel block and each encoding mode, and the generated code amount and coding distortion of the pixel block of the input image signal are estimated using the estimated parameter, and the estimated generated code amount and code A moving picture coding method, wherein an optimal coding mode is determined based on a coding distortion. A moving image signal is divided into a plurality of first pixel blocks, the first pixel block is divided into a plurality of second pixel blocks, and one code is generated from a plurality of encoding modes for each size of the pixel block. In the moving picture coding method for selecting a coding mode and performing coding for each pixel block in the selected coding mode,
For the second pixel block, using the estimation parameters generated for each pixel block and each encoding mode, the generated code amount and encoding distortion of the pixel block are estimated, and the estimated generated code Based on the amount and coding distortion, determine one candidate from the plurality of coding modes, or a combination candidate of the plurality of coding modes,
With respect to the first pixel block, the code using one candidate or a plurality of combination candidates from the plurality of coding modes based on the generated code amount and the actual value of the coding distortion in each coding mode. A moving picture coding method characterized by determining a coding mode. Each image frame of the input moving image signal is divided into a plurality of pixel blocks, one mode is selected from a plurality of encoding modes for each pixel block, and each pixel block is encoded in the selected encoding mode. In a moving image encoding method for performing
A first encoding distortion of an encoded pixel block adjacent to an encoding target pixel block in the image frame is detected, and an already encoded image frame at the same position as the encoding target pixel block has been encoded. Detecting a second coding distortion of the pixel block, and determining a coding mode in which an error between at least one of the first and second coding distortions and the coding distortion of the coding target pixel block is reduced; A video encoding method characterized by the above. Means for generating different estimation parameters for each pixel block and for each encoding mode;
Means for estimating a generated code amount and encoding distortion of a pixel block of an input image signal using the estimation parameter;
Means for determining an optimum coding mode from a plurality of coding modes based on the estimated generated code amount and coding distortion;
Means for encoding each pixel block in the optimal encoding mode;
A moving picture encoding apparatus comprising: Means for dividing a moving image signal into a plurality of first pixel blocks, and dividing the first pixel block into a plurality of second pixel blocks;
Means for generating different estimation parameters for each pixel block and for each encoding mode;
With respect to the second pixel block, using the estimation parameters generated for each pixel block and each coding mode, means for estimating the generated code amount and coding distortion of the pixel block;
Means for determining one candidate from the plurality of encoding modes or a combination candidate of the plurality of encoding modes based on the estimated generated code amount and encoding distortion;
With respect to the first pixel block, the code using one candidate or a plurality of combination candidates from the plurality of coding modes based on the generated code amount and the actual value of the coding distortion in each coding mode. Means for determining the activation mode;
Means for encoding for each pixel block in the determined encoding mode;
A moving picture encoding apparatus comprising: Each image frame of the input moving image signal is divided into a plurality of pixel blocks, one mode is selected from a plurality of encoding modes for each pixel block, and each pixel block is encoded in the selected encoding mode. In a video encoding device that performs
Means for detecting a first encoding distortion of an encoded pixel block adjacent to an encoding target pixel block in an image frame;
Means for detecting a second encoding distortion of an encoded pixel block of an already encoded image frame at the same position as the encoding target pixel block;
Means for determining an encoding mode in which an error between at least one of the first and second encoding distortions and the encoding distortion of the encoding target pixel block is reduced;
Means for encoding each pixel block in the determined encoding mode;
A moving picture encoding apparatus comprising:

Images