JP5943757B2 - Image encoding apparatus and method - Google Patents

Description

  The present invention relates to an image encoding apparatus and method thereof, and more specifically to an image encoding apparatus and method for compressing and encoding a moving image.

  2. Description of the Related Art Conventionally, a digital video camera is well known as a camera-integrated moving image recording apparatus that captures a subject and compresses and records moving image data obtained by the shooting. In recent years, recording media for recording moving image data have changed from conventional magnetic tapes to highly convenient disk media such as random accessibility and semiconductor memories. In addition, as a compression method, the MPEG2 method capable of compressing at a high compression rate by using motion prediction between frames is generally used, and in recent years, H.264 that can be compressed to a higher compression is used. The H.264 system is also used. H. In the H.264 system, it is defined in the standard that motion compensation is performed by subdividing a macroblock into smaller submacroblocks.

  An image feature or the like is used to determine a block size as a unit of encoding. A method for adaptively determining the block size according to the feature of the image is described in Patent Document 1. Patent Document 2 describes a method for adaptively determining a block size according to camera information.

Japanese Patent Laid-Open No. 09-130801 JP 2006-254370 A

  If the size of the coding block can be adaptively changed, the coding efficiency is improved and the image quality is improved. However, the quantization coefficient is determined for each size of the encoded block, and the code amount error (difference between the target code amount and the generated code amount) for each block increases, which causes image quality degradation.

  It is an object of the present invention to provide an image encoding apparatus and method for performing quantization control that eliminates such inconvenience and suppresses a code amount error.

An image encoding device according to the present invention is an image encoding device that encodes image data, the quantization means for quantizing the image data in units of encoding blocks whose size can be changed, and the quantization An encoding unit that encodes the image data of the encoded block, a code amount error calculating unit that calculates a code amount error based on a difference between a target code amount and a generated code amount by the encoding unit, and the code amount Quantization coefficient calculation means for calculating a quantization coefficient of the quantization means of the subsequent coding block according to an error and quantization control sensitivity, the quantization coefficient calculation means depending on the size of the coding block And changing the quantization control sensitivity.
An image encoding method according to the present invention is an image encoding method for encoding image data, the quantization step for quantizing the image data in units of encoding blocks whose size can be changed, and An encoding step for encoding image data of a quantized encoding block; a code amount error calculating step for calculating a code amount error based on a difference between a target code amount and a generated code amount in the encoding step; A quantization coefficient calculation step of calculating a quantization coefficient of a subsequent coding block according to a code amount error and a quantization control sensitivity, and in the quantization coefficient calculation step, the quantization coefficient is calculated according to a size of the coding block. The control sensitivity is changed.

  According to the present invention, by suppressing the code amount error by changing the sensitivity of the quantization control according to the size of the coding block, the quantization control becomes appropriate, and the compression rate or the image quality can be improved.

It is a schematic block diagram of one Example of this invention. It is an example of the division | segmentation of an encoding block. It is a schematic diagram which shows an example of target code amount, a quantization coefficient, and generated code amount. It is a schematic diagram of the example of a change following FIG. It is a schematic diagram which shows the example of a change of the target code amount for every encoding block. It is a schematic diagram of the example of a change following FIG. It is an operation | movement flowchart of a quantization control apparatus. It is another operation | movement flowchart of a quantization control apparatus.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a schematic block diagram of an embodiment of an image encoding apparatus according to the present invention. The moving image coding apparatus 100 includes intra coding (intra-screen coding) for coding only with image data in a frame, and inter coding (inter-screen coding) for coding including prediction between frames. Are selectively available. Inter coding includes P (unidirectional predictive coding) pictures and B (bidirectional predictive coding) pictures, and intra coding includes I pictures. The P picture is encoded by unidirectional prediction in which a macro block serving as a motion compensation unit is predicted with one reference frame. The B picture is encoded by bidirectional prediction in which prediction with up to two reference frames is performed. An I picture is encoded using image data in the same frame (or screen).

  The order of frames to be encoded is different from the order of input frames. This is to enable prediction (backward prediction) with not only past frames but also temporally future frames. In an inter coding frame, inter prediction or intra prediction can be selected for each block to be coded. On the other hand, only inter prediction is employed in intra-coded frames.

  The imaging unit 101 is a camera unit including a lens, an imaging sensor, and the like, and outputs a subject image signal to the moving image encoding apparatus 100 at a moving image rate. In the moving image encoding device 100, the image signal from the imaging unit 101 is temporarily stored in the frame memory 102. The frame memory 102 includes a reference image memory 114 that stores images (reference images) of temporally different frames used for inter prediction. Image data from the imaging unit 101 is sequentially stored in the frame memory 102 in the order of the first frame, the second frame, the third frame,. From the frame memory 102, for example, the third frame, the first frame, the second frame,... Are read in the order of encoding.

  For an intra-encoded frame, image data of an encoding target block that is a maximum encoding unit is read from the frame memory 102 to the intra-prediction device 105. In addition, data of pixels adjacent to the read target block to be encoded is also read from the frame memory 102 and input to the intra prediction device 105. The intra prediction device 105 generates a plurality of intra prediction image data in a plurality of prediction directions from the data of pixels adjacent to the encoding target block, and performs block matching with the image data of the encoding target block. Then, the intra prediction device 105 outputs the prediction direction (or prediction mode) corresponding to the intra prediction image data having the highest correlation in this block matching to the prediction method selection device 104.

  In an intra-encoded frame, the prediction method selection device 104 always selects the intra prediction device 105 and notifies the prediction image generation device 106 of data indicating the prediction direction from the intra prediction device 105. The predicted image generation device 106 performs intra predicted image data from reconstructed image data (output image data of the adder 112) previously encoded and decoded in the same frame in accordance with the prediction direction notified from the prediction method selection device 104. Is generated. The generated intra prediction image data is supplied to the subtracter 107 and the adder 112.

  The subtracter 107 subtracts the intra prediction image data from the prediction image generation device 106 from the image data of the encoding target block read from the frame memory 102, and outputs the obtained difference image data to the integer conversion device 108. The integer conversion device 108 performs integer conversion on the difference image data from the subtractor 107 and supplies the conversion coefficient data to the quantization device 109. The quantization device 109 quantizes the transform coefficient data from the integer transform device 108 in a quantization step set by the quantization control device 116.

  On the other hand, in the frame of inter coding, the image data of the block to be coded which is the maximum coding unit is read from the frame memory 102 to the inter prediction device 103. The inter prediction device 103 reads the reference image data from the reference image memory 114, detects a motion vector from the image data to be encoded and the reference image data, and notifies the prediction method selection device 104 of the motion vector.

  In the inter coding frame, either inter prediction or intra prediction can be selected for each block to be coded, so that the intra prediction apparatus 105 also operates as described in the intra coding frame. The intra prediction apparatus 105 notifies the prediction method selection apparatus 104 of the result of intra prediction (here, the prediction direction having the strongest correlation and its prediction error value).

  The prediction method selection device 104 compares the prediction result of the inter prediction device 103 and the prediction result of the intra prediction device 105, selects, for example, a prediction with a smaller prediction error, and notifies the prediction image generation device 106 of the prediction. When the intra prediction is notified from the prediction method selection apparatus 104, the predicted image generation apparatus 106 generates intra predicted image data from the reconstructed image data of the same frame as described in the frame of intra coding. When inter prediction is notified from the prediction method selection device 104, the predicted image generation device 106 uses the reference image data referred to by the inter prediction device 103 as predicted image data.

  The subtracter 107 subtracts the predicted image data from the predicted image generation device 106 from the image data of the encoding target block read from the frame memory 102, and outputs the obtained difference image data to the integer conversion device 108. The integer conversion device 108 performs integer conversion on the difference image data from the subtractor 107 and supplies the conversion coefficient data to the quantization device 109. The quantization device 109 quantizes the transform coefficient data from the integer transform device 108 in a quantization step set by the quantization control device 116.

  The entropy encoder 115 entropy-encodes the transform coefficient quantized by the quantizer 109, and generates and outputs stream data in a predetermined format to which motion compensation motion vector information and the like are added. The entropy encoding device 115 also supplies data indicating the generated code amount to the code amount control device 117 and the quantization control device 116.

  The transform coefficient quantized by the quantization device 109 is also input to the inverse quantization device 110 for local decoding. The inverse quantization device 110 inversely quantizes the quantized transform coefficient, and the inverse integer transform device 111 performs inverse integer transform on the output of the inverse quantization device 110. The output of the inverse integer transform device 111 corresponds to the local decoded value of the output of the subtractor 107.

  The adder 112 adds the intra prediction image data generated by the prediction image generation device 106 to the output data of the inverse integer conversion device 111. The output of the adder 112 is reconstructed image data reconstructed by local decoding, and is input to the predicted image generation device 106 and used to generate intra predicted image data. Further, the reconstructed image data is subjected to encoding distortion reduction processing by the in-loop filter 113 and is stored in the reference image memory 114 as reference image data used for inter coding.

  The code amount control device 117 determines a target code amount that will be a predetermined code amount in the long term in accordance with the generated code amount from the entropy encoding device 115 and supplies it to the quantization control device 116. The quantization control device 116 holds the initial quantization coefficient or the quantization coefficient used in the previous block, and inputs information on the coding block size determined by the prediction method selection device 104 to the quantization control device 116. The quantization control device 116 performs quantization in the quantization of the subsequent block according to the initial quantization coefficient, the quantization coefficient used in the previous block, the coding block size, the generated code amount, and the target code amount. The quantization coefficient to be applied is determined and set in the quantization device 109.

  The output data of the entropy encoding device 115 becomes the output of the moving image encoding device 100. The recording device 118 records the stream data output from the entropy encoding device 115 on the recording medium 119.

  FIG. 2 is a schematic diagram illustrating an example of coding block division. FIG. 2 shows the maximum size of the encoded block (maximum encoded block). Blocks obtained by subdividing the maximum coding block as shown in FIG. 2 are adaptively determined, and blocks numbered 1 to 22 are to be quantized. Of course, a coding block having the maximum size may be a block to be quantized. A plurality of such encoding blocks or maximum encoding blocks are arranged in a frame. In the example shown in FIG. 2, when the size of the maximum coding block is 32 pixels × 32 pixels, the size of the minimum quantization target block is 4 pixels × 4 pixels.

FIG. 2 shows an example in which attention is paid to one certain maximum coding block, and each number is called a coding block number. In intra prediction and inter prediction, the block size to be quantized is determined so that the difference value is minimized in the maximum coding block. However, since block discrimination information is added when the number of blocks increases, the block size is determined in consideration of the amount of header information. For example, if the difference value is D in an arbitrary block A divided into blocks, when the quantization coefficient Q and the header information amount I are, the evaluation value H of the block A is
H = D / Q + I
It is expressed. However, Q shows a quantization coefficient and I shows header information amount. The block to be quantized and its size are determined so that the evaluation value H of each block is minimized. Of course, the quantization target block and its size may be determined by another evaluation method or a division method different from FIG.

FIG. 3 is a schematic diagram showing the relationship between the target code amount, the quantization coefficient, and the generated code amount of each block. In FIG. 3, ● indicates the accumulated code amount at the previous block address, and the generated code amount at the position of ● is indicated by □. An oblique broken line is drawn between the accumulated code amount and the target picture code amount. The block target code amount is
{(Target picture code amount)-(Integrated code amount)} / (Remaining block equivalent size)
It can be obtained more. However, the remaining block equivalent size is a value obtained by converting an uncoded block by the block size of the current block.

Encoding is performed in units of blocks, and the difference between the block target code amount and the block generated code amount becomes a block code amount error err. The quantization control device 116 functions as a code amount error calculating means for calculating the block code amount error err. The quantization control device 116 sets the quantization coefficient Qp to
Qp = Q + Σerr × α (1)
And functions as a quantization coefficient calculation means for this. α is an arbitrary value. The quantization coefficient Qp can be finely changed by increasing the control sensitivity. However, if the intensity is increased too much, quantization distortion will occur, and it will be necessary to reduce the sensitivity. This is adjusted by a coefficient α. As the coefficient α increases, the control sensitivity increases.

  FIG. 4 is an example of a target code amount, a quantization coefficient, and a generated code amount of a coding block following the CU 10 in the example shown in FIG. 3, and is a schematic diagram when the CU 11 is encoded by taking over the quantization coefficient of the CU 10 The figure is shown. Since the block size is different, the code amount error increases.

  FIG. 5 shows an example of a change in the target code amount for each coding block. Assuming that the difference between the broken line and the accumulated code amount is the control code amount error, in the example shown in FIG. 5, the control sensitivity of quantization is enhanced by updating the broken line for each block. This control increases the control code amount error in the CU10.

  FIG. 6 shows an example of the relationship between the target code amount, quantization coefficient, and generated code amount of the block following the example shown in FIG. Since the quantization coefficient is different, the generated code amount is different from the example shown in FIG. When the block size is large, the amount of generated code varies greatly depending on the quantization coefficient, so it is necessary to increase the quantization control sensitivity and suppress the code amount error.

  FIG. 7 is an operation flowchart of the quantization controller 116. The operation of the quantization control device 116 will be described with reference to FIG.

  The quantization control device 116 sets the initial quantization coefficient or the last quantization coefficient of the previous picture of the same type, the current block size, and the code amount error as initial values (S701). At the beginning of quantization, since there is no last quantization coefficient of the immediately preceding picture of the same type, the initial quantization coefficient is adopted. When encoding progresses and the last quantized coefficient of the previous picture of the same type is present, the last quantized coefficient of the previous picture of the same type is adopted.

  The quantization control device 116 subtracts the generated code amount from the target picture code amount to the previous block, and calculates the remaining picture code amount (S702). The quantization controller 116 also calculates the minimum number of remaining blocks when the entire picture is the minimum block.

  The quantization control device 116 calculates a target code amount from the picture remaining code amount, the minimum number of remaining blocks, and the block size (S703).

  Next, the quantization control device 116 determines whether or not the block size is larger than 16 × 16 pixels (S704). If the block size is larger than 16 × 16 pixels (S704), the quantization control device 116 performs quantization. The control sensitivity is increased (S705). It is possible to increase the quantization control sensitivity by increasing the value of the coefficient α in the equation (1).

  The quantization controller 116 calculates a Q value that is a quantization coefficient (S706).

  As described above, in this embodiment, when the block size is large, control for suppressing the code amount error is performed.

  In the above description, the quantization control sensitivity is increased when the block is larger than 16 × 16 pixels, which is the first threshold value related to the block size. However, the quantization control sensitivity is changed or controlled based on another block size. Also good.

  In the present embodiment, as described above, since the code amount error is suppressed by changing the quantization control sensitivity according to the size of the coding block, it is possible to perform more appropriate quantization control than before, and Or compression efficiency can be improved.

  FIG. 8 is another operation flowchart of the quantization control device 116. In FIG. 8, steps having the same processing contents as those in FIG. 7 are denoted by the same reference numerals, and different portions will be described in detail. Specifically, a conditional branch step S807 is inserted between steps S703 and S704.

  In step S807, the quantization control apparatus 116 determines whether or not the number of blocks when the unencoded portion of the encoding target frame is the minimum divided block is greater than the threshold value TB (second threshold value). When the number of blocks is larger than the threshold value TB (S807), the quantization control apparatus 116 proceeds to step S704, and executes step S704 and subsequent steps as in FIG.

  When the number of blocks is equal to or less than the threshold value TB (S807), the quantization control apparatus 116 increases the control sensitivity in accordance with the block size so that there is no large difference between the target code amount of the picture and the generated code amount (S807). S808). This is to suppress the code amount error by increasing the control sensitivity as the block is larger.

  As described above, since the code amount error is suppressed by changing the quantization control sensitivity according to the size of the coding block, it is possible to perform more appropriate quantization control than before, improving image quality or compression efficiency it can.

Claims (6)

An image encoding device for encoding image data,
Quantization means for quantizing the image data in units of a coding block whose size can be changed;
Encoding means for encoding image data of the quantized encoding block;
A code amount error calculating unit that calculates a code amount error based on a difference between a target code amount and a generated code amount by the encoding unit;
A quantization coefficient calculation unit that calculates a quantization coefficient of the quantization unit of a subsequent coding block according to the code amount error and the quantization control sensitivity;
The image coding apparatus, wherein the quantization coefficient calculation unit changes the quantization control sensitivity according to a size of the coding block.   The image coding apparatus according to claim 1, wherein the quantization coefficient calculation unit increases the quantization control sensitivity when the size of the coding block is larger than a first threshold.   The said quantization coefficient calculation means strengthens the said quantization control sensitivity, when the number of blocks of the non-encoded part of the encoding object frame is larger than a second threshold value. Image coding apparatus. An image encoding method for encoding image data, comprising:
A quantization step of quantizing the image data in units of coding blocks whose size can be changed;
An encoding step of encoding image data of the quantized encoding block;
A code amount error calculating step of calculating a code amount error based on a difference between the target code amount and the generated code amount in the encoding step;
A quantization coefficient calculating step of calculating a quantization coefficient of a subsequent coding block according to the code amount error and the quantization control sensitivity,
In the quantization coefficient calculating step, the quantization control sensitivity is changed according to the size of the encoding block.   5. The image encoding method according to claim 4, wherein, in the quantization coefficient calculation step, the quantization control sensitivity is increased when the size of the encoding block is larger than a first threshold. 6. The quantization control sensitivity is increased in the quantization coefficient calculation step when the number of blocks of an unencoded portion of a frame to be encoded is larger than a second threshold value. Image coding method.

Images