JP4942208B2 - Encoder - Google Patents

Description

  The present invention relates to an encoding apparatus that encodes a moving image, and encodes an image by dividing the image into a plurality of blocks, such as MPEG (Moving Pictures of Experts Group) and H.264 / AVC (Advanced Video Coding). The present invention relates to an encoding device.

  Along with the development of multimedia in recent years, various video compression encoding methods have been proposed. Typical examples are MPEG-1, 2, 4 and H.264. There is something like H.264. In these compression encoding processes, an original image (image) included in a moving image is divided into predetermined regions called blocks, and encoding processing such as motion compensation prediction and DCT conversion processing is performed in units of blocks. It is something to apply. When performing motion compensation prediction, an image obtained by local decoding of already encoded image data is used as a reference image, and thus decoding processing is necessary.

  In addition, when image compression encoding is performed in accordance with the MPEG system, the amount of codes often varies greatly depending on the spatial frequency characteristics, scenes, and quantization scale values that are characteristics of the images themselves. An important technique for making it possible to obtain a decoded image with good image quality in realizing an encoding apparatus having such encoding characteristics is code amount control.

  TM5 (Test Model 5) is generally used as one of the code amount control algorithms. The code amount control algorithm based on TM5 is composed of the following three steps, and the code amount is controlled so that the bit rate is constant for each GOP (Group Of Picture).

(Step 1)
The target code amount of the picture to be encoded from now is determined. The code amount Rgop that can be used in the current GOP is calculated by the following equation (1). That is,
Rgop = (ni + np + nb) x (bits_rate / picture_rate) (1)
Here, ni, np, and nb indicate the number of remaining pictures in the current GOP of I, P, and B pictures, respectively. bits_rate indicates the target bit rate. picture_rate indicates the picture rate.

Further, for each of the I, P and B pictures, the picture complexity complexity Xi,
Xp and Xb are obtained by the following equation (2). That is,
Xi = Ri × Qi
Xp = Rp × Qp (2)
Xb = Rb × Qb
Here, Ri, Rp, and Rb indicate code amounts obtained as a result of encoding I, P, and B pictures, respectively. Qi, Qp, and Qb are average values of the Q scale in all macroblocks in the I, P, and B pictures, respectively. From the equations (1) and (2), the target code amounts Ti, Tp, and Tb for each of the I, P, and B pictures can be obtained by the following equation (3). That is,
Ti = max {(Rgop / (1 + ((Np × Xp) / (Xi × Kp)) + ((Nb × Xb) / (Xi × Kb)))), (bit_rate / (8 × picture_rate))}
Tp = max {(Rgop / (Np + (Nb × Kp × Xb) / (Kb × Xp))), (bit_rate / (8 × picture_rate))} (3)
Tb = max {(Rgop / (Nb + (Np × Kb × Xp) / (Kp × Xb))), (bit_rate / (8 × picture_rate))}
Np and Nb indicate the remaining number of P and B pictures in the current GOP, respectively. The constants Kp = 1.0 and Kb = 1.4.

(Step 2)
Three virtual buffers are used for each of the I, P, and B pictures, and the difference between the target code amount obtained by Expression (3) and the generated code amount is managed. The data accumulation amount of the virtual buffer is fed back, and the reference value of the Q scale is set for the macroblock to be encoded next so that the actual generated code amount approaches the target code amount based on the data accumulation amount. For example, when the current picture type is a P picture, the difference between the target code amount and the generated code amount can be obtained by arithmetic processing according to the following equation (4). That is,
d _{p, j} = d _{p, 0} + B _{p, j-1} -((Tp × (j-1)) / MB_cnt) (4)
Here, the subscript j indicates the number of the macroblock in the picture. d _{p, 0} indicates the initial fullness of the virtual buffer. B _{p, j} represents the total code amount up to the j-th macroblock. MB_cnt indicates the number of macroblocks in the picture.

Next, using d _{p, j} (hereinafter referred to as d _j ), the reference value of the Q scale in the j-th macroblock is obtained.
Q _j = (d _j × 31) / r (5)
here,
r = 2 × bits_rate / picture_rate (6)
It is.

(Step 3)
The quantization scale is finally determined based on the spatial activity of the macroblock to be encoded so that the visual characteristics, that is, the image quality of the decoded image is improved. That is,
ACT _j = 1 + min (vblk ₁ , vblk ₂ , ..., vblk ₈ ) ... (7)
In Expression (7), vblk _{1 to} vblk ₄ indicate spatial activities in 8 × 8 sub-blocks in a macroblock having a frame structure. vblk _{5 to} vblk ₈ indicate spatial activities of 8 × 8 sub-blocks in the field-structured macroblock. Here, the space activity vblk is in accordance with the following equations (8) and (9):
vblk = Σ (P _i -Pbar) 2 (8)
Pbar = (1/64) × ΣP _i (9)
It can ask for. Here, P _i is a pixel value in the i-th macroblock, and Σ in equations (8) and (9) is an operation of i = 1 to 64.

Next, the activity ACT _j obtained by the equation (7) is normalized according to the following equation (10). That is,
N_ACT _j = (2 * ACT _j + AVG_ACT) / (ACT _j + 2 * AVG_ACT) (10)
Here, AVG_ACT is a reference value of ACT _j in a previously encoded picture. The final quantization scale (Q scale value) MQUANT _j is
MQUANT _j = Q _j × N_ACT _j (11)
Is required.

  According to the above TM5 algorithm, a large amount of code is allocated to the I picture by the processing in step 1, and the amount of code is large in a flat portion (a portion where spatial activity is low) that tends to visually deteriorate in the picture. Distributed. That is, it is possible to perform code amount control and quantization control while suppressing deterioration in image quality within a predetermined bit rate.

In addition, another technique for performing quantization control according to the image characteristics as in TM5 has been proposed (see, for example, Patent Document 1).
Japanese Patent Laid-Open No. 11-196417

  The TM5 system performs quantization control so as to obtain a predetermined target code amount by extracting features in units of macroblocks and performing adaptive quantization that changes the quantization parameter based on the features. .

  Further, in Patent Document 1, when the number of blocks with high complexity and a high quantization parameter is small, deterioration of blocks with high complexity is conspicuous despite the increase in the amount of generated codes, so adaptive quantization is not performed. Control is performed. The same applies when there are many blocks with low complexity and low quantization parameters. However, this algorithm has a problem in that the image quality fluctuates because of the presence or absence of adaptive quantization between frames.

  Further, even in an image having many blocks for which the quantization parameter is lowered, there is a block in which the code amount does not increase even if the quantization parameter is lowered, such as a flat portion, depending on the characteristics of the block. Originally, adaptive quantization is not applied to an image to be subjected to adaptive quantization.

  The present invention has been made in view of the above-described problems, and an encoding apparatus capable of suppressing image quality deterioration by performing adaptive quantization in consideration of the degree of deterioration of an encoded image and the characteristics of a block. The purpose is to propose.

An encoding apparatus according to the present invention includes: transforming means for orthogonally transforming input image data divided into a plurality of blocks in units of blocks, and outputting transform coefficient data; and quantizing the transform coefficient data output from the transform means. Quantization means for performing quantization in block units according to the quantization parameter; and feature extraction means for extracting a block having a predetermined feature that is easily noticeable from the plurality of blocks, and outputting the number of extracted blocks in the screen; the set control sensitivity for each feature of the block extracted by the feature extraction means, comprising a quantization control means for determining the quantization parameter for each block in accordance with the control sensitivity, the feature extracting means, the predetermined as a block having the features of, and extracted the block having a flat portion, and a block having an edge portion or the skin color portion, the quantization Control means, for a block having the edge portion or the skin color portion is extracted by the feature extraction means changes the value of the control sensitivity according to the number of blocks thereof extracted, the block having the flat portion Is characterized in that the value of the control sensitivity is not significantly changed in accordance with the number of extracted blocks .

  According to the present invention, since the quantization parameter is determined in accordance with the number of blocks extracted as blocks that are visually noticeable, even with the same generated code amount, it is possible to improve the image quality of blocks that are visually noticeable.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a schematic block diagram of an embodiment of an encoding apparatus according to the present invention. FIG. 2 shows a schematic block diagram of the feature extraction unit, and FIGS. 3 and 4 are explanatory diagrams of its operation.

  The input image data 100 is composed of image data in units of macroblocks having a predetermined size. The block size is determined by the encoding method, and is, for example, 16 × 16 pixels or 8 × 8 pixels in MPEG. Here, it is assumed that the size of the macroblock is 16 × 16 pixels.

  The rearrangement unit 101 rearranges the input image data 100 in the frame order corresponding to the encoded picture type, and outputs the image data in units of blocks for transform encoding. The subtracter 102 outputs the image data from the rearrangement unit 101 to the orthogonal transformation unit 103 as it is when the encoded picture pair type is an intra-frame coding (intra coding) method. In the case of the inter-frame coding (inter-coding) method, the subtracter 102 subtracts the prediction data from the reference image from the image data from the rearrangement unit 101, and outputs the obtained difference image data to the orthogonal transformation unit 103. .

  The orthogonal transform unit 103 performs orthogonal transform on the output data (image data or difference image data) of the subtractor 102 and outputs transform coefficient data to the quantization unit 104. The quantization unit 104 quantizes the transform coefficient data from the orthogonal transform unit 103 according to the quantization scale from the quantization control unit 112.

  The variable length coding unit 110 performs variable length coding on the quantized transform coefficient data from the quantization unit 104. The variable length encoding unit 110 multiplexes motion vector data for motion compensation described later on code data obtained by variable length encoding and supplies the multiplexed data to the buffer 114. The buffer 114 temporarily stores the data from the variable length encoding unit 110 and sequentially reads the data to the output terminal 116.

  The inverse quantization unit 105 inversely quantizes the output data from the quantization unit 104, and the inverse orthogonal transform unit 106 performs inverse orthogonal transform on the output data from the inverse quantization unit 105. The adder 108 outputs the output data of the inverse orthogonal transform unit 106 as it is in the case of intra coding, and adds the prediction data obtained in the previous decoding in the case of inter coding. By adding the prediction data, the image data is restored from the difference image data. The output data of the adder 108 corresponds to locally decoded image data.

  The video buffer 109 stores the locally decoded image data output from the adder 108 for a plurality of frames. The motion prediction motion compensation unit 107 compares the current image from the rearrangement unit 101 with the reference image stored in the video buffer 109 to predict motion, and calculates motion compensated predicted image data. The motion prediction motion compensation unit 107 outputs the calculated predicted image data to the subtractor 102 and the adder 108, and outputs motion vector information of the predicted image data to the variable length encoding unit 110.

  The code amount control unit 111 refers to the accumulated data amount in the buffer 114 and acquires the frame generation code amount. The code amount control unit 111 determines a target code amount according to the frame generation code amount from the buffer 114 and the feature amount from the feature extraction unit 113. More specifically, the allocated bit amount for each picture in the GOP is distributed based on the bit amount for a picture that has not yet been encoded in the GOP including the allocation target picture. This distribution is repeated in the order of the encoded pictures in the GOP, and a picture target code amount is set for each picture.

  The quantization control unit 112 acquires the generated code amount (block generated code amount) in units of macroblocks output from the variable length encoding unit 110 from the buffer 114. The quantization control unit 112 determines the quantization parameter according to the block generation code amount, the target code amount from the code amount control unit 111, and the activity from the feature extraction unit 113. More specifically, in order to match the target code amount for each picture with the actual generated code amount, the reference value of the quantization scale is determined from the block generated code amount. Based on the activity calculated by the feature extraction unit 113 with respect to the reference value of the quantization scale, the quantization parameter used by the quantization unit 104 is determined using Equation (11). If this activity is a small value, a larger amount of code can be allocated by reducing the quantization parameter.

  The above operation corresponds to Steps 1 to 3 described in the background art.

FIG. 2 shows a schematic block diagram of the feature extraction unit 113. The flatness detection unit 201 detects the degree (level) of the flat part from the image data in units of macroblocks from the rearrangement unit 101. Similarly, the edge detection unit 202 detects the degree (level) of the edge from the image data from the rearrangement unit 101. The skin color detection unit 203 detects the level (level) of the skin color from the image data from the rearrangement unit 101. FIG. 3 shows the detection characteristics or leveling functions of the detection units 201, 202, and 203. 3 (a) shows the level of characteristics for flat detection unit 201. FIG. 3B shows leveling characteristics for the edge detection unit 202. FIG. 3C shows leveling characteristics for the skin color detection unit 203.

  The flatness detection unit 201 calculates the variance for the image data of the macroblock, and sets the level to 1 if the variance is less than or equal to the threshold th1F, and infinite if the variance is greater than or equal to the threshold th2F. In the range from the threshold th1F to the threshold th2F, the level is calculated from a function connecting (th1F, 1) and (th2F, pre_avg). pre_avg is an average value of activities in a picture encoded one frame before.

  The edge detection unit 202 further divides the 16 × 16 pixel image data into 8 × 8 pixel sub-blocks, and calculates a variance for each sub-block. The difference between the maximum value and the minimum value of the obtained variance is calculated. If the difference value is equal to or greater than the threshold th2E, the level is set to 1, and if the difference value is equal to or less than the threshold th1E, the level is set to infinity. In the range from the threshold th1E to the threshold th2E, the level is calculated from the function connecting (th1E, pre_avg) and (th2E, 1).

  The flesh color detection unit 203 counts pixels that are flesh color from the luminance component and the color difference component in the macroblock image data. If the number of pixels is equal to or greater than the threshold th2S, the level is set to 1. If the number of pixels is equal to or less than the threshold th1S, the level is set to infinity. In the range from the threshold th1S to the threshold th2S, the level is calculated from the function connecting (th1S, pre_avg) and (th2S, 1).

  The minimum value detection unit 204 detects the minimum value among the levels calculated by the detection units 201, 202, and 203 and outputs the minimum value to the normalization unit 205. When the levels calculated by the detection units 201, 202, and 203 are all infinite, the minimum value detection unit 204 outputs the variance value of the block.

  The normalization unit 205 calculates a normalization activity using a predetermined control sensitivity (reaction parameter) for the pre-normalization activity from the minimum value detection unit 204. The control sensitivity can be set for each feature to be detected. In this embodiment, a flat portion control sensitivity 206, an edge portion control sensitivity 207, and a flesh color portion control sensitivity 208 are prepared corresponding to the detection units 201, 202, and 203. The control sensitivity is a parameter that determines the magnitude of the quantization parameter. If the control sensitivity is large, the control sensitivity acts to be smaller than the quantization parameter.

  The accumulating unit 209 counts the number of macroblocks detected by the detecting units 201, 202, and 203 for each type, and supplies the number of extracted blocks to the quantization control unit 112. With reference to FIG.4 and FIG.5, the relationship between each control sensitivity and the integrating | accumulating part 209 is demonstrated.

  FIG. 4 shows example images for explanation with different feature ratios. FIG. 4A shows an image in which the flat portion in the screen is 20%, the edge portion is 20%, and the skin color portion is 20% when feature extraction is performed. FIG. 4B shows an image in which the flat portion is 30%, the edge portion is 20%, and the skin color portion is 40%. Macroblocks that do not belong to any type shall fall into other categories. Those that are flesh-colored and flat like a human cheek belong to the flat part, and those that are flesh-colored and edge like a face outline belong to the edge part.

  FIG. 5 schematically shows the relationship between the number of macroblocks detected in the screen and the control sensitivity. FIG. 5A shows the control sensitivity for the flat portion, where the horizontal axis indicates the number of detected flat portions, and the vertical axis indicates the control sensitivity. FIG. 5B shows the control sensitivity for the edge portion. FIG. 5C shows the control sensitivity for the skin color portion.

  In the flat portion, the image quality varies by changing the quantization parameter, but the code amount does not vary much. That is, when the same quantization parameter is applied to the image example shown in FIG. 4A and the image example shown in FIG. 4B, the code amount of the entire flat portion does not change significantly. Therefore, in this embodiment, as shown in FIG. 5A, the control sensitivity for the flat portion is not changed greatly according to the number of detections.

  On the other hand, in the edge portion and the flesh color portion, the image quality fluctuates by changing the quantization parameter, but the code amount also fluctuates greatly. Although it is possible to reduce mosquito noise by lowering the quantization parameter of the edge portion, the amount of generated code increases accordingly. Similarly, it is possible to reduce the deterioration of the face by lowering the quantization parameter of the flesh color part, but the amount of generated code increases accordingly. If too many edges and flesh-color blocks are detected, a large amount of code is assigned to the edges and flesh-color blocks, and the amount of code assigned to blocks including high frequencies is reduced. Blocks containing high frequencies are somewhat rough, and visual degradation is considered to be inconspicuous even when quantized. However, if the amount of codes to be allocated is too small, the image quality is greatly reduced in blocks including high frequencies. Moreover, since the code amount of the entire image also increases, it becomes higher than a predetermined rate.

  In this embodiment, the control sensitivity is changed according to the number of detections for the edge portion and the skin color portion. As shown in FIGS. 5B and 5C, if the number of macroblocks detected as an edge portion or a flesh color portion is small, the control sensitivity is increased, thereby greatly reducing the quantization parameter. If the number of macroblocks detected as an edge portion or a skin color portion is large, the control sensitivity is reduced, thereby suppressing the quantization parameter from being lowered excessively. 5B and 5C show curves having the same tendency, but these curves can be set arbitrarily.

  In the image example shown in FIG. 4A and the image example shown in FIG. 4B, there is no difference in the number of detected edge portions, and therefore the quantization parameter is determined using a large control sensitivity. On the other hand, the number of detections for the skin color portion is larger for the image example shown in FIG. 4B than for the image example shown in FIG. Therefore, a larger control sensitivity is adopted for the image example shown in FIG. 4A, and a smaller control sensitivity is adopted for the image example shown in FIG. decide.

  The detection method of each detection unit is not limited to the method described here, and dispersion serving as an element for performing detection can be substituted by frequency conversion or the like. Moreover, although the flat part, the edge part, and the skin color part were mentioned as a kind to detect in this Example, it is good also considering this other kind as a component. If the type is a flat part and the code amount does not change much even if the quantization parameter is changed, the same control as that for the flat part is adopted. If the code amount changes greatly when the quantization parameter is changed, the same control as that of the edge portion and the skin color portion is adopted.

  It should be noted that each processing in the above-described embodiment can be realized by hardware, software, or a combination thereof. Part or all of the program constituting the software is stored in a nonvolatile recording medium such as an HDD, an optical disk, or a flash memory. It is clear that the functions necessary for the present invention can be realized by reading the software recorded on such a recording medium into a computer.

  Further, the software constituting the present invention may be transmitted to a remote computer via a transmission medium. The transmission medium is a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.

  The program may realize a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer, what is called a difference file (difference program) may be sufficient.

  The embodiment of the present invention has been described in detail with reference to the drawings, but the specific configuration is not limited to this embodiment, and a configuration within the scope of the present invention is also within the technical scope of the present invention. included.

It is a schematic block diagram of one Example of this invention. It is a schematic block diagram of a feature extraction unit. It is explanatory drawing of the leveling in a feature extraction part. It is an example of an explanatory image with a different feature ratio. It is a figure explaining the relationship between the detected number of features and control sensitivity.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Input image signal 101 ... Rearrangement part 102 ... Subtractor 103 ... Orthogonal transformation part 104 ... Quantization part 105 ... Inverse quantization part 106 ... Inverse orthogonal transformation part 107 ... motion prediction motion compensation unit 108 ... adder 109 ... video buffer (frame memory)
110: Variable length encoding unit 111: Code amount control unit 112 ... Quantization control unit 113 ... Feature extraction unit 114 ... Buffer 116 ... Output terminal

Claims (5)

Transform means for orthogonally transforming input image data divided into a plurality of blocks in units of blocks and outputting transform coefficient data;
Quantization means for quantizing the transform coefficient data output from the transform means in block units according to a quantization parameter;
A feature extraction means for extracting a block having a predetermined feature that is prominent in visual deterioration from the plurality of blocks, and outputting the number of extracted blocks in the screen;
Setting a control sensitivity for each feature of the block extracted by the feature extraction means, and comprising a quantization control means for determining the quantization parameter for each block according to the control sensitivity,
The feature extraction unit extracts a block having a flat portion and a block having an edge portion or a skin color portion as the block having the predetermined feature,
For the block having the edge portion or the flesh color portion extracted by the feature extraction portion, the quantization control means changes the value of the control sensitivity according to the number of extracted blocks , and the flat portion is changed. An encoding apparatus, wherein the control sensitivity value is not changed greatly according to the number of extracted blocks . The feature extraction means extracts a block having an edge as a block having the predetermined feature,
When the number of extracted blocks related to the edge portion is less than a threshold value, the quantization control means greatly changes the value of the control sensitivity set for the edge portion so as to lower the quantization parameter. When the number of extracted blocks related to a part is equal to or greater than the threshold value, the control sensitivity value set for the edge part is changed to a small value to suppress an excessive decrease in the quantization parameter. The encoding device according to 1. The feature extraction unit extracts a block having a flesh color part as the block having the predetermined feature,
When the number of extracted blocks related to the skin color portion is less than a threshold value, the quantization control means greatly changes the value of the control sensitivity set for the skin color portion to lower the quantization parameter, and the skin color The excessive reduction of the quantization parameter is suppressed by changing the value of the control sensitivity set for the skin color portion to be small when the number of extracted blocks related to the portion is equal to or greater than the threshold value. The encoding device according to 1. The feature extraction means extracts blocks having a flat portion, an edge portion, and a skin color portion, respectively, as the blocks having the predetermined feature,
The quantization control means changes the value of the control sensitivity set for each of the edge portion and the skin color portion according to the number of extracted blocks from the feature extraction means, and sets it for the flat portion. The encoding device according to claim 1, wherein the control sensitivity is set to a constant value regardless of the number of extracted blocks from the feature extraction unit . The encoding apparatus according to claim 1 , wherein the control sensitivity is a reaction parameter for calculating a normalized activity that is referred to when the quantization parameter is determined .

Images