JP2021129128A - Encoding device, encoding method, and program - Google Patents

Abstract

To flexibly control a quantization value within a frame by a smaller amount of codes.SOLUTION: The quantization value is determined in accordance with a determination method based on a technique for predicting a predictive image that is used in encoding a to-be-encoded unit, which is a unit of processing for performing an encoding process on an image, so that a quantization value for each to-be-encoded unit changes according to the image. During encoding of the quantization value determined in accordance with the determination method, a quantization parameter pertaining to the change in the quantization value for each to-be-encoded unit is excluded as for a P picture and a B picture from the object of transmission, and an encoded stream is generated. The present technique may be applied, for example, to an encoding device that performs encoding on the basis of HEVC.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to a coding device and a coding method, and a program, in particular, a coding device and a coding method that enable flexible control of a quantization value within a frame with a smaller amount of code, and a program. Regarding.

Conventionally, the amount of coding generated by coding a moving image can be adjusted by increasing or decreasing the quantization value. Therefore, adjusting the quantization value is an important factor in determining the transmission rate of the encoded stream, and at the same time, it is also important in determining the image quality of the encoded image.

Further, in the coding of the moving image, the information is compressed by using the entropy coding after removing the information that becomes redundant in expressing the moving image. Therefore, subjectively excellent coded image quality is required by utilizing the visual characteristics of human beings even under the same coded distortion index.

For example, according to human visual characteristics, it is known that when noise is added to a region having a complicated pattern and a region having a simple pattern, the noise is more likely to be recognized in the region having a simple pattern. Therefore, ideally, quantization is performed so as to dynamically change the quantization value according to the complexity of the pattern of the coding unit (hereinafter referred to as adaptive quantization), thereby improving the subjective image quality. Desirable above.

For example, in Patent Document 1, the coding efficiency of the quantization parameter is improved by selecting the quantization parameter having the smallest difference from the quantization parameter set for the coded block as the predictive quantization parameter. An image coding device to be used is disclosed.

Japanese Unexamined Patent Publication No. 2012-170042

By the way, in the conventional standardized coding method, when it is desired to fluctuate the quantization value for each coding unit for the purpose of adaptive quantization, the quantization parameter (difference quantum) related to the quantization value is used for each coding unit. It was necessary to transmit the quantized value cu_qp_delta_abs, the code cu_qp_delta_sign_flag). Therefore, the more the dynamic quantization value is controlled within the frame, the more the code amount related to the quantization parameter increases.

For example, the flag cu_qp_delta_enabled_flag, which indicates whether the quantization parameter value can be changed at the coding unit level, is multiplexed in the picture parameter set. Then, when it is determined that the quantization parameter for each coding unit is not transmitted based on the determination according to the flag cu_qp_delta_enabled_flag, the increase in the code amount can be suppressed. However, in that case, since the same quantization parameter is used through the coding units in the same frame, it becomes difficult to perform adaptive quantization. Further, in the standardization of moving image coding in the future, it is strongly required to improve the subjective image quality and reduce the coding amount.

The present disclosure has been made in view of such a situation, and makes it possible to flexibly control the quantization value in a frame with a smaller amount of code.

The coding device according to one aspect of the present disclosure is the coding target unit so that the quantization value of each coding target unit, which is a processing unit for performing coding processing on an image, fluctuates according to the image. A quantization unit that determines the quantization value according to a determination method based on a method for predicting a predicted image used when encoding, and a quantization unit that determines the quantization value according to the determination method. The predetermined image includes a coding unit that generates a coding stream by excluding the quantization parameter related to the variation of the quantization value for each coding target unit from the transmission target.

In the coding method of one aspect of the present disclosure, the coding device that performs the coding process on the image adapts the quantization value for each coding target unit that is the processing unit that performs the coding process to the image. The quantization value is determined according to a determination method based on a method for predicting a predicted image used when encoding the coded target unit, and the quantum determined according to the determination method. When encoding the coded value, it includes excluding the quantization parameter related to the variation of the quantization value for each coded target unit from the transmission target to generate a coded stream.

In the program of one aspect of the present disclosure, in the computer of the coding apparatus that performs the coding process on the image, the quantization value for each coding target unit that is the processing unit that performs the coding process is applied to the image. The quantization value is determined according to a determination method based on a method for predicting a predicted image used when encoding the coded target unit, and the quantum determined according to the determination method. When encoding the coded value, a process including excluding the quantization parameter related to the variation of the quantization value for each coded target unit from the transmission target and generating a coded stream is executed.

In one aspect of the present disclosure, when the coded unit is encoded so that the quantization value of each coded unit, which is a processing unit for performing the coding process on the image, fluctuates according to the image. The quantization value is determined according to the determination method based on the method for predicting the predicted image used in, and when the quantization value determined according to the determination method is encoded, the variation of the quantization value for each coded unit. A coded stream is generated by excluding the quantization parameters related to.

According to one aspect of the present disclosure, the quantization value can be flexibly controlled in the frame with a smaller amount of code.

The effects described here are not necessarily limited, and may be any of the effects described in the present disclosure.

It is a block diagram which shows the structural example of one Embodiment of the coding apparatus to which this technique is applied. It is a figure which shows an example of a picture parameter set. It is a figure which shows an example of a picture parameter set. It is a figure which shows an example of the delta QP syntax. It is a figure which shows the method of determining the quantization value at the time of intra prediction. It is a figure which shows the prediction mode in the screen of HEVC. It is a figure explaining the 1st determination method which determines the quantization value. It is a figure explaining the 2nd determination method which determines the quantization value. It is a figure explaining the 3rd determination method which determines the quantization value. It is a figure explaining the 3rd determination method by taking the coding target unit of 8 pixels × 8 pixels as an example. It is a figure explaining how to obtain the quantization value in the prediction mode 3. It is a figure explaining how to obtain the quantization value in the prediction mode 19. It is a figure which shows the comparison about the 1st to 3rd determination methods. It is a figure which shows the method of determining the quantization value at the time of inter prediction. It is a figure which shows an example of the quantized value of I picture, P picture, and B picture. It is a figure explaining how to obtain the quantization value when the coding target unit of a P picture is a coding target. It is a figure explaining how to obtain the quantization value when the coding target unit of a B picture is a coding target. It is a block diagram which shows the structural example of one Embodiment of the decoding apparatus to which this technique is applied. It is a flowchart explaining the adaptive quantization value determination process in a coding apparatus. It is a flowchart explaining the adaptive quantization value determination process in a decoding apparatus. It is a block diagram which shows the structural example of one Embodiment of the computer to which this technique is applied.

Hereinafter, specific embodiments to which the present technology is applied will be described in detail with reference to the drawings.

<Configuration example of coding device>
FIG. 1 is a block diagram showing a configuration example of an embodiment of a coding device to which the present technology is applied.

As shown in FIG. 1, the coding device 11 includes a calculation unit 21, a quantization value memory unit 22, a frequency conversion quantization unit 23, an inverse quantization inverse frequency conversion unit 24, an arithmetic unit 25, a loop filter unit 26, and a frame. It includes a memory 27, an in-screen prediction unit 28, a motion compensation prediction unit 29, a prediction image selection unit 30, an entropy coding unit 31, and a coding control unit 32.

Image data of the input image supplied to the coding device 11 is input to the calculation unit 21, the predicted image supplied from the in-screen prediction unit 28 or the motion compensation prediction unit 29 is subtracted, and the difference information is frequency-converted. It is supplied to the quantization unit 23.

The quantization value memory unit 22 holds the quantization value supplied from the frequency conversion quantization unit 23.

The frequency conversion quantization unit 23 acquires the conversion coefficient by frequency-converting the difference information supplied from the calculation unit 21. Then, the frequency conversion quantization unit 23 quantizes the conversion coefficient with reference to the quantization value (or the quantization value supplied from the coding control unit 32) held in the quantization value memory unit 22. The quantization value is obtained and supplied to the entropy coding unit 31.

Inverse quantization The inverse frequency conversion unit 24 is supplied with the quantization value supplied from the frequency conversion quantization unit 23 to the entropy coding unit 31. Then, the inverse quantization inverse frequency conversion unit 24 acquires the difference information before the frequency conversion and the quantization are performed in the frequency conversion quantization unit 23 by inverse quantization and the inverse frequency conversion of the quantization value. , Supply to the calculation unit 25.

Difference information is supplied to the calculation unit 25 from the inverse quantization inverse frequency conversion unit 24, and the prediction image selected by the prediction image selection unit 30 is supplied. Then, the calculation unit 25 generates a reconstructed image in which the input image is reconstructed by adding the difference information to the predicted image, and supplies the reconstructed image to the loop filter unit 26 and the in-screen prediction unit 28.

The loop filter unit 26 appropriately filters the reconstructed image shared by the calculation unit 25, and then supplies the reconstructed image to the frame memory 27.

The frame memory 27 holds the reconstructed image supplied from the loop filter unit 26, and supplies the reconstructed image necessary for generating the predicted image in the motion compensation prediction unit 29 to the motion compensation prediction unit 29.

The in-screen prediction unit 28 refers to the reconstructed image supplied from the calculation unit 25, performs intra-prediction on the image data of the input image, and generates a prediction image.

The motion compensation prediction unit 29 refers to the reconstructed image (past frame) supplied from the frame memory 27 and the input image (current frame) input to the coding device 11, and image data of the input image. Inter-prediction is performed for and a prediction image is generated.

The prediction image selection unit 30 selects one of the prediction image generated by the in-screen prediction unit 28 and the prediction image generated by the motion compensation prediction unit 29, and supplies the prediction image to the calculation unit 21 and the calculation unit 25. do.

The entropy coding unit 31 entropy-encodes the quantization value supplied from the frequency conversion quantization unit 23, and outputs a coded stream.

The coding control unit 32 controls each block constituting the coding device 11 in order to code the input image in the coding device 11. For example, when the coding device 11 does not perform the adaptive quantization value determination process described later, the coding control unit 32 appropriately determines the quantization value and supplies it to the frequency conversion quantization unit 23.

When the quantization value is changed in the frame, the coding device 11 configured in this way controls the code amount and the image quality of the I picture by determining the quantization parameter on the coding side as before. , Quantization parameters for each coding unit are transmitted. On the other hand, the coding device 11 can reduce the amount of coding by not transmitting the quantization parameter for each coding unit for the P picture and the B picture.

At this time, even if the coding device 11 adaptively changes the quantization value according to the image quality, the decoding side can autonomously obtain the quantization value without using the quantization parameter for each coding unit. A process of determining the quantization value (hereinafter, referred to as an adaptive quantization value determination process) is performed so as to be possible, and the P picture and the B picture are encoded. Therefore, the decoding side can also autonomously determine the quantization value of the P picture and the B picture by performing the adaptive quantization value determination process. As a result, even if the coding device 11 does not transmit the quantization parameter for each coding unit, the decoding side autonomously determines the quantization value and controls the image quality while adaptively changing the quantization value. It can be carried out.

Therefore, the coding device 11 can flexibly control the quantization value in the frame with a smaller amount of code.

<Explanation of syntax elements>
In FIGS. 2 and 3, HEVC (High Efficiency Video Coding), which is one of the video coding methods, is taken as an example, and a picture parameter set to which a syntax element for realizing adaptive quantization value determination processing is added is added. It is shown. Similarly, FIG. 4 shows an example of the Delta QP syntax.

For example, in the picture parameter set and the delta QP syntax, the syntax element is the flag pps_autonomous_qp_flag indicating whether or not to execute the adaptive quantization value determination process as shown in bold in FIGS. 2 and 4. It is provided as. That is, when the flag pps_autonomous_qp_flag is 1, it indicates that the adaptive quantization value determination process is executed. On the other hand, when the flag pps_autonomous_qp_flag is 0, it means that the process of encoding and transmitting the quantization parameter is executed as in the conventional coding method without executing the adaptive quantization value determination process.

Further, this flag pps_autonomous_qp_flag is a syntax element that is valid only in the P picture and the B picture, but is invalid in the I picture. Therefore, when the P picture and the B picture are the objects of coding and the flag pps_autonomous_qp_flag is 0, the flag cu_qp_delta_enabled_flag indicating whether or not the quantization parameter value can be changed at the coding unit level. The conventional quantization parameter syntax is transmitted according to the above. On the other hand, when the P picture and the B picture are the objects to be encoded and the flag pps_autonomous_qp_flag is 1, the quantization parameters (cu_qp_delta_abs, cu_qp_delta_sign_flag) are not transmitted even if the flag cu_qp_delta_enabled_flag is 1.

Here, how to use the syntax in each of the I picture and the P picture and the B picture will be described in detail.

For example, when the I picture is the object of coding, the flag pps_autonomous_qp_flag indicating whether or not to execute the adaptive quantization value determination process is invalid, and the same coding control and coding as before is performed. ..

That is, when it is desired to dynamically change the quantization value in the coding unit in the frame, 1 is set in the flag cu_qp_delta_enabled_flag indicating whether or not the quantization parameter value can be changed at the coding unit level. do. As a result, the quantization parameter suitable for each coding unit is determined in consideration of the image feature amount of the coding unit, and the quantization parameter (cu_qp_delta_abs, cu_qp_delta_sign_flag) is transmitted according to the quantization parameter.

On the other hand, when the quantization parameter is not controlled in the coding unit unit in the frame, the flag cu_qp_delta_enabled_flag indicating whether or not the quantization parameter value can be changed at the coding unit level is set. Set to 0. As a result, the quantization parameters (cu_qp_delta_abs, cu_qp_delta_sign_flag) are not transmitted, and the quantization parameters in the frame become constant.

Further, when the P picture and the B picture are to be coded and the flag pps_autonomous_qp_flag indicating whether or not to execute the adaptive quantization value determination process is 0, the coding control similar to the conventional one is performed. And perform coding.

That is, when the flag cu_qp_delta_enabled_flag is 1, the quantization parameter suitable for each coding unit is determined from the feature amount of the image of the coding unit as in the case of the I picture. Then, the quantization parameters (cu_qp_delta_abs, cu_qp_delta_sign_flag) are transmitted according to the determination. On the other hand, when the flag cu_qp_delta_enabled_flag is 0, the quantization parameter control for each coding unit is not performed, and the quantization parameter in the frame does not change.

Then, when the flag pps_autonomous_qp_flag indicating whether or not to execute the adaptive quantization value determination process is 1, the adaptive quantization value determination process is executed. In that case, the quantization parameters (cu_qp_delta_abs, cu_qp_delta_sign_flag) are not transmitted regardless of the value of the flag cu_qp_delta_enabled_flag. In particular, when the flag cu_qp_dleta_enabled_flag is 1, the quantization value of the coded unit is autonomously derived using the quantization value of the peripheral coded unit without using the quantization parameter. This enables adaptive quantization control in which the quantization value of each unit in the picture is variable without transmitting the quantization parameters (cu_qp_delta_abs, cu_qp_delta_sign_flag).

Regarding the syntax elements (pps_qp_delta_p, pps_qp_delta_bi, pps_qp_delta_bp) related to the adjustment of the picture average quantization value of the P picture and the B picture in the picture parameter set shown in FIG. 2, the adaptive quantization value is determined according to the prediction type. The syntax elements (cu_qp_delta_coded_type) related to the applicability of the process will be described later together with their usage.

<Method of determining the quantization value of P picture and B picture at the time of intra prediction>
A method of determining the quantization value of the P picture and the B picture at the time of intra prediction will be described with reference to FIGS. 5 to 13.

For example, as shown in FIG. 5, when the coded unit is an in-screen prediction (intra-prediction), the quantization value of the coded unit is set using the quantization value of the peripheral coded units. It is determined. Further, in the following, the first to third determination methods used as a method for determining the quantization value in the adaptive quantization value determination processing will be described in detail, taking the in-screen prediction mode of HEVC as shown in FIG. 6 as an example. explain.

The first determination method used in the adaptive quantization value determination process will be described with reference to FIG. 7.

In the first determination method, as shown in FIG. 7, the position of the upper left pixel of the coded target unit of N pixels × N pixels is set as the coordinates (0,0). Then, as a unit adjacent to the unit to be encoded, the adjacent unit A including the pixel of the coordinate (-1,0), the adjacent unit B including the pixel of the coordinate (0, -1), and the coordinate (N, -1). An adjacent unit C including pixels, an adjacent unit D including pixels at coordinates (-1, -1), and an adjacent unit E including pixels at coordinates (-1, N) are set.

Here, in the formula used in the following description, integer division is performed so that the result of the operation is rounded down toward 0. For example, the operation of 7/4 and -7 / -4 gives a result of 1, and the operation of -7/4 and 7 / -4 gives a result of -1.

When the planer prediction (INTRA_PLANAR) is set as the prediction mode (predModeIntra = 0), the average value of the quantization values of the adjacent unit A, the adjacent unit B, the adjacent unit C, and the adjacent unit E is set to the coded unit. Used as a quantization value. That is, the quantization value Q _target to be coded unit, the quantized value Q _A of the adjacent units _A, the quantization value Q _B of the adjacent units _B, the quantization value Q _C of the adjacent unit _C, and of the adjacent units E Quantum using reduction value Q _E, it is determined according to the following equation (1).

When DC prediction (INTRA_DC) is set as the prediction mode (predModeIntra = 1), the average value of the quantization values of the adjacent unit A and the adjacent unit B is used as the quantization value of the unit to be encoded. That is, the quantization value Q _target to be coded unit using the quantization value Q _B of the quantization values Q _A of the adjacent units _A, and the adjacent units B, it is determined according to the following equation (2).

When the prediction directions 2 to 9 (INTRA_ANGULAR2 to 9) of the direction prediction are set as the prediction mode (predModeIntra = 2 to 9), the average value of the quantized values of the adjacent unit A and the adjacent unit E is encoded. Used as the quantization value of the target unit. That is, the quantization value Q _target to be coded unit using the quantization value Q _E of the quantization values Q _A of the adjacent units _A, and the adjacent unit E, is determined according to the following equation (3).

When the prediction direction 10 (INTRA_ANGULAR10) of the direction prediction is set as the prediction mode (predModeIntra = 10), the quantization value of the adjacent unit A is used as the quantization value of the coded target unit. That is, the quantization value Q _target of the coded target unit is obtained according to the following equation (4) using _{the quantization value Q A} of the adjacent unit A.

When the prediction direction 11 to 25 (INTRA_ANGULAR 11 to 25) of the direction prediction is set as the prediction mode (predModeIntra = 11 to 25), the average value of the quantization values of the adjacent unit A, the adjacent unit B, and the adjacent unit D. Is used as the quantization value of the unit to be encoded. That is, the quantization value Q _target of the coded target unit is the following, using the quantization value Q _{A of} the adjacent unit A, the quantization value Q _{B of} the adjacent unit B, and the quantization value Q _{D of the adjacent unit D.} It is obtained according to the formula (5).

When the prediction direction 26 (INTRA_ANGULAR26) of the direction prediction is set as the prediction mode (predModeIntra = 26), the quantization value of the adjacent unit B is used as the quantization value of the coded target unit. That is, the quantization value Q _target of the coded target unit is obtained according to the following equation (6) using _{the quantization value Q B} of the adjacent unit B.

When the prediction direction 27 to 34 (INTRA_ANGULAR27 to 34) of the direction prediction is set as the prediction mode (predModeIntra = 27 to 34), the average value of the quantized values of the adjacent unit B and the adjacent unit C is encoded. Used as the quantization value of the target unit. That is, the quantization value Q _target to be coded unit using the quantization value Q _C of the quantized value Q _B of the adjacent units _B, and the adjacent unit C, determined according to the following equation (7).

As described above, in the first determination method used in the adaptive quantization value determination process, any of the above-mentioned equations (1) to (7) is selected according to the prediction mode, and the coded unit is subjected to. The quantization value is determined.

A second determination method used in the adaptive quantization value determination process will be described with reference to FIG.

In the second determination method, as shown in FIG. 8, h adjacent to the left side as a unit adjacent to the unit to be encoded and containing pixels necessary for generating an in-screen predicted image. An adjacent unit A (0) to A (h-1) is set. Similarly, i adjacent units B (0) to B (i-1) adjacent to the upper side, j adjacent units C (0) to C (j-1) adjacent to the upper right side, and adjacent to the upper left side. Adjacent units D and k adjacent units E (0) to E (k-1) adjacent to the lower left side are set.

When planar prediction (INTRA_PLANAR) is set as the prediction mode (predModeIntra = 0), adjacent units A (0) to A (h-1), adjacent units B (0) to B (i-1), adjacent units The average value of the quantization values of C (0) and the adjacent unit E (0) is used as the quantization value of the unit to be encoded. That is, the quantization value Q _target of the unit to be encoded is the quantization value Q _{A (0) to} Q _{A (h-1) of} the adjacent units A (0) to A (h-1), and the adjacent unit B (0). ) To B (i-1) quantized values Q _{B (0) to} Q _{B (i-1)} , the quantized values QC _{(0) of} the adjacent unit C (0), and the adjacent unit E (0). It is obtained according to the following equation (8) using the quantized value QE _(0).

When DC prediction (INTRA_DC) is set as the prediction mode (predModeIntra = 1), the quanta of adjacent units A (0) to A (h-1) and adjacent units B (0) to B (i-1). The average value of the coded values is used as the quantization value of the coded unit. _{That is, the quantization value Q target} of the coded target unit is the quantization values Q _{A (0) to} Q _{A (h-1) of} the adjacent units A (0) to A (h-1), and the adjacent unit B ( It is obtained according to the following equation (9) using the quantized values Q _{B (0) to} Q _{B (i-1) of 0) to B (i-1).}

When the prediction directions 2 to 9 (INTRA_ANGULAR2 to 9) of the directional prediction are set as the prediction modes (predModeIntra = 2 to 9), the adjacent units A (0) to A (h-1), and the adjacent units E ( The average value of the quantization values of 0) to E (k-1) is used as the quantization value of the unit to be encoded. _{That is, the quantized value Q target} of the coded target unit is the quantized values Q _{A (0) to} Q _{A (h-1) of} the adjacent units A (0) to A (h-1), and the adjacent unit E ( It is obtained according to the following equation (10) using the quantized values Q _{E (0) to} Q _{E (k-1) of 0) to E (k-1).}

When the prediction direction 10 (INTRA_ANGULAR10) of the direction prediction is set as the prediction mode (predModeIntra = 10), the quantization value of the adjacent units A (0) to A (h-1) is quantized in the unit to be encoded. Use as a value. That is, the quantization value Q _target of the coded target unit is the following, using _{the quantization values Q A (0) to} Q _{A (h-1)} of the adjacent units A (0) to A (h-1). It is obtained according to the formula (11).

When the prediction directions 11 to 25 (INTRA_ANGULAR 11 to 25) of the direction prediction are set as the prediction mode (predModeIntra = 11 to 25), the adjacent units A (0) to A (h-1) and the adjacent units B (0). ) To B (i-1) and the average value of the quantized values of the adjacent unit D are used as the quantized values of the unit to be encoded. That is, the quantization value Q _target of the unit to be encoded is the quantization value Q _{A (0) to} Q _{A (h-1) of} the adjacent units A (0) to A (h-1), and the adjacent unit B (0). ) To B (i-1) quantized values Q _{B (0) to} Q _{B (i-1) , and the quantized values Q D} of the adjacent unit D, and obtained according to the following equation (12).

When the prediction direction 26 (INTRA_ANGULAR26) of the direction prediction is set as the prediction mode (predModeIntra = 26), the quantization value of the adjacent units B (0) to B (i-1) is quantized in the unit to be encoded. Use as a value. That is, the quantized value Q _{target of the coded target unit uses the quantized values Q B (0) to} Q _{B (i-1)} of the adjacent units B (0) to B (i-1) as follows. It is obtained according to the formula (13).

When the prediction directions 27 to 34 (INTRA_ANGULAR27 to 34) of the directional prediction are set as the prediction modes (predModeIntra = 27 to 34), the adjacent units B (0) to B (i-1), and the adjacent units C ( The average value of the quantization values of 0) to C (j-1) is used as the quantization value of the unit to be encoded. _{That is, the quantized value Q target} of the coded target unit is the quantized values Q _{B (0) to} Q _{B (i-1) of} the adjacent units B (0) to B (i-1), and the adjacent unit C ( It is obtained according to the following equation (14) using the quantized values QC _{(0) to QC (j-1) of 0) to C (j-1).}

As described above, in the second determination method used in the adaptive quantization value determination process, any of the above-mentioned equations (8) to (14) is selected according to the prediction mode, and the coded unit is selected. The quantization value is determined.

A third determination method used in the adaptive quantization value determination process will be described with reference to FIGS. 9 to 12.

In the third determination method, as shown in FIG. 9, as n units adjacent to the unit to be coded and including pixels necessary for generating a predicted image by intra-prediction, the lower left. Adjacent units (0) to (n-1) are set in order from to the upper right.

At this time, the quantization values Q _target of the coded target unit are the quantization values Q _{0 to} Q _n-1 of the adjacent units (0) to (n-1) and the adjacent units (0) to (n-1). Of the number of pixels in contact with the unit to be encoded included in, the number of pixels Cnt (0) to Cnt (n-1) that contribute to the generation of the predicted image in the intra prediction is used according to the following equation (15). Desired.

Here, in FIGS. 10 to 12, a method of obtaining the quantized value using the third determination method will be described by taking an coded target unit of 8 pixels × 8 pixels as an example.

As shown in FIG. 10, it is assumed that the number of adjacent units including the pixels required for generating the predicted image by the intra prediction is 7 (n = 7) for the coded target unit of 8 pixels × 8 pixels. , Adjacent unit (0) to adjacent unit (6) are set. _{At this time, the quantized values Q 0 to} Q _{6 of} the adjacent unit (0) to the adjacent unit (6) are set.

In FIG. 10, the circles shown by solid lines are pixels included in adjacent units, which may be used to generate a prediction image in intra-prediction, and the numbers in parentheses are the pixels. Represents the number of adjacent units to which it belongs.

FIG. 11 shows an intra-prediction when the prediction direction 3 (prediction mode 3) of the directional prediction is set as the prediction mode for the coded target unit shown in FIG. In FIG. 11, the pixel represented by the solid line indicates a pixel that contributes to the generation of the predicted pixel in the prediction mode 3.

As shown in the figure, in the prediction mode 3, the number of pixels contributing to the generation of the intra prediction image is 7 pixels for the adjacent unit (0), 4 pixels for the adjacent unit (1), and 4 pixels for the adjacent unit (2). Is included. On the other hand, for the adjacent units (3) to (6) other than these, there are no pixels that contribute to the generation of the predicted image. From this, the number of pixels Cnt (0) = 7, the number of pixels Cnt (1) = 4, the number of pixels Cnt (2) = 4, and the number of pixels Cnt (3) to Cnt (6) = 0 are set.

Thus, in the example shown in FIG. 11, the quantization value Q _target to be coded unit, the quantized value Q ₀ of the adjacent units _(0), the quantized values to Q ₁ adjacent units _(1), and adjacent unit (2 ) using the quantization value _{Q 2} of, it is determined according to the following equation (16).

FIG. 12 shows an intra-prediction when the prediction direction 19 (prediction mode 19) of the directional prediction is set as the prediction mode for the coded target unit shown in FIG. In FIG. 12, the pixel represented by the solid line indicates a pixel that contributes to the generation of the predicted pixel in the prediction mode 19. Further, in the case of the prediction mode 19, since the angle (intraPredAngle) shown in FIG. 6 is negative, the pixels for generating the prediction pixels are transferred and used as shown in FIG.

As a result, in the prediction mode 19, the number of pixels that contributes to the generation of the intra prediction image is 3 pixels for the adjacent unit (1), 3 pixels for the adjacent unit (2), 1 pixel for the adjacent unit (3), and adjacent units. The unit (4) contains 8 pixels. On the other hand, with respect to the adjacent unit (0), the adjacent unit (5), and the adjacent unit (6) other than these, there are no pixels that contribute to the generation of the predicted image. From this, the number of pixels Cnt (0) = 0, the number of pixels Cnt (1) and Cnt (2) = 3, the number of pixels Cnt (3) = 1, the number of pixels Cnt (4) = 8, and the number of pixels Cnt. (5) and Cnt (6) = 0 are set.

Thus, in the example shown in FIG. 12, the quantization value Q _target to be coded unit, the quantized values to Q ₁ adjacent units _(1), the quantization value Q ₂ of the adjacent unit _(2), adjacent unit (3) It is obtained according to the following equation (17) using the quantization value Q _{3 of} the above and the quantization value Q _{4 of the adjacent unit (4).}

As described above, in the third determination method used in the adaptive quantization value determination process, the quantum of the unit to be encoded is based on the above-mentioned equation (15) based on the number of pixels that contribute to the generation of the predicted image in the intra prediction. The conversion value is determined.

By the way, in the first to third determination methods, when there is no encoded unit and there is no peripheral quantization value, the initial value is used. On the other hand, in other cases, that is, when there is a coded unit in which the quantized value does not exist, it is copied from the unit in which the quantized value exists and used as described below.

For example, when the quantization value of the unit E of FIG. 5 described above does not exist, the unit A, the unit D, the unit B, and the unit C are scanned in this order, and the quantization value of the unit existing at the beginning is copied. If the quantized value of unit A does not exist, the quantized value of unit E is copied. Similarly, if the quantized value of unit D does not exist, the quantized value of unit A is copied, and if the quantized value of unit B does not exist, the quantized value of unit D is copied. When there is at least one encoded unit in this way, the quantization value of the unit in which the quantization value exists can be used.

Here, with reference to FIG. 13, the first to third determination methods used in the adaptive quantization value determination process will be compared and described.

The first determination method limits the adjacent units used for obtaining the quantization value of the coded unit to one unit for each of the left, upper, upper right, upper left, and lower left of the coded unit. Then, the number of pixels that contributes to the generation of the intra-predicted image included in each adjacent unit is not evaluated (considered).

The second determination method uses all units including the pixels required for intra-predicted image generation of the coded unit for adjacent units used to determine the quantization value of the coded unit. Then, the number of pixels that contributes to the generation of the intra-predicted image included in each adjacent unit is not evaluated.

The third determination method uses all the units including the pixels necessary for generating the intra-predicted image of the coded unit for the adjacent unit used to obtain the quantization value of the coded unit. Then, the number of pixels that contributes to the generation of the intra-predicted image included in each adjacent unit is evaluated.

For example, when a certain intra prediction mode is selected, it is considered that the spatial correlation with the direction is high. Therefore, the quantization value of the adjacent unit in that direction is also emphasized. It is considered that this makes it possible to derive the quantization value of the coded unit with higher accuracy.

Therefore, it is a derivation means that emphasizes spatial correlation in the order of the first determination method, the second determination method, and the third determination method. Therefore, while it is possible to derive the quantization value of the coded target unit with high accuracy in this order, the calculation cost increases in this order. That is, in order to improve the accuracy, there is a trade-off in which the cost for mounting is high.

In the new coding method, for which standardization work is currently underway, there are rectangular prediction blocks other than squares in the in-screen prediction compared to HEVC shown as the existing method, and the number of directional prediction modes is large. Improvements such as an increase in the number of filter taps for pixels used for predictive pixel generation have been added. Here, the first to third determination methods are the same as the existing method in that peripheral encoded pixels are used, and can be applied to a new coding method.

<Method of determining the quantization value of P-picture and B-picture at the time of inter-prediction>
A method of determining the quantization value of the P picture and the B picture at the time of inter-prediction will be described with reference to FIGS. 14 to 17.

For example, as shown in FIG. 14, when the coding target unit is inter-screen prediction (inter-prediction), the quantization value of the coding unit including the upper left pixel of the reference position is copied and used.

Further, FIG. 15 shows an example of the quantized values of the I picture, the P picture, and the B picture when the interval M of the P pictures is 3 frames and the interval N of the I pictures is 15 frames (M = 3 /). N = 15) is shown. In general coding control, the quantization value of the I picture, the P picture, and the B picture is determined from the viewpoint of, for example, the importance as a reference image and the code amount control. Therefore, as shown in FIG. 15, as for the average value of the quantization value in each picture, the average value of the quantization value of the I picture is the smallest, the average value of the quantization value of the P picture is the next smallest, and B It is often controlled so that the average value of the quantization value of the picture is the largest.

Then, in the adaptive quantization value determination process performed by the coding apparatus 11, when the unit to be coded is inter-screen prediction (inter-prediction), the quantization value of the I picture is copied and used in the P picture. .. Further, in the B picture, the quantized value of the I picture, the P picture, or the B picture is copied and used.

Therefore, for example, as the quantization value of the inter-screen prediction unit in the P picture, the quantization value of the I picture is used as it is, and it becomes impossible to cause a difference in the quantization value between the pictures. Therefore, the adaptive quantization value determination process performed by the coding apparatus 11 is a syntax element (pps_qp_delta_p, pps_qp_delta_bi, pps_qp_delta_bp) related to the adjustment of the picture average quantization value of the P picture and the B picture as shown in FIG. To use. As a result, it is possible to generate a difference from the quantization value used in the coding unit that is the copy source in the inter-screen prediction unit, that is, the quantization value of the reference picture.

That is, the syntax element pps_qp_delta_p is used so as to be added when the in-screen prediction (inter-prediction) unit to be encoded in the P picture copies and uses the quantization value. Further, the syntax element pps_qp_delta_bi is used so that the in-screen prediction (inter-prediction) unit to be encoded in the B picture adds the quantization value of the I picture when it is copied and used. Similarly, the syntax element pps_qp_delta_bp is used so that the in-screen prediction (inter-prediction) unit to be encoded in the B picture adds when copying and using the quantized value of the P picture.

With reference to FIGS. 16 and 17, a method of obtaining the quantization value Q _target of the coded target unit using the syntax elements (pps_qp_delta_p, pps_qp_delta_bi, pps_qp_delta_bp) will be described.

For example, as shown in FIG. 16, when the coded target unit of the P picture is the coded target, the quantization value Q _{target of the coded target unit is the quantization value Q _ref_I} of the copy source in the referenced I picture. Is obtained as shown in the following equation (18).

Further, as shown in FIG. 17, when the coded target unit of the B picture is the coded target and the one-way prediction with reference to the I picture _{is performed, the quantization value Q target} of the coded target unit is the copy source. Using the quantized value Q _{_ref_I} of the I picture, it is obtained as shown in the following equation (19).

Similarly, when the coded target unit of the B picture is the coded target and the one-way prediction with reference to the P picture _{is performed, the quantization value Q target} of the coded target unit is the quantum of the P picture that is the copy source. _{Using the quantized} value Q _ref_P, it is calculated as shown in the following equation (20).

Further, when the coded target unit of the B picture is the coded target and the twin prediction refers to the I picture and the P picture, the quantization value Q _target of the coded target unit is the copy source I picture. using the quantization value Q _{_Ref_P} quantized value Q _{_Ref_I} and P pictures, is obtained as shown in the following equation (21).

Here, as described with reference to FIG. 15, the quantized values of the I picture, the P picture, and the B picture have the smallest average value of the quantized values of the I picture, and the average of the quantized values of the P picture. It is often controlled so that the value is the next smallest and the average value of the quantization value of the B picture is the largest. Therefore, the syntax elements (pps_qp_delta_p, pps_qp_delta_bi, pps_qp_delta_bp) take only positive values.

When the B picture is referred to in the B picture, it is often the same picture type and the tendency of the quantization value control does not change significantly. Further, when the coded target unit is an in-screen prediction (intra) unit, the quantized value in the same picture is used. Therefore, it is not necessary to adjust the quantization value for use.

<Switching of applicable units>
The syntax element cu_qp_delta_coded_type shown in the picture parameter set of FIG. 2 and the delta QP syntax of FIG. 4 applies the adaptive quantization value determination process performed by the coding device 11 only to the in-screen prediction (intra) unit. Shows switching between doing, applying only to interscreen prediction (inter) units, and applying to all coding units.

For example, the syntax element cu_qp_delta_coded_type is set to a value of 0, 1, or 2. Then, when the syntax element cu_qp_delta_coded_type is 0, the adaptive quantization value determination process is performed only for the in-screen prediction unit. When the syntax element cu_qp_delta_coded_type is 1, the adaptive quantization value determination process is performed only for the inter-screen prediction unit. When the syntax element cu_qp_delta_coded_type is 2, the adaptive quantization value determination process is performed for both the in-screen prediction unit and the inter-screen prediction unit.

In addition, when applied only to the in-screen prediction unit, or when applied only to the inter-screen prediction unit, the coding unit on the non-applicable side performs accurate quantization value control according to the characteristics of the coding unit. It is possible. Therefore, the parameter related to the quantization value is not transmitted in the coding unit on the applied side. Therefore, the amount of code increases as compared with the case where it is applied to both the in-screen prediction unit and the inter-screen prediction unit, but the quantization value is larger than that when it is applied to both the in-screen prediction unit and the inter-screen prediction unit. The control accuracy is high. Therefore, it is possible to perform an operation that is intermediate between the conventional method and the case where it is applied to both the in-screen prediction unit and the inter-screen prediction unit.

<Configuration example of decoding device>
FIG. 18 is a block diagram showing a configuration example of an embodiment of a decoding device to which the present technology is applied.

As shown in FIG. 18, the decoding device 12 includes an entropy decoding unit 41, a quantization value memory unit 42, an inverse quantization inverse frequency conversion unit 43, an arithmetic unit 44, a loop filter unit 45, a frame memory 46, and an in-screen prediction unit. It includes 47, a motion compensation prediction unit 48, and a prediction image selection unit 49.

In the decoding device 12, the quantization value memory unit 42, the inverse quantization inverse frequency conversion unit 43, the calculation unit 44, the loop filter unit 45, the frame memory 46, the in-screen prediction unit 47, the motion compensation prediction unit 48, and the prediction. The image selection unit 49 includes a quantization value memory unit 22, an inverse quantization inverse frequency conversion unit 24, a calculation unit 25, a loop filter unit 26, a frame memory 27, and an in-screen prediction unit 28 included in the coding device 11 of FIG. It is configured in the same manner as the motion compensation prediction unit 29 and the prediction image selection unit 30.

That is, in the decoding device 12, the coded stream transmitted from the coding device 11 is supplied to the entropy decoding unit 41. The entropy decoding unit 41 uses the quantization value memory unit 42, the inverse quantization inverse frequency conversion unit 43, the loop filter unit 45, the in-screen prediction unit 47, and the in-screen prediction unit 47 to obtain the quantization value obtained by entropy decoding the encoded stream. It is supplied to the motion compensation prediction unit 48.

The quantization value memory unit 42 holds the quantization value supplied from the entropy decoding unit 41. Inverse quantization Inverse frequency conversion unit 43 acquires a conversion coefficient by dequantizing the quantization value held in the quantization value memory unit 42 or the quantization value decoded by the entropy decoding unit 41. , The difference information is acquired by reverse frequency conversion of the conversion coefficient and supplied to the calculation unit 44.

The calculation unit 44 generates a decoded image by adding the difference information supplied from the inverse quantization inverse frequency conversion unit 43 and the prediction image supplied from the in-screen prediction unit 47 or the motion compensation prediction unit 48. It is supplied to the loop filter unit 45 and the in-screen prediction unit 47. The loop filter unit 45 appropriately filters the decoded image supplied from the calculation unit 44, removes block distortion and improves the image quality, and then outputs the filtered image. .. Further, the decoded image output from the loop filter unit 45 is supplied to the frame memory 46, the decoded image is held, and the decoded image required for the motion compensation prediction unit 48 to generate the predicted image is moved. It is supplied to the compensation prediction unit 48.

The in-screen prediction unit 47 generates a prediction image by performing intra-prediction on the decoded image supplied from the calculation unit 44 using the quantization value supplied from the entropy decoding unit 41. Further, the motion compensation prediction unit 48 generates a prediction image by performing inter-prediction on the decoded image supplied from the frame memory 46 using the quantization value supplied from the entropy decoding unit 41. Then, the prediction image selection unit 49 determines the prediction image generated by the in-screen prediction unit 47 or the prediction generated by the motion compensation prediction unit 48 according to the prediction used when the image to be decoded is encoded. One of the images is selected and supplied to the calculation unit 44.

In the decoding device 12 configured as described above, when the decoding target unit is the in-screen prediction (intra) unit, the P-picture and the B-picture are based on the in-screen prediction mode decoded by the entropy decoding unit 41. The quantization value of the decoding target unit can be determined by any of the first to third determination methods described above. On the other hand, in the decoding device 12, when the decoding target unit is an inter-screen prediction (inter) unit, the determination method described with reference to FIG. 16 or FIG. 17 based on the motion vector decoded by the entropy decoding unit 41. The quantization value of the decoding target unit can be determined by.

<Code processing and decoding processing>
FIG. 19 shows a flowchart illustrating an adaptive quantization value determination process for adaptively determining a quantization value among the coding processes performed in the coding apparatus 11 of FIG. 1.

In step S11, the frequency conversion quantization unit 23 determines whether or not 1 is set in the flag pps_autonomous_qp_flag indicating whether or not the adaptive quantization value determination process is executed for the coded target unit. do.

If the frequency conversion quantization unit 23 determines in step S11 that the flag pps_autonomous_qp_flag is set to 1, the process proceeds to step S12, and the frequency conversion quantization unit 23 has an I-picture as the unit to be encoded. Judge whether or not.

If the frequency conversion quantization unit 23 determines in step S12 that the unit to be encoded is not an I picture, the process proceeds to step S13. In step S13, the frequency conversion quantization unit 23 determines whether to apply the inter-prediction or the intra-prediction to the coded unit.

In step S14, the frequency conversion quantization unit 23 determines whether or not to apply the intra prediction to the coded target unit according to the determination result performed in step S13.

If the frequency conversion quantization unit 23 determines in step S14 that the intra prediction is applied to the coded unit, the process proceeds to step S15. In step S15, the frequency conversion quantization unit 23 determines whether or not 0 or 2 is set in the syntax element cu_qp_delta_coded_type.

If the frequency conversion quantization unit 23 determines in step S15 that either 0 or 2 is set in the syntax element cu_qp_delta_coded_type, the process proceeds to step S16. In step S16, the frequency conversion quantization unit 23 determines whether to apply the planer prediction, the DC prediction, or the direction prediction as the prediction mode in the intra prediction.

In step S17, the frequency conversion quantization unit 23 determines whether or not to apply the planar prediction as the prediction mode in the intra prediction according to the determination result performed in step S16.

If the frequency conversion quantization unit 23 determines in step S17 that the planar prediction is applied as the prediction mode in the intra prediction, the process proceeds to step S18. In step S18, the frequency conversion quantization unit 23 obtains the quantization value at the time of planer prediction according to any one of the first to third determination methods described above, and then the process is terminated.

On the other hand, if the frequency conversion quantization unit 23 determines in step S17 that the planar prediction is not applied as the prediction mode in the intra prediction, the process proceeds to step S19.

In step S19, the frequency conversion quantization unit 23 determines whether or not to apply the DC prediction as the prediction mode in the intra prediction according to the determination result performed in step S16.

If the frequency conversion quantization unit 23 determines in step S19 that DC prediction is applied as the prediction mode in the intra prediction, the process proceeds to step S20. In step S20, the frequency conversion quantization unit 23 obtains the quantization value at the time of DC prediction according to any one of the first to third determination methods described above, and then the process is terminated.

On the other hand, if the frequency conversion quantization unit 23 determines in step S19 that DC prediction is not applied as the prediction mode in the intra prediction, the process proceeds to step S21. In step S21, the frequency conversion quantization unit 23 obtains the quantization value at the time of direction prediction according to any one of the first to third determination methods described above, and then the process is terminated.

On the other hand, if the frequency conversion quantization unit 23 determines in step S14 that the intra prediction is not applied to the coded unit, the process proceeds to step S22. In step S22, the frequency conversion quantization unit 23 determines whether or not 1 or 2 is set in the syntax element cu_qp_delta_coded_type.

If the frequency conversion quantization unit 23 determines in step S22 that the syntax element cu_qp_delta_coded_type is set to 1 or 2, the process proceeds to step S23. In step S23, the frequency conversion quantization unit 23 determines whether or not the coded unit is a P picture.

If the frequency conversion quantization unit 23 determines in step S23 that the unit to be encoded is a P-picture, the process proceeds to step S24. In step S24, the frequency conversion quantization unit 23 obtains the quantization value including the upper left pixel of the reference position according to the above equation (18) as described with reference to FIG. 16, and then the process is terminated. NS.

On the other hand, if the frequency conversion quantization unit 23 determines in step S23 that the unit to be encoded is not a P picture, the process proceeds to step S25. That is, in this case, the unit to be coded is the B picture, and in step S25, the frequency conversion quantization unit 23 determines whether or not it is a bi-prediction that refers to both the I picture and the P picture.

If the frequency conversion quantization unit 23 determines in step S25 that it is not bi-prediction, the process proceeds to step S24, and the quantization value is obtained as described above. In this case, the frequency conversion quantization unit 23 obtains the quantization value including the upper left pixel of the reference position according to the above-mentioned equation (19) or equation (20) as described with reference to FIG. The process is terminated.

On the other hand, if the frequency conversion quantization unit 23 determines in step S25 that the prediction is bi-prediction, the process proceeds to step S26. In step S26, as described with reference to FIG. 17, the frequency conversion quantization unit 23 obtains the average of the quantization values including the upper left pixel of the reference position according to the above equation (21), and then the process is performed. It will be terminated.

On the other hand, if it is determined in step S11 that 1 is not set in the flag pps_autonomous_qp_flag, if it is determined in step S12 that the unit to be encoded is an I picture, in step S15, the syntax element cu_qp_delta_coded_type is 0. If it is determined that neither 1 or 2 is set, or if it is determined in step S22 that neither 1 nor 2 is set in the syntax element cu_qp_delta_coded_type, the process proceeds to step S27.

In step S27, the frequency conversion quantization unit 23 obtains the quantization value by the conventional method, and then the process is terminated.

FIG. 20 shows a flowchart for explaining the adaptive quantization value determination process for adaptively determining the quantization value among the decoding processes performed in the decoding device 12 of FIG.

In step S31, the entropy decoding unit 41 acquires the quantization value by entropy decoding the coded stream and supplies it to the inverse quantization inverse frequency conversion unit 43.

In step S32, the inverse quantization inverse frequency conversion unit 43 determines whether or not 1 is set in the flag pps_autonomous_qp_flag indicating whether or not the adaptive quantization value determination process is executed for the decoding target unit. judge.

If the inverse quantization inverse frequency conversion unit 43 determines in step S32 that the flag pps_autonomous_qp_flag is set to 1, the process proceeds to step S33, and the inverse quantization inverse frequency conversion unit 43 has the decoding target unit I. Determine if it is a picture.

If the inverse quantization inverse frequency conversion unit 43 determines in step S33 that the decoding target unit is not an I picture, the process proceeds to step S34.

In step S34, the inverse quantization inverse frequency conversion unit 43 determines whether or not the decoding target unit is encoded by the intra prediction.

If the inverse quantization inverse frequency conversion unit 43 determines in step S34 that the decoding target unit is encoded by the intra prediction, the process proceeds to step S35. In step S35, the inverse quantization inverse frequency conversion unit 43 determines whether or not 0 or 2 is set in the syntax element cu_qp_delta_coded_type.

If the inverse quantization inverse frequency converter 43 determines in step S35 that the syntax element cu_qp_delta_coded_type is set to either 0 or 2, the process proceeds to step S36.

In step S36, the inverse quantization inverse frequency conversion unit 43 determines whether or not the decoding target unit is encoded by applying the planar prediction.

If the inverse quantization inverse frequency conversion unit 43 determines in step S36 that the decoding target unit is encoded by applying the planar prediction, the process proceeds to step S37. In step S37, the inverse quantization inverse frequency conversion unit 43 obtains the quantization value at the time of planer prediction according to any one of the first to third determination methods described above, and then the process is terminated.

On the other hand, in step S36, when the inverse quantization inverse frequency conversion unit 43 determines that the decoding target unit is not encoded by applying the planar prediction, the process proceeds to step S38.

In step S38, the inverse quantization inverse frequency conversion unit 43 determines whether or not the decoding target unit is encoded by applying DC prediction.

In step S38, when the inverse quantization inverse frequency conversion unit 43 determines that the decoding target unit is encoded by applying the DC prediction, the process proceeds to step S39. In step S39, the inverse quantization inverse frequency conversion unit 43 obtains the quantization value at the time of DC prediction according to any one of the first to third determination methods described above, and then the process is terminated.

On the other hand, in step S38, when the inverse quantization inverse frequency conversion unit 43 determines that the decoding target unit is not encoded by applying the DC prediction, the process proceeds to step S40. In step S40, the inverse quantization inverse frequency conversion unit 43 obtains the quantization value at the time of direction prediction according to any one of the first to third determination methods described above, and then the process is terminated.

On the other hand, in step S34, when the dequantization reverse frequency conversion unit 43 determines that the decoding target unit is not encoded by the intra prediction, the process proceeds to step S41. In step S41, the inverse quantization inverse frequency conversion unit 43 determines whether or not 1 or 2 is set in the syntax element cu_qp_delta_coded_type.

If the inverse quantization inverse frequency conversion unit 43 determines in step S41 that the syntax element cu_qp_delta_coded_type is set to 1 or 2, the process proceeds to step S42. In step S42, the frequency conversion quantization unit 23 determines whether or not the decoding target unit is a P picture.

If the inverse quantization inverse frequency conversion unit 43 determines in step S42 that the decoding target unit is a P picture, the process proceeds to step S43. In step S43, the inverse quantization inverse frequency conversion unit 43 obtains the quantization value including the upper left pixel of the reference position according to the above-mentioned equation (18) as described with reference to FIG. 16, and then the process is performed. It will be terminated.

On the other hand, in step S42, when the dequantization reverse frequency conversion unit 43 determines that the decoding target unit is not a P picture, the process proceeds to step S44. That is, in this case, the unit to be coded is a B picture, and in step S44, the inverse quantization inverse frequency conversion unit 43 determines whether or not it is a bi-prediction that refers to both the I picture and the P picture. ..

If the inverse quantization inverse frequency conversion unit 43 determines in step S44 that it is not bi-prediction, the process proceeds to step S43, and the quantization value is obtained as described above. In this case, the inverse quantization inverse frequency conversion unit 43 obtained the quantization value including the upper left pixel of the reference position according to the above-mentioned equation (19) or equation (20) as described with reference to FIG. After that, the process is finished.

On the other hand, if the inverse quantization inverse frequency conversion unit 43 determines in step S44 that the prediction is bi-prediction, the process proceeds to step S45. In step S45, the inverse quantization inverse frequency conversion unit 43 obtains the average of the quantization values including the upper left pixel of the reference position according to the above-mentioned equation (21) as described with reference to FIG. The process is terminated.

On the other hand, if it is determined in step S32 that 1 is not set in the flag pps_autonomous_qp_flag, if it is determined in step S33 that the unit to be encoded is an I picture, in step S35, the syntax element cu_qp_delta_coded_type is 0. If it is determined that neither 1 or 2 is set, or if it is determined in step S41 that neither 1 nor 2 is set in the syntax element cu_qp_delta_coded_type, the process proceeds to step S46.

In step S46, the inverse quantization inverse frequency conversion unit 43 obtains the quantization value by the conventional method, and then the process is terminated.

When it is desired to use a non-uniform quantization value within the frame by performing the adaptive quantization value determination processing as described above, in the I picture, the quantization parameter is determined on the coding side as before and the code amount is determined. Quantization parameters for each coding unit can be transmitted by controlling and controlling the image quality. On the other hand, in the P picture and the B picture, the quantization value can be autonomously determined on the decoding side without transmitting the quantization parameter for each coding unit.

As a result, the coding device 11 and the decoding device 12 can reduce the amount of coding and can flexibly control the quantization value within the frame.

<Computer configuration example>
Next, the series of processes described above can be performed by hardware or software. When a series of processes is performed by software, the programs constituting the software are installed on a general-purpose computer or the like.

FIG. 21 is a block diagram showing a configuration example of an embodiment of a computer in which a program for executing the above-mentioned series of processes is installed.

The program can be recorded in advance on the hard disk 105 or ROM 103 as a recording medium built in the computer.

Alternatively, the program can be stored (recorded) in the removable recording medium 111 driven by the drive 109. Such a removable recording medium 111 can be provided as so-called package software. Here, examples of the removable recording medium 111 include a flexible disc, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disc, a DVD (Digital Versatile Disc), a magnetic disc, and a semiconductor memory.

The program can be installed on the computer from the removable recording medium 111 as described above, or can be downloaded to the computer via a communication network or a broadcasting network and installed on the built-in hard disk 105. That is, for example, the program transfers wirelessly from a download site to a computer via an artificial satellite for digital satellite broadcasting, or transfers to a computer by wire via a network such as LAN (Local Area Network) or the Internet. be able to.

The computer has a built-in CPU (Central Processing Unit) 102, and the input / output interface 110 is connected to the CPU 102 via the bus 101.

When a command is input by the user by operating the input unit 107 or the like via the input / output interface 110, the CPU 102 executes a program stored in the ROM (Read Only Memory) 103 accordingly. .. Alternatively, the CPU 102 loads the program stored in the hard disk 105 into the RAM (Random Access Memory) 104 and executes it.

As a result, the CPU 102 performs a process according to the above-mentioned flowchart or a process performed according to the above-mentioned block diagram configuration. Then, the CPU 102 outputs the processing result from the output unit 106, transmits it from the communication unit 108, and further records it on the hard disk 105, if necessary, via the input / output interface 110, for example.

The input unit 107 is composed of a keyboard, a mouse, a microphone, and the like. Further, the output unit 106 is composed of an LCD (Liquid Crystal Display), a speaker, or the like.

Here, in the present specification, the processing performed by the computer according to the program does not necessarily have to be performed in chronological order in the order described as the flowchart. That is, the processing performed by the computer according to the program also includes processing executed in parallel or individually (for example, parallel processing or processing by an object).

Further, the program may be processed by one computer (processor) or may be distributed processed by a plurality of computers. Further, the program may be transferred to a distant computer and executed.

Further, in the present specification, the system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether or not all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and a device in which a plurality of modules are housed in one housing are both systems. ..

Further, for example, the configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units). On the contrary, the configurations described above as a plurality of devices (or processing units) may be collectively configured as one device (or processing unit). Further, of course, a configuration other than the above may be added to the configuration of each device (or each processing unit). Further, if the configuration and operation of the entire system are substantially the same, a part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit). ..

Further, for example, the present technology can have a cloud computing configuration in which one function is shared and jointly processed by a plurality of devices via a network.

Further, for example, the above-mentioned program can be executed in any device. In that case, the device may have necessary functions (functional blocks, etc.) so that necessary information can be obtained.

Further, for example, each step described in the above-mentioned flowchart can be executed by one device or can be shared and executed by a plurality of devices. Further, when a plurality of processes are included in one step, the plurality of processes included in the one step can be executed by one device or shared by a plurality of devices. In other words, a plurality of processes included in one step can be executed as processes of a plurality of steps. On the contrary, the processes described as a plurality of steps can be collectively executed as one step.

In the program executed by the computer, the processing of the steps for describing the program may be executed in chronological order in the order described in this specification, or may be executed in parallel or called. It may be executed individually at a necessary timing such as time. That is, as long as there is no contradiction, the processing of each step may be executed in an order different from the above-mentioned order. Further, the processing of the step for writing this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

It should be noted that the present techniques described in the present specification can be independently implemented independently as long as there is no contradiction. Of course, any plurality of the present technologies can be used in combination. For example, some or all of the techniques described in any of the embodiments may be combined with some or all of the techniques described in other embodiments. It is also possible to carry out a part or all of any of the above-mentioned techniques in combination with other techniques not described above.

<Example of configuration combination>
The present technology can also have the following configurations.
(1)
A predicted image used when encoding the coded target unit so that the quantization value of each coded target unit, which is a processing unit for performing the coding process on the image, fluctuates according to the image. A quantization unit that determines the quantization value according to a determination method based on the prediction method, and
When encoding the quantization value determined according to the determination method, for the predetermined image, the quantization parameter relating to the variation of the quantization value for each coding target unit is excluded from the transmission target. A coding device including a coding unit that generates a coded stream.
(2)
When the image to be coded is a first type of image encoded by using only the information in the image, the quantization unit is used for each of the coded units according to the quantization parameter. Determine the quantization value and
The coding device according to (1) above, wherein the coding unit encodes the quantization value determined according to the quantization parameter, and encodes and transmits the quantization parameter.
(3)
The quantization unit is a case where the image to be encoded is a second type of image encoded by referring to the information of another image ahead of the image in time, or. When the image to be encoded is a third type of image encoded by referring to the information of another image located at least one of the front and the rear of the image in time, the determination method. The coding apparatus according to (2) above, which determines the quantization value for each unit to be coded according to the above.
(4)
When the predicted image is predicted by applying the intra prediction, the quantization unit is another unit adjacent to the coded target unit, and is among other units that have already been coded. The coding device according to (3) above, wherein the quantization value of the coded target unit is determined using the set quantization value of the adjacent unit.
(5)
The quantization unit according to the first determination method, which is the determination method.
A left adjacent unit that includes pixels arranged to the left of the upper left corner pixel of the coded unit and is adjacent to the left side of the coded unit.
An upper adjacent unit that includes pixels arranged above and adjacent to the upper left corner pixel of the coded unit and is adjacent to the upper side of the coded unit.
An adjacent unit on the upper right side, which includes a pixel arranged on the upper right side of the pixel in the upper right corner of the unit to be coded and is adjacent to the upper right side of the unit to be coded.
An upper left adjacent unit that includes pixels arranged to the upper left of the upper left corner pixel of the code target unit and is adjacent to the upper left side of the code target unit.
And, the lower left adjacent unit including the pixel arranged next to the lower left corner pixel of the coded target unit and adjacent to the lower left side of the coded target unit is set, and the adjacent units are quantized. The coding device according to (4) above, which determines the quantization value of the coded unit according to the prediction mode defined by HEVC (High Efficiency Video Coding) based on the value.
(6)
When the prediction mode is planar prediction, the quantization unit has a quantization value of the left adjacent unit, a quantization value of the upper adjacent unit, a quantization value of the upper right adjacent unit, and the left lower adjacent unit. The coding apparatus according to (5) above, wherein an average value obtained from the quantization value of the unit is determined as the quantization value of the coding target unit.
(7)
When the prediction mode is DC prediction, the quantization unit calculates the quantization value of the left adjacent unit and the average value obtained from the quantization value of the upper adjacent unit into the quantization target unit. The coding apparatus according to (5) above, which is determined as a value.
(8)
When the prediction mode is directional prediction, the quantization unit determines the quantization value of the coded target unit from the quantized value of the adjacent unit on the prediction direction side in the directional prediction. The coding apparatus according to 5).
(9)
The quantization unit follows the second determination method, which is the determination method.
A plurality of left adjacent units adjacent to the left side of the coded unit,
A plurality of upper adjacent units adjacent to the upper side of the coded unit,
A plurality of upper right adjacent units adjacent to the upper right side of the coded unit and arranged from the upper right adjacent unit toward the right side,
The upper left adjacent unit adjacent to the upper left side of the coded unit,
Then, a plurality of lower left adjacent units adjacent to the lower left side of the coded unit and arranged downward from the left adjacent unit are set, and based on the quantization value of those adjacent units, HEVC is used. The coding apparatus according to (4) above, which determines the quantization value of the coding target unit according to the prediction mode defined in the standard.
(10)
When the prediction mode is planar prediction, the quantization unit includes an average value of the quantization values of the plurality of left adjacent units, an average value of the quantization values of the plurality of upper adjacent units, and a plurality of the upper right adjacent units. The quantized value of the upper right adjacent unit in the vicinity of the coded unit among the units, and the quantum of the lower left adjacent unit in the vicinity of the coded adjacent unit among the plurality of lower left adjacent units. The coding apparatus according to (9) above, wherein the average value obtained from the quantization value is determined as the quantization value of the coding target unit.
(11)
When the prediction mode is DC prediction, the quantization unit obtains an average value obtained from the average value of the quantization values of the left adjacent unit and the average value of the quantization values of the upper adjacent unit. The coding apparatus according to (9) above, which is determined as the quantization value of the unit to be converted.
(12)
When the prediction mode is directional prediction, the quantization unit determines the quantization value of the coded target unit from the quantized value of the adjacent unit on the prediction direction side in the directional prediction. The coding apparatus according to 9).
(13)
The quantization unit according to the third determination method, which is the determination method.
A predetermined number of adjacent units adjacent to the lower left side, left side, upper left side, upper side, and upper right side of the coded target unit are set, and the quantization value of the predetermined number of adjacent units and the adjacent unit The total value of the pixels in contact with the coded target unit among the pixels of the above, which are the pixels that contribute to predicting the predicted image by intra-prediction, and the number of pixels of each adjacent unit of the contributing pixels. The coding apparatus according to (4) above, wherein a value obtained by dividing a value by the total number of contributing pixels is determined as a quantization value of the coding target unit.
(14)
When the predicted image is predicted by applying the inter-prediction and the second type of image is the object of coding processing, the quantization unit is referred to in the first type of image at the time of coding. The quantization value of the coding unit at the reference position according to the motion vector between the images and the syntax element related to the adjustment of the average of the quantization values in the second type of image to be encoded are added. The coding apparatus according to any one of (3) to (13) above, wherein the quantization value is determined.
(15)
In the quantization unit, when the predicted image is predicted by applying inter-prediction and the third type of image is the object of coding processing, and the predicted image is temporally ahead of the third type of image. Or, when referring to the other image on one of the rear sides, the reference position is in accordance with the motion vector between the images in the first type image or the second type image referred to at the time of coding. The quantization value is determined by adding the quantization value of the coding unit and the syntax element related to the adjustment of the average of the quantization value in the third type of image to be encoded (3). The coding apparatus according to any one of (13) to (13).
(16)
In the quantization unit, when the predicted image is predicted by applying inter-prediction and the third type of image is the object of coding processing, and the predicted image is temporally ahead of the third type of image. When referring to the other images both at the rear and behind, they are at reference positions according to the motion vector between the images in the first type image and the second type image referred to at the time of coding. Quantize the average value of the values obtained by adding each of the quantization values of the coding unit and each of the syntax elements involved in adjusting the average of the quantization values in the third type of image to be encoded. The encoding device according to any one of (3) to (13) above, which is determined as a value.
(17)
The quantization unit is a flag that specifies one of a first process of determining the quantization value according to the determination method and a second process of determining the quantization value using the quantization parameter. The coding apparatus according to any one of (1) to (16) above, which switches between the first process and the second process according to the above.
(18)
The quantization unit is
Determining the quantization value according to the determination method only for the coded target unit encoded using the predicted image predicted by applying intra-prediction.
Determining the quantization value according to the determination method only for the coded target unit encoded using the predicted image predicted by applying the inter-prediction.
The coded object unit encoded by using the predicted image predicted by applying intra prediction, and the coded object encoded by using the predicted image predicted by applying inter prediction. The above (17), wherein the first process and the second process are switched according to a syntax element that specifies either to determine the quantization value according to the determination method for both of the units. Coding device.
(19)
An encoding device that performs coding processing on an image
A method of predicting a predicted image used when encoding the coded target unit so that the quantization value of each coded target unit, which is a processing unit for performing the coding process, fluctuates according to the image. To determine the quantization value according to the determination method based on
When encoding the quantization value determined according to the determination method, a coding stream is generated by excluding the quantization parameter related to the variation of the quantization value for each coding target unit from the transmission target. A coding method that includes and.
(20)
To the computer of the coding device that performs the coding process on the image
A method of predicting a predicted image used when encoding the coded target unit so that the quantization value of each coded target unit, which is a processing unit for performing the coding process, fluctuates according to the image. To determine the quantization value according to the determination method based on
When encoding the quantization value determined according to the determination method, a coding stream is generated by excluding the quantization parameter related to the variation of the quantization value for each coding target unit from the transmission target. A program for executing processing including and.
(21)
A decoding unit that decodes a coded stream in which a quantization value determined to adapt to the image and fluctuates for each coded unit that is a processing unit that performs coding processing on the image is encoded.
The dequantization unit that dequantizes the quantization value obtained by decoding by the decoding unit according to a determination method based on a method for predicting a predicted image used when encoding the coded target unit. Decoding device with and.
(22)
The decoding device that performs the coding process on the image
Decoding a coded stream in which a quantization value determined to adapt to the image and fluctuates for each coded target unit, which is a processing unit for performing the coding process, is encoded.
A decoding method including dequantizing the quantization value obtained by decoding according to a determination method based on a method for predicting a predicted image used when encoding the coded target unit.
(23)
To the computer of the decoding device that encodes the image
Decoding a coded stream in which a quantization value determined to adapt to the image and fluctuates for each coded target unit, which is a processing unit for performing the coding process, is encoded.
A process including dequantizing the quantized value obtained by decoding according to a determination method based on a method for predicting a predicted image used when encoding the coded target unit is executed. Program for.

The present embodiment is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present disclosure. Further, the effects described in the present specification are merely examples and are not limited, and other effects may be obtained.

11 Encoding device, 12 Decoding device, 21 Calculation unit, 22 Quantized value memory unit, 23 Frequency conversion quantization unit, 24 Inverse quantization inverse frequency conversion unit, 25 Calculation unit, 26 Loop filter unit, 27 Frame memory, 28 In-screen prediction unit, 29 motion compensation prediction unit, 30 prediction image selection unit, 31 entropy coding unit, 32 coding control unit, 41 entropy decoding unit, 42 quantization value memory unit, 43 inverse quantization inverse frequency conversion unit, 44 Calculation unit, 45 Loop filter unit, 46 Frame memory, 47 In-screen prediction unit, 48 Motion compensation prediction unit, 49 Prediction image selection unit

Claims (20)

A predicted image used when encoding the coded target unit so that the quantization value of each coded target unit, which is a processing unit for performing the coding process on the image, fluctuates according to the image. A quantization unit that determines the quantization value according to a determination method based on the prediction method, and
When encoding the quantization value determined according to the determination method, for the predetermined image, the quantization parameter relating to the variation of the quantization value for each coding target unit is excluded from the transmission target. A coding device including a coding unit that generates a coded stream. When the image to be coded is a first type of image encoded by using only the information in the image, the quantization unit is used for each of the coded units according to the quantization parameter. Determine the quantization value and
The coding device according to claim 1, wherein the coding unit encodes the quantization value determined according to the quantization parameter, and encodes and transmits the quantization parameter. The quantization unit is a case where the image to be encoded is a second type of image encoded by referring to the information of another image ahead of the image in time, or. When the image to be encoded is a third type of image encoded by referring to the information of another image located at least one of the front and the rear of the image in time, the determination method. The coding apparatus according to claim 2, wherein the quantization value is determined for each coding target unit according to the above. When the predicted image is predicted by applying the intra prediction, the quantization unit is another unit adjacent to the coded target unit, and is among other units that have already been coded. The coding device according to claim 3, wherein the quantization value of the coded target unit is determined by using the set quantization value of the adjacent unit. The quantization unit according to the first determination method, which is the determination method.
A left adjacent unit that includes pixels arranged to the left of the upper left corner pixel of the coded unit and is adjacent to the left side of the coded unit.
An upper adjacent unit that includes pixels arranged above and adjacent to the upper left corner pixel of the coded unit and is adjacent to the upper side of the coded unit.
An adjacent unit on the upper right side, which includes a pixel arranged on the upper right side of the pixel in the upper right corner of the unit to be coded and is adjacent to the upper right side of the unit to be coded.
An upper left adjacent unit that includes pixels arranged to the upper left of the upper left corner pixel of the code target unit and is adjacent to the upper left side of the code target unit.
And, the lower left adjacent unit including the pixel arranged next to the lower left corner pixel of the coded target unit and adjacent to the lower left side of the coded target unit is set, and the adjacent units are quantized. The coding device according to claim 4, wherein the quantization value of the coded unit is determined based on the value according to the prediction mode defined by the standard in HEVC (High Efficiency Video Coding). When the prediction mode is planar prediction, the quantization unit has a quantization value of the left adjacent unit, a quantization value of the upper adjacent unit, a quantization value of the upper right adjacent unit, and the lower left adjacent unit. The coding apparatus according to claim 5, wherein the average value obtained from the quantization value of the unit is determined as the quantization value of the coding target unit. When the prediction mode is DC prediction, the quantization unit calculates the quantization value of the left adjacent unit and the average value obtained from the quantization value of the upper adjacent unit into the quantization target unit. The coding apparatus according to claim 5, which is determined as a value. A claim that the quantization unit determines the quantization value of the coded target unit from the quantization value of the adjacent unit on the prediction direction side in the direction prediction when the prediction mode is the direction prediction. The coding apparatus according to 5. The quantization unit follows the second determination method, which is the determination method.
A plurality of left adjacent units adjacent to the left side of the coded unit,
A plurality of upper adjacent units adjacent to the upper side of the coded unit,
A plurality of upper right adjacent units adjacent to the upper right side of the coded unit and arranged from the upper right adjacent unit toward the right side,
The upper left adjacent unit adjacent to the upper left side of the coded unit,
Then, a plurality of lower left adjacent units adjacent to the lower left side of the coded unit and arranged downward from the left adjacent unit are set, and based on the quantization value of those adjacent units, HEVC is used. The coding apparatus according to claim 4, wherein the quantization value of the coding target unit is determined according to the prediction mode defined in the standard. When the prediction mode is planar prediction, the quantization unit includes an average value of the quantization values of the plurality of left adjacent units, an average value of the quantization values of the plurality of upper adjacent units, and a plurality of the upper right adjacent units. The quantized value of the upper right adjacent unit in the vicinity of the coded unit among the units, and the quantum of the lower left adjacent unit in the vicinity of the coded adjacent unit among the plurality of lower left adjacent units. The coding apparatus according to claim 9, wherein the average value obtained from the quantization value is determined as the quantization value of the coding target unit. When the prediction mode is DC prediction, the quantization unit obtains an average value obtained from the average value of the quantization values of the left adjacent unit and the average value of the quantization values of the upper adjacent unit. The coding apparatus according to claim 9, which is determined as the quantization value of the unit to be converted. A claim that the quantization unit determines the quantization value of the coded target unit from the quantization value of the adjacent unit on the prediction direction side in the direction prediction when the prediction mode is the direction prediction. 9. The coding apparatus according to 9. The quantization unit according to the third determination method, which is the determination method.
A predetermined number of adjacent units adjacent to the lower left side, left side, upper left side, upper side, and upper right side of the coded target unit are set, and the quantization value of the predetermined number of adjacent units and the adjacent unit The total value of the pixels in contact with the coded target unit among the pixels of the above, which are the pixels that contribute to predicting the predicted image by intra-prediction, and the number of pixels of each adjacent unit of the contributing pixels. The coding device according to claim 4, wherein the value divided by the total number of the contributing pixels is determined as the quantization value of the coding target unit. When the predicted image is predicted by applying the inter-prediction and the second type of image is the object of coding processing, the quantization unit is referred to in the first type of image at the time of coding. The quantization value of the coding unit at the reference position according to the motion vector between the images and the syntax element related to the adjustment of the average of the quantization values in the second type of image to be coded are added. The coding apparatus according to claim 3, wherein the quantization value is determined. In the quantization unit, when the predicted image is predicted by applying inter-prediction and the third type of image is the object of coding processing, and the predicted image is temporally ahead of the third type of image. Or, when referring to the other image on one of the rear sides, the reference position is in accordance with the motion vector between the images in the first type image or the second type image referred to at the time of coding. The third aspect of claim 3 is to add the quantization value of the coding unit and the syntax element related to the adjustment of the average of the quantization value in the third type of image to be encoded to determine the quantization value. The encoding device described. The quantization unit predicts the predicted image by applying inter-prediction, and when the third type of image is the object of coding processing and is ahead of the third type of image in time. When referring to the other images both at the rear and behind, they are at reference positions according to the motion vector between the images in the first type image and the second type image referred to at the time of encoding. Quantize the average value of the values obtained by adding each of the quantization values of the coding unit and each of the syntax elements involved in adjusting the average of the quantization values in the third type of image to be encoded. The encoding device according to claim 3, which is determined as a value. The quantization unit is a flag that specifies one of a first process of determining the quantization value according to the determination method and a second process of determining the quantization value using the quantization parameter. The coding apparatus according to claim 1, wherein the first process and the second process are switched according to the above. The quantization unit is
Determining the quantization value according to the determination method only for the coded target unit encoded using the predicted image predicted by applying intra-prediction.
Determining the quantization value according to the determination method only for the coded target unit encoded using the predicted image predicted by applying the inter-prediction.
The coded object unit encoded by using the predicted image predicted by applying intra prediction, and the coded object encoded by using the predicted image predicted by applying inter prediction. 13. The reference numeral 17 of claim 17, which switches between the first process and the second process according to a syntax element that specifies either of the units to determine the quantization value according to the determination method. Device. An encoding device that performs coding processing on an image
A method of predicting a predicted image used when encoding the coded target unit so that the quantization value of each coded target unit, which is a processing unit for performing the coding process, fluctuates according to the image. To determine the quantization value according to the determination method based on
When encoding the quantization value determined according to the determination method, a coding stream is generated by excluding the quantization parameter related to the variation of the quantization value for each coding target unit from the transmission target. A coding method that includes and. To the computer of the coding device that performs the coding process on the image
A method of predicting a predicted image used when encoding the coded target unit so that the quantization value of each coded target unit, which is a processing unit for performing the coding process, fluctuates according to the image. To determine the quantization value according to the determination method based on
When encoding the quantization value determined according to the determination method, a coding stream is generated by excluding the quantization parameter related to the variation of the quantization value for each coding target unit from the transmission target. A program for executing processing including and.

Images