JP2013074528A - Image processing device and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable fast encoding and decoding of images.SOLUTION: An arithmetic decoding unit 522 performs an arithmetic decoding process on encoded data obtained by encoding syntax elements, namely, first information, second information, and third information, encoded in that order. The absolute value of a post-quantization transformation coefficient is compared to a threshold "m", and if said absolute value is greater than the threshold "m", the threshold is incremented by one, namely, "m+1" is regarded as new "m", for another comparison. The first information represents the comparison result for each threshold from "m=1" to a prescribed value. The second information corresponds to the amount by which the absolute value exceeds the prescribed value. The third information represents the encoding for the transformation coefficient. A multivalued-data conversion unit 523 converts binary data obtained from the arithmetic decoding process performed by the arithmetic decoding unit 522 into multivalued data and generates a pre-encoding transformation coefficient.

Description

  This technique relates to an image processing apparatus and an image processing method. Specifically, the image encoding process and the decoding process can be efficiently performed.

  Conventionally, H.M. In the H.264 / AVC (Advanced Video Coding) image coding system, CABAC (Context-based Adaptive Binary Arithmetic Coding) and CAVLC (Context-based Adaptive Variable Length Coding) are defined as entropy coding. Among these, CABAC is a binary arithmetic coding system that performs adaptive coding according to a context.

  The arithmetic coding method repeats the process of dividing a numerical section according to the occurrence probability for each symbol, and generates encoded data by a binary expression of the occurrence probability of the divided numerical section. In addition, the arithmetic coding method needs to sequentially update the symbol occurrence probability table for each coding process. In arithmetic decoding, processing reverse to that of arithmetic coding is performed.

  H. As a next-generation image coding method following H.264 / AVC (Advanced Video Coding), standardization of HEVC (High Efficiency Video Coding) is in progress. The basic encoding technique used by HEVC is H.264. It is the same as H.264 / AVC. However, in the entropy coding method, H.264 is used. A method different from CAVLC and CABAC in H.264 / AVC entropy coding has been proposed (see Non-Patent Document 1 below).

"Working Draft 4 of High-Efficiency Video Coding" Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11 6th Meeting: Torino, IT, 14-22 July, 2011 . (JCTVC-F803_d0)

  By the way, in CABAC, a coding part that performs coding with reference to a context (Context) and a coding part that performs coding without referring to the context are mixed. The encoding part that refers to the context is complicated in processing because it refers to the context. Further, the process becomes more complicated due to the mixing of the encoded part that refers to the context and the encoded part that does not refer to the context, and it may be difficult to efficiently perform the encoding process and the decoding process.

  The present technology has been made in view of such a situation, and an object thereof is to provide an image processing apparatus and an image processing method capable of efficiently performing CABAC encoding processing and decoding processing.

  The first aspect of this technique is to compare the absolute value of the quantized transform coefficient with a threshold value “m” and compare the threshold value as “m = m + 1” when the absolute value is greater than the threshold value “m”. First information indicating a comparison result until the threshold value reaches a predetermined value from “m = 1”, second information according to an excess of the absolute value with respect to the predetermined value, and a sign of the conversion coefficient An arithmetic decoding processing unit that performs arithmetic decoding processing on encoded data that is encoded in the order of the first information, the second information, and the third information, with each of the third information indicating An image processing apparatus includes a multi-value quantization unit that multi-values binarized data obtained by performing arithmetic decoding processing in the arithmetic decoding processing unit and generates the transform coefficient before encoding.

  In this technique, for example, the absolute value of a transform coefficient in units of sub-blocks after quantization is compared with a threshold value “m”, and if the absolute value is larger than the threshold value “m”, the threshold value is set to “m = m + 1”. First information indicating a comparison result from the threshold value “m = 1” to a predetermined value, for example, “m = 2”, second information corresponding to an excess of the absolute value with respect to the predetermined value, and conversion Arithmetic decoding processing is performed on the encoded data obtained by encoding the first information, the second information, and the third information in the order of the third information indicating the sign of the coefficient as a syntax element. Done in The sub-block unit is a block unit obtained by dividing a transform unit (TU: transform_unit), which is a unit of orthogonal transformation, for each predetermined number of coefficients, for example, 16 coefficients.

  In the arithmetic decoding process, the decoding process of the first information is performed in the mode that refers to the context representing the probability state and the dominant probability symbol, and the decoding process of the second information and the third information is performed in the mode that does not refer to the context. Done. Also, the binarized data obtained by performing the arithmetic decoding process is multi-valued, and transform coefficients before encoding are generated.

  In the second aspect of this technique, the absolute value of the transform coefficient after quantization is compared with the threshold “m”, and when the absolute value is greater than the threshold “m”, the threshold is set to “m = m + 1”. First information indicating a comparison result until the threshold value reaches a predetermined value from “m = 1”, second information according to an excess of the absolute value with respect to the predetermined value, and a sign of the conversion coefficient And performing the arithmetic decoding process on the encoded data encoded in the order of the first information, the second information, and the third information, using the third information indicating Performing the multi-value conversion of the binarized data obtained by performing the processing, and generating the transform coefficient before encoding.

  The third aspect of this technique compares the absolute value of the quantized transform coefficient with a threshold “m” and compares the threshold value with “m = m + 1” when the absolute value is greater than the threshold “m”. First information indicating a comparison result until the threshold value reaches a predetermined value from “m = 1”, second information according to an excess of the absolute value with respect to the predetermined value, and a sign of the conversion coefficient The binarization unit that binarizes in order of the first information, the second information, and the third information, with the third information indicating each as a syntax element, and the binary obtained by the binarization unit An image processing apparatus includes an arithmetic encoding processing unit that performs arithmetic encoding processing of encoded data and generates encoded data.

  In this technique, for example, the absolute value of a transform coefficient in units of sub-blocks after quantization is compared with a threshold value “m”, and if the absolute value is larger than the threshold value “m”, the threshold value is set to “m = m + 1”. First information indicating a comparison result from the threshold value “m = 1” to a predetermined value, for example, “m = 2”, second information corresponding to an excess of the absolute value with respect to the predetermined value, and conversion The third information indicating the sign of the coefficient is used as a syntax element, and binarized in the order of the first information, the second information, and the third information. An arithmetic encoding process is performed on the binarized data by an arithmetic encoding processing unit to generate encoded data. In the arithmetic encoding process, the first information is encoded in a mode that refers to a context that represents a probability state and a dominant probability symbol, and the second information and the third information are encoded in a mode that does not refer to the context. Processing is performed.

  The fourth aspect of this technique is to compare the absolute value of the quantized transform coefficient with a threshold value “m” and compare the threshold value as “m = m + 1” when the absolute value is greater than the threshold value “m”. First information indicating a comparison result until the threshold value reaches a predetermined value from “m = 1”, second information according to an excess of the absolute value with respect to the predetermined value, and a sign of the conversion coefficient Each of the third information indicating a syntax element, and a binarization process in the order of the first information, the second information, and the third information, and arithmetic of the binarized data obtained by the binarization An image processing method including a step of performing an encoding process and generating encoded data.

  According to this technique, the absolute value of the transform coefficient after quantization and the threshold “m” are compared, and when the absolute value is larger than the threshold “m”, the threshold is set to “m = m + 1”. First information indicating a comparison result from “m = 1” to a predetermined value, second information corresponding to an excess of the absolute value with respect to the predetermined value, and third information indicating the sign of the transform coefficient, respectively As syntax elements, generation and decoding of encoded data encoded in the order of the first information, the second information, and the third information are performed. By setting the order of such syntax elements, transform coefficients can be determined in order at the time of decoding. Also, it is possible to perform information encoding processing and decoding processing together. Therefore, CABAC encoding processing and decoding processing can be performed efficiently.

It is the figure which illustrated the structure of the image coding apparatus. It is a flowchart which shows operation | movement of an image coding apparatus. It is the figure which illustrated the structure of the CABAC encoding part. It is a figure for demonstrating binarization operation | movement. It is a figure which shows the specific example of binarization operation | movement. It is a figure which shows the specific example of the binarization operation | movement which enabled it to perform a process efficiently. It is the figure which compared the operation | movement at the time of decoding. It is a flowchart which shows operation | movement of a CABAC encoding part. It is the figure which illustrated the syntax of CABAC. It is the figure which illustrated the structure of the image decoding apparatus. It is a flowchart which shows operation | movement of an image decoding apparatus. It is the figure which illustrated the structure of the CABAC decoding part. It is a flowchart which shows operation | movement of a CABAC decoding part. It is the figure which illustrated schematic structure of the television apparatus. It is the figure which illustrated schematic structure of the mobile phone. It is the figure which illustrated schematic structure of the recording / reproducing apparatus. It is the figure which illustrated schematic structure of the imaging device.

Hereinafter, embodiments for carrying out the present technology will be described. The description will be given in the following order.
1. 1. Configuration of image encoding device 2. Operation of image encoding device 3. Configuration of CABAC encoding unit Operation of CABAC encoding unit 4-1. About CABAC encoding / decoding processing 4-2. 3. CABAC encoding processing operation 4. Operation of lossless encoding unit 5. Configuration of image decoding device 6. Operation of image decoding device Configuration of CABAC decoding unit 8. 8. Operation of CABAC decoding unit In case of software processing 10. When applied to electronic equipment

<1. Configuration of Image Encoding Device>
FIG. 1 illustrates the configuration of an image encoding device. The image encoding device 10 includes an analog / digital conversion unit (A / D conversion unit) 11, a screen rearrangement buffer 12, a calculation unit 13, an orthogonal transformation unit 14, a quantization unit 15, a lossless encoding unit 16, and a storage buffer 17. The rate control unit 18 is provided. Furthermore, the image encoding device 10 includes an inverse quantization unit 21, an inverse orthogonal transform unit 22, a calculation unit 23, a deblocking filter 24, a frame memory 25, an intra prediction unit 31, a motion prediction / compensation unit 32, a predicted image / optimum A mode selection unit 33 is provided.

  The A / D converter 11 converts an analog image signal into digital image data and outputs the digital image data to the screen rearrangement buffer 12. The screen rearrangement buffer 12 rearranges the frames of the image data output from the A / D conversion unit 11. The screen rearrangement buffer 12 rearranges the frames according to the GOP (Group of Pictures) structure related to the encoding process, and outputs the rearranged image data to the calculation unit 13, the rate control unit 18, and the intra prediction unit 31. The result is output to the motion prediction / compensation unit 32.

  The calculation unit 13 is supplied with the image data output from the screen rearrangement buffer 12 and the predicted image data selected by the predicted image / optimum mode selection unit 33 described later. The calculation unit 13 is a difference image image data (prediction error) that is a difference between the image data of the input image output from the screen rearrangement buffer 12 and the image data of the prediction image supplied from the prediction image / optimum mode selection unit 33. Data) is output to the orthogonal transform unit 14.

  The orthogonal transform unit 14 performs orthogonal transform processing such as discrete cosine transform (DCT) and Karoonen-Labe transform on the prediction error data output from the computation unit 13. The orthogonal transform unit 14 outputs transform coefficient data obtained by performing the orthogonal transform process to the quantization unit 15.

  The quantization unit 15 is supplied with transform coefficient data output from the orthogonal transform unit 14 and a quantization parameter (quantization scale) from a rate control unit 18 described later. The quantization unit 15 quantizes the transform coefficient data and outputs the quantized data to the lossless encoding unit 16 and the inverse quantization unit 21. Further, the quantization unit 15 changes the bit rate of the quantized data according to the quantization parameter set by the rate control unit 18.

  The lossless encoding unit 16 is supplied with quantized data of transform coefficients from the quantization unit 15, prediction mode information from the intra prediction unit 31, prediction mode information and motion vector information from the motion prediction / compensation unit 32, and the like. Also, information indicating whether the optimal mode is intra prediction or inter prediction is supplied from the predicted image / optimum mode selection unit 33. Note that the prediction mode information includes a prediction mode, block size information, and the like according to intra prediction or inter prediction.

  The lossless encoding unit 16 performs a lossless encoding process on the quantized data of the transform coefficient by, for example, CAVLC or CABAC, and generates an encoded stream and outputs the encoded stream to the accumulation buffer 17. Moreover, the lossless encoding part 16 performs the lossless encoding of the prediction mode information supplied from the intra prediction part 31, when the optimal mode is intra prediction. Further, when the optimum mode is inter prediction, the lossless encoding unit 16 performs lossless encoding of prediction mode information, motion vector information, and the like supplied from the motion prediction / compensation unit 32. Furthermore, the lossless encoding part 16 performs the lossless encoding of the information regarding a quantization parameter.

  The accumulation buffer 17 accumulates the encoded stream from the lossless encoding unit 16. The accumulation buffer 17 outputs the accumulated encoded stream at a transmission rate corresponding to the transmission path.

  The rate control unit 18 monitors the free capacity of the storage buffer 17, and when the free capacity is low, the bit rate of the quantized data decreases, and when the free capacity is sufficiently large, the bit of the quantized data Set the quantization parameter to increase the rate. The rate control unit 18 outputs the set quantization parameter to the quantization unit 15.

  The inverse quantization unit 21 performs an inverse quantization process on the quantized data supplied from the quantization unit 15. The inverse quantization unit 21 outputs transform coefficient data obtained by performing the inverse quantization process to the inverse orthogonal transform unit 22.

  The inverse orthogonal transform unit 22 performs an inverse orthogonal transform process on the transform coefficient data supplied from the inverse quantization unit 21 and outputs the obtained data to the calculation unit 23.

  The computing unit 23 adds the data supplied from the inverse orthogonal transform unit 22 and the predicted image data supplied from the predicted image / optimum mode selection unit 33 to generate image data of the local decoded image, and the deblocking filter 24. And output to the intra prediction unit 31. In addition, the local decoded image produced | generated by the calculating part 23 is used as a reference image in intra prediction or inter prediction.

  The deblocking filter 24 performs a filter process for reducing block distortion that occurs when an image is encoded. The deblocking filter 24 performs a filter process for removing block distortion from the image data supplied from the calculation unit 23, and stores the image data after the filter process in the frame memory 25.

  The intra prediction unit 31 uses the input image data of the encoding target image supplied from the screen rearrangement buffer 12 and the image data (reference image data) supplied from the calculation unit 23 to perform prediction in a candidate intra prediction mode. To determine the optimal intra prediction mode. For example, the intra prediction unit 31 calculates the cost function value in each intra prediction mode, and sets the intra prediction mode in which the coding efficiency is the best based on the calculated cost function value as the optimal intra prediction mode. The intra prediction unit 31 outputs the predicted image data generated in the optimal intra prediction mode and the cost function value in the optimal intra prediction mode to the predicted image / optimum mode selection unit 33. Further, the intra prediction unit 31 outputs prediction mode information indicating the optimal intra prediction mode to the lossless encoding unit 16.

  The motion prediction / compensation unit 32 uses the input image data of the encoding target image supplied from the screen rearrangement buffer 12 and the reference image data read from the frame memory 25 to perform prediction in a candidate inter prediction mode, Determine the optimal inter prediction mode. For example, the motion prediction / compensation unit 32 calculates the cost function value in each inter prediction mode, and sets the inter prediction mode in which the coding efficiency is the best based on the calculated cost function value as the optimal inter prediction mode. The motion prediction / compensation unit 32 outputs the predicted image data generated in the optimal inter prediction mode and the cost function value in the optimal inter prediction mode to the predicted image / optimum mode selection unit 33. Further, the motion prediction / compensation unit 32 outputs prediction mode information and motion vector information regarding the optimal inter prediction mode to the lossless encoding unit 16.

  The predicted image / optimum mode selection unit 33 compares the cost function value supplied from the intra prediction unit 31 with the cost function value supplied from the motion prediction / compensation unit 32, and encodes the one having the smaller cost function value. Select the optimal mode with the best efficiency. Further, the predicted image / optimum mode selection unit 33 outputs the predicted image data generated in the optimal mode to the calculation unit 13 and the calculation unit 23. Further, the predicted image / optimum mode selection unit 33 outputs information indicating whether the optimal mode is the intra prediction mode or the inter prediction mode to the lossless encoding unit 16. Note that the predicted image / optimum mode selection unit 33 switches between intra prediction and inter prediction in units of slices.

<2. Operation of Image Encoding Device>
Next, the operation of the image coding apparatus will be described using the flowchart shown in FIG. In step ST11, the A / D converter 11 performs A / D conversion on the input image signal.

  In step ST12, the screen rearrangement buffer 12 performs image rearrangement. The screen rearrangement buffer 12 stores the image data supplied from the A / D conversion unit 11, and rearranges from the display order of each picture to the encoding order.

  In step ST13, the calculation unit 13 generates prediction error data. The calculation unit 13 calculates a difference between the image data of the input image rearranged in step ST12 and the image data of the predicted image selected by the predicted image / optimum mode selection unit 33, and the image data of the difference image, that is, the prediction Generate error data. The prediction error data has a smaller data amount than the original image data. Therefore, the data amount can be encoded as compared with the case where the image is encoded as it is.

  In step ST14, the orthogonal transform unit 14 performs an orthogonal transform process. The orthogonal transform unit 14 performs orthogonal transform on the prediction error data supplied from the calculation unit 13. Specifically, orthogonal transformation such as discrete cosine transformation and Karhunen-Loeve transformation is performed on the prediction error data, and transformation coefficient data is output.

  In step ST15, the quantization unit 15 performs a quantization process. The quantization unit 15 quantizes the transform coefficient data. At the time of quantization, rate control is performed as described in the process of step ST25 described later.

  In step ST16, the inverse quantization unit 21 performs an inverse quantization process. The inverse quantization unit 21 inversely quantizes the transform coefficient data quantized by the quantization unit 15 with characteristics corresponding to the characteristics of the quantization unit 15.

  In step ST17, the inverse orthogonal transform unit 22 performs an inverse orthogonal transform process. The inverse orthogonal transform unit 22 performs inverse orthogonal transform on the transform coefficient data inversely quantized by the inverse quantization unit 21 with characteristics corresponding to the characteristics of the orthogonal transform unit 14.

  In step ST18, the calculation unit 23 generates reference image data. The calculation unit 23 adds the predicted image data supplied from the predicted image / optimum mode selection unit 33 and the data after inverse orthogonal transformation of the position corresponding to the predicted image to generate image data (reference image) of the local decoded image. Data).

  In step ST19, the deblocking filter 24 performs a deblocking filter process. The deblocking filter 24 filters the decoded image data output from the calculation unit 23 to remove block distortion.

  In step ST20, the frame memory 25 stores the decoded image data. The frame memory 25 stores decoded image data that has been filtered by the deblocking filter.

  In step ST21, the intra prediction unit 31 and the motion prediction / compensation unit 32 each perform a prediction process. That is, the intra prediction unit 31 performs intra prediction processing in the intra prediction mode, and the motion prediction / compensation unit 32 performs motion prediction / compensation processing in the inter prediction mode. By the prediction process, the prediction process in the candidate prediction mode is performed, and the cost function value in the candidate prediction mode is calculated. Then, based on the calculated cost function value, the optimal intra prediction mode and the optimal inter prediction mode are selected, and the prediction image generated in the selected prediction mode and its cost function and prediction mode information are predicted image / optimum mode. It is supplied to the selector 33.

  In step ST22, the predicted image / optimum mode selection unit 33 selects predicted image data. The predicted image / optimum mode selection unit 33 determines the optimal mode with the best coding efficiency based on the cost function values output from the intra prediction unit 31 and the motion prediction / compensation unit 32. Further, the predicted image / optimum mode selection unit 33 outputs the predicted image data of the determined optimal mode to the calculation unit 13 and the calculation unit 23. As described above, the predicted image data is used for the calculations in steps ST13 and ST18.

  In step ST23, the lossless encoding unit 16 performs a lossless encoding process. The lossless encoding unit 16 performs lossless encoding on the quantized data of the transform coefficient output from the quantization unit 15. That is, lossless encoding such as CAVLC or CABAC is performed on the quantized data to encode the data. In addition, the lossless encoding unit 16 performs lossless encoding of prediction mode information or the like corresponding to the prediction image data selected in step ST25, and adds the prediction mode to the encoded stream generated by lossless encoding of the quantized data. Lossless encoded data such as information is included.

  In step ST24, the accumulation buffer 17 performs accumulation processing. The accumulation buffer 17 accumulates the encoded stream output from the lossless encoding unit 16. The encoded stream stored in the storage buffer 17 is appropriately read and transmitted to the decoding side via the transmission path.

  In step ST25, the rate control unit 18 performs rate control. When accumulating the encoded stream in the accumulation buffer 17, the rate control unit 18 sets the quantization parameter QP so that overflow or underflow does not occur in the accumulation buffer 17 and outputs the quantization parameter QP to the quantization unit 15. Control the rate of activation.

<3. Configuration of CABAC Encoding Unit>
FIG. 3 illustrates the configuration of a CABAC encoding unit that performs CABAC processing in the lossless encoding unit 16. The CABAC encoding unit 160 includes a binarizing unit 161, a context calculating unit 162, and an arithmetic encoding processing unit 163.

  The binarization unit 161 performs binarization processing on the quantized transform coefficient, motion vector information, prediction mode information, and other data, and generates binarized data having a syntax defined by the standard. The result is output to the arithmetic coding processing unit 163.

  The context calculation unit 162 generates a context index based on the syntax. The context index is information for selecting a context, and in the context, information on a dominant symbol having a high occurrence probability and the occurrence probability of the dominant symbol is indicated.

  The context calculation unit 162 selects a context corresponding to the context index and outputs the selected context to the arithmetic coding processing unit 163. In addition, the context calculation unit 162 updates the occurrence probability based on the arithmetic coding result of the arithmetic coding processing unit 163.

  The arithmetic coding processing unit 163 repeats the process of dividing the numerical interval according to the occurrence probability based on the binarized data and the context, and uses the binary expression of the occurrence probability of the divided numerical interval as a result of the arithmetic coding. Some encoded data is generated and output to the accumulation buffer 17.

  The arithmetic encoding processing unit 163 is provided with an encoding decision process and an encoding bypass process in the arithmetic encoding process. In the arithmetic coding process, an encode / terminate process is provided, but a description thereof is omitted. Encode decision processing is an encoding mode that references contexts that represent probability states and dominant probability symbols. The encoding bypass process is an encoding mode that does not refer to a context. The arithmetic coding processing unit 163 performs coding by selecting a coding mode according to the syntax.

<4. Operation of CABAC Encoding Unit>
<4-1. About CABAC encoding / decoding processing>
The CABAC encoding unit 160 obtains encoded data by performing arithmetic encoding processing after performing binarization for converting a symbol to be encoded into an intermediate format called bin. For example, the CABAC encoding unit 160 compares the absolute value of the quantized data of the transform coefficient (hereinafter simply referred to as transform coefficient information) with the threshold “m”. Here, when the absolute value is larger than the threshold value “m”, the comparison is performed with the threshold value “m = m + 1”, and the comparison is performed from “m = 1” to the predetermined value. The first information indicating the comparison result thus obtained, the second information indicating the excess of the absolute value with respect to the predetermined value, and the third information indicating the sign of the transform coefficient are used as syntax elements, respectively. . Binarized data of this syntax element is generated and arithmetic coding processing is performed to generate encoded data.

  In CABAC encoding of transform coefficient information, in HEVC (High Efficiency Video Coding), a transform unit (TU: transform_unit), which is a unit of orthogonal transform, is divided into sub-blocks for every predetermined number of coefficients, for example, every 16 coefficients, The transform coefficient is encoded on a sub-block basis.

  Further, in CABAC encoding of transform coefficients in HEVC, it is considered that a predetermined value is “m = 2” and that the syntax elements include significant_coeff_flag, coeff_abs_level_greater1_flag, coeff_abs_level_greater2_flag, coeff_abs_level_minus3, coeff_sign_flag, and the like.

  significant_coeff_flag is a syntax element of a flag indicating whether or not the conversion coefficient is “0”. significant_coeff_flag is significant_coeff_flag = 0 when the conversion coefficient is “0”, and significant_coeff_flag = 1 when the conversion coefficient is not “0”.

  coeff_abs_level_greater1_flag is a syntax element of a flag indicating whether or not the absolute value of the transform coefficient is larger than the threshold “1”. coeff_abs_level_greater1_flag is coeff_abs_level_greater1_flag = 1 when the conversion coefficient is larger than “1”, and coeff_abs_level_greater1_flag = 0 when the conversion coefficient is “1” or less.

  coeff_abs_level_greater2_flag is a syntax element of a flag indicating whether or not the absolute value of the transform coefficient is larger than the threshold “2”. coeff_abs_level_greater2_flag is coeff_abs_level_greater2_flag = 1 when the conversion coefficient is larger than “2”, and coeff_abs_level_greater2_flag = 0 when the conversion coefficient is “2” or less.

  coeff_abs_level_minus3 is a syntax element indicating a value obtained by subtracting “3” from the absolute value of the transform coefficient, and is information according to an excess of the absolute value with respect to a predetermined value. When the coeff_abs_level_greater2_flag is “1”, the absolute value of the conversion coefficient is a value larger than the threshold “2”, that is, an integer value of “3” or more. Therefore, a value obtained by subtracting “3” from the absolute value of the conversion coefficient is used as a syntax element. The coeff_abs_level_minus3 is binarized by a method using a Golomb-Rice code, for example, in exponential Golomb coding.

  coeff_sign_flag is a syntax element indicating the sign of the transform coefficient. coeff_sign_flag is coeff_sign_flag = 0 when the conversion coefficient is a positive value, and coeff_sign_flag = 1 when the conversion coefficient is a negative value.

  Significant_coeff_flag, coeff_abs_level_greater1_flag, and coeff_abs_level_greater2_flag are subjected to encoding / decision processing. Further, coeff_abs_level_minus3 and coeff_sign_flag are subjected to encoding / bypass processing. The encoding bypass process is a process that simplifies the sign operation by fixing the occurrence probability to “0.5”.

  Although explanation is omitted, other syntax elements are provided in the CABAC syntax.

  FIG. 4 shows the order of CABAC syntax elements in the encoded stream. In FIG. 4, “b0” indicates encoded data of coeff_abs_level_greater1_flag. Further, “b1” indicates encoded data of coeff_abs_level_greater2_flag, “Lm3” indicates encoded data of coeff_abs_level_minus3, and “s” indicates encoded data of coeff_sign_flag. Although not shown, the encoded data of significant_coeff_flag is provided before the encoded data of coeff_abs_level_greater1_flag.

  In HM3.0 which is a test model of HEVC, the order of CABAC syntax elements is the order shown in FIG. In the case of HM3.0, a syntax element on which encoding / decision processing is performed and a syntax element on which encoding / bypass processing is performed are mixed.

  When performing the encoding / decision process, it is necessary to update the probability of occurrence based on the arithmetic encoding result, and the process is complicated. In addition, since the occurrence probability is fixed to “0.5” in the encoding / bypass processing, the processing is simple and, for example, a plurality of data encoding processing and decoding processing can be performed in one clock cycle. Therefore, if a syntax element for which encoding / decision processing is performed and a syntax element for performing encoding / bypass processing are mixed, the efficiency and speed of the encoding / decoding processing cannot be improved. . For this reason, "" Parallel Context Processing of Coefficient Level "Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11 6th Meeting: Torino, IT, 14-22 July, In 2011 (JCTVC-F130), as shown in FIG. 4B, it is proposed to separate a syntax element that is subjected to encoding / decision processing from a syntax element that is subjected to encoding / bypass processing. Yes.

  Incidentally, when decoding the encoded stream in the order shown in FIG. 4B, for example, in the decoding of the syntax element of coeff_abs_level_minus3, it is determined whether the bin obtained by decoding corresponds to a code break bit by bit. The Therefore, it is not possible to continuously process syntax elements for which encoding bypass processing has been performed. For this reason, in HM4.0 which is a test model of HEVC, as shown in (C) of FIG. 4, it is proposed that the syntax elements of coeff_sign_flag are put together, and then the syntax elements of coeff_abs_level_minus3 are put together. In such an order, the number of coeff_sign_flags can be determined from the preceding significant_coeff_flag, so that it is possible to continuously decode coeff_sign_flag without determining code breaks.

  With respect to these orders, in the present technology, as shown in FIG. 4D, the encoded data of coeff_abs_level_greater1_flag is followed by the encoded data of coeff_abs_level_greater2_flag. Thereafter, the encoded data of coeff_abs_level_minus3 and then the encoded data of coeff_sign_flag are arranged in this order, thereby further improving efficiency and speed compared to (A) to (C) of FIG.

  FIG. 5 illustrates the binarization processing operation of HM4.0. FIG. 5A illustrates the value of the conversion coefficient. FIG. 5B shows coeff_abs_level_greater1_flag. coeff_abs_level_greater1_flag is set to coeff_abs_level_greater1_flag = 1 when the absolute value of the transform coefficient is larger than the threshold “1”, and coeff_abs_level_greater1_flag = 0 when the absolute value of the conversion coefficient is equal to or smaller than “1”.

  FIG. 5C illustrates coeff_abs_level_greater2_flag. coeff_abs_level_greater2_flag is set to coeff_abs_level_greater2_flag = 0 when the absolute value of coeff_abs_level_greater1_flag = 1 is coeff_abs_level_greater2_flag = 1 when the absolute value is larger than the threshold “2” and equal to or less than “2”.

  (D) of FIG. 5 has shown coeff_sign_flag. coeff_sign_flag is set to coeff_sign_flag = 0 when the conversion coefficient is a positive value, and coeff_sign_flag = 1 when the conversion coefficient is a negative value.

  FIG. 5E shows coeff_abs_level_minus3. coeff_abs_level_minus3 is a value obtained by subtracting “3” from the absolute value for the transform coefficient with coeff_abs_level_greater2_flag = 1.

  FIG. 6 illustrates the binarization processing operation of the present technology. FIG. 6A illustrates the value of the conversion coefficient. FIG. 6B shows coeff_abs_level_greater1_flag. coeff_abs_level_greater1_flag is set to coeff_abs_level_greater1_flag = 1 when the absolute value of the transform coefficient is larger than the threshold “1”, and coeff_abs_level_greater1_flag = 0 when the absolute value of the conversion coefficient is equal to or smaller than “1”.

  (C) of FIG. 6 has shown coeff_abs_level_greater2_flag. The coeff_abs_level_greater2_flag is set to coeff_abs_level_greater1_flag = 0 when the absolute value is greater than the threshold “2” and coeff_abs_level_greater2_flag = 1 when the absolute value is greater than the threshold “2”.

  (D) of FIG. 6 has shown coeff_sign_flag. coeff_sign_flag is set to coeff_sign_flag = 0 when the conversion coefficient is a positive value, and coeff_sign_flag = 1 when the conversion coefficient is a negative value.

  (E) of FIG. 6 has shown coeff_abs_level_minus3. coeff_abs_level_minus3 is a value obtained by subtracting 3 from the absolute value for the transform coefficient with coeff_abs_level_greater2_flag = 1.

  FIG. 7 is a diagram comparing the decoding operation of HM4.0 and the present technology. FIG. 7A illustrates the encoded stream shown in the binarization processing operation shown in FIG. Moreover, (B) of FIG. 7 has shown the decoding result of the encoding stream shown to (A) of FIG. In the order shown in FIG. 7A, coeff_abs_level_greater1_flag is decoded in sub-block 1, and then coeff_abs_level_greater2_flag is decoded. At this time, the conversion coefficient is not fixed. Next, when coeff_sign_flag is decoded, conversion coefficients of ± 1 and ± 2 are determined. In other words, in the case of FIG. 5, the second conversion coefficient “2” to the tenth conversion coefficient “1” is determined. However, the first transform coefficient “6” is not fixed unless the decoding of coeff_abs_level_minus3 is completed. Further, since the first transform coefficient has not been determined, in order to output the transform coefficients after decoding in the correct order, the transform coefficient “2” determined before the transform coefficient “6” is determined. , -1, 1... 1 ”is stored in a buffer, for example. Further, it is necessary to output the stored conversion coefficients in the correct order after the conversion coefficient “6”.

  In this way, since the determined transform coefficient is temporarily stored in the buffer or the like, if the capacity of the buffer or the like is small, the transmission of the subblock 1 is completed, and the decoding of the subblock 2 is advanced until the buffer or the like is empty. I can't. Therefore, the efficiency of the decoding process is reduced. In FIG. 7B, a waiting time is provided between the decoding of coeff_abs_level_greater1_flag and the decoding of coeff_sign_flag, and the decoding of coeff_sign_flag is completed at the end of transmission of sub-block 1, and the determined transform coefficient is stored in the buffer. The case of memorizing is illustrated.

  FIG. 7C illustrates an encoded stream indicated by the binarization processing operation shown in FIG. Moreover, (D) of FIG. 7 has shown the decoding result of the encoding stream shown to (C) of FIG. In the order shown in FIG. 7C, coeff_abs_level_greater1_flag is decoded in sub-block 1, and then coeff_abs_level_greater2_flag is decoded. At this time, the conversion coefficient is not fixed. Thereafter, when coeff_abs_level_minus3 is decoded, the absolute value of each transform coefficient is determined. Next, when coeff_sign_flag is decoded, the transform coefficients are determined in the order of output, and the transform coefficients are sequentially output in the correct order as shown in FIG. 7D without waiting for the determination of other transform coefficients. it can.

  In this way, if the encoded streams are in the order shown in FIG. 7C, it is not necessary to wait for decoding of the next sub-block until the buffer becomes free as shown in FIG. 7B. In addition, the syntax element for which encoding / decision processing is performed and the syntax element for which encoding bypass processing is performed are separated, and coeff_sign_flag is connected, so that processing can be performed efficiently and at high speed. .

  Furthermore, in a processing system that decodes a plurality of coeff_sign_flags in a single cycle in order to process coeff_sign_flag in parallel, if the transform coefficient output interface is parallelized, it can be output immediately when the transform coefficient is determined, and the sub-blocks are continuously connected And can be decrypted.

<4-2. CABAC encoding processing operation>
Next, as illustrated in (D) of FIG. 4 and (C) of FIG. 7, an operation of the CABAC encoding unit 160 that generates encoded data in the order of coeff_abs_level_greater1_flag, coeff_abs_level_greater2_flag, coeff_abs_level_minus3, and coeff_sign_flag will be described.

  FIG. 8 is a flowchart showing the encoding processing operation in the CABAC encoding unit 160. In step ST31, the CABAC encoding unit 160 encodes coeff_abs_level_greater1_flag for the number of coefficients. The CABAC encoding unit 160 compares the absolute value of each transform coefficient in the sub-block with the threshold “1”, performs arithmetic encoding of coeff_abs_level_greater1_flag indicating the comparison result, and proceeds to step ST32.

  In step ST32, the CABAC encoding unit 160 encodes coeff_abs_level_greater2_flag for the number of coeff_abs_level_greater1_flag = 1. The CABAC encoding unit 160 compares the absolute value of the transform coefficient with coeff_abs_level_greater1_flag = 1 and the threshold “2”, performs arithmetic encoding of coeff_abs_level_greater2_flag indicating the comparison result, and proceeds to step ST33.

  In step ST33, the CABAC encoding unit 160 determines whether the transform coefficient is coeff_abs_level_greater2_flag = 1. The CABAC encoding unit 160 proceeds to step ST34 when the transform coefficient is coeff_abs_level_greater2_flag = 1, and proceeds to step ST35 when the coeff_abs_level_greater2_flag = 1 is not satisfied.

  In step ST34, the CABAC encoding unit 160 encodes coeff_abs_level_minus3. The CABAC encoding unit 160, when coeff_abs_level_greater2_flag = 1, has an absolute value of the transform coefficient larger than “2”. Therefore, the CABAC encoding unit 160 encodes coeff_abs_level_minus3, which is a value obtained by subtracting “3” from the absolute value of the transform coefficient. The CABAC encoding part 160 performs the encoding bypass process of coeff_abs_level_minus3, and progresses to step ST35.

  In step ST35, the CABAC encoding unit 160 determines whether or not the encoding for the sub-block has been completed. The CABAC encoding unit 160 returns to step ST33 when the encoding of the transform coefficient in the subblock is not completed, and proceeds to step ST36 when the encoding of the transform coefficient in the subblock is completed.

  In step ST36, the CABAC encoding unit 160 encodes coeff_sign_flag for the number of coefficients. The CABAC encoding unit 160 performs coeff_sign_flag encoding / bypass processing for the number of coefficients with coeff_sign_flag = 0 when the transform coefficient is a positive value and coeff_sign_flag = 1 when the transform coefficient is a negative value.

  Furthermore, the CABAC encoding unit 160 generates encoded data in the data order shown in FIG. 4D by repeatedly performing the processing from step ST31 to step ST36 in units of sub-blocks.

  FIG. 9 illustrates the CABAC syntax. Note that the number at the left end of each line is a line number given for explanation. As shown in the 18th to 37th lines, the encoded data has syntax elements in the order of coeff_abs_level_greater1_flag, coeff_abs_level_greater2_flag, coeff_abs_level_minus3, and coeff_sign_flag.

  As described above, the CABAC encoding unit 160 generates encoded data in which the syntax elements are in the order of coeff_abs_level_greater1_flag, coeff_abs_level_greater2_flag, coeff_abs_level_minus3, and coeff_sign_flag. That is, the CABAC encoding unit 160 can generate encoded data so that the processing efficiency and speed can be increased during decoding.

<5. Configuration of Image Decoding Device>
Next, an image decoding apparatus that performs decoding processing of an encoded stream output from the image encoding apparatus will be described. An encoded stream generated by encoding an input image is supplied to an image decoding device via a predetermined transmission path, a recording medium, and the like and decoded.

  FIG. 10 illustrates the configuration of an image decoding apparatus that decodes an encoded stream. The image decoding device 50 includes a storage buffer 51, a lossless decoding unit 52, an inverse quantization unit 53, an inverse orthogonal transform unit 54, a calculation unit 55, a deblocking filter 56, a screen rearrangement buffer 57, a digital / analog conversion unit (D / A conversion unit) 58. Further, the image decoding device 50 includes a frame memory 61, an intra prediction unit 71, a motion compensation unit 72, and a selector 73.

  The accumulation buffer 51 accumulates the transmitted encoded stream. The lossless decoding unit 52 decodes the encoded stream supplied from the accumulation buffer 51 by a method corresponding to the encoding method of the CABAC encoding unit 160 shown in FIG.

  The lossless decoding unit 52 performs lossless decoding of the encoded stream and obtains various information. For example, the lossless decoding unit 52 outputs the quantized data of the transform coefficient obtained by decoding the encoded stream to the inverse quantization unit 53. Further, the lossless decoding unit 52 outputs prediction mode information obtained by decoding the encoded stream to the intra prediction unit 71. Further, the lossless decoding unit 52 outputs prediction mode information and motion vector information obtained by decoding the encoded stream to the motion compensation unit 72.

  The inverse quantization unit 53 performs inverse quantization on the quantized data supplied from the lossless decoding unit 52 in a method corresponding to the quantization method of the quantization unit 15 illustrated in FIG. The inverse orthogonal transform unit 54 performs inverse orthogonal transform on the output of the inverse quantization unit 53 by a method corresponding to the orthogonal transform method of the orthogonal transform unit 14 illustrated in FIG.

  The computing unit 55 adds the data after inverse orthogonal transformation and the predicted image data supplied from the selector 73 to generate decoded image data, and outputs the decoded image data to the deblocking filter 56 and the intra prediction unit 71.

  The deblocking filter 56 performs a filter process for reducing block distortion that occurs during image coding. The deblocking filter 56 performs a filter process for removing block distortion from the image data supplied from the calculation unit 55, and outputs the image data after the filter process to the screen rearrangement buffer 57 and the frame memory 61.

  The screen rearrangement buffer 57 rearranges images. That is, the frame order rearranged for the encoding order by the screen rearrangement buffer 12 shown in FIG. 1 is rearranged in the original display order and output to the D / A conversion unit 58.

  The D / A conversion unit 58 performs D / A conversion on the image data supplied from the screen rearrangement buffer 57, and outputs the decoded image to a display (not shown).

  The intra prediction unit 71 generates predicted image data based on the prediction mode information supplied from the lossless decoding unit 52 and the decoded image data supplied from the calculation unit 55, and outputs the generated predicted image data to the selector 73. .

  The motion compensation unit 72 performs motion compensation using the image data read from the frame memory 61 based on the prediction mode information and the motion vector information supplied from the lossless decoding unit 52, and generates predicted image data. The motion compensation unit 72 outputs the generated predicted image data to the selector 73.

  Based on the prediction mode information supplied from the lossless decoding unit 52, the selector 73 selects the intra prediction unit 71 for intra prediction and the motion compensation unit 72 for inter prediction. The selector 73 outputs the predicted image data generated by the selected intra prediction unit 71 or motion compensation unit 72 to the calculation unit 55.

<6. Operation of Image Decoding Device>
Next, the operation of the image decoding device 50 will be described with reference to the flowchart of FIG. In step ST51, the accumulation buffer 51 accumulates the supplied encoded stream. In step ST52, the lossless decoding unit 52 performs lossless decoding processing. The lossless decoding unit 52 decodes the encoded stream supplied from the accumulation buffer 51 by a method corresponding to the lossless encoding unit 16 shown in FIG. The lossless decoding unit 52 outputs the quantized data of the transform coefficient obtained by decoding the encoded stream to the inverse quantization unit 53. Further, the lossless decoding unit 52 outputs prediction mode information obtained by decoding the encoded stream to the intra prediction unit 71. Further, the lossless decoding unit 52 outputs prediction mode information and motion vector information obtained by decoding the encoded stream to the motion compensation unit 72.

  In step ST53, the inverse quantization unit 53 performs an inverse quantization process. The inverse quantization unit 53 inversely quantizes the quantized data decoded by the lossless decoding unit 52 with characteristics corresponding to the characteristics of the quantization unit 15 illustrated in FIG.

  In step ST54, the inverse orthogonal transform unit 54 performs an inverse orthogonal transform process. The inverse orthogonal transform unit 54 performs inverse orthogonal transform on the transform coefficient data inversely quantized by the inverse quantization unit 53 with characteristics corresponding to the characteristics of the orthogonal transform unit 14 illustrated in FIG.

  In step ST55, the calculation unit 55 generates decoded image data. The computing unit 55 adds the data obtained by performing the inverse orthogonal transform process and the predicted image data selected in step ST59 described later to generate decoded image data. As a result, the original image is decoded.

  In step ST56, the deblocking filter 56 performs a deblocking filter process. The deblocking filter 56 filters the decoded image data output from the calculation unit 55 to remove block distortion.

  In step ST57, the frame memory 61 stores data. The frame memory 61 stores decoded image data that has been filtered by the deblocking filter.

  In step ST58, the intra prediction unit 71 and the motion compensation unit 72 generate predicted image data. The intra prediction unit 71 and the motion compensation unit 72 generate prediction image data corresponding to the prediction mode information supplied from the lossless decoding unit 52, respectively.

  That is, when prediction mode information for intra prediction is supplied from the lossless decoding unit 52, the intra prediction unit 71 generates predicted image data based on the prediction mode information. When inter prediction mode information is supplied from the lossless decoding unit 52, the motion compensation unit 72 performs motion compensation based on the prediction mode information and motion vector information to generate predicted image data.

  In step ST59, the selector 73 selects predicted image data. The selector 73 selects the prediction image data supplied from the intra prediction unit 71 and the prediction image data supplied from the motion compensation unit 72, supplies the selected prediction image data to the calculation unit 55, and reverses in step ST55. It is added to the output of the orthogonal transformation unit 54.

  In step ST60, the screen rearrangement buffer 57 performs image rearrangement. That is, the screen rearrangement buffer 57 rearranges the order of frames rearranged for encoding by the screen rearrangement buffer 12 of the image encoding device 10 shown in FIG. 1 to the original display order.

  In step ST61, the D / A converter 58 D / A converts the image data from the screen rearrangement buffer 57. This image is output to a display (not shown), and the image is displayed.

<7. Configuration of CABAC Decoding Unit>
FIG. 12 illustrates the configuration of a CABAC decoding unit that performs CABAC decoding processing in the lossless decoding unit 52. The CABAC decoding unit 520 receives the encoded data from the accumulation buffer 51, decodes it into binary data, and converts it into multilevel data. The CABAC decoding unit 520 includes a context calculation unit 521, an arithmetic decoding processing unit 522, and a multi-value conversion unit 523.

  The context calculation unit 521 generates a context index based on the syntax to be decoded. Further, the context calculation unit 521 selects a context corresponding to the context index and outputs it to the arithmetic decoding processing unit 522. The context calculation unit 521 updates the occurrence probability based on the arithmetic decoding result obtained by the arithmetic decoding processing unit 522.

  The arithmetic decoding processing unit 522 performs reverse processing on the arithmetic coding processing unit 163 based on the encoded data and context input from the accumulation buffer 51 to generate binary data. The arithmetic decoding processing unit 522 is provided with a decoding decision (DecodeDecision) process that is a decoding mode that refers to a context, and a decoding bypass (DecodeBypass) process that is a decoding mode that does not refer to a context. Note that, as a decoding mode, a decode termination process is provided, but a description thereof is omitted. The arithmetic decoding processor 522 selects a decoding mode according to the syntax and decodes the encoded data. For example, when decoding the encoded data of one coefficient, the arithmetic decoding processing unit 522 performs a decoding process based on the context supplied from the context calculation unit 521 for the syntax element that has been subjected to the encoding / decision processing. Do. Further, the decoding bypass process is performed for the syntax element that has been subjected to the encode bypass process. The arithmetic decoding processing unit 522 outputs the binarized data that is the result of the arithmetic decoding to the multileveling unit 523. The arithmetic decoding processing unit 522 outputs the arithmetic decoding result to the context calculating unit 521.

  The multi-value quantization unit 523 performs processing corresponding to the binarization unit 161 of the CABAC encoding unit 160, and the quantized data before binarization of the binarized data supplied from the arithmetic decoding processing unit 522 is performed. And output to the inverse quantization unit 53. In addition, prediction mode information and the like obtained by performing the lossless decoding process are output to the intra prediction unit 71 and the motion compensation unit 72.

<8. Operation of CABAC decoding unit>
FIG. 13 is a flowchart showing the operation of the CABAC decoding unit in the lossless decoding unit. In step ST71, the CABAC decoding unit 520 decodes coeff_abs_level_greater1_flag for the number of coefficients. CABAC decoding section 520 performs decoding / decision processing, generates coeff_abs_level_greater1_flag for the number of coefficients, and proceeds to step ST72.

  In step ST72, the CABAC decoding unit 520 decodes coeff_abs_level_greater2_flag for the number of coeff_abs_level_greater1_flag = 1. The CABAC decoding unit 520 performs decoding / decision processing, generates coeff_abs_level_greater1_flag for the number of coeff_abs_level_greater1_flag = 1, and proceeds to step ST73.

  In step ST <b> 73, the CABAC decoding unit 520 determines whether the conversion coefficient is coeff_abs_level_greater2_flag = 1. CABAC decoding section 520 proceeds to step ST74 when coeff_abs_level_greater2_flag = 1, and proceeds to step ST76 when coeff_abs_level_greater2_flag = 1.

  In step ST74, the CABAC decoding unit 520 performs 1-bit decoding. The CABAC decoding unit 520 performs 1-bit decoding / bypass processing and proceeds to step ST75.

  In step ST75, the CABAC decoding unit 520 determines whether or not the decoding of coeff_abs_level_minus3 has been completed. CABAC decoding section 520 returns to step ST74 when the decoding result of coeff_abs_level_minus3 is not obtained, and proceeds to step ST76 when the decoding result of coeff_abs_level_minus3 is obtained.

  In step ST76, the CABAC decoding unit 520 determines whether the decoding for the sub-block has been completed. The CABAC decoding unit 520 returns to step ST73 when the decoding processing result for the sub-block is not obtained, and proceeds to step ST77 when the decoding processing result for the sub-block is obtained.

  In step ST77, the CABAC decoding unit 520 decodes coeff_sign_flag for the number of coefficients. The CABAC decoding unit 520 performs decoding / bypass processing to decode the coeff_sign_flag for the number of coefficients.

  Further, CABAC decoding section 520 correctly decodes the encoded data in the data order shown in FIG. 4D by repeatedly performing the processing from step ST71 to step ST77 in units of sub-blocks.

  In addition, since the transform coefficients are sequentially determined in the correct order every time coeff_sign_flag is decoded, the CABAC decoding unit 520 does not need to store in the buffer in order to output the already determined transform coefficients in the correct order. Processing can be performed efficiently. Also, the coeff_sign_flag can be processed together to perform a decoding process at high speed.

<9. For software processing>
The series of processes described in the specification can be executed by hardware, software, or a combined configuration of both. When processing by software is executed, a program in which a processing sequence is recorded is installed and executed in a memory in a computer incorporated in dedicated hardware. Alternatively, the program can be installed and executed on a general-purpose computer capable of executing various processes.

  For example, the program can be recorded in advance on a hard disk or ROM (Read Only Memory) as a recording medium. Alternatively, the program is temporarily or permanently stored on a removable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, or a semiconductor memory. It can be stored (recorded). Such a removable recording medium can be provided as so-called package software.

  The program is installed on the computer from the removable recording medium as described above, or is wirelessly transferred from the download site to the computer, or is wired to the computer via a network such as a LAN (Local Area Network) or the Internet. . The computer can receive the program transferred in this way and install it on a recording medium such as a built-in hard disk.

  The step of describing the program includes not only the processing that is performed in time series in the order described, but also the processing that is not necessarily performed in time series but is executed in parallel or individually.

<10. When applied to electronic devices>
Further, the image encoding device 10 and the image decoding device 50 described above can be applied to any electronic device. Examples thereof will be described below.

  FIG. 14 illustrates a schematic configuration of a television apparatus to which the present technology is applied. The television apparatus 90 includes an antenna 901, a tuner 902, a demultiplexer 903, a decoder 904, a video signal processing unit 905, a display unit 906, an audio signal processing unit 907, a speaker 908, and an external interface unit 909. Furthermore, the television apparatus 90 includes a control unit 910, a user interface unit 911, and the like.

  The tuner 902 selects and demodulates a desired channel from the broadcast wave signal received by the antenna 901, and outputs the obtained encoded bit stream to the demultiplexer 903.

  The demultiplexer 903 extracts video and audio packets of the program to be viewed from the encoded bit stream, and outputs the extracted packet data to the decoder 904. Further, the demultiplexer 903 supplies a packet of data such as EPG (Electronic Program Guide) to the control unit 910. If scrambling is being performed, descrambling is performed by a demultiplexer or the like.

  The decoder 904 performs packet decoding processing, and outputs video data generated by the decoding processing to the video signal processing unit 905 and audio data to the audio signal processing unit 907.

  The video signal processing unit 905 performs noise removal, video processing according to user settings, and the like on the video data. The video signal processing unit 905 generates video data of a program to be displayed on the display unit 906, image data by processing based on an application supplied via a network, and the like. The video signal processing unit 905 generates video data for displaying a menu screen for selecting an item and the like, and superimposes the video data on the video data of the program. The video signal processing unit 905 generates a drive signal based on the video data generated in this way, and drives the display unit 906.

  The display unit 906 drives a display device (for example, a liquid crystal display element or the like) based on a drive signal from the video signal processing unit 905 to display a program video or the like.

  The audio signal processing unit 907 performs predetermined processing such as noise removal on the audio data, performs D / A conversion processing and amplification processing on the processed audio data, and outputs the audio data by supplying the audio data to the speaker 908. .

  The external interface unit 909 is an interface for connecting to an external device or a network, and performs data transmission / reception such as video data and audio data.

  A user interface unit 911 is connected to the control unit 910. The user interface unit 911 includes an operation switch, a remote control signal receiving unit, and the like, and supplies an operation signal corresponding to a user operation to the control unit 910.

  The control unit 910 is configured using a CPU (Central Processing Unit), a memory, and the like. The memory stores a program executed by the CPU, various data necessary for the CPU to perform processing, EPG data, data acquired via a network, and the like. The program stored in the memory is read and executed by the CPU at a predetermined timing such as when the television device 90 is activated. The CPU controls each unit so that the television device 90 operates according to the user operation by executing the program.

  Note that the television device 90 is provided with a bus 912 for connecting the tuner 902, the demultiplexer 903, the video signal processing unit 905, the audio signal processing unit 907, the external interface unit 909, and the control unit 910.

  In the television device configured as described above, the decoder 904 is provided with the function of the image processing device (image processing method) of the present technology. For this reason, by using the function of the image processing apparatus of the present technology on the broadcasting station side, the encoded stream can be efficiently decoded by the television apparatus.

  FIG. 15 illustrates a schematic configuration of a mobile phone to which the present technology is applied. The cellular phone 92 includes a communication unit 922, an audio codec 923, a camera unit 926, an image processing unit 927, a demultiplexing unit 928, a recording / reproducing unit 929, a display unit 930, and a control unit 931. These are connected to each other via a bus 933. An antenna 921 is connected to the communication unit 922, and a speaker 924 and a microphone 925 are connected to the audio codec 923. Further, an operation unit 932 is connected to the control unit 931.

  The mobile phone 92 performs various operations such as transmission / reception of voice signals, transmission / reception of e-mail and image data, image shooting, and data recording in various modes such as a voice call mode and a data communication mode.

  In the voice call mode, the voice signal generated by the microphone 925 is converted into voice data and compressed by the voice codec 923 and supplied to the communication unit 922. The communication unit 922 performs audio data modulation processing, frequency conversion processing, and the like to generate a transmission signal. The communication unit 922 supplies a transmission signal to the antenna 921 and transmits it to a base station (not shown). In addition, the communication unit 922 performs amplification, frequency conversion processing, demodulation processing, and the like of the reception signal received by the antenna 921, and supplies the obtained audio data to the audio codec 923. The audio codec 923 performs data expansion of the audio data and conversion to an analog audio signal and outputs the result to the speaker 924.

  In addition, when mail transmission is performed in the data communication mode, the control unit 931 accepts character data input by operating the operation unit 932 and displays the input characters on the display unit 930. In addition, the control unit 931 generates mail data based on a user instruction or the like in the operation unit 932 and supplies the mail data to the communication unit 922. The communication unit 922 performs mail data modulation processing, frequency conversion processing, and the like, and transmits the obtained transmission signal from the antenna 921. In addition, the communication unit 922 performs amplification, frequency conversion processing, demodulation processing, and the like of the reception signal received by the antenna 921, and restores mail data. This mail data is supplied to the display unit 930 to display the mail contents.

  Note that the mobile phone 92 can also store the received mail data in a storage medium by the recording / playback unit 929. The storage medium is any rewritable storage medium. For example, the storage medium is a removable medium such as a semiconductor memory such as a RAM or a built-in flash memory, a hard disk, a magnetic disk, a magneto-optical disk, an optical disk, a USB memory, or a memory card.

  When transmitting image data in the data communication mode, the image data generated by the camera unit 926 is supplied to the image processing unit 927. The image processing unit 927 performs encoding processing of image data and generates encoded data.

  The demultiplexing unit 928 multiplexes the encoded data generated by the image processing unit 927 and the audio data supplied from the audio codec 923 by a predetermined method, and supplies the multiplexed data to the communication unit 922. The communication unit 922 performs modulation processing and frequency conversion processing of multiplexed data, and transmits the obtained transmission signal from the antenna 921. In addition, the communication unit 922 performs amplification, frequency conversion processing, demodulation processing, and the like of the reception signal received by the antenna 921, and restores multiplexed data. This multiplexed data is supplied to the demultiplexing unit 928. The demultiplexing unit 928 performs demultiplexing of the multiplexed data, and supplies the encoded data to the image processing unit 927 and the audio data to the audio codec 923. The image processing unit 927 performs a decoding process on the encoded data to generate image data. The image data is supplied to the display unit 930 and the received image is displayed. The audio codec 923 converts the audio data into an analog audio signal, supplies the analog audio signal to the speaker 924, and outputs the received audio.

  In the mobile phone device configured as described above, the image processing unit 927 is provided with the function of the image processing device (image processing method) of the present technology. Therefore, communication data can be generated so that decoding can be efficiently performed when communicating image data. Also, it is possible to efficiently perform the decoding process of communication data.

  FIG. 16 illustrates a schematic configuration of a recording / reproducing apparatus to which the present technology is applied. The recording / reproducing apparatus 94 records, for example, audio data and video data of a received broadcast program on a recording medium, and provides the recorded data to the user at a timing according to a user instruction. The recording / reproducing device 94 can also acquire audio data and video data from another device, for example, and record them on a recording medium. Furthermore, the recording / reproducing device 94 decodes and outputs the audio data and video data recorded on the recording medium, thereby enabling image display and audio output on the monitor device or the like.

  The recording / reproducing apparatus 94 includes a tuner 941, an external interface unit 942, an encoder 943, an HDD (Hard Disk Drive) unit 944, a disk drive 945, a selector 946, a decoder 947, an OSD (On-Screen Display) unit 948, a control unit 949, A user interface unit 950 is included.

  The tuner 941 selects a desired channel from a broadcast signal received by an antenna (not shown). The tuner 941 outputs an encoded bit stream obtained by demodulating the received signal of a desired channel to the selector 946.

  The external interface unit 942 includes at least one of an IEEE 1394 interface, a network interface unit, a USB interface, a flash memory interface, and the like. The external interface unit 942 is an interface for connecting to an external device, a network, a memory card, and the like, and receives data such as video data and audio data to be recorded.

  The encoder 943 performs encoding by a predetermined method when the video data and audio data supplied from the external interface unit 942 are not encoded, and outputs an encoded bit stream to the selector 946.

  The HDD unit 944 records content data such as video and audio, various programs, and other data on a built-in hard disk, and reads them from the hard disk at the time of reproduction or the like.

  The disk drive 945 records and reproduces signals with respect to the mounted optical disk. An optical disc, for example, a DVD disc (DVD-Video, DVD-RAM, DVD-R, DVD-RW, DVD + R, DVD + RW, etc.), a Blu-ray (registered trademark) disc, or the like.

  The selector 946 selects one of the encoded bit streams from the tuner 941 or the encoder 943 when recording video or audio, and supplies the selected bit stream to either the HDD unit 944 or the disk drive 945. Further, the selector 946 supplies the encoded bit stream output from the HDD unit 944 or the disk drive 945 to the decoder 947 at the time of reproduction of video and audio.

  The decoder 947 performs a decoding process on the encoded bitstream. The decoder 947 supplies the video data generated by performing the decoding process to the OSD unit 948. The decoder 947 outputs audio data generated by performing the decoding process. The OSD unit 948 generates video data for displaying a menu screen for selecting an item and the like, and superimposes the video data on the video data output from the decoder 947 and outputs the video data.

  A user interface unit 950 is connected to the control unit 949. The user interface unit 950 includes an operation switch, a remote control signal receiving unit, and the like, and supplies an operation signal corresponding to a user operation to the control unit 949.

  The control unit 949 is configured using a CPU, a memory, and the like. The memory stores programs executed by the CPU and various data necessary for the CPU to perform processing. The program stored in the memory is read and executed by the CPU at a predetermined timing such as when the recording / reproducing apparatus 94 is activated. The CPU executes the program to control each unit so that the recording / reproducing device 94 operates in accordance with the user operation.

  In the recording / reproducing apparatus configured as described above, the function of the image processing apparatus (image processing method) of the present technology is provided in the encoder 943 and the decoder 947, so that video can be recorded and reproduced at high speed.

  FIG. 17 illustrates a schematic configuration of an imaging apparatus to which the present technology is applied. The imaging device 96 images a subject and displays an image of the subject on a display unit, or records it on a recording medium as image data.

  The imaging device 96 includes an optical block 961, an imaging unit 962, a camera signal processing unit 963, an image data processing unit 964, a display unit 965, an external interface unit 966, a memory unit 967, a media drive 968, an OSD unit 969, and a control unit 970. Have. In addition, a user interface unit 971 is connected to the control unit 970. Furthermore, the image data processing unit 964, the external interface unit 966, the memory unit 967, the media drive 968, the OSD unit 969, the control unit 970, and the like are connected via a bus 972.

  The optical block 961 is configured using a focus lens, a diaphragm mechanism, and the like. The optical block 961 forms an optical image of the subject on the imaging surface of the imaging unit 962. The imaging unit 962 is configured using a CCD or CMOS image sensor, generates an electrical signal corresponding to the optical image by photoelectric conversion, and supplies the electrical signal to the camera signal processing unit 963.

  The camera signal processing unit 963 performs various camera signal processes such as knee correction, gamma correction, and color correction on the electrical signal supplied from the imaging unit 962. The camera signal processing unit 963 supplies the image data after the camera signal processing to the image data processing unit 964.

  The image data processing unit 964 performs an encoding process on the image data supplied from the camera signal processing unit 963. The image data processing unit 964 supplies the encoded data generated by performing the encoding process to the external interface unit 966 and the media drive 968. Further, the image data processing unit 964 performs a decoding process on the encoded data supplied from the external interface unit 966 and the media drive 968. The image data processing unit 964 supplies the image data generated by performing the decoding process to the display unit 965. Further, the image data processing unit 964 superimposes the processing for supplying the image data supplied from the camera signal processing unit 963 to the display unit 965 and the display data acquired from the OSD unit 969 on the image data. To supply. The OSD unit 969 generates display data such as a menu screen or an icon made up of symbols, characters, or graphics and outputs it to the image data processing unit 964.

  The external interface unit 966 includes, for example, a USB input / output terminal, and is connected to a printer when printing an image. In addition, a drive is connected to the external interface unit 966 as necessary, a removable medium such as a magnetic disk or an optical disk is appropriately mounted, and a computer program read from them is installed as necessary. Furthermore, the external interface unit 966 has a network interface connected to a predetermined network such as a LAN or the Internet. For example, the control unit 970 reads the encoded data from the memory unit 967 in accordance with an instruction from the user interface unit 971, and supplies the encoded data to the other device connected via the network from the external interface unit 966. it can. Also, the control unit 970 may acquire encoded data and image data supplied from another device via the network via the external interface unit 966 and supply the acquired data to the image data processing unit 964. it can.

  As a recording medium driven by the media drive 968, any readable / writable removable medium such as a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory is used. The recording medium may be any type of removable medium, and may be a tape device, a disk, or a memory card. Of course, a non-contact IC card or the like may be used.

  Further, the media drive 968 and the recording medium may be integrated and configured by a non-portable storage medium such as a built-in hard disk drive or an SSD (Solid State Drive).

  The control unit 970 is configured using a CPU, a memory, and the like. The memory stores programs executed by the CPU, various data necessary for the CPU to perform processing, and the like. The program stored in the memory is read and executed by the CPU at a predetermined timing such as when the imaging device 96 is activated. The CPU executes the program to control each unit so that the imaging device 96 operates according to the user operation.

  In the imaging apparatus configured as described above, the image data processing unit 964 is provided with the function of the image processing apparatus (image processing method) of the present technology. Therefore, when a captured image is recorded in the memory unit 967, a recording medium, or the like, or when a recorded captured image is reproduced, recording and reproduction can be performed efficiently.

  Further, the present technology should not be construed as being limited to the embodiments of the technology described above. The embodiments of this technology disclose the present technology in the form of examples, and it is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the gist of the present technology. That is, in order to determine the gist of the present technology, the claims should be taken into consideration.

Note that the image processing apparatus of the present technology may also have the following configuration.
(1) The absolute value of the transform coefficient after quantization is compared with the threshold “m”, and when the absolute value is greater than the threshold “m”, the threshold is set to “m = m + 1”. = 1 ”to a predetermined value, first information indicating a comparison result, second information corresponding to an excess of the absolute value with respect to the predetermined value, and third information indicating a sign of the conversion coefficient Each of which is a syntax element, an arithmetic decoding processing unit that performs arithmetic decoding processing on encoded data encoded in the order of the first information, the second information, and the third information;
An image processing apparatus comprising: a multi-value quantization unit that multi-values binarized data obtained by performing arithmetic decoding processing in the arithmetic decoding processing unit and generates the transform coefficient before encoding.
(2) The arithmetic decoding processing unit performs the decoding process of the first information in a mode that refers to a context representing a probability state and a dominant probability symbol, and performs the second information and the third in a mode that does not refer to the context. (1) The image processing apparatus according to (1).
(3) The encoded data is the image processing apparatus according to (1) or (2), wherein the encoded data is data generated with the predetermined number being “m = 2”.
(4) The image processing device according to any one of (1) to (3), wherein the encoded data is data obtained by encoding the transform coefficient in subblock units.
(5) The quantized transform coefficient absolute value is compared with the threshold “m”, and when the absolute value is greater than the threshold “m”, the threshold is set to “m = m + 1”. = 1 ”to a predetermined value, first information indicating a comparison result, second information corresponding to an excess of the absolute value with respect to the predetermined value, and third information indicating a sign of the conversion coefficient Each of which is a syntax element, and a binarization unit that binarizes the first information, the second information, and the third information in order,
An image processing apparatus comprising: an arithmetic encoding processing unit that performs arithmetic encoding processing of binarized data obtained by the binarizing unit and generates encoded data.
(6) The arithmetic coding processing unit performs coding processing of the first information in a mode that refers to a context representing a probability state and a dominant probability symbol, and performs the second information and a mode that does not refer to the context. The image processing device according to (5), wherein the third information is encoded.
(7) The image processing apparatus according to (5) or (6), wherein the binarization unit generates the binarized data by setting the predetermined number to “m = 2”.
(8) The image processing device according to any one of (5) to (7), wherein the binarization unit and the arithmetic coding processing unit perform binarization and coding processing of the transform coefficient in units of sub-blocks. .

  In the image processing apparatus and the image processing method of the present technology, the absolute value of the transform coefficient after quantization and the threshold “m” are compared, and when the absolute value is larger than the threshold “m”, the threshold is set to “m = m + 1”. The first information indicating the comparison result until the threshold value reaches the predetermined value from “m = 1”, the second information according to the absolute value exceeding the predetermined value, and the sign of the transform coefficient Using the third information as a syntax element, generation and decoding of encoded data encoded in the order of the first information, the second information, and the third information are performed. By setting the order of such syntax elements, transform coefficients can be determined in order at the time of decoding. Also, it is possible to perform information encoding processing and decoding processing together. Therefore, CABAC encoding processing and decoding processing can be performed efficiently. In addition, the present technology provides an electronic device, an optical disc, a magnetic disc, and the like that transmits / receives an encoded stream obtained by performing encoding in block units via network media such as satellite broadcasting, cable TV, the Internet, and a mobile phone. It is suitable for an electronic device that records and reproduces an image using a storage medium such as a flash memory.

  DESCRIPTION OF SYMBOLS 10 ... Image coding apparatus, 11 ... A / D conversion part, 12, 57 ... Screen rearrangement buffer, 13, 23, 55 ... Calculation part, 14 ... Orthogonal transformation part, 15 ... Quantizer, 16 ... Lossless encoder, 17 ... Accumulation buffer, 18 ... Rate controller, 21, 53 ... Inverse quantizer, 22, 54 ... Inverse orthogonal Conversion unit, 24, 56 ... deblocking filter, 25, 61 ... frame memory, 31 ... intra prediction unit, 32 ... motion prediction / compensation unit, 33 ... prediction image / optimum mode selection , 50 ... Image decoding device, 51 ... Accumulation buffer, 52 ... Lossless decoding part, 53 ... Dequantization part, 58 ... D / A conversion part, 71 ... Intra prediction , 72... Motion compensation unit, 73, 946... Selector, 90. Device: 92, mobile phone, 94: recording / playback device, 96: imaging device, 160: CABAC encoding unit, 161: binarization unit, 162, 521 ... context calculation 163 ... arithmetic encoding processing unit, 520 ... CABAC decoding unit, 522 ... arithmetic decoding processing unit, 523 ... multi-value quantization unit, 901, 921 ... antenna, 902 ... Tuner 903... Demultiplexer 904 947 Decoder 905 Video signal processing unit 906 930 965 Display unit 907 Audio signal processing unit 908 924 ..Speaker, 909, 942, 966... External interface unit, 910, 931, 949, 970... Control unit, 911, 950, 971. 33,972 ... Bus, 922 ... Communication unit, 923 ... Audio codec, 925 ... Microphone, 926 ... Camera unit, 927 ... Image processing unit, 928 ... Demultiplexing unit , 929... Recording / playback unit, 932... Operation unit, 941... Tuner, 943... Encoder, 944... HDD unit, 945. , 961... Optical block, 962... Imaging unit, 963... Camera signal processing unit, 964... Image data processing unit, 967.

(D) of FIG. 6 has shown coeff_abs_level_minus3. coeff_abs_level_minus3 is a value obtained by subtracting 3 from the absolute value for the transform coefficient with coeff_abs_level_greater2_flag = 1.

(E) of FIG. 6 has shown coeff_sign_flag. coeff_sign_flag is set to coeff_sign_flag = 0 when the conversion coefficient is a positive value, and coeff_sign_flag = 1 when the conversion coefficient is a negative value.

Claims (10)

When the absolute value of the transform coefficient after quantization is compared with the threshold “m” and the absolute value is larger than the threshold “m”, the comparison is performed with “m = m + 1” as the threshold, and the threshold is “m = 1”. First information indicating the comparison result from the first to the predetermined value, second information corresponding to the excess of the absolute value with respect to the predetermined value, and third information indicating the sign of the transform coefficient As a tax element, an arithmetic decoding processing unit that performs arithmetic decoding processing on encoded data encoded in the order of the first information, the second information, and the third information;
An image processing apparatus comprising: a multi-value quantization unit that multi-values binarized data obtained by performing arithmetic decoding processing in the arithmetic decoding processing unit and generates the transform coefficient before encoding. The arithmetic decoding processing unit performs the decoding process of the first information in a mode referring to a context representing a probability state and a dominant probability symbol, and the second information and the third information in a mode not referring to the context. The image processing apparatus according to claim 1, wherein decoding processing is performed. The image processing apparatus according to claim 1, wherein the encoded data is data generated with the predetermined number being “m = 2”. The image processing apparatus according to claim 1, wherein the encoded data is data obtained by encoding the transform coefficient in subblock units. When the absolute value of the transform coefficient after quantization is compared with the threshold “m” and the absolute value is larger than the threshold “m”, the comparison is performed with “m = m + 1” as the threshold, and the threshold is “m = 1”. First information indicating the comparison result from the first to the predetermined value, second information corresponding to the excess of the absolute value with respect to the predetermined value, and third information indicating the sign of the transform coefficient Performing an arithmetic decoding process on the encoded data encoded in the order of the first information, the second information, and the third information as a tax element;
An image processing method including: performing binarization of the binarized data obtained by performing the arithmetic decoding process, and generating the transform coefficient before encoding. When the absolute value of the transform coefficient after quantization is compared with the threshold “m” and the absolute value is larger than the threshold “m”, the comparison is performed with “m = m + 1” as the threshold, and the threshold is “m = 1”. First information indicating the comparison result from the first to the predetermined value, second information corresponding to the excess of the absolute value with respect to the predetermined value, and third information indicating the sign of the transform coefficient As a tax element, a binarization unit that binarizes first information, second information, and third information in this order;
An image processing apparatus comprising: an arithmetic encoding processing unit that performs arithmetic encoding processing of binary data obtained by the binarizing unit and generates encoded data. The arithmetic coding processing unit performs the coding process of the first information in a mode that refers to a context representing a probability state and a dominant probability symbol, and performs the second information and the third information in a mode that does not refer to the context. The image processing apparatus according to claim 6, wherein the information encoding process is performed. The image processing apparatus according to claim 6, wherein the binarization unit generates the binarized data with the predetermined number being “m = 2”. The image processing apparatus according to claim 6, wherein the binarization unit and the arithmetic coding processing unit perform binarization and coding processing of the transform coefficient in units of sub blocks. When the absolute value of the transform coefficient after quantization is compared with the threshold “m” and the absolute value is larger than the threshold “m”, the comparison is performed with “m = m + 1” as the threshold, and the threshold is “m = 1”. First information indicating the comparison result from the first to the predetermined value, second information corresponding to the excess of the absolute value with respect to the predetermined value, and third information indicating the sign of the transform coefficient As a tax element, binarizing in the order of the first information, the second information, and the third information;
An image processing method including a step of performing arithmetic encoding processing of binary data obtained by the binarization to generate encoded data.

Images