WO2014002445A1

WO2014002445A1 - Image coding apparatus, image coding method, and recording medium, and image decoding apparatus, image decoding method, and recording medium

Info

Publication number: WO2014002445A1
Application number: PCT/JP2013/003861
Authority: WO
Inventors: Satoshi Naito
Original assignee: Canon Kabushiki Kaisha
Priority date: 2012-06-27
Filing date: 2013-06-20
Publication date: 2014-01-03
Also published as: JP2014011481A

Abstract

A received image is segmented into blocks for each of which a predictive residue is generated from a reference pixel. The predictive residue is transformed into a transformation coefficient which is quantized to generate coefficient data. Entropy coding is performed on the generated coefficient data. When the amount of the coefficient data which has been subjected to entropy coding exceeds a predetermined threshold, data indicating the image obtained through the segmentation into blocks is selected. When the amount of the coefficient data does not exceed a predetermined threshold, the coefficient data which has been subjected to entropy coding is selected. The selected data is output as a code string.

Description

IMAGE CODING APPARATUS, IMAGE CODING METHOD, AND RECORDING MEDIUM, AND IMAGE DECODING APPARATUS, IMAGE DECODING METHOD, AND RECORDING MEDIUM

The present invention relates to an image coding apparatus, an image coding method, and a recording medium, and an image decoding apparatus, an image decoding method, and a recording medium, and particularly to a process of bypassing entropy coding.

A known coding system used in compression and recording of a moving image is H.264/MPEG-4 AVC (hereinafter, referred to as H.264; see NPL 1). In H.264, not only are 16 x 16 pixel macroblocks which have been subjected to entropy coding output, but also a mode called the I-PCM mode in which original image data, as it is, is output can be selected for each macroblock. H.264 has a restriction that the number of bits of a macroblock should not exceed 3200 bits. When the data amount of a macroblock which has been subjected to entropy coding exceeds 3200 bits, it is necessary to select the I-PCM mode or, for example, to perform coding again with an increased quantization scale value.

H.264 employs two entropy coding systems, context adaptive variable length coding (CAVLC) and context-based adaptive binary arithmetic coding (CABAC). Which entropy coding system is to be used can be selected for each picture.

In CABAC, each of the syntax elements which are to be subjected to entropy coding is binarized so that binary data is generated. A table describing occurrence probabilities (hereinafter, referred to as an occurrence probability table) is provided for each of the syntax elements in advance, and the binary data is subjected to arithmetic coding on the basis of the occurrence probability table. Every time coding is performed, the occurrence probability table is updated on the basis of statistical information describing whether or not the binary data which has been encoded is a symbol having a high occurrence probability.

Compared with CAVLC, CABAC has an advantage of having a high compression rate, but has a disadvantage of having low processing performance per clock cycle because the above-described processes have to be performed. In particular, since the processing time per macroblock of CABAC changes depending on the number of bits of binary data, CABAC may have insufficient decoding processing performance when the compression rate is low. Therefore, CAVLC is selected for a picture for which occurrence of a macroblock having a compression rate resulting in the number of bits of the macroblock which is close to 3200 bits described above is expected from analysis of an original image or setting of quantization values, preventing decoding processing performance from being insufficient.

Recently, international standardization for a higher efficient coding system that is a successor to H.264 has started, and the International Organization for Standardization (ISO)/the International Electrotechnical Commission (IEC) and the International Telecommunications Union-Telecommunication Standardization Sector (ITU-T) have founded the Joint Collaborative Team on Video Coding (JCT-VC). In the JCT-VC, standardization for high efficiency video coding (hereinafter, referred to as HEVC) is being pursued.

In the standardization of HEVC, various coding tools are being studied from a wide viewpoint including ease of implementation and reduction in processing time as well as increase in coding efficiency. From the viewpoint of reduction in processing time, methods are being devised which increase parallelism under the assumption that such tools are operated by multi-core central processing units (CPUs) or the like. Examples of such methods include a method for segmenting an image into units, such as slices, tiles, or block lines, to perform entropy coding/decoding in parallel.

In contrast, the studies are progressing based on the premise that only CABAC is used as an entropy coding system. As described above, a method for performing entropy coding/decoding in parallel has been introduced. Therefore, the processing performance required for CABAC to perform coding/decoding on image data having a predetermined size has a wide range.

Further, coding in HEVC is performed using, as a unit, a block called a largest coding unit (LCU) instead of a macroblock in H.264. The size of an LCU can be selected from 16 x 16 pixels, 32 x 32 pixels, and 64 x 64 pixels. An LCU can be further segmented into subblocks each called a coding unit (CU), and the I-PCM mode can be selected by using a CU as a unit.

As described above, the studies in HEVC are progressing based on the premise that only CABAC is used in entropy coding. Therefore, even when a subblock which requires a long decoding processing time may be expected to occur because the subblock has a low compression rate, it is not possible to switch to a fast entropy coding system. In the application field in which application to real-time compression processing in a digital video camera or the like is assumed, it is difficult to encode the above-described subblock again by increasing a quantization scaling value, from the viewpoint of required processing performance.

One method for solving such a problem is a method in which an upper limit which is determined for the data amount of a subblock and which serves as a threshold for determining whether or not the I-PCM mode is to be selected is set lower than one of the related art because, in the I-PCM mode in which original image data, as it is, is output as a code string, entropy decoding is bypassed in a decoding process.

For example, a threshold used when a 16 x 16 pixel subblock is subjected to entropy coding is set to 500 bits instead of 3200 bits which is the corresponding threshold in H.264. In contrast, when coding is performed in the I-PCM mode, a subblock having more than 500 bits is allowed to be processed. As described above, the processing time per subblock of CABAC changes depending on the number of bits of binary data, and the processing is typically performed at a rate such that about one bit of binary data is processed in one cycle. Therefore, when the above-described threshold is equal to 500 bits, it can be estimated that entropy decoding is performed on a 16 x 16 pixel subblock in about 500 cycles.

However, as described above, a method for performing entropy coding/decoding in parallel is introduced in HEVC. Accordingly, processing performance required for entropy decoding units achieving parallel processing on the decoding apparatus side is not constant. Needless to say, entropy decoding performance also depends on the performance of a CPU when decoding is implemented through software executed on the CPU, and also depends on the clock frequency of a large-scale integrated circuit (LSI) when the decoding is implemented using dedicated circuits in the LSI.

However, when the threshold for determining whether or not the I-PCM mode is to be selected is set lower in a uniform way, the I-PCM mode may be selected more than necessary even when an entropy decoding unit has sufficient processing performance. When the I-PCM mode is selected more than necessary, many subblocks of original image data which is not subjected to compression are output to a code string, resulting in reduction in compression efficiency.

In contrast, when the above-described threshold is set high, a decoding apparatus which does not support parallel entropy decoding has insufficient entropy decoding performance, resulting in possibility of frames being dropped in a decoded moving image.

ITU-T H.264 (03/2010) Advanced video coding for generic audiovisual services JCT-VC contribution JCTVC-H1003 internet <http://phenix.int-evry.fr/jct/doc_end_user/documents/8_San%20Jose/wg11/>

Therefore, the present invention achieves coding/decoding in which decoded image frames can be prevented from being dropped due to insufficient entropy decoding performance, and in which data which has bypassed entropy coding can be prevented from being output to a code string more than necessary.

To solve the above-described problem, an image coding apparatus in the present invention has the following configuration. That is, the image coding apparatus includes a prediction unit, a quantization unit, an entropy coding unit, a selection unit, and a code-string output unit. The prediction unit segments a received image into multiple blocks and generates a predictive residue from a reference pixel. The quantization unit transforms the predictive residue into a transformation coefficient and quantizes the generated transformation coefficient so as to generate coefficient data. The entropy coding unit performs entropy coding on the generated coefficient data. The selection unit selects data indicating the image obtained through the segmentation into blocks when a data amount of the coefficient data which has been subjected to entropy coding exceeds a predetermined threshold, and selects the coefficient data which has been subjected to entropy coding when the data amount of the coefficient data does not exceed the predetermined threshold. The code-string output unit outputs the selected data as a code string.

Further, an image decoding apparatus in the present invention has the following configuration. That is, the image decoding apparatus that decodes a code string corresponding to an image which has been subjected to entropy coding on a block-by-block basis includes a decoding unit and a determination unit. The decoding unit decodes, from the code string, a profile identifier for identifying a profile indicating a combination of coding processes. The determination unit determines, from the code string, a coding type identifier for identifying, on a block-by-block basis, which one among data which has been subjected to entropy coding and data which has not been subjected to entropy coding is included. The profile is used to derive an upper limit per block for an amount of the data which has been subjected to entropy coding.

The image coding apparatus in the present invention selects whether or not entropy coding is to be bypassed on the basis of a profile indicating a combination of decoding capabilities or a coding level defining the number of pixels to be processed per unit time which are included in a code string. Therefore, a code string optimized for the entropy decoding performance of the image decoding apparatus can be output. Thus, decoded image frames can be prevented from being dropped due to insufficient entropy decoding performance, and data which has bypassed entropy coding can be prevented from being output to a code string more than necessary, resulting in an increase in the compression rate.

Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.

Fig. 1 is a block diagram illustrating the configuration of an image coding apparatus according to a first embodiment of the present invention. Fig. 2 is a flowchart for describing a procedure of a data selection unit according to the first embodiment. Fig. 3 is a table for describing thresholds used by the data selection unit according to the first embodiment. Fig. 4 is a table for describing thresholds used by the data selection unit in a modification example of the first embodiment. Fig. 5 is a table for describing thresholds used by the data selection unit in a modification example of the first embodiment. Fig. 6 is a table for describing thresholds used by the data selection unit in a modification example of the first embodiment. Fig. 7 is a block diagram illustrating the configuration of an entropy coding unit according to a second embodiment of the present invention. Fig. 8 is a flowchart for describing a procedure of the data selection unit according to the second embodiment. Fig. 9 is a table for describing thresholds used by the data selection unit according to the second embodiment. Fig. 10 is a table for describing thresholds used by the data selection unit in a modification example of the second embodiment. Fig. 11 is a table for describing thresholds used by the data selection unit in a modification example of the second embodiment. Fig. 12 is a flowchart for describing a procedure of the data selection unit according to a third embodiment of the present invention. Fig. 13 is a flowchart for describing a procedure of the data selection unit according to a fourth embodiment of the present invention. Fig. 14 is a table for describing thresholds used by the data selection unit according to the fourth embodiment. Fig. 15 is a table for describing thresholds used by the data selection unit in a modification example of the fourth embodiment. Fig. 16 is a table for describing thresholds used by the data selection unit in a modification example of the fourth embodiment. Fig. 17 is a block diagram illustrating the configuration of an image decoding apparatus according to a fifth embodiment of the present invention. Fig. 18 is a flowchart for describing a procedure of an overall controller according to the fifth embodiment. Fig. 19 is a flowchart for describing a procedure of the overall controller in a modification example of the fifth embodiment. Fig. 20 is a flowchart for describing a procedure of entropy decoding units according to the fifth embodiment. Fig. 21 is a flowchart for describing a procedure of the entropy decoding units in a modification example of the fifth embodiment. Fig. 22 is a block diagram illustrating the configuration of the entropy decoding units according to the fifth embodiment. Fig. 23 is a flowchart for describing a procedure of the entropy decoding units according to a sixth embodiment of the present invention. Fig. 24 is a flowchart for describing a determination procedure performed by an entropy decoding controller according to a seventh embodiment of the present invention. Fig. 25 is a flowchart for describing a determination procedure performed by the entropy decoding controller according to an eighth embodiment of the present invention. Fig. 26A is a diagram illustrating an example of segmentation into slices, tiles, and block lines. Fig. 26B is a diagram illustrating an example of segmentation into slices, tiles, and block lines. Fig. 27 is a table for describing thresholds derived by the data selection unit in a modification example of the fourth embodiment. Fig. 28 is a block diagram illustrating an exemplary hardware configuration of a computer applicable to the image coding apparatus and the image decoding apparatus in the present invention.

Embodiments of the present invention will be described below in detail referring to the attached drawings. Configurations described in the embodiments described below are merely examples, and the present invention is not limited to the illustrated configurations.

First Embodiment

An image coding apparatus according to a first embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a block diagram illustrating the configuration of the image coding apparatus according to the first embodiment.

A block segmentation unit 100 segments original image data received in a block-by-block basis into subblocks having a predetermined size in accordance with the image characteristics, and outputs the subblocks. The received image data has one of three block sizes of 64 x 64 pixels, 32 x 32 pixels, and 16 x 16 pixels. Instead of segmenting a block into subblocks, a received block may be used as a subblock without any segmentation. The minimum size of a subblock is 8 x 8 pixels. Therefore, the size of subblocks which are output from the block segmentation unit 100 is one of the four sizes of 64 x 64 pixels, 32 x 32 pixels, 16 x 16 pixels, and 8 x 8 pixels.

An original image buffer 101 stores the original image data which is obtained through the block segmentation and which is output from the block segmentation unit 100.

A predictive residue generation unit 102 performs intra-frame prediction or inter-frame prediction on the original image data which is obtained through the block segmentation and which is output from the block segmentation unit 100, and outputs predictive residue data. When intra-frame prediction is performed, the predictive residue generation unit 102 generates reference image data on the basis of pixels adjacent to a target block, and calculates the difference between the reference image data and the original image data. When inter-frame prediction is performed, the predictive residue generation unit 102 reads out reference image data from a frame memory 106, and calculates the difference between the reference image data and the original image data. At that time, the subblock described above may be further segmented into prediction blocks. The size of a prediction block is known from NPL 2, and will not be described in detail. The predictive residue generation unit 102 also outputs header data, such as an intra-prediction mode and a motion vector.

A transform/quantization unit 103 transforms a predictive residue which is output from the predictive residue generation unit 102, and quantizes and then outputs coefficient data obtained through the transformation. At that time, the above-described subblock may be further segmented into transform blocks. The size of a transform block is known from NPL 2, and will not be described in detail.

An inverse transform/quantization unit 104 performs inverse quantization and inverse transform on the coefficient data which is output from the transform/quantization unit 103.

For a subblock which has been subjected to inter-frame prediction coding, a localized decoding unit 105 adds the data which is output from the inverse transform/quantization unit 104 to reference image data which is read out from the frame memory 106. For a subblock which has been subjected to intra-frame prediction coding, the localized decoding unit 105 adds the data which is output from the inverse transform/quantization unit 104 to reference image data based on adjacent pixels. The image data decoded through addition is written into the frame memory 106 to be used for coding of subsequent original image data.

An entropy coding unit 107 performs entropy coding on the coefficient data which is output from the transform/quantization unit 103. The entropy coding unit 107 also performs entropy coding on the header data which is output from the predictive residue generation unit 102.

A threshold setting unit 110 derives thresholds used by a data selection unit 108 described below, on the basis of a profile indicating a combination of decoding capabilities of an image decoding apparatus, and sets the thresholds in the data selection unit 108.

The data selection unit 108 includes a buffer (not illustrated) for storing data which has been subjected to entropy coding by the entropy coding unit 107, on a subblock-by-subblock basis. The data selection unit 108 selects data to be output on a subblock-by-subblock basis, from the original image data stored in the original image buffer 101 and the data which has been subjected to entropy coding, on the basis of the thresholds which are set. That is, the data selection unit 108 selects either one of the data which has bypassed entropy coding and the data which has been subjected to entropy coding.

When the data which has bypassed entropy coding (original image data) is selected, the original image data is referred to in predictive coding performed on a subsequent subblock or frame. In this case, the predictive residue generation unit 102 and the localized decoding unit 105 receive the data which is output from the original image buffer 101. A code-string output unit 109 outputs data selected by the data selection unit 108 to the outside.

How to select data for a subblock by using the data selection unit 108 will be described with reference to the flowchart in Fig. 2. In step S101, the data selection unit 108 compares the number of bits of a subblock with a threshold. If the number of bits of the subblock is larger than the threshold (YES in step S101), the data selection unit 108 selects the data which has bypassed entropy coding (original image data) in step S102. Otherwise (NO in step S101), the data selection unit 108 selects the data which has been subjected to entropy coding in step S103.

In HEVC, the range of the size of a subblock for which the data which has bypassed entropy coding (original image data) is selected depends on parameters belonging to the sequence parameter set. When a subblock having a size which is out of the range is to be processed, the data selection unit 108 always selects the data which has been subjected to entropy coding.

An exemplary method for deriving thresholds by using the threshold setting unit 110 will be described with reference to Fig. 3. Fig. 3 is a table indicating thresholds for each of the profiles. A profile is set in advance from the outside in the threshold setting unit 110 before coding of an image is started. In Fig. 3, the column "parallel decoding capability" describes processes which can be performed in parallel, in a combination of the decoding capabilities of an image decoding apparatus.

The profile names, the parallel decoding capabilities, and the thresholds in the present invention are not limited to the specific values illustrated in Fig. 3. For example, the profile names, the parallel decoding capabilities, and the thresholds illustrated in Fig. 4 may be used.

In the baseline profile, processes only for a slice can be performed in parallel. In the main profile, processes for a slice, a tile, or a block line can be performed in parallel. A slice is a portion obtained by segmenting a screen in coding order. A tile is a portion obtained by segmenting a screen into rectangles. A block line is a portion obtained by segmenting a screen into lines having a height of one block.

These can be used at the same time in an image in a hierarchical manner, increasing the concurrency of processes compared with the baseline profile. For example, as illustrated in Fig. 26A, one slice may include multiple tiles. As illustrated in Fig. 26B, one slice may include multiple block lines.

The increase in concurrency decreases processing performance required per entropy decoding unit in an image decoding apparatus compared with the baseline profile. Therefore, a threshold higher than that in the baseline profile is derived. Needless to say, thresholds may be derived regardless of the number of the processes which can be performed in parallel. For example, in Fig. 3, the types of processes which can be performed in parallel are the same in the main profile and the high profile. However, thresholds higher than those in the main profile are derived for the high profile.

As in illustrated in Fig. 5, no thresholds are set in a higher profile such as the high profile. Accordingly, the data selection unit 108 may select any data regardless of the comparison result between a data amount and a threshold.

As described above, in the image coding apparatus according to the first embodiment, a threshold for the data amount for one subblock is derived on the basis of a profile. When a data amount exceeds the threshold, data which has bypassed entropy coding is output.

As in the description of the related art, an image decoding apparatus can obtain data which has bypassed entropy coding, from a code string at a high rate. Therefore, the data can be decoded at a high rate. In contrast, decoding of data which has been subjected to entropy coding depends on the data amount. Accordingly, the more the data amount per subblock is, the longer the decoding processing time is. Therefore, the threshold is changed in accordance with a profile indicating the presence or the absence of processes which can be performed in parallel by an image decoding apparatus, enabling output of a code string optimized for entropy-decoding processing performance of the image decoding apparatus. For example, for an image decoding apparatus which does not support parallel entropy decoding, occurrence of blocks which cause an extreme increase in the decoding processing time is suppressed, preventing dropping of decoded image frames due to insufficient processing performance of an entropy decoding unit. In contrast, for an image decoding apparatus which supports parallel entropy decoding, data which has bypassed entropy coding is prevented from being output into a code string more than necessary, achieving an increase in the compression rate.

The present invention is not limited to the restriction for the data amount per subblock using a profile. For example, the threshold setting unit 110 may derive thresholds from coding levels for defining the number of pixels to be processed per unit time included in a code string as illustrated in Fig. 6. In Fig. 6, the column "the number of pixels per unit time" indicates the number of pixels which can be processed by an image decoding apparatus per second. An image decoding apparatus which supports Level 1 can process 552,960 pixels per second, whereas image decoding apparatuses which support Level 4 and Level 4.1 have to process 62,668,800 pixels per second. The more the number of pixels to be processed per unit time is, the less the processing time per subblock allowed for an entropy decoding unit is. Therefore, thresholds are set in such a manner that a lower data amount per subblock is set to a threshold in a higher level so that occurrence of subblocks which cause an extreme increase in the decoding processing time is suppressed, preventing dropping of decoded image frames due to insufficient processing performance of an entropy decoding unit.

In the case where a profile includes processes which can be performed in parallel, even when the number of pixels to be processed per unit time is increased, processing time per subblock allowed for an entropy decoding unit increases. In this case, the restriction of a threshold for a data amount per subblock may be eased. Accordingly, a threshold may be raised so as to be above that in Level 4, as in Level 4.1 in Fig. 6. Alternatively, no thresholds may be set and the data selection unit 108 may select any data as in Level 4.2.

Second Embodiment

An image coding apparatus according to a second embodiment of the present invention will be described. The overall configuration of the image coding apparatus according to the second embodiment is the same as that of the image coding apparatus according to the first embodiment illustrated in Fig. 1. However, the entropy coding unit 107 has a configuration which is illustrated by the block diagram in Fig. 7. The image coding apparatus according to the second embodiment will be described below with reference to Fig. 7.

An entropy coding controller 201 provides the data type of a coding target (i.e., the type of a syntax element) for a binarization unit 202, an arithmetic coding unit 203, and a code selection unit 204 which are described below.

The binarization unit 202 transforms coefficient data and header data which are received into binary data. Methods for transformation into binary data, which are different depending on data types, are known from NPL 2, and will not be described. The binarization unit 202 also outputs the number of bits of the binary data to the data selection unit 108.

The arithmetic coding unit 203 selects an occurrence probability table in accordance with the data type of a coding target, and performs arithmetic coding on the binary data on the basis of the selected occurrence probability table.

The code selection unit 204 selects either one of the binary data and the data which has been subjected to arithmetic coding, in accordance with the data type, and outputs it to the data selection unit 108.

The method for selecting data on a subblock-by-subblock basis by using the data selection unit 108 according to the second embodiment will be described with reference to Fig. 8. In step S201, the data selection unit 108 accumulates the number of bits of binary data in a subblock. In step S202, the data selection unit 108 compares the number of bits in the subblock with a threshold. If the number of bits of binary data in the subblock is larger than the threshold (YES in step S202), the data selection unit 108 selects data which has bypassed entropy coding (original image data) in step S203. Otherwise (NO in step S202), the data selection unit 108 selects data which has been subjected to entropy coding in S204.

The method for deriving a threshold by using the threshold setting unit 110 according to the second embodiment will be described. Fig. 9 is a table indicating thresholds for the number of bits of binary data for each of the profiles. Similarly to the table in Fig. 3 which is used for description about the image coding apparatus according to the first embodiment, the column "parallel decoding capability" describes processes which can be performed in parallel, in a combination of the decoding capabilities of an image decoding apparatus. Similarly to the first embodiment, according to the second embodiment, a threshold is derived on the basis of a profile. Needless to say, the present invention is not limited to this. No thresholds may be set as in the high profile as in Fig. 10, and the data selection unit 108 may select any data.

The present invention is not limited to this, and thresholds may be derived from coding levels for defining the number of pixels to be processed per unit time included in a code string as illustrated in Fig. 11. The present invention is not limited to the specific thresholds illustrated in Figs. 9. 10, and 11, and other values may be used.

As described above, in the image coding apparatus according to the second embodiment, a threshold for the number of bits of binary data per subblock is derived on the basis of a profile or a level. When the number of bits exceeds the threshold, data which has bypassed entropy coding is output. Typically, arithmetic decoding is performed at a rate such that one bit of binary data is processed in one clock cycle. Therefore, decoding processing time for a subblock depends on the number of bits of binary data in the subblock. Consequently, occurrence of subblocks having binary data which causes an extreme increase in the decoding processing time is suppressed, preventing dropping of decoded image frames due to insufficient processing performance of an entropy decoding unit.

The present invention is not limited to this, and the data selection unit 108 may accumulate only the number of bits of binary data corresponding to the data types for which the code selection unit 204 selects arithmetic coding, in step S201. That is, only the number of bits of binary data which is subjected to arithmetic coding may be accumulated in step S201.

Third Embodiment

An image coding apparatus according to a third embodiment of the present invention will be described. The overall configuration of the image coding apparatus according to the third embodiment is the same as that of the image coding apparatus according to the first embodiment illustrated in Fig. 1. The entropy coding unit 107 has the same configuration as that of the image coding apparatus according to the second embodiment which is illustrated in Fig. 7.

The method for deriving a threshold by using the threshold setting unit 110 is different from those used in the image coding apparatuses according to the first and second embodiments. Instead of using a profile or a level, a value defined in accordance with the size of a subblock is derived. However, the present invention is not limited to this, and a different value may be derived in accordance with a profile or a level.

The method for selecting data on a subblock-by-subblock basis by using the data selection unit 108 is different from that used in the image coding apparatus according to the second embodiment. The method for selecting data on a subblock-by-subblock basis in the image coding apparatus according to the third embodiment will be described with reference to the flowchart in Fig. 12.

In step S301, the data selection unit 108 initializes a variable BinTotal which represents an accumulated value for the number of bits of binary data in a subblock to 0. In step S302, the data selection unit 108 obtains the number of bits, BinSize, for the data type (syntax element) of a coding target.

If the data type corresponding to the obtained BinSize is a data type which is to be subjected to arithmetic coding (YES in step S303), the data selection unit 108 doubles BinSize in step S304. Otherwise (NO in step S303), the process proceeds to step S305. The present invention is not limited to the operation of doubling BinSize. Another way of changing weighting of the number of bits of binary data in accordance with the type of the binary data may be employed. In step S305, BinSize is added to BinTotal.

If entropy coding of the subblock is completed (YES in step S306), the process proceeds to step S307. Otherwise (NO in step S306), the process proceeds to step S302, and the data selection unit 108 obtains the number of bits of binary data for the subsequent data type, and adds the obtained number of bits to BinTotal.

In step S307, the data selection unit 108 compares BinTotal with a threshold. If BinTotal is larger than the threshold (YES in step S307), the data selection unit 108 selects data which has bypassed entropy coding in step S308. Otherwise (NO in step S307), the data selection unit 108 selects data which has been subjected to entropy coding in step S309.

As described above, in the image coding apparatus according to the third embodiment, weighting of the number of bits of binary data is changed in accordance with the type of the binary data, and the number of bits of a subblock is accumulated. Then, the accumulated number of bits is compared with the threshold. The rate of entropy decoding of a code string which is data which has bypassed arithmetic coding is different from that of a code string which is data which has been subjected to arithmetic coding. Thus, by changing weighting of the number of bits of binary data, a code string which can be decoded with stable processing performance by an entropy decoding unit can be generated, preventing dropping of decoded image frames.

Fourth Embodiment

An image coding apparatus according to a fourth embodiment of the present invention will be described. The image coding apparatus according to the fourth embodiment is the same as that according to the second embodiment. However, a threshold derived by the threshold setting unit 110 and a process performed by the data selection unit 108 are different.

As illustrated in Fig. 14, the threshold setting unit 110 derives first thresholds and second thresholds on the basis of the subblock size, and sets them to the data selection unit 108. The process performed by the data selection unit 108 will be described using the flowchart in Fig. 13.

In step S401, the data selection unit 108 accumulates the number of bits of entropy-coded data in a subblock. In step S402, the data selection unit 108 accumulates the number of bits of binary data in the subblock. In step S403, the data selection unit 108 compares the number of bits of entropy-coded data in the subblock with a first threshold. If the number of bits of entropy-coded data in the subblock is larger than the first threshold (YES in step S403), the process proceeds to step S406. Otherwise (NO in step S403), the process proceeds to step S404. In step S404, the data selection unit 108 compares the number of bits of binary data in the subblock with a second threshold. If the number of bits of binary data in the subblock is larger than the second threshold (YES in step S404), the process proceeds to step S406. Otherwise (NO in step S404), the process proceeds to step S405. In step S405, the data selection unit 108 selects entropy-coded data and outputs the selected data to the code-string output unit 109. In step S406, the data selection unit 108 selects data which has bypassed entropy coding (original image data) and outputs the selected data to the code-string output unit 109.

As described above, in the image coding apparatus according to the fourth embodiment, when the number of bits of entropy-coded data in a subblock exceeds a first threshold, data which has bypassed entropy coding is output. When the number of bits of binary data in a subblock exceeds a second threshold, data which has bypassed entropy coding is also output.

In the case where only the number of bits of binary data in a subblock is suppressed, even when processing performance of an image decoding apparatus is satisfied, a code string having a low compression rate may be generated. However, in the image coding apparatus according to the fourth embodiment, the number of bits of entropy-coded data in a subblock is also suppressed in such a manner as to be equal to or less than a threshold, achieving satisfaction of processing performance of an image decoding apparatus and output of a code string having a high compression rate.

In addition, the execution order of step S403 and step S404 may be switched. That is, first, it is determined whether or not the number of bits of binary data in a subblock is larger than a second threshold. If the number of bits of binary data in the subblock is larger than the second threshold, step S406 is executed. Otherwise, it is determined whether or not the number of bits of entropy-coded data in the subblock is larger than a first threshold. If the number of bits of entropy-coded data in the subblock is larger than the first threshold, step S406 is executed. Otherwise, step S405 is executed.

Further, the threshold setting unit 110 may derive first thresholds and second thresholds on the basis of the profiles as illustrated in Fig. 15, and may set them to the data selection unit 108. Furthermore, the threshold setting unit 110 may derive first thresholds and second thresholds on the basis of the levels as illustrated in Fig. 16.

In addition, the present invention is not limited to the specific thresholds illustrated in Figs. 14, 15, and 16, and other values may be used.

The present invention is not limited to this. In step S402, only the number of bits of binary data in an arithmetic coding target may be accumulated. In this case, for example, as illustrated in Fig. 27, a second threshold is set to be smaller than a first threshold, whereby the number of bits of data which is to be subjected to arithmetic decoding and which has low processing performance per clock cycle can be reduced, achieving satisfaction of processing performance of an image decoding apparatus and output of a code string having a high compression rate.

Fifth Embodiment

An image decoding apparatus according to a fifth embodiment of the present invention will be described with reference to Fig. 17. Fig. 17 is a block diagram illustrating the configuration of the image decoding apparatus according to the fifth embodiment. The image decoding apparatus according to the fifth embodiment has a configuration which enables parallel decoding processing and in which two code strings can be decoded in parallel. The present invention is not limited to this. A configuration may be employed in which parallel decoding processing can be performed on more than two code strings, or a configuration may be employed which does not support parallel decoding processing. Specific elements which can be processed in parallel are the same as those described with reference to Figs. 26A and 26B in the first embodiment of the present invention.

A code string buffer 500 temporarily stores a code string which is input from the outside. An overall controller 509 decodes syntax elements representing, for example, a moving image sequence parameter and a picture parameter in the code string stored in the code string buffer 500, and controls other units. As disclosed in NPL 2, syntax elements included in a sequence parameter set which is a set of sequence parameters include a profile and a level.

The flow in which the overall controller 509 decodes a sequence parameter set will be described with reference to the flowchart in Fig. 18.

In step S501, the overall controller 509 decodes one of parameters from the sequence parameter set in a code string stored in the code string buffer 500. The overall controller 509 determines whether or not the parameter is a profile identifier indicating a profile in step S502. If the header is not a profile identifier (NO in step S502), the process proceeds to step S505. If the header is a profile identifier (YES in step S502), the process proceeds to step S503.

In step S503, the overall controller 509 determines whether or not the image decoding apparatus according to the fifth embodiment can decode code strings, on the basis of the profile. Specifically, the overall controller 509 sets a threshold defined for the profile illustrated in Fig. 4 as an upper limit per block for the number of bits of data which has been subjected to entropy coding. If the upper limit exceeds an upper limit for the amount of data which can be processed by

entropy decoding units

502 and 512 described below, the overall controller 509 determines that the profile is one for which decoding cannot be performed (NO in step S503). In this case, the overall controller 509 notifies the outside that the image decoding apparatus according to the fifth embodiment cannot decode the code string in step S504, and the process ends. If it is determined that the profile is one for which decoding can be performed (YES in step S503), the process proceeds to step S505.

In step 505, the overall controller 509 determines whether or not decoding of the sequence parameter set is completed. If decoding of the sequence parameter set is completed (YES in step S505), the process ends. Otherwise (NO in step S505), the process proceeds to step S501.

The present invention is not limited to the profile names, the parallel decoding capabilities, and the specific thresholds illustrated in Fig. 4. For example, those illustrated in Fig. 3 may be used. In addition, as illustrated in Fig. 5, there may be a profile having no thresholds. In this case, if the image decoding apparatus supports the high profile, the determination in step S503 always results in "YES".

After decoding the parameter set having various parameters, the overall controller 509 sets the start addresses of target code strings to be subjected to decoding by the

entropy decoding units

502 and 512, to code-

string input units

501 and 511. Specifically, if the profile indicates that decoding processing can be performed in parallel, the overall controller 509 decodes a syntax element for deriving the start addresses of code strings which can be processed in parallel. The overall controller 509 derives the start addresses of code strings on the basis of the syntax element, calculates memory addresses in the code string buffer 500 which correspond to the start addresses, and sets the memory addresses to the code-

string input units

501 and 511.

The code-

string input units

501 and 511 will be described. The code-

string input units

501 and 511 sequentially read out code strings which are to be decoded, starting from the memory address of the code string buffer 500 which is set by the overall controller 509, and input them to the

entropy decoding units

502 and 512, respectively.

The

entropy decoding units

502 and 512 perform entropy decoding on received code strings. In entropy decoding, specifically, arithmetic decoding and inverse binarization are performed. The internal configuration of the

entropy decoding units

502 and 512 will be described below. Decoding of a subblock performed by the

entropy decoding units

502 and 512 will be described with reference to the flowchart in Fig. 20.

In step S521, an entropy decoding unit decodes a coding type identifier. If the coding type identifier indicates data which has bypassed entropy coding (YES in step S522), the entropy decoding unit reads out image data from the code string, and outputs the read-out image data to a corresponding one of image buffers 506 and 516 in step S523. Then, the process ends. Otherwise (NO in step S522), the process proceeds to step S524. In step S524, the entropy decoding unit decodes header data representing, for example, a motion vector and a prediction mode, and outputs the decoded data to a corresponding one of

predictive decoding units

504 and 514. In step S525, the entropy decoding unit decodes a transformation coefficient, and outputs the decoded coefficient to a corresponding one of inverse transform/

quantization units

503 and 513. Then, the process of decoding a subblock ends.

The inverse transform/

quantization units

503 and 513 perform inverse quantization/transform on a transformation coefficient, and output predictive residue data. The

predictive decoding units

504 and 514 decode the predictive residue data on the basis of the header data. When the prediction mode indicates inter-frame prediction coding, the

predictive decoding units

504 and 514 read out reference data from a frame memory 505, and decode the predictive residue data by referring to the reference data. When the prediction mode indicates intra-frame prediction coding, the

predictive decoding units

504 and 514 decode the predictive residue data by referring to pixels located around the subblock which is to be decoded.

The image buffers 506 and 516 store image data which is output from the

entropy decoding units

502 and 512, respectively. Decoding

controllers

508 and 518 input the coding type identifiers which have been decoded by the

entropy decoding units

502 and 512 to image

output units

507 and 517, respectively. When the coding type indicates data which has bypassed entropy coding, the

image output units

507 and 517 read out image data stored in the image buffers 506 and 516, respectively, and output it to the outside. Otherwise, the

image output units

507 and 517 output image data which has been decoded by the

predictive decoding units

504 and 514, respectively, to the outside.

The internal configuration of the

entropy decoding units

502 and 512 which embody the process described above with reference to Fig. 20 will be described with reference to Fig. 22.

A shifter 5022 locates the start of a received code string. An arithmetic decoding unit 5023 selects an occurrence probability table in accordance with the data type (syntax element type) of a decoding target, performs arithmetic decoding on a code string on the basis of the selected occurrence probability table, and outputs binary data.

An inverse binarization unit 5024 performs inverse binarization on binary data which has been subjected to arithmetic decoding or binary data included in the code string so as to transform it into header data or transformation coefficient data in accordance with the data type of the decoding target. The transformation coefficient is output to a corresponding one of the inverse transform/

quantization units

503 and 513, whereas the header data representing, for example, a motion vector and a prediction mode is output to a corresponding one of the

predictive decoding units

504 and 514.

An image-data reading unit 5025 reads out image data from the shifter 5022 when the coding type indicates data which has bypassed entropy coding. The read-out image data is output to a corresponding one of the image buffers 506 and 516.

An entropy decoding controller 5021 controls the arithmetic decoding unit 5023 and the inverse binarization unit 5024 on the basis of the data type (syntax element) of a current decoding target. A process performed by the entropy decoding controller 5021 will be described.

When the coding type which has been subjected to inverse binarization indicates data which has been subjected to entropy coding, the entropy decoding controller 5021 instructs the arithmetic decoding unit 5023 and the inverse binarization unit 5024 to decode a subsequent code string.

As described above, the image decoding apparatus according to the fifth embodiment derives an upper limit for the data amount per subblock on the basis of a profile, and determines whether or not a received code string can be decoded on the basis of the upper limit. Therefore, the possibility of frames being dropped during decoding due to insufficient processing performance of an entropy decoding unit is reduced.

The present invention is not limited to this. A level identifier for identifying a coding level defining the number of pixels to be processed per unit time which can be decoded may be decoded, and an upper limit per subblock for the number of bits of data which has been subjected to entropy coding may be derived on the basis of the level. This will be described with reference to the flowchart in Fig. 19.

Steps in the flowchart in Fig. 19 are the same as corresponding steps in the flowchart in Fig. 18 except for steps S512 and S513. In step S512, the overall controller 509 determines whether or not the decoded parameter is a level identifier. In step S513, the overall controller 509 determines whether or not the level corresponds to a level for which the image decoding apparatus according to the fifth embodiment can perform decoding. Specifically, the overall controller 509 sets a threshold derived on the basis of the levels illustrated in Fig. 6 as an upper limit per subblock for the number of bits of data which has been subjected to entropy coding, and the overall controller 509 determines whether or not decoding can be performed on the basis of the upper limit.

The present invention is not limited to the levels and the specific thresholds illustrated in Fig. 6. In addition, the

decoding controllers

508 and 518 may receive the number of bits per subblock for data which has been subjected to entropy coding and which is included in a received code string from the

entropy decoding units

502 and 512, and may compare the number of bits with the threshold on a subblock-by-subblock basis. This will be described with reference to the flowchart in Fig. 21.

In Fig. 21, steps S531 to S535 are the same as corresponding steps in Fig. 20. In step S536, if the number of bits of data in the subblock is larger than the threshold (YES in step S536), the entropy decoding unit notifies the outside of this information in step S537, and the process ends. Otherwise (NO in step S536), the process ends without any operations.

Thus, when data which has been subjected to entropy coding and whose amount exceeds the upper limit derived from a profile or a level causes decoding processing time to be increased so that a problem such as dropping of frames arises, a cause of the problem can be easily analyzed.

Sixth Embodiment

An image decoding apparatus according to a sixth embodiment of the present invention will be described. The overall configuration of the image decoding apparatus according to the sixth embodiment is the same as that of the image decoding apparatus according to the fifth embodiment illustrated in Fig. 17, and the configuration of the

entropy decoding units

502 and 512 is the same as that illustrated in Fig. 22.

The process flow performed by the overall controller 509 is the same as that in Fig. 18 or 19, but the definition of a threshold is different. Specifically, the overall controller 509 sets a threshold derived in accordance with a profile or a level illustrated in either of Figs. 9 to 11 as an upper limit per subblock for the number of bits of binary data. When the upper limit for the number of bits of the binary data exceeds a value to which an entropy decoding unit can perform decoding, similarly to step S504 in Fig. 18 and step S514 in Fig. 19, the overall controller 509 notifies the outside that the image decoding apparatus according to the sixth embodiment cannot decode the binary data, and the process ends.

The entropy decoding controller 5021 may accumulate the number of bits of binary data per subblock for data which has been subjected to entropy coding, and may compare the accumulated number with a threshold. Fig. 23 is a flowchart for this example.

Steps in the flowchart in Fig. 23 are the same as corresponding steps in the flowchart in Fig. 20 except for steps S606 and S607. In step S606, the entropy decoding unit compares the number of bits of binary data in a subblock with the threshold. If the number of bits of binary data in the subblock is larger than the threshold (YES in step S606), the entropy decoding unit notifies the outside of this information in step 607, and the process ends. Otherwise (NO in step S606), the process ends without any operations.

Thus, the image decoding apparatus according to the sixth embodiment determines whether or not decoding can be performed on the basis of the number of bits of binary data. As described above, arithmetic decoding is typically performed at a rate such that one bit of binary data is processed in one clock cycle. Therefore, decoding processing time for a subblock depends on the number of bits of binary data of the subblock. Consequently, occurrence of subblocks having a large amount of binary data which causes extreme increase in the decoding processing time can be detected in advance.

Seventh Embodiment

An image decoding apparatus according to a seventh embodiment of the present invention will be described. The overall configuration of the image decoding apparatus according to the seventh embodiment is the same as that illustrated in Fig. 17, and the configuration of an entropy decoding unit is the same as that illustrated in Fig. 22. The process performed by the overall controller 509 is the same as that in the image decoding apparatus according to the sixth embodiment, but the present invention is not limited to this. For example, an upper limit per subblock for the number of bits of binary data may be constant regardless of a profile or a level.

In contrast, the processes performed by the entropy decoding controller 5021 include determination on a subblock-by-subblock basis which is illustrated in the flowchart in Fig. 24, in addition to the control over the units illustrated in Fig. 22. The determination on a subblock-by-subblock basis will be described with reference to Fig. 24.

In step S701, the entropy decoding controller 5021 initializes a variable BinTotal which represents the sum for the number of bits of binary data of a subblock to 0. In step S702, the entropy decoding controller 5021 obtains the number of bits, BinSize, for the data type (syntax element) of a decoding target.

If the data type corresponding to the obtained BinSize is a data type which has been subjected to arithmetic coding (YES in step S703), the entropy decoding controller 5021 doubles BinSize in step S704. Otherwise (NO in step S703), the process proceeds to step S705. The present invention is not limited to the operation of doubling BinSize. Another way of changing weighting of the number of bits of binary data in accordance with the type of the binary data may be employed.

In step S705, the entropy decoding controller 5021 adds BinSize to BinTotal. If entropy decoding of the subblock is completed (YES in step S706), the process proceeds to step S707. Otherwise (NO in step S706), the process proceeds to step S702, and the entropy decoding controller 5021 obtains the number of bits of binary data for the subsequent data type so as to add it to BinTotal.

In step S707, the entropy decoding controller 5021 compares BinTotal with a threshold. If BinTotal is larger than the threshold (YES in step S707), the entropy decoding controller 5021 notifies the outside of this information in step S708, and the process for the subblock ends. Otherwise (NO in step S707), the process for the subblock ends without any operations.

As described above, in the image coding apparatus according to the seventh embodiment, weighting of the number of bits of binary data is changed in accordance with the type of the binary data, and the number of bits of a subblock is accumulated. Then, the accumulated number of bits is compared with the threshold. The rate of entropy decoding of a code string for data which has bypassed arithmetic coding is different from that for data which has been subjected to arithmetic coding. Thus, by changing weighting of the number of bits of binary data, a cause of, for example, dropping of decoded image frames which may occur due to insufficient decoding processing performance can be analyzed.

Eighth Embodiment

An image decoding apparatus according to an eighth embodiment of the present invention will be described. The overall configuration of the image decoding apparatus according to the eighth embodiment is the same as that illustrated in Fig. 17, and the configuration of an entropy decoding unit is the same as that illustrated in Fig. 22. The process flow performed by the overall controller 509 is the same as that in the image decoding apparatus according to the sixth embodiment, but, as illustrated in Fig. 14, there is a difference in that first thresholds and second thresholds are derived.

In contrast, the processes performed by the entropy decoding controller 5021 include determination on a subblock-by-subblock basis which is illustrated in the flowchart in Fig. 25, in addition to the control over the units illustrated in Fig. 22. The determination on a subblock-by-subblock basis will be described with reference to Fig. 25.

In step S801, the entropy decoding controller 5021 accumulates the number of bits of entropy-coded data of a subblock. In step S802, the entropy decoding controller 5021 accumulates the number of bits of binary data of the subblock. In step S803, the entropy decoding controller 5021 compares the number of bits of entropy-coded data of the subblock with a first threshold. If the number of bits of entropy-coded data of the subblock is larger than the first threshold (YES in step S803), the process proceeds to step S804. Otherwise (NO in step S803), the process proceeds to step S805.

In step S804, the entropy decoding controller 5021 notifies the outside that a subblock having the number of bits of entropy-coded data which is larger than the first threshold is detected, and the process ends. In step S805, the entropy decoding controller 5021 compares the number of bits of binary data in the subblock with a second threshold. If the number of bits of binary data in the subblock is larger than the second threshold (YES in step S805), the process proceeds to step S806. Otherwise (NO in step S805), the process ends. In step S806, the entropy decoding controller 5021 notifies the outside that a subblock having the number of bits of binary data which is larger than the second threshold is detected, and the process ends.

As described above, in the image coding apparatus according to the eighth embodiment, when the number of bits of entropy-coded data in a subblock exceeds a first threshold or when the number of bits of binary data in the subblock exceeds a second threshold, notification of this information is transmitted to the outside. Thus, a cause of, for example, dropping of decoded image frames which may occur due to insufficient decoding processing performance can be analyzed.

In addition, the execution order of step S803 and step S805 may be switched. That is, the entropy decoding controller 5021 first determines whether or not the number of bits of binary data in a subblock is larger than a second threshold. If the number of bits of the binary data in the subblock is larger than the second threshold, the entropy decoding controller 5021 notifies the outside of this information, and the process ends. Otherwise, the entropy decoding controller 5021 determines whether or not the number of bits of entropy-coded data in the subblock is larger than a first threshold. If the number of bits of entropy-coded data in the subblock is larger than the first threshold, the entropy decoding controller 5021 notifies the outside of this information, and the process ends.

In addition, the overall controller 509 may derive first thresholds and second thresholds on the basis of the profiles as illustrated in Fig. 15, or may derive first thresholds and second thresholds on the basis of the levels as illustrated in Fig. 16. The present invention is not limited to these specific values. Further, the present invention is not limited to this. In step S802, the entropy decoding controller 5021 may accumulate only the number of bits of binary data to be subjected to arithmetic coding. In this case, for example, thresholds illustrated in Fig. 27 may be used.

Ninth Embodiment

In the embodiments described above, the units illustrated in Figs. 1, 7, 17, and 22 are described assuming that the units are configured with hardware. However, the processes performed by the units illustrated in Figs. 1, 7, 17, and 22 may be configured using computer programs.

Fig. 28 is a block diagram illustrating an exemplary hardware configuration of a computer applicable to an image coding apparatus and an image decoding apparatus according to the above-described embodiments.

A CPU 2801 controls the entire computer by using computer programs and data stored in a random-access memory (RAM) 2802 and a read-only memory (ROM) 2803, and also executes the processes described as those performed by the image coding apparatuses and the image decoding apparatuses according to the above-described embodiments. That is, the CPU 2801 functions as the units illustrated in Figs. 1, 7, 17, and 22.

The RAM 2802 has an area for temporarily storing computer programs and data which are loaded from an external storage apparatus 2806, data obtained from the outside via an interface (I/F) 2807, and the like. Further, the RAM 2802 has a work area used when the CPU 2801 executes various processes. That is, the RAM 2802 may be assigned, for example, as a frame memory, or may be provided as other various types of area as appropriate.

The ROM 2803 stores setting data of the computer, boot programs, and the like. An operation unit 2804 includes a keyboard and a mouse. A user of the computer operates the operation unit 2804, thereby inputting various instructions to the CPU 2801. An output unit 2805 outputs processing results from the CPU 2801. The output unit 2805 may be provided with a display apparatus such as a liquid crystal display so as to display the processing results.

The external storage apparatus 2806 is a large-capacity information storage typified by a hard disk drive. The external storage apparatus 2806 stores the operating system (OS) and computer programs for achieving the functions of the units illustrated in Figs. 1, 7, 17, and 22 by using the CPU 2801. Further, the external storage apparatus 2806 may store images to be processed.

The computer programs and data stored in the external storage apparatus 2806 are loaded to the RAM 2802 as appropriate in accordance with the control performed by the CPU 2801 so as to be processed by the CPU 2801. Networks, such as a local-area network (LAN) and the Internet, and other apparatuses, such as a projector and a display, may be connected to the I/F 2807. The computer can receive/transmit various types of information via the I/F 2807. A bus 2808 connects the above-described units to each other.

Operations described in the above-described flowcharts are performed using the above-described configuration, and are controlled mainly by the CPU 2801.

Other Embodiments

Aspects of the present invention can also be realized by a computer of a system or apparatus (or devices such as a CPU or MPU) that reads out and executes a program recorded on a memory device to perform the functions of the above-described embodiment(s), and by a method, the steps of which are performed by a computer of a system or apparatus by, for example, reading out and executing a program recorded on a memory device to perform the functions of the above-described embodiment(s). For this purpose, the program is provided to the computer for example via a network or from a recording medium of various types serving as the memory device (e.g., computer-readable medium).

While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

This application claims the benefit of Japanese Patent Application No. 2012-144321, filed June 27, 2012, which is hereby incorporated by reference herein in its entirety.

Claims

An image coding apparatus comprising:
a prediction unit configured to segment a received image into a plurality of blocks and to generate a predictive residue from a reference pixel;
a quantization unit configured to transform the predictive residue into a transformation coefficient and to quantize the generated transformation coefficient so as to generate coefficient data;
an entropy coding unit configured to perform entropy coding on the generated coefficient data;
a selection unit configured to select data indicating the image obtained through the segmentation into blocks when a data amount of the coefficient data which has been subjected to entropy coding exceeds a predetermined threshold, and to select the coefficient data which has been subjected to entropy coding when the data amount of the coefficient data does not exceed the predetermined threshold; and
a code-string output unit configured to output the selected data as a code string.
The image coding apparatus according to Claim 1, wherein the predetermined threshold is set by a threshold setting unit configured to set a threshold based on a profile indicating a combination of decoding capabilities allowed in an image decoding method.
The image coding apparatus according to Claim 2, wherein the threshold setting unit derives a threshold on the basis of a size of a block.
The image coding apparatus according to Claim 2, wherein the threshold setting unit derives a threshold for a data amount of a block on the basis of a profile indicating presence or absence of processes in decoding processing, the processes being allowed to be performed as parallel processing.
The image coding apparatus according to Claim 4, wherein a unit processed in the parallel processing is a slice obtained by segmenting a screen in coding order.
The image coding apparatus according to Claim 4, wherein a unit processed in the parallel processing is a tile obtained by segmenting a screen into rectangles.
The image coding apparatus according to Claim 4, wherein a unit processed in the parallel processing is a block line obtained by segmenting a screen into lines having a height of one block.
The image coding apparatus according to Claim 2, wherein the profile includes a combination which enables the selection unit to select any data from the data indicating the image obtained through the segmentation into blocks and the coefficient data which has been subjected to entropy coding, regardless of the comparison result between the threshold and the data amount of the coefficient data which has been subjected to entropy coding on a block-by-block basis.
The image coding apparatus according to Claim 1, wherein the predetermined threshold is set by a threshold setting unit configured to set a threshold on the basis of a coding level defining the number of pixels to be processed per unit time, the pixels being included in the code string.
The image coding apparatus according to Claim 1, wherein the selection unit compares the number of bits of the coefficient data which has been subjected to entropy coding with the predetermined threshold.
The image coding apparatus according to Claim 1, wherein the entropy coding unit includes
a binarization unit configured to transform data to be encoded, into binary data, and
an arithmetic coding unit configured to perform arithmetic coding on the binary data,
wherein the selection unit compares the number of bits of the binary data with the predetermined threshold.
The image coding apparatus according to Claim 11, wherein the entropy coding unit further includes a code selection unit configured to select either one of the data which has been subjected to the arithmetic coding and the binary data, in accordance with a type of data to be encoded, and to output the selected data, and
wherein the selection unit compares the threshold with the number of bits of binary data corresponding to a data type according to which the data which has been subjected to the arithmetic coding is selected.
An image decoding apparatus that decodes a code string corresponding to an image which has been subjected to entropy coding on a block-by-block basis, the apparatus comprising:
a decoding unit configured to decode, from the code string, a profile identifier for identifying a profile indicating a combination of coding processes; and
a determination unit configured to determine, from the code string, a coding type identifier for identifying, on a block-by-block basis, which one among data which has been subjected to entropy coding and data which has not been subjected to entropy coding is included,
wherein the profile is used to derive an upper limit per block for an amount of the data which has been subjected to entropy coding.
The image decoding apparatus according to Claim 13, wherein the amount of the data which has been subjected to entropy coding is represented by the number of bits of the data which has been subjected to entropy coding.
The image decoding apparatus according to Claim 13, wherein the code string is generated by transforming data to be encoded, into binary data and performing arithmetic coding on the binary data, and the amount of the data which has been subjected to entropy coding is represented by the number of bits of the binary data.
The image decoding apparatus according to Claim 13, wherein the upper limit per block for the amount of the data which has been subjected to entropy coding is derived on the basis of a block size.
An image coding method performed in an image coding apparatus, the method comprising:
segmenting a received image into a plurality of blocks and generating a predictive residue from a reference pixel;
transforming the predictive residue into a transformation coefficient and quantizing the generated transformation coefficient so as to generate coefficient data;
performing entropy coding on the generated coefficient data;
selecting data indicating the image obtained through the segmentation into blocks when a data amount of the coefficient data which has been subjected to entropy coding exceeds a predetermined threshold, and selecting the coefficient data which has been subjected to entropy coding when the data amount of the coefficient data does not exceed the predetermined threshold; and
outputting the selected data as a code string.
An image decoding method for decoding a code string corresponding to an image which has been subjected to entropy coding on a block-by-block basis, the method comprising:
decoding, from the code string, a profile identifier for identifying a profile indicating a combination of coding processes; and
determining, from the code string, a coding type identifier for identifying, on a block-by-block basis, which one among data which has been subjected to entropy coding and data which has not been subjected to entropy coding is included,
wherein the profile is used to derive an upper limit per block for an amount of the data which has been subjected to entropy coding.
A recoding medium that records a program which is read out and executed by a computer, the program causing the computer to function as the image coding apparatus according to Claim 1.
A recoding medium that records a program which is read out and executed by a computer, the program causing the computer to function as the image decoding apparatus according to Claim 13.