WO2014002422A1 - Image encoding apparatus, image encoding method and program, image decoding apparatus, and image decoding method and program - Google Patents

Image encoding apparatus, image encoding method and program, image decoding apparatus, and image decoding method and program Download PDF

Info

Publication number
WO2014002422A1
WO2014002422A1 PCT/JP2013/003760 JP2013003760W WO2014002422A1 WO 2014002422 A1 WO2014002422 A1 WO 2014002422A1 JP 2013003760 W JP2013003760 W JP 2013003760W WO 2014002422 A1 WO2014002422 A1 WO 2014002422A1
Authority
WO
Grant status
Application
Patent type
Prior art keywords
coding mode
enhancement layer
image
unit
base layer
Prior art date
Application number
PCT/JP2013/003760
Other languages
French (fr)
Inventor
Mitsuru Maeda
Masato Shima
Original Assignee
Canon Kabushiki Kaisha
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/187Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scalable video layer
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103Selection of coding mode or of prediction mode
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/184Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being bits, e.g. of the compressed video stream
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Abstract

When scalable coding based on bit depth is performed, in regeneration of a pixel of an I_PCM coded block, uncompressed coding of an image including a necessary number of bits from the most significant bit is performed even in an enhancement layer. Thus, the most significant bits in the enhancement layer are the same as those of an image sent in a base layer, and the code becomes redundant. The coding mode of a to-be-encoded block, serving as a target of encoding, of an enhancement layer is obtained. On the basis of the obtained coding mode, the to-be-encoded block of the enhancement layer is encoded. When the obtained coding mode is an uncompressed coding mode, an image including the least significant (m-n) bits is generated from the to-be-encoded block of the enhancement layer, and the generated image is encoded.

Description

IMAGE ENCODING APPARATUS, IMAGE ENCODING METHOD AND PROGRAM, IMAGE DECODING APPARATUS, AND IMAGE DECODING METHOD AND PROGRAM

The present invention relates to an image encoding apparatus, an image encoding method and program, an image decoding apparatus, and an image decoding method and program, and more particularly to an encoding method and a decoding method for non-compressed encoded blocks.

H.264/Moving Picture Experts Group (MPEG)-4 Advanced Video Coding (AVC) (hereinafter referred to as "H.264") is known as an encoding system for compressing and recording moving images (NPL 1). H.264 can use the technology of including an input image as it is into a bit stream, without compressing the pixels of the input image. This technology, which is uncompressed coding, is referred to as intra macroblock pulse code modulation (I_PCM) coding. For example, in the case where the bit depth of a pixel of an image to be encoded is 8 bits and 4:2:0, if the image is subjected to I_PCM coding, 256 luma elements and each 64 chroma elements are included as a bit stream of a macroblock.

H.264 sets I_PCM coding in units of macroblocks. Further, when the encoding amount of a macroblock to be encoded exceeds a predetermined value, H.264 performs I_PCM coding, which is uncompressed coding, instead of encoding the to-be-encoded macroblock again.

Annex G of H.264 discusses extension to scalable coding. Scalable coding encodes to-be-encoded macroblocks of a base layer and an enhancement layer. The macroblocks encoded in the individual layers are multiplexed to generate a bit stream. Encoding a to-be-encoded macroblock of the enhancement layer uses predictive coding that performs prediction based on the to-be-encoded macroblock that has been encoded in the base layer and then encodes the to-be-encoded macroblock of the enhancement layer. Since I_PCM coding does not perform prediction, images are inserted into a bit stream in the order of scanning similarly in the enhancement layer.

Further, an activity for internationally standardizing a high efficiency coding system has been started as a successor to H.264 in recent years. International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) and International Telecommunication Union's Telecommunication Standardization Sector (ITU-T) have established a Joint Collaborative Team on Video Coding (JCT-VC) to develop the High Efficiency Video Coding (hereinafter referred to as "HEVC") standard. NPL 2 describes that I_PCM coding, which is uncompressed coding, can also be used in HEVC.

A bit stream generated using I_PCM coding includes header information and picture data. In HEVC, bit_depth_luma_minus8 code and bit_depth_chroma_minus8 code included in the header information of a bit stream, which is referred to as a seq_parameter_set header, represent the bit depth. The bit depth of a pixel of an image subjected to I_PCM coding (PCM pixel depth information), which is uncompressed coding, is represented by pcm_bit_depth_luma_minus1 code and pcm_bit_depth_chroma_minus1 code. The picture data, which includes encoded images, is subsequent to the header information. The picture data includes a mixture of I_PCM coded pixel data and predictive-coded pixel data.

In contrast, like H.264, HEVC is expected to handle an image with a bit depth of 8 bits or above, as described in NPL 3. To handle an image with a bit depth of 8 bits or above, bit-depth scalable coding may be used in order to be compatible with an image with a bit depth of 8 bits.

In H.264, scalable coding sends encoded data of an I_PCM coded macroblock as it is in any of the layers, as described above. That is, redundancy occurs since the same encoded data is sent in all the layers.

Though not realized in H.264, the same applies to bit-depth scalable coding. For example, out of input images, a layer of an image whose number of bits is less than pixel depth information serves as a base layer, and a layer of an image whose number of bits is greater than that serves as an enhancement layer. If the same configuration as that of H.264 scalable coding is adopted, an image whose number of bits corresponds to the base layer is sent in each of the layers. Therefore, the number of bits corresponding to the base layer is sent with redundancy. If bit-depth scalable coding is performed in multiple enhancement layers, the same image of a lower layer among the enhancement layers is sent, which is redundant.

ITU-T H.264 (03/2010) Advanced video coding for generic audiovisual services JCT-VC contribution JCTVC-H1003 JCT-VC contribution JCTVC-I0108

The present invention provides an image encoding apparatus that performs scalable coding of an image divided in units of blocks by using a plurality of layers including an n-bit base layer and an m-bit enhancement layer, m being greater than or equal to n. The image encoding apparatus includes an enhancement layer coding mode determining unit configured to determine a coding mode of a to-be-encoded block, serving as a target of encoding, of the enhancement layer; and an enhancement layer encoding unit configured to encode the to-be-encoded block of the enhancement layer on the basis of the coding mode determined by the enhancement layer coding mode determining unit. When the coding mode determined by the enhancement layer coding mode determining unit is an uncompressed coding mode, the enhancement layer encoding unit generates an image including the least significant m-n bits from the to-be-encoded block of the enhancement layer, and encodes the generated image in the uncompressed coding mode.

Further, the present invention provides an image decoding apparatus that decodes a bit stream generated by performing scalable coding of an image divided in units of blocks by using a plurality of layers including an n-bit base layer and an m-bit enhancement layer, m being greater than or equal to n. The image decoding apparatus includes an enhancement layer coding mode obtaining unit configured to obtain a coding mode of a to-be-decoded block, serving as a target of decoding, of the enhancement layer; and an enhancement layer decoding unit configured to decode the to-be-decoded block of the enhancement layer on the basis of the coding mode obtained by the enhancement layer coding mode obtaining unit. When the coding mode obtained by the enhancement layer coding mode obtaining unit is an uncompressed coding mode, the enhancement layer decoding unit decodes a bit stream including the least significant m-n bits from the to-be-decoded block of the enhancement layer, and does not decode a bit stream including the most significant n bits.

Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.

According to the present invention, even in I_PCM coding, the coding efficiency can be improved by sending an image corresponding to the number of differential bits with a lower layer, without performing redundant coding and generating redundant code. Decoding can be quickly and productively performed because no encoded data is redundantly decoded.

Fig. 1 is a block diagram illustrating the configuration of an image encoding apparatus according to a first embodiment. Fig. 2 is a block diagram illustrating the details of an enhancement layer image reconstruction unit according to the first embodiment and a second embodiment. Fig. 3 is a flowchart illustrating an image encoding process performed by the image encoding apparatus according to the first embodiment. Fig. 4 is a block diagram illustrating another configuration of the image encoding apparatus according to the first embodiment. Fig. 5 is a block diagram illustrating the configuration of an image decoding apparatus according to the second embodiment. Fig. 6 is a flowchart illustrating an image decoding process performed by the image decoding apparatus according to the second embodiment. Fig. 7 is a block diagram illustrating another configuration of the image decoding apparatus according to the second embodiment. Fig. 8A is a diagram illustrating an example of the configuration of a bit stream generated by the image encoding apparatus according to the present invention. Fig. 8B is a diagram illustrating an example of the configuration of a bit stream generated by the image encoding apparatus according to the present invention. Fig. 9 is a block diagram illustrating the configuration of an image encoding apparatus according to a third embodiment. Fig. 10 is a flowchart illustrating an image encoding process performed by the image encoding apparatus according to the third embodiment. Fig. 11 is a block diagram illustrating the configuration of an image encoding apparatus according to a fourth embodiment. Fig. 12 is a flowchart illustrating an image encoding process performed by the image encoding apparatus according to the fourth embodiment. Fig. 13 is a block diagram illustrating the configuration of an image decoding apparatus according to a fifth embodiment. Fig. 14 is a flowchart illustrating an image decoding process performed by the image decoding apparatus according to the fifth embodiment. Fig. 15A is a diagram illustrating an example of the configuration of a bit stream generated by the image encoding apparatus according to the present invention. Fig. 15B is a diagram illustrating an example of the configuration of a bit stream generated by the image encoding apparatus according to the present invention. Fig. 16 is a block diagram illustrating an example of the hardware configuration of a computer applicable to the image encoding apparatus or the image decoding apparatus according to the present invention.

Hereinafter, the invention of the present application will be described in detail on the basis of exemplary embodiments with reference to the attached drawings. The configurations discussed in the following exemplary embodiments are only exemplary, and the present invention is not limited to the illustrated configurations.

First Embodiment

Hereinafter, a first embodiment of the present invention will be described using Fig. 1. Fig. 1 is a block diagram illustrating the configuration of an image encoding apparatus according to the first embodiment. The image encoding apparatus according to the first embodiment encodes an image in units of blocks.

Referring to Fig. 1, a terminal 100 is an input terminal that inputs an image to the interior of the image encoding apparatus according to the first embodiment. It is assumed that the bit depth of a pixel of an image input to the image encoding apparatus according to the first embodiment is 14 bits. However, the present invention is not limited to this case. Hereinafter, the bit depth of a pixel of an image is referred to as "pixel depth information".

A layer separator 101 divides an image input from the terminal 100 (hereinafter referred to as an "input image") into blocks. Each of the blocks, obtained by division performed by the layer separator 101, has the most significant n bits as an image of a base layer (hereinafter referred to as a "base layer image") and the input image itself (m bits) as an image of an enhancement layer (hereinafter referred to as an "enhancement layer image"). That is, pixel depth information of the base layer image is n, and pixel depth information of the enhancement layer image is m (m is greater than or equal to n). Since pixel depth information of the input image is 14 bits in the first embodiment, pixel depth information of the enhancement layer image is 14 bits (m = 14). The layer separator 101 outputs the base layer image to a base layer coding mode determining unit 102 and the enhancement layer image to an enhancement layer coding mode determining unit 111 in units of blocks.

The base layer coding mode determining unit 102 receives, as an input, the base layer image of a block to be processed (to-be-processed block) and determines the coding mode of the base layer image. Coding modes determined by the base layer coding mode determining unit 102 include, for example, intra predictive coding (hereinafter referred to as "intra coding"), inter predictive coding (hereinafter referred to as "inter coding"), and I_PCM coding. A method of determining the coding mode is not particularly limited. The coding mode may be determined on the basis of, for example, a feature amount of the to-be-processed block, the result of estimation of the amount of coding, or the like.

A selector 103 selects the output destination of the base layer image of the to-be-processed block, on the basis of the output of the base layer coding mode determining unit 102. If the output of the base layer coding mode determining unit 102 is other than the I_PCM coding mode, which is an uncompressed coding mode, the selector 103 selects a prediction unit 104 as the output destination of the base layer image of the to-be-processed block. In contrast, if the output of the base layer coding mode determining unit 102 is the I_PCM coding mode, the selector 103 selects a base layer image reconstruction unit 107 and a base layer second encoding unit 109 as the output destinations of the base layer image of the to-be-processed block.

The prediction unit 104 performs intra prediction or inter prediction based on the output of the base layer coding mode determining unit 102, and calculates a prediction error in units of blocks. If intra coding is selected by the base layer coding mode determining unit 102, the prediction unit 104 performs prediction by referring to the pixel values of surrounding encoded blocks stored in the base layer image reconstruction unit 107. If inter coding is selected by the base layer coding mode determining unit 102, the prediction unit 104 performs prediction by referring to images of different frames stored in the base layer image reconstruction unit 107. The prediction unit 104 calculates a prediction error by comparing a predicted value with an image input from the selector 103.

A transformation/quantization unit 105 calculates a quantization coefficient by performing orthogonal transformation and quantization of the prediction error calculated by the prediction unit 104. At first, the transformation/quantization unit 105 performs orthogonal transformation. After calculating an orthogonal transformation coefficient, the transformation/quantization unit 105 further performs quantization of the orthogonal transformation coefficient, and calculates a quantization coefficient.

A base layer first encoding unit 106 encodes the quantization coefficient calculated by the transformation/quantization unit 105. A method of encoding the quantization coefficient is not particularly limited. Coding such as Golomb coding, arithmetic coding, Huffman coding, or the like may be used.

A base layer coding mode encoding unit 108 encodes the coding mode determined by the base layer coding mode determining unit 102. A method of encoding the coding mode is not particularly limited. Coding such as Golomb coding, arithmetic coding, Huffman coding, or the like may be used.

The base layer second encoding unit 109 generates I_PCM code by performing uncompressed coding of the base layer image input from the layer separator 101 via the selector 103.

The base layer image reconstruction unit 107 receives, as inputs, the base layer image output from the layer separator 101 via the selector 103, the quantization coefficient calculated by the transformation/quantization unit 105, and the coding mode determined by the base layer coding mode determining unit 102. The base layer image reconstruction unit 107 generates an image which has been locally decoded (hereinafter referred to as a "locally decoded image") from the input base layer image, quantization coefficient, and coding mode, and stores the locally decoded image for reference.

A base layer integrating encoding unit 110 generates the code of the base layer image (hereinafter referred to as the "base layer code") by integrating the code output from the base layer coding mode encoding unit 108, the base layer first encoding unit 106, and the base layer second encoding unit 109. The base layer integrating encoding unit 110 further generates and encodes, in units of frames, header information representing that the code is of the base layer image, and adds the header information to the base layer code.

The enhancement layer coding mode determining unit 111 receives, as an input, the enhancement layer image of a to-be-processed block and determines the coding mode of the enhancement layer image. Coding modes determined by the enhancement layer coding mode determining unit 111 include, for example, intra coding, inter coding between enhancement layer images, predictive coding that refers to the base layer image (hereinafter referred to as "inter-layer coding"), and I_PCM coding. To simplify the description, the first embodiment describes only the case in which the coding mode is one of the intra coding mode, the inter-layer coding mode, and the I_PCM coding mode. A method of determining the coding mode is not particularly limited. The coding mode may be determined on the basis of, for example, a feature amount of the to-be-processed block, the result of estimation of the amount of coding, or the like.

A selector 112 selects the output destination of the enhancement layer image of the to-be-processed block, on the basis of the output of the enhancement layer coding mode determining unit 111. If the output of the enhancement layer coding mode determining unit 111 is other than the I_PCM coding mode, the selector 112 selects a prediction unit 113 as the output destination of the enhancement layer image of the to-be-processed block. In contrast, if the output of the enhancement layer coding mode determining unit 111 is the I_PCM coding mode, the selector 112 selects an enhancement layer image reconstruction unit 116 and an enhancement layer dividing unit 118 as the output destinations of the enhancement layer image of the to-be-processed block.

The prediction unit 113 performs intra prediction or inter prediction based on the output of the enhancement layer coding mode determining unit 111, and calculates a prediction error in units of blocks. If inter-layer coding is selected by the enhancement layer coding mode determining unit 111, the prediction unit 113 expands the n-bit base layer image of the base layer image reconstruction unit 107 to m bits, and obtains a prediction error based on the m-bit expanded image as a reference image. A method of calculating a prediction error by the prediction unit 113 is not particularly limited. For example, a predicted value (image) may be obtained by shifting the pixel data of the base layer image, stored in the base layer image reconstruction unit 107, (m-n) bits to the left.

A transformation/quantization unit 114 calculates a quantization coefficient by performing orthogonal transformation and quantization of the prediction error calculated by the prediction unit 113. At first, the transformation/quantization unit 114 performs orthogonal transformation. After calculating an orthogonal transformation coefficient, the transformation/quantization unit 114 further performs quantization of the orthogonal transformation coefficient, and calculates a quantization coefficient.

An enhancement layer first encoding unit 115 encodes the quantization coefficient calculated by the transformation/quantization unit 114. A method of encoding the quantization coefficient is not particularly limited. Coding such as Golomb coding, arithmetic coding, Huffman coding, or the like may be used.

An enhancement layer coding mode encoding unit 117 encodes the coding mode determined by the enhancement layer coding mode determining unit 111. A method of encoding the coding mode is not particularly limited. Coding such as Golomb coding, arithmetic coding, Huffman coding, or the like may be used.

The enhancement layer dividing unit 118 generates an (m-n)-bit image by removing the n-bit base layer image from the m-bit enhancement layer image.

An enhancement layer second encoding unit 119 generates I_PCM code by performing uncompressed coding of the (m-n)-bit image, which is generated by division performed by the enhancement layer dividing unit 118, as an enhancement layer image.

The enhancement layer image reconstruction unit 116 receives, as inputs, the (m-n)-bit enhancement layer image generated by the enhancement layer dividing unit 118, and the base layer image stored for reference in the base layer image reconstruction unit 107. Also, the enhancement layer image reconstruction unit 116 receives, as inputs, the quantization coefficient calculated by the transformation/quantization unit 114 and the coding mode determined by the enhancement layer coding mode determining unit 111, and generates a locally decoded image. Further, the enhancement layer image reconstruction unit 116 stores the generated locally decoded image for reference. The prediction unit 113 refers to the locally decoded image, which is stored in the enhancement layer image reconstruction unit 116, when the coding mode determined by the enhancement layer coding mode determining unit 111 is intra coding or inter coding.

An enhancement layer integrating encoding unit 120 generates the code of the enhancement layer image (hereinafter referred to as the "enhancement layer code") by integrating the code output from the enhancement layer coding mode encoding unit 117, the enhancement layer first encoding unit 115, and the enhancement layer second encoding unit 119. The enhancement layer integrating encoding unit 120 further generates and encodes, in units of frames, header information representing that the code is of the enhancement layer image, and adds the header information to the enhancement layer code.

A layer integrating unit 121 generates header information of the entire sequence. The layer integrating unit 121 generates a bit stream by integrating the base layer code of the base layer integrating encoding unit 110 and the enhancement layer code of the enhancement layer integrating encoding unit 120. If the enhancement layer code is unnecessary at the decoding side, the layer integrating unit 121 may output only the base layer code.

A terminal 122 outputs the bit stream generated by the layer integrating unit 121 to the outside of the image encoding apparatus.

An image encoding method performed by the above-described image encoding apparatus will be described below. Referring to Fig. 1, the image encoding apparatus according to the first embodiment outputs an m-bit image, which is input to the terminal 100, to the layer separator 101. The layer separator 101 divides the most-significant n bits of the m-bit input image into blocks and inputs the blocks to the base layer coding mode determining unit 102 and the selector 103. It is assumed that pixel depth information n of a base layer image is 8 bits. The layer separator 101 also divides the 14-bit input image, input from the terminal 100, into blocks, and inputs the blocks to the enhancement layer coding mode determining unit 111 and the selector 112.

Firstly, encoding the base layer image will be described.

The base layer coding mode determining unit 102 determines the coding mode of a to-be-encoded block from the image input from the layer separator 101. To simplify the description, the first embodiment assumes that the coding mode determined by the base layer coding mode determining unit 102 is one of the following coding modes. That is, the coding mode determined by the base layer coding mode determining unit 102 is any of the intra coding mode using intra prediction, the inter coding mode using inter prediction that performs motion compensation, and the I_PCM coding mode performing uncompressed coding.

The base layer coding mode determining unit 102 outputs the determined coding mode to the selector 103, the prediction unit 104, the transformation/quantization unit 105, the base layer coding mode encoding unit 108, and the base layer image reconstruction unit 107. The base layer coding mode encoding unit 108 generates the code of the coding mode (hereinafter referred to as the "coding mode code"), input from the base layer coding mode determining unit 102, by encoding the coding mode, and outputs the coding mode code to the base layer integrating encoding unit 110.

Firstly, the case in which the base layer coding mode determining unit 102 selects a mode other than the I_PCM coding mode will be described.

Since the output of the base layer coding mode determining unit 102 is a mode other than the I_PCM mode, the selector 103 outputs the base layer image, which is a target of encoding, input from the layer separator 101, to the prediction unit 104.

The prediction unit 104 generates a predicted value by performing prediction based on the output of the base layer coding mode determining unit 102. If the output of the base layer coding mode determining unit 102 is the intra coding mode, the prediction unit 104 generates a predicted value by performing prediction by referring to the pixel values of surrounding encoded blocks stored in the base layer image reconstruction unit 107. Alternatively, if the output of the base layer coding mode determining unit 102 is the inter coding mode, the prediction unit 104 generates a predicted value by performing prediction by referring to images of different frames stored in the base layer image reconstruction unit 107. The prediction unit 104 calculates a prediction error by comparing the predicted value with the image input from the selector 103. The prediction unit 104 outputs the calculated prediction error to the transformation/quantization unit 105.

The transformation/quantization unit 105 calculates a quantization coefficient by performing orthogonal transformation and quantization of the prediction error input from the prediction unit 104. The transformation/quantization unit 105 outputs the calculated quantization coefficient to the base layer first encoding unit 106 and the base layer image reconstruction unit 107.

The base layer image reconstruction unit 107 regenerates an orthogonal transformation coefficient by dequantizing the quantization coefficient input from the transformation/quantization unit 105, and regenerates a prediction error by performing inverse orthogonal transformation of the orthogonal transformation coefficient. The base layer image reconstruction unit 107 obtains a predicted value by referring to the locally decoded image stored in the base layer image reconstruction unit 107 in accordance with the coding mode determined by the base layer coding mode determining unit 102, and obtains a regenerated image by adding the regenerated prediction error to the predicted value. The base layer image reconstruction unit 107 stores the obtained regenerated image in the base layer image reconstruction unit 107 for future reference.

The base layer first encoding unit 106 generates quantization coefficient code by encoding the quantization coefficient input form the transformation/quantization unit 105.

The base layer coding mode encoding unit 108 generates coding mode code by encoding the coding mode determined by the base layer coding mode determining unit 102.

The base layer integrating encoding unit 110 generates the code of the to-be-processed block by integrating the coding mode code generated by the base layer coding mode encoding unit 108, the quantization coefficient code generated by the base layer first encoding unit 106, and information relating to the prediction. In particular, this code will be referred to as "base layer predictive coding pixel information". The base layer predictive coding pixel information is code included in the base layer code.

Next, the case in which the base layer coding mode determining unit 102 selects the I_PCM coding mode will be described.

Since the output of the base layer coding mode determining unit 102 is the I_PCM coding mode, the selector 103 outputs the base layer image, which is a target of encoding, input from the layer separator 101, to the base layer image reconstruction unit 107 and the base layer second encoding unit 109.

The base layer image reconstruction unit 107 generates a regenerated image from the base layer image input from the selector 103. If the I_PCM coding mode is selected by the base layer coding mode determining unit 102, a regenerated image generated by the base layer image reconstruction unit 107 is the same image as that input to the base layer image reconstruction unit 107. Therefore, the base layer image reconstruction unit 107 stores this regenerated image in the base layer image reconstruction unit 107.

The base layer second encoding unit 109 generates I_PCM code by collectively performing uncompressed coding of the base layer image input from the selector 103.

The base layer coding mode encoding unit 108 generates coding mode code by encoding the coding mode determined by the base layer coding mode determining unit 102.

The base layer integrating encoding unit 110 generates the code of the to-be-processed block by integrating the coding mode code generated by the base layer coding mode encoding unit 108 and the I_PCM code generated by the base layer second encoding unit 109. In particular, this code will be referred to as "base layer I_PCM pixel information". The base layer I_PCM pixel information is code included in the base layer code.

Next, encoding the enhancement layer image will be described.

The enhancement layer coding mode determining unit 111 determines the coding mode of a to-be-encoded block from the enhancement layer image input from the layer separator 101. To simplify the description, the first embodiment assumes that the coding mode determined by the enhancement layer coding mode determining unit 111 is one of the following coding modes. That is, the coding mode determined by the enhancement layer coding mode determining unit 111 is any of the intra coding mode using intra prediction, the inter-layer coding mode referring to the base layer image, and the I_PCM coding mode performing uncompressed coding. The enhancement layer coding mode determining unit 111 outputs the determined coding mode to the selector 112, the prediction unit 113, the transformation/quantization unit 114, the enhancement layer coding mode encoding unit 117, and the enhancement layer image reconstruction unit 116.

The enhancement layer coding mode encoding unit 117 generates coding mode code by encoding the coding mode input from the enhancement layer coding mode determining unit 111, and outputs the coding mode code to the enhancement layer integrating encoding unit 120.

Firstly, the case in which the enhancement layer coding mode determining unit 111 selects a mode other than the I_PCM coding mode will be described.

The selector 112 outputs the enhancement layer image, which is a target of encoding, input from the layer separator 101, to the prediction unit 113.

The prediction unit 113 performs prediction based on the output of the enhancement layer coding mode determining unit 111. If the output of the enhancement layer coding mode determining unit 111 is the intra coding mode, the prediction unit 113 performs prediction by referring to the pixel values of surrounding blocks stored in the enhancement layer image reconstruction unit 116. If the output of the enhancement layer coding mode determining unit 111 is the inter-layer coding mode, the prediction unit 113 performs prediction by referring to a corresponding image stored in the base layer image reconstruction unit 107. Here, a 14-bit image is generated by shifting the 8-bit base layer image, stored in the base layer image reconstruction unit 107, 6 bits to the left, and this 14-bit image serves as a predicted image. However, a method of generating a predicted image is not limited to this method. The prediction unit 113 calculates a prediction error by comparing the generated predicted image with the enhancement layer image input from the selector 112. Further, the prediction unit 113 outputs the calculated prediction error to the transformation/quantization unit 114.

The transformation/quantization unit 114 calculates a quantization coefficient by performing orthogonal transformation and quantization of the input prediction error. The transformation/quantization unit 114 outputs the calculated quantization coefficient to the enhancement layer first encoding unit 115 and the enhancement layer image reconstruction unit 116.

Fig. 2 illustrates a detailed configuration of the enhancement layer image reconstruction unit 116. Referring to Fig. 2, a terminal 200 receives, as an input, the n-bit, that is, 8-bit image, to which reference is made, from the base layer image reconstruction unit 107. A terminal 201 receives, as an input, the quantization coefficient of the prediction error from the transformation/quantization unit 114. A terminal 202 receives, as an input, the (m-n)-bit, that is, 6-bit image from the enhancement layer dividing unit 118. A terminal 203 receives, as an input, the coding mode from the enhancement layer coding mode determining unit 111. A shifting unit 204 shifts the pixel value of the image, input from the base layer image reconstruction unit 107 via the terminal 200, 6 bits to the left (in the most significant bit direction). A de-quantization/inverse transformation unit 205 regenerates an orthogonal transformation coefficient by dequantizing the quantization coefficient input from the terminal 201, and regenerates a prediction error by performing inverse orthogonal transformation of the orthogonal transformation coefficient. A selector 206 and a selector 207 select their input destinations in accordance with the coding mode input from the terminal 203. An adder 208 adds (combines) the inputs from the selector 206 and the selector 207 and outputs the added result to a frame memory 209. The frame memory 209 stores a locally decoded image for image reference. A terminal 210 outputs, as an image to be referred to, the locally decoded image, which is stored in the frame memory 209, to the prediction unit 113.

In the above-described configuration, the terminal 203 receives, as an input, the coding mode from the enhancement layer coding mode determining unit 111.

If the coding mode input to the terminal 203 is the intra coding mode, the selector 206 sets the locally decoded image, stored in the frame memory 209, to be output to the adder 208. Also in this case, the selector 207 sets the prediction error, regenerated by the de-quantization/inverse transformation unit 205, to be output to the adder 208. Then, the de-quantization/inverse transformation unit 205 outputs the quantization coefficient of the prediction error, obtained by intra prediction, from the transformation/quantization unit 114 via the terminal 201, and regenerates the input quantization coefficient as a prediction error. The de-quantization/inverse transformation unit 205 outputs the regenerated prediction error to the adder 208 via the selector 207. The selector 206 also outputs the locally decoded image, input from the frame memory 209, to the adder 208. The adder 208 regenerates a locally decoded image by adding the image input from the selector 206 and the prediction error input from the selector 207, and outputs the regenerated locally decoded image to the frame memory 209. The frame memory 209 stores the locally decoded image, regenerated by the adder 208, in a certain region.

If the coding mode input to the terminal 203 is the inter-layer coding mode, the selector 206 sets the image from the shifting unit 204 to be output to the adder 208. Also in this case, the selector 207 sets the prediction error, regenerated by the de-quantization/inverse transformation unit 205, to be output to the adder 208. Then, the de-quantization/inverse transformation unit 205 outputs the quantization coefficient of the prediction error, obtained by inter-layer prediction, from the transformation/quantization unit 114 via the terminal 201, and regenerates the input quantization coefficient as a prediction error. The de-quantization/inverse transformation unit 205 outputs the regenerated prediction error to the adder 208 via the selector 207. Also, the shifting unit 204 receives, as an input, the 8-bit reference image, stored in the base layer image reconstruction unit 107, via the terminal 200. The shifting unit 204 shifts the reference image 6 bits to the left, embeds 0s to the least significant 6 bits, and generates a 14-bit image. The shifting unit 204 further outputs the generated 14-bit image to the adder 208 via the selector 206. The adder 208 regenerates a locally decoded image by adding the image input from the selector 206 and the prediction error input from the selector 207, and outputs the regenerated locally decoded image to the frame memory 209. The frame memory 209 stores the locally decoded image, regenerated by the adder 208, in a certain region. The transformation/quantization unit 114 inputs the calculated quantization coefficient to the enhancement layer first encoding unit 115 and the enhancement layer image reconstruction unit 116.

Referring to Fig. 1, the enhancement layer first encoding unit 115 generates the code of the quantization coefficient calculated by the transformation/quantization unit 114 (hereinafter referred to as the "quantization coefficient code") by encoding the quantization coefficient. The enhancement layer integrating encoding unit 120 integrates the coding mode code generated by the enhancement layer coding mode encoding unit 117, the image quantization coefficient code generated by the enhancement layer first encoding unit 115, information relating to the prediction, and information relating to reference to the base layer image. The enhancement layer integrating encoding unit 120 regards the integrated code and information as the code of the to-be-processed block. The code of the to-be-processed block is referred to as "enhancement layer predictive coding pixel information". The enhancement layer predictive coding pixel information is code included in the enhancement layer code.

Next, the case in which the enhancement layer coding mode determining unit 111 selects the I_PCM coding mode will be described.

The selector 112 outputs the enhancement layer image, which is a target of encoding, input from the layer separator 101, to the enhancement layer dividing unit 118.

The enhancement layer dividing unit 118 deletes the most significant 8 bits from the input 14-bit image, and generates the least significant 6-bit image. The enhancement layer dividing unit 118 outputs the generated 6-bit image to the enhancement layer image reconstruction unit 116 and the enhancement layer second encoding unit 119.

A detailed process performed by the enhancement layer image reconstruction unit 116 when the I_PCM coding mode is selected will be described using Fig. 2. Referring to Fig. 2, the terminal 202 receives, as an input, the 6-bit image from the enhancement layer dividing unit 118. The terminal 203 outputs the I_PCM coding mode from the enhancement layer coding mode determining unit 111 to the selectors 206 and 207. The selectors 206 and 207 select their input destinations in accordance with the coding mode input from the terminal 203. If the I_PCM coding mode is input from the terminal 203, the selector 206 selects the shifting unit 204 as the input destination, and the selector 207 selects the terminal 202 as the input destination.

The shifting unit 204 receives, as an input, the 8-bit reference image, stored in the base layer image reconstruction unit 107, from the base layer image reconstruction unit 107 via the terminal 200. The shifting unit 204 shifts the input image 6 bits to the left, embeds 0s to the least significant 6 bits, and generates a 14-bit image. The selector 206 receives, as an input, the 14-bit image generated by the shifting unit 204, and outputs the 14-bit image to the adder 208. In contrast, the selector 207 receives, as an input, the 6-bit image, generated by the enhancement layer dividing unit 118, via the terminal 202, and outputs the 6-bit image to the adder 208. The adder 208 regenerates a locally decoded image by adding the inputs from the selector 206 and the selector 207, and outputs the locally decoded image to the frame memory 209. The frame memory 209 stores the locally decoded image, regenerated by the adder 208, in a certain region.

Referring to Fig. 1, the enhancement layer second encoding unit 119 generates I_PCM code by integrating the 6-bit image input from the enhancement layer dividing unit 118.

The enhancement layer integrating encoding unit 120 generates the code of the to-be-processed block by integrating the coding mode code generated by the enhancement layer coding mode encoding unit 117 and the I_PCM code generated by the enhancement layer second encoding unit 119. In particular, this code will be referred to as "enhancement layer I_PCM pixel information". The enhancement layer I_PCM pixel information is code included in the enhancement layer code.

At first, the layer integrating unit 121 generates header information of the base layer. Encoded pixel depth information of the base layer, and encoded pixel depth information of the base layer in the I_PCM coding mode are embedded in the header information. Although encoding of these pieces of information is not particularly limited, bit_depth_luma_minus8 code and bit_depth_chroma_minus8 code of H.264 may be used. Further, pcm_bit_depth_luma_minus1 code and pcm_bit_depth_chroma_minus1 code may be used. Subsequent to the header information, the layer integrating unit 121 integrates, as picture data, the base layer predictive coding pixel information and the base layer I_PCM pixel information, which is code input from the base layer integrating encoding unit 110.

Fig. 3 is a flowchart illustrating an image encoding process performed by the image encoding apparatus according to the first embodiment.

In step S300, the image encoding apparatus according to the first embodiment receives, as an input, an image to be encoded, in which m bits are used per pixel, from the terminal 100 illustrated in Fig. 1.

In step S301, the image encoding apparatus according to the first embodiment generates and encodes a header representing the entire sequence. This header includes information representing whether to perform bit-depth scalable coding (bitdepth_enhancement_flag). Further, the image encoding apparatus according to the first embodiment generates and encodes a header representing information of a base layer and a header representing information of an enhancement layer. These headers include pixel depth information of their layers.

In step S302, the layer separator 101 extracts an image of a block to be encoded, from the image input from the terminal 100.

In step S303, the layer separator 101 generates a base layer image by further extracting a pixel including the most significant n bits from the input m-bit pixel.

In step S304, the base layer coding mode determining unit 102 determines the coding mode of the block of the base layer.

In step S305, the selector 103 determines whether the coding mode determined in step S304 is the I_PCM coding mode. If the coding mode is the I_PCM coding mode, the process proceeds to step S306; otherwise, the process proceeds to step S308.

In step S306, the base layer coding mode encoding unit 108 encodes the I_PCM coding mode and outputs the encoded I_PCM coding mode to the base layer integrating encoding unit 110.

In step S307, the base layer second encoding unit 109 performs uncompressed coding of the base layer image and outputs the uncompressed encoded base layer image to the base layer integrating encoding unit 110.

In step S308, the base layer coding mode encoding unit 108 encodes the coding mode determined in step S304, and outputs the encoded coding mode to the base layer integrating encoding unit 110.

In step S309, the prediction unit 104 calculates a prediction error by performing prediction, such as intra prediction or inter prediction, of the base layer image. Further, the transformation/quantization unit 105 generates a quantization coefficient by performing orthogonal transformation and quantization of the prediction error calculated by the prediction unit 104.

In step S310, the base layer first encoding unit 106 encodes coding control information, such as the coding mode or the motion vector necessary for prediction, and encodes and outputs the quantization coefficient, generated in step S309 by the transformation/quantization unit 105.

In step S311, the enhancement layer coding mode determining unit 111 determines the coding mode of a block of the enhancement layer.

In step S312, the selector 112 determines whether the coding mode determined in step S311 by the enhancement layer coding mode determining unit 111 is the I_PCM coding mode. If the coding mode is the I_PCM coding mode, the process proceeds to step S313. If the coding mode is not the I_PCM coding mode, the process proceeds to step S315.

In step S313, the enhancement layer coding mode encoding unit 117 encodes the I_PCM coding mode, and outputs the encoded I_PCM coding mode to the enhancement layer integrating encoding unit 120.

In step S314, the enhancement layer dividing unit 118 extracts the least significant (m-n) bits from the enhancement layer image. The enhancement layer image reconstruction unit 116 performs uncompressed coding of the (m-n)-bit image, extracted by the enhancement layer dividing unit 118, and outputs the encoded (m-n)-bit image.

In step S315, the enhancement layer coding mode encoding unit 117 encodes and outputs the coding mode determined in step S311.

In step S316, the prediction unit 113 calculates a prediction error by performing prediction, such as intra prediction or inter-layer prediction, of the enhancement layer image. Further, the transformation/quantization unit 114 generates a quantization coefficient by performing orthogonal transformation and quantization of the prediction error calculated by the prediction unit 113.

In step S317, the enhancement layer first encoding unit 115 encodes coding control information, such as the coding mode or the motion vector necessary for prediction, and encodes and outputs the quantization coefficient, generated in step S316 by the transformation/quantization unit 114.

In step S318, the image encoding apparatus according to the first embodiment determines whether the process of encoding all the blocks of the input image input from the terminal 100 is completed. If the process on all the blocks of the input image is completed, the encoding process ends. If the process is not completed, the process returns to step S302, and the encoding process continues on the next block as a to-be-processed block.

Figs. 8A and 8B illustrate bit streams generated as described above. Fig. 8A is a diagram representing a bit stream of a base layer. A bit stream in the first embodiment includes header information and picture data. The header information of the base layer includes pixel depth information of the base layer. The picture data of the base layer includes code of each block. The code includes base layer predictive coding pixel information and base layer I_PCM pixel information. Further, the base layer predictive coding pixel information includes coding mode code and quantization coefficient code. The base layer I_PCM pixel information includes coding mode code and I_PCM code.

Fig. 8B is a diagram representing a bit stream of an enhancement layer. The bit stream of the enhancement layer in the first embodiment includes header information and picture data. The header information of the enhancement layer includes an enhancement layer header representing that the bit stream is in the enhancement layer, and pixel depth information of the enhancement layer. Further, the header of the enhancement layer includes an index indicating a reference layer to be referred to, as in H.264.

The picture data of the enhancement layer includes code of each block. The code includes enhancement layer predictive coding pixel information and enhancement layer I_PCM pixel information. Further, the enhancement layer predictive coding pixel information includes coding mode code and quantization coefficient code. Also, the pixel depth information (n) of the base layer can be recognized by looking at the index representing the reference layer, included in the header of the enhancement layer. Since the enhancement layer pixel depth information (m) is known, it can be judged that the enhancement layer I_PCM pixel information includes coding mode code and I_PCM code in which (m-n) bits are used per pixel.

With the above configuration and operation, scalable coding in the I_PCM coding mode can separately encode a base layer image and an enhancement layer image including the least significant bits excluding bits represented in the base layer. At the same time, scalable coding becomes possible even in the I_PCM coding mode. In particular, the number of bits of an image sent in the enhancement layer can be reduced since an image including the least significant bits excluding bits represented in the base layer is encoded. As a result, the coding efficiency is improved.

Although the bit streams are illustrated in Figs. 8A and 8B in the first embodiment, the configuration of a bit stream is not limited to these bit streams.

Although the base layer and one enhancement layer are described in the first embodiment, the number of layers is not limited to this case. For example, a configuration illustrated in Fig. 4 may be adopted.

Fig. 4 illustrates the configuration in which an enhancement layer coding mode determining unit 411 to an enhancement layer integrating encoding unit 420 are added to the image encoding apparatus illustrated in Fig. 1. Referring to Fig. 4, for example, it is assumed that an input image input to the terminal 100 is a 16-bit image, a base layer image is an 8-bit image, an enhancement layer image thereafter is a 14-bit image, and the last enhancement layer image is a 16-bit image. At this time, a layer separator 401 receives, as inputs, images corresponding to these pieces of pixel depth information. That is, the base layer coding mode determining unit 102 to the base layer integrating encoding unit 110 encode the most significant 8-bit image. The enhancement layer coding mode determining unit 111 to the enhancement layer integrating encoding unit 120 encode the most significant 14-bit image. However, the enhancement layer second encoding unit 119 encodes a 6-bit image including the most significant 9-th bit to the 14-th bit. Further, the enhancement layer coding mode determining unit 411 to the integrating encoding unit 420 encode the most significant 16-bit image. However, the enhancement layer second encoding unit 419 encodes a 2-bit image including the most significant 15-th bit and the 16-th bit.

As described above, the number of enhancement layers may be increased as necessary. To encode more enhancement layers, it is clear that adding units from an enhancement layer coding mode determining unit to an enhancement layer integrating encoding unit makes it possible to handle more layers. Pixel depth information in the first embodiment is not limited to that described above. Also, pixel depth information may be individually set for the I_PCM coding mode. Accordingly, it is possible to set, in the I_PCM coding mode, pixel depth information different from that of a block in other coding modes.

Second Embodiment

Hereinafter, a second embodiment of the present invention will be described using Fig. 5. Fig. 5 is a block diagram illustrating the configuration of an image decoding apparatus according to the second embodiment.

Referring to Fig. 5, a terminal 500 is an input terminal that inputs an image to the interior of the image decoding apparatus according to the second embodiment. It is assumed that a bit stream to be decoded in the second embodiment is generated by the image encoding apparatus according to the first embodiment.

A layer code decoder/separator 501 decodes each piece of header information of the bit stream input from the terminal 500, separates the code of individual layers, and outputs the code to a subsequent stage. In the second embodiment, the layer code decoder/separator 501 outputs the code of the individual layers to a base layer decoding section that decodes base layer code and an enhancement layer decoding section that decodes enhancement layer code.

A base layer integrating decoding unit 502 receives, as an input, a bit stream of a base layer from the layer code decoder/separator 501, and decodes the header information of the input bit stream of the base layer. The base layer integrating decoding unit 502 also separates the picture data of the bit stream, in units of blocks, into coding mode code of the base layer, base layer predictive coding pixel information, and base layer I_PCM pixel information. The base layer integrating decoding unit 502 further outputs the separated coding mode code of the base layer to a base layer coding mode decoding unit 503, and the base layer predictive coding pixel information and the base layer I_PCM pixel information to a selector 504.

The base layer coding mode decoding unit (base layer coding mode obtaining unit) 503 generates a coding mode by decoding the coding mode code of the base layer, which is input from the base layer integrating decoding unit 502. The base layer coding mode decoding unit 503 outputs the generated coding mode to a subsequent stage.

The selector 504 selects an output destination of the base layer predictive coding pixel information, on the basis of the output of the base layer coding mode decoding unit 503. If the output of the base layer coding mode decoding unit 503 is other than the I_PCM coding mode, the selector 504 selects a base layer first decoding unit 505 as the output destination. If the output of the base layer coding mode decoding unit 503 is the I_PCM coding mode, the selector 504 selects a base layer second encoding unit 506 as the output destination of the base layer I_PCM pixel information.

The base layer first decoding unit 505 receives, as an input, the base layer predictive coding pixel information separated by the base layer integrating decoding unit 502, via the selector 504, and decodes the base layer predictive coding pixel information. The base layer first decoding unit 505 obtains a quantization coefficient of a prediction error by decoding quantization coefficient code included in the base layer predictive coding pixel information. The base layer first decoding unit 505 further outputs the obtained quantization coefficient to a base layer image reconstruction unit 507.

The base layer second encoding unit 506 receives, as an input, the base layer I_PCM pixel information separated by the base layer integrating decoding unit 502, via the selector 504, and performs uncompressed decoding of the base layer I_PCM pixel information. The base layer second encoding unit 506 outputs, as a base layer image, an n-bit image regenerated by decoding the base layer I_PCM pixel information, to a subsequent stage.

The base layer image reconstruction unit 507 regenerates a prediction error by performing dequantization and inverse orthogonal transformation of the quantization coefficient input from the base layer first decoding unit 505. The base layer image reconstruction unit 507 receives, as inputs, the regenerated prediction error, the base layer image regenerated by the base layer second encoding unit 506, and the coding mode generated by the base layer coding mode decoding unit 503, and regenerates a decoded image as a base layer image.

A terminal 508 outputs the base layer image regenerated by the base layer image reconstruction unit 507 to the outside of the image decoding apparatus.

An enhancement layer integrating decoding unit 510 receives, as an input, a bit stream of an enhancement layer from the layer code decoder/separator 501. The enhancement layer integrating decoding unit 510 separates the picture data of the bit stream, in units of blocks, into coding mode code of the enhancement layer, enhancement layer predictive coding pixel information, and enhancement layer I_PCM pixel information, and outputs the coding mode code of the enhancement layer, the enhancement layer predictive coding pixel information, and the enhancement layer I_PCM pixel information to a subsequent stage.

An enhancement layer coding mode decoding unit (enhancement layer coding mode obtaining unit) 511 generates a coding mode by decoding the coding mode code of the enhancement layer, which is input from the enhancement layer integrating decoding unit 510.

A selector 512 selects an output destination on the basis of the output of the enhancement layer coding mode decoding unit 511. If the output of the enhancement layer coding mode decoding unit 511 is other than the I_PCM coding mode, the selector 512 selects an enhancement layer first decoding unit 513 as the output destination of the enhancement layer predictive coding pixel information. If the output of the enhancement layer coding mode decoding unit 511 is the I_PCM coding mode, the selector 512 selects an enhancement layer second encoding unit 514 as the output destination of the enhancement layer I_PCM pixel information.

The enhancement layer first decoding unit 513 receives, as an input, the enhancement layer predictive coding pixel information separated by the enhancement layer integrating decoding unit 510, via the selector 512, and decodes the enhancement layer predictive coding pixel information. The enhancement layer first decoding unit 513 obtains a quantization coefficient of a prediction error by decoding quantization coefficient code included in the enhancement layer predictive coding pixel information. The enhancement layer first decoding unit 513 further outputs the obtained quantization coefficient to an enhancement layer image reconstruction unit 515.

The enhancement layer second encoding unit 514 receives, as an input, the enhancement layer I_PCM pixel information separated by the enhancement layer integrating decoding unit 510, via the selector 512, and performs uncompressed decoding of the enhancement layer I_PCM pixel information. The enhancement layer second encoding unit 514 outputs, as an enhancement layer image, an (m-n)-bit image regenerated by decoding the enhancement layer I_PCM pixel information, to a subsequent stage.

The enhancement layer image reconstruction unit 515 regenerates a prediction error by performing dequantization and inverse orthogonal transformation of the quantization coefficient input from the enhancement layer first decoding unit 513. The enhancement layer image reconstruction unit 515 receives, as inputs, the regenerated prediction error, the base layer image regenerated by the base layer image reconstruction unit 507, the enhancement layer image regenerated by the enhancement layer second encoding unit 514, and the coding mode generated by the enhancement layer coding mode decoding unit 511. The enhancement layer image reconstruction unit 515 regenerates a decoded image as an enhancement layer image.

A terminal 516 outputs the enhancement layer image regenerated by the enhancement layer image reconstruction unit 515 to the outside of the image decoding apparatus.

An image decoding method performed by the above-described image decoding apparatus will be described below. Referring to Fig. 5, the image decoding apparatus according to the second embodiment outputs a bit stream input to the terminal 500 to the layer code decoder/separator 501. The layer code decoder/separator 501 decodes each piece of header information. The header information includes pixel depth information of a base layer and pixel depth information of an enhancement layer. By decoding these pieces of pixel depth information, the layer code decoder/separator 501 regenerates pixel depth information, which is n bits, of the base layer, and pixel depth information, which is m bits, of the enhancement layer. The layer code decoder/separator 501 further outputs a bit stream of the base layer to the base layer integrating decoding unit 502, and outputs a bit stream of the enhancement layer to the enhancement layer integrating decoding unit 510.

Firstly, decoding base layer code will be described.

The base layer integrating decoding unit 502 decodes the header information of the bit stream of the base layer, which is input from the layer code decoder/separator 501, and divides the picture data of the bit stream into code in units of blocks. The base layer integrating decoding unit 502 further separates coding mode code from the divided code, and outputs the separated coding mode code to the base layer coding mode decoding unit 503. The base layer integrating decoding unit 502 also outputs the remaining quantization coefficient code or I_PCM code to the selector 504.

The base layer coding mode decoding unit 503 generates a coding mode by decoding the coding mode code input from the base layer integrating decoding unit 502. The base layer coding mode decoding unit 503 outputs the generated coding mode to the selector 504, the base layer first decoding unit 505, and the base layer image reconstruction unit 507. To simplify the description, like the first embodiment, the second embodiment assumes that the coding mode generated by the base layer coding mode decoding unit 503 is one of the following decoding modes. That is, the coding mode generated by the base layer coding mode decoding unit 503 is any of the intra coding mode using intra prediction, the inter coding mode using inter prediction that performs motion compensation, and the I_PCM coding mode performing uncompressed coding.

Firstly, the case in which the base layer coding mode decoding unit 503 generates a mode other than the I_PCM decoding mode will be described.

On the basis of the output of the base layer coding mode decoding unit 503, the selector 504 outputs the base layer predictive coding pixel information, input from the base layer integrating decoding unit 502, to the base layer first decoding unit 505.

The base layer first decoding unit 505 receives, as an input, the base layer predictive coding pixel information separated by the base layer integrating decoding unit 502, via the selector 504, and decodes the base layer predictive coding pixel information. The base layer first decoding unit 505 decodes coding control information, such as the coding mode or the motion vector necessary for prediction, included in the input base layer predictive coding pixel information. The base layer first decoding unit 505 generates a quantization coefficient of a prediction error by decoding quantization coefficient code included in the input base layer predictive coding pixel information. The base layer first decoding unit 505 further outputs the generated quantization coefficient to the base layer image reconstruction unit 507.

The base layer image reconstruction unit 507 regenerates a prediction error by performing dequantization and inverse orthogonal transformation of the quantization coefficient input from the base layer first decoding unit 505. In accordance with the coding mode generated by the base layer coding mode decoding unit 503, the base layer image reconstruction unit 507 further obtains a regenerated image from the regenerated prediction error and the decoded image stored in the base layer image reconstruction unit 507. At this time, the base layer image reconstruction unit 507 can generate a regenerated image by referring to the decoded image, stored in the base layer image reconstruction unit 507, as a predicted value, and adding the decoded image serving as a predicted value to the regenerated prediction error. The base layer image reconstruction unit 507 further outputs the generated regenerated image to the outside of the image decoding apparatus via the terminal 508, and stores the regenerated image in the base layer image reconstruction unit 507 for future reference.

Next, the case in which the base layer coding mode decoding unit 503 generates the I_PCM coding mode will be described.

The selector 504 outputs the base layer I_PCM pixel information, which is a target of decoding, input from the base layer integrating decoding unit 502, to the base layer second encoding unit 506.

The base layer second encoding unit 506 regenerates each pixel value from I_PCM code included in the base layer I_PCM pixel information input from the selector 504, and outputs each pixel value to the base layer image reconstruction unit 507.

Since each pixel value input from the base layer second encoding unit 506 is a decoded image itself, the base layer image reconstruction unit 507 outputs each pixel value, input from the base layer second encoding unit 506, as a regenerated image to the outside of the image decoding apparatus via the terminal 508. The base layer image reconstruction unit 507 further stores the regenerated image in the base layer image reconstruction unit 507 for future reference.

Next, decoding enhancement layer code will be described.

The enhancement layer integrating decoding unit 510 decodes the header information of the bit stream of the enhancement layer, which is input from the layer code decoder/separator 501, and divides the picture data of the bit stream into code in units of blocks. The enhancement layer integrating decoding unit 510 separates coding mode code from the divided code, and outputs the separated coding mode code to the enhancement layer coding mode decoding unit 511. The enhancement layer integrating decoding unit 510 also outputs the remaining quantization coefficient code or I_PCM code to the selector 512.

The enhancement layer coding mode decoding unit 511 generates a coding mode by decoding the coding mode code input from the enhancement layer integrating decoding unit 510. The enhancement layer coding mode decoding unit 511 outputs the generated coding mode to the selector 512, the enhancement layer first decoding unit 513, and the enhancement layer image reconstruction unit 515. To simplify the description, like the first embodiment, the second embodiment assumes that the coding mode generated by the enhancement layer coding mode decoding unit 511 is one of the following decoding modes. That is, the coding mode generated by the enhancement layer coding mode decoding unit 511 is any of the intra coding mode using intra prediction, the inter-layer coding mode referring to the base layer image, and the I_PCM coding mode performing uncompressed coding.

Firstly, the case in which the enhancement layer coding mode decoding unit 511 generates a mode other than the I_PCM decoding mode will be described.

On the basis of the output of the enhancement layer coding mode decoding unit 511, the selector 512 outputs the enhancement layer predictive coding pixel information, input from the enhancement layer integrating decoding unit 510, to the enhancement layer first decoding unit 513.

The enhancement layer first decoding unit 513 receives, as an input, the enhancement layer predictive coding pixel information separated by the enhancement layer integrating decoding unit 510, via the selector 512, and decodes the enhancement layer predictive coding pixel information. The enhancement layer first decoding unit 513 decodes coding control information, such as the coding mode code or information relating to reference to the base layer image, necessary for prediction, included in the input enhancement layer predictive coding pixel information. The enhancement layer first decoding unit 513 generates a quantization coefficient of a prediction error by decoding quantization coefficient code included in the input enhancement layer predictive coding pixel information. The enhancement layer first decoding unit 513 further outputs the generated quantization coefficient to the enhancement layer image reconstruction unit 515.

The enhancement layer image reconstruction unit 515 regenerates a prediction error by performing dequantization and inverse orthogonal transformation of the quantization coefficient input from the enhancement layer first decoding unit 513. In accordance with the coding mode input from the enhancement layer coding mode decoding unit 511, the enhancement layer image reconstruction unit 515 further calculates a predicted value. The enhancement layer image reconstruction unit 515 also calculates a predicted value by referring to the decoded image stored in the enhancement layer image reconstruction unit 515 and the base layer image stored in the base layer image reconstruction unit 507. The enhancement layer image reconstruction unit 515 obtains a regenerated image by adding the predicted value calculated as above and the regenerated prediction error. The enhancement layer image reconstruction unit 515 further outputs the generated regenerated image to the outside of the image decoding apparatus via the terminal 516, and stores the regenerated image in the enhancement layer image reconstruction unit 515 for future reference.

A detailed process performed by the enhancement layer image reconstruction unit 515 when the enhancement layer coding mode decoding unit 511 generates a mode other than the I_PCM coding mode will be described using Fig. 2. Referring to Fig. 2, the terminal 200 receives, as an input, an 8-bit image to be referred to, from the base layer image reconstruction unit 507. The terminal 201 receives, as an input, the quantization coefficient of the prediction error from the enhancement layer first decoding unit 513. The terminal 202 receives, as an input, an (m-n)-bit, that is, 6-bit image from the enhancement layer image reconstruction unit 515. The terminal 203 receives, as an input, the coding mode from the enhancement layer coding mode decoding unit 511. The shifting unit 204 shifts the pixel value of the image, input from the base layer image reconstruction unit 507 via the terminal 200, 6 bits to the left. The selector 206 and the selector 207 select their input destinations in accordance with the coding mode input from the enhancement layer coding mode decoding unit 511 via the terminal 203. The adder 208 adds the inputs from the selector 206 and the selector 207 and outputs the added result to the frame memory 209. The frame memory 209 stores therein a locally decoded image in order to refer to the images input from the selector 206 and the selector 207. The terminal 210 outputs the locally decoded image, stored in the frame memory 209, to the outside of the image decoding apparatus via the terminal 516.

In the above-described configuration, the terminal 203 receives, as an input, the coding mode from the enhancement layer coding mode decoding unit 511.

If the coding mode input to the terminal 203 is the intra coding mode, the selector 206 sets the locally decoded image, stored in the frame memory 209, to be output as a predicted image to the adder 208. Also in this case, the selector 207 sets the prediction error, regenerated by the de-quantization/inverse transformation unit 205, to be output to the adder 208. Then, the de-quantization/inverse transformation unit 205 receives, as an input, the quantization coefficient of the prediction error, obtained by intra prediction, from the enhancement layer first decoding unit 513 via the terminal 201, and regenerates the input quantization coefficient as a prediction error. The de-quantization/inverse transformation unit 205 outputs the regenerated prediction error to the adder 208 via the selector 207. The selector 206 also outputs the locally decoded image, input from the frame memory 209, to the adder 208. The adder 208 generates a decoded image by adding the locally decoded image input from the selector 206 and the prediction error input from the selector 207, and stores the decoded image in a certain region of the frame memory 209. The frame memory 209 further outputs the decoded image, generated by the adder 208, to the outside of the image decoding apparatus via the terminal 210 and further via the terminal 516 illustrated in Fig. 5.

If the coding mode input to the terminal 203 is the inter-layer coding mode, the selector 206 receives, as an input, the image from the shifting unit 204 and sets the image to be output to the adder 208. Also in this case, the selector 207 sets the prediction error, regenerated by the de-quantization/inverse transformation unit 205, to be output to the adder 208. Then, the de-quantization/inverse transformation unit 205 receives, as an input, the quantization coefficient of the prediction error, obtained by inter-layer prediction, from the enhancement layer first decoding unit 513 via the terminal 201, and regenerates the input quantization coefficient as a prediction error. The de-quantization/inverse transformation unit 205 outputs the regenerated prediction error to the adder 208 via the selector 207. Also, the shifting unit 204 receives, as an input, the 8-bit reference image, stored in the base layer image reconstruction unit 507 illustrated in Fig. 5, via the terminal 200. The shifting unit 204 shifts the input reference image 6 bits to the left, embeds 0s to the least significant 6 bits, and generates a 14-bit image. The shifting unit 204 further outputs the generated 14-bit image to the adder 208 via the selector 206. The adder 208 regenerates a locally decoded image by adding the image input from the selector 206 and the prediction error input from the selector 207, and outputs the regenerated locally decoded image to the frame memory 209. The frame memory 209 stores the locally decoded image, regenerated by the adder 208, in a certain region. The frame memory 209 further outputs the decoded image, regenerated by the adder 208, to the outside of the image decoding apparatus via the terminal 210 and further via the terminal 516 illustrated in Fig. 5.

Next, the case in which the enhancement layer coding mode decoding unit 511 generates the I_PCM coding mode will be described.

On the basis of the output of the enhancement layer coding mode decoding unit 511, the selector 512 outputs the enhancement layer I_PCM pixel information, input from the enhancement layer integrating decoding unit 510, to the enhancement layer second encoding unit 514.

The enhancement layer second encoding unit 514 decodes the I_PCM code included in the enhancement layer I_PCM pixel information input from the selector 512, regenerates the (m-n)-bit image as an enhancement layer image, and outputs the enhancement layer image to the enhancement layer image reconstruction unit 515.

The enhancement layer image reconstruction unit 515 regenerates a decoded image on the basis of the coding mode input from the enhancement layer coding mode decoding unit 511. The enhancement layer image reconstruction unit 515 also regenerates a decoded image by referring to the 6-bit image input from the enhancement layer second encoding unit 514 and the 8-bit base layer image stored in the base layer image reconstruction unit 507. The enhancement layer image reconstruction unit 515 further outputs the regenerated decoded image to the outside of the image decoding apparatus via the terminal 516, and stores the decoded image in the enhancement layer image reconstruction unit 515 for future reference.

A detailed process performed by the enhancement layer image reconstruction unit 515 when the enhancement layer coding mode decoding unit 511 generates the I_PCM coding mode will be described using Fig. 2. Referring to Fig. 2, the terminal 202 receives, as an input, a 6-bit image from the enhancement layer second encoding unit 514. The terminal 203 receives, as an input, the I_PCM coding mode from the enhancement layer coding mode decoding unit 511. At this time, the selector 206 selects its input destination as the shifting unit 204, and the selector 207 selects its input destination as the terminal 202. The shifting unit 204 receives, as an input, the 8-bit reference image, stored in the base layer image reconstruction unit 507, via the terminal 200. The shifting unit 204 shifts the input 8-bit image 6 bits to the left, embeds 0s to the least significant 6 bits, and generates a 14-bit image. The shifting unit 204 further outputs the generated 14-bit image to the adder 208 via the selector 206. In contrast, the selector 207 receives, as an input, the 6-bit image, regenerated by the enhancement layer second encoding unit 514, via the terminal 202, and outputs the image to the adder 208. The adder 208 regenerates a decoded image by adding the inputs from the selector 206 and the selector 207, and outputs the decoded image to the frame memory 209. The frame memory 209 stores the decoded image, input from the adder 208, in a certain region, and outputs the regenerated decoded image to the outside of the image decoding apparatus via the terminal 210 and the terminal 516.

Fig. 6 is a flowchart illustrating an image decoding process performed by the image decoding apparatus according to the second embodiment.

In step S600, the image decoding apparatus according to the second embodiment receives, as an input, a bit stream to be decoded, from the terminal 500 illustrated in Fig. 5.

In step S601, the image decoding apparatus according to the second embodiment decodes a header representing the entire sequence, and obtains information indicating whether the bit stream input from the terminal 500 has been subjected to bit-depth scalable coding (bitdepth_enhancement_flag). If this information is 1, it is indicated that an enhancement layer is an enhancement layer of bit-depth layering. If this information is 0, it is indicated that an enhancement layer is an enhancement layer of other layering. Further, the image decoding apparatus according to the second embodiment decodes a header representing information of the base layer and a header representing information of the enhancement layer, and regenerates pieces of pixel depth information indicting the bit depth of the individual layers. Here, the regenerated pixel depth information of the base layer is base_layer_bit_depth, and the regenerated pixel depth information of the enhancement layer is enhancement_layer_bit_depth.

In step S602, the layer code decoder/separator 501 extracts code relating to a to-be-decoded block of the base layer from the bit stream, and outputs the code to the base layer integrating decoding unit 502.

In step S603, the base layer integrating decoding unit 502 extracts coding mode code of the block, from the code relating to the block of the base layer, input from the layer code decoder/separator 501. The base layer integrating decoding unit 502 generates a coding mode by decoding the extracted coding mode code.

In step S604, the selector 504 determines whether the coding mode of the block of the base layer, generated in step S603, is the I_PCM coding mode. If the coding mode of the block is the I_PCM coding mode, the process proceeds to step S605; otherwise, the process proceeds to step S606.

In step S605, the base layer second encoding unit 506 extracts the I_PCM code subsequent to the coding mode code, from the code relating to the block of the base layer, input from the base layer integrating decoding unit 502 via the selector 504, and regenerates the value of each n-bit pixel.

In step S606, the base layer first decoding unit 505 decodes coding control information, such as the coding mode or the motion vector necessary for prediction, decodes the subsequent quantization coefficient code, and regenerates a quantization coefficient of a prediction error.

In step S607, the base layer image reconstruction unit 507 regenerates a prediction error by performing dequantization and inverse orthogonal transformation of the quantization coefficient regenerated in step S606.

In step S608, the base layer image reconstruction unit 507 obtains the pixel value regenerated in step S605 as a regenerated image. Alternatively, the base layer image reconstruction unit 507 regenerates a pixel value by adding the prediction error regenerated in step S607 to a predicted value generated using the coding information regenerated in step S606, and obtains this pixel value as a regenerated image. The base layer image reconstruction unit 507 stores the regenerated pixel value as a regenerated image.

In step S609, the layer code decoder/separator 501 extracts code relating to a to-be-decoded block of the enhancement layer from the bit stream, and outputs the code to the enhancement layer integrating decoding unit 510.

In step S610, the enhancement layer integrating decoding unit 510 extracts coding mode code of the block of the enhancement layer, from the code relating to the block, input from the layer code decoder/separator 501. The enhancement layer integrating decoding unit 510 generates a coding mode by decoding the extracted coding mode code.

In step S611, the selector 512 determines whether the coding mode of the block of the enhancement layer, generated in step S610, is the I_PCM coding mode. If the coding mode of the block is the I_PCM coding mode, the process proceeds to step S612; otherwise, the process proceeds to step S613.

In step S612, the enhancement layer second encoding unit 514 extracts the I_PCM code subsequent to the coding mode code, from the code relating to the block of the enhancement layer, input from the enhancement layer integrating decoding unit 510 via the selector 512, and regenerates the value of each (m-n)-bit pixel.

In step S613, the enhancement layer first decoding unit 513 decodes coding control information, such as the coding mode or the motion vector necessary for prediction, decodes the subsequent quantization coefficient code, and regenerates a quantization coefficient of a prediction error.

In step S614, the enhancement layer image reconstruction unit 515 regenerates a prediction error by performing dequantization and inverse orthogonal transformation of the quantization coefficient regenerated in step S613.

In step S615, the enhancement layer image reconstruction unit 515 regenerates a pixel value by adding a pixel value obtained by shifting the n-bit pixel value of the base layer, which is at the same pixel position as the (m-n)-bit pixel value regenerated in step S612, (m-n) bits to the left. Alternatively, the enhancement layer image reconstruction unit 515 regenerates a pixel value by adding the prediction error regenerated in step S614 to a predicted value generated using the coding information regenerated in step S613. The enhancement layer image reconstruction unit 515 stores the regenerated pixel value as a regenerated image.

In step S616, the image decoding apparatus according to the second embodiment determines whether the process of decoding all the blocks of the bit stream input from the terminal 500 is completed. If the process on all the blocks of the input bit stream is completed, the decoding process ends. If the process is not completed, the process returns to step S602, and the decoding process continues on the next block as a to-be-processed block.

Table 1 describes the operation of the image decoding apparatus according to the above-described second embodiment in accordance with the format of the H.264 standard.

Figure JPOXMLDOC01-appb-T000001

In Table 1, macroblock_layer_in_scalable_extension( ) represents decoding of a block of the enhancement layer image. InCropWindow( CurrMbAddr ) indicates that a to-be-decoded block indicated by the address CurrMbAddr is inside a decoded image. adaptive_base_mode_flag is a flag described in "G.7.4.3.4 Slice header in scalable extension semantics" of NPL 1. base_mode_flag is described in "G.7.4.6 Macroblock layer in scalable extension semantics" of NPL 1. This flag is a flag that represents whether a block of the enhancement layer image and a block of the base layer image have the same coding mode or the like. With pcm_alignment_zero_bit, a bit that equals 0 is filled up to the byte boundary.

If bitdepth_enhancmenet_flag is 0, the 10th to 13th lines of macroblock_layer_in_scalable_extension( ) represent decoding of an m-bit I_PCM coded image represented by the enhancement layer. That is, pcm_sample_luma[ i ] does not represent I_PCM code of the number of differential bits, but represents I_PCM code of the m-bit luma. From this I_PCM code, the pixel value of luma of the m-bit base layer image, corresponding to the block, is regenerated. Similarly, pcm_sample_chroma[ i ] represents I_PCM code of chroma. From this I_PCM code, the m-bit pixel value of chroma, corresponding to the block, is regenerated.

If bitdepth_enhancmenet_flag is 1, the 15th to 22nd lines of macroblock_layer_in_scalable_extension( ) represent decoding of a differential-number-of-bit I_PCM coded image of the enhancement layer. pcm_sample_luma_value represents I_PCM code of luma of the enhancement layer image. From this I_PCM code, the (m-n)-bit pixel value of luma, corresponding to the block, is regenerated. Similarly, pcm_sample_chroma_value represents I_PCM code of chroma. From this I_PCM code, the (m-n)-bit pixel value of chroma, corresponding to the block, is regenerated. This (m-n) is represented as number_of_bit_depth. Accordingly, regarding the regenerated pixel value of the enhancement layer image, the luma is calculated by the following equation.

pcm_sample_luma[ i ] = (pcm_sample_luma[ i ] << number_of_bit_depth) + pcm_sample_luma_value ...(1)
where << represents an arithmetic operation that shifts bits the number of bits indicated by the value thereafter to the left and fills 0s. Similarly, the chroma is calculated by the following equation.

pcm_sample_chroma[ i ] = (pcm_sample_chroma[ i ] << number_of_bit_depth) + pcm_sample_chroma_value ...(2)
In this manner, the image decoding apparatus according to the second embodiment regenerates a decoded image of the enhancement layer.

Although Table 1 uses bitdepth_enhamcnent_flag, this may be omitted. Table 2 indicates that case. Table 2 is different from Table 1 in the 9th line. That is, in Table 2, determination is performed by comparing base_layer_bit_depth and enhancement_layer_bit_depth included in header information. In Table 2, if enhancement_layer_bit_depth is greater than base_layer_bit_depth, the enhancement layer image is code of the number of differential bits.

Figure JPOXMLDOC01-appb-T000002

With the above configuration and operation, decoding code that has been subjected to scalable coding (hereinafter referred to as "scalable code") in the I_PCM coding mode makes it possible to decode code in which the code is separated into the base layer image and an image including the least significant bits excluding bits represented in the base layer and these images are individually encoded. Further, layered decoding of the scalable code becomes possible also in the I_PCM coding mode which performs uncompressed coding. Also, even in a bit stream in which the number of bits of an image is reduced by encoding an image including bits lower in the order than the base layer, the same image quality as that in the case where the number of bits is not reduced can be achieved.

Although the base layer and one enhancement layer are described in the second embodiment, the number of layers is not limited to this case. For example, a configuration illustrated in Fig. 7 may be adopted. An image decoding apparatus illustrated in Fig. 7 is a configuration obtained by adding an enhancement layer integrating decoding unit 710 to a terminal 716 to the image decoding apparatus illustrated in Fig. 5.

For example, it is assumed that a bit stream input to the image decoding apparatus uses 16 bits per pixel value at maximum, a base layer uses 8 bits, an enhancement layer thereafter uses 14 bits, and the last enhancement layer uses 16 bits. At this time, the layer code decoder/separator 501 inputs the bit stream with these layers to respective layers. The base layer integrating decoding unit 502 and onward decode the most significant 8 bit-image. The enhancement layer integrating decoding unit 510 and onward decode the most significant 14-bit image. At this time, the enhancement layer second encoding unit 514 decodes a 6-bit image including the most significant 9-th bit to the 14-th bit. At last, the enhancement layer integrating decoding unit 710 and onward decode the most significant 16-bit image. At this time, the enhancement layer second encoding unit 714 decodes a 2-bit image including the most significant 15-th bit and the 16-th bit.

As described above, it is clear that adding units from an enhancement layer integrating decoding unit to an enhancement layer image reconstruction unit makes it possible to handle more layers. Also, the bit depths in the second embodiment are not limited to those described above. Also, the second embodiment is not limited to the encoding and decoding formats described in Tables 1 and 2.

Although pixel depth information in the I_PCM coding mode is pixel depth information in which the luma is the same as the chroma, the luma and the chroma may take different values, as described above.

Although it has been described that processing from the 10th to 13th lines is performed when base_layer_bit_depth is equal to enhancement_layer_bit_depth in the 9th line of Table 2, the second embodiment is not limited to this case. For example, when base_layer_bit_depth is equal to enhancement_layer_bit_depth, the I_PCM code may be omitted, and the value of the base layer image may be used as it is.

Third Embodiment

Hereinafter, a third embodiment of the present invention will be described using Fig. 9. Fig. 9 is a block diagram illustrating the configuration of an image encoding apparatus according to the third embodiment.

In Fig. 9, blocks that serve the same functions as those illustrated in Fig. 1 according to the first embodiment are given the same numerals, and descriptions thereof are omitted.

An enhancement layer coding mode determining unit 911 is different from the enhancement layer coding mode determining unit 111 illustrated in Fig. 1 in the point that the enhancement layer encoding mode determining unit 911 receives, as an input, the result of the base layer coding mode determining unit 102.

An image encoding operation performed by the image encoding apparatus will be described below. As in the first embodiment, the base layer coding mode determining unit 102 determines the coding mode of a to-be-encoded block. Also in the third embodiment, the coding mode determined by the base layer coding mode determining unit 102 is any of the intra coding mode, the inter coding mode, and the I_PCM coding mode. However, the present invention is not limited to this case. The coding mode determined by the base layer coding mode determining unit 102 is input to the enhancement layer coding mode determining unit 911, besides the output destinations in the first embodiment. Thereafter, a base layer image is encoded, as in the first embodiment.

Next, encoding of an enhancement layer image will be described. The enhancement layer coding mode determining unit 911 determines the coding mode of a to-be-encoded block of an enhancement layer, on the basis of the coding mode of the base layer of an input image input to the image encoding apparatus according to the third embodiment. To simplify the description, as in the first embodiment, the third embodiment assumes that the coding mode is any of the intra coding mode, the inter coding mode, the inter-layer coding mode, and the I_PCM coding mode. If the output from the base layer coding mode determining unit 102 is the I_PCM coding mode, the enhancement layer coding mode determining unit 911 determines the I_PCM coding mode as the coding mode of the to-be-encoded block of the enhancement layer image. If the output from the base layer coding mode determining unit 102 is a mode other than the I_PCM coding mode, the enhancement layer coding mode determining unit 911 determines the coding mode of the enhancement layer on the basis of, for example, a feature amount of the block of the enhancement layer image, the result of estimation of the amount of coding, or the like.

Further, as in the enhancement layer coding mode determining unit 111 of the first embodiment, the enhancement layer coding mode determining unit 911 outputs the determined coding mode to a subsequent stage. That is, the enhancement layer coding mode determining unit 911 outputs the determined coding mode to the selector 112, the prediction unit 113, the transformation/quantization unit 114, the enhancement layer coding mode encoding unit 117, and the enhancement layer image reconstruction unit 116.

The enhancement layer coding mode encoding unit 117 generates coding mode code by encoding the coding mode input from the enhancement layer coding mode determining unit 911, and outputs the coding mode code to the enhancement layer integrating encoding unit 120. Thereafter, as in the first embodiment, the enhancement layer image is encoded in accordance with the coding mode determined by the enhancement layer coding mode determining unit 911.

Fig. 10 is an image encoding process performed by the image encoding apparatus according to the third embodiment. In Fig. 10, steps that serve the same functions as those illustrated in Fig. 3 according to the first embodiment are given the same numerals, and descriptions thereof are omitted.

In steps S300 and S301, an image is input to the image encoding apparatus according to the third embodiment, and header information of the input image is encoded. Next, in steps S302 to S310, a block of a base layer image is encoded.

In step S1001, the enhancement layer coding mode determining unit 911 determines whether the coding mode of block of the base layer image, determined in step S304, is the I_PCM coding mode. If the coding mode of the block of the base layer image is the I_PCM coding mode, the process proceeds to step S313; otherwise, the process proceeds to step S311.

Thereafter, in steps S311 to S317, a block of an enhancement layer image is encoded.

In particular, the third embodiment is characteristic in the point that, if the coding mode determined by the base layer coding mode determining unit 102 is referred to and the coding mode is the I_PCM coding mode in step S1001, the I_PCM coding mode is similarly regarded as the coding mode of the enhancement layer.

The configuration of a bit stream generated by the image encoding apparatus according to the third embodiment is the same as that illustrated in Figs. 8A and 8B. However, the contents of the coding mode code in the enhancement layer I_PCM pixel information are different. Therefore, a bit stream generated by the image encoding apparatus according to the third embodiment can be decoded by the image decoding apparatus described in the second embodiment. In this case, referring to Fig. 6, step S611 proceeds to step S612 if the coding mode of the block of the base layer image is the I_PCM coding mode in step S604.

With the above configuration and operation, scalable coding can facilitate the determination of the coding mode of the enhancement layer by determining the coding mode of the enhancement layer as the I_PCM coding mode in the case where the coding mode of the base layer is the I_PCM coding mode. As a result, the speed of processing can be increased. In general, the I_PCM coding mode is used in an image in which improvement of the coding efficiency is unrealizable in predictive coding. For example, a white noise image is conceivable. That is, the I_PCM coding mode is selected in the base layer because normal encoding is impossible, and it is highly likely that the same applies to the enhancement layer including the least significant bits of the base layer. The third embodiment is based on this idea.

Fourth Embodiment

Hereinafter, a fourth embodiment of the present invention will be described using Fig. 11. Fig. 11 is a block diagram illustrating the configuration of an image encoding apparatus according to the fourth embodiment.

In Fig. 11, blocks that serve the same functions as those illustrated in Fig. 9 according to the third embodiment are given the same numerals, and descriptions thereof are omitted.

An enhancement layer coding mode encoding unit 1117 encodes the coding mode determined by the enhancement layer coding mode determining unit 911. The enhancement layer coding mode encoding unit 1117 is different from the enhancement layer coding mode encoding unit 117 illustrated in Fig. 9 in the point that the enhancement layer coding mode encoding unit 1117 receives, as an input, the result of the base layer coding mode determining unit 102.

An image encoding operation performed by the image encoding apparatus will be described below. As in the first and third embodiments, the base layer coding mode determining unit 102 illustrated in Fig. 11 determines the coding mode of a to-be-encoded block. Also in the fourth embodiment, the coding mode determined by the base layer coding mode determining unit 102 is one of the intra coding mode, the inter coding mode, and the I_PCM coding mode. However, the fourth embodiment is not limited to this case. Also, the coding mode determined by the base layer coding mode determining unit 102 is input to the enhancement layer coding mode determining unit 911 and the enhancement layer coding mode encoding unit 1117, besides the output destinations in the first embodiment. Thereafter, as in the first and third embodiments, a base layer image is encoded. Next, encoding of an enhancement layer image will be described.

The enhancement layer coding mode determining unit 911 determines the coding mode of a to-be-encoded block of an enhancement layer, on the basis of the coding mode of the base layer of an input image input to the image encoding apparatus according to the fourth embodiment, as in the third embodiment. To simplify the description, as in the first embodiment, the fourth embodiment assumes that the coding mode is any of the intra coding mode, the inter-layer coding mode, and the I_PCM coding mode. If the output from the base layer coding mode determining unit 102 is the I_PCM coding mode, the enhancement layer coding mode determining unit 911 determines the I_PCM coding mode as the coding mode of the to-be-encoded block of the enhancement layer image. If the output from the base layer coding mode determining unit 102 is a mode other than the I_PCM coding mode, the enhancement layer coding mode determining unit 911 determines the coding mode of the enhancement layer on the basis of, for example, a feature amount of the block of the enhancement layer image, the result of estimation of the amount of coding, or the like.

The enhancement layer coding mode encoding unit 1117 receives, as inputs, the coding mode of the enhancement layer, determined by the enhancement layer coding mode determining unit 911, and the coding mode of the base layer, determined by the base layer coding mode determining unit 102. If the input coding mode of the base layer is a mode other than the I_PCM coding mode, the enhancement layer coding mode encoding unit 1117 generates coding mode code by encoding these coding modes, as in the first and third embodiments. The enhancement layer coding mode encoding unit 1117 outputs the generated coding mode code to the enhancement layer integrating encoding unit 120. If the input coding mode of the base layer is the I_PCM coding mode, the enhancement layer coding mode encoding unit 1117 regards that the coding mode of the enhancement layer is the I_PCM coding mode, and does not encode the coding mode. Also, the enhancement layer coding mode encoding unit 1117 does not encode base_mode_flag described in Tables 1 and 2. Thereafter, the enhancement layer image is encoded in accordance with the coding mode, as in the third embodiment.

Fig. 12 is a flowchart illustrating an image encoding process performed by the image encoding apparatus according to the fourth embodiment. In Fig. 12, steps that serve the same functions as those illustrated in Fig. 3 according to the first embodiment are given the same numerals, and descriptions thereof are omitted.

In steps S300 and S301, the image encoding apparatus according to the fourth embodiment receives, as an input, an image and encodes header information of the input image. Next, in steps S302 to S310, the image encoding apparatus according to the fourth embodiment encodes a block of a base layer image.

In step S1201, the enhancement layer coding mode determining unit 911 determines whether the coding mode of the block of the base layer image, determined in step S304, is the I_PCM coding mode. If the coding mode of the block of the base layer image is the I_PCM coding mode, the process proceeds to step S314; otherwise, the process proceeds to step S311.

Thereafter, in steps S311 to S317, a block of an enhancement layer image is encoded.

In particular, the fourth embodiment is characteristic in the point that, if the coding mode determined by the base layer coding mode determining unit 102 is referred to and the coding mode is the I_PCM coding mode in step S1201, the I_PCM coding mode is similarly regarded as the coding mode of the enhancement layer. Further, in this case, encoding of the coding mode of the enhancement layer in step S313 is not performed.

Figs. 15A and 15B illustrate bit streams generated as above. Fig. 15A is a diagram illustrating a bit stream of the base layer. The bit stream of the base layer illustrated in Fig. 15A is the same as the bit stream illustrated in Fig. 8A according to the first embodiment. Fig. 15B is a diagram illustrating a bit stream of the enhancement layer. Fig. 15B illustrates three pieces of enhancement layer I_PCM pixel information. Referring to Fig. 15B, the enhancement layer I_PCM pixel information on the left includes the coding mode code, as in Fig. 8B, since the coding mode of the base layer is not the I_PCM coding mode. However, the enhancement layer I_PCM pixel information on the right in Fig. 15B includes no coding mode code since the coding mode of the base layer is the I_PCM coding mode.

With the above configuration and operation, scalable coding can facilitate the determination of the coding mode of the enhancement layer by determining the coding mode of the enhancement layer as the I_PCM coding mode in the case where the coding mode of the base layer is the I_PCM coding mode. As a result, the speed of encoding processing can be increased. Further, when the coding mode of the base layer is the I_PCM coding mode, omitting the coding mode code of the enhancement layer improves the coding efficiency by not sending redundant code. As in the third embodiment, the I_PCM coding mode is selected in the base layer because normal encoding is impossible, and it is highly likely that the same applies to the enhancement layer including the least significant bits of the base layer. The fourth embodiment is based on this idea.

Fifth Embodiment

Hereinafter, a fifth embodiment of the present invention will be described using Fig. 13. Fig. 13 is a block diagram illustrating the configuration of an image decoding apparatus according to the fifth embodiment.

In Fig. 13, blocks that serve the same functions as those illustrated in Fig. 5 according to the second embodiment are given the same numerals, and descriptions thereof are omitted. Also in the fifth embodiment, it is assumed that a bit stream to be decoded is generated in the fourth embodiment.

An enhancement layer integrating decoding unit 1310 receives, as inputs, a bit stream of the enhancement layer, separated by the layer code decoder/separator 501, and the coding mode of the base layer, decoded by the base layer coding mode decoding unit 503. On the basis of these inputs, the enhancement layer integrating decoding unit 1310 separates the bit stream, in units of blocks, into coding mode code, enhancement layer predictive coding pixel information, and enhancement layer I_PCM pixel information, and outputs the coding mode code, the enhancement layer predictive coding pixel information, and the enhancement layer I_PCM pixel information to a subsequent stage. The enhancement layer integrating decoding unit 1310 is different from the enhancement layer integrating decoding unit 510 illustrated in Fig. 5 in the point that the enhancement layer integrating decoding unit 1310 receives, as an input, the coding mode decoded by the base layer coding mode decoding unit 503. An enhancement layer coding mode decoding unit 1311 receives, as an input, the coding mode code of the enhancement layer, and decodes the coding mode. The enhancement layer coding mode decoding unit 1311 is different from the enhancement layer coding mode decoding unit 511 illustrated in Fig. 5 in the point that the enhancement layer coding mode decoding unit 1311 receives, as an input, the coding mode decoded by the base layer coding mode decoding unit 503.

An image decoding operation performed by the image decoding apparatus will be described below.

Firstly, decoding base layer code will be described.

As in the second embodiment, the base layer coding mode decoding unit 503 generates a coding mode by decoding the coding mode code included in the bit stream of the base layer, input from the base layer integrating decoding unit 502. The base layer coding mode decoding unit 503 outputs the generated coding mode to the enhancement layer coding mode decoding unit 1311, besides the output destinations in the second embodiment.

Thereafter, the base layer code is decoded, as in the second embodiment.

Next, decoding enhancement layer code will be described.

The enhancement layer integrating decoding unit 1310 decodes the header information of a bit stream of the enhancement layer, input from the layer code decoder/separator 501, and divides the picture data of the bit stream into code in units of blocks.

If the coding mode of the base layer, decoded by the base layer coding mode decoding unit 503, is a mode other than the I_PCM coding mode, the enhancement layer integrating decoding unit 1310 separates coding mode code from the divided code, as in the second embodiment. Further, the enhancement layer integrating decoding unit 1310 outputs the separated coding mode code to the enhancement layer coding mode decoding unit 1311.

If the coding mode of the base layer, decoded by the base layer coding mode decoding unit 503, is the I_PCM coding mode, the enhancement layer integrating decoding unit 1310 determines that the coding mode of the enhancement layer is the I_PCM coding mode. Therefore, the bit stream of the enhancement layer includes no coding mode code. That is, the enhancement layer integrating decoding unit 1310 outputs nothing to the enhancement layer coding mode decoding unit 1311.

Also, if the coding mode of the base layer is the I_PCM coding mode, the enhancement layer coding mode decoding unit 1311 inputs the fact that the coding mode is the I_PCM coding mode, regardless of code, to the selector 512, the enhancement layer first decoding unit 513, and the enhancement layer image reconstruction unit 515.

Thereafter, the enhancement layer code is decoded on the basis of the coding mode, as in the second embodiment.

Fig. 14 is a flowchart illustrating an image decoding process performed by the image decoding apparatus according to the fifth embodiment. In Fig. 14, steps that serve the same functions as those illustrated in Fig. 6 according to the second embodiment are given the same numerals, and descriptions thereof are omitted.

In steps S600 and S601, the image decoding apparatus according to the fifth embodiment receives, as an input, a bit stream and decodes the header information of the input bit stream. Next, in steps S602 to S608, the image decoding apparatus according to the fifth embodiment decodes base layer code. Further, in step S609, the layer code decoder/separator 501 extracts code relating to a to-be-decoded block of the enhancement layer from the bit stream input to the image decoding apparatus according to the fifth embodiment.

In step S1401, the base layer coding mode decoding unit 503 determines whether the coding mode of the block of the base layer image, decoded in step S603, is the I_PCM coding mode. If the coding mode of the block of the base layer image is the I_PCM coding mode, the process proceeds to step S612; otherwise, the process proceeds to step S610.

Thereafter, in steps S610 to S615, the block of the enhancement layer image is decoded.

In particular, the fifth embodiment is characteristic in the point that, if the coding mode generated by the base layer coding mode decoding unit 503 is referred to and the coding mode is the I_PCM coding mode in step S1401, the I_PCM coding mode is similarly regarded as the coding mode of the enhancement layer. Further, in this case, decoding of the coding mode code of the enhancement layer in step S610 is not performed.

Table 3 describes the above operation in accordance with the format of the H.264 standard.

Figure JPOXMLDOC01-appb-T000003

In comparison with Table 1, a portion of Table 3 that is different from Table 1 will be described.

base_is_ipcm[CurrMbAddr] is a sequence that represents whether the coding mode of a to-be-decoded block of the base layer, indicated by the address CurrMbAddr, is the I_PCM coding mode. In decoding the coding mode of the to-be-decoded block of the base layer, the image decoding apparatus according to the fifth embodiment stores whether the coding mode is the I_PCM coding mode. Only if base_is_ipcm[CurrMbAddr] on the 2nd line is false, that is, only if the coding mode of the base layer is not the I_PCM coding mode, base_mode_flag and mb_type are decoded. Conversely, if the coding mode of the base layer is the I_PCM coding mode, the code such as base_mode_flag and mb_type is omitted. Since portions thereafter are the same as Table 1, descriptions thereof are omitted.

Tables 4 and 5 describe the operation of the image decoding apparatus according to the fifth embodiment in accordance with the format of the draft standard of HEVC (NPL 2). Table 4 represents decoding of coding_unit, which is the unit of coding in HEVC blocks.

Figure JPOXMLDOC01-appb-T000004

In Table 4, the image decoding apparatus according to the fifth embodiment refers to base_is_ipcm[CurrMbAddr], as in H.264. Only if the coding mode of the base layer is not the I_PCM coding mode, the image decoding apparatus according to the fifth embodiment performs decoding based on base_mode_flag and prediction. If the coding mode of the base layer is the I_PCM coding mode, the image decoding apparatus decodes the I_PCM code of the enhancement layer based on scalable_pcm_sample( x0, y0, log2CbSize ).

Table 5 describes a process of scalable_pcm_sample( x0, y0, log2CbSize ).

Figure JPOXMLDOC01-appb-T000005

In Table 5, the image decoding apparatus according to the fifth embodiment obtains number_of_bit_depth from the pixel depth information n of the base layer and the pixel depth information m of the enhancement layer, as in H.264. Further, the image decoding apparatus decodes pcm_sample_luma_value, and calculates the luma of the enhancement layer based on equation (1) using pcm_sample_luma[i], which is the decoded pixel value of the base layer, and number_of_bit_depth. The same applies to chroma. That is, the image decoding apparatus decodes pcm_sample_chroma_value, and calculates the chroma of the enhancement layer based on equation (2) using pcm_sample_chroma[i], which is the decoded pixel value of the base layer, and number_of_bit_depth.

In this manner, the image decoding apparatus according to the fifth embodiment regenerates a decoded image of the enhancement layer.

With the above configuration and operation, when the I_PCM coding mode is used in decoding the scalable code, the scalable code is separated into a base layer image and an image including the least significant bits excluding bits represented by the base layer image, and code generated by encoding these images can be decoded. Also, decoding the scalable code becomes possible even in the I_PCM coding mode. Further, when the coding mode of the block of the base layer is referred to and the coding mode is the I_PCM coding mode, even if coding mode code is omitted in the enhancement layer, the image can be correctly decoded. Accordingly, decoding can be performed even more efficiently.

Although the fifth embodiment is described using H.264 and HEVC, the fifth embodiment is not limited to these standards.

Sixth Embodiment

The first to fifth embodiments describe that the processors illustrated in Figs. 1, 2, 4, 5, 7, 9, 11, and 13 are configured using hardware. However, processes performed by these processors illustrated in these drawings may be configured with computer programs.

Fig. 16 is a block diagram illustrating an example of the hardware configuration of a computer applicable to the image encoding apparatuses and the image decoding apparatuses (hereinafter collectively referred to as an "image processing apparatus") according to the first to fifth embodiments.

A central processing unit (CPU) 1601 controls the entire computer using computer programs and data stored in a random-access memory (RAM) 1602 and a read-only memory (ROM) 1603, and executes the above-described processes described as being performed by the image processing apparatus according to the first to fifth embodiments. That is, the CPU 1601 functions as the processors illustrated in Figs. 1, 2, 4, 5, 7, 9, 11, and 13.

The RAM 1602 has an area for temporarily storing computer programs and data loaded from an external storage device 1606 and data obtained from the outside via an interface (I/F) 1607. The RAM 1602 further has a work area used when the CPU 1601 executes various processes. That is, the RAM 1602 can be assigned as, for example, a frame memory, or can appropriately provide other various areas.

The ROM 1603 stores the setting data, boot program, or the like of the computer.

An operation unit 1604 includes a keyboard, a mouse, or the like, and, in response to an operation performed by the user of the computer, the operation unit 1604 can input various instructions to the CPU 1601.

An output unit 1605 outputs the result of processing performed by the CPU 1601. The output unit 1605 includes, for example, a liquid crystal display.

The external storage device 1606 is a large-capacity information storage device represented by a hard disk drive. The external storage device 1606 saves am operating system (OS) and a computer program for causing the CPU 1601 to realize the functions of the units illustrated in Figs. 1 and 2. The external storage device 1606 may further save images to be processed.

The computer program and data saved in the external storage device 1606 are appropriately loaded to the RAM 1602 under control of the CPU 1601 and are to be processed by the CPU 1601.

The I/F 1607 can connect a network, such as a local area network (LAN) or the Internet, or another device, such as a projector or a display, to the computer. The computer can obtain and send various pieces of information via the I/F 1607.

A bus 1608 connects the above-described units.

The operation performed by the above-described configuration is the operation described using the above-described flowcharts, which is principally controlled by the CPU 1601.

Other Embodiments

An object of the present invention can be achieved when a storage medium recording the code of a computer program realizing the above-described functions is supplied to a system, and then the system reads and executes the code of the computer program. In this case, the code itself of the computer program read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the code of the computer program constitutes the present invention. The invention also includes the case in which the above-described functions are realized by an operating system (OS) or the like, running on a computer, which performs part or entirety of the actual processing based on instructions of the code of the program.

Furthermore, the present invention may be realized in the following form. That is, the computer program code read from the storage medium is written to a memory included in a feature expansion card inserted into the computer or a feature expansion unit connected to the computer. A CPU or the like, included in the feature expansion card or the feature expansion unit, performs part or entirety of the actual processing based on instructions of the code of the program and realizes the above-described functions.

When the present invention is applied to the above-described storage medium, the storage medium stores the code of a computer program corresponding to the previously-described flowcharts.

While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Embodiments of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions recorded on a storage medium (e.g., non-transitory computer-readable storage medium) to perform the functions of one or more of the above-described embodiments of the present invention, and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiments. The computer may comprise one or more of a central processing unit (CPU), micro processing unit (MPU), or other circuitry, and may include a network of separate computers or separate computer processors. The computer executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)TM), a flash memory device, a memory card, and the like.

This application claims the benefit of Japanese Patent Application No. 2012-147151, filed June 29, 2012, which is hereby incorporated by reference herein in its entirety.

Claims (16)

  1. An image encoding apparatus that performs scalable coding of an image divided in units of blocks by using a plurality of layers including an n-bit base layer and an m-bit enhancement layer, m being greater than or equal to n, comprising:
    an enhancement layer coding mode determining unit configured to determine a coding mode of a to-be-encoded block, serving as a target of encoding, of the enhancement layer; and
    an enhancement layer encoding unit configured to encode the to-be-encoded block of the enhancement layer on the basis of the coding mode determined by the enhancement layer coding mode determining unit,
    wherein, when the coding mode determined by the enhancement layer coding mode determining unit is an uncompressed coding mode, the enhancement layer encoding unit generates an image including the least significant m-n( bits from the to-be-encoded block of the enhancement layer, and encodes the generated image in the uncompressed coding mode.
  2. The image encoding apparatus according to Claim 1, further comprising a base layer coding mode determining unit configured to determine a coding mode of a to-be-encoded block, serving as a target of encoding, of the base layer,
    wherein, when the coding mode determined by the base layer coding mode determining unit is the uncompressed coding mode, the enhancement layer encoding unit performs uncompressed coding of the to-be-encoded block of the enhancement layer.
  3. The image encoding apparatus according to Claim 2, wherein, when the coding mode determined by the base layer coding mode determining unit is the uncompressed coding mode,
    the enhancement layer coding mode determining unit determines the coding mode of the to-be-encoded block of the enhancement layer as the uncompressed coding mode, and
    the enhancement layer encoding unit performs uncompressed coding of the to-be-encoded block of the enhancement layer without encoding the coding mode determined by the enhancement layer coding mode determining unit.
  4. The image encoding apparatus according to Claim 2, wherein the enhancement layer encoding unit refers to the to-be-encoded block of the base layer and encodes the to-be-encoded block of the enhancement layer.
  5. The image encoding apparatus according to Claim 1, wherein the uncompressed coding mode is a mode in which the to-be-encoded block is encoded without being compressed, and inserted into a bit stream generated by encoding the image divided in units of blocks.
  6. An image decoding apparatus that decodes a bit stream generated by performing scalable coding of an image divided in units of blocks by using a plurality of layers including an n-bit base layer and an m-bit enhancement layer, m being greater than or equal to n, comprising:
    an enhancement layer coding mode obtaining unit configured to obtain a coding mode of a to-be-decoded block, serving as a target of decoding, of the enhancement layer; and
    an enhancement layer decoding unit configured to decode the to-be-decoded block of the enhancement layer on the basis of the coding mode obtained by the enhancement layer coding mode obtaining unit,
    wherein, when the coding mode obtained by the enhancement layer coding mode obtaining unit is an uncompressed coding mode, the enhancement layer decoding unit decodes a bit stream including the least significant m-n bits from the to-be-decoded block of the enhancement layer, and does not decode a bit stream including the most significant n bits.
  7. The image decoding apparatus according to Claim 6, further comprising a base layer coding mode obtaining unit configured to obtain a coding mode of a to-be-decoded block, serving as a target of decoding, of the base layer,
    wherein, when the coding mode obtained by the base layer coding mode obtaining unit is the uncompressed coding mode, the enhancement layer decoding unit decodes a block including the uncompressed-encoded least significant m-n bits of the to-be-decoded block of the enhancement layer.
  8. The image decoding apparatus according to Claim 7, wherein, when the coding mode obtained by the base layer coding mode obtaining unit is the uncompressed coding mode,
    the enhancement layer coding mode obtaining unit regards the coding mode of the to-be-decoded block of the enhancement layer as the uncompressed coding mode, and
    the enhancement layer decoding unit decodes a block including the uncompressed-encoded least significant m-n bits of the to-be-decoded block of the enhancement layer.
  9. The image decoding apparatus according to Claim 7, further comprising:
    a base layer decoding unit configured to decode the to-be-decoded block of the base layer on the basis of the coding mode obtained by the base layer coding mode obtaining unit; and
    a regenerating unit configured to regenerate a decoded image by combining the to-be-decoded block decoded by the base layer decoding unit and the to-be-decoded block decoded by the enhancement layer decoding unit,
    wherein, when the coding mode obtained by the enhancement layer coding mode obtaining unit is the uncompressed coding mode, the regenerating unit shifts the to-be-decoded block, decoded by the base layer decoding unit, m-n bits in the most significant bit direction, and combines the shifted to-be-decoded block and the to-be-decoded block decoded by the enhancement layer decoding unit.
  10. The image decoding apparatus according to Claim 7, wherein the enhancement layer decoding unit refers to the to-be-decoded block of the base layer, and decodes the to-be-decoded block of the enhancement layer.
  11. An image encoding method of performing scalable coding of an image divided in units of blocks by using a plurality of layers including an n-bit base layer and an m-bit enhancement layer, m being greater than or equal to n, comprising:
    an enhancement layer coding mode determining step of determining a coding mode of a to-be-encoded block, serving as a target of encoding, of the enhancement layer; and
    an enhancement layer encoding step of encoding the to-be-encoded block of the enhancement layer on the basis of the coding mode determined in the enhancement layer coding mode determining step,
    wherein, when the coding mode determined in the enhancement layer coding mode determining step is an uncompressed coding mode, the enhancement layer encoding step generates an image including the least significant m-n bits from the to-be-encoded block of the enhancement layer, and encodes the generated image in the uncompressed coding mode.
  12. The image encoding method according to Claim 11, further comprising a base layer coding mode determining step of determining a coding mode of a to-be-encoded block, serving as a target of encoding, of the base layer,
    wherein, when the coding mode determined in the base layer coding mode determining step is the uncompressed coding mode, the enhancement layer encoding step performs uncompressed coding of the to-be-encoded block of the enhancement layer.
  13. An image decoding method of decoding a bit stream generated by performing scalable coding of an image divided in units of blocks by using a plurality of layers including an n-bit base layer and an m-bit enhancement layer, m being greater than or equal to n, comprising:
    an enhancement layer coding mode obtaining step of obtaining a coding mode of a to-be-decoded block, serving as a target of decoding, of the enhancement layer; and
    an enhancement layer decoding step of decoding the to-be-decoded block of the enhancement layer on the basis of the coding mode obtained in the enhancement layer coding mode obtaining step,
    wherein, when the coding mode obtained in the enhancement layer coding mode obtaining step is an uncompressed coding mode, the enhancement layer decoding step decodes a bit stream including the least significant m-n bits from the to-be-decoded block of the enhancement layer, and does not decode a bit stream including the most significant n bits.
  14. The image decoding method according to Claim 13, further comprising a base layer coding mode obtaining step of obtaining a coding mode of a to-be-decoded block, serving as a target of decoding, of the base layer,
    wherein, when the coding mode obtained in the base layer coding mode obtaining step is the uncompressed coding mode, the enhancement layer decoding step decodes a block including the uncompressed-encoded least significant m-n bits of the to-be-decoded block of the enhancement layer.
  15. A non-transitory computer-readable storage medium storing a program for causing a computer to execute image encoding, the program comprising:
    computer-executable instructions that perform scalable coding of an image divided in units of blocks by using a plurality of layers including an n-bit base layer and an m-bit enhancement layer, m being greater than or equal to n;
    computer-executable instructions that determine a coding mode of a to-be-encoded block, serving as a target of encoding, of the enhancement layer;
    computer-executable instructions that encode the to-be-encoded block of the enhancement layer on the basis of the determined coding mode; and
    computer-executable instructions that, when the determined coding mode is an uncompressed coding mode, generates an image including the least significant m-n bits from the to-be-encoded block of the enhancement layer, and encodes the generated image in the uncompressed coding mode.
  16. A non-transitory computer-readable storage medium storing a program for causing a computer to execute image decoding, the program comprising:
    computer-executable instructions that decode a bit stream generated by performing scalable coding of an image divided in units of blocks by using a plurality of layers including an n-bit base layer and an m-bit enhancement layer, m being greater than or equal to n;
    computer-executable instructions that obtain a coding mode of a to-be-decoded block, serving as a target of decoding, of the enhancement layer;
    computer-executable instructions that decode the to-be-decoded block of the enhancement layer on the basis of the obtained coding mode; and
    computer-executable instructions that, when the obtained coding mode is an uncompressed coding mode, decodes a bit stream including the least significant m-n bits from the to-be-decoded block of the enhancement layer, and does not decode a bit stream including the most significant n bits.
PCT/JP2013/003760 2012-06-29 2013-06-17 Image encoding apparatus, image encoding method and program, image decoding apparatus, and image decoding method and program WO2014002422A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2012-147151 2012-06-29
JP2012147151A JP6041555B2 (en) 2012-06-29 2012-06-29 The image coding apparatus, image coding method, and program, image decoding apparatus, image decoding method, and program

Publications (1)

Publication Number Publication Date
WO2014002422A1 true true WO2014002422A1 (en) 2014-01-03

Family

ID=48874454

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2013/003760 WO2014002422A1 (en) 2012-06-29 2013-06-17 Image encoding apparatus, image encoding method and program, image decoding apparatus, and image decoding method and program

Country Status (2)

Country Link
JP (1) JP6041555B2 (en)
WO (1) WO2014002422A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004045217A1 (en) * 2002-11-13 2004-05-27 Koninklijke Philips Electronics N.V. Transmission system with colour depth scalability
US20050259729A1 (en) * 2004-05-21 2005-11-24 Shijun Sun Video coding with quality scalability
EP1601204A1 (en) * 2004-05-27 2005-11-30 Daeyang Foundation Apparatus for scalable encoding/decoding of moving image and method thereof
US20100091840A1 (en) * 2007-01-10 2010-04-15 Thomson Licensing Corporation Video encoding method and video decoding method for enabling bit depth scalability
JP2012147151A (en) 2011-01-11 2012-08-02 Nippon Telegr & Teleph Corp <Ntt> Optical subsystem and optical subsystem control method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004045217A1 (en) * 2002-11-13 2004-05-27 Koninklijke Philips Electronics N.V. Transmission system with colour depth scalability
US20050259729A1 (en) * 2004-05-21 2005-11-24 Shijun Sun Video coding with quality scalability
EP1601204A1 (en) * 2004-05-27 2005-11-30 Daeyang Foundation Apparatus for scalable encoding/decoding of moving image and method thereof
US20100091840A1 (en) * 2007-01-10 2010-04-15 Thomson Licensing Corporation Video encoding method and video decoding method for enabling bit depth scalability
JP2012147151A (en) 2011-01-11 2012-08-02 Nippon Telegr & Teleph Corp <Ntt> Optical subsystem and optical subsystem control method

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
BAO ET AL: "A Low-Complexity AVC-based Scalable Video Codec", 71. MPEG MEETING; 17-01-2005 - 21-01-2005; HONG KONG; (MOTION PICTUREEXPERT GROUP OR ISO/IEC JTC1/SC29/WG11),, no. M11751, 13 January 2005 (2005-01-13), XP030040496, ISSN: 0000-0250 *
CHONO K ET AL: "Pulse code modulation mode for HEVC", 4. JCT-VC MEETING; 95. MPEG MEETING; 20-1-2011 - 28-1-2011; DAEGU;(JOINT COLLABORATIVE TEAM ON VIDEO CODING OF ISO/IEC JTC1/SC29/WG11AND ITU-T SG.16 ); URL: HTTP://WFTP3.ITU.INT/AV-ARCH/JCTVC-SITE/,, no. JCTVC-D044, 15 January 2011 (2011-01-15), XP030008085, ISSN: 0000-0015 *
WIEGAND T ET AL: "Overview of the H.264/AVC video coding standard", IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, IEEE SERVICE CENTER, PISCATAWAY, NJ, US, vol. 13, no. 7, 1 July 2003 (2003-07-01), pages 560 - 576, XP011221093, ISSN: 1051-8215, DOI: 10.1109/TCSVT.2003.815165 *

Also Published As

Publication number Publication date Type
JP2014011637A (en) 2014-01-20 application
JP6041555B2 (en) 2016-12-07 grant

Similar Documents

Publication Publication Date Title
US20040136457A1 (en) Method and system for supercompression of compressed digital video
US20040179610A1 (en) Apparatus and method employing a configurable reference and loop filter for efficient video coding
US8606028B2 (en) Pixel bit depth conversion in image encoding and decoding
US20110292999A1 (en) Super macro block based intra coding method and apparatus
US20060146191A1 (en) Color image encoding and decoding method and apparatus using a correlation between chrominance components
US20160080753A1 (en) Method and apparatus for processing video signal
US20100177820A1 (en) Video encoding device, video decoding device, video encoding method, video decoding method, video encoding or decoding program
JP2009513056A (en) Encoding / decoding apparatus and method of video using the adaptive scan
US20160150236A1 (en) Image encoding apparatus, image encoding method, recording medium and program, image decoding apparatus, image decoding method, and recording medium and program
JP2010232720A (en) Image encoding method and image decoding method
US20090279612A1 (en) Methods and apparatus for multi-view video encoding and decoding
US20130101037A1 (en) Video encoding device, video decoding device, video encoding method, video decoding method, and program
US20100220792A1 (en) Encoding device and decoding device
WO2008024345A1 (en) Adaptive region-based flipping video coding
US20070086516A1 (en) Method of encoding flags in layer using inter-layer correlation, method and apparatus for decoding coded flags
US8472731B2 (en) Image signal decoding apparatus, image signal decoding method, image signal encoding apparatus, image signal encoding method, and program
WO2012070232A1 (en) Image encoding device, image decoding device, image encoding method, image decoding method, and program
US20150381980A1 (en) Image processing device, image processing method, and program
US20120128069A1 (en) Image processing apparatus and method
US20090219989A1 (en) Coding device and editing device
US20090103613A1 (en) Method for Decoding Video Signal Encoded Using Inter-Layer Prediction
JP2002058028A (en) Device and method for encoding picture, method and device for decoding picture and picture processor
US20080175316A1 (en) Method and apparatus for encoding and decoding based on intra prediction
US20010013952A1 (en) Data structure for image transmission, image coding method, and image decoding method
US20110002392A1 (en) Method and apparatus for multi-view video encoding and method and apparatus for multiview video decoding

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13740375

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13740375

Country of ref document: EP

Kind code of ref document: A1