WO2013001755A1

WO2013001755A1 - Image encoding apparatus, image encoding method and program, image decoding apparatus, image decoding method, and program

Info

Publication number: WO2013001755A1
Application number: PCT/JP2012/004026
Authority: WO
Inventors: Masato Shima
Original assignee: Canon Kabushiki Kaisha
Priority date: 2011-06-29
Filing date: 2012-06-21
Publication date: 2013-01-03
Also published as: JP2013012887A

Abstract

A method for encoding an image executed in an image encoding apparatus, the method comprising, dividing an input image into a plurality of blocks, generating a prediction error on a block-by-block basis with respect to the divided blocks; performing orthogonal transform on the prediction error to generate transform coefficients, storing in a storage unit quantization matrices to be used when the transform coefficients are quantized, determining a scanning method for scanning the quantized transform coefficients in order to encode the quantized transform coefficients, selecting one of the plurality of quantization matrices stored in the storage unit based on the determined scanning method, scanning transform coefficients by the determined scanning method and generating quantized coefficients from the transform coefficients using the selected quantization matrix, and encoding the quantized coefficients.

Description

IMAGE ENCODING APPARATUS, IMAGE ENCODING METHOD AND PROGRAM, IMAGE DECODING APPARATUS, IMAGE DECODING METHOD, AND PROGRAM

The present invention relates to an image encoding apparatus, an image encoding method and a program, an image decoding apparatus, an image decoding method and program, and more particularly, relates to an encoding method and a decoding method of a quantization matrix in an image.

As a compression and recording method of a moving image, H.264/MPEG-4 AVC (hereinafter, H.264) is known. (International Telecommunication Union Telecommunication Standardization Sector (ITU-T) H.264 (03/2010) Advanced Video Coding for Generic Audiovisual Services) In the H.264, each element of a quantization matrix can be changed to an arbitrary value by encoding scaling_list information. According to Section 7.3.2.1.1.1 described in the above literature, each element of the quantization matrix can take an arbitrary value by adding delta_scale which is a difference value between a changing target element and a preceding element.

In recent years, international standardization activities for higher efficient encoding method as a succession standard to H.264 have been started. The Joint Collaborative Team on Video Coding (JCT-VC) has been founded between International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) and International Telecommunication Union Telecommunication Standardization Sector (ITU-T), and the standardizations are underway as High Efficiency Video Coding (HEVC) encoding method (hereinafter, HEVC).

For the purpose of improvements of encoding efficiency, scanning methods of coefficients after encoding processing of transformation/quantization, etc. such as horizontal scan and vertical scan have been studied in addition to a conventional zigzag scan method. Figs. 2A to 2C illustrate states of the scanning methods of coefficients. A solid frame 1000 represents a block as a unit to perform transformation, quantization, or the like, and for ease of description, the block is composed of 4 * 4 pixels, and every square within the solid frame represents a coefficient after transformation/quantization. Fig. 2A illustrates a conventional zigzag scan, and as indicated by arrows, the coefficients are scanned in a direction starting from upper-left toward lower-right within the block. On the other hand, Fig. 2B and Fig. 2C illustrate horizontal scan and vertical scan respectively, and as indicated by arrows, the coefficients are scanned in a horizontal direction or a vertical direction starting from upper-left of the block. In this way, encoding processing is performed using a plurality of the scanning methods for coefficients. (JCT-VC Document JCTVC-E603.doc Internet <http://phenix.int-evry.fr/jct/doc_end_user/documents/5_Geneva/wg11/>)

In HEVC, it is also considered to introduce a mechanism such that each element of the quantization matrix can be set to an arbitrary value similarly to the H.264. However, it is not considered to perform control of the quantization matrix corresponding to a scanning method of coefficients. It is desirable from viewpoint of image quality to perform control of the quantization matrix corresponding to the scanning method of the coefficients.

However, in a case where the quantization matrix can be controlled for each scanning method of the coefficients, it is necessary to encode the quantization matrix which is optimal to each scanning method, and there arises an issue that a encoding amount will increase.

ITU-T H.264 (03/2010) Advanced Video Coding for Generic Audiovisual Services JCT-VC Document JCTVC-E603.doc Internet < http://phenix.int-evry.fr/jct/doc_end_user/documents/5_Geneva/wg11/ >

The present invention relates to using quantization matrices adaptively according to scanning methods of coefficients.

According to an aspect of the present invention, an image encoding apparatus includes a division unit configured to divide an input image into a plurality of blocks, a prediction unit configured to generate a prediction error on a block-by-block basis with respect to the blocks divided by the division unit, a transform unit configured to perform orthogonal transform on the prediction error to generate a transform coefficient, a quantization matrix storage unit configured to store a quantization matrix to be used when the transform coefficient is quantized, a scanning method determination unit configured to determine a scanning method for scanning the quantized transform coefficient in order to encode the quantized transform coefficient, a quantization matrix selection unit configured to select a plurality of quantization matrices stored in the quantization matrix storage unit based on the scanning method determined by the scanning method determination unit, a quantization unit configured to scan a transform coefficient by the determined scanning method and generate a quantized coefficient from the transform coefficient using the selected quantization matrices, and a coefficient encoding unit configured to encode the quantized coefficient.

According to the present invention, a quantization matrix can be set for each scanning method of coefficients, and thus an optimal quantization processing can be performed by each scanning method

Further features and aspects of the present invention will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings.

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the invention and, together with the description, serve to explain the principles of the invention.

Fig. 1 is a block diagram illustrating a configuration of an image encoding apparatus according to a first exemplary embodiment of the present invention. Fig. 2A illustrates an example of a scanning method of coefficients in a block. Fig. 2B illustrates an example of a scanning method of coefficients in a block. Fig. 2C illustrates an example of a scanning method of coefficients in a block. Fig. 3 is a block diagram illustrating a configuration of an image decoding apparatus according to a second exemplary embodiment. Fig. 4A illustrates an example of a quantization matrix. Fig. 4B illustrates an example of a quantization matrix. Fig. 4C illustrates an example of a quantization matrix. Fig. 4D illustrates an example of a quantization matrix. Fig. 5 is a flowchart illustrating image encoding processing in the image encoding apparatus according to the first exemplary embodiment. Fig. 6 is a flowchart illustrating image decoding processing in the image decoding apparatus according to the second exemplary embodiment. Fig. 7A illustrates an example of a structure of bitstreams. Fig. 7B illustrates an example of a structure of bitstreams. Fig. 8 is a block diagram illustrating a configuration of an image encoding apparatus according to a third exemplary embodiment. Fig. 9 is a block diagram illustrating a configuration of an image decoding apparatus according to a fourth exemplary embodiment. Fig. 10 is a flowchart illustrating image encoding processing of the image encoding apparatus according to the third exemplary embodiment. Fig. 11 is a flowchart illustrating image decoding processing in the image decoding apparatus according to the fourth exemplary embodiment. Fig. 12 is a block diagram illustrating a configuration of an image encoding apparatus according to a fifth exemplary embodiment. Fig. 13 is a block diagram illustrating a configuration of an image decoding apparatus according to a sixth exemplary embodiment. Fig. 14 is a flowchart illustrating image encoding processing in the image encoding apparatus according to the fifth exemplary embodiment. Fig. 15 is a flowchart illustrating image decoding processing in the image decoding apparatus according to the sixth exemplary embodiment. Fig. 16 is a block diagram illustrating an example of a hardware configuration of a computer applicable to the image encoding apparatus and the decoding apparatus according to the present invention.

Various exemplary embodiments, features, and aspects of the invention will be described in detail below with reference to the drawings.

The configurations illustrated in exemplary embodiments described below are only examples, and the present invention is not limited to the illustrated configurations.

Hereinbelow, a first exemplary embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a block diagram illustrating an image encoding apparatus according to the present exemplary embodiment.

In Fig. 1, a block division unit 1 divides an input image into a plurality of blocks.

A prediction unit 2 performs prediction, by the block, on respective blocks divided by the block division unit 1, determines a prediction method, performs calculation of a difference value in accordance with the prediction method, and calculates a prediction error. It is assumed to perform intra prediction in case of a still image or in an intra frame of a moving image, and to perform also a motion compensation prediction in case of a moving image. The intra prediction is generally realized by selecting a method for referring to reference pixels (prediction method) used for calculating the prediction value from data of neighboring pixels, with respect to a plurality of reference methods.

A coefficient scanning method determination unit 7 determines a coefficient scanning method based on the prediction method determined by the prediction unit 2. Types of the coefficient scanning methods is not limited especially, however in the present exemplary embodiment, three types of the method: a zigzag scan, a horizontal scan, and a vertical scan in Fig. 2 are used.

A quantization matrix storage unit 9 generates a quantization matrix and temporarily stores the quantization matrix. A generation method of a quantization matrix to be stored is not limited especially. A user may input a quantization matrix, or a quantization matrix may be calculated from characteristics of the input image, or of course, a quantization matrix specified in advance as an initial value may be used.

A transform unit 3 performs orthogonal transform on a prediction error of each block. More specifically, the transform unit 3 performs orthogonal transform in a unit of an input block size or its subdivided block size, and calculates orthogonal transform coefficients. Hereinbelow, a block which is subjected to orthogonal transform is referred to as a transform block. With regard to the orthogonal transform, it is not limited especially, and a discrete cosine transform, a Hadamard transform, or the like may be used. Further, in the present exemplary embodiment, for convenience of description, it is assumed to halve vertically and horizontally the prediction errors on an 8 * 8 pixel block basis, to perform orthogonal transform on a 4 * 4 pixel transform block basis, but a size of the transform block is not limited to this. The orthogonal transform may be performed using the transform block with identical size to that of the block, or the orthogonal transform may be performed using the transform block obtained by dividing the block in even finer unit than halving vertically and horizontally.

A quantization matrix selection unit 8 receives inputs of the coefficient scanning method determined by the coefficient scanning method determination unit 7 and the transform block size determined the transform unit 3, and selects a quantization matrix to be applied from among the quantization matrices stored in the quantization matrix storage unit 9.

A quantization unit 4 scans the orthogonal transform coefficients using the coefficient scanning method determined by the coefficient scanning method determination unit 7, and quantizes the orthogonal transform coefficients using the quantization matrix selected by the quantization matrix selection unit 8. Thus, the quantized coefficients can be obtained by the quantization. According to the present exemplary embodiment, the order is set in which quantization is performed after the orthogonal transform coefficients have been scanned, however the order is not limited to this. The quantized coefficients may be scanned after the quantization has been performed. If each element within the quantization matrix is appropriately rearranged, it is possible to calculate identical quantized coefficients regardless of the order of scanning and quantization of the coefficients.

A coefficient encoding unit 5 encodes the quantized coefficients obtained in this way to generate quantized coefficient code data. For an encoding method, Huffman coding, arithmetic coding, or the like can be used.

A quantization matrix encoding unit 10 encodes the quantization matrix stored in the quantization matrix storage unit 9, to generate the quantization matrix code data. An encoding method for the quantization matrix is not limited especially, and Huffman coding, arithmetic coding, or the like can be used for a value of each element of the quantization matrix itself, or a difference from a value of the preceding element or a difference from other quantization matrix.

In the present exemplary embodiment, the quantization matrix is generated and encoded for each transform block size, but encoding may be performed using the quantization matrix corresponding to other transform block size. For example, the quantization matrix of 8 * 8 pixels may be encoded by the element, and the quantization matrix of 4 * 4 pixels may be obtained by taking average values of 2 * 2 elements within the quantization matrix of 8 * 8 pixels as prediction values, and encoding the differences thereof.

An integration unit 6 generate codes related to header information, prediction, and transform, and integrates the quantized coefficient code data generated by the coefficient encoding unit 5 and the quantization matrix code data generated by the quantization matrix encoding unit 10. The codes related to prediction and transform are assumed to be codes of, for example, the selected prediction method or a state of division of a transform block.

Encoding operations of an image according to the above-described image encoding apparatus will be described below. In the present exemplary embodiment, it is configured to input moving image data by the frame, but it may be configured to input one frame of still image data. In the present exemplary embodiment, for ease of description, only intra prediction encoding processing will be described. However, the present invention is not limited to this configuration, and inter prediction encoding processing is also applicable. In the present exemplary embodiment, for convenience of description, it is described that the block division unit 1 divides an image into blocks of 8 * 8 pixels, however, the present exemplary embodiment is not limited to this example.

The one frame of the image data is input into the block division unit 1, and is divided into blocks of 8 * 8 pixels. The divided image data is input into the prediction unit 2.

The prediction unit 2 performs predictions by the block to generate coefficient scanning method selection auxiliary information and a prediction error. In the present exemplary embodiment, the coefficient scanning method selection auxiliary information assumes information of the intra prediction method, for example, information of the prediction mode. However, the coefficient scanning method selection auxiliary information is not limited to this, and information of the motion compensation prediction method or information of the transform may be also used.

The transform unit 3 determines a transform block size for the prediction error generated by the prediction unit 2 to perform orthogonal transform, and generates orthogonal transform coefficients. The transform unit 3 inputs the transform block size into the quantization matrix selection unit 8, and inputs the orthogonal transform coefficients into the quantization unit 4.

On the other hand, the coefficient scanning method determination unit 7 determines a coefficient scanning method using the coefficient scanning method selection auxiliary information generated by the prediction unit 2, and outputs the coefficient scanning method to the quantization matrix selection unit 8 and the quantization unit 4. A determination method of the coefficient scanning method in the present exemplary embodiment, in a case where prediction is performed from left-hand block in the blocks, that is, in a case where a prediction direction is from left to right, uses the vertical scan illustrated in Fig. 2C. In a case where prediction is performed from an upper block of the block, the determination method uses the horizontal scan illustrated in Fig. 2B. In other cases, the determination method is assumed to use the zigzag scan illustrated in Fig. 2A, and a combination of the prediction methods and the scanning methods is not limited to this.

The quantization matrix storage unit 9 stores quantization matrices used for quantization of the frame, more specifically, quantization matrices corresponding to each of the scanning methods and transform block sizes. In the present exemplary embodiment, the quantization matrix storage unit 9 stores a first quantization matrix corresponding to 4 * 4 pixels zigzag scan, a second quantization matrix corresponding to 4 * 4 pixels horizontal scan, and a third quantization matrix corresponding to 4 * 4 pixels vertical scan. In the present exemplary embodiment, a matrix illustrated in Fig. 4A is a first quantization matrix, a matrix illustrated in Fig. 4B is a second quantization matrix, and a matrix illustrated in Fig. 4C is a third quantization matrix. Of course, the elements within each quantization matrix are not limited to these examples.

In the present exemplary embodiment, the transform block size is available in one type, i.e., 4 * 4 pixels, and the scanning methods are available in three types, i.e., zigzag scan, vertical scan, and horizontal scan. Therefore, the quantization matrices to be stored are available in three types, but actually the quantization matrices are stored equivalent to the number of combinations of the transform block sizes and the scanning methods. For example, in a case where the transform block size of 8 * 8 pixels and the corresponding zigzag scan of 8 * 8 pixels in addition to the present exemplary embodiment are also used, a fourth quantization matrix corresponding to the 8 * 8 pixels zigzag scan will be stored.

The quantization matrix encoding unit 10 reads out and encodes the quantization matrices in order according to scanning methods and the transform block sizes from the quantization matrix storage unit 9, to generate the quantization matrix code data. An example of a specific encoding method of the quantization matrix will be described with reference to Figs. 4A to 4D. A solid frame 1100 indicates the quantization matrix. For ease of description, the quantization matrix has a configuration of 16 pixels corresponding to the transform blocks of 4 * 4 pixels, and each square within the solid frame represents an element. In this case, four types of the quantization matrices illustrated in Figs. 4A to 4D are assumed to be encoded.

Fig. 4A is an example where each element of the quantization matrix starts with 1 and is incremented by one. For example, when the quantization matrix in Fig. 4A is encoded, values from 1 to 16 are encoded one by one. Further, after encoding the first element 1, a difference from the preceding element, i.e., a difference value 1 in this example may be encoded 15 times. Furthermore, similar to differential pulse code modulation (DPCM), after encoding the first element 1 and the difference 1 between the second element and the first element, 0 as a prediction difference may be encoded.

Fig. 4B is an example where each element of the quantization matrix starts with 3 and is incremented by one. For example, when the quantization matrix in Fig. 4B is encoded, the values from 3 to 18 are encoded one by one. Further, after encoding the first element 3, a difference from the preceding element, i.e., a difference value 1 in this example may be encoded 15 times. Furthermore, a difference from a value of each element of the quantization matrix illustrated in Fig. 4A, i.e., a difference value 2 in this example may be encoded 16 times.

Fig. 4C is an example where each element of the quantization matrix starts with 3, and is incremented by one up to 10, then 10 is repeated three times, subsequently is incremented by one to 16. For example, when the quantization matrix in Fig. 4C is encoded, similar to Fig. 4A or Fig. 4B, values of respective elements are encoded one by one. A difference from the preceding element, or a difference from elements of other quantization matrix may be encoded. In a case where a difference between the value of each element in Fig. 4C and the value of each element in Fig. 4A is encoded, a difference value 2 is encoded from the first element 3 to the first 10, and a difference value 1 is encoded at the second 10. Then, since the difference values are 0 from the third 10 to the last element, the difference value 0 is encoded 7 times. Further, a method for reducing code amounts of the difference values by encoding the code which indicates aborting of encoding may be used.

Fig. 4D is an example when each element of the quantization matrix starts with 3 and is incremented by one up to 10, and thereafter continues to be 10 to the end. For example, when the quantization matrix in Fig. 4D is encoded, similar to Fig. 4A or Fig. 4B, values of respective elements are encoded one by one. A difference from the preceding element, or a difference from elements of other quantization matrix may be encoded. When a difference between the value of each element in Fig. 4D and the value of the preceding element thereof is encoded, the difference value 1 is encoded from the first element 3 to the first 10. Then, since the difference values are 0 to the last element, the difference value 0 is encoded 7 times. Further, a method for reducing code amounts of the difference values by encoding the code which indicates aborting of encoding may be used.

The quantization matrix selection unit 8 selects a quantization matrix used for quantization of the block, using the coefficient scanning method determined by the coefficient scanning method determination unit 7 and the transform block size determined by the transform unit 3. More specifically, the quantization matrix selection unit 8 selects a quantization matrix to be used for quantization of the transform block from among the quantization matrices stored in the quantization matrix storage unit 9. For example, in the present exemplary embodiment, in a case where the coefficient scanning method is the horizontal scan and the transform block size is 4 * 4 pixels, the second quantization matrix is input from the quantization matrix storage unit 9, and is output to the quantization unit 4.

In the quantization unit 4, the orthogonal transform coefficients output from the transform unit 3 are scanned using the scanning method determined by the coefficient scanning method determination unit 7, and are quantized using the quantization matrices selected by the quantization matrix selection unit 8 to generate quantized coefficients. The generated quantized coefficients are input into the coefficient encoding unit 5.

The coefficient encoding unit 5 encodes the quantized coefficients generated by the quantization unit 4 to generate quantized coefficient code data and outputs it to the integration unit 6.

On the other hand, the quantization matrix encoding unit 10 encodes each quantization matrix stored in the quantization matrix storage unit 9 to generate the quantization matrix code data and outputs it to the integration unit 6.

The integration unit 6 generates codes such as headers of sequences, frames, pictures, and slices of an image. Further, the integration unit 6 inserts the quantization matrix code data generated by the quantization matrix encoding unit 10 into either of these headers. The integration unit 6 integrates the codes of such header portions and the quantized coefficient code data generated by the coefficient encoding unit 5 to generate and output a bitstream.

Fig. 7A is an example of a bitstream output in the first exemplary embodiment. In the present exemplary embodiment, it is configured such that the quantization matrix is encoded in a sequence header portion in the bitstream, however, a position of encoding is not limited to this configuration. Of course, it may be configured such that the quantization matrix is encoded in a picture header portion or other header portions. Further, in a case where the quantization matrix is changed in one sequence, the quantization matrix can be updated by newly encoding the quantization matrix. At this time, all quantization matrices may be rewritten, or it is also possible to change the part of the quantization matrices by specifying a scanning method and a transform block size of the quantization matrices to be rewritten.

Fig. 5 is a flowchart illustrating image encoding processing in the image encoding apparatus according to the first exemplary embodiment. First, in step S001, the quantization matrix storage unit 9 generates one or more quantization matrices.

In step S002, the quantization matrix encoding unit 10 encodes each quantization matrix generated by the quantization matrix storage unit 9. In step S003, the integration unit 6 encodes and outputs a header portion of a bitstream. In step S004, the block division unit 1 divides an input image per frame into blocks (8 * 8 pixels).

In step S005, the prediction unit 2 performs prediction by the block, and generates coefficient scanning method selection auxiliary information and prediction errors. In step S006, the coefficient scanning method determination unit 7 determines a coefficient scanning method using the coefficient scanning method selection auxiliary information generated in step S005. In step S007, the transform unit 3 determines a transform block size for the prediction errors generated in step S005 and performs orthogonal transform to generate orthogonal transform coefficients.

In step S008, the quantization matrix selection unit 8 selects one quantization matrix from the quantization matrices generated in step S001 using the coefficient scanning method determined in step S006 and the transform block size determined in step S007. In step S009, the quantization unit 4 scans the orthogonal transform coefficients generated in step S007 using the coefficient scanning method selected in step S006. Further, the quantization unit 4 quantizes the orthogonal transform coefficients to generate quantized coefficients using the quantization matrix selected in step S008.

In step S010, the coefficient encoding unit 5 encodes the quantized coefficients generated in step S009 to generate quantized coefficient code data. In step S011, the image encoding apparatus determines whether all transform blocks in the block are encoded. If all transform blocks are encoded (YES in step S011), the process proceeds to step S012. If all transform blocks are not yet encoded (NO in step S011), the process returns to step S007 to target the next transform block.

In step S012, the image encoding apparatus determines whether all blocks are encoded. If all blocks are encoded (YES in step S012), the image encoding apparatus stops all operations and ends the process. If all blocks are not yet encoded (NO in step S012), the process returns to step S004 to target the next block.

According to the above-described configuration and operations, especially through the processing in steps S001 and S008, a bitstream can be generated which is subjected to quantization processing using an optimal quantization matrix according to the coefficient scanning method.

In the present exemplary embodiment, descriptions are made taking a frame which uses only the intra prediction as an example, however, the present exemplary embodiment is also applicable to in a frame which can use inter prediction.

Further, in the present exemplary embodiment, for convenience of description, the blocks are set to 8 * 8 pixels, and the transform blocks are set to 4 * 4 pixels, however, the present exemplary embodiment is not limited to this example. For example, a block size can be changed to such as 16 * 16 pixels or 32 * 32 pixels, and a shape of a block is not limited to a square, and a rectangle of 16 * 8 pixels or the like may be used.

According to the present exemplary embodiment, generation, storage, and selection of the quantization matrices are performed according to transform block sizes, however, the present exemplary embodiment is not limited to this example. If a transform block size is only one, it is clear that the transform block size is not always necessary.

In the present exemplary embodiment, three types of scanning methods, namely the zigzag scan, the horizontal scan, and the vertical scan illustrated in Figs. 2A to 2C are used, however, the present exemplary embodiment is not limited to this example. A method for scanning in an oblique direction or a method for scanning non-adjacent coefficients may be used.

Further, the transform block size is half of the block size each vertically and horizontally, however, the transform block size may be the same size of the block size, and even finer size than half each vertically and horizontally is also acceptable. The method for encoding each element is described in the encoding of the quantization matrix, however, the present exemplary embodiment is not limited to this method.

In the present exemplary embodiment, three types of scanning methods, namely the zigzag scan, the horizontal scan, and the vertical scan illustrated in Figs. 2A to 2C can be applied to the transform blocks of 4 * 4 pixels. However, if an aspect ratio is 1:1, an image becomes line symmetry in the horizontal/vertical directions. Therefore, encoding is performed on two types of the quantization matrices, that is the quantization matrix for the zigzag scan and the quantization matrix for the horizontal/vertical scans. With respect to the horizontal/vertical scans which are line symmetric each other, a method for sharing the quantization matrix can be used. Further improvement of compression ratio can be realized by using this method. In the present exemplary embodiment, a method for setting a coefficient scanning method on a block basis is described, however, the present exemplary embodiment is not limited to this method. Of course, the coefficient scanning method may be set, for example, on a transform block basis.

Fig. 3 is a block diagram illustrating a configuration of an image decoding apparatus according to a second exemplary embodiment of the present invention. In the present exemplary embodiment, decoding of the bitstream generated in the first exemplary embodiment illustrated in Fig. 1 will be described.

In Fig. 3, a decoding/separation unit 101 decodes header information of an input bitstream, separates necessary codes from the bitstream, and outputs them to a later stage. A quantization matrix decoding unit 108 extracts and decodes quantization matrix code data, which is an encoded quantization matrix, from the header information of the bitstream.

A quantization matrix storage unit 109 temporarily stores the quantization matrices decoded by the quantization matrix decoding unit 108. A coefficient scanning method extraction unit 106 decodes information of prediction on a block basis from the codes separated by the decoding/separation unit 101, and determines a coefficient scanning method based on the information of prediction. A transform size extraction unit 107 decodes information of the transform block sizes from the codes separated by the decoding/separation unit 101.

A quantization matrix selection unit 110 selects one of the quantization matrices stored by the quantization matrix storage unit 109 based on the transform block size decoded by the transform size extraction unit 107 and the coefficient scanning method determined by the coefficient scanning method extraction unit 106.

A coefficient decoding unit 102 decodes quantized coefficient codes from the codes separated by the decoding/separation unit 101 for each transform block size decoded by the transform size extraction unit 107, and reconstructs the quantized coefficients.

An inverse quantization unit 103 performs inverse quantization on the quantized coefficients using the quantization matrix selected by the quantization matrix selection unit 110, performs inverse scanning on the quantized coefficients using the coefficient scanning method determined by the coefficient scanning method extraction unit 106 to reconstruct orthogonal transform coefficients.

Similar to the quantization unit 4 in Fig. 1, the order of the inverse scanning and the inverse quantization is not limited to this example, and the inverse scanning may be performed after the inverse quantization is performed. By appropriately rearranging each element in the quantization matrix, identical orthogonal transform coefficients can be reconstructed. However, if the quantization unit 4 in Fig. 1 and the inverse quantization unit 103 use identical quantization matrix, the order of the scanning and the quantization in the quantization unit 4 in Fig. 1, and the order of the inverse scanning and the inverse quantization in the quantization unit 103 need to be reversed. For example, in the quantization unit 4 in Fig. 1, if the quantization is performed after scanning, the inverse quantization unit 103 needs to perform inverse scanning after performing inverse quantization.

An inverse transform unit 104 performs inverse orthogonal transform which is inverse to the transform unit 3 in Fig. 1, and reconstructs prediction errors. A prediction/reconstruction unit 105 reconstructs image data of the blocks from the prediction errors and decoded image data.

Decoding operations of an image according to the above-described image decoding apparatus will be described below. In the present exemplary embodiment, it is configured such that a moving image bitstream generated in the first exemplary embodiment is input on a frame basis, however it may be configured to input one frame a still image bitstream. In the present exemplary embodiment, for ease of description, only the intra prediction decoding processing will be described. However, the present exemplary embodiment is not limited to this example and is also applicable in inter prediction decoding processing.

In Fig. 3, the input stream data equivalent to one frame is input into the decoding/separation unit 101. In the decoding/separation unit 101, the header information necessary for reconstructing the image is decoded, and the codes to be used at a later stage are separated and output. The quantization matrix code data contained in the header information is input into the quantization matrix decoding unit 108, and the quantization matrix corresponding to the inverse scanning method is reconstructed as the quantization matrix which will be used in the inverse quantization processing at a later stage. According to the present exemplary embodiment, three types of the quantization matrices are reconstructed, that is the first quantization matrix corresponding to 4 * 4 pixels zigzag scan, the second quantization matrix corresponding to 4 * 4 pixels horizontal scan, and the third quantization matrix corresponding to 4 * 4 pixels vertical scan.

According to the present exemplary embodiment, since the transform block size is available in one type, i.e., 4 * 4 pixels, and the scanning methods are available in three types, i.e., zigzag scan, vertical scan, and horizontal scan, and the quantization matrices to be reconstructed are supposed to be available in three types. Actually, however, the quantization matrices equivalent to the number of combinations of the transform block sizes and the scanning methods are reconstructed. For example, in a case where a transform block size of 8 * 8 pixels and the corresponding zigzag scan of 8 * 8 pixels are also used in addition to the present exemplary embodiment, a fourth quantization matrix corresponding to 8 * 8 pixels zigzag scan will be reconstructed. The reconstructed quantization matrices are input into the quantization matrix storage unit 109, and are temporarily stored therein.

Information related to prediction among the codes separated by the decoding/separation unit 101 is input into the coefficient scanning method extraction unit 106, and information indicating coefficient scanning method is generated and output. In the present exemplary embodiment, information generated based on the information related to prediction indicates either coefficient scanning method of the zigzag scan, the horizontal scan, and the vertical scan. For example, if prediction is performed from left-hand of the block, the vertical scan illustrated in Fig. 2C is selected.

On the other hand, among the codes separated by the decoding/separation unit 101, information related to sizes of the transform blocks is input into the transform size extraction unit 107, and a transform block size is extracted. The extracted transform block size is output to the quantization matrix selection unit 110 and the coefficient decoding unit 102.

The quantization matrix selection unit 110 selects one quantization matrix from among the quantization matrices stored in the quantization matrix storage unit 109 according to the input information of the coefficient scanning method and transform block size, and outputs the selected quantization matrix to the inverse quantization unit 103. For example, according to the present exemplary embodiment, if the coefficient scanning method is the horizontal scan and the transform block size is 4 * 4 pixels, the second quantization matrix is selected from the quantization matrix storage unit 109, and is output to the inverse quantization unit 103.

Among the codes separated by the decoding/separation unit 101, the quantized coefficient code data is input into the coefficient decoding unit 102. Then, the coefficient decoding unit 102 receives inputs of the transform block size extracted by the transform size extraction unit 107, decodes the quantized coefficient code data for each transform block based on the transform block size, reconstructs quantized coefficients, and outputs them to the inverse quantization unit 103.

The inverse quantization unit 103 receives inputs of the information of the coefficient scanning method determined by the coefficient scanning method extraction unit 106, the quantization matrix selected by the quantization matrix selection unit 110, and the quantized coefficients reconstructed by the coefficient decoding unit 102. Then, the inverse quantization unit 103 performs inverse quantization using the quantization matrix, reconstructs the orthogonal transform coefficients, performs inverse scanning on the orthogonal transform coefficients using the coefficient scanning method, and outputs the inverse-scanned orthogonal transform coefficients to the inverse transform unit 104.

The inverse transform unit 104 receives the inverse-scanned orthogonal transform coefficients, and performs inverse orthogonal transform which is inverse to the transform unit 3 in Fig. 1, reconstructs prediction errors, and outputs the reconstructed prediction errors to the prediction/reconstruction unit 10. The prediction/reconstruction unit 105 performs prediction on the input prediction errors in accordance with the intra prediction method decoded from the decoded neighboring pixel data to reconstruct and output the block-based image data.

Fig. 6 is a flowchart illustrating image decoding processing in the image decoding apparatus according to the second exemplary embodiment. First, in step S101, the decoding/separation unit 101 decodes header information.

In step S102, the quantization matrix decoding unit 108 decodes quantization matrix code data contained in the header information, and reconstructs the quantization matrix to be used in inverse quantization processing at a later stage. According to the present exemplary embodiment, the quantization matrix decoding unit 108 sequentially decodes and reconstructs three types of the quantization matrices corresponding to respective scanning methods of the zigzag scan, the vertical scan, and the horizontal scan of 4 * 4 pixels. It is assumed that, for example, there are two types of the transform block size, i.e., 8 * 8 pixels and 4 * 4 pixels, and three types of scanning methods, i.e., the zigzag scan, the vertical scan, and the horizontal scan, are applied for respective transform blocks. In that case, first the quantization matrix decoding unit 108 sequentially decodes each element of three types of the quantization matrices corresponding to the respective scanning methods with respect to the transform block of 4 * 4 pixels. Then, the quantization matrix decoding unit 108 sequentially decodes each element of three types of the quantization matrices corresponding to the respective scanning methods with respect to the transform block of 8 * 8 pixels. However, decoding order is not limited to this example.

In step S103, the transform size extraction unit 107 decodes information related to dimensions of each transform block in the block to be decoded, and extracts a transform block size. In step S104, the coefficient scanning method extraction unit 106 decodes information related to the intra prediction to generate coefficient scanning method information from the content thereof.

In step S105, the quantization matrix selection unit 110 selects a quantization matrix to be applied to the transform block based on the transform block size extracted in step S103 and the coefficient scanning method information extracted in step S104.

In step S106, the coefficient decoding unit 102 decodes quantized coefficient code data on a transform block basis to reconstruct quantized coefficients. In step S107, the inverse quantization unit 103 performs inverse quantization on the quantized coefficients reconstructed in step S106 using the quantization matrix selected in step S105. Further, the inverse quantization unit 103 performs inverse scanning on the quantized coefficients based on the coefficient scanning method information generated in step S104 to reconstruct orthogonal transform coefficients.

In step S108, the inverse transform unit 104 performs inverse orthogonal transform on the orthogonal transform coefficients reconstructed in step S107 to reconstruct the prediction errors. In step S109, the image decoding apparatus determines whether all transform blocks in the block are decoded. If all transform blocks are decoded (YES in step S109), the process proceeds to step S110. If all transform blocks are not yet decoded (NO in step S109), the process returns to step S106 to target the next transform block.

In step S110, the prediction/reconstruction unit 105 performs prediction from the decoded neighboring pixel data, and adds the result to the prediction errors reconstructed in step S108 to reconstruct decoded image of the block. In step S111, the image decoding apparatus determines whether all blocks are decoded. If all blocks are decoded (YES in step S111), the image decoding apparatus stops all operations and ends the process. If all blocks are not yet decoded (NO in step S111), the process returns to step S103 to target the next block.

According to the above-described configuration and operations, a bitstream generated by the quantization processing using the optimal quantization matrix according to the coefficient scanning method generated in the first exemplary embodiment can be decoded, and the reconstructed image can be obtained.

Further, similar to the first exemplary embodiment, a size of a block, a size of a transform block, and a shape of a block are not limited to the above examples. Further, according to the present exemplary embodiment, a method for setting a coefficient scanning method on a block-by-block basis is described, however, the present exemplary embodiment is not limited to this example, and may be set on, for example, a transform block basis of course. In this case, the processing in step S110 will be performed before step S109.

Further, according to the present exemplary embodiment, decoding, storage, and selection of the quantization matrix are performed according to a transform block size, however, the present exemplary embodiment is not limited to this example, and it is clear that the transform block size is not always necessary in case of one transform block size.

Further, if a plurality of pieces of the quantization matrix code data is included in a bitstream of one sequence, the quantization matrix may be updated. The quantization matrix code data is detected by the decoding/separation unit 101, and is decoded by the quantization matrix decoding unit 108. Data of the decoded quantization matrix is replaced by the relevant quantization matrix in the quantization matrix storage unit 109. At this time, all quantization matrices may be rewritten, or by discriminating the scanning method and the transform block size of the quantization matrix to be rewritten and a part thereof may be changed.

Fig. 8 is a block diagram illustrating an image encoding apparatus according to a third exemplary embodiment. In Fig. 8, same reference numerals are assigned to the parts which perform similar functions to those in Fig. 1 according to the first exemplary embodiment, and thus the descriptions thereof will not be repeated. A quantization unit 24 quantizes the quantization matrix selected by the quantization matrix selection unit 8. The quantized coefficients can be obtained by the quantization. The difference from the quantization unit 4 in Fig. 1 is that the coefficient scanning method selection auxiliary information is not input from the coefficient scanning method determination unit 7, and scanning is not performed.

A coefficient encoding unit 25 scans and encodes the quantized coefficients using the coefficient scanning method determined by the coefficient scanning method determination unit 7 to generate quantized coefficient code data. The encoding method is not limited especially, and Huffman coding or arithmetic coding can be used. The difference from the coefficient encoding unit 5 in Fig. 1 is that the coefficient scanning method selection auxiliary information is input from the coefficient scanning method determination unit 7, and the scanning of the quantized coefficients is performed.

A quantization matrix storage unit 29 stores the quantization matrix for each scanning method and transform block size. The difference from the quantization matrix storage unit 9 in Fig. 1 is that elements of the quantization matrix are stored in a two-dimensional array in correspondence with the transform block, instead of a one-dimensional array.

A quantization matrix selection unit 28 inputs the coefficient scanning method selection auxiliary information and the transform block size from the coefficient scanning method determination unit 7, and selects the relevant quantization matrix stored in the quantization matrix storage unit 29. The difference from the quantization matrix selection unit 8 in Fig. 1 is that the quantization matrix storage unit 29 selects and outputs the two-dimensional quantization matrix.

A quantization matrix encoding unit 30 differs from the quantization matrix encoding unit 10 Fig. 1 in that the quantization matrix encoding unit 30 arranges the elements of the quantization matrix in a one-dimensional array in accordance with each corresponding scanning method and then encodes the elements of the quantization matrix.

Encoding operations of an image in the above-described image encoding apparatus will be described below. The block division unit 1, the prediction unit 2, the transform unit 3, the coefficient scanning method determination unit 7, and the quantization matrix selection unit 8 operate in a manner as described in the first exemplary embodiment, to generate the coefficient scanning method selection auxiliary information and the transform block sizes.

The quantization matrix storage unit 29 stores the two-dimensional quantization matrices for each scanning method and transform block size. The quantization matrix encoding unit 30 sequentially reads out the quantization matrices according to the scanning method and transform block sizes from the quantization matrix storage unit 29, and scans and then encodes the quantization matrices in accordance with the scanning method to generate the quantization matrix code data.

The quantization matrix selection unit 28 receives the inputs of the quantization matrices based on the coefficient scanning method selection auxiliary information and the transform block size, and outputs them to the quantization unit 24.

The quantization unit 24 quantizes the orthogonal transform coefficients output from the transform unit 3 as two-dimensional data using the quantization matrix selected by the quantization matrix selection unit 28 to generate quantized coefficients. The generated quantized coefficients are input into the coefficient encoding unit 25.

The coefficient encoding unit 25 scans and encodes the quantized coefficients generated by the quantization unit 24 using the scanning method determined by the coefficient scanning method determination unit 7 to generate quantized coefficient code data and outputs the quantized coefficient code data to the integration unit 6.

Fig. 10 is a flowchart illustrating image decoding processing in the image encoding apparatus according to the third exemplary embodiment. In Fig. 10, same reference numerals are assigned to the parts which perform similar functions to those in Fig. 5 according to the first exemplary embodiment, and thus the descriptions thereof will not be repeated.

In step S021, the quantization matrix storage unit 29 generates one or more two-dimensional quantization matrices. In step S022, the quantization matrix encoding unit 30 scans the respective quantization matrices generated by the quantization matrix storage unit 29 according to each scanning method, and encodes the quantization matrices. In step S029, the quantization unit 24 quantizes the orthogonal transform coefficients generated in step S007 using the quantization matrix selected in step S008 to generate the quantized coefficients.

In step S030, the coefficient encoding unit 25 scans and encodes the quantized coefficients generated in step S029 using the coefficient scanning method selected in step S006 to generate quantized coefficient code data.

According to the above-described configuration and operations, similar to the first exemplary embodiment, even when quantized coefficient scanning is performed after quantization, a bitstream, which is subjected to quantization processing, can be generated using an optimal quantization matrix according to the coefficient scanning method. According to the present exemplary embodiment, generation, storage, and selection of the quantization matrices are performed according to transform block sizes, however, the present exemplary embodiment is not limited to this example. If a transform block size is only one, it is clear that the transform block size is not always necessary.

Fig. 9 is a block diagram illustrating an image decoding apparatus according to a fourth exemplary embodiment. In Fig. 9, same reference numerals are assigned to the parts which perform similar functions those in Fig. 3 according to the second exemplary embodiment, and thus the descriptions thereof will not be repeated.

A coefficient decoding unit 122 decodes the quantized coefficients from the codes separated by the decoding/separation unit 101, for each transform block size decoded by the transform size extraction unit 107, performs inverse scanning thereon using the coefficient scanning method determined by the coefficient scanning method extraction unit 106, and reconstructs the quantized coefficients. An inverse quantization unit 123 performs inverse quantization on the quantized coefficients using the quantization matrix selected by the quantization matrix selection unit 110, and reconstructs the orthogonal transform coefficients. A quantization matrix decoding unit 128 extracts and decodes the quantization matrix code data from header information of the bitstream. The difference from the quantization matrix decoding unit 108 in Fig. 3 is that the quantization matrix decoding unit 128 generates two-dimensional quantization matrices in accordance with the scanning method after decoding.

A quantization matrix storage unit 129 temporarily stores the two-dimensional quantization matrices decoded by the quantization matrix decoding unit 128. The difference from the quantization matrix storage unit 109 in Fig. 3 is that elements of the quantization matrix are stored in a two-dimensional array instead of one-dimensional array, in correspondence with the transform block.

A quantization matrix selection unit 130 selects one of the quantization matrices stored in the quantization matrix storage unit 129 based on the transform block size decoded by the transform size extraction unit 107 and the coefficient scanning method determined by the coefficient scanning method extraction unit 106. The difference from the quantization matrix selection unit 110 in Fig. 3 is that the quantization matrix selection unit 130 selects and outputs the two-dimensional quantization matrix.

Decoding operations of an image in the above-described image decoding apparatus will be described below.

The quantization matrix code data contained in the header information separated by the decoding/separation unit 101 is input into the quantization matrix decoding unit 128, and the two-dimensional quantization matrix corresponding to the inverse scanning method is reconstructed as the quantization matrix to be used in the quantization processing at a later stage. The reconstructed quantization matrix is input and temporarily stored in the quantization matrix storage unit 129.

The coefficient decoding unit 122 receives inputs of the quantized coefficient code data, information of the transform block size extracted by the transform size extraction unit 107, and the coefficient scanning method determined by the coefficient scanning method extraction unit 106. Then, the coefficient decoding unit 122 decodes the quantized coefficient code data for each transform block based on the transform block size, reconstructs the quantized coefficients, performs inverse scanning using the determined coefficient scanning method, and outputs the two-dimensional quantized coefficients to the inverse quantization unit 123.

The quantization matrix selection unit 130 selects one from among the two-dimensional quantization matrices stored in the quantization matrix storage unit 129 according to the input information of coefficient scanning method and transform block size, and outputs the selected two-dimensional quantization matrix to the inverse quantization unit 123.

The inverse quantization unit 123 receives inputs of the quantization matrix selected by the quantization matrix selection unit 130 and the quantized coefficients reconstructed by the coefficient decoding unit 122. Then, the inverse quantization unit 123 performs inverse quantization using the quantization matrix to reconstruct the orthogonal transform coefficients, and outputs them to the inverse transform unit 104.

Fig. 11 is a flowchart illustrating image decoding processing in the image decoding apparatus according to the fourth exemplary embodiment. Same reference numerals are assigned to the parts which perform similar functions to those in Fig. 6 according to the second exemplary embodiment, and thus the descriptions thereof will not be repeated.

In step S122, the quantization matrix decoding unit 128 decodes the quantization matrix code data, and stores the quantization matrices subjected to the inverse scanning in accordance with the corresponding scanning method in the quantization matrix storage unit 129.

In step S126, the coefficient decoding unit 122 decodes the quantized coefficient code data on a transform block basis. Further, the coefficient decoding unit 122 performs inverse scanning on the quantized coefficient code data based on the coefficient scanning method information generated in step S104 to reconstruct the quantized coefficients. In step S127, the inverse quantization unit 123 performs inverse quantization on the quantized coefficients reconstructed in step S106 using the quantization matrix selected in step S105 to reconstruct the orthogonal transform coefficients.

According to the above-described configuration and operations, a bitstream generated by the quantization processing using the optimal quantization matrix according to the coefficient scanning method generated in the third exemplary embodiment can be decoded, and the reconstructed image can be obtained.

Fig. 12 is a block diagram illustrating an image encoding apparatus according to a fifth exemplary embodiment. In Fig. 12, same reference numerals are assigned to the parts which perform the functions similar to those in Fig. 1 according to the first exemplary embodiment, and thus the descriptions thereof will not be repeated.

An encoding control information generation unit 51 generates quantization matrix encoding method information indicating how to encode each quantization matrix. A quantization matrix encoding unit 40 encodes the quantization matrices stored in the quantization matrix storage unit 9 based on the quantization matrix encoding method information generated by the encoding control information generation unit 51 to generate quantization matrix code data.

An integration unit 46 is similar to the integration unit 6 in Fig. 1 and generates codes related to header information, prediction, and transform. The difference from the integration unit 6 is that the integration unit 46 inputs the quantization matrix encoding method information from the encoding control information generation unit 51, and encodes it.

Encoding operations of an image in the above-described image encoding apparatus will be described below.

The encoding control information generation unit 51, first, generates the quantization matrix encoding method information indicating how to encode each quantization matrix. According to the present exemplary embodiment, if the quantization matrix encoding method information is zero, a method for directly encoding each element of the quantization matrix is used. If the quantization matrix encoding method information is one, a method for encoding a difference from the preceding element is used. Further if the quantization matrix encoding method information is two, a method for encoding a difference from an element at the same position of the element of the previously encoded quantization matrix is used.

However, the encoding method of each element of the quantization matrix is not limited to this example, and in the same way as the DPCM, a prediction difference may be encoded after encoding the first element and a difference between the second element and the first element. Further, a combination of the quantization matrix encoding method information and the encoding methods of the quantization matrices is not limited to this example. The generation method for the quantization matrix encoding method information is not limited especially. The quantization matrix encoding method information may be input by the user, or may use the one specified in advance as a fixed value, or may be calculated from characteristics of the quantization matrices stored in the quantization matrix storage unit 9. The generated quantization matrix encoding method information is input into the quantization matrix encoding unit 40 and the integration unit 46.

The quantization matrix encoding unit 40 encodes each quantization matrix stored in the quantization matrix storage unit 9 based on the input quantization matrix encoding method information, and generates and outputs quantization matrix code data to the integration unit 46.

The integration unit 46 encodes the quantization matrix encoding method information generated by the encoding control information generation unit 51, generates a quantization matrix encoding information code, and outputs the quantization matrix encoding information code by adding in the header information or the like. The encoding method is not limited especially, and Huffman coding or arithmetic coding can be used. Fig. 7B illustrates an example of a bitstream containing the quantization matrix encoding information code. The quantization matrix encoding information code may be entered into any of the sequence header, the picture header, or the like, but exists at the front of each piece of the quantization matrix code data.

Fig. 14 is a flowchart illustrating image decoding processing in the image encoding apparatus according to the fifth exemplary embodiment. In Fig. 14, same reference numerals are assigned to the parts which perform the functions similar to those in Fig. 5 according to the first exemplary embodiment, and thus the descriptions thereof will not be repeated.

In step S051, the encoding control information generation unit 51 determines the quantization matrix encoding method to be used in step S052 at a later stage. In step S052, the quantization matrix encoding unit 40 encodes the quantization matrices generated in step S001 based on the quantization matrix encoding method determined in step S051. In step S053, the quantization matrix encoding unit 40 encodes the quantization matrix encoding method information to generate the quantization matrix encoding information code, and outputs the quantization matrix encoding information code by adding in the header portion similarly to other codes.

According to the above-described configuration and operations, each quantization matrix can be encoded by an optimal encoding method, and a bitstream can be generated, which is subjected to quantization processing using an optimal quantization matrix according to a coefficient scanning method.

Fig. 13 is a block diagram illustrating an image decoding apparatus according to a sixth exemplary embodiment. In Fig. 13, same reference numerals are assigned to the parts which perform the functions similar to those in Fig. 3 according to the second exemplary embodiment, and thus the descriptions thereof will not be repeated.

A decoding/separation unit 131 decodes header information of the input bitstream, separates a necessary code from the bitstream, and outputs the code to a later stage. The difference from the decoding/separation unit 101 in Fig. 3 is that the decoding/separation unit 131 separates the quantization matrix encoding information code from the header information of the bitstream and outputs the code to a later stage.

A decoding control information decoding unit 151 decodes the quantization matrix encoding information code separated by the decoding/separation unit 131 to reconstruct information of the quantization matrix encoding method. A quantization matrix decoding unit 138 decodes the quantization matrix code data separated by the decoding/separation unit 131 from the header information of the bitstream based on the information of the quantization matrix encoding method.

In Fig. 13, the input stream data equivalent to one frame is input into the decoding/separation unit 131, and the header information necessary for reconstructing an image is decoded. Further, the code to be used at a later stage is separated and output. The quantization matrix encoding information code contained in the header information is input into the decoding control information decoding unit 151, and information of the encoding method of the quantization matrix is reconstructed. Then, the information of the encoding method of the reconstructed quantization matrix is input into the quantization matrix decoding unit 138.

On the other hand, the quantization matrix code data contained in the header information is input into the quantization matrix decoding unit 138. Then, the quantization matrix decoding unit 138 decodes the quantization matrix code data based on the information of the encoding method of the input quantization matrices, to reconstruct the quantization matrix to be used in inverse quantization processing at a later stage.

Fig. 15 is a flowchart illustrating the image decoding processing in the image decoding apparatus according to the sixth exemplary embodiment. Same reference numerals are assigned to the parts which perform the functions similar to those in Fig. 6 according to the second exemplary embodiment, and thus the descriptions thereof will not be repeated.

In step S101, the decoding/separation unit 131 decodes the header information. In step S151, the decoding control information decoding unit 151 decodes the quantization matrix encoding information code contained in the header information to reconstruct the information of the encoding method of the quantization matrix. In step S152, the quantization matrix decoding unit 138 decodes the quantization matrix code data contained in the header information based on the information of the encoding method of the quantization matrix reconstructed in step S151, and reconstructs the quantization matrix.

According to the above-described configuration and operations, each quantization matrix, which is generated in the fifth exemplary embodiment, can be encoded by an optimal encoding method. Further, a bitstream, which is subjected to quantization processing using an optimal quantization matrix according to a coefficient scanning method can be decoded, and the reconstructed image can be obtained.

According to the above-described exemplary embodiments, it is assumed that the respective processing units illustrated in Fig. 1 and Fig. 3 are constituted by hardware. However, the processing performed by the respective processing units illustrated in Fig. 1 and Fig. 3 may be constituted by a computer program.

Fig. 16 is a block diagram illustrating an example of a hardware configuration of a computer applicable to an image display device according to the above-described exemplary embodiments.

The CPU 1401 performs control of the entire computer using a computer program or data stored in a RAM 1402 or a ROM 1403, executes each processing described above as to be performed by an image processing apparatus according to the above-described exemplary embodiments. In other words, the CPU 1401 will function as respective processing units illustrated in Fig. 1 and Fig. 3.

The RAM 1402 includes an area for temporarily storing a computer program and data loaded from an external storage device 1406, and data acquired from the outside via an interface (I/F) 1407. Further, the RAM 1402 includes a working area which is used when the CPU 1401 executes various types of processing. In other words, the RAM 1402 can be allocated as, for example, a frame memory, or can provide other various areas as appropriate.

In the ROM 1403, setting data and a boot program of the computer are stored. An operation unit 1404 includes a keyboard, a mouse, and the like, and a user of the computer can operate the CPU 1401 to input various types of instructions thereto. A display unit 1405 displays processing results obtained by the CPU 1401. Further, the display unit 1405 includes a display device, such as a liquid crystal display.

The external storage device 1406 is a high-capacity information storage device as represented by a hard disk drive device. The external storage device 1406 stores an operating system (OS) and a computer program for causing the CPU 1401 to implement the functions of the respective units illustrated in Fig. 1 and Fig. 3. Further, the external storage device 1406 may store each image data to be a processing target.

The computer program and the data stored in the external storage device 1406 are loaded into the RAM 1402 as appropriate, in accordance with the control by the CPU 1401, and becomes the processing target by the CPU 1401. To the I/F 1407, networks such as a local area network (LAN) and an Internet, other devices such as a projection device and a display device can be connected, and the computer can acquire and deliver various information pieces via the I/F 1407. A bus 1408 connects the above-described respective units to one another.

The operations of the above-described configurations are controlled by the CPU 1401 taking the initiative to perform the control of the operations described in the above-described flowcharts.

The objective of the present invention is also achieved by supplying a storage medium which records codes of a computer program that implements the above-described functions to a system, and by causing the system to read out and execute the codes of the computer program. In this case, the codes themselves of the computer program read out from the storage medium implement the functions of the above-described exemplary embodiments, and the storage medium that stores the codes of the computer program constitutes the present invention. Further, a case where an operating system (OS) or the like running on the computer performs a part or the whole of actual processing based on instructions of the codes of the program, and the above-described functions are implemented by the processing is also included in the present invention.

Further, the present invention may be also realized by the following forms. In other words, computer program codes read out from a storage medium are written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. Then, a case where the CPU provided in the function expansion card or the function expansion unit performs a part or the whole of actual processing based on the instructions of the codes of the computer program, and the above-described functions are implemented is also included in the present invention.

In a case where the present invention is applied to the above-described storage medium, the codes of the computer program corresponding to the processing in the above described flowcharts will be stored in the storage medium.

While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all modifications, equivalent structures, and functions.

This application claims the benefit of Japanese Patent Application No. 2011-144069, filed June 29, 2011, which is hereby incorporated by reference herein in its entirety.

Claims

An image encoding apparatus comprising:
a division unit configured to divide an input image into a plurality of blocks;
a prediction unit configured to generate a prediction error on a block-by-block basis with respect to the blocks divided by the division unit;
a transform unit configured to perform orthogonal transform on the prediction error to generate transform coefficients;
a quantization matrix storage unit configured to store quantization matrices to be used when the transform coefficients are quantized;
a scanning method determination unit configured to determine a scanning method for scanning the quantized transform coefficients in order to encode the quantized transform coefficients;
a quantization matrix selection unit configured to select one of the plurality of quantization matrices stored in the quantization matrix storage unit based on the scanning method determined by the scanning method determination unit;
a quantization unit configured to scan transform coefficients by the determined scanning method and generate quantized coefficients from the transform coefficients using the selected quantization matrix; and
a coefficient encoding unit configured to encode the quantized coefficients.
The image encoding apparatus according to claim 1, wherein the scanning method determination unit determines a scanning method based on a prediction direction of a target block of prediction by the prediction unit.
The image encoding apparatus according to claim 1, further comprising:
a quantization matrix encoding unit configured to encode the quantization matrices; and
an integration unit configured to integrate the quantized coefficients encoded by the coefficient encoding unit and the quantization matrices encoded by the quantization matrix encoding unit.
The image encoding apparatus according to claim 3, wherein the quantization matrix encoding unit determines whether to encode the quantization matrices or to encode a difference from a first quantization matrix with respect to the plurality of quantization matrices.
An image decoding apparatus comprising:
a separation unit configured to decode header information of an input bitstream and separate necessary codes;
a scanning method determination unit configured to determine a scanning method from the codes separated by the separation unit;
a coefficient decoding unit configured to decode quantized coefficients from the code separated by the separation unit;
a quantization matrix selection unit configured to select a quantization matrix based on the determined scanning method; and
an inverse quantization unit configured to perform inverse quantization on the decoded quantized coefficients using the selected quantization matrix to generate transform coefficients and perform inverse scanning on the transform coefficients by a determined inverse scanning method.
A method for encoding an image executed in an image encoding apparatus, the method comprising:
dividing an input image into a plurality of blocks;
generating a prediction error on a block-by-block basis with respect to the divided blocks;
performing orthogonal transform on the prediction error to generate transform coefficients;
storing in a storage unit quantization matrices to be used when the transform coefficients are quantized;
determining a scanning method for scanning the quantized transform coefficients in order to encode the quantized transform coefficients;
selecting one of the plurality of quantization matrices stored in the storage unit based on the determined scanning method;
scanning transform coefficients by the determined scanning method and generating quantized coefficients from the transform coefficients using the selected quantization matrix; and
encoding the quantized coefficients.
A method for decoding an image executed in an image decoding apparatus, the method comprising:
decoding header information of an input bitstream and separating necessary codes;
determining a scanning method from the separated codes;
decoding quantized coefficients from the separated code;
selecting a quantization matrix based on the determined scanning method; and
performing inverse quantization on the decoded quantized coefficients using the selected quantization matrix to generate transform coefficients and performing inverse scanning on the transform coefficients by a determined inverse scanning method.
A program causing a computer to read out and execute the program to function as an image encoding apparatus according to claim 1.
A program causing a computer to read out and execute the program to function as an image decoding apparatus according to claim 5.