JP6541763B2 - Image decoding apparatus, image decoding method and program - Google Patents

Description

The present invention relates to an image decoding apparatus, an image decoding method, and a program.

As a method of compressing and recording moving images, H. H.264 / MPEG-4 AVC (hereinafter H.264) is known. (Non-patent document 1)
H. In H.264, each coefficient of the quantization matrix can be changed to an arbitrary value by encoding scaling_list information. According to Non-Patent Document 1, each coefficient of the quantization matrix can have an arbitrary value by adding delta_scale which is a difference value to the immediately preceding coefficient.

Recently, H. As a successor to H.264, activities have been launched to carry out international standardization of more efficient coding schemes. JCT-VC (Joint Collaborative Team on Video Coding) was established between ISO / IEC and ITU-T. In this JCT-VC, standardization is advanced as a High Efficiency Video Coding (HEVC) coding scheme (hereinafter, HEVC).

In this method, in order to improve the coding efficiency, in addition to the conventional discrete cosine-based orthogonal transform method, an orthogonal transform method such as discrete sine transform is being studied. In addition, methods such as a conversion skip technique that does not perform orthogonal transformation are being studied in order to improve the coding efficiency in artificial pictures such as PC screens and games as well as natural pictures shot with a camera or the like. (Non-patent document 2)

ITU-T H.2. 264 (03/2010) Advancedvideo coding for generic audiovisual services JCT-VC contribution JCTVC-I 1003. doc Internet <http: // phenix. int-evry. fr / jct / doc_end_user / documents / 9_Geneva / wg11 />

Generally, in image compression, to improve the subjective image quality, among the conversion coefficients after orthogonal transformation, the coefficient corresponding to the low frequency component is quantized with a smaller value, and the coefficient corresponding to the high frequency component is larger It is often quantized. In this case, it is general to perform weighted quantization by frequency components using a quantization matrix as shown in FIG. 5 (a). The quantization matrix 500 has smaller coefficients corresponding to low frequency components and larger coefficients corresponding to high frequency components.

On the other hand, in the case of HEVC, orthogonal transform is not performed particularly when the above-described transform skip technique is used, and therefore all coefficients to be quantized have the same frequency component. For this reason, it is not preferable to perform the above-mentioned weighted quantization when using the conversion skip technique, and use a flat quantization matrix 501 in which all coefficients have the same value as shown in FIG. 5 (b). It is necessary to perform quantization processing. Therefore, it is desirable to be able to encode the information of the flat quantization matrix 501 with a smaller code amount, but in the above-mentioned HEVC in particular, there is no mechanism that can efficiently encode, and as a result, the code amount is unnecessarily large. It had become.

Therefore, the present invention has been made to solve the above-mentioned problems, and by introducing a method for efficiently encoding a flat quantization matrix with a smaller code amount, highly efficient quantization matrix coding -The purpose is to realize decryption.

In order to solve the above-mentioned problems, the image decoding apparatus of the present invention has the following configuration. That is, acquisition means for acquiring quantization coefficients and conversion skip determination information indicating whether or not the quantization coefficients are orthogonally transformed coefficients from a bit stream, and all coefficients are identical from the bit stream Inverse quantization means for inversely quantizing the quantization coefficient based on the quantization matrix obtained according to the acquired reference information when acquiring reference information indicating a quantization matrix having a value of
An adjustment to adjust a bit depth of the coefficient inversely quantized by the inverse quantization unit, when the conversion skip determination information acquired by the acquisition unit indicates that the quantization coefficient is a coefficient that has not been orthogonally transformed Means, and decoding means for decoding an image from the data obtained by the adjusting means;
And the inverse quantization means may orthogonally transform the quantization coefficient even when the conversion skip determination information acquired by the acquisition means indicates a coefficient to which the quantization coefficient is orthogonally transformed. Even when it is indicated that there is no coefficient, when acquiring the reference information, the quantization coefficient is inversely quantized based on a quantization matrix obtained according to the reference information.

Furthermore, the image decoding method of the present invention has the following configuration. That is, an acquisition step of acquiring quantization coefficients and conversion skip determination information indicating whether or not the quantization coefficients are orthogonally transformed coefficients from the bit stream, and all coefficients are identical from the bit stream In the case of acquiring reference information indicating a quantization matrix having a value of 1, the inverse quantization step of inversely quantizing the quantization coefficient based on the quantization matrix obtained according to the acquired reference information, and the acquisition step Adjusting the bit depth of the coefficient inversely quantized in the inverse quantization step, if the obtained conversion skip determination information indicates that the quantization coefficient is a coefficient that has not been orthogonally transformed; Decoding the image from the data obtained in the adjusting step; There are, converted skip decision information obtained in said obtaining step, in the case shown the coefficients the quantized coefficients are orthogonally transformed even, in the case shown in that the quantization coefficient is a coefficient that has not been orthogonally converted Also, when acquiring the reference information, the quantization coefficient is inversely quantized based on a quantization matrix obtained according to the reference information.

According to the present invention, the amount of code for quantization matrix coding can be reduced, and highly efficient coding / decoding becomes possible.

Block diagram showing the configuration of the image coding apparatus according to the first embodiment Block diagram showing the configuration of the image decoding apparatus in the second embodiment Block diagram showing the configuration of the image coding apparatus in the third embodiment Block diagram showing the configuration of the image decoding apparatus in the fourth embodiment (A), (b) A diagram showing an example of a quantization matrix Flow chart showing image coding process in the image coding apparatus according to the first embodiment A flowchart showing an image decoding process in the image decoding apparatus according to the second embodiment Flow chart showing image coding processing in the image coding apparatus according to the third embodiment The flowchart which shows the picture decoding processing in the picture decoding device which relates to the execution form 4 (A), (b), (c) A diagram showing an example of a bitstream structure Diagram showing an example of the relationship between a quantization matrix to be encoded and quantization matrix reference information A block diagram showing a configuration example of hardware of a computer applicable to the image encoding device and the decoding device of the present invention

Hereinafter, the present invention will be described in detail based on its preferred embodiments with reference to the attached drawings. In addition, the structure shown in the following embodiment is only an example, and this invention is not limited to the illustrated structure.

First Embodiment
Hereinafter, embodiments of the present invention will be described using the drawings. FIG. 1 is a block diagram showing an image coding apparatus according to the present embodiment.

In FIG. 1, reference numeral 101 denotes a terminal for inputting image data (hereinafter referred to as an image). A block division unit 102 divides the input image into a plurality of blocks. A prediction unit 103 predicts each block divided by the block division unit 102 in units of blocks, determines a prediction method, calculates a difference value according to the prediction method, and calculates a prediction error. The prediction unit 103 also performs intra prediction or motion compensation prediction. In general, intra prediction is realized by selecting an optimum one of a plurality of reference methods as a prediction method in order to calculate a prediction value from surrounding pixel data.

A transformation unit 104 performs orthogonal transformation on the prediction error of each block, and performs orthogonal transformation on the basis of the size of the input block or the size obtained by dividing it, and calculates orthogonal transformation coefficients. Below, the block which performs orthogonal transformation is called a conversion block. The orthogonal transform is not particularly limited, but discrete cosine transform, Hadamard transform, or the like may be used. Further, in the present embodiment, for the purpose of explanation, the prediction error of the block unit of 8 × 8 pixels is divided into vertical and horizontal, and orthogonal transformation is performed in units of 4 × 4 pixel conversion block. The shape is not limited to this. Orthogonal transformation may be performed using a transformation block of the same size as the block, or orthogonal transformation may be performed using a transformation block obtained by dividing the block into units smaller than the vertical and horizontal division.

A quantization matrix holding unit 107 generates and stores a quantization matrix. There is no particular limitation on the method of generating the quantization matrix to be stored, but the user may input the quantization matrix, or even if it is calculated from the characteristics of the input image, the one specified in advance is used as the initial value. Of course it is good. In this embodiment, it is assumed that a two-dimensional quantization matrix corresponding to the 4 × 4 pixel conversion block as shown in FIGS. 5A and 5B is generated and stored.

A quantization unit 105 quantizes the orthogonal transformation coefficient using the quantization matrix stored in the quantization matrix storage unit 107. A quantization coefficient can be obtained by this quantization. A coefficient coding unit 106 encodes the quantization coefficient obtained in this way to generate quantization coefficient code data. Although the method of encoding is not particularly limited, Huffman coding, arithmetic coding, etc. can be used. A quantization matrix determination unit 108 determines whether the quantization matrix stored in the quantization matrix storage unit 107 matches a predetermined quantization matrix. More specifically, it is determined whether the quantization matrix to be encoded is an already encoded quantization matrix, a predetermined quantization matrix, or a flat quantization matrix in which all the coefficients have the same value. judge. It is assumed that this predetermined quantization matrix is stored in the quantization matrix holding unit 107. However, it is not limited to this. For example, it may be stored in the quantization matrix determination unit 108.

A quantization matrix header encoding unit 109 encodes information indicating the determination in the quantization matrix determination unit 108 and other information related to the quantization matrix as a header to generate a quantization matrix header code. A quantization matrix coefficient encoding unit 110 encodes each coefficient of a quantization matrix to be encoded to generate a quantization matrix coefficient code.

An integrated coding unit 111 generates header information and codes relating to prediction and conversion, and integrates code data generated by the above-described coding units to form and output a bit stream. Specifically, the quantization coefficient code data generated by the coefficient coding unit 106, the quantization matrix header code generated by the quantization matrix header coding unit 109, and the quantization matrix coefficient code generated by the quantization matrix coefficient coding unit 110. Integrate quantization matrix coefficient codes. Here, the code relating to prediction and conversion is, for example, a code such as the selected prediction method or the state of division of the conversion block. Reference numeral 112 denotes a terminal for outputting the bit stream generated by the integration coding unit 111 to the outside.

The image encoding operation of the image encoding apparatus will be described below. Although moving image data is input in units of pictures in the present embodiment, still image data of one picture may be input. Further, in the present embodiment, only the process of intra prediction encoding is described for ease of explanation, but the present invention is not limited to this and is also applicable to the process of inter prediction encoding. Although the block division unit 102 is described as being divided into blocks of 8 × 8 pixels for the purpose of description in the present embodiment, the present invention is not limited to this.

<Coding of quantization matrix coefficient>
First, the quantization matrix holding unit 107 generates a quantization matrix. The quantization matrix is determined according to the transform block size which is the size of the orthogonal transform used for the encoding process. The method of determining the coefficients of the quantization matrix is not particularly limited. For example, a predetermined initial value may be used or may be set individually. Also, it may be set and generated according to the characteristics of the image.

The quantization matrix holding unit 107 holds the quantization matrix generated in this manner. FIGS. 5A and 5B are examples of quantization matrices corresponding to 4 × 4 pixel conversion blocks. In order to simplify the description, it is assumed that the configuration has 16 pixels corresponding to a 4 × 4 pixel conversion block, and each square in a bold frame represents a coefficient. In this embodiment, the quantization matrix shown in FIGS. 5A and 5B is assumed to be held in a two-dimensional shape, but the coefficients in the quantization matrix are of course not limited to this. For example, when a transform block size of 8 × 8 pixels is used in addition to the present embodiment, another quantization matrix corresponding to the 8 × 8 pixel transform block is held.

The quantization matrix determination unit 108 sequentially reads out the quantization matrix to be encoded from the quantization matrix holding unit 107. Then, it is determined whether or not it matches the predetermined quantization matrix, and matching information indicating that the quantization matrix to be encoded matches the predetermined quantization matrix is generated. In this embodiment, whether or not it matches the encoded quantization matrix, the default quantization matrix shown in FIG. 5 (a) or the flat quantization matrix shown in FIG. 5 (b). Determine The encoded quantization matrix is stored in the quantization matrix holding unit 107, and the quantization matrices shown in FIGS. 5A and 5B are also stored in the quantization matrix holding unit 107 in advance. It is assumed that If it is determined that the quantization matrix to be encoded matches any, reference information is further generated indicating which quantization matrix matches. This reference information is output to the quantization matrix header encoding unit 109 together with the coincidence information. In this case, nothing is output to the quantization matrix coefficient encoding unit 110 and it does not operate. On the other hand, when neither of the quantization matrices match, only the match information is output to the quantization matrix header encoding unit 109, and the quantization matrix to be encoded is output to the quantization matrix coefficient encoding unit 110. In the present embodiment, the match information is 1 when there is a predetermined quantization matrix that matches the quantization matrix to be encoded, and is 0 otherwise. The combination with is not limited to this.

FIG. 11 is a view showing an example of the relationship between a quantization matrix to be encoded and reference information in the present embodiment. In this embodiment, the first three corresponding to the quantization of the Y component of the image input in the YCbCr color space, the second corresponding to the quantization of the Cb component, and the third three types corresponding to the quantization of the Cr component It is assumed that a quantization matrix is present, but is not limited thereto. More quantization matrices may be defined according to the combination of conversion block size, prediction method, and color space. Further, in this embodiment, the quantization matrix shown in FIG. 5A is assumed to be the default 4 × 4 and the quantization matrix shown in FIG. 5B is assumed to be the flat 4 × 4 quantization matrix. However, the quantization matrix used is not limited to these.

For example, when the encoding target quantization matrix is the first (4 × 4Y) and matches the default 4 × 4, the reference information is set to 0. Further, when the flat 4 × 4 matches, the reference information is set to 1. Matching with such flat 4 × 4 is flat information indicating that all coefficients of the quantization matrix to be encoded have the same value. Similarly, in the case where the encoding target quantization matrix is the second (4x4Cb), the reference information is 0 if the default 4x4 matches, and the reference information is 1 if the first encoding (4x4Y) matches. Become. If the flat 4 × 4 matches, the reference information is 2. The same applies to the case where the encoding target quantization matrix is the third (4 × 4 Cr).

The quantization matrix header encoding unit 109 encodes the matching information input from the quantization matrix determination unit 108 and the reference information as needed to generate a quantization matrix header code. In this embodiment, when the quantization matrix to be encoded matches either the quantization matrix stored in advance in the encoding apparatus or the encoded quantization matrix, both the coincidence information and the reference information are Encode. On the other hand, in the case where none of the above quantization matrices match, only the match information is encoded. Although the method of encoding is not particularly limited, Huffman coding, arithmetic coding, etc. can be used. The generated quantization matrix header code is output to the integration coding unit 111.

The quantization matrix coefficient encoding unit 110 encodes the quantization matrix to be encoded input from the quantization matrix determination unit 108 to generate a quantization matrix coefficient code. In the present embodiment, the quantization matrix coefficient encoding unit 110 is configured only when the quantization matrix to be encoded does not match the quantization matrix stored in advance in the encoding apparatus or the already encoded quantization matrix. Operate. As a coding method, in this embodiment, a method is used such as coding each coefficient as it is, coding a difference with the immediately preceding coefficient, DPCM, and coding a difference with another quantization matrix. However, the scheme is not limited to these. In addition, in the case of encoding a flat quantization matrix not stored in advance in the encoding apparatus, only the representative value may be encoded. In addition, when the same value continues together with these, the code amount of the quantization matrix can also be suppressed by inserting a code for terminating the coding.

The integration encoding unit 111 encodes header information necessary for encoding an image, and integrates the input quantization matrix header code and the input quantization matrix coefficient code.

The quantization matrix determination unit 108, the quantization matrix header encoding unit 109, and the quantization matrix coefficient encoding unit 110 may function as a quantization matrix encoding unit which is one processing unit.

<Coding of image>
The image for one picture input from the terminal 101 is input to the block division unit 102 and divided into blocks of 8 × 8 pixels. The divided image is input to the prediction unit 103. The prediction unit 103 performs block-based prediction to generate a prediction error. The transform unit 104 divides the prediction error generated by the prediction unit 103 into transform block sizes, performs orthogonal transform, and generates orthogonal transform coefficients. Then, the orthogonal transformation coefficient is input to the quantization unit 105. In this embodiment, it is assumed that the orthogonal transformation is performed by dividing the prediction error of the 8 × 8 pixel block unit into the 4 × 4 pixel conversion block unit. The quantization unit 105 quantizes the orthogonal transformation coefficient output from the transformation unit 104 using the quantization matrix stored in the quantization matrix holding unit 107 to generate a quantization coefficient. The generated quantization coefficient is input to the coefficient coding unit 106. The coefficient coding unit 106 codes the quantization coefficient generated by the quantization unit 105, generates quantization coefficient code data, and outputs the coded coefficient code data to the integration coding unit 111. The integration coding unit 111 generates a code relating to prediction and conversion in block units, integrates the code in block units and the quantization coefficient code data generated in the coefficient coding unit 106 together with the coded data of the header, A stream is generated and output from the terminal 112.

FIG. 10 shows an example of the bit stream output in the first embodiment. The sequence header includes the code data of the quantization matrix, and is composed of the header code and the encoding result of each coefficient. FIG. 10 (a) is an example of a bit stream in which all of the first to third quantization matrices are different and are different from the flat quantization matrix and the default quantization matrix. That is, 0 of coincidence information is encoded in each of the first to third quantization matrix header codes, and the coefficient of each quantization matrix is encoded in each quantization matrix coefficient code. However, the position of encoding is not limited to this. Of course, the configuration may be such that the picture header portion and other header portions are encoded.

FIG. 10 (b) is an example of a bit stream when all of the first to third quantization matrices are flat quantization matrices. In this case, 1 of coincidence information and 1 of reference information indicating a flat quantization matrix are encoded in the first quantization matrix header code. In the second quantization matrix header code, 1 of matching information and 2 of reference information indicating a flat quantization matrix are encoded. In this case, even if the reference information is 1, since the first quantization matrix to be referenced is also a flat quantization matrix, the decoding results of the bit stream are the same. Similarly, in the third quantization matrix header code, 1 of coincidence information and 2 of reference information indicating a flat quantization matrix are encoded. Also in this case, even if the reference information is 1 or 2, since the first or second quantization matrix to be referred to is also a flat quantization matrix, the decoding results of the bit stream are the same.

In FIG. 10C, the first quantization matrix is different from the flat quantization matrix and the default quantization matrix, but the second and third quantization matrices are identical to the first quantization matrix. It is an example of a bitstream. In this case, 0 of the match information is encoded in the first quantization matrix header code, and the coefficients of the first quantization matrix are encoded in the first quantization matrix coefficient code. In the second quantization matrix header code, 1 of the matching information and 1 of reference information indicating a reference to the first quantization matrix are encoded. Similarly, in the third quantization matrix header code, 1 of the matching information and 1 of reference information indicating a reference to the second quantization matrix are encoded.

FIG. 6 is a flowchart showing an image coding process in the image coding apparatus according to the first embodiment. First, in step S601, the quantization matrix holding unit 107 generates a quantization matrix. In step S602, the quantization matrix determination unit 108 determines whether or not the quantization matrix to be encoded matches a predetermined quantization matrix, and generates match information. In the present embodiment, the encoded quantization matrix, the default quantization matrix shown in FIG. 5 (a) stored in advance in the encoding apparatus, or the flat quantization shown in FIG. 5 (b) It is determined whether it matches any of the matrices. If it is determined that the quantization matrix to be encoded matches any of the above quantization matrices, reference information is further generated which indicates which quantization matrix matches.

In step S603, the quantization matrix header encoding unit 109 encodes the match information and the reference information generated in step S602 to generate a quantization matrix header code. In step S604, the image coding apparatus determines whether to code the coefficients of the quantization matrix to be coded, based on the above-mentioned matching information. If the coefficients of the target quantization matrix are to be encoded, the process proceeds to step S605; otherwise, the process proceeds to step S606.

In step S605, the quantization matrix coefficient encoding unit 110 encodes the quantization matrix generated in step S601 to generate a quantization matrix coefficient code. In step S606, the image coding apparatus determines whether all the quantization matrices have been coded, and if completed, the process proceeds to step S607, otherwise the process proceeds to step S601 for the next quantization matrix. Return to In step S 607, integrated coding section 111 encodes the header portion of the bit stream and outputs it. In this embodiment, it is assumed that the sequence header and the picture header of the example shown in FIG. 10 are encoded and output.

In step S608, the block division unit 102 divides the input image in units of pictures into units of blocks. In step S609, the prediction unit 103 performs prediction on a block basis to generate a prediction error. In step S610, the transform unit 104 divides the prediction error generated in step S609 into transform block sizes, performs orthogonal transform, and generates orthogonal transform coefficients. In step S611, the quantization unit 105 quantizes the orthogonal transformation coefficient generated in step S610 using the quantization matrix generated in step S601 and stored in the quantization matrix holding unit 107, and quantizes the quantization coefficient. Generate In step S612, the coefficient encoding unit 106 encodes the quantization coefficient generated in step S611 to generate quantization coefficient code data. In step S613, the image coding apparatus determines whether coding of all transform blocks in the block is completed. If completed, the process advances to step S614, and if not completed, the next process is performed. The process returns to step S610 for the conversion block. In step S614, the image coding apparatus determines whether or not coding of all blocks is completed, and if so, all operations are stopped and processing is terminated, and otherwise the next The process returns to step S608 for the block of.

With the above configuration and operation, especially when encoding a flat quantization matrix in steps S602 to S604, if it matches with another quantization matrix, there is no need to send a redundant code, so the code amount A reduced bitstream can be constructed. This makes it possible to generate a bitstream with a smaller amount of code than encoding the coefficients of the flat quantization matrix one by one.

In the present embodiment, a picture using only intra prediction has been described as an example, but it is apparent that it is possible to cope with a picture that can use inter prediction. Furthermore, in the present embodiment, the block is 8 × 8 pixels and the conversion block is 4 × 4 pixels for the purpose of explanation, but the present invention is not limited to this. For example, the block size can be changed to 16 × 16 pixels or 32 × 32 pixels, and the shape of the block is not limited to a square, and may be a rectangle such as 16 × 8 pixels. Although the conversion block size is set to half of the block size in the vertical and horizontal directions, the same size may be used, and the size may be smaller than the half in both the vertical and horizontal directions.

Second Embodiment
FIG. 2 is a block diagram showing a configuration of an image decoding apparatus according to Embodiment 2 of the present invention. In this embodiment, decoding of the bit stream generated in Embodiment 1 will be described.

In FIG. 2, reference numeral 201 denotes a terminal for inputting a coded bit stream. A decoding / separation unit 202 decodes header information of the input bit stream, separates necessary codes from the bit stream, and outputs the code to the subsequent stage. The decoding / separating unit 202 performs the reverse operation of the integrated coding unit 111 of FIG.

A quantization matrix header decoding unit 203 decodes the quantization matrix header code separated from the header information of the bit stream and restores the match information and the reference information. A quantization matrix coefficient decoding unit 204 decodes the quantization matrix coefficient code separated from the bit stream and restores each coefficient of the quantization matrix. Reference numeral 205 denotes a quantization matrix reconstruction unit that reconstructs a quantization matrix to be decoded from the obtained coincidence information and reference information or coefficients of the quantization matrix. A quantization matrix holding unit 206 stores the quantization matrix obtained by the quantization matrix reconstruction unit 205. It is assumed that the predetermined quantization matrix stored in the quantization matrix determination unit 108 of the first embodiment is held in this. However, it is not limited to this. It may be stored in the quantization matrix reconstruction unit 205. Further, the quantization matrix header decoding unit 203, the quantization matrix coefficient decoding unit 204, and the quantization matrix reconstruction unit 205 may function as one processing unit, ie, a quantization matrix decoding unit.

On the other hand, reference numeral 207 denotes a coefficient decoding unit that decodes a quantization coefficient code from the quantization coefficient code data separated by the decoding / separation unit 202 and restores the quantization coefficient. An inverse quantization unit 208 performs inverse quantization on the quantization coefficient using the quantization matrix stored in the quantization matrix storage unit 206 to obtain an orthogonal transformation coefficient. An inverse transformation unit 209 performs inverse orthogonal transformation that is the inverse of the transformation unit 104 in FIG. 1 to obtain a prediction error. Reference numeral 210 denotes a prediction reconstruction unit that restores a block image from the prediction error and the decoded image.

The image decoding operation in the image decoding apparatus will be described below. In this embodiment, the moving image bitstream generated in the first embodiment is input in units of pictures. However, a still image bitstream of one picture may be input. Further, in the present embodiment, only the intra prediction decoding process is described in order to facilitate the description, but the present invention is not limited to this and is applicable to the inter prediction decoding process.

In FIG. 2, a bit stream for one picture input from a terminal 201 is input to a decoding / separation unit 202, header information necessary for reproducing an image is decoded, and a code used in a later stage is separated. Output. The quantization matrix header code included in the header information is output to the quantization matrix header decoding unit 203. In this embodiment, it is assumed that the sequence header and the picture header of the bit stream example shown in FIG. 10 are input and decoded, and the code is output.

The quantization matrix header decoding unit 203 first decodes the input quantization matrix header code to restore the match information. If the match information is 1 which means that there is a matrix matching the quantization matrix to be decoded, the reference information is further reproduced from the quantization matrix header code. The restored reference information is output to the quantization matrix reconstruction unit 205. On the other hand, when the match information is 0, which means that there is no matching matrix, information indicating that there is a need to decode the coefficients of the quantization matrix to be decoded is output to the decoding / separating unit 202 Do. In the present embodiment, the coincidence information is the above-described information, but the form is not particularly limited as long as the information has the same meaning.

When the decoding / separation unit 202 receives, from the quantization matrix header decoding unit 203, information indicating that there is a need to decode the coefficients of the quantization matrix to be decoded, quantization is performed on the quantization matrix coefficient decoding unit 204. Output matrix coefficient code data. The quantization matrix coefficient decoding unit 204 decodes the input quantization matrix coefficient code data, and restores each coefficient of the quantization matrix to be decoded. Each coefficient of the reproduced quantization matrix is output to the quantization matrix reconstruction unit 205.

The quantization matrix reconstruction unit 205 reconstructs the quantization matrix to be decoded from the reference information input from the quantization matrix header decoding unit 203 or each coefficient of the quantization matrix input from the quantization matrix coefficient decoding unit 204. Do. When each coefficient of the quantization matrix is input from the quantization matrix coefficient decoding unit 204, the quantization matrix is reconstructed using each input coefficient and is output to the quantization matrix holding unit 206. On the other hand, when the reference information is input from the quantization matrix header decoding unit 203, the quantization matrix of the reference destination is read out from the quantization matrix holding unit 206 based on the reference information, and the quantization matrix to be decoded is read again. Configure. Then, the reconstructed quantization matrix is output to the quantization matrix holding unit 206. The quantization matrix holding unit 206 stores the inputted quantization matrix, and outputs it to the quantization matrix reconstruction unit 205 and the inverse quantization unit 208 as necessary.

Among the codes separated by the decoding / separation unit 202, quantized coefficient code data is input to the coefficient decoding unit 207. Then, the coefficient decoding unit 207 decodes the quantization coefficient code data for each transform block, restores the quantization coefficient, and outputs it to the inverse quantization unit 208. The inverse quantization unit 208 receives the quantization coefficient restored by the coefficient decoding unit 207 and the quantization matrix stored in the quantization matrix holding unit 206. Then, inverse quantization is performed using the quantization matrix to restore orthogonal transformation coefficients, which are output to the inverse transformation unit 209. The inverse transform unit 209 receives the orthogonal transform coefficient, performs inverse orthogonal transform that is the reverse of the transform unit 104 in FIG. 1, restores the prediction error, and outputs the restored prediction error to the prediction reconstruction unit 210. The prediction reconstruction unit 210 predicts the input prediction error from the decoded surrounding pixel data to restore a block unit image, and outputs the image from a terminal 211.

FIG. 7 is a flowchart showing an image decoding process in the image decoding apparatus according to the second embodiment.

First, in step S701, the decoding / separating unit 202 decodes header information and separates the code for output to a subsequent stage. In this embodiment, it is assumed that the sequence header and the picture header of the bit stream example shown in FIG. 10 are input and decoded, and the subsequent code is output to the corresponding block. In step S702, the quantization matrix header decoding unit 203 first decodes the separated quantization matrix header code to restore the match information. If the match information is 1 which means that there is a matrix matching the quantization matrix to be decoded, the reference information is further reproduced from the quantization matrix header code. Conversely, if it is 0, which means that there is no matrix that matches the quantization matrix to be decoded, no further operation is performed in this step.

In step S703, the image decoding apparatus determines whether to decode the coefficients of the quantization matrix to be decoded based on the matching information generated in step S702. If the coefficient of the target quantization matrix is to be decoded, the process proceeds to step S704, and if not, the process proceeds to step S705. In step S704, the quantization matrix coefficient decoding unit 204 decodes the quantization matrix coefficient code data, and reproduces each coefficient of the quantization matrix to be decoded. In step S705, the quantization matrix reconstruction unit 205 reconstructs the quantization matrix to be decoded from the reference information reconstructed in step S702 or the coefficients of the quantization matrix reconstructed in step S704.

In step S706, the image decoding apparatus determines whether or not all quantization matrices have been decoded, and if completed, the process proceeds to step S707, otherwise the next quantization matrix is used as a target. It returns to S702. In step S 707, the coefficient decoding unit 207 decodes the quantization coefficient code data separated from the bit stream in units of transform blocks, and restores the quantization coefficients. In step S 708, the dequantization unit 208 dequantizes the quantization coefficient reproduced in step S 707 using the quantization matrix reproduced in step S 705, and restores the orthogonal transformation coefficient. In step S709, the inverse transform unit 209 performs inverse orthogonal transform on the orthogonal transform coefficient restored in step S708 to restore a prediction error.

In step S710, the image decoding apparatus determines whether decoding of all conversion blocks in the block is completed, and if completed, the process proceeds to step S711, and if not completed, the next conversion block The process returns to step S 707 for. In step S711, the prediction reconstruction unit 210 performs prediction from the decoded surrounding pixel data, and adds the prediction error to the prediction error restored in step S709 to generate a decoded image of the block. In step S712, the image decoding apparatus determines whether or not decoding of all the blocks is completed, and if so, stops all operations and ends the process, otherwise the next block The process returns to step S 707.

With the above-described configuration and operation, it is possible to decode a bitstream including a code represented by a small code amount such as the match information generated in the first embodiment, and obtain a reproduced image.

As in the first embodiment, the block size, the conversion block size, and the block shape are not limited thereto. It is also possible to update the quantization matrix by including the quantization matrix code data multiple times in the bitstream of one sequence. The decoding / separating unit 202 detects the quantization matrix header code, and the quantization matrix header decoding unit 203 reproduces the match information and the reference information as needed. Furthermore, according to the coincidence information, the quantization matrix coefficient decoding unit 204 restores the coefficients of the quantization matrix to be decoded. Next, the quantization matrix reconstruction unit 205 restores the quantization matrix to be decoded from the reference information or each coefficient of the quantization matrix. Then, the data of the restored quantization matrix is replaced with the corresponding quantization matrix of the quantization matrix holding unit 206. At this time, all the quantization matrices may be rewritten, or part of the quantization matrices may be changed by determining the quantization matrix to be rewritten.

Further, in the present embodiment, the method of performing processing after accumulating code data for one picture has been described as an example, but the present invention is not limited to this. For example, an input method may be used such as a block unit or a slice unit configured of a plurality of blocks. Furthermore, it may be divided into fixed length packets and the like instead of blocks and the like.

Embodiment 3
FIG. 3 is a block diagram showing an image coding apparatus according to the present embodiment. In FIG. 3, the same reference numerals are given to parts performing the same functions as in FIG. 1 of the first embodiment, and the description will be omitted.

Similar to the prediction unit 103 in FIG. 1, the prediction unit 303 predicts each block divided by the block division unit 102 in units of blocks, determines a prediction method, calculates a difference value according to the prediction method, and calculates a prediction error. It is. 321 is a conversion control unit that determines whether or not to perform orthogonal transformation on the prediction error generated by the prediction unit 303, and outputs the prediction error to the subsequent stage and outputs the determination result to the subsequent stage as conversion skip determination information is there. Similar to the transform unit 104 in FIG. 1, 304 is a transform unit that performs orthogonal transform on the prediction error of each block, and performs orthogonal transform in units of the size of the input block or a block obtained by dividing it Calculate the coefficient.

A bit depth adjustment unit 322 adjusts the bit depth of the prediction error in order to achieve consistency with the quantization processing in the quantization unit 305 in the latter stage. The degree of bit depth adjustment here is determined by the bit depth of the input image, the bit depth adjustment with orthogonal transformation in the transform unit 304, the bit depth adjustment with quantization processing in the quantization unit 305, the coefficient value of the quantization matrix, etc. It shall be. For example, in the present embodiment, bit depth adjustment is performed by bit-shifting the prediction error to the left by the difference between the predetermined value 13 and the bit depth of the input image. That is, if the bit depth of the input image is 8 bits, it is assumed that the prediction error is shifted to the left by 5 bits. However, the value used for bit depth adjustment is not limited to this.

A quantization unit 305 quantizes the orthogonal transformation coefficient generated by the conversion unit 304 or the prediction error adjusted by the bit depth adjustment unit 322 using the quantization matrix stored in the quantization matrix holding unit 107. is there. An integrated coding unit 311 generates header information and codes relating to prediction and conversion, and integrates code data generated by the above-described coding units to form and output a bit stream. The integrated coding unit 111 of FIG. 1 is different from that of FIG. 1 in that the conversion skip determination information is further coded to generate code data, and a bit stream is formed and output.

The image encoding operation of the image encoding apparatus will be described below.

The prediction unit 303 first performs block-based prediction to generate a prediction error. The generated prediction error is output to the conversion control unit 321. The transformation control unit 321 determines whether or not to perform orthogonal transformation on the input prediction error. Then, the prediction error is output to the conversion unit 304 when performing orthogonal transformation, and the prediction error is output to the bit depth adjustment unit 322 when performing no orthogonal conversion. Although the method of determining whether or not to perform orthogonal transformation is not particularly limited, it may be determined from the characteristics of the image, the size of the transform coefficient after orthogonal transformation, and the like. In addition, the determined result is output to the integrated coding unit 311 as conversion skip determination information. The transform unit 304 performs orthogonal transform on the prediction error input from the transform control unit 321 in units of transform block size to generate orthogonal transform coefficients.

On the other hand, the bit depth adjustment unit 322 performs bit depth adjustment of the prediction error input from the conversion control unit 321 in units of conversion blocks, and generates a prediction error adjusted in bit depth. In the quantization unit 305, the quantization matrix stored in the quantization matrix holding unit 107 with respect to the orthogonal transformation coefficient output from the conversion unit 304 or the prediction error with bit depth adjustment output from the bit depth adjustment unit 322. Quantize using to generate quantized coefficients. The generated quantization coefficient is input to the coefficient coding unit 106.

The integration encoding unit 311 encodes header information necessary for encoding an image, and integrates the input quantization matrix header code and the input quantization matrix coefficient code. The integrated encoding unit 311 further encodes the conversion skip determination information input from the conversion control unit 321 in addition to the header information encoded by the integrated encoding unit 111 in FIG. 1.

FIG. 8 is a flowchart showing an image encoding process in the image encoding device according to the third embodiment. In FIG. 8, the same reference numerals are given to parts performing the same functions as in FIG. 6 of the first embodiment, and the description will be omitted.

In step S820, the conversion control unit 321 determines whether or not to perform orthogonal conversion on the prediction error generated in step S609, and generates conversion skip determination information. If orthogonal transformation is to be performed based on the conversion skip determination information in step S821, the process proceeds to step S610, and if not, the process proceeds to step S822. In step S822, the bit depth adjustment unit 322 performs bit depth adjustment of the prediction error generated in step S609 on a conversion block basis to generate a bit depth adjusted prediction error. In step S811, the quantization unit 305 quantizes the orthogonal transformation coefficient generated in step S610 or the bit depth adjusted prediction error generated in step S822 using the quantization matrix generated in step S601. Generate quantization coefficients. In step S823, the integration coding unit 311 encodes the conversion skip determination information generated in step S821.

With the above configuration and operation, there is no need to send a redundant code when encoding a flat quantization matrix corresponding to conversion skip, particularly in steps S820 to S823, so a bit stream with a reduced code amount can be used. It can be configured.

In this embodiment, although the conversion skip determination information is encoded in units of conversion blocks, the unit in which the conversion skip determination information is encoded is not limited to this. Encoding may be performed in block units, or a configuration may be adopted in which the code amount is saved using conversion skip determination information common to each color component. Although the method of encoding is not particularly limited, Huffman coding, arithmetic coding, etc. can be used.

Fourth Embodiment
FIG. 4 is a block diagram showing an image decoding apparatus according to the present embodiment. In FIG. 4, the same reference numerals are given to parts performing the same functions as in FIG. 2 of the second embodiment, and the description will be omitted. In this embodiment, decoding of the bitstream generated in Embodiment 3 will be described.

A decoding / demultiplexing unit 402 decodes header information of an input bit stream, separates necessary codes from the bit stream, and outputs the code to a subsequent stage. The decoding / separating unit 402 performs the reverse operation of the integrated coding unit 311 in FIG. 3. The decoding / separating unit 202 in FIG. 2 is different from the decoding / separating unit 202 in that the conversion skip determination information is further decoded and reproduced and output to the subsequent stage. An inverse quantization unit 408 performs inverse quantization on the quantization coefficient using the quantization matrix stored in the quantization matrix storage unit 206 and reproduces the inverse quantization coefficient. The inverse quantization coefficient in this embodiment corresponds to the orthogonal transformation coefficient or bit depth adjusted prediction error in the third embodiment.

An inverse transformation control unit 421 determines whether to perform inverse orthogonal transformation based on the transformation skip determination information input from the decoding / demultiplexing unit 402, and outputs an inverse quantization coefficient to the subsequent stage. An inverse transform unit 409 performs inverse orthogonal transform which is the reverse of the transform unit 304 in FIG. 3 and reproduces a prediction error. A bit depth inverse adjustment unit 422 performs bit depth adjustment that is the reverse of the bit depth adjustment unit 322 in FIG. 3 and reproduces a prediction error. Here, the degree of bit depth inverse adjustment may be the bit depth of the output image, the bit depth adjustment with inverse orthogonal transformation in inverse transformation unit 409, the bit depth adjustment with quantization processing in inverse quantization unit 408, and the quantization matrix It shall be determined by the coefficient value etc. For example, in this embodiment, the bit depth reverse adjustment is performed by bit shifting the coefficient to the right by the difference between 13 and the bit depth of the output image. That is, if the bit depth of the output image is 8 bits, it is assumed that the prediction error is shifted to the right by 5 bits. However, the value used for bit depth reverse adjustment is not limited to this. Reference numeral 410 denotes a prediction reconstruction unit that reproduces a block image from the prediction error and the decoded image.

The image decoding operation in the image decoding apparatus will be described below.

In FIG. 4, a bit stream for one picture input from a terminal 201 is input to a decoding / separation unit 402, header information necessary for reproducing an image is decoded, and a code used in a later stage is separated. , As appropriate. The quantization matrix header code included in the header information is output to the quantization matrix header decoding unit 203. When the decoding / separation unit 402 receives, from the quantization matrix header decoding unit 203, information indicating that there is a need to decode the coefficients of the quantization matrix to be decoded, the decoding / separation unit 402 transmits the information to the quantization matrix coefficient decoding unit 204. Output quantization matrix coefficient code data. Further, the decoding / separating unit 402 decodes and reproduces conversion skip determination information indicating whether to perform inverse orthogonal transformation in units of transformation blocks from the bit stream and outputs the information to the inverse transformation control unit 421.

The inverse quantization unit 408 receives the quantization coefficient reproduced by the coefficient decoding unit 207 and the quantization matrix stored in the quantization matrix holding unit 206. Then, inverse quantization is performed using the quantization matrix, the inverse quantization coefficient is regenerated, and the result is output to the inverse transformation control unit 421. The inverse transform control unit 421 determines whether to perform inverse orthogonal transform on the input inverse quantization coefficient based on the conversion skip determination information input from the decoding / separation unit 402. When the inverse orthogonal transform is performed, the inverse quantization coefficient is output to the inverse transform unit 409, and when the inverse orthogonal transform is not performed, the inverse quantization coefficient is output to the bit depth inverse adjustment unit 422. The inverse transformation unit 209 performs inverse orthogonal transformation that is the inverse of the transformation unit 304 in FIG. 3 on the orthogonal transformation coefficient that is the input inverse quantization coefficient, reproduces a prediction error, and outputs the prediction error to the prediction reconstruction unit 410 Do.

On the other hand, the bit depth inverse adjustment unit 422 performs bit depth inverse adjustment in units of conversion blocks on the bit depth adjusted prediction error that is the inverse quantization coefficient input from the inverse transform control unit 421, and reproduces the prediction error, It is output to the prediction reconstruction unit 410. The prediction reconstruction unit 410 performs prediction based on the input prediction error based on the decoded surrounding pixel data, reproduces a block unit image, and outputs the image from a terminal 211.

FIG. 9 is a flowchart showing an image decoding process in the image decoding apparatus according to the fourth embodiment. The same reference numerals are given to parts that perform the same functions as in FIG. 7 of the second embodiment, and the description will be omitted.
In step S 921, the decoding / demultiplexing unit 402 decodes / reproduces conversion skip determination information indicating whether to perform inverse orthogonal transformation in units of transformation blocks from the bit stream. In step S922, determination based on the conversion skip determination information generated in step S921 is performed. If the inverse orthogonal transform is to be performed, the process advances to step S709; otherwise, the process advances to step S923. In step S923, the bit depth inverse adjustment unit 422 performs bit depth inverse adjustment in units of transform blocks on the dequantized coefficient generated in step S708, and reproduces a prediction error.

With the above configuration and operation, even in the image decoding apparatus having the conversion skip process, decoding of the flat quantization matrix represented by a small code amount by the quantization matrix matching information or the like generated in the third embodiment is performed. , You can get a reproduced image.

Further, in the present embodiment, conversion skip determination information is provided for each conversion block, but a flag may be provided in the header portion, and thereby it may be possible to switch whether the decoding apparatus has conversion skip processing or not. . In that case, a flat quantization matrix may be stored and referred to only when the conversion skip processing is used, and otherwise it may be excluded from the reference.

Fifth Embodiment
In the above embodiments, each processing unit shown in FIG. 1, FIG. 2, FIG. 3, and FIG. 4 is described as being configured by hardware. However, the processing performed by each processing unit shown in FIG. 1, FIG. 2, FIG. 3, and FIG. 4 may be configured by a computer program.

FIG. 12 is a block diagram showing a configuration example of hardware of a computer applicable to the image display device according to each of the above embodiments.

The CPU 1201 controls the entire computer using computer programs and data stored in the RAM 1202 and the ROM 1203 and executes the processes described above as being performed by the image processing apparatus according to each of the above embodiments. That is, the CPU 1201 functions as each processing unit shown in FIG. 1, FIG. 2, FIG. 3, and FIG.

The RAM 1202 has an area for temporarily storing a computer program or data loaded from the external storage device 1206, data acquired from the outside via an I / F (interface) 1207, and the like. Furthermore, the RAM 1202 has a work area used when the CPU 1201 executes various processes. That is, the RAM 1202 can be allocated, for example, as a picture memory, or can appropriately provide various other areas.

The ROM 1203 stores setting data of this computer, a boot program, and the like. The operation unit 1204 includes a keyboard, a mouse, and the like, and can input various instructions to the CPU 1201 by the operation of the user of the computer. The output unit 1205 outputs the processing result of the CPU 1201. Also, the output unit 1205 can be configured by a display device such as a liquid crystal display, for example, to display the processing result.

The external storage device 1206 is a large-capacity information storage device represented by a hard disk drive. The external storage device 1206 stores an operating system (OS) and a computer program for causing the CPU 1201 to realize the functions of the units shown in FIG. 1, FIG. 2, FIG. 3, and FIG. Furthermore, the external storage device 1206 may store each image to be processed.

Computer programs and data stored in the external storage device 1206 are appropriately loaded into the RAM 1202 under control of the CPU 1201 and are to be processed by the CPU 1201. A network such as a LAN or the Internet, or other devices such as a projection device or a display device can be connected to the I / F 1207, and the computer acquires or sends various information via the I / F 1207. Can be A bus 1208 connects the above-described units.

In the operation having the above-described configuration, the CPU 1201 mainly controls the operation described in the above-described flowchart.

<Other Embodiments>
The object of the present invention is also achieved by supplying a storage medium storing computer program code for realizing the functions described above to a system, and the system reads out and executes the computer program code. In this case, the code itself of the computer program read from the storage medium implements the functions of the above-described embodiments, and the storage medium storing the code of the computer program constitutes the present invention. In addition, an operating system (OS) or the like operating on a computer performs part or all of the actual processing based on the instruction of the code of the program, and the above-described functions are realized by the processing. .

Furthermore, it may be realized in the following form. That is, the computer program code read out from the storage medium is written in a memory provided in a function expansion card inserted in the computer or a function expansion unit connected to the computer. Then, based on the instruction of the code of the computer program, the CPU included in the function expansion card or the function expansion unit may execute part or all of the actual processing to realize the above-described function.

When the present invention is applied to the storage medium, the storage medium stores the code of the computer program corresponding to the flowchart described above.

Claims (6)

Acquisition means for acquiring, from a bit stream, quantization coefficients and conversion skip determination information indicating whether the quantization coefficients are orthogonally transformed coefficients;
When acquiring reference information indicating a quantization matrix in which all coefficients have the same value from the bit stream, the quantization coefficient is inversely quantized based on the quantization matrix obtained according to the acquired reference information. Inverse quantization means,
An adjustment to adjust a bit depth of the coefficient inversely quantized by the inverse quantization unit, when the conversion skip determination information acquired by the acquisition unit indicates that the quantization coefficient is a coefficient that has not been orthogonally transformed Means,
Decoding means for decoding an image from the data obtained by the adjusting means;
Have
The inverse quantization means is a coefficient in which the quantization coefficient is not orthogonally transformed even when the conversion skip determination information acquired by the acquisition means indicates a coefficient to which the quantization coefficient is orthogonally transformed. In the case where the reference information is to be acquired, the image decoding device is characterized in that the quantization coefficient is inversely quantized based on the quantization matrix obtained according to the reference information.   The data obtained by the inverse orthogonal transformation means, or the data obtained by the adjustment means is a prediction error which is a difference value between pixels of a block to be referred to and pixels of a block to be decoded. The image decoding apparatus according to 1. An image decoding method,
Acquiring, from a bitstream, a quantization coefficient and conversion skip determination information indicating whether the quantization coefficient is an orthogonally transformed coefficient;
When acquiring reference information indicating a quantization matrix in which all coefficients have the same value from the bit stream, the quantization coefficient is inversely quantized based on the quantization matrix obtained according to the acquired reference information. Inverse quantization step,
When the conversion skip determination information acquired in the acquisition step indicates that the quantization coefficient is a coefficient that has not been orthogonally transformed, an adjustment that adjusts the bit depth of the inverse quantized coefficient in the inverse quantization step Step and
Decoding the image from the data obtained in the adjusting step;
In the inverse quantization step, even when the conversion skip determination information acquired in the acquisition step indicates a coefficient to which the quantization coefficient is orthogonally transformed, the quantization coefficient is a coefficient not to be orthogonally transformed. In the case where the reference information is indicated, the image decoding method further includes the step of inversely quantizing the quantization coefficient based on a quantization matrix obtained according to the reference information.   The data obtained in the inverse orthogonal transformation step or the data obtained in the adjustment step is a prediction error which is a difference value between pixels of a block to be referred to and pixels of a block to be decoded. The image decoding method according to 3.   A program that causes a computer to function as the image decoding device according to any one of claims 1 and 2 by being read and executed by the computer.   A computer readable storage medium storing the program of claim 5.

Images