JP6891325B2 - Image coding method - Google Patents

Description

The present invention relates to an image coding method.

As a method for compressing and recording moving images, H.I. 264 / MPEG-4 AVC (hereinafter referred to as H.264) is known. (Non-Patent Document 1)
H. In 264, each coefficient of the quantization matrix can be changed to an arbitrary value by encoding the scaling_list information. According to Non-Patent Document 1, each coefficient of the quantization matrix can take an arbitrary value by adding the difference value delta_scale to the immediately preceding coefficient.

In recent years, H. As a successor to 264, activities to carry out international standardization of more efficient coding methods have started. JCT-VC (Joint Collaboration Team on Video Coding) was established between ISO / IEC and ITU-T. In this JCT-VC, standardization is being promoted as a HEVC (High Efficiency Video Coding) coding method (hereinafter referred to as HEVC).

In this method, in order to improve the coding efficiency, an orthogonal transform method such as a discrete sine transform is being studied in addition to the conventional discrete cosine-based orthogonal transform method. Further, in order to improve the coding efficiency not only in natural images taken by a camera or the like but also in artificial images such as PC screens and games, a method such as a conversion skip technique that does not perform orthogonal conversion is being studied. (Non-Patent Document 2)

ITU-T H. 264 (03/2010) Advancedvideo coding for generic audiovisual services JCT-VC Contribution JCTVC-I1003. doc internet <http: // phenix. int-every. fr / junction / doc_end_user / documents / 9_Geneva / wg11 />

Generally, in image compression, in order to improve subjective image quality, among the conversion coefficients after orthogonal transformation, the coefficient corresponding to the low frequency component is quantized with a smaller value, and the coefficient corresponding to the high frequency component is set to a larger value. Often quantized. In this case, it is common to perform weighted quantization by frequency components using a quantization matrix as shown in FIG. 5A. The quantization matrix 500 has a smaller coefficient corresponding to a low frequency component and a larger coefficient corresponding to a high frequency component.

On the other hand, in HEVC, especially when the above-mentioned conversion skip technique is used, orthogonal transformation is not performed, so that the coefficients to be quantized all have the same frequency component. Therefore, when the conversion skip technique is used, it is not preferable to perform the above-mentioned weighted quantization, and a flat quantization matrix 501 in which all the coefficients have the same value as shown in FIG. 5B is used. It is necessary to perform quantization processing. Therefore, it is desirable that the information of the flat quantization matrix 501 can be encoded with a smaller amount of code. However, especially in the above-mentioned HEVC, there is no mechanism for efficiently coding, and as a result, the amount of code is unnecessarily large. It had become.

Therefore, the present invention has been made to solve the above-mentioned problems, and by introducing a method for efficiently coding a flat quantization matrix with a smaller amount of code, highly efficient quantization matrix coding can be performed. -The purpose is to realize decoding.

In order to solve the above-mentioned problems, the image coding method of the present invention has the following configuration. That is, a conversion step for orthogonally transforming an image, an adjustment step for adjusting the bit depth of the image without orthogonally transforming the image, and a coefficient obtained by orthogonally transforming the image, or for orthogonally transforming the image. The quantization step of obtaining the quantization coefficient by quantization based on the quantization matrix in which all the coefficients have the same value with respect to the coefficient whose bit depth is adjusted without the above, the quantization coefficient, and the quantum. Conversion skip determination information indicating whether or not the conversion coefficient is an orthogonally converted coefficient, a coding step for encoding reference information indicating whether or not all the coefficients are quantization matrices having the same value, and Have,
In the quantization step, when quantization is performed using a quantization matrix in which all the coefficients have the same value with respect to the coefficient whose bit depth is adjusted without orthogonally transforming the image, the quantization coefficient is intra. Quantization is performed using a quantization matrix in which all the coefficients are the same value regardless of whether they are encoded using prediction or inter-prediction. In the quantization step, conversion skip determination information indicating that the quantization coefficient is not an orthogonally converted coefficient and reference information indicating whether or not all the coefficients are quantization matrices having the same value are encoded. It is characterized by that.

INDUSTRIAL APPLICABILITY According to the present invention, the amount of coding for quantization matrix coding can be reduced, and highly efficient coding / decoding becomes possible.

The block diagram which shows the structure of the image coding apparatus in Embodiment 1. The block diagram which shows the structure of the image decoding apparatus in Embodiment 2. The block diagram which shows the structure of the image coding apparatus in Embodiment 3. The block diagram which shows the structure of the image decoding apparatus in Embodiment 4. (A), (b) The figure which shows an example of the quantization matrix. A flowchart showing an image coding process in the image coding apparatus according to the first embodiment. A flowchart showing an image decoding process in the image decoding apparatus according to the second embodiment. A flowchart showing an image coding process in the image coding apparatus according to the third embodiment. A flowchart showing an image decoding process in the image decoding apparatus according to the fourth embodiment. (A), (b), (c) The figure which shows an example of the bit stream structure. The figure which shows an example of the relationship between the quantization matrix to be encoded and the quantization matrix reference information. A block diagram showing a configuration example of computer hardware applicable to the image coding device and decoding device of the present invention.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings, based on its preferred embodiments. The configuration shown in the following embodiments is only an example, and the present invention is not limited to the illustrated configuration.

<Embodiment 1>
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an image coding apparatus of this embodiment.

In FIG. 1, 101 is a terminal for inputting image data (hereinafter referred to as an image). Reference numeral 102 denotes a block dividing unit that divides the input image into a plurality of blocks. Reference numeral 103 denotes a prediction unit that predicts each block divided by the block division unit 102 in block units, determines a prediction method, calculates a difference value according to the prediction method, and calculates a prediction error. The prediction unit 103 also performs intra prediction or motion compensation prediction. Intra-prediction is generally realized by selecting the most suitable prediction method from a plurality of reference methods in order to calculate a prediction value from surrounding pixel data.

Reference numeral 104 denotes a conversion unit that performs orthogonal conversion with respect to the prediction error of each block, performs orthogonal conversion in units of the size of the input block or the size obtained by subdividing it, and calculates the orthogonal conversion coefficient. Hereinafter, the block that performs orthogonal transformation is referred to as a conversion block. The orthogonal transform is not particularly limited, but a discrete cosine transform, a Hadamard transform, or the like may be used. Further, in the present embodiment, for the sake of explanation, the prediction error in the block unit of 8 × 8 pixels is divided into two in the vertical and horizontal directions, and the orthogonal transformation is performed in the conversion block unit of 4 × 4 pixels. The shape is not limited to this. Orthogonal transformation may be performed using a conversion block having the same size as the block, or orthogonal transformation may be performed using a conversion block in which the block is divided into smaller units than the vertical and horizontal divisions.

Reference numeral 107 denotes a quantization matrix holding unit that generates and stores a quantization matrix. The method of generating the stored quantization matrix is not particularly limited, but the user may input the quantization matrix, or even if it is calculated from the characteristics of the input image, the one specified in advance as the initial value is used. But of course it's good. In the present embodiment, it is assumed that a two-dimensional quantization matrix corresponding to a conversion block of 4 × 4 pixels as shown in FIGS. 5A and 5B is generated and stored.

Reference numeral 105 denotes a quantization unit that quantizes the orthogonal transformation coefficient by the quantization matrix stored in the quantization matrix holding unit 107. The quantization coefficient can be obtained by this quantization. Reference numeral 106 denotes a coefficient coding unit that encodes the quantization coefficient thus obtained to generate the quantization coefficient code data. The coding method is not particularly limited, but a Huffman code, an arithmetic code, or the like can be used. Reference numeral 108 denotes a quantization matrix determination unit for determining whether or not the quantization matrix stored in the quantization matrix holding unit 107 matches a predetermined quantization matrix. More specifically, whether or not the quantization target to be encoded is an already encoded quantization matrix, a predetermined quantization matrix, or a flat quantization matrix in which all the coefficients have the same value. judge. It is assumed that this predetermined quantization matrix is stored in the quantization matrix holding unit 107. However, it is not limited to this. For example, it may be stored in the quantization matrix determination unit 108.

Reference numeral 109 denotes a quantization matrix header coding unit that generates a quantization matrix header code by encoding information indicating a determination in the quantization matrix determination unit 108 and other information related to the quantization matrix as a header. Reference numeral 110 denotes a quantization matrix coefficient coding unit that encodes each coefficient of the quantization matrix to be encoded to generate a quantization matrix coefficient code.

Reference numeral 111 denotes an integrated coding unit that generates a code related to header information, prediction, and conversion, and integrates the code data generated by each of the above-mentioned coding units to form a bit stream and output the code. Specifically, the quantization coefficient code data generated by the coefficient coding unit 106, the quantization matrix header code generated by the quantization matrix header coding unit 109, and the quantization matrix coefficient coding unit 110 are generated. Integrate the quantization matrix coefficient code. Here, the code related to prediction and conversion is, for example, a code such as a selected prediction method or a state of division of conversion blocks. Reference numeral 112 denotes a terminal for outputting the bit stream generated by the integrated coding unit 111 to the outside.

The image coding operation in the image coding apparatus will be described below. In the present embodiment, the moving image data is input in units of pictures, but the still image data for one picture may be input. Further, in the present embodiment, only the intra-predictive coding process will be described for ease of explanation, but the present embodiment is not limited to this, and can be applied to the inter-predictive coding process. In the present embodiment, for the sake of explanation, the block division unit 102 will be described as being divided into blocks of 8 × 8 pixels, but the present embodiment is not limited to this.

<Code-coding of quantization matrix coefficient>
First, the quantization matrix holding unit 107 generates a quantization matrix. The quantization matrix is determined according to the conversion block size, which is the size of the orthogonal transformation used in the coding process. The method for determining the coefficients of the quantization matrix is not particularly limited. For example, a predetermined initial value may be used or may be set individually. Further, it may be set and generated according to the characteristics of the image.

The quantization matrix holding unit 107 holds the quantization matrix thus generated. 5 (a) and 5 (b) are examples of a quantization matrix corresponding to a conversion block of 4 × 4 pixels. For the sake of simplicity, it is assumed that the configuration is 16 pixels corresponding to the conversion block of 4 × 4 pixels, and each square in the thick frame represents a coefficient. In the present embodiment, it is assumed that the quantization matrix shown in FIGS. 5A and 5B is held in a two-dimensional shape, but the coefficients in the quantization matrix are of course not limited to this. For example, when a conversion block size of 8 × 8 pixels is also used in addition to the present embodiment, another quantization matrix corresponding to the 8 × 8 pixel conversion block is held.

The quantization matrix determination unit 108 reads out the quantization matrix to be encoded in order from the quantization matrix holding unit 107. Then, it is determined whether or not it matches the predetermined quantization matrix, and matching information indicating that the quantization matrix to be encoded matches the predetermined quantization matrix is generated. In this embodiment, whether it matches either the encoded quantization matrix, the default quantization matrix shown in FIG. 5 (a), or the flat quantization matrix shown in FIG. 5 (b). To judge. The encoded quantization matrix is stored in the quantization matrix holding unit 107, and the quantization matrix shown in FIGS. 5 (a) and 5 (b) is also stored in the quantization matrix holding unit 107 in advance. It is assumed that there is. If it is determined that the quantization matrix to be encoded matches any of them, reference information indicating which quantization matrix matches is further generated. This reference information is output to the quantization matrix header coding unit 109 together with the matching information. In this case, nothing is output to the quantization matrix coefficient coding unit 110 and the operation does not occur. On the other hand, if it does not match any of the quantization matrices, only the match information is output to the quantization matrix header coding unit 109, and the quantization target to be encoded is output to the quantization matrix coefficient coding unit 110. In the present embodiment, if there is a predetermined quantization matrix that matches the quantization target quantization matrix, the match information is 1, and if not, it is 0. However, the value and the state are set. The combination with is not limited to this.

FIG. 11 is a diagram showing an example of the relationship between the quantization matrix to be encoded and the reference information in the present embodiment. In this embodiment, there are three types: the first corresponding to the quantization of the Y component of the image input in the YCbCr color space, the second corresponding to the quantization of the Cb component, and the third corresponding to the quantization of the Cr component. It is assumed that a quantization matrix exists, but the present invention is not limited to this. More quantization matrices may be defined by the combination of conversion block size, prediction method, and color space. Further, in the present embodiment, the quantization matrix shown in FIG. 5 (a) is a default 4 × 4, and the quantization matrix shown in FIG. 5 (b) is a flat 4 × 4 quantization matrix. However, the quantization matrix used is not limited to these.

For example, if the quantization target quantization matrix is the first (4x4Y) and matches the default 4x4, the reference information is set to 0. If it matches the flat 4 × 4, the reference information is set to 1. Matching with such flat 4 × 4 provides flat information indicating that the coefficients of the quantization matrix to be coded are all the same value. Similarly, when the coded object quantization matrix is the second (4x4Cb), the reference information is set to 0 if it matches the default 4x4, and the reference information is set to 1 if it matches the coded first (4x4Y). Become. If it matches the flat 4 × 4, the reference information is set to 2. The same applies when the quantization target quantization matrix is the third (4x4Cr).

The quantization matrix header coding unit 109 encodes the matching information input from the quantization matrix determination unit 108 and the reference information as needed to generate the quantization matrix header code. In the present embodiment, when the quantization matrix to be coded matches either the quantization matrix stored in the coding device or the coded quantization matrix, both the matching information and the reference information are displayed. To encode. On the other hand, if it does not match any of the above quantization matrices, only the match information is encoded. The coding method is not particularly limited, but a Huffman code, an arithmetic code, or the like can be used. The generated quantization matrix header code is output to the integrated coding unit 111.

The quantization matrix coefficient coding unit 110 encodes the quantization target quantization matrix input from the quantization matrix determination unit 108 to generate a quantization matrix coefficient code. In the present embodiment, the quantization matrix coefficient coding unit 110 sets the quantization matrix coefficient coding unit 110 only when the quantization matrix to be coded does not match the quantization matrix stored in advance in the coding device or the coded quantization matrix. Operate. As the coding method, in the present embodiment, a method of coding each coefficient as it is, coding the difference from the immediately preceding coefficient, and coding the difference from DPCM or another quantization matrix is used. However, the method is not limited to these. Further, when coding a flat quantization matrix that is not stored in advance in the coding device, a method of coding only the representative value may be adopted. Further, when the same value continues in combination with these, the code amount of the quantization matrix can be suppressed by inserting a code for terminating the coding.

The integrated coding unit 111 encodes the header information necessary for coding the image, and integrates the input quantization matrix header code and the quantization matrix coefficient code.

The quantization matrix determination unit 108, the quantization matrix header coding unit 109, and the quantization matrix coefficient coding unit 110 may function as one processing unit, the quantization matrix coding unit.

<Image coding>
The image for one picture input from the terminal 101 is input to the block division unit 102, and is divided into block units of 8 × 8 pixels. The divided image is input to the prediction unit 103. The prediction unit 103 makes a block-by-block prediction, and a prediction error is generated. The conversion unit 104 divides the prediction error generated by the prediction unit 103 into conversion block sizes, performs orthogonal transformation, and generates an orthogonal transformation coefficient. Then, the orthogonal transformation coefficient is input to the quantization unit 105. In the present embodiment, the prediction error in block units of 8 × 8 pixels is divided into conversion block units of 4 × 4 pixels to perform orthogonal conversion. The quantization unit 105 generates a quantization coefficient by quantizing the orthogonal transformation coefficient output from the conversion unit 104 using the quantization matrix stored in the quantization matrix holding unit 107. The generated quantization coefficient is input to the coefficient coding unit 106. The coefficient coding unit 106 encodes the quantization coefficient generated by the quantization unit 105, generates the quantization coefficient code data, and outputs the quantization coefficient code data to the integrated coding unit 111. The integrated coding unit 111 generates a code related to prediction and conversion in block units, integrates the code in block units and the quantization coefficient code data generated by the coefficient coding unit 106 together with the coding data of the header, and bits. A stream is generated and output from the terminal 112.

FIG. 10 shows an example of the bit stream output in the first embodiment. The sequence header contains the code data of the quantization matrix, and is composed of the header code and the coded result of each coefficient. FIG. 10A is an example of a bitstream in which all of the first to third quantization matrices are different and different from the flat quantization matrix and the default quantization matrix. That is, 0 of the matching information is encoded in each of the first to third quantization matrix header codes, and the coefficient of each quantization matrix is encoded in each quantization matrix coefficient code. However, the coding position is not limited to this. Of course, a configuration in which the picture header portion and other header portions are encoded may be used.

FIG. 10B is an example of a bit stream when all of the first to third quantization matrices are flat quantization matrices. In this case, 1 of the match information and 1 of the reference information indicating the flat quantization matrix are encoded in the first quantization matrix header code. Further, in the second quantization matrix header code, 1 of match information and 2 of reference information indicating a flat quantization matrix are encoded. In this case, even if the reference information is 1, the first quantization matrix to be referred to is also a flat quantization matrix, so that the decoding results of the bitstreams are the same. Similarly, in the third quantization matrix header code, 1 of the match information and 2 of the reference information indicating the flat quantization matrix are encoded. In this case as well, even if the reference information is 1 or 2, the first or second quantization matrix to be referred to is also a flat quantization matrix, so that the decoding result of the bit stream is the same.

FIG. 10C shows a case where the first quantization matrix is different from the flat quantization matrix and the default quantization matrix, but the second and third quantization matrices are the same as the first quantization matrix. This is an example of a bit stream. In this case, 0 of the matching information is encoded in the first quantization matrix header code, and the coefficient of the first quantization matrix is encoded in the first quantization matrix coefficient code. Further, in the second quantization matrix header code, 1 of the match information and 1 of the reference information indicating a reference to the first quantization matrix are encoded. Similarly, in the third quantization matrix header code, 1 of the match information and 1 of the reference information indicating a reference to the second quantization matrix are encoded.

FIG. 6 is a flowchart showing an image coding process in the image coding apparatus according to the first embodiment. First, in step S601, the quantization matrix holding unit 107 generates a quantization matrix. In step S602, the quantization matrix determination unit 108 determines whether or not the quantization matrix to be encoded matches a predetermined quantization matrix, and generates match information. In this embodiment, the encoded quantization matrix, the default quantization matrix shown in FIG. 5 (a) stored in advance in the coding apparatus, or the flat quantization shown in FIG. 5 (b). Determine if it matches any of the matrices. When it is determined that the quantization matrix to be encoded matches any of the above quantization matrices, reference information indicating which quantization matrix matches is further generated.

In step S603, the quantization matrix header coding unit 109 encodes the matching information and the reference information generated in step S602 to generate a quantization matrix header code. In step S604, the image coding apparatus determines whether or not to encode the coefficient of the quantization matrix to be encoded based on the above-mentioned matching information. If the coefficients of the target quantization matrix are encoded, the process proceeds to step S605, otherwise the process proceeds to step S606.

In step S605, the quantization matrix coefficient coding unit 110 encodes the quantization matrix generated in step S601 to generate a quantization matrix coefficient code. In step S606, the image coding apparatus determines whether or not all the quantization matrices have been encoded, and if it is completed, the process proceeds to step S607, and if not, the next quantization matrix is targeted in step S601. Return to. In step S607, the integrated coding unit 111 encodes the header part of the bit stream and outputs it. In the present embodiment, the sequence header and the picture header of the example shown in FIG. 10 are encoded and output.

In step S608, the block division unit 102 divides the input image for each picture into block units. In step S609, the prediction unit 103 makes a block-by-block prediction and generates a prediction error. In step S610, the conversion unit 104 divides the prediction error generated in step S609 into conversion block sizes and performs orthogonal conversion to generate an orthogonal conversion coefficient. In step S611, the quantization unit 105 quantizes the orthogonal transformation coefficient generated in step S610 using the quantization matrix generated in step S601 and stored in the quantization matrix holding unit 107 to obtain the quantization coefficient. Generate. In step S612, the coefficient coding unit 106 encodes the quantization coefficient generated in step S611 and generates the quantization coefficient code data. In step S613, the image coding apparatus determines whether or not the coding of all the conversion blocks in the block is completed, and if it is completed, the process proceeds to step S614, and if not, the next The process returns to step S610 for the conversion block. In step S614, the image coding apparatus determines whether or not the coding of all the blocks is completed, and if so, stops all operations and ends the process, otherwise the next Return to step S608 for the block of.

With the above configuration and operation, especially when encoding a flat quantization matrix from step S602 to step S604, if it matches with another quantization matrix, it is not necessary to send a redundant code, so the code amount can be reduced. A reduced bitstream can be configured. This makes it possible to generate a bitstream with a smaller amount of code than encoding the coefficients of a flat quantization matrix one by one.

In the present embodiment, a picture using only intra-prediction has been described as an example, but it is clear that a picture in which inter-prediction can be used can also be used. Further, in the present embodiment, the block is 8 × 8 pixels and the conversion block is 4 × 4 pixels for the sake of explanation, but the present invention is not limited to this. For example, the block size can be changed to 16 × 16 pixels or 32 × 32 pixels, and the shape of the block is not limited to a square, and may be a rectangle such as 16 × 8 pixels. Further, although the conversion block size is halved in each of the vertical and horizontal directions, the same size may be used, and of course, the size may be smaller than half in each of the vertical and horizontal directions.

<Embodiment 2>
FIG. 2 is a block diagram showing a configuration of an image decoding device according to a second embodiment of the present invention. In this embodiment, decoding of the bitstream generated in the first embodiment will be described.

In FIG. 2, 201 is a terminal for inputting an encoded bit stream. Reference numeral 202 denotes a decoding / separation unit that decodes the header information of the input bit stream, separates the necessary code from the bit stream, and outputs it to the subsequent stage. The decoding / separation unit 202 performs the reverse operation of the integrated coding unit 111 of FIG.

Reference numeral 203 denotes a quantization matrix header decoding unit that decodes the quantization matrix header code separated from the bitstream header information and restores the match information and the reference information. Reference numeral 204 denotes a quantization matrix coefficient decoding unit that decodes the quantization matrix coefficient code separated from the bit stream and restores each coefficient of the quantization matrix. Reference numeral 205 denotes a quantization matrix reconstruction unit that reconstructs the quantization matrix to be decoded from the obtained match information and reference information or each coefficient of the quantization matrix. Reference numeral 206 denotes a quantization matrix holding unit for storing the quantization matrix obtained by the quantization matrix reconstruction unit 205. It is assumed that the predetermined quantization matrix stored in the quantization matrix determination unit 108 of the first embodiment is held therein. However, the present invention is not limited to this. It may be stored in the quantization matrix reconstruction unit 205. Further, the quantization matrix header decoding unit 203, the quantization matrix coefficient decoding unit 204, and the quantization matrix reconstruction unit 205 may function as one processing unit, the quantization matrix decoding unit.

On the other hand, 207 is a coefficient decoding unit that decodes the quantization coefficient code from the quantization coefficient code data separated by the decoding / separation unit 202 and restores the quantization coefficient. Reference numeral 208 denotes an inverse quantization unit that performs inverse quantization on the quantization coefficient using the quantization matrix stored in the quantization matrix holding unit 206 to obtain an orthogonal conversion coefficient. Reference numeral 209 denotes an inverse conversion unit that performs an inverse orthogonal transformation opposite to that of the conversion unit 104 in FIG. 1 to obtain a prediction error. Reference numeral 210 denotes a prediction reconstruction unit that restores the block image from the prediction error and the decoded image.

The image decoding operation in the image decoding device will be described below. In the present embodiment, the moving image bitstream generated in the first embodiment is input in units of pictures, but a still image bitstream for one picture may be input. Further, in the present embodiment, only the intra-predictive decoding process will be described for ease of explanation, but the present embodiment is not limited to this and can be applied to the inter-predictive decoding process.

In FIG. 2, the bit stream for one picture input from the terminal 201 is input to the decoding / separation unit 202, the header information necessary for reproducing the image is decoded, and the code used in the subsequent stage is further separated. Is output. The quantization matrix header code included in the header information is output to the quantization matrix header decoding unit 203. In the present embodiment, it is assumed that the sequence header and the picture header of the bitstream example shown in FIG. 10 are input / decoded and the code is output.

The quantization matrix header decoding unit 203 first decodes the input quantization matrix header code and restores the matching information. If the match information is 1, which means that a matrix that matches the quantization matrix to be decoded exists, the reference information is further reproduced from the quantization matrix header code. The restored reference information is output to the quantization matrix reconstruction unit 205. On the other hand, if the match information is 0, which means that there is no matching matrix, information indicating that it is necessary to decode the coefficient of the quantization matrix to be decoded is output to the decoding / separation unit 202. To do. In the present embodiment, it is assumed that the matching information is the above-mentioned information, but the form is not particularly limited as long as the information has the same meaning.

In the decoding / separation unit 202, when information indicating that it is necessary to decode the coefficient of the quantization matrix to be decoded is input from the quantization matrix header decoding unit 203, the quantization matrix coefficient decoding unit 204 is quantized. Outputs matrix coefficient code data. In the quantization matrix coefficient decoding unit 204, the input quantization matrix coefficient code data is decoded, and each coefficient of the quantization matrix to be decoded is restored. Each coefficient of the regenerated quantization matrix is output to the quantization matrix reconstruction unit 205.

The quantization matrix reconstruction unit 205 reconstructs the quantization target to be decoded from the reference information input from the quantization matrix header decoding unit 203 or each coefficient of the quantization matrix input from the quantization matrix coefficient decoding unit 204. To do. When each coefficient of the quantization matrix is input from the quantization matrix coefficient decoding unit 204, the quantization matrix is reconstructed using each input coefficient and output to the quantization matrix holding unit 206. On the other hand, when the reference information is input from the quantization matrix header decoding unit 203, the reference destination quantization matrix is read from the quantization matrix holding unit 206 based on the reference information, and the quantization target quantization matrix is re-read. Configure. Then, the reconstructed quantization matrix is output to the quantization matrix holding unit 206. The quantization matrix holding unit 206 stores the input quantization matrix and outputs it to the quantization matrix reconstruction unit 205 and the inverse quantization unit 208 as needed.

Of the codes separated by the decoding / separation unit 202, the quantization coefficient code data is input to the coefficient decoding unit 207. Then, the coefficient decoding unit 207 decodes the quantization coefficient code data for each conversion block, restores the quantization coefficient, and outputs the quantization coefficient to the inverse quantization unit 208. The inverse quantization unit 208 inputs the quantization coefficient restored by the coefficient decoding unit 207 and the quantization matrix stored in the quantization matrix holding unit 206. Then, inverse quantization is performed using the quantization matrix, the orthogonal transformation coefficient is restored, and the orthogonal transformation coefficient is output to the inverse transformation unit 209. The inverse conversion unit 209 inputs the orthogonal transformation coefficient, performs the inverse orthogonal transformation which is the reverse of the conversion unit 104 in FIG. 1, restores the prediction error, and outputs the prediction error to the prediction reconstruction unit 210. The prediction reconstruction unit 210 predicts the input prediction error from the peripheral pixel data that has been decoded, restores the image in block units, and outputs the image from the terminal 211.

FIG. 7 is a flowchart showing an image decoding process in the image decoding apparatus according to the second embodiment.

First, in step S701, the decoding / separation unit 202 decodes the header information and separates the header information in order to output the code to the subsequent stage. In the present embodiment, it is assumed that the sequence header and the picture header of the bitstream example shown in FIG. 10 are input / decoded, and the subsequent codes are output to the corresponding block. In step S702, the quantization matrix header decoding unit 203 first decodes the separated quantization matrix header code and restores the matching information. If the match information is 1, which means that a matrix that matches the quantization matrix to be decoded exists, the reference information is further reproduced from the quantization matrix header code. On the contrary, if it is 0, which means that there is no matrix matching the quantization matrix to be decoded, no further operation is performed in this step.

In step S703, the image decoding apparatus determines whether or not to decode the coefficient of the quantization matrix to be decoded based on the matching information generated in step S702. If the coefficient of the target quantization matrix is decoded, the process proceeds to step S704, otherwise the process proceeds to step S705. In step S704, the quantization matrix coefficient decoding unit 204 decodes the quantization matrix coefficient code data and reproduces each coefficient of the quantization matrix to be decoded. In step S705, the quantization matrix reconstruction unit 205 reconstructs the quantization matrix to be decoded from the reference information restored in step S702 or each coefficient of the quantization matrix restored in step S704.

In step S706, the image decoding apparatus determines whether or not all the quantization matrices have been decoded, and if it is completed, the process proceeds to step S707. If not, the next quantization matrix is targeted at the step. Return to S702. In step S707, the coefficient decoding unit 207 decodes the quantization coefficient code data separated from the bit stream in units of conversion blocks, and restores the quantization coefficient. In step S708, the inverse quantization unit 208 inversely quantizes the quantization coefficient reproduced in step S707 using the quantization matrix reproduced in step S705, and restores the orthogonal transformation coefficient. In step S709, the inverse conversion unit 209 performs inverse orthogonal transformation on the orthogonal transformation coefficient restored in step S708 to restore the prediction error.

In step S710, the image decoding apparatus determines whether or not the decoding of all the conversion blocks in the block is completed, and if it is completed, the process proceeds to step S711. If not, the next conversion block is not completed. Return to step S707. In step S711, the prediction reconstruction unit 210 makes a prediction from the peripheral pixel data that has been decoded, adds it to the prediction error restored in step S709, and generates a decoded image of the block. In step S712, the image decoding device determines whether or not the decoding of all blocks has been completed, and if so, stops all operations and ends the process, otherwise the next block Return to step S707.

With the above configuration and operation, a bit stream containing a code represented by a small amount of code such as the match information generated in the first embodiment can be decoded to obtain a reproduced image.

As in the first embodiment, the block size, the conversion block size, and the block shape are not limited to this. It is also possible to update the quantization matrix by including the quantization matrix code data a plurality of times in the bit stream of one sequence. The decoding / separation unit 202 detects the quantization matrix header code, and the quantization matrix header decoding unit 203 reproduces the matching information and the reference information as needed. Further, according to the matching information, the quantization matrix coefficient decoding unit 204 restores the coefficients of the quantization matrix to be decoded. Next, the quantization matrix reconstruction unit 205 restores the quantization matrix to be decoded from the reference information or each coefficient of the quantization matrix. Then, the restored quantization matrix data is replaced with the corresponding quantization matrix of the quantization matrix holding unit 206. At this time, all the quantization matrices may be rewritten, or a part of the quantization matrix may be changed by determining the quantization matrix to be rewritten.

Further, in the present embodiment, a method of accumulating code data for one picture and then performing processing has been described as an example, but the present invention is not limited to this. For example, an input method such as a block unit or a slice unit composed of a plurality of blocks may be used. Further, it may be divided into fixed-length packets or the like instead of a block or the like.

<Embodiment 3>
FIG. 3 is a block diagram showing an image coding apparatus of this embodiment. In FIG. 3, the same numbers are assigned to the parts that perform the same functions as those in FIG. 1 of the first embodiment, and the description thereof will be omitted.

Similar to the prediction unit 103 of FIG. 1, the prediction unit 303 predicts each block divided by the block division unit 102 in block units, determines a prediction method, calculates a difference value according to the prediction method, and calculates a prediction error. Is. 321 is a conversion control unit that determines whether or not to perform orthogonal transformation with respect to the prediction error generated by the prediction unit 303, outputs the prediction error in the subsequent stage, and outputs the determination result as conversion skip determination information in the subsequent stage. is there. Like the conversion unit 104 in FIG. 1, 304 is a conversion unit that performs orthogonal conversion with respect to the prediction error of each block, and performs orthogonal conversion in units of the input block size or the subdivided blocks to perform orthogonal conversion. Calculate the coefficient.

Reference numeral 322 denotes a bit depth adjusting unit that adjusts the bit depth of the prediction error in order to obtain consistency with the quantization process in the quantization unit 305 in the subsequent stage. The degree of bit depth adjustment here is determined by the bit depth of the input video, the bit depth adjustment associated with the orthogonal conversion in the conversion unit 304, the bit depth adjustment associated with the quantization process in the quantization unit 305, the coefficient value of the quantization matrix, and the like. Shall be. For example, in the present embodiment, the bit depth is adjusted by bit-shifting the prediction error to the left by the difference between the predetermined value 13 and the bit depth of the input image. That is, if the bit depth of the input image is 8 bits, the prediction error by 5 bits is bit-shifted to the left. However, the value used for bit depth adjustment is not limited to this.

Reference numeral 305 is a quantization unit that quantizes the orthogonal transformation coefficient generated by the conversion unit 304 or the prediction error adjusted by the bit depth adjustment unit 322 by the quantization matrix stored in the quantization matrix holding unit 107. is there. Reference numeral 311 is an integrated coding unit that generates a code related to header information, prediction, and conversion, and integrates the code data generated by each of the above-mentioned coding units to form a bit stream and output the code. It differs from the integrated coding unit 111 of FIG. 1 in that the conversion skip determination information is further encoded to generate code data, and a bit stream is formed and output.

The image coding operation in the image coding apparatus will be described below.

First, the prediction unit 303 makes a block-by-block prediction, and a prediction error is generated. The generated prediction error is output to the conversion control unit 321. The conversion control unit 321 determines whether or not to perform orthogonal conversion with respect to the input prediction error. Then, when the orthogonal transformation is performed, the prediction error is output to the conversion unit 304, and when the orthogonal transformation is not performed, the prediction error is output to the bit depth adjusting unit 322. The method for determining whether or not to perform the orthogonal transformation is not particularly limited, but it may be determined from the characteristics of the image, the magnitude of the conversion coefficient after the orthogonal transformation, and the like. Further, the determined determination result is output to the integrated coding unit 311 as conversion skip determination information. The conversion unit 304 performs orthogonal conversion on the prediction error input from the conversion control unit 321 in units of conversion block sizes to generate an orthogonal conversion coefficient.

On the other hand, the bit depth adjustment unit 322 adjusts the bit depth of the prediction error input from the conversion control unit 321 in units of conversion blocks, and generates a bit depth-adjusted prediction error. In the quantization unit 305, the quantization matrix stored in the quantization matrix holding unit 107 with respect to the orthogonal conversion coefficient output from the conversion unit 304 or the bit depth adjusted prediction error output from the bit depth adjustment unit 322. Quantize using to generate the quantization coefficient. The generated quantization coefficient is input to the coefficient coding unit 106.

The integrated coding unit 311 encodes the header information necessary for coding the image, and integrates the input quantization matrix header code and the quantization matrix coefficient code. In the integrated coding unit 311, in addition to the header information encoded by the integrated coding unit 111 of FIG. 1, the conversion skip determination information input from the conversion control unit 321 is further encoded.

FIG. 8 is a flowchart showing an image coding process in the image coding apparatus according to the third embodiment. In FIG. 8, the same numbers are assigned to the portions that perform the same functions as those in FIG. 6 of the first embodiment, and the description thereof will be omitted.

In step S820, the conversion control unit 321 determines whether or not to perform orthogonal transformation with respect to the prediction error generated in step S609, and generates conversion skip determination information. In step S821, if orthogonal transformation is performed based on the conversion skip determination information, the process proceeds to step S610, and if not, the process proceeds to step S822. In step S822, the bit depth adjustment unit 322 adjusts the bit depth of the prediction error generated in step S609 in units of conversion blocks, and generates a bit depth-adjusted prediction error. In step S811, the quantization unit 305 quantizes the orthogonal transformation coefficient generated in step S610 or the bit depth-adjusted prediction error generated in step S822 using the quantization matrix generated in step S601. Generate a quantization coefficient. In step S823, the integrated coding unit 311 encodes the conversion skip determination information generated in step S821.

With the above configuration and operation, especially when encoding a flat quantization matrix corresponding to conversion skip in steps S820 to S823, it is not necessary to send a redundant code, so that a bit stream with a reduced amount of code can be obtained. Can be configured.
In the present embodiment, the conversion skip determination information is encoded in units of conversion blocks, but the unit in which the conversion skip determination information is encoded is not limited to this. It may be encoded in block units, or it is also possible to adopt a configuration that saves the code amount by using the conversion skip determination information common to each color component. The coding method is not particularly limited, but a Huffman code, an arithmetic code, or the like can be used.

<Embodiment 4>
FIG. 4 is a block diagram showing an image decoding device of the present embodiment. In FIG. 4, the same numbers are assigned to the portions that perform the same functions as those in FIG. 2 of the second embodiment, and the description thereof will be omitted. In this embodiment, decoding of the bitstream generated in the third embodiment will be described.

Reference numeral 402 denotes a decoding / separation unit that decodes the header information of the input bit stream, separates the necessary code from the bit stream, and outputs it to the subsequent stage. The decoding / separation unit 402 performs the reverse operation of the integrated coding unit 311 of FIG. It differs from the decoding / separation unit 202 of FIG. 2 in that the conversion skip determination information is further decoded / reproduced and output to the subsequent stage. Reference numeral 408 is an inverse quantization unit that performs inverse quantization on the quantization coefficient using the quantization matrix stored in the quantization matrix holding unit 206 and reproduces the inverse quantization coefficient. The inverse quantization coefficient in the present embodiment corresponds to the orthogonal transformation coefficient or the bit depth-adjusted prediction error in the third embodiment.

Reference numeral 421 is an inverse conversion control unit that determines whether or not to perform inverse orthogonal transformation based on the conversion skip determination information input from the decoding / separation unit 402 and outputs the inverse quantization coefficient in the subsequent stage. Reference numeral 409 is an inverse conversion unit that performs an inverse orthogonal transformation that is the reverse of the conversion unit 304 of FIG. 3 and reproduces a prediction error. Reference numeral 422 is a bit depth reverse adjustment unit that adjusts the bit depth, which is the reverse of the bit depth adjustment unit 322 of FIG. 3, and reproduces the prediction error. The degree of bit depth reverse adjustment here is the bit depth of the output image, the bit depth adjustment associated with the inverse orthogonal conversion in the inverse conversion unit 409, the bit depth adjustment associated with the quantization processing in the inverse quantization unit 408, and the quantization matrix. It shall be determined by the coefficient value and the like. For example, in the present embodiment, the bit depth reverse adjustment is performed by bit-shifting the coefficient to the right by the difference between 13 and the bit depth of the output image. That is, if the bit depth of the output image is 8 bits, the prediction error by 5 bits is bit-shifted to the right. However, the value used for the bit depth reverse adjustment is not limited to this. Reference numeral 410 denotes a prediction reconstruction unit that reproduces a block image from the prediction error and the decoded image.

The image decoding operation in the image decoding device will be described below.

In FIG. 4, the bit stream for one picture input from the terminal 201 is input to the decoding / separation unit 402, the header information necessary for reproducing the image is decoded, and the code used in the subsequent stage is further separated. , Is output as appropriate. The quantization matrix header code included in the header information is output to the quantization matrix header decoding unit 203. Further, in the decoding / separation unit 402, when information indicating that it is necessary to decode the coefficient of the quantization matrix to be decoded is input from the quantization matrix header decoding unit 203, the quantization matrix coefficient decoding unit 204 receives information. Quantization matrix Coefficient code data is output. Further, the decoding / separation unit 402 decodes / reproduces the conversion skip determination information indicating whether or not the inverse orthogonal transformation is performed in units of conversion blocks from the bit stream, and outputs the information to the inverse conversion control unit 421.

In the inverse quantization unit 408, the quantization coefficient reproduced by the coefficient decoding unit 207 and the quantization matrix stored in the quantization matrix holding unit 206 are input. Then, the inverse quantization is performed using the quantization matrix, the inverse quantization coefficient is reproduced, and it is output to the inverse transformation control unit 421. The inverse transformation control unit 421 determines whether or not to perform inverse orthogonal transformation on the input inverse quantization coefficient based on the conversion skip determination information input from the decoding / separation unit 402. Then, when the inverse orthogonal transformation is performed, the inverse quantization coefficient is output to the inverse conversion unit 409, and when the inverse quantization coefficient is not performed, the inverse quantization coefficient is output to the bit depth inverse adjustment unit 422. The inverse conversion unit 209 performs an inverse orthogonal transformation that is the reverse of the conversion unit 304 of FIG. 3 with respect to the input orthogonal conversion coefficient, reproduces the prediction error, and outputs the prediction error to the prediction reconstruction unit 410. To do.

On the other hand, the bit depth reverse adjustment unit 422 performs bit depth reverse adjustment in units of conversion blocks for the bit depth adjusted prediction error, which is the inverse quantization coefficient input from the reverse conversion control unit 421, and reproduces the prediction error. Output to the prediction reconstruction unit 410. The prediction reconstruction unit 410 predicts the input prediction error from the peripheral pixel data that has been decoded, reproduces the image in block units, and outputs the image from the terminal 211.

FIG. 9 is a flowchart showing an image decoding process in the image decoding apparatus according to the fourth embodiment. The same numbers are assigned to the parts that perform the same functions as those in FIG. 7 of the second embodiment, and the description thereof will be omitted.
In step S921, the decoding / separation unit 402 decodes / reproduces the conversion skip determination information indicating whether or not the inverse orthogonal transformation is performed from the bit stream in units of conversion blocks. In step S922, a determination is made based on the conversion skip determination information generated in step S921. If the inverse orthogonal transform is performed, the process proceeds to S709, otherwise the process proceeds to S923. In step S923, the bit depth reverse adjustment unit 422 reverse-adjusts the bit depth of the inverse quantization coefficient generated in step S708 in units of conversion blocks, and reproduces the prediction error.

With the above configuration and operation, even in the image decoding apparatus having the conversion skip processing, the flat quantization matrix generated in the third embodiment and represented by a small amount of code in the quantization matrix matching information or the like is decoded. , A reproduced image can be obtained.

Further, in the present embodiment, the conversion skip determination information exists for each conversion block, but it may be configured to have a flag in the header portion and thereby switch whether or not the decoding device has the conversion skip processing. .. In that case, the flat quantization matrix may be stored and used as a reference target only when the conversion skip process is used, and may be excluded from the reference target otherwise.

<Embodiment 5>
Each of the processing units shown in FIGS. 1, 2, 3 and 4 has been described in the above embodiment as being configured by hardware. However, the processing performed by each of the processing units shown in FIGS. 1, 2, 3 and 4 may be configured by a computer program.

FIG. 12 is a block diagram showing a configuration example of computer hardware applicable to the image display device according to each of the above embodiments.

The CPU 1201 controls the entire computer by using the computer programs and data stored in the RAM 1202 and the ROM 1203, and also executes the above-described processes as those performed by the image processing apparatus according to each of the above embodiments. That is, the CPU 1201 functions as each processing unit shown in FIGS. 1, 2, 3, and 4.

The RAM 1202 has an area for temporarily storing computer programs and data loaded from the external storage device 1206, data acquired from the outside via the I / F (interface) 1207, and the like. Further, the RAM 1202 has a work area used by the CPU 1201 to execute various processes. That is, the RAM 1202 can be allocated as a picture memory, for example, or can provide various other areas as appropriate.

The ROM 1203 stores the setting data of the computer, the boot program, and the like. The operation unit 1204 is composed of a keyboard, a mouse, and the like, and can be operated by a user of the computer to input various instructions to the CPU 1201. The output unit 1205 outputs the processing result by the CPU 1201. Further, the output unit 1205 can be configured by a display device such as a liquid crystal display to display the processing result.

The external storage device 1206 is a large-capacity information storage device typified by a hard disk drive device. The external storage device 1206 stores an OS (operating system) and a computer program for realizing the functions of the respective parts shown in FIGS. 1, 2, 3, and 4 in the CPU 1201. Further, each image as a processing target may be stored in the external storage device 1206.

The computer programs and data stored in the external storage device 1206 are appropriately loaded into the RAM 1202 according to the control by the CPU 1201, and are processed by the CPU 1201. A network such as a LAN or the Internet, or other devices such as a projection device or a display device can be connected to the I / F 1207, and the computer acquires and sends various information via the I / F 1207. Can be done. Reference numeral 1208 is a bus connecting the above-mentioned parts.

The CPU 1201 plays a central role in controlling the operation described in the above-mentioned flowchart for the operation having the above configuration.

<Other Embodiments>
An object of the present invention is also achieved by supplying a storage medium in which a computer program code that realizes the above-mentioned functions is recorded to a system, and the system reads and executes the computer program code. In this case, the computer program code itself read from the storage medium realizes the function of the above-described embodiment, and the storage medium storing the computer program code constitutes the present invention. It also includes cases where the operating system (OS) running on the computer performs part or all of the actual processing based on the instructions in the code of the program, and the processing realizes the above-mentioned functions. ..

Further, it may be realized in the following form. That is, the computer program code read from the storage medium is written to the memory provided in the function expansion card inserted in the computer or the function expansion unit connected to the computer. Then, based on the instruction of the code of the computer program, the function expansion card, the CPU provided in the function expansion unit, or the like performs a part or all of the actual processing to realize the above-mentioned function.

When the present invention is applied to the storage medium, the storage medium stores the code of the computer program corresponding to the flowchart described above.

Claims (1)

It is an image coding method
A conversion step that converts an image orthogonally, and
An adjustment step that adjusts the bit depth of the image without orthogonal transformation of the image,
Quantization is performed based on a quantization matrix in which all the coefficients are the same value with respect to the coefficient obtained by orthogonally transforming the image or the coefficient for which the bit depth is adjusted without orthogonally transforming the image. The quantization step to obtain the quantization coefficient by
The quantization coefficient, the conversion skip determination information indicating whether or not the quantization coefficient is an orthogonally converted coefficient, and the reference information indicating whether or not all the coefficients are the same value are coded. With a coding step to quantize,
In the quantization step, when quantization is performed using a quantization matrix in which all the coefficients have the same value with respect to the coefficient whose bit depth is adjusted without orthogonally transforming the image, the quantization coefficient is intra. Quantization is performed using a quantization matrix in which all coefficients are the same value regardless of whether they are encoded using prediction or inter-prediction.
In the coding step, the conversion skip determination information indicating that the quantization coefficient is not an orthogonally converted coefficient and the reference information indicating whether or not all the coefficients are the same value in the quantization matrix are coded. An image coding method characterized by being quantized.

Images