JP2017060177A - Image encoding apparatus, image encoding method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a decoding method of an efficiently decoded quantization matrix.SOLUTION: A decoding method includes: inverse quantization means of acquiring a quantization matrix from a bit stream, and inversely quantizes a quantization coefficient on the basis of the quantization matrix regardless of determination information showing whether or not the quantization coefficient is an orthogonal transformation coefficient; inverse orthogonal transformation means of inversely orthogonal transforming the inverse quantization coefficient by the inverse quantization when the determination information shows that the quantization coefficient is the orthogonal transformation coefficient; adjusting means of adjusting a bit depth of the inverse quantization coefficient by the inverse quantization means when the determination information shows that the quantization coefficient is the orthogonal transformation coefficient; and decoding means of decoding an image from data acquired by the inverse orthogonal transformation means or data decoding the image form the data acquired by the adjusting means.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image decoding apparatus, an image decoding method, and a program, and more particularly, to a quantization matrix encoding method and decoding method.

As a method of compressing and recording moving images, H.264 H.264 / MPEG-4 AVC (hereinafter referred to as H.264) is known. (Non-Patent Document 1)
H. In H.264, each coefficient of the quantization matrix can be changed to an arbitrary value by encoding scaling_list information. According to Non-Patent Document 1, each coefficient of the quantization matrix can take an arbitrary value by adding delta_scale which is a difference value to the immediately preceding coefficient.

  In recent years, H.C. As a successor to H.264, activities to start international standardization of a more efficient coding method have started. JCT-VC (Joint Collaborative Team on Video Coding) was established between ISO / IEC and ITU-T. In this JCT-VC, standardization is being advanced as a High Efficiency Video Coding (HEVC) encoding method (hereinafter referred to as HEVC).

  In this method, in order to improve coding efficiency, an orthogonal transform method such as a discrete sine transform is being studied in addition to a conventional discrete cosine-based orthogonal transform method. Also, in order to improve the coding efficiency of not only natural images taken with a camera, but also artificial images such as PC screens and games, a method such as a conversion skip technique that does not perform orthogonal transform is also being studied. (Non-Patent Document 2)

ITU-TH. H.264 (03/2010) Advanced video coding for generic audiovisual services JCT-VC contribution JCTVC-I1003. doc Internet <http: // phenix. int-evry. fr / jct / doc_end_user / documents / 9_Geneva / wg11 />

  In general, in image compression, in order to improve subjective image quality, among the transform coefficients after orthogonal transform, the coefficient corresponding to the low frequency component is quantized with a smaller value, and the coefficient corresponding to the high frequency component is increased with a larger value. It is often quantized. In this case, it is common to perform weighted quantization with frequency components using a quantization matrix as shown in FIG. The quantization matrix 500 has a smaller coefficient corresponding to a low frequency component and a larger coefficient corresponding to a high frequency component.

  On the other hand, in HEVC, especially when the above-described transform skip technique is used, orthogonal transform is not performed, so that all the coefficients to be quantized have the same frequency component. For this reason, when the conversion skip technique is used, it is not preferable to perform the above-described weighted quantization, and a flat quantization matrix 501 in which all the coefficients have the same value as shown in FIG. 5B is used. It is necessary to perform quantization processing. For this reason, it is desirable that the information of the flat quantization matrix 501 can be encoded with a smaller code amount. However, particularly in the above-described HEVC, there is no mechanism for efficient encoding, and as a result, the code amount is unnecessarily large. It had become.

  Therefore, the present invention has been made to solve the above-described problem, and by introducing a method for efficiently encoding a flat quantization matrix with a smaller code amount, high-efficiency quantization matrix coding is achieved. -The purpose is to realize decryption.

In order to solve the above-described problems, the image decoding apparatus of the present invention has the following configuration. That is, acquisition means for acquiring a quantization coefficient and determination information indicating whether or not the quantization coefficient is an orthogonally-transformed coefficient from the bit stream, acquiring a quantization matrix from the bit stream, Regardless of the determination information, the quantization coefficient is inversely quantized based on the acquired quantization matrix, and the determination information acquired by the acquisition means is obtained by orthogonally transforming the quantization coefficient. The inverse quantization transform means for inverse orthogonal transformation of the coefficient inversely quantized by the inverse quantization means, and the determination information obtained by the obtaining means are such that the quantization coefficient is orthogonal transformed. In the case of indicating that the coefficient has not been processed, the adjustment means for adjusting the bit depth of the coefficient inversely quantized by the inverse quantization means, and the inverse orthogonal transform means Having a decoding means for decoding an image from data obtained by the data obtained or the adjustment means, I.

  Furthermore, the image decoding method of the present invention has the following configuration. That is, an acquisition step of acquiring a quantization coefficient and determination information indicating whether or not the quantization coefficient is an orthogonally transformed coefficient from the bit stream, acquiring a quantization matrix from the bit stream, Regardless of the determination information, the quantization coefficient is inversely quantized based on the acquired quantization matrix, and the determination information acquired in the acquisition step is obtained by orthogonally transforming the quantization coefficient. The inversely orthogonal transform step for inversely orthogonally transforming the coefficient inversely quantized in the inverse quantization step, and the determination information acquired in the acquisition step, the quantized coefficient is orthogonally transformed. An adjustment that adjusts the bit depth of the dequantized coefficient in the dequantization step when indicating that the coefficient is not It has a step, and a decoding step of decoding the image from the obtained data in the obtained data or the adjustment step, in the inverse orthogonal transform step

  According to the present invention, the amount of codes for quantization matrix coding can be reduced, and highly efficient coding / decoding becomes possible.

1 is a block diagram illustrating a configuration of an image encoding device according to a first embodiment. FIG. 3 is a block diagram illustrating a configuration of an image decoding device according to Embodiment 2. FIG. 9 is a block diagram illustrating a configuration of an image encoding device according to Embodiment 3. FIG. 7 is a block diagram illustrating a configuration of an image decoding device according to Embodiment 4. (A), (b) The figure which shows an example of a quantization matrix 7 is a flowchart showing image encoding processing in the image encoding apparatus according to the first embodiment. 7 is a flowchart showing image decoding processing in the image decoding apparatus according to the second embodiment. 10 is a flowchart showing image encoding processing in the image encoding apparatus according to the third embodiment. 10 is a flowchart showing image decoding processing in the image decoding apparatus according to the fourth embodiment. (A), (b), (c) The figure which shows an example of a bit stream structure The figure which shows an example of the relationship between the quantization matrix of encoding object, and quantization matrix reference information The block diagram which shows the structural example of the hardware of the computer applicable to the image coding apparatus of this invention, and a decoding apparatus.

  Hereinafter, the present invention will be described in detail based on the preferred embodiments with reference to the accompanying drawings. The configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

<Embodiment 1>
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an image encoding apparatus according to this embodiment.

  In FIG. 1, reference numeral 101 denotes a terminal for inputting image data (hereinafter referred to as an image). A block dividing unit 102 divides the input image into a plurality of blocks. A prediction unit 103 predicts each block divided by the block dividing unit 102 in units of blocks, determines a prediction method, calculates a difference value according to the prediction method, and calculates a prediction error. The prediction unit 103 also performs intra prediction or motion compensation prediction. Intra prediction is generally realized by selecting an optimum prediction method from a plurality of reference methods in order to calculate a prediction value from surrounding pixel data.

  Reference numeral 104 denotes a transform unit that performs orthogonal transform on the prediction error of each block, and performs orthogonal transform in units of the size of the input block or a size obtained by subdividing it to calculate orthogonal transform coefficients. Below, the block which performs orthogonal transformation is called a transformation block. Although it does not specifically limit regarding orthogonal transformation, you may use discrete cosine transformation, Hadamard transformation, etc. Further, in the present embodiment, for the sake of explanation, the prediction error in units of 8 × 8 pixels is divided into two in the vertical and horizontal directions, and orthogonal transform is performed in units of 4 × 4 pixels, but the size of the transform block, The shape is not limited to this. Orthogonal transformation may be performed using a transform block having the same size as the block, or orthogonal transform may be performed using a transform block obtained by dividing the block in units smaller than vertical and horizontal divisions.

  Reference numeral 107 denotes a quantization matrix holding unit that generates and stores a quantization matrix. The method for generating the stored quantization matrix is not particularly limited, but the user may input the quantization matrix or use the one specified in advance as the initial value even if it is calculated from the characteristics of the input image. Of course it is good. In the present embodiment, it is assumed that a two-dimensional quantization matrix corresponding to a 4 × 4 pixel conversion block as shown in FIGS. 5A and 5B is generated and stored.

  Reference numeral 105 denotes a quantization unit that quantizes the orthogonal transform coefficient using a quantization matrix stored in the quantization matrix holding unit 107. A quantization coefficient can be obtained by this quantization. Reference numeral 106 denotes a coefficient encoding unit that encodes the quantized coefficient thus obtained to generate quantized coefficient code data. The encoding method is not particularly limited, but a Huffman code or an arithmetic code can be used. Reference numeral 108 denotes a quantization matrix determination unit that determines whether or not the quantization matrix stored in the quantization matrix holding unit 107 matches a predetermined quantization matrix. More specifically, whether the quantization matrix to be encoded is an already encoded quantization matrix, a predetermined quantization matrix, or a flat quantization matrix having all the same values. judge. It is assumed that this predetermined quantization matrix is stored in the quantization matrix holding unit 107. However, it is not limited to this. For example, it may be stored in the quantization matrix determination unit 108.

  Reference numeral 109 denotes a quantization matrix header encoding unit that encodes information indicating the determination in the quantization matrix determination unit 108 and other information related to the quantization matrix as a header to generate a quantization matrix header code. Reference numeral 110 denotes a quantization matrix coefficient encoding unit that encodes each coefficient of a quantization matrix to be encoded to generate a quantization matrix coefficient code.

  Reference numeral 111 denotes an integrated coding unit that generates codes related to header information, prediction, and conversion, and forms and outputs a bitstream by integrating the code data generated by each of the above-described encoding units. Specifically, the quantization coefficient code data generated by the coefficient encoding unit 106, the quantization matrix header code generated by the quantization matrix header encoding unit 109, and the quantization matrix coefficient encoding unit 110 Integrate quantization matrix coefficient codes. Here, the codes relating to prediction and conversion are, for example, codes such as a selected prediction method and a state of conversion block division. Reference numeral 112 denotes a terminal for outputting the bit stream generated by the integrated encoding unit 111 to the outside.

  An image encoding operation in the image encoding apparatus will be described below. In this embodiment, moving image data is input in units of pictures, but still image data for one picture may be input. Further, in the present embodiment, only the intra prediction encoding process will be described for ease of explanation, but the present invention is not limited to this and can be applied to the inter prediction encoding process. In the present embodiment, for the sake of explanation, the block dividing unit 102 will be described as dividing into 8 × 8 pixel blocks, but the present invention is not limited to this.

<Encoding of quantization matrix coefficient>
First, the quantization matrix holding unit 107 generates a quantization matrix. The quantization matrix is determined according to the transform block size that is the size of the orthogonal transform used for the encoding process. The method for determining the quantization matrix coefficient is not particularly limited. For example, a predetermined initial value may be used, or may be set individually. Further, it may be set and generated according to the characteristics of the image.

  The quantization matrix holding unit 107 holds the quantization matrix generated in this way. FIGS. 5A and 5B are examples of quantization matrices corresponding to 4 × 4 pixel conversion blocks. In order to simplify the description, it is assumed that the configuration is for 16 pixels corresponding to a 4 × 4 pixel conversion block, and each square in the thick frame represents a coefficient. In this embodiment, it is assumed that the quantization matrix shown in FIGS. 5A and 5B is held in a two-dimensional shape, but the coefficients in the quantization matrix are of course not limited to this. For example, when a transform block size of 8 × 8 pixels is used in addition to the present embodiment, another quantization matrix corresponding to the 8 × 8 pixel transform block is held.

  The quantization matrix determination unit 108 sequentially reads out the quantization matrix to be encoded from the quantization matrix holding unit 107. Then, it is determined whether or not it matches a predetermined quantization matrix, and matching information indicating that the quantization matrix to be encoded matches the predetermined quantization matrix is generated. In the present embodiment, whether or not the encoded quantization matrix matches any of the default quantization matrix shown in FIG. 5A or the flat quantization matrix shown in FIG. Determine. The encoded quantization matrix is stored in the quantization matrix holding unit 107, and the quantization matrix shown in FIGS. 5A and 5B is also stored in the quantization matrix holding unit 107 in advance. It shall be. When it is determined that the quantization matrix to be encoded matches any one, reference information indicating which quantization matrix further matches is generated. This reference information is output to the quantization matrix header encoding unit 109 together with the coincidence information. In this case, nothing is output to the quantization matrix coefficient encoding unit 110 and the operation is not performed. On the other hand, if none of the quantization matrices match, only the matching information is output to the quantization matrix header encoding unit 109 and the quantization matrix to be encoded is output to the quantization matrix coefficient encoding unit 110. In the present embodiment, the matching information is 1 if there is a predetermined quantization matrix that matches the quantization matrix to be encoded, and 0 otherwise. The combination with is not limited to this.

  FIG. 11 is a diagram illustrating an example of a relationship between a quantization matrix to be encoded and reference information in the present embodiment. In the present embodiment, the first three types corresponding to the quantization of the Y component of the image input in the YCbCr color space, the second corresponding to the quantization of the Cb component, and the third three types corresponding to the quantization of the Cr component. Although a quantization matrix is assumed to exist, the present invention is not limited to this. More quantization matrices may be defined depending on combinations of transform block size, prediction method, and color space. In this embodiment, the quantization matrix shown in FIG. 5A is a default 4 × 4 quantization matrix, and the quantization matrix shown in FIG. 5B is a flat 4 × 4 quantization matrix. However, the quantization matrix used is not limited to these.

  For example, when the encoding target quantization matrix is the first (4 × 4Y) and matches the default 4 × 4, the reference information is set to 0. In addition, when it matches flat 4 × 4, the reference information is 1. Matching with such a flat 4 × 4 is flat information indicating that all the coefficients of the quantization matrix to be encoded have the same value. Similarly, when the encoding target quantization matrix is the second (4 × 4Cb), the reference information is 0 if it matches the default 4 × 4, and the reference information is 1 if it matches the encoded first (4 × 4Y). Become. The reference information is 2 if it matches the flat 4 × 4. The same applies when the encoding target quantization matrix is the third (4 × 4Cr).

  The quantization matrix header encoding unit 109 encodes the matching information input from the quantization matrix determination unit 108 and the reference information as necessary to generate a quantization matrix header code. In this embodiment, when the quantization matrix to be encoded matches either the quantization matrix stored in advance in the encoding device or the encoded quantization matrix, both the match information and the reference information are displayed. Encode. On the other hand, if it does not match any of the above quantization matrices, only the matching information is encoded. The encoding method is not particularly limited, but a Huffman code or an arithmetic code can be used. The generated quantization matrix header code is output to the integrated encoding unit 111.

  The quantization matrix coefficient encoding unit 110 encodes the quantization matrix to be encoded input from the quantization matrix determination unit 108 to generate a quantization matrix coefficient code. In the present embodiment, the quantization matrix coefficient encoding unit 110 is used only when the quantization matrix to be encoded does not match the quantization matrix stored in advance in the encoding device or the encoded quantization matrix. Operate. As an encoding method, in this embodiment, a method is used in which each coefficient is encoded as it is, a difference from the immediately preceding coefficient is encoded, and a difference from DPCM or another quantization matrix is encoded. However, the method is not limited to these. Further, when a flat quantization matrix that is not stored in advance in the encoding device is encoded, a method may be adopted in which only the representative value is encoded. In addition, when the same value continues with these, the code amount of the quantization matrix can also be suppressed by inserting a code that terminates the encoding.

  The integrated encoding unit 111 encodes header information necessary for image encoding, and integrates the input quantization matrix header code and quantization matrix coefficient code.

  Note that the quantization matrix determination unit 108, the quantization matrix header encoding unit 109, and the quantization matrix coefficient encoding unit 110 may function as a quantization matrix encoding unit that is one processing unit.

<Image coding>
An image for one picture input from the terminal 101 is input to the block dividing unit 102 and divided into blocks of 8 × 8 pixels. The divided image is input to the prediction unit 103. The prediction unit 103 performs prediction in units of blocks and generates a prediction error. The transform unit 104 divides the prediction error generated by the prediction unit 103 into transform block sizes and performs orthogonal transform to generate orthogonal transform coefficients. Then, the orthogonal transform coefficient is input to the quantization unit 105. In the present embodiment, it is assumed that orthogonal transformation is performed by dividing a prediction error of a block unit of 8 × 8 pixels into a conversion block unit of 4 × 4 pixels. The quantization unit 105 quantizes the orthogonal transform coefficient output from the transform unit 104 using the quantization matrix stored in the quantization matrix holding unit 107 to generate a quantized coefficient. The generated quantized coefficient is input to the coefficient encoding unit 106. The coefficient encoding unit 106 encodes the quantization coefficient generated by the quantization unit 105, generates quantized coefficient code data, and outputs it to the integrated encoding unit 111. The integrated encoding unit 111 generates a code related to prediction and conversion in units of blocks, integrates the encoded data of the header and the quantized coefficient code data generated by the coefficient encoding unit 106 together with the encoded data of the header, A stream is generated and output from the terminal 112.

  FIG. 10 shows an example of the bit stream output in the first embodiment. The sequence header includes code data of a quantization matrix, and includes a header code and a result of encoding each coefficient. FIG. 10A shows an example of a bitstream in which all of the first to third quantization matrices are different and are different from a flat quantization matrix and a default quantization matrix. That is, the first to third quantization matrix header codes are encoded with the matching information 0, and each quantization matrix coefficient code is encoded with the coefficient of each quantization matrix. However, the encoding position is not limited to this. Of course, it does not matter even if it takes the structure encoded in the picture header part and other header parts.

  FIG. 10B shows an example of a bit stream in the case where all of the first to third quantization matrices are flat quantization matrices. In this case, in the first quantization matrix header code, 1 of matching information and 1 of reference information indicating a flat quantization matrix are encoded. The second quantization matrix header code is encoded with the match information 1 and the reference information 2 indicating a flat quantization matrix. In this case, even if the reference information is 1, since the first quantization matrix to be referenced is also a flat quantization matrix, the bitstream decoding results are the same. Similarly, in the third quantization matrix header code, 1 of matching information and 2 of reference information indicating a flat quantization matrix are encoded. Also in this case, even if the reference information is 1 or 2, since the first or second quantization matrix to be referred to is also a flat quantization matrix, the decoding result of the bit stream is the same.

  FIG. 10C shows a case where the first quantization matrix is different from the flat quantization matrix and the default quantization matrix, but the second and third quantization matrices are the same as the first quantization matrix. It is an example of a bit stream. In this case, the first quantization matrix header code is encoded with the matching information 0, and the first quantization matrix coefficient code is encoded with the coefficient of the first quantization matrix. In addition, in the second quantization matrix header code, 1 of matching information and 1 of reference information indicating a reference to the first quantization matrix are encoded. Similarly, in the third quantization matrix header code, 1 of coincidence information and 1 of reference information indicating a reference to the second quantization matrix are encoded.

  FIG. 6 is a flowchart showing an image encoding process in the image encoding apparatus according to the first embodiment. First, in step S601, the quantization matrix holding unit 107 generates a quantization matrix. In step S602, the quantization matrix determination unit 108 determines whether the quantization matrix to be encoded matches a predetermined quantization matrix, and generates matching information. In the present embodiment, the encoded quantization matrix, the default quantization matrix shown in FIG. 5 (a) stored in advance in the encoding device, or the flat quantization shown in FIG. 5 (b). It is determined whether or not it matches any of the matrices. When it is determined that the quantization matrix to be encoded matches any of the above-described quantization matrices, reference information indicating which quantization matrix is further matched is generated.

  In step S603, the quantization matrix header encoding unit 109 encodes the match information and the reference information generated in step S602 to generate a quantization matrix header code. In step S604, the image encoding apparatus determines whether to encode the coefficient of the quantization matrix to be encoded based on the above-described coincidence information. If the coefficient of the target quantization matrix is encoded, the process proceeds to step S605. Otherwise, the process proceeds to step S606.

  In step S605, the quantization matrix coefficient encoding unit 110 encodes the quantization matrix generated in step S601 to generate a quantization matrix coefficient code. In step S606, the image coding apparatus determines whether or not all the quantization matrices have been coded. If completed, the process proceeds to step S607. Otherwise, the next quantization matrix is processed in step S601. Return to. In step S607, the integrated encoding unit 111 encodes and outputs the header portion of the bitstream. In the present embodiment, it is assumed that the sequence header and picture header of the example shown in FIG. 10 are encoded and output.

  In step S608, the block dividing unit 102 divides the input image in units of pictures into units of blocks. In step S609, the prediction unit 103 performs block unit prediction and generates a prediction error. In step S610, the transform unit 104 divides the prediction error generated in step S609 into transform block sizes, performs orthogonal transform, and generates orthogonal transform coefficients. In step S611, the quantization unit 105 quantizes the orthogonal transform coefficient generated in step S610 using the quantization matrix generated in step S601 and stored in the quantization matrix holding unit 107, and obtains the quantization coefficient. Generate. In step S612, the coefficient encoding unit 106 encodes the quantized coefficient generated in step S611 to generate quantized coefficient code data. In step S613, the image coding apparatus determines whether or not the coding of all the transform blocks in the block has been finished. If finished, the process proceeds to step S614. The process returns to step S610 for the conversion block. In step S614, the image encoding apparatus determines whether or not encoding of all blocks has been completed, and if completed, stops all operations and ends the process. Returning to the block S608, the process returns to step S608.

  With the above configuration and operation, particularly when encoding a flat quantization matrix by steps S602 to S604, if it matches with other quantization matrices, it is not necessary to send a redundant code. A reduced bitstream can be constructed. As a result, a bit stream can be generated with a smaller code amount than when the coefficients of the flat quantization matrix are encoded one by one.

  In the present embodiment, a picture using only intra prediction has been described as an example. However, it is obvious that a picture that can use inter prediction can also be used. Further, in this embodiment, for the sake of explanation, the block is 8 × 8 pixels and the conversion block is 4 × 4 pixels, but the present invention is not limited to this. For example, the block size can be changed to 16 × 16 pixels or 32 × 32 pixels, and the shape of the block is not limited to a square, but may be a rectangle such as 16 × 8 pixels. Further, although the conversion block size is half of the block size in the vertical and horizontal directions, it may be the same size, or of course a size finer than half of the vertical and horizontal directions.

<Embodiment 2>
FIG. 2 is a block diagram showing the configuration of the image decoding apparatus according to Embodiment 2 of the present invention. In the present embodiment, decoding of the bitstream generated in the first embodiment will be described.

  In FIG. 2, 201 is a terminal for inputting an encoded bit stream. A decoding / separating unit 202 decodes header information of the input bitstream, separates necessary codes from the bitstream, and outputs them to the subsequent stage. The decoding / separating unit 202 performs the reverse operation of the integrated encoding unit 111 in FIG.

  Reference numeral 203 denotes a quantization matrix header decoding unit that decodes the quantization matrix header code separated from the header information of the bit stream and restores the matching information and the reference information. A quantization matrix coefficient decoding unit 204 decodes the quantization matrix coefficient code separated from the bit stream and restores each coefficient of the quantization matrix. Reference numeral 205 denotes a quantization matrix reconstruction unit that reconstructs a quantization matrix to be decoded from the obtained matching information and reference information or each coefficient of the quantization matrix. A quantization matrix holding unit 206 stores the quantization matrix obtained by the quantization matrix reconstruction unit 205. It is assumed that the predetermined quantization matrix stored in the quantization matrix determination unit 108 of the first embodiment is held in this. However, it is not limited to this. It may be stored in the quantization matrix reconstruction unit 205. Further, the quantization matrix header decoding unit 203, the quantization matrix coefficient decoding unit 204, and the quantization matrix reconstruction unit 205 may function as a quantization matrix decoding unit that is one processing unit.

  On the other hand, reference numeral 207 denotes a coefficient decoding unit that decodes the quantized coefficient code from the quantized coefficient code data separated by the decoding / separating unit 202 and restores the quantized coefficient. Reference numeral 208 denotes an inverse quantization unit that performs inverse quantization on the quantization coefficient using the quantization matrix stored in the quantization matrix holding unit 206 to obtain an orthogonal transform coefficient. Reference numeral 209 denotes an inverse transform unit that performs an inverse orthogonal transform that is the inverse of the transform unit 104 in FIG. 1 to obtain a prediction error. A prediction reconstruction unit 210 restores a block image from a prediction error and a decoded image.

  An image decoding operation in the image decoding apparatus will be described below. In the present embodiment, the moving image bitstream generated in the first embodiment is input in units of pictures, but a still image bitstream for one picture may be input. Further, in the present embodiment, only the intra prediction decoding process will be described for ease of explanation, but the present invention is not limited to this and can be applied to the inter prediction decoding process.

  In FIG. 2, the bit stream for one picture input from the terminal 201 is input to the decoding / separating unit 202, the header information necessary for reproducing the image is decoded, and the code used in the subsequent stage is further separated. Is output. The quantization matrix header code included in the header information is output to the quantization matrix header decoding unit 203. In the present embodiment, it is assumed that the sequence header and the picture header in the example of the bit stream shown in FIG. 10 are input / decoded and a code is output.

  First, the quantization matrix header decoding unit 203 decodes the input quantization matrix header code to restore the match information. If the match information is 1, which means that there is a matrix that matches the quantization matrix to be decoded, the reference information is further reproduced from the quantization matrix header code. The restored reference information is output to the quantization matrix reconstruction unit 205. On the other hand, if the matching information is 0, which means that there is no matching matrix, information indicating that it is necessary to decode the coefficient of the quantization matrix to be decoded is output to the decoding / separating unit 202. To do. In the present embodiment, the coincidence information is the above-described information, but the form is not particularly limited as long as the information has the same meaning.

  In the decoding / separation unit 202, when information indicating that it is necessary to decode the coefficient of the quantization matrix to be decoded is input from the quantization matrix header decoding unit 203, the quantization matrix coefficient decoding unit 204 performs quantization. Outputs matrix coefficient code data. The quantization matrix coefficient decoding unit 204 decodes the input quantization matrix coefficient code data and restores each coefficient of the quantization matrix to be decoded. Each coefficient of the reproduced quantization matrix is output to the quantization matrix reconstruction unit 205.

  The quantization matrix reconstruction unit 205 reconstructs the quantization matrix to be decoded from the reference information input from the quantization matrix header decoding unit 203 or each coefficient of the quantization matrix input from the quantization matrix coefficient decoding unit 204. To do. When each coefficient of the quantization matrix is input from the quantization matrix coefficient decoding unit 204, the quantization matrix is reconfigured using each input coefficient, and is output to the quantization matrix holding unit 206. On the other hand, when reference information is input from the quantization matrix header decoding unit 203, based on the reference information, the reference destination quantization matrix is read from the quantization matrix holding unit 206, and the quantization matrix to be decoded is re-read. Configure. The reconstructed quantization matrix is output to the quantization matrix holding unit 206. The quantization matrix holding unit 206 stores the input quantization matrix and outputs it to the quantization matrix reconstruction unit 205 and the inverse quantization unit 208 as necessary.

  Among the codes separated by the decoding / separating unit 202, the quantized coefficient code data is input to the coefficient decoding unit 207. Then, the coefficient decoding unit 207 decodes the quantized coefficient code data for each transform block, restores the quantized coefficient, and outputs it to the inverse quantization unit 208. The inverse quantization unit 208 receives the quantization coefficient restored by the coefficient decoding unit 207 and the quantization matrix stored in the quantization matrix holding unit 206. Then, inverse quantization is performed using the quantization matrix, the orthogonal transform coefficient is restored, and output to the inverse transform unit 209. The inverse transform unit 209 receives the orthogonal transform coefficient, performs inverse orthogonal transform that is the reverse of the transform unit 104 in FIG. 1, restores the prediction error, and outputs the prediction error to the prediction reconstruction unit 210. The prediction reconstruction unit 210 performs prediction from the surrounding pixel data that has been decoded on the input prediction error, restores an image in block units, and outputs the image from the terminal 211.

  FIG. 7 is a flowchart illustrating image decoding processing in the image decoding apparatus according to the second embodiment.

  First, in step S701, the decoding / separating unit 202 decodes the header information and separates the code for output to the subsequent stage. In the present embodiment, it is assumed that the sequence header and picture header in the example of the bit stream shown in FIG. 10 are input and decoded, and the subsequent code is output to the corresponding block. In step S702, the quantization matrix header decoding unit 203 first decodes the separated quantization matrix header code to restore the match information. If the match information is 1, which means that there is a matrix that matches the quantization matrix to be decoded, the reference information is further reproduced from the quantization matrix header code. On the other hand, if it is 0, which means that there is no matrix that matches the quantization matrix to be decoded, no further operation is performed in this step.

  In step S703, the image decoding apparatus determines whether to decode the coefficient of the quantization matrix to be decoded based on the coincidence information generated in step S702. If the coefficient of the target quantization matrix is decoded, the process proceeds to step S704. Otherwise, the process proceeds to step S705. In step S704, the quantization matrix coefficient decoding unit 204 decodes the quantization matrix coefficient code data, and reproduces each coefficient of the quantization matrix to be decoded. In step S705, the quantization matrix reconstruction unit 205 reconstructs a quantization matrix to be decoded from the reference information restored in step S702 or each coefficient of the quantization matrix restored in step S704.

  In step S706, the image decoding apparatus determines whether all the quantization matrices have been decoded. If completed, the process proceeds to step S707. Otherwise, the next decoding matrix is processed as a target. The process returns to S702. In step S707, the coefficient decoding unit 207 decodes the quantized coefficient code data separated from the bit stream in units of transform blocks, and restores the quantized coefficients. In step S708, the inverse quantization unit 208 performs inverse quantization on the quantization coefficient reproduced in step S707 using the quantization matrix reproduced in step S705, and restores the orthogonal transform coefficient. In step S709, the inverse transform unit 209 performs inverse orthogonal transform on the orthogonal transform coefficient restored in step S708 to restore the prediction error.

  In step S710, the image decoding apparatus determines whether or not decoding of all the transform blocks in the block has been completed. If it has been completed, the process proceeds to step S711. The process returns to step S707. In step S711, the prediction reconstruction unit 210 performs prediction from the decoded surrounding pixel data, and adds the prediction error restored in step S709 to generate a decoded image of the block. In step S712, the image decoding apparatus determines whether or not the decoding of all the blocks has been completed. If the decoding has been completed, the image decoding apparatus stops all the operations and ends the process. The process returns to step S707.

  With the above configuration and operation, it is possible to decode a bitstream including a code represented by a small code amount such as the matching information generated in Embodiment 1 and obtain a reproduced image.

  As in the first embodiment, the block size, the transform block size, and the block shape are not limited thereto. It is also possible to update the quantization matrix by including the quantization matrix code data a plurality of times in one sequence of bit stream. The decoding / separating unit 202 detects the quantization matrix header code, and the quantization matrix header decoding unit 203 reproduces the matching information and, if necessary, the reference information. Further, according to the coincidence information, the quantization matrix coefficient decoding unit 204 restores the coefficients of the quantization matrix to be decoded. Next, the quantization matrix reconstruction unit 205 restores the decoding target quantization matrix from the reference information or each coefficient of the quantization matrix. Then, the restored quantization matrix data is replaced with a corresponding quantization matrix in the quantization matrix holding unit 206. At this time, all the quantization matrices may be rewritten, or a part of them can be changed by determining the quantization matrix to be rewritten.

  In the present embodiment, the method of performing processing after accumulating code data for one picture has been described as an example. However, the present invention is not limited to this. For example, an input method such as a block unit or a slice unit composed of a plurality of blocks may be used. Furthermore, instead of a block or the like, it may be divided into fixed-length packets or the like.

<Embodiment 3>
FIG. 3 is a block diagram showing an image encoding apparatus according to this embodiment. In FIG. 3, the same numbers are assigned to portions that perform the same functions as those in FIG.

  Similar to the prediction unit 103 in FIG. 1, a prediction unit 303 predicts each block divided by the block division unit 102 in units of blocks, determines a prediction method, calculates a difference value accordingly, and calculates a prediction error. It is. A conversion control unit 321 determines whether or not to perform orthogonal transformation on the prediction error generated by the prediction unit 303, outputs the prediction error to the subsequent stage, and outputs the determination result to the subsequent stage as conversion skip determination information. is there. Similar to the conversion unit 104 in FIG. 1, 304 is a conversion unit that performs orthogonal conversion on the prediction error of each block, and performs orthogonal conversion on the basis of the input block size or a block obtained by subdividing the block, and performs orthogonal conversion Calculate the coefficient.

  Reference numeral 322 denotes a bit depth adjustment unit that adjusts the bit depth of the prediction error in order to maintain consistency with the quantization processing in the quantization unit 305 in the subsequent stage. The degree of bit depth adjustment here is determined by the bit depth of the input video, the bit depth adjustment accompanying orthogonal transformation in the transform unit 304, the bit depth adjustment accompanying quantization processing in the quantization unit 305, the coefficient value of the quantization matrix, and the like. Shall. For example, in the present embodiment, the bit depth adjustment is performed by bit-shifting the prediction error to the left by the difference between the predetermined value 13 and the bit depth of the input image. That is, if the bit depth of the input image is 8 bits, the prediction error is bit-shifted to the left by 5 bits. However, the value used for bit depth adjustment is not limited to this.

  A quantization unit 305 quantizes the orthogonal transformation coefficient generated by the conversion unit 304 or the prediction error adjusted by the bit depth adjustment unit 322 by the quantization matrix stored in the quantization matrix holding unit 107. is there. Reference numeral 311 denotes an integrated encoding unit that generates codes related to header information, prediction, and conversion, and also integrates the code data generated by each of the above-described encoding units to form and output a bitstream. 1 is different from the integrated encoding unit 111 of FIG. 1 in that conversion skip determination information is further encoded to generate code data, and a bit stream is formed and output.

  An image encoding operation in the image encoding apparatus will be described below.

  In the prediction unit 303, first, prediction in block units is performed, and a prediction error is generated. The generated prediction error is output to the conversion control unit 321. The transformation control unit 321 determines whether or not to perform orthogonal transformation on the input prediction error. If orthogonal transformation is performed, the prediction error is output to the conversion unit 304, and if orthogonal transformation is not performed, the prediction error is output to the bit depth adjustment unit 322. The method for determining whether or not to perform orthogonal transform is not particularly limited, but may be determined based on the characteristics of the image, the size of the transform coefficient after orthogonal transform, and the like. Further, the determined determination result is output to the integrated encoding unit 311 as conversion skip determination information. The transform unit 304 performs orthogonal transform on the prediction error input from the transform control unit 321 in transform block size units to generate orthogonal transform coefficients.

  On the other hand, the bit depth adjustment unit 322 performs bit depth adjustment on the prediction error input from the conversion control unit 321 in units of conversion blocks, and generates a prediction error that has been adjusted in bit depth. In the quantization unit 305, the quantization matrix stored in the quantization matrix holding unit 107 for the orthogonal transformation coefficient output from the conversion unit 304 or the prediction error adjusted in bit depth output from the bit depth adjustment unit 322. Is used to generate a quantized coefficient. The generated quantized coefficient is input to the coefficient encoding unit 106.

  The integrated encoding unit 311 encodes header information necessary for image encoding, and integrates the input quantization matrix header code and quantization matrix coefficient code. The integrated encoding unit 311 further encodes the conversion skip determination information input from the conversion control unit 321 in addition to the header information encoded by the integrated encoding unit 111 of FIG.

  FIG. 8 is a flowchart showing image coding processing in the image coding apparatus according to the third embodiment. In FIG. 8, the same numbers are assigned to portions that perform the same functions as those in FIG. 6 of the first embodiment, and description thereof is omitted.

  In step S820, the conversion control unit 321 determines whether or not to perform orthogonal transform on the prediction error generated in step S609, and generates conversion skip determination information. In step S821, based on the conversion skip determination information, if orthogonal transform is to be performed, the process proceeds to step S610, and if not, the process proceeds to step S822. In step S822, the bit depth adjustment unit 322 adjusts the bit depth of the prediction error generated in step S609 in units of transform blocks, and generates a prediction error that has been adjusted in bit depth. In step S811, the quantization unit 305 quantizes the orthogonal transformation coefficient generated in step S610 or the prediction error adjusted in bit depth generated in step S822 using the quantization matrix generated in step S601. Generate quantization coefficients. In step S823, the integrated encoding unit 311 encodes the conversion skip determination information generated in step S821.

  With the above-described configuration and operation, it is not necessary to send a redundant code when encoding a flat quantization matrix corresponding to conversion skip, particularly in steps S820 to S823. Can be configured.

  In this embodiment, the conversion skip determination information is encoded in units of conversion blocks. However, the unit in which the conversion skip determination information is encoded is not limited to this. The encoding may be performed in units of blocks, and a configuration that saves the code amount by using the conversion skip determination information common to each color component may be employed. The encoding method is not particularly limited, but a Huffman code or an arithmetic code can be used.

<Embodiment 4>
FIG. 4 is a block diagram showing an image decoding apparatus according to this embodiment. In FIG. 4, the same numbers are assigned to portions that perform the same functions as those in FIG. 2 of the second embodiment, and descriptions thereof are omitted. In the present embodiment, decoding of the bitstream generated in the third embodiment will be described.

  A decoding / separating unit 402 decodes header information of the input bitstream, separates necessary codes from the bitstream, and outputs them to the subsequent stage. The decoding / separating unit 402 performs the reverse operation of the integrated encoding unit 311 in FIG. 2 differs from the decoding / separating unit 202 in that the conversion skip determination information is further decoded / reproduced and output to the subsequent stage. Reference numeral 408 denotes an inverse quantization unit that performs inverse quantization on the quantization coefficient using the quantization matrix stored in the quantization matrix holding unit 206 and reproduces the inverse quantization coefficient. The inverse quantization coefficient in the present embodiment corresponds to the orthogonal transform coefficient in the third embodiment or the prediction error with the bit depth adjusted.

  Reference numeral 421 denotes an inverse transform control unit that determines whether to perform inverse orthogonal transform based on the transform skip determination information input from the decoding / separating unit 402 and outputs an inverse quantization coefficient to the subsequent stage. Reference numeral 409 denotes an inverse transform unit that performs inverse orthogonal transform that is the inverse of the transform unit 304 in FIG. 3 and reproduces a prediction error. Reference numeral 422 denotes a bit depth reverse adjustment unit that performs bit depth adjustment that is the reverse of the bit depth adjustment unit 322 of FIG. 3 and reproduces a prediction error. Here, the degree of bit depth reverse adjustment is the bit depth of the output image, bit depth adjustment accompanying inverse orthogonal transformation in the inverse transform unit 409, bit depth adjustment accompanying quantization processing in the inverse quantization unit 408, and quantization matrix It is determined by the coefficient value. For example, in this embodiment, bit depth reverse adjustment is performed by bit-shifting the coefficient to the right by the difference between 13 and the bit depth of the output image. That is, if the bit depth of the output image is 8 bits, the prediction error is bit-shifted to the right by 5 bits. However, the value used for bit depth reverse adjustment is not limited to this. A prediction reconstruction unit 410 reproduces a block image from the prediction error and the decoded image.

  An image decoding operation in the image decoding apparatus will be described below.

  In FIG. 4, the bit stream for one picture input from the terminal 201 is input to the decoding / separating unit 402, the header information necessary for reproducing the image is decoded, and the code used in the subsequent stage is further separated. Are output as appropriate. The quantization matrix header code included in the header information is output to the quantization matrix header decoding unit 203. Also, in the decoding / separating unit 402, when information indicating that it is necessary to decode the coefficient of the quantization matrix to be decoded is input from the quantization matrix header decoding unit 203, the quantization matrix coefficient decoding unit 204 Quantization matrix coefficient code data is output. Further, the decoding / separating unit 402 decodes / reproduces conversion skip determination information indicating whether or not to perform inverse orthogonal transform in units of transform blocks from the bit stream, and outputs the decoding skip determination information to the inverse transform control unit 421.

  In the inverse quantization unit 408, the quantization coefficient reproduced by the coefficient decoding unit 207 and the quantization matrix stored in the quantization matrix holding unit 206 are input. Then, inverse quantization is performed using the quantization matrix, the inverse quantization coefficient is reproduced, and output to the inverse transform control unit 421. Based on the transform skip determination information input from the decoding / separating unit 402, the inverse transform control unit 421 determines whether to perform inverse orthogonal transform on the input inverse quantization coefficient. Then, when performing inverse orthogonal transform, the inverse quantization coefficient is output to the inverse transform unit 409, and when not performed, the inverse quantization coefficient is output to the bit depth inverse adjustment unit 422. The inverse transform unit 209 performs inverse orthogonal transform that is the reverse of the transform unit 304 in FIG. 3 on the orthogonal transform coefficient that is the input inverse quantization coefficient, reproduces the prediction error, and outputs the prediction error to the prediction reconstruction unit 410. To do.

  On the other hand, the bit depth reverse adjustment unit 422 performs bit depth reverse adjustment on a transform block basis for the bit depth adjusted prediction error, which is the inverse quantization coefficient input from the inverse conversion control unit 421, and reproduces the prediction error. The result is output to the prediction reconstruction unit 410. The prediction reconstruction unit 410 performs prediction from the surrounding pixel data decoded on the input prediction error, reproduces an image in block units, and outputs it from the terminal 211.

FIG. 9 is a flowchart showing image decoding processing in the image decoding apparatus according to the fourth embodiment. The same numbers are assigned to portions that perform the same functions as those in FIG. 7 of the second embodiment, and descriptions thereof are omitted.
In step S921, the decoding / separating unit 402 decodes / reproduces conversion skip determination information indicating whether or not to perform inverse orthogonal transform in units of transform blocks from the bitstream. In step S922, determination based on the conversion skip determination information generated in step S921 is performed. If the inverse orthogonal transform is to be performed, the process proceeds to S709; otherwise, the process proceeds to S923. In step S923, the bit depth inverse adjustment unit 422 performs bit depth inverse adjustment of the inverse quantization coefficient generated in step S708 on a transform block basis to reproduce a prediction error.

  With the above configuration and operation, the image decoding apparatus having the conversion skip processing also decodes the flat quantization matrix expressed in the quantization matrix matching information generated in the third embodiment and represented by a small code amount. A reproduced image can be obtained.

  Further, in this embodiment, the conversion skip determination information exists for each conversion block. However, a configuration may be adopted in which a flag is included in the header portion and thereby the decoding apparatus switches whether to have conversion skip processing. . In that case, a configuration may be adopted in which a flat quantization matrix is stored and used as a reference object only when conversion skip processing is used, and is excluded from the reference object otherwise.

<Embodiment 5>
The above embodiments have been described on the assumption that the processing units shown in FIGS. 1, 2, 3, and 4 are configured by hardware. However, the processing performed by each processing unit shown in FIGS. 1, 2, 3, and 4 may be configured by a computer program.

  FIG. 12 is a block diagram illustrating a configuration example of computer hardware applicable to the image display device according to each of the embodiments.

  The CPU 1201 controls the entire computer using computer programs and data stored in the RAM 1202 and the ROM 1203, and executes each process described above as performed by the image processing apparatus according to each of the above embodiments. That is, the CPU 1201 functions as each processing unit shown in FIG. 1, FIG. 2, FIG. 3, and FIG.

  The RAM 1202 has an area for temporarily storing computer programs and data loaded from the external storage device 1206, data acquired from the outside via an I / F (interface) 1207, and the like. Further, the RAM 1202 has a work area used when the CPU 1201 executes various processes. That is, the RAM 1202 can be allocated as, for example, a picture memory or can provide various other areas as appropriate.

  The ROM 1203 stores setting data for the computer, a boot program, and the like. The operation unit 1204 is configured by a keyboard, a mouse, and the like, and can input various instructions to the CPU 1201 when operated by a user of the computer. The output unit 1205 outputs the processing result by the CPU 1201. The output unit 1205 can be configured by a display device such as a liquid crystal display to display the processing result.

  The external storage device 1206 is a large-capacity information storage device represented by a hard disk drive device. The external storage device 1206 stores an OS (Operating System) and computer programs for causing the CPU 1201 to realize the functions of the units illustrated in FIGS. 1, 2, 3, and 4. Furthermore, each image as a processing target may be stored in the external storage device 1206.

  Computer programs and data stored in the external storage device 1206 are appropriately loaded into the RAM 1202 under the control of the CPU 1201 and are processed by the CPU 1201. The I / F 1207 can be connected to a network such as a LAN or the Internet, and other devices such as a projection device and a display device. The computer can acquire and transmit various information via the I / F 1207. Can be. Reference numeral 1208 denotes a bus connecting the above-described units.

  The operation having the above-described configuration is controlled by the CPU 1201 as the operation described in the above flowchart.

<Other embodiments>
The object of the present invention can also be achieved by supplying a storage medium storing a computer program code for realizing the above-described functions to the system, and the system reading and executing the computer program code. In this case, the computer program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the computer program code constitutes the present invention. In addition, the operating system (OS) running on the computer performs part or all of the actual processing based on the code instruction of the program, and the above-described functions are realized by the processing. .

  Furthermore, you may implement | achieve with the following forms. That is, the computer program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. Then, based on the instruction of the code of the computer program, the above-described functions are realized by the CPU or the like provided in the function expansion card or function expansion unit performing part or all of the actual processing.

  When the present invention is applied to the above storage medium, the computer program code corresponding to the flowchart described above is stored in the storage medium.

Claims (7)

Acquisition means for acquiring, from a bitstream, determination information indicating whether or not the quantization coefficient and the quantization coefficient are orthogonally transformed coefficients;
An inverse quantization means for obtaining a quantization matrix from the bitstream and dequantizing the quantization coefficient based on the obtained quantization matrix regardless of the determination information;
When the determination information acquired by the acquisition unit indicates that the quantized coefficient is a coefficient obtained by orthogonal transform, an inverse orthogonal transform unit that performs inverse orthogonal transform on the coefficient inversely quantized by the inverse quantization unit; ,
An adjustment unit that adjusts the bit depth of the coefficient inversely quantized by the inverse quantization unit when the determination information acquired by the acquisition unit indicates that the quantization coefficient is a coefficient that has not been orthogonally transformed; ,
Decoding means for decoding an image from the data obtained by the inverse orthogonal transform means or the data obtained by the adjustment means;
An image decoding apparatus comprising: The obtaining means further obtains coincidence information indicating whether or not the orthogonally transformed quantization coefficient and the quantization coefficient not orthogonally transformed are quantized with the same quantization matrix,
The dequantization means relates to the determination information when the coincidence information indicates that the quantized coefficient obtained by orthogonal transform and the quantized coefficient not orthogonally transformed are quantized by the same quantization matrix. The image decoding apparatus according to claim 1, wherein the quantization coefficient is inversely quantized based on the acquired quantization matrix.   The data obtained by the inverse orthogonal transform unit or the data obtained by the adjustment unit is a prediction error which is a difference value between a pixel of a block to be referenced and a pixel of a block to be decoded. The image decoding device according to 1 or 2. An image decoding method,
An acquisition step of acquiring, from the bitstream, determination information indicating a quantization coefficient and whether the quantization coefficient is an orthogonally transformed coefficient;
An inverse quantization step of obtaining a quantization matrix from the bitstream and dequantizing the quantization coefficient based on the obtained quantization matrix regardless of the determination information;
When the determination information acquired in the acquisition step indicates that the quantized coefficient is a coefficient obtained by orthogonal transform, an inverse orthogonal transform step that performs inverse orthogonal transform on the coefficient dequantized in the inverse quantization step; ,
When the determination information acquired in the acquisition step indicates that the quantized coefficient is a coefficient that has not been orthogonally transformed, an adjustment step of adjusting the bit depth of the coefficient inversely quantized in the inverse quantization step; ,
A decoding step of decoding an image from the data obtained in the inverse orthogonal transform step or the data obtained in the adjustment step;
An image decoding method characterized by comprising: In the acquisition step, further, acquisition of coincidence information indicating whether or not the quantized coefficient that has been orthogonally transformed and the quantized coefficient that has not been orthogonally transformed have been quantized with the same quantization matrix,
In the inverse quantization step, when the coincidence information indicates that the quantized coefficient obtained by orthogonal transformation and the quantized coefficient not orthogonally transformed are quantized by the same quantization matrix, An image decoding method characterized by dequantizing the quantization coefficient based on the acquired quantization matrix.   The data obtained in the inverse orthogonal transform step or the data obtained in the adjustment step is a prediction error which is a difference value between a pixel of a block to be referenced and a pixel of a block to be decoded. Method.   A program that causes the computer to function as the image decoding device according to any one of claims 1 to 3 by being read and executed by the computer.

Images