JP2021150723A - Image encoding device, image encoding method and program, and image decoding device, image decoding method and program - Google Patents

Abstract

To provide an image encoding device and an image decoding device that can reduce a code amount of data for indicating a quantization matrix.SOLUTION: An image coding device 100 encodes information indicating whether or not a second quantization matrix corresponding to a first color difference component and a third quantization matrix corresponding to a second color difference component have been encoded into a bit stream, into the bit stream, and in a case where an image to be encoded has a luminescence component, the first color difference component, and the second color difference component, when the information indicates that the second quantization matrix and the third quantization matrix have not been encoded, quantifies a prediction error of a luminescence component of a block to be encoded, a prediction error of the first color difference component of the block to be encoded, and a prediction error of the second color difference component of the block to be encoded, using the first quantization matrix.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image coding technique.

As a coding method for compressed recording of moving images, a HEVC (High Efficiency Video Coding) coding method (hereinafter referred to as HEVC) is known. In HEVC, in order to improve coding efficiency, a basic block having a size larger than that of a conventional macroblock (16 × 16 pixels) has been adopted. This large-sized basic block is called a CTU (Coding Tree Unit), and its size is up to 64 × 64 pixels. The CTU is further divided into sub-blocks that serve as units for prediction and conversion.

Further, in HEVC, the quantization process is performed on the coefficient after the orthogonal transformation is performed (hereinafter, referred to as the orthogonal transformation coefficient). In this quantization process, the quantization step of the quantization process in each frequency component is determined by weighting according to the frequency component using the quantization matrix. As a result, it is possible to improve the compression efficiency while maintaining the image quality by further reducing the data of the high frequency component whose deterioration is inconspicuous to human vision. Patent Document 1 discloses a technique for encoding such a quantization matrix.

In recent years, as a successor to HEVC, activities to carry out international standardization of more efficient coding methods have been started. JVET (Joint Video Experts Team) was established by ISO / IEC and ITU-T, and standardization is being promoted as a VVC (Versatile Video Coding) coding method (hereinafter, VVC). In order to improve coding efficiency, the maximum size of the basic block in VVC is 128 x 128 pixels, and in addition to the conventional method of dividing into square subblocks, a method of dividing into rectangular subblocks is also considered. Has been done.

Japanese Unexamined Patent Publication No. 2013-38758

In VVC as well, the introduction of a quantization matrix is being studied as in HEVC. Further, in VVC, more kinds of subblock division methods than HEVC, including a rectangular shape, are being studied. Since the distribution of the orthogonal transformation coefficients corresponding to each subblock division differs depending on the size and shape of the orthogonal transformation, it is desirable to define the optimum quantization matrix according to the size and shape of the subblocks. However, if the quantization matrix is defined individually for each of the shapes of all the sub-blocks, the code amount of the data indicating the quantization matrix will be unnecessarily increased.

The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to suppress the code amount of data showing a quantization matrix.

In order to solve the above-mentioned problems, the image coding apparatus of the present invention has the following configuration. That is, it is an image coding device capable of encoding an image having a brightness component, a first color difference component, and a second color difference component into a bit stream, and divides the image to be encoded into a plurality of blocks. The dividing means to be used, the prediction means for performing prediction processing on the block to be encoded and deriving the prediction error of the block to be encoded, and the conversion means for orthogonally converting the prediction error to derive the orthogonal conversion coefficient. The coding means has a quantization means for quantizing the orthogonal conversion coefficient using at least the first quantization matrix, and a coding means for encoding at least the first quantization matrix. Provides information indicating whether the second quantization matrix corresponding to the first color difference component and the third quantization matrix corresponding to the second color difference component are encoded in the bit stream. The second quantization matrix is encoded in the bit stream, and the quantization means has the brightness component, the first color difference component, and the second color difference component in the image to be encoded. When the information indicates that the third quantization matrix and the third quantization matrix are encoded in the bit stream, the prediction error of the brightness component of the coded block is determined by using the first quantization matrix. The prediction error of the first color difference component of the block to be encoded is quantized using the second quantization matrix, and the prediction error of the second color difference component of the block to be encoded is quantized. Is quantized using the third quantization matrix, and when the image to be encoded has the brightness component, the first color difference component, and the second color difference component, the second quantization matrix is used. When the information indicates that the third quantization matrix and the third quantization matrix are not encoded in the bit stream, the prediction error of the brightness component of the coded block and the coded block of the coded block. The prediction error of the first color difference component and the prediction error of the second color difference component of the block to be encoded are quantized using the first quantization matrix.

In addition, the image decoding device of the present invention has the following configuration. That is, it is an image decoding device capable of decoding a bit stream in which an image is encoded and having a brightness component, a first color difference component, and a second color difference component, and is at least the first. Using the decoding means for decoding the quantization matrix of 1 from the bit stream and at least the first quantization matrix, the residual coefficient of the block to be decoded in the image to be decoded is inversely quantized to obtain the orthogonal conversion coefficient. An inverse quantization means to be derived, an inverse conversion means to derive a prediction error by inversely orthogonalizing the orthogonal conversion coefficient, and a generation that performs prediction processing on the prediction error to generate an image of the block to be decoded. The decoding means has means, and the decoding means has a second quantization matrix corresponding to the first color difference component and a third quantization matrix corresponding to the second color difference component in the bit stream. When the information indicating whether the code is encoded is decoded from the bit stream, and the inverse quantization means has the brightness component, the first color difference component, and the second color difference component in the bit stream to be decoded. In, when the information indicates that the second quantization matrix and the third quantization matrix are encoded in the bit stream, the residual coefficient of the brightness component of the block to be decoded is calculated. The block to be decoded is dequantized using the first quantization matrix, and the residual coefficient of the first color difference component of the block to be decoded is dequantized using the second quantization matrix. The residual coefficient of the second color difference component is inversely quantized using the third quantization matrix, and the brightness component, the first color difference component, and the second color difference component are converted into the bit stream to be decoded. When the information indicates that the second quantization matrix and the third quantization matrix are not encoded in the bit stream, the brightness component of the block to be decoded A first quantization matrix is obtained by combining the residual coefficient, the residual coefficient of the first color difference component of the block to be decoded, and the residual coefficient of the second color difference component of the block to be decoded. Is dequantized using.

According to the present invention, it is possible to suppress the code amount of the data indicating the quantization matrix.

It is a block diagram which shows the structure of the image coding apparatus in embodiment. It is a block diagram of the image decoding apparatus in embodiment. It is a flowchart which shows the image coding process in the image coding apparatus in embodiment. It is a flowchart which shows the image decoding processing in the image decoding apparatus in embodiment. It is a block diagram in the case where the embodiment is realized by hardware. It is a figure which shows the example of the bit stream structure generated by the coding apparatus in embodiment, and is decoded by the image decoding apparatus. It is a figure which shows the example of the sub-block division in an embodiment. It is a figure which shows the example of the quantization matrix used in embodiment. It is a figure which shows the scanning method of the element of the quantization matrix used in embodiment. It is a figure which shows the matrix of the difference value of the quantization matrix generated in embodiment. It is a figure which shows the example of the coding table used for coding the difference value of a quantization matrix. It is a figure which shows the example of the syntax table of the quantization matrix used in embodiment. It is a figure which shows the example of another quantization matrix used in embodiment.

Embodiments of the present invention will be described with reference to the accompanying drawings. The configuration shown in the following embodiments is an example, and the present invention is not limited to the configurations described in the following embodiments. It should be noted that the names such as basic block and sub-block are names used for convenience in each embodiment, and other names may be used as appropriate as long as their meanings do not change. For example, a basic block or subunit may be referred to as a basic unit or subunit, or may simply be referred to as a block or unit. Further, as an example, the basic block may be a block of 128 × 128 pixels, or a block of 64 × 64 pixels or 32 × 32 pixels may be a basic block. Further, a smaller block may be used as a basic block. The basic block is, for example, a coding tree unit or a coding unit. The basic block may be a unit that can be further subdivided into subblocks.

Further, when the term "Cb / Cr" is used in each embodiment, it means that it is at least one of the Cb component and the Cr component. That is, "Cb / Cr" may be both a Cb component and a Cr component, or may be only one of the Cb component and the Cr component.

Further, although a plurality of configurations are described in each embodiment, not all of these plurality of configurations are indispensable. Further, these plurality of configurations may be arbitrarily combined.

Further, in the attached drawings, the same or similar configurations are designated by the same reference numerals, and the description of overlapping portions will be omitted.

<Embodiment>
Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of the image coding device 100 of the present embodiment. The image coding device 100 has a control unit 150 that controls the entire image coding device 100. Further, the image coding device 100 includes an input terminal 101, a block division unit 102, a quantization matrix holding unit 103, a prediction unit 104, a conversion / quantization unit 105, and an inverse quantization / inverse conversion unit 106. Further, the image coding device 100 includes an image reproduction unit 107, a frame memory 108, an in-loop filter unit 109, a coding unit 110, an integrated coding unit 111, an output terminal 112, and a quantization matrix coding unit 113. .. The coding unit 110, the integrated coding unit 111, and the quantization matrix coding unit 113 cooperate with each other to function as a coding means for coding each data into a bit stream.

The input terminal 101 inputs the image data to be encoded in frame units. The image data is acquired from an imaging device, a file server or a storage medium that stores the image data to be encoded, but the type thereof does not matter. Further, the output terminal 112 outputs the coded data to the output destination device, and the output destination device is not particularly limited to a storage medium, a file server, or the like.

The block division unit 102 divides the image of the input frame (picture) into a plurality of basic blocks, and outputs one of the basic blocks to the prediction unit 104 in the subsequent stage in order.

The quantization matrix holding unit 103 generates a plurality of quantization matrices prior to coding and holds them in an internal memory (not shown). The method of generating the quantization matrix is not particularly limited. The user may input data indicating the quantization matrix, or the image coding apparatus 100 may calculate the quantization matrix from the characteristics of the input image. Moreover, you may use the quantization matrix specified in advance as an initial value. The quantization matrix holding unit 103 in the present embodiment is a two-dimensional quantization corresponding to the orthogonal transformation of 8 × 8 pixels, 4 × 4 pixels, or 2 × 2 pixels shown in FIGS. 8 (a) to 8 (e). Hold the matrix 800-804.

Here, the quantization matrix is for weighting the quantization with respect to the orthogonal transformation coefficient according to the frequency component. The quantization step for each orthogonal transformation coefficient in the quantization described later is weighted by multiplying the reference parameter value (quantization parameter) by the value of each element in the quantization matrix.

The prediction unit 104 determines the sub-block division for the image data in the basic block unit. That is, it is determined whether or not to divide the basic block into subblocks, and if so, how to divide it. If not divided into subblocks, the subblock will be the same size as the basic block. The subblock may be a square or a rectangle other than a square.

Then, the prediction unit 104 performs intra-frame prediction, which is intra-frame prediction, inter-frame prediction, which is inter-frame prediction, and the like in the divided sub-block units, and generates prediction image data for each sub-block unit. For example, the prediction unit 104 selects a prediction method for a certain subblock from intra-prediction, inter-prediction, and prediction that combines intra-prediction and inter-prediction, and performs the selected prediction. Generate predicted image data for subblocks.

Further, the prediction unit 104 calculates and outputs a prediction error in pixel units from the input image data and the predicted image data in subblock units. For example, the prediction unit 104 calculates the difference between each pixel value of the image data of the subblock and each pixel value of the predicted image data generated by the prediction for the subblock, and calculates it as a prediction error.

Further, the prediction unit 104 provides information necessary for prediction, for example, information indicating subblock division (division state from basic block to subblock) (information indicating how the basic block is divided into subblocks). Output together with the prediction error. Further, the prediction unit 104 also outputs information such as a prediction mode and a motion vector used in the prediction of the sub-block together with the prediction error. Hereinafter, the information necessary for this prediction will be referred to as prediction information.

The conversion / quantization unit 105 orthogonally transforms the prediction error input from the prediction unit 104 in subblock units to obtain an orthogonal transformation coefficient representing each frequency component of the prediction error. Further, the conversion / quantization unit 105 uses the quantization matrix stored in the quantization matrix holding unit 103 and the quantization parameters to quantize the orthogonal transformation coefficient, and the residual coefficient (after quantization). Orthogonal transformation factor). The function of performing orthogonal transformation and the function of performing quantization may be configured separately.

The inverse quantization / inverse conversion unit 106 inputs a residual coefficient from the conversion / quantization unit 105, and inversely quantizes using the quantization matrix stored in the quantization matrix holding unit 103 and the quantization parameter. , Reproduce the orthogonal conversion coefficient. The inverse quantization / inverse transformation unit 106 further performs inverse orthogonal transformation of the orthogonal transformation coefficient to reproduce the prediction error. In this way, the process of reproducing (deriving) the orthogonal transformation coefficient using the quantization matrix and the quantization parameter is referred to as inverse quantization. The function of performing inverse quantization and the function of performing inverse orthogonal transformation may be configured separately. In addition, the information for the image decoding apparatus to derive the quantization parameter is also encoded in the bit stream by the coding unit 110.

The image reproduction unit 107 generates the prediction image data by appropriately referring to the frame memory 108 based on the prediction information output from the prediction unit 104. The image reproduction unit 107 adds the prediction error input from the inverse quantization / inverse conversion unit 106 to the predicted image data to generate the reproduced image data (reconcluded picture) and stores it in the frame memory 108.

The in-loop filter 109 performs in-loop filter processing such as a deblocking filter and a sample adaptive offset on the reproduced image stored in the frame memory 108, and stores the filtered image data in the frame memory 108 again.

The coding unit 110 encodes the residual coefficient output from the conversion / quantization unit 105 and the prediction information output from the prediction unit 104, generates code data, and outputs the code data to the integrated coding unit 111.

The quantization matrix coding unit 113 encodes the quantization matrix held in the quantization matrix holding unit 103, generates the quantization matrix code data, and outputs the quantization matrix code data to the integrated coding unit 111.

The integrated coding unit 111 generates header code data including the quantization matrix code data from the quantization matrix coding unit 113. Then, the integrated coding unit 111 causes the header code data to follow the code data output from the coding unit 110 to form a bit stream. Then, the integrated coding unit 111 outputs the formed bit stream via the output terminal 112.

Here, the image coding operation in the image coding apparatus 100 will be described in more detail below. In the present embodiment, the moving image data in the 4: 2: 0 color format is input in frame units from the input terminal 101, but the still image data for one frame may be input. Further, for the sake of explanation in the present embodiment, the block division unit 101 will be described as dividing the image data input from the input terminal into basic blocks of 8 × 8 pixels. That is, in the basic block of 8 × 8 pixels, the pixels of the luminance (Y) component of 8 × 8 pixels and the color difference component of 4 × 4 pixels (including the first color difference component Cb and the second color difference component Cr) are included. Will be included. It should be noted that this is for facilitating understanding, and is not limited to the above numerical values (size). Further, the basic block referred to here is, for example, a coding tree unit. As an example, an example in which the size of the coding tree unit is 8 × 8 pixels will be described, but other sizes may be used. For example, the size may be any one of 32 × 32 pixels to 128 × 128 pixels.

Prior to image coding, a quantization matrix is generated.

The quantization matrix holding unit 103 first generates and holds a quantization matrix. Specifically, a quantization matrix is generated according to the size of the encoded subblock and the type of prediction method. In the present embodiment, the quantization matrix corresponding to the 8 × 8 pixel basic block which is not divided into the sub-blocks shown in FIG. 7 (a) and the basic block shown in FIG. 7 (b) are divided into quadtrees. A quantization matrix corresponding to a subblock of 4 × 4 pixels is generated. That is, the quantization matrix holding unit 103 includes an 8 × 8 quantization matrix and a 4 × 4 quantization matrix for the brightness (Y) component, a 4 × 4 quantization matrix for the color difference (Cb and Cr) components, and the like. Generate a 2x2 quantization matrix. However, the generated quantization matrix is not limited to this, and a quantization matrix corresponding to the shape of the subblock, such as 4 × 8 or 8 × 4, may be generated. The method for determining each element (value for weighting) constituting the quantization matrix is not particularly limited. For example, a predetermined initial value may be used or may be set individually. Further, it may be generated according to the characteristics of the image.

The quantization matrix holding unit 103 holds the quantization matrix thus generated in an internal memory (not shown). FIG. 8A shows an 8 × 8 quantization matrix 800 for the Y component using intra-prediction. Further, FIG. 8B shows an 8 × 8 quantization matrix 801 for the Y component using inter-prediction. FIG. 8 (c) shows a 4 × 4 quantization matrix 802 for the Y / Cb / Cr component using intra-prediction. Similarly, FIG. 8 (d) shows a 4 × 4 quantization matrix 803 for the Y / Cb / Cr component using inter-prediction. Further, FIG. 8E shows a 2 × 2 quantization matrix 804 for the Cb / Cr component, which also uses inter-prediction.

That is, in the present embodiment, when the subblock sizes are the same, the same quantization matrix is used regardless of the color components such as Y, Cb, and Cr. In the present embodiment, when the intra prediction is used in the basic block of 8 × 8 pixels, the brightness component is divided into the subblocks of 4 × 4 pixels even if the quadtree division shown in FIG. 7 (b) is performed. Although it is divided, it is assumed that the color difference component is not divided into 2 × 2 pixel subblocks. That is, in this case, since the color difference component is processed in units of 4 × 4 pixels, a 4 × 4 quantization matrix 802 is used. For the sake of simplicity, the configuration is 8 × 8 64 pixels, 4 × 4 16 pixels, and 2 × 2 4 pixels, and each square in the thick frame constitutes the quantization matrix. It shall represent an element. In the present embodiment, it is assumed that the five types of quantization matrices shown in FIGS. 8A to 8E are held in a two-dimensional shape, but each element in the quantization matrix is of course included in this. Not limited. It is also possible to hold multiple quantization matrices for the same color component and the same prediction mode, depending on the size of the subblock.

In general, in order to realize the quantization process according to the human visual characteristics, the quantization matrix is an element of a low frequency portion corresponding to the upper left portion of the quantization matrix as shown in FIGS. Is small, and the element of the high frequency part corresponding to the lower right part is large. As the value of the element increases, the quantization step also increases and the amount of code is suppressed.

The quantization matrix coding unit 113 reads out the two-dimensional shape quantization matrix held in the quantization matrix holding unit 106 in order, scans each element, calculates the difference value between the elements, and calculates the difference value between the elements. Encode the difference information that is a set.

FIG. 12 is a syntax table used for coding the quantization matrix in the present embodiment. The image coding device 100 encodes at least the syntax shown in this syntax table. Using this syntax table, the coding process of each quantization matrix shown in FIGS. 8A to 8E in the present embodiment will be specifically described. However, FIG. 12 is a table used for coding 10 types of quantization matrices corresponding to 2 × 2 to 8 × 8 subblocks in the present embodiment, and if the types of subblock sizes increase, the coding will occur. The types of quantization matrix to be performed also increase.

First, the structure of the syntax table of FIG. 12 will be described. First, scaling_matrix_for_lfnst_disabled_flag is a flag indicating the following. That is, it is a flag indicating whether to perform quantization using a quantization matrix for a subblock that is further subjected to a secondary orthogonal transformation to the orthogonal transformation coefficient generated by subjecting the prediction error to the orthogonal transform. In this embodiment, since the secondary orthogonal transformation is not used, there is no influence by scaling_matrix_for_lfnst_disclosed_flag, but 0 is encoded.

Next, scaling_list_chroma_present_flag is a flag indicating whether or not to encode the quantization matrix (chroma_scaling_lists) corresponding to the color difference component. When this flag is 1, the quantization matrix corresponding to the color difference component is encoded, while when this flag is 0, the coding of the quantization matrix corresponding to the color difference component is omitted.

Even when the image data to be encoded has only the Y component (monochrome), the coding unit 110 may encode the scaling_list_chroma_present_flag into a bit stream. That is, the coding unit 110 may encode the scaling_list_chroma_present_flag into a bitstream regardless of whether it is monochrome or not. Even when the image data has only the Y component, encoding the scaling_list_chroma_present_flag into a bit stream has the following effects. That is, the contents of the quantization process and the coding process of the quantization matrix can be determined based on scaling_list_chroma_present_flag, regardless of whether the image data is monochrome or not. Therefore, it is possible to prevent the processing from becoming complicated.

In the present embodiment, the 4 × 4 quantization matrix 802 for the Cb / Cr component using the intra prediction is the same as the 4 × 4 quantization matrix 802 for the Y component that also uses the intra prediction. Similarly, the 4 × 4 quantization matrix 803 for the Cb / Cr component using the inter-prediction is the same as the 4 × 4 quantization matrix 803 for the Y component also using the inter-prediction. Furthermore, the 2x2 quantization matrix 804 for the Cb / Cr component using the inter-prediction is each element of the even-numbered rows and even-columns of the 4x4 quantization matrix 803 for the Y component using the inter-prediction. It can be generated by extracting 10, 20, 20, 27).

In other words, in the present embodiment, the quantization matrix for all the color difference components can be generated by using the quantization matrix for the luminance component. Therefore, in the present embodiment, instead of directly encoding the data indicating the quantization matrix for each color difference component into a bit stream, 0 can be encoded as scaling_list_chroma_present_flag. Thereby, the coding of the quantization matrix for the color difference component can be omitted.

On the other hand, unlike the present embodiment, when at least one quantization matrix for the color difference component cannot be generated from the quantization matrix for the luminance component, 1 is encoded as scaling_list_chroma_present_flag. Then, the quantization matrix for the color difference component is individually encoded. As described above, when the image data to be encoded has a Y component, a Cb component, and a Cr component, and the scaling_list_chroma_present_flag is 1, the result is as follows. That is, the quantization matrix coding unit 113 encodes the quantization matrix for the Y component and the quantization matrix for the Cb / Cr component. Then, the conversion / quantization unit 105 quantizes the Y component using the quantization matrix for the Y component. Then, the conversion / quantization unit 105 quantizes the Cb / Cr component using the quantization matrix for the Cb / Cr component.

Next, in the syntax table of FIG. 12, there is a for loop with a parameter called id. id indicates the size and type of the quantization matrix, and the quantization matrix of the same size is continuously encoded.

Specifically, id = 0 corresponds to a 2 × 2 quantization matrix for the Cb component using inter-prediction, and id = 1 corresponds to a 2 × 2 quantization matrix for the Cr component using inter-prediction. id = 2 corresponds to a 4 × 4 quantization matrix for the Y component using intra-prediction. id = 3 corresponds to a 4 × 4 quantization matrix for the Cb component, also using intra-prediction. id = 4 corresponds to a 4 × 4 quantization matrix for the Cr component using intra-prediction.

Similarly, id = 5 corresponds to a 4 × 4 quantization matrix for the Y component using inter-prediction. id = 6 corresponds to a 4 × 4 quantization matrix for the Cb component using inter-prediction. id = 7 corresponds to a 4 × 4 quantization matrix for the Cr component using inter-prediction.

Furthermore, id = 8 corresponds to an 8 × 8 quantization matrix for the Y component using intra-prediction. id = 9 corresponds to an 8 × 8 quantization matrix for the Y component using inter-prediction.

Therefore, in the present embodiment, it means that the 2 × 2 quantization matrix group is encoded, and then the 4 × 4 quantization matrix group and the 8 × 8 quantization matrix group are encoded in this order. ing. In VVC, orthogonal transformation with a minimum magnitude of 2 × 2 to 64 × 64 is used, and it is also possible to encode a quantization matrix corresponding to each. However, in this embodiment, since the maximum orthogonal transformation size, that is, the size of the subblock is 8 × 8, the upper limit of the for loop corresponds to the 8 × 8 quantization matrix for the Y component using inter-prediction. Id = 9. In this way, by setting the upper limit of the for loop based on the maximum value of the orthogonal transform size actually used, unnecessary coding of the quantization matrix can be omitted and generation of redundant codes can be prevented. Can be done.

Subsequently, the coding procedure of each quantization matrix will be described. First, the quantization matrix coding unit 113 makes a determination based on scaling_list_chroma_present_flag and id. When the remainder obtained by dividing the id by 3 is 2, or when the id is 9, since it corresponds to the quantization matrix for any Y component, the coding for the target Y component is performed. Move to processing. On the other hand, when id is other than that, the object to be encoded is the quantization matrix for the Cb or Cr component. Therefore, only when the scalaring_list_chroma_present_flag is 1, the quantization matrix for the target color difference component is encoded. ..

In this embodiment, since scaling_list_chroma_present_flag is 0, only the quantization matrix for the Y component corresponding to id = 2, 5, 8 and 9 is encoded. As described above, when the image data to be encoded has a Y component, a Cb component, and a Cr component, and the scaling_list_chroma_present_flag is 0, the result is as follows. That is, the quantization matrix coding unit 113 encodes only the quantization matrix for the Y component, and does not encode the quantization matrix for the Cb / Cr component. Then, the conversion / quantization unit 105 quantizes the Cb / Cr component using the quantization matrix for the Y component.

In the coding of each quantization matrix, in the syntax table of FIG. 12, first, scaling_list_copy_mode_flag is encoded. If the quantization matrix to be encoded is the same as the encoded quantization matrix, 1 is encoded as this value, and 0 is encoded otherwise. In the present embodiment, the scalaring_list_copy_mode_flag is 0 because not all the elements are the same in each of the quantization matrices shown in FIGS. 8A to 8E to be encoded.

Subsequently, when scaling_list_copy_mode_flag is 0, scaling_list_pred_mode_flag indicating whether to use the predicted difference coding with each element of the encoded quantization matrix is encoded. When predictive difference coding is used, scaling_list_pred_mode_flag is 1, otherwise it is 0. In this embodiment, since the prediction difference coding is not used for coding each quantization matrix, 0 is coded as scaling_list_pred_mode_flag.

Next, when the scaling_list_copy_mode_flag or the scaling_list_pred_mode_flag is 1, the scaling_list_pred_id_delta is encoded. scaling_list_pred_id_delta means the identifier of the coded quantization matrix to be referenced. In this embodiment, neither the scaling_list_copy_mode_flag nor the scaling_list_pred_mode_flag is 0, so the scaling_list_pred_id_delta is not encoded.

Next, we move on to coding the difference of each element of the quantization matrix to be coded. The quantization matrix coding unit 113 scans each element of the quantization matrix to be coded using any of FIGS. 9A to 9C according to the size of the quantization matrix to obtain a difference. Calculate and place in a one-dimensional matrix. Then, the quantization matrix coding unit 113 encodes each difference value arranged in the one-dimensional matrix as the difference information. This corresponds to scaling_list_delta_coef in the syntax table of FIG.

For example, in the coding of the quantization matrix 802 of FIG. 8 (c), the quantization matrix coding unit 113 scans the quantization matrix 802 of FIG. 8 (c) using FIG. 9 (b). For each element, the difference from the immediately preceding element is sequentially calculated in the scanning order. The quantization matrix coding unit 113 arranges the calculated difference in a one-dimensional matrix, and obtains the one-dimensional difference value matrix shown in FIG.

Here, for example, the quantization matrix for 4 × 4 pixels in FIG. 8 (c) is scanned by the scanning method shown in FIG. 9 (b), but immediately after the first element 6 located in the upper left. The element 13 located below is scanned, and the difference +7 is calculated. Further, in the coding of the first element of the quantization matrix (6 in the present embodiment), the difference from a predetermined initial value (for example, 8) is calculated, but of course, the difference is not limited to this, and is arbitrary. The difference from the value or the value of the first element itself may be used. In short, it suffices if the initial value is the same as that of the image decoding device 200. Then, the quantization matrix coding unit 113 encodes each difference value in FIG. 10.

The quantization matrix coding unit 113 generates the quantization matrix code data as the coding result of each quantization matrix generated as described above. The quantization matrix coding unit 113 of the present embodiment encodes and quantizes each element of the one-dimensional difference matrix by assigning a code word using the coding table shown in FIG. 11 (a). Generate matrix code data. The coding table is not limited to this, and for example, the coding table shown in FIG. 11B may be used. In this way, the quantization matrix coding unit 113 outputs the generated quantization matrix code data to the integrated coding unit 111 in the subsequent stage.

Returning to FIG. 1, the integrated coding unit 111 integrates the code data of the quantization matrix into the header information required for encoding the image data.

Subsequently, the image data is encoded. The image data for one frame input from the input terminal 101 is supplied to the block division unit 102.

The block division unit 102 divides the input one-frame image data into a plurality of basic blocks, and outputs the image data for each basic block to the prediction unit 104. In the present embodiment, image data in basic block units of 8 × 8 pixels is supplied to the prediction unit 104.

The prediction unit 104 executes a prediction process on the image data of the basic block unit input from the block division unit 102. Specifically, the subblock division state when the basic block is divided into finer subblocks is determined as appropriate, and the prediction mode such as intra prediction or inter prediction is further determined for each subblock.

In intra-prediction, the predicted pixels of the coded block are generated using the coded pixels located in the spatial periphery of the coded block, and the prediction pixels of the coded block are used among the intra-prediction methods such as horizontal prediction, vertical prediction, and DC prediction. It also generates an intra-prediction mode that shows the intra-prediction method. In the inter-prediction, the predicted pixels of the coded block are generated using the encoded pixels of the frame that is different in time from the coded block, and the motion information indicating the frame to be referred to, the motion vector, and the like is also generated.

The sub-block division method will be described with reference to FIG. 7. The thick frames of the blocks 700 to 705 of FIGS. 7A to 7F have the same size of 8 × 8 pixels as the basic block. Each rectangle in the thick frame represents a subblock. FIG. 7B shows an example of the conventional square subblock division, and the 8 × 8 pixel basic block 701 is divided into four 4 × 4 pixel subblocks.

On the other hand, FIGS. 7 (c) to 7 (f) show an example of rectangular sub-block division. FIG. 7C shows that the basic block 702 is divided into two subblocks (longitudinal in the vertical direction) having a size of 4 × 8 pixels. FIG. 7D shows that the basic block 703 is divided into two subblocks (horizontally longitudinal) having a size of 8 × 4 pixels. 7 (e) and 7 (f) show that the basic blocks 704 and 705 are divided into three rectangular sub-blocks at a ratio of 1: 2: 1, although the division method is different. In this way, not only the square but also the rectangular sub-block is used for the coding process.

In the present embodiment, either FIG. 7 (a) in which the basic block having an 8 × 8 pixel size is not divided into sub-blocks or FIG. 7 (b) in which the basic block having a size of 8 × 8 pixels is divided into sub-blocks is used. The subblock division method is not limited to this. A ternary tree division as shown in FIGS. 7 (e) and 7 (f) or a binary tree division as shown in FIGS. 7 (c) and 7 (d) may be used. When the sub-block partitioning other than those shown in FIGS. 7 (a) and 7 (b) is also used, the quantization matrix corresponding to the sub-block used in the quantization matrix holding unit 103 is generated. Further, the generated quantization matrix will be encoded by the quantization matrix coding unit 113.

The prediction unit 104 generates prediction image data from the determined prediction mode and the encoded area stored in the frame memory 108, and further predicts an error in pixel units from the input image data and the prediction image data. Is calculated, and the error is output to the conversion / quantization unit 105. Further, the prediction unit 104 outputs information such as subblock division and prediction mode as prediction information to the coding unit 110 and the image reproduction unit 107.

The conversion / quantization unit 105 performs orthogonal transformation / quantization on the prediction error input from the prediction unit 104 to generate a residual coefficient. Specifically, the conversion / quantization unit 105 first performs an orthogonal transformation process corresponding to the size of the subblock on the prediction error to generate an orthogonal transformation coefficient. Then, the conversion / quantization unit 105 quantizes the orthogonal transformation coefficient using the quantization matrix stored in the quantization matrix holding unit 103 according to the prediction mode and the color component, and the quantization parameter, and the residual Generate a coefficient. In the present embodiment, when the intra prediction mode is used without subblock division and the size of the basic block is used, the orthogonal transformation coefficient of the Y component is shown in FIG. 8A, and the orthogonal transformation of the Cb / Cr component is performed. It is assumed that the quantization matrix of FIG. 8 (c) is used for the coefficient. Similarly, when the inter-prediction mode is used instead of the size of the basic block without subblock division, the orthogonal transformation coefficient of the Y component is shown in FIG. 8 (b), and the orthogonality of the Cb / Cr components is orthogonal. It is assumed that the quantization matrix shown in FIG. 8D is used for the conversion coefficient. On the other hand, when the sub-block division of FIG. 7 (b) is performed and the inter-prediction mode is used, the orthogonal transformation coefficient of the Y component is shown in FIG. 8 (d), and the orthogonal transformation coefficient of the Cb / Cr component is shown in FIG. 8 (e). ) Quantization matrix shall be used. However, the quantization matrix used is not limited to this. The generated residual coefficient and color difference integrated information are output to the coding unit 110 and the inverse quantization / inverse conversion unit 106.

The inverse quantization / inverse conversion unit 106 inversely quantizes the residual coefficient input from the conversion / quantization unit 105 by using the quantization matrix stored in the quantization matrix holding unit 103 and the quantization parameter. By doing so, the orthogonal conversion coefficient is reproduced. The inverse quantization / inverse transformation unit 106 further performs inverse orthogonal transformation of the reproduced orthogonal transformation coefficient to reproduce the prediction error.

Similar to the conversion / quantization unit 105, the inverse quantization process uses a quantization matrix corresponding to the size and color component of the subblock to be encoded. Specifically, the inverse quantization / inverse conversion unit 106 performs inverse quantization using the same quantization matrix used in the conversion / quantization unit 105. That is, when the intra prediction mode is used without subblock division, the quantization matrix of FIG. 8 (a) is used for the orthogonal transformation coefficient of the Y component, and FIG. 8 (c) is used for the orthogonal transformation coefficient of the Cb / Cr component. Is used. Similarly, when the inter-prediction mode is used instead of subblock division, the orthogonal transformation coefficient of the Y component is shown in FIG. 8 (b), and the orthogonal transform coefficient of the Cb / Cr component is shown in FIG. 8 (d). Quantization matrix is used.

On the other hand, when the sub-block division of FIG. 7 (b) is performed and the inter-prediction mode is used, the orthogonal transformation coefficient of the Y component is shown in FIG. 8 (d), and the orthogonal transformation coefficient of the Cb / Cr component is shown in FIG. 8 (e). ) Quantization matrix is used.

The prediction error reproduced by applying the inverse orthogonal transformation to the orthogonal transformation coefficient reproduced in this way is output to the image reproduction unit 107.

The image reproduction unit 107 appropriately refers to the frame memory 108 based on the prediction information input from the prediction unit 104, and reproduces the predicted image. Then, the image reproduction unit 107 generates the reproduced image data based on the reproduced predicted image and the prediction error reproduced by the inverse quantization / inverse conversion unit 106, and stores the reproduced image data in the frame memory 108.

The in-loop filter unit 109 reads the reproduced image data from the frame memory 108 and performs in-loop filter processing such as a deblocking filter. Then, the in-loop filter unit 109 re-stores the filtered image data in the frame memory 108.

The coding unit 110 entropy-codes the residual coefficient and color difference integrated information for each subblock generated by the conversion / quantization unit 105, and the prediction information input from the prediction unit 104 to generate code data. The method of entropy coding is not particularly specified, but Golomb coding, arithmetic coding, Huffman coding, etc. can be used. The coding unit 110 outputs the generated code data to the integrated coding unit 111.

The integrated coding unit 111 forms a bit stream by multiplexing the code data input from the coding unit 110 together with the code data of the header described above. Then, the integrated coding unit 111 outputs the formed bit stream from the output terminal 112 to the outside (storage medium, network, etc.).

FIG. 6A is an example of the data structure of the bitstream output in the present embodiment. The sequence header contains the code data of the quantization matrix, and is composed of the coded results of each quantization matrix. However, the encoded position is not limited to this, and of course, a configuration in which the image header portion as shown in FIG. 6B or the header portion spanning a plurality of pictures may be encoded may be used. Further, when the quantization matrix is changed in one sequence, it can be updated by newly encoding the quantization matrix. At this time, all the quantization matrices may be rewritten, or some of them may be changed by specifying the size, prediction mode, and color component of the quantization matrix corresponding to the rewritten quantization matrix. Is.

FIG. 3 is a flowchart showing the coding process for one frame of the control unit 150 in the image coding device 100 of the embodiment.

First, prior to image coding, in S301, the control unit 150 controls the quantization matrix holding unit 103 to generate and hold a two-dimensional quantization matrix. The quantization matrix holding unit 103 of the present embodiment corresponds to a block of 8 × 8 to 2 × 2 pixels, and quantizes corresponding to each color component and prediction mode shown in FIGS. 8 (a) to 8 (e). It will generate and hold a matrix.

In S302, the control unit 150 controls the quantization matrix coding unit 113 to encode the quantization matrix generated and held in S301. The specific operation of the quantization matrix coding unit 113 here has already been described and will be omitted. In the present embodiment, the control unit 150 controls the quantization matrix coding unit 113, and the quantization matrices 800 to 804 shown in FIGS. 8 (a) to 8 (e) are based on the syntax table of FIG. Encoding is performed and quantization matrix code data is generated.

In S303, the control unit 150 controls the integrated coding unit 111 to encode and output the header information necessary for coding the image data together with the generated quantization matrix code data.

In step S304, the control unit 150 controls the block division unit 102 to divide the input image in frame units into basic block units.

In S305, the control unit 150 controls the prediction unit 104 to execute prediction processing on the image data of the basic block unit generated in S304, and predictive information such as sub-block division information and prediction mode and Generate predicted image data. Further, the control unit 150 controls the prediction unit 104 to calculate the prediction error from the input image data and the prediction image data.

In S306, the control unit 150 controls the conversion / quantization unit 105 to perform orthogonal transformation on the prediction error calculated in S305, and generate an orthogonal transformation coefficient. Further, the control unit 150 controls the conversion / quantization unit 105 to perform quantization using the quantization matrix generated / held in S301 and the quantization parameters to generate a residual coefficient. In this embodiment, it is assumed that the quantization matrix of FIGS. 8 (a) to 8 (e) is used according to the size of the subblock, the prediction mode, and the color component.

In S307, the control unit 150 controls the inverse quantization / inverse conversion unit 106, and uses the quantization matrix generated / held in S301 and the quantization parameter for the residual coefficient generated in S306. Inverse quantization is performed to regenerate the orthogonal conversion coefficient. In this step, the same quantization matrix used in S306 is used, and the inverse quantization process is performed. Then, the reproduced orthogonal transformation coefficient is inversely orthogonally converted, and the prediction error is reproduced.

In S308, the control unit 150 controls the image reproduction unit 107, reproduces the predicted image based on the prediction information generated in S305, and generates image data from the reproduced predicted image and the prediction error generated in S307. It is reproduced and stored in the frame memory 108.

In S309, the control unit 150 controls the coding unit 110 to encode the prediction information generated in S305 and the residual coefficient generated in S306 to generate code data. Further, the coding unit 110 outputs the generated code data to the integrated coding unit 111. The integrated coding unit 111 positions and outputs the coded data from the coding unit 110 so as to follow the previously generated header.

In S310, the control unit 150 determines whether or not the coding of all the basic blocks in the frame of interest has been completed. The control unit 150 proceeds to S311 when it determines that it has finished, returns processing to S304 when it determines that an uncoded basic block remains, and continues encoding for the next basic block. Let me.

In S311 the control unit 150 controls the in-loop filter unit 109, performs in-loop filter processing on the image data reproduced in S308, generates a filtered image, and ends the processing.

In the above configuration and operation, especially in S302, the amount of code generated by the quantization matrix is suppressed by omitting the coding of the quantization matrix corresponding to the color difference component and coding only the quantization matrix corresponding to the luminance component. Can be made to.

In the present embodiment, the 2 × 2 quantization matrix 804 corresponding to the color difference component is generated by thinning out the even-numbered rows / even-column components of the 4 × 4 quantization matrix 803 corresponding to the luminance component. The generation method is not limited to this. For example, a configuration using a flat quantization matrix 805 in which all the elements are uniform, which is shown in FIG. 8 (f) and serves as a reference when the quantization matrix is not used, may be used. As a result, the arithmetic processing for generating the 2 × 2 quantization matrix is reduced, and further, the weighting at the time of quantization for each frequency component in the 2 × 2 orthogonal transformation in which the frequency decomposition performance is relatively inferior is reduced. be able to.

Further, in the present embodiment, the quantization matrix corresponding to the color difference component has been described as the same as or can be generated as the quantization matrix corresponding to the luminance component, but the quantization matrix used is not limited thereto.

For example, it is also possible to use the quantization matrix group as shown in FIG. 13 in this embodiment. FIG. 13 (a) shows an 8 × 8 quantization matrix 1300 for the Y component using intra-prediction. Further, FIG. 13D shows an 8 × 8 quantization matrix 1303 for the Y component using inter-prediction. FIG. 13 (b) shows a 4 × 4 quantization matrix 1301 for the Cb component using intra-prediction, and FIG. 13 (c) shows a 4 × 4 quantization matrix for the Cr component using intra-prediction. It shows 1302.

Similarly, FIG. 13 (e) shows a 4x4 quantization matrix 1304 for the Cb component using inter-prediction, and FIG. 13 (f) shows 4x4 for the Cr component using inter-prediction. The quantization matrix 1305 is shown.

Further, FIG. 13 (g) shows a 4 × 4 quantization matrix 1306 for the Y component using intra-prediction, and FIG. 13 (h) shows a 4 × 4 quantum for the Y component using inter-prediction. Quantization matrix 1307 is shown.

Further, FIG. 13 (i) shows a 2 × 2 quantization matrix 1308 for the Cb component using inter-prediction, and FIG. 13 (j) shows a 2 × 2 quantum for the Cr component using inter-prediction. Quantization matrix 1309 is shown. In other words, FIGS. 13 (a), 13 (d), 13 (g), and 13 (h) are quantization matrix groups corresponding to the luminance components. Further, FIGS. 13 (b), 13 (c), 13 (e), 13 (f), 13 (i), and 13 (j) can be regarded as a quantization matrix corresponding to the color difference component. can. In this case, since it is not possible to generate all of the quantization matrix groups corresponding to the color difference component using the quantization matrix group corresponding to the luminance component, it is necessary to encode the quantization matrix group corresponding to the color difference component. ..

Therefore, the quantization matrix coding unit 113 encodes 1 as scaling_list_chroma_present_flag, and encodes the quantization matrix for the color difference component corresponding to id = 0, 1, 3, 4, 6, and 7. Thereby, when it is desired to apply different quantization control to the luminance component and the color difference component, it is possible to set a different quantization matrix for each color component and encode it.

FIG. 2 is a block diagram of an image decoding device 200 that decodes the coded image data (bit stream) generated by the image coding device 100 described in the present embodiment. Hereinafter, the configuration and its operation related to the decoding process will be described with reference to FIG.

The image decoding device 200 has a control unit 250 that controls the entire device. Further, the image decoding device 200 includes an input terminal 201, a separation decoding unit 202, a decoding unit 203, an inverse quantization / inverse conversion unit 204, an image reproduction unit 205, a frame memory 206, an in-loop filter unit 207, an output terminal 208, and the like. , Has a quantization matrix decoding unit 209. The separation decoding unit 202, the decoding unit 203, and the quantization matrix decoding unit 209 cooperate with each other to function as a decoding means for decoding each data from the bit stream.

The input terminal 201 inputs a coded bit stream, and the input source is, for example, a storage medium storing the coded stream, but it may be input from a network, and the type thereof does not matter.

The separation / decoding unit 202 separates the bitstream into code data related to information related to the decoding process and coefficients, and decodes the code data existing in the header part of the bitstream. The separation / decoding unit 202 of the present embodiment separates the quantization matrix code data and outputs it to the quantization matrix decoding unit 209. Further, the separation / decoding unit 202 outputs the code data of the image to the decoding unit 203. That is, the separation / decoding unit 202 performs the reverse operation of the integrated coding unit 111 of FIG.

The quantization matrix decoding unit 209 reproduces and holds the quantization matrix by decoding the quantization matrix code data supplied from the separation decoding unit 202. Specifically, the quantization matrix decoding unit 209 reproduces each element in the quantization matrix by sequentially adding each difference value in the difference matrix of the quantization matrix from the above-mentioned initial value. Then, the quantization matrix decoding unit 209, for example, associates each of the reproduced one-dimensional elements with each element of the two-dimensional quantization matrix in order according to the scanning method shown in FIG. 9, thereby forming a two-dimensional quantum. Regenerate the quantization matrix.

The decoding unit 203 decodes the code data of the image output from the separation decoding unit 202, and reproduces the residual coefficient and the prediction information.

Similar to the inverse quantization / inverse conversion unit 106 of FIG. 1, the inverse quantization / inverse conversion unit 204 performs inverse quantization on the residual coefficient using the regenerated quantization matrix and the quantization parameter. , Obtain the orthogonal conversion coefficient after dequantization. The inverse quantization / inverse conversion unit 204 further reproduces the prediction error by executing the inverse orthogonal transformation with respect to the orthogonal transformation coefficient. As described above, the process of reproducing (deriving) the orthogonal transform coefficient using the quantization matrix and the quantization parameter is referred to as inverse quantization. Further, the function of performing inverse quantization and the function of performing inverse orthogonal transformation may be configured separately. The information for deriving the quantization parameter is also decoded from the bit stream by the decoding unit 203.

The image reproduction unit 205 appropriately refers to the frame memory 206 based on the input prediction information to generate the prediction image data. The image reproduction unit 205 generates predicted image data in sub-block units by using one of two types of prediction methods, intra-prediction and inter-prediction. In addition, a prediction method that combines intra-prediction and inter-prediction may be used.

Then, the image reproduction unit 205 generates the reproduced image data from the predicted image data and the prediction error reproduced by the inverse quantization / inverse conversion unit 204. Specifically, the image reproduction unit 205 reproduces the image data by adding the predicted image data and the prediction error, and stores the image data in the frame memory 206.

Similar to the in-loop filter unit 109 of FIG. 1, the in-loop filter unit 207 performs in-loop filter processing such as a deblocking filter on the reproduced image data stored in the frame memory 206, and frames the processed image data. Restore in memory 206.

The output terminal 208 sequentially outputs the frame images stored in the frame memory 206 to the outside. The output destination is generally a display device, but other devices may be used.

The operation related to image decoding of the image decoding apparatus 200 of the above embodiment will be described in more detail. In the present embodiment, the encoded bit stream is input in frame units.

In FIG. 2, the bit stream for one frame input from the input terminal 201 is supplied to the separation / decoding unit 202. The separation / decoding unit 202 separates the bitstream into information related to the decoding process and code data related to the coefficient, and decodes the code data existing in the header unit of the bitstream.

Then, the separation / decoding unit 202 supplies the quantization matrix code data included in the header unit to the quantization matrix decoding unit 209, and supplies the code data of the image data to the decoding unit 203. Specifically, the separation / decoding unit 202 first extracts the quantization matrix code data from the sequence header of the bitstream shown in FIG. 6A and outputs the quantization matrix code data to the quantization matrix decoding unit 209. In the present embodiment, the quantization matrix code data corresponding to the quantization matrix shown in FIGS. 8A to 8E is extracted and output. Subsequently, the code data of the basic block unit of the picture data is extracted and output to the decoding unit 203.

The quantization matrix decoding unit 209 first decodes the input quantization matrix code data, and based on the syntax table shown in FIG. 12, obtains the coding result of each quantization matrix generated on the coding side. Reproduce. First, the quantization matrix decoding unit 209 decodes scaling_matrix_for_lfnst_dismade_flag and obtains 0 as a value. Since the quadratic inverse transformation is not used in this embodiment, this value has no effect. Subsequently, scaling_list_chroma_present_flag is decoded from the bitstream. Here, it is assumed that the value is 0. From this, it can be seen that the quantization matrix corresponding to the color difference component is not encoded in the bit stream. As described above, when the image data to be decoded has a Y component, a Cb component, and a Cr component, and the scaling_list_chroma_present_flag is 0, for example, it is as follows. That is, the quantization matrix decoding unit 209 decodes only the quantization matrix for the Y component and does not decode the quantization matrix for the Cb / Cr component in order to process the block to be encoded. Then, the inverse quantization / inverse transformation unit 204 also performs the inverse quantization of the Cb / Cr component by using the quantization matrix for the Y component.

Further, even when the image data to be decoded has only the Y component (monochrome), the decoding unit 203 may decode the scaling_list_chroma_present_flag from the bit stream. That is, the decoding unit 203 may decode the scaling_list_chroma_present_flag into a bitstream regardless of whether it is monochrome or not. Even when the image data has only the Y component, decoding the scaling_list_chroma_present_flag from the bitstream has the following effects. That is, the contents of the inverse quantization process and the decoding process of the quantization matrix can be determined based on scaling_list_chroma_present_flag, regardless of whether the image data is monochrome or not. Therefore, it is possible to prevent the processing from becoming complicated.

Next, since there is a for loop with parameters based on the size and type of the quantization matrix called id, the decoding process of the quantization matrix corresponding to the id value is performed.

Specifically, id = 0 corresponds to a 2 × 2 quantization matrix for the Cb component using inter-prediction, and id = 1 corresponds to a 2 × 2 quantization matrix for the Cr component using inter-prediction. There is. id = 2 is a 4x4 quantization matrix for the Y component using intra-prediction, id = 3 is a 4x4 quantization matrix for the Cb component that also uses intra-prediction, and id = 4 is an intra-prediction. The 4 × 4 quantization matrix for the Cr component used corresponds.

Similarly, id = 5 is a 4 × 4 quantization matrix for the Y component using inter-prediction, id = 6 is a 4 × 4 quantization matrix for the Cb component using inter-prediction, and id = 7 is inter-prediction. A 4 × 4 quantization matrix for the Cr component using is supported. Furthermore, id = 8 corresponds to an 8 × 8 quantization matrix for the Y component using intra-prediction, and id = 9 corresponds to an 8 × 8 quantization matrix for the Y component using inter-prediction.

Therefore, when the for loop processing by this id is performed, the quantization matrix of the same size is continuously decoded. That is, in the present embodiment, it means that after the 2 × 2 quantization matrix group is decoded, the 4 × 4 quantization matrix group and the 8 × 8 quantization matrix group are decoded in this order. .. In VVC, orthogonal transformation with a minimum magnitude of 2 × 2 to 64 × 64 is used, and it is also possible to encode a quantization matrix corresponding to each. However, in this embodiment, since the maximum orthogonal transformation size, that is, the size of the subblock is 8 × 8, the upper limit of the for loop corresponds to the 8 × 8 quantization matrix for the Y component using inter-prediction. Id = 9.

In this way, by setting the upper limit of the for loop based on the maximum value of the orthogonal transform size actually used, unnecessary coding of the quantization matrix is omitted and the occurrence of redundant codes is prevented. The stream can be decoded.

Subsequently, the decoding procedure of each quantization matrix will be described. First, the quantization matrix decoding unit 209 makes a determination based on scaling_list_chroma_present_flag and id. When the remainder obtained by dividing id by 3 is 2, or when id is 9, since it corresponds to the quantization matrix for any Y component, the process of decoding for the target Y component is performed. Move to. On the other hand, when id is other than that, the target of decoding is the quantization matrix for the Cb or Cr component. Therefore, only when the scalaring_list_chroma_present_flag is 1, the quantization matrix for the target color difference component is decoded. In this embodiment, since scaling_list_chroma_present_flag is 0, only the quantization matrix for the Y component corresponding to id = 2, 5, 8 and 9 is decoded.

In decoding each quantization matrix, first, scaling_list_copy_mode_flag is decoded in the syntax table of FIG. When the obtained value is 1, it means that the quantization matrix to be decoded is the same as the decoded quantization matrix, and when it is 0, it means that it is not. In the present embodiment, the scalaring_list_copy_mode_flag corresponding to the quantization matrix to be decoded is always 0, which means that there is no identical quantization matrix that has been decoded.

Subsequently, when scaling_list_copy_mode_flag is 0, the scaling_list_pred_mode_flag indicating whether to use the predicted difference coding with each element of the encoded quantization matrix is decoded. When the obtained value is 1, it means that it is a predicted difference coding mode for decoding the difference value with each element of the decoded quantization matrix, and when it is 0, it means that each element in the quantization matrix and its element. It means that it is a mode to decode the difference value from the immediately preceding element. In the present embodiment, since 0 is obtained as scaling_list_pred_mode_flag, the difference value between each element in the quantization matrix and the element immediately before it is decoded in the mode of decoding.

Next, when the scaling_list_copy_mode_flag or the scaling_list_pred_mode_flag is 1, the scaling_list_pred_id_delta is encoded. scaling_list_pred_id_delta means the identifier of the coded quantization matrix to be referenced. In this embodiment, neither the scaling_list_copy_mode_flag nor the scaling_list_pred_mode_flag is 0, so the scaling_list_pred_id_delta is not encoded.

Next, we move on to the differential coding of each element of the quantization matrix to be decoded. For example, decoding of a 4 × 4 quantization matrix for the Y component using intra-prediction corresponding to id = 2 will be described. The quantization matrix decoding unit 209 decodes the difference information, that is, scaling_list_delta_coef in the syntax table of FIG. 12 by the number of elements of the quantization matrix, and reproduces the one-dimensional difference matrix shown in FIG. The quantization matrix decoding unit 209 is arranged in two dimensions using the scanning method of FIG. 9B while adding each difference value in the one-dimensional difference matrix of FIG. 10 to the immediately preceding element, and is arranged in two dimensions in FIG. 8 (b). The quantization matrix 802 of c) is reproduced. Similarly, FIGS. 8 (a), 8 (b), 8 (c), and 8 (d), which are quantization matrix groups corresponding to the luminance components corresponding to id = 2, 5, 8, and 9. ) Quantization matrix is regenerated.

On the other hand, the quantization matrix decoding unit 209 reproduces the quantization matrix group corresponding to the color difference component whose decoding process is omitted in the present embodiment by using the quantization matrix group of the decoded luminance component. First, the 2 × 2 quantization matrix for the Cb / Cr component using the inter-prediction corresponding to id = 0 and 1, is an even number of rows of the quantization matrix of FIG. 8 (d) corresponding to the decoded id = 5. Only the elements of the even number of columns are extracted, and the quantization matrix of FIG. 8 (e) is reproduced.

As the 4 × 4 quantization matrix for the Cb / Cr component using the intra prediction corresponding to id = 3 and 4, the quantization matrix of FIG. 8 (c) corresponding to id = 2 is used. Similarly, as the 4 × 4 quantization matrix for the Cb / Cr component using the inter-prediction corresponding to id = 6, 7, the quantization matrix of FIG. 8 (d) corresponding to id = 5 is used.

In the present embodiment, the coding table shown in FIG. 11A is used in decoding the difference information, but the coding table shown in FIG. 11B may be used. In short, the same one as the coding side may be used. Then, the quantization matrix decoding unit 209 is reproduced in this way and holds the quantization matrix 800 to 804. Here, the operation opposite to the operation of the quantization matrix coding unit 113 on the coding side is performed.

The decoding unit 203 decodes the code data supplied from the separation decoding unit 202, reproduces the prediction information, and further reproduces the residual coefficient. First, the decoding unit 203 reproduces the prediction information and acquires the prediction mode used in the subblock. The decoding unit 203 outputs the reproduced residual coefficient to the inverse quantization / inverse conversion unit 204, and outputs the reproduced prediction information to the image reproduction unit 205.

The inverse quantization / inverse conversion unit 204 performs inverse quantization on the input residual coefficient using the quantization matrix reproduced by the quantization matrix decoding unit 209 and the quantization parameter, and performs the orthogonal conversion coefficient. Is generated, and the prediction error is reproduced by performing inverse orthogonal transformation.

Similar to the inverse quantization / inverse conversion unit 106 on the coding side, the inverse quantization / inverse conversion unit 204 uses a quantization matrix corresponding to the size and color component of the decoding target subblock and a quantization parameter to reverse the quantization. Quantization will be performed. That is, when the intra prediction mode is used without subblock division and with the size of the basic block, the orthogonal transformation coefficient of the Y component is shown in FIG. 8A, and the orthogonal transformation coefficient of the Cb / Cr component is used. The quantization matrix of FIG. 8 (c) is used. Similarly, when the inter-prediction mode is used instead of the sub-block division, the orthogonal transformation coefficient of the Y component is shown in FIG. 8 (b), and the orthogonal transform coefficient of the Cb / Cr component is shown in FIG. 8 (d). ) Quantization matrix is used. On the other hand, when the sub-block division of FIG. 7B is performed and the inter-prediction mode is used, the orthogonal transformation coefficient of the Y component is shown in FIG. 8D, and the orthogonal transformation coefficient of the Cb / Cr component is shown in FIG. The quantization matrix of e) is used. The prediction error reproduced by applying the inverse orthogonal transformation to the orthogonal transformation coefficient reproduced in this way is output to the image reproduction unit 205. However, the quantization matrix used is not limited to this, and may be the same as the quantization matrix used in the conversion / quantization unit 105 and the inverse quantization / inverse conversion unit 106 on the coding side.

The image reproduction unit 205 appropriately refers to the frame memory 206 based on the prediction information input from the decoding unit 203, and reproduces the predicted image. The image reproduction unit 205 of the present embodiment uses intra-prediction and inter-prediction as in the prediction unit 104 on the coding side. Since the specific prediction processing is the same as that of the prediction unit 104 on the coding side, the description thereof will be omitted. The image reproduction unit 205 reproduces the image data from the predicted image and the prediction error input from the inverse quantization / inverse conversion unit 204, and stores the image data in the frame memory 206. The stored image data is used as a reference when making a prediction.

Like the in-loop filter unit 109 on the coding side, the in-loop filter unit 207 reads the reproduced image from the frame memory 206 and performs in-loop filter processing such as a deblocking filter. Then, the in-loop filter unit 207 re-stores the filtered image in the frame memory 206.

The reproduced image stored in the frame memory 206 is finally output from the output terminal 208 to the outside (for example, a display device).

FIG. 4 is a flowchart showing a decoding process of the control unit 205 in the image decoding device 200 according to the embodiment.

First, in S401, the control unit 250 controls the separation / decoding unit 202 to separate the bit stream into information related to the decoding process and code data related to the coefficient, and decode the code data of the header portion. More specifically, the separation / decoding unit 202 supplies the quantization matrix code data to the quantization matrix decoding unit 209, and supplies the image code data to the decoding unit 203.

In S402, the control unit 250 controls the quantization matrix decoding unit 209 to decode the quantization matrix code data reproduced in S401 based on the syntax table of FIG. The specific operation of the quantization matrix decoding unit 209 here has already been described and will be omitted. In the present embodiment, the control unit 250 controls the quantization matrix decoding unit 209 to decode the quantization matrix code data reproduced in S401 based on the syntax table of FIG. 12, and FIGS. The quantization matrix 800 to 804 shown in e) is regenerated and held.

In S403, the control unit 250 controls the decoding unit 203, decodes the code data separated in S401, reproduces the prediction information, and reproduces the residual coefficient.

In S404, the control unit 250 controls the inverse quantization / inverse conversion unit 204, performs inverse quantization using the quantization matrix reproduced in S402 with respect to the residual coefficient, and the quantization parameter, and is orthogonal to each other. Generate a conversion factor. The inverse quantization / inverse transformation unit 204 further performs inverse orthogonal transformation to reproduce the prediction error. In the present embodiment, the quantization matrix used in the inverse quantization process is determined according to the color component and size of the subblock to be decoded. That is, the inverse quantization / inverse conversion unit 204 uses the quantization matrix of FIGS. 8A to 8E and the quantization parameter according to the size of the subblock, the prediction mode, and the color component. Inverse quantize. However, the quantization matrix used is not limited to this, and may be the same as the quantization matrix used on the coding side.

In S405, the control unit 250 controls the image reproduction unit 205 to reproduce the image based on the prediction information generated in S403. Specifically, the image reproduction unit 205 reproduces the predicted image with reference to the frame memory 206 based on the prediction information. At this time, the image reproduction unit 205 uses intra-prediction and inter-prediction as in S305 on the coding side. Then, the image reproduction unit 205 reproduces the image data from the reproduced predicted image and the prediction error generated in S404, and stores the reproduced image data in the frame memory 206.

In S406, the control unit 250 determines whether or not the decoding of all the basic blocks in the frame of interest has been completed, and if so, proceeds to S407, and if there is an unsigned basic block. Returns the process for decoding the next basic block back to S403.

In S407, the control unit 250 controls the in-loop filter unit 207, performs in-loop filter processing on the image data reproduced in S405, generates a filtered image, and ends the processing.

With the above configuration and operation, the coding of the coded bit stream generated by the image coding device 100 described above, that is, the quantization matrix corresponding to the color difference component is omitted, and the code amount generated by the quantization matrix is omitted. It is possible to decode a bitstream with suppressed.

In the present embodiment, the 2 × 2 quantization matrix 804 corresponding to the color difference component is generated by thinning out the even components of the 4 × 4 quantization matrix 803 corresponding to the luminance component. The method for generating the corresponding 2 × 2 quantization matrix is not limited to this. For example, a configuration using a flat quantization matrix 805 in which all the elements are uniform, which is shown in FIG. 8 (f) and serves as a reference when the quantization matrix is not used, may be used. As a result, the arithmetic processing for generating the 2 × 2 quantization matrix is reduced, and the weighting at the time of quantization for each frequency component in the 2 × 2 orthogonal transformation in which the frequency decomposition performance is relatively inferior is reduced. The bitstream can be decoded.

Further, in the present embodiment, the decoding of the bit stream generated in a form in which the quantization matrix corresponding to the color difference component is the same as or can be generated in the same manner as the quantization matrix corresponding to the luminance component is described, but the quantum used. The quantization matrix is not limited to this.

For example, it is also possible to decode a bitstream encoded using the quantization matrix group as shown in FIG. In this case, since 1 is encoded in the bitstream as scaling_list_chroma_present_flag on the coding side, decoding of the quantization matrix corresponding to the color difference component is not omitted on the decoding side. That is, when the image data to be decoded has a Y component, a Cb component, and a Cr component, and the scaling_list_chroma_present_flag is 1, the result is as follows. That is, the quantization matrix decoding unit 209 decodes the quantization matrix for the Y component and the quantization matrix for the Cb / Cr component, respectively. Then, the inverse quantization / inverse transformation unit 204 quantizes the Y component using the quantization matrix for the Y component. Then, the inverse quantization / inverse transformation unit 204 quantizes the Cb / Cr component using the quantization matrix for the Cb / Cr component. As a result, each quantization matrix is decoded, and the quantization matrix group shown in FIG. 13 is decoded and reproduced. As a result, when it is desired to apply different quantization controls to the luminance component and the color difference component, it is possible to set a different quantization matrix for each color component and decode the encoded bit stream.

Each processing unit included in the image coding device 100 and the image decoding device 200 of the above embodiment may be realized by using the hardware shown in FIG.

FIG. 5 is a block diagram showing a configuration example of computer hardware applicable to the image coding device 100 and the image decoding device 200 according to the above embodiment.

The CPU 501 controls the entire computer by using the computer programs and data stored in the RAM 502 and the ROM 503, and executes each of the above-described processes as performed by the image processing apparatus according to the above embodiment. That is, the CPU 501 functions as each processing unit shown in FIGS. 1 and 2.

The RAM 502 has an area for temporarily storing data acquired from the outside via the external storage device 506 and the I / F (interface) 507. Further, the RAM 502 is also used as a work area used by the CPU 501 when executing various processes. The RAM 502 can be allocated as a frame memory, for example, or various other areas can be provided as appropriate.

The ROM 503 stores the setting data of the computer, the boot program, and the like. The operation unit 504 is composed of a keyboard, a mouse, and the like, and can be operated by a user of the computer to input various instructions to the CPU 501. The output unit 505 displays the processing result by the CPU 501. Further, the output unit 505 is composed of, for example, a liquid crystal display.

The external storage device 506 is a large-capacity information storage device typified by a hard disk drive device. The external storage device 506 stores an OS (operating system) and a computer program (application program) for realizing the functions of the respective parts shown in FIGS. 1 and 2 in the CPU 501. Further, each image data as a processing target may be stored in the external storage device 506.

The computer programs and data stored in the external storage device 506 are appropriately loaded into the RAM 502 according to the control by the CPU 501, and are processed by the CPU 501. A network such as a LAN or the Internet, or other devices such as a projection device or a display device can be connected to the I / F 507, and the computer acquires and sends various information via the I / F 507. Can be done. Reference numeral 508 is a bus connecting the above-mentioned parts.

In the above configuration, when the power is turned on to the present device, the CPU 501 executes the boot program stored in the ROM 503, loads the OS stored in the external storage device 506 into the RAM 502, and executes the boot program. Then, under the control of the OS, the CPU 501 loads the application program related to coding or decoding from the external storage device 506 into the RAM 502 and executes it. As a result, the CPU 501 functions as each processing unit of FIG. 1 or 2, and this device functions as an image coding device 100 or an image decoding device 200.

(Other Examples)
It is also realized by supplying a program that realizes one or more functions of the above-described embodiment to a system or a device via a network or a storage medium, and reading and executing the program by one or more processors in the computer of the system or the device. It is possible. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Each embodiment is also achieved by supplying a storage medium in which the code of the computer program that realizes the above-mentioned functions is recorded to the system, and the system reads and executes the code of the computer program. In this case, the computer program code itself read from the storage medium realizes the function of the above-described embodiment, and the storage medium storing the computer program code constitutes the present invention. It also includes cases where the operating system (OS) running on the computer performs part or all of the actual processing based on the instructions in the code of the program, and the processing realizes the above-mentioned functions. ..

Further, it may be realized in the following form. That is, the computer program code read from the storage medium is written to the memory provided in the function expansion card inserted in the computer or the function expansion unit connected to the computer. Then, based on the instruction of the code of the computer program, the function expansion card, the CPU provided in the function expansion unit, or the like performs a part or all of the actual processing to realize the above-mentioned function.

When the present invention is applied to the storage medium, the storage medium stores the code of the computer program corresponding to the flowchart described above.

101 Input terminal 102 Block division 103 Quantization matrix holding unit 104 Prediction unit 105 Conversion / quantization unit 106 Inverse quantization / inverse conversion unit 107 Image reproduction unit 108 Frame memory 109 In-loop filter unit 110 Coding unit 111 Integrated coding Part 112 Output terminal 113 Quantization matrix coding part

Claims (10)

An image coding device capable of encoding an image having a luminance component, a first color difference component, and a second color difference component into a bit stream.
A partitioning means for dividing the image to be encoded into a plurality of blocks, and
A prediction means that performs prediction processing on the block to be coded and derives a prediction error of the block to be coded.
A conversion means for deriving the orthogonal conversion coefficient by orthogonally converting the prediction error,
A quantization means for quantizing the orthogonal transformation coefficient using at least the first quantization matrix, and
It has at least a coding means for encoding the first quantization matrix.
The coding means is
Information indicating whether the second quantization matrix corresponding to the first color difference component and the third quantization matrix corresponding to the second color difference component are encoded in the bit stream is provided. Encoded into a bitstream
The quantization means is
When the image to be encoded has the luminance component, the first color difference component, and the second color difference component, the second quantization matrix and the third quantization matrix are included in the bit stream. If the information indicates that it is encoded,
The prediction error of the luminance component of the block to be encoded is quantized by using the first quantization matrix.
The prediction error of the first color difference component of the block to be encoded is quantized by using the second quantization matrix.
The prediction error of the second color difference component of the block to be encoded is quantized by using the third quantization matrix.
When the image to be encoded has the luminance component, the first color difference component, and the second color difference component, the second quantization matrix and the third quantization matrix are included in the bit stream. If the information indicates that it is unencoded,
The prediction error of the brightness component of the block to be coded, the prediction error of the first color difference component of the block to be coded, and the second color difference component of the block to be coded. An image coding device characterized in that a prediction error is quantized using a first quantization matrix. The coding means is
When the image to be encoded has the brightness component, the first color difference component, and the second color difference component, the second quantization matrix and the third quantization matrix are combined in the bit stream. The image coding according to claim 1, wherein when the information indicates that it is encoded, the second quantization matrix and the third quantization matrix are encoded in the bit stream. Device. The coding means is
The invention according to claim 1 or 2, wherein the information is encoded in the bit stream even when the image to be encoded does not have the first color difference component and the second color difference component. Image encoding device. The image coding apparatus according to any one of claims 1 to 3, wherein the information is a flag. The image coding apparatus according to claim 4, wherein the flag is scaling_list_chroma_present_flag. An image decoding device capable of decoding a bitstream in which an image is encoded and having a luminance component, a first color difference component, and a second color difference component.
By using a decoding means for decoding at least the first quantization matrix from the bit stream and at least the first quantization matrix, the residual coefficient of the block to be decoded in the image to be decoded is inversely quantized and orthogonally transformed. Inverse quantization means to derive coefficients and
An inverse transformation means for deriving a prediction error by inversely transforming the orthogonal transformation coefficient,
It has a generation means that performs prediction processing on the prediction error and generates an image of the block to be decoded.
The decoding means
Information indicating whether the second quantization matrix corresponding to the first color difference component and the third quantization matrix corresponding to the second color difference component are encoded in the bit stream is provided. Decrypt from bitstream
The inverse quantization means
When the bit stream to be decoded has the brightness component, the first color difference component, and the second color difference component, the second quantization matrix and the third quantization matrix are coded in the bit stream. If the above information indicates that it has been quantized,
The residual coefficient of the luminance component of the block to be decoded is inversely quantized using the first quantization matrix.
The residual coefficient of the first color difference component of the block to be decoded is inversely quantized using the second quantization matrix.
The residual coefficient of the second color difference component of the block to be decoded is inversely quantized using the third quantization matrix.
When the bit stream to be decoded has the luminance component, the first color difference component, and the second color difference component, the second quantization matrix and the third quantization matrix are included in the bit stream. If the information indicates that it is unencoded,
The residual coefficient of the luminance component of the block to be decoded, the residual coefficient of the first color difference component of the block to be decoded, and the residual of the second color difference component of the block to be decoded. An image decoding device characterized in that the difference coefficient is inversely quantized using the first quantization matrix. The decoding means
When the bit stream to be decoded has the brightness component, the first color difference component, and the second color difference component, the second quantization matrix and the third quantization matrix are included in the bit stream. The image decoding apparatus according to claim 6, wherein when the information indicates that the information is encoded, the second quantization matrix and the third quantization matrix are decoded from the bit stream. The decoding means
The image decoding according to 6 or 7, wherein the information is decoded from the bit stream even when the bit stream to be decoded does not have the first color difference component and the second color difference component. Device. The image decoding apparatus according to any one of claims 6 to 8, wherein the information is a flag. The image decoding apparatus according to claim 9, wherein the flag is scaling_list_chroma_present_flag.

Images