WO2020028998A1

WO2020028998A1 - Method and device for coding image data, and method and device for decoding image data

Info

Publication number: WO2020028998A1
Application number: PCT/CN2018/098900
Authority: WO
Inventors: Sato Kazushi
Original assignee: Huawei Technologies Co., Ltd.
Priority date: 2018-08-06
Filing date: 2018-08-06
Publication date: 2020-02-13

Abstract

A method or the like for coding image data is provided. This method includes: generating a sequence parameter set and a picture parameter set which are associated with the image data, and both include a first syntax element indicating a minimum block size in orthogonal transform and a second syntax element indicating a maximum block size in orthogonal transform; and outputting a bitstream including the sequence parameter set and the picture parameter set.

Description

METHOD AND DEVICE FOR CODING IMAGE DATA, AND METHOD AND DEVICE FOR DECODING IMAGE DATA

Technical Field

The present disclosure relates to techniques for coding and decoding image data.

Background Art

Video coding techniques, such as H. 264/AVC (Advanced Video Coding) and H. 265/HEVC (High Efficiency Video Coding) , employ a hybrid coding scheme using both block-by-block intra and inter prediction. In this scheme, a prediction residual signal derived after intra and inter prediction is a target for orthogonal transform. Quantization is performed on an output through orthogonal transform.

In the video coding techniques, different quantization matrices are designated frequency by frequency in quantization. Information on the designated quantization matrices is supplied from a device that codes a video to a device that decodes the video. The information on the designated quantization matrices is used in an inverse quantization process that is performed at the time of decoding.

Summary of Invention

Under the above circumstances, technical advantages are provided by embodiments disclosed below.

A first aspect of an embodiment provides a method for coding image data, including: generating a sequence parameter set and a picture parameter set which are associated with the image data, and both include a first syntax element indicating a minimum block size to be subjected to orthogonal transform and a second syntax element indicating a maximum block size to be subjected to orthogonal transform; and outputting a bitstream including the sequence parameter set and the picture parameter set.

The first aspect can avoid redundant transmission of quantization matrix data.

A second aspect of an embodiment provides a method for decoding image data, including: receiving a bitstream, the bitstream including a sequence parameter set and a picture parameter set which are associated with the image data, and both include a first syntax element indicating a minimum block size to be subjected to orthogonal transform and a second syntax element indicating a maximum block size to be subjected to orthogonal transform; and decoding the bitstream, in which the first and second syntax elements included in the sequence parameter set and the picture parameter set are referred to.

The second aspect can avoid redundant reception of quantization matrix data.

A third aspect of an embodiment provides an image coding device, including: a function unit for coding a sequence parameter set and a picture parameter set which are associated with the image data, and both include a first syntax element indicating a minimum block size to be subjected to orthogonal transform and a second syntax element indicating a maximum block size to be subjected to orthogonal transform; and an output unit for outputting a bitstream including the sequence parameter set and the picture parameter set.

The third aspect can avoid redundant transmission of quantization matrix data.

A fourth aspect of an embodiment provides an image decoding device, including: a buffer unit for receiving a bitstream, the bitstream including a sequence parameter set and a picture parameter set which are associated with the image data, and both include a first syntax element indicating a minimum block size to be subjected to orthogonal transform and a second syntax element indicating a maximum block size to be subjected to orthogonal transform; and a decoding unit for decoding the bitstream, the decoding unit referring to the first and second syntax elements included in the sequence parameter set and the picture parameter set.

The fourth aspect can avoid redundant reception of quantization matrix data.

A fifth aspect of an embodiment provides a program for allowing a computer device to execute the method according to the first aspect or the second aspect.

The fifth aspect can avoid redundant transmission and reception of quantization matrix data.

A sixth aspect of an embodiment provides a computer readable recording medium storing a program for allowing a computer device to execute the method according to the first aspect or the second aspect.

The sixth aspect can avoid redundant transmission and reception of quantization matrix data.

A seventh aspect of an embodiment provides an image processing system including: the image coding device according to the third aspect; and the image decoding device according to the fourth aspect.

The seventh aspect can avoid redundant transmission and reception of quantization matrix data.

Brief Description of Drawings

To describe the technical solutions in the embodiments more clearly, the following briefly describes the accompanying drawings required for describing the present embodiments. Apparently, the accompanying drawings in the following description depict merely some of the possible embodiments, and a person of ordinary skill in the art may still derive other drawings, without creative efforts, from these accompanying drawings, in which:

[Fig. 1] Fig. 1 is a diagram showing one example of the configuration of an image processing system according to one embodiment;

[Fig. 2] Fig. 2 is a diagram illustrating an example of descriptions in a sequence parameter set according to one embodiment;

[Fig. 3] Fig. 3 is a diagram illustrating an example of descriptions in a picture parameter set according to one embodiment;

[Fig. 4] Fig. 4 is a diagram showing one example of the schematic structure of scaling list data according to one embodiment;

[Fig. 5] Fig. 5 is a diagram showing a general sequence parameter set conforming to H. 265/HEVC;

[Fig. 6] Fig. 6 is a diagram showing a general picture parameter set conforming to H. 265/HEVC;

[Fig. 7] Fig. 7 is a diagram showing general scaling list data conforming to H. 265/HEVC;

[Fig. 8] Fig. 8 is a diagram showing one example of the configuration of an image coding device according to one embodiment;

[Fig. 9] Fig. 9 is a diagram showing one example of the configuration of an image decoding device according to one embodiment;

[Fig. 10A] Fig. 10A is a diagram showing default quantization matrices;

[Fig. 10B] Fig. 10B is a diagram showing default quantization matrices;

[Fig. 10C] Fig. 10C is a diagram showing default quantization matrices;

[Fig. 11] Fig. 11 is a diagram showing how to expand a quantization matrix;

[Fig. 12] Fig. 12 is a diagram showing a default quantization matrix expanded in the image decoding device;

[Fig. 13] Fig. 13 is a flow diagram illustrating one example of a process in the image coding device according to one embodiment;

[Fig. 14] Fig. 14 is a flow diagram illustrating one example of a process in the image decoding device according to one embodiment; and

[Fig. 15] Fig. 15 is a diagram showing one example of the hardware configuration each of the image coding device and the image decoding device.

Description of Embodiments

The following will clearly describe technical solutions of embodiments of the present disclosure referring to the accompanying drawings of the present embodiments. Apparently, described embodiments are not all but just some of the present embodiments. It is to be noted that all other embodiments which may be obtained by a person skilled in the art based on the present embodiments without creative efforts shall fall within the protection scope of embodiments of the present disclosure.

(Configuration of Image Processing System 1)

Fig. 1 is a diagram showing a configurational example of an image processing system 1 according to the present embodiment. The image processing system 1 according to the present embodiment includes an image coding device 10, an image decoding device 50, and an image reproducing device 4.

The image coding device 10 may be a computer, such as a PC (Personal Computer) , a server device, a work station, a tablet terminal, or a smartphone. The image coding device 10 may also be an imaging device, such as a monitoring camera, a digital still camera equipped with a video-shooting capability, or a digital video camera. The image decoding device 50 may be a computer, such as a PC, a server device, a work station, a tablet terminal, or a smartphone. The image coding device 10 and the image decoding device 50 may be realized by using a recording medium, such as an optical recording medium, magnetic recording medium, magneto-optical recording medium, or a semiconductor memory.

When an input image signal is input to the image coding device 10, the image coding device 10 performs coding on the input image signal using a specific coding scheme. The coded signal is transmitted over a cable or radio transmission path 3 to the image decoding device 50.

The image decoding device 50 decodes the coded signal from the image coding device 10, and outputs the decoded signal to the image reproducing device 4 such as a display. As a result, an image is reproduced.

Although the present embodiment is described, as an example, in a case of conforming H. 265/HEVC, the image processing system 1 according to the present embodiment may be applied to the present video coding scheme or other video coding schemes available in future.

According to the configurational example shown in Fig. 1, although the image decoding device 50 and the image reproducing device 4 are shown as separate devices, those devices may be realized by a single device. Alternatively, the

individual devices

4, 10, and 50 may be realized by a single device. Further, the coded image data may be provided to the image decoding device 50 via, for example, a recording medium.

(Outline of Coding Scheme)

The following describes the coding scheme usable by the system 1 to realize the system 1 with reference to Figs. 2 to 7. Fig. 2 is a diagram illustrating an example of descriptions in a sequence parameter set according to one embodiment. Fig. 3 is a diagram illustrating an example of descriptions in a picture parameter set according to one embodiment. Fig. 4 is a diagram showing one example of the schematic structure of scaling list data according to one embodiment. Fig. 5 is a diagram showing a general sequence parameter set conforming to H.265/HEVC. Fig. 6 is a diagram showing a general picture parameter set conforming to H. 265/HEVC. Fig. 7 is a diagram showing general scaling list data conforming to H. 265/HEVC.

In H. 264/AVC which is one of standards for the video coding scheme, different quantization steps can be used for the respective components of the orthogonal transform coefficient in a profile equal to or greater than a high profile at the time of quantizing image data. Specifically, in H. 264/AVC, there are two sizes of orthogonal transform units (hereinafter, simply referred to as "transform unit" ) , namely, 4×4 and 8×8, for each of an intra prediction mode and an inter prediction mode, so that predetermined values (default values) for four types of quantization matrices are defined.

H. 265/HEVC (High Efficiency Video Coding) is one of video compression standards following H. 264/AVC, in which there are four types of orthogonal transform blocks (TU: Transform Unit) of 4×4, 8×8, 16×16, and 32×32. That is, there are twenty types of quantization matrices in H. 265/HEVC. This is expressed by the following relational equation (1) .

20= "3" (block size: 4×4, 8×8, 16×16) × "3" (luminance and color-difference signals: Y, Cb, Cr) × "2" (mode: intra prediction mode, inter prediction mode) + "1" (block size: 32×32) × "1" (luminance signal: Y) × "2" (mode: intra prediction mode, inter prediction mode)

Equation (1)

where the aforementioned "Y" indicates the luminance signal (brightness) , and "Cb" and "Cr" indicate components representing the color-difference signal (intensities of color) .

In the equation, Y=0.299R (red) +0.587G (green) +0.114B (blue) , Cb= {B-Y} ×0.5/0.886=-0.169R-0.331G+0.500B, and Cr= {R-Y} ×0.5/0.701=0.500R-0.419G-0.0081B.

For bitstreams that conform to the conventional H. 265/HEVC, various syntaxes are defined as shown in Figs. 5 to 7.

As shown in Fig. 5, "seq_parameter_set_rbsp () " is given as a description corresponding to a sequence parameter set 500, and includes a syntax element 501 indicating a minimum block size to be subjected to orthogonal transform, and a syntax element 502 indicating a maximum block size to be subjected to inverse quantization. In the example of Fig. 5, the syntax element 501 is shown as "log2_min_transform_block_size_minus2" , and the syntax element 502 is shown as "log2_diff_max_min_transform_block_size" . For example, when the syntax element 501 is "0" (log2_min_transform_block_size_minus2=0) , "minTr" which indicates the minimum block size becomes "4" ( "minTr" =2 ²=4) as the value of "log2 (minTr) " is "2" . That is, the minimum block size is 4×4. Likewise, when the syntax element 501 is "1" (log2_min_transform_block_size_minus2=1) , the minimum block size "minTr" becomes "8" (block size: 8×8) , and when the syntax element 501 is "2" (log2_min_transform_block_size_minus2=2) , the minimum block size "minTr" becomes "16" (block size: 16×16) . The block size which may be indicated by the syntax element 501 depends on the block sizes which may be adopted in the video coding scheme to which the image processing method according to the present embodiment is applied (in the case of H. 265/HEVC, for example, 4×4, 8×8, 16×16, and 32×32) .

The syntax element 502 ( "log2_diff_max_min_transform_block_size" ) related to the maximum block size indicates the difference from the minimum block size. For example, if the syntax element 501 is "0" (log2_min_transform_block_size_minus2=0) (minimum block size: 4×4) , and if the syntax element 502 is "1" (log2_diff_max_min_transform_block_size=1) , the value of "log2 (maxTr) " becomes "3" (the value of "log2 (minTr) " is "2" , and the difference from the value is "1" , so that 2+1=3) , so that "maxTr" indicating the maximum block size becomes "8" ( "maxTr" =2 ³=8) . In other words, the maximum block size becomes 8×8. In the case of the present example, therefore, the block sizes of 4×4 and 8×8 are adopted.

Further, the sequence parameter set 500 includes a flag 503, which is "scaling_list_enabled_flag" , and a flag 504, which is "sps_scaling_list_data_present_flag" .

The flag 503 indicates whether or not to change the quantization matrix frequency by frequency. When the flag 503 is "0" (scaling_list_enabled_flag=0) , for example, it means that frequency-by-frequency control is not performed. In this case, a flat matrix is applied. When the flag 503 is "1" (scaling_list_enabled_flag=1) , however, it means that the quantization matrix is changed frequency by frequency.

Further, with the flag 503 being "1" (scaling_list_enabled_flag=1) , the flag 504 being "0" (sps_scaling_list_data_present_flag=0) means that a default quantization matrix is used. With the flag 503 being "1" (scaling_list_enabled_flag=1) , the flag 504 being "1" (sps_scaling_list_data_present_flag=1) means that a user-defined quantization matrix is used. This user definition may allow for the use of only one or more specific quantization matrices.

In Fig. 5, scaling list data 505 is used to scale a quantization matrix. Quantization matrices may be obtained by this scaling list data 505.

In Fig. 6, a picture parameter set 600 includes a flag 601, which is "pps_scaling_list_data_present_flag" . This flag 601 serves in a similar way as the flag 504 ( "sps_scaling_list_data_present_flag" ) in the sequence parameter set 500 shown in Fig. 5. That is, the flag 601 being "0" (pps_scaling_list_data_present_flag=0) means that the default quantization matrix is used. The flag 601 being "1" (pps_scaling_list_data_present_flag=1) means that a user-defined quantization matrix is used. This user definition may allow for the use of only one or more specific quantization matrices.

Fig. 7 shows scaling list data 700, which allows data of all of the twenty types of quantization matrices to be obtained.

When coding is carried out by the scheme shown in Figs. 5 to 7, all the types of quantization matrices are transmitted to the decoding side. In other words, quantization matrices which will not be used in coding and decoding are actually also transmitted to the decoding side.

In the example of the image processing method according to the present embodiment shown in Figs. 2 to 4, by way of contrast, only information on specific quantization matrices among all the types of quantization matrices may be transmitted to the decoding side. That is, data of the quantization matrices which will be used in coding and decoding are transmitted to the decoding side, so that redundant data transmission can be avoided.

(Sequence Parameter Set)

Fig. 2 shows an example of a sequence parameter set according to the present embodiment. In Fig. 2, a sequence parameter set 200, like the one shown in Fig. 5, is described as "seq_parameter_set_rbsp () " , and includes a syntax element 201 indicating the minimum block size, and a syntax element 202 indicating the maximum block size. In the example of Fig. 2, the syntax element 201 is shown as "sps_log2_min_transform_block_size_minus2" , and the syntax element 202 is shown as "sps_log2_diff_max_min_transform_block_size" . Those

elements

201 and 202 have functions similar to the

elements

501 and 502 shown in Fig. 5. Further, the sequence parameter set 200, like the one shown in Fig. 5, includes a flag 203 ( "scaling_list_enabled_flag" ) and a flag 204 ( "sps_scaling_data_present_flag" ) , and these flags have functions similar to the

flags

503 and 504 shown in Fig. 5.

(Picture Parameter Set)

Fig. 3 shows an example of a picture parameter set according to the present embodiment. In Fig. 3, the picture parameter set 300, like the one shown in Fig. 6, is described as "pic_parameter_set_rbsp () " . However, the picture parameter set 300, unlike the one shown in Fig. 6, further includes two

syntax elements

301 and 302.

The

elements

301 and 302 are similar to the

aforementioned elements

201 and 202. That is, "pps_log2_min_transform_block_size_minus2" representing a parameter of the minimum block size is described as the element 301. Further, "pps_log2_diff_max_min_transform_block_size" representing a parameter of the maximum block size is described as the element 302. Accordingly, the minimum and maximum block sizes are designated in the picture parameter set 300 as well as in the sequence parameter set 200. In this case, different block sizes may be designated for respective sequence intervals or picture by picture. While block sizes of 4×4 to 32×32 may be set for a certain interval, for example, block sizes of only 16×16 and 32×32 may be set for another interval. Therefore, when coding is performed by using a specific block size, an unnecessary scaling list will not be transmitted in the image coding device 10. Moreover, the picture parameter set 300 in Fig. 3, like the one shown in Fig. 6, includes a flag 303 ( "pps_scaling_list_data_present_flag" ) , which has a function similar to the flag 601 shown in Fig. 6.

As mentioned above, when the values of the flag 204 and the flag 303 respectively in the sequence parameter set 200 and the picture parameter set 300 are "1" , a user-defined quantization matrix is used. At this time, the scaling

list data

205, 304 is called up, and a process for scaling list data 400 shown in Fig. 4 is performed.

(Scaling List Data)

Fig. 4 shows one example of scaling list data according to the present embodiment. The scaling list data 400 shown in Fig. 4 is a syntax for referring to quantization matrix data at the time of performing quantization. In Fig. 4, the scaling list data 400 includes two parameters 401 and 402 ( "min_sizeId" and "max_sizeId" ) . "sizeId" is a variable designating the block size of orthogonal transform. Further, "matrixId" is a variable designating a quantization matrix which may be taken for the designated block size. (It is to be noted that the names of those parameters are illustrative only, and may be changed as needed in practice. )

The value of the parameter 401 ( "min_sizeId" ) is determined according to the values of the element 201 and 301 (minimum block size) shown in Figs. 2 and 3. The element 201 ( "sps_log2_min_transform_block_size_minus2" ) and the element 301 ( "pps_log2_min_transform_block_size_minus2" ) indicate the minimum block size to be subjected to orthogonal transform as mentioned above. When the minimum block size derived from the values of the

elements

201 and 301 is 4×4, for example, the value of the parameter 401 ( "min_sizeId" ) becomes "0" . When the minimum block size is 8×8, 16×16, or 32×32, for example, the value of the parameter 401 ( "min_sizeId" ) becomes "1" , "2" or "3" , respectively.

The value of the parameter 402 ( "max_sizeId" ) is determined according to the values of the element 202 and 302 (maximum block size) shown in Figs. 2 and 3. The element 202 ( "sps_log2_diff_max_min_transform_block_size" ) and the element 302 ( "pps_log2_diff_max_min_transform_block_size" ) indicate the maximum block size to be subjected to orthogonal transform as mentioned above. When the maximum block size derived from the values of the

elements

202 and 302 is 4×4, 8×8, 16×16, and 32×32, for example, the value of the parameter 402 ( "max_sizeId" ) becomes "0" , "1" , "2" or "3" , respectively.

Further, "matrixId" is a variable designating a quantization matrix which may be taken for the designated block size. In H. 265/HEVC, for example, there are twenty types of quantization matrices as expressed by the relational equation (1) . "matrixId" in the scaling list data 400 is a variable for obtaining matrices of { "3" (Y, Cb, Cr) × "2" (intra prediction mode, inter prediction mode) =6 types} expressed by the relational equation (1) for each block size the variable "sizeId" may take (in a case of a block size of 32×32, however, (Y) × (intra prediction mode, inter prediction mode) = 2 types) .

A parameter 403 ( "scaling_list_comp_enable_flag" ) is a variable indicating whether frequency-by-frequency control is performed or not. The parameter 403 being "0" (scaling_list_comp_enable_flag=0) means that frequency-by-frequency control is not performed (a flat matrix is used) , and the parameter 403 being "1" (scaling_list_comp_enable_flag=1) means that frequency-by-frequency control is performed.

Furthermore, a parameter 404 ( "scaling_list_comp_data_present_flag" ) is a variable indicating whether a default quantization matrix is used or a user-defined quantization matrix is used. The parameter 404 being "0" (scaling_list_comp_data_present_flag=0) means that a default quantization matrix is used, and the parameter 404 being "1" (scaling_list_comp_data_present_flag=1) means that a user-defined quantization matrix is used.

According to the present embodiment, for each of the quantization matrices designated by the aforementioned combinations of "sizeId" and "matrixId" , when the parameter 403 is "0" (scaling_list_comp_enable_flag [sizeId] [matrixId] =0) , frequency-by-frequency control is not performed (a flat matrix is applied) . On the other hand, when the parameter 403 is "1" (scaling_list_comp_enable_flag [sizeId] [matrixId] =1) , and when the parameter 404 is "0" (scaling_list_comp_data_present_flag [sizeId] [matrixId] =0) , a default quantization matrix is applied.

When the parameter 403 is "1" (scaling_list_comp_enable_flag [sizeId] [matrixId] =1) , and when the parameter 404 is "1" (scaling_list_comp_data_present_flag [sizeId] [matrixId] =1) , a user-defined quantization matrix is applied. Then, only the quantization matrix applied is transmitted, along with the sequence parameter set 200, the picture parameter set 300 and the scaling list data 400, to the image decoding device 50.

The values to be set in the aforementioned individual parameters and flags are just illustrative, and other values may be adopted as a matter of design choice.

The sequence parameter set 200, the picture parameter set 300, and the scaling list data 400, which are applied to each sequence and each picture, as well as the quantization matrix data have the parameters and flags set as described above (for example, user definitions or the like in each sequence and each picture) , and are stored in advance in a recording medium like a semiconductor memory.

The following describes in detail the configurations of the image coding device 10 and the image decoding device 50 which use the coding scheme shown in Figs. 2 to 4.

(Configuration of Image Coding Device 10)

Fig. 8 is a diagram sowing one example of the configuration of the image coding device 10. The image coding device 10 includes a re-ordering buffer 11, a subtracter 12, an orthogonal transform unit 13, a quantizer 14, an entropy coding unit 15, and a storage buffer 16. The image coding device 10 further includes a scaling list setting unit 17, a rate controller 18, an inverse quantizer 19, an inverse orthogonal transform unit 20, an adder 21, a loop filter 22, a frame memory 23, an intra prediction unit 24, and an inter prediction unit 25.

The re-ordering buffer 11 re-orders input image data according to a GOP (Group of Pictures) structure. The re-ordered image data is output to the subtracter 12, the intra prediction unit 24, and the inter prediction unit 25.

Image data input from the re-ordering buffer 11 and predictive image data from the intra prediction unit 24 or the inter prediction unit 25 are supplied to the subtracter 12. The subtracter 12 calculates prediction error data which is the difference between the image data from the re-ordering buffer 11 and the predictive image data, and outputs the calculated prediction error data to the orthogonal transform unit 13.

The orthogonal transform unit 13 performs orthogonal transform on the prediction error data input from the subtracter 12, and generates transform coefficient data which is the result of transform of a pixel region in the image to a frequency region. The generated transform coefficient data is output to the quantizer 14. The orthogonal transform that is performed by the orthogonal transform unit 13 may be, for example, discrete cosine transform (DCT) , Karhunen-Loéve transform or the like. The orthogonal transform unit 13 selects one or more transform units (TU) which used in orthogonal transform on image data to be coded. Candidates for the block size for the transform unit which may be selected then may be all of block sizes that may be adopted in the video coding scheme to which the image processing system 1 of the present embodiment is applied (i.e., 4×4, 8×8, 16×16, and 32×32 in the example of H. 265/HEVC) .

Which block size is selected is determined by referring to the sequence parameter set 200 (Fig. 2) applied to each sequence in the input image signal to be coded, or the picture parameter set 300 (Fig. 3) applied to each picture. Suppose that the input image signal includes one or more sequences and one or more pictures in the video coding scheme adopted in the image processing system 1 of the present embodiment. The sequence parameter set 200 includes a sequence parameter set (SPS) ID, and this SPS_ID is an identifier designating a sequence to which the sequence parameter set 200 should be applied. Likewise, the picture parameter set 300 includes an SPS_ID and a picture parameter set (PPS) ID; the SPS_ID indicates a sequence to which a picture to which the picture parameter set 300 should be applied belongs, and PPS_ID is an identifier designating a picture in the sequence. Based on the SPS_ID and PPS_ID, the sequence parameter set 200 or the picture parameter set 300 which is to be applied to each sequence and each picture in the input image signal may be determined. (It is to be noted that the picture parameter set 300 may not exist for all the pictures, and if there is not a corresponding picture parameter set 300, a latest picture parameter set 300, for example, may be referred to. )

The sequence parameter set 200 and the picture parameter set 300 to be referred to, as described above, respectively include the

syntax elements

201, 301 each indicating the minimum block size, and the

syntax elements

202, 302 each indicating the maximum block size. The quantizer 14 to be described later refers to a quantization matrix only for a specific block size in the scaling list data 400 based on those values, and performs quantization on the specific block size.

Information indicating the block size of the transform unit selected by the orthogonal transform unit 13 is also output to the quantizer 14, the entropy coding unit 15, and the inverse quantizer 19.

The quantizer 14 performs quantization on transform coefficient data output from the orthogonal transform unit 13. At this time, the quantizer 14 performs quantization by using the scaling list data 400 transmitted from the scaling list setting unit 17 to be described later. The quantized transform coefficient data is output to the entropy coding unit 15 and the inverse quantizer 19. The bit rate of the quantized data output from the quantizer 14 is controlled based on a rate control signal from the rate controller 18.

The quantizer 14 also quantizes the transform coefficient data generated by the orthogonal transform unit 13 by using the quantization matrix corresponding to the transform unit, based on the information, output from the orthogonal transform unit 13, which indicates the block size of the transform unit selected by the orthogonal transform unit 13.

Further, the quantizer 14 stores only data of a quantization matrix 90 in Fig. 11 and Fig. 12 to be described later, which is determined based on the scaling list data 400 as described above and is actually used in quantization, in a storage device such as a semiconductor memory.

Fig. 10 shows an example where a default quantization matrix is used in the case of H. 265/HEVC. Fig. 10A shows a default quantization matrix with a 4×4 block size which is used in intra prediction mode and inter prediction mode, Fig. 10B shows a default quantization matrix with a block size ranging from 8×8 to 32×32 which is used in intra prediction mode, and Fig. 10C shows a default quantization matrix with a block size ranging from 8×8 to 32×32 which is used in inter prediction mode.

Although Figs. 10B and 10C each show a quantization matrix with a block size of 8×8, H. 265/HEVC defines only quantization matrices with block sizes of 4×4 and 8×8 to reduce the amount of a memory in use.

Fig. 11 shows an example of a quantization matrix 90 which is transmitted to the decoding side via the storage buffer 16 from the quantizer 14, separately from a DC coefficient 91. The quantization matrix 90 in Fig. 11 may be a user-defined quantization matrix other than the default quantization matrices shown in Fig. 10. For the quantization matrices with block sizes of 16×16 and 32×32, a quantization matrix 90 with a block size of 8×8 (for example, the quantization matrix in Fig. 10B or Fig. 10C) is expanded to quantization matrices 93 with a 16×16 block size and a 32×32 block size, respectively, for use in the decoding processing in the image decoding device 50. Fig. 11 shows, as an example, the quantization matrix 93 which is the quantization matrix 90 with a block size of 8×8 expanded to a block size of 16×16 in the image decoding device 50 in a case where the block size of a quantization matrix for use in decoding processing is 16×16.

Fig. 12 shows a default quantization matrix with a block size of 16×16, which is one type of the quantization matrix 90 in Fig. 11 and is generated by expanding the default quantization matrix with a block size of 8×8 shown in Fig. 10B. For the aforementioned, expanded quantization matrix, the processing in the image decoding device 50 is performed as follows:

d [x] [y] =Clip3 (-32768, 32767, ( (TransCoeffLevel [xTbY] [yTbY] [cIdx] [x] [y] *m [x] [y] *levelScal e [qP%6] << (qP/6) + (1<< (bdShift-1) ) ) >>bdShift) where the scaling factor m [x] [y] is derived as follows: if scalling_list_enabled_flag=0, m [x] [y] =16, and otherwise m [x] [y] =ScallingFactor [sizeId] [matrixId] [x] [y] .

The above equation is also defined in H. 265/HEVC. d [x] [y] is a scaled transform coefficient, and may be determined by the equation. The scaling factor m [x] [y] indicates each value in the quantization matrix 93. In this embodiment, while only data of the quantization matrix that is actually used in the coding processing and the decoding processing is transmitted to the decoding side, d [x] [y] can be determined by also using the above equation as defined in H. 265/HEVC.

H. 266, which will be formulated in future, also has proposed block sizes larger than 32×32. In general, a larger block size is applied to a flat area in an image where most of non-zero transform coefficients are in a low-frequency domain, so that with such larger block sizes in use, the transform coefficients are pre-determined as zero.

The entropy coding unit 15 codes the sequence parameter set 200 and the picture parameter set 300 which are associated with image data. Quantized data input from the quantizer 14, the sequence parameter set 200, the picture parameter set 300, the scaling list data 400, and information for generating a quantization matrix on the decoding side, as well as information on intra prediction and inter prediction, are supplied to the entropy coding unit 15. The information on intra prediction may include, for example, prediction mode information indicating an optimal intra prediction mode for each block. Likewise, the information on inter prediction may include, for example, prediction mode information for predicting a block-by-block motion vector, differential motion vector information, and reference image information.

The entropy coding unit 15 performs entropy coding on quantized data to generate a coded bitstream. Coding by the entropy coding unit 15 may be, for example, variable length coding, arithmetic coding or the like. In addition, the entropy coding unit 15 combines information for generating quanti zation matrices within the header of a bitstream (for example, the sequence parameter set 200 and the picture parameter set 300) . Further, the entropy coding unit 15 also combines the aforementioned information on intra prediction and information on inter prediction within the header of a coded bitstream. The entropy coding unit 15 then outputs the generated bitstream to the storage buffer 16.

The storage buffer 16 outputs a bitstream including the sequence parameter set 200 and the picture parameter set 300. The storage buffer 16 temporarily stores the bitstream output from the entropy coding unit 15. The storage buffer 16 then outputs the stored bitstream at a rate matching the bandwidth of the transmission path to the image decoding device. The storage buffer 16 may be constituted by a recording medium such as a semiconductor memory.

The scaling list setting unit 17 sets the scaling list data 400 for a bitstream. This setting may be carried out, for example, through a user input or the like. Further, the scaling list setting unit 17 transmits the scaling list data 400 to the quantizer 14 and the inverse quantizer 19. The transmitted scaling list data 400 is used in quantization and inverse quantization of the transform coefficient data. The scaling list data is also transmitted to the entropy coding unit 15 to be coded so that the scaling list data is referred to within the sequence parameter set 200 or the picture parameter set 300.

The rate controller 18 monitors the free area of the storage buffer 16. Then, the rate controller 18 generates a rate control signal according to the free area of the storage buffer 16, and outputs the generated rate control signal to the quantizer 14. When the free area of the storage buffer 16 is small, for example, the rate controller 18 generates the rate control signal to reduce the bit rate for quantized data. When the free area of the storage buffer 16 is sufficiently large, on the other hand, the rate controller 18 generates the rate control signal to increase the bit rate for quantized data.

The inverse quantizer 19 performs inverse quantization on the quantized data input from the quantizer 14. At this time, the inverse quantizer 19 performs inverse quantization by using the scaling list data transmitted from the scaling list setting unit 17. The inverse quantizer 19 then outputs the transform coefficient data obtained through the inverse quantization to the inverse orthogonal transform unit 20.

The inverse orthogonal transform unit 20 performs inverse quantization on the transform coefficient data input from the inverse quantizer 19 to restore prediction error data. The inverse orthogonal transform unit 20 then outputs the restored prediction error data to the adder 21.

The adder 21 generates decoded image data by adding the restored prediction error data input from the inverse orthogonal transform unit 20 and the predictive image data input from the intra prediction unit 24 or the inter prediction unit 25 together. The generated decoded image data is output to the loop filter 22 and the frame memory 23.

The loop filter 22 performs filtering to reduce coding distortion which is caused at the time of coding an image. The loop filter 22 eliminates the coding distortion by filtering the decoded image data input from the adder 21, and outputs the filtered decoded image data to the frame memory 23.

The frame memory 23 stores the decoded image data input from the adder 21 and the filtered decoded image data input from the loop filter 22. Specifically, the frame memory 23 may be constituted by, for example, a recording medium such as a semiconductor memory. The frame memory 23 supplies the decoded image data before filtering, which is used for intra prediction, as reference image data to the intra prediction unit 24, or supplies the filtered decoded image data, which is used for inter prediction, as reference image data to the inter prediction unit 25.

The intra prediction unit 24 performs intra prediction in each intra prediction mode based on the image data to be coded, input from the re-ordering buffer 11, and the decoded image data supplied from the frame memory 23. For example, the intra prediction unit 24 evaluates the result of the prediction in each intra prediction mode by using a predetermined cost function. The intra prediction unit 24 then selects the intra prediction mode that minimizes the cost function value, i.e., the intra prediction mode that maximizes the compression rate, as an optimal intra prediction mode. Further, the intra prediction unit 24 outputs information related to intra prediction, such as the prediction mode information indicating the optimal intra prediction mode, the predictive image data and the cost function value.

The inter prediction unit 25 performs inter prediction (interframe prediction) based on the image data to be coded, input from the re-ordering buffer 11, and the decoded image data supplied from the frame memory 23. For example, the inter prediction unit 25 evaluates the result of the prediction in each inter prediction mode by using a predetermined cost function. The inter prediction unit 25 then selects the inter prediction mode that minimizes the cost function value, i.e., the inter prediction mode that maximizes the compression rate, as an optimal inter prediction mode. Further, the inter prediction unit 25 generates predictive image data according to the optimal inter prediction mode. Then, the inter prediction unit 25 outputs information related to inter prediction, such as the prediction mode information representing the optimal inter prediction mode, the predictive image data and the cost function value.

The cost function value related to intra prediction output from the intra prediction unit 24 is compared with the cost function value related to inter prediction output from the inter prediction unit 25 to select the intra prediction or the inter prediction, whichever provides a smaller cost function value. When the intra prediction is selected, the information related to intra prediction is output to the entropy coding unit 15, and the predictive image data is output to the subtracter 12 and the adder 21. When the inter prediction is selected, on the other hand, the information related to inter prediction is output to the entropy coding unit 15, and the predictive image data is output to the subtracter 12 and the adder 21.

(Configuration of Image Decoding Device 50)

Fig. 9 is a diagram showing one example of the configuration of the image decoding device 50. The image decoding device 50 includes a storage buffer 51, an entropy decoding unit 52, an inverse quantizer 53, an inverse orthogonal transform unit 54, an adder 55, a loop filter 56, and a re-ordering buffer 57, a frame memory 58, an intra prediction unit 59, an inter prediction unit 60, and a scaling list buffer 61. The image decoding device 50 basically performs a process opposite to the process performed in the image coding device 10 to restore original video data.

The storage buffer 51 temporarily stores an input bitstream input from the image coding device 10 over the transmission path 3. The storage buffer 51 may be constituted by, for example, a recording medium such as a semiconductor memory.

The entropy decoding unit 52 decodes the input bitstream input from the storage buffer 51. The entropy decoding unit 52 refers to the

syntax elements

201 and 202, and 301 and 302 respectively included in the sequence parameter set 200 and the picture parameter set 300. The entropy decoding unit 52 decodes information combined in the header area of the input bitstream. The information combined in the header area of the input bitstream may include, for example, the sequence parameter set 200, the picture parameter set 300, and the scaling list data 400, mentioned above, the basic matrix information and differential matrix information for generating a quantization matrix, and information related to intra prediction and information related to inter prediction, which are located within a block header. The entropy decoding unit 52 outputs quantized data after decoding and information for generating a quantization matrix to the inverse quantizer 53. The entropy decoding unit 52 also outputs information related to intra prediction to the intra prediction unit 59. The entropy decoding unit 52 further outputs information related to inter prediction to the inter prediction unit 60. The decoded scaling list data 400 is stored in the scaling list buffer 61.

The scaling list buffer 61 stores the scaling list data 400 decoded from the input bitstream in the entropy decoding unit 52. Based on the values of "sizeId" and "matrixId" in the scaling list data 400 for the TU currently being processed, the quantization matrix to be used in inverse quantization is transmitted to the inverse quantizer 53. The transmitted scaling list data 400 is used in inverse quantization in the inverse quantizer 53.

The inverse quantizer 53 and the inverse orthogonal transform unit 54 generate prediction error data by respectively performing inverse quantization and inverse orthogonal transform on quantized data input from the entropy decoding unit 52. Specifically, the inverse quantizer 53 performs inverse quantization by using the scaling list data 400 transmitted from the scaling list buffer 61. The inverse orthogonal transform unit 54 outputs the generated prediction error data to the adder 55. The inverse quantizer 53 and the inverse orthogonal transform unit 54 perform inverse processes to the processes performed by the orthogonal transform unit 13 and the quantizer 14 in the image coding device 10. That is, the inverse quantizer 53 and the inverse orthogonal transform unit 54 perform inverse quantization and inverse orthogonal transform by using the sequence parameter set 200 and the picture parameter set 300 corresponding to a sequence or a picture to be processed.

The adder 55 generates decoded image data by adding the prediction error data input from the inverse orthogonal transform unit 54 and the predictive image data input from the intra prediction unit 59 or the inter prediction unit 60. The generated decoded image data is output to the loop filter 56 and the frame memory 58.

The loop filter 56 eliminates the coding di stortion by filtering the decoded image data input from the adder 55, and outputs the filtered decoded image data to the re-ordering buffer 57 and the frame memory 58.

The re-ordering buffer 57 re-orders the images input from the loop filter 56 to generate a sequence of time-sequential image data. The image data generated by the re-ordering buffer 57 is output to the image reproducing device 4 for display.

The frame memory 58 stores the unfiltered, decoded image data input from the adder 55 and the filtered decoded image data input from the loop filter 56. Specifically, the frame memory 58 may be constituted by, for example, a recording medium such as a semiconductor memory.

(Processing Flow of Image Coding)

Fig. 13 is a flow diagram illustrating one example of the coding processing in the image coding device 10. This flow includes: generating a sequence parameter set 200 and a picture parameter set 300, which are associated with image data, and both respectively include a

syntax element

201, 301 indicating a minimum block size to be subjected to orthogonal transform and a

syntax element

202, 302 indicating a maximum block size to be subjected to orthogonal transform; and outputting a bitstream including the sequence parameter set 200 and the picture parameter set 300. The processing exemplified in Fig. 13 is performed mainly in the scaling list setting unit 17.

First, as the

syntax elements

201 and 202 in the sequence parameter set 200 in Fig. 2, values of "sps_log2_min_transform_block_size_minus2" and "sps_log2_diff_max_min_transform_block_size" are set according to a user input or the like. Further, as the

syntax elements

301 and 302 in the picture parameter set 300 in Fig. 3, values of "pps_log2_min_transform_block_size_minus2" and "pps_log2_diff_max_min_transform_block_size" are set according to a user input or the like (step S102) . Based on the values of the

syntax elements

201, 202, 301 and 302, values of "min_sizeId" and "max_sizeId" (

parameters

401 and 402 in the scaling list data 400 in Fig. 4) are determined (step S104) .

The processes of the following steps S108 to S124 are repeated by incrementing a variable i (corresponding to "sizeId" in the scaling list data 400) by one at a time while the value of the variable i is equal to or greater than "min_sizeId" and is equal to or less than "max_sizeId" (step S106) . (For example, according to H. 265/HEVC, the values of the variable i=0, 1, 2, 3 determine the block sizes of 4×4, 8×8, 16×16, and 32×32, respectively. )

When the value of "matrixId" is equal to or greater than "0" and equal to or less than "6" (in case of sizeId=3, however, "matrixId" is equal to or greater than "0" and equal to or less than "2" ) in the loop process of step S106, the processes of the following steps S110 to S124 are repeated (step S108) .

Through the processes of steps S110 to S124, data of quantization matrices having block sizes corresponding to the range of values equal to or greater than the value of "min_sizeId" and equal to or less than the value of "max_sizeId" can be transmitted to the image decoding device 50. As described earlier, in the example of H. 265/HEVC, for the block sizes of 4×4, 8×8, and 16×16, "matrixId" represents a variable for obtaining six types of matrices. For the block size of 32×32, "matrixId" represents a variable for obtaining two types of matrices.

In the loop process starting at step S108, for each of the combinations of the values of "sizeId" and "matrixId" , first, it is determined whether a flat matrix is applied to the combination of the values of "sizeId" and "matrixId" or not (control for quantization is performed or not) (step S110) . When the flat matrix is applied (step S110: Yes) , the flag 403 in the scaling list data 400 in Fig. 4 is set as "0" (scaling_list_comp_enabled_flag [sizeId] [matrixId] =0) (step S112) . When the flat matrix is not applied (frequency-by-frequency control is performed) (step S110: No) , however, the flag 403 is set as "1" (scaling_list_comp_enabled_flag [sizeId] [matrixId] =1) (step S114) .

Following step S114, it is then determined whether a default matrix is applied or not (step S116) . When the default matrix is applied (step S116: Yes) , the flag 404 in the scaling list data 400 in Fig. 4 is set as "0" (scaling_list_comp_present_flag [sizeId] [matrixId] =0) (step S118) . When the default matrix is not applied (step S116: No) , however, the flag 404 is set as "1" (scaling_list_comp_present_flag [sizeId] [matrixId] =1) (step S120) .

It is further determined whether the value of the TU specified by [sizeId] [matrixId] after orthogonal transform is "0" or not (step S122) . When this value is "0" (step S122: Yes) , the scaling list data 400 is not transmitted, and the processing returns to step S106 or S108. In this case, since the scaling list data 400 is not transmitted, transmission of data from the image coding device 10 may be suppressed. This can suppress the amount of transmission data in the case of performing coding on a relative large block size.

When the value of the TU specified by [sizeId] [matrixId] after orthogonal transform is not "0" (step S122: No) , on the other hand, the scaling list data 400 is transmitted to the quantizer 14, the entropy coding unit 15 and the inverse quantizer 19 (step S124) .

Thereafter, coding of the input image signal is performed using the determined scaling list data 400 by other modules, such as the orthogonal transform unit 13, the quantizer 14, and the entropy coding unit 15.

(Processing Flow of Image Decoding)

Fig. 14 is a flow diagram illustrating one example of the decoding processing in the image decoding device 50. This flow includes: receiving a bitstream, the bitstream including a sequence parameter set 200 and a picture parameter set 300, which are associated with the image data, and both respectively include a

first syntax element

second syntax element

202, 302 indicating a maximum block size to be subjected to orthogonal transform; and decoding the bitstream, in which the

first syntax element

201, 301 and the

second syntax element

202, 302 respectively included in the sequence parameter set 200 and the picture parameter set 300 are referred to.

First, the input bitstream received from the image coding device 10 is temporarily stored in the storage buffer 51. The entropy decoding unit 52 decodes the sequence parameter set 200, the picture parameter set 300, and the scaling list data 400 in the input bitstream, as well as other various kinds of data. The decoded scaling list data 400 is stored in the scaling list buffer 61. In this manner, the syntax elements 201 and 202 ( "sps_log2_min_transform_block_size_minus2" and "sps_log2_diff_max_min_transform_block_size" in Fig. 2) of the sequence parameter set 200 and the syntax elements 301 and 302 ( "pps_log2_min_transform_block_size_minus2" and "pps_log2_diff_max_min_transform_block_size" in Fig. 2) of the picture parameter set 300 are received (step S202) . Based on the values of the

syntax elements

201 and 202 or the

syntax elements

301 and 302, the values of "min_sizeId" and "max_sizeId" (the

parameters

401 and 402 in the scaling list data 400 in Fig. 4) are determined (step S204) . The subsequent processes are performed mainly by the inverse quantizer 53.

The processes of the following steps S208 to S220 are repeated by incrementing the variable i (corresponding to "sizeId" in the scaling list data 400) by one at a time while the value of the variable i is equal to or greater than "min_sizeId" and is equal to or less than "max_sizeId" (step S206) . (For example, according to H. 265/HEVC, the variable i may take the values of 0 to 3. In this case, i=0, 1, 2, 3 may correspond to the block sizes of 4×4, 8×8, 16×16, and 32×32, respectively. )

When the value of "matrixId" is equal to or greater than "0" and equal to or less than "6" (in case of sizeId=3, however, "matrixId" is equal to or greater than "0" and equal to or less than "2" ) while the value is incremented by "1" at a time in the loop process of step S206, the processes of the following steps S210 to S220 are repeated (step S208) .

In the loop process starting at step S208, for each of the combinations of the values of "sizeId" and "matrixId" , it is determined whether or not the flag 403 in the scaling list data 400 in Fig. 4 is "0" (scaling_list_comp_enabled_flag [sizeId] [matrixId] =0) (step S210) . When the flag 403 is "0" (step S210: Yes) , a flat scaling list is applied for the combination of the values of "sizeId" and "matrixId" in this case (step S212) . When the flag 403 is not "0" (scaling_list_comp_enabled_flag [sizeId] [matrixId] =0) (step S210: No) , it is determined whether or not the flag 404 in the scaling list data 400 in Fig. 4 is "0" (scaling_list_comp_present_flag [sizeId] [matrixId] =0) (step S214) . When the flag 404 is "0" (step S214: Yes) , a default scaling list is applied for the combination of the values of "sizeId" and "matrixId" in this case (step S212) . When the flag 404 is not "0" (step S214: No) , it is further determined whether the value of the TU specified by [sizeId] [matrixId] after orthogonal transform is "0" or not (step S218) . When this value is "0" (step S218: Yes) , the scaling list data 400 is not received, and the processing returns to step S206 or S208. This may further reduce the amount of transmission/reception data, which is particularly useful for large block sizes. When the value of the TU specified by [sizeId] [matrixId] after orthogonal transform is not "0" (step S2118: No) , however, quantization matrix data is received by the inverse quantizer 53 (step S220) . Thereafter, the inverse quantizer 53 performs inverse quantization by using the received quantization matrix data, after which the processing in the inverse orthogonal transform unit 54 is performed.

Although the possible values of the variable i are 0 to 3 in the foregoing embodiment, the values have just been given in conformity with H. 265/HEVC for the sake of simple description. The possible values of the variable i may vary according to the image coding scheme to which the image processing system according to the present embodiment is applied. The same is true of "sizeId" and "matrixId" .

In the foregoing embodiment, two syntax elements "log2_min_transform_block_size_minus2" and "log2_diff_max_min_transform_block_size" not only in the sequence parameter set but also in the picture parameter set are transmitted. In addition, which quantization matrix in the bitstream is to be transmitted may be determined based on the values of those two syntax elements.

According to the foregoing embodiment, it is possible to avoid redundant transmission of quantization matrix data at the time of transmitting the scaling list data in the bitstream to the image decoding device, thereby enhancing the coding efficiency.

According to the foregoing embodiment, the number of bits needed for each component of the scaling list is 8 bits in the worst case, and the number of bits needed to transmit scaling list data is 8000 bits in total. However, when a user-defined scaling list is applied for the 4×4 block size× (Y, Cb, Cr) × (intra prediction, inter prediction) , and a flat scaling list is applied for the other block sizes, the number of necessary bits is only 782 bits.

For future image coding standards such as H. 266, a larger DCT size of 64×64, for example, may be adopted. The image processing system and the image processing method according to the foregoing embodiment can be applied to such future image coding standards. Further, the quantization matrix may be designed to be transmitted only for low-frequency components whose values are not set to zero in advance (see step S122 in Fig. 13 and step S218 in Fig. 14) . This modification is particularly useful for future image coding schemes for which larger block sizes may be adopted.

According to the configuration of the foregoing embodiment, log2_min_transform_block_size_minus2 and log2_diff_max_min_transform_block_size in both of the sequence parameter set 200 and the picture parameter set 300 are transmitted. Accordingly, when a commercial video in active motion is broadcasted in a TV program in not active motion, for example, it is possible to implement an application of designating quantization matrices of small block sizes in the sequence parameter set at the top of the TV program and designating quantization matrices of large block sizes in the picture parameter set at the beginning of the commercial video in active motion.

It is to be noted that min_sizeId and max_sizeId may be defined separately in intra prediction mode and in inter prediction mode. That is, min_sizeId and max_sizeId may be set in such a way that, for example, while 4×4, 8×8, and 16×16 can be applied to min_sizeId and max_sizeId in intra prediction mode, only 8×8 and 16×16 can be applied to min_sizeId and max_sizeId in inter prediction mode.

Fig. 15 is a diagram showing one example of the hardware configuration of a control device 100 to which the image coding device 10 and the image decoding device 50 are applied.

The control device 10 includes a processor 100a, a memory 100b, a display interface 100c, a communication interface 100d with the communication path 3, and a connection interface 100e to a recording medium 100f.

The processor 100a is connected to the individual components by buses, and performs a process of transferring control signals and data. The processor 100a also runs various programs for achieving the general operation of the control device 100 and performs arithmetic operations, timing control and the like. The programs may be stored in the computer readable recording medium 100f such as DVD-ROM (Read Only Memory) or CD (Compact Disk) -ROM.

The memory 100b includes a ROM and RAM (Random Access Memory) . The memory 100b may store programs for the operating system needed for the general operational control of the control device 100 and various kinds of data. The memory 100b may be provided with a recording area for temporarily storing data and a program, and retains such a program and data, and other data necessary for the processing of the control device 10.

A display device which is connected to the display interface 100c may be a flat panel display, such as a liquid crystal display or EL (Electro-Luminescence) display.

The individual functions of the image coding device 10, the image decoding device 50, and the image reproducing device 4 according to the foregoing embodiment may be implemented by reading the program stored in, for example, the computer readable recording medium 100 f of the computer device 100 shown in Fig. 15 and running the program by means of the processor 100a. In other words, a computer program that achieves the processes in the foregoing image coding device 10 and image decoding device 50 also falls within the protection scope of the present invention. Further, a computer readable recording medium on which the computer program is stored also falls within the protection scope of the present invention.

Although one embodiment has been described so far, the present invention is not limited to the foregoing embodiment, and may of course be worked out in various different modes within the scope of the technical concept of the present invention.

The scope of the present invention is not limited to the example embodiment illustrated and described, and shall include all embodiments that bring about advantages equivalent to intended advantages of the present invention. Moreover, it is to be noted that the scope of the present invention is not limited to combinations of the features of the invention as set forth in the individual claims, but may be specified by any desirable combination of specific features among all the disclosed features of the present invention.

Claims

A method for coding image data, comprising:

generating a sequence parameter set and a picture parameter set which are associated with the image data, and both include a first syntax element indicating a minimum block size to be subjected to orthogonal transform and a second syntax element indicating a maximum block size to be subjected to orthogonal transform; and

outputting a bitstream including the sequence parameter set and the picture parameter set.
The method according to claim 1, wherein the first syntax element is log2_min_transform_block_size_minus2, and the second syntax element is log2_diff_max_min_transform_block_size.
The method according to claim 2, further comprising determining a quantization matrix on which block sizes to which orthogonal transform is applied are transmitted based on the minimum and maximum block sizes indicated by the first and second syntax elements.
The method according to any one of claims 1 to 3, further comprising independently designating, in the bitstream, whether a flat quantization matrix is to be applied to each of combinations of values of a variable indicating block sizes in the orthogonal transform and values of a variable designating quantization matrices applicable to the block sizes.
The method according to any one of claims 1 to 4, further comprising independently designating, in the bitstream, whether a default quantization matrix is to be applied to each of combinations of values of a variable indicating block sizes in the orthogonal transform and values of a variable designating quantization matrices applicable to the block sizes.
The method according to any one of claims 1 to 5, wherein in the outputting the bitstream, quantization matrix data is sent only for an orthogonal transform coefficient which is not preset to zero.
A method for decoding image data, comprising:

receiving a bitstream, the bitstream including a sequence parameter set and a picture parameter set which are associated with the image data, and both include a first syntax element indicating a minimum block size to be subjected to orthogonal transform and a second syntax element indicating a maximum block size to be subjected to orthogonal transform; and

decoding the bitstream, in which the first and second syntax elements included in the sequence parameter set and the picture parameter set are referred to.
The method according to claim 7, wherein the first syntax element is log2_min_transform_block_size_minus2, and the second syntax element is log2_diff_max_min_transform_block_size.
The method according to claim 8, further comprising determining a quantization matrix on which block sizes to which orthogonal transform is applied are received based on the minimum and maximum block sizes indicated by the first and second syntax elements.
The method according to any one of claims 7 to 9, wherein whether a flat quantization matrix is to be applied to each of combinations of variables indicating block sizes in the orthogonal transform and variables designating quantization matrices applicable to the block sizes is independently designated in the bitstream.
The method according to any one of claims 7 to 10, wherein whether a default quantization matrix is to be applied to each of combinations of variables indicating block sizes in the orthogonal transform and variables designating quantization matrices applicable to the block sizes is independently designated in the bitstream.
The method according to any one of claims 7 to 11, wherein quantization matrix data is included in the bitstream only for an orthogonal transform coefficient which is not preset to zero.
An image coding device, comprising:

a function unit for coding a sequence parameter set and a picture parameter set which are associated with the image data, and both include a first syntax element indicating a minimum block size to be subjected to orthogonal transform and a second syntax element indicating a maximum block size to be subjected to orthogonal transform; and

an output unit for outputting a bitstream including the sequence parameter set and the picture parameter set.
An image decoding device, comprising:

a buffer unit for receiving a bitstream, the bitstream including a sequence parameter set and a picture parameter set which are associated with the image data, and both include a first syntax element indicating a minimum block size to be subjected to orthogonal transform and a second syntax element indicating a maximum block size to be subjected to orthogonal transform; and

a decoding unit for decoding the bitstream, the decoding unit referring to the first and second syntax elements included in the sequence parameter set and the picture parameter set.
A program for allowing a computer device to execute the method as set forth in any one of claims 1 to 12.
A computer readable recording medium storing a program for allowing a computer device to execute the method as set forth in any one of claims 1 to 12.
An image processing system comprising:

the image coding device as set forth in claim 13; and

the image decoding device as set forth in claim 14.