KR20130084236A - Image processing device and method - Google Patents

Abstract

TECHNICAL FIELD This technique relates to the image processing apparatus and method which make it possible to perform quantization processing or inverse quantization processing which are more suitable for the content of an image. The reversible decoding unit 202 decodes the encoded data read out from the accumulation buffer 201 at a predetermined timing. The sub macroblock inverse quantization unit 221 obtains the quantization value for each sub macroblock using the quantization parameter supplied from the inverse quantization unit 203, and returns it to the inverse quantization unit 203. The inverse quantization unit 203 dequantizes the quantization coefficient obtained by decoding by the reversible decoding unit 202 using the quantization value for each sub macroblock supplied from the sub macroblock dequantization unit 221. The present technology can be applied to, for example, an image processing apparatus.

Description

[0001] IMAGE PROCESSING DEVICE AND METHOD [0002]

TECHNICAL FIELD This technique relates to an image processing apparatus and method, and relates to the image processing apparatus and method which perform a quantization process or a dequantization process.

Recently, MPEG is dealt with image information as digital, and then compressed by orthogonal transform and motion compensation such as discrete cosine transform, using redundancy peculiar to image information for the purpose of transferring and accumulating highly efficient information. A device based on a method such as (Moving Picture Experts Group) is widely used both for distributing information such as broadcasting stations and receiving information in a general household.

Nowadays, in order to further compress high-compression ratios, such as to compress an image of about 4096x2048 pixels, which is four times higher than a high-vision image, or to transmit a high-vision image in an environment with limited transmission capacity, such as the Internet. The demand for it is increasing. For this reason, the VCEG under ITU-T continues to consider improvement of the coding efficiency.

The macroblock which is a partial region of an image which becomes a division unit (encoding processing unit) of the image at the time of image coding in MPEG1, MPEG2, and ITU-T H.264, MPEG4-AVC which is the conventional image coding system. All pixel sizes were 16x16 pixels. On the other hand, according to Non-Patent Document 1, as an element technology of the next-generation image coding standard, a proposal has been made to extend the number of pixels in the horizontal and vertical directions of a macroblock. According to this proposal, in addition to the pixel size of the 16 × 16 pixel macroblock specified in MPEG1, MPEG2, and ITU-T H.264, MPEG4-AVC, etc., a macroblock consisting of 32 × 32 pixels and 64 × 64 pixels is used. It is also proposed. This is expected to increase the pixel size in the horizontal and vertical directions of an image to be encoded in the future, for example, UHD (Ultra High Definition (4000 pixels x 2000 pixels), but in that case, the motions are similar to each other. In the area, the object is to improve the coding efficiency by performing motion compensation and orthogonal transformation in units of a larger area.

In the non-patent document 1, by employing a hierarchical structure, a larger block is defined as a superset of 16 × 16 pixel blocks or less while maintaining compatibility with the macroblock in the current AVC.

Non-Patent Document 1 proposes to apply an extended macroblock to an inter slice, but in Non-Patent Document 2, it is proposed to apply an extended macroblock to an intra slice.

: Peisong Chenn, Yan Ye, Marta Karczewicz, "Video Coding Using Extended Block Sizes", COM16-C123-E, Qualcomm Inc : Sung-Chang Lim, Hahyun Lee, Jinho Lee, Jongho Kim, Haechul Choi, Seyoon Jeong, Jin Soo Choi, "Intra coding using extended block size", VCEG-AL28, July 2009

By the way, when macroblocks of extended sizes, as proposed in Non-Patent Document 1 or Non-Patent Document 2, are applied, there is a high possibility that flat regions and regions containing textures are mixed in a single macro block. .

However, in Non-Patent Document 1 or Non-Patent Document 2, since only a single quantization parameter can be specified for one macroblock, it is necessary to perform adaptive quantization according to the characteristics of each area in the plane. It becomes difficult.

This technology is made in view of such a situation, and aims at suppressing the reduction of subjective image quality of a decoded image by performing more appropriate quantization process.

One aspect of the present technology is directed to a decoding unit that decodes an encoded stream to generate quantized data, and a coding unit that is lower than a reference coding unit in a reference layer of a coding unit that is a coding processing unit when encoding image data. A quantization parameter used for inverse quantization of the quantized data generated by the decoder, and a quantization data generated by the decoder using the quantization parameter set by the setting unit. An image processing apparatus including an inverse quantization unit to quantize.

The setting unit uses the current coding unit by using a differential quantization parameter representing a difference value between a quantization parameter set in the current coding unit to be subjected to inverse quantization processing and a quantization parameter set in the coding unit in the same layer as the current coding unit. The quantization parameter of can be set.

The difference quantization parameter may be a difference value between the quantization parameter set in the current coding unit and the quantization parameter set in the coding unit decoded before the current coding unit in decoding processing order.

The differential quantization parameter may be a difference value between the quantization parameter set in the current coding unit and the quantization parameter set in the coding unit decoded one time before the current coding unit in decoding processing order.

The reference coding unit may be a maximum coding unit that is a coding unit of a top layer.

A receiving unit which receives the minimum coding unit size data indicating the minimum size of the coding unit for setting the coded stream and the differential quantization parameter, wherein the setting unit according to the minimum coding unit size data received by the receiving unit The quantization parameter of the current coding unit may be set.

The receiving unit may acquire the minimum coding unit size data from a slice header of the encoded stream.

When the size indicated by the minimum coding unit size data is 16 pixels, the differential quantization parameter of the coding unit whose size is less than 16 pixels can be set to zero.

The setting unit sets a quantization parameter of the current coding unit by using a differential quantization parameter representing a difference value between the quantization parameter set in the current coding unit to be decoded and the quantization parameter set in the slice to which the current coding unit belongs. Can be.

The setting unit, when the current coding unit is the first in a decoding processing order in the hierarchy of the reference coding unit, sets the difference value between the quantization parameter set in the current coding unit and the quantization parameter set in the slice to which the current coding unit belongs. The quantization parameter of the current coding unit can be set using the differential quantization parameter indicated.

The reference coding unit may be a maximum coding unit that is a coding unit of a top layer.

A receiving unit which receives the minimum coding unit size data indicating the minimum size of the coding unit for setting the coded stream and the differential quantization parameter, wherein the setting unit according to the minimum coding unit size data received by the receiving unit The quantization parameter of the current coding unit may be set.

The receiving unit may acquire the minimum coding unit size data from a slice header of the encoded stream.

When the size indicated by the minimum coding unit size data is 16 pixels, the differential quantization parameter of the coding unit whose size is less than 16 pixels can be set to zero.

The setting unit targets a coding unit that is lower than the reference coding unit, and when the value of the differential quantization parameter is 0, the coding unit having the quantization parameter set in the reference coding unit lower than the reference coding unit. It can be set as a quantization parameter set in.

The receiver further includes a receiving unit that receives differential identification data identifying whether the value of the differential quantization parameter is 0, targeting a coding unit that is lower than the reference coding unit, and the setting unit receives the difference received by the receiving unit. Using the identification data, the quantization parameter set in the reference coding unit can be set as the quantization parameter set in the coding unit lower than the reference coding unit.

One aspect of the present technology further targets a coding unit in a lower layer than a reference coding unit in a reference layer of a coding unit which is a coding processing unit when decoding a coded stream to generate quantized data and encoding image data. And an quantization parameter used for inverse quantization of the generated quantized data, and an inverse quantization of the generated quantized data using the set quantization parameter.

According to another aspect of the present technology, a quantization parameter used when quantizing image data is set for a coding unit that is lower than a reference coding unit in a reference layer of a coding unit that is a coding processing unit when encoding image data. A quantizer for quantizing image data to generate quantized data using the quantization parameter set by the setter, and a encoder for encoding a quantized data generated by the quantizer to generate an encoded stream. It is an image processing apparatus provided.

The setting unit sets a differential quantization parameter indicating a difference value between the quantization parameter set in the current coding unit to be subjected to the coding process and the quantization parameter set in the coding unit in the same layer as the current coding unit, and by the setting unit The apparatus may further include a transmitter configured to transmit the set differential quantization parameter and the encoded stream generated by the encoder.

The setting unit may set, as the difference quantization parameter, a difference value between the quantization parameter set in the current coding unit and the quantization parameter set in the coding unit encoded before the current coding unit in the coding processing order.

The setting unit may set, as the difference quantization parameter, a difference value between the quantization parameter set in the current coding unit and the quantization parameter set in the coding unit coded one earlier in the coding processing order than the current coding unit.

The reference coding unit may be a maximum coding unit that is a coding unit of a top layer.

The setting unit may set minimum coding unit size data indicating a minimum size of a coding unit for setting the differential quantization parameter, and the transmitting unit may transmit minimum coding unit size data set by the setting unit.

The transmission unit may add, as a slice header, the minimum coding unit size data set by the setting unit to the syntax of the encoded stream generated by the encoding unit.

In the case where the size indicated by the minimum coding unit size data is set to 16 pixels, the setting unit may set a differential quantization parameter of a coding unit whose size is less than 16 pixels to zero.

The setting unit sets a differential quantization parameter indicating a difference value between the quantization parameter set in the current coding unit to be subjected to the coding process and the quantization parameter set in the slice to which the current coding unit belongs, and the differential quantization parameter set by the setting unit. And a transmitter for transmitting the encoded stream generated by the encoder.

The setting unit, when the current coding unit is the first in the coding processing order in the layer of the reference coding unit, sets the difference value between the quantization parameter set in the current coding unit and the quantization parameter set in the slice to which the current coding unit belongs. Can be set as the differential quantization parameter.

The reference coding unit may be a maximum coding unit that is a coding unit of a top layer.

The setting unit may set minimum coding unit size data indicating a minimum size of a coding unit for setting the differential quantization parameter, and the transmitting unit may transmit minimum coding unit size data set by the setting unit.

The transmission unit may add, as a slice header, the minimum coding unit size data set by the setting unit to the syntax of the encoded stream generated by the encoding unit.

When the size indicated by the minimum coding unit size data is set to 16 pixels, the setting unit may set a differential quantization parameter of a coding unit whose size is less than 16 pixels to 0.

The setting unit includes a quantization parameter set in the reference coding unit lower than the reference coding unit when the value of the differential quantization parameter is set to 0 for a coding unit located lower than the reference coding unit. It can be set as a quantization parameter set in the coding unit.

The setting unit sets difference identification data for identifying whether the value of the differential quantization parameter is 0, targeting a coding unit located lower than the reference coding unit, and sets the difference identification data and the coding unit set by the setting unit. The apparatus may further include a transmission unit configured to transmit the encoded stream generated by the PDU.

According to another aspect of the present technology, a quantization parameter used when quantizing image data is aimed at a coding unit that is lower than a reference coding unit in a reference layer of a coding unit that is a coding processing unit when encoding image data. And a quantization parameter to quantize the image data to generate quantized data, and encode the generated quantized data to generate an encoded stream.

In one aspect of the present technology, a coding unit is decoded to generate quantized data, and a coding unit that is lower than a reference coding unit in a reference layer of a coding unit, which is a coding processing unit when image data is encoded, is targeted. A quantization parameter used when the generated quantized data is inverse quantized is set, and the set quantization parameter is used to dequantize the generated quantized data.

According to another aspect of the present technology, a quantization parameter used when quantizing image data is applied to a coding unit that is lower than a reference coding unit in a reference layer of a coding unit that is a coding processing unit when encoding image data. The set and set quantization parameters are used to quantize the image data to generate quantized data, and the generated quantized data is encoded to generate an encoded stream.

According to the present technology, the quantization process or the inverse quantization process can be performed more appropriately.

1 is a block diagram illustrating a main configuration example of a picture coding apparatus to which the present technology is applied.
2 is a diagram showing an example of a correspondence relationship between a quantization parameter of a luminance signal and a quantization parameter of a chrominance signal.
3 is a diagram illustrating an example of a macro block.
4 is a diagram illustrating another example of a macro block.
5 is a block diagram illustrating a detailed configuration example of a quantization unit.
6 is a diagram for explaining an example of an image in macroblock units.
7 is a flowchart for explaining an example of the flow of an encoding process.
8 is a flowchart for explaining an example flow in a quantization parameter calculation process.
9 is a block diagram illustrating a main configuration example of an image decoding device to which the present technology is applied.
10 is a block diagram illustrating a detailed configuration example of an inverse quantization unit.
11 is a flowchart for explaining an example of the flow of a decoding process.
12 is a flowchart for explaining an example of the flow of inverse quantization processing.
13 is a flowchart for explaining another example of the flow of a quantization parameter calculation process.
14 is a flowchart for explaining another example of the flow of inverse quantization processing.
15 is a diagram illustrating a configuration example of a coding unit.
FIG. 16 shows an example of quantization parameters allocated to each coding unit. FIG.
17 is a diagram illustrating an example of syntax.
18 is a block diagram illustrating another configuration example of a picture coding apparatus to which the present technology is applied.
19 is a block diagram illustrating a detailed configuration example of a coding unit quantization unit and a rate control unit.
20 is a flowchart for explaining another example of the flow of a quantization parameter calculation process;
21 is a block diagram showing another configuration example of an image decoding device to which the present technology is applied.
22 is a block diagram illustrating a detailed configuration example of a coding unit dequantization unit.
FIG. 23 is a flowchart showing still another example of the flow of inverse quantization processing; FIG.
Fig. 24 is a diagram comparing the characteristics of each calculation method of the quantization parameter dQP.
25 is a diagram illustrating an example of quantization parameters allocated for each coding unit.
Fig. 26 is a diagram illustrating an example of syntax of a slice header.
27 is a diagram for explaining an example of an activity calculation method.
28 is a diagram illustrating a relationship between a quantization parameter and a quantization scale.
29 is a block diagram showing yet another example of the configuration of a picture coding apparatus to which the present technology is applied.
30 is a block diagram illustrating a detailed configuration example of a coding unit quantization unit, a quantization unit, and a rate control unit;
31 is a flowchart for explaining another example of the flow of an encoding process.
32 is a flowchart for explaining an example flow in a quantization process.
33 is a diagram illustrating an example of a multi-view picture coding method.
34 is a diagram illustrating an example of a main configuration of a multiview image coding apparatus to which the present technology is applied.
35 is a diagram illustrating a main configuration example of a multiview image decoding device to which the present technology is applied.
36 is a diagram illustrating an example of a hierarchical image coding scheme.
37 is a diagram illustrating an example of a main configuration of a hierarchical image coding apparatus to which the present technology is applied.
38 is a diagram illustrating a main configuration example of a hierarchical image decoding device to which the present technology is applied.
Fig. 39 is a block diagram showing a main configuration example of a computer to which the present technology is applied.
40 is a block diagram illustrating a main configuration example of a television device to which the present technology is applied.
Fig. 41 is a block diagram showing a main configuration example of a mobile terminal to which the present technology is applied.
Fig. 42 is a block diagram showing a main configuration example of a recording and playback apparatus to which the present technology is applied.
43 is a block diagram illustrating a main configuration example of an imaging device to which the present technology is applied.

EMBODIMENT OF THE INVENTION Hereinafter, the form (henceforth embodiment) for implementing this technique is demonstrated. The description will be made in the following order.

1. First Embodiment (Picture Coding Apparatus)

2. Second Embodiment (Image Decoding Apparatus)

3. Third embodiment (image coding apparatus, image decoding apparatus)

4. Fourth embodiment (image coding apparatus, image decoding apparatus)

5. Fifth Embodiment (Image Coding Device)

6. Sixth embodiment (multi-view image coding / multi-view image decoding device)

7. Seventh embodiment (hierarchical image coding / layer image decoding device)

8. 8th Embodiment (Application Example)

<1. First embodiment>

[Image coding device]

Fig. 1 shows a configuration of an embodiment of an image coding apparatus as an image processing apparatus to which the present technology is applied.

The picture coding apparatus 100 shown in FIG. 1 is, for example, H.264 and MPEG (Moving Picture Experts Group) 4 Part 10 (AVC (Advanced Video Coding)) (hereinafter referred to as H.264 / AVC) schemes. Similarly, it is an encoding device for encoding an image. However, the image encoding device 100 specifies quantization parameters for each sub macroblock.

A macroblock is a partial area of the said image used as a processing unit at the time of encoding an image. A sub macro block is a small area which divides the macro block into a plurality.

In the example of FIG. 1, the image encoding apparatus 100 includes an A / D (Analog / Digital) conversion unit 101, a screen rearrangement buffer 102, an operation unit 103, an orthogonal conversion unit 104, and quantization. A unit 105, a reversible encoder 106, and an accumulation buffer 107 are provided. In addition, the image encoding apparatus 100 includes an inverse quantization unit 108, an inverse orthogonal transform unit 109, an operation unit 110, a deblocking filter 111, a frame memory 112, a selection unit 113, and an intra. The predictor 114, the motion predictor / compensator 115, the selector 116, and the rate controller 117 are provided.

The picture coding apparatus 100 further includes a sub macroblock quantization unit 121 and a sub macroblock dequantization unit 122.

The A / D conversion unit 101 performs A / D conversion on the input image data, outputs it to the screen rearrangement buffer 102, and stores it.

The screen rearrangement buffer 102 rearranges the images of frames stored in the display order in the order of the frames for encoding according to the GOP (Group of Picture) structure. The screen rearrangement buffer 102 supplies the image rearranging the order of the frames to the calculation unit 103. In addition, the screen rearrangement buffer 102 supplies the image rearranged in the order of the frames to the intra predictor 114 and the motion predictor / compensator 115.

The calculation unit 103 subtracts the prediction image supplied from the intra prediction unit 114 or the motion prediction / compensation unit 115 through the selection unit 116 from the image read out from the screen rearrangement buffer 102. The difference information is output to the orthogonal transformation unit 104.

For example, in the case of an image to which intra coding is performed, the calculating part 103 subtracts the predictive image supplied from the intra predicting part 114 from the image read out from the screen rearrangement buffer 102. For example, in the case of an image subjected to inter encoding, the calculation unit 103 subtracts the prediction image supplied from the motion prediction / compensation unit 115 from the image read out from the screen rearrangement buffer 102. do.

The orthogonal transform unit 104 performs orthogonal transform such as discrete cosine transform, Karuneen-Lube transform, and the like on the difference information supplied from the calculating unit 103, and supplies the transform coefficient to the quantization unit 105.

The quantization unit 105 quantizes the transform coefficients output from the orthogonal transformation unit 104. [ The quantization unit 105 sets the quantization parameter for each sub macroblock that is an area smaller than the macro block while cooperating with the sub macroblock quantization unit 121 based on the information supplied from the rate control unit 117, and performs quantization. Do it. The quantization unit 105 supplies the quantized transform coefficients to the reversible encoding unit 106.

The reversible coding unit 106 performs reversible coding, such as variable length coding and arithmetic coding, on the quantized transform coefficients.

The reversible coding unit 106 obtains information indicating intra prediction from the intra predicting unit 114, and obtains information indicating the inter prediction mode, motion vector information, and the like from the motion predicting / compensating unit 115. In addition, the information which shows intra prediction (intra picture prediction) is also called intra prediction mode information hereafter. In addition, the information which shows the information mode which shows inter prediction (inter prediction) is also called inter prediction mode information hereafter.

The reversible coding unit 106 encodes the quantized transform coefficients and makes various kinds of information such as filter coefficients, intra prediction mode information, inter prediction mode information, and quantization parameters as part of header information of the encoded data (multiplexing). box). The reversible encoding unit 106 supplies and stores the encoded data obtained by encoding the accumulation buffer 107.

For example, in the reversible coding unit 106, reversible coding processes such as variable length coding or arithmetic coding are performed. Examples of variable length coding include CAVLC (Context-Adaptive Variable Length Coding) determined by the H.264 / AVC method. Examples of the arithmetic coding include CABAC (Context-Adaptive Binary Arithmetic Coding).

The storage buffer 107 temporarily holds the encoded data supplied from the reversible encoding unit 106 and is a coded image encoded by the H.264 / AVC system at a predetermined timing. Output to a recording apparatus, transmission path, etc. not shown.

The transform coefficients quantized in the quantization unit 105 are also supplied to the inverse quantization unit 108. The inverse quantization unit 108 dequantizes the quantized transform coefficients by a method corresponding to the quantization by the quantization unit 105. The inverse quantization unit 108 performs inverse quantization using the quantization parameter for each sub macroblock set in the quantization unit 105, in association with the sub macroblock inverse quantization unit 122. The inverse quantization unit 108 supplies the obtained conversion coefficient to the inverse orthogonal transformation unit 109.

The inverse orthogonal transform unit 109 inversely transforms the supplied transform coefficients by a method corresponding to the orthogonal transform process by the orthogonal transform unit 104. [ The inverse orthogonal transformed output (restored difference information) is supplied to the calculation unit 110.

The calculating part 110 performs the intra prediction part 114 or the motion prediction / compensation part 115 to the inverse orthogonal transform result supplied from the inverse orthogonal transform part 109, ie, the restored difference information, via the selection part 116. ) Is added to obtain a predicted image, and a locally decoded image (decoded image) is obtained.

For example, when the difference information corresponds to an image on which intra encoding is performed, the calculation unit 110 adds the predictive image supplied from the intra prediction unit 114 to the difference information. For example, when difference information corresponds to the image to which inter encoding is performed, the calculating part 110 adds the prediction image supplied from the motion prediction / compensation part 115 to the difference information.

The addition result is supplied to the deblocking filter 111 or the frame memory 112.

The deblock filter 111 removes block distortion of the decoded image by appropriately performing a deblocking filter process, and improves image quality by appropriately performing a loop filter process using, for example, a Wiener filter. The deblocking filter 111 classifies each pixel and performs appropriate filter processing for each class. The deblocking filter 111 supplies the filter processing result to the frame memory 112.

The frame memory 112 outputs the accumulated reference image to the intra predictor 114 or the motion predictor / compensator 115 through the selector 113 at a predetermined timing.

For example, in the case of an image subjected to intra encoding, the frame memory 112 supplies the reference image to the intra prediction unit 114 through the selection unit 113. For example, when inter coding is performed, the frame memory 112 supplies the reference image to the motion prediction / compensation unit 115 via the selection unit 113.

The selector 113 supplies the reference picture to the intra predictor 114 when the reference picture supplied from the frame memory 112 is an intra coded picture. In addition, when the reference picture supplied from the frame memory 112 is an inter coded picture, the selection unit 113 supplies the reference picture to the motion prediction / compensation unit 115.

The intra prediction unit 114 performs intra prediction (intra prediction) for generating a predictive image using pixel values in the screen. The intra prediction unit 114 performs intra prediction in a plurality of modes (intra prediction modes).

The intra prediction unit 114 generates predictive images in all intra prediction modes, evaluates each predictive image, and selects an optimal mode. When the intra prediction unit 114 selects the optimal intra prediction mode, the intra prediction unit 114 supplies the predicted image generated in the optimum mode to the operation unit 103 or the operation unit 110 through the selection unit 116.

In addition, as described above, the intra prediction unit 114 appropriately supplies the reversible coding unit 106 with information such as intra prediction mode information indicating the adopted intra prediction mode.

The motion prediction / compensation unit 115 is an input image supplied from the screen rearrangement buffer 102 and a reference image supplied from the frame memory 112 via the selection unit 113 with respect to the image on which inter encoding is performed. Motion estimation is performed, motion compensation processing is performed according to the detected motion vector, and a predictive image (inter prediction image information) is generated.

The motion prediction / compensation unit 115 performs inter prediction processing of all inter prediction modes as candidates, and generates a predicted image. The motion prediction / compensation unit 115 supplies the generated predicted image to the calculation unit 103 or the calculation unit 110 through the selection unit 116.

The motion prediction / compensation unit 115 supplies the reversible coding unit 106 with inter prediction mode information indicating the adopted inter prediction mode and motion vector information indicating the calculated motion vector.

The selecting unit 116 supplies the output of the intra prediction unit 114 to the calculating unit 103 or the calculating unit 110 in the case of an image to be intra-coded, and the motion predicting and compensating unit in the case of an image to be inter-coded. The output of 115 is supplied to arithmetic unit 103 or arithmetic unit 110.

The rate control unit 117 controls the rate of the quantization operation of the quantization unit 105 so that overflow or underflow does not occur based on the compressed image accumulated in the accumulation buffer 107. The rate control unit 117 supplies the quantization unit 105 with information indicating the complexity of the image for each sub macroblock, which is a small area in which the macroblock is divided into a plurality of regions.

For example, the rate control unit 117 provides the quantization unit 105 with an activity that is information indicating dispersion of pixel values as information indicating the complexity of this image. Of course, any information may be sufficient as the information which shows the complexity of this image.

The sub-macroblock quantization unit 121 obtains, from the quantization unit 105, information indicating the complexity of the image for each sub-macroblock, and sets the quantization value (quantization step) for each sub-macroblock based on the information. The value is returned to the quantization unit 105.

The sub macroblock inverse quantization unit 122 obtains quantization parameters from the inverse quantization unit 108, obtains quantization values for each sub macroblock using those values, and returns them to the inverse quantization unit 108.

[Quantization of AVC]

Here, as a conventional quantization process, quantization determined in AVC (Advanced Video Coding) will be described as an example.

The integer conversion matrix [H] determined in the AVC does not satisfy the conditions of the orthogonal transformation matrix represented by the following equation (1), but after integer conversion, different quantization processing is performed on each component, and integer conversion and By combining the quantizations, an orthogonal transformation process is performed.

[H] [H] ^T = [I]... (One)

In AVC, in order to perform quantization, it is possible to define the quantization parameter QP (Quantization Parameter) which can take the value of "0"-"51" with respect to each macroblock.

For example, suppose that it is a value which satisfy | fills following formula (2) irrespective of the value of A (QP), B (QP) k, and QP.

A (QP) * B (QP) = 2m + n... (2)

Orthogonal transformation and inverse orthogonal transformation in AVC can be implement | achieved by operation similar to following formula (3) and formula (4).

d = c * A (QP) / 2 ^m ... (3)

c '= d * B (QP) / 2 ⁿ ... (4)

C is an orthogonal transform coefficient before quantization, d is an orthogonal transform coefficient after quantization, and c 'is an orthogonal transform coefficient after inverse quantization.

By performing such a process, in AVC, it is possible to realize quantization and inverse quantization processing only by shift operation, not division.

In addition, the value of A and B will have a different value for every component.

The quantization parameter QP is designed such that, for example, from 6 to 12, when the value is increased by 6, a roughly doubled quantization process is performed.

In addition, especially at a lower bit rate, i.e., higher QP, the degradation in the color difference signal is likely to be noticeable. Therefore, for the quantization parameter QP _Y for the luminance signal, the default quantization parameter QP _C for the chrominance signal is defined as shown in the table shown in FIG.

The user can control this relationship by setting information about ChromaQPOffset included in the image compression information.

In addition, above the High Profile, it is possible to independently set the quantization parameter for the Cb / Cr component using ChromaQPOffset and 2ndChromaQPOffset.

[Quantification Parameter Calculation]

In the AVC coding method and the coding methods described in Non-Patent Document 1 and Non-Patent Document 2, the quantization parameter MB_QP for each macro block is calculated as follows.

That is, first, QpBdOffset _Y is calculated from the bit_depth_luma_minus8 present in the sequence parameter set as in the following formula (5).

QpBdOffset _Y = 6 * bit_depth_luma_minus8... (5)

Next, the initial value of the quantization parameter in each picture is specified by pic_init_qp_minus26 in the picture parameter set.

Next, the quantization parameter SliceQP _Y in the slice is calculated by the following formula (6) by slice_qp_delta defined in the slice layer.

SliceQP _Y = 26 + pic_init_qp_minus26 + slice_qp_delta... (6)

Finally, using mb_qp_delta in the macroblock layer, the quantization parameter MB_QP for each macroblock is calculated as in the following equation (7).

MB_QP = ((MB_QP _Prev + mb_qp_delta + 52 + 2 * QpBdOffsetY _Y )% (52 + QpBdOffset _Y ))-QpBdOffset _Y ... (7)

MB_QP _Prev is a quantization parameter in the immediately preceding macroblock.

In the present technology, in addition to this, the information on the submb_qp_delta is included in the sub macroblock during image compression.

Using this information, the quantization parameter SubMB_QP for each sub macroblock is calculated as in the following equation (8).

SubMB_QP = Clip (0,51, MB_QP + submb_qp_delta)... (8)

Here, Clip (min, max, value) is a function having a return value as in Equation 1 below.

That is, the quantization parameter SubMB_QP for each sub macroblock is calculated as shown in Equation 2 below. However, the predefined minimum quantization parameter is minQP, and the predefined maximum quantization parameter is maxQP.

When submb_qp_delta does not exist in the image compression information, the value is "0", and the quantization parameter in the macroblock is also applied to the submacroblock.

[Quantification Department]

5 is a block diagram illustrating a detailed configuration example of the quantization unit 105 of FIG. 1. As shown in FIG. 5, the quantization unit 105 includes a sub macroblock activity buffer 151, a quantization parameter calculation unit 152, and a quantization processing unit 153.

The sub macroblock activity buffer 151 holds an activity supplied from the rate control unit 117. In the AVC coding scheme, although adaptive quantization is performed based on an activity as defined in the MPEG-2 Test Model, the rate control unit 117 is configured to perform an activity (also referred to as a sub macroblock activity) for each submacroblock. The calculation is performed. The method of calculating the sub macroblock activity is the same as in the conventional case of calculating the activity for each macroblock.

The sub macro block activity buffer 151 holds sub macro block activities supplied from the rate control unit 117, and stores the sub macro block activities held for each predetermined amount (for example, one screen). Supply to the quantization unit 121.

The sub macro block quantization unit 121 calculates a quantization value for each sub macro block by using the sub macro block activity supplied from the sub macro block activity buffer 151. The quantization value for each sub macroblock can be calculated by the same method as that for calculating the quantization value for each macroblock from the activity for each macroblock.

When the quantization value is obtained for each sub macro block, the sub macro block quantization unit 121 supplies the quantization value for each sub macro block to the quantization parameter calculation unit 152.

The quantization parameter calculator 152 calculates various quantization parameters by using quantization values for each sub macroblock supplied from the sub macroblock quantization unit 121.

For example, the quantization parameter calculator 152 calculates quantization parameters such as pic_init_qp_minus26, slice_qp_delta, and mb_qp_delta. Since the quantization parameter calculator 152 can obtain the quantization value for each macroblock from the quantization value for each submacroblock, it calculates and sets these various quantization parameters as in the case of the conventional AVC coding method. do.

The quantization parameter calculator 152 further obtains a quantization parameter submb_qp_delta indicating the difference between the quantization parameter MB_QP for each macroblock and the quantization parameter SubMB_QP for each submacroblock. The quantization parameter for each sub macroblock needs to be transmitted to the decoding side. Therefore, by setting the difference value in this manner, the code amount of the quantization parameter for each sub macroblock can be reduced. In other words, this quantization parameter submb_qp_delta is a format for transmission of the quantization parameter SubMB_QP. The quantization parameter SubMB_QP for each sub macroblock is obtained by converting the quantization value for each sub macroblock. Similarly, the quantization parameter MB_QP for each macro block is obtained by converting the quantization value for each macro block. The quantization parameter calculator 152 calculates submb_qp_delta for each sub macroblock using, for example, the above equation (35).

The quantization parameter calculator 152 supplies the quantization value for each sub macroblock to the quantization processor 153. In addition, the quantization parameter calculator 152 supplies the calculated quantization parameters (specifically, pic_init_qp_minus26, slice_qp_delta, mb_qp_delta, etc.) to the reversible coding unit 106, and transmits the image together with the encoded coded stream. . As described above, when the value of submb_qp_delta is "0", transmission of submb_qp_delta is omitted. That is, in that case, quantization parameters other than submb_qp_delta are supplied to the reversible coding unit 106.

The quantization parameter calculator 152 also supplies the quantization value for each sub macroblock to the dequantizer 108.

The quantization processing unit 153 quantizes the orthogonal transform coefficients supplied from the orthogonal transform unit 104 using quantization values for each sub macroblock.

The quantization processing unit 153 supplies the quantized orthogonal transform coefficients to the reversible coding unit 106 and the inverse quantization unit 108.

In addition, the inverse quantization unit 108 uses the sub macroblock inverse quantization unit 122 to inverse quantize the orthogonal transform coefficients quantized by the quantization unit 105 described above. Also in the image decoding apparatus corresponding to the image encoding apparatus 100, the same processing as that of the inverse quantization processing is performed, so that the details of the inverse quantization will be described at the time of explaining the image decoding apparatus.

In the conventional case such as the AVC coding scheme, only one quantization parameter can be set for one macroblock. Therefore, when flat regions and regions containing textures are mixed in one macro block, it becomes difficult to set appropriate quantization parameters in both regions.

In particular, as the macroblock (extended partial region) proposed in Non-Patent Document 2 and the like increases in size, the likelihood that images having different features are mixed in the region increases, and the respective regions It becomes more difficult to perform adaptive quantization according to the characteristics of.

In contrast, the image encoding apparatus 100 calculates an index indicating the complexity of the image for each sub macroblock in the rate control unit 117, and calculates a quantization value for each sub macroblock in the sub macroblock quantization unit 121. can do. In other words, the quantization processing unit 153 can perform quantization processing using an appropriate quantization value for each sub macroblock.

As a result, the image encoding apparatus 100 can perform quantization processing more suited to the content of the image. In particular, even when the macroblock size is expanded and includes both a flat area and an area including a texture in a single macroblock, the image encoding apparatus 100 performs adaptive quantization processing suitable for each area. The degradation of subjective image quality of the decoded image can be suppressed.

For example, in the image 160 shown in FIG. 6, only the flat area is included in the macro block 161. Thus, for example, even if the image encoding apparatus 100 performs a quantization process using a single quantization parameter on such a macroblock 161, there is no problem in image quality in particular.

In contrast, the macro block 162 includes both a flat region and a texture region. In the quantization process using a single quantization parameter, appropriate adaptive quantization cannot be performed on both the flat region and the texture region. Therefore, for example, when the image encoding apparatus 100 performs a quantization process using a single quantization parameter for such a macroblock 161, subjective image quality of the decoded image may be reduced.

Even in such a case, since the image encoding apparatus 100 can calculate the quantization value in units of sub macroblocks as described above, it is possible to perform a more appropriate quantization process to suppress the reduction in subjective image quality of the decoded image.

In addition, in the accumulation buffer 107, control by the quantization parameter is performed even when the total code amount for each picture is about to overflow. Therefore, as described above, the quantization unit 105 calculates the quantization value in units of sub-macroblocks and performs quantization, so that the image encoding apparatus 100 controls the overflow countermeasure in finer units. I can do it.

In addition, when the value of submb_qp_delta is "0", since the transmission of the submb_qp_delta is omitted, unnecessary reduction in coding efficiency can be suppressed. When the value of submb_qp_delta is "0", since the quantization parameter SubMB_QP for each sub macroblock and the quantization parameter MB_QP for each macro block are the same, on the decoding side, the quantization parameter MB_QP for each macro block is quantized for each sub macroblock. Since the parameter SubMB_QP can be used, the value of submb_qp_delta ("0") is unnecessary. Therefore, transmission of submb_qp_delta can be omitted as described above. Of course, submb_qp_delta whose value is "0" can also be transmitted. However, by omitting the transmission of submb_qp_delta, the coding efficiency can be improved accordingly.

[Flow of encoding process]

Next, the flow of each process performed by the above-described image coding apparatus 100 will be described. First, an example of the flow of an encoding process is demonstrated with reference to the flowchart of FIG.

In step S101, the A / D converter 101 A / D converts the input image. In step S102, the screen rearrangement buffer 102 stores the A / D-converted image, and rearranges from the order displayed by each picture to the order to be encoded.

In step S103, the calculating part 103 calculates the difference of the image rearranged by the process of step S102, and the predictive image. The predicted image is supplied from the motion prediction / compensation unit 115 in the case of inter prediction to the calculation unit 103 from the intra prediction unit 114 in the case of intra prediction through the selection unit 116.

The difference data is reduced in data amount compared to the original image data. Therefore, the data amount can be compressed as compared with the case where the image is encoded as it is.

In step S104, orthogonal transformation unit 104 orthogonally transforms the difference information generated in step S103. Specifically, orthogonal transformations, such as a discrete cosine transform and a Karune-Lube transform, are performed, and a transform coefficient is output.

In step S105, the quantization unit 105 and the sub macroblock quantization unit 121 obtain quantization parameters. The detail of the flow of a quantization parameter calculation process is mentioned later.

In step S106, the quantization processing unit 153 of the quantization unit 105 quantizes the orthogonal transform coefficients obtained by the processing in step S104, using the quantization value for each sub macroblock calculated by the processing in step S105. .

The difference information quantized by the process of step S106 is locally decoded as follows. That is, in step S107, the inverse quantization unit 108 dequantizes the quantized orthogonal transform coefficients (also referred to as quantization coefficients) generated by the process of step S106 to characteristics corresponding to the characteristics of the quantization unit 105. In step S108, the inverse orthogonal transform unit 109 inversely orthogonally transforms the orthogonal transform coefficient obtained by the process of step S107 to a characteristic corresponding to that of the orthogonal transform unit 104.

In step S109, the calculating part 110 adds a predictive image to the locally decoded difference information, and produces | generates a locally decoded image (image corresponding to the input to the calculating part 103). In step S110, the deblocking filter 111 filters the image generated by the process of step S109. As a result, block distortion is eliminated.

In step S111, the frame memory 112 stores an image from which block distortion has been removed by the process of step S110. The frame memory 112 is also supplied with and stored by the computing unit 110 an image which has not been filtered by the deblocking filter 111.

In step S112, the intra prediction unit 114 performs intra prediction processing in the intra prediction mode. In step S113, the motion prediction / compensation unit 115 performs inter motion prediction processing that performs motion prediction and motion compensation in the inter prediction mode.

In step S114, the selection unit 116 determines the optimum prediction mode based on the respective cost function values output from the intra prediction unit 114 and the motion prediction / compensation unit 115. That is, the selection unit 116 selects either the predicted image generated by the intra predictor 114 or the predicted image generated by the motion predictor / compensator 115.

In addition, selection information indicating which predictive image is selected is supplied to the selected one of the intra prediction unit 114 and the motion prediction / compensation unit 115. When the predictive image of the optimal intra prediction mode is selected, the intra prediction unit 114 supplies the reversible coding unit 106 with information indicating the optimal intra prediction mode (that is, intra prediction mode information).

When the predictive image of the best inter prediction mode is selected, the motion prediction / compensation unit 115 outputs information indicating the best inter prediction mode and information according to the best inter prediction mode to the reversible coding unit 106 as necessary. do. Examples of the information according to the optimal inter prediction mode include motion vector information, flag information, reference frame information, and the like.

In step S115, the reversible coding unit 106 encodes the transform coefficients quantized by the process of step S106. That is, reversible coding such as variable length coding and arithmetic coding is performed on the difference image (second difference image in the case of the inter).

The reversible encoding unit 106 also encodes the quantization parameter calculated in step S105 and adds it to the encoded data.

In addition, the reversible encoding unit 106 encodes information about the prediction mode of the predicted image selected by the process of step S114, and adds it to the encoded data obtained by encoding the differential image. That is, the reversible encoding unit 106 also encodes the intra prediction mode information supplied from the intra prediction unit 114 or the information according to the optimal inter prediction mode supplied from the motion prediction / compensation unit 115, and the like. Add to

In step S116, the accumulation buffer 107 accumulates the encoded data output from the reversible encoding unit 106. The coded data accumulated in the accumulation buffer 107 is appropriately read and transmitted to the decoding side via the transmission path.

In step S117, the rate control unit 117 performs the quantization operation of the quantization unit 105 so that overflow or underflow does not occur based on the compressed image accumulated in the accumulation buffer 107 by the process of step S116. Control the rate.

When the process of step S117 ends, the encoding process ends.

[Flow of quantization parameter calculation processing]

Next, an example of the flow of the quantization parameter calculation process performed in step S105 of FIG. 7 will be described with reference to the flowchart of FIG. 8.

When the quantization parameter calculation process is started, in step S131, the sub macroblock activity buffer 151 acquires the submacroblock activity supplied from the rate control unit 117. The sub macroblock activity buffer 151 holds the acquired submacroblock activity for one screen, for example.

In step S132, the sub macroblock quantization unit 121 acquires, for example, one screen of the sub macroblock activity from the submacroblock activity buffer 151. Then, the sub macroblock quantization unit 121 calculates the quantization value for each sub macroblock using the obtained submacroblock activity.

In step S133, the quantization parameter calculator 152 obtains the quantization parameter pic_init_qp_minus26 using the quantization value for each sub macroblock calculated in step S132.

In step S134, the quantization parameter calculator 152 calculates the quantization parameter slice_qp_delta using the quantization value for each sub macroblock calculated in step S132.

In step S135, the quantization parameter calculator 152 obtains the quantization parameter mb_qp_delta using the quantization value for each sub macroblock calculated in step S132.

In step S136, the quantization parameter calculator 152 calculates the quantization parameter submb_qp_delta using the quantization value for each sub macroblock calculated in step S132.

When the various quantization parameters are obtained as described above, the quantization unit 105 ends the quantization parameter calculation processing, returns the processing to step S105 in FIG. 7, and executes the processing after step S106.

Since the encoding process and the quantization parameter calculation process are performed as described above, the image encoding apparatus 100 can set the quantization value for each sub macroblock, and can perform more appropriate quantization process.

In addition, since the quantization parameter calculated as described above is transmitted to the image decoding apparatus, the image encoding apparatus 100 can cause the image decoding apparatus to obtain a quantization value for each sub macroblock, and use it to perform inverse quantization.

<2. Second Embodiment>

[Image decoding device]

9 is a block diagram showing a main configuration example of an image decoding device to which the present technology is applied. The image decoding device 200 shown in FIG. 9 is a decoding device corresponding to the image coding device 100.

The coded data encoded by the picture coding apparatus 100 is transmitted to the picture decoding apparatus 200 corresponding to this picture coding apparatus 100 through a predetermined transmission path, and is decoded.

As shown in FIG. 9, the image decoding device 200 includes an accumulation buffer 201, a reversible decoding unit 202, an inverse quantization unit 203, an inverse orthogonal conversion unit 204, an operation unit 205, and a decode. The block filter 206, the screen rearrangement buffer 207, and the D / A converter 208 are provided. The image decoding device 200 also includes a frame memory 209, a selection unit 210, an intra prediction unit 211, a motion prediction / compensation unit 212, and a selection unit 213.

The image decoding device 200 also includes a sub macroblock dequantization unit 221.

The accumulation buffer 201 accumulates the transmitted encoded data. This coded data is coded by the picture coding apparatus 100. The reversible decoding unit 202 decodes the encoded data read out from the accumulation buffer 201 at a predetermined timing in a manner corresponding to the encoding method of the reversible encoding unit 106 in FIG. 1.

The inverse quantization unit 203 operates in conjunction with the sub-macroblock inverse quantization unit 221 and uses the quantization unit 105 of FIG. 1 to obtain coefficient data (quantization coefficients) obtained by decoding by the reversible decoding unit 202. Dequantize in a manner corresponding to the quantization scheme of. In other words, the inverse quantization unit 203 uses the quantization parameter calculated for each sub macroblock supplied from the image encoding apparatus 100 to perform the inverse of the quantization coefficient in the same manner as the inverse quantization unit 108 of FIG. 1. Quantization is performed.

The inverse quantization unit 203 supplies dequantized coefficient data, that is, orthogonal transform coefficients, to the inverse orthogonal transform unit 204. The inverse orthogonal transform unit 204 inversely orthogonally transforms the orthogonal transform coefficients in a manner corresponding to the orthogonal transform method of the orthogonal transform unit 104 in FIG. 1 before being orthogonally transformed in the image encoding apparatus 100. Decoded residual data corresponding to the residual data is obtained.

The decoding residual data obtained by inverse orthogonal transformation is supplied to the calculating part 205. In addition, the calculation unit 205 supplies a predictive image from the intra prediction unit 211 or the motion prediction / compensation unit 212 through the selection unit 213.

The calculating unit 205 adds the decoding residual data and the predictive image, and obtains decoded image data corresponding to the image data before the predictive image is subtracted by the calculating unit 103 of the image coding apparatus 100. The calculation unit 205 supplies the decoded image data to the deblocking filter 206.

The deblocking filter 206 removes block distortion of the supplied decoded image and then supplies it to the screen rearrangement buffer 207.

The screen rearrangement buffer 207 rearranges the images. That is, the order of the frames rearranged for the encoding order by the screen rearrangement buffer 102 of FIG. 1 is rearranged in the order of the original display. The D / A conversion unit 208 D / A converts the image supplied from the screen rearrangement buffer 207, outputs it to a display (not shown), and displays it.

The output of the deblock filter 206 is also supplied to the frame memory 209.

The frame memory 209, the selector 210, the intra predictor 211, the motion predictor / compensator 212, and the selector 213 are selected from the frame memory 112 of the image encoding apparatus 100. Corresponding to the unit 113, the intra predictor 114, the motion predictor / compensator 115, and the selector 116, respectively.

The selector 210 reads the inter-processed image and the referenced image from the frame memory 209 and supplies the motion predictor / compensator 212. The selection unit 210 reads an image used for intra prediction from the frame memory 209 and supplies the image to the intra prediction unit 211.

The intra prediction unit 211 is appropriately supplied with information indicating the intra prediction mode obtained by decoding the header information from the reversible decoding unit 202. The intra prediction unit 211 generates a predictive image from the reference image acquired from the frame memory 209 based on this information, and supplies the generated predictive image to the selection unit 213.

The motion prediction / compensation unit 212 obtains information (prediction mode information, motion vector information, reference frame information, flags, various parameters, etc.) obtained by decoding the header information from the reversible decoding unit 202.

The motion prediction / compensation unit 212 generates a predictive image from the reference image acquired from the frame memory 209 based on those information supplied from the reversible decoding unit 202, and selects the generated predictive image 213. Supplies).

The selection unit 213 selects the predicted image generated by the motion prediction / compensation unit 212 or the intra prediction unit 211, and supplies it to the calculation unit 205.

The sub macroblock inverse quantization unit 221 obtains a quantization parameter from the inverse quantization unit 203, obtains a quantization value for each sub macroblock by using Equation 2, and returns it to the inverse quantization unit 203. Send it back.

Inverse quantization department

10 is a block diagram illustrating a detailed configuration example of the inverse quantization unit 203.

As shown in FIG. 10, the inverse quantization unit 203 includes a quantization parameter buffer 251, an orthogonal transform coefficient buffer 252, and an inverse quantization processing unit 253.

The parameters related to quantization in each layer, such as a picture parameter set of coded data supplied from the picture coding apparatus 100 and a slice header, are decoded in the reversible decoding unit 202, and the quantization parameter buffer ( 251). The quantization parameter buffer 251 appropriately holds the quantization parameter and supplies it to the sub macroblock dequantization unit 221 at a predetermined timing.

The sub macroblock inverse quantization unit 221 uses the quantization parameter supplied from the quantization parameter buffer 251, for example, as shown in equations (5) to (2), for each sub macroblock. SubMB_QP is calculated, it is converted into quantization values for each sub macroblock, and it is supplied to inverse quantization processing unit 253.

In addition, as described above in the first embodiment, when the value of submb_qp_delta is "0", submb_qp_delta is not transmitted. When submb_qp_delta does not exist in the quantization parameter supplied from the quantization parameter buffer 251, the sub macroblock dequantization unit 221 converts the value of the quantization parameter MB_QP for each macroblock to the quantization parameter SubMB_QP for each submacroblock. Apply.

In the reversible decoding unit 202, the quantized orthogonal transform coefficients obtained by decoding the encoded data supplied from the image encoding apparatus 100 are supplied to the orthogonal transform coefficient buffer 252. The orthogonal transform coefficient buffer 252 appropriately holds the quantized orthogonal transform coefficient and supplies it to the inverse quantization processing unit 253 at a predetermined timing.

The inverse quantization processing unit 253 inverse quantizes the quantized orthogonal transform coefficients supplied from the orthogonal transform coefficient buffer 252 using quantization values for each sub macroblock supplied from the sub macroblock inverse quantization unit 221. do. The inverse quantization processing unit 253 supplies the orthogonal transformation coefficient obtained by inverse quantization to the inverse orthogonal transformation unit 204.

As described above, the inverse quantization unit 203 can perform inverse quantization processing using the quantization value calculated for each sub macroblock. As a result, the image decoding device 200 can perform inverse quantization processing more suited to the contents of the image. In particular, even when the macro block size is extended and includes both flat areas and areas including textures in a single macro block, the image decoding device 200 performs adaptive dequantization processing suitable for each area. In this way, degradation in subjective image quality of the decoded image can be suppressed.

In addition, the inverse quantization unit 108 of the picture coding apparatus 100 shown in FIG. 1 also has the same configuration as the inverse quantization unit 203 and performs the same processing. However, the inverse quantization unit 108 obtains the quantization parameter and the quantized orthogonal transform coefficients supplied from the quantization unit 105 and performs inverse quantization.

In addition, the inverse quantization unit 108 provides a quantization parameter to the sub macroblock dequantization unit 122 that performs the same processing as the sub macroblock inverse quantization unit 221, and provides a quantization value for each sub macroblock. Create

[Flow of Decoding Process]

Next, the flow of each process performed by the image decoding apparatus 200 as described above will be described. First, with reference to the flowchart of FIG. 11, an example of the flow of a decoding process is demonstrated.

When the decoding process is started, in step S201, the accumulation buffer 201 accumulates the transmitted encoded data. In step S202, the reversible decoding unit 202 decodes the encoded data supplied from the accumulation buffer 201. That is, the I picture, P picture, and B picture coded by the reversible encoder 106 of FIG. 1 are decoded.

At this time, motion vector information, reference frame information, prediction mode information (intra prediction mode or inter prediction mode), and information such as a flag and a quantization parameter are also decoded.

When the prediction mode information is intra prediction mode information, the prediction mode information is supplied to the intra prediction unit 211. When the prediction mode information is the inter prediction mode information, the motion vector information corresponding to the prediction mode information is supplied to the motion prediction / compensation unit 212.

In step S203, the inverse quantization unit 203 dequantizes the quantized orthogonal transform coefficients obtained by being decoded by the reversible decoding unit 202. In step S204, the inverse orthogonal transform unit 204 inversely orthogonally transforms the orthogonal transform coefficient obtained by inverse quantization by the inverse quantization unit 203 in a manner corresponding to the orthogonal transform unit 104 in FIG. As a result, the difference information corresponding to the input (output of the calculation unit 103) of the orthogonal transform unit 104 in FIG. 1 is decoded.

In step S205, the calculating part 205 adds a predictive image to the difference information obtained by the process of step S204. As a result, the original image data is decoded.

In step S206, the deblock filter 206 appropriately filters the decoded image obtained by the process of step S205. As a result, block distortion is appropriately removed from the decoded image.

In step S207, the frame memory 209 stores the filtered decoded image.

In step S208, the intra prediction unit 211 or the motion prediction / compensation unit 212 performs image prediction processing in response to the prediction mode information supplied from the reversible decoding unit 202, respectively.

That is, when intra prediction mode information is supplied from the reversible decoding unit 202, the intra prediction unit 211 performs intra prediction processing in the intra prediction mode. In addition, when inter prediction mode information is supplied from the reversible decoding unit 202, the motion prediction / compensation unit 212 performs motion prediction processing in the inter prediction mode.

In step S209, the selection unit 213 selects the predictive image. In other words, the prediction unit generated by the intra prediction unit 211 or the prediction image generated by the motion prediction / compensation unit 212 is supplied to the selection unit 213. The selection unit 213 selects the side to which the prediction image is supplied, and supplies the prediction image to the calculation unit 205. This predictive image is added to difference information by the process of step S205.

In step S210, the screen rearrangement buffer 207 rearranges the frames of the decoded image data. That is, the order of the frames rearranged for encoding by the screen rearrangement buffer 102 (FIG. 1) of the image encoding apparatus 100 in the decoded image data is rearranged in the order of the original display.

In step S211, the D / A converter 208 performs D / A conversion on the decoded image data in which the frames are rearranged in the screen rearrangement buffer 207. This decoded image data is output to a display not shown, and the image is displayed.

[Dequantization processing]

Next, with reference to the flowchart of FIG. 12, an example of the flow of inverse quantization process is demonstrated.

When the inverse quantization process is started, the quantization parameter buffer 251 acquires the quantization parameter pic_init_qp_minus26 supplied from the reversible decoding unit 202 in step S231.

In step S232, the quantization parameter buffer 251 acquires the quantization parameter slice_qp_delta supplied from the reversible decoding unit 202.

In step S233, the quantization parameter buffer 251 acquires the quantization parameter mb_qp_delta supplied from the reversible decoding unit 202.

In step S234, the quantization parameter buffer 251 acquires the quantization parameter submb_qp_delta supplied from the reversible decoding unit 202. However, if submb_qp_delta does not exist, the process of step S234 is skipped.

In step S235, the sub macroblock dequantization unit 221 calculates a quantization value for each sub macroblock using various quantization parameters obtained by the processes of steps S231 to S234. However, when submb_qp_delta is not supplied from the image coding apparatus 100 and the process of step S234 is omitted, the sub macroblock dequantization unit 221 determines the quantization value for each macroblock and the quantization value for each submacroblock. Applies to

In step S236, the inverse quantization processing unit 253 inverses the quantized orthogonal transform coefficient held in the orthogonal transform coefficient buffer 252 using the quantization value for each sub macroblock calculated by the process of step S235. Quantize.

When the process of step S236 ends, the inverse quantization unit 203 returns the process to step S203 and executes the process after step S204.

As described above, by performing the decoding process and the inverse quantization process, the image decoding device 200 can perform the inverse quantization process using the quantization value calculated for each sub-macroblock, which is more suitable for the content of the image. Can be done.

<3. Third embodiment>

[submb_qp_present_flag]

In the above description, submb_qp_delta is appropriately transmitted as the quantization parameter. However, a flag for notifying the presence or absence of submb_qp_delta for each macroblock may be transmitted.

In that case, the structure of the image encoding apparatus 100 is the same as that of the structural example shown in FIG. In addition, the structure of the quantization part 105 is the same as that of the structural example shown in FIG. However, the quantization parameter calculator 152 further calculates submb_qp_present_flag, which is flag information indicating whether or not there is a submb_qp_delta whose value is not "0" for each macroblock. When one submb_qp_delta of the sub macroblock belonging to the macroblock has a value other than "0", the submb_qp_present_flag is set to "1", for example. In addition, when submb_qp_delta of all the sub macroblocks which belong to this macroblock is "0", submb_qp_present_flag is set to "0", for example.

Of course, the value of submb_qp_present_flag is arbitrary, and any value may be sufficient as long as it can identify the case where one submb_qp_delta has a value other than "0", and the case where the submb_qp_delta of all the sub macroblocks is "0".

If the quantization parameter calculator 152 sets the values in this manner, the quantization parameter calculator 152 supplies the submb_qp_present_flag as one of the quantization parameters to the reversible encoder 106. The reversible coding unit 106 adds this submb_qp_present_flag to, for example, a macroblock header and encodes it. In other words, the submb_qp_present_flag is transmitted together with the coded data similarly to other quantization parameters.

Therefore, the encoding process in this case is performed similarly to the above-mentioned with reference to the flowchart of FIG. In addition, an example of the flow of a quantization parameter calculation process in this case is demonstrated with reference to the flowchart of FIG. In this case, the quantization parameter calculation process is basically performed similarly to the case described with reference to the flowchart of FIG. 8.

That is, each process of step S331-S336 is performed similarly to each process of step S131-step S136 of FIG. In this case, however, the quantization parameter calculator 152 calculates the quantization parameter submb_qp_present_flag again in step S337.

As described above, the quantization parameter submb_qp_present_flag is calculated and transmitted.

That is, submb_qp_present_flag exists in each macroblock header of data. Submb_qp_delta exists in the sub macroblock header of the macroblock whose submb_qp_present_flag value is "1", and submb_qp_delta does not exist in the submacroblock header of the macroblock in which the value of submb_qp_present_flag is "0".

Such encoded data is transmitted from the image encoding apparatus 100 to the image decoding apparatus 200.

The configuration of the image decoding device 200 in this case is the same as the configuration example shown in FIG. 9. In addition, the structure of the inverse quantization part 203 is the same as that of the structural example shown in FIG. However, the sub-macroblock dequantization unit 221 calculates the quantization value for each macroblock without waiting for the submb_qp_delta to be supplied to the macroblock in which the submb_qp_present_flag is set to "0". Applies to quantization values.

In other words, the sub macroblock dequantization unit 221 acquires submb_qp_delta only when submb_qp_present_flag is "1", and calculates the quantization value for each sub macroblock.

The decoding process in this case is performed similarly to the above-mentioned with reference to the flowchart of FIG. In this case, an example of the flow of the inverse quantization process will be described with reference to the flowchart of FIG. 14. Inverse quantization processing is also basically performed in this case as in the case described with reference to the flowchart of FIG. 12.

That is, each process of step S431-step S433 is performed similarly to each process of step S231-step S233 of FIG. In this case, however, the quantization parameter buffer 251 obtains the quantization parameter submb_qp_present_flag stored in the macroblock header in step S434.

In step S435, the sub macroblock dequantization unit 221 determines whether or not the value of the quantization parameter submb_qp_present_flag is "1". When the value of the quantization parameter submb_qp_present_flag is "1", the quantization parameter buffer 251 acquires the quantization parameter submb_qp_delta in step S436. In step S437, the sub macroblock dequantization unit 221 calculates a quantization value for each sub macroblock. That is, the same process as that of step S234 and step S235 of FIG. 12 is performed.

In addition, when it is determined in step S435 that the value of the quantization parameter submb_qp_present_flag is "0", the sub-macroblock dequantization unit 221 calculates the quantization value for each macroblock in step S438, and sub-macro It is applied as a quantization value for each block.

When the quantization value is calculated as described above, the dequantization processing unit 253 performs dequantization using the quantization value in step S439.

As described above, by transmitting the submb_qp_present_flag indicating the presence or absence of the quantization parameter submb_qp_delta for each macroblock and using it for inverse quantization, the image decoding apparatus 200 can more easily grasp the presence or absence of the quantization parameter submb_qp_delta. The quantization value can be calculated more easily without unnecessary processing such as searching for a submb_qp_delta that does not exist.

As mentioned above, although the image coding apparatus which performs the encoding by the system based on AVC, and the image decoding apparatus which performs the decoding by the system based on AVC were demonstrated as an example in 1st Embodiment thru | or 3rd embodiment, The range of application is not limited to this, and it is possible to apply to all the picture coding apparatuses and the picture decoding apparatuses which perform the coding process based on the block having a hierarchical structure as shown in FIG.

In addition, the various quantization parameters described above may be added to arbitrary positions of the encoded data, for example, or may be transmitted to the decoding side unlike the encoded data. For example, the reversible coding unit 106 may describe these information as syntax in the bit stream. In addition, the reversible encoding unit 106 may store these information as auxiliary information in a predetermined area and transmit the information. For example, the information may be stored in a parameter set (e.g., a header of a sequence or a picture) such as SEI (Suplemental Enhancement Information).

In addition, the reversible coding unit 106 may transmit these information from the image coding apparatus to the image decoding apparatus separately from the encoded data (as another file). In that case, it is necessary to clarify the relationship between these pieces of information and the encoded data (so that the decoding side can grasp), but the method is arbitrary. For example, table information indicating a correspondence relationship may be created separately, or link information indicating data of a correspondence destination may be embedded in each other's data.

In addition, the quantization (calculation of the quantization parameter for each sub macroblock) using the quantization value for each sub macroblock described above may be performed only for an extended macroblock of 32x32 or more.

For example, the rate control unit 117 calculates an activity for each sub macroblock only when the macroblock to be processed is an extended macroblock, and the conventional 16 is defined in an existing coding standard such as AVC. For macro blocks of x 16 or less, the activity is calculated for each macro block.

For example, the sub-macroblock quantization unit 121 calculates the quantization value for each sub-macroblock only for the extended macroblock, and the quantization value for each macroblock for the conventional 16 × 16 or less macroblock. Calculate.

The quantization parameter calculator 152, for example, calculates the quantization parameter Submb_qp_delta only for the extended macroblock, and does not calculate the quantization parameter submb_qp_delta for the conventional 16x16 or less macroblock.

The quantization processing unit 153 performs quantization using, for example, only the extended macroblock using the quantization value for each submacroblock. For the conventional macroblock of 16 × 16 or less, the quantization value for each macroblock is determined. To perform quantization.

By doing the above, the image encoding apparatus 100 performs quantization using the quantization value for each sub-macroblock only for the extended macroblock of a large area which can sufficiently anticipate the deterioration effect of the subjective image quality of the decoded image, For a macro block of a conventional size, where the expectation of the effect is relatively small, quantization using the quantization value for each macro block can be performed. As a result, the image encoding apparatus 100 can suppress an increase in load caused by quantization using the quantization value for each sub macroblock.

In this case, of course, the image decoding apparatus 200 may perform inverse quantization using the quantization value for each sub-macroblock only for the extended macroblock, similarly to the image encoding apparatus 100.

For example, the sub macroblock dequantization unit 221 calculates a quantization value for each sub macroblock only for the extended macroblock, and for a macroblock of 16 × 16 or less in the related art, the quantization value for each macroblock is calculated. Calculate.

Therefore, the inverse quantization processing unit 253 performs inverse quantization using, for example, only the extended macroblock using quantization values for each submacroblock, and for each macroblock of a conventional 16 × 16 or less macroblock. Inverse quantization is performed using the quantization value of.

By doing in this way, the image decoding apparatus 200 performs inverse quantization using the quantization value for each sub macroblock only for the extended macroblock of a large area which can fully anticipate the suppression effect of the subjective image quality of the decoded image, In the case of a macro block of a conventional size having a relatively small expectation of the effect, inverse quantization using quantization values for each macro block can be performed. Thereby, the image decoding apparatus 200 can suppress the increase of the load by performing inverse quantization using the quantization value for every sub macroblock.

As in the third embodiment, when the submb_qp_present_flag is transmitted, this quantization parameter submb_qp_present_flag may be transmitted only for the extended macroblock. In other words, transmission of this quantization parameter submb_qp_present_flag can be omitted for a macro block of a conventional size. Of course, a quantization parameter submb_qp_present_flag of a value indicating that there is no quantization parameter submb_qp_delta whose value is other than "0" may be transmitted to a macro block of a conventional size.

<4. Fourth Embodiment>

[summary]

In the above, it was demonstrated to specify the quantization parameter in the unit of the sub macroblock, but the method of assigning the quantization parameter to the sub macroblock may be other than that described above. For example, the quantization parameter SubMB_QP assigned to each sub macroblock is defined using the quantization parameter submb_qp_delta for each sub macroblock and the quantization parameter previous_qp of the submacroblock coded immediately before, as shown in Equation (11) below. You may do so.

SubMB_QP = Clip (0,51, previous_qp + submb_qp_delta)... (11)

[Coding unit]

Such a method will be described below, but in the following description, a unit called a coding unit is described instead of the above-described macroblock or sub-macroblock.

For example, in "Test Model Under Consideration" (JCTVC-B205), the extended macroblock described with reference to FIG. 4 is defined by the concept of a coding unit.

A coding unit is a division unit of an image (one picture) which becomes a processing unit, such as encoding process of image data. That is, the coding unit is a block (partial region) in which a picture (one picture) is divided into a plurality. In other words, the coding unit corresponds to the above-described macro block or sub macro block.

15 is a diagram illustrating a configuration example of a coding unit. As shown in FIG. 15, the coding unit can divide the area | region further into several, and can make each area | region into a coding unit below one layer. That is, the coding unit may be configured to be hierarchical (configured in a tree structure). In other words, the size of the coding unit is arbitrary, and coding units of different sizes may exist in one picture.

In the example of FIG. 15, the size of the coding unit of the uppermost layer (Depth = 0) is 128 pixels x 128 pixels, and the area of 64 pixels x 64 pixels, which is divided into two equally (four divisions) vertically and horizontally, is located below one layer ( A coding unit of Depth = 1), and similarly, the hierarchization of the coding unit is repeated, and the area of 8 pixels x 8 pixels is the coding unit of the lowest layer (Depth = 4).

In this case, the coding unit of the uppermost layer is referred to as a large coding unit (LCU), and the coding unit of the lowermost layer is referred to as a smallest coding unit (SCU). That is, the LCU corresponds to a macro block, and the coding unit of the lower layer corresponds to the sub macro block.

In addition, the size, shape, and number of layers of the coding unit of each layer are arbitrary. In other words, the size and shape of the LCU and the SCU do not have to coincide in the image (one picture). The number of hierarchies of the coding units may be different depending on the position in the image, and the method of segmentation of the region is also arbitrary. In other words, the tree structure of the coding unit can be any structure.

Of course, for example, the division method of the regions is common, and the degree of freedom of the hierarchical structure of the coding unit may be partially limited, such that only the number of hierarchies can be different. For example, as shown in FIG. 15, at any position, one area (one picture or one coding unit) is divided into two equally (ie, four divisions) horizontally and horizontally, and the size of the LCU and SCU at each position is defined. By doing this, the hierarchical structure of the coding unit may be defined.

The sizes of the LCU and the SCU can be specified, for example, in the sequence parameter set of the image compression information. Of course, it may be specified in other metadata.

[Assignment of Quantization Parameters]

In this embodiment, the quantization parameter submb_qp_delta is assigned to each coding unit instead of the macro block or the sub macro block. In this case, however, the quantization parameter submb_qp_delta is not a difference value between the quantization parameter MB_QP for each macroblock and the quantization parameter SubMB_QP for each submacroblock, but the quantization parameter previous_qp of the coding unit encoded before, and the coding for the current processing target. The difference value of the quantization parameter SubMB_QP of the unit.

In other words, each coding unit is assigned a quantization parameter previous_qp used for the previous encoding and a quantization parameter submb_qp_delta indicating the difference between the quantization parameters SubMB_QP of the coding unit to be processed. That is, the quantization parameter submb_qp_delta satisfying the above expression (11) is assigned to each coding unit.

In addition, since it is only necessary to quantize the entire area in the image, in practice, the quantization parameter submb_qp_delta is allocated to some coding units, for example, only the SCU.

Similar to the other embodiments described above, the quantization parameter SubMB_QP of the coding unit to be processed currently can be obtained by converting the quantization value obtained from the activity of the coding unit. Therefore, the quantization parameter submb_qp_delta for each coding unit can be calculated using Formula (11).

16 shows an example of the configuration of a coding unit in 1LCU and an example of a quantization parameter assigned to each coding unit. As illustrated in FIG. 16, each coding unit CU is assigned a quantization parameter previous_qp used for the previous encoding and a difference value ΔQP of the quantization parameter SubMB_QP of the coding unit to be processed currently as a quantization parameter.

More specifically, the quantization parameter ΔQP ₀ is assigned to coding unit 0 on the upper left side of this LCU. The quantization parameter ΔQP ₁₀ is assigned to the coding unit 10 (Coding Unit 10) on the upper left side among the four coding units on the upper right side in this LCU. The quantization parameter ΔQP ₁₁ is assigned to the coding unit 11 at the upper right side among the four coding units at the upper right side in this LCU. The quantization parameter ΔQP ₁₂ is assigned to the coding unit 12 on the lower left side among the four coding units on the upper right side in this LCU. The quantization parameter ΔQP ₁₃ is assigned to the coding unit 13 at the lower right among the four coding units at the upper right in the LCU.

The quantization parameter ΔQP ₂₀ is assigned to the coding unit 20 at the upper left of the four coding units at the lower left in the LCU. The quantization parameter ΔQP ₂₁ is assigned to the coding unit 21 at the upper right side among the four coding units at the lower left side in this LCU. The quantization parameter ΔQP ₂₂ is assigned to the coding unit 22 at the lower left of the four coding units at the lower left in this LCU. The quantization parameter ΔQP ₂₃ is assigned to the coding unit 23 on the lower right side among the four coding units on the lower left side in this LCU. And, the encoding unit 3 of the lower right (Coding Unit 3) the quantization parameter ΔQP ₃ are assigned to in the LCU.

The quantization parameter of the coding unit processed immediately before this LCU is called PrevQP. In addition, the coding unit 0 (Coding Unit 0) in the upper left part of this LCU is a coding unit processed first in this LCU, and it is assumed that it is a current process object.

The quantization parameter CurrentQP of the coding unit to be processed currently is calculated as shown in Equation (12) below.

CurrentQP = PrevQP + ΔQP ₀ ... (12)

It is assumed that the coding unit to be processed next to the coding unit 0 is the coding unit 10 (Coding Unit 10) at the upper left of the four coding units at the upper right in the LCU shown in FIG.

When this coding unit 10 becomes a processing object, the quantization parameter CurrentQP of the coding unit of the current processing object is calculated as in the following equations (13) and (14).

PrevQP = CurrentQP... (13)

CurrentQP = PrevQP + ΔQP10... (14)

In this way, since the quantization parameter assigned to each coding unit is a difference value between the quantization parameter of the coding unit encoded immediately before and the quantization parameter of the current processing target, it is not necessary to calculate the quantization parameter for each macro block. The quantization process can be performed more easily.

When the difference value between the quantization parameter of the coding unit to be encoded and the quantization parameter of the current processing target is calculated, the coding unit coded before the coding unit of the current processing target (than the coding unit encoded immediately before in the LCU) It is also possible to calculate the difference value with the previously encoded coding unit). However, it is suitable to take the difference value between the quantization parameter of the coding unit encoded immediately before and the quantization parameter of the current processing target.

In other words, when calculating the difference value between the quantization parameter of the coding unit encoded immediately before and the quantization parameter of the current processing target, only the quantization parameter of the coding unit encoded immediately before may be stored in the memory. The quantization parameter may be managed in a first-in, first-out (form). Therefore, when calculating the difference value of a quantization parameter, management of a quantization parameter becomes easy and the amount of memory used also becomes small, and there exists an advantage in a mounting surface.

In addition, the quantization parameter cu_qp_delta for each coding unit is defined in the syntax of the coding unit, for example, as shown in FIG. 17 and transmitted to the decoding side. In other words, the quantization parameter cu_qp_delta for each coding unit corresponds to the quantization parameter sub_qp_delta described above.

[Image coding device]

18 is a block diagram illustrating a main configuration example of a picture coding apparatus to which the present technology is applied. As described above, the picture coding apparatus 300 illustrated in FIG. 18 assigns the quantization parameter cu_qp_delta to each coding unit.

As shown in FIG. 18, the picture coding apparatus 300 basically has the same configuration as that of the picture coding apparatus 100 of FIG. 1. However, the picture coding apparatus 300 replaces the quantization unit 105, the rate control unit 117, and the sub macroblock quantization unit 121 of the picture coding apparatus 100 with the coding unit quantization unit 305 and the rate. It has a control unit 317. In addition, the image coding apparatus 300 includes a coding unit inverse quantization unit 308 instead of the inverse quantization unit 108 and the sub macroblock inverse quantization unit 122 of the image encoding apparatus 100.

The rate control unit 317 controls the rate of the quantization operation of the coding unit quantization unit 305 so that overflow or underflow does not occur based on the compressed image accumulated in the accumulation buffer 107. The rate control unit 317 also provides the coding unit quantization unit 305 with information indicating the complexity of the image for each coding unit. The coding unit quantization unit 305 uses the activity to perform quantization for each coding unit. In addition, the coding unit quantization unit 305 calculates a quantization parameter for each coding unit. The coding unit quantization unit 305 supplies the reversible coding unit 106 with the orthogonal transform coefficients (coefficient data) quantized for each coding unit and the calculated quantization parameter for each coding unit, encodes and transmits them. The coding unit quantization unit 305 also provides the coding unit dequantization unit 308 with the orthogonal transform coefficients (coefficient data) quantized for each coding unit and the quantization parameter for each coding unit calculated.

The coding unit dequantization unit 308 performs inverse quantization for each coding unit using the quantization parameter for each coding unit supplied from the coding unit quantization unit 305. The coding unit dequantization unit 308 supplies the inverse orthogonal transform coefficient (coefficient data) dequantized for each coding unit to the inverse orthogonal transform unit 109. The details of the coding unit dequantization unit 308 will be described later in the description of the image decoding device.

[Detailed Configuration on Quantization]

19 is a block diagram illustrating a detailed configuration example of the rate control unit 317 and the coding unit quantization unit 305.

As shown in FIG. 19, the rate control unit 317 has an activity calculating unit 321 and an activity buffer 322.

The activity calculating unit 321 acquires an image (coding unit to be processed) of an encoding process target from the screen rearrangement buffer 102, and is an activity that is information indicating dispersion of pixel values as information indicating the complexity of the image. Calculate In other words, the activity calculating unit 321 calculates the activity for each coding unit. In addition, since the quantization process may be performed for the entire image, the activity may be calculated only for some coding units, for example, only for SCU.

The activity buffer 322 holds an activity for each coding unit calculated by the activity calculating unit 321, and provides it to the quantization unit 105 at a predetermined timing. The activity buffer 322 holds, for example, one screen of activities for each of the acquired coding units.

The activity calculation method is arbitrary, for example, the method similar to the MPEG-2 Test Model mentioned above may be sufficient. In addition, the content of the information which shows the complexity of an image is also arbitrary, and information other than such an activity may be sufficient.

The coding unit quantization unit 305 includes a coding unit quantization value calculator 331, a picture quantization parameter calculator 332, a slice quantization parameter calculator 333, a coding unit quantization parameter calculator 334, and a coding unit. It has a quantization unit 335.

The coding unit quantization value calculator 331 calculates a quantization value for each coding unit based on the activity (information indicating the complexity of the image for each coding unit) for each coding unit supplied from the rate control unit 317. The quantization value for each coding unit can be calculated by the same method as that for calculating the quantization value for each LCU from the activity for each LCU. In addition, since the quantization process may be performed for the entire image, the quantization value for each coding unit may be calculated for only one coding unit. In the following, as an example, the quantization value for each coding unit is calculated only for the SCU.

When the quantization value is obtained for each coding unit, the coding unit quantization value calculation unit 331 supplies the quantization value for each coding unit to the picture quantization parameter calculation unit 332.

The picture quantization parameter calculator 332 obtains the quantization parameter pic_init_qp_minus26 for each picture using the quantization value for each coding unit.

The slice quantization parameter calculator 333 obtains a quantization parameter slice_qp_delta for each slice using the quantization value for each coding unit.

The coding unit quantization parameter calculator 334 calculates the quantization parameter cu_qp_delta for each coding unit using the quantization parameter prevQP used for the previous encoding.

The quantization parameter generated by the picture quantization parameter calculator 332 or the coding unit quantization parameter calculator 334 is supplied to the reversible encoder 106, encoded, transmitted to the decoding side, and the coding unit dequantizer Also supplied to 308.

The coding unit quantization unit 335 quantizes the orthogonal transform coefficients of the coding unit to be processed, using the quantization value for each coding unit.

The coding unit quantization unit 335 supplies the orthogonal transform coefficients quantized for each coding unit to the reversible coding unit 106 and the coding unit inverse quantization unit 308.

[Flow of encoding process]

The image encoding apparatus 300 performs encoding processing basically similarly to the case of the image encoding apparatus 100 of FIG. 1 described with reference to FIG. 6.

[Flow of quantization parameter calculation processing]

An example of the flow of the quantization parameter calculation process performed in the encoding process will be described with reference to the flowchart in FIG. 20.

When the quantization parameter calculation process is started, in step S531, the coding unit quantization value calculation unit 331 acquires the activity for each coding unit supplied from the rate control unit 317.

In step S532, the coding unit quantization value calculator 331 calculates the quantization value for each coding unit using the activity for each coding unit.

In step S533, the picture quantization parameter calculator 332 obtains the quantization parameter pic_init_qp_minus26 using the quantization value for each coding unit calculated in step S532.

In step S534, the slice quantization parameter calculator 333 calculates the quantization parameter slice_qp_delta using the quantization value for each coding unit calculated in step S532.

In step S535, the coding unit quantization parameter calculator 334 calculates the quantization parameters cu_qp_delta (ΔQP ₀ to ΔQP ₂₃ in FIG. 16, etc.) for each coding unit using the quantization parameter prevQP used for the previous encoding.

When various quantization parameters are obtained as described above, the coding unit quantization unit 305 ends the quantization parameter calculation process and performs subsequent processing of the encoding process.

Since the encoding process and the quantization parameter calculation process are performed as described above, the image encoding apparatus 300 can set the quantization value for each coding unit, and can perform more appropriate quantization processing according to the content of the image.

In addition, since the quantization parameter calculated as described above is transmitted to the image decoding apparatus, the image encoding apparatus 300 can cause the image decoding apparatus to dequantize each coding unit.

In addition, the coding unit dequantization unit 308 included in the image encoding apparatus 300 performs the same processing as the coding unit dequantization unit included in the image decoding apparatus corresponding to the image encoding apparatus 300. That is, the picture coding apparatus 300 can also perform dequantization for each coding unit.

[Image decoding device]

21 is a block diagram illustrating a main configuration example of an image decoding device to which the present technology is applied. The image decoding apparatus 400 shown in FIG. 21 corresponds to the image encoding apparatus 300 described above, correctly decodes a code stream (coded data) generated by the image encoding apparatus 300 by encoding the image data, Generates a decoded image.

As shown in FIG. 21, the image decoding apparatus 400 basically has the same structure as the image decoding apparatus 200 of FIG. 8, and performs the same process. However, the image decoding device 400 has a coding unit dequantization unit 403 instead of the inverse quantization unit 203 and the sub macroblock dequantization unit 221 of the image decoding device 200.

The coding unit dequantization unit 403 dequantizes the orthogonal transform coefficients quantized for each coding unit in the image coding apparatus 300 using quantization parameters for each coding unit supplied from the image coding apparatus 300, and the like. do.

22 is a block diagram illustrating a main configuration example of the coding unit dequantization unit 403. As shown in FIG. 22, the coding unit dequantization unit 403 includes a quantization parameter buffer 411, an orthogonal transform coefficient buffer 412, a coding unit quantization value calculation unit 413, and a coding unit dequantization processing unit ( 414).

The quantization parameter in each layer, such as a picture parameter set of the coded data supplied from the picture coding apparatus 300 and a slice header, is decoded in the reversible decoding unit 202, and the quantization parameter buffer 411 Supplied to. The quantization parameter buffer 411 appropriately holds the quantization parameter and supplies the quantization parameter to the coding unit quantization value calculator 413 at a predetermined timing.

The coding unit quantization value calculation unit 413 uses the quantization parameter supplied from the quantization parameter buffer 411, for example, in the same manner as in the equations (36) to (39), for each coding unit. Is calculated and supplied to the coding unit dequantization processing unit 414.

In the reversible decoding unit 202, the quantized orthogonal transform coefficients obtained by decoding the encoded data supplied from the image coding apparatus 300 are supplied to the orthogonal transform coefficient buffer 412. The orthogonal transform coefficient buffer 412 appropriately holds the quantized orthogonal transform coefficient and supplies it to the coding unit dequantization processing unit 414 at a predetermined timing.

The coding unit inverse quantization processing unit 414 inverses the quantized orthogonal transform coefficients supplied from the orthogonal transform coefficient buffer 412 using the quantization values for each coding unit supplied from the coding unit quantization value calculation unit 413. Quantize. The coding unit dequantization processing unit 414 supplies the orthogonal transform coefficients obtained by inverse quantization to the inverse orthogonal transform unit 204.

As described above, the coding unit dequantization unit 403 can perform inverse quantization processing using the quantization value calculated for each coding unit. Thereby, the image decoding device 400 can perform inverse quantization processing more suitable for the content of an image. In particular, even when the macroblock size is extended (large LCU size), even if the single LCU includes both a flat area and an area including a texture, the image decoding device 400 is provided in each area. Appropriate adaptive dequantization processing can be performed to suppress deterioration in subjective image quality of the decoded image.

In addition, the coding unit dequantization unit 308 of the picture coding apparatus 300 shown in FIG. 18 also has the same configuration as that of the coding unit dequantization unit 403, and performs the same processing. However, the coding unit inverse quantization unit 308 obtains the quantization parameter and the quantized orthogonal transform coefficients supplied from the coding unit quantization unit 305 and performs inverse quantization.

[Flow of Decoding Process]

The image decoding apparatus 400 basically performs a decoding process similarly to the case of the image decoding apparatus 200 of FIG. 8 described with reference to the flowchart of FIG. 10.

[Flow of Dequantization Process]

An example of the flow of the inverse quantization process performed in the decoding process by the image decoding device 400 will be described with reference to the flowchart in FIG. 23.

When the dequantization process is started, the quantization parameter buffer 411 acquires the quantization parameter pic_init_qp_minus26 supplied from the reversible decoding unit 202 in step S631.

In step S632, the quantization parameter buffer 411 acquires the quantization parameter slice_qp_delta supplied from the reversible decoding unit 202.

In step S633, the quantization parameter buffer 411 acquires the quantization parameter cu_qp_delta supplied from the reversible decoding unit 202.

In step S634, the coding unit quantization value calculation unit 413 uses the various quantization parameters acquired by the processes of steps S631 to S633 and the quantization parameter PrevQP used immediately before, to determine the quantization value for each coding unit. Calculate.

In step S635, the coding unit dequantization processing unit 414 uses the quantized orthogonal transform coefficients held in the orthogonal transform coefficient buffer 412, using the quantized values for each coding unit calculated by the processing of step S634. Dequantize.

When the process of step S635 ends, the coding unit dequantization unit 403 returns the process to the decoding process and executes the subsequent process.

As described above, by performing the decoding process and the inverse quantization process, the image decoding device 400 can perform the inverse quantization process using the quantization value calculated for each coding unit, thereby performing inverse quantization process that is more suitable for the content of the image. I can do it.

As described above, in order to reduce the code amount of the quantization parameter for each coding unit (sub macroblock), the difference value dQP (quantization) between the predetermined quantization parameter and the quantization parameter SubMB_QP is not transmitted, instead of transmitting the quantization parameter SubMB_QP itself. Get the parameter submb_qp_delta) and send it. In the above, the two methods shown by following formula (15) and formula (16) were demonstrated as a calculation method of this quantization parameter dQP.

dQP = CurrentQP-LCUQP. (15)

dQP = CurrentQP-PreviousQP. (16)

In Formulas (15) and (16), CurrentQP is a quantization parameter of the coding unit CU to be processed. In addition, LCUQP is a quantization parameter of the LCU (that is, the LCU to be processed) to which the CU to be processed belongs. In addition, PreviousQP is a quantization parameter of the CU processed immediately before the CU to be processed.

In other words, in the equation (15), the difference between the quantization parameter of the LCU to be processed and the quantization parameter of the CU to be processed is transmitted. In addition, in the formula (16), the difference value between the quantization parameter of the immediately processed CU and the quantization parameter of the CU currently being processed is transmitted.

The calculation method of such quantization parameter dQP for transmission is arbitrary, and may be other than the two examples mentioned above.

For example, as shown in equation (17) below, the difference between the quantization parameter SliceQP of the slice (that is, the slice to be processed) to which the CU to be processed belongs and the quantization parameter of the CU to be processed is transmitted. You may also

dQP = CurrentQP-SliceQP. (17)

The quantization parameter CurrentQP can be obtained, for example, by the coding unit quantization parameter calculator 334 of FIG. 19 converting the quantization value of the CU to be processed calculated by the coding unit quantization value calculator 331. In addition, the quantization parameter SliceQP is obtained, for example, by the slice quantization parameter calculator 333 of FIG. 19 using the quantization parameter pic_init_qp_minus26 obtained by the picture quantization parameter calculator 332 and the quantization parameter slice_qp_delta obtained by the self. Can be.

Therefore, for example, the coding unit quantization parameter calculator 334 of FIG. 19 can obtain the quantization parameter dQP using these values. The coding unit quantization parameter calculator 334 supplies the quantization parameter dQP to the reversible encoder 106 and transmits it to the decoding side.

The quantization parameter pic_init_qp_minus26 and the quantization parameter slice_qp_delta are defined, for example, in "Test Model Under Consideration" (JCTVC-B205) and can be set in the same manner as in the conventional coding scheme.

On the decoding side, the quantization parameter of the CU can be obtained from the quantization parameter dQP transmitted from the encoding side.

For example, the coding unit quantization value calculation unit 413 obtains the quantization parameter SubMB_QP of the CU from the quantization parameter dQP, as shown in Equation (18) below, and converts it to obtain a quantization value.

SubMB_QP = Clip (minQP, maxQP, SliceQP + submb_qp_delta)... (18)

In Formula (18), minQP is a predefined minimum quantization parameter, and maxQP is a predefined maximum quantization parameter.

In this manner, when the quantization parameter SliceQP is used to obtain the quantization parameter dQP, quantization and inverse quantization can be performed similarly to the case of the two methods described above. That is, not only can the quantization and inverse quantization be more suitable for the content of the image, but the code amount of the quantization parameter can be reduced.

A table comparing the characteristics of the processing of each of these methods is shown in FIG. 24. In the table shown in FIG. 24, the uppermost method (called the first method) is a method of obtaining the quantization parameter dQP using the quantization parameter of the LCU. The second method (called the second method) above is a method of obtaining the quantization parameter dQP using the quantization parameter of the CU processed immediately before the CU to be processed. The lowest method (called the third method) is a method of obtaining a quantization parameter dQP using the quantization parameter of a slice to be processed.

The table in Fig. 24 compares the ease of pipeline processing and the coding efficiency as a feature of each method. As shown in the table of FIG. 24, the first method is easier for the pipeline processing than the second method. In addition, the third method is easier to process the pipeline than the first method. Also, the first method has better coding efficiency than the third method. In addition, the second method has better coding efficiency than the first method.

That is, generally, the correlation with the area | region (a coding unit, a sub macroblock, etc.) of a process target is so high that it is a region nearer to the process target area. Therefore, the coding efficiency of the quantization parameter dQP can be further improved by obtaining the quantization parameter dQP using the area closer to the area to be processed.

In general, the further the area is spaced from the area to be processed, the faster the processing order is. Therefore, the time until the area to be processed is processed becomes long. That is, the allowable time for processing delay and the like becomes long. Therefore, by obtaining the quantization parameter dQP using an area further separated from the area to be processed, delay is less likely to occur, which is advantageous for pipeline processing.

As mentioned above, since each method has a different characteristic from each other, which method is more suitable depends on the conditions on which it is prioritized. Moreover, you may be able to select each method. This selection method is arbitrary. For example, a user may decide in advance which method to apply, for example. For example, either method may be adaptively selected according to an arbitrary condition (for example, at every processing unit or at an event occurrence).

In addition, when one method is appropriately selected, flag information indicating which method is selected may be generated, and the flag information may be transmitted from the encoding side (quantization side) to the decoding side (inverse quantization side). do. In that case, the decoding side (inverse quantization side) can select the same method as the encoding side (quantization side) by referring to the flag information.

In addition, the calculation method of the quantization parameter dQP is arbitrary, and may be other than the above-mentioned. The number of calculation methods to prepare is arbitrary. In addition, the number may be made variable. The information defining the quantization parameter dQP may be transmitted from the encoding side (quantization side) to the decoding side (inverse quantization side).

Considering the features of the above-described methods, a method of calculating the difference value of the quantization parameter is exemplified. 25 shows an example of the configuration of the LCU and the CU. (Number) shows the processing sequence of coding (decoding) of a coding unit.

In LCU (0), the coding order of the coding units is as follows:

In this case, the difference value of the quantization parameter is as follows:

In the LCU, the first coding unit CU (0) uses equation (17) to quantize the quantization parameter SliceQP of the slice to which CU (0) belongs (that is, the slice to be processed) and the CU (0) to be processed. Send the difference value of the parameter.

dQP (CU (0)) = CurrentQP (CU0) -SliceQP

Next, the coding units CU 10 to CU 3 other than the head in the LCU use the equation (16) to determine the difference between the quantization parameter (CurrentCU) of the CU to be processed and the CU (PrevisousCU) encoded immediately before. Send the value.

dQP = CurrentQP (CUi) -PreviousQP (CUi-1)

That is, using FIG. 25, the difference value of the quantization parameter is as follows:

dQP (CU (10)) = CurrentQP (CU (10))-PrevisouQP (CU (0))

dQP (CU (11)) = CurrentQP (CU (11))-PrevisouQP (CU (10))

dQP (CU (12)) = CurrentQP (CU (12))-PrevisouQP (CU (11))

dQP (CU (13)) = CurrentQP (CU (13))-PrevisouQP (CU (12))

dQP (CU (20)) = CurrentQP (CU (20))-PrevisouQP (CU (13))

dQP (CU (21)) = CurrentQP (CU (21))-PrevisouQP (CU (20))

dQP (CU (30)) = CurrentQP (CU (30))-PrevisouQP (CU (21))

dQP (CU (31)) = CurrentQP (CU (31))-PrevisouQP (CU (30))

dQP (CU (32)) = CurrentQP (CU (32))-PrevisouQP (CU (31))

dQP (CU (33)) = CurrentQP (CU (33))-PrevisouQP (CU32))

dQP (CU (23)) = CurrentQP (CU (23))-PrevisouQP (CU33))

dQP (CU (3)) = CurrentQP (CU (3))-PrevisouQP (CU23)

Similarly for the other LCUs 1 to LCU (N), the difference value of the quantization parameter is calculated and transmitted.

In this way, by calculating and transmitting the difference value of the quantization parameter, by adopting the advantages (double circle in the figure) of the characteristics of each method, both the ease of pipeline processing and the coding efficiency can be achieved together.

In addition, from an implementation point of view, when performing closed control in LUC, the head coding unit CU (0) in the LCU may calculate the difference value of the quantization parameter using equation (15).

In addition, the quantization parameter dQP described so far does not need to be set in every coding unit, but only a CU which wants to set a value different from the quantization parameter as a reference such as LCUQP, PreviousQP, SliceQP, and the like.

Therefore, for example, a syntax of MinCUForDQPCoded may be added to the slice header (SliceHeader).

26 is a diagram illustrating an example of syntax of a slice header. The number at the left end of each line is a line number appended for explanation.

In the example of FIG. 26, MinCUForDQPCoded is set in the 22nd line. This MinCUForDQPCoded specifies the minimum CU size for setting dQP. For example, even if the minimum size of a CU is 8 × 8, when MinCUForDQPCoded = 16, the coding unit quantization parameter calculation unit 334 of the picture coding apparatus 300 only uses a CU of size 16 × 16 or more. Set dQP, and do not set dQP in 8x8 CU. That is, in this case, dQP of a CU of size 16 × 16 or more is transmitted. MinCUForDQPCoded is a method of designating a minimum CU size for setting dQP, and dQP is selected from CU sizes (4 × 4, 8 × 8, 16 × 16, 32 × 32, etc.) set during encoding (decoding). It may be set as a flag (for example, 0: 4x4, 1: 8x8, 2: 16x16, etc.) which identifies (selects) the minimum CU size to be set.

For example, if the person making the encoder wants only control of a 16 × 16 CU, the 8 × 8 CU needs to transmit all the dQPs to 0, thereby reducing the coding efficiency. There is.

Therefore, by setting such syntax MinCUForDQPCoded, transmission of dQP of 8x8 CU in this case can be omitted, and a reduction in coding efficiency can be suppressed.

The coding unit quantization value calculation unit 413 of the image decoding device 400 determines that the dQP of the 8x8 CU is not transmitted according to this syntax, and quantizes as a reference such as LCUQP, PreviousQP, SliceQP, or the like. The parameter is used to calculate the quantization value.

MinCUForDQPCoded may be stored in addition to the slice header. For example, it may be stored in a picture parameter set (PictureParameterSet). By storing in a slice header or a picture parameter set, it is possible to cope with an operation such as changing this value after a scene change, for example.

However, since MinCUForDQPCoded is stored in the slice header, the picture can be multi-sliced and processed in parallel for each slice, which is more preferable.

<5. Fifth Embodiment>

[summary]

In the above description, the quantization parameter for each sub macroblock (coding unit smaller than the LCU) has been described so as to be transmitted from the picture coding apparatus to the picture decoding apparatus. It must be possible to obtain a quantization parameter for each) and use it to perform quantization for each sub macroblock (coding unit smaller than the LCU).

Thus, although the image coding apparatus performs quantization processing for each sub macroblock (coding unit smaller than LCU), the quantization parameter is set for each macroblock LCU, and the image decoding is performed for the quantization parameter for each macroblock LCU. It may be provided to the device.

For example, when the picture coding apparatus calculates the activity of each macro block (LCU) by TestModel 5 described above, even if the size of the macro block (LCU) is 64x64 or 128x128, the macroblock ( Activity is calculated in units of blocks (coding units), such as 8 × 8 or 16 × 16, smaller than LCU).

Then, the image encoding apparatus determines the quantization parameter value in 8 × 8 blocks or 16 × 16 blocks based on the method of TestModel5 based on the activities in units of 8 × 8 or 16 × 16 blocks.

However, the quantization parameter is set for each macro block (LCU).

For example, as shown in FIG. 27, it is assumed that the size of the LCU (macro block) is 64x64 pixels. When the picture coding apparatus calculates an activity for each 16x16 coding unit and calculates a quantization parameter for this LCU, the activity of each coding unit (block) is assumed to be QP ₀₀ to QP ₃₃ .

In the case of AVC, as shown in Fig. 28, the quantization parameter QP is designed such that, for example, from 6 to 12, when the value increases by 6, roughly doubled quantization processing is performed.

In addition, especially at a lower bit rate, i.e., higher QP, the degradation in the color difference signal is likely to be noticeable. Thus, for the quantization parameter QPY for the luminance signal, the default quantization parameter QPC for the chrominance signal is predefined.

The user can control this relationship by setting the information about ChromaQPOffset included in the image compression information.

In contrast, in the present embodiment, the image coding device determines, as the first step, the quantization parameter QP _MB for the macroblock as in the following equation (3).

2 as carried out by a step, quantization of the blocks, use the value of QP QP ₀₀ to _33. As a result, in each block, the position of the coefficient to be non-zero (zero) is stored in the memory.

As a third step, quantization processing of each block is performed using the value of QP _MB .

As a 4th step, only the value of the coefficient position which is a non-zero coefficient also in the 2nd step among the ratio 0 obtained by a 3rd step is transmitted as reversible coding information as encoding information.

By performing such a process, only QP _MB is the quantization parameter transmitted to the image compression information. However, each block is pseudo-simulated by using values of QP ₀₀ to QP ₃₃ to realize adaptive quantization and output. It is possible to improve the subjective image quality of the resulting image compression information.

[Image coding device]

29 is a block diagram illustrating a main configuration example of a picture coding apparatus to which the present technology is applied. As shown in FIG. 29, the image coding apparatus 500 in this case basically has the same structure as the image coding apparatus 100 of FIG. 1, and performs the same process.

However, the image coding apparatus 500 replaces the rate control unit 317 and the coding unit quantization instead of the quantization unit 105, the rate control unit 117, and the sub macroblock quantization unit 121 of the image coding apparatus 100. And a quantization unit 505.

In addition, the image encoding apparatus 500 includes only the inverse quantization unit 108, whereas the image encoding apparatus 100 of FIG. 1 includes the sub macroblock inverse quantization unit 122 in addition to the inverse quantization unit 108. Have That is, inverse quantization processing is performed for each LCU (macro block) similarly to the conventional AVC. This also applies to the image decoding apparatus corresponding to this image encoding apparatus 500.

The coding unit quantization unit 504 quantizes each coding unit (for example, SCU) using the activities for each coding unit obtained by the rate control unit 317.

The quantization unit 505 obtains a quantization parameter for each LCU and uses it to quantize each coding unit. The quantization unit 505 quantizes the non-zero coefficients of the quantized orthogonal transform coefficients of the respective coding units obtained by the coding unit quantization unit 504 by the quantization unit 505 at the same position. Replace with the processing result (quantized orthogonal transformation coefficient).

The result of this substitution is supplied to the reversible coding unit 106 or the inverse quantization unit 108 as a quantization result. The quantization parameter for each LCU calculated by the quantization unit 505 is supplied to the reversible coding unit 106 or the inverse quantization unit 108.

The inverse quantization unit 108 and the inverse quantization unit (not shown) of the image decoding device perform inverse quantization using quantization parameters for each LCU, as in the case of the conventional AVC.

[Configuration of rate control unit, coding unit quantization unit, quantization unit]

30 is a block diagram illustrating a detailed configuration example of a rate control unit, a coding unit quantization unit, and a quantization unit in FIG. 29.

As shown in FIG. 30, the coding unit quantization unit 504 includes a coding unit quantization parameter determination unit 511, a coding unit quantization processing unit 512, and a non-zero coefficient position buffer 513.

The coding unit quantization parameter determination unit 511 uses the activity for each coding unit (for example, SCU) lower than the LCU, which is supplied from the activity buffer 322 of the rate control unit 317, to code the lower layer than the LCU. The quantization parameter CU_QP is determined for each unit (e.g., SCU). The coding unit quantization parameter determination unit 511 supplies the quantization parameter CU_QP for each coding unit to the LCU quantization parameter determination unit 522 of the coding unit quantization processing unit 512 and the quantization unit 505.

The coding unit quantization processing unit 512 uses the orthogonal transform supplied from the orthogonal transform coefficient buffer 521 of the quantization unit 505 using the quantization parameter CU_QP for each coding unit supplied from the coding unit quantization parameter determination unit 511. Coefficients are quantized for each coding unit (e.g., SCU) lower than the LCU. The coding unit quantization processing unit 512 sets the non-zero coefficient position buffer 513 to the position of a coding unit whose value is nonzero (nonzero coefficient) among the quantized orthogonal transform coefficients for each coding unit obtained by the quantization. ), And hold.

The non-zero coefficient position buffer 513 supplies the position of the non-zero coefficient retained at the predetermined timing to the coefficient substitution unit 524 of the quantization unit 505.

As shown in FIG. 30, the quantization unit 505 includes an orthogonal transform coefficient buffer 521, an LCU quantization parameter determination unit 522, an LCU quantization processing unit 523, and a coefficient replacement unit 524.

The orthogonal transform coefficient buffer 521 holds orthogonal transform coefficients supplied from the orthogonal transform unit 104, and stores the orthogonal transform coefficients held at the predetermined timing in the coding unit quantization processing unit 512 and the LCU quantization processing unit ( 523).

The LCU quantization parameter determiner 522 quantizes the minimum value in the LCU of the quantization parameter CU_QP for each coding unit supplied from the coding unit quantization parameter determiner 511, as described above in Expression (19). Determined by the parameter LCU_QP. The LCU quantization parameter determination unit 522 supplies the LCU quantization processing unit 523 with the quantization parameter LCU_QP (minimum value of CU_QP in the processing target LCU) for each LCU.

The LCU quantization processing unit 523 uses the quantization parameter LCU_QP for each LCU supplied from the LCU quantization parameter determination unit 522 to convert the orthogonal transform coefficients supplied from the orthogonal transform coefficient buffer 521 to a lower layer coding unit than the LCU. Quantize each (for example, SCU). The LCU quantization processing unit 523 supplies the coefficient substitution unit 524 with the quantized orthogonal transform coefficients for each coding unit obtained by the quantization.

The coefficient substitution unit 524 is a nonzero coefficient supplied from the nonzero coefficient position buffer 513 among the coefficients (nonzero coefficients) whose values are not zero of the orthogonal transform coefficients quantized by the LCU quantization processing unit 523. The coefficient of the position different from the position of is replaced with zero.

In other words, the coefficient substitution unit 524 has a non-zero value in both of the quantization using the quantization parameter CU_QP determined for each coding unit of the lower layer than the LCU and the quantization using the quantization parameter LCU_QP determined for each LCU. Only for the coding unit (which is lower than the LCU), the value of the quantization result is adopted as the quantized orthogonal transform coefficient. On the other hand, for other coding units (lower than the LCU), the coefficient replacement unit 524 sets all the values of the quantized orthogonal transform coefficients to zero.

The coefficient replacement unit 524 supplies the quantized orthogonal transform coefficients, which have been appropriately substituted with values, to the reversible coding unit 106 and the inverse quantization unit 108 together with the quantization parameter LCU_QP determined for each LCU. .

The reversible encoding unit 106 encodes the supplied coefficient data and the quantization parameter and corresponds to the image encoding apparatus 500 (which can decode the encoded data generated by the image encoding apparatus 500). Supply to. The image decoding apparatus performs dequantization using the quantization parameter LCU_QP for each LCU supplied from the image encoding apparatus 500, similarly to the conventional AVC.

The inverse quantization unit 108 also performs inverse quantization of the coefficient data supplied from the coefficient replacement unit 524 using the quantization parameter LCU_QP for each LCU supplied from the coefficient replacement unit 524.

The inverse quantization unit 108 has the same configuration as the inverse quantization unit 203 described with reference to FIG. 10. However, in the case of the inverse quantization unit 108, the inverse quantization processing unit 253 uses the quantization parameter (quantization parameter LCU_QP for each LCU) supplied from the quantization parameter buffer 251 to decode the quadrature transform coefficient buffer 252. Inversely quantizes the supplied quantized orthogonal transform coefficients.

[Flow of encoding process]

Next, an example of the flow of the encoding process performed by this image encoding apparatus 500 will be described with reference to the flowchart in FIG. 31. In this case, each process of the encoding process is basically performed similarly to each process of the encoding process demonstrated with reference to the flowchart of FIG.

That is, similarly to each process of step S101-step S104 of FIG. 7, each process of step S701-step S704 is performed. However, instead of step S105 and step S106 of FIG. 7, the quantization process of step S705 is performed. In addition, similar to each process of step S106 to step S117, each process of step S706 to step S716 is performed.

[Flow of Quantization Processing]

Next, with reference to the flowchart of FIG. 32, the example of the flow of the quantization process performed in step S705 of FIG. 31 is demonstrated.

When the quantization process is started, in step S731, the activity calculating unit 321 calculates an activity for each coding unit.

In step S732, the coding unit quantization parameter determination unit 511 determines the quantization parameter CU_QP in the coding unit unit of the lower layer than the LCU.

In step S733, the LCU quantization parameter determination unit 522 determines the quantization parameter LCU_QP in LCU units.

In step S734, the coding unit quantization processing unit 512 performs quantization using the quantization parameter CU_QP in units of coding units in the lower layer than the LCU.

In step S735, the non-zero coefficient position buffer 513 holds the position of the non-zero coefficient generated by the quantization process in step S734.

In step S736, the LCU quantization processing unit 523 performs quantization using the quantization parameter LCU_QP in LCU units.

In step S737, the coefficient replacing unit 524 replaces the value of the quantized orthogonal transform coefficient of the coding unit of the lower layer than the LCU with a position different from the position of the non-zero coefficient held by the process of step S735 to zero. do.

When the substitution ends, the quantization process ends, the process returns to step S705 in FIG. 31, and the process subsequent to step S706 is executed.

As described above, in the image information encoding apparatus which outputs the image compression information based on the encoding method using the expanded macroblock and the image information decoding apparatus which is the input, in the code unit (sub macroblock) unit lower than the LCU, By quasi-quantizing the quantization process, even in the case where a flat area and a texture area are mixed in a single LCU (macro block), the subjective picture quality can be improved by performing adaptive quantization based on the respective characteristics.

<6. Sixth Embodiment >

[Application to multi-view image coding and multi-view image decoding]

The series of processes described above can be applied to multiview image encoding and multiview image decoding. 33 shows an example of a multiview image coding method.

As shown in FIG. 33, a multi-view image contains the image of several viewpoints, The image of the predetermined | prescribed one viewpoint among the some viewpoint is designated as the image of a base view. Images at each viewpoint other than the image of the base view are treated as images of the non-base view.

In the case of performing multi-view image coding as shown in Fig. 33, in each view (same view), the difference of the quantization parameter may be taken:

(1) base-view ：

  (1-1) dQP (base view) = Current_CU_QP (base view) -LCU_QP (base view)

  (1-2) dQP (base view) = Current_CU_QP (base view) -Previsous_CU_QP (base view)

  (1-3) dQP (base view) = Current_CU_QP (base view) -Slice_QP (base view)

(2) non-base-view ：

  (2-1) dQP (non-base view) = Current_CU_QP (non-base view) -LCU_QP (non-base view)

  (2-2) dQP (non-base view) = CurrentQP (non-base view) -PrevisousQP (non-base view)

  (2-3) dQP (non-base view) = Current_CU_QP (non-base view) -Slice_QP (non-base view)

When performing multi-view image coding, in each view (different view), the difference of the quantization parameter may be taken:

(3) base-view / non-base view:

  (3-1) dQP (inter-view) = Slice_QP (base view) -Slice_QP (non-base view)

  (3-2) dQP (inter-view) = LCU_QP (base view) -LCU_QP (non-base view)

(4) non-base view / non-base view:

  (4-1) dQP (inter-view) = Slice_QP (non-base view i) -Slice_QP (non-base view j)

  (4-2) dQP (inter-view) = LCU_QP (non-base view i) -LCU_QP (non-base view j)

In this case, you may use combining said (1)-(4). For example, in non-base view, a method of taking the difference of quantization parameter at slice level between base view and non-base view (combining 3-1 and 2-3), between base view and non-base view A method (combining 3-2 and 2-1) is taken to take the difference of the quantization parameter at the LCU level. In this way, by applying the difference repeatedly, the coding efficiency can be improved even when multi-view coding is performed.

Similar to the above-described method, for each of the dQPs described above, a flag may be set for identifying whether or not a dQP having a value other than zero exists.

Multi-view Image Coding Apparatus

FIG. 34 is a diagram illustrating a multi-view image coding device which performs the multi-view image coding described above. As shown in FIG. 34, the multi-viewpoint image encoding apparatus 600 includes an encoder 601, an encoder 602, and a multiplexer 603.

The encoding unit 601 encodes the base view image and generates a base view image encoded stream. The encoding unit 602 encodes a non-base view image and generates a non-base view image encoded stream. The multiplexing unit 603 multiplexes the base view image encoded stream generated in the encoding unit 601 and the non-base view image encoded stream generated in the encoding unit 602 to generate a multi-view image encoded stream .

With respect to the coding unit 601 and the coding unit 602 of the multi-view image coding apparatus 600, the image coding apparatus 100 (FIG. 1), the image coding apparatus 300 (FIG. 18), or the image coding apparatus 500 (FIG. 29) can be applied. In this case, the multi-viewpoint image encoding apparatus 600 sets and transmits a difference value between the quantization parameter set by the encoder 601 and the quantization parameter set by the encoder 602.

[View point image decoding device]

35 is a diagram illustrating a multi-view image decoding device which performs the above-described multi-view image decoding. As shown in FIG. 35, the multiview image decoding device 610 includes a demultiplexer 611, a decoder 612, and a decoder 613.

The demultiplexer 611 demultiplexes a multiview image coded stream in which the base view picture coded stream and the non-base view picture coded stream are multiplexed, and extracts the base view picture coded stream and the non-base view picture coded stream. The decoding unit 612 decodes the base view image encoded stream extracted by the demultiplexing unit 611 to obtain a base view image. The decoding unit 613 decodes the non-base-view coded stream extracted by the demultiplexing unit 611 and obtains a non-base-view image.

The image decoding device 200 (FIG. 9) or the image decoding device 400 (FIG. 21) may be applied to the decoding unit 612 and the decoding unit 613 of the multi-view image decoding device 610. Can be. In this case, the multi-viewpoint image decoding device 610 sets quantization parameters from the difference between the quantization parameter set by the encoder 601 and the quantization parameter set by the encoder 602 to perform inverse quantization.

<7. Seventh Embodiment >

[Application to Hierarchical Image Coding and Hierarchical Image Decoding]

The above-described series of processes can be applied to hierarchical image coding and hierarchical image decoding. 36 shows an example of a multiview image coding method.

As shown in Fig. 36, the hierarchical image includes an image of a plurality of hierarchies (resolutions), and an image of a predetermined one of the plurality of resolutions is designated as an image of the base layer. Images of each layer other than the image of the base layer are treated as images of the non-base layer.

When performing hierarchical image coding (spatial scalability) as shown in FIG. 36, in each layer (same layer), a difference of quantization parameters may be taken:

(1) base-layer:

  (1-1) dQP (base layer) = Current_CU_QP (base layer) -LCU_QP (base layer)

  (1-2) dQP (base layer) = Current_CU_QP (base layer) -Previsous_CU_QP (base layer)

  (1-3) dQP (base layer) = Current_CU_QP (base layer) -Slice_QP (base layer)

(2) non-base-layer:

  (2-1) dQP (non-base layer) = Current_CU_QP (non-base layer) -LCU_QP (non-base layer)

  (2-2) dQP (non-base layer) = CurrentQP (non-base layer) -PrevisousQP (non-base layer)

  (2-3) dQP (non-base layer) = Current_CU_QP (non-base layer) -Slice_QP (non-base layer)

In the case of hierarchical coding, in each layer (different layers), the difference of the quantization parameter may be taken:

(3) base-layer / non-base layer:

  (3-1) dQP (inter-layer) = Slice_QP (base layer) -Slice_QP (non-base layer)

  (3-2) dQP (inter-layer) = LCU_QP (base layer) -LCU_QP (non-base layer)

(4) non-base layer / non-base layer:

  (4-1) dQP (inter-layer) = Slice_QP (non-base layer i) -Slice_QP (non-base layer j)

  (4-2) dQP (inter-layer) = LCU_QP (non-base layer i) -LCU_QP (non-base layer j)

In this case, you may use combining said (1)-(4). For example, in the non-base layer, a method of taking the difference of quantization parameters at the slice level between the base layer and the non-base layer (combining 3-1 and 2-3), between the base layer and the non-base layer A method (combining 3-2 and 2-1) is taken to take the difference of the quantization parameter at the LCU level. In this way, by applying the difference repeatedly, the coding efficiency can be improved even when hierarchical coding is performed.

Similar to the above-described method, for each of the dQPs described above, a flag may be set for identifying whether or not a dQP having a value other than zero exists.

Hierarchical Image Coding Apparatus

FIG. 37 is a diagram illustrating a hierarchical image coding apparatus that performs the above-described hierarchical image coding. As shown in FIG. 37, the hierarchical image coding apparatus 620 includes an encoding unit 621, an encoding unit 622, and a multiplexing unit 623.

The encoder 621 encodes the base layer image and generates a base layer image encoded stream. The encoding unit 622 encodes the non-base layer picture and generates a non-base layer picture coded stream. The multiplexing unit 623 multiplexes the base layer picture coded stream generated in the coding unit 621 and the non-base layer picture coded stream generated in the coding unit 622 to generate a layered picture coded stream.

With respect to the coding unit 621 and the coding unit 622 of the hierarchical image coding apparatus 620, the image coding apparatus 100 (FIG. 1), the image coding apparatus 300 (FIG. 18), or the image coding apparatus ( 500 (FIG. 29) can be applied. In this case, the hierarchical image coding apparatus 600 sets and transmits a difference value between the quantization parameter set by the encoder 621 and the quantization parameter set by the encoder 622.

Hierarchical Image Decoding Device

Fig. 38 is a diagram showing a hierarchical image decoding device which performs the above-described hierarchical image decoding. As shown in FIG. 38, the hierarchical image decoding device 630 includes a demultiplexer 631, a decoder 632, and a decoder 633.

The demultiplexer 631 demultiplexes the hierarchical image coded stream obtained by multiplexing the base layer picture coded stream and the non-base layer picture coded stream, and extracts the base layer picture coded stream and the non-base layer picture coded stream. The decoder 632 decodes the base layer image encoded stream extracted by the demultiplexer 631 to obtain a base layer image. The decoding unit 633 decodes the non-base layer picture coded stream extracted by the demultiplexing unit 631 to obtain a non-base layer picture.

The image decoding device 200 (FIG. 9) or the image decoding device 400 (FIG. 21) can be applied to the decoding unit 632 and the decoding unit 633 of the hierarchical image decoding device 630. have. In this case, the hierarchical image decoding device 630 sets the quantization parameter from the difference value between the quantization parameter set by the encoding unit 631 and the quantization parameter set by the encoding unit 632 to perform inverse quantization.

<8. Eighth Embodiment >

[computer]

The series of processes described above may be executed by hardware or by software. In this case, for example, it may be configured as a computer as shown in FIG.

In FIG. 39, a central processing unit (CPU) 701 of the personal computer 700 is a program stored in a ROM (Read Only Memory) 702 or a random access memory (RAM) from a storage unit 713. Various processes are executed in accordance with the program loaded in 703. The RAM 703 also appropriately stores data required for the CPU 701 to perform various processes.

The CPU 701, the ROM 702, and the RAM 703 are connected to each other via the bus 704. An input / output interface 710 is also connected to this bus 704.

The input / output interface 710 includes an input unit 711 consisting of a keyboard, a mouse, a display, a display made of a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display), or the like, an output unit 712 made of a speaker, or the like, a hard disk, or the like. The memory unit 713 configured as a communication unit, and the communication unit 714 configured as a modem are connected. The communication unit 714 performs communication processing via a network including the Internet.

A drive 715 is also connected to the input / output interface 710 as necessary, and a removable medium 721 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is appropriately mounted and read out from them. The created computer program is installed in the storage unit 713 as necessary.

When the series of processes described above is executed by software, a program constituting the software is installed from a network or a recording medium.

This recording medium is, for example, a magnetic disk (including a flexible disk) on which a program is recorded, which is distributed to transmit a program to a user, separately from the apparatus main body, as shown in FIG. 39, and an optical disk. (Removable media 721) comprising a compact disc-read only memory (CD-ROM), a digital versatile disc (DVD), a magneto-optical disc (including a mini disc), a semiconductor memory, or the like. It is composed of a ROM 702 in which a program is recorded, a hard disk included in the storage unit 713, and the like, which are not only constituted by the above, but also transmitted to a user in a state of being pre-built in the apparatus main body.

The program executed by the computer may be a program in which the processing is performed in time series according to the procedure described herein, or may be a program in which processing is performed at a necessary timing such as when the call is made in parallel or when a call is made.

In addition, in this specification, the step which describes the program recorded on a recording medium includes the process performed not only in time series but also in parallel or separately, even if it does not necessarily process in time series according to the described procedure. It is.

In addition, in this specification, a system represents the whole apparatus comprised by the some device (device).

In addition, in the above, you may divide the structure demonstrated as one apparatus (or processing part), and may comprise as a some apparatus (or processing part). On the contrary, the structure described as a plurality of devices (or processing units) may be integrated as one device (or processing unit). It is of course possible to add a configuration other than the above to the configuration of each device (or each processing unit). If the configuration and operation of the system as a whole are substantially the same, a part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit). That is, embodiment of this technology is not limited to embodiment mentioned above, A various change is possible in the range which does not deviate from the summary of this technology.

The picture coding apparatus 100 (FIG. 1), the picture coding apparatus 300 (FIG. 18), the picture coding apparatus 500 (FIG. 29), and the multi-view image coding apparatus 600 (FIG. 34) which concern on the above-mentioned embodiment ), Hierarchical image coding apparatus 620 (FIG. 37), image decoding apparatus 200 (FIG. 9), image decoding apparatus 400 (FIG. 21), multi-view image decoding apparatus 610 (FIG. 35), and The hierarchical image decoding device 630 (FIG. 38) includes a transmitter or receiver, an optical disk, a magnetic disk, and the like in satellite broadcasting, cable broadcasting such as cable TV, distribution on the Internet, and distribution to a terminal by cellular communication. The present invention can be applied to various electronic devices such as a recording device for recording an image on a medium such as a flash memory, or a reproducing device for reproducing an image from these storage media. Hereinafter, four application examples will be described.

[TV apparatus]

40 shows an example of a schematic configuration of a television device to which the above-described embodiment is applied. The television device 900 includes an antenna 901, a tuner 902, a demultiplexer 903, a decoder 904, a video signal processor 905, a display unit 906, an audio signal processor 907, and a speaker 908. And an external interface 909, a controller 910, a user interface 911, and a bus 912.

The tuner 902 extracts a signal of a desired channel from the broadcast signal received through the antenna 901, and demodulates the extracted signal. The tuner 902 then outputs the coded bit stream obtained by demodulation to the demultiplexer 903. That is, the tuner 902 has a role as a transmission means in the television device 900 which receives an encoded stream in which an image is encoded.

The demultiplexer 903 separates the video stream and the audio stream of the program to be viewed from the encoded bit stream, and outputs the separated streams to the decoder 904.

Further, the demultiplexer 903 extracts supplementary data such as EPG (Electronic Program Guide) from the coded bit stream, and supplies the extracted data to the control unit 910. In addition, the demultiplexer 903 may descramble when the coded bit stream is scrambled.

The decoder 904 decodes the video stream and the audio stream input from the demultiplexer 903. The decoder 904 then outputs the video data generated by the decoding process to the video signal processing unit 905. The decoder 904 also outputs the audio data generated by the decoding process to the audio signal processing unit 907.

The video signal processing unit 905 reproduces the video data input from the decoder 904 and causes the display unit 906 to display the video. The video signal processing unit 905 may display the application screen supplied through the network on the display unit 906. In addition, the video signal processing unit 905 may perform additional processing, such as noise removal, on the video data, depending on the setting. The video signal processing unit 905 may generate an image of a graphical user interface (GUI) such as a menu, a button, or a cursor, for example, and superimpose the generated image on an output image.

The display unit 906 is driven by a drive signal supplied from the video signal processing unit 905 to display an image of a display device (for example, a liquid crystal display, a plasma display, or an organic electroluminescence display (OLED) or the like). Display an image or image on the surface.

The audio signal processing unit 907 performs reproduction processing such as D / A conversion and amplification on the audio data input from the decoder 904 and outputs the audio from the speaker 908. In addition, the audio signal processing unit 907 may perform additional processing such as noise removal on the audio data.

The external interface 909 is an interface for connecting the television apparatus 900 to an external device or a network. For example, the video stream or audio stream received via the external interface 909 may be decoded by the decoder 904. That is, the external interface 909 also receives the encoded stream in which the image is encoded, and serves as a transmission means in the television apparatus 900.

The control unit 910 has a processor such as a CPU and a memory such as a RAM and a ROM. The memory stores programs executed by the CPU, program data, EPG data, data acquired via the network, and the like. The program stored in the memory is read and executed by the CPU, for example, when the television device 900 starts up. By executing a program, the CPU controls the operation of the television apparatus 900 according to, for example, an operation signal input from the user interface 911.

The user interface 911 is connected to the control unit 910. The user interface 911 has, for example, a button and a switch for the user to operate the television device 900, a receiver of a remote control signal, and the like. The user interface 911 detects operation by a user through these components, generates an operation signal, and outputs the generated operation signal to the control unit 910.

The bus 912 connects the tuner 902, the demultiplexer 903, the decoder 904, the video signal processor 905, the audio signal processor 907, the external interface 909, and the controller 910 to each other.

In the television device 900 configured as described above, the decoder 904 includes the image decoding device 200 (FIG. 9), the image decoding device 400 (FIG. 21), and the multi-view image decoding device according to the above-described embodiment. 610 (FIG. 35), or hierarchical image decoding device 630 (FIG. 38). Therefore, for the video decoded by the television apparatus 900, the quantization value is calculated for each sub macroblock using quantization parameters such as submb_qp_delta supplied from the encoding side, and inverse quantization is performed. Therefore, a dequantization process more suited to the content of the image can be performed, and deterioration in subjective image quality of the decoded image can be suppressed.

[Mobile phone]

41 shows an example of a schematic configuration of a mobile phone to which the above-described embodiment is applied. The mobile phone 920 includes an antenna 921, a communication unit 922, a voice codec 923, a speaker 924, a microphone 925, a camera unit 926, an image processing unit 927, and a multiple separation unit 928. ), A recording / reproducing unit 929, a display unit 930, a control unit 931, an operation unit 932, and a bus 933.

The antenna 921 is connected to the communication unit 922. The speaker 924 and the microphone 925 are connected to an audio codec 923. The operation unit 932 is connected to the control unit 931. The bus 933 includes a communication unit 922, an audio codec 923, a camera unit 926, an image processing unit 927, a multiple separation unit 928, a recording and playback unit 929, a display unit 930, and a control unit ( 931 are connected to each other.

The mobile phone 920 is capable of transmitting and receiving voice signals, sending and receiving e-mail or image data, capturing images, recording data, and the like, in various operation modes including a voice call mode, a data communication mode, a shooting mode, and a television telephone mode. Performs the operation of.

In the voice call mode, the analog voice signal generated by the microphone 925 is supplied to the voice codec 923. The audio codec 923 converts an analog audio signal into audio data, and A / D converts and compresses the converted audio data. The audio codec 923 then outputs the compressed audio data to the communication unit 922. The communication unit 922 encodes and modulates voice data and generates a transmission signal. The communication unit 922 then transmits the generated transmission signal to a base station (not shown) via the antenna 921. In addition, the communication unit 922 amplifies and frequency-converts the radio signal received through the antenna 921 and acquires the received signal. The communication unit 922 demodulates and decodes the received signal to generate voice data, and outputs the generated voice data to the voice codec 923. The speech codec 923 decompresses and D / A-converts the speech data to generate an analog speech signal. The audio codec 923 then supplies the generated audio signal to the speaker 924 to output audio.

Further, in the data communication mode, for example, the control unit 931 generates text data constituting an electronic mail in accordance with an operation by a user who has passed through the operation unit 932. In addition, the control unit 931 displays the characters on the display unit 930. The control unit 931 generates electronic mail data in accordance with a transmission instruction from the user who has passed through the operation unit 932, and outputs the generated electronic mail data to the communication unit 922. The communication unit 922 encodes and modulates the electronic mail data, and generates a transmission signal. The communication unit 922 then transmits the generated transmission signal to a base station (not shown) via the antenna 921. In addition, the communication unit 922 amplifies and frequency-converts the radio signal received through the antenna 921 and acquires the received signal. The communication unit 922 demodulates and decodes the received signal to restore the electronic mail data, and outputs the restored electronic mail data to the control unit 931. The control unit 931 displays the contents of the electronic mail on the display unit 930 and stores the electronic mail data in the storage medium of the recording / reproducing unit 929.

The recording / reproducing section 929 has any storage medium that can be read and written. For example, the storage medium may be a built-in storage medium such as RAM or flash memory, or may be an external storage type storage medium such as a hard disk, a magnetic disk, a magneto-optical disk, an optical disk, a USB memory, or a memory card.

In addition, in the shooting mode, for example, the camera unit 926 photographs a subject to generate image data, and outputs the generated image data to the image processing unit 927. The image processing unit 927 encodes the image data input from the camera unit 926 and stores the encoded stream in the storage medium of the recording / reproducing unit 929.

In the telephony mode, for example, the multiplexing section 928 multiplexes and multiplexes the video stream encoded by the image processing section 927 and the audio stream input from the audio codec 923. Is output to the communication unit 922. The communication unit 922 encodes and modulates a stream and generates a transmission signal. The communication unit 922 then transmits the generated transmission signal to a base station (not shown) via the antenna 921. In addition, the communication unit 922 amplifies and frequency-converts the radio signal received through the antenna 921 and acquires the received signal. These transmission signals and reception signals may include an encoded bit stream. The communication unit 922 demodulates and decodes the received signal to restore the stream, and outputs the restored stream to the multiple separation unit 928. The multiple separation unit 928 separates the video stream and the audio stream from the input stream, and outputs the video stream to the image processing unit 927 and the audio stream to the audio codec 923. The image processing unit 927 decodes the video stream and generates video data. The video data is supplied to the display unit 930, and a series of images is displayed by the display unit 930. The speech codec 923 decompresses and D / A-converts the speech stream and generates an analog speech signal. The audio codec 923 then supplies the generated audio signal to the speaker 924 to output audio.

In the mobile phone 920 configured as described above, the image processing unit 927 includes the image coding apparatus 100 (FIG. 1), the image coding apparatus 300 (FIG. 18), and the image coding apparatus ( 500 (FIG. 29), the function of the multiview image coding apparatus 600 (FIG. 34), or the hierarchical image coding apparatus 620 (FIG. 37), the image decoding apparatus 200 (FIG. 9), and the image decoding apparatus. 400 (FIG. 21), a multi-view image decoding device 610 (FIG. 35), or a hierarchical image decoding device 630 (FIG. 38). Therefore, for the video coded and decoded by the cellular phone 920, a quantization value is calculated for each sub macroblock, and the orthogonal transform coefficient is quantized using the quantization value for each sub macroblock. By doing in this way, the quantization process which is more suitable for the content of an image can be performed, and encoded data can be produced so that degradation of subjective image quality of a decoded image can be suppressed. Further, using a quantization parameter such as submb_qp_delta supplied from the encoding side, a quantization value is calculated for each sub macroblock, and inverse quantization is performed. Therefore, a dequantization process more suited to the content of the image can be performed, and deterioration in subjective image quality of the decoded image can be suppressed.

In the above description, the mobile phone 920 has been described as a mobile phone 920. For example, the mobile phone 920 includes PDAs (Personal Digital Assistants), smart phones, UMPCs (Ultra Mobile Personal Computers), netbooks, and notebook personal computers. As long as the device has an imaging function and a communication function similar to the above, any device can be applied to an image coding device and an image decoding device to which the present technology is applied, similarly to the case of the mobile phone 920.

[Recording / Reproducing Apparatus]

42 shows an example of a schematic configuration of a recording / reproducing apparatus to which the above-described embodiment is applied. The recording and reproducing apparatus 940, for example, encodes the received audio data and the video data of the broadcast program, and records them on the recording medium. In addition, the recording / reproducing apparatus 940 may encode, for example, audio data and video data obtained from another apparatus and record the same on a recording medium. In addition, the recording and reproducing apparatus 940 reproduces the data recorded on the recording medium on the monitor and the speaker, for example, in accordance with a user's instruction. At this time, the recording and reproducing apparatus 940 decodes the audio data and the video data.

The recording and reproducing apparatus 940 includes a tuner 941, an external interface 942, an encoder 943, a hard disk drive (HDD) 944, a disk drive 945, a selector 946, a decoder 947, An on-screen display (OSD) 948, a controller 949, and a user interface 950.

The tuner 941 extracts a signal of a desired channel from a broadcast signal received through an antenna (not shown), and demodulates the extracted signal. The tuner 941 then outputs the coded bit stream obtained by demodulation to the selector 946. In other words, the tuner 941 has a role as a transmission means in the recording / reproducing apparatus 940.

The external interface 942 is an interface for connecting the recording / reproducing apparatus 940 to an external device or a network. The external interface 942 may be, for example, an IEEE1394 interface, a network interface, a USB interface, a flash memory interface, or the like. For example, video data and audio data received through the external interface 942 are input to the encoder 943. That is, the external interface 942 has a role as a transmission means in the recording / reproducing apparatus 940.

The encoder 943 encodes the video data and the audio data when the video data and audio data input from the external interface 942 are not coded. The encoder 943 then outputs the encoded bit stream to the selector 946.

The HDD 944 records encoded bit streams, various programs, and other data in which content data such as video and audio are compressed, on an internal hard disk. The HDD 944 also reads these data from the hard disk at the time of reproduction of video and audio.

The disk drive 945 writes and reads data to and from a mounted recording medium. The recording medium loaded in the disc drive 945 may be, for example, a DVD disc (DVD-Video, DVD-RAM, DVD-R, DVD-RW, DVD + R, DVD + RW, etc.) or Blu-ray (registered trademark). ) Disk or the like.

The selector 946 selects an encoded bit stream input from the tuner 941 or the encoder 943 at the time of recording video and audio, and outputs the selected encoded bit stream to the HDD 944 or the disk drive 945. do. In addition, the selector 946 outputs the encoded bit stream input from the HDD 944 or the disk drive 945 to the decoder 947 when the video and audio are reproduced.

The decoder 947 decodes the encoded bit stream and generates video data and audio data. The decoder 947 then outputs the generated video data to the OSD 948. The decoder 904 also outputs the generated audio data to an external speaker.

The OSD 948 reproduces the video data input from the decoder 947 and displays the video. Further, the OSD 948 may superimpose a GUI image such as a menu, a button, or a cursor on an image to be displayed.

The control unit 949 has a processor such as a CPU and a memory such as a RAM and a ROM. The memory stores programs, program data, and the like executed by the CPU. The program stored in the memory is read and executed by the CPU at the start of the recording / playback apparatus 940, for example. By executing the program, the CPU controls the operation of the recording / playback apparatus 940 according to, for example, an operation signal input from the user interface 950.

The user interface 950 is connected to the control unit 949. The user interface 950 has, for example, a button and a switch for the user to operate the recording / playback apparatus 940, a receiving section of a remote control signal, and the like. The user interface 950 detects operation by a user through these components, generates an operation signal, and outputs the generated operation signal to the control unit 949.

In the recording and reproducing apparatus 940 configured as described above, the encoder 943 includes the image encoding apparatus 100 (FIG. 1), the image encoding apparatus 300 (FIG. 18), and the image encoding apparatus ( It has the function of 500 (FIG. 29), the multiview image coding apparatus 600 (FIG. 34), or the hierarchical image coding apparatus 620 (FIG. 37). In addition, the decoder 947 includes the image decoding apparatus 200 (FIG. 9), the image decoding apparatus 400 (FIG. 21), the multi-view image decoding apparatus 610 (FIG. 35), or the above. It has the function of the hierarchical image decoding device 630 (Fig. 38). Therefore, for the video coded and decoded by the recording / reproducing apparatus 940, the quantization value is calculated for each sub macroblock, and the orthogonal transform coefficient is quantized using the quantization value for each sub macroblock. By doing in this way, the quantization process which is more suitable for the content of an image can be performed, and encoded data can be produced so that degradation of subjective image quality of a decoded image can be suppressed. Further, using a quantization parameter such as submb_qp_delta supplied from the encoding side, a quantization value is calculated for each sub macroblock, and inverse quantization is performed. Therefore, a dequantization process more suited to the content of the image can be performed, and deterioration in subjective image quality of the decoded image can be suppressed.

[Imaging device]

43 shows an example of a schematic configuration of an imaging device to which the above-described embodiment is applied. The imaging device 960 captures an image of a subject to generate an image, encodes image data, and records the image data on a recording medium.

The imaging device 960 includes an optical block 961, an imaging unit 962, a signal processing unit 963, an image processing unit 964, a display unit 965, an external interface 966, a memory 967, a media drive ( 968, OSD 969, control unit 970, user interface 971, and bus 972.

The optical block 961 is connected to the imaging unit 962. The imaging unit 962 is connected to the signal processing unit 963. The display unit 965 is connected to the image processing unit 964. The user interface 971 is connected to the control unit 970. The bus 972 interconnects the image processing unit 964, the external interface 966, the memory 967, the media drive 968, the OSD 969, and the control unit 970.

The optical block 961 has a focus lens, an aperture mechanism, and the like. The optical block 961 forms an optical image of the subject on the imaging surface of the imaging unit 962. The imaging unit 962 has an image sensor such as a CCD or a CMOS, and converts the optical image formed on the imaging surface into an image signal as an electric signal by photoelectric conversion. The imaging unit 962 then outputs an image signal to the signal processing unit 963.

The signal processing unit 963 performs various camera signal processing such as knee correction, gamma correction, and color correction on the image signal input from the imaging unit 962. The signal processing unit 963 outputs the image data after the camera signal processing to the image processing unit 964.

The image processing unit 964 encodes the image data input from the signal processing unit 963 and generates encoded data. The image processing unit 964 then outputs the generated coded data to the external interface 966 or the media drive 968. The image processing unit 964 decodes the encoded data input from the external interface 966 or the media drive 968 to generate image data. The image processing unit 964 then outputs the generated image data to the display unit 965. In addition, the image processing unit 964 may display the image by outputting the image data input from the signal processing unit 963 to the display unit 965. In addition, the image processing unit 964 may superimpose the display data acquired from the OSD 969 on the image output to the display unit 965.

The OSD 969 generates an image of a GUI such as a menu, a button or a cursor, and outputs the generated image to the image processing unit 964.

The external interface 966 is configured, for example, as a USB input / output terminal. The external interface 966 connects the imaging device 960 and a printer, for example, when printing an image. In addition, a drive is connected to the external interface 966 as necessary. In the drive, for example, a removable medium such as a magnetic disk or an optical disk is mounted, and a program read from the removable media can be installed in the imaging device 960. In addition, the external interface 966 may be configured as a network interface connected to a network such as a LAN or the Internet. That is, the external interface 966 has a role as a transmission means in the imaging device 960.

The recording medium attached to the media drive 968 may be any removable medium that can be read and written, such as a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory. In addition, a recording medium may be fixedly mounted to the media drive 968, and a non-reversible storage unit such as, for example, an internal hard disk drive or a solid state drive (SSD) may be configured.

The control unit 970 has a processor such as a CPU and a memory such as a RAM or a ROM. The memory stores programs, program data, and the like executed by the CPU. The program stored in the memory is read and executed by the CPU at the start of the imaging device 960, for example. By executing a program, the CPU controls the operation of the imaging device 960 according to, for example, an operation signal input from the user interface 971.

The user interface 971 is connected to the control unit 970. The user interface 971 includes, for example, a button, a switch, and the like for the user to operate the imaging device 960. The user interface 971 detects an operation by a user through these components, generates an operation signal, and outputs the generated operation signal to the control unit 970.

In the imaging device 960 configured as described above, the image processing unit 964 includes the image coding apparatus 100 (FIG. 1), the image coding apparatus 300 (FIG. 18), and the image coding apparatus ( 500 (FIG. 29), the function of the multiview image coding apparatus 600 (FIG. 34), or the hierarchical image coding apparatus 620 (FIG. 37), the image decoding apparatus 200 (FIG. 9), and the image decoding apparatus. 400 (FIG. 21), a multi-view image decoding device 610 (FIG. 35), or a hierarchical image decoding device 630 (FIG. 38). Therefore, for the video coded and decoded by the imaging device 960, the quantization value is calculated for each sub macroblock, and the orthogonal transform coefficient is quantized using the quantization value for each sub macroblock. By doing in this way, the quantization process which is more suitable for the content of an image can be performed, and encoded data can be produced so that degradation of subjective image quality of a decoded image can be suppressed. Further, using a quantization parameter such as submb_qp_delta supplied from the encoding side, a quantization value is calculated for each sub macroblock, and inverse quantization is performed. Therefore, a dequantization process more suited to the content of the image can be performed, and deterioration in subjective image quality of the decoded image can be suppressed.

Of course, the picture coding apparatus and the picture decoding apparatus to which the present technology is applied can be applied to devices and systems other than those described above.

In this specification, an example in which the quantization parameter is transmitted from the encoding side to the decoding side has been described. The method of transmitting the quantization matrix parameter may be transmitted or recorded as separate data associated with the encoded bit stream without being multiplexed into the encoded bit stream. Here, the term "associate" means that an image (which may be part of an image such as a slice or a block) included in the bit stream and information corresponding to the image can be linked at the time of decoding. That is, the information may be transmitted on a transmission path different from that of the image (or bit stream). In addition, the information may be recorded in a recording medium different from the image (or bit stream) (or another recording area of the same recording medium). In addition, the information and the image (or bit stream) may be associated with each other in any unit such as a plurality of frames, one frame, or a part of the frame.

As mentioned above, although preferred embodiment of this indication was described in detail, referring an accompanying drawing, the technical scope of this indication is not limited to this example. It is obvious that any person having ordinary knowledge in the technical field of the present disclosure can conceive various modifications or modifications within the scope of the technical idea described in the claims. It is understood that it belongs to the technical scope of the disclosure.

100: image encoding device
105: quantization unit
108: inverse quantization unit
117: rate control unit
121: sub macroblock quantization unit
122: sub macroblock dequantization unit
151: Sub macro block activity buffer
152: quantization parameter calculator
153: quantization processing unit
200: image decoding device
203: inverse quantization unit
221: sub macroblock inverse quantization unit
251 quantization parameter buffer
252: Cartesian transform coefficient buffer
253 inverse quantization processing unit

Claims (34)

A decoding unit for decoding the encoded stream to generate quantized data;
A quantization parameter used for inverse quantization of quantized data generated by the decoding unit, targeting a coding unit that is lower than a reference coding unit in a reference layer of a coding unit that is a coding processing unit when encoding image data. Setting section to set,
And an inverse quantization unit which inversely quantizes the quantization data generated by the decoding unit using the quantization parameter set by the setting unit. The method of claim 1,
The setting unit uses the current coding unit by using a differential quantization parameter representing a difference value between the quantization parameter set in the current coding unit to be subjected to inverse quantization processing and the quantization parameter set in the coding unit in the same layer as the current coding unit. An image processing apparatus for setting a quantization parameter. The method of claim 2,
And the differential quantization parameter is a difference value between the quantization parameter set in the current coding unit and the quantization parameter set in the coding unit decoded before the current coding unit in decoding processing order. The method of claim 3,
And the difference quantization parameter is a difference value between the quantization parameter set in the current coding unit and the quantization parameter set in the coding unit decoded one before the current coding unit in decoding processing order. 5. The method of claim 4,
And the reference coding unit is a maximum coding unit that is a coding unit of a top layer. The method of claim 5,
A receiving unit which receives minimum coding unit size data indicating a minimum size of a coding unit for setting the coded stream and the differential quantization parameter,
And the setting unit sets the quantization parameter of the current coding unit according to the minimum coding unit size data received by the receiving unit. The method according to claim 6,
And the receiving unit obtains the minimum coding unit size data from a slice header of the encoded stream. The method of claim 7, wherein
And the difference quantization parameter of the coding unit whose size is less than 16 pixels is set to 0 when the size indicated by the minimum coding unit size data is 16 pixels. The method of claim 1,
The setting unit sets a quantization parameter of the current coding unit by using a differential quantization parameter indicating a difference value between the quantization parameter set in the current coding unit to be decoded and the quantization parameter set in the slice to which the current coding unit belongs. An image processing apparatus. 10. The method of claim 9,
The setting unit, when the current coding unit is the first in a decoding processing order in the hierarchy of the reference coding unit, sets the difference value between the quantization parameter set in the current coding unit and the quantization parameter set in the slice to which the current coding unit belongs. An image processing apparatus for setting quantization parameters of the current coding unit using differential quantization parameters indicated. The method of claim 10,
And the reference coding unit is a maximum coding unit that is a coding unit of a top layer. 10. The method of claim 9,
A receiving unit which receives minimum coding unit size data indicating a minimum size of a coding unit for setting the coded stream and the differential quantization parameter,
And the setting unit sets the quantization parameter of the current coding unit according to the minimum coding unit size data received by the receiving unit. The method of claim 12,
And the receiving unit obtains the minimum coding unit size data from a slice header of the encoded stream. The method of claim 13,
And the difference quantization parameter of the coding unit whose size is less than 16 pixels is set to 0 when the size indicated by the minimum coding unit size data is 16 pixels. The method of claim 1,
The setting unit targets a coding unit that is lower than the reference coding unit, and when the value of the differential quantization parameter is 0, the coding unit having the quantization parameter set in the reference coding unit lower than the reference coding unit. An image processing apparatus that is set as a quantization parameter set in FIG. 16. The method of claim 15,
A receiving unit which receives differential identification data for identifying whether the value of the differential quantization parameter is 0, targeting a coding unit in a lower layer than the reference coding unit,
And the setting unit sets the quantization parameter set in the reference coding unit as the quantization parameter set in a coding unit lower than the reference coding unit using the difference identification data received by the receiving unit. Decode the encoded stream to generate quantized data,
A quantization parameter used for inverse quantization of the generated quantized data is set for a coding unit that is lower than a reference coding unit in a reference layer of a coding unit that is a coding processing unit when encoding image data,
An image processing method for dequantizing the generated quantized data using the set quantization parameter. A setting unit for setting a quantization parameter used for quantizing image data, targeting a coding unit that is lower than a reference coding unit in a reference layer of a coding unit that is a coding processing unit when encoding image data;
A quantization unit for quantizing image data to generate quantization data using the quantization parameter set by the setting unit;
And an encoding unit for encoding the quantized data generated by the quantization unit to generate an encoded stream. 19. The method of claim 18,
The setting unit sets a differential quantization parameter indicating a difference value between the quantization parameter set in the current coding unit to be subjected to the coding process and the quantization parameter set in the coding unit in the same layer as the current coding unit,
And a transmission unit for transmitting the differential quantization parameter set by the setting unit and the encoded stream generated by the encoding unit. 20. The method of claim 19,
And the setting unit sets, as the difference quantization parameter, a difference value between the quantization parameter set in the current coding unit and the quantization parameter set in the coding unit encoded before the current coding unit in the coding processing order. 21. The method of claim 20,
And the setting unit sets, as the difference quantization parameter, a difference value between the quantization parameter set in the current coding unit and the quantization parameter set in the coding unit coded one earlier in the coding processing order than the current coding unit. 21. The method of claim 20,
And the reference coding unit is a maximum coding unit that is a coding unit of a top layer. The method of claim 22,
The setting unit sets minimum coding unit size data indicating a minimum size of a coding unit for setting the differential quantization parameter,
And the transmission unit transmits the minimum coding unit size data set by the setting unit. 24. The method of claim 23,
And the transmission unit adds, as a slice header, the minimum coding unit size data set by the setting unit to the syntax of the encoded stream generated by the encoding unit. 25. The method of claim 24,
And the setting unit sets the differential quantization parameter of a coding unit whose size is less than 16 pixels to 0 when the size indicated by the minimum coding unit size data is set to 16 pixels. 19. The method of claim 18,
The setting unit sets a differential quantization parameter indicating a difference value between the quantization parameter set in the current coding unit to be subjected to the coding process and the quantization parameter set in the slice to which the current coding unit belongs,
And a transmission unit for transmitting the differential quantization parameter set by the setting unit and the encoded stream generated by the encoding unit. The method of claim 26,
The setting unit, when the current coding unit is the first in the coding processing order in the layer of the reference coding unit, sets the difference value between the quantization parameter set in the current coding unit and the quantization parameter set in the slice to which the current coding unit belongs. And setting as the difference quantization parameter. 28. The method of claim 27,
And the reference coding unit is a maximum coding unit that is a coding unit of a top layer. 29. The method of claim 28,
The setting unit sets minimum coding unit size data indicating a minimum size of a coding unit for setting the differential quantization parameter,
And the transmission unit transmits the minimum coding unit size data set by the setting unit. 30. The method of claim 29,
And the transmission unit adds, as a slice header, the minimum coding unit size data set by the setting unit to the syntax of the encoded stream generated by the encoding unit. 31. The method of claim 30,
And the setting unit sets the differential quantization parameter of a coding unit whose size is less than 16 pixels to 0 when the size indicated by the minimum coding unit size data is set to 16 pixels. 19. The method of claim 18,
The setting unit includes a quantization parameter set in the reference coding unit lower than the reference coding unit when the value of the differential quantization parameter is set to 0 for a coding unit located lower than the reference coding unit. An image processing apparatus set as a quantization parameter set in a coding unit. 33. The method of claim 32,
The setting unit sets differential identification data for identifying whether a value of the differential quantization parameter is 0, targeting a coding unit located lower than a reference coding unit,
And a transmission unit which transmits the difference identification data set by the setting unit and the encoded stream generated by the encoding unit. A quantization parameter used when quantizing image data is set for a coding unit that is lower than a reference coding unit in a reference layer of a coding unit that is a coding processing unit at the time of encoding image data,
Using the set quantization parameter, quantize the image data to generate quantized data,
An image processing method for generating an encoded stream by encoding the generated quantized data.

Images