JP2008061122A - Control device, control method, encoder, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent deterioration in image quality of a decoded image obtained by encoding and decoding an image to which editing such as dithering is applied. <P>SOLUTION: A prediction unit 1 determines a prediction error between a source image and a predictive image, a conversion unit 2 converts the prediction error into a frequency component, and a quantization unit 43 quantizes the frequency component. On the other hand, a dithering information acquisition unit 41 acquires dithering information representing whether dithering is applied to the source image and on the basis of the dither information, a dithering quantization step control unit 42 controls a quantization step in the case where the quantization unit 43 quantizes the frequency component. The present invention may be applicable to an encoder which encodes an image using an encoding method compliant to H. 264/AVC or the like. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a control device, a control method, an encoding device, and a program, and in particular, can prevent deterioration in the image quality of a decoded image obtained by encoding and decoding an image that has been subjected to editing such as dithering. The present invention relates to a control device, a control method, an encoding device, and a program that can be used.

  FIG. 1 shows a configuration of an example of a conventional encoding apparatus that encodes an image.

  In FIG. 1, the encoding device includes a prediction unit 1, a conversion unit 2, a quantization unit 3, a generation unit 4, and a rate control unit 5. The prediction unit 1 includes a moving image to be encoded. An original image, which is an image, is supplied.

  The prediction unit 1 obtains a prediction error between the original image supplied thereto and a prediction image obtained by predicting the original image, and supplies the prediction error to the conversion unit 2.

  That is, the prediction unit 1 divides the original image supplied thereto into predetermined blocks, and obtains a predicted image of the block of the original image. Then, the prediction unit 1 subtracts the prediction image from the block of the original image to obtain a prediction error between the block of the original image and the prediction image, and supplies the prediction error to the conversion unit 2 in units of blocks.

  The conversion unit 2 converts the prediction error block from the prediction unit 1 into a frequency component block that is information in the frequency domain, and supplies the block to the quantization unit 3.

  Here, as a conversion method for converting a prediction error into a frequency component, for example, orthogonal transform such as DCT (Discrete Cosine Transform) or Hadamard transform can be employed.

  The quantization unit 3 is supplied with a block of frequency components from the conversion unit 2 and is also supplied with quantization control information from the rate control unit 5.

  The quantization unit 3 determines a quantization step when quantizing the frequency component based on the quantization control information supplied from the rate control unit 5, and the block unit from the conversion unit 2 in the quantization step , And the quantized coefficient that is the result of the quantization is supplied to the generating unit 4 in units of blocks.

  In addition to the quantization coefficient supplied from the quantization unit 3, the generation unit 4 is also supplied with quantization control information used to determine the quantization step in the quantization unit 3 from the rate control unit 5. .

  The generation unit 4 generates an encoded bit stream including at least the block-unit quantization coefficient from the quantization unit 3 and the quantization control information from the rate control unit 5 and outputs the encoded bit stream as an original image encoding result. To do.

  On the other hand, the rate control unit 5 monitors the encoded bit stream output from the generation unit 4, and controls the quantization step of the quantization unit 3 based on the rate of the encoded bit stream.

  That is, when the rate of the encoded bit stream output from the generation unit 4 is high, the rate control unit 5 performs coarse quantization (quantization with coarse quantization granularity) in the quantization unit 3 in order to reduce the rate. Quantization control information for determining a quantization step to perform is generated. Further, when the rate of the encoded bit stream output from the generation unit 4 is low, the rate control unit 5 performs fine quantization (quantization with a fine quantization granularity) in the quantization unit 3 in order to increase the rate. Quantization control information for determining a quantization step to perform is generated.

  As described above, the rate control unit 5 supplies the quantization control information generated based on the rate of the encoded bit stream output from the generation unit 4 to the quantization unit 3, thereby enabling the rate of the encoded bit stream. Is controlled to be a predetermined value (a value within a predetermined range).

  Note that, as described above, the quantization control information generated by the rate control unit 5 is supplied to the generation unit 4 in addition to the quantization unit 3 and included in the encoded bitstream.

  By the way, there is a dither as a technique for representing an intermediate gradation in an image in a pseudo manner.

  FIG. 2 shows the dither.

  Dither is a pixel value v and a pixel value v + 1 (or pixel value v-1) that differs from the pixel value v by 1 when the pixel value takes an integer value of 0 or more. A pseudo-representation of an intermediate value v + 0.5 between pixel values v and v + 1 (an intermediate value v-0.5 between pixel values v and v-1) that cannot be taken as a pixel value. For example, by arranging 0 and 1 as a pixel value of a region of an image in a grid pattern, the region appears to a person as a region having a pixel value of 0.5.

  Dither is often applied to CG (Computer Graphics) or a composite image of CG and the actually captured image. In order to improve gradation expression, dithering may be intentionally performed as preprocessing (PreProcess) (see, for example, Non-Patent Document 1).

  Here, for example, dithering may be performed to reduce the quantization roughness of the image. However, in the gradation area where the gradation changes gradually, the step at the quantization bit switching portion is conspicuous. , It may be displayed as a striped pattern. A method for detecting such a step is described in Patent Document 1, for example.

  In the dithered image, as described above, in principle, a certain pixel value v and a pixel value v + 1 that differs from the pixel value v by 1 are arranged in a grid pattern.

  However, the dithered image is an HD (High Definition) image, and the SD (Standard Definition) image is obtained by thinning out every other row of pixels in the HD image, for example, horizontally aligned. When generated, in the SD image, the pixel values v and v + 1 are not arranged in a grid pattern, but are arranged alternately only in the horizontal direction, and in the vertical direction, the pixel value v or v Only v + 1 is placed.

  Similarly, when an SD image is generated by thinning out every other column of pixels in the vertical direction of the dithered HD image, the pixel values v and v + 1 Are not arranged in a grid pattern, but are arranged alternately only in the vertical direction, and only the pixel values v or v + 1 are arranged in the horizontal direction.

  An image in which the pixel values v and v + 1 are alternately arranged in only one direction in the horizontal direction or the vertical direction as in the above-described SD image is also included in the dithered image.

JP 2006-154452 JP

"Mach band removal system -MAPS-service start", [online], August 17, 2005, IMAGICA, Inc. [search August 22, 2006], Internet <URL: http: // www .imagica.com / newsrelease / 00069.html>

  As in the encoding apparatus of FIG. 1, the dither is applied to the original image that is the encoding target of the encoding apparatus that converts the prediction error into a frequency component and quantizes the frequency component. If so, the image quality of the decoded image obtained by decoding the encoded stream obtained as a result of encoding may deteriorate.

  Deterioration of the image quality of the decoded image will be described with reference to FIGS.

  3 and 4 show data obtained in the process of encoding and decoding the dithered original image.

  For example, in the encoding apparatus shown in FIG. 1, the original image is encoded in units of 256 pixel blocks of horizontal × vertical (horizontal × vertical) of 16 × 16 pixels. As a predicted image of a target block that is a block of an image, for example, an image of a block that is already encoded and so-called locally decoded, for example, a block adjacent to the left of the target block is used.

In FIG. 3, the block P ₁ is, for example, the leftmost block of the original image, and the block P ₃ is, for example, the block on the right side of the block P ₁ of the original image.

In each of the blocks P ₁ and P _{3 of the} original image, for example, 22 pixel values and 23 pixel values different from the pixel value by 1 are alternately arranged only in the horizontal direction, and in the vertical direction, Only 22 or 23 pixel values are arranged and therefore dithered.

Here, in the blocks P ₁ and P _{3 of the} original image in which the pixel values of 22 and 23 are alternately arranged in the horizontal direction and only the pixel values of 22 or 23 are arranged in the vertical direction, Since there are the same number of pixel values, the DC (Direct Current) component that is the average value of the pixel values is 22.5.

The block P ₂ is a block of a decoded image obtained by encoding the block P _{1 of the} original image with the encoding device in FIG. 1 and performing local decoding. In FIG. 3, the pixel values of the block P ₂ of the decoded image are all 22. Therefore, the DC component is 22.

If the encoding apparatus of FIG. 1 encodes the block P _{3 of the} original image as a target block, the prediction unit 1 encodes the block P ₁ adjacent to the left of the block P ₃ of the original image. the block P ₂ of the decoded image locally decoded, as it is, as the predicted image P ₄ blocks P ₃ of the original image, obtains the prediction error P _5.

In block P ₃ of the original image, as described above, in the horizontal direction, pixel values of 22 and 23 are alternately arranged in the vertical direction, only the pixel value of 22 also 23 are arranged, the predicted image P ₄ As described above, since all the pixel values are the decoded image block P ₂ of 22, the prediction error P ₅ between the block P _{3 of the} original image and the prediction image P ₄ in the horizontal direction, A block in which pixel values of 0 and 1 are alternately arranged, and only pixel values of 0 or 1 are arranged in the vertical direction.

The prediction error P ₅ is converted into a frequency component by the conversion unit 2. Assuming that the conversion unit 2 performs, for example, DCT (two-dimensional DCT) as conversion to frequency components, the prediction error P ₅ is roughly converted into a block of DCT coefficients such as the block P _6. Is done.

Here, the prediction error P ₅ is a block in which pixel values of 0 and 1 are alternately arranged in the horizontal direction and only pixel values of 0 or 1 are arranged in the vertical direction. Therefore, the prediction error P ₅ In this case, since the same number of pixel values of 0 and 1 exist, the DC component is 0.5.

Further, the prediction error P _5, 0 and so the first pixel values are arranged alternately, the prediction error P ₅ is closed the largest frequency component blocks of 16 × 16 pixels can have an (spatial frequency components) The magnitude (absolute value) is 1.

Therefore, in block P ₆ of the DCT coefficients, the DCT coefficients of the upper left 0.5 is a DC component, which is the highest frequency component, for example, the DCT coefficients of the lower right 1, the other DCT coefficients 0, Each has become.

The quantization unit 3 quantizes the DCT coefficient block P ₆ as described above, and outputs a quantization coefficient block P ₇ .

  Here, the quantization unit 3 uses, for example, the fact that human vision is less sensitive to high frequency components than low frequency components, and so on, for the DCT coefficients of low frequency components, fine quantization is performed. The coarse quantization is performed on the DCT coefficient of the high frequency component.

Therefore, in the quantization coefficient block P ₇ , the DCT coefficient 0.5 of the upper left DCT coefficient of the DCT coefficient block P ₆ is 1, and the lower right DCT coefficient 1 of the highest frequency component is 1, The other DCT coefficients 0 are quantized to 0 and 0, respectively. That is, the quantization coefficient block P ₇ is a block in which the quantization coefficient of the DC component is 1 and the quantization coefficient of the other frequency component (AC (Alternating Current) component) is 0.

In the encoding device of FIG. 1, an encoded stream including the block P ₇ of quantized coefficients as described above is output and decoded by a decoding device (not shown).

That is, in the decoding apparatus, the difference between the decoded image and the predicted image corresponding to the prediction error P ₅ (hereinafter, referred to as a decoding error as appropriate) is obtained by performing inverse quantization and inverse DCT on the block P ₇ of the quantized coefficients. The decoded image is obtained by adding the decoding error and the predicted image. Note that a decoded image is obtained in the same manner in local decoding.

As described above, the quantization coefficient block P ₇ is a block in which the quantization coefficient of the DC component is 1 and the quantization coefficient of the other frequency component is 0. Therefore, the quantization coefficient block P ₇ Is inversely quantized and subjected to inverse DCT, for example, a block P ₈ with a decoding error of which the average value of the pixel values is 1 and the pixel values are all 1 is obtained.

Further, the predicted image P ₄ blocks P ₃ to the block P ₇ coded original image quantized coefficients, as described above, a block of all the pixel values 22.

Therefore, the decoded image block P ₉ of the original image block P ₃ obtained by adding the decoded error block P ₈ and the predicted image P ₄ has a pixel value of 23 and a DC component of 23. It becomes.

The encoding apparatus in FIG. 1 encodes the block P ₃ of the original image with the block on the right as the block of interest. The encoding will be described with reference to FIG.

In FIG. 4, the original image block P ₁₁ is a block on the right side of the original image block P ₃ (FIG. 3), and the block P ₁₁ is the same as the original image blocks P ₁ and P _{3 of} FIG. In addition, 22 pixel values and 23 pixel values different from the pixel value by 1 are alternately arranged only in the horizontal direction, and only 22 or 23 pixel values are arranged in the vertical direction.

Therefore, the original image block P ₁₁ is dithered in the same manner as the original image blocks P ₁ and P ₃ in FIG. 3, and its DC component is 22.5.

When the encoding apparatus in FIG. 1 encodes the original image block P ₁₁ as a target block, the prediction unit 1 encodes the block P ₃ (FIG. 3) adjacent to the left of the original image block P ₁₁ . the block P ₉ of the decoded image locally decoded turned into, as it is, as a predictive picture P ₁₂ of the block P ₁₁ of the original image, obtains the prediction error P _13.

In block P ₁₁ of the original image, as described above, in the horizontal direction, pixel values of 22 and 23 are alternately arranged in the vertical direction, only the pixel value of 22 also 23 are arranged, the predicted image P ₁₂ As described above, since all the pixel values are the decoded image block P ₉ of 23, the prediction error P ₁₃ between the block P _{11 of the} original image and the prediction image P ₁₂ is in the horizontal direction, The pixel values of -1 and 0 are alternately arranged, and only the pixel values of -1 or 0 are arranged in the vertical direction.

The prediction error P ₁₃ is DCTed by the conversion unit 2, and as a result, is roughly converted into a block of DCT coefficients such as the block P ₁₄ .

Here, the prediction error P ₁₃ is the horizontal direction, pixel values of -1 and 0 are alternately arranged in the vertical direction, a block in which only the pixel value of -1 or 0 is placed, therefore, the prediction error in P _13, the pixel values of -1 and 0 are present by the same number, DC component is -0.5.

Further, the prediction error P _13, the pixel values of -1 and 0 are alternately arranged, the prediction error P ₁₃ is the highest frequency component blocks of 16 × 16 pixels can have an (spatial frequency components) And its magnitude (absolute value) is 1.

Therefore, in block P ₁₄ of the DCT coefficient is the DCT coefficient at the upper left is a DC component -0.5, the highest spatial frequency components, for example, the DCT coefficients of the lower right -1, the other DCT coefficients 0, respectively.

The quantization unit 3 quantizes the DCT coefficient block P ₁₄ as described above, and outputs a quantization coefficient block P ₁₅ .

  As described with reference to FIG. 3, the quantization unit 3 performs fine quantization on the low-frequency component DCT coefficients, and performs coarse quantization on the high-frequency component DCT coefficients.

For this reason, in the quantization coefficient block P ₁₅ , the DC component of the upper left DCT coefficient −0.5 of the DCT coefficient block P ₁₄ is −1, and the DCM coefficient of the lower right DCT coefficient of the highest spatial frequency component is −1. -1 is quantized to 0, and 0 of other DCT coefficients are quantized to 0, respectively. That is, the quantization coefficient block P ₁₅ is a block in which the quantization coefficient of the DC component is −1 and the quantization coefficient of the other frequency component (AC component) is 0.

In the encoding apparatus of FIG. 1, an encoded stream including the block P ₁₅ of quantization coefficients as described above is output and decoded by a decoding apparatus (not shown).

That is, the decoding apparatus obtains a decoding error block P ₁₆ corresponding to the prediction error P ₁₃ by inverse quantization and inverse DCT of the quantization coefficient block P ₁₅ .

Specifically, as described above, the quantization coefficient block P ₁₅ is a block in which the quantization coefficient of the DC component is −1 and the quantization coefficient of the other frequency components is 0. By performing inverse quantization and inverse DCT on the quantization coefficient block P ₁₅ , for example, a decoding error block P ₁₆ having pixel values of all −1 and having an average pixel value of −1 is obtained.

Further, the predicted image P ₁₂ of the block P ₁₁ encoded original image into blocks P ₁₅ quantized coefficients, as described above, a block of all the pixel values 23.

Therefore, the decoded image block P ₁₇ of the original image block P ₁₁ obtained by adding the decoding error block P ₁₆ and the predicted image P ₁₂ is a block having all the pixel values of 22 and the DC component of 22. It becomes.

  3 and 4 and FIG. 11 to be described later are diagrams for explaining in an easy-to-understand manner how the data obtained in the process of encoding and decoding the dithered original image is changed. In the figure, the value of the DCT coefficient obtained by DCT of the prediction error and the value of the decoding error obtained by inverse DCT of the DCT coefficient are not accurate values.

As described with reference to FIGS. 3 and 4, when dithering is applied to the original image, a block of quantization coefficients in which the AC component is 0 and only the DC component is 1 due to quantization ( For example, block P ₇ ) in FIG. 3 and quantization coefficient blocks (for example, block P _{15 in} FIG. 4) in which the AC component is 0 and only the DC component is −1 appear alternately. As a result, as a block of a decoded image, a block having a certain value for all pixels and a block having a value different from that value by 1 appear alternately.

  That is, FIG. 5 shows a dithered original image and a decoded image obtained by encoding the original image with the encoding device of FIG. 1 and decoding with the decoding device.

  The left side of FIG. 5 shows the dithered original image, in which a certain pixel value v and a pixel value v + 1 that is 1 larger than the pixel value v are arranged in a grid pattern, so that an intermediate gradation is obtained. The pixel value v + 0.5 is expressed in a pseudo manner.

  On the other hand, the right side of FIG. 5 shows a decoded image obtained by encoding the original image on the left side of FIG. 5 with the encoding device of FIG. 1 and decoding it with the decoding device. , And blocks of value v + 1 appear alternately.

  As described above, when dither is applied to the original image to be encoded, all pixel values are set to values in the decoded image obtained by decoding the encoded stream obtained as a result of encoding. The block of v and the block of value v + 1 appear alternately, resulting in an unnatural image, that is, the image quality of the decoded image may be deteriorated.

  The present invention has been made in view of such a situation, and can prevent deterioration in image quality of a decoded image obtained by encoding and decoding an image subjected to editing such as dithering. To do.

  A control device according to a first aspect of the present invention includes a prediction unit that obtains a prediction error between an image and a prediction image obtained by predicting the image, a conversion unit that converts the prediction error into a frequency component, and a quantum component for the frequency component. An image encoding device including quantization means for converting the frequency component into a control device for controlling a quantization step when the frequency component is quantized, and obtaining editing information indicating whether or not the image has been edited Editing information acquisition means for editing, and editing quantization step control means for controlling a quantization step when the frequency component is quantized based on the editing information.

  The control method or program according to the first aspect of the present invention includes a prediction unit that obtains a prediction error between an image and a prediction image obtained by predicting the image, a conversion unit that converts the prediction error into a frequency component, and the frequency. An image encoding device including a quantization unit that quantizes a component is a control method for controlling a quantization step when the frequency component is quantized, or a program for causing a computer to execute a control process, Editing information indicating whether or not editing has been performed on the image, and controlling a quantization step when quantizing the frequency component based on the editing information.

  An encoding apparatus according to a second aspect of the present invention is an encoding apparatus that encodes an image, a prediction unit that obtains a prediction error between the image and a predicted image obtained by predicting the image, and the prediction error as a frequency. Based on the editing information, conversion means for converting into components, quantization means for quantizing the frequency components, editing information acquisition means for acquiring editing information indicating whether the image has been edited, Editing quantization step control means for controlling a quantization step when the frequency component is quantized.

  In the first and second aspects of the present invention, a quantization step when the editing information indicating whether or not the image has been edited is acquired and the frequency component is quantized based on the editing information Is controlled.

  The program can be transmitted via a transmission medium, or can be recorded on a recording medium.

  According to the first and second aspects of the present invention, it is possible to improve the image quality of a decoded image obtained by encoding and decoding an image subjected to editing such as dithering.

  Embodiments of the present invention will be described below. Correspondences between the constituent elements of the present invention and the embodiments described in the specification or the drawings are exemplified as follows. This description is intended to confirm that the embodiments supporting the present invention are described in the specification or the drawings. Therefore, even if there is an embodiment which is described in the specification or the drawings but is not described here as an embodiment corresponding to the constituent elements of the present invention, that is not the case. It does not mean that the form does not correspond to the constituent requirements. Conversely, even if an embodiment is described here as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. It's not something to do.

The control device according to the first aspect of the present invention includes:
Prediction means (for example, the prediction unit 1 in FIG. 7) for obtaining a prediction error between an image and a predicted image obtained by predicting the image;
Conversion means for converting the prediction error into a frequency component (for example, the conversion unit 2 in FIG. 7);
When an image encoding device (for example, the image encoding unit 31 in FIG. 7) including a quantization unit (for example, the quantization unit 43 in FIG. 7) that quantizes the frequency component quantizes the frequency component Is a control device (for example, the quantization step control unit 32 in FIG. 7) that controls the quantization step of
Editing information acquisition means (for example, a dither information acquisition unit 41 in FIG. 7) for acquiring editing information indicating whether the image has been edited;
Editing quantization step control means for controlling a quantization step when the frequency component is quantized based on the editing information (for example, a dither quantization step control unit in FIG. 7).

In the control device according to the first aspect,
The image encoding apparatus includes rate control means for controlling the quantization step based on a rate of an encoded bitstream including a quantization coefficient obtained by quantizing the frequency component (for example, rate control in FIG. 7). Part 5) can be further provided,
In the editing quantization step control means, when the image is edited, the rate control means performs the quantization step based on quantization step control information for controlling the quantization step. Can be controlled.

A control method or program according to the first aspect of the present invention includes:
Prediction means (for example, the prediction unit 1 in FIG. 7) for obtaining a prediction error between an image and a predicted image obtained by predicting the image;
Conversion means for converting the prediction error into a frequency component (for example, the conversion unit 2 in FIG. 7);
When an image encoding device (for example, the image encoding unit 31 in FIG. 7) including a quantization unit (for example, the quantization unit 43 in FIG. 7) that quantizes the frequency component quantizes the frequency component A control method for controlling the quantization step of the above, or a program for causing a computer to execute control processing,
Edit information indicating whether the image has been edited (for example, step S11 in FIG. 9),
Based on the editing information, a quantization step for quantizing the frequency component is controlled (for example, step S12 in FIG. 9).
Includes steps.

The encoding device according to the second aspect of the present invention provides:
An encoding device that encodes an image (for example, the encoding device 22 in FIG. 6);
Prediction means for obtaining a prediction error between the image and a prediction image obtained by predicting the image (for example, the prediction unit 1 in FIG. 7);
Conversion means for converting the prediction error into a frequency component (for example, the conversion unit 2 in FIG. 7);
Quantization means for quantizing the frequency component (for example, the quantization unit 43 in FIG. 7);
Editing information acquisition means (for example, a dither information acquisition unit 41 in FIG. 7) for acquiring editing information indicating whether the image has been edited;
Editing quantization step control means for controlling a quantization step when the frequency component is quantized based on the editing information (for example, a dither quantization step control unit in FIG. 7).

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 6 shows an authoring system to which the present invention is applied (a system is a logical collection of a plurality of devices, regardless of whether or not the devices of each configuration are in the same casing). The structural example of embodiment is shown.

  In FIG. 6, the authoring system includes an authoring device 21, an encoding device 22, and a recording control device 23.

  The authoring device 21 is supplied with an image (moving image) as a material to be edited. The authoring device 21 performs processing corresponding to the operation of the operator of the authoring system, that is, processing such as dithering and special effects, for the image as the material supplied thereto, and the image as a result of authoring Is supplied to the encoding device 22.

  The encoding device 22 encodes the original image that is the result of the authoring from the authoring device 21 as an original image to be encoded, and controls the recording of the encoded bitstream obtained as a result of the encoding. Supply to device 23.

  That is, the encoding device 22 encodes the original image from the authoring device 21 by converting the prediction error of the original image into a frequency component and quantizing the frequency component as in the encoding device of FIG. The encoded bit stream is supplied to the recording control device 23.

  The recording control device 23 records the encoded bit stream from the encoding device 22 on a recording medium 24 such as a disc (for example, a Blu-ray disc or a DVD (Digital Versatile Disc)) or a semiconductor memory.

  FIG. 7 shows a configuration example of the encoding device 22 of FIG.

  In the figure, portions corresponding to those of the encoding device of FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate.

  The encoding device 22 includes an image encoding unit 31 and a quantization step control unit 32.

  The image encoding unit 31 includes a prediction unit 1, a conversion unit 2, a rate control unit 5, a quantization unit 43, and a generation unit 44, and is an encoding target supplied from the authoring device 21 of FIG. An original image is encoded and an encoded bit stream is output.

  In other words, the prediction unit 1 is supplied with the original image to be encoded from the authoring device 21 of FIG. As described with reference to FIG. 1, the prediction unit 1 divides an original image supplied thereto into predetermined blocks and obtains a predicted image of the block of the original image. Further, the prediction unit 1 obtains a prediction error between the block of the original image and the prediction image, and supplies the prediction error to the conversion unit 2 in units of blocks.

  As described with reference to FIG. 1, the transform unit 2 transforms the prediction error block from the predictor 1 into a block of frequency components by transform such as orthogonal transform such as DCT or Hadamard transform, and the quantizer 43.

  As described with reference to FIG. 1, the rate control unit 5 monitors the encoded bit stream output from the generation unit 44 and controls the quantization step of the quantization unit 43 based on the rate of the encoded bit stream. .

  That is, when the rate of the encoded bit stream output from the generation unit 44 is high, the rate control unit 5 determines a quantization step for performing rough quantization in the quantization unit 43 in order to reduce the rate. Generate quantization control information. In addition, when the rate of the encoded bit stream output from the generation unit 44 is low, the rate control unit 5 determines a quantization step for performing detailed quantization in the quantization unit 43 in order to increase the rate. Generate quantization control information.

  Then, the rate control unit 5 supplies the quantization control information generated based on the rate of the encoded bit stream output from the generation unit 44 to the quantization unit 43, whereby the rate of the encoded bit stream is set to a predetermined value. The quantization step is controlled to be a value of (a value within a predetermined range).

  Here, in the encoding device 22, the quantization control information for controlling the quantization step of the quantization unit 43 is generated not only by the rate control unit 5 but also by a dither quantization step control unit 42 described later. . Therefore, in order to distinguish between the quantization control information generated by the rate control unit 5 and the quantization control information generated by the dithering quantization step control unit 42, the rate control unit 5 generates the following appropriately. The quantization control information is referred to as first quantization control information, and the quantization control information generated by the dither quantization step control unit 42 is referred to as second quantization control information.

  The rate control unit 5 supplies the first quantization control information to the quantization unit 43 as well as to the dither quantization step control unit 42 and the generation unit 44.

  The quantization unit 43 is supplied with a block of frequency components from the conversion unit 2, is supplied with first quantization control information from the rate control unit 5, and is also supplied with a second from the dither quantization step control unit 42. Quantization control information is supplied.

  When the quantization unit 43 quantizes the frequency component based on the first quantization control information from the rate control unit 5 and the second quantization control information from the dither quantization step control unit 42 The quantization step is determined, the frequency component of the block unit from the transform unit 2 is quantized in the quantization step, and the quantization coefficient resulting from the quantization is supplied to the generation unit 44 in block unit To do.

  In addition to the quantization coefficient supplied from the quantization unit 43 to the generation unit 44, the first quantization control information used to determine the quantization step by the quantization unit 43 is received from the rate control unit 5. The second quantization control information used to determine the quantization step by the quantization unit 43 is supplied from the dither quantization step control unit 42.

  The generation unit 44, the block-unit quantization coefficient from the quantization unit 43, the first quantization control information from the rate control unit 5, and the second quantization control from the dither quantization step control unit 42 An encoded bit stream including at least information is generated and output as an encoding result of the original image.

  The quantization step control unit 32 includes a dither information acquisition unit 41 and a dither quantization step control unit 42, and the image encoding unit 31 (the quantization unit 43 thereof) performs quantization when the frequency component is quantized. Control the step.

  That is, the dither information acquisition unit 41 acquires dither information indicating whether or not dither has been applied to the original image that is to be encoded by the image encoding unit 31, and supplies the dither information to the dither quantization step control unit 42. To do.

  The dither quantization step control unit 42 controls the quantization step when the quantization unit 43 quantizes the frequency component based on the dither information supplied from the dither information acquisition unit 41.

  That is, the dither quantization step control unit 42 recognizes based on the dither information whether the quantizing unit 43 has dithered the target block that is the block whose frequency component is the quantization target. If no dither is applied to the block of interest, the quantization unit 43 generates, for example, second quantization control information in which the same quantization step as that of the encoding device in FIG. 1 is determined.

  Also, the dither quantization step control unit 42 determines the second quantization step for performing fine quantization on the high frequency component in the quantization unit 43 when the target block is dithered. The quantization control information is generated based on the first quantization information supplied from the rate control unit 5.

  Then, the dither quantization step control unit 42 supplies the second quantization control information to the quantization unit 43, thereby controlling the quantization step used in the quantization performed by the quantization unit 43. .

  The dither quantization step control unit 42 supplies the second quantization control information to the generation unit 44 in addition to supplying the quantization unit 43. As described above, the generation unit 44 includes the second quantization control information from the dither quantization step control unit 42 in the encoded bitstream.

  Next, as described above, the dither information acquisition unit 41 acquires dither information indicating whether or not dither has been applied to the original image. The acquisition method will be described.

  Dithering is performed on the original image in accordance with the operation of the operator in the authoring device 21 of FIG. Therefore, when the authoring device 21 outputs information related to dither performed according to the operation of the operator, that is, for example, information indicating the position of the original image subjected to dither, the dither information acquisition unit 41 By receiving information on dither output from the authoring device 21, the information can be acquired as dither information.

7 encodes the original image by converting the prediction error of the original image into a frequency component and quantizing the frequency component, like the encoding device of FIG. However, assuming that, for example, a DCT coefficient is used as the frequency component of the prediction error of the original image, if the block where the conversion unit 2 converts the prediction error into the DCT coefficient is dithered, the block of DCT coefficients obtained in part 2, for example, and blocks P ₆ in FIG. 3, as shown in block P ₁₄ in FIG. 4, the highest frequency component, for example, DCT coefficients of the lower right block Becomes 1 or -1.

  Therefore, the dither information acquisition unit 41 can acquire dither information by determining whether or not dither is applied to the original image based on the frequency component of the prediction error obtained by the conversion unit 2.

  That is, the dither information acquisition unit 41 determines whether the lower right DCT coefficient of the DCT coefficient block obtained by the conversion unit 2 is 1 or −1, and the lower right DCT coefficient of the DCT coefficient block is 1. Alternatively, when it is −1, it is recognized that the block is dithered, and the dither information to that effect can be supplied to the dither quantization step control unit 42.

  Here, FIG. 8 shows the basis of DCT when DCT is performed on a block of horizontal × vertical 8 × 8.

  Now, assuming that the DCT coefficient in the i-th row from the top and the j-th column from the left is the (i, j) component, if the original image is dithered, In the image prediction error block, the pixel values of 0 and 1 and the pixel values of -1 and 0 are arranged in a lattice form, as described with reference to FIGS.

  Then, when a block in which the pixel values of 0 and 1 and the pixel values of -1 and 0 are arranged in a grid is DCTed in this way, in the block of DCT coefficients obtained as a result of the DCT, The (8,8) component which is the lower DCT coefficient is 1 or -1.

  Therefore, the dither information acquisition unit 41 can determine whether the block is dithered based on whether the DCT coefficient at the lower right of the DCT coefficient block is 1 or -1.

  Note that the dithered image is an HD image, and when the SD image is generated by thinning out every other row of pixels of the HD image that are lined up horizontally, dithering is performed. In the case where an SD image is generated by thinning out every other column of pixels arranged in the vertical direction of the HD image, when the SD image is used as an original image to be encoded, In the image prediction error block, pixel values of 0 and 1 and pixel values of -1 and 0 are not arranged in a grid pattern, but are alternately arranged in only one of the horizontal direction and the vertical direction.

  When such a prediction error block is DCT, the DCT coefficient block obtained as a result of the DCT is the (1,8) component which is the DCT coefficient at the right end of the block or the DCT coefficient at the lower end (8.1). The component is 1 or -1.

  Therefore, in the dither information acquisition unit 41, when the block of the DCT coefficient is an 8 × 8 block, the (8,8) component is 1 or -1, and the (1,8) component is , (8.1) Even if the component is 1 or -1, it recognizes that the dithering is applied to the block of the original image, and supplies the dither information to that effect to the dithering quantization step control unit 42. Can be.

  The dither information acquisition unit 41 is not limited to the method of recognizing whether the original image is dithered based on the frequency component of the prediction error.

  That is, the dither information acquisition unit 41 determines whether the lower right frequency component of the frequency component block is a predetermined value based on the frequency component of the pixel value of the original image itself, not the prediction error of the original image. Whether the original image block is dithered by determining whether two pixel values differing by 1 are arranged in a grid pattern based on the pixel value of the original image itself I can recognize.

  Here, since the frequency component of the prediction error is obtained by the conversion unit 2 of the image coding unit 31, it is not necessary to obtain it separately only to recognize whether or not dithering is applied to the block of the original image.

  Further, when recognizing (determining) whether or not dither is applied to a block of the original image based on a specific frequency component of the block, only the specific frequency component of the block may be referred to. The frequency component (distribution) of the entire block may be referred to.

  Next, processing of the encoding device 22 in FIG. 7 will be described.

  In the encoding device 22 of FIG. 7, based on the dither information, the dither quantization step control process for controlling the quantization step of the quantization unit 43, the prediction error of the original image is converted into a frequency component, and the frequency component Is encoded, and an encoding process for encoding the original image is performed.

  FIG. 9 is a flowchart for explaining the dither quantization step control process performed by the encoding device 22 of FIG.

  The dither quantization step control process of FIG. 9 is performed by the quantization step control unit 32 of the encoding device 22 of FIG.

  That is, in the quantization step control unit 32, in step S11, the dither information acquisition unit 41 dithers to a target block that is a block of frequency components that are quantized by the quantization unit 43 of the image encoding unit 31. Is obtained, supplied to the dither quantization step control unit 42, and the process proceeds to step S12.

  In step S12, the dither quantization step control unit 42 controls the quantization step when the quantization unit 43 quantizes the frequency component based on the dither information supplied from the dither information acquisition unit 41, and This completes the quantization step control process.

  That is, the dither quantization step control unit 42 determines, based on the dither information from the dither information acquisition unit 41, for example, a second quantization step that is determined by the quantization unit 43 as in the encoding apparatus of FIG. Quantization control information or second quantization control information for determining a quantization step for performing fine quantization on high frequency components is generated and supplied to the quantization unit 43.

  After that, after waiting for a new block of frequency components to be supplied as a target block from the transform unit 2 to the quantization unit 43, the process returns to step S11, and the processes of steps S11 and S12 are repeated.

  The dither quantization step control unit 42 is supplied from the rate control unit 5 when the dither information from the dither information acquisition unit 41 indicates that the target block is dithered. Based on the quantization control information of 1, the second quantization control information for controlling the quantization step of the quantization unit 43 is generated.

  Here, the quantization unit 43 performs the quantization step based on both the first quantization control information from the rate control unit 5 and the second quantization control information from the dither quantization step control unit 42. To decide.

  For example, the first quantization control information is represented as A, the second quantization control information is represented as B, and the first quantization control information A and the second quantization control information B are arguments. If the function is expressed as f (A, B), the quantization unit 43 determines the value of the function f (A, B) as a quantization step.

  On the other hand, in the case where dither is applied to the target block, as described in FIGS. 3 and 4, the highest frequency component of the target block is −1 or 1, that is, for example, If the target block is an 8 × 8 block, as described in FIG. 8, the highest frequency component (1,8) component, (8,1) component, or (8,8) component is 1 or −1, but the highest frequency component is quantized to 0, so that the image quality of the decoded image is deteriorated.

  Accordingly, when the dither information from the dither information acquisition unit 41 indicates that the target block has been dithered, the dither quantization step control unit 42 determines the value of the highest frequency component of the target block. The second quantization control information for controlling the quantization step in the quantization unit 43 is dynamically changed so that -1 or 1 is quantized into a quantization coefficient that can be restored by inverse quantization. To generate.

  That is, since the quantization step in the quantization unit 43 is determined based on both the first and second quantization control information as described above, the dither quantization step control unit 42 When the frequency component of the target block is quantized in the quantization step determined based on the second quantization control information, the highest frequency component value of -1 or 1 is obtained by inverse quantization. Based on the first quantization control information, second quantization control information is generated so as to be quantized into a quantization coefficient that can be restored.

  Next, FIG. 10 is a flowchart illustrating an encoding process performed by the encoding device 22 of FIG.

  The image encoding unit 31 in FIG. 7 performs the encoding process in FIG.

  That is, in the image encoding unit 31, the prediction unit 1 is supplied with the original image to be encoded from the authoring device 21 in FIG. As described with reference to FIG. 1, the prediction unit 1 divides an original image supplied thereto into predetermined blocks, and selects a block to be encoded as a target block.

  And the prediction part 1 calculates | requires the prediction image of an attention block in step S31, and calculates | requires the prediction error of the attention block and prediction image of an original image. Furthermore, the prediction unit 1 supplies the prediction error obtained for the block of interest to the conversion unit 2, and proceeds to step S32.

  In step S32, the conversion unit 2 converts the prediction error for the target block from the prediction unit 1 into a frequency component, supplies the frequency component for the target block to the quantization unit 43, and proceeds to step S33.

  In step S <b> 33, the quantization unit 43 receives attention based on the first quantization control information supplied from the rate control unit 5 and the second quantization control information from the dither quantization step control unit 42. A quantization step for quantizing the frequency component of the block is determined, and in that quantization step, each frequency component of the block of interest from the transform unit 2 is quantized, and the quantization coefficient that is the result of the quantization is determined. The block is supplied to the generation unit 44, and the process proceeds to step S34.

  That is, the rate control unit 5 monitors the encoded bit stream output from the generating unit 44, and controls the quantization step of the quantization unit 43 based on the rate of the encoded bit stream. The quantization control information is generated and supplied to the dither quantization step control unit 42, the quantization unit 43, and the generation unit 44.

  Further, as described with reference to FIG. 9, the dither quantization step control unit 42 performs dither information on the block of interest in which the frequency component is supplied from the transform unit 2 to the quantization unit 43, and further, if necessary, the rate. Based on the first quantization control information from the control unit 5, second quantization control information for controlling the quantization step of the quantization unit 43 is generated and supplied to the quantization unit 43 and the generation unit 44. To do.

  In step S33, the quantization unit 43 performs the first quantization control information supplied from the rate control unit 5 as described above and the second quantization control supplied from the dither quantization step control unit 42. Based on the information, the quantization step for quantizing the frequency component of the block of interest is determined. Then, in the quantization step, the quantization unit 43 quantizes each frequency component of the target block from the conversion unit 2, and supplies the quantization coefficient block resulting from the quantization to the generation unit 44. The process proceeds from step S33 to step S34.

  In step S <b> 34, the generation unit 44 performs the block-unit quantization coefficient from the quantization unit 43, the first quantization control information from the rate control unit 5, and the second dither quantization step control unit 42. The encoded bit stream including at least the quantization control information is generated and output as the encoding result of the original image.

  Thereafter, the process waits for the prediction unit 1 to select a new block of the original image as the block of interest, returns to step S31, and repeats the processes of steps S31 to S34.

  As described above, in the encoding device 22, when the original image is encoded by converting the prediction error of the original image into a frequency component and quantizing the frequency component, the quantization is performed based on the dither information. Control the step, i.e. the highest frequency component of the block of the original image is -1 or 1, so when the block of the original image is dithered, the value of the highest frequency component of the block is Since the quantization step is controlled so that a certain -1 or 1 is quantized to a quantized coefficient that can be recovered by inverse quantization, the highest frequency component is specifically quantized to 0, Thus, it is possible to prevent the image quality of the decoded image of the original image that has been dithered from deteriorating.

  In this way, it will be further described with reference to FIG. 11 that the encoding device 22 can prevent degradation of the image quality of the decoded image of the original image that has been dithered.

  FIG. 11 shows data obtained in the process of encoding and decoding (local decoding) the dithered original image in the encoding device 22.

  For example, as in the case described with reference to FIG. 3 above, the encoding device 22 encodes the original image in units of 256-pixel blocks of horizontal × vertical 16 × 16 pixels. As a predicted image of a target block that is a block of an original image to be converted, an image of a block adjacent to the left of the target block that has already been encoded and locally decoded is used.

In FIG. 11, a block P ₁₀₁ is, for example, the leftmost block of the original image, and a block P ₁₀₃ is, for example, a block adjacent to the right of the block P ₁₀₁ of the original image.

In both blocks P ₁₀₁ and P _{103 of the} original image, for example, 22 pixel values and 23 pixel values obtained by adding 1 to the pixel value are alternately arranged only in the horizontal direction, and in the vertical direction. Only 22 or 23 pixel values are arranged, and therefore dithered.

Here, since the same number of pixel values 22 and 23 exist in the blocks P ₁₀₁ and P ₁₀₃ of the original image, the DC component that is the average value of the pixel values is 22.5.

The block P ₁₀₂ is a block of a decoded image obtained by encoding and locally decoding the block P _{101 of the} original image with the encoding device 22 (the image encoding unit 31) of FIG. 11, the pixel values of the block P ₁₀₂ of the decoded image are all become 22, therefore, the DC component is 22.

When the encoding device 22 performs encoding using the original image block P ₁₀₃ as a target block, the prediction unit 1 encodes the block P ₁₀₁ adjacent to the left of the original image block P _{103 and} performs local decoding. The decoded error block P ₁₀₂ is used as it is as the predicted image P ₁₀₄ of the original image block P ₁₀₃ to obtain a prediction error P ₁₀₅ .

In block P ₁₀₃ of the original image, as described above, in the horizontal direction, pixel values of 22 and 23 are alternately arranged in the vertical direction, only the pixel value of 22 also 23 are arranged, the predicted image P ₁₀₄ As described above, since all the pixel values are the decoded image block P ₁₀₂ of 22, the prediction error P ₁₀₅ between the block P _{103 of the} original image and the prediction image P ₁₀₄ is in the horizontal direction, A block in which pixel values of 0 and 1 are alternately arranged, and only pixel values of 0 or 1 are arranged in the vertical direction.

The prediction error P ₁₀₅ is converted into a frequency component by the conversion unit 2. Assuming that the conversion unit 2 performs, for example, DCT as the conversion to the frequency component, the prediction error P ₁₀₅ is roughly converted into a DCT coefficient block such as the block P ₁₀₆ .

Here, the prediction error P ₁₀₅ is in the horizontal direction, 0 and 1 of the pixel values are arranged alternately in the vertical direction, a block in which only the pixel value of 0 or 1 is placed, therefore, the prediction error P ₁₀₅ In this case, since the same number of pixel values of 0 and 1 exist, the DC component is 0.5.

In addition, in the prediction error P ₁₀₅ , since the pixel values of 0 and 1 are alternately arranged, the prediction error P105 has the maximum frequency component that a block of 16 × 16 pixels can have, and its size is , 1

Therefore, in block P ₁₀₆ of the DCT coefficient is the DCT coefficient of the upper left 0.5 is a DC component, which is the highest frequency component, for example, the DCT coefficients of the lower right 1, the other DCT coefficients 0, Each has become.

The quantization unit 43 quantizes the DCT coefficient block P ₁₀₆ as described above, and outputs a quantization coefficient block P ₁₀₇ .

  Here, as described above, in the encoding device of FIG. 1, for example, using the fact that human vision is less sensitive to high frequency components than low frequency components, the DCT coefficients of low frequency components are used. 7, fine quantization is performed, and DCT coefficients of high frequency components are subjected to coarse quantization. However, in principle, the encoding device 22 in FIG. 7 has the same quantum as the encoding device in FIG. 1. Is done.

  However, in the encoding device 22, when dithering is performed on the block of the original image, the dithering quantization step control unit 42 determines that the value of the highest frequency component of the block being dithered is −1 or The quantization step of the quantization unit 43 is controlled so that 1 is quantized to a quantization coefficient that can be restored by inverse quantization.

Therefore, in the quantization coefficient block P ₁₀₇ , the DCT coefficient 0.5 of the upper left DCT coefficient of the DCT coefficient block P ₁₀₆ is 1, and the lower right DCT coefficient 1 of the highest frequency component is 1, The other DCT coefficients 0 are quantized to 1 and 0, respectively.

The encoding device 22 outputs an encoded stream including the block P ₁₀₇ of quantized coefficients as described above, and is decoded by a decoding device (not shown).

That is, the decoding device obtains a decoding error corresponding to the prediction error P ₁₀₅ , which is a difference between the decoded image and the predicted image, by dequantizing the block P ₁₀₇ of the quantized coefficient and performing inverse DCT. By adding the decoding error and the predicted image, a decoded image is obtained.

As described above, the quantization coefficient block P ₁₀₇ is a block in which the quantization coefficient of the DC component and the highest frequency component is 1, and the quantization coefficient of the other frequency component is 0. As a result of inverse quantization and inverse DCT of the quantization coefficient block P ₁₀₇ , for example, similarly to the prediction error P ₁₀₅ , pixel values of 0 and 1 are alternately arranged in the horizontal direction, and 0 in the vertical direction. Alternatively, a decoding error block P ₁₀₈ in which only one pixel value is arranged is obtained.

Further, as described above, the predicted image P ₁₀₄ of the original image block P ₁₀₃ encoded in the quantization coefficient block P ₁₀₇ is a block having all the pixel values of 22.

Thus, obtained by adding the block P ₁₀₈ of the decoding error and the predicted image P _104, block P ₁₀₉ of the decoded image of the block P ₁₀₃ of the original image, similarly to the block P ₁₀₃ of the original source image, the 22 A pixel value and 23 pixel values that differ from the pixel value by 1 are arranged alternately only in the horizontal direction, and only 22 or 23 pixel values are arranged in the vertical direction. This is a block that maintains the dither applied to the block P ₁₀₃ of the image.

  Next, the encoding device 22 in FIG. 7 can be applied to an encoding method for encoding an original image by converting the prediction error of the original image into a frequency component and quantizing the frequency component.

  Examples of such an encoding method include MPEG (Moving Picture Experts Group) 2, MPEG4 visual, and H.264 / AVC (Advanced Video Coding).

  FIG. 12 shows a configuration example of the encoding device 22 when H.264 / AVC is adopted as the encoding method.

  In the figure, portions corresponding to those in FIG. 7 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate.

  The original images from the authoring device 21 (FIG. 6) are supplied to the rearrangement unit 61 in the display order.

  The rearrangement unit 61 rearranges the original images in the display order from the authoring device 21 in the encoding (decoding) order, and sequentially supplies the images to the calculation unit 62 and the inter prediction unit 71.

  The calculation unit 62 divides the original image from the rearrangement unit 61 into predetermined blocks, and selects a target block to be encoded. Further, the calculation unit 62 obtains a prediction error between the block of interest and the predicted image of the block of interest supplied from the mode determination unit 72 described later as needed, and supplies the prediction error to the orthogonal transform unit 63. The orthogonal transform unit 63 performs orthogonal transform processing such as DCT and, if necessary, Hadamard transform on the prediction error of the target block from the calculation unit 62, and the orthogonal transform coefficient of the target block obtained as a result Is supplied to the quantization unit 64.

  The quantization unit 64 is supplied with the orthogonal transform coefficient of the block of interest from the orthogonal transform unit 63, and receives a quantization parameter Qp as first quantization control information from the rate quantization step control unit 75 described later. In addition, the scaling list as the second quantization control information is supplied from the dithering quantization step control unit 42.

  The quantization unit 64 determines a quantization step based on the quantization parameter Qp from the rate quantization step control unit 75 and the scaling list from the dither quantization step control unit 42, and the quantization step Thus, the orthogonal transform coefficient of the target block from the orthogonal transform unit 63 is quantized, and the quantized coefficient of the target block obtained as a result is supplied to the entropy encoding unit 73.

  The entropy encoding unit 73 includes a quantization coefficient supplied from the quantization unit 64, a quantization parameter Qp supplied from the rate quantization step control unit 75, and a scaling supplied from the dither quantization step control unit 42. Codes can be applied to lists and other data by processing such as CAVLC (Context-Adaptive Variable Length Coding) or other variable-length coding, or CABAC (Context-Adaptive Binary Arithmetic Coding). A generated bit stream is generated and supplied to the buffer 74.

  The buffer 74 temporarily stores the encoded bit stream from the entropy encoding unit 73, reads it out as appropriate, and supplies it to the recording control device 23 (FIG. 6).

  Further, the buffer 74 supplies the storage amount of the encoded bit stream to the rate quantization step control unit 75 as appropriate.

  The rate quantization step control unit 75 controls the quantization step of the quantization unit 64 based on the storage amount of the encoded bit stream from the buffer 74.

  That is, when the storage amount of the encoded bitstream is large, the rate quantization step control unit 75 performs a quantization step for performing rough quantization by the quantization unit 64 in order to reduce the storage amount. The quantization parameter Qp as the first quantization control information to be determined is determined. Also, the rate quantization step control unit 75 has a quantization step for performing fine quantization in the quantization unit 64 in order to increase the storage amount when the storage amount of the encoded bitstream is small. The quantization parameter Qp as the first quantization control information to be determined is determined.

  The rate quantization step control unit 75 supplies the quantization parameter Qp as the first quantization control information generated based on the storage amount of the encoded bitstream to the quantization unit 64 as described above. Thus, the quantization step is controlled so that the rate of the encoded bit stream becomes a predetermined value (a value within a predetermined range).

  Here, in H.264 / AVC, the larger the quantization parameter Qp, the larger the quantization step (quantization granularity). Therefore, when the storage amount of the encoded bitstream is large, the quantization parameter Qp is determined to be a large value, and the quantization step is also determined to be a large value. As a result, coarse quantization is performed, and the rate (data amount) of the encoded bit stream becomes small. On the other hand, when the storage amount of the encoded bitstream is small, the quantization parameter Qp is determined to be a small value, and the quantization step is also determined to be a small value. As a result, fine quantization is performed and the rate of the encoded bit stream is increased.

  As described above, in the conventional coding apparatus that performs coding in accordance with H.264 / AVC, when the storage amount of the coded bitstream is large, the quantization parameter Qp is determined to be a large value, Since the conversion step is also determined to be a large value, as described with reference to FIGS. 3 and 4 above, 1 or −1 of the right lower orthogonal transform coefficient (DCT coefficient, etc.) which is the highest frequency component in the block Is quantized to 0.

  In H.264 / AVC, the quantization parameter Qp is determined for each so-called macroblock.

  The rate quantization step control unit 75 supplies the quantization parameter Qp as the first quantization control information to the quantization unit 64, and also supplies the dither quantization step control unit 42 and the entropy encoding unit 73. Supply.

  On the other hand, the dither information acquisition unit 41 acquires dither information indicating whether or not the target block to be quantized by the quantization unit 64 is dithered and supplies it to the dither quantization step control unit 42. To do.

  The dither quantization step control unit 42 controls the quantization step when the quantization unit 64 quantizes the frequency component based on the dither information from the dither information acquisition unit 41.

  That is, when the dither information from the dither information acquisition unit 41 does not indicate that the target block is dithered, the dither quantization step control unit 42 uses the second dither control information as the second quantization control information. The scaling value of each scaling matrix in the set of scaling matrices (Scaling Matrix) that is the scaling list is set to a standard value (default).

  Here, in H.264 / AVC, a scaling matrix is prepared so that the quantization granularity can be changed for each orthogonal transform coefficient of a block (DCT Matrix) subject to orthogonal transform such as DCT. .

  When the block to be orthogonally transformed is, for example, a 4 × 4 block, the scaling matrix is a matrix having 4 × 4 components, and each component is called a scaling value. In H.264 / AVC, the quantization step is determined based on both of the above-described quantization parameter Qp and the scaling value, and in this quantization step, the orthogonal transform coefficient is quantized. In, the orthogonal transform coefficient is divided by the scaling value.

  Therefore, if the scaling value is small, the quantization step is small and fine quantization is performed. On the other hand, if the scaling value is large, the quantization step is large and coarse quantization is performed.

  In H.264 / AVC, the scaling value can be set to an integer value in the range of 1 to 255, and the standard value is 16.

  Therefore, if the dither information from the dither information acquisition unit 41 does not indicate that the target block has been dithered, the dither quantization step control unit 42 sets the scaling value of the scaling matrix to the standard value. Set the value to 16.

  In H.264 / AVC, there are a luminance signal block, a color signal Cb block, and a color signal Cr block as blocks to be subjected to orthogonal transformation, and the luminance signal block includes 4 × 4. There are two types of blocks, 8 × 8 blocks.

  In H.264 / AVC, a scaling matrix can be specified for each of a block to be intra-encoded (intra-frame prediction) and a block to be inter-encoded (inter-frame prediction).

  Therefore, the scaling matrix is for intra coding and inter coding for each of the 4 × 4 block of the luminance signal, the 8 × 8 block of the luminance signal, the block of the color signal Cb, and the block of the color signal Cr. Exists. That is, there are eight scaling matrices. A set of these eight scaling matrices is called a scaling list.

  When the dither information from the dither information acquisition unit 41 does not indicate that the target block is dithered, the dither quantization step control unit 42 uses a scaling value of a standard value as a component as a scaling matrix. Is supplied to the quantization unit 64 and the entropy encoding unit 73 as second quantization control information.

  In addition, when the dither information from the dither information acquisition unit 41 indicates that the target block has been dithered, the dither quantization step control unit 42 starts from the rate quantization step control unit 75. Based on the supplied quantization parameter Qp as the first quantization control information, a scaling list as second quantization control information for controlling the quantization step of the quantization unit 64 is generated.

  That is, when the dither information from the dither information acquisition unit 41 indicates that the target block is dithered, the dither quantization step control unit 42 is orthogonal to the highest frequency component of the target block. Second quantization control for controlling the quantization step in the quantization unit 64 so that the transform coefficient value −1 or 1 is quantized into a quantization coefficient that can be restored by inverse quantization. Generate a scaling list as information.

  In H.264 / AVC, the quantization step is determined based on both the quantization parameter Qp and the scaling value, so that when the quantization step is performed, the highest frequency component is obtained. Based on the quantization parameter Qp, the scaling value is set so that −1 or 1 which is the value of the orthogonal transform coefficient is quantized to a quantization coefficient that can be restored by inverse quantization.

  Here, as described above, the quantization step increases as the quantization parameter Qp increases, and decreases as the scaling value decreases.

  For this reason, the dithering quantization step control unit 42 sets a small value as the scaling value when the quantization parameter Qp is large, and as the scaling value when the quantization parameter Qp is small. A large value is set. As a result, the value of the orthogonal transform coefficient of -1 or 1 of the highest frequency component of the block of interest is quantized by the quantization unit 64 so that it is quantized into a quantized coefficient that can be restored by inverse quantization. A scaling value is set that controls the quantization step.

  That is, FIG. 13 shows a scaling value set by the dithering quantization step control unit 42 based on the quantization parameter Qp when the original image is dithered.

  As described above, the dithering quantization step control unit 42 sets a small scaling value when the quantization parameter Qp is large, and sets a large value when the quantization parameter Qp is small. Set the scaling value.

  Returning to FIG. 12, when the dithering quantization step control unit 42 sets a scaling value, a scaling list that is a set of scaling matrices having the scaling value as a component is used as the second quantization control information, and the quantization unit. 64 and the entropy encoding unit 73.

  Here, the entropy encoding unit 73 includes the scaling list supplied from the dither quantization step control unit 42 in the encoded bitstream. In H.264 / AVC, the scaling list is the encoded bitstream ( It is included in SPS (Sequence Parameter Set) or PPS (Picture Parameter Set) in the syntax (Syntax). This encoded bit stream is compliant with H.264 / AVC, and therefore can be decoded by a conventional decoding device compliant with H.264 / AVC.

  The H.264 / AVC scaling list corresponds to an MPEG2 or MPEG4 visual quantization matrix.

  As described above, the quantization coefficient of the block of interest obtained by the quantization unit 64 is supplied to the entropy encoding unit 73 and included in the encoded bitstream, but is also supplied to the inverse quantization unit 65. The

  The inverse quantization unit 65 inversely quantizes the quantization coefficient of the target block from the quantization unit 64 and supplies the orthogonal transform coefficient of the target block obtained as a result to the inverse orthogonal transform unit 66.

  The inverse orthogonal transform unit 66 performs an inverse orthogonal transform process on the transform coefficient supplied from the inverse quantization unit 65, thereby obtaining a decoding error corresponding to the prediction error of the block of interest and supplying the decoding unit 67 with the decoding error. .

  The calculation unit 67 adds the prediction image supplied from the mode determination unit 72 and used to obtain the prediction error of the block of interest and the decoding error from the inverse orthogonal transform unit 66 as necessary. The decoded image of the block of interest is locally decoded. The decoded image obtained by the calculation unit 67 is supplied to the filter 68.

  The filter 68 performs a deblocking process to remove block distortion on the decoded image of the block of interest as necessary, and supplies the result to the frame memory 69.

  The frame memory 69 stores the decoded image from the filter 68 as a reference image used for generating a predicted image. This reference image is referred to when the intra prediction unit 70 and the inter prediction unit 71 generate a prediction image.

  In other words, the intra prediction unit 70 obtains a prediction error from the reference image, which is a decoded image that has already been encoded, locally decoded, and stored in the frame memory 69 among the pictures of the block of interest. An intra prediction image is generated by extracting an image of a block adjacent to the target block to be supplied, and supplied to the mode determination unit 72.

  In addition, the inter prediction unit 71 includes a reference image that is a decoded image that has already been encoded, locally decoded, and stored in the frame memory 69 among pictures different from the picture (frame or field) of the block of interest. Detects a motion vector representing the motion between the block of interest, and performs motion compensation on the reference image based on the motion vector (MotionEstimation / Compensation) to generate an inter prediction image and a mode determination unit 72.

  The mode determination unit 72 selects a prediction mode to be applied to the block of interest (ModeDecision), and according to the prediction mode, the intra prediction image from the intra prediction unit 70 or the inter prediction image from the inter prediction unit 71 One is supplied to the calculation units 62 and 67 as a predicted image of the block of interest.

  As described above, the calculation unit 62 obtains a prediction error using the predicted image from the mode determination unit 72, and the calculation unit 67 obtains a decoded image using the prediction image from the mode determination unit 72. .

  As described above, when dithering is applied to the original image, the highest frequency component, which is the frequency component generated by the dithering, is quantized into a quantization coefficient that can be restored by inverse quantization. As described above, since the quantization step is controlled separately from the control of the quantization step based on the encoded bit stream, it is possible to prevent deterioration of the image quality of the decoded image.

  Furthermore, when the dither is applied to the original image, the quantization step is controlled using a scaling value that can be set for each frequency component, and only the frequency component generated by the dither is dequantized. Since the scaling value is set so as to be quantized into a quantized coefficient that can be restored, an increase in the data amount of the encoded bitstream can be suppressed.

  Here, controlling the quantization step depending on whether or not dither has been applied to the original image as described above includes the case where a prediction error is obtained using an inter prediction image and the case where prediction is performed using an intra prediction image. The present invention is applicable to both cases where an error is obtained, but is particularly useful when a prediction error is obtained using an intra-predicted image.

  In FIG. 12, rearrangement unit 61, calculation unit 62, inverse quantization unit 65, inverse orthogonal transform unit 66, calculation unit 67, filter 68, frame memory 69, intra prediction unit 70, inter prediction unit 71, and mode The determination unit 72 corresponds to the prediction unit 1 in FIG. 7, and the orthogonal transformation unit 63 corresponds to the conversion unit 2 in FIG. Further, the quantization unit 64 corresponds to the quantization unit 43 in FIG. 7, and the entropy encoding unit 73 corresponds to the generation unit 44 in FIG. 7. Further, the buffer 74 and the rate quantization step control unit 75 correspond to the rate control unit 5 of FIG.

  Next, FIG. 14 shows a configuration example of another embodiment of an authoring system to which the present invention is applied.

  In the figure, portions corresponding to those in FIG. 6 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate.

  That is, the authoring system of FIG. 14 is the same as the authoring system of FIG. 6 in that it has an authoring device 21, an encoding device 22, and a recording control device 23, and an encoding device 91, a decoding device 92, a display device 93, 6 is different from the authoring system of FIG. 6 in that an operation device 94 is newly provided.

  As described with reference to FIG. 6, the authoring device 21 performs processing according to the operation of the operator of the authoring system on the image as the material supplied thereto, and the image as a result of the authoring is encoded. 22 and 91.

  The encoding device 91 encodes the image from the authoring device 21 as an original image to be encoded by the same encoding method as the encoding device 22, that is, the encoding device 22 converts the original image, for example, When encoding with an encoding method compliant with H.264 / AVC, the encoding device 91 also encodes the original image with an encoding method compliant with H.264 / AVC, and the resulting code The encoded bit stream is supplied to the decoding device 92.

  Here, when the encoding device 22 encodes an original image with, for example, an encoding method compliant with H.264 / AVC, the encoding device 91 performs encoding according to H.264 / AVC. A conventional encoding device that performs encoding by a method can be used.

  Also, the encoding device 91 can have the same configuration as the encoding device 22. However, when the encoding device 91 has the same configuration as that of the encoding device 22, it is necessary to prevent the quantization step from being controlled in the encoding device 91 depending on whether or not the original image is dithered. .

  The decoding device 92 decodes the encoded bit stream from the encoding device 91 and supplies the decoded image obtained as a result to the display device 93 for display.

  The operator of the authoring system looks at the decoded image displayed on the display device 93 and, as described with reference to FIGS. 3 to 5, in the decoded image, all the pixel values of the block having the value v and the value v + 1 The operation device 94 is operated so as to designate a portion where blocks appear alternately, that is, a portion which is unnatural due to dithering being applied to the original image.

  The operation device 94 supplies an operation signal corresponding to the operation of the operator to the dither information acquisition unit 41 (FIG. 7) of the encoding device 22. In the encoding device 22, the dither information acquisition unit 41 acquires dither information by recognizing whether or not the original image is dithered in accordance with an operation signal from the operation device 94, and dither quantization. This is supplied to the step controller 42.

  Thereafter, the encoding device 22 and the recording control device 23 perform the same processing as in the case of FIG. 6, whereby an encoded bit stream obtained by encoding the original image is recorded on the recording medium 24.

  Next, a series of processing performed by the image encoding unit 31 and the quantization step control unit 32 of the encoding device 22 (FIG. 7) can be performed by dedicated hardware or can be performed by software. When a series of processing is performed by software, a program constituting the software is installed in a general-purpose computer or the like.

  Accordingly, FIG. 15 shows a configuration example of an embodiment of a computer in which a program for executing the series of processes described above is installed.

  The program can be recorded in advance on a hard disk 105 or a ROM 103 as a recording medium built in the computer.

  Alternatively, the program is stored temporarily on a removable recording medium 111 such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), a MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, or a semiconductor memory. It can be stored permanently (recorded). Such a removable recording medium 111 can be provided as so-called package software.

  The program is installed in the computer from the removable recording medium 111 as described above, or transferred from the download site to the computer wirelessly via a digital satellite broadcasting artificial satellite, LAN (Local Area Network), The program can be transferred to a computer via a network such as the Internet, and the computer can receive the program transferred in this way by the communication unit 108 and install it in the built-in hard disk 105.

  The computer includes a CPU (Central Processing Unit) 102. An input / output interface 110 is connected to the CPU 102 via the bus 101, and the CPU 102 operates an input unit 107 including a keyboard, a mouse, a microphone, and the like by the user via the input / output interface 110. When a command is input as a result, the program stored in a ROM (Read Only Memory) 103 is executed accordingly. Alternatively, the CPU 102 also transfers from a program stored in the hard disk 105, a program transferred from a satellite or a network, received by the communication unit 108 and installed in the hard disk 105, or a removable recording medium 111 attached to the drive 109. The program read and installed in the hard disk 105 is loaded into a RAM (Random Access Memory) 104 and executed. Thus, the CPU 102 performs processing according to the above-described flowchart or processing performed by the configuration of the above-described block diagram. Then, the CPU 102 outputs the processing result from the output unit 106 configured with an LCD (Liquid Crystal Display), a speaker, or the like, for example, via the input / output interface 110, or from the communication unit 108 as necessary. Transmission and further recording on the hard disk 105 are performed.

  Here, in this specification, the processing steps for describing a program for causing a computer to perform various types of processing do not necessarily have to be processed in time series according to the order described in the flowchart, but in parallel or individually. This includes processing to be executed (for example, parallel processing or processing by an object).

  Further, the program may be processed by a single computer, or may be processed in a distributed manner by a plurality of computers. Furthermore, the program may be transferred to a remote computer and executed.

  The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

  That is, in the above-described embodiment, dither information indicating whether dither has been applied to the original image is acquired, and the quantization step when quantizing the frequency component is controlled based on the dither information. However, for example, the editing information indicating whether the original image has undergone editing other than dithering such as processing related to CG (Computer Graphics) or processing related to image composition is obtained, and based on the editing information. Thus, the quantization step when the frequency component is quantized can be controlled.

It is a block diagram which shows the structure of an example of the conventional encoding apparatus. It is a figure explaining a dither. It is a figure which shows the data obtained in the process of encoding and decoding the dithered original image. It is a figure which shows the data obtained in the process of encoding and decoding the dithered original image. It is a figure which shows the original image to which the dither was given, and the decoded image obtained by encoding and decoding the original image. It is a block diagram which shows the structural example of one Embodiment of the authoring system to which this invention is applied. 3 is a block diagram illustrating a configuration example of an encoding device 22. FIG. It is a figure which shows the base of DCT. It is a flowchart explaining the quantization step control process for dithering. It is a flowchart explaining an encoding process. It is a figure which shows the data obtained in the process of encoding and decoding the dithered original image. It is a block diagram which shows the structural example of the encoding apparatus 22 which performs the encoding based on H.264 / AVC. It is a figure which shows the scaling value set based on the quantization parameter Qp. It is a block diagram which shows the structural example of other embodiment of the authoring system to which this invention is applied. It is a block diagram which shows the structural example of one Embodiment of the computer to which this invention is applied.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Prediction part, 2 conversion part, 5 rate control part, 21 authoring apparatus, 22 encoding apparatus, 23 recording control apparatus, 24 recording medium, 31 image encoding part, 32 quantization step control part, 41 dither information acquisition part, 42 dither quantization step control unit, 43 quantization unit, 44 generation unit, 61 rearrangement unit, 62 operation unit, 63 orthogonal transform unit, 64 quantization unit, 65 inverse quantization unit, 66 inverse orthogonal transform unit, 67 Arithmetic unit, 68 filter, 69 frame memory, 70 intra prediction unit, 71 inter prediction unit, 72 mode determination unit, 73 entropy coding unit, 74 buffer, 75 rate quantization step control unit, 91 coding device, 92 decoding Device, 93 display device, 94 operation device, 101 bus, 102 CPU, 103 ROM, 104 RAM, 105 hard disk, 106 output unit, 107 input unit, 108 communication unit, 109 drive, 110 input / output interface, 111 removable recording medium

Claims (8)

A prediction means for calculating a prediction error between an image and a predicted image obtained by predicting the image;
Conversion means for converting the prediction error into a frequency component;
In a control device for controlling a quantization step when the image encoding device including a quantization means for quantizing the frequency component quantizes the frequency component,
Editing information acquisition means for acquiring editing information indicating whether the image has been edited;
A control apparatus comprising: an editing quantization step control unit that controls a quantization step when the frequency component is quantized based on the editing information. The control device according to claim 1, wherein the editing information acquisition unit acquires the editing information by determining whether the image is edited based on the frequency component. The image encoding device further includes rate control means for controlling the quantization step based on a rate of an encoded bitstream including a quantization coefficient obtained by quantizing the frequency component,
The editing quantization step control means controls the quantization step based on quantization step control information for the rate control means to control the quantization step when the image is edited. The control device according to claim 1. The converting means converts a prediction error of a predetermined block of the image into a frequency component,
The rate control means controls the quantization step for each predetermined block;
The control apparatus according to claim 3, wherein the editing quantization step control means controls the quantization step for each frequency component of the predetermined block. The control apparatus according to claim 1, wherein the editing information is information indicating whether a process related to dithering, CG (Computer Graphics), or a process related to image composition is performed. A prediction means for calculating a prediction error between an image and a predicted image obtained by predicting the image;
Conversion means for converting the prediction error into a frequency component;
In a control method for controlling a quantization step when the image encoding device including a quantization means for quantizing the frequency component quantizes the frequency component,
Obtain editing information indicating whether the image has been edited;
A control method including a step of controlling a quantization step when the frequency component is quantized based on the editing information. A prediction means for calculating a prediction error between an image and a predicted image obtained by predicting the image;
Conversion means for converting the prediction error into a frequency component;
In a program for causing a computer to execute a control process for controlling a quantization step when the image encoding device including a quantization unit that quantizes the frequency component quantizes the frequency component,
Obtain editing information indicating whether the image has been edited;
A program for causing a computer to execute a control process including a step of controlling a quantization step when quantizing the frequency component based on the editing information. In an encoding device for encoding an image,
Prediction means for obtaining a prediction error between the image and a prediction image obtained by predicting the image;
Conversion means for converting the prediction error into a frequency component;
Quantization means for quantizing the frequency component;
Editing information acquisition means for acquiring editing information indicating whether the image has been edited;
An encoding apparatus comprising: an editing quantization step control unit that controls a quantization step when the frequency component is quantized based on the editing information.

Images