JP2008219205A - Picture information encoder and picture information encoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To minimize quantization degradation of mosquito noise or the like while faithfully reproducing input picture information while reducing a calculation amount. <P>SOLUTION: By an orthogonal transformation size judgement part 13, an orthogonal transformation size is judged utilizing the feature amounts of the input picture information such as macroblock activity, macroblock dispersion and the number of edges included in a macroblock, and the orthogonal transformation size in an orthogonal transformation part 15 is determined beforehand. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image information encoding apparatus and image information encoding method for outputting compressed image information obtained by orthogonally transforming moving image information in units of variable orthogonal transform sizes, and in particular, MPEG, H.26x, etc. As described above, when receiving image information (bitstream) compressed by orthogonal transformation such as discrete cosine transformation or Karhunen-Labe transformation and motion compensation via network media such as satellite broadcasting, cable TV, the Internet, and cellular phones. The present invention also relates to an image information encoding apparatus and an image information encoding method used when processing on a storage medium such as an optical, magnetic disk, or flash memory.

  In recent years, image information has been handled as digital data. At that time, MPEG is used for the purpose of efficient transmission and storage of information, and compression is performed by orthogonal transform such as discrete cosine transform and motion compensation using redundancy unique to image information. An apparatus conforming to the above-described method is becoming widespread in both information distribution of broadcasting stations and information reception in general households.

  In particular, MPEG2 (ISO / IEC 13818-2) is defined as a general-purpose image coding system, and includes both interlaced scanning images and progressive scanning images, standard definition (SD) images, and high definition (HD: HD). High Definition) A standard that covers images and is currently widely used in a wide range of professional and consumer applications. By using the MPEG2 compression method, for example, a standard resolution interlaced scanning image having 720 × 480 pixels is 4 to 8 Mbps, and a high resolution interlaced scanning image having 1920 × 1088 pixels is 18 to 22 Mbps. (Bit rate) can be assigned to achieve a high compression rate and good image quality.

  MPEG2 was mainly intended for high-quality encoding suitable for broadcasting, but did not support encoding methods with a lower code amount (bit rate) than MPEG1, that is, a higher compression rate. With the widespread use of mobile terminals, the need for such an encoding system is expected to increase in the future, and the MPEG4 encoding system has been standardized accordingly. Regarding the image coding system, the standard was approved as an international standard as ISO / IEC 14496-2 in December 1998.

  Furthermore, in recent years, the standardization of a standard called H.26L (ITU-T Q6 / 16 VCEG) has been advanced for the purpose of image coding for an initial video conference. H. 26L is known to achieve higher encoding efficiency, although a larger amount of computation is required for encoding and decoding than conventional encoding methods such as MPEG2 and MPEG4. In addition, as part of MPEG4 activities, this H.264 Based on H.26L Standardization that incorporates functions not supported by H.26L and achieves higher coding efficiency is performed as Joint Model of Enhanced-Compression Video Coding. As a standardization schedule, in March 2003, H.264 H.264 and MPEG-4 Part 10 (Advanced Video Coding) have become international standards.

  Here, a schematic configuration of the image information encoding apparatus 100 that outputs image compression information based on the AVC standard is shown in a block diagram of FIG.

  The image information encoding apparatus 100 includes an A / D conversion unit 101, a screen rearrangement buffer 102, an adder 103, an orthogonal transformation unit 104, a quantization unit 105, a lossless encoding unit 106, an accumulation buffer 107, and an inverse quantization unit. 108, an inverse orthogonal transform unit 109, a deblocking filter 110, a frame memory 111, an intra prediction unit 112, a motion prediction / compensation unit 113, a rate control unit 114, and the like.

  In the image information encoding apparatus 100 shown in FIG. 2, the A / D conversion unit 101 converts an input image signal into a digital signal and supplies it to the screen rearrangement buffer 102. Then, the screen rearrangement buffer 102 rearranges the frames in accordance with the GOP (Group of Pictures) structure of the compressed image information output from the image information encoding device 100.

  Here, with respect to image information that is intra-encoded, that is, encoded using a single frame, the difference between the input image information and the pixel value generated by the intra-prediction unit 112 is sent to the orthogonal transform unit 104. The orthogonal transform unit 104 performs orthogonal transform such as discrete cosine transform and Karhunen-Loeve transform. The orthogonal transform unit 104 supplies transform coefficients obtained by the orthogonal transform to the quantization unit 105.

  The quantization unit 105 performs a quantization process on the transform coefficient supplied from the orthogonal transform unit 104 and supplies the quantized transform coefficient to the lossless encoding unit 106.

  The lossless encoding unit 106 performs lossless encoding such as variable length encoding and arithmetic encoding on the quantized transform coefficient supplied from the quantization unit 105. The transform coefficient losslessly encoded by the lossless encoding unit 106 is stored in the storage buffer 107 and output as image compression information.

  The behavior of the quantization unit 105 is controlled by the rate control unit 114. Further, the quantization unit 105 supplies the quantized transform coefficient to the inverse quantization unit 108. Further, the inverse orthogonal transform unit 109 performs inverse orthogonal transform processing to obtain decoded image information. After the block distortion is removed by the deblocking filter 110, the information is stored in the frame memory 111. In the intra prediction unit 112, information on the intra prediction mode applied to the block / macro block is transmitted to the lossless encoding unit 106 and encoded as part of header information in the image compression information.

  On the other hand, with respect to image information that is inter-encoded, that is, encoded using a plurality of frames, the image information from the screen rearrangement buffer 102 is input to the motion prediction / compensation unit 113. The motion prediction / compensation unit 113 reads image information that is simultaneously referenced from the frame memory 111, performs motion prediction / compensation processing, generates reference image information, and supplies the reference image information to the adder 103. The adder 103 converts the image information from the screen rearrangement buffer 102 into a difference signal from the reference image information. The motion prediction / compensation unit 113 supplies motion vector information to the lossless encoding unit 106 at the same time. The lossless encoding unit 106 performs lossless encoding processing such as variable length encoding and arithmetic encoding on the motion vector information, and forms information to be inserted into the header portion of the image compression information. Other processes are the same as the image compression information subjected to intra coding.

  Next, a block diagram of FIG. 3 shows a schematic configuration of an image information decoding apparatus 200 that realizes image compression by orthogonal transformation such as discrete cosine transformation or Karhunen-Labe transformation and motion compensation.

  The image information decoding apparatus 200 includes a storage buffer 201, a lossless decoding unit 202, an inverse quantization unit 203, an inverse orthogonal transform unit 204, an adder 205, a screen rearrangement buffer 206, a D / A conversion unit 207, a frame memory 208, It includes a motion compensation / compensation unit 209, an intra prediction unit 210, a deblock filter 211, and the like.

  In the image information decoding device 200 shown in FIG. 3, the accumulation buffer 201 temporarily stores the input image compression information and transfers it to the lossless decoding unit 202. The lossless decoding unit 202 performs processing such as variable length decoding and arithmetic decoding on the compressed image information transferred from the accumulation buffer 201 based on the determined format of the compressed image information. Further, when the frame is intra-coded, the lossless decoding unit 202 also decodes the intra prediction mode information stored in the header portion of the image compression information, and supplies the information to the intra prediction unit 210. . Further, when the frame is inter-coded, the lossless decoding unit 202 also decodes the motion vector information stored in the header portion of the image compression information and supplies the information to the motion prediction / compensation unit 209. To do.

  The inverse quantization unit 203 performs inverse quantization on the quantized transform coefficient supplied from the lossless decoding unit 202 and supplies the quantized transform coefficient to the inverse orthogonal transform unit 204 as a transform coefficient. The inverse orthogonal transform unit 204 performs a fourth-order inverse orthogonal transform on the transform coefficient supplied from the inverse quantization unit 203 based on a determined method.

  Here, when the frame is intra-coded, the image information subjected to the inverse orthogonal transform process is supplied to the adder 205 and combined with the predicted image information generated by the intra prediction unit 210. Further, after the block distortion is removed by the deblocking filter 211, it is stored in the screen rearrangement buffer 206, and is output after D / A conversion processing by the D / A conversion unit 207.

  On the other hand, if the frame is inter-coded, the motion prediction / compensation unit 209 stores the motion vector information subjected to the lossless decoding process by the lossless decoding unit 202 and the frame memory 208. Reference image information is generated based on the image information and supplied to the adder 205. The adder 205 synthesizes this reference image information and the output of the inverse orthogonal transform unit 204. Other processing is the same as that of the intra-coded frame.

JP 2005-341184 A JP-A-2005-39743 JP 2002-185914 A

  By the way, in AVC, only 4 × 4 DCT existed in the first place. This is because the propagation of mosquito noise can be suppressed by reducing the DCT size. However, although the effect is high in a standard definition (SD) image or an image size smaller than that, a texture such as grain noise cannot be faithfully reproduced in a 4 × 4 DCT in a high definition (HD) image. It was later found that x8DCT can be reproduced more faithfully.

  Therefore, in 2004, High Profile was added as a Fidelity Range Extension in addition to the conventional profile for the purpose of reproducing the original image more faithfully. A major difference from the conventional Main Profile is that a quantization matrix called a scaling list matrix that can switch the DCT size between 8 × 8 and 4 × 4 in units of macroblocks can be used.

  With the introduction of 8 × 8 DCT, the reproducibility of texture (pattern) such as Grain noise in a high definition (HD) image is improved. The introduction of Scaling List Matrix has improved image quality suitable for human visual characteristics.

  In order to switch between 8 × 8 DCT and 4 × 4 DCT, it is necessary to set the transform size flag (transform_size — 8 × 8_flag) in PPS (Picture parameter set) to [1]. As a result, the DCT size can be switched for each macroblock.

  The DCT size of the macro block can be switched by transform_size_8 × 8_flag in Macroblock layer syntax. If this flag is [0], it is 4 × 4 DCT, and if it is [1], it is 8 × 8 DCT. In the case of 8 × 8 DCT, the block size of motion compensation must be 8 × 8 or more.

  In this way, the High Profile is added and the DCT size can be switched. However, in HD, mosquito noise due to the use of 8 × 8 DCT is observed in the edge portion or extremely complex texture, and the subjective image quality The phenomenon that is bad occurs. There is a problem that the DCT size must be switched in order to minimize the quantization deterioration such as mosquito noise while reproducing the input image information more faithfully. In addition, since motion prediction and intra prediction considering both 8 × 8 DCT and 4 × 4 DCT must be performed, there is a problem that the amount of calculation increases.

Therefore, in view of the conventional problems as described above, an object of the present invention is to minimize quantization degradation such as mosquito noise while reducing the amount of calculation and reproducing the input image information more faithfully. Another object of the present invention and specific advantages obtained by the present invention are further apparent from the description of embodiments described below. To be.

  In the present invention, in order to solve the above-described problem, the orthogonal transform size determination means determines the orthogonal transform size using the feature amount of the input image information, and determines the orthogonal transform size in advance, thereby determining the DCT size. Introduce a judgment unit. The feature amount may be anything that can represent the edge portion of the image or a complex texture in the input macroblock, such as macroblock activity, macroblock variance, and the number of edges included in the macroblock.

  That is, the present invention is an image information encoding device that outputs image compression information obtained by encoding and transforming moving image information in units of variable orthogonal transform sizes. The input moving image information includes a predetermined number of pixels. An orthogonal transform size determining unit that determines the orthogonal transform size based on a unit image feature amount is provided, and the orthogonal transform is performed by switching the orthogonal transform size according to a determination result by the orthogonal transform size determining unit And

  The present invention is also an image information encoding method for outputting compressed image information obtained by encoding moving image information by performing orthogonal transform in variable orthogonal transform size units, and for input moving image information, a predetermined number of pixels is provided. The orthogonal transform size is determined based on the unit image feature amount, and the orthogonal transform is performed by switching the orthogonal transform size according to the determination result.

  In the present invention, the orthogonal transform size determination means determines the orthogonal transform size using the feature amount of the input image information such as the macroblock activity, the macroblock variance, the number of edges included in the macroblock, and the orthogonal transform in advance. By determining the size, it is possible to minimize the deterioration of quantization such as mosquito noise while reproducing the input image information more faithfully while reducing the amount of calculation.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Needless to say, the present invention is not limited to the following examples, and can be arbitrarily changed without departing from the gist of the present invention.

  The present invention is applied to, for example, an image information encoding apparatus 10 configured as shown in FIG.

  The image information encoding device 10 includes an A / D conversion unit 11, a screen rearrangement buffer 12, a DCT size determination unit 13, an adder 14, an orthogonal transformation unit 15, a quantization unit 16, a lossless encoding unit 17, and a storage buffer. 18, an inverse quantization unit 19, an inverse orthogonal transform unit 20, a deblock filter 21, a frame memory 22, an intra prediction unit 23, a motion prediction / compensation unit 24, a rate control unit 25, and the like.

  In the image information encoding apparatus 10 shown in FIG. 1, the A / D conversion unit 11 converts the input moving image signal into a digital signal and supplies the digital signal to the screen rearrangement buffer 12.

  The screen rearrangement buffer 12 is configured to convert the input image information converted into a digital signal by the A / D conversion unit 11 according to a GOP (Group of Pictures) structure of image compression information output from the image information encoding device 10. The rearranged input image information is supplied to the adder 14, the orthogonal transform unit 15, the intra prediction unit 23, and the motion prediction / compensation unit 24 through the DCT size determination unit 13.

  The DCT size determination unit 13 determines the DCT size of the input image information supplied from the screen rearrangement buffer 12 using, for example, feature quantities such as macroblock activity, macroblock variance, and the number of edges included in the macroblock. Then, the conversion size flag (transform_size_8 × 8_flag) indicating the DCT size is changed according to the determination result.

  The adder 14 generates a difference value between the input image information supplied via the DCT size determination unit 13 and the intra or inter predicted image information for each macroblock.

  Here, with respect to image information that is intra-encoded, that is, encoded using a single frame, the difference value between the input image information and the intra-predicted image information generated by the intra-prediction unit 23 is an orthogonal transform unit. 15 is input. Also, for image information that is inter-encoded, that is, encoded using a plurality of frames, the difference value between the input image information and the reference image information generated by the motion compensation / prediction unit 24 is the orthogonal transform unit 15. Is input.

  The orthogonal transform unit 15 performs orthogonal transform such as discrete cosine transform and Karhunen-Loeve transform on the difference value for each macroblock supplied from the adder 14 in a variable transform size unit, here discrete cosine transform (DCT). The obtained orthogonal transform (DCT) coefficient is supplied to the quantization unit 16. The variable transform size in the orthogonal transform unit 15 is switched to the transform size indicated by the transform size flag (transform_size_8 × 8_flag) given by the DCT size determination unit 13. Here, the transform size flag (transform_size_8 × 8_flag) indicates that the orthogonal transform size is 8 × 8 in the case of [1], and the orthogonal transform size is 4 × 4 in the case of [0]. It shows that.

  The quantization unit 16 performs a quantization process on the transform coefficient supplied from the orthogonal transform unit 15 and supplies the quantized transform coefficient to the lossless encoding unit 17 and the inverse quantization unit 19.

  The behavior of the quantization unit 16 is controlled by the rate control unit 25.

  The lossless encoding unit 17 performs lossless encoding such as variable length encoding and arithmetic encoding on the quantized transform coefficient supplied from the quantization unit 16, for example, CABAC (Context-Adaptive Binary Arithmetic Coding). Encode. The transform coefficient losslessly encoded by the lossless encoding unit 17 is accumulated in the accumulation buffer 18 and output as image compression information.

  The inverse quantization unit 19 performs an inverse quantization process on the quantized orthogonal transform coefficient supplied from the quantization unit 16 and supplies the orthogonal transform coefficient to the inverse orthogonal transform unit 20.

  The inverse orthogonal transform unit 20 performs an inverse orthogonal transform process on the orthogonal transform coefficient supplied from the inverse quantization unit 19 and supplies the obtained decoded image information to the frame memory 22 via the deblock filter 21.

  The deblocking filter 21 removes block distortion included in the decoded image information.

  The frame memory 22 stores decoded image information.

  The intra prediction unit 23 extracts adjacent, already encoded image information from the frame memory 22 and performs only intra prediction processing suitable for the DCT size based on the extracted image information. Here, when the transform size flag (transform_size_8 × 8_flag) given by the DCT size determination unit 13 is [1], the intra prediction unit 23 performs intra prediction only for Intra8 × 8.

  The motion prediction / compensation unit 24 searches the reference image information for the motion vector and generates the inter prediction image information for only the necessary block size according to the determination result by the DCT size determination unit 13, and the DCT size is set in advance. If it is not known, the block size is fed back to the DCT size determination unit 13. Here, when the transform size flag (transform_size_8 × 8_flag) given by the DCT size determination unit 13 is [1], the motion prediction / compensation unit 24 performs motion prediction only on the 8 × 8 size, and When the transform size flag (transform_size_8 × 8_flag) given by the DCT size determination unit 13 is [0], intra 4 × 4 or intra 16 × 16 intra prediction is performed.

  The rate control unit 25 controls the operation of the quantization unit 16 by feedback control, and controls the code amount of image compression information to be output.

  Next, the operation principle of the DCT size determination unit 13 that is a feature of the present invention will be described.

  The DCT size determination unit 13 determines the DCT size from the feature quantity in units of macroblocks. The feature amount is, for example, macroblock activity, macroblock variance, the number of edges included in the macroblock, and the like.

  The macroblock activity MBACT is calculated as follows.

Here, P _k is a pixel value in the luminance signal block of the original image. Here, sblk represents four 8 × 8 pixel blocks included in the macroblock. The DCT size is determined using a certain threshold Th _act as follows, for example.

  The macroblock distribution MBVAR is calculated as follows.

Here, P _k is a pixel value in the luminance signal block of the original image. The determination of the DCT size is performed as follows using, for example, the threshold value Th _var .

  The macroblock edge MBEDGE is calculated as follows. Any method may be used for edge detection, but here a sobel operation is shown.

When g _hsij and g _hvij are obtained by multiplying the above-mentioned coefficients by i rows and j columns of the input image information and obtaining the sum as g _hsij and g _hvij , the pixel value g _ij after the sobel operation of the target pixel is expressed by the following equation.

Here, _{MBEDGE in which} the number of pixel values greater than or equal to the threshold value Th _sobel is tabulated for each macroblock is expressed as follows.

Here, E _k takes the value [1] when the pixel value after the intra-block sobel operation is _{equal to} or greater than the threshold Th _sobel [0]. The DCT size is determined using, for example, a certain threshold Th _edge as follows.

  The edge detection method used for this process is not limited to the sobel operation, and various edge detection methods can be used. Further, not only the number of edges in the macroblock but also the number of vertical and horizontal edges may be evaluated separately.

  Next, the operation principle in the intra prediction unit 23 will be described.

  When the value of the transform size flag (transform_size_8 × 8_flag) given by the DCT size determination unit 13 is [0], since the DCT size is 4 × 4, the intra 8 × 8 prediction with the DCT size of 8 × 8 is performed. In the case of [1], since the DCT size is 8 × 8, prediction of Intra4 × 4 and Inra16 × 16 with a DCT size of 4 × 4 is not performed. Thereby, the amount of calculation can be reduced.

  Further, the operation principle in the motion prediction / compensation unit 24 will be described.

  When the value of the transform size flag (transform_size_8 × 8_flag) given by the DCT size determination unit 13 is [1], the DCT size is 8 × 8, and the motion compensation block size must be 8 × 8 or more. . Therefore, since it is not necessary to perform motion prediction of less than 8 × 8, it is possible to reduce calculation.

  However, regarding the case where motion compensation is performed prior to the DCT size determination in the DCT size determination unit 13 by pipeline processing due to the hardware configuration, if the selected motion compensation block size is less than 8 × 8, Since the transform size flag (transform_size_8 × 8_flag) needs to be [0], the information needs to be fed back to the DCT size determination unit 13. This makes it possible to perform DCT size determination that is syntactically consistent.

It is a block diagram which shows the structural example of the image information encoding apparatus which concerns on this invention. It is a block diagram which shows schematic structure of the image information encoding apparatus which outputs the image compression information based on an AVC encoding system. It is a block diagram which shows schematic structure of the image information decoding apparatus which inputs the image compression information based on an AVC encoding system.

Explanation of symbols

11 A / D conversion unit, 12 screen rearrangement buffer, 13 DCT size determination unit, 14 adder, 15 orthogonal transform unit, 16 quantization unit, 17 lossless encoding unit, 18 accumulation buffer, 19 inverse quantization unit, 20 Inverse orthogonal transform unit, 21 deblock filter, 22 frame memory, 23 intra prediction unit, 24 motion prediction / compensation unit, 25 rate control unit,

Claims (10)

An image information encoding device that outputs compressed image information obtained by encoding and transforming moving image information in units of variable orthogonal transform sizes,
The input moving image information includes an orthogonal transform size determination unit that determines the orthogonal transform size based on an image feature amount in a predetermined number of pixels.
An image information encoding apparatus, wherein orthogonal transform is performed by switching the orthogonal transform size according to a determination result by the orthogonal transform size determining means. An image information encoding device that outputs image compression information based on MPEG-4, AVC encoding method, etc.
The orthogonal transform size determination means determines the orthogonal transform size of the input moving image information based on the image feature amount in units of macroblocks, and sets a transform size flag (transform_size_8 × 8_flag) according to the determination result. 2. The image information encoding apparatus according to claim 1, wherein the orthogonal transform size is switched to 4 × 4 or 8 × 8 by changing.   3. The image information encoding apparatus according to claim 2, wherein the orthogonal transform size determination unit switches the orthogonal transform using a macroblock activity MBACT as the image feature amount.   3. The image information encoding apparatus according to claim 2, wherein the orthogonal transform size determination unit switches the orthogonal transform using a macroblock distributed MBVAR as the image feature amount.   3. The image information encoding apparatus according to claim 2, wherein the orthogonal transform size determination unit switches the orthogonal transform using an edge MBEDGE included in a macroblock as an image feature amount. Predicting / compensating means for performing motion prediction / compensation processing on the moving image information to be orthogonally transformed,
By determining the orthogonal transform size based on the determination result by the orthogonal transform size determination unit before performing the motion prediction by the prediction / compensation unit, the prediction / compensation unit has the orthogonal transform size of 8 × 8. 3. The image information encoding apparatus according to claim 2, wherein when the transform size flag (transform_size — 8 × 8_flag) indicating that it is “1”, motion prediction only for the 8 × 8 size is performed. 4. Predicting / compensating means for performing motion prediction / compensation processing on the moving image information to be orthogonally transformed,
When the orthogonal transform size is determined after motion prediction is performed by the prediction / compensation unit, the orthogonal transform size determination unit determines that the transform size flag (transform_size_8 × 8_flag) is [1] and is determined by motion prediction. 3. The image information encoding apparatus according to claim 2, wherein the transform size flag (transform_size_8 × 8_flag) is changed to [0] when the size of the motion compensation performed is less than the 8 × 8 block size. Intra prediction means for performing motion intra prediction processing on the moving image information to be orthogonally transformed,
By determining the DCT size based on the determination result by the orthogonal transform size determination means before performing the intra prediction by the intra prediction means, the intra prediction means has the transform size flag (transform_size_8 × 8_flag) set to [ 3. The image information encoding apparatus according to claim 2, wherein intra prediction of only Intra 8 × 8 is performed in the case of 1]. Intra prediction means for performing motion intra prediction processing on the moving image information to be orthogonally transformed,
By determining the DCT size based on the determination result by the orthogonal transform size determination means before performing the intra prediction by the intra prediction means, the intra prediction means has the transform size flag (transform_size_8 × 8_flag) set to [ 0], the intra prediction of Intra4 × 4 or Intra16 × 16 is performed. An image information encoding method for outputting compressed image information obtained by orthogonally transforming and encoding moving image information in variable orthogonal transform size units,
For the input video image information, the orthogonal transformation size is determined based on the image feature amount in a predetermined number of pixels,
An image information encoding method, wherein orthogonal transform is performed by switching the orthogonal transform size according to the determination result.

Images