JP4250553B2 - Image data processing method and apparatus - Google Patents

Description

  The present invention relates to an image data processing method and apparatus, and more particularly, to an encoding / decoding method and apparatus for compressing a moving image with a reduced resolution and performing resolution conversion at the time of decoding.

  When moving images are distributed in a compressed stream as in BS digital broadcasting, a request to recompress the received compressed moving image with a different compression rate and store it in a smaller amount of data than the original stream is It always exists because the capacity of the storage medium is finite. In order to meet this demand, many compression methods for changing the compression rate and recompressing have been proposed. In particular, a method of storing an HD (High Definition) video stream by changing the resolution to an SD (Standard Definition) level is common (Patent Document 1, Patent Document 2, etc.). When viewing a moving image stored using these methods, the SD level moving image is converted in resolution and displayed. Most of the resolution conversion methods in this case use general resolution conversion. For example, in Patent Document 1, an inverse discrete cosine transform method, nearest neighbor interpolation method, bilinear interpolation method, cubic convolution interpolation method, and Hadamard transform, which are general resolution conversion methods, are cited as enlargement methods.

However, when these general enlargement methods are used, information lost due to reduction during compression cannot be used for enlargement during decoding, and enlargement is interpolated only from the remaining information, and information is lost. It is clear that That is, the image quality deteriorates.
JP-A-9-331529 Japanese Patent Laid-Open No. 2001-16589

  The present invention has been made to solve the above-mentioned problem, and when recompressing a compressed image by reducing the resolution, the resolution conversion processing at the time of decoding is made into a high-quality image close to the original signal. With the goal.

To achieve the above object, the present invention, a) remove a predetermined frequency component from the input orthogonal transform image data, obtaining a reduced Cartesian converted image data, wherein the reduced orthogonal transformation image the interpolation pattern used to restore the high frequency components in decoding the data, a selecting step of selecting from among a plurality of interpolation patterns prepared in advance, a parameter indicating the selected interpolation pattern in the selection step, in addition to the orthogonal transform image data the reduced, a step of creating a re-compressed image data composed of the a reduced orthogonal transformation image data and the parameter, the re-compression step comprising, b) the re-compressed image from the data, detecting the parameter, an acquisition step of acquiring a high-frequency component by using the interpolation patterns corresponding to the detected parameters, the obtained high frequency component Decoding including a high frequency component adding step of adding the orthogonal transform image data reduced contained in the re-compressed image data, a step of inverse orthogonal transformation on the orthogonal transformation image data to which the high-frequency component is reduced is added, the And an image data processing method.

  Here, the orthogonal transformation image data refers to image data subjected to orthogonal transformation. However, image data that has been subjected to orthogonal transformation and then subjected to quantization and encoding, and image data that has been subjected to decoding and inverse quantization are also included.

The present invention is also an image data processing apparatus including an image recompression unit and an image decoding unit, wherein the image recompression unit deletes a predetermined high-frequency component from the input orthogonal transform image data and reduces the image data. and an image compressing portion for obtaining a Cartesian conversion image data, the interpolation pattern used to restore the high frequency components in decoding the orthogonal transformation image data the reduced, from the plurality of interpolation patterns prepared in advance a selection selecting section you select, a parameter indicating a correction pattern selected by the selection unit, in addition to the orthogonal transform image data the reduced, comprising the the reduced orthogonal transformation image data and the parameter re anda creating unit for creating compressed image data, the image decoding unit, from said re-compressed image data, using an interpolation pattern for detecting the parameter corresponding to the detected parameters A resulting portion preparative you get a high frequency component, the obtained high frequency component, the a high frequency component adding unit for adding the orthogonal image converted image data reduced contained in the re-compressed image data, the high frequency component is added and inverse orthogonal transform unit said you inverse orthogonal transformation on the reduced orthogonal transformation image data has an image data processing apparatus characterized by having a.

  According to the present invention, when a compressed image is recompressed by reducing the resolution, the resolution conversion process at the time of decoding can be made to be a high-quality image close to the original signal.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing an image data processing apparatus according to an embodiment of the present invention.

  In FIG. 1, 101 is an input unit for inputting an MPEG2Video stream, 102 is an image recompression unit, 103 is a first bitstream analysis unit for analyzing an MPEG2 bitstream, and 104 is a Huffman decoding process for an MPEG2Video macroblock. One Huffman decoding unit, 105 is a first inverse quantization unit that performs inverse quantization of the MPEG2 stream, 106 is a first IDCT unit that performs IDCT on the DCT coefficients after the inverse quantization in accordance with the standard, and 107 is an inverse quantum. The high-frequency region pattern selection unit 108 applies a pattern to be described later in order to the high-frequency region with respect to the low-frequency region of the DCT coefficient that has been converted into a pattern, and the pattern 108 selected by the high-frequency region pattern selection unit 107 in the low-frequency region The second IDCT unit 109 that performs IDCT together with the high frequency region to which is applied is the result of the first IDCT unit 105 and the result of the second IDCT unit 107. A pattern evaluation unit that calculates and evaluates the function, 110 is a quantization unit that quantizes only the low-frequency region of the DCT coefficient, 111 is a Huffman encoding unit that Huffman-encodes the quantized DCT block, and 112 is a high-frequency signal conforming to the MPEG2 standard A bit stream creation unit that creates a bit stream to which an area pattern is added, 113 is a storage device that accumulates the recompressed stream compressed by the bit stream creation unit 112, 114 is an image decoding unit, and 115 is a first unit that analyzes the recompressed stream. A second bit stream analyzing unit 116 is a second Huffman decoding unit that performs a Huffman decoding process of a macroblock of the recompressed stream, 117 is a second dequantizing unit that performs inverse quantization of the recompressed stream, and 118 is an inverse unit High frequency domain pattern is applied from the quantized DCT coefficient and high frequency domain pattern coefficient. A high frequency region interpolating unit for interpolating the frequency region, 119 is a third IDCT unit for performing IDCT according to the standard, 120 is a motion compensating unit for performing motion compensation using a motion vector, and 121 is an output unit.

  In the present embodiment, the MPEG2 Video stream input from the input unit 101 is recompressed by the recompressing unit 102 using the DCT coefficient low frequency region and a parameter for estimating the high frequency region to increase the compression rate. Are stored in the storage device 113. In the present embodiment, the recompression unit 102 constitutes the image data encoding device of the present invention. The storage device 113 corresponds to the encoded data storage unit of the present invention. The recompressed stream has a high frequency region estimation parameter added thereto, and becomes a recompressed stream that is out of the MPEG2 standard. The accumulated recompressed stream is decoded by the image decoding unit 114 with the same size and the same image quality as the original image from the low frequency region and high frequency region estimation parameter of the DCT coefficient to be decoded. In the present embodiment, the image decoding unit 114 constitutes an image data decoding device of the present invention.

  First, the operation flow of the recompression unit 102 will be described.

The MPEG2 Video stream input from the input unit 101 is analyzed by the first bit stream analysis unit 103. Details of the decoding process of the MPEG2 Video stream will not be described in this embodiment. Only the operation of macroblocks and blocks that are a group of DCT coefficients will be described.

  Of the analyzed bitstream, the macroblock information is subjected to Huffman decoding processing by the first Huffman decoding unit 104. The macro block of the MPEG2 Video stream subjected to the Huffman decoding is dequantized by the first dequantization unit 105 to obtain four 8 × 8 luminance DCT coefficients and two 8 × 8 color difference DCT coefficients. For simplicity, the description related to the DCT coefficient will be described below using only one luminance DCT coefficient, but other luminance DCT coefficients and color difference DCT coefficients can be handled in exactly the same way.

  The DCT coefficient for which inverse quantization has been completed is sent to the first IDCT unit 106, the high frequency region pattern selection unit 107, and the quantization unit 110.

  The first IDCT unit 106 performs normal IDCT, and an 8 × 8 pixel value or a pixel difference value for motion compensation is obtained.

  The high frequency region pattern selection unit 107 uses the 4 × 4 low frequency region of the 8 × 8 DCT coefficient to interpolate the high frequency region component with reference to the interpolation pattern of the high frequency region described later. With respect to the DCT coefficient obtained by interpolating the high frequency region, normal IDCT is performed by the second IDCT unit 108, and an 8 × 8 pixel value or a pixel difference value for motion compensation in which the high frequency region is interpolated is obtained.

  The pattern evaluation unit 109 compares the correlation between the normal result of the first IDCT unit 106 and the result of the second IDCT unit 108 in which the high frequency region is interpolated, and stores the correlation value. A series of operations from the high frequency region pattern selection unit 107 to the pattern evaluation unit 109 is repeated for the number of high frequency region patterns prepared in advance, and the pattern having the highest correlation value is selected.

  The quantization unit 110 quantizes only 4 × 4 in the low frequency region among the 16 × 16 DCT coefficients that are the result of the first inverse quantization unit 105. The Huffman encoding unit 111 performs Huffman encoding on the result output from the quantization unit 110 in accordance with the MPEG2 standard. Here, the image compression unit of the present invention includes the quantization unit 110 or the quantization unit 110 and the Huffman encoding unit 111.

  The bitstream creation unit 112 performs original recompression by adding a parameter indicating the high-frequency region pattern selected by the pattern evaluation unit 109 to the MPEG2 format bitstream together with the original bitstream information and the DCT coefficient encoded by the Huffman Create a bitstream. The created recompressed bitstream is stored in the storage device 113. In the present embodiment, the bit stream generation unit 112 corresponds to the selection information addition unit of the present invention. Further, the bit stream generation unit 112 and the Huffman encoding unit 111 correspond to the reduced orthogonal transform image data encoding unit of the present invention.

  In this embodiment, a technique for selecting a pattern for interpolating a high frequency region is important. The process of determining the high frequency region pattern will be described in more detail.

  First, as one conventional example of a method for estimating a high frequency region from a low frequency region of a DCT coefficient, there is a method of coefficient folding. In this method, for example, in FIG. 2, when an 8 × 8 DCT coefficient is created by interpolating a high frequency area A1 to A3 from a 4 × 4 low frequency area DCT coefficient A0, A0 is horizontally set for A1. When A2 is folded back in the vertical direction and A3 is folded back horizontally and vertically, a good result image can be obtained.

  Although this method can obtain a good result to some extent, there is also a problem that noise is generated because a value is entered even when there is originally no value in the high frequency region.

  In the present embodiment, (pattern 0) 0 insertion, (pattern 1) copy, (pattern 2) horizontal wrapping, (pattern 3) vertical wrapping, (pattern 4) horizontal Prepare a total of 5 patterns of vertical folding, compare the result of IDCT with the original DCT coefficient and the result of IDCT with the DCT coefficient interpolated in the high frequency region by the method 1-5, and select the best pattern Features.

  A high frequency region interpolation pattern for a 4 × 4 low frequency region is shown in FIG.

  FIG. 4 is a block diagram showing the configuration of the high frequency region pattern determining unit, and FIG. 5 shows an algorithm for determining the high frequency region pattern. In the present embodiment, the high frequency region pattern determination unit, that is, the first IDCT unit 106, the high frequency region pattern selection unit 107, the second IDCT unit 108, and the pattern evaluation unit 109 constitute the high frequency component selection unit of the present invention. The

  First, the high frequency region pattern selection unit 107 extracts 4 × 4, which is a low frequency region, from the original 8 × 8 DCT coefficient (S1). What was extracted in this way is A0 in FIG. Next, a high frequency region pattern is applied to A1 to A3 (S2). The application of the high-frequency region pattern is executed by the number of combinations of the high-frequency region patterns. In the present embodiment, there are 125 combinations of patterns, which are combinations of applying five high frequency region patterns to three of A1 to A3.

  The DCT coefficient obtained by interpolating the high frequency region by the high frequency region pattern selection unit 107 is IDCTed by the second IDCT unit 108 and becomes a luminance value (S3). This process is exactly the same as general IDCT.

  As described above, the first IDCT unit 106 performs normal IDCT (S4). The pattern evaluation unit 109 compares the results of the first IDCT unit 106 and the second IDCT unit 108. The method for calculating the correlation between the two IDCTs is arbitrary, but in this embodiment, the sum of the luminance value differences and the maximum difference value are used as the evaluation values. In this embodiment, 125 patterns are evaluated in order. That is, the difference values are summed up in each pattern (S5), and the maximum difference value is acquired (S6). This is repeated (S7, S8). Once 125 evaluation values have been calculated, select a pattern. As the first evaluation criterion, the one with the smallest difference is selected (S9). For the one selected by the first evaluation criterion, the maximum value of the difference is examined as the second evaluation criterion (S10, S11), and if it is not more than the threshold value, it is selected as the best pattern.

  The high frequency region pattern determined in this way is added to the compressed bit stream by the bit stream creation unit 112. The bit stream creation unit 112 performs recompression according to the MPEG2 Video standard, but adds a 9-bit parameter shown in FIG. 6 before the block indicating the DCT coefficient. The part to which the parameter is added is described in the C language style as shown in FIG. FIG. 7 is a macroblock syntax described in ISO / IEC 13818-2 with the parameters of the present invention added. The contents of the description other than the parameters of the present invention are described in ISO / IEC 13818-2.

  A compressed stream is generated by the method described above.

  Next, the operation flow of the decoding unit 114 will be described.

  The recompressed bit stream read from the storage device 113 is analyzed by the second bit stream analysis unit 115. In the present embodiment, the second bit stream analysis unit 115 corresponds to an encoded data acquisition unit.

  Since the recompressed bitstream is the same as MPEG2 except that a parameter indicating a high frequency region pattern is added, the description of the decoding unit is also omitted except for the decoding flow of the macroblock. The macro block of the recompressed bit stream is input to the second Huffman decoding unit 116 and subjected to Huffman decoding. The Huffman-decoded macroblock is inversely quantized by the inverse quantization unit 117. The DCT coefficient of the inversely quantized macroblock is 4 × 4 in the low frequency region of the original image. Since the parameter indicating the interpolation pattern in the high frequency region is added to the recompressed bitstream, the high frequency region interpolation unit 118 restores the high frequency region according to the interpolation pattern indicated by the parameter. As a result, the 8 × 8 DCT coefficient is restored, and the normal IDCT is performed by the third IDCT unit 119 to obtain an 8 × 8 luminance value or a luminance difference value used for motion compensation. When motion compensation is necessary, the motion compensation unit 120 performs motion compensation, and the output unit 121 outputs a decoded image. In the present embodiment, the high frequency region interpolation unit 118 constitutes a high frequency component acquisition unit and a high frequency component addition unit of the present invention. The third IDCT unit 119 corresponds to the image data inverse orthogonal transform unit of the present invention.

  The decoding unit 114 is a device for decoding the recompressed stream created by the recompressing unit 102. The decoding unit for decoding MPEG2 is basically characterized in that it recognizes a parameter indicating the high frequency region interpolation pattern added by the recompression unit 102 and interpolates the high frequency region to create a DCT coefficient.

  In the present embodiment, five types of patterns for interpolating the high frequency region are used, but the present invention is not limited to this. It is possible to execute with fewer patterns, and more patterns may be used. For example, the effect can be obtained by using only the pattern 0 and the pattern 4 for the purpose of reducing the processing amount. Further, in order to further increase the accuracy of interpolation, it is possible to add a code to each pattern and use a total of 10 patterns.

  In the present embodiment, the method using the sum of the pixel difference values and the maximum difference value for evaluating the result of the first IDCT unit 106 and the result of the second IDCT unit 108 has been described. Not limited to this. Any method can be used as long as the result of the first IDCT unit 106 and the result of the second IDCT unit 108 can be correlated.

1 is a block diagram showing an embodiment of a resolution conversion apparatus according to the present invention. Explanatory drawing explaining the conventional high frequency domain DCT coefficient interpolation system. Explanatory drawing explaining operation | movement of the high frequency area | region pattern selection part in FIG. Explanatory drawing explaining the operation | movement of the high frequency area | region pattern selection in embodiment of this invention. Explanatory drawing explaining the operation | movement of the high frequency area | region pattern selection in embodiment of this invention. Explanatory drawing explaining operation | movement of the bit stream preparation part in FIG. Explanatory drawing explaining operation | movement of the bit stream preparation part in FIG.

Explanation of symbols

101 Input Unit 102 Image Recompression Unit 103 First Bitstream Analysis Unit 104 First Huffman Decoding Unit 105 First Inverse Quantization Unit 106 First IDCT Unit 107 High Frequency Area Pattern Selection Unit 108 Second IDCT Unit 109 Pattern evaluation unit 110 Quantization unit 111 Huffman encoding unit 112 Bit stream creation unit 113 Storage device 114 Image decoding unit 115 Second bit stream analysis unit 116 Second Huffman decoding unit 117 Second inverse quantization unit 118 High frequency domain interpolation Unit 119 third IDCT unit 120 motion compensation unit 121 output unit

Claims (4)

and a) remove a predetermined frequency component from the input orthogonal transform image data to obtain a reduced Cartesian converted image data process,
A selection step of selecting from among a plurality of interpolation patterns interpolation pattern, prepared in advance to be used to restore the high frequency components in decoding the orthogonal transformation image data the reduced,
The parameter indicating the interpolation pattern selected in said selecting step, a step of adding the orthogonal transform image data the reduced, to create a re-compressed image data composed of the a reduced orthogonal transformation image data and the parameter ,
A recompression process comprising:
b) an acquisition step of detecting the parameter from the recompressed image data and acquiring a high-frequency component using an interpolation pattern corresponding to the detected parameter ;
A high frequency component adding step of adding the orthogonal transform image data reduced include the acquired high-frequency component in the re-compressed image data,
A decoding step including the steps of inverse orthogonal transformation on the orthogonal transformation image data to which the high-frequency component is reduced is added,
An image data processing method comprising:   An inverse quantization process for performing an inverse quantization process on the input orthogonal transformation image data;
  The selection step includes
  The first IDCT process for IDCT processing the DCT coefficient obtained by the inverse quantization process in the inverse quantization process
When,
  A low frequency component is extracted from the DCT coefficients obtained in the inverse quantization process, and each of the plurality of interpolation patterns prepared in advance is applied to the low frequency components of the DCT coefficients, and each interpolation pattern is applied. An interpolation process for interpolating the high-frequency component of the DCT coefficient;
  A second IDCT process for IDCT processing DCT coefficients interpolated with high frequency components in the interpolation process;
  A pattern selection step of evaluating each interpolation pattern using the result obtained in the first IDCT step and the result obtained in the second IDCT step, and selecting one from the plurality of interpolation patterns,
The image data processing method according to claim 1, further comprising: An image data processing device comprising an image recompression unit and an image decoding unit,
The image recompression unit includes:
Remove the predetermined high-frequency component from the input orthogonal transform image data, an image compression unit for obtaining a reduced Cartesian converted image data,
A selection selecting section you selected from the plurality of interpolation patterns interpolation pattern used to restore the high frequency components, prepared in advance when decoding the orthogonal transformation image data the reduced,
The parameter indicating the correction pattern selected by the selecting section, wherein in addition to reduced orthogonal transformation image data, created to create a re-compressed image data composed of the a reduced orthogonal transformation image data and the parameter part When,
Have
The image decoding unit
Wherein the recompressed image data, detects the parameters, and the resulting unit preparative you get a high-frequency component by using the interpolation patterns corresponding to the detected parameters,
A high frequency component adding unit that adds the obtained high-frequency component, the orthogonal image converted image data reduced contained in the re-compressed image data,
And inverse orthogonal transform unit you inverse orthogonal transformation on the orthogonal transformation image data to which the high-frequency component is the reduced is added,
Have
An image data processing apparatus.   An inverse quantization unit that performs an inverse quantization process on the input orthogonal transform image data;
  The selection unit includes:
  A first IDCT unit that performs IDCT processing on a DCT coefficient obtained by inverse quantization processing in the inverse quantization unit;
  A low frequency component is extracted from the DCT coefficients obtained by the inverse quantization unit, and each of the plurality of interpolation patterns prepared in advance is applied to the low frequency component of the DCT component. A second IDCT unit that interpolates the high-frequency component of the DCT coefficient and IDCT-processes the DCT coefficient interpolated with the high-frequency component;
  A pattern evaluation unit that evaluates each interpolation pattern using the result obtained in the first IDCT unit and the result obtained in the second IDCT unit, and selects one from the plurality of interpolation patterns;
The image data processing apparatus according to claim 3, further comprising:

Images