JP2007311837A - Method and apparatus for reducing coded block distortion - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and apparatus for reducing coded block distortion for reducing block-shaped distortion caused in a reproduced image when an image is coded/decoded in processing in the unit of a square block by processing the reproduced image. <P>SOLUTION: The block distortion reducing apparatus includes: matching estimate means (7, 9) using a one-dimentional pixel sequence of pixels adjacent to a target block as a reference pixel sequence and determines an optimum moving and reduction/magnification value providing best matching between a reference pixel sequence and a pixel sequence in the target block adjacent to the reference pixel sequence while applying movement and reduction/magnification to the pixel sequence in a direction of the pixel sequence; pixel sequence control means (8, 10) that determine a pixel shift amount obtained by applying the movement and reduction/magnification to each pixel sequence from which the optimum movement and reduction/magnification value is obtained as to a plurality of pixel sequences adjacent to each other in the target block; and pixel shift means (3, 5) for resampling each pixel in each pixel sequence in the direction of the pixel sequence according to the pixel shift amount of each pixel sequence determined by correction amount control. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  In order to efficiently transmit, store, and display moving image signals such as television images with high image quality, processing is performed in units of square blocks in connection with high-efficiency encoding that encodes moving image signals with a smaller amount of information. Belongs to a coding block distortion reduction process for reducing block-like distortion that occurs in a reproduced image when the image is encoded and decoded by the image processing on the reproduced image.

<Outline of image coding / decoding and block distortion>
As for coding of a moving image signal, motion compensation interframe predictive coding is generally used. This is also the case for international standards represented by MPEG. In this case, motion compensation processing is performed in units of square blocks, and DCT (discrete cosine transform), which is processing in units of square blocks, is performed on the prediction residual. These encodings are basically irreversible encodings and roughen the quantization according to the bit rate. As a result, the quantization error increases and the reproduced image quality deteriorates. However, the deterioration often becomes a rectangular block shape that is a unit of encoding processing. The cause of such block distortion is both when the block distortion caused by motion compensation remains and when each coefficient is not properly reproduced in DCT.

  Still image encoding is not necessarily performed in units of square blocks in new standards such as JPEG2000, but digital still cameras, which are still mainstream still images, are compliant with the JPEG standard using DCT and have an information compression rate. When the height is increased, block distortion occurs as in the case of moving images.

<Outline of conventional coding block distortion reduction>
FIG. 3 shows a conventional configuration for reducing coding block distortion. This process is general, but a specific example is H.264. Some H.264 (MPEG-4AVC) standards are used as deblocking filters.

  The code string coming from the code string input 1 is decoded by the image decoder 21 to become a reproduced image, and is given to a variable horizontal LPF (low-pass filter) 22. In the variable horizontal LPF 22, a filter is applied in the horizontal direction to the block boundary of the reproduced image and can be temporarily stored in the image buffer 4. Subsequently, the reproduced image read in the vertical direction from the image buffer 4 is filtered in the vertical direction with respect to the block boundary by the variable vertical LPF 23. The filtered image is output from the image output 6.

  The filtering of the variable horizontal LPF 22 and the variable vertical LPF 23 is performed only on the boundary portion between the blocks, and the central portion of the block other than the boundary portion is not filtered. The degree of suppression of the high frequency component of the LPF is changed according to the control information provided from the correction amount determiner 24. The filter used here is generally a non-linear filter, and its action is limited when the pixel value changes greatly due to filtering.

  On the other hand, the image decoder 21 is multiplexed with a code string, and outputs information related to an encoding (decoding) process that is also used for decoding. Specifically, such information is given to the correction amount determination unit 24 in the inter-picture prediction (macroblock) mode, the quantization step width, and the like. The correction amount determination unit 24 determines an appropriate degree of filtering based on the encoding (decoding) information, and gives the information to the variable horizontal LPF 22 and the variable vertical LPF 23.

  Such control is performed because filtering for reducing block distortion also acts on the reproduced image itself, so that if the filtering is strong, the details are likely to disappear. Further, if only the block boundary position is filtered, the position is not related to the motion of the image, so that an unnatural image may be obtained. In order to reduce these side effects, it is desirable to control with the encoding (decoding) method, but it is necessary to take out the information from the inside of the decoding process and supply it in synchronization with the post-processing. It's not easy.

  Conventional coding block distortion reduction is smoothing of block boundaries based on LPF, and reduces discontinuity of pixel values (levels) between blocks. Block distortion at a relatively flat portion not including the detail shown as level (shading) distortion in FIG. 4A is reduced by such processing, but the shape shown in FIG. Some are shown as (position) distortion. Such shape (position) distortion occurs when motion compensation of a square block is applied to an image that is deformed by zooming or the like in video coding, or when the prediction residual is not reproduced due to insufficient motion compensation accuracy. It is easy to occur. Still image coding is likely to occur when a diagonal high frequency component is deleted by DCT.

  Such shape (position) distortion is often not reduced in the smoothing of pixel values, which is the conventional block distortion reduction, because the difference in values at the boundary is large. Moreover, even if it acts, it does not reduce distortion because it filters in a direction orthogonal to the boundary line.

  The present invention has been made paying attention to the above points, and at the block boundary portion of the reproduced image, the optimum movement and enlargement / reduction values are determined by matching movement and enlargement / reduction between opposing pixel arrays, and the pixel array By re-sampling each pixel and changing the shape of the block itself, it is possible to reduce even block distortion that is discontinuous in shape, and an encoded block distortion reduction method that provides a high-quality reproduced image, and An object of the present invention is to provide an encoded block distortion reduction apparatus.

  The present invention has been made in view of the above-described problems of the prior art, and the present invention relates to a code of a decoded image in reducing block distortion of an image that is irreversibly encoded and decoded in units of square blocks. In a block corresponding to a block, a pixel column that is a one-dimensional column of pixels adjacent to the processing target block is used as a reference pixel column, and the pixel column in the processing target block adjacent to the processing target block is moved and reduced or enlarged in the pixel column direction. While determining the optimum value by matching, the pixel shift amount for each pixel column is determined based on the positional relationship between the pixel column from which the optimum value was obtained and the pixel column for a plurality of adjacent pixel columns in the processing target block. Encoding block distortion in which pixel shift is performed by resampling each pixel in each pixel column in accordance with the pixel shift amount of the pixel column in the processing target block A reduction method and the encoding block distortion reducing apparatus.

  Further, the present invention provides the coded block distortion reducing method, wherein a block is determined based on a relationship between a difference between adjacent pixel values in the entire block and a difference between pixel values at a block boundary portion in a direction spatially orthogonal to the pixel column. An encoding block distortion reduction method and an encoding block distortion reduction apparatus that obtain a shape distortion amount and perform the pixel shift only when the distortion amount is larger than a predetermined value.

<First Embodiment of Encoding Block Distortion Reduction>
The coding block distortion reduction according to the first embodiment of the present invention will be described. FIG. 1 shows the configuration thereof, and the components thereof are greatly different from the conventional example of FIG. Compared with FIG. 3, FIG. 1 does not have an image decoder 21 because this embodiment uses encoding (decoding) information. The variable horizontal LPF 22 and the variable vertical LPF 23 are replaced with a horizontal pixel shifter 3 and a main straight pixel shifter 5. The control system is quite different.

  In the conventional example, the pixel value at the boundary portion is smoothed by filtering. However, the coding block distortion reduction according to the first embodiment of the present invention is performed by changing the block shape by pixel shift, thereby enabling continuity between blocks. Is to improve. As shown in FIG. 5 (a) one-way sequential, the relationship between the blocks is processed in the same block position and order as the blocks in encoding (decoding). For the processing target block, the processed upper and left blocks are used for matching estimation, and the unprocessed lower and right blocks are not used.

<Processing flow of main processing system>
The image signal coming from the image input 1 is once stored in the image buffer 2 and then given to the horizontal pixel shifter 3 and the horizontal matching estimator 7. The output of the image buffer 2 is in block units as necessary, but the block size is 8 × 8 pixels. The horizontal pixel shifter 3 divides 8 × 8 pixels in the vertical direction, sets the horizontal 8 pixels as one pixel column, and forms eight pixel columns in the vertical direction. Pixel shift is performed for each pixel column, but pixels other than 8 pixels are also used in the resampling process. The content of the pixel shift is changed for each pixel column in accordance with the pixel shift information given from the pixel column controller 8.

  In this way, the image signal deformed in the horizontal direction is temporarily stored in the image buffer 4, read out in the vertical direction, and supplied to the vertical pixel shifter 5 and the vertical matching estimator 9. The vertical pixel shifter 3 divides 8 × 8 pixels in the horizontal direction, sets the vertical 8 pixels as one pixel column, and forms eight pixel columns in the horizontal direction. The concept of processing is common except that the horizontal pixel shifter 3 is in the vertical direction. However, it is desirable that the output of the vertical pixel shifter 5 is not a block unit but a normal raster scan order. The image signal deformed in the vertical direction in this way is output from the image output 6 as an image signal with reduced block distortion.

<Detailed description of processing from matching estimation to pixel shift>
The horizontal alignment estimator 7 obtains an image signal including the pixels of the adjacent blocks from the image buffer 2. The horizontal alignment estimator 7 detects a positional shift of the boundary portion. In the boundary portion of the adjacent block, the pixel row on the adjacent block side is used as a reference pixel row, and the pixel row of the block to be processed is moved and scaled. The obtained plurality of signals are obtained and compared with the reference pixel column. The sum of the mean square error or absolute value error is obtained between the pixel columns, and the smallest movement and enlargement / reduction are set as the optimum values. In this process, the motion estimation based on block matching performed on a two-dimensional block is replaced with a one-dimensional pixel column. The obtained optimum movement and enlargement / reduction values are given to the pixel column controller 8.

  The movement and enlargement / reduction to be searched correspond to the horizontal change in FIG. The range and accuracy of the search is a range in which the end pixel moves 1.0 pixel to the left and right, and is performed in increments of (1/4) pixels. In this case, since there are 9 movements at the left end (0, ± 0.25, ± 0.5, ± 0.75, ± 1.0) and 9 movements at the right end, there are 81 patterns. Each pixel in the pixel column is evenly allocated according to the movement of the end pixel.

  The pixel column controller 8 determines the pixel shift of the pixel column in each block from the obtained optimal movement and enlargement / reduction value, and supplies the pixel shift to the horizontal pixel shifter 3. FIG. 7 shows how to obtain the pixel shift. The reference pixel column on the adjacent block side is set to the optimum value, and the pixel column 8 on the opposite side in the target block is not shifted. The pixel columns 1 to 7 in between are obtained by linear interpolation according to the position of the pixel column. Since there are eight pixel columns and the pixel column opposite to the reference pixel column is eight pixels away, the pixel value adjacent to the reference pixel column changes when the pixel column changes by one (1/8) times the optimum value. Is a shift amount (7/8) times the optimum value. Since the reference pixel column itself is not the target block, pixel shift is not performed, and there is no pixel column shifted by the optimum value. The pixel column controller 10 performs the same processing as the pixel column controller 8 except for the point that the pixel column to be used is in the direction perpendicular to the reference pixel column. For this reason, detailed description is abbreviate | omitted in this specification.

  The horizontal pixel shifter 7 resamples each pixel column in accordance with a given pixel shift. In the pixel shift, since the display position of the output pixel on the screen is only fixed, the filter coefficient acting on the input image is dynamically changed. As the simplest example, FIG. 9 shows the case of linear interpolation. However, since image quality is likely to deteriorate in linear interpolation, it is desirable to process with a resample filter using 4 to 8 pixels in the vicinity. An example of forming a pixel at the center position from 6 pixels is shown in FIG. The accuracy of the pixel position is about (1/8) pixel unit. The maximum pixel shift amount is less than one pixel.

<Second Embodiment of Encoding Block Distortion Reduction>
A second embodiment of motion compensated image coding according to the present invention will be described. FIG. 2 shows the configuration, and the same reference numerals are given to the same components as those of the first embodiment of FIG. In FIG. 2, a vertical distortion amount evaluator 15 and a horizontal distortion amount determiner 16 are added as compared with FIG. The input / output of the image buffers 11 and 12 and the pixel column controllers 13 and 14 are different from those of the image buffers 2 and 4 and the pixel column controllers 8 and 10 in FIG. The processing operation is different in that the correction is not performed when the distortion amount is observed and the distortion amount is small.

  The image signal coming from the image input 1 is once stored in the image buffer 11 and then given to the horizontal pixel shifter 3, the horizontal alignment estimator 7 and the vertical distortion amount evaluator 16. The operations of the image shifter 3 and the horizontal alignment estimator 7 are the same.

  The vertical distortion evaluator 15 obtains the image signal for the target block and the adjacent block above from the image buffer 2. As shown in FIG. 11, the processing is performed by taking a difference between pixels adjacent in the vertical direction between the upper half of the target block and the lower half of the adjacent block, and using the mean square error or the average absolute value difference as a reference. Calculate as a difference. Next, a difference is similarly obtained between pixels at the boundary portions of the two blocks, and an average square error or an average absolute value difference (matching with the reference) is obtained as a boundary difference. The ratio (boundary difference / reference difference) is obtained and compared with a predetermined value. When the amount of distortion is smaller than the predetermined value, information is given to the pixel array controller 13 so that pixel shift is not performed. Specifically, the predetermined value is 2.0 or the like. The pixel column controller 13 controls the presence or absence of the pixel shift based on the vertical distortion amount information given from the vertical distortion evaluator 11.

  On the other hand, the horizontal distortion evaluator 16 obtains image signals for the target block and its left block from the image buffer 12. The processing content of the horizontal distortion evaluator 16 is the same except that the vertical distortion evaluator 15 changes the vertical process in the horizontal direction. When the amount of distortion is smaller than a predetermined value, information is given to the pixel column controller 14 so that the pixel shift is not performed. The pixel column controller 14 controls the presence / absence of a pixel shift based on the information regarding the distortion in the horizontal direction given from the horizontal distortion evaluator 16.

<Third Embodiment of Encoding Block Distortion Reduction>
A third embodiment of coding block distortion reduction according to the present invention will be described. The configuration diagram is the same as that of the first embodiment of FIG. 1 and the second embodiment of FIG. 2, but the relationship between the non-processing block and the adjacent block is different. The relationship between the blocks is shown in “(b) Bidirectional checkered flag” in FIG. 5, and there are alternately processed blocks and unprocessed blocks. This is a pattern called a checkered flag. In this case, the upper, lower, left, and right blocks of the block to be processed are not processed. In the block to be processed, the position of matching estimation and the pixel column control method are performed in both directions unlike the first embodiment.

  The horizontal movement estimator 7 and the vertical alignment estimator 9 are basically the same as the first embodiment in how to obtain the optimum movement and the enlargement / reduction value. In addition to the adjacent portion, it is attached to the adjacent portion to the lower block, and the vertical matching estimator 9 is also attached to the adjacent portion to the right block in addition to the adjacent portion to the left block. The pixel column shift setting in the pixel column controllers 8 and 10 is that the pixel column on the opposite side is fixed as it is in the first embodiment, whereas in the third embodiment, FIG. As shown in Fig. 2, both are moved, and in the middle, both are connected.

  In the third embodiment, pixel shift is not performed at all for the upper, lower, left, and right adjacent blocks that are not the target block. That is, correction is more positively performed in the half block, but no correction is performed in the remaining half block.

  As described above, the present invention takes the matching of movement and enlargement / reduction between pixel columns at the block boundary of the reproduced image, shifts the pixel by resampling for each pixel column, and changes the block shape. In moving image coding, when motion compensation of a square block is applied to an image that is deformed by zooming or the like, or when the accuracy of motion compensation is insufficient and the prediction residual cannot be reproduced, still image coding uses DCT. Block distortion in which the shape (position) that tends to occur when the oblique high-frequency component is deleted can be reduced, and a high-quality reproduced image can be obtained.

  Also, with respect to the direction that is spatially orthogonal to the pixel column, the amount of distortion is evaluated by comparing the difference between the block boundary portions with the difference between other pixels, and if the amount of distortion is small, pixel shift is not performed. Therefore, it is possible to accurately reduce block distortion without avoiding unnecessary shape deformation and without causing image quality deterioration due to erroneous deformation and shape deformation due to noise.

  The encoded block distortion reduction method and apparatus of the present invention is preferably implemented as an LSI for an MPEG decoder, but can also be realized by a software module that executes the same processing. It can be stored and distributed in a device-readable recording medium such as a hard disk, CD-ROM, DVD, MO, flash memory, or EEPROM.

It is a figure which shows the structural example of the encoding block distortion reduction in 1st Embodiment of this invention. It is a figure which shows the structural example of the encoding block distortion reduction in 2nd Embodiment of this invention. It is a figure which shows the structural example of the encoding block distortion reduction of a prior art example. It is a figure which shows the mode of two types of block distortion. It is a figure which shows the mode of the block relationship in the encoding block distortion reduction process of this invention. It is a figure which shows the mode of a block shape deformation | transformation in the encoding block distortion reduction process of this invention. It is a figure which shows the block shape change of 1st Embodiment of this invention, and the mode of a pixel shift. It is a figure which shows the mode of a block shape change and pixel shift of 2nd Embodiment of this invention. It is a figure which shows the mode of the pixel shift by the phase filter of embodiment of this invention. It is a figure which shows the mode of the pixel shift by the high order filter in the encoding block distortion reduction of this invention. It is a figure which shows the mode of the amount of boundary distortion detection in the encoding block distortion reduction of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Image input terminal 2, 4, 11, 12 ... Image buffer, 3 ... Horizontal pixel shifter, 5 ... Vertical pixel shifter, 6 ... Image output, 7 ... Horizontal matching estimator, 8, 10, 13, 14 ... Pixel Column controller, 9 ... vertical matching estimator, 11, 15 ... vertical distortion evaluator, 12, 16 ... horizontal distortion evaluator, 21 ... image decoder, 22 ... variable horizontal LPF, 23 ... variable vertical LPF, 24 ... Correction amount determiner

Claims (4)

In a method for reducing block distortion of an image that is irreversibly encoded and decoded in units of square blocks,
In a block corresponding to an encoded block of a decoded image, a pixel column that is a one-dimensional column of pixels adjacent to the processing target block is set as a reference pixel column, and a pixel column in the processing target block adjacent to the reference pixel column A matching estimation step for obtaining matching with the reference pixel column while moving and reducing / enlarging in the direction of the pixel column, and determining an optimal movement and reduced / enlarged value that provides the best matching;
For a plurality of pixel columns adjacent to each other in the processing target block, pixels that are moved and reduced / enlarged for each pixel column based on the positional relationship between the pixel column that has obtained the optimal movement and reduced / enlarged values and each pixel column A pixel column control step for determining a shift amount;
An image shift step for performing pixel shift by resampling each pixel in each pixel column in the direction of the pixel column in accordance with the pixel shift amount of each pixel column in the processing target block. Encoding block distortion reduction method. The encoded block distortion reduction method of claim 1, comprising:
A distortion evaluation step for performing distortion evaluation for obtaining a distortion amount of a block shape from a relationship between a difference between adjacent pixel values in a whole block and a difference between pixel values of a block boundary part in a direction orthogonal to the pixel row,
The encoded block distortion reduction method, wherein the pixel shift step is executed when the distortion amount of the block shape is larger than a predetermined value. In an apparatus for reducing block distortion of an image that has been lossy encoded and decoded in units of square blocks,
In a block corresponding to an encoded block of a decoded image, a pixel column that is a one-dimensional column of pixels adjacent to the processing target block is set as a reference pixel column, and a pixel column in the processing target block adjacent to the reference pixel column A matching estimation means for obtaining matching with the reference pixel column while moving and reducing / enlarging in the direction of the pixel column, and determining an optimal movement and reduced / enlarged value that provides the best matching;
For a plurality of pixel columns adjacent to each other in the processing target block, pixels that are moved and reduced / enlarged for each pixel column based on the positional relationship between the pixel column that has obtained the optimal movement and reduced / enlarged values and each pixel column Pixel column control means for determining the shift amount;
A coding block distortion reduction apparatus comprising: pixel shift means for resampling each pixel in each pixel column in the direction of the pixel column in accordance with the pixel shift amount of each pixel column in the processing target block. The encoded block distortion reduction apparatus according to claim 3,
A distortion evaluation means for obtaining a distortion amount of a block shape from a relationship between a difference between adjacent pixel values in the whole block and a difference between pixel values of a block boundary portion in a direction orthogonal to the pixel row,
An encoded block distortion reduction apparatus that performs resampling by a pixel shift unit and shifts an image when a distortion amount of the block shape is larger than a predetermined value.

Images