KR20010101882A - Blocking artifact reduction in the DCT domain - Google Patents

Abstract

A method and apparatus for removing blocking artifacts in the frequency domain. Two frequency domain blocks A and B are received in the video stream. The third frequency domain block is computed directly in the transform domain that overlaps the boundary between the first and second frequency domain blocks. The coefficients of the third block are modified in the frequency domain to remove blocking artifacts, which is then transformed into a modification of two frequency domain blocks A and B in the frequency domain.

Description

Blocking artifact reduction in the DCT domain

Compression of an image causes the image to be transmitted in coded form over a communication channel using fewer data bits than in uncompressed form. Discrete cosine transform (DCT) coding is one well known compression scheme. The image is divided into small square inverses or "blocks". Each block is transform coded and transmitted over a communication channel. At the receiver, the blocks are decoded and reassembled into the original image. Typically these blocks are formed in an array of 8 x 8 pixels. DCT is applied to each block and then each block is quantized.

DCT is a linear transform that creates a block of new pixels, where each new pixel is a linear combination of all the input pixels of the original block. DCT-based block image coding techniques cause degradation of the received image in the form of blocking artifacts. When the image is block coded, the reconstructed blocks can be seen in the reconstructed image, which allows the viewer to see the block boundaries, which is usually due to uncorrelated quantization noise. Quantization noise is independent between blocks that yield jumps or steps at block boundaries. Typically, the greater the compression, the greater the blocking artifacts. The blocking effect is very visible to the viewer and very annoying since the eyes are very sensitive to "steps" at the block boundary.

This problem has been observed before and research has been proposed to solve this problem. Since blocking artifacts are caused by sudden discontinuities between neighboring blocks, eliminating this discontinuity can reduce the problem. Many solutions already proposed work in the spatial (pixel) domain. One technique involves using a spatially varying low pass filter in the pixel domain of block boundaries. The problem with such an approach is that it involves decompression of the image before the filter can be applied. Typically, however, it is useful to store the image in a compressed form after reduction of block artifacts and thus this technique requires the image to be decompressed.

FIELD OF THE INVENTION The present invention relates generally to the reduction of blocking artifacts in image data, and more particularly to a method and apparatus for reducing blocking artifacts of image data using only frequency domain operations.

FIG. 1A shows a third pixel block 'c' overlapping two 8 × 8 pixel blocks 'a' and 'b' and 'a' and 'b' and covering a boundary 'e'.

1B shows an example of elements of blocks 'a', a _ij and 'b', b _ij in a first embodiment of the invention;

FIG. 2A illustrates how pixel values of block 'c' change in a block. FIG.

2B shows ramp edges of pixel values of block 'c'.

3 shows a DCT matrix.

4A and 4B show matrices M ₁ and M ₂ , respectively.

5A and 5B show a matrix.

FIG. 6A shows a pixel matrix 'c' with step edges in columns 4 and 5; FIG.

FIG. 6B shows a matrix newc including linear interpolation of values of the pixel matrix c of FIG. 6A.

7A and 7B show a matrix.

8 illustrates the transpose matrix of D. FIG.

9A and 9B show blocks A and B. FIGS.

10 shows a change in coefficients of a frequency domain block δC.

11 shows a change in coefficients of a frequency domain block δA.

12 and 13 show changes for coefficients of blocks A and B (A + δA, B + δB, respectively).

14 illustrates a system using the method and / or apparatus of the present invention.

A method and apparatus are provided for reducing blocking artifacts by smoothing abrupt discontinuities at block boundaries in the frequency domain to analyze DCT characteristics of the boundary between two blocks and reduce blocking artifacts. The first frequency domain block A and the second frequency domain block B are received in a video stream. Then, a third frequency domain block C is calculated that overlaps the first and second frequency domain blocks at the boundary of the first and second blocks. The third block is used to smooth the discontinuity at the boundary of blocks A and B by adjusting the DCT coefficients of the third block. The calculated change of the coefficients of the third block is then converted into a change of the coefficients of the first and second blocks A, B.

It is an object of the present invention to perform this smoothing only in the frequency domain by using a third block (C).

Another object of the present invention is to change the first and second blocks of the following equation,

δC = δAM ₁ + δBM ₂ ,

Where δC is the change of the third block, δA and δB are the change of each of the first and second blocks, and M ₁ and M ₂ are the matrices with known values, respectively.

Another object of the present invention is to select δA and δB to minimize changes to the first and second blocks.

Another object of the present invention is to make δ A which is substantially the opposite of δ B.

It is another object of the present invention to determine whether the first and second blocks have relatively uniform pixel values in each block having a jump of pixel values across the boundary between the two blocks.

Another object of the invention relates to blocks A and B when blocks A and B are not relatively uniform. This object is achieved by reducing the large DCT coefficients of the high frequency components of the third block (C).

Yet another object of the present invention is to change the coefficients of the third block C by reducing the large high frequency coefficients of the third block C if the respective coefficients of the first and second blocks are substantially zero.

Still other objects and advantages of the invention will be apparent from the specification. Accordingly, the various steps of the present invention and the relationship therebetween, as well as combinations of elements and devices for eliciting a particular effect, will be apparent from the following detailed description, and the scope of the invention is indicated in the appended claims.

In general, embodiments of the present invention are directed to reducing blocking artifacts by analyzing the DCT characteristics of the boundary between two blocks, and by smoothing the sudden discontinuity of the block boundary in the frequency domain. Two frequency domains A and B are received in the video stream and the third block C is calculated to overlap at the boundary between the two blocks A and B. The third block is used to smooth the discontinuities at the boundaries of the blocks A and B by changing the DCT coefficients of the third block and converting this change into the difference of the coefficients of the blocks A and B. This blocking artifact reduction is described with reference to two blocks A and B in the first example that each of them has a uniform pixel value in the spatial domain but a step or jump in the pixel value occurs at the boundary of the block. The second example is described with reference to two blocks A and B that do not need to have uniform pixels in the spatial domain.

Figure 1a shows a third pixel block 'c' overlapping two 8x8 pixel blocks 'a' and 'b' and 'a' and 'b' and covering the boundary 'e'. An example of elements of blocks 'a', a _ij and 'b', b _ij in the first embodiment of the invention is shown, in this example, a _ij is the same for all (i, j) and such The cases are represented as ∂, where a _ij = ∂ (similarly b _ij is the same for all (i, j), where i = 0 · 7 and j = 0 · 7 And b _{ij =} ↓ when i = 0 · 7 and j = 0 · 7), hereinafter a _ij and b _ij are referred to as single pixel values. At high compression, the block boundary 'e' between 'a' and 'b' shows a step, i.e. all pixel values a _ij and b _ij are constant at the left and right of the boundary, with large discontinuities Exists in 2A shows how the pixel values of block 'c' change in the block. As shown graphically in FIG. 2A, there is a step edge in column 4. This step edge causes blocking artifacts. To remove this blocking artifact, the step edge must be smoothed with a ramp edge as shown in FIG. 2B. The object of the present invention is to smooth the DCT of block 'c' in the frequency domain, thus eliminating a step in the middle of block 'c' and converting this smoothing into a change in the coefficients of DCT blocks A and B. It is.

When receiving a bitstream compressed using JPEG or MPEG compression, the DCT blocks A and B are easily used from the compressed bitstream after performing variable length decoding, but the DCT block C is different. Therefore, in order to smooth the DCT block C, it is necessary to first calculate the DCT coefficients of the block C. This calculation is the DCT value of the right part of block A and the DCT of the left part of block B, since each DCT block is a linear combination of all input pixels of the block and thus greatly complicates the calculation of DCT block C. Involves merging values. One way to find the DCT coefficients of block C is to transform blocks A and B into the spatial domain and compute block C by linearly combining all the pixels of block 'c'. This is a cumbersome technique and requires decompressing the block. In the present invention, the DCT block C is obtained from the DCT blocks A and B without performing inverse DCT.

Since the blocks 'a, b' and 'c' can be considered as a matrix having elements constituting the pixel values a _ij, b _ij , the matrix 'c' is divided into the matrix 'a' and 'b' as follows. Can be described.

c = aK ₁ + bK ₂ (1)

Where K ₁ and K ₂ are an 8 X 8 matrix. The matrix K ₁ is given by

Where 0 is a 4 × 4 matrix for all zeros, and I is a 4 × 4 matrix with all diagonal entries equal to 1 and non-diagonal entries equal to zero. K ₂ = K ₁ ^T (ie, K ₂ is the transpose of K ₁ ). DCT and C of 'c' are obtained by the following multiplication using a DCT matrix. To get the DCT of the matrix we need to multiply it by

A = DaD ^T

B = DbD ^T

C = DcD^T (2)

(Where D is the DCT matrix shown in FIG. 3 and D ^T is the transpose matrix of D shown in FIG. 8.

The original matrices ('a', 'b', 'c') can be obtained from the DCT matrix using the following equation. In other words,

a = D ^T AD (3)

b = D ^T BD (4)

c = D ^T CD

If c is replaced by equation (1) in equation (2),

C = D (aK ₁ + bK ₂ ) D ^T (5)

Then, when formulas (3) and (4) are substituted for formula (5) for 'a' and 'b',

C = D (D ^T ADK ₁ + D ^T BDK ₂ ) D ^T

= (DD ^T ADK ₁ + DD ^T BDK ₂ ) D ^T

= DD ^T ADK ₁ D ^T + DD ^T BDK ₂ D ^T (6)

DD ^T = D ^T D = I know

C = ADK ₁ D ^T + BDK ₂ D ^T (7)

This expression can be rewritten as

C = AM ₁ + BM ₂ (8)

As can be seen from equation (7), M ₁ and M ₂ are fixed and do not depend on the matrix 'a', 'b' or 'c'.

4A and 4B show the matrices M ₁ and M ₂ , respectively. Looking at this matrix, the first, third, fifth, and seventh elements of M ₁ in odd rows are the same as in M ₂ , and the second, fourth, sixth, and eighth of M ₁ It is a negative value of M ₂ . Even rows are reversible. Therefore, M ₁ and M ₂ can be written as

M ₁ = Com + Dif (9)

M ₂ = Com-Dif (10)

Here, Com includes only common elements of M ₁ and M ₂ and all other elements are zero, and Dif contains only other elements, as shown in FIGS. 5A and 5B (all other elements are zero). Substituting Eq. (9) and Eq. (10) into Eq. (8) for M ₁ and M ₂ , respectively, the DCT matrix (C) converts from the DCT matrices (A and B) to the spatial domain as follows. You don't need to).

C = A (Com + Dif) + B (Com-Dif)

= (A + B) Com + (A -B) Dif (11)

This is less than 32 non-zero elements because the matrix Com contains only common elements. Similarly, matrix Dif contains only 32 non-zero entries. Comparing C using equation (11) instead of equation (8) saves computation. This is because Equation (8) requires two matrix multiplications, and as in Eq. (11), Equation (11) performs multiplication with matrix elements that are mostly zero.

Therefore, equation (11) provides the DCT coefficient of the DCT ring of block C, which converts blocks A and B back to the pixel domain.

Once the DCT coefficients of block C have been calculated, this block must be smoothed to remove step edges that exist in the middle of the block in the spatial domain. In order to smooth the block C in the DCT domain without converting to the pixel domain, the coefficient of the block C is changed when it is changed from having a block 'c' step edge in the spatial domain to showing a ramped edge. Analyze to see how.

6A shows the pixel matrix 'c' with step edges in columns 4 and 5. FIG. 6B shows a matrix newc including linear interpolation of the values of the pixel matrix c of FIG. 6A. This newc is a smoothed block in the pixel domain representing a ramped edge on the block. In FIG. 6B, the first column includes the pixel value aij of block 'a' and the last column includes the pixel value b _ij of block 'b'. The pixel values in columns 2-7 vary with linear interpolation from left to right. As can be seen in FIG. 6B, there is no longer a step edge between columns 4 and 5. Obviously linear interpolation is not only a method of eliminating this step edge but it is chosen for convenience of explanation.

The DCT of newc is NEWC as shown in FIG. 7C. DCT of 'c' is C as shown in FIG. 7A. The only difference between NEWC and C is that the first row of coefficients has been modified. Since the DCT of the block can be found by the equation DcD ^T , where matrices D and D ^T are shown in FIGS. 3 and 8, the first row of C can be rewritten using the following equation (where all remaining elements of C are A _ij = ∂ for all i = 0 · 7 and j = 0 · 7 and zero when b _ij = ↓ for i = 0 · 7 and j = 0 · 7)

C ₀₀ = 4 (∂ + ↓), C ₀₁ = 2 ^1/2 2.563 (∂- ↓), C ₀₃ = 2 ^1/2 0.9 (∂- ↓)

C ₀₅ = 2 _1/2 0.610 (∂- ↓), C ₀₇ = 2 ^1/2 0.511 (∂- ↓)

(The above equation for calculating C ₀₀ -C ₀₇ can be used for two blocks with uniform pixel values for all pixels a _ij and all pixel values b _ij )

Substituting newc shown in Fig. 4B into c in equation (2),

DnewcD ^T = NEWC

The first line of NEWC can be rewritten as follows (again, all remaining elements of NEWC are zero):

NEWC ₀₀ = 4 (∂ + ↓), NEWC ₀₁ = 2 ^1/2 1.841 (∂- ↓), NEWC ₀₃ = 2 ^1/2 0.1918 (∂- ↓)

NEWC ₀₅ = 2 ^1/2 0.0576 (∂- ↓), NEWC ₀₇ = 2 ^1/2 0.5139 (∂- ↓) (12)

Equation (12) requires spatial domain values a _ij and ↓ _ij , but these can easily be obtained from the coefficients of A and B without converting to the spatial domain. In this case a _ij = ∂ = 40 and b _ij = ↓ = 70 for all i = 0 · 7 and j = 0 · 7, but the ∂ and ↓ values depend on the video frames received in the video stream. 9A and 9B show blocks A and B (the DCT transformation of blocks 'a' and 'b' is shown in FIG. 1B). In this example, the DCT blocks A and B have only nonzero coefficients and are divided into eight, and have pixel values ∂ and ↓ of the respective domain blocks 'a' and 'b'. The spatial domain pixel values a _ij and b _ij can be obtained directly from the blocks A and B by dividing the DC coefficients of A and B into eight, respectively. To change C to NEWC, the first row of C's coefficients must be changed using Equation (12). This smooths the step edge to the ramp edge by acting block C in the frequency domain.

As mentioned above, block C does not exist in the video stream and was calculated to remove block artifacts. Therefore smoothing block C must be converted to a change in the coefficients of blocks A and B, which can be found in the video stream. From the formula C = AM ₁ + BM ₂ , the following can be seen.

C + δC = (A + δA) M ₁ + (B + δB) M ₂

δC = δAM ₁ + δBM ₂

δC (where δC is the change that must be made to block C in order to convert to NEWC) is taken as NEWC-C.

NEWC-C = δAM ₁ + δBM ₂

Therefore, the value for solving this equation for δA and δB smoothes the boundary between blocks A and B and eliminates blocking artifacts. There are many ways to solve these expressions. However, for the best results in image quality, the minimum amount of change for each block is minimized and a change occurs in one block opposite in the direction of the change of the other block, so that a smooth ramp edge occurs between the blocks. Preferred embodiments of the present invention are δ A = − δ B. This equation is as follows.

NEWC-C = δAM ₁ + δAM ₂ ,

NEWC-C = δA (M ₁ -M ₂₎

(NEWC-C) (M _1- M ₂ ) ^-1 = δ A

δA is shown in FIG.

Changes to the coefficients of blocks A and B (A + δA, B + δB) are shown in FIGS. 12 and 13, respectively. By modifying the blocks A and B in this way, the edge between the blocks A and B is smoothed without conversion into the spatial domain.

A system using the method and / or apparatus of the present invention is shown in FIG. The variable length decoder 10 decodes the input video stream and the inverse quantizer 20 calculates a DCT block with block artifacts. The block artifact removal system 30 removes blocking artifacts from the DCT block and provides the DCT block for display or storage.

The foregoing description reduces blocking artifacts when blocks in the spatial domain contain uniform pixels. When a block in the spatial domain contains a high frequency component, various various methods are used as described below.

The first way to reduce blocking artifacts when the block does not contain uniform pixels is to reduce the high frequency component of block C which is the result of the block boundary. When calculating block C using the method described herein, there are a number of high frequency DCT components of block C due to the block boundary.

Smoothing can be obtained by setting many high frequency coefficient values of block C to zero or reducing them. Once this modification to C has been performed, the change to C is converted to a change in blocks A and B as described above. This method can be used for uniform pixel blocks, but more precise results for uniform blocks are obtained from the method described above using linear interpolation.

Another embodiment of the present invention is smoothed by varying the individual coefficient values in C to zero when the corresponding values of blocks A and B are zero. The reason for this smoothing is that the blocks A and B belong to the same image region and thus the frequency characteristics of A and B are the same. Since the block C belongs to the same image region, the frequency characteristic of the block C should be the same as the frequency characteristic of the blocks A and B. However, since there is a boundary leading to the block C, there is a high frequency characteristic of the block C of the block C indicating the boundary and is not present in the blocks A and B and should be set to zero. Therefore, the high frequency component that is zero in A and B is set to zero in block C. This change in block C is translated into a change in blocks A and B as described above.

Although the present description is limited to the vertical boundary between blocks A and B, it may also extend to the horizontal boundary. In addition, the above-described DCT coefficient correction may be repeatedly performed on a block of the image until a predetermined smoothing is obtained.

Those skilled in the art will appreciate that various modifications, changes and variations are possible without departing from the spirit and scope of the invention. It is to be understood that the matters shown in the description and drawings are for illustrative purposes only and are not intended to limit the rights of the invention.

Claims (12)

In a method of reducing blocking artifacts, Receiving a first frequency domain block (FIG. 9A); Receiving a second frequency domain block (FIG. 9B); Calculating (30) a third frequency domain block (FIG. 7A) that overlaps the boundary (e) between the first (FIG. 9A) and second (FIG. 9B) frequency domain blocks; In order to reduce blocking artifacts at the boundary e between respective spatial domain blocks (FIG. 1B) of the first (FIG. 9A) and second (FIG. 9B) frequency domain blocks, the third frequency domain block δC ( Calculating a change in the coefficients of FIG. 10; Calculating (30) corresponding changes in the first and second frequency domain blocks (δA, δB) resulting in changes in the coefficients of the third frequency domain block (δC). The method of claim 1, Computing corresponding changes in the first and second frequency domain blocks includes the following condition: δC = δAM ₁ + δBM ₂ And M ₁ and M ₂ are constants. The method of claim 2, Wherein δA and δB are selected to minimize the change in the first and second frequency domain blocks. The method of claim 2, And wherein δA is substantially opposite to δB. The method of claim 1, Calculating a change in coefficients of the third frequency domain block (δC) is performed by reducing large coefficients of the high frequency components of the third frequency domain block and subtracting the third frequency domain block. The method of claim 1, Calculating a change in the coefficients of the third block comprises modifying the coefficients of the third frequency domain blocks to reduce high frequency coefficients if each of the coefficients of the first and second frequency domain blocks is substantially zero, and then the first A method for reducing blocking artifacts, performed by subtracting three frequency domain blocks. In an apparatus for reducing blocking artifacts, A receiver (FIG. 14) for receiving first and second frequency domain blocks; a) calculating a third frequency domain block (FIG. 7A) that overlaps the boundary e between the first (FIG. 9A) and second (FIG. 9B) frequency domain blocks, and b) the first and second Calculate a change in the coefficients of the third frequency domain block δC to reduce the blocking artifact at the boundary between each spatial domain blocks of the frequency domain blocks, c) of the coefficients of the third frequency domain block δC And a processor (30) for calculating corresponding changes in the first and second frequency domain blocks (δA, δB) resulting in a change. The method of claim 7, wherein The processor is subject to the following conditions, δC = δAM ₁ + δBM ₂ , where M ₁ and M ₂ are constants And calculate corresponding changes in the first and second frequency domain blocks to satisfy. The method of claim 8, And the processor calculates δA and δB to minimize changes in the first and second frequency domain blocks. The method of claim 8, And the processor calculates δ A, which is substantially inverse to δ B. The method of claim 7, wherein And the processor calculates a change in the coefficients of the third frequency domain block by reducing the large coefficients of the high frequency components of the third frequency domain block (δC) and subtracting the third frequency domain block. The method of claim 7, wherein Calculating a change in the coefficients of the third block may modify the coefficients of the third frequency domain blocks to reduce high frequency coefficients if each coefficient of the first and second frequency domain blocks is substantially zero and then the first A blocking artifact reduction apparatus, performed by subtracting three frequency domain blocks.