JP2003516054A - Method and apparatus for reducing blocking artifacts in DCT domain - Google Patents

Abstract

(57) Abstract: A method and apparatus for removing blocking artifacts in the frequency domain. Two frequency domain blocks (A and B) are received in the video stream. The third frequency domain block is calculated directly in the transform domain that overlaps the first and second frequency domain blocks. The coefficients of the third block are modified in the frequency domain to remove blocking artifacts, which are then transformed in the frequency domain into modifications of two frequency domain blocks (A and B).

Description

Detailed Description of the Invention

The present invention relates to reducing blocking artifacts in image data, and more particularly to methods and apparatus for reducing blocking artifacts in image data using only frequency domain operations.

Image compression allows images to be transmitted in encoded form over a communication channel using less bits of data than when transmitting images in uncompressed form. The Discrete Cosine Transform (DCT) is one well known compression technique.
The image is divided into small rectangular areas, or "blocks". Each block is transform-coded and transmitted through a communication transmission line. At the receiver, the blocks are decoded and assembled into the original image. Generally, these blocks are formed by an array of 8x8 pixels. DCT is applied to each block and then each block is quantized.

The DCT is a linear transform that forms a new block of pixels, where each new pixel is a linear combination of all the incoming pixels of the original block. Block image coding techniques based on DCT cause degradation of the received image in the form of blocking artifacts. When the image is block coded, the reconstructed blocks are visible in the reconstructed image and the observer will typically see block boundaries due to uncorrelated quantization noise. Quantization noise is block independent and causes jumps or steps at block boundaries. Generally, the greater the compression, the greater the blocking artifacts. The blocking effect is quite annoying because it is fairly visible to the observer and the eye is sensitive to "steps" at the block boundaries.

This problem has been observed for some time and attempts have been made to solve it. Since blocking artifacts are caused by sharp discontinuities between adjacent blocks, removing the discontinuities can reduce the blocking artifacts. Many of the proposals so far have been made in the spatial (pixel) domain. One technique uses a low pass filter that spatially varies in the pixel domain at the block boundaries. The problem with such an approach is that the image is decompressed before the filter can be applied. However, it is generally convenient to store the image in some form of compression after reducing blocking artifacts, and this technique therefore requires the image to be recompressed.

According to the present invention, there is provided a method and apparatus for analyzing DCT characteristics of a boundary between two blocks and reducing blocking artifacts by smoothing abrupt discontinuities at block boundaries in the frequency domain. To be done. The first frequency domain block (A) and the second frequency domain block (B) are received in the video stream. Next, a third frequency domain block (C) overlapping the first and second frequency domain blocks at the boundary between the first and second blocks.
Is calculated. The third block is used to smooth the discontinuities at the boundaries of blocks A and B by adjusting the DCT coefficients of the third block. Then, the calculated change in the coefficients of the third block is calculated as the first and second blocks (
A, B) is converted into a change in coefficient.

An object of the present invention is to perform such smoothing in the frequency domain only by using the third block C.

[0007]   In the present invention, δC is a change of the third block, δA is the first block, and δB is
The change in the second block is M₁And M_TwoLet be a matrix with known values
, The change of the first block and the second block is   δC = δAM₁+ ΔBM_Two The purpose is to meet.

The present invention attempts to minimize the change in the first block and the second block by δA.
And δB for another purpose.

[0009]   It is a further object of the present invention that δA is substantially opposite to δB.

The present invention determines whether the first block and the second block have relatively uniform pixel values within each block when having a jump in pixel values across the boundary between two blocks. That is another purpose.

The present invention aims to deal with blocks A and B when they are relatively non-uniform. This objective is achieved by reducing the large DCT coefficients of the high frequency components in the third block C.

The present invention reduces the large high frequency coefficients of the third block C by reducing the respective coefficients in the first block and the second block to be substantially zero. Another purpose is to change the coefficient of.

Other objects of the invention will be apparent from the following description. Accordingly, the invention is directed to a number of steps, the interrelationship of one or more of these steps, and the features of features, combinations of elements, and arrangements of parts adapted to carry out such steps. And apparatus embodying the invention, all of which are set forth in the following description and the scope of the invention is set forth in the following claims.

In general, embodiments of the present invention analyze DCT features at the boundary between two blocks,
It relates to reducing blocking artifacts by smoothing sharp discontinuities at block boundaries in the frequency domain. Two frequency domain blocks A and B are received in the video stream and a third block C is calculated which overlaps the boundary between the two blocks A and B. The third block is the third
Used to smooth the discontinuity at the boundary of blocks A and B by changing the DCT coefficient of the block of blocks and transforming this change into changes of blocks A and B. Reducing blocking artifacts is described first in the case where two blocks A and B each have uniform pixel values in the spatial domain, but pixel value steps or jumps occur across the boundaries of the blocks, Second, the case of two blocks A and B that do not necessarily have uniform pixels in the spatial domain will be described.

FIG. 1 a shows two 8 × 8 pixel blocks “a” and “b” and “a” and “b”.
FIG. 6 is a diagram showing a third pixel block “c” that covers the overlapping boundary “e” with FIG. Figure 1b
Is a diagram showing a first example of element b _ij elements a _ij and the block of the block "a" in Example "b" of the present invention. In this case, a _ij is the same for all (i, j),

[0016]

[Outer 1] And i = 0 to 7 and j = 0 to 7

[0017]

[Equation 1] (Similarly, b _ij is the same for all (i, j), and such a case is denoted by ↓, and b _ij = when i = 0 to 7 and j = 0 to 7) ↓),
Hereinafter, a _ij and b _ij are each shown as a single pixel value. For high compression,
The block boundary “e” between “a” and “b” indicates a step, ie all pixel values a _ij and b _ij are constant on the left and right sides of the boundary, respectively, and there is a large discontinuity at the boundary “e”. There is a nature. FIG. 2a is a diagram showing how the pixel values in block “c” change throughout the block. As can be seen from the graph in Figure 2a, there is a step edge in column 4. This step edge causes blocking artifacts. To remove this blocking artifact,
The step edges must be smoothed into slanted edges as shown in Figure 2b. The present invention smooths the DCT C of block “c” in the frequency domain, thereby eliminating steps in the middle of block “c” and transforming this smoothing into changes in the coefficients of DCT blocks A and B. With the goal.

When receiving a bitstream compressed using JPEG or MPEG compression, DCT blocks A and B are readily available after performing variable length decoding from the compressed bitstream, while DCT blocks C is not readily available. Therefore, in order to smooth the DCT block C, first the DCT of the block C
It is necessary to calculate the coefficients. This calculation is simply the DCT of the right part of block A, since each DCT block is a linear combination of all the incoming pixels of the block.
It does not merge the values with the DCT values in the left part of block B, but rather complicates the calculation of DCT block C. One way to find the DCT coefficients of block C is to transform block A and B into the spatial domain and compute block C by linearly combining all the pixels of block "c". This is a tricky technique and requires block decompression. In the present invention, DCT block C is obtained from DCT blocks A and B without performing an inverse DCT.

Since the blocks “a”, “b”, and “c” can be considered to be matrices each having an element composed of pixel values a _ij and b _ij , the matrix “c” is the matrix “a”. , B can be written as c = aK ₁ + bK ₂ (1), where K ₁ and K ₂ are 8 × 8 matrices. The matrix K ₁ is

[0020]

[Equation 2] Where 0 is a 4x4 matrix with all zeros and I is all
It is a matrix in which elements on the diagonal line are 1 and elements on the diagonal line are 0. K₂=₁ ^TK (
That is, K₂Is K₁Is the transposed matrix of). C, which is the DCT of “c”, is a DCT matrix
It is obtained by the following multiplication used. To obtain the matrix DCT, the following equation,   A = DaD^T   B = DbD^T   C = DcD^T                              (2) (However, D is the DCT matrix shown in FIG.^TIs a transposed matrix of D, and FIG.
It is known to have to be applied).

[0021]   The original matrix (“a”, “b”, “c”) is the following equation from the DCT matrix:   a = D^TAD (3)   b = D^TBD (4)   c = D^TCD Substituting c of Equation 2 into Equation 1,   C = D (aK₁+ BK₂) D^T                (5) Then, by substituting the equations 3 and 4 into “a” and “b” of the equation 5, respectively,   C = D (D^TADK₁+ D^TBDK₂) D^T     = (DD^TADK₁+ DD^TBDK₂) D^T     = DD^TADK₁D^T+ DD^TBDK₂D^T      (6) Becomes

Since DD ^T = D ^T D = I, C = ADK ₁ D ^T + BDK ₂ D ^T (7). This equation can be written as C = AM ₁ + BM ₂ (8). As can be seen from Equation 7, M ₁ and M ₂ are constant, and the matrix “
It does not depend on any of "a", "b", and "c".

[0023]   4a and 4b show the matrix M, respectively.₁And M₂FIG. These matrices
Looking, M₁The 1st, 3rd, 5th, and 7th elements in the odd row of are M₂Same as the element of
The same, M₁The second, fourth, sixth and eighth elements of M are₂Is negative of
I understand. The opposite is true for even rows. Therefore, M₁And M₂Is
The formula below,   M₁= Com + Dif (9)   M₂= Com + Dif (10) Where Com is M, as shown in FIGS. 5a and 5b, respectively. ₁ And M₂Contains only the common elements of both, all other elements are zero, and Dif is
Contains only different elements (all other elements are zero). Equation 9 and Equation 10 are M₁
And M₂Substituting into Equation 8 as DCT matrix C from DCT matrices A and B (
Without converting back to the spatial domain),   C = A (Com + Dif) + B (Com-Dif)     = (A + B) Com + (AB) Dif (11) Can be found by The matrix Com contains less than 32 because it contains only common elements.
Have non-zero elements. Similarly, the matrix Dif contains only 32 non-zero elements.
Although Equation 8 requires two matrix multiplications and Equation 11 also requires two matrix multiplications,
, Equation 11 performs multiplication using matrix elements that are almost zero, so Equation 1 is replaced by Equation 1
Computing C using 1 saves computation.

Therefore, equation 11 gives the DCT coefficients of the DCT transform of block c without having to transform blocks A and B back into the pixel domain.

Once the DCT coefficients of block C have been calculated, this block must be “smoothed” to remove the step edges that lie in the center of the block in the spatial domain. To smooth block C in the DCT domain without transforming it back into the pixel domain, to see if block "c" in the spatial domain changes from having step edges to showing slanted edges. Analysis was done.

FIG. 6 a shows a pixel matrix “c” with step edges in columns 4 and 5. FIG. 6b is a diagram showing a matrix newc that includes linear interpolation of the values of the pixel matrix c of FIG. 6a. This newc is a smoothed block in the pixel domain, showing slanted edges across the block. In FIG. 6b, the first column contains the pixel values a _ij of block “a” and the last column contains the pixel values b _ij of block “b”. The pixel values in columns 2-7 change according to linear interpolation from left to right. As can be seen from FIG. 6b, there is no longer a step edge between columns 4 and 5. Obviously, linear interpolation is not the only way to eliminate step edges and is chosen here for ease of explanation.

FIG. 7 c is a diagram showing NEWC, which is DCC of newc. DCT of "c"
Is C as shown in FIG. 7a. The only difference between NEWC and C is that the first row of coefficients has changed. The DCT of a block is found by the equation DcD _T using the matrices D and D _T shown in FIGS. 3 and 8 respectively and the matrix “c” shown in FIG. 6a, so the first row of C is Can be written using. (Note that the remaining elements of C are zero, but for all values of i = 0 to 7 and j = 0 to 7

[0028]

[Equation 3] And b _ij = ↓ for all values i = 0 to 7 and j = 0 to 7).

[0029]

[Equation 4] (The above formula for calculating C ₀₀ -C ₀₇ can be used for any two blocks with uniform pixel values respectively for all pixels a _ij and all values b _ij )
. Substituting newc shown in FIG. 4b into c in Equation 2, DnewcD _T = NEWC, and the first row of NEWC (again, all remaining elements of NEWC are zero),

[0030]

[Equation 5] Can be written. Equation 12 is the spatial domain value

[0031]

[Outside 2] , Which are easily obtained from the A and B coefficients without spatial domain transformation. In this example, for all values i = 0 and j = 0

[0032]

[Equation 6] In Although,

[0033]

[Outside 3] Depends on the frames of the video received in the video stream. 9a and 9b are diagrams showing blocks A and B (DCT transform of blocks "a" and "b" shown in FIG. 1b). DCT blocks A and B, when divided by eight, are of the respective spatial domain blocks "a" and "b".

[0034]

[Outside 4] For example, when i = 0 to 7 and j = 0 to 7 in this example,

[0035]

[Equation 7] Has only one non-zero coefficient that yields ↓ = 70 = b _ij for i = 0 to 7 and j = 0 to 7. Therefore, the spatial domain pixel values a _ij and b _ij are simply A
And B are obtained directly from blocks A and B by dividing the DCT coefficients by 8. To change C to NEWC, the first row of C's coefficients must be changed using Equation 12. It smooths step edges into slanted edges by operating on block C in the frequency domain only.

[0036]   As mentioned above, block C is not present in the video stream and is blocked.
-Calculated to eliminate artifacts. Therefore, the smoothing of block C is
, Converted to changes in the coefficients of blocks A and B found in the video stream.
I have to do it. Formula C = AM₁+ BM₁From   C + δC = (A + δA) M₁+ (B + δB) M₂   δC = δAM₁+ ΔBM₂ ΔC (where δC is for converting block C to NEWC
If the changes that must be made) are NEWC-C,   NEWC-C = δAM₁+ ΔBM₂ Can be written.

[0037]   Thus, for any value that solves the equations for δA and δB, the blocks A and B
The boundaries between are smoothed and the blocking artifacts are removed. This formula
There are many ways to solve, but to achieve the best results in terms of image quality,
Try to minimize the amount of variation for each block, and
Change in one block is opposite to the change in the other block so that
Should be done. In the preferred embodiment of the invention, δA = −δB.
Then the formula is as follows:   NEWC-C = δAM₁+ ΔAM₂   NEWC-C = δA (M₁-M₂)   (NEWC-C) (M₁-M₂)^-1= ΔA Can be written. δA is shown in FIG.

12 and 13 are diagrams showing changes in the coefficients of blocks A and B (A + δA, B + δB), respectively. By changing blocks A and B in this way,
The edges between blocks A and B are smoothed without conversion to the spatial domain.

FIG. 14 is a diagram showing a system using the method and / or apparatus of the present invention. The variable length decoder 10 decodes the incoming video stream and the dequantizer 20 computes a DCT block with blocking artifacts. The blocking artifact removal system 30 removes blocking artifacts from DCT blocks and provides DCT blocks for display or storage.

The above description relates to reducing blocking artifacts when blocks in the spatial domain contain uniform pixels. If the blocks in the spatial domain contain many high frequency components, different methods are used, as described below.

A first way to reduce blocking artifacts when a block does not contain uniform pixels is to reduce high frequency components in block C that result from block boundaries. When calculating block C using the method described herein, there are numerous high frequency DCT components in block C due to block boundaries.

Smoothing involves setting large high frequency component values in block C to zero, or
Obtained by reducing their values. Once this change to C is performed, the change to C is converted to the change in blocks A and B as described above. Although this method can be used for uniform pixels, the method described above with linear interpolation gives more accurate results for uniform blocks.

In another embodiment of the invention, block C is smoothed by changing the individual coefficient values in C to zero when their corresponding values in blocks A and B are also zero. It This gives a smoothing because blocks A and B belong to the same image region and therefore the frequency features of A and B should be similar to each other. Since the block C also belongs to the same image area, the frequency characteristics of the block C should be similar to those of the blocks A and B. However, since the boundary is introduced in block C, it has some high frequency features that represent the boundary,
These are not found in blocks A and B and should therefore be set to zero. Therefore, high frequency components that are zero in both blocks A and B are set to zero in block C. This change in block C is converted to the change in blocks A and B as described above.

Although the above description was limited to the vertical boundary between blocks A and B, it is equally applicable to the vertical boundary. Further, the DCT coefficient modification described above may be iteratively performed on blocks of the image until the desired smoothness is achieved.

Thus, the foregoing objectives have been effectively achieved as set forth in the foregoing description, and there is some requisite for performing and implementing the methods described above without departing from the spirit and scope of the invention. It is to be understood that the matter contained in the above description and shown in the accompanying drawings is illustrative and not limiting, as changes may be made to

[Brief description of drawings]

FIG. 1a shows two 8 × 8 pixel blocks “a” and “b” and a third pixel block “c” that overlaps “a” and “b” and covers a boundary “e”.

Figure 1b   Element a of block "a" of the first embodiment of the present invention_ijAnd element “b” of block “b” _ij It is a figure which shows the example of.

FIG. 2a is a diagram showing how pixel values in block “c” change across blocks.

Figure 2b   It is a figure which shows an inclined edge.

[Figure 3]   It is a figure which shows D which is a DCT matrix.

FIG. 4a is a diagram showing a matrix M ₁ .

FIG. 4b is a diagram showing a matrix M ₂ .

FIG. 5a: Com containing only the common elements of both M ₁ and M ₂ , all other elements being zero.
FIG.

FIG. 5b is a diagram showing Dif containing only different elements (all other elements are zero).

FIG. 6a   FIG. 6 shows a pixel matrix “c” with step edges in columns 4 and 5.

FIG. 6b   FIG. 6b is a diagram showing a matrix newc that includes linear interpolation of the values of the pixel matrix c of FIG. 6a.

FIG. 7a   It is a figure which shows C which is DCT of "c".

FIG. 7b   It is a figure which shows NEWC which is DCT of newc.

FIG. 8 is a diagram showing a transposed matrix D ^{T of} D which is a DCT matrix.

FIG. 9a   It is a figure which shows the block A.

FIG. 9b   It is a figure which shows the block B.

[Figure 10]   It is a figure which shows deltaC.

FIG. 11   It is a figure which shows deltaA.

[Fig. 12]   It is a figure which shows the change of the coefficient of block A.

[Fig. 13]   It is a figure which shows the change of the coefficient of block B.

FIG. 14   FIG. 6 illustrates a system using the method and / or apparatus of the present invention.

[Procedure amendment]

[Submission date] August 10, 2001 (2001.08.10)

[Procedure Amendment 1]

[Document name to be amended] Statement

[Name of item to be corrected] 0027

[Correction method] Change

[Contents of correction]

FIG. 7 b is a diagram showing NEWC, which is the DCT of newc. DCT of "c"
Is C as shown in FIG. 7a. The only difference between NEWC and C is that the first row of coefficients has changed. The DCT of a block is found by the equation DcD _T using the matrices D and D _T shown in FIGS. 3 and 8 respectively and the matrix “c” shown in FIG. 6a, so the first row of C is Can be written using. (Note that the remaining elements of C are zero, but for all values of i = 0 to 7 and j = 0 to 7

─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Abdel-Mottareb, Mohamed             Netherlands, 5656 Earth Ardine             Fen, Plov Holstran 6 F term (reference) 5C059 KK03 LC03 MA00 MA23 MC11                       UA02                 5C078 AA04 BA57 CA21 DA02                 5J064 AA02 BA16 BC08 BC09 BC21                       BD01

Claims (12)

[Claims] 1. A step of receiving a first frequency domain block, a step of receiving a second frequency domain block, and a third step of overlapping a boundary between the first frequency domain block and the second frequency domain block. Calculating frequency domain blocks of the third frequency domain block and changing the coefficient (δC) of the third frequency domain block to reduce blocking artifacts at boundaries between respective spatial domain blocks of the first and second frequency domain blocks. And computing corresponding changes (δA, δB) in the first and second frequency domain blocks that result in a change (δC) in the coefficients of the third frequency domain block. Blocking artifacts How to reduce. 2. Corresponding variations of the first and second frequency domain blocks.
The step of calculating₁And M₂Is a constant,   δC = δAM₁+ ΔBM₂ The method of claim 1, wherein: 3. The method of claim 2, wherein δA and δB are selected to minimize changes in the first and second frequency domain blocks. 4. The method of claim 2, wherein δA is substantially the opposite of δB. 5. The step of calculating the change (δC) of the coefficient of the third frequency domain block reduces the large coefficient of the high frequency component of the third frequency domain block and subtracts the third frequency domain block. The method of claim 1, wherein the method is performed by. 6. The step of calculating the change in the coefficient of the third block includes a third step for reducing the high frequency coefficient if the respective coefficient in the first and second frequency domain blocks is substantially zero. 2. The method of claim 1, wherein the method is performed by changing the coefficients of the frequency domain blocks of, and then subtracting the third frequency domain block. 7. A receiver for receiving the first and second frequency domain blocks, and (a) calculating a third frequency domain block overlapping the boundary between the first and second frequency domain blocks, b) calculating the coefficient change (δC) of the third frequency domain block to reduce blocking artifacts at the boundary between the respective spatial domain blocks of the first and second frequency domain blocks; A processor for calculating corresponding changes (δA, δB) in the first and second frequency domain blocks that cause changes in the coefficients (δC) of the three frequency domain blocks. 8. The processor includes the first and second frequency domain blocks.
The corresponding change in₁And M₂Is a constant,   δC = δAM₁+ ΔBM₂ The device according to claim 7, which calculates to satisfy 9. The apparatus of claim 8, wherein the processor calculates δA and δB to minimize changes in the first and second frequency domain blocks. 10. The processor is such that δ is substantially opposite to δB.
The apparatus according to claim 8, which calculates A. 11. The processor changes a coefficient (δC) of the third frequency domain block by reducing a large coefficient of a high frequency component of the third frequency domain block and subtracting the third frequency domain block. The apparatus of claim 7, which calculates 12. The step of calculating the variation of the coefficients of the third block includes a third step for reducing the high frequency coefficient if each coefficient in the first and second frequency domain blocks is substantially zero. Change the coefficients of the frequency domain block of
8. The apparatus of claim 7, performed by subtracting the frequency domain blocks of