CA2440013C - Signal adaptive filtering method, signal adaptive filter and computer readable medium for storing program therefor - Google Patents

Abstract

A signal adaptive filtering method for reducing blocking effect and ringing noise, a signal adaptive filter, and a computer readable medium. The signal adaptive filtering method capable of reducing blocking effect and ringing noise of image data when a frame is composed of blocks of a predetermined size includes the steps of: (a) generating blocking information for reducing the blocking effect and ringing information for reducing the ringing noise, from coefficients of predetermined pixels of the upper and left boundary regions of the data block when a frame obtained by deconstructing a bitstream image data for inverse quantization is an intraframe; and (b) adaptively filtering the image data passed through inverse quantization and inverse discrete cosine transform according to the generated blocking information and ringing information. Therefore, the blocking effect and ringing noise can be eliminated from the image restored from the block-based image, thereby enhancing the image restored from compression:

Description

This application is a division of Canadian Application I~o. 2,415,375 filed June 18, 1998 for Signal Adaptive Filtering nllethod, Signa! Adaptive Filter and Computer Readable ll~edium for Storing Program Therefor.
Technical Field The present invention relates to data filtering, and more particularly, to a signal adaptive filtering method for reducing blocking effect and ringing noise, a signal adaptive filter and a computer readable medium for storing a program therefor.
Background Art Generally, picture encoding standards such as MPEG of the International Standardization Organization (ISO) and H.263 recommended by the International Telecommunication Union (ITU) adopt block-based motion estimation and discrete cosine transform (DCT) blocks. V1lhen an image is highly compressed, the block-based coding may cause the well-known blocking effect. A typical blocking effect is grid noise in a homogeneous area in which adjacent pixels have relatively similar pixel values. Another blocking effect is staircase noise which has a staircase shape and is generated along the edges of the image. Also, ringing noise is a typical Gibb's phenomenon occurring by truncation when the coefficients of the DCT are quantized so as to highly compress the image.
Grid noise shows traces of the block-based process at the edges between blocks when the compressed data is displayed on a screen after being restored. Thus, one can identify the edges between blocks. Also, staircase noise has a staircase shape at the edges of the image, so that one can notice a bumpy edge on the image. Also, one can notice overlapping of images with a predetermined interval due to the ringing noise.
In order to reduce the blocking effect and the ringing noise occurring when block-based coding is performed, several methods have been suggested. According to H.261 encoding, a simple 3 ~ 3 low-pass filter (LPF) is used as a loop filter to reduce the blocking effect [°'Video Codes for Audiovisual Services at f' X 62 kbitls'°, . . . . . . , CCITT Recommendation H.~6I, December 14~, 3990]. Also, a simple edge loop filter has been suggested . so as to reduce the blocking effect and mosquito noise (G.Bjontegaard, "A Simple Fdge Loop Filter to Reduce .~lockir~g cznd Ivlosquito Noise", ISOJIEC TTCIJSc29/WG11 I~iFEG96/0617, January, 1996, and '°A
Simple S Edge Loop Filter to Reduce docking rznd .Mosquito Noise", ITU SC15 LBC
Expert Group ITU-LBC-96-032, .Ianuary, 1996]. The edge loop filter makes linear values of two pixels adjacent to the block boundary and replaces the two pixel values by the linearized values. Such edge loop filter can reduce the blocking effect but not the ringing noise. In order to reduce the ringing noise, a non-linear filter using a binary index has been suggested [Y.Itoh, "Detail Preserving Nonlinear Filter using Binary Index, "ISUJIEC JTC1lSC291WCa11 Mfl'EG95/0357, November, 1995j.
However, the non-linear filter cannot reduce the blocking effect.
Disclosure of the Invention To solve the above problems, it is an object of the present invention is to provide an signal adaptive filtering method for reducing blocking effect and ringing noise in high compression encoding, a signal adaptive filter, and ~a computer readable medium.
According to an aspect of the present invention, there is provided a signal adaptive filtering method capable of.reducing blocking effect and ringing noise of image data when a frame is composed of blocks of a predetermined size, the method comprising the steps of. (a) generating blocking information for reducing the blocking effect and ringing information for reducing the ringing noise, from coefficients of predetermined pixels of the upper and left boundary regions of the data block when a frame obtained by deconstructing a bitstream image data for inverse quantizarion is an intraframes and (b) adaptively filtering the image data passed through inverse quantization and inverse discrete cosine transform according to the generated blocking information and ringing information:
Preferably the step (a) further comprises the step of setting the blocking information and ringing information of the previous frame corresponding to a morion vector as blocking information and ringing information of the-current frame if the frame is an interframe, and setting the ringing information to " 1 "
which represents the image data requiring filtering if a residual signal of the inverse-quantized current block exists, and the blocking information and ringing information are determined according to coefficients of a pixel A located at the upper left comer of the block, a pixel ft located to the right of the pixel A
and a pixel C located below the pixel A.
Preferably, the blocking information is constituted of horizontal blocking information and vertical blocking information, and the horizontal blocking information is set to "1'° which means the image data requiring filtering when only the coefficient of the pixel A is not equal to "0" or any coefficient of the pixels of the left boundary region of the block is not equal to °'0", and the vertical blocking information is set to " 1 " which means the image data requiring filtering when only the coefficient of~the pixel A is not equal to "0'° or any coefficient of the pixels of the upper boundary region of the block is not equal to "0", and the ringing information is set to "1" which means the image data requiring filtering when any coe~cient of the pixels other than the pixels A, P and C of the block is not equal to "0" .
Preferably, the blocking information is constituted of horizontal blocking information and vertical blocking information, and the horizontal blocking information is set to "1" which means the image data requiring .filtering when all coefficients of the pixels A, ~ and C of the block are not equal to "0" or any coefficient of the pixels of the left boundary region of the block is nbt equal _ to "0" , and tie vertical blocking information is set to °' 9.'" which means the image data requiring filtering when all coefficients of the pixels A, B and.C are not equal to "0" or any coefficient. of the pixels of the upper boundary region of the block is not equal too "0", and the ringing information is set to "1" which means the image data requiring filtering when any coefficient of the pixels other than the pixels A; B and C of the block is not equal to "~" .
Preferably, in order to reduce the blocking effect, in the step (b), the horizontal (or vertical) filtering is performed using a weighted filter having a predetermined weighted value when the horizontal (or vertical) blocking information of the block is equal to " 1 " and the ringing information is equal to "0", and when the horizontal (or vertical) blocking information of the block is not equal to °' 1 " or the ringing inforanation is not equal to °'0" , an absolute value of the difference between adjacent pixels and a Q value used as a dividend for quantizing the block are compared, and then filtering is performed with a predetermined value according to the result of the comparison.
According to another aspect of the present invention, there is provided a signal adaptive filter capable of reducing blocking effect- and ringing noise of image data when a frame is composed of blocks of a predetermined size, comprising: a mode flag checking unit for checking a flag to determine whether or not a frame is an intraframe or an interframe when a bitstream image data is deconstructed for inverse quantization; an infra filtering information generator for generating blocking information for reducing the Mocking effect and zinging information for reducing the ringing noise, from coefficients of predetermined pixels of the upper and left boundary regions of the data block when the frame is . determined as an intraframe by the mode flag checking unit; an inter filtering information generator for setting the blocking information and ringing . information of the previous frame corresponding to a motion vector as blocking information and ringing information of the current frame if the frame is an interframe, and setting the ringing information to "i" if a residual signal of the inverse-quantized current block exists;
and an adaptive filter for adaptively filtering the image data passed through an inverse quantizer and an inverse discrete cosine transformer according to the blocking information and the ringing information generated by the infra filtering information generator and the inter faltering information generator.
The invention may be embodied in a general purpose digital computer that is running a program from a computer usable medium, including but ndt limited to storage media such as magnetic storage media (e.g., i2~M's, floppy disks, hard disks, etc.), optically readable media (e.g., CIA-It~Ms, L)V'Ds, etc.) and carrier waves (e.g., transmissions over the Internet). kience, the present invention may be embodied as a computer usable medium.
According eo still another aspect of the present invention, there is provided a computer readable medium having embodied thereon a computer program for a signal adaptive filtering capable of reducing blocking effect and ringing noise of image data when a frame is composed of blocks of a predetermined size, wherein the signal adaptive filtering comprises the steps of: (a) generating blocking information for reducing the blocking effect and ringing information for reducing the ringing noise, from coefficients of predetermined pixels of the upper and left boundary regions of the data block when a frame obtained by deconstructing a bitstream image data for inverse quantization is ~an intraframe; (b) setting the blocking information and ringing information of the previous frame corresponding to a motion vector as blocking information and ringing information of the cura-ent frame if the frame is an interframe, and setting the ringing informati~n 'to "1"
which represents the image data requiring filtering if a residual signal of the inverse-quantized current block exists; and (c) adaptively filtering the image data passed through the inverse quantization and inverse discrete cosine transform based on the block according to the generated blocking information and ringing information.
Also, there is provided a computer readable medium having embodied thereon a computer program for a method of filtering ringing noise caused when decoding a block-based compressed image data, wherein the ringing noisy filtering method comprises the steps of: Via) performing a gradient operation on the block subjected to inverse tpuantization and inverse discrete cosine transform using predetermined one-dimensional horizontal and vertical gradient operators; (b) generating a binary edge map representing whether or not each pixel is an edge pixel, using an absolute value of the d'iff'erence betv~een the gradient-operated value of one pixel azid the value of the adjacent pixel, and a C2 value used as a dividend for quantizing the block; and (c) performing a filtering by applying a predetermined Biter window to the generated binary edge map.
Brief Description of the Drawings FIG. 1 is a block diagram of a signal adaptive f°~lter for reducing blocking effect and ringing noise according to the present invention;
FIG. 2 is a flowchart illustrating a signal adaptive filtering method according to a preferred embodiment of the present invention;
FIG. 3 shows an inverse-quantized block having 8 x 8 pixels FIG: 4 is a flowchart illustrating the step of generating information used to filter an intraframe;
FIG. 5 is a flowchart illustrating the step of generating inforrr~ation used to filter an interframe;
FIG. 6 shows the location of pmxels adjacent to the block boundary for illustrating the filtering for reducing blocking effect; and FIG. 7 shows the location of pixels to be processed in the current block.
Best mode for carr~rin~ out the Invention _ In FIG. 1, a signal adaptive filter for reducing blocking effect and ringing noise according to the present invention includes a mode flag checking unit 120, an infra filtering information generator 130, an inter filtering information generator 140 and an adaptive filter unit 1~0. '6~hen bitstx~ea~n image data is deconstructed for inverse quantization, the mode flag checking unit 120 checks whether or not the frame is an infra frame or an inter fine. The infra filtering information generator 130 generates blocking information for reducing blocking effect and ringing information from coefficients of predetermined pixels of the upper and left boundary regions of the data block when the frame is determined as an infra frame by the mode flag checking unit 120. then the mode flag checking unit 120 determines the frame as an inter frame, the inter filtering information generator 140 generates blocking anformatioar and ringing information of the previous frame corresponding to a motion vector as blocking information and ringing information of the current -frame. If a residual signal of the: inverse-quantized current block exists, the ringing information is set to "1". a adaptive filtering unit 150 adaptively filters the image data of the block which has passed an inverse quantizer (Q-') 100 and an inverse discrete cosine transformer (I)CT'-' ) i 10 according to the blocking information and ringing information generated by the infra filtering information generator 130 and the inter, filtering information generator 140.
Meanwhile, .a signal adaptive filtering method according to a preferred embodiment of the present ynvention will be described. FIG. 2 is a flowchart illustrating the signal adaptive filtering method according to the present invention.
Bitstream image data coded by an encoder is decoded by a decoder to be reproduced. To end this, the bitstream data is deconstructed and then inverse-quantized by the inverse quantizer 100 (step 200). I-Iere, the image data is constituted of a plurality of frames, and each frame is constituted of a plurality of blocks. FIG. 3 shows an inverse-quantized block having ~ x8 pixels which form the frame.
S Before filtering the frame data with the inverse discrete cosine transform (IDCT), a flag is checked in order to make a determination as to whether or not the frame is an intraframe or an interframe (step 2I0). If the frame is an intraframe (step 220), information used to filter the intraframe is generated (step 230).
If the frame is an interfrar~ae, information used to filter the interframe is generated {step I0 240). Then, the frame data which has passed through the IDCT 110 are adaptively filtered according to the gene: ated filtering inforrrtation, thereby eliminating blocking effect and ringing noise (step 250).
FIG. 4 is a flowchart illustrating in detail the step ~f generating information used to filter the intraframe. As shown in FIG. 4, if the frame is determined as an 15 intraframe by the anode flag checking unit 120, the coefficient of pixel A
of FIG.
3 is checked (step 400). If only the coefficient of the pixel A is not equal to "0", horizontal blocking information (HBI) and vertical blocking information (VBI) are set to "1" {step 41~). If any coefficient of pixels (~ pixels including the pixels A
and B) belonging to the upper boundary region 3of the block shown in FICi. 3 20 is not equal to "0" (step 420), the VBI is set to "1 ~' (step 430).
Otherwise, the VBI
is set to "0" (seep 440). Also, if any coefficient of pixels {8 pixels including the pixels A and C) belonging to the left boundary region 310 of the block shown in FIG. 3 is not equal to '°0" {step 450), the HBI is.set to "1" (step 460). Otherwise, the HBI is set to "~" (seep 470).
25 After the HBI and I are set, ringing information {ItI) used to filter the ringing noise is generated. t is, if any coe~cient of pixels other than the pixels A, B and C of the block shown in FI(i. 3 is not equal to "0" {step 4~0); the RI is set to °'I" {step 490). Otherwise, the I2T is set to "0" {step 495).
Here, the HBI
and VBI are set to "1'° when only flee coefficient of the pixel A is not equal to "0"
30 (step 400). However, by setting the I and VBI to °' I " even if all coefficients of the pixels A, B and C are not equal to '°0°', favorable effect to some extent can be obtained when a signal adaptive filtering is performed later.

g FIG. S is a flowchart illustrating the step of generating inforrtr,ation used to falter the interframe. If the frame is determined as an interframe by the mode flag checking unit 120, the HBI, ~J'BI and It.I of the intraframe are transferred to IiBI, VBI and RI of the interframe according to the motion vector (step S00). Also, if S a residual signal exists after motion compensation (step 510), the laI is updated (step S20).
~Ihen the blocking information and ringing information for filtering are generated as described above, filtering is adaptively performed according to the information. First, a filtering method for reducing blocking effect will be described. °The filtering for reducing the blocking effect is classified as one of horizontal filtering and vertical filtering. Flere, the horizontal filtering will be explained. FIG. b shows the location of pixels adjacent to the block boundary, for illustrating the filtering for reducing the blocking effect. A determination is made as to whether or not the HBI and R.I of blocks I and J of F$G. 6 are equal to ''0".
iS If the HBI and RI of blocks I and J of FIG. 6 are e~.qual to "~", a weighted filtering is performed on pixels A, B, C, IJ, E and F of FIG. 6 using a 7-tap (1,1,1,2,1,1,1) low-pass filter (I,PF).
If the HBI or the ItI of blocks I and 3 cof FIG. 6 is not equal to "0'°, a filtering is performed on the pixels B, C, I3 and F using the following algorithm.
d=D-C;
If (ABS(d) S Q) {
I~= D - (dl2); C = C + (dl2);
2S if (ABS(d) ~ ) E = E - (dl4);
d=C--°B;
if (ABS(d) <_ ~) B = B + (dl4);
else {
if (ABS(d/2) < 2Q) If (d > 0) {
-. dl2)) 9 C = C + {Q - ABS(d/2));
else {
D = D -t- {Q - ABS(d/2});
S C = C + (Q - ABS(d/2));
d~E_D~
if (ABS(d) ~ Q) E = E - (d/4);
d=C-By IO if (ABS(d) < Q) B = B - (d/4);
In the above algorithm, ABS represents an absolute value, and Q represents a dividend used when the blocks forming the frame are quantized.
IS In detail, the absolute value (ABS{d)) of the difference (d) between the pixels D and C is equal to or less than Q, the current pixel value of the pixel D is set by subtracting dl2 from the current pixel value, and the current pixel value of the pixel C is set by adding dt2 to the current pixel value. Also, the absolute value (ABS(d)) of the difference (d) between the pixels E and D is equal to or less than 20 Q, the current pixel value of the pixels is set by subtracting d~4 from the current pixtl value. Also, if the absolute value (ABS(d)) of the difference (d) between the pixels C and B is equal to or less than Q, the current pixel values of the pixel B is set by subtracting d14 fr~an the current pixel value. In the same ynanner as the above, the pixel values of the pixels B, C, D and E are set according to the 25 algorithm other than the above, which is obvious to those skilled in the art, thus the explanation thereof will be omitted. Also, vertical filtering is performed according to the same principle as the horizontal faltering.
1'lext, a filtering method for reducing ringing noise will be described.
First, the generated ringing inforrnadon is checked. If the ringing information is 'set to 30 "I", the filtering is performed. C,therwise, the filtering is not performed. 'lo end this, edge pixels of the block which have been sub3ected to inverse quantization and IDC'T are determined. hi order to determine flee edge pixels, a gradient operation is performed on the blocks which have been subjected to inverse quantization and IDCT using one-dimensional horizontal and vertical gradient operators.
Then, an absolute value of the difference between one gradient-operated pixel value and the adjacent pixel value, and Q value used as a dividend when 5 quantizing the block, are used to generate a binary edge map representing the edge of each pixel. Here, the block has 8 x8 pixels, and the size of the binary edge map is represented as a two-dimensional array edge[10)~lU] as shown in FIG. 7.
In order to generate the binary edge map, vertical edge detection and horizontal edge detection are performed. Algorithms for vertical edge and 10 horizontal edge detecLions are as follows.
l * Vertical edge detection *!
Al = ABS(Ptrimage[0] - PtrImage[1]);
A2 = (ABS(PtrImage[tI] - Ptrlmage[-1]);
if (({Al > Th)&&{A2 > Th)) ~ ~ (A1 > 5*Th/2) ~ ~ (A2) > 5*Th/2)) ~ge(ml f nl = I ~ l * edge *!
else ~ /~ Horizontal edge detection */
A'1 = ABS(PtrImage[0] - Ptrlmage[width]);
A'2 = (ABS(Ptrlmage[0] - PtrImage[-width]);
if t((A'~'Th)&8~dA'2>'1'h))II(A'1'S"~T'hl2)II~I~'2)'S'rt-hl2)>
Edge[m][n] = 1; / '" edge '~/
For the vertical edge detection, an absolute value (A1) of the difference between the gradient-operated results of the piacel (Ptrlmage[0]) on which a determination as to whether or not the pixel is an edge of the block is made, and the right pixel (PtrImage[I]) of the pixel {Ptrlmage[OJ) is calculate. Then, an °'" ~ absolute value (A2) of the difference between the gradient-operated results of the PtrImage[0] and the left pixel (PtrImage[-1]) of the pixel (PtrImage[0]) is calculated. Then, a determination as to whether or not the pixel is an edge is made acco~~ding to the logic values obtained after the absolute values A1 and A2 are-compared with a predetermined threshold value Th, and then the above process 'is Il performed on all pixels of the block. The vertical edge detection is performed according to a logical formula of (A1 > Th}&&{A2 > Th} [ j {A1 > S*Thl2) [ J
{A2) > S*Th/2). If the logical formula is true, the pixel is determined as a vertical edge. Otherwise, the pixel is determined to not be a vertical edge.
o.
S The horizontal edge detection is performed according to the same principle of the horizontal edge detection. First, an absolute value (A°1) of the difference between the gradient-operated results of the pixel (PtrImagej0]) on which a determination as to whether or not the pixel is edge of the block is nnade, and the Iower pixel (Ftrlmage[width]) of the pixel (Ptrlmage[0]) is calculated. Then, an absolute value (A'2) of the difference between the gradient-operated results of the pixel (PtrImage[0]) and the upper pixel (PtrImage[-width]) of the pixel (PtrIrnagej0]) is calculated. Then, a determination as to whether or not the pixel is an edge is made according to the Iogic values obtained after the absolute values A' I and A'2 are compared with a predetermined threshold value Th, and then the 1 S above process is performed on all pixels of the block. The horizontal edge detection is performed according to a logical formula of (A' 1 > Th)&&(A'2 > Th) ( I (A' 1 > 5 *Th/2) [ ~ (A'2) > S*Th/2). if the logical formula is true, the pixel is determined as a horizontal edge. Otherwise, the pixel is determined to not be a horizontal edge. Here, '&&' represents logical AND, and ' [ ~' represents logical ~I~.
Next, filtering is perfoamed by applying a predetermined filter window to the generated binary edge snap. The filtering may be perfornned by a general filtering method by applying a flier window having a predetermined size.
However, in this embodiment, filtering is not performed if the central pixel of the filter window is an edge, while the filtering is performed if the central pixel is not an edge. The filter window array be a general filter window. In this embodiment, a 4-connectivity filter window having five pixels arranged in a cross shape centering one central pixel, as shown in FIG. 7,, is used. In FIG. 7, ~C
represents an edge pixel, and the regions other than the regions with "X" represent non-edge pixels.
Also, if the filter window has no edge pixel, ordinary filtering is performed, while weighted filtering is performed if the edge pixel exists. An example of weighted filtering is shown in FIG. 7. In FIG. 7, " < < " represents a shift to the left, and " > > " represents a shift to the right.
The invention rnay be embodied in a general purpose digital computer that is running a p<-ogram from a computer usable medium, including but not limited to storage media such as magnetic storage media (e.g., R~M's, floppy disks, hard disks, etc.), optically readable media (e.g., CD-IZOMs, DVDs, etc.) and carrier waves (e.g., transmissions over the Internet). I-Ience, the present invention may be embodied as a computer usable medium having a computer readable program code unit embodied therein for signal adaptive filtering, the coynputer readable program code means in the computer usable medium comprising: computer readable program code means for causing a computer to effect generating blocking information for reducing the blocking effect and ringing information for reducing the ringing noise, from coefficients of predetermined pixels of the upper and left boundary regions of the data block when a frame obtained by deconstructing a i5 bitstream image data for inverse quantization is an intraframe; computer readable program code means for causing a computer to effect setting the blocking information and ringing informatioru of the previous frame corresponding to a motion vector as blocking information and ringing information of the current frame if the frame is an interframe, and setting the ringing information to "I"
which represents the image data requiring filtering if a residual signal of the inverse-quantized current block exists; and computer readable program code means for causing a computer to effect adaptively filtering the image data passed through the inverse quantization and inverse discrete cosine transform based on the block according to the generated blocking information and ringing information, for instance. A functional program, code and code segments, used to implement the present invention can be derived by a skilled computer programmer from the description of the invention contained hereiri_ Industrial An~alicability 3m As described above, the present invention can remove the blocking noise and the ringing noise from an image restored from an image compressed based on blocks, thereby enhancing the image restored from compression.

Claims (2)

CLAIMS:

1. An image data filtering method for reducing blocking effect and noise when a frame of the image data is composed of data blocks of predetermined size, the method comprising:
obtaining flag information indicating whether a mode of a current block is an intra mode or an inter mode, from bitstream image data;
generating filtering information on the current block using a motion vector and a residual signal of the current block when the flag information indicates the mode of the current block is the inter mode;
reconstructing the current block by performing inverse quantization and inverse transform; and filtering the reconstructed current block according to the generated filtering information.

2. An image data filtering apparatus for reducing blocking effect and noise when a frame of the image data is composed of data blocks of predetermined size, the apparatus comprising:
a mode flag checking unit checking flag information indicating whether a mode of a current block is an intra mode or an inter mode, from bitstream image data;
an inter filtering information generating unit coupled to the mode flag checking unit and generating filtering information on the current block using a motion vector and a residual signal of the current block when the flag information indicates the mode of the current block is the inter mode; and an adaptive filtering unit filtering a reconstructed current block reconstructed by inverse quantizing and inverse transforming the current block according to the generated filtering information.