USRE39541E1  Methods and apparatuses for recovering compressed motion picture  Google Patents
Methods and apparatuses for recovering compressed motion picture Download PDFInfo
 Publication number
 USRE39541E1 USRE39541E1 US11081073 US8107305A USRE39541E US RE39541 E1 USRE39541 E1 US RE39541E1 US 11081073 US11081073 US 11081073 US 8107305 A US8107305 A US 8107305A US RE39541 E USRE39541 E US RE39541E
 Authority
 US
 Grant status
 Grant
 Patent type
 Prior art keywords
 pixel
 α
 image
 pixels
 respect
 Prior art date
 Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
 Active
Links
Images
Classifications

 H—ELECTRICITY
 H04—ELECTRIC COMMUNICATION TECHNIQUE
 H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
 H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
 H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
 H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
 H04N19/117—Filters, e.g. for preprocessing or postprocessing

 H—ELECTRICITY
 H04—ELECTRIC COMMUNICATION TECHNIQUE
 H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
 H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
 H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
 H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
 H04N19/136—Incoming video signal characteristics or properties
 H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
 H04N19/139—Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability

 H—ELECTRICITY
 H04—ELECTRIC COMMUNICATION TECHNIQUE
 H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
 H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
 H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
 H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
 H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
 H04N19/159—Prediction type, e.g. intraframe, interframe or bidirectional frame prediction

 H—ELECTRICITY
 H04—ELECTRIC COMMUNICATION TECHNIQUE
 H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
 H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
 H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
 H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
 H04N19/192—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding the adaptation method, adaptation tool or adaptation type being iterative or recursive

 H—ELECTRICITY
 H04—ELECTRIC COMMUNICATION TECHNIQUE
 H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
 H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
 H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using preprocessing or postprocessing specially adapted for video compression
 H04N19/86—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using preprocessing or postprocessing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness
Abstract
Description
Notice: More than one reissue application has been filed for the reissue of U.S. Pat. No. 6,535,643. The reissue applications are applications Ser. Nos. 11/081,073 (the subject application), 11/081,075 and 11/338,905, all of which are divisional reissues of U.S. Pat. No. 6,535,643.
1. Field of the Invention
The present invention relates to an image processing technique, and in particular to a method for recovering a compressed video signal and an apparatus therefor.
2. Description of the Prior Art
The image compression technique of MPEG, MPEG2, H261, H263, etc. is implemented by a Hybrid MC DCT (Motion Compensation Discrete Cosine Transform) technique. This hybrid MC DCT is classified into an encoding process and decoding processes. In the encoding process, the original image is divided into a plurality of blocks for compressing the information of a spacious region, and a twodimensional DCT is performed with respect to each block, and a redundancy is decreased in the image of between the images using a correlation on a time axis between the images for decreasing the information of the time region. In addition, in the decoding process, the reverse sequence of the decoding process is performed. In order to implement the MCDCT technique, an encoder and decoder are required.
While the video signal is being coded, the information may be lost during the quantizing process. Therefore, the video signals reconstructed by the decoder may cause blocking artifacts and ring effects. The block artifacts occur when quantizing a low frequency DCT coefficient, and the ring effects occur due to the information loss of the original video in the quantizing process for a high frequency DCT coefficient.
Namely, in the case of the coding technique using a DCT in a system which is capable of coding a still picture or a motion picture, the entire image is divided into a plurality of small images (for example, 8×8 blocks), and then a transforming operation is performed with respect to the divided blocks, and the original image is processed based on a DCT, and an important information of the original image based on a result of the conversion is included in the low frequency component. As the component becomes high frequency, the important information is decreased. The low frequency component includes an information related to the neighboring block. The DCT transform is performed without considering a correlation between the blocks. Namely, the low frequency components are quantized by the blocks, so that a continuity between the neighboring blocks is lost. This phenomenon is called as the referred to as blocking artifacts.
In addition, when quantizing the coefficients obtained when performing from the DCT operation, as the interval of the quantizing operation is increased, the components to be coded is decreased. Therefore, the number of bits which will be processed is decreased, so that a distortion occurs in the reconstructed original image. This phenomenon is called as the referred to as ring effects. The ring effects, which occur when increasing the intervals of the quantizing operations, are increased at a contour line of an object among in the images.
As a technique for removing the abovedescribed block artifacts and ring effects, a low pass filter technique and a regularization recovering technique are generally used.
The low pass filter sets a filter tap or a filter coefficient based on or by selecting (filter mask) a plurality of pixels near a certain pixel and obtaining an average of the pixels. The recovered images are over smoothed in accordance with the kinds of images, and a compression ratio.
In the regularization recovering method, the block artifacts are adaptively processed in accordance with the statistical characteristic of the images. Namely, a nonuniform information is all computed at all direction boundary areas and in the interior of the block. However, since the computed values have a matrix form, it is impossible to implement a real time computation due to a the large amount of computation. In addition, with an exception except for the amount of nonuniformity, since an average is comprehensively adapted based on a result of the computation of the nonuniform information, in the block having a large amount of nonuniformity, the degree of the nonuniformity is decreased. On the contrary, the degree of the nonuniformity may be increased. Therefore, it is hard to say whether it is well adaptive adapted well to the system.
The abovedescribed two techniques have advantages and disadvantages in view of a complexity and performance increase of the system. Namely, the low pass filter technique has requires less computation amount compared to the regularization recovering technique and has , but has a small capacity for adaptively processing the images, so that the information is lost at an edge portion. The regularization recovering method has an excellent performance and , but requires a large amount of computation when computing regularization parameters.
Accordingly, in the present invention, it is possible to removing a block artifact and ring effect which occur in a decoded video signal.
In addition, it is possible to define a cost function having a directional feature by the unit of pixels during a decoding operation and obtain a regularization parameter based on the cost function.
To achieve the above objects, there is provided a method for recovering a compressed motion picture according to an embodiment of the invention, comprising the steps of defining a cost function having a smoothing degree of an image and a reliability with respect to an original image in consideration of the directional characteristics of the pixels which will be recovered and a plurality of pixels near the recovering pixels, obtaining a regularization parameter variable having a weight value of a reliability with respect to an original image based on the cost function, and approximating the regularization parameter variable using the compressed pixel and obtaining a recovering pixel.
These and other objects of the present application will become more readily apparent from the detailed description given hereinafter. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the invention, are given by way of illustration only, since various changes and modifications within the spirit and scope of the invention will become apparent to those skilled in the art from this detailed description.
Accordingly, in the present invention, it is possible to remove a block artifact and ring effect which occur in a decoded video signal.
In addition, in at least one embodiment, it is possible to define a cost function having a directional feature by the unit of pixels during a decoding operation and obtain a regularization parameter based on the cost function.
One embodiment provides a method for recovering a compressed motion picture, comprising the steps of defining a cost function having a smoothing degree of an image and a reliability with respect to an original image in consideration of the directional characteristics of the pixels which will be recovered and a plurality of pixels near the recovering pixels, obtaining a regularization parameter variable having a weight value of a reliability with respect to an original image based on the cost function, and approximating the regularization parameter variable using the compressed pixel and obtaining a recovering pixel.
At least one embodiment of the present invention also provides a method of filtering an image. In one embodiment, a filtering method is selected from filtering methods having different filtering strengths based on whether a pixel being filtered is in an intracoded portion of an image. For example, a filtering method may be selected based on whether the pixel being filtered is in an intracoded macro block.
Furthermore, in one embodiment, the selected filtering method adjusts a degree of filtering based on a difference value where the difference value is based on the pixel being filtered and a neighboring pixel. For example, the neighboring pixel may be a pixel adjacent to the pixel being filtered.
Additionally, in one embodiment, the selected filtering method filters the pixel based on a quantization parameter used in processing a portion of an image including the pixel.
In yet another embodiment, the selected filtering method includes determining at least one boundary value based on a quantization parameter of a portion of the image including the pixel.
The present invention will become more fully understood from the detailed description given hereinbelow and the accompanying drawings which are given by way of illustration only, and thus as not limitative of the present invention and wherein:
A first embodiment of the present invention will be explained with reference to the accompanying drawings.
In the first embodiment of the present invention, a cost function having a directional feature by the unit of pixels is defined, and a regularization parameter is obtained based on the cost function. A recoverable pixel is obtained using a value which is actually adapted to the regularization parameter and is processed based on a DCT and a projection. Then a resultant data is processed based on a reverse DCT for thereby recovering an image similar to the original image. The abovedescribed operation will be explained in detail.
When the original image f is compressed and transmitted, the image g which is reconstructed by the decoder 201 may be expressed as follows.
g=f+n (1)
where, g, f, and n have a size of MM×1 rearranged in a scanning sequence, and n represents a quantizing difference.
In order to process the original image f by the unit of pixels, the original pixels f(i,j) having a certain position information(i,j) is adapted. The recovered pixel g(i,j) may be expressed using the original pixel(i,j) and a quantizing difference n(i,j) with respect to the original pixel(i,j).
g(i,j)=f(i,j)+n(i,j) (2)
As seen Equation 2, a smoothing which represents a nonuniformity degree with respect to the original pixel f(i,j) and the neighboring pixels of the original pixel f(i,j) and a cost function including includes a reliability with respect to the original pixel f(i,j) and the compressed pixel g(i,j).
First, in order to consider the directional features of four pixels f(i,j+1), f(i+1,j), f(i,j−1), and f(i−1,j) with respect to the original pixel f(i,j), the cost functions of M_{HL }(f(i,j)), M_{HR }(f(i,j)), M_{VT }(f(i,j)), M_{VD }(f(i,j)), M_{T }(f(i,j)) are defined with respect to the neighboring pixels. In order to set a time based region relationship of the original pixel f(i,j), the cost function M_{T }(f(i,j)) is defined. Next, the cost functions of M_{HL }(f(i,j)), M_{HR }(f(i,j)), M_{VT }(f(i,j)), M_{VD }(f(i,j)), M_{T }(f(i,j)) with respect to the neighboring pixels and the cost function M_{T}(f(i,j)) of the time region are summed, so that it is possible to obtain the cost function M(f(i,j)) with respect to the original pixel f(i,j) may be obtained based on as shown in Equation (3).
M(f(i,j))=M_{HL }(f(i,j))+M_{HR}(f(i,j))+M_{VT}(f(i,j))+M_{VD}(f(i,j))+M_{T}(f(i,j)) (3)
where M_{HL }represents a cost function having a relationship between the pixel f(i,j) and the left side neighboring pixel f(i,j−1), M_{HR}(f(i,j)) represents a cost function having a relationship between the pixel f(i,j) and the right side neighboring pixel f(i,j+1), M_{VT}(f(i,j)) represents a cost function having a relationship between the pixel f(i,j) and the upper side neighboring pixel f(i−1,j), M_{VD}(f(i,j)) represents a cost function having a relationship between the pixel f(i,j) and the lower side neighboring pixel f(i+1,j), and M_{T}(f(i,j)) represents a cost function having a relationship of the time region.
The cost function having a smoothing degree and reliability may be expressed as the following equation 4.
M_{HL }(f(i,j))=[f(i,j)−f(i,j−1)]^{2}+α_{HL}[g(i,j)−f(i,j)]^{2 }
M_{HR}(f(i,j))=[f(i,j)−f(i,j+1)]^{2}+α_{HR}[g(i,j)−f(i,j)]^{2 }
M_{VT}(f(i,j))=[f(i,j)−f(i−1,j)]^{2}+α_{VT}[g(i,j)−f(i,j)]^{2 }
M_{VD}(f(i,j))=[f(i,j)−f(i+1,j)]^{2}+α_{VD}[g(i,j)−f(i,j)]^{2 }
MT(f(i,j))=[f(i,j)−f_{MC}(i,j)]^{2}+α_{T}[g(i,j)−f(i,j)]^{2} (4)
As seen in Equation 4, the first term of the right side of each cost function represents a smoothing degree with respect to the original pixel and the neighboring pixel, and the second term of the right side represents a reliability with respect to the original pixel and the recovered pixel.
The first term of the right side of the cost function M_{HL}(f(i,j)) represents a square value of the difference between the original pixel f(i,j) and the left side neighboring pixel f(i,j−1) and represents a uniformity degree, namely, a smoothed degree of the original pixel f(i,j) and the left side neighboring pixel f(i,j−1) based on the error component between the original pixel f(i,j) and the left side neighboring pixel f(i,j−1). In addition, the second term of the right side of the cost function M_{HL}(f(i,j)) represents a square value of the difference between the original pixel f(i,j) and the compressed pixel g(i,j) and represents a value for comparing whether a certain difference exists between the compressed pixel g(i,j) and the original pixel f(i,j) based on a difference component between the original pixel f(i,j) and the compressed pixel g(i,j) and represents a reliability of the original pixel f(i,j) and the compressed pixel g(i,j).
In addition, the first term of the right side of M_{HR}(f(i,j)) represents a smoothing degree of the original pixel f(i,j) and the right side neighboring pixel f(i,j+1), and the second term of the right side represents a reliability of the original pixel f(i,j) and the compressed pixel g(i,j). The first term of the right side of the cost function M_{VT}(f(i,j)) represents a smoothing degree of the original pixel f(i,j) and the upper side neighboring pixel f(i−1,j), and the second term of the right side represents a reliability of the original pixel, and the compressed pixel g(i,j). The first term of the right side of the cost function M_{VT}(f(i,j)) represents a smoothing degree of the original pixel f(i,j) and the lower side neighboring pixel f(i+1,j), and the second term of the right side represents a reliability of the original pixel f(i,j) and the compressed pixel g(i,j). M_{T}(f(i,j)) represents a cost function for setting a relationship of the time region.
The values of α_{HL}, α_{HR}, α_{VT}, α_{VD }α_{T }of the second term of the right side represents a regularization parameter and a ratio of a smoothing degree and reliability. These values represent a different component. In addition, these values represent a weight value with respect to the reliability. As these values are increased, the reliability is enhanced. Since the smoothing degree and the reliability are opposite to each other, the ratio of the smoothing degree and reliability is determined when the regularization parameter is determined. Each regularization parameter may be expressed as the following Equation 5.
In the above Equation 5, the denominators of the aboveequations represents a difference between the original pixel and the compressed pixel, and the numerator represents a difference between the original pixel and the neighboring pixel.
Computation of Recovering Pixels Based on Cost Function
It is needed to obtain the recovering pixels which is the original pixels. However, the cost function includes a square with respect to the original pixel. Therefore, the cost function is partially differentiated with respect to the original pixel, so that it is possible to obtain the original pixels based on the differentiated values. The cost function M(f(i,j)) may be differentiated based on Equation 3.
Each term of the right side of the cost function with respect to the neighboring pixels is as follows.
The values of Equation 7 are substituted for Equation 6, and the pixels which will be finally recovered are in the following Equation 8.
The pixels expressed by Equation 8 are the pixels included in the inter macro block. However, the pixels of the macro block coded into the intra macro type based on Equation 6 is
because there is not a motion information on tile time axis. Therefore, the pixels included in the intra macro block may be expressed in the following Equation 9.
Therefore, the pixels included in the inter macro block are obtained based on Equation 8, and the pixels included in the intra macro block are obtained based on Equation 9. Whether the pixels of the macro block are coded in the intra macro type or in the inter macro type are determined by the intra inter information (p=mtype).
As seen in Equation 8 and 9, the recovering pixels include a regularization parameter α, and each regularization parameter variable is approximated as follows.
Approximation of Regularization Parameter Variable
As seen in Equation 5, each regularization parameter variable includes an original pixel, a neighboring pixel, and a recovering pixel (compressed pixel). In addition, since the original pixel f(i,j) and four neighboring pixels f(i,j−1), f(i,j+1), f(i−1,j), f(i+1,j) are the original pixels, these values do not exist in the decoder. Therefore, the pixels f(i,j), f(i,j−1), f(i,j+1), f(i−1,j), f(i+1,j) may not be used for an actual computation. Therefore, in order to actually use the pixels f(i,j), f(i,j−1), f(i,j+1), f(i−1,j), f(i+1,j), the compressed pixels g(i,j), g(i,j−1), g(i,j+1), g(i−1,j), g(i+1,j) must be approximated. To implement the abovedescribed approximation, the following three cases are assumed.
First, the quantizing maximum difference of the macro block unit is a quantizing variable (Qp).
Second, a quantizing difference of each DCT coefficient is uniformly allocated to each pixel of a corresponding macro block,
Third, the nonuniform values between two pixels of the original image are statistically similar to the nonuniform values between two pixels of the compressed image.
As seen in the following Equation 10, each regularization variable is approximated based on the abovedescribed three cases.

 where 1 represents the 1th macro block, and Qp1 represents a quantizing variable of the 1th macro block. As seen in Equation 10, the difference between the original pixel which is the denominator component of each regularization parameter variable and the compressed pixel is approximated based on the quantizing maximum difference, and the difference between the original pixel which is the numerator component and the compressed pixel is approximated based on the difference with respect to the difference value between the compressed pixel and the neighboring pixel.
The thusly approximated regularization parameter variable is substituted for Equation 8 or 9 for thereby obtaining a result value f(i,j).
As shown therein, in Step ST1, whether the processing pixels are referred to the pixels of the intra macro block or the pixels of the inter macro block is judged. As a result of the judgement, in Steps ST2 and ST3, the regularization parameter variable is obtained. Namely, if the processing pixels are referred to the pixels of the intra macro block, in Step ST2, the regularization parameter variables α_{HL}, α_{HR}, α_{VT}, α_{VDar }are obtained based on Equation 9. In addition, if the processing pixels are referred to the pixels of the inter macro block, the regularization parameter variables α_{HL}, α_{HR}, α_{VT}, α_{VD}, α_{T }are obtained in Step ST3. In addition, the pixel f(i,j) is obtained in Step ST4 based on the obtained regularization parameter variable. At this time, if the processing pixels are referred to the pixels of the inter macro block, and the pixels are obtained based on Equation 8, and if the processing pixels are referred to the pixels of the inter macro block, the pixels are obtained based on Equation9.
Recovering the Images Using a Projection Technique
In Step ST5, a DCT is performed with respect to the pixel f(ij) f(i,j), and then a quantizing process is performed therefor. Here, the DCT coefficient of the pixel f(i,j) may be expressed as F(u,v).
The value G(u,v) which is DCTprocessed with respect to the compressed image g(i,j) may be expressed in the DCT region based on the following Equation 11.
G=QBf (11)

 where B represents a DCT process, and Q represents a quanitizing process.
The DCT coefficient of the original image and the DCT coefficient of the compressed image have the following interrelationship as seen in Equation 12.
G(u,v)−Qp1≦F(u,v)≦G(u,v)+Qp1 (12)

 where G(u,v) represents a (u,u)th value of the twodimensional DCT coefficient of the compressed image, F(u,v) represents a (u,v)th value of the twodimensional DCT coefficient of the original image, Qp1 represents the quantizing maximum difference of the 1th macro block, and each DCT coefficient value represents a subset for setting the range of the DCT coefficient of the recovered images. Therefore, the recovered images must be projected based on the subset of Equation 12, and this process is performed in Step ST6 as seen in the following Equation 13.
P(F(u,v))=G(u,v)−Qp1 if F(u,v)<G(u,v)−Qp1
P(F(u,v))=G(u,v)+Qp1 if F(u,v)>G(u,v)+Qp1
P(F(u,v))=F(u,v) otherwise (13)
 where G(u,v) represents a (u,u)th value of the twodimensional DCT coefficient of the compressed image, F(u,v) represents a (u,v)th value of the twodimensional DCT coefficient of the original image, Qp1 represents the quantizing maximum difference of the 1th macro block, and each DCT coefficient value represents a subset for setting the range of the DCT coefficient of the recovered images. Therefore, the recovered images must be projected based on the subset of Equation 12, and this process is performed in Step ST6 as seen in the following Equation 13.
The Equation 13 will be explained in detail.
If F(u,v) is smaller than G(u,v)−Qp1, the projected recovering image P(F(u,v) is mapped based on G(u,v)−Qp1, and if F(u,v) is larger than G(u,v)−Qp1, the projected recovering image P(F(u,v)) is mapped based on G(u,v)+Qp1, otherwise P(F(u,v)) is directly mapped based on the projected recovering image F(u,v).
The mapped image P(F(u,v)) is reversely DCTprocessed in the spacious region in Step ST7, and the finally recovered image may be expressed by the following Equation 14.
f=B^{T}PBf=B^{T}PBK(g) (14)

 where K(g) represents a computation of the recovering pixels of Equation 8 or 9, BK(g) represents a block DCT coefficient, PBK(g) represents a projected block DCT coefficient, and B^{T}PBK(g) represents that the projected block DCT coefficient is recovered in the spacious region. The recovered image is stored in the image memory and is outputted.
In the present invention, it is possible to eliminate a block artifact and ring effect based on an nonuniform degree and reliability of the recovered image using a plurality of information from the decoder.
Repetition Technique
If the block artifact and ring effect are not fully eliminated from the recovered pixels, he the abovedescribed processes may be repeatedly performed. As the process for eliminating the block artifact and ring effect is repeatedly performed, the block artifact and ring effect of the recovering image is more further eliminated. In this case, a blurring phenomenon occurs in the edge region of the image. Therefore, the number of repetition must be determined based on the block artifact and ring effect and the blurring phenomenon which is opposite thereto.
The recovering image f_{k+1}(i,j) is as follows based on Equations 15 and 16 by repeating the abovedescribed process by ktimes.
The image expressed based on Equation 15 is related to the pixels included in the inter macro block, and the image expressed based on Equation 16 is related to the pixels included in the intra macro block.
The regularization parameter variables are obtained based on Equations 15) 15 and 16, and the DCT is performed with respect thereto, and the projection technique is adapted, and then . Then the reverse DCT is performed therefor, so that the final recovering image is obtained based on Equation 17.
f_{k+1}′=B^{T}PB{circumflex over (f)}=B^{T}PBK(g_{k+1}) (17)
Namely, the block artifact and ring effects are eliminated from the recovered images by an adaptive decoding operation, so that a real time process is implemented in the digital video apparatus. In particular, it is possible to enhance the resolution in the compression images which require a low bit ration ratio or high speed process.
Next, another embodiment of the present invention will be explained.
This embodiment of the present invention is basically directed to decreasing the computation amount and time compared to the earlier embodiment of the present invention. The operation thereof is performed by the recovering apparatus, as shown in
M(f(i,j))=M_{L}(f(i,j))+M_{R}(f(i,j))+M_{U}(f(i,j))+M_{D}(f(i,j)) (18)

 where M_{L }represents a cost function having an interrelationship between the pixel f(i,j) and the left side neighboring pixel f(i,j−1), M_{R}(f(i,j)) represents a cost function having an interrelationship between the pixel f(i,j) and the right side neighboring pixel f(i,j+1), M_{U}(f(i,j)) represents a cost function having an interrelationship between the pixel f(i,j) and the upper side neighboring pixel f(i−1,j), and M_{D}(f(i,j)) is a cost function having an interrelationship between the pixel f(i,j) and the lower side neighboring pixel f(i+1,j).
Next, the cost functions including a smoothing degree and reliability are defined. The regularization parameter variable is included in only the portion (the second term of the right side in Equation 4) of the reliability with respect to the original pixel and recovered pixel. Differently from this construction, in another embodiment of the present invention, the regularization parameter variable is included in the portion which represents a reliability of the original pixel and recovered pixel as well as is included in the portion which represents the smoothing degree with respect to the original pixel and the neighboring pixel. In addition, the smoothing degree and the reliability of the pixel are opposite each other inn in their meaning. Each cost function may be expressed based on Equation 19. Equation 19. as follows:
M_{L}(f(i,j))=α_{L}(f(i,j))[f(i,j)−f(i,j−1)]^{2}+(1−α_{L}(f(i,j)))[g(i,j)−f(i,j)]^{2 }
M_{R}(f(i,j))=α_{R}(f(i,j))[f(i,j)−f(i,j+1)]^{2}+(1−α_{R}(f(i,j)))[g(i,j)−f(i,j)]^{2 }
M_{U}(f(i,j))=α_{U}(f(i,j))[i f(i,j)−f(i−1i,j)]^{2}+(1−α_{U}(f(i,j)))[g(i,j)−f(i,j)]^{2}
M_{U}(f(i,j))=α_{U}(f(i,j))[f(i,j)−f(i−1,j)]^{2}+( 1−α _{U}(f(i,j)))[g(i,j)−f(i,j]^{2}
M_{D}(f(i,j))=α_{D}(f(i,j))[f(i,j)−f(i+1,j)]^{2}+(1−α_{D}(f(i,j)))[g(i,j)−f(i,j)]^{2} (19)
As seen in Equation 19, the first term of the right side represents a smoothing degree with respect to the original pixel and the neighboring pixel, and the second term of the right side represents a reliability with respect to the original pixel and the recovered pixel. Here, α_{L}, α_{R}, α_{U}, α_{D }represent a regularization parameter variable with respect to each cost function and represent a ratio of a smoothing degree and reliability as a difference component. For example, α_{L }represents a weight value with respect to the smoothing degree, and 1−α_{L }represents a weight value with respect to the reliability. Therefore, as the regularization parameter variable is increased, the smoothing degree is increased, and the reliability is decreased. Since the regularization includes the right side first term and the left side term of the cost function, it is possible to implement more stable smoothing degree and reliability compared to the earlier embodiment of the present invention.
Next, in order to obtain the recovering pixel, the cost function is partially differentiated with respect to the original pixel. The thusly differentiated value is obtained by the following Equation 20.
The terms of the right side of Equation 20 are as follows:
When the values expressed based on Equation 21 are substituted for Equation 20, the finally recovered pixels are obtained based on the following Equation 22.
In addition, in the macro type(mtype), the bit value which is defined as COD is included. This COD includes an information of the macro block. If COD value is ‘0’, it means the coded macro block, and if COD value is ‘1’, it means the noncoded macro block (not coded). Namely, it is possible to Recognize recognize whether the pixels of the current macro block are the same as the pixels of the previously transmitted macro block. If COD value is ‘0’, it means that the macro block of the previous compressed image is different from the macro block of the current image, and if COD value is ‘1’, it means that the macro block of the previous image is the same as the macro block of the current image. Therefore, if COD value is ‘0’, the value is recovered based on Equation 22, and if COD value is ‘1’, as seen in Equation 23, the recovered pixel value fp(i,j) is substituted for the current pixel value with respect to the macro block of the previous image.
f(i,j)=fp(i,j) (23)
Next, as seen in Equation 22, the recovering pixel includes a regularization parameter variable α, and each regularization parameter variable is obtained as follows.
The regularization parameter variable is obtained based on Equation 19. Namely, since the smoothing degree and reliability are opposite to each other, the regularization parameter variable may be arranged according to Equation 24 as follows based on a ratio of the smoothing degree and the reliability. Equation 24 may be expressed as follows.
In order to obtain the regularization parameter variable expressed as Equation 24, the pixels f(i,j), f(ij−1), f(i,j+1), f(i−1,j), f(i+1,j) must be approximated based on the compressed pixels g(i,j), g(i,j−1), g(i,j+1), g(i−1,j), g(i+1,j) which may be actually used. For implementing the abovedescribed operation, the following three cases are assumed.
First, a quantization difference of each pixel is a function of a quantization variable Qp which is set by the unit of macro blocks.
Second, since the block artifacts generating at a block boundary has a certain nonuniformity degree which is larger than the ring effect occurring in the interior of the block, the difference with respect to the pixels positioned at the block boundary is more largely reflected compared to the pixels positioned in the interior of the block. Namely, a weight value is provided to the difference based on the position of the pixels.
Equation 24 is approximated to Equation 25 based on the abovedescribed two assumptions.

 where Φ(Qp) is a function of the quantizating variable Qp and is different based on the position of a pixel.
Therefore, with consideration of the position of each pixel in the function Φ(Qp), Φ(Qp) may be expressed as K_{L}QP^{2 }with respect to α_{L}, and Φ(Qp) is expressed as K_{R}Qp^{2 }with respect to α_{R}, and Φ(Qp) is expressed as K_{U}Qp^{2}, with respect to α_{U}, and Φ(Qp) is expressed as K_{D}Qp^{2 }with respect to α_{D}. Here, constants K_{L}, K_{R}, K_{U}, K_{D }are weight values and are different based on whether the neighboring pixel is positioned at the block boundary or in the interior of the block. With consideration to the position of each pixel, type regularization parameter variable is approximated based on the following Equation 26.
Assuming that one block is formed of 8×8 number of pixels, namely, assuming that I and j of f(i,j) is 8, respectively, the weight values K_{L}, K_{R}, K_{U}, K_{D }may be expressed as follows.
K_{L}={9, if j mod 8=0; 1, otherwise}
K_{R}={9, if j mod 8=7; 1, otherwise}
K_{U}={9, if i mod 8=0; 1, otherwise}
K_{D}={9, if i mod 8=7; 1, otherwise}
For example, in the Equation related to K_{L}, if the residual is 0 when dividing j by 8, K_{L }is 9, and otherwise, K_{L }is 1.
When the approximated regularization parameter values are substituted for Equation 22, it is possible to obtain a resultant value f(i,j).
In Step ST10, it is judged whether the pixels of the current macro block are the same as the pixels of the previously transmitted macro block based on the COD value. If they are same, in Step ST11, the recovering pixel values are substituted for the pixel values which are previously recovered based on Equation 23. If they are not the same, in Step ST12, the regularization parameter variables α_{L}, α_{R}, α_{U}, α_{D }are obtained based on Equation 26, and the recovering pixel f(i,j) is obtained based on Equation 22 in Step ST13.
As described above, in the present invention, a certain weight is provided to the regularization parameter variable, which will be approximated, based on the position of the pixels in consideration with the reliability and smoothing degree as well as the regularization parameter variables, so that it is possible to obtain a value which is near the actual pixel value. Therefore, in the present invention, it is not needed to perform a projection method and a repetition method. In addition, in the present invention, the computation amount and time are significantly decreased.
The invention being thus described, it will be obvious that the same may be varied in many ways. Such variations are not to be regarded as a departure from the spirit and scope of the invention, and all such modifications as would be obvious to one skilled in the art are intended to be included within the scope of the following claims invention.
Claims (26)
M_{HL}(f(i,j))=[f(i,j)−f(i,j−1)]^{2}+α_{HL}[g(i,j)−f(i,j)]^{2 }
M_{HR}(f(i,j))=[f(i,j)−f(i,j−1)]^{2}+α_{HR}[g(i,j)−f(i,j)]^{2}M_{HR}(f(i,j))=[f(i,j)−f(i,j+1 )]^{2}+α_{HR} [g(i,j)−f(i,j)]^{2}
M_{VT}(f(i,j))=[f(i,j)−f(i,j−1)]^{2}+α_{VT}[g(i,j)−f(i,j)]^{2}M_{VT}(f(i,j))=[f(i,j)−f(i−1,j)]^{2}+α_{VT} [g(i,j)−f(i,j)]^{2}
M_{VD}(f(i,j))=[f(i,j)−f(i,j+1)]^{2}+α_{VD}[g(i,j)−f(i,j)]^{2}M_{VD}(f(i,j))=[f(i,j)−f(i+1,j)]^{2}+α_{VD} [g(i,j)−f(i,j)]^{2}
M_{T}(f(i,j))=[f(i,j)−f_{MC}(i,j)]^{2}+α_{T}[g(i,j)−f(i,j)]^{2}
M_{L}(f(i,j))=α_{L}(f(i,j))[f(i,j)−f(i,j−1)]^{2}+(1−α_{L}(f(i,j)))[g(i,j)−f(i,j)]^{2 }
M_{R}(f(i,j))=α_{R}(f(i,j))[f(i,j)−f(i,j+1)]^{2}+(1−α_{R}(f(i,j)))[g(i,j)−f(i,j)]^{2 }
M_{U}(f(i,j))=α_{U}(f(i,j))[f(i,j)−f(i−1,j)]^{2}+(1−α_{U}(f(i,j)))[g(i,j)−f(i,j)]^{2 }
M_{D}(f(i,j))=α_{D}(f(i,j))[f(i,j)−f(i−1,j)]^{2}+(1−α_{D}(f(i,j)))[g(i,j)−f(i,j)]^{2}
M_{D}(f(i,j))=α _{D}(f(i,j))[f(i,j)−f(i+1,j)]^{2} +( 1−α _{D}(f(i,j)))[g(i,j)−f(i,j)]^{2}
Priority Applications (4)
Application Number  Priority Date  Filing Date  Title 

KR19980046895A KR100287529B1 (en)  19981103  Method the developing block and the ring Reduction of the video signal  
KR19990028137A KR100304897B1 (en)  19990713  19990713  Method of removal block effect and ringing effect for real time peocessing of compressed video signal 
US09430747 US6535643B1 (en)  19981103  19991029  Method for recovering compressed motion picture for eliminating blocking artifacts and ring effects and apparatus therefor 
US11081073 USRE39541E1 (en)  19981103  20050316  Methods and apparatuses for recovering compressed motion picture 
Applications Claiming Priority (1)
Application Number  Priority Date  Filing Date  Title 

US11081073 USRE39541E1 (en)  19981103  20050316  Methods and apparatuses for recovering compressed motion picture 
Related Parent Applications (1)
Application Number  Title  Priority Date  Filing Date  

US09430747 Reissue US6535643B1 (en)  19981103  19991029  Method for recovering compressed motion picture for eliminating blocking artifacts and ring effects and apparatus therefor 
Publications (1)
Publication Number  Publication Date 

USRE39541E1 true USRE39541E1 (en)  20070403 
Family
ID=26634279
Family Applications (4)
Application Number  Title  Priority Date  Filing Date 

US09430747 Active US6535643B1 (en)  19981103  19991029  Method for recovering compressed motion picture for eliminating blocking artifacts and ring effects and apparatus therefor 
US11081073 Active USRE39541E1 (en)  19981103  20050316  Methods and apparatuses for recovering compressed motion picture 
US11081075 Active USRE42716E1 (en)  19981103  20050316  Method for filtering an image 
US11338905 Active USRE42677E1 (en)  19981103  20060125  Method for filtering an image 
Family Applications Before (1)
Application Number  Title  Priority Date  Filing Date 

US09430747 Active US6535643B1 (en)  19981103  19991029  Method for recovering compressed motion picture for eliminating blocking artifacts and ring effects and apparatus therefor 
Family Applications After (2)
Application Number  Title  Priority Date  Filing Date 

US11081075 Active USRE42716E1 (en)  19981103  20050316  Method for filtering an image 
US11338905 Active USRE42677E1 (en)  19981103  20060125  Method for filtering an image 
Country Status (2)
Country  Link 

US (4)  US6535643B1 (en) 
JP (1)  JP3710974B2 (en) 
Families Citing this family (9)
Publication number  Priority date  Publication date  Assignee  Title 

US6594400B1 (en)  19980831  20030715  Lg Electronics Inc.  Method of removing block phenomenon and ring phenomenon in compression coded image 
KR100335055B1 (en) *  19991208  20020502  구자홍  Method of removal block effect and ringing effect of compressed video signal 
KR100525785B1 (en)  20010615  20051103  엘지전자 주식회사  Filtering method for pixel of image 
EP2899977A1 (en) *  20020131  20150729  Samsung Electronics Co., Ltd  Filtering method and apparatus for reducing block artifacts or ringing noise 
US6898328B2 (en) *  20021023  20050524  Sony Corporation  Method and apparatus for adaptive pixel estimation under high error rate conditions 
US7643679B2 (en) *  20030213  20100105  Ati Technologies Ulc  Method and apparatus for block based image compression with multiple nonuniform block encodings 
KR100936034B1 (en)  20030811  20100111  삼성전자주식회사  Deblocking method for blockcoded digital images and display playback device thereof 
CN101911704A (en) *  20080114  20101208  汤姆森许可贸易公司  Methods and apparatus for deartifact filtering using multilattice sparsitybased filtering 
US8821641B2 (en)  20110930  20140902  Samsung Electronics Co., Ltd.  Nozzle unit, and apparatus and method for treating substrate with the same 
Citations (22)
Publication number  Priority date  Publication date  Assignee  Title 

US5283646A (en)  19920409  19940201  Picturetel Corporation  Quantizer control method and apparatus 
US5488570A (en)  19931124  19960130  Intel Corporation  Encoding and decoding video signals using adaptive filter switching criteria 
US5563813A (en) *  19940601  19961008  Industrial Technology Research Institute  Area/timeefficient motion estimation micro core 
US5611000A (en) *  19940222  19970311  Digital Equipment Corporation  Splinebased image registration 
US5748795A (en)  19950207  19980505  Canon Kabushiki Kaisha  Image decoder using adjustable filtering 
US5790131A (en) *  19960515  19980804  Iterated Systems, Inc.  System and method for lossy compression of data with output file size control 
WO1999004497A2 (en)  19970716  19990128  Samsung Electronics Co., Ltd.  Signal adaptive filtering method, signal adaptive filter and computer readable medium for storing program therefor 
US5878166A (en) *  19951226  19990302  CCube Microsystems  Field frame macroblock encoding decision 
US5940536A (en)  19950905  19990817  Matsushita Electric Industrial Co., Ltd.  Ringing detector and filter 
US6041145A (en)  19951102  20000321  Matsushita Electric Industrial Co., Ltd.  Device and method for smoothing picture signal, device and method for encoding picture and device and method for decoding picture 
US6058210A (en) *  19970915  20000502  Xerox Corporation  Using encoding cost data for segmentation of compressed image sequences 
US6108455A (en)  19980529  20000822  Stmicroelectronics, Inc.  Nonlinear image filter for filtering noise 
US6178205B1 (en)  19971212  20010123  Vtel Corporation  Video postfiltering with motioncompensated temporal filtering and/or spatialadaptive filtering 
US6195632B1 (en) *  19981125  20010227  Matsushita Electric Industrial Co., Ltd.  Extracting formantbased sourcefilter data for coding and synthesis employing cost function and inverse filtering 
US6226050B1 (en)  19970404  20010501  Samsung Electronics Co., Ltd.  Signal adaptive filtering method for reducing ringing noise and signal adaptive filter 
US6259823B1 (en)  19970215  20010710  Samsung Electronics Co., Ltd.  Signal adaptive filtering method and signal adaptive filter for reducing blocking effect and ringing noise 
US6385245B1 (en) *  19970923  20020507  Us Philips Corporation  Motion estimation and motioncompensated interpolition 
US6529638B1 (en)  19990201  20030304  Sharp Laboratories Of America, Inc.  Block boundary artifact reduction for blockbased image compression 
US6594400B1 (en)  19980831  20030715  Lg Electronics Inc.  Method of removing block phenomenon and ring phenomenon in compression coded image 
US6665346B1 (en)  19980801  20031216  Samsung Electronics Co., Ltd.  Loopfiltering method for image data and apparatus therefor 
US20050147319A1 (en)  20040106  20050707  Sharp Laboratories Of America, Inc.  System and method for removing ringing artifacts 
US20050201633A1 (en)  20040311  20050915  Daeyang Foundation  Method, medium, and filter removing a blocking effect 
Family Cites Families (2)
Publication number  Priority date  Publication date  Assignee  Title 

JP3095140B2 (en) *  19970310  20001003  三星電子株式会社  Dimensional signal adaptive filter and filtering method for reducing blocking effect 
KR100525785B1 (en)  20010615  20051103  엘지전자 주식회사  Filtering method for pixel of image 
Patent Citations (23)
Publication number  Priority date  Publication date  Assignee  Title 

US5283646A (en)  19920409  19940201  Picturetel Corporation  Quantizer control method and apparatus 
US5488570A (en)  19931124  19960130  Intel Corporation  Encoding and decoding video signals using adaptive filter switching criteria 
US5611000A (en) *  19940222  19970311  Digital Equipment Corporation  Splinebased image registration 
US5563813A (en) *  19940601  19961008  Industrial Technology Research Institute  Area/timeefficient motion estimation micro core 
US5748795A (en)  19950207  19980505  Canon Kabushiki Kaisha  Image decoder using adjustable filtering 
US5940536A (en)  19950905  19990817  Matsushita Electric Industrial Co., Ltd.  Ringing detector and filter 
US6041145A (en)  19951102  20000321  Matsushita Electric Industrial Co., Ltd.  Device and method for smoothing picture signal, device and method for encoding picture and device and method for decoding picture 
US5878166A (en) *  19951226  19990302  CCube Microsystems  Field frame macroblock encoding decision 
US5790131A (en) *  19960515  19980804  Iterated Systems, Inc.  System and method for lossy compression of data with output file size control 
US6259823B1 (en)  19970215  20010710  Samsung Electronics Co., Ltd.  Signal adaptive filtering method and signal adaptive filter for reducing blocking effect and ringing noise 
US6226050B1 (en)  19970404  20010501  Samsung Electronics Co., Ltd.  Signal adaptive filtering method for reducing ringing noise and signal adaptive filter 
WO1999004497A2 (en)  19970716  19990128  Samsung Electronics Co., Ltd.  Signal adaptive filtering method, signal adaptive filter and computer readable medium for storing program therefor 
US6631162B1 (en)  19970716  20031007  Samsung Electronics Co., Ltd.  Signal adaptive filtering method, signal adaptive filter and computer readable medium for storing program therefor 
US6058210A (en) *  19970915  20000502  Xerox Corporation  Using encoding cost data for segmentation of compressed image sequences 
US6385245B1 (en) *  19970923  20020507  Us Philips Corporation  Motion estimation and motioncompensated interpolition 
US6178205B1 (en)  19971212  20010123  Vtel Corporation  Video postfiltering with motioncompensated temporal filtering and/or spatialadaptive filtering 
US6108455A (en)  19980529  20000822  Stmicroelectronics, Inc.  Nonlinear image filter for filtering noise 
US6665346B1 (en)  19980801  20031216  Samsung Electronics Co., Ltd.  Loopfiltering method for image data and apparatus therefor 
US6594400B1 (en)  19980831  20030715  Lg Electronics Inc.  Method of removing block phenomenon and ring phenomenon in compression coded image 
US6195632B1 (en) *  19981125  20010227  Matsushita Electric Industrial Co., Ltd.  Extracting formantbased sourcefilter data for coding and synthesis employing cost function and inverse filtering 
US6529638B1 (en)  19990201  20030304  Sharp Laboratories Of America, Inc.  Block boundary artifact reduction for blockbased image compression 
US20050147319A1 (en)  20040106  20050707  Sharp Laboratories Of America, Inc.  System and method for removing ringing artifacts 
US20050201633A1 (en)  20040311  20050915  Daeyang Foundation  Method, medium, and filter removing a blocking effect 
NonPatent Citations (4)
Title 

"Iterative Procedures for Reduction of Blocking Effects in Transform Image Coding", Zakhor, IEEE Transactions on Circuits and Systems for Video Technology, vol. 2, No. 1, IEEE Mar. 1993, pp. 9195. * 
"Iterative Projection Algorithms For Removing The Blocking Artifacts Of BockDCT Compressed Images", Yang et al., IEEE 1993, pp. V405V408. * 
Korean Office Action dated Jul. 18, 2005, with English Translation. 
Pang, Khee K. et al. "Optimum Loop Filter in Hybrid Coolers." IEEE Circuit and Systems For Video Technology, vol. 4, No. 2, Apr. 1994, pp. 158167. 
Also Published As
Publication number  Publication date  Type 

USRE42677E1 (en)  20110906  grant 
JP2000152241A (en)  20000530  application 
JP3710974B2 (en)  20051026  grant 
US6535643B1 (en)  20030318  grant 
USRE42716E1 (en)  20110920  grant 
Similar Documents
Publication  Publication Date  Title 

US5146326A (en)  Coded picture information decoding apparatus having means for improving picture distortion  
US5731836A (en)  Method of video encoding by accumulated error processing and encoder therefor  
US5610729A (en)  Noise reduction apparatus for reducing noise in decoding of moving picture sequence  
US6198848B1 (en)  Method and apparatus for compressing and storing data indicative of a fullcolor image  
US5218435A (en)  Digital advanced television systems  
US6148027A (en)  Method and apparatus for performing hierarchical motion estimation using nonlinear pyramid  
US5974197A (en)  Loop filter and loop filtering method  
US6246719B1 (en)  Temporal tile staggering for block based video compression  
US5287200A (en)  Block adaptive linear predictive coding with multidimensional adaptive gain and bias  
US6037986A (en)  Video preprocessing method and apparatus with selective filtering based on motion detection  
US6658157B1 (en)  Method and apparatus for converting image information  
US6728414B1 (en)  Deblocking method and apparatus  
US5479211A (en)  Imagesignal decoding apparatus  
US7027507B2 (en)  Movingpicture coding and decoding method and apparatus with reduced computational cost  
US6104434A (en)  Video coding apparatus and decoding apparatus  
US7262886B2 (en)  Method of reducing a blocking artifact when coding moving picture  
US6480670B1 (en)  Video signal encoding method and system  
US5025482A (en)  Image transformation coding device with adaptive quantization characteristic selection  
US5565921A (en)  Motionadaptive image signal processing system  
US6985635B2 (en)  System and method for providing a singlelayer video encoded bitstreams suitable for reducedcomplexity decoding  
US5546129A (en)  Method for encoding a video signal using feature point based motion estimation  
US5757969A (en)  Method for removing a blocking effect for use in a video signal decoding apparatus  
US5892548A (en)  Adaptive quantizer with modification of high frequency coefficients  
US5253075A (en)  Image signal coding/decoding system using adaptive quantization  
US7349473B2 (en)  Method and system for selecting interpolation filter type in video coding 
Legal Events
Date  Code  Title  Description 

FPAY  Fee payment 
Year of fee payment: 8 