WO2002069261A1 - Procede et appareil permettant de traiter une image numerique, dispositif de codage d'images et dispositif de decodage d'images - Google Patents

Procede et appareil permettant de traiter une image numerique, dispositif de codage d'images et dispositif de decodage d'images Download PDF

Info

Publication number
WO2002069261A1
WO2002069261A1 PCT/SG2001/000036 SG0100036W WO02069261A1 WO 2002069261 A1 WO2002069261 A1 WO 2002069261A1 SG 0100036 W SG0100036 W SG 0100036W WO 02069261 A1 WO02069261 A1 WO 02069261A1
Authority
WO
WIPO (PCT)
Prior art keywords
picture
points
function
coding information
post
Prior art date
Application number
PCT/SG2001/000036
Other languages
English (en)
Inventor
Susu Yao
Original Assignee
Nanyang Technological University, Centre For Signal Processing School Of Electrical And Electronic Engineering
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanyang Technological University, Centre For Signal Processing School Of Electrical And Electronic Engineering filed Critical Nanyang Technological University, Centre For Signal Processing School Of Electrical And Electronic Engineering
Priority to PCT/SG2001/000036 priority Critical patent/WO2002069261A1/fr
Publication of WO2002069261A1 publication Critical patent/WO2002069261A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/527—Global motion vector estimation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20004—Adaptive image processing
    • G06T2207/20012—Locally adaptive
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20172—Image enhancement details
    • G06T2207/20192—Edge enhancement; Edge preservation

Definitions

  • the invention relates to a method and an apparatus for processing a digital picture, a picture encoding device and a picture decoding device.
  • the most noticeable artefact after the decoding operation is the blocking artefact that is caused by independent block processing in a picture encoding device such as a picture encoding device encoding a picture according to a JPEG or an MPEG standard.
  • a ringing artefact is another annoying artefact that often appears in a wavelet and subband based compression scheme such as the compression scheme according to the JPEG 2000 scheme.
  • the coding artefacts lead to a noticeable deterioration in the subjective quality of the decoded picture that increases with increasing compression rate of the picture data.
  • [3] discloses the decoding process as a maximum a posterior (MAP) estimation problem, which results in a convex optimization problem that is solved by gradient descent search iteratively.
  • MAP maximum a posterior
  • POCS projection onto convex sets
  • the object is achieved with a method and an apparatus for processing a digital picture, a picture encoding device and a picture decoding device with the features according to the independent claims .
  • an iterative filtering for at least a part of the picture points of the picture using a regularization function is applied.
  • the filtering process which comprises an arbitrary number of iterations, in each iteration, the filtering is performed for two picture points respectively.
  • the regularization function includes a term that ensures the fidelity of the picture and a smoothness constraint .
  • An apparatus for processing a digital picture comprising picture points, wherein coding information is assigned to the picture points comprises a processing unit which is adapted to perform the steps described above.
  • a picture encoding device for encoding a digital picture comprising picture points, wherein coding information is assigned to the picture points comprises a picture encoding unit for encoding the digital picture and a picture decoding unit for decoding the encoded picture, for example in an encoding scheme for encoding moving pictures, e.g. according to MPEG1, MPEG2, MPEG-4, H.263, H.263+ or H.263L.
  • the picture encoding device comprises a postprocessing unit for post-processing the decoded picture, the post-processing unit being adapted to perform the following steps :
  • the regularization function includes a term that ensures the fidelity of the picture and a smoothness constraint .
  • a picture decoding device for decoding an encoded digital picture comprising picture points, wherein coding information is assigned to the picture points, comprises - a picture decoding unit for decoding the encoded picture, - a post-processing unit for post-processing the decoded picture, the post-processing unit being adapted to perform the following steps:
  • the regularization function includes a term that ensures the fidelity of the picture and a smoothness constraint .
  • the invention may be seen in that the post-processing is formulated as a picture restoration and estimation problem, which leads to a constrained optimization problem.
  • the constraints are expressed through a regularization function, which is also called the potential function in Markov random field approach.
  • a non-quadratic potential function is preferably used.
  • the regularization function is a half-quadratic regularization function .
  • the regularization function includes an edge-preserving constraint .
  • the regularization function is
  • ⁇ (x, y) ⁇ _( ⁇ , y) + ⁇ r (x) with
  • ⁇ (x, y) being the term that ensures the fidelity of the picture
  • • ⁇ is a potential real function having edge-preserving properties
  • • i is a first index for denoting a place in the picture array in a first direction
  • • j is a second index for denoting a place in the picture array in a second direction, which second direction is different from the first direction
  • • S is the amount of all picture points in the picture array.
  • the regularization function preferably comprises an auxiliary variable b such that the regularization function is an infimum function according to the following formula
  • ⁇ (x, y, b) being a quadratic function in x when the auxiliary variable b is fixed.
  • the quadratic function ⁇ (x, y, b) may result fr om
  • auxiliary variables bj_ j and the estimated coding information x are optimised in each iteration alternately, wherein
  • the quadratic function ⁇ (x, y, bj is solved using a divide and conquer process for decomposing and recomposing the quadratic function ⁇ (x, y, b) .
  • the estimated coding information i,j at the place (i,j) in the picture array and XJ_ + ] _ at the place (i,j+l) in the picture array is determined using the following formulas:
  • x i,j+l Yi,j+1 + ⁇ i,j(yi,j - V i,j+l)'
  • the first direction and the second direction are perpendicular to each other .
  • a third direction being angled to the first direction with an angle of 45° and a fourth direction being angled to the second direction with an angle of 45° are taken into account.
  • a third direction first order difference operator D- -x and a fourth direction first order difference operator D. .x are used,
  • D. -x is determined according to the following formula:
  • D- -x is determined according to the following formulc - ⁇ J
  • the picture is encoded using a wavelet transformation.
  • Figure 1 shows a flow chart of the post-processing on the decompressed picture according to the preferred embodiment of the invention
  • Figure 2 shows a flow chart of the line processing along a horizontal direction according to the preferred embodiment of the invention
  • Figure 3 shows a block diagram illustrating a video encoder according to a preferred embodiment of the invention
  • Figure 4 shows the block diagram of the pixel-based processing unit according to a preferred embodiment of the invention.
  • Fig.3 shows a video encoder 300 for encoding a digital video sequence 301.
  • the digital video sequence 301 is a sequence of subsequent digital pictures 302 (digital frames) comprising picture elements .
  • each picture 302 is grouped in picture blocks, each picture block comprising 8 x 8 picture elements or 16 x 16 picture elements.
  • the invention is not restricted to a specific form of picture blocks, the picture elements are grouped in, in particular, the invention is not restricted to a specific number of picture elements in a picture block.
  • each picture 302 comprises 16 x 16 macroblocks .
  • Luminance information and chrominance information is assigned to one picture element or to a plurality of picture element (in the case of sub-sampling) as coding information.
  • each picture 302 is selected and it is determined whether it is to be coded in a so called INTER- mode or the so called INTRA-mode.
  • a motion estimation is provided in a motion estimation unit 304 with reference to a reference picture.
  • the reference picture may be a preceding picture or a following picture of the first picture in the video sequence 301.
  • a motion vector describing the displacement of the macroblock in the current picture compared with the selected best matched macroblock in the reference picture is determined.
  • the coding mode 305 for the selected picture 301 and the determined motion vectors 306 are provided to and stored in a picture storage element 307.
  • the motion estimation will further be described in detail.
  • the data 308 in the macroblocks (the coding information) is provided to a subtracting unit 309, where the data 310 of the macroblock from the reference picture, which macroblock has been matched and selected according to the motion estimation is subtracted, thereby generating motion estimation error coefficients 311.
  • the INTRA-coding mode no subtraction and no motion estimation is provided.
  • the entire luminance information and the entire chrominance information of the selected picture to be coded is coded, not only a differential signal as in the INTER-coding mode.
  • INTRA-coefficients 311 are output by the subtracting unit 309.
  • the motion estimation error coefficients 311 or the INTRA coefficients 311 are provided to a transformation unit 312.
  • a discrete cosine transformation is provided to the input motion estimation error coefficients 311 or the INTRA coefficients 311, thereby generating transformation coefficients 313, which are provided to a quantizing unit 314.
  • any suitable type of transformation for generating the transformation coefficients 313 can be used according to the invention, e.g. the wavelet transformation.
  • the transformation coefficients 313 are quantized and scanned, thereby generating quantized transformation coefficients 315, which are provided to a entropy coding unit 316 and to an inverse quantizing unit 318.
  • a variable length encoding such as a Lempel-Ziv-Encoding or a Huffman encoding is performed on the quantized transformation coefficients 315.
  • a run length encoding algorithm is performed on the variable length encoded quantized transformation coefficients .
  • the resulting coded data stream 317 is output by the video encoder 300 and, for example, transmitted to a video decoder unit .
  • the quantized transformation coefficients 315 are inversely quantized, thereby generating inverse quantized transformation coefficients 319, which are provided to an inverse transformation unit 320.
  • an inverse discrete cosine transformation generally the respective inverse transformation to the transformation used for generating the transformation coefficients, e.g. the inverse wavelet transformation, is performed on the inverse quantized transformation coefficients 319, thereby generating inverse transformed coefficients 321, which are provided to a first input 322 of an adder 323.
  • IDCT inverse discrete cosine transformation
  • the data 310 of the macroblock from the reference picture, which macroblock has been matched and selected according to the provided motion estimation is provided to a second input 324 of the adder 323, thus being added to the inverse transformed coefficients 321 in the INTER-mode, thereby generating a reconstructed signal 325.
  • the reconstructed signal 325 output by the adder 323 is stored in the picture storage element 307.
  • the thus reconstructed picture stored in the picture storage element 307 is further provided to a post-processing unit 326 as a picture signal 327.
  • the post-processing unit 326 performs a post-processing on the picture signal 327 as will be later described in detail.
  • the resulting post-processed picture signal 328 is provided to and stored in the picture storage element 307 as a post- processed picture 329.
  • the invention may be applied to any other encoding architecture and decoding architecture, for example in an object-oriented encoding scheme like MPEG4 or in a hybrid block-based encoding scheme and object-oriented encoding scheme, a still picture encoding scheme and decoding scheme such as the JPEG scheme or the JPEG 2000 scheme. It should be noted that the described post-processing can be applied to the decoded pictures without altering the coder and/or decoder scheme and requiring any aside information.
  • One aspect of the invention may be seen in that an edge- preserving and smoothness constraint is imposed on the picture points while retaining fidelity to the picture.
  • auxiliary variable in the regularization function permits us to linearize the problem and to derive a deterministic method based on alternative minimizations on the picture variable and the auxiliary variable.
  • the basic idea of the DAC strategy is to divide a large problem into pieces, to solve the divided sub-problems separately and then to combine the sub-solutions into a full solution .
  • a picture is regarded as a degraded version from a blurred and noisy observation.
  • the artefact reduction problem is considered as an estimation problem, i.e. to estimate or recover a 2-D signal from a noisy signal, which can be represented as follows:
  • y is a given picture array and ⁇ is a mixture of outliers and the nominal noise distribution.
  • the goal of artefact reduction is to estimate the "ideal" picture, further denoted as the picture to be estimated x, from the given picture array y .
  • the pictures are considered as lexicographically ordered vectors.
  • ⁇ is a white Gaussian noise
  • the restored picture is the maximum a posteriori (MAP) estimation.
  • MAP maximum a posteriori
  • ⁇ (x, y) ⁇ (x,y)+ ⁇ • ⁇ r (x) : 3 )
  • ⁇ r (x) is a roughness penalty regularization, which is expressed as the following formula:
  • the second term ⁇ r (x) imposes a smoothness constraints or roughness penalty.
  • the first term ⁇ (x, y) ensures fidelity to the picture, ⁇ is a positive parameter that balances these two terms.
  • the picture points are processed with an edge preservation.
  • ⁇ '(x, y) is the derivative of ⁇ (x, y) .
  • Equation (7) can be written as
  • T is a matrix that represents a weighted discrete approximation of the Laplacian operator.
  • the weights of the Laplacian operator are given by the function _1_ _ d ⁇ (t) 2t dt
  • the corresponding weights vanish and thus the edge is preserved. Also, it seems natural that there should be a one-to-one correspondence between values of the gradient and values of the weight. This requires that the weighting function must be strictly monotonous on [0,+ ⁇ ) . There are many functions that satisfy those .conditions. In this preferred embodiment of the invention, the following function is chosen as the potential function.
  • the MAP criterion can be written as the minimum of a dual energy
  • ⁇ (x, y, b) is a quadratic function when the auxiliary variable b is fixed.
  • the minimum value of the auxiliary variable b is unique and is given by the following formula:
  • minimizing the MAP energy is equivalent to minimizing the augmented energy with the auxiliary variable b .
  • x n is fixed
  • b n is computed using equation (12) .
  • the complete post-processing includes a line process along + 45° and -45° directions, i.e. diagonal directions.
  • Fig.l shows the separate steps for the post-processing on the decompressed picture in summary.
  • the picture data is input into the post-processing unit .326.
  • a first post-processing is performed along the horizontal direction of the picture array.
  • a second postprocessing is performed along the horizontal direction of the picture array.
  • a third post-processing is performed again along the +45° direction of the picture array and in a fourth and last stage (block 400) , a fourth post-processing is performed along the -45° direction of the picture array.
  • the post-processed picture data is output by the post-processing unit 326.
  • Fig.2 shows an example of the post-processing along the horizontal direction of the picture array.
  • indices i, j for denoting the line and the column of the picture array are initialised to a value "0", respectively.
  • step 102 two adjacent image picture points are input sequentially for post-processing into the post-processing unit 326.
  • a pixel-based processing in other words, a pixel-based filtering is performed.
  • a pixel-based filtering is performed.
  • step 1014 it is determined whether
  • the column index j is increased by 2 (step 105) , i.e.
  • the post-processing is performed for the next two consecutive picture points of the current row i.
  • step 103 the pixel-based processing (step 103) is again performed with an altered initial column index j .
  • step 107 it is determined whether
  • the post-processing is performed for the next two consecutive picture points of the current row i.
  • step 110 it is further determined (step 110) whether the iteration has been finished.
  • the method goes on to the second stage (stage 200).
  • the second stage (stage 200), the third stage (stage 300) and the fourth stage (stage 400) include the same steps compared to the first stage (stage 100) .
  • the row index and the column index is exchanged in the respective steps, i.e. for example, that the row index i is increased by 2 in the respective steps and the post-processing is performed column by column instead of the post-processing row by row according to the first stage (stage 100) .
  • Fig.4 shows the filtering step 103 in more detail.
  • the filtering step is an implementation of formula (15) described above .
  • Two adjacent picture points (with coding information Y r j , Y ⁇ , j +l ) are read from the picture storage medium 307 (step 401) .
  • the first difference (y ⁇ j + i - Y ⁇ , j ) is multiplied by the weighting coefficient ⁇ - j 403 resulting in the term [ ⁇ lf -, (y lf -,+ ⁇ - y X/ )].
  • step 406 the post-process picture points are stored in the picture storage medium 307.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

L'invention concerne un procédé permettant de traiter une image numérique comprenant des points d'image, les informations codées étant affectées aux points d'image. Ce procédé consiste: - à appliquer un filtrage itératif à au moins quelques points de l'image par une fonction de régularisation. Dans chaque itération, le filtrage est effectué respectivement pour deux points d'image et la fonction de régularisation comporte un terme qui assure la fidélité de l'image et une contrainte lisse.
PCT/SG2001/000036 2001-02-27 2001-02-27 Procede et appareil permettant de traiter une image numerique, dispositif de codage d'images et dispositif de decodage d'images WO2002069261A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/SG2001/000036 WO2002069261A1 (fr) 2001-02-27 2001-02-27 Procede et appareil permettant de traiter une image numerique, dispositif de codage d'images et dispositif de decodage d'images

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/SG2001/000036 WO2002069261A1 (fr) 2001-02-27 2001-02-27 Procede et appareil permettant de traiter une image numerique, dispositif de codage d'images et dispositif de decodage d'images

Publications (1)

Publication Number Publication Date
WO2002069261A1 true WO2002069261A1 (fr) 2002-09-06

Family

ID=20428913

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SG2001/000036 WO2002069261A1 (fr) 2001-02-27 2001-02-27 Procede et appareil permettant de traiter une image numerique, dispositif de codage d'images et dispositif de decodage d'images

Country Status (1)

Country Link
WO (1) WO2002069261A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1991012590A1 (fr) * 1990-02-09 1991-08-22 Aware, Inc. Appareil et procede ameliores de representation d'une image
US5343309A (en) * 1992-08-12 1994-08-30 Xerox Corporation Image processing system and method employing adaptive filtering to provide improved reconstruction of continuous tone images from halftone images including those without a screen structure
US5359676A (en) * 1993-07-19 1994-10-25 Xerox Corporation Decompression of standard ADCT-compressed document images
US5521718A (en) * 1994-12-05 1996-05-28 Xerox Corporation Efficient iterative decompression of standard ADCT-compressed images
EP0629080B1 (fr) * 1993-06-09 2000-02-23 STMicroelectronics S.r.l. Méthode adaptive pour supprimer les échos d'un signal vidéo

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1991012590A1 (fr) * 1990-02-09 1991-08-22 Aware, Inc. Appareil et procede ameliores de representation d'une image
US5343309A (en) * 1992-08-12 1994-08-30 Xerox Corporation Image processing system and method employing adaptive filtering to provide improved reconstruction of continuous tone images from halftone images including those without a screen structure
EP0629080B1 (fr) * 1993-06-09 2000-02-23 STMicroelectronics S.r.l. Méthode adaptive pour supprimer les échos d'un signal vidéo
US5359676A (en) * 1993-07-19 1994-10-25 Xerox Corporation Decompression of standard ADCT-compressed document images
US5521718A (en) * 1994-12-05 1996-05-28 Xerox Corporation Efficient iterative decompression of standard ADCT-compressed images

Similar Documents

Publication Publication Date Title
Ozcelik et al. Image and video compression algorithms based on recovery techniques using mean field annealing
CN100364338C (zh) 估计图像噪声的方法和设备和消除噪声的方法
US5731836A (en) Method of video encoding by accumulated error processing and encoder therefor
KR0165497B1 (ko) 블럭화현상 제거를 위한 후처리장치 및 그 방법
JP5259396B2 (ja) ハイブリッド映像圧縮のための非線形予測フィルタ
KR20010089526A (ko) 화상 경계에 걸친 화소값들의 차이를 감소시키기 위한화상 처리회로 및 방법
EP0680219B1 (fr) Méthode améliorée de post-traitement pour être utilisée dans un dispositif de décodage de signal d'image
GB2287153A (en) Noise reduction apparatus
US20010003545A1 (en) Method for eliminating blocking effect in compressed video signal
US20040228406A1 (en) Advanced DCT-based video encoding method and apparatus
JPH08265761A (ja) 映像データ後処理方法
EP1629436B1 (fr) Procede de codage de trame residuelle de mouvement base sur une transformee sur-complete et appareil de compression video
JP3710974B2 (ja) ブロック化現象及びリング現象を除去するための圧縮動画像の復元方法及びその装置
KR100319557B1 (ko) 블럭 단위로 부호화된 영상의 블럭 경계 잡음 성분 제거 방법
KR100386639B1 (ko) 정규화된 역양자화기를 이용한 영상 및 동영상의 복원 방법
JPH10224790A (ja) 圧縮伸張された画像中のブロック状ノイズを除去するフィルタおよびフィルタ方法
JPH11187400A (ja) 歪除去方法および歪除去フィルタ
WO2002069261A1 (fr) Procede et appareil permettant de traiter une image numerique, dispositif de codage d'images et dispositif de decodage d'images
EP0886972B1 (fr) Post-filtrage ameliore pour reduire les effets de bloc sur un signal d'image decode
Fan et al. Reducing artifacts in JPEG decompression by segmentation and smoothing
Hsieh et al. Block effect reduction by the 1-D gray polynomial interpolation
Joung et al. Postprocessing of interframe coded images based on convex projection and regularization
Krishnamoorthy et al. Reduction of blocking artifact in JPEG images with orthogonal polynomials
Jung et al. Restoration of differential images for enhancement of compressed video
Yang et al. An adaptive edge-preserving artifacts removal filter for video post-processing

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CR CU CZ DE DK DM DZ EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP