WO2004064406A1 - Defining interpolation filters for error concealment in a coded image - Google Patents

Defining interpolation filters for error concealment in a coded image Download PDF

Info

Publication number
WO2004064406A1
WO2004064406A1 PCT/US2003/021515 US0321515W WO2004064406A1 WO 2004064406 A1 WO2004064406 A1 WO 2004064406A1 US 0321515 W US0321515 W US 0321515W WO 2004064406 A1 WO2004064406 A1 WO 2004064406A1
Authority
WO
WIPO (PCT)
Prior art keywords
interpolation filter
mode
derived
intra
coding technique
Prior art date
Application number
PCT/US2003/021515
Other languages
French (fr)
Inventor
Cristina Gomila
Original Assignee
Thomson Licensing S.A.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Thomson Licensing S.A. filed Critical Thomson Licensing S.A.
Priority to BR0317966-4A priority Critical patent/BR0317966A/en
Priority to AU2003248913A priority patent/AU2003248913A1/en
Priority to JP2004566420A priority patent/JP4474288B2/en
Priority to EP03815173A priority patent/EP1645135A4/en
Priority to US10/541,929 priority patent/US20060072676A1/en
Priority to MXPA05007449A priority patent/MXPA05007449A/en
Publication of WO2004064406A1 publication Critical patent/WO2004064406A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H04N19/89—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving methods or arrangements for detection of transmission errors at the decoder
    • H04N19/895—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving methods or arrangements for detection of transmission errors at the decoder in combination with error concealment
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/11—Selection of coding mode or of prediction mode among a plurality of spatial predictive coding modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation

Definitions

  • This invention relates to a technique for defining directional interpolation filters for the concealment of errors within a coded video stream.
  • video streams undergo compression (coding) to facilitate storage and transmission.
  • coding compression
  • block-based coding schemes such as the proposed ISO/ITU H.2.64 coding technique.
  • coded video streams incur data losses or become corrupted during transmission because of channel errors and/or network congestion.
  • the loss/corruption of data manifests itself as missing/corrupted pixel values that give rise to image artifacts.
  • a decoder will "conceal" such missing/corrupted pixel values by estimating such values from other macroblocks in the same image or from other images.
  • conceal is a somewhat of a misnomer because the decoder does not actually hide missing/corrupted pixel values.
  • Spatial concealment seeks to derive the missing/corrupted pixel values by using pixel values from other areas in the same image relying on the similarity between neighboring regions in the spatial domain.
  • spatial concealment techniques achieve lower performance than temporal error concealment techniques that rely on information from other transmitted pictures.
  • An error concealment algorithm should invoke spatial interpolation only in those instances where no temporal option is available, that is, when losses affect intra-coded pictures, intra refresh pictures or when no temporal information is available.
  • the quality of future inter-coded frames that use a concealed image as a reference will depend on the quality of the spatial concealment. When the spatial concealment yields a relatively poor intra-coded picture, each resultant inter-coded picture will likewise have poor quality.
  • concealment of errors in a coded image comprised of an array of macroblocks commences by first identifying macroblocks within the image having missing/corrupted pixel values. For each identified macroblock, at least one intra- prediction mode is derived from neighboring macroblocks.
  • intra-coded coded macroblocks can be predicted for coding purposes either as a whole block of 16x16 pixels, or on blocks of 4x4 pixels. For a whole 16x16 block, there exists one intra-prediction mode. In contrast, an intra-prediction mode exists for each sub-macroblock of 4x4 pixels within the macroblock.
  • an interpolation filter is selected to define the manner in which pixel values are estimated from neighboring blocks selected by proceeding in a direction prescribed by the identified intra-prediction mode.
  • the macroblocks having missing/corrupted pixel vales are concealed using the estimated pixel values obtained in accordance with the selected interpolation filter.
  • the interpolation filter established for concealment purposes constitutes the filter prescribed in the H.264 coding technique for the intra 4x4 prediction mode. Since different orders of concealment can exist, a mirrored version of the interpolation filters defined in the H.264 coding technique serves to adapt to available samples when neighboring left and above pixels are unavailable.
  • FIGURE 1 depicts a coded picture partitioned into macroblocks, with each macroblock partitioned into blocks, and each block partitioned into pixels;
  • FIGURE 2 illustrates the Intra_4x4 prediction modes described in the proposed H.264 coding technique
  • FIGURES 3A-3F each depict the position of the sets of reference pixels (A, B, C, D and
  • FIGURES 4A-4F each depict the position of the sets of reference pixels (A',B',C',D' and I', J', K', U) as defined for a first set of mirrored interpolation filters corresponding to the Intra_4x4 prediction modes illustrated in FIG. 2.
  • FIGURES 5C-5F each depict the position of the sets of reference pixels (A', B', C, D' and I', J', K', U) as defined for a second set of mirrored interpolation filters corresponding to the Intra_4x4 prediction modes illustrated in FIG. 2.
  • FIGURE 6C depicts the position of the sets of reference pixels (A', B', C, D' and F, J', K', U as defined for a third set of mirrored interpolation filters corresponding to the L tra_4x4 prediction modes illustrated in FIG. 2.
  • Block-based video compression techniques operate by dividing a picture into slices, each slice comprising a set of macroblocks or macroblock pairs, with each macroblock coded in accordance with the coding technique.
  • a macroblocks typically comprises a squared region of 16x16 pixels.
  • macroblocks can be further partitioned into sub-macroblocks not necessarily squared.
  • Each sub-macroblock can have a different coding mode when the macroblock is encoded.
  • a block will be referred to as a sub-macroblock of 4x4 pixels.
  • FIGURE 1 depicts the partitioning of a coded picture 100 into macroblocks 110, with each macroblock 110 partitioned into blocks 120, and each block partitioned into pixels 130. Note that the number of macroblocks within a picture varies depending on the size of the picture, while the number of blocks within a macroblock remains constant.
  • each macroblock 110 within the partitioned image 100 information from already transmitted macroblocks can s yield a prediction of the coding of an individual macroblock. In this case, only the prediction error and the prediction mode require transmission.
  • the video coding technique employed to code the picture 100 will specify the process for deriving the predicted pixel values in order to ensure that the encoder (not shown) and the decoder (not shown) both obtain the same estimation.
  • individual macroblocks can be intra-predicted either as a single partition of 16x16 pixels (Intra_16xl6 type coding) or as a partition of 16 blocks of 4x4 pixels (Intra_4x4 type coding).
  • the ISO/ITU H.264 coding technique specifies four intra-prediction modes: Mode 0, vertical prediction; Mode 1, horizontal prediction; Mode 2, DC prediction; Mode 3, plane prediction.
  • the ISO/ITU H.264 coding technique specifies nine intra-prediction modes: Mode 0, vertical prediction; Mode 1, horizontal prediction; Mode 2, DC prediction; Mode 3, diagonal down-left prediction; Mode 4, diagonal down-right prediction; Mode 5, vertical right prediction; Mode 6, horizontal down prediction; Mode 7, vertical left prediction; and Mode 8, horizontal up prediction.
  • FIGURE 2 depicts each of the Intra_4x4 coding type prediction modes in tabular form as well as a vector display that indicates the direction of each of the intra-prediction Modes 0-8.
  • each intra-prediction mode has an associated interpolation filter that prescribes how to obtain a predicted coding value when proceeding in the direction defined by the intra-prediction mode.
  • the interpolation filters defined by the H.264 can also provide a mechanism for estimating pixel values for error concealment purposes. As described in greater detail below, the H.264 interpolation filters can be used in their exact form for error concealment when error concealment proceeds in the decoding order.
  • FIGURES 3A-3F each depict the position of the sets of reference pixels (A, B, C, D and I,J,K,L) used for the interpolation filters corresponding to the Intra 4x4 prediction modes illustrated in FIG. 2. (Note that in some instances, two different interpolation filters associated to two different intra-prediction modes may use the same set of reference pixels.)
  • FIGS. 3A-3F there appears a sub-macroblock 200 having missing/corrupted pixels requiring concealment using values estimated from the pixel values within a neighboring row and/or column.
  • each intra-prediction mode there exists one interpolation filter that prescribes exactly how to obtain an estimate for each missing/corrupted pixel in the sub-macroblock 200 from the neighboring pixel values.
  • FIG. 3 A depicts error concealment for Mode 0 using the interpolation filter prescribed by the H.264 coding technique for that mode.
  • the interpolation filter prescribed by the H.264 coding technique defines the mechanism for obtaining coding prediction values.
  • the interpolation filter prescribed by the H.264 coding technique also provides a mechanism for obtaining error concealment values.
  • the 4x4 pixel sub-macroblock 200 contains pixels a-p each of which requires concealment.
  • the values of the pixels A-D in a neighboring row of pixels 210, which lies above the upper row of pixels a-d in the sub-macroblock 200, provide values from which to estimate a concealment value for each of the pixels a-p using the H.264 coding technique interpolation filter associated with Mode 0.
  • the value of the pixel A in row 210 provides a concealment estimate for each of the pixels a, e, i, and m in the first (left hand-most) column of the sub-macroblock 200 in accordance with the interpolation filter prescribed for Mode 0 by the H.264 coding technique.
  • the pixel B in row 210 provides a concealment estimate for each of the pixels b, f, j, and n in the second column.
  • the pixels C and D in the row 210 each provide an estimate for the pixels in the third and fourth columns, respectively, in the sub-macroblock 200.
  • one or more of the pixels A-D in row 210 may have missing values, and thus provide a poor estimate for the pixels a-p in the sub-macroblock 200.
  • a "mirrored" interpolation filter for Mode 1 serves to prescribe the manner in which to obtain such pixel concealment values.
  • the pixels A-D in row 210 may have missing values, and thus provide a poor estimate for the pixels a-p in the sub-macroblock 200.
  • Mode 1 H.264 coding technique interpolation filter which makes use of the top neighboring row 210 to provide concealment values as seen in FIG. 3 A
  • the mirrored interpolation filter of the present principles makes use of a bottom neighboring row 220 of pixels A' , B' , C and D' for error concealment purposes as seen in FIG. 4A.
  • the mirrored interpolation filter employs the pixel A' in row 220.
  • the pixels B', C and D' in row 220' provide estimates for the concealment values for the pixels in second, third and fourth columns, respectively, of the sub- macroblock 200 using the mirrored interpolation filter for Mode 0.
  • Table 1 summarizes the H.264 coding technique interpolation filter and the mirrored interpolation filter for providing error concealment values for Mode 0.
  • FIGURE 3B depicts error concealment for Mode 1 using the Mode 1 interpolation filter prescribed by the H.264 coding technique.
  • Each of the pixels I-Lin each row of a neighboring column 210' to the left of the sub-macroblock 200 provides an error concealment estimate for each of the pixels in a corresponding row of the sub-macroblock.
  • the pixel I in the first (upper) row of the column 210 provides a concealment estimate for each of the pixels a, b, c, and d in the first (upper most) row of the sub-macroblock 200.
  • the pixel "J" in column 210' provides a concealment estimate for the pixels e, f, g and h in the second row of the sub-macroblock 200.
  • the pixels K and L provide concealment estimates for the pixels in the third and fourth rows, respectively, of the sub-macroblock 200.
  • FIG. 4B depicts error concealment for Mode 1 using a mirrored interpolation filter.
  • the mirrored interpolation filer for Mode 1 makes use of the pixels I', J', K' and L' in a right hand neighboring column 220' to provide estimates of the concealment values for the pixels in the first (top), second, third and fourth rows, respectively, of the sub-macroblock 200.
  • Table 2 summarizes the H.264 coding technique interpolation filter and the mirrored interpolation filter for estimating concealment values for Mode 1.
  • FIGURE 3C depicts error concealment for the DC intra-prediction mode.
  • the DC mode interpolation filter for coding prediction computes the average of the pixels (A + B + C + D + I + J + K + L + 4) /8, whenever all these samples are available, where the pixels A, B, C and D lie in a neighboring row 210 above the sub-macroblock 200 and the pixels I, J, K and L lie in a neighboring column 210' to the left of the sub- macroblock.
  • FIG. 4C, 5C and 6C depict the mirrored versions of the set of reference pixels shown in FIG 3C. These mirrored versions can be used for error concealment purposes when the block at the left and/or above the missing block is also corrupted.
  • Table 2 summarizes the H.264 coding technique interpolation filter and the mirrored interpolation filters for estimating concealment values for Mode 2.
  • the DC intra-prediction mode interpolation filter prescribed by the H.264 coding technique doesn't provide a good prediction for error concealment purposes.
  • the H.264 coding technique interpolation filter specified for the DC mode provides a very rough prediction that creates flat zones in the concealed image. For that reason, its use for error concealment purposes is recommended only for applications allowing low quality results.
  • another type of interpolation filter classically known as weighted interpolation, could serve to provide a better prediction for the error concealment values.
  • the estimated value of each pixel within the sub-macroblock 200 is obtained independently as the weighted sum of the nearest pixel values from a neighboring column and a neighboring row in the vertical and horizontal directions, respectively, that has been either correctly received or already concealed.
  • Pixel( i, j) W0 * Pixel( iO-1, j) + WI * Pixel( iO, jO-l)
  • WO and WI weigh the influence of the pixel values used as references.
  • WO and WI each represent the distance between the missing pixel and its references.
  • Tables 3 A-D depict the weighted interpolation filters for the DC intra-prediction mode defined depending on which rows/columns of neighboring pixels are used as reference.
  • FIGURE 3D depicts the position of the set of reference pixels to be used for error concealment for both Mode 3 (diagonal down left) and Mode 7 (vertical left) using the H.264 coding technique interpolation filter.
  • the corresponding interpolation filter prescribed by the H.264 coding technique makes use of a separate use weighted average of the pixels A, B. C, D, E, F, and G in a neighboring row 210 above the sub-macroblock 200.
  • FIGURE 4D depicts the position of the set of reference pixels to be used for error concealment using a mirrored interpolation filters for both Mode 3 (diagonal down left) and Mode 7 (vertical left).
  • the corresponding mirrored interpolation filter makes use of a separate weighted average of the pixels H' G', F' E' D' C B' and A' in an extended row neighboring row 210' lying below the sub-macroblock 200.
  • Table 4 summarizes the H.264 coding technique interpolation filter and the mirrored interpolation filter for providing error concealment values for Mode 3.
  • the interpolation filter prescribed by the H.264 coding technique for predicting coding values provides that pixel a in the sub-macroblock 200 can be estimated from the values of the pixels A, B and C using the relationship (A + 2B + C + 2) /4 where the pixels A, B and C each lie in the neighboring row 210 above the sub-macroblock 200.
  • the mirrored interpolation filter for Mode 3 provides an error concealment estimate for pixel a in the sub-macroblock 200 from the values of the pixels G' and H' in accordance with the relationship (G' + 3H' + 2) I A.
  • the remaining pixels b-p can likewise be estimated for error concealment purposes in accordance with the relationship set forth in Table 4.
  • Table 5 summarizes the H.264 coding technique interpolation filter and the mirrored interpolation filter for providing error concealment values for Mode 7 TABLE 5
  • FIGURE 3E depicts the position of the set of reference pixels to be used for error concealment for Modes 4 (diagonal Down Right), Mode 5 (Vertical Right) and Mode 6 (Horizontal Down) using the interpolation filter prescribed by the H.264 coding technique. Because these interpolation filters are defined to require reference pixels on both the left neighboring column and the above neighboring column, their mirroring for error concealment purposes would require the definition of four different cases as for the DC mode. To reduce the number of cases, we propose an alternative definition avoiding to use reference pixels from the left column.
  • FIGURE 4E depicts the position of the set of reference pixels to be used for error concealment for Modes 4, 5 and 6 using a mirrored version of the previous interpolation filters. The filter in FIG.
  • FIG. 4E is an alternative to the filter in FIG. 3E as defined by the H.264 video compression standard.
  • the other mirrored interpolation filter in FIG. 5E is required to allow the error concealment to proceed out of the decoding order.
  • the one proposed in this embodiment locates all the reference pixels in only one neighboring row or only one neighboring column.
  • Such mirroring has two main advantages: first, it facilitates the memory access; second, it reduces the number of cases for which the filter has to be specified. [Note: this applies to all the mirrored interpolated filters defined in this invention]
  • Table 6 summarizes the H.264 coding technique interpolation filter and the mirrored interpolation filter for providing error concealment values for Mode 4. TABLE 6
  • Table 7 summarizes the H.264 coding technique interpolation filter and the mirrored interpolation filter for providing error concealment values for Mode 5.
  • Table 8 summarizes the H.264 coding technique interpolation filter and the mirrored interpolation filter for providing error concealment values for Mode 6.
  • FIGURE 3F depicts error concealment for Mode 8 (Horizontal Up) using the interpolation filter prescribed by the H.264 coding technique.
  • the definition of the mirrored filter in FIG. 4F is proposed as an alternative to the H.264 interpolation filter with the previously mentioned advantages.
  • Table 9 summarizes the H.264 coding technique interpolation filter and the mirrored interpolation filter for providing error concealment values for Mode 8. TABLE 9
  • the foregoing describes a technique for defining directional interpolation filters that establish the mechanism by which to conceal of errors within a coded video stream.

Abstract

Concealment of errors in a coded image (100) occurs by first selecting an intra-prediction mode in accordance with the coding of the image. The selected intra-prediction mode, while ordinarily used to specify the direction for obtaining a coding prediction value, also serves to specify the direction for obtaining estimated values for error concealment. An interpolation filter defines the manner of obtaining estimated pixel values along the direction specified by the intra-prediction mode. Like the intra-prediction mode, the interpolation filter is derived in accordance with the coding of the image. Concealment of the image is achieved using the estimated values obtained in the manner prescribed by the interpolation filter.

Description

DEFINING INTERPOLATION FILTERS FOR ERROR CONCEALMENT IN A CODED
IMAGE
CROSS REFERENCE TO RELATED APPLICATION
This application claims priority under 35 U.S.C. 119(e) to U.S. Provisional Patent Application Serial No. 60/439,185 filed January 10, 2003, the teachings of which are incorporated herein.
TECHNICAL FIELD
This invention relates to a technique for defining directional interpolation filters for the concealment of errors within a coded video stream.
BACKGROUND ART
In many instances, video streams undergo compression (coding) to facilitate storage and transmission. Presently, there exist a variety of coding schemes, including block-based coding schemes such as the proposed ISO/ITU H.2.64 coding technique. Not infrequently, such coded video streams incur data losses or become corrupted during transmission because of channel errors and/or network congestion. Upon decoding, the loss/corruption of data manifests itself as missing/corrupted pixel values that give rise to image artifacts. To reduce such artifacts, a decoder will "conceal" such missing/corrupted pixel values by estimating such values from other macroblocks in the same image or from other images. The term conceal is a somewhat of a misnomer because the decoder does not actually hide missing/corrupted pixel values.
Spatial concealment seeks to derive the missing/corrupted pixel values by using pixel values from other areas in the same image relying on the similarity between neighboring regions in the spatial domain. Typically, at the same level of complexity, spatial concealment techniques achieve lower performance than temporal error concealment techniques that rely on information from other transmitted pictures. An error concealment algorithm should invoke spatial interpolation only in those instances where no temporal option is available, that is, when losses affect intra-coded pictures, intra refresh pictures or when no temporal information is available. The quality of future inter-coded frames that use a concealed image as a reference will depend on the quality of the spatial concealment. When the spatial concealment yields a relatively poor intra-coded picture, each resultant inter-coded picture will likewise have poor quality.
Various techniques exist for accomplishing spatial error concealment. These techniques include: (a) Block copy (BC), (b) Pixel Domain Interpolation (PDI), (c) Multi-directional Interpolation (MDI), (d) Maximally Smooth Recovery (MSR) and (e) Projection On Convex Sets (POCS). Moreover, there now exists a proposal to make use of the intra-prediction modes computed for blocks of 4x4 pixels in accordance with the H.264 technique for error concealment. In accordance with this proposal, the same intra-prediction modes that provide a direction for estimating the coding value from neighboring blocks can also provide a direction for estimating missing/corrupted pixels values for error concealment. Having established the desirability of using the same intra-prediction mode to provide the direction for error concealment as coding prediction, there exists a need to define an appropriate mechanism to derive estimated pixel values for concealment when proceeding in the direction defined by the intra-prediction mode.
BRIEF SUMMARY OF THE INVENTION
Briefly, in accordance with the present principles, concealment of errors in a coded image comprised of an array of macroblocks commences by first identifying macroblocks within the image having missing/corrupted pixel values. For each identified macroblock, at least one intra- prediction mode is derived from neighboring macroblocks. When the image is coded in accordance with the ISO/ITU H.264 coding technique, intra-coded coded macroblocks can be predicted for coding purposes either as a whole block of 16x16 pixels, or on blocks of 4x4 pixels. For a whole 16x16 block, there exists one intra-prediction mode. In contrast, an intra-prediction mode exists for each sub-macroblock of 4x4 pixels within the macroblock. In connection with the derived intra-prediction mode, an interpolation filter is selected to define the manner in which pixel values are estimated from neighboring blocks selected by proceeding in a direction prescribed by the identified intra-prediction mode. The macroblocks having missing/corrupted pixel vales are concealed using the estimated pixel values obtained in accordance with the selected interpolation filter. When the macroblocks in the coded image are coded in accordance with the H.264 coding technique and the concealment order is equal to the decoding order, the interpolation filter established for concealment purposes constitutes the filter prescribed in the H.264 coding technique for the intra 4x4 prediction mode. Since different orders of concealment can exist, a mirrored version of the interpolation filters defined in the H.264 coding technique serves to adapt to available samples when neighboring left and above pixels are unavailable.
BRIEF SUMMARY OF THE DRAWINGS
FIGURE 1 depicts a coded picture partitioned into macroblocks, with each macroblock partitioned into blocks, and each block partitioned into pixels;
FIGURE 2 illustrates the Intra_4x4 prediction modes described in the proposed H.264 coding technique; FIGURES 3A-3F each depict the position of the sets of reference pixels (A, B, C, D and
I, J, K, L) as defined for the interpolation filters corresponding to the Intra_4x4 prediction modes illustrated in FIG. 2; and
FIGURES 4A-4F each depict the position of the sets of reference pixels (A',B',C',D' and I', J', K', U) as defined for a first set of mirrored interpolation filters corresponding to the Intra_4x4 prediction modes illustrated in FIG. 2.
FIGURES 5C-5F each depict the position of the sets of reference pixels (A', B', C, D' and I', J', K', U) as defined for a second set of mirrored interpolation filters corresponding to the Intra_4x4 prediction modes illustrated in FIG. 2.
FIGURE 6C depicts the position of the sets of reference pixels (A', B', C, D' and F, J', K', U as defined for a third set of mirrored interpolation filters corresponding to the L tra_4x4 prediction modes illustrated in FIG. 2.
DETAILED DESCRIPTION
Block-based video compression techniques, such as embodied in the proposed ISO/ITU H.264 coding technique, operate by dividing a picture into slices, each slice comprising a set of macroblocks or macroblock pairs, with each macroblock coded in accordance with the coding technique. A macroblocks typically comprises a squared region of 16x16 pixels. For coding purposes, macroblocks can be further partitioned into sub-macroblocks not necessarily squared. Each sub-macroblock can have a different coding mode when the macroblock is encoded. For ease of discussion, a block will be referred to as a sub-macroblock of 4x4 pixels. FIGURE 1 depicts the partitioning of a coded picture 100 into macroblocks 110, with each macroblock 110 partitioned into blocks 120, and each block partitioned into pixels 130. Note that the number of macroblocks within a picture varies depending on the size of the picture, while the number of blocks within a macroblock remains constant.
To reduce the cost of individually coding each macroblock 110 within the partitioned image 100, information from already transmitted macroblocks can s yield a prediction of the coding of an individual macroblock. In this case, only the prediction error and the prediction mode require transmission. The video coding technique employed to code the picture 100 will specify the process for deriving the predicted pixel values in order to ensure that the encoder (not shown) and the decoder (not shown) both obtain the same estimation. In accordance with the ISO/ITU H.264 coding technique, individual macroblocks can be intra-predicted either as a single partition of 16x16 pixels (Intra_16xl6 type coding) or as a partition of 16 blocks of 4x4 pixels (Intra_4x4 type coding).
For the Intra_16xl6 type coding, the ISO/ITU H.264 coding technique specifies four intra-prediction modes: Mode 0, vertical prediction; Mode 1, horizontal prediction; Mode 2, DC prediction; Mode 3, plane prediction. For the Intra_4x4 coding type, the ISO/ITU H.264 coding technique specifies nine intra-prediction modes: Mode 0, vertical prediction; Mode 1, horizontal prediction; Mode 2, DC prediction; Mode 3, diagonal down-left prediction; Mode 4, diagonal down-right prediction; Mode 5, vertical right prediction; Mode 6, horizontal down prediction; Mode 7, vertical left prediction; and Mode 8, horizontal up prediction. FIGURE 2 depicts each of the Intra_4x4 coding type prediction modes in tabular form as well as a vector display that indicates the direction of each of the intra-prediction Modes 0-8. (Note that Mode 2, corresponding to the DC mode, has no direction, since it uniformly predicts the content of a block within a homogeneous region.) The other modes 0-1 and 3-8 predict the content of a macroblock along a separate one of the eight quantized directions. The proposed H.264 coding technique provides that each intra-prediction mode has an associated interpolation filter that prescribes how to obtain a predicted coding value when proceeding in the direction defined by the intra-prediction mode. In accordance with the present principles, the interpolation filters defined by the H.264 can also provide a mechanism for estimating pixel values for error concealment purposes. As described in greater detail below, the H.264 interpolation filters can be used in their exact form for error concealment when error concealment proceeds in the decoding order. Alternatively, when the error concealment proceeds in a different order, a mirrored version of the H.264 interpolation filters has to be considered. FIGURES 3A-3F each depict the position of the sets of reference pixels (A, B, C, D and I,J,K,L) used for the interpolation filters corresponding to the Intra 4x4 prediction modes illustrated in FIG. 2. (Note that in some instances, two different interpolation filters associated to two different intra-prediction modes may use the same set of reference pixels.) In each of FIGS. 3A-3F, there appears a sub-macroblock 200 having missing/corrupted pixels requiring concealment using values estimated from the pixel values within a neighboring row and/or column. For each intra-prediction mode, there exists one interpolation filter that prescribes exactly how to obtain an estimate for each missing/corrupted pixel in the sub-macroblock 200 from the neighboring pixel values. To better understand the nature of each such interpolation filter, refer to FIG. 3 A, which depicts error concealment for Mode 0 using the interpolation filter prescribed by the H.264 coding technique for that mode. Ordinarily, the interpolation filter prescribed by the H.264 coding technique defines the mechanism for obtaining coding prediction values. In accordance with the present principles, the interpolation filter prescribed by the H.264 coding technique also provides a mechanism for obtaining error concealment values. As seen in FIG 3 A, the 4x4 pixel sub-macroblock 200 contains pixels a-p each of which requires concealment. The values of the pixels A-D in a neighboring row of pixels 210, which lies above the upper row of pixels a-d in the sub-macroblock 200, provide values from which to estimate a concealment value for each of the pixels a-p using the H.264 coding technique interpolation filter associated with Mode 0. For Mode 0 (vertical), the value of the pixel A in row 210 provides a concealment estimate for each of the pixels a, e, i, and m in the first (left hand-most) column of the sub-macroblock 200 in accordance with the interpolation filter prescribed for Mode 0 by the H.264 coding technique. Likewise, the pixel B in row 210 provides a concealment estimate for each of the pixels b, f, j, and n in the second column. In a similar fashion , the pixels C and D in the row 210 each provide an estimate for the pixels in the third and fourth columns, respectively, in the sub-macroblock 200.
In some instances, one or more of the pixels A-D in row 210 may have missing values, and thus provide a poor estimate for the pixels a-p in the sub-macroblock 200. h accordance with another aspect of the present principles, a "mirrored" interpolation filter for Mode 1 serves to prescribe the manner in which to obtain such pixel concealment values. In contrast to the
Mode 1 H.264 coding technique interpolation filter which makes use of the top neighboring row 210 to provide concealment values as seen in FIG. 3 A, the mirrored interpolation filter of the present principles makes use of a bottom neighboring row 220 of pixels A' , B' , C and D' for error concealment purposes as seen in FIG. 4A. Thus, instead of using value of pixel A in row 210 to estimate each of the pixels a, e, i and m, the mirrored interpolation filter employs the pixel A' in row 220. Likewise, the pixels B', C and D' in row 220' provide estimates for the concealment values for the pixels in second, third and fourth columns, respectively, of the sub- macroblock 200 using the mirrored interpolation filter for Mode 0.
Table 1 summarizes the H.264 coding technique interpolation filter and the mirrored interpolation filter for providing error concealment values for Mode 0.
TABLE 1 Mode 0 (Vertical)
Figure imgf000008_0001
FIGURE 3B depicts error concealment for Mode 1 using the Mode 1 interpolation filter prescribed by the H.264 coding technique. Each of the pixels I-Lin each row of a neighboring column 210' to the left of the sub-macroblock 200 provides an error concealment estimate for each of the pixels in a corresponding row of the sub-macroblock. Thus, for example, the pixel I in the first (upper) row of the column 210 provides a concealment estimate for each of the pixels a, b, c, and d in the first (upper most) row of the sub-macroblock 200. Likewise, the pixel "J" in column 210' provides a concealment estimate for the pixels e, f, g and h in the second row of the sub-macroblock 200. Similarly, the pixels K and L provide concealment estimates for the pixels in the third and fourth rows, respectively, of the sub-macroblock 200.
FIG. 4B depicts error concealment for Mode 1 using a mirrored interpolation filter. Rather than making use of the pixels I, J, K and L in the left-hand column 210', the mirrored interpolation filer for Mode 1 makes use of the pixels I', J', K' and L' in a right hand neighboring column 220' to provide estimates of the concealment values for the pixels in the first (top), second, third and fourth rows, respectively, of the sub-macroblock 200. Table 2 summarizes the H.264 coding technique interpolation filter and the mirrored interpolation filter for estimating concealment values for Mode 1.
TABLE 2
Mode 1 (Horizontal)
Figure imgf000009_0001
FIGURE 3C depicts error concealment for the DC intra-prediction mode. As defined in the H.264 coding technique, the DC mode interpolation filter for coding prediction computes the average of the pixels (A + B + C + D + I + J + K + L + 4) /8, whenever all these samples are available, where the pixels A, B, C and D lie in a neighboring row 210 above the sub-macroblock 200 and the pixels I, J, K and L lie in a neighboring column 210' to the left of the sub- macroblock. In other words, all the pixels a-p inside the sub-macroblock 200 are predicted for coding purposes with the same value corresponding to the average of the neighboring pixel values of the column and row to the left and above, respectively, of the sub-macroblock. FIG. 4C, 5C and 6C depict the mirrored versions of the set of reference pixels shown in FIG 3C. These mirrored versions can be used for error concealment purposes when the block at the left and/or above the missing block is also corrupted.
Table 2 summarizes the H.264 coding technique interpolation filter and the mirrored interpolation filters for estimating concealment values for Mode 2.
TABLE 3
Mode 2 ( DC)
Figure imgf000010_0001
Figure imgf000010_0002
However, unlike the other intra-prediction modes, the DC intra-prediction mode interpolation filter prescribed by the H.264 coding technique doesn't provide a good prediction for error concealment purposes. The H.264 coding technique interpolation filter specified for the DC mode provides a very rough prediction that creates flat zones in the concealed image. For that reason, its use for error concealment purposes is recommended only for applications allowing low quality results. For the other applications, another type of interpolation filter, classically known as weighted interpolation, could serve to provide a better prediction for the error concealment values. When this technique applies to the DC mode, the estimated value of each pixel within the sub-macroblock 200 is obtained independently as the weighted sum of the nearest pixel values from a neighboring column and a neighboring row in the vertical and horizontal directions, respectively, that has been either correctly received or already concealed.
Classically, the following relationship prescribes the weighted interpolation of the pixel value at the (i,j) position:
Pixel( i, j) = W0 * Pixel( iO-1, j) + WI * Pixel( iO, jO-l) where WO and WI weigh the influence of the pixel values used as references. Typically, WO and WI each represent the distance between the missing pixel and its references. In the illustrated embodiment, WO = (i - iO) and WI = (j - jO). Using the same notation employed to describe the other interpolation filters defined by the H.264 coding technique, Tables 3 A-D depict the weighted interpolation filters for the DC intra-prediction mode defined depending on which rows/columns of neighboring pixels are used as reference.
TABLE 3A
Mode 2 (DC) Weighted interpolation in FIG.3C a=(4A + 4I)/8 e = (3A + 4J)/7 i = (2A + 4K)/6 m = (l A + 4L)/5 b = (4B + 3I)/7 f= (3B + 3J)/6 j = (2B + 3K)/5 n = (lB+3L)/4 c = (4C + 2I)/6 g = (3C + 2J)/5 k = (2C + 2K)/4 o = (lC + 2L)/3 d = (4D + U)/5 h = (3D + U)/4 l = (2D+lK)/3 p = ( 1 D + 1 L ) / 2
TABLE 3B
Mode 2 (DC) Weighted interpolation in FIG.4C a = (4A+U')/5 e = (3A+U')/4 i = (2A+lK')/3 m = ( 1 A + 1 L' ) / 2 b = (4B+2I')/6 f= (3B + 2J')/5 j = (2B+2K')/4 n = (lB+2L')/3 c = (4C + 3F)/7 g = (3C + 3J')/6 k = (2C + 3K')/5 o = (lC + 3L')/4 d = (4D + 4I')/8 h = (3D + 4J')/7 l = (2D + 4K')/6 p = (lD + 4L')/5
TABLE 3C
Mode 2 (DC) Weighted interpolation in Fig.5C a = (l A' + 4I)/5 e = (2A' + 4J)/6 i = (3A' + 4K)/7 m = (4A' + 4L)/8 b = (lB' + 3I)/4 f= (2B' + 3J)/5 j = (3B'+3K)/6 n = (4B' + 3L)/7 c = (lC' + 2I)/3 g = (2C' + 2J)/4 k = (3C' + 2K)/5 o = (4C' + 2L)/6 d = (lD'+H)/2 h = (2D'+l J)/3 l = (3D'+lK)/4 p = (4D'+lL)/5 TABLE 3D Mode 2 (DC) Weighted interpolation in Fig.6C a=(l A'+H')/2 e = (2A'+l J 3 i = (3A'+lK')/4 m = (4A'+lL')/5 b = (lB' + 2I')/3 f = (2B' + 2J')/4 j = (3B' + 2K')/5 n = (4B' + 2L')/6 c = (lC'+3I')/4 g = (2C'+3J')/5 k = (3C' + 3K')/6 o = (4C'+3L')/7 d = (lD' + 4I')/5 h = (2D' + 4J')/6 l = (3D' + 4K')/7 p = (4D'+4L')/8
FIGURE 3D depicts the position of the set of reference pixels to be used for error concealment for both Mode 3 (diagonal down left) and Mode 7 (vertical left) using the H.264 coding technique interpolation filter. For each of Modes 3 and 7, the corresponding interpolation filter prescribed by the H.264 coding technique makes use of a separate use weighted average of the pixels A, B. C, D, E, F, and G in a neighboring row 210 above the sub-macroblock 200. Similarly, FIGURE 4D depicts the position of the set of reference pixels to be used for error concealment using a mirrored interpolation filters for both Mode 3 (diagonal down left) and Mode 7 (vertical left). For each of Modes 3 and 7, the corresponding mirrored interpolation filter makes use of a separate weighted average of the pixels H' G', F' E' D' C B' and A' in an extended row neighboring row 210' lying below the sub-macroblock 200.
Table 4 summarizes the H.264 coding technique interpolation filter and the mirrored interpolation filter for providing error concealment values for Mode 3.
TABLE 4
Mode 3 (Diagonal down-left)
Figure imgf000013_0001
By way of example, the interpolation filter prescribed by the H.264 coding technique for predicting coding values, and utilized in accordance with the present principles for estimating error concealment values, provides that pixel a in the sub-macroblock 200 can be estimated from the values of the pixels A, B and C using the relationship (A + 2B + C + 2) /4 where the pixels A, B and C each lie in the neighboring row 210 above the sub-macroblock 200. Similarly, the mirrored interpolation filter for Mode 3 provides an error concealment estimate for pixel a in the sub-macroblock 200 from the values of the pixels G' and H' in accordance with the relationship (G' + 3H' + 2) I A. The remaining pixels b-p can likewise be estimated for error concealment purposes in accordance with the relationship set forth in Table 4.
Table 5 summarizes the H.264 coding technique interpolation filter and the mirrored interpolation filter for providing error concealment values for Mode 7 TABLE 5
Mode 7. (Vertical left)
Figure imgf000014_0001
FIGURE 3E depicts the position of the set of reference pixels to be used for error concealment for Modes 4 (diagonal Down Right), Mode 5 (Vertical Right) and Mode 6 (Horizontal Down) using the interpolation filter prescribed by the H.264 coding technique. Because these interpolation filters are defined to require reference pixels on both the left neighboring column and the above neighboring column, their mirroring for error concealment purposes would require the definition of four different cases as for the DC mode. To reduce the number of cases, we propose an alternative definition avoiding to use reference pixels from the left column. FIGURE 4E depicts the position of the set of reference pixels to be used for error concealment for Modes 4, 5 and 6 using a mirrored version of the previous interpolation filters. The filter in FIG. 4E is an alternative to the filter in FIG. 3E as defined by the H.264 video compression standard. The other mirrored interpolation filter in FIG. 5E is required to allow the error concealment to proceed out of the decoding order. Although different mirroring procedures could be considered, the one proposed in this embodiment locates all the reference pixels in only one neighboring row or only one neighboring column. Such mirroring has two main advantages: first, it facilitates the memory access; second, it reduces the number of cases for which the filter has to be specified. [Note: this applies to all the mirrored interpolated filters defined in this invention] Table 6 summarizes the H.264 coding technique interpolation filter and the mirrored interpolation filter for providing error concealment values for Mode 4. TABLE 6
Mode 4. Diagonal down-right.
Figure imgf000015_0001
Table 7 summarizes the H.264 coding technique interpolation filter and the mirrored interpolation filter for providing error concealment values for Mode 5.
TABLE 7
Mode 5. Vertical right.
Figure imgf000015_0002
Table 8 summarizes the H.264 coding technique interpolation filter and the mirrored interpolation filter for providing error concealment values for Mode 6.
Table 8
Mode 6. Horizontal down.
Figure imgf000016_0001
FIGURE 3F depicts error concealment for Mode 8 (Horizontal Up) using the interpolation filter prescribed by the H.264 coding technique. FIGURES 4E and 5E depict error concealment for Mode 8 using two mirrored interpolation filter. As for the Modes 4, 5, and 6, the definition of the mirrored filter in FIG. 4F is proposed as an alternative to the H.264 interpolation filter with the previously mentioned advantages. Table 9 summarizes the H.264 coding technique interpolation filter and the mirrored interpolation filter for providing error concealment values for Mode 8. TABLE 9
Mode 8 (Horizontal up)
Figure imgf000017_0001
The foregoing describes a technique for defining directional interpolation filters that establish the mechanism by which to conceal of errors within a coded video stream.

Claims

1. A method for concealing errors in a coded image formed of an array of macroblocks, comprising the steps of: identifying macroblocks within the array having missing/corrupted pixel values; deriving at least one intra-prediction mode for each identified macroblock to define a concealment direction, the at least one intra-prediction mode derived in accordance with the coded image; establishing an interpolation filter for the identified intra-prediction mode for estimating concealment values for each identified macroblock along the concealment direction; and concealing the identified macroblock in accordance with the estimated concealment values.
2. The method according to claim 1 wherein the image is coded in accordance with the H.264 coding technique and wherein the step of deriving the at least one intra-prediction mode further comprises the step of deriving an Intra_4x4 prediction mode prescribed by the H.264 coding technique.
3. The method according to claim 2 wherein step of establishing the interpolation filter further comprises selecting the interpolation filter prescribed by the H.264 coding technique for the derived Intra_4x4 prediction mode.
4. The method according to claim 2 wherein step of establishing the interpolation filter further comprises the step of deriving a interpolation filter mirroring the interpolation filter prescribed by the H.264 coding technique for the derived Intra_4x4 prediction mode.
6. The method according to claim 2 wherein the derived Intra_4x4 prediction mode comprises Mode 0 (vertical) and wherein the derived interpolation filter comprises the interpolation filter prescribed by the H.264 coding technique for Mode 0.
7. The method according to claim 4 wherein the derived Intra_4x4 prediction mode comprises Mode 1 (horizontal) and wherein the derived interpolation filter comprises the interpolation filter prescribed by the H.264 coding technique for Mode 1.
8. The method according to claim 2 wherein the derived Intra_4x4 prediction mode comprises Mode 2 (DC) and wherein the step of establishing the interpolation filter further comprises the step independently weighing a sum of pixel values from a neighboring column and a neighboring row in a vertical direction and a horizontal direction, respectively.
9. The method according to claim 4 wherein the derived Intra_4x4 prediction mode comprises Mode 3 (Diagonal down left) and wherein the derived interpolation filter comprises the interpolation filter prescribed by the H.264 coding technique for Mode 3.
10. The method according to claim 4 wherein the derived Intra_4x4 prediction mode comprises Mode 7 (vertical left) and wherein the derived interpolation filter comprises the interpolation filter prescribed by the H.264 coding technique for Mode 7.
11. The method according to claim 4 wherein the derived Intra_4x4 prediction mode comprises Mode 4 (Diagonal down right) and wherein the derived interpolation filter comprises the interpolation filter prescribed by the H.264 coding technique for Mode 4.
12. The method according to claim 4 wherein the derived Intra_4x4 prediction mode comprises Mode 5 (Vertical right) and wherein the derived interpolation filter comprises the interpolation filter prescribed by the H.264 coding technique for Mode 5.
13. The method according to claim 4 wherein the derived Intra_4x4 prediction mode comprises Mode 6 (horizontal down) and wherein the derived interpolation filter comprises the interpolation filter prescribed by the H.264 coding technique for Mode 6.
14. The method according to claim 4 wherein the derived Intra_4x4 prediction mode comprises Mode 8 (horizontal up) and wherein the derived interpolation filter comprises the interpolation filter prescribed by the H.264 coding technique for Mode 8.
15. A method for concealing errors in a coded image comprised of an array of macroblocks, the image coded in accordance with the H.264 coding technique, the method, comprising the steps of: identifying macroblocks within the array having missing/corrupted pixel values; deriving at least one Intra_4x4 prediction mode in accordance with the H.264 coding technique for each identified macroblock to define a concealment direction; establishing an interpolation filter for the identified intra-prediction mode for estimating concealment values for each identified macroblock the along the concealment direction; and concealing the identified macroblock in accordance with the estimated concealment values.
16. The method according to claim 15 wherein step of establishing the interpolation filter further comprises selecting the interpolation filter prescribed by the H.264 coding technique for the derived Intra_4x4 prediction mode.
17. The method according to claim 15 wherein step of establishing the interpolation filter further comprises the step of deriving a interpolation filter mirroring the interpolation filter prescribed by the H.264 coding technique for the derived Intra_4x4 prediction mode.
18. The method according to claim 15 wherein the derived Intra_4x4 prediction mode comprises Mode 1 (horizontal) and wherein the derived interpolation filter comprises the interpolation filter prescribed by the H.264 coding technique for Mode 1.
19. The method according to claim 15 wherein the derived Intra_4x4 prediction mode comprises Mode 3 (Diagonal down left) and wherein the derived interpolation filter comprises the interpolation filter prescribed by the H.264 coding technique for Mode 3.
20. The method according to claim 15 wherein the derived Intra_4x4 prediction mode comprises Mode 7 (vertical left) and wherein the derived interpolation filter comprises the interpolation filter prescribed by the H.264 coding technique for Mode 7.
21. The method according to claim 15 wherein the derived Intra_4x4 prediction mode comprises Mode 4 (Diagonal down right) and wherein the derived interpolation filter comprises the interpolation filter prescribed by the H.264 coding technique for Mode 4.
22. The method according to claim 15 wherein the derived Intra_4x4 prediction mode comprises Mode 5 (Vertical right) and wherein the derived interpolation filter comprises the interpolation filter prescribed by the H.264 coding technique for Mode 5.
23. The method according to claim 15 wherein the derived Intra_4x4 prediction mode comprises Mode 6 (horizontal down) and wherein the derived interpolation filter comprises the interpolation filter prescribed by the H.264 coding technique for Mode 6.
24. The method according to claim 15 wherein the derived Intra_4x4 prediction mode comprises Mode 8 (horizontal up) and wherein the derived interpolation filter comprises the interpolation filter prescribed by the H.264 coding technique for Mode 8.
PCT/US2003/021515 2003-01-10 2003-07-10 Defining interpolation filters for error concealment in a coded image WO2004064406A1 (en)

Priority Applications (6)

Application Number Priority Date Filing Date Title
BR0317966-4A BR0317966A (en) 2003-01-10 2003-07-10 Defining tween filters to hide errors in an encoded image
AU2003248913A AU2003248913A1 (en) 2003-01-10 2003-07-10 Defining interpolation filters for error concealment in a coded image
JP2004566420A JP4474288B2 (en) 2003-01-10 2003-07-10 Defining an interpolation filter for error concealment in coded images
EP03815173A EP1645135A4 (en) 2003-01-10 2003-07-10 Defining interpolation filters for error concealment in a coded image
US10/541,929 US20060072676A1 (en) 2003-01-10 2003-07-10 Defining interpolation filters for error concealment in a coded image
MXPA05007449A MXPA05007449A (en) 2003-01-10 2003-07-10 Defining interpolation filters for error concealment in a coded image.

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US43918503P 2003-01-10 2003-01-10
US60/439,185 2003-01-10

Publications (1)

Publication Number Publication Date
WO2004064406A1 true WO2004064406A1 (en) 2004-07-29

Family

ID=32713446

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2003/021515 WO2004064406A1 (en) 2003-01-10 2003-07-10 Defining interpolation filters for error concealment in a coded image

Country Status (9)

Country Link
US (1) US20060072676A1 (en)
EP (1) EP1645135A4 (en)
JP (1) JP4474288B2 (en)
KR (1) KR20050090451A (en)
CN (1) CN100584024C (en)
AU (1) AU2003248913A1 (en)
BR (1) BR0317966A (en)
MX (1) MXPA05007449A (en)
WO (1) WO2004064406A1 (en)

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006020019A1 (en) * 2004-07-15 2006-02-23 Qualcomm Incorporated H.264 spatial error concealment based on the intra-prediction direction
JP2006101440A (en) * 2004-09-30 2006-04-13 Toshiba Corp Information processing apparatus and method
WO2006106935A1 (en) * 2005-04-01 2006-10-12 Matsushita Electric Industrial Co., Ltd. Image decoding apparatus and image decoding method
WO2007038727A2 (en) * 2005-09-27 2007-04-05 Qualcomm Incorporated Video encoding method enabling highly efficient partial decoding of h.264 and other transform coded information
JP2007096679A (en) * 2005-09-28 2007-04-12 Kddi R & D Laboratories Inc Method of encoding and decoding moving picture
EP1811789A2 (en) * 2006-01-23 2007-07-25 Samsung Electronics Co., Ltd. Video coding using directional interpolation
EP1761064A3 (en) * 2005-09-06 2009-09-02 Samsung Electronics Co., Ltd. Methods and apparatus for video intraprediction encoding and decoding
EP2210421A2 (en) * 2007-10-16 2010-07-28 LG Electronics Inc. A method and an apparatus for processing a video signal
WO2011083439A1 (en) * 2010-01-08 2011-07-14 Nokia Corporation An apparatus, a method and a computer program for video coding
JP5032344B2 (en) * 2006-02-06 2012-09-26 パナソニック株式会社 Image decoding apparatus and image decoding method
JP2012231532A (en) * 2005-07-13 2012-11-22 Polycom Inc Video error concealment method
EP2388999A3 (en) * 2010-05-17 2013-01-23 Lg Electronics Inc. New intra prediction modes
WO2013067435A1 (en) * 2011-11-04 2013-05-10 Huawei Technologies Co., Ltd. Differential pulse code modulation intra prediction for high efficiency video coding
KR20140007465A (en) * 2011-04-25 2014-01-17 엘지전자 주식회사 Intra-prediction method, and encoder and decoder using same
US9055298B2 (en) 2005-07-15 2015-06-09 Qualcomm Incorporated Video encoding method enabling highly efficient partial decoding of H.264 and other transform coded information
US9516349B2 (en) 2011-07-12 2016-12-06 Futurewei Technologies, Inc. Pixel-based intra prediction for coding in HEVC
KR20180034665A (en) * 2015-09-10 2018-04-04 엘지전자 주식회사 Method and apparatus for intra prediction in video coding system

Families Citing this family (43)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1569462A1 (en) * 2004-02-13 2005-08-31 France Telecom Method for determining the optimal prediction direction for video intra-prediction coding
US20060045190A1 (en) * 2004-09-02 2006-03-02 Sharp Laboratories Of America, Inc. Low-complexity error concealment for real-time video decoder
DE102004059978B4 (en) * 2004-10-15 2006-09-07 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Apparatus and method for generating a coded video sequence and decoding a coded video sequence using interlayer residue prediction, and a computer program and computer readable medium
KR100667808B1 (en) * 2005-08-20 2007-01-11 삼성전자주식회사 Method and apparatus for intra prediction encoding and decoding for image
KR100718135B1 (en) * 2005-08-24 2007-05-14 삼성전자주식회사 apparatus and method for video prediction for multi-formet codec and the video encoding/decoding apparatus and method thereof.
KR100706720B1 (en) * 2005-09-15 2007-04-12 한양대학교 산학협력단 A Spatial Error Concealment Technique Using Edge-Oriented Interpolation
RU2369038C1 (en) * 2005-11-30 2009-09-27 Кабусики Кайся Тосиба Image encoding/decoding method, image encoding/decoding device
US8238442B2 (en) 2006-08-25 2012-08-07 Sony Computer Entertainment Inc. Methods and apparatus for concealing corrupted blocks of video data
KR101380843B1 (en) * 2006-12-28 2014-04-07 삼성전자주식회사 Method for generating reduced image of compressed image including blocks encoded according to intra prediction and apparatus thereof
JP2009094828A (en) * 2007-10-10 2009-04-30 Hitachi Ltd Device and method for encoding image, and device and method for decoding image
KR101361005B1 (en) 2008-06-24 2014-02-13 에스케이 텔레콤주식회사 Intra Prediction Method and Apparatus and Image Encoding/Decoding Method and Apparatus Using Same
US8867854B2 (en) 2008-10-01 2014-10-21 Electronics And Telecommunications Research Institute Image encoder and decoder using undirectional prediction
KR101356448B1 (en) 2008-10-01 2014-02-06 한국전자통신연구원 Image decoder using unidirectional prediction
US8774541B2 (en) * 2008-11-05 2014-07-08 Sony Corporation Intra prediction with adaptive interpolation filtering for image compression
JP5238523B2 (en) * 2009-01-13 2013-07-17 株式会社日立国際電気 Moving picture encoding apparatus, moving picture decoding apparatus, and moving picture decoding method
US8743970B2 (en) * 2009-04-13 2014-06-03 Freescale Semiconductor, Inc. Video decoding with error detection and concealment
JP5169978B2 (en) * 2009-04-24 2013-03-27 ソニー株式会社 Image processing apparatus and method
KR101682147B1 (en) * 2010-04-05 2016-12-05 삼성전자주식회사 Method and apparatus for interpolation based on transform and inverse transform
KR20110113561A (en) 2010-04-09 2011-10-17 한국전자통신연구원 Method and apparatus for intra prediction encoding and decoding using adaptive filter
DK3457689T3 (en) 2010-05-25 2020-10-12 Lg Electronics Inc NEW PLANAR PREDICTION MODE
KR101387467B1 (en) * 2010-06-07 2014-04-22 (주)휴맥스 Methods For Encoding/Decoding High Definition Image And Apparatuses For Performing The Same
KR20110138098A (en) * 2010-06-18 2011-12-26 삼성전자주식회사 Apparatus and method for image intra prediction and apparatus and method for decoding image using the intra prediction
US9008175B2 (en) 2010-10-01 2015-04-14 Qualcomm Incorporated Intra smoothing filter for video coding
JP5645589B2 (en) * 2010-10-18 2014-12-24 三菱電機株式会社 Video encoding device
KR101772046B1 (en) * 2010-11-04 2017-08-29 에스케이텔레콤 주식회사 Video Encoding/Decoding Method and Apparatus for Intra-Predicting Using Filtered Value of Pixel According to Prediction Mode
US10142630B2 (en) * 2010-12-10 2018-11-27 Texas Instruments Incorporated Mode adaptive intra prediction smoothing in video coding
WO2012081895A1 (en) 2010-12-13 2012-06-21 한국전자통신연구원 Intra prediction method and apparatus
KR102086145B1 (en) 2010-12-13 2020-03-09 한국전자통신연구원 Method for intra prediction and apparatus thereof
US20130294511A1 (en) * 2011-01-14 2013-11-07 Telefonaktiebolaget L M Ericsson (Publ) Methods and Devices for Intra Coding of Video
WO2012134046A2 (en) * 2011-04-01 2012-10-04 주식회사 아이벡스피티홀딩스 Method for encoding video
KR20120140181A (en) * 2011-06-20 2012-12-28 한국전자통신연구원 Method and apparatus for encoding and decoding using filtering for prediction block boundary
US9131239B2 (en) * 2011-06-20 2015-09-08 Qualcomm Incorporated Unified merge mode and adaptive motion vector prediction mode candidates selection
KR101956284B1 (en) * 2011-06-30 2019-03-08 엘지전자 주식회사 Interpolation Method And Prediction method thereof
KR102055451B1 (en) * 2011-07-01 2019-12-13 엠앤케이홀딩스 주식회사 Apparatus and method for decoding image
US10904551B2 (en) * 2013-04-05 2021-01-26 Texas Instruments Incorporated Video coding using intra block copy
JP2015185897A (en) 2014-03-20 2015-10-22 パナソニックＩｐマネジメント株式会社 Image encoding method and device
CN110225360A (en) * 2014-04-01 2019-09-10 联发科技股份有限公司 The method that adaptive interpolation filters in Video coding
KR101582501B1 (en) * 2014-06-13 2016-01-07 에스케이텔레콤 주식회사 Intra Prediction Method and Apparatus and Image Encoding/Decoding Method and Apparatus Using Same
KR101534048B1 (en) * 2014-06-13 2015-07-07 에스케이텔레콤 주식회사 Intra Prediction Method and Apparatus and Image Encoding/Decoding Method and Apparatus Using Same
KR101533434B1 (en) * 2014-06-13 2015-07-03 에스케이텔레콤 주식회사 Intra Prediction Method and Apparatus and Image Encoding/Decoding Method and Apparatus Using Same
JP6669622B2 (en) * 2016-09-21 2020-03-18 Kddi株式会社 Moving image decoding device, moving image decoding method, moving image encoding device, moving image encoding method, and computer-readable recording medium
CN113766246A (en) * 2020-06-05 2021-12-07 Oppo广东移动通信有限公司 Image encoding method, image decoding method and related device
CN113965764B (en) * 2020-07-21 2023-04-07 Oppo广东移动通信有限公司 Image encoding method, image decoding method and related device

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5353059A (en) * 1992-01-09 1994-10-04 Sony United Kingdom Ltd. Data error concealment
US5532837A (en) * 1992-12-18 1996-07-02 Matsushita Electric Industrial Co., Ltd. Digital video signal processing apparatus
US5621467A (en) * 1995-02-16 1997-04-15 Thomson Multimedia S.A. Temporal-spatial error concealment apparatus and method for video signal processors

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5353059A (en) * 1992-01-09 1994-10-04 Sony United Kingdom Ltd. Data error concealment
US5532837A (en) * 1992-12-18 1996-07-02 Matsushita Electric Industrial Co., Ltd. Digital video signal processing apparatus
US5621467A (en) * 1995-02-16 1997-04-15 Thomson Multimedia S.A. Temporal-spatial error concealment apparatus and method for video signal processors

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP1645135A4 *

Cited By (68)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006020019A1 (en) * 2004-07-15 2006-02-23 Qualcomm Incorporated H.264 spatial error concealment based on the intra-prediction direction
JP2006101440A (en) * 2004-09-30 2006-04-13 Toshiba Corp Information processing apparatus and method
JP4559811B2 (en) * 2004-09-30 2010-10-13 株式会社東芝 Information processing apparatus and information processing method
US8885707B2 (en) 2005-04-01 2014-11-11 Panasonic Corporation Image decoding apparatus and image decoding method
WO2006106935A1 (en) * 2005-04-01 2006-10-12 Matsushita Electric Industrial Co., Ltd. Image decoding apparatus and image decoding method
JP2012231532A (en) * 2005-07-13 2012-11-22 Polycom Inc Video error concealment method
US9661376B2 (en) 2005-07-13 2017-05-23 Polycom, Inc. Video error concealment method
US9055298B2 (en) 2005-07-15 2015-06-09 Qualcomm Incorporated Video encoding method enabling highly efficient partial decoding of H.264 and other transform coded information
EP1761064A3 (en) * 2005-09-06 2009-09-02 Samsung Electronics Co., Ltd. Methods and apparatus for video intraprediction encoding and decoding
JP2009510938A (en) * 2005-09-27 2009-03-12 クゥアルコム・インコーポレイテッド H. H.264 and other video coding methods enabling efficient partial decoding of transform coding information
WO2007038727A3 (en) * 2005-09-27 2007-08-02 Qualcomm Inc Video encoding method enabling highly efficient partial decoding of h.264 and other transform coded information
JP2012231505A (en) * 2005-09-27 2012-11-22 Qualcomm Inc Video encoding method enabling highly efficient partial decoding of h.264 and other transform coded information
WO2007038727A2 (en) * 2005-09-27 2007-04-05 Qualcomm Incorporated Video encoding method enabling highly efficient partial decoding of h.264 and other transform coded information
JP4577778B2 (en) * 2005-09-28 2010-11-10 株式会社Ｋｄｄｉ研究所 Video coding and decoding method
JP2007096679A (en) * 2005-09-28 2007-04-12 Kddi R & D Laboratories Inc Method of encoding and decoding moving picture
EP1811789A2 (en) * 2006-01-23 2007-07-25 Samsung Electronics Co., Ltd. Video coding using directional interpolation
EP1811789A3 (en) * 2006-01-23 2012-07-11 Samsung Electronics Co., Ltd. Video coding using directional interpolation
JP5032344B2 (en) * 2006-02-06 2012-09-26 パナソニック株式会社 Image decoding apparatus and image decoding method
US8750369B2 (en) 2007-10-16 2014-06-10 Lg Electronics Inc. Method and an apparatus for processing a video signal
EP2210421A4 (en) * 2007-10-16 2013-12-04 Lg Electronics Inc A method and an apparatus for processing a video signal
US10306259B2 (en) 2007-10-16 2019-05-28 Lg Electronics Inc. Method and an apparatus for processing a video signal
US9813702B2 (en) 2007-10-16 2017-11-07 Lg Electronics Inc. Method and an apparatus for processing a video signal
US8750368B2 (en) 2007-10-16 2014-06-10 Lg Electronics Inc. Method and an apparatus for processing a video signal
US10820013B2 (en) 2007-10-16 2020-10-27 Lg Electronics Inc. Method and an apparatus for processing a video signal
US8761242B2 (en) 2007-10-16 2014-06-24 Lg Electronics Inc. Method and an apparatus for processing a video signal
US8867607B2 (en) 2007-10-16 2014-10-21 Lg Electronics Inc. Method and an apparatus for processing a video signal
EP2210421A2 (en) * 2007-10-16 2010-07-28 LG Electronics Inc. A method and an apparatus for processing a video signal
US8724692B2 (en) 2010-01-08 2014-05-13 Nokia Corporation Apparatus, a method and a computer program for video coding
WO2011083439A1 (en) * 2010-01-08 2011-07-14 Nokia Corporation An apparatus, a method and a computer program for video coding
EP2388999A3 (en) * 2010-05-17 2013-01-23 Lg Electronics Inc. New intra prediction modes
US9083974B2 (en) 2010-05-17 2015-07-14 Lg Electronics Inc. Intra prediction modes
KR101645787B1 (en) 2011-04-25 2016-08-04 엘지전자 주식회사 Intra-prediction method, and encoder and decoder using same
KR20140007465A (en) * 2011-04-25 2014-01-17 엘지전자 주식회사 Intra-prediction method, and encoder and decoder using same
US11910010B2 (en) 2011-04-25 2024-02-20 Lg Electronics Inc. Intra-prediction method, and encoder and decoder using same
JP2016077006A (en) * 2011-04-25 2016-05-12 エルジー エレクトロニクス インコーポレイティド Intra-prediction method and encoder and decoder using same
RU2550539C1 (en) * 2011-04-25 2015-05-10 ЭлДжи ЭЛЕКТРОНИКС ИНК. Internal prediction method and encoder and decoder using same
EP4224844A3 (en) * 2011-04-25 2023-09-27 LG Electronics Inc. Intra-prediction method for video decoding and video encoding
KR102350988B1 (en) 2011-04-25 2022-01-13 엘지전자 주식회사 Intra-prediction method, and encoder and decoder using same
EP2704435A4 (en) * 2011-04-25 2014-11-26 Lg Electronics Inc Intra-prediction method, and encoder and decoder using same
KR101763496B1 (en) 2011-04-25 2017-07-31 엘지전자 주식회사 Intra-prediction method, and encoder and decoder using same
KR20210054045A (en) * 2011-04-25 2021-05-12 엘지전자 주식회사 Intra-prediction method, and encoder and decoder using same
EP2704435A2 (en) * 2011-04-25 2014-03-05 LG Electronics Inc. Intra-prediction method, and encoder and decoder using same
US11006146B2 (en) 2011-04-25 2021-05-11 Lg Electronics Inc. Intra-prediction method, and encoder and decoder using same
GB2505345B (en) * 2011-04-25 2018-05-09 Lg Electronics Inc Intra-prediction method, and encoder and decoder using same
KR101855951B1 (en) 2011-04-25 2018-05-09 엘지전자 주식회사 Intra-prediction method, and encoder and decoder using same
KR102250105B1 (en) 2011-04-25 2021-05-10 엘지전자 주식회사 Intra-prediction method, and encoder and decoder using same
KR20180051656A (en) * 2011-04-25 2018-05-16 엘지전자 주식회사 Intra-prediction method, and encoder and decoder using same
KR20200118242A (en) * 2011-04-25 2020-10-14 엘지전자 주식회사 Intra-prediction method, and encoder and decoder using same
KR101968325B1 (en) 2011-04-25 2019-04-12 엘지전자 주식회사 Intra-prediction method, and encoder and decoder using same
KR20190040088A (en) * 2011-04-25 2019-04-16 엘지전자 주식회사 Intra-prediction method, and encoder and decoder using same
AU2012249024B2 (en) * 2011-04-25 2015-07-23 Lg Electronics Inc. Intra-prediction method, and encoder and decoder using same
KR102164571B1 (en) 2011-04-25 2020-10-12 엘지전자 주식회사 Intra-prediction method, and encoder and decoder using same
US10368090B2 (en) 2011-04-25 2019-07-30 Lg Electronics Inc. Intra-prediction method, and encoder and decoder using same
KR102081350B1 (en) 2011-04-25 2020-02-25 엘지전자 주식회사 Intra-prediction method, and encoder and decoder using same
US10645415B2 (en) 2011-04-25 2020-05-05 Lg Electronics Inc. Intra-prediction method, and encoder and decoder using same
KR20200020986A (en) * 2011-04-25 2020-02-26 엘지전자 주식회사 Intra-prediction method, and encoder and decoder using same
US10244262B2 (en) 2011-07-12 2019-03-26 Futurewei Technologies, Inc. Pixel-based intra prediction for coding in HEVC
US9516349B2 (en) 2011-07-12 2016-12-06 Futurewei Technologies, Inc. Pixel-based intra prediction for coding in HEVC
WO2013067435A1 (en) * 2011-11-04 2013-05-10 Huawei Technologies Co., Ltd. Differential pulse code modulation intra prediction for high efficiency video coding
US9813733B2 (en) 2011-11-04 2017-11-07 Futurewei Technologies, Inc. Differential pulse code modulation intra prediction for high efficiency video coding
US9503750B2 (en) 2011-11-04 2016-11-22 Futurewei Technologies, Inc. Binarization of prediction residuals for lossless video coding
US9253508B2 (en) 2011-11-04 2016-02-02 Futurewei Technologies, Inc. Differential pulse code modulation intra prediction for high efficiency video coding
US10574984B2 (en) 2015-09-10 2020-02-25 Lg Electronics Inc. Intra prediction method and device in video coding system
KR102160667B1 (en) 2015-09-10 2020-09-28 엘지전자 주식회사 Intra prediction method and apparatus in video coding system
EP3349448A4 (en) * 2015-09-10 2019-06-26 LG Electronics Inc. Intra prediction method and device in video coding system
CN108028923A (en) * 2015-09-10 2018-05-11 Lg电子株式会社 Intra-frame prediction method and equipment in video coding system
KR20180034665A (en) * 2015-09-10 2018-04-04 엘지전자 주식회사 Method and apparatus for intra prediction in video coding system
CN108028923B (en) * 2015-09-10 2021-08-20 Lg电子株式会社 Intra-frame prediction method and apparatus in video coding system

Also Published As

Publication number Publication date
EP1645135A4 (en) 2010-03-03
CN1720747A (en) 2006-01-11
MXPA05007449A (en) 2005-09-12
CN100584024C (en) 2010-01-20
EP1645135A1 (en) 2006-04-12
AU2003248913A1 (en) 2004-08-10
US20060072676A1 (en) 2006-04-06
BR0317966A (en) 2005-11-29
JP4474288B2 (en) 2010-06-02
JP2006513635A (en) 2006-04-20
KR20050090451A (en) 2005-09-13

Similar Documents

Publication Publication Date Title
WO2004064406A1 (en) Defining interpolation filters for error concealment in a coded image
CN1323553C (en) Spatial error concealment based on the intra-prediction modes transmitted in a coded stream
US10158862B2 (en) Method and apparatus for intra prediction within display screen
EP2920964B1 (en) Method of cross color intra prediction
JP4357590B2 (en) Image encoding system and image decoding system
US8224100B2 (en) Method and device for intra prediction coding and decoding of image
KR101563834B1 (en) Image decoding device, image decoding method, image encoding device, and image encoding method
US20130034159A1 (en) Decoder, encoder, method for decoding and encoding, data stream
US20060110056A1 (en) Deblocker for postprocess deblocking
EP4140130A1 (en) Compresssed picture-in-picture signaling

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
WWE Wipo information: entry into national phase

Ref document number: 2635/DELNP/2005

Country of ref document: IN

WWE Wipo information: entry into national phase

Ref document number: 2003815173

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: PA/a/2005/007449

Country of ref document: MX

Ref document number: 20038257750

Country of ref document: CN

Ref document number: 2004566420

Country of ref document: JP

Ref document number: 1020057012838

Country of ref document: KR

ENP Entry into the national phase

Ref document number: 2006072676

Country of ref document: US

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 10541929

Country of ref document: US

WWP Wipo information: published in national office

Ref document number: 1020057012838

Country of ref document: KR

ENP Entry into the national phase

Ref document number: PI0317966

Country of ref document: BR

WWP Wipo information: published in national office

Ref document number: 10541929

Country of ref document: US

WWP Wipo information: published in national office

Ref document number: 2003815173

Country of ref document: EP