US20010020906A1 - Deblocking filter for encoder/decoder arrangement and method with divergence reduction - Google Patents

Deblocking filter for encoder/decoder arrangement and method with divergence reduction Download PDF

Info

Publication number
US20010020906A1
US20010020906A1 US09/734,280 US73428000A US2001020906A1 US 20010020906 A1 US20010020906 A1 US 20010020906A1 US 73428000 A US73428000 A US 73428000A US 2001020906 A1 US2001020906 A1 US 2001020906A1
Authority
US
United States
Prior art keywords
loop
inverse transformer
image
encoder
terminal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
US09/734,280
Other versions
US6404361B2 (en
Inventor
Barry Andrews
Stephane Bryant
May Chiang
Ruili Hu
Katherine Kwan
Paul Ning
Paul Voois
Bryan Martin
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
8X8 Inc
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US07/838,382 external-priority patent/US5379351A/en
Priority claimed from US09/005,053 external-priority patent/US6124882A/en
Priority claimed from US09/064,397 external-priority patent/US6215425B1/en
Priority claimed from US09/098,106 external-priority patent/US6441842B1/en
Priority to US09/734,280 priority Critical patent/US6404361B2/en
Application filed by Individual filed Critical Individual
Publication of US20010020906A1 publication Critical patent/US20010020906A1/en
Assigned to NETERGY MICROELECTRONICS, INC. reassignment NETERGY MICROELECTRONICS, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: 8X8, INC.
Publication of US6404361B2 publication Critical patent/US6404361B2/en
Application granted granted Critical
Assigned to 8X8, INC. reassignment 8X8, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: NETERGY MICROELECTRONICS, INC.
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/14Systems for two-way working
    • H04N7/15Conference systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00Image coding
    • G06T9/007Transform coding, e.g. discrete cosine transform
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H04N19/86Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/14Systems for two-way working
    • H04N7/141Systems for two-way working between two video terminals, e.g. videophone

Definitions

  • the present invention relates to data compression and decompression in an encoding/decoding system.
  • the present invention has particular application for image coding/decoding systems where communicating terminals employ reconstruction algorithms based on previously decoded images.
  • Videocommunication equipment such as videoconferencing systems and videophone devices, have enabled people to communicate visually without having to travel to a common location. As a result, communication participants can be separated by large distances.
  • a typical videoconferencing uses a video camera to capture a series of images of a target, such as a meeting participant or a document.
  • the series of images is encoded as a data stream and transmitted over a communications channel to a remote location.
  • the data stream may be transmitted over a phone line, an integrated services digital network (ISDN) line, or the Internet.
  • ISDN integrated services digital network
  • the encoding process is typically implemented using a digital video coder/decoder (codec), which divides the images into blocks and compresses the blocks according to a video compression standard, such as the H.263 and H.261 recommendations by the Telecommunication Standardization Sector of the International Telecommunication Union (ITU-T).
  • a block may be compressed independent of the previous image or as a difference between the block and part of the previous image.
  • the data stream is received at a remote location, where it is decoded into a series of images, which may be viewed at the remote location. Depending on the equipment used, this process typically occurs at a rate of one to thirty frames per second.
  • a method for transmitting a representation of an image from a first terminal to a second terminal uses a communications channel on which communication has been established between the first terminal and the second terminal.
  • the method operates each of the first and second terminals using an inverse transformer loop having a decoding algorithm that accumulates at least one error, and selectively uses a loop filter in the inverse transformer loop based on at least one predetermined condition.
  • the predetermined condition includes at least one of: surpassing a quantization parameter threshold; and processing chroma channel data.
  • the present invention provides a method for encoding and decoding a representation of an image for communication between a first terminal to a second terminal.
  • the method comprises: operating each of the first and second terminals using an inverse transformer loop having a decoding algorithm; and preventing unacceptable accumulation of an error within the inverse transformer loop by at least one of: pre-processing input data before using the inverse transformer loop to process the input data; using the inverse transformer in the loop according to a bit-exact specification between the encoder and decoder in the resepctive first and second terminals; using the inverse transformer in the loop according to one of a plurality of bit-exact specifications between the encoder and decoder in the respective first and second terminals and negotiating a common decision on the particular specification through encoder/decoder negotiation; providing a quantization value in the inverse transformer loop that takes into account that the effect of the quantization values on loop filter divergence so as to be less prone to divergence; adjusting or providing transform coefficients to
  • Certain other aspects of the invention are directed to encoding and decoding equipment and circuits constructed and arranged to operate according to the above methodology.
  • FIG. 1 illustrates a videoconferencing system implementing an embodiment of the present invention
  • FIGS. 2, 2A and 2 B illustrate in block diagram form various additional embodiments of an encoding arrangement according to the present invention
  • FIG. 3 illustrates in block diagram form an embodiment of a decoding arrangement, according to the present invention, that is useful for decoding images sourced from the encoding arrangement of FIG. 2 and generally consistent with other ones of the illustrated encoding/decoding methods and processes discussed below;
  • FIG. 4 illustrates in block diagram form another embodiment of an encoding arrangement according to the present invention
  • FIG. 5 illustrates in block diagram form another embodiment of an encoding arrangement according to the present invention
  • FIG. 6 illustrates in block diagram form another embodiment of an encoding arrangement according to the present invention
  • FIG. 7 illustrates in block diagram form another embodiment of an encoding/decoding arrangement according to the present invention.
  • FIGS. 8A and 8B illustrate in graphic form characteristics of respective transform loops according to particular embodiments of the present invention.
  • FIG. 9 illustrates in block diagram form another embodiment of an encoding/decoding arrangement according to the present invention.
  • the present invention is believed to be applicable to a variety of systems and arrangements that transmit and decode compressed images.
  • the present invention has been found to be particularly advantageous in videoconferencing applications in which the reconstruction algorithms of the compressor encoder and compressor decoder are not identical.
  • a particular example application is an application intending to be compliant with the ITU-T H.263 Annex J recommendation where the accuracy of the prediction operation at the communicating terminals is defined using statistical data, rather than a bit-exact or operation-specific methodology at both ends. While the present invention is not necessarily limited to such applications, an appreciation of various aspects of the invention is best gained through a discussion of example arrangements operating in this type of application.
  • the problem can be best understood by referring to an example image-communication system, such as the videoconferencing system 10 of FIG. 1 and the other illustrated encoding and decoding arrangements.
  • FIG. 1 illustrates an example videoconferencing system 10 particularly suited for use in connection with the present invention.
  • a first videoconferencing station 12 includes a first terminal 14 that receives images from a camera 16 .
  • the images are processed by a codec in the terminal 14 and are transmitted over a communications channel 20 .
  • the images may be transmitted over a telephone line, an ISDN line, an Internet channel, or a similar communications network.
  • the images are then received by a second videoconferencing station 22 , including a second terminal configured to be compatible with the first terminal.
  • the first and second terminals 14 and 24 are respectively referred to as local and remote terminals. It should be understood, however, that the features described can be incorporated into either or both terminals.
  • the remote videoconferencing station 22 may further include a camera 26 and a codec 28 .
  • the remote videoconferencing station 22 receives the images over the communications channel 20 from the local videoconferencing station 12 and decodes them using the codec 28 .
  • the decoded images may be displayed on a display monitor 30 . In this manner, meetings can be conducted, for example, with participants separated by long distances with all meeting participants able to see each other.
  • FIG. 2 illustrates another embodiment of an encoding arrangement 300 that is particularly suited for use in the present invention.
  • An original image is stored in a memory 301 , such as a frame buffer.
  • the original image is received by a summing element 302 , which is configured and arranged to calculate difference information representing the difference between the original image and an image stored in a memory 304 , such as a frame buffer.
  • a transformation block 306 When no image is initially stored in the memory 304 , the original image is passed to a transformation block 306 , which transforms the image into coefficient data selected from a continuous range of values.
  • the transformation block 306 performs a discrete cosine transform (DCT) on the original image.
  • DCT discrete cosine transform
  • the coefficient data is further transformed by a transformation block, illustrated as a quantizer 308 , which is controlled by a coding control block 310 .
  • the quantizer 308 maps the coefficient data to a discrete set of values by dividing the continuous range of values into a set of non-overlapping subranges. Each subrange is mapped to a single value, such that whenever a coefficient falls within a given subrange, the quantizer 308 generates the corresponding discrete value.
  • the size of the subranges and the quality of the quantization are controlled by the coding control block 310 .
  • the quantized data thus generated is encoded by an encoding block, illustrated as a Huffman encoder 312 .
  • the quantized data may be encoded as a run length vector.
  • the encoded data is stored in a memory 314 for transmission over a communications channel 316 .
  • the quantized data is also received by a transformation block 318 .
  • the transformation block 318 implemented as an inverse quantizer in the illustrated example, maps the quantized data to a continuous range of values.
  • the quality of the inverse quantization is controlled by the coding control block 310 .
  • a second transformation block 320 further transforms the quantized data using an inverse discrete cosine transform (IDCT) to reconstruct the image as it would be seen at a remote videoconferencing station.
  • IDCT inverse discrete cosine transform
  • the reconstructed image is incorporated into the image previously stored in the memory 304 by a summing element 322 , and the image thus produced is stored in the memory 304 as the new reference frame for encoding the next frame.
  • the memory 304 provides the reconstructed image to the summing element 302 , and the process is repeated, either by continuing to encode successive frames of video, or by progressively improving a single still image.
  • FIG. 2A illustrates another example embodiment of a decoding arrangement 350 .
  • the functional blocks illustrated in FIG. 2A correspond and perform as described in connection with the corresponding blocks of FIG. 2, with certain exceptions relating to the coding control block 310 of FIG. 2.
  • the coding control block 310 ′ is used not only to control the quality of the quantization and inverse quantization as discussed in connection with FIG. 2, but also to determine when the characteristics of the incoming data merit use of the deblocking filter 303 ′ of FIG. 2A.
  • divergence can be reduced by disabling the deblocking filter for those frames with picture or average quantizations that are less than a selected threshold.
  • the perceptual advantage provided by the deblocking filter is present only at higher quantizations.
  • the perceptual advantage provided by the deblocking filter is present when the quantization parameter (PQUANT) is at least five.
  • the coding control block 310 ′ is coupled to an enable/disable port of the deblocking filter 303 ′.
  • the coding control block 310 ′ uses a comparator block 315 to determine whether or not PQUANT is less than the selected threshold, “five” in this instance.
  • the selected threshold is checked on a frame-by-frame basis (after encoding each frame) or, alternatively, using an average of a number of previously coded frames. In a particular embodiment, several previously coded frames are used to form the average.
  • the coding control block 310 ′ monitors PQUANT and uses the average PQUANT value over some number of frames in order to compare with the threshold. This provides for less-frequent enable/disable transitions for the deblocking filter.
  • FIG. 2A Any of the above-described embodiments of FIG. 2A is advantageous because each realizes a significant reduction in both divergence artifacts and computation.
  • FIG. 2B illustrates yet another example embodiment of a decoding arrangement 370 in which a deblocking filter 303 ′′ includes an enable/disable port controlled by a channel selector 372 .
  • the channel selector 372 includes conventional channel selection addressing for sequentially selecting the luma channel and chroma channels for each frame in the memory 301 .
  • the channel selector 372 disables the deblocking filter 303 ′′ when the chroma channels of the memory 301 are selected for encoding.
  • This implementation realizes an elimination of prominent divergence artifacts, including a lack of significant blockiness. Although luma-based divergence artifacts remain, such artifacts are much less severe than chroma-based artifacts.
  • the processing of the luma and chroma channels for each frame in the memory 301 is performed on a parallel-path basis (not shown), rather than the above-described sequential or alternating luma/chroma channel processing method.
  • a deblocking filter is used only for the portion of the encoding arrangement that encodes the luma channel data, and not for the portion of the encoding arrangement that encodes the chroma channel data.
  • two or more of the above criteria are used at the same time to minimize and/or remove divergence artifacts in the encoding arrangement.
  • the deblocking filter is disabled when PQUANT is less than the selected threshold and/or when chroma channel data is being encoded.
  • FIG. 3 illustrates another example embodiment of a decoding arrangement 400 , also including a loop filter like the loop filter of FIG. 2, that is also particularly suited for use in the present invention.
  • a data stream received over a communications channel 402 is stored in a memory 404 , implemented as a buffer in the illustrated example.
  • a decoder illustrated as a Huffman decoder 406 , decodes the data stream into image data and provides the image data to a transformation block 408 .
  • the transformation block 408 implemented as an inverse quantizer in the illustrated example, maps the image data to a continuous range of values.
  • a second transformation block 410 further transforms the continuous value using an inverse discrete cosine transform (IDCT) to reconstruct the difference information received from the local terminal.
  • IDCT inverse discrete cosine transform
  • a summing element 414 integrates the difference information into the image previously stored in a memory 412 , depicted as a frame buffer, and the image is filtered by a loop filter 411 (constructed similar to the loop filter 303 of FIG. 2). The image thus produced is stored in the memory 412 and displayed by a display device 413 . Additional data received over the communications channel 402 is integrated into the stored image to improve its quality.
  • Encoder UBC encoder tmn-3.1.2 using their floating point IDCT, with Annex J enabled. For simplicity, only enable loop filter with Annex J, and keep unrestricted motion vector mode and four motion vector modes OFF.
  • Decoder UBC decoder tmndec-3.1.2 using a modified fixed-point IDCT.
  • the fixed-point IDCT was modified to make it H.263 Annex A compliant and is listed in Appendix A.
  • Test sequence A one hundred frame natural sequence showing a stationary color grid is included in a file entitled “rawseq” as discussed in the above-referenced paper submitted to the ITU-T.
  • the threshold is reached, divergence may have already leaked into other MBs under the following conditions: (1) uncoded blocks that border coded blocks are filtered based on the pixels in the coded blocks; these uncoded blocks are not fixed by the refresh since their threshold are never triggered; and (2) MVs of neighboring MBs may reference areas of divergence, the neighboring MBs are not fixed by the refresh if they stop referencing the refreshed MB.
  • Condition (1) is specific to the loop filter and gives rise to an interesting scenario. Refreshing a MB may not even fix divergence within the refreshed MB since the loop filter may immediately pull in divergence errors just outside the MB boundaries.
  • the observed visual artifacts indicate that the loop filter is not stable in a control sense; i.e., a small difference in the encoder/decoder is amplified by the filter. This amplification is demonstrated for a specific range of the filter in the following example.
  • the present invention provides a method for encoding and decoding involving the use of an inverse transform that is specified bit-exactly in encoder and decoder.
  • the encoding/decoding arrangement of FIGS. 2 and 3 provide this approach, using a specified bit-exact match. This avoids encoder/decoder inverse transform mismatch, thereby eliminating divergence artifacts.
  • the present invention provides a method for encoding and decoding involving the use of an inverse transform that is specified bit-exactly in encoder and decoder, as described above, but the inverse transform is selected at the run-time from a set of available inverse transforms.
  • the communications terminals then negotiate to reach a common inverse transform so that encoder/decoder inverse transforms are matched, thereby eliminating divergence artifacts.
  • FIGS. 2 and 3 illustrate an alternative encoding/decoding arrangement, according to the present invention, using an optional inverse transform selector at each end to provide the appropriate matching.
  • the present invention provides a method for encoding and decoding involving the step of pre-processing image data before encoding in the loop. For example, by applying a filter to the entire image, or selectively in regions that may give rise to divergence artifacts, any potential inverse transform mismatch from being significantly enhanced by the loop filter is prevented to reduce divergence artifacts.
  • a filter for example, by applying a filter to the entire image, or selectively in regions that may give rise to divergence artifacts, any potential inverse transform mismatch from being significantly enhanced by the loop filter is prevented to reduce divergence artifacts.
  • the present invention provides a method for encoding and decoding involving the use of a modified encoding arrangement to reduce errors.
  • Example implementations include one or more of the following: an encoding arrangement that selects a quantization value which is less prone to divergence; an encoding arrangement that adjusts transform coefficients to limit divergence; an encoding arrangement that adjusts the encoding mode decision for macroblocks; and an encoding arrangement that uses combinations of the above approaches.
  • Such an implementation is shown in FIG. 5, where an anti-divergence encoding adjustment block 460 is used to provide one or more of these adjustments or selections.
  • another embodiment of the present invention involves the encoding arrangement determining at block 470 which portions of the image are diverging.
  • the portions are INTRA coded (refreshed) to terminate any divergence.
  • the refresh is accomplished by coding the regions in INTRA mode, with no dependencies on previous reference data.
  • a conventional encoding arrangement is modified as above to refresh portions of the image that are diverging and coding the regions in INTRA mode, but based on information conveyed from the decoder via a feedback channel, such as a channel 472 carrying control data between encoder refresh controller 474 and decoder deblocking feedback controller 476 .
  • the information may include, for example, copies of the decoder's reference frame at a particular time, or portions thereof. This similarly terminates divergence artifacts before they become significant.
  • the modified set of loop filter characteristics shown in FIG. 8B are used rather than the ITU-T set of FIG. 8A so that inverse transform mismatches are not amplified.
  • One or more of the loop filter equations, the strength table, and the definition of the UpDownRamp curve, are adjusted to reduce loop filter amplification of inverse transform mismatch.
  • the encoding arrangement is adaptively modified depending on the motion in the scene.
  • a more particular embodiment involves such adaptive modification applied non-uniformly to luma and chroma components of the image data.
  • such adaptive modification is applied uniformly to luma and chroma components of the image data.
  • the loop filter is applied to the frame buffer image just before sending the data to a display circuit, but the results are not written back to the frame buffer. This moves the loop filter out of the codec loop and preserves some of the benefits of deblocking, but any deblocking amplification of inverse transform mismatch is not fed back to avoid error build up and to eliminate divergence artifacts.
  • an encoding/decoding method and arrangement uses an inverse transform loop having a decoding algorithm that: is susceptible to accumulating error due to inverse transformer mismatch with the method and arrangement providing control to maintain the accumulation below an acceptable threshold; and/or eliminates the possibility of such an error being accumulated within the loop.

Abstract

According to one aspect of the disclosure, the present invention provides methods and arrangements for the removal or reduction of divergence artifacts between a transmitting codec and a receiving codec. One of a number of implementations includes using a deblocking filter in an inverse transformer loop and selectively disabling the filter upon certain conditions.

Description

  • RELATED APPLICATIONS [0001]
  • This is a continuation-in-part of U.S. patent application Ser. No. 09/064,397, filed on Apr. 22, 1998 (Docket No. 8X8S.208PA).[0002]
  • FIELD OF THE INVENTION
  • The present invention relates to data compression and decompression in an encoding/decoding system. The present invention has particular application for image coding/decoding systems where communicating terminals employ reconstruction algorithms based on previously decoded images. [0003]
  • BACKGROUND OF THE INVENTION
  • Videocommunication equipment, such as videoconferencing systems and videophone devices, have enabled people to communicate visually without having to travel to a common location. As a result, communication participants can be separated by large distances. [0004]
  • A typical videoconferencing uses a video camera to capture a series of images of a target, such as a meeting participant or a document. The series of images is encoded as a data stream and transmitted over a communications channel to a remote location. For example, the data stream may be transmitted over a phone line, an integrated services digital network (ISDN) line, or the Internet. The encoding process is typically implemented using a digital video coder/decoder (codec), which divides the images into blocks and compresses the blocks according to a video compression standard, such as the H.263 and H.261 recommendations by the Telecommunication Standardization Sector of the International Telecommunication Union (ITU-T). In standards of this type, a block may be compressed independent of the previous image or as a difference between the block and part of the previous image. [0005]
  • In a typical videoconferencing system, the data stream is received at a remote location, where it is decoded into a series of images, which may be viewed at the remote location. Depending on the equipment used, this process typically occurs at a rate of one to thirty frames per second. [0006]
  • In some videoconferencing applications, it is desirable to transmit a high quality still image. Until the image is completely received and decoded, the receiving terminal is often unaware of its content. Some decoders decode and display a block only after they have received the complete image. With the image being transmitted as a series of blocks, considerable delay is often involved in transmitting the entire image. For example, in applications where the available bandwidth for transmitting data is small, transmission of a 352×288 pixel image may require up to a minute. In order to transmit still images more quickly, the image may be highly compressed. [0007]
  • The above-mentioned Telecommunication Standardization Sector recently revised ITU-T H.263 recommendation by appending thereto Annex J: Deblocking Filter Mode. This annex describes an optional loop filter (also referred to as block edge filter or deblocking filter) to be used within the prediction loop used for coding in each of the send and receive terminals in image communicating system. The main purpose of the loop filter is to reduce blocking artifacts. Such artifacts often appear at boundaries between different image blocks. The above-mentioned annex, not unlike other recommendations by the ITU, was adopted after much research and consideration for the purpose of providing communicating image terminals of various types and manufacturers the ability to communicate images accurately. [0008]
  • In connection with the present invention, a significant discovery has been made. After intense research, it has been discovered that respective implementations of two image communication terminals, each fully compliant with the above-mentioned annex to the ITU-T H.263 recommendation result in visually apparent artifacts which result from divergence between sending and receiving terminals. Such artifacts appear as long as the loop filter portion of Annex J is used, irrespective of whether the other portions of Annex J (Unrestricted Motion Vectors, 4MV) are also used. The response by the ITU-T was one of surprise. [0009]
  • SUMMARY OF THE INVENTION
  • Generally, the present invention provides methods and arrangements for the removal or reduction of divergence artifacts between the transmitting codec and the receiving code. In one general embodiment of the invention, a method for transmitting a representation of an image from a first terminal to a second terminal uses a communications channel on which communication has been established between the first terminal and the second terminal. The method operates each of the first and second terminals using an inverse transformer loop having a decoding algorithm that accumulates at least one error, and selectively uses a loop filter in the inverse transformer loop based on at least one predetermined condition. In a more specific embodiment, the predetermined condition includes at least one of: surpassing a quantization parameter threshold; and processing chroma channel data. [0010]
  • In another method embodiment, the present invention provides a method for encoding and decoding a representation of an image for communication between a first terminal to a second terminal. The method comprises: operating each of the first and second terminals using an inverse transformer loop having a decoding algorithm; and preventing unacceptable accumulation of an error within the inverse transformer loop by at least one of: pre-processing input data before using the inverse transformer loop to process the input data; using the inverse transformer in the loop according to a bit-exact specification between the encoder and decoder in the resepctive first and second terminals; using the inverse transformer in the loop according to one of a plurality of bit-exact specifications between the encoder and decoder in the respective first and second terminals and negotiating a common decision on the particular specification through encoder/decoder negotiation; providing a quantization value in the inverse transformer loop that takes into account that the effect of the quantization values on loop filter divergence so as to be less prone to divergence; adjusting or providing transform coefficients to limit divergence; choosing an encoding mode for macro-blocks of input data; modifying the encoder to refresh portions of the image that are diverging; modify the loop filter characteristics so that inverse transform mismatches are not amplified; applying the loop filter to input data corresponding to a frame buffer image just before sending to a display circuit without writing back the results to a frame buffer in the loop. [0011]
  • Certain other aspects of the invention are directed to encoding and decoding equipment and circuits constructed and arranged to operate according to the above methodology. [0012]
  • The above summary is not intended to characterize each embodiment of the present invention. Other aspects of the present invention will become apparent upon review of the figures and corresponding “Detailed Description”.[0013]
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • Various aspects and advantages of the present invention will become apparent upon reading the following detailed description of various embodiments and upon reference to the drawings in which: [0014]
  • FIG. 1 illustrates a videoconferencing system implementing an embodiment of the present invention; [0015]
  • FIGS. 2, 2A and [0016] 2B illustrate in block diagram form various additional embodiments of an encoding arrangement according to the present invention;
  • FIG. 3 illustrates in block diagram form an embodiment of a decoding arrangement, according to the present invention, that is useful for decoding images sourced from the encoding arrangement of FIG. 2 and generally consistent with other ones of the illustrated encoding/decoding methods and processes discussed below; [0017]
  • FIG. 4 illustrates in block diagram form another embodiment of an encoding arrangement according to the present invention; [0018]
  • FIG. 5 illustrates in block diagram form another embodiment of an encoding arrangement according to the present invention; [0019]
  • FIG. 6 illustrates in block diagram form another embodiment of an encoding arrangement according to the present invention; [0020]
  • FIG. 7 illustrates in block diagram form another embodiment of an encoding/decoding arrangement according to the present invention; [0021]
  • FIGS. 8A and 8B illustrate in graphic form characteristics of respective transform loops according to particular embodiments of the present invention; and [0022]
  • FIG. 9 illustrates in block diagram form another embodiment of an encoding/decoding arrangement according to the present invention.[0023]
  • While the invention is amenable to various modifications and alternative forms, specifics thereof have been shown by way of example in the drawings and will be described in detail. It should be understood, however, that the intention is not to limit the invention to the particular embodiment described. On the contrary, the invention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined by the appended claims. [0024]
  • Detailed Description [0025]
  • The present invention is believed to be applicable to a variety of systems and arrangements that transmit and decode compressed images. The present invention has been found to be particularly advantageous in videoconferencing applications in which the reconstruction algorithms of the compressor encoder and compressor decoder are not identical. A particular example application is an application intending to be compliant with the ITU-T H.263 Annex J recommendation where the accuracy of the prediction operation at the communicating terminals is defined using statistical data, rather than a bit-exact or operation-specific methodology at both ends. While the present invention is not necessarily limited to such applications, an appreciation of various aspects of the invention is best gained through a discussion of example arrangements operating in this type of application. [0026]
  • In connection with the present invention, it has been discovered that the nature of the ITU-TH.263 Annex J recommendation permits the accumulation of seemingly negligible differences between transmitting codec's prediction frames and the receiving codec's prediction frames, therefore resulting in significant distortion of the receiving codec's decoded images. This occurs even though mismatched inverse transformations, in this case IDCTs, are compliant according to ITU-T H.263 Annex A (“Inverse Transform Accuracy Specification”). [0027]
  • The problem can be best understood by referring to an example image-communication system, such as the [0028] videoconferencing system 10 of FIG. 1 and the other illustrated encoding and decoding arrangements.
  • FIG. 1 illustrates an [0029] example videoconferencing system 10 particularly suited for use in connection with the present invention. A first videoconferencing station 12 includes a first terminal 14 that receives images from a camera 16. The images are processed by a codec in the terminal 14 and are transmitted over a communications channel 20. For example, the images may be transmitted over a telephone line, an ISDN line, an Internet channel, or a similar communications network. The images are then received by a second videoconferencing station 22, including a second terminal configured to be compatible with the first terminal. To facilitate the discussion below, the first and second terminals 14 and 24 are respectively referred to as local and remote terminals. It should be understood, however, that the features described can be incorporated into either or both terminals. The remote videoconferencing station 22 may further include a camera 26 and a codec 28. The remote videoconferencing station 22 receives the images over the communications channel 20 from the local videoconferencing station 12 and decodes them using the codec 28. The decoded images may be displayed on a display monitor 30. In this manner, meetings can be conducted, for example, with participants separated by long distances with all meeting participants able to see each other.
  • FIG. 2 illustrates another embodiment of an [0030] encoding arrangement 300 that is particularly suited for use in the present invention. An original image is stored in a memory 301, such as a frame buffer. The original image is received by a summing element 302, which is configured and arranged to calculate difference information representing the difference between the original image and an image stored in a memory 304, such as a frame buffer. When no image is initially stored in the memory 304, the original image is passed to a transformation block 306, which transforms the image into coefficient data selected from a continuous range of values. In the illustrated example, the transformation block 306 performs a discrete cosine transform (DCT) on the original image. To encode a stream of images, the original image is replaced with successive frames after encoding each frame.
  • The coefficient data is further transformed by a transformation block, illustrated as a [0031] quantizer 308, which is controlled by a coding control block 310. The quantizer 308 maps the coefficient data to a discrete set of values by dividing the continuous range of values into a set of non-overlapping subranges. Each subrange is mapped to a single value, such that whenever a coefficient falls within a given subrange, the quantizer 308 generates the corresponding discrete value. The size of the subranges and the quality of the quantization are controlled by the coding control block 310.
  • The quantized data thus generated is encoded by an encoding block, illustrated as a [0032] Huffman encoder 312. For example, the quantized data may be encoded as a run length vector. The encoded data is stored in a memory 314 for transmission over a communications channel 316.
  • The quantized data is also received by a [0033] transformation block 318. The transformation block 318, implemented as an inverse quantizer in the illustrated example, maps the quantized data to a continuous range of values. The quality of the inverse quantization is controlled by the coding control block 310. A second transformation block 320 further transforms the quantized data using an inverse discrete cosine transform (IDCT) to reconstruct the image as it would be seen at a remote videoconferencing station.
  • Using a [0034] loop filter 303 to smooth out transitions at image block boundaries, the reconstructed image is incorporated into the image previously stored in the memory 304 by a summing element 322, and the image thus produced is stored in the memory 304 as the new reference frame for encoding the next frame. The memory 304 provides the reconstructed image to the summing element 302, and the process is repeated, either by continuing to encode successive frames of video, or by progressively improving a single still image.
  • FIG. 2A illustrates another example embodiment of a [0035] decoding arrangement 350. The functional blocks illustrated in FIG. 2A correspond and perform as described in connection with the corresponding blocks of FIG. 2, with certain exceptions relating to the coding control block 310 of FIG. 2. In FIG. 2A, the coding control block 310′ is used not only to control the quality of the quantization and inverse quantization as discussed in connection with FIG. 2, but also to determine when the characteristics of the incoming data merit use of the deblocking filter 303′ of FIG. 2A. In accordance with the present invention, it has been discovered that divergence can be reduced by disabling the deblocking filter for those frames with picture or average quantizations that are less than a selected threshold.
  • Further, in connection with the present invention it has been discovered that the perceptual advantage provided by the deblocking filter is present only at higher quantizations. For example, the perceptual advantage provided by the deblocking filter is present when the quantization parameter (PQUANT) is at least five. [0036]
  • Accordingly, in the [0037] example decoding arrangement 350 of FIG. 2A, the coding control block 310′ is coupled to an enable/disable port of the deblocking filter 303′. The coding control block 310′ uses a comparator block 315 to determine whether or not PQUANT is less than the selected threshold, “five” in this instance. The selected threshold is checked on a frame-by-frame basis (after encoding each frame) or, alternatively, using an average of a number of previously coded frames. In a particular embodiment, several previously coded frames are used to form the average. Using this data flow arrangement, the coding control block 310′ monitors PQUANT and uses the average PQUANT value over some number of frames in order to compare with the threshold. This provides for less-frequent enable/disable transitions for the deblocking filter.
  • Any of the above-described embodiments of FIG. 2A is advantageous because each realizes a significant reduction in both divergence artifacts and computation. [0038]
  • FIG. 2B illustrates yet another example embodiment of a [0039] decoding arrangement 370 in which a deblocking filter 303″ includes an enable/disable port controlled by a channel selector 372. The channel selector 372 includes conventional channel selection addressing for sequentially selecting the luma channel and chroma channels for each frame in the memory 301. In addition, the channel selector 372 disables the deblocking filter 303″ when the chroma channels of the memory 301 are selected for encoding. This implementation realizes an elimination of prominent divergence artifacts, including a lack of significant blockiness. Although luma-based divergence artifacts remain, such artifacts are much less severe than chroma-based artifacts.
  • Alternatively, the processing of the luma and chroma channels for each frame in the [0040] memory 301 is performed on a parallel-path basis (not shown), rather than the above-described sequential or alternating luma/chroma channel processing method. In this alternative implementation, a deblocking filter is used only for the portion of the encoding arrangement that encodes the luma channel data, and not for the portion of the encoding arrangement that encodes the chroma channel data.
  • According to other example embodiments of the present invention, two or more of the above criteria are used at the same time to minimize and/or remove divergence artifacts in the encoding arrangement. In one implementation, for example, the deblocking filter is disabled when PQUANT is less than the selected threshold and/or when chroma channel data is being encoded. [0041]
  • FIG. 3 illustrates another example embodiment of a [0042] decoding arrangement 400, also including a loop filter like the loop filter of FIG. 2, that is also particularly suited for use in the present invention. A data stream received over a communications channel 402 is stored in a memory 404, implemented as a buffer in the illustrated example. A decoder, illustrated as a Huffman decoder 406, decodes the data stream into image data and provides the image data to a transformation block 408. The transformation block 408, implemented as an inverse quantizer in the illustrated example, maps the image data to a continuous range of values. A second transformation block 410 further transforms the continuous value using an inverse discrete cosine transform (IDCT) to reconstruct the difference information received from the local terminal. A summing element 414 integrates the difference information into the image previously stored in a memory 412, depicted as a frame buffer, and the image is filtered by a loop filter 411 (constructed similar to the loop filter 303 of FIG. 2). The image thus produced is stored in the memory 412 and displayed by a display device 413. Additional data received over the communications channel 402 is integrated into the stored image to improve its quality.
  • It has been discovered in connection with the present invention that visually apparent divergence artifacts occurs and that the visual artifacts worsen over time when the ITU-T H.263 Annex J loop filter is used in the case where the encoder IDCT differs from the decoder IDCT. Differences between sending codec and receiving codec IDCTs may arise due to difference in how many bits of precision are used in the arithmetic operations of the specific implementation of the IDCT, even though both IDCTs are H.263 Annex A compliant. Furthermore, the artifacts do not occur if the loop filter is turned off. It is therefore likely that the loop filter may enhance the mismatch between encoder and decoder IDCT. [0043]
  • To reproduce these described divergence artifacts, reference may be made to document Q15-D-41 submitted to the ITU-T Study Group 16 (paper dated Apr. 13, 1998). These divergence artifacts were discovered using the following setup: [0044]
  • Encoder: UBC encoder tmn-3.1.2 using their floating point IDCT, with Annex J enabled. For simplicity, only enable loop filter with Annex J, and keep unrestricted motion vector mode and four motion vector modes OFF. [0045]
  • >tmn -i rawseq -B bs-q I-a 0 -b 99 -q 2 -A 2 -k 0 -1 [0046]
  • Decoder: UBC decoder tmndec-3.1.2 using a modified fixed-point IDCT. The fixed-point IDCT was modified to make it H.263 Annex A compliant and is listed in Appendix A. [0047]
  • >tmndec -o4 -x - 1 bs-q I -j [0048]
  • Test sequence: A one hundred frame natural sequence showing a stationary color grid is included in a file entitled “rawseq” as discussed in the above-referenced paper submitted to the ITU-T. [0049]
  • The above simulations were used on the included dataset for QP values ranging from 1 to 31. Included in ql5d4lab.tar.gz are coded bitstreams where Annex J is both on and off (bs-ql-j, bs-ql), with QP=1. [0050]
  • For low QPs (O<QP<5), obvious divergence patterns appear along edges. For QP=1 (bs-ql-j) a vertical line along the right edge of the yellow patch, as well as spots along the 8×8 block boundaries in the lower right blue patch appear. When QP is increased, the patterns become blurred, but divergence patterns are still noticeable along the edges of the green center-low patch and the blue right-low patch up to QP=4. [0051]
  • Note that the INTRA MB refresh rule (threshold=132 coded MBs) does not prevent artifacts with the test sequence since the threshold is not reached. In general, even if the threshold is reached, divergence may have already leaked into other MBs under the following conditions: (1) uncoded blocks that border coded blocks are filtered based on the pixels in the coded blocks; these uncoded blocks are not fixed by the refresh since their threshold are never triggered; and (2) MVs of neighboring MBs may reference areas of divergence, the neighboring MBs are not fixed by the refresh if they stop referencing the refreshed MB. [0052]
  • Condition (1) is specific to the loop filter and gives rise to an interesting scenario. Refreshing a MB may not even fix divergence within the refreshed MB since the loop filter may immediately pull in divergence errors just outside the MB boundaries. [0053]
  • The observed visual artifacts indicate that the loop filter is not stable in a control sense; i.e., a small difference in the encoder/decoder is amplified by the filter. This amplification is demonstrated for a specific range of the filter in the following example. [0054]
  • The deblock equations are: [0055]
  • Bi=clip(B+di)
  • Cl=clip(C−di)
  • Al=A−d2
  • DI=D+d2
  • d=(A−4B+4C−D)/g
  • dI =UpDownRamp(d, STRENGTH)
  • d2=clipdI((A−D)/4, d ½)
  • Consider the case where A=D, and for simplicity no clipping is required. Then the equations are: [0056]
  • BI=B+dI
  • Cl=C−d1
  • Al=A
  • DI=D
  • d=(C−B)/2
  • dI=UpDownRainp((C−B)/2, STRENGTH)
  • d2=0
  • Note that only B and C get updated by the deblock filter. [0057]
  • Assume the encoder has [0058]
  • C′−B′>0
  • and the decoder has [0059]
  • C″−B″>0
  • and that there is a slight encoder/decoder IDCT mismatch such that [0060] ( C - B ) 2 d - ( C - B ) 2 d = A > 0
    Figure US20010020906A1-20010913-M00001
  • If these (d′, d″) are in the negative slope range of the UpDownRamp, then di′>di″. Now since [0061]
  • C1′−B1′=(C′−B′)−2d1′
  • C1″−BI″=(C″−B″)−2d1″
  • the encoder/decoder mismatch after the deblock filter will be: [0062]
  • A1(C1″−BI″)−(C1′−BI′)
  • (C″−B″)−(C′−B′)−2di″+2d1
  • A+2(d1′−di″)
  • The quantity (d I′−dI′) is positive and therefore [0063]
  • A1>A
  • which implies that the filter amplifies the encoder/decoder mismatch. [0064]
  • The present invention addresses this discovered problem using one or more of the embodiments that are discussed below. [0065]
  • According to one implementation, the present invention provides a method for encoding and decoding involving the use of an inverse transform that is specified bit-exactly in encoder and decoder. For example, the encoding/decoding arrangement of FIGS. 2 and 3 provide this approach, using a specified bit-exact match. This avoids encoder/decoder inverse transform mismatch, thereby eliminating divergence artifacts. [0066]
  • According to another implementation, the present invention provides a method for encoding and decoding involving the use of an inverse transform that is specified bit-exactly in encoder and decoder, as described above, but the inverse transform is selected at the run-time from a set of available inverse transforms. The comunicating terminals then negotiate to reach a common inverse transform so that encoder/decoder inverse transforms are matched, thereby eliminating divergence artifacts. FIGS. 2 and 3 illustrate an alternative encoding/decoding arrangement, according to the present invention, using an optional inverse transform selector at each end to provide the appropriate matching. [0067]
  • According to another implementation, the present invention provides a method for encoding and decoding involving the step of pre-processing image data before encoding in the loop. For example, by applying a filter to the entire image, or selectively in regions that may give rise to divergence artifacts, any potential inverse transform mismatch from being significantly enhanced by the loop filter is prevented to reduce divergence artifacts. Such an implementation is shown in FIG. 4, where the input data supplied by the [0068] camera 450 is filtered at filter 452 before presentation to the encoder 454 to avoid divergence build-up.
  • According to another implementation, the present invention provides a method for encoding and decoding involving the use of a modified encoding arrangement to reduce errors. Example implementations include one or more of the following: an encoding arrangement that selects a quantization value which is less prone to divergence; an encoding arrangement that adjusts transform coefficients to limit divergence; an encoding arrangement that adjusts the encoding mode decision for macroblocks; and an encoding arrangement that uses combinations of the above approaches. Such an implementation is shown in FIG. 5, where an anti-divergence [0069] encoding adjustment block 460 is used to provide one or more of these adjustments or selections.
  • As shown by way of example in the encoding arrangement of FIG. 6, another embodiment of the present invention involves the encoding arrangement determining at [0070] block 470 which portions of the image are diverging. The portions are INTRA coded (refreshed) to terminate any divergence. The refresh is accomplished by coding the regions in INTRA mode, with no dependencies on previous reference data.
  • In yet another embodiment of the present invention and as shown by way of example in FIG. 7, a conventional encoding arrangement is modified as above to refresh portions of the image that are diverging and coding the regions in INTRA mode, but based on information conveyed from the decoder via a feedback channel, such as a [0071] channel 472 carrying control data between encoder refresh controller 474 and decoder deblocking feedback controller 476. The information may include, for example, copies of the decoder's reference frame at a particular time, or portions thereof. This similarly terminates divergence artifacts before they become significant.
  • According to another implementation and as shown by way of example in FIGS. 8A and 8B, the modified set of loop filter characteristics shown in FIG. 8B are used rather than the ITU-T set of FIG. 8A so that inverse transform mismatches are not amplified. One or more of the loop filter equations, the strength table, and the definition of the UpDownRamp curve, are adjusted to reduce loop filter amplification of inverse transform mismatch. [0072]
  • In another embodiment, the encoding arrangement is adaptively modified depending on the motion in the scene. A more particular embodiment involves such adaptive modification applied non-uniformly to luma and chroma components of the image data. In an alternative particular embodiment, such adaptive modification is applied uniformly to luma and chroma components of the image data. Using either of these particular implementations, the adjustments to the encoder behavior reduce divergence artifacts. [0073]
  • According to yet another implementation implementation and as shown by way of example in FIG. 9, the loop filter is applied to the frame buffer image just before sending the data to a display circuit, but the results are not written back to the frame buffer. This moves the loop filter out of the codec loop and preserves some of the benefits of deblocking, but any deblocking amplification of inverse transform mismatch is not fed back to avoid error build up and to eliminate divergence artifacts. [0074]
  • Accordingly, the present invention provides various implementations that solve a significant problem discovered in connection herewith. According to particular implementations, an encoding/decoding method and arrangement uses an inverse transform loop having a decoding algorithm that: is susceptible to accumulating error due to inverse transformer mismatch with the method and arrangement providing control to maintain the accumulation below an acceptable threshold; and/or eliminates the possibility of such an error being accumulated within the loop. [0075]
  • For further information concerning the construction and operation of the encoding and decoding components and blocks illustrated in connection with the figures, reference may be made to U.S. patent application Ser. No. 08/944,311, filed Oct. 6, 1997, and No. 09/005,053, filed Jan. 9, 1998 (Docket Nos. 11611.14USI1, and 11611.51USO1), incorporated by reference in their entirety. [0076]
  • The various embodiments described above are provided by way of illustration only and should not be construed to limit the invention. Those skilled in the art will readily recognize various modifications and changes which may be made to the present invention. For example, various ones of the above approaches to eliminating or reducing artifacts are compatible and can therefore be combined. Such modifications may not strictly follow the above discussion of the exemplary embodiments and applications. However, these modifications do not depart from the teaching, or the true spirit and scope, of the present invention. The invention is set forth in the following claims. [0077]

Claims (13)

What is claimed is:
1. A method for transmitting a representation of an image from a first terminal to a second terminal using a communications channel, comprising:
operating each of the first and second terminals using an inverse transformer loop having a decoding algorithm that accumulates at least one error; and
selectively using a loop filter in the inverse transformer loop based on at least one predetermined condition.
2. A method, according to
claim 1
, wherein said at least one predetermined condition includes at least one of: surpassing a quantization parameter threshold; and processing chroma channel data.
3. A method, according to
claim 1
, wherein said at least one predetermined condition includes a quantization parameter threshold approximately equal to 5.
4. A method, according to
claim 3
, wherein said quantization parameter threshold being equal to 5 corresponds to an average of several quantization parameters.
5. A method, according to
claim 1
, further including refreshing.
6. A method for encoding and decoding a representation of an image for communication between a first terminal and a second terminal, comprising:
operating each of the first and second terminals using an inverse transformer loop; and preventing unacceptable accumulation of an error within the inverse transformer loop by at least one of: pre-processing input data before using the inverse transformer loop to process the input data; using the inverse transformer in the loop according to a bit-exact specification between the encoder and decoder in the respective first and second terminals; using the inverse transformer in the loop according to one of a plurality of bit-exact specifications between the encoder and decoder in the respective first and second terminals and negotiating a common decision on the particular specification through encoder/decoder negotiation; providing a quantization value in the inverse transformer loop that is less prone to divergence; adjusting or providing transform coefficients to limit divergence; choosing an encoding mode for macro-blocks of input data; modifying the encoder to refresh portions of the image that are diverging; modifying the loop filter characteristics so that inverse transform mismatches are not amplified; applying the loop filter to input data corresponding to a frame buffer image just before sending to a display circuit without writing back the results to a frame buffer in the loop; selectively using a loop filter in each inverse transformer loop based on at least one predetermined condition.
7. A method, according to
claim 6
, wherein said at least one predetermined condition includes at least one of: surpassing a quantization parameter threshold, and processing chroma channel data.
8. An image communication terminal having a circuit constructed and arranged to operate according to
claim 6
.
9. An image communication terminal having a circuit constructed and arranged to operate according to
claim 1
.
10. An image communication terminal having a circuit constructed and arranged to operate according to
claim 2
.
11. An image communication terminal having a circuit constructed and arranged to operate according to
claim 3
.
12. For communicating a representation of an image between a first terminal and a second terminal using a communications channel, an arrangement in at least one of the terminals, comprising:
means for operating the terminal using an inverse transformer loop having a decoding algorithm that accumulates at least one error; and
means for selectively using a loop filter in each inverse transformer loop based on at least one predetermined condition.
13. For communicating a representation of an image between a first terminal and a second terminal using a communications channel, an arrangement in at least one of the terminals, comprising:
means for operating the terminal using an inverse transformer loop; and
means for preventing unacceptable accumulation of an error within the inverse transformer loop by at least one of: pre-processing input data before using the inverse transformer loop to process the input data; using the inverse transformer in the loop according to a bit-exact specification between the encoder and decoder in the respective first and second terminals; using the inverse transformer in the loop according to one of a plurality of bit-exact specifications between the encoder and decoder in the respective first and second terminals and negotiating a common decision on the particular specification through encoder/decoder negotiation; providing a quantization valuer in the inverse transformer loop that is less prone to divergence; adjusting or providing transform coefficients to limit divergence; choosing an encoding mode for macro-blocks of input data; modifying the encoder to refresh portions of the image that are diverging; modifying the loop filter characteristics so that inverse transform mismatches are not amplified; applying the loop filter to input data corresponding to a frame buffer image just before sending to a display circuit without writing back the results to a frame buffer in the loop; selectively using a loop filter in each inverse transformer loop based on at least one predetermined condition.
US09/734,280 1992-02-19 2000-12-11 Deblocking filter for encoder/decoder arrangement and method with divergence reduction Expired - Lifetime US6404361B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US09/734,280 US6404361B2 (en) 1992-02-19 2000-12-11 Deblocking filter for encoder/decoder arrangement and method with divergence reduction

Applications Claiming Priority (9)

Application Number Priority Date Filing Date Title
US07/838,382 US5379351A (en) 1992-02-19 1992-02-19 Video compression/decompression processing and processors
US30397394A 1994-09-09 1994-09-09
US65891796A 1996-05-31 1996-05-31
US08/908,826 US5790712A (en) 1992-02-19 1997-08-08 Video compression/decompression processing and processors
US09/005,053 US6124882A (en) 1992-02-19 1998-01-09 Videocommunicating apparatus and method therefor
US09/064,397 US6215425B1 (en) 1992-02-19 1998-04-22 Deblocking filter for encoder/decoder arrangement and method with divergence reduction
US09/098,106 US6441842B1 (en) 1992-02-19 1998-06-16 Video compression/decompression processing and processors
US09/112,612 US6160503A (en) 1992-02-19 1998-07-09 Deblocking filter for encoder/decoder arrangement and method with divergence reduction
US09/734,280 US6404361B2 (en) 1992-02-19 2000-12-11 Deblocking filter for encoder/decoder arrangement and method with divergence reduction

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US09/112,612 Continuation US6160503A (en) 1992-02-19 1998-07-09 Deblocking filter for encoder/decoder arrangement and method with divergence reduction

Publications (2)

Publication Number Publication Date
US20010020906A1 true US20010020906A1 (en) 2001-09-13
US6404361B2 US6404361B2 (en) 2002-06-11

Family

ID=27567337

Family Applications (2)

Application Number Title Priority Date Filing Date
US09/112,612 Expired - Lifetime US6160503A (en) 1992-02-19 1998-07-09 Deblocking filter for encoder/decoder arrangement and method with divergence reduction
US09/734,280 Expired - Lifetime US6404361B2 (en) 1992-02-19 2000-12-11 Deblocking filter for encoder/decoder arrangement and method with divergence reduction

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US09/112,612 Expired - Lifetime US6160503A (en) 1992-02-19 1998-07-09 Deblocking filter for encoder/decoder arrangement and method with divergence reduction

Country Status (1)

Country Link
US (2) US6160503A (en)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030206664A1 (en) * 2002-05-01 2003-11-06 Cristina Gomila Deblocking filter conditioned on pixel brightness
US20030219074A1 (en) * 2002-01-31 2003-11-27 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US20050013360A1 (en) * 2003-07-18 2005-01-20 Samsung Electronics Co., Ltd. Image encoding and decoding apparatus and method
WO2005122588A1 (en) * 2004-06-14 2005-12-22 Tandberg Telecom As Method for chroma deblocking
US20060008013A1 (en) * 2004-07-08 2006-01-12 Oskar Pelc Method and system for performing deblocking filtering
US20070140355A1 (en) * 2005-12-21 2007-06-21 Kabushiki Kaisha Toshiba Information processing apparatus, control method, and program
US7277592B1 (en) 2003-10-21 2007-10-02 Redrock Semiconductory Ltd. Spacial deblocking method using limited edge differences only to linearly correct blocking artifact
US7319415B2 (en) 2002-05-01 2008-01-15 Thomson Licensing Chroma deblocking filter
US20090022415A1 (en) * 2005-11-22 2009-01-22 Freescale Semiiconductor, Inc. Method and system for filtering image data
US20090052555A1 (en) * 2007-08-21 2009-02-26 David Mak-Fan System and method for providing dynamic deblocking filtering on a mobile device
US20090238477A1 (en) * 2008-03-24 2009-09-24 Megachips Corporation Image processor
US20150181211A1 (en) * 2006-09-15 2015-06-25 Freescale Semiconductor, Inc. Video information processing system with selective chroma deblock filtering
US9445128B2 (en) 2006-12-08 2016-09-13 Freescale Semiconductor, Inc. System and method of determining deblocking control flag of scalable video system for indicating presentation of deblocking parameters for multiple layers
US9961372B2 (en) 2006-12-08 2018-05-01 Nxp Usa, Inc. Adaptive disabling of deblock filtering based on a content characteristic of video information

Families Citing this family (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7822188B1 (en) 1999-04-01 2010-10-26 Callwave, Inc. Methods and apparatus for providing expanded telecommunications service
US7103167B2 (en) 2002-05-20 2006-09-05 Callwave, Inc. Systems and methods for call screening
JP3489735B2 (en) * 2000-08-16 2004-01-26 松下電器産業株式会社 Deblocking filter operation unit
US6931063B2 (en) * 2001-03-26 2005-08-16 Sharp Laboratories Of America, Inc. Method and apparatus for controlling loop filtering or post filtering in block based motion compensationed video coding
US7929610B2 (en) * 2001-03-26 2011-04-19 Sharp Kabushiki Kaisha Methods and systems for reducing blocking artifacts with reduced complexity for spatially-scalable video coding
EP1303143A3 (en) * 2001-10-16 2004-01-14 Koninklijke Philips Electronics N.V. Blocking detection method
JP3936335B2 (en) * 2001-11-30 2007-06-27 株式会社エヌ・ティ・ティ・ドコモ Moving picture encoding apparatus, moving picture decoding apparatus, moving picture encoding method, moving picture decoding method, program, and computer-readable recording medium storing program
CN101448162B (en) * 2001-12-17 2013-01-02 微软公司 Method for processing video image
US7386049B2 (en) * 2002-05-29 2008-06-10 Innovation Management Sciences, Llc Predictive interpolation of a video signal
US7068722B2 (en) * 2002-09-25 2006-06-27 Lsi Logic Corporation Content adaptive video processor using motion compensation
US6922492B2 (en) * 2002-12-27 2005-07-26 Motorola, Inc. Video deblocking method and apparatus
US20130107938A9 (en) * 2003-05-28 2013-05-02 Chad Fogg Method And Apparatus For Scalable Video Decoder Using An Enhancement Stream
US7310371B2 (en) * 2003-05-30 2007-12-18 Lsi Corporation Method and/or apparatus for reducing the complexity of H.264 B-frame encoding using selective reconstruction
US20050013494A1 (en) * 2003-07-18 2005-01-20 Microsoft Corporation In-loop deblocking filter
US7724827B2 (en) 2003-09-07 2010-05-25 Microsoft Corporation Multi-layer run level encoding and decoding
US7616829B1 (en) 2003-10-29 2009-11-10 Apple Inc. Reducing undesirable block based image processing artifacts by DC image filtering
US20050281339A1 (en) * 2004-06-22 2005-12-22 Samsung Electronics Co., Ltd. Filtering method of audio-visual codec and filtering apparatus
JP4191729B2 (en) * 2005-01-04 2008-12-03 三星電子株式会社 Deblock filtering method considering intra BL mode and multi-layer video encoder / decoder using the method
KR100679035B1 (en) * 2005-01-04 2007-02-06 삼성전자주식회사 Deblocking filtering method considering intra BL mode, and video encoder/decoder based on multi-layer using the method
US8855107B1 (en) 2005-07-01 2014-10-07 Callwave Communications, Llc Methods and systems for call routing via a telephone number
WO2007032602A1 (en) * 2005-07-29 2007-03-22 Samsung Electronics Co., Ltd. Deblocking filtering method considering intra-bl mode and multilayer video encoder/decoder using the same
KR100678958B1 (en) * 2005-07-29 2007-02-06 삼성전자주식회사 Deblocking filtering method considering intra BL mode, and video encoder/decoder based on multi-layer using the method
US8548447B1 (en) 2006-10-06 2013-10-01 Callwave Communications, Llc Methods and systems for blocking unwanted telecommunications
US20080084932A1 (en) * 2006-10-06 2008-04-10 Microsoft Corporation Controlling loop filtering for interlaced video frames
US8325801B2 (en) * 2008-08-15 2012-12-04 Mediatek Inc. Adaptive restoration for video coding
US8787443B2 (en) 2010-10-05 2014-07-22 Microsoft Corporation Content adaptive deblocking during video encoding and decoding
US9042458B2 (en) 2011-04-01 2015-05-26 Microsoft Technology Licensing, Llc Multi-threaded implementations of deblock filtering
US10110926B2 (en) 2015-10-15 2018-10-23 Cisco Technology, Inc. Efficient loop filter for video codec

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6215425B1 (en) * 1992-02-19 2001-04-10 Netergy Networks, Inc. Deblocking filter for encoder/decoder arrangement and method with divergence reduction
JP3224926B2 (en) * 1993-12-28 2001-11-05 沖電気工業株式会社 Quantization / inverse quantization circuit
US5604502A (en) * 1994-03-21 1997-02-18 Lucent Technologies Inc. Motion video compression system with inverse discrete cosine transform mismatch control
JPH08289302A (en) * 1995-04-14 1996-11-01 Toshiba Corp Image decoding device
US5872866A (en) * 1995-04-18 1999-02-16 Advanced Micro Devices, Inc. Method and apparatus for improved video decompression by predetermination of IDCT results based on image characteristics
JP3094390B2 (en) * 1995-11-29 2000-10-03 三星電子株式会社 Transform coding apparatus for block including boundary of object of arbitrary form

Cited By (61)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8457438B2 (en) 2002-01-31 2013-06-04 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US9467714B2 (en) 2002-01-31 2016-10-11 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US8208749B2 (en) * 2002-01-31 2012-06-26 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US8290039B2 (en) 2002-01-31 2012-10-16 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US9538201B2 (en) 2002-01-31 2017-01-03 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US9538202B2 (en) 2002-01-31 2017-01-03 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US9538203B2 (en) 2002-01-31 2017-01-03 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US9538204B2 (en) 2002-01-31 2017-01-03 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US20070223833A1 (en) * 2002-01-31 2007-09-27 Samsung Electronics Co., Ltd. Filtering Method for Removing block Artifacts and/or Ringing Noise and Apparatus Therefor
US9532078B2 (en) 2002-01-31 2016-12-27 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US8699568B2 (en) 2002-01-31 2014-04-15 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US9532079B2 (en) 2002-01-31 2016-12-27 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US9532077B2 (en) 2002-01-31 2016-12-27 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US8160138B2 (en) 2002-01-31 2012-04-17 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US7561623B2 (en) * 2002-01-31 2009-07-14 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US20090190657A1 (en) * 2002-01-31 2009-07-30 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US9467717B2 (en) 2002-01-31 2016-10-11 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US20030219074A1 (en) * 2002-01-31 2003-11-27 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US9467716B2 (en) 2002-01-31 2016-10-11 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US9467715B2 (en) 2002-01-31 2016-10-11 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US9258574B2 (en) 2002-01-31 2016-02-09 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US9161059B2 (en) 2002-01-31 2015-10-13 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US9060105B2 (en) 2002-01-31 2015-06-16 Samsung Electronics Co., Ltd. Filtering method for removing block artifacts and/or ringing noise and apparatus therefor
US7319415B2 (en) 2002-05-01 2008-01-15 Thomson Licensing Chroma deblocking filter
US6907079B2 (en) 2002-05-01 2005-06-14 Thomson Licensing S.A. Deblocking filter conditioned on pixel brightness
US20030206664A1 (en) * 2002-05-01 2003-11-06 Cristina Gomila Deblocking filter conditioned on pixel brightness
EP1959689A3 (en) * 2003-07-18 2010-05-12 Samsung Electronics Co., Ltd Image encoding and decoding apparatus and method
EP2323400A3 (en) * 2003-07-18 2011-08-10 Samsung Electronics Co., Ltd. Image encoding and decoding apparatus and method
US8270474B2 (en) 2003-07-18 2012-09-18 Samsung Electronics Co., Ltd. Image encoding and decoding apparatus and method
EP2323396A3 (en) * 2003-07-18 2011-08-10 Samsung Electronics Co., Ltd. Image encoding and decoding apparatus and method
US8345748B2 (en) 2003-07-18 2013-01-01 Samsung Electronics Co., Ltd. Image encoding and decoding apparatus and method
US10602172B2 (en) 2003-07-18 2020-03-24 Samsung Electronics Co., Ltd. Image encoding and decoding apparatus and method
US9706216B2 (en) 2003-07-18 2017-07-11 Samsung Electronics Co., Ltd. Image encoding and decoding apparatus and method
US9961356B2 (en) 2003-07-18 2018-05-01 Samsung Electronics Co., Ltd. Image encoding and decoding apparatus and method
EP2323395A3 (en) * 2003-07-18 2011-08-10 Samsung Electronics Co., Ltd. Image encoding and decoding apparatus and method
US20050013360A1 (en) * 2003-07-18 2005-01-20 Samsung Electronics Co., Ltd. Image encoding and decoding apparatus and method
US9042443B2 (en) 2003-07-18 2015-05-26 Samsung Electronics Co., Ltd. Image encoding and decoding apparatus and method
US9042442B2 (en) 2003-07-18 2015-05-26 Samsung Electronics Co., Ltd. Image encoding and decoding apparatus and method
EP2323401A3 (en) * 2003-07-18 2011-08-10 Samsung Electronics Co., Ltd. Image encoding and decoding apparatus and method
US9729892B2 (en) 2003-07-18 2017-08-08 Samsung Electronics Co., Ltd. Image encoding and decoding apparatus and method
EP2323397A3 (en) * 2003-07-18 2011-08-10 Samsung Electronics Co., Ltd. Image encoding and decoding apparatus and method
US9706215B2 (en) 2003-07-18 2017-07-11 Samsung Electronics Co., Ltd. Image encoding and decoding apparatus and method
US20110235710A1 (en) * 2003-07-18 2011-09-29 Samsung Electronics Ltd., Co. Image encoding and decoding apparatus and method
US7277592B1 (en) 2003-10-21 2007-10-02 Redrock Semiconductory Ltd. Spacial deblocking method using limited edge differences only to linearly correct blocking artifact
US7822125B2 (en) 2004-06-14 2010-10-26 Tandberg Telecom As Method for chroma deblocking
WO2005122588A1 (en) * 2004-06-14 2005-12-22 Tandberg Telecom As Method for chroma deblocking
US7738563B2 (en) 2004-07-08 2010-06-15 Freescale Semiconductor, Inc. Method and system for performing deblocking filtering
US20060008013A1 (en) * 2004-07-08 2006-01-12 Oskar Pelc Method and system for performing deblocking filtering
US20090022415A1 (en) * 2005-11-22 2009-01-22 Freescale Semiiconductor, Inc. Method and system for filtering image data
US8532424B2 (en) 2005-11-22 2013-09-10 Freescale Semiconductor, Inc. Method and system for filtering image data
EP1802128A2 (en) * 2005-12-21 2007-06-27 Kabushiki Kaisha Toshiba Image processing apparatus, control method, and program
EP1802128A3 (en) * 2005-12-21 2008-10-15 Kabushiki Kaisha Toshiba Image processing apparatus, control method, and program
US20070140355A1 (en) * 2005-12-21 2007-06-21 Kabushiki Kaisha Toshiba Information processing apparatus, control method, and program
US9374586B2 (en) * 2006-09-15 2016-06-21 North Star Innovations Inc. Video information processing system with selective chroma deblock filtering
US20150181211A1 (en) * 2006-09-15 2015-06-25 Freescale Semiconductor, Inc. Video information processing system with selective chroma deblock filtering
US9445128B2 (en) 2006-12-08 2016-09-13 Freescale Semiconductor, Inc. System and method of determining deblocking control flag of scalable video system for indicating presentation of deblocking parameters for multiple layers
US9961372B2 (en) 2006-12-08 2018-05-01 Nxp Usa, Inc. Adaptive disabling of deblock filtering based on a content characteristic of video information
US20090052555A1 (en) * 2007-08-21 2009-02-26 David Mak-Fan System and method for providing dynamic deblocking filtering on a mobile device
US8913670B2 (en) * 2007-08-21 2014-12-16 Blackberry Limited System and method for providing dynamic deblocking filtering on a mobile device
US20090238477A1 (en) * 2008-03-24 2009-09-24 Megachips Corporation Image processor
US8571336B2 (en) * 2008-03-24 2013-10-29 Megachips Corporation Image processor for inhibiting noise

Also Published As

Publication number Publication date
US6404361B2 (en) 2002-06-11
US6160503A (en) 2000-12-12

Similar Documents

Publication Publication Date Title
US6404361B2 (en) Deblocking filter for encoder/decoder arrangement and method with divergence reduction
US6215425B1 (en) Deblocking filter for encoder/decoder arrangement and method with divergence reduction
USRE47261E1 (en) Method and apparatus for controlling loop filtering or post filtering in block based motion compensationed video coding
US20220103851A1 (en) Multidimensional quantization techniques for video coding/decoding systems
Chin et al. A software-only videocodec using pixelwise conditional differential replenishment and perceptual enhancements
RU2350040C1 (en) Method and device for instructions of quantiser parameters in video coding system
US7580584B2 (en) Adaptive multiple quantization
CN113632490A (en) Nonlinear adaptive loop filtering in video processing
US6075554A (en) Progressive still frame mode
US6028965A (en) Method and apparatus for intelligent codec system
WO2005079073A1 (en) Encoding and decoding of video images based on a quantization with an adaptive dead-zone size
US6865229B1 (en) Method and apparatus for reducing the “blocky picture” effect in MPEG decoded images
US20120087411A1 (en) Internal bit depth increase in deblocking filters and ordered dither
US7095448B2 (en) Image processing circuit and method for modifying a pixel value
US6225923B1 (en) Encoder arrangement and bit-exact IDCT protocol
KR100229796B1 (en) Image decoding system including compensation at detoriorate image
Naccari et al. Intensity dependent spatial quantization with application in HEVC
Chai et al. Foreground/background video coding scheme
Ngan et al. Very low bit rate video coding using H. 263 coder
JPH0998421A (en) Image encoding/decoding device
US8175151B2 (en) Encoders and image encoding methods
Ilgin et al. Image and Video Compression Artifact Reduction at Low Bit Rates Using Local Cosine Transform in Conjunction with Significance Map Coding
Halbach et al. SNR scalability by transform coefficient refinement for block-based video coding
Ng et al. A modified H. 263 algorithm using bit allocation buffer control algorithm
Wu Reducing inverse quantization numbers for video transcoding

Legal Events

Date Code Title Description
AS Assignment

Owner name: NETERGY MICROELECTRONICS, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:8X8, INC.;REEL/FRAME:012668/0984

Effective date: 20020211

STCF Information on status: patent grant

Free format text: PATENTED CASE

CC Certificate of correction
CC Certificate of correction
AS Assignment

Owner name: 8X8, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NETERGY MICROELECTRONICS, INC.;REEL/FRAME:013835/0116

Effective date: 20030311

FPAY Fee payment

Year of fee payment: 4

FPAY Fee payment

Year of fee payment: 8

FPAY Fee payment

Year of fee payment: 12