US7567619B2 - Video deblocking - Google Patents
Video deblocking Download PDFInfo
- Publication number
- US7567619B2 US7567619B2 US11/191,220 US19122005A US7567619B2 US 7567619 B2 US7567619 B2 US 7567619B2 US 19122005 A US19122005 A US 19122005A US 7567619 B2 US7567619 B2 US 7567619B2
- Authority
- US
- United States
- Prior art keywords
- discontinuities
- image unit
- client device
- video
- filtering
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related, expires
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
- H04N19/86—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness
Definitions
- This disclosure relates in general to video deblocking and in particular, by way of example but not limitation, to post-processing of video information to reduce blockiness in the display thereof.
- Television-based entertainment systems are expanding the programming and services that they offer.
- television service providers are adding on-demand video, as well as other interactive services, features, and applications.
- Such content and additional information are downloaded over a network for display, use, and/or storage on client-side set-top boxes or similar devices.
- downloads that include audio and/or video information are transmitted in a coded or compressed format. In other words, to reduce the amount of data that is transmitted, the information is typically compressed from a first size to a second smaller size.
- the data is decompressed/decoded into a semblance of the original audio and video information.
- the compression-decompression cycle is typically lossy, the decompressed version of the audio and video information differs from the original version of the audio and video information. With video information, for example, such differences can result in the presentation quality of the video information being visibly decreased.
- Video deblocking can be implemented in video content processing and delivery environments when displaying decompressed/decoded video information. Discontinuities are identified and smoothed to reduce blockiness, for example at macroblock boundaries in conjunction with Moving Picture Expert Group (MPEG)-based schemes.
- MPEG Moving Picture Expert Group
- a client device detects discontinuities at or near block boundaries, determines which discontinuities are artificial, and smooths at least those discontinuities that are determined to be artificial.
- These actions may be accomplished using (i) one or more Laplacian of Gaussian (LoG) operations as applied to both an image and a quantization matrix and (ii) windowed comparisons between values and a threshold resulting from the LoG operations. Alternatively, these actions may be accomplished using a spatio-temporally varying filter.
- intra image units may be deblocked using the former approach while non-intra image units may be deblocked using the latter approach.
- FIG. 1 illustrates an exemplary television system architecture in which the systems and methods for video deblocking can be implemented.
- FIG. 2 illustrates an exemplary client device, a television, and various input devices that interact with the client device.
- FIG. 3 is a block diagram that illustrates components of the exemplary client devices shown in FIGS. 1 and 2 .
- FIG. 4 is a set of graphs that illustrate an exemplary discontinuity and smoothing thereof.
- FIG. 5 is a flow diagram that illustrates an exemplary method for video deblocking.
- FIG. 6 is a flow diagram that illustrates an exemplary process for directly deblocking video.
- FIG. 7 illustrates an exemplary window for deblocking analysis around a macroblock boundary.
- FIG. 8 illustrates an exemplary Laplacian of Gaussian (LoG) operation in diagrammatic form.
- FIG. 9 is a graph that illustrates exemplary values resulting from an LoG operation on a quantization matrix.
- FIG. 10 illustrates pixels around a macroblock boundary for an exemplary visual adjustment procedure.
- FIGS. 11A , 11 B, and 11 C are exemplary diagrams of a portion of a video information presentation across multiple frames of a group of pictures (GOP).
- GOP group of pictures
- FIG. 12 is a graph that illustrates filters across macroblocks and macroblock boundaries in an exemplary approach for indirectly deblocking video using spatial filtering.
- FIGS. 13A and 13B illustrate an exemplary approach for indirectly deblocking video using temporal filtering across multiple frames of a GOP.
- FIG. 14 is a flow diagram of an exemplary approach to block-type-dependent filtering under a Moving Pictures Expert Group (MPEG)-compliant compression/coding scheme.
- MPEG Moving Pictures Expert Group
- video content processing and delivery systems such as interactive TV networks, cable/satellite networks that utilize electronic program guides and other applications, and Web-enabled TV networks.
- Client devices in such systems range from full-resource clients with substantial memory and processing resources, such as TV-enabled personal computers and TV recorders equipped with hard-disks, to low-resource clients with limited memory and/or processing resources, such as traditional set-top boxes and personal digital assistants (PDAs) or mobile phones.
- PDAs personal digital assistants
- video deblocking as described herein may additionally be used in other environments such as in streaming (e.g., over the Internet), generally in post-processing of compression and decompression cycles, and so forth. While aspects of the described systems and methods can be used in any of these environments and for any types of client devices, they are described primarily in the context of the following exemplary environment.
- FIG. 1 illustrates an exemplary television entertainment system 100 that is an architecture in which video deblocking may be implemented.
- System 100 facilitates distribution of content and other information to multiple viewers.
- System 100 includes one or more content providers 102 , one or more other information providers 104 , a content distribution system 106 , and multiple client devices 108 ( 1 ), 108 ( 2 ), . . . , 108 (N) coupled to content distribution system 106 via a broadcast network 110 .
- Content provider 102 includes a content server 112 and stored content 114 , such as movies, television programs, commercials, music, and similar audio and/or video content.
- Content server 112 controls distribution of stored content 114 from content provider 102 to content distribution system 106 . Additionally, content server 112 may control distribution of live content (e.g., content that was not previously stored, such as live feeds) and/or content stored at other locations to content distribution system 106 .
- live content e.g., content that was not previously stored, such as live feeds
- Other information provider 104 includes other information database 116 and other information server 118 .
- Other information database 116 stores information that may be provided to client devices 108 . Such information includes software modules, files, images, text, executable programs, moving video, gaming or other interactive information, and so forth. The information may also include content, especially content of an irregular, one-of-a-kind, or similar nature, or content from smaller independent providers. Part or all of the information from other information database 116 may ultimately be displayed to subscribers after traversing one or more low-bandwidth transmission mediums, regardless of whether such transmission medium(s) are located upstream or downstream of content distribution system 106 .
- Other information server 118 processes the other information from other information database 116 prior to distribution to generate one or more files that are optimized for, or at least capable of, transmission to content distribution system 106 .
- Content distribution system 106 includes a broadcast transmitter 128 , one or more content processors 130 , and one or more other information processors 132 .
- Broadcast transmitter 128 can alternatively be a transceiver if bi-directional communication is desired (e.g., in situations in which broadcast network 110 is a bi-directional network 110 ).
- Broadcast transmitter 128 transmits (e.g., broadcasts) signals, such as cable/satellite television signals, across broadcast network 110 .
- Broadcast network 110 can include a cable television network, RF, microwave, satellite, and/or data network, such as the Internet, and may also include wired or wireless media using any transmission format or protocol. Additionally, broadcast network 110 can be any type of network (including a bi-directional network), using any type of network topology and any network communication protocol, and can be represented or otherwise implemented as a combination of two or more networks.
- Content processor 130 processes the content received from content provider 102 prior to transmitting the content across broadcast network 110 .
- other information processor 132 processes the other information that is received from other information provider 104 prior to transmission of the other information across broadcast network 110 .
- a particular content processor 130 may encode, or otherwise process, the received content into a format that is understood by the multiple client devices 108 ( 1 ), 108 ( 2 ), . . . , 108 (N) that are coupled to broadcast network 110 .
- FIG. 1 shows a single content provider 102 , a single other information provider 104 , and a single content distribution system 106
- the exemplary system 100 can include any number of content providers and/or other information providers coupled to any number of content distribution systems.
- content distribution system 106 , content provider 102 , and/or other information provider 104 are individually or jointly representative of a headend service that provides content and other information to multiple subscribers.
- Client devices 108 can be implemented in a number of ways. For example, a client device 108 ( 1 ) receives content and other information from a satellite-based transmitter via a satellite dish 134 . Client device 108 ( 1 ) is also referred to as a set-top box or a satellite receiving device. Client device 108 ( 1 ) is coupled to a television 136 ( 1 ) for presenting the content and other information (e.g., audio information, video information, and/or data information) that are received by the client device 108 ( 1 ), as well as for presenting a graphical user interface. A particular client device 108 can be coupled to any number of televisions 136 and/or similar devices that can be implemented to display or otherwise render content. Similarly, any number of client devices 108 can be coupled to a single television 136 .
- a television 136 receives content and other information from a satellite-based transmitter via a satellite dish 134 .
- Client device 108 ( 1 ) is also referred to as a set-top box
- Client device 108 ( 2 ) is also coupled to receive content and other information from broadcast network 110 and to provide the received content and other information to associated television 136 ( 2 ).
- Client device 108 (N) is an example of a combination television 138 and integrated set-top box 140 .
- Set-top box 140 that is integrated into television 138 can receive signals (e.g., broadcast signals) via a satellite dish (similar to satellite dish 134 ) and/or directly via broadcast network 110 .
- client devices 108 may receive signals via the Internet or any other network, especially those network mediums that are broadcast-capable.
- client devices 108 may also engage in video deblocking prior to displaying video information (whether content video information or other video information) that is received from a low bandwidth medium, such as a memory storage, other client devices, possibly broadcast network 110 itself, and so forth.
- a low bandwidth medium such as a memory storage, other client devices, possibly broadcast network 110 itself, and so forth.
- the exemplary system 100 also includes information from other networks/network providers 142 , which may provide information such as information streamed over the Internet, information received directly from a provider of the information, and so forth.
- Information from other networks/network providers 142 may be accessible over broadcast network 110 (i.e., a network that also provides content information and other information from content distribution system 106 ).
- broadcast network 110 i.e., a network that also provides content information and other information from content distribution system 106
- information from other networks/network providers 142 may be accessible over a different network, including a wide area network (WAN), the Internet, a public or private telecommunications network, and so forth.
- WAN wide area network
- the Internet a public or private telecommunications network, and so forth.
- FIG. 2 illustrates an exemplary implementation 200 of a client device 108 shown as a standalone unit that connects to a television 136 and communicates with various input devices 204 , 206 , and 208 .
- Client device 108 can be implemented in any number of embodiments, including as a set-top box, a satellite receiver, a TV recorder with a hard disk, a digital video record (DVR) and playback system, a game console, an information appliance, and so forth.
- DVR digital video record
- Client device 108 includes a wireless port 202 , such as an infrared (IR) or Bluetooth wireless port, for receiving wireless communications from a remote control device 204 , a handheld input device 206 , or any other wireless device, such as a wireless keyboard.
- Handheld input device 206 can be a personal digital assistant (PDA), handheld computer, wireless phone, or the like.
- PDA personal digital assistant
- a wired keyboard 208 can be coupled to communicate with client device 108 .
- remote control device 204 , handheld device 206 , and/or keyboard 208 may use an RF communication link or other mode of transmission to communicate with client device 108 .
- Client device 108 receives one or more (e.g., broadcast) signals 210 from one or more broadcast sources, such as from a satellite or a cable or a broadcast network, including a broadcast implementation of network 110 (of FIG. 1 ).
- Client device 108 includes hardware and/or software for receiving and decoding a broadcast signal 210 , such as an NTSC, PAL, SECAM or other TV system video signal.
- Client device 108 also includes hardware and/or software for providing the user with a graphical user interface by which the user can, for example, access various network services, configure client device 108 , and perform other functions, including requesting video delivery.
- Client device 108 can communicate with other devices via one or more connections including a conventional telephone line 212 , an ISDN link 214 , a cable link 216 , an Ethernet link 218 , a DSL link 220 , and the like. Client device 108 may use any one or more of the various communication links 212 - 220 at a particular instant to communicate with any number of other devices. For example, in addition to receiving information over any one of communication links 212 - 220 , client device 108 may provide (e.g., transmit) information over communication links 212 - 220 or any other type of communication link.
- Such other communication links may include links capable of interfacing with a local network such as a local area network (LAN), a Bluetooth® network, an IEEE 802.11b-compliant network, or other wired or wireless network type.
- a client device 108 that receives information from broadcast network 110 may forward the information over the local network to one or more other client devices 108 .
- the forwarding may be effectuated, for example, over a lower-bandwidth transmission medium that introduces blockiness.
- Client device 108 generates video signal(s) 222 and audio signal(s) 224 , both of which are communicated to television 136 .
- Video signals 222 and audio signals 224 can be communicated from client device 108 to television 136 via an RF (radio frequency) link, S-video link, composite video link, component video link, co-axial cable link, or other communication link.
- the video signals 222 may include deblocked video signals, for example.
- client device 108 may include one or more lights or other indicators identifying the current status of the device. Additionally, the client device may include one or more control buttons, switches, or other selectable controls for controlling operation of the device.
- FIG. 3 illustrates selected components of exemplary client device 108 shown in FIGS. 1 and 2 .
- Client device 108 includes a first tuner 300 and an optional second tuner 302 .
- the tuners 300 and 302 are representative of one or more in-band tuners that tune to various frequencies or channels to receive television signals, as well as at least one out-of-bound (OOB) tuner that tunes to the broadcast channel(s) over which data information is broadcast (e.g., carouseled or otherwise transmitted) to client device 108 .
- OOB out-of-bound
- Client device 108 also includes one or more processors 304 which process various instructions to control the operation of client device 108 and to communicate with other electronic and computing devices.
- Client device 108 can be implemented with one or more memory components, examples of which include a random access memory (RAM) 306 , a disk drive 308 , another mass storage component 310 , and a non-volatile memory 312 (e.g., ROM, Flash, EPROM, EEPROM, etc.).
- the memory components e.g., RAM 306 , disk drive 308 , mass storage 310 , and non-volatile memory 312 ) store various instructions and/or information such as received content, programs, configuration information for client device 108 , graphical user interface information, and/or video data or information.
- client device 108 can include a range of processing and memory capabilities, and may include more or fewer types of memory components than those illustrated in FIG. 3 .
- full-resource clients can be implemented with substantial memory and processing resources, including the disk drive 308 to store content for replay by the viewer.
- Low-resource clients may have limited processing and memory capabilities, such as a limited amount of RAM 306 , no disk drive 308 , and limited processing capabilities of a processor 304 .
- An operating system 314 and one or more programs as represented by general instructions 316 may be stored in non-volatile memory 312 (and/or other memory component(s)) and executed on processor 304 to provide a runtime environment.
- a runtime environment facilitates extensibility of client device 108 by allowing various interfaces to be defined that, in turn, allow the programs to interact with client device 108 .
- these programs may be installed when client device 108 is manufactured, they may also be received via broadcast network 110 from content distribution system 106 (of FIG. 1 ).
- Also stored in non-volatile memory 312 (and/or other memory component(s)) are electronically-executable instructions for video deblocking 318 , as described further herein.
- Client device 108 also includes a decoder 320 to decode a broadcast video signal, such as an NTSC, PAL, SECAM or other TV system video signal.
- Processor 304 along with tuner(s) 300 and 302 and/or decoder 320 , also enables client device 108 to reconstruct audio and video from an MPEG-2 stream or other digital packet signal, whether compressed or uncompressed.
- Client device 108 can also include other components pertaining to a television entertainment system which are not illustrated in this example. For instance, client device 108 can include a user interface application and user interface lights, buttons, controls, and the like to facilitate viewer interaction with the device.
- Client device 108 further includes a wireless interface 322 , a network interface 324 , a serial and/or parallel interface 326 , and a modem 328 .
- Wireless interface 322 allows client device 108 to receive input commands and other information from a user-operated input device, such as from a remote control device or from another IR, Bluetooth, or similar RF input device.
- Network interface 324 and serial and/or parallel interface 326 allows client device 108 to interact and communicate with other electronic and computing devices via various communication links, including local network communication links to other client devices 108 .
- client device 108 may also include other types of data communication interfaces to communicate with other devices.
- Modem 328 facilitates communication by client device 108 with other electronic and computing devices via a conventional telephone line.
- Client device 108 also includes an audio output 330 and a video output 332 that provide signals to a television or other device that processes and/or displays or otherwise renders the audio and video information, including deblocked video information.
- client device 108 may be implemented together in an application specific integrated circuit (ASIC).
- ASIC application specific integrated circuit
- electronically-executable instructions ( 318 ) and associated processing abilities for video deblocking may be integrated together onto one or more ASICs instead of using more-general memory (e.g., 306 , 312 , etc.) and/or processing (e.g., 304 , 320 , etc.) resources of a client device ( 108 ).
- a system bus typically connects the various components within client device 108 .
- a system bus can be implemented as one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, or a local bus using any of a variety of bus architectures.
- bus architectures can include an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnects (PCI) bus also known as a Mezzanine bus.
- ISA Industry Standard Architecture
- MCA Micro Channel Architecture
- EISA Enhanced ISA
- VESA Video Electronics Standards Association
- PCI Peripheral Component Interconnects
- Video information is often compressed or otherwise coded to facilitate the handling thereof.
- the coding of the video information into video data can reduce the bandwidth required to display, store, transmit, or otherwise manipulate the video information. For example, it can be beneficial to reduce the bit rate of video information for storing a video stream to disk or streaming it over a noisy, bandwidth-limited channel.
- compression and decompression of the video information typically causes visual artifacts that degrade the visual quality.
- One artifact that degrades visual quality is blockiness. In other words, low bit rate video information is often plagued with blockiness that seriously degrades visual quality.
- Video deblocking usually entails reducing such blocky artifacts to enhance the visual presentation quality of the video information. Implementing a good deblocking mechanism helps to mitigate the deleterious byproducts of a lower bit rate video stream and thereby justifies using a lower bit rate that enables the aforementioned low-bandwidth applications.
- Deblocking is implemented after previously encoded video data is decoded into at least a semblance of the original video information.
- One set of coding standards that may be used on the original video information are Moving Picture Expert Group (MPEG)-compliant standards.
- MPEG-based encoding/decoding algorithms involve performing a discrete cosine transform (DCT) on original video information and quantizing the DCT coefficients thereof.
- the resulting video data is de-quantized and an inverse DCT is performed to reacquire at least a semblance of the original video information.
- Reducing the bit rate of the video data stream may be achieved by more coarsely quantizing the DCT coefficients. In doing so, however, blockiness that is introduced along macroblock boundaries tends to seriously degrade the visual quality.
- the visual presentation quality may be at least partially restored by using deblocking during post-processing. Specifically, the deblocking may be used to reduce the blockiness around macroblock boundaries.
- FIG. 4 is a set of graphs 400 that illustrate an exemplary discontinuity and smoothing thereof.
- Graphs 400 include graphs 400 A, 400 B, 400 C, and 400 D.
- Each of graphs 400 A- 400 D includes two macroblocks of a video picture. Although the macroblocks are shown as being rectangular, actual macroblocks may instead be square.
- Graph 400 A illustrates original video information 402 (as a solid line). There is no break in original video information 402 between the left macroblock and the right macroblock.
- Graph 400 B illustrates original video information 402 and adds thereto an illustration of discontinuity 404 (as a large dashed line).
- Discontinuity 404 represents an exemplary discontinuity or break in the seamlessness of original video information 402 . This discontinuity arises at the macroblock boundary of the left and right macroblocks as a result of the encoding/decoding process. Discontinuity 404 may be visible as a blocky artifact. In fact, discontinuity 404 along with other discontinuities in the overall picture may be visible as a blockiness that reduces the overall visual quality.
- Graph 400 C illustrates original video information 402 and adds thereto an illustration of smoothed video information 406 (as a small dashed line). Smoothed video information 406 results from application of video deblocking to discontinuity 404 . While smoothed video information 406 is not likely to be an exact replica of original video information 402 , smoothed video information 406 reduces the blocky artifact of discontinuity 404 and generally more-closely approximates original video information 402 . It should be noted that smoothed video information 406 is representative of deblocking mechanisms/approaches in general, including visual adjustment, filter smoothing, spatial-temporal filtering, and so forth.
- Graph 400 D includes original video information 402 , discontinuity 404 , and smoothed video information 406 . All three are included together to illustrate (i) the discontinuity that may be formed from coarse quantization of original video information or other deleterious coding/decoding factors and (ii) the resulting video information after the discontinuity has been smoothed in post-processing. This post-processing video deblocking is described further herein.
- deblocking in an MPEG-based environment in which discontinuities appear at macroblock boundaries.
- deblocking may be employed after decompression/decoding in accordance with other standards and approaches and along other boundaries. It should therefore be understood that deblocking may be employed with video data/information in general and regardless of the size/type of blocks that are employed to segregate the video picture.
- discontinuity is used herein, other such terms include “edge” and “boundary”.
- An edge, boundary, or discontinuity in a video picture may be “artificial” or “real”.
- Artificial discontinuities arise, for example, from coarse quantization of DCT coefficients.
- Other terms for artificial discontinuities include, but are not limited to, quantization discontinuities, false discontinuities, fake discontinuities, and quantization noise discontinuities.
- Real discontinuities arise from true boundaries in a video picture from one element to another element within the picture.
- Other terms for real discontinuities include, but are not limited to, actual discontinuities, true discontinuities, and natural discontinuities. These real discontinuities may also lie along a macroblock boundary.
- Video deblocking may be described in the general context of electronically-executable instructions.
- electronically-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types.
- Video deblocking may also be practiced in distributed computing environments where functions are performed by remote processing devices that are linked through a communications network.
- electronically-executable instructions may be located in both local and remote storage media.
- FIGS. 5 , 6 , and 14 are illustrated in flow diagrams divided into multiple blocks. However, the order in which the methods and processes are described is not intended to be construed as a limitation, and any number of the described blocks can be combined in any order to implement one or more methods or processes for video deblocking. Furthermore, although the methods and processes are described below with reference to the television entertainment environments 100 and 200 and client devices 108 where applicable, the methods and processes can be implemented in any suitable hardware, software, firmware, or combination thereof and using any suitable mathematical alternatives.
- FIG. 5 is a flow diagram 500 that illustrates an exemplary method for video deblocking.
- Flow diagram 500 includes blocks 502 , 504 , and 506 that may be implemented, for instance, by client devices 108 (of FIGS. 1-3 ).
- Blocks 502 , 504 , and 506 may correspond to electronically-executable instructions that are stored in one or more memories of a client device 108 and executed using one or more processors thereof.
- a discontinuity is detected. For example, macroblock boundaries are analyzed to detect whether a discontinuity exists for video information between a first macroblock boundary and a second macroblock boundary. An exemplary window around a macroblock boundary for such an analysis is described further below with reference to FIG. 7 .
- the discontinuity is examined to determine whether it is a real discontinuity or an artificial discontinuity.
- Real edges of actual picture elements may fall on a macroblock boundary, but smoothing real edges usually decreases video presentation quality. Exemplary mechanisms for differentiating between real discontinuities and artificial discontinuities are described further below especially with reference to FIGS. 8 and 9 .
- the artificial discontinuity or discontinuities are smoothed.
- the smoothing of the artificial discontinuities reduces visually-apparent blockiness. It should be understood that smoothing may encompass a variety of mechanisms/approaches designed to reduce or eliminate the visibility of discontinuities. Exemplary mechanisms for smoothing artificial discontinuities are described further below especially with reference to FIG. 10 et seq.
- Flow diagram 500 is directed to both direct deblocking and indirect deblocking.
- Direct deblocking of video is described especially with reference to FIGS. 6-10 .
- Indirect deblocking of video is described especially with reference to FIGS. 11A-13B .
- the description of FIG. 14 below is directed to a video deblocking approach that utilizes both direct and indirect deblocking mechanisms.
- FIG. 6 is a flow diagram 600 that illustrates an exemplary process for directly deblocking video.
- Flow diagram 600 includes six (6) blocks 602 - 612 .
- Blocks 602 and 604 pertain to input data for the process of flow diagram 600 .
- Block 602 provides the scaled quantization matrix that was used to quantize the original video information during the coding/compressing process.
- This scaled quantization matrix of block 602 may be related to, for example, the “quant_matrix” parameter of MPEG-compliant coding schemes.
- the quant_matrix is determined by the encoder and encoded into the bit stream.
- a scaled quant matrix may be created as the product of the quant matrix and the quantization scale, which is another parameter that is encoded into the bit stream by the encoder.
- Block 604 provides the image that is formed from the decoded video information.
- the image may include artificial discontinuities (e.g., at macroblock boundaries) and real discontinuities (e.g., either at or away from macroblock boundaries).
- the artificial discontinuities at the macroblock boundaries can arise from noise attributable to the quantization matrix during the quantization process.
- Block 606 performs a Laplacian of Gaussian (LoG) operation on each of the scaled quantization matrix of block 602 and the image video information of block 604 to produce two different results.
- the LoG operation may be defined as convolving the argument with the LoG kernel. From the two different results of applying the LoG operation, the type of discontinuity is determined at block 608 .
- the type of discontinuity (if any) at the macroblock boundary under consideration is determined by comparing the result of the LoG operation as performed on the image of block 604 with a (quantization) threshold determined responsive to the result of the LoG operation as performed on the scaled quantization matrix of block 602 .
- the LoG operation of block 606 and the discontinuity type determination of block 608 are described further below especially with reference to FIGS. 8 and 9 .
- block 610 may optionally perform a visual adjustment procedure at the macroblock boundary. An exemplary visual adjustment procedure is described further below especially with reference to FIG. 10 . If, on the other hand, the discontinuity at the macroblock boundary is determined to be an artificial discontinuity (at block 608 ), block 612 performs a filter smoothing procedure at the macroblock boundary. An exemplary filter smoothing procedure is described further below primarily after the description of FIG. 10 .
- FIG. 7 illustrates an exemplary window 702 for deblocking analysis around a macroblock boundary 706 .
- Two macroblocks 708 L and 708 R are illustrated. Each macroblock 708 is surrounded by four (4) macroblock boundaries 706 . Within each macroblock 708 are a set of pixels. Although macroblocks 708 are shown as being blocks of eight-by-eight (8 ⁇ 8) pixels, they may alternatively be blocks of 16 ⁇ 16 pixels, 24 ⁇ 24 pixels, 30 ⁇ 30 pixels, or any other size. Because the illustrated macroblocks are of an eight-by-eight (8 ⁇ 8) size, window 702 includes sixteen (16) pixels to be analyzed for possible discontinuities at the macroblock boundary 706 that window 702 surrounds. However, the width of window 702 may alternatively be greater than two pixels (i.e., the width can be more than one pixel within each macroblock 708 ).
- Window 702 is located between the left macroblock 708 L and the night macroblock 708 R. Similar “vertical” windows 702 are used at the other “vertical” macroblock boundaries 706 . Furthermore, “horizontal” windows (not shown) are used at macroblock boundaries between upper macroblocks and adjacent lower macroblocks. In other words, to address a two-dimensional (2-D) image, video deblocking is performed in both the vertical direction and the horizontal direction.
- the following algorithm may be employed: Let Q represent the quantization matrix used in quantizing an image I. Because quantization is usually performed in the DCT domain, the effect of using Q in the time domain is first determined. Thus, an inverse DCT is performed on Q to generate Q t . Next, the discontinuities (e.g., edges, boundaries, etc.) are detected in the image I. Any such discontinuities may be detected using the Laplacian of Gaussian (LoG) operator to filter I and generate I LG .
- the LoG operator may be considered a concatenation of the Laplacian operator, which is given by:
- FIG. 8 illustrates an exemplary Laplacian of Gaussian (LoG) operation 800 in diagrammatic form.
- An image 802 is filtered using LoG operation 804 to produce an edge-biased image 806 .
- discontinuities e.g., edges
- each discontinuity in each window 702 is a real discontinuity (e.g., from an edge of an image element) or an artificial discontinuity (e.g., from quantization or other deleterious compression/decompression factors).
- the LoG operation detects both kinds of discontinuities, but it does not clearly differentiate between them by itself.
- Quant_threshold A variable that may be termed “quant_threshold” is created to determine which discontinuities are artificial. The amount of blockiness attributable to quantization noise is effectively assessed. In order to assess the amount of blockiness attributable to quantization, the time-domain quantization matrix divided by two
- Q t 2 ( Q t 2 ) is passed through the LoG filter to obtain Q tLG .
- the quantization matrix Q t is divided by 2 because half the quantization matrix Q t represents the actual loss of data due to round-off/truncation.
- FIG. 9 is a graph 900 that illustrates exemplary values Q tLG resulting from a LoG operation on a time-domain quantization matrix Q t .
- a plot 902 graphs different quantization matrix values (along the abscissa axis) versus corresponding LoG operation output values Q tLG (along the ordinate axis).
- Maximum absolute values for the Q tLG values are denoted as one hundred percent (100%). Any portion of the 100% value may be selected as the threshold between real discontinuities and artificial discontinuities.
- the threshold may be selected, for example, such that the majority of the visually displeasing artificial discontinuities are subject to smoothing while few if any of the real discontinuities are subject to smoothing (therein mitigating “fuzzying” of the original image).
- An optimum value for the threshold for any given type of video information may be selected through experimental viewing of deblocked video. In the illustrated graph 900 , eighty-five percent (85%) is selected as the threshold between real discontinuities and artificial discontinuities. Furthermore, this threshold may be tunable subject to user/viewer preferences.
- the threshold is used in conjunction with an analysis of macroblock boundaries 706 in windows 702 .
- the analysis is focused in windows 702 because blockiness that is due to quantization noise is centered on macroblock boundaries 706 .
- the values of I LG are compared to the threshold within each window 702 . If more than a certain fraction, ⁇ , of the I LG values in a window 702 are above the selected threshold, then that discontinuity is labeled as a real discontinuity. If not, then that discontinuity is labeled as an artificial discontinuity.
- the fraction, ⁇ may also be selected experimentally, for example by viewing different types of video that has been deblocked using different values for the fraction, ⁇ , until a visually-appealing value is selected.
- the parameter, ⁇ may be experimentally determined and fine tuned to maximize visual quality.
- discontinuities After discontinuities have been (i) detected and (ii) determined to be (and “labeled” as) either real discontinuities or artificial discontinuities, appropriate deblocking procedures can be implemented. As described above with reference to blocks 608 , 610 , and 612 (of FIG. 6 ), real discontinuities may be subjected to a visual adjustment procedure, and artificial discontinuities may be subjected to a filter smoothing procedure. In an exemplary visual adjustment procedure, pixel values are adjusted slightly by moving them towards each other by an amount proportional to the threshold so as to reduce the effects of quantization.
- FIG. 10 illustrates pixels 704 around a macroblock boundary 706 for an exemplary visual adjustment procedure.
- Eight (8) adjacent pixels 704 from p 0 to p 7 are divided by the macroblock boundary 706 that is between left macroblock 708 L and right macroblock 708 R.
- Pixels p 0 , p 1 , p 2 , and p 3 are in left macroblock 708 L.
- Pixels p 4 , p 5 , p 6 , and p 7 are in right macroblock 708 R.
- the values of the pixels p 0 to p 7 are adjusted slightly by moving them towards each other by an amount proportional to the selected threshold so as to reduce the effects of quantization. This adjustment is performed recursively to the adjacent pixels.
- d min ⁇ ( p 3 - p 4 2 , quant_threshold )
- p 3 ′ p 3 - d
- p 4 ′ p 4 + d
- p 2 ′ p 2 + p 3 ′ 2
- p 5 ′ p 5 + p 4 ′ 2 ⁇ ⁇ ⁇
- time-domain pixels are designated by “p” (p without the prime) and the adjusted pixels are represented by “p” (p with the prime).
- a filter smoothing procedure is performed.
- pixels 704 (of FIG. 7 ) of a window 702 are filtered around boundary 706 with a smoothing filter.
- a smoothing filter such as stretched and/or scaled version(s) of the Gaussian or tent filter.
- An example of a Gaussian/tent filter is:
- any smoothing filter can be used with the degree of smoothing set to accommodate the desired amount of blurring.
- a less rigid and more indirect mechanism is used to detect discontinuities, to differentiate between real and artificial discontinuities, and to smooth the detected discontinuities.
- This second mechanism includes both spatial and temporal aspects.
- the mechanism entails associating an increased likelihood of needing deblocking (e.g., of a discontinuity being an artificial discontinuity) with increased proximity to a macroblock boundary.
- the mechanism entails associating the likelihood of blockiness at a macroblock boundary to the position of the picture within a GOP. For example, it is assumed that an I-frame of the GOP is more susceptible to blockiness at the macroblock boundary. On the other hand, a B-frame downstream in the GOP is less likely to experience blockiness at the macroblock boundary.
- FIGS. 11A , 11 B, and 11 C are exemplary diagrams 1100 of a portion of video information that is presented across multiple frames of a GOP.
- frames are organized into GOPs that start with an intra (I) frame and are followed by predicted (P) frames and bi-directional (B) frames/modes.
- I-frames are usually more susceptible to blockiness at macroblock boundaries than those frames that are downstream in the GOP as is explained by the description of the diagrams 1100 A-C.
- diagram 1100 A represents an intra-frame with many macroblocks 708 and macroblock boundaries 706 .
- Diagrams 1100 B and 1100 C represent non-intra frames.
- Each non-intra block in non-intra frames includes two components: the predicted term and the difference term.
- the predicted term dominates, and the difference term is relatively small.
- the predicted term is usually not macroblock aligned in the reference frame (as shown in diagram 1100 B)
- any macroblocking effects in the reference frame are seen in the interior of the reconstructed macroblock in the current frame.
- the magnitude and location of blockiness tends to be diffused (as shown in diagram 1100 C). In other words, downstream in a GOP, images tend to be both less crisp as well as less blocky.
- the position of a frame in a GOP tends to serve as an indicator of the extent of blockiness. This tendency is reflected in the temporal aspect of the second mechanism.
- the extent of quantization-caused blockiness tends to increase as pixel location approaches a macroblock boundary.
- the second mechanism thus involves filtering the image with a spatio-temporally varying filter.
- the degree of filtering increases for pixels near macroblock boundaries and decreases for pixels away from macroblock boundaries. This spatial filtering is described further below with reference to FIG. 12 . Also, the degree of filtering is higher at the beginning of a GOP while the degree of filtering decreases downstream in the GOP. This temporal filtering is described further below with reference to FIGS. 13A and 13B . Combining the spatial filtering with the temporal filtering creates a spatio-temporally varying filter that indirectly detects discontinuities, differentiates between discontinuity types, and smooths the detected discontinuities.
- FIG. 12 is a graph 1200 that illustrates filters 1202 across macroblocks 708 and macroblock boundaries 706 in an exemplary approach for indirectly deblocking video using spatial filtering.
- Filters 1202 include three (3) different filtering zones labeled zone # 1 , zone # 2 , and zone # 3 . While each of the three different filtering zones may have filters of similar or identical energy levels (to avoid changing image intensity), each of the filters of the three different filtering zones filters pixels of the macroblocks 708 differently.
- Filtering zone # 1 filters to the greatest degree in an area around macroblock boundaries 706 .
- Filtering zone # 3 filters the least in an area around the center of macroblocks 708 .
- Filters of filtering zones # 3 may be implemented as an impulse function so that the pixels are changed very slightly or not at all.
- the centers of macroblocks 708 are the least likely to exhibit blockiness due to quantization noise.
- Located spatially between the filtering zones # 1 and # 3 are filtering zones # 2 .
- the filters of filtering zones # 2 filter the corresponding physically-intermediate pixels to a degree that is also intermediate to that of the filters of filtering zones # 1 and # 3 .
- filter 1202 includes an exemplary three filtering zones, four or more or less than three filtering zones may alternatively be employed.
- An example of a suitable spatial filter is:
- the filters can be normalized in order to achieve a gain of 1 (or any desired value).
- Such spatial filtering thus concentrates filtering near macroblock boundaries and away from the center of macroblocks. Using a spatial filter designed accordingly effectively tends to indirectly filter blockiness due to quantization noise.
- FIGS. 13A and 13B illustrate an exemplary approach for indirectly deblocking video using temporal filtering across multiple frames of a GOP.
- the extent of spatial filtering that is necessary and/or desirable around macroblock boundaries tends to decrease as frame position number proceeds through a GOP, as is explained above with reference to FIGS. 11A-11C .
- Chart 1300 A illustrates a GOP and exemplary temporal filtering coefficients designated as ⁇ .
- the GOP position number (GPN) runs from 1 to n, where n may be any natural number, but is often set to fifteen (15).
- the MPEG frames are designated by I, B, or P in an exemplary sequence that runs: I, B, B, B, P, B, B, B, P, B, B, B, P, B, B, B, P, B, B, B, B. Other sequences may alternatively be used.
- Three different sets of temporal filtering coefficients ⁇ are included and are designated as ⁇ 1 , ⁇ 2 , and ⁇ 3 .
- the temporal filtering coefficient ⁇ may be applied to the spatial filtering as described above with reference to FIG. 12 .
- the temporal filtering coefficient ⁇ serves to reduce the filtering by, and therefore the impact of, the spatial filter 1202 as frames progress downward through the GOP. Consequently, each ⁇ may be determined responsive to the GPN of the frame under consideration.
- Table 1300 B provides a general GPN-dependent formula for determining an ⁇ for each of the three temporal filtering coefficient ⁇ examples. Each of the ⁇ examples decrease in value as the GPN increases to reduce the impact of the spatial filter. The rapidity at which each a decreases in value as the GPN increases from 1 to n scales from a low with ⁇ 1 to a high with ⁇ 3 .
- other formulas for determining ⁇ may alternatively be employed. Such other formulas, or even other sets of ⁇ values that are not necessarily formulaic, need not use an actual GPN numeral to set the ⁇ .
- the temporally varying parameter, ⁇ may be used to vary the degree of filtering using, for example, the following approach:
- G ⁇ ⁇ [ ⁇ , n ] K ⁇ 1 2 ⁇ ⁇ ⁇ ⁇ ⁇ e - 1 2 ⁇ ( ⁇ ⁇ ⁇ ⁇ ) 2 ⁇ n 2
- K is chosen to normalize the energy in the filter.
- the ⁇ formula may be set up so as to increase with increasing GPN.
- ⁇ 1 is the set of numbers ⁇ 1, 14/15, 13/15, 12/15, 11/15, . . . , 2/15, 1/15 ⁇
- ⁇ 2 is the set of numbers ⁇ 1, 1/2, 1/3, 1/4, 1/5, . . . , 1/14, 1/15 ⁇
- ⁇ 3 is the set of numbers ⁇ 1, 1/4, 1/9, 1/16, 1/25, . . . , 1/196, 1/225 ⁇ .
- An ⁇ formula or set of values that results in visually-pleasing video may be determined experimentally for any given type of video information.
- Use of the temporal filtering coefficient ⁇ as a modifier to a spatial filter 1202 enables application of a spatio-temporally varying filter that indirectly detects, differentiates between types of, and smoothes discontinuities.
- FIG. 14 is a flow diagram 1400 of an exemplary approach to block-type-dependent filtering under an MPEG-compliant compression/coding scheme.
- the second mechanism as described above with particular reference to FIGS. 11A-13B , may be applied to all frames of a GOP.
- the approach of flow diagram 1400 may be employed in which the video deblocking mechanism that is being applied depends on the frame/block type.
- Flow diagram 1400 is described in terms of blocks such as macroblocks, but it may instead be implemented on a frame-only basis. More generally, blocks, frames, macroblocks, etc. may be considered units of an image.
- each MPEG GOP starts with an I-frame.
- This I-frame has relatively hard and certain macroblock boundaries.
- Non-I-frames e.g., B-frames and P-frames
- the first mechanism (which is more direct) may be applied to I-frames while the second mechanism (which is more indirect) may be applied to non-I-frames.
- the deblocking under such a scheme is frame-type-dependent.
- individual macroblocks in a non-I-frame may be equivalent to an I-frame.
- a determination as to which mechanism is to be used under an MPEG-compliant coding scheme may be block-type-dependent.
- a flow diagram block 1402 determines whether a block under consideration is (i) an intra block or (ii) a predicted or a bi-directional block. For example, prior to actual smoothing, each macroblock of a frame of video information may be analyzed to determine whether it is an I-block or a P-block/B-block. If it is determined to be an I-block, then a visual adjustment procedure or a filter smoothing procedure is implemented at flow diagram block 1404 . Whether a visual adjustment procedure or a filter smoothing procedure is implemented is dependent on discontinuity type. As described further above with reference especially to FIGS. 6-10 , the visual adjustment procedure is applied to real discontinuities and the filter smoothing procedure is applied to artificial discontinuities.
- the block under consideration is determined to be a P-block or a B-block (e.g., a non-I-block)
- a spatio-temporally varying filtering procedure is implemented for the block under consideration at flow diagram block 1406 .
- the extent of filtering for any given pixel or set of pixels is dependent on the spatial position of the pixel or pixels within the block.
- the extent of filtering for the given pixel or set of pixels is also dependent on the temporal-based GPN of the frame of which the block forms a part.
- Such a spatio-temporally varying filtering procedure is described further above with reference especially to FIGS. 11A-13B .
- the different types of MPEG frames or blocks may therefore be handled using different deblocking mechanisms.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Description
and the Gaussian operator, which is given by:
thereby yielding:
The filtered image is therefore given by:
I LG =LoG*I.
is passed through the LoG filter to obtain QtLG. The quantization matrix Qt is divided by 2 because half the quantization matrix Qt represents the actual loss of data due to round-off/truncation. After obtaining QtLG from passing
through the LoG filter, the maximum and minimum values of QtLG are examined to determine the quant_threshold.
In general, any smoothing filter can be used with the degree of smoothing set to accommodate the desired amount of blurring.
Zone 1: | σ = 1; | ||
Zone 2: | σ = 0.707; and | ||
Zone 3: | σ = 0.5. | ||
Furthermore, the filters can be normalized in order to achieve a gain of 1 (or any desired value). Such spatial filtering thus concentrates filtering near macroblock boundaries and away from the center of macroblocks. Using a spatial filter designed accordingly effectively tends to indirectly filter blockiness due to quantization noise.
where the number K is chosen to normalize the energy in the filter. Also, if the filtering ability of the
Claims (14)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/191,220 US7567619B2 (en) | 2002-06-24 | 2005-07-27 | Video deblocking |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/179,825 US20030235250A1 (en) | 2002-06-24 | 2002-06-24 | Video deblocking |
US11/191,220 US7567619B2 (en) | 2002-06-24 | 2005-07-27 | Video deblocking |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/179,825 Division US20030235250A1 (en) | 2002-06-24 | 2002-06-24 | Video deblocking |
Publications (2)
Publication Number | Publication Date |
---|---|
US20050281340A1 US20050281340A1 (en) | 2005-12-22 |
US7567619B2 true US7567619B2 (en) | 2009-07-28 |
Family
ID=29734992
Family Applications (4)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/179,825 Abandoned US20030235250A1 (en) | 2002-06-24 | 2002-06-24 | Video deblocking |
US11/191,131 Expired - Fee Related US7567618B2 (en) | 2002-06-24 | 2005-07-27 | Video deblocking |
US11/191,220 Expired - Fee Related US7567619B2 (en) | 2002-06-24 | 2005-07-27 | Video deblocking |
US11/191,204 Expired - Fee Related US7660351B2 (en) | 2002-06-24 | 2005-07-27 | Video deblocking |
Family Applications Before (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/179,825 Abandoned US20030235250A1 (en) | 2002-06-24 | 2002-06-24 | Video deblocking |
US11/191,131 Expired - Fee Related US7567618B2 (en) | 2002-06-24 | 2005-07-27 | Video deblocking |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/191,204 Expired - Fee Related US7660351B2 (en) | 2002-06-24 | 2005-07-27 | Video deblocking |
Country Status (1)
Country | Link |
---|---|
US (4) | US20030235250A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100033633A1 (en) * | 2006-12-28 | 2010-02-11 | Gokce Dane | Detecting block artifacts in coded images and video |
Families Citing this family (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
ATE527591T1 (en) * | 2002-11-15 | 2011-10-15 | Qualcomm Inc | APPARATUS AND METHOD FOR MULTIPLE DESCRIPTION CODING |
US7995849B2 (en) * | 2003-03-17 | 2011-08-09 | Qualcomm, Incorporated | Method and apparatus for improving video quality of low bit-rate video |
US8625680B2 (en) * | 2003-09-07 | 2014-01-07 | Microsoft Corporation | Bitstream-controlled post-processing filtering |
US7724827B2 (en) * | 2003-09-07 | 2010-05-25 | Microsoft Corporation | Multi-layer run level encoding and decoding |
US20090064242A1 (en) * | 2004-12-23 | 2009-03-05 | Bitband Technologies Ltd. | Fast channel switching for digital tv |
KR100679035B1 (en) * | 2005-01-04 | 2007-02-06 | 삼성전자주식회사 | Deblocking filtering method considering intra BL mode, and video encoder/decoder based on multi-layer using the method |
CN101147163B (en) * | 2005-01-19 | 2011-01-12 | Nxp股份有限公司 | Device for and method of providing operating data and/or data associated with playback data to a remote device |
US8340098B2 (en) * | 2005-12-07 | 2012-12-25 | General Instrument Corporation | Method and apparatus for delivering compressed video to subscriber terminals |
US20070171980A1 (en) * | 2006-01-26 | 2007-07-26 | Yen-Lin Lee | Method and Related Apparatus For Decoding Video Streams |
KR100771879B1 (en) * | 2006-08-17 | 2007-11-01 | 삼성전자주식회사 | Method of deblocking filtering decreasing inner memory storage and a video processing device using the method |
GB2442256A (en) * | 2006-09-28 | 2008-04-02 | Tandberg Television Asa | Position-dependent spatial filtering |
US8700792B2 (en) * | 2008-01-31 | 2014-04-15 | General Instrument Corporation | Method and apparatus for expediting delivery of programming content over a broadband network |
JP5050158B2 (en) * | 2008-06-02 | 2012-10-17 | 株式会社メガチップス | Transcoder |
US20090304086A1 (en) * | 2008-06-06 | 2009-12-10 | Apple Inc. | Method and system for video coder and decoder joint optimization |
US8752092B2 (en) | 2008-06-27 | 2014-06-10 | General Instrument Corporation | Method and apparatus for providing low resolution images in a broadcast system |
US9357244B2 (en) * | 2010-03-11 | 2016-05-31 | Arris Enterprises, Inc. | Method and system for inhibiting audio-video synchronization delay |
US8976856B2 (en) * | 2010-09-30 | 2015-03-10 | Apple Inc. | Optimized deblocking filters |
US8787443B2 (en) | 2010-10-05 | 2014-07-22 | Microsoft Corporation | Content adaptive deblocking during video encoding and decoding |
US9042458B2 (en) | 2011-04-01 | 2015-05-26 | Microsoft Technology Licensing, Llc | Multi-threaded implementations of deblock filtering |
WO2012148238A2 (en) * | 2011-04-28 | 2012-11-01 | 삼성전자 주식회사 | Method and apparatus for adjusting a data transmission rate in a wireless communication system |
US8982948B2 (en) | 2011-12-21 | 2015-03-17 | Sony Corporation | Video system with quantization matrix coding mechanism and method of operation thereof |
US9667996B2 (en) * | 2013-09-26 | 2017-05-30 | Qualcomm Incorporated | Sub-prediction unit (PU) based temporal motion vector prediction in HEVC and sub-PU design in 3D-HEVC |
US9762927B2 (en) | 2013-09-26 | 2017-09-12 | Qualcomm Incorporated | Sub-prediction unit (PU) based temporal motion vector prediction in HEVC and sub-PU design in 3D-HEVC |
Citations (46)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0381067A2 (en) | 1989-01-31 | 1990-08-08 | Schlumberger Technologies, Inc. | A method for registration of CAD model to video images with added clutter |
US5142592A (en) | 1990-12-17 | 1992-08-25 | Moler Keith E | Method and apparatus for detection of parallel edges in image processing |
US5446804A (en) | 1994-04-14 | 1995-08-29 | Hewlett-Packard Company | Magnifying digital image using edge mapping |
US5512956A (en) * | 1994-02-04 | 1996-04-30 | At&T Corp. | Adaptive spatial-temporal postprocessing for low bit-rate coded image sequences |
US5548662A (en) | 1993-02-08 | 1996-08-20 | Lg Electronics Inc. | Edge extracting method and apparatus using diffusion neural network |
US5668598A (en) | 1995-03-27 | 1997-09-16 | International Business Machines Corporation | Motion video compression system with guaranteed bit production limits |
US5684894A (en) | 1994-11-28 | 1997-11-04 | Eastman Kodak Company | Scale specific and robust line/edge encoding of images |
US5796875A (en) * | 1996-08-13 | 1998-08-18 | Sony Electronics, Inc. | Selective de-blocking filter for DCT compressed images |
US5802213A (en) * | 1994-10-18 | 1998-09-01 | Intel Corporation | Encoding video signals using local quantization levels |
US5805221A (en) * | 1994-10-31 | 1998-09-08 | Daewoo Electronics Co., Ltd. | Video signal coding system employing segmentation technique |
US5850294A (en) | 1995-12-18 | 1998-12-15 | Lucent Technologies Inc. | Method and apparatus for post-processing images |
US5852475A (en) | 1995-06-06 | 1998-12-22 | Compression Labs, Inc. | Transform artifact reduction process |
US5903673A (en) | 1997-03-14 | 1999-05-11 | Microsoft Corporation | Digital video signal encoder and encoding method |
US5995080A (en) | 1996-06-21 | 1999-11-30 | Digital Equipment Corporation | Method and apparatus for interleaving and de-interleaving YUV pixel data |
US6014693A (en) | 1996-03-29 | 2000-01-11 | Mitsubishi Denki Kabushiki Kaisha | System for delivering compressed stored video data by adjusting the transfer bit rate to compensate for high network load |
US6040861A (en) | 1997-10-10 | 2000-03-21 | International Business Machines Corporation | Adaptive real-time encoding of video sequence employing image statistics |
US6104434A (en) | 1996-10-24 | 2000-08-15 | Fujitsu Limited | Video coding apparatus and decoding apparatus |
US6178205B1 (en) * | 1997-12-12 | 2001-01-23 | Vtel Corporation | Video postfiltering with motion-compensated temporal filtering and/or spatial-adaptive filtering |
US6181742B1 (en) | 1998-01-26 | 2001-01-30 | International Business Machines Corporation | Single pass target allocation for video encoding |
US6278735B1 (en) | 1998-03-19 | 2001-08-21 | International Business Machines Corporation | Real-time single pass variable bit rate control strategy and encoder |
US6281942B1 (en) | 1997-08-11 | 2001-08-28 | Microsoft Corporation | Spatial and temporal filtering mechanism for digital motion video signals |
US6285801B1 (en) | 1998-05-29 | 2001-09-04 | Stmicroelectronics, Inc. | Non-linear adaptive image filter for filtering noise such as blocking artifacts |
US20010019634A1 (en) | 2000-01-21 | 2001-09-06 | Nokia Mobile Phones Ltd. | Method for filtering digital images, and a filtering device |
US6320905B1 (en) * | 1998-07-08 | 2001-11-20 | Stream Machine Company | Postprocessing system for removing blocking artifacts in block-based codecs |
US6373482B1 (en) | 1998-12-23 | 2002-04-16 | Microsoft Corporation | Method, system, and computer program product for modified blending between clip-map tiles |
US6449255B1 (en) | 1999-04-26 | 2002-09-10 | Cisco Technology, Inc. | Method and apparatus for managing packets using a real-time feedback signal |
US20020159096A1 (en) * | 2001-03-29 | 2002-10-31 | Sharp Laboratories Of America, Inc. | Adaptive image filtering based on a distance transform |
US6504873B1 (en) * | 1997-06-13 | 2003-01-07 | Nokia Mobile Phones Ltd. | Filtering based on activities inside the video blocks and at their boundary |
US20030035586A1 (en) * | 2001-05-18 | 2003-02-20 | Jim Chou | Decoding compressed image data |
US6539060B1 (en) | 1997-10-25 | 2003-03-25 | Samsung Electronics Co., Ltd. | Image data post-processing method for reducing quantization effect, apparatus therefor |
US20030058944A1 (en) | 2001-09-24 | 2003-03-27 | Macinnis Alexander G. | Method and apparatus for performing deblocking filtering with interlace capability |
US6611503B1 (en) | 1998-05-22 | 2003-08-26 | Tandberg Telecom As | Method and apparatus for multimedia conferencing with dynamic bandwidth allocation |
US6665346B1 (en) | 1998-08-01 | 2003-12-16 | Samsung Electronics Co., Ltd. | Loop-filtering method for image data and apparatus therefor |
US6668095B2 (en) | 1998-12-03 | 2003-12-23 | Koninklijke Philips Electronics N.V. | Systems and methods for compressing and decompressing images |
US6690838B2 (en) | 1998-11-30 | 2004-02-10 | Equator Technologies, Inc. | Image processing circuit and method for reducing a difference between pixel values across an image boundary |
US6728414B1 (en) | 1998-11-25 | 2004-04-27 | Samsung Electronics Co., Ltd. | De-blocking method and apparatus |
US6816166B2 (en) | 2000-02-25 | 2004-11-09 | International Business Machines Corporation | Image conversion method, image processing apparatus, and image display apparatus |
US6898321B1 (en) * | 1998-10-09 | 2005-05-24 | Snell & Wilcox Limited | Method and apparatus for blocking effect reduction |
US6950473B2 (en) | 2002-06-21 | 2005-09-27 | Seiko Epson Corporation | Hybrid technique for reducing blocking and ringing artifacts in low-bit-rate coding |
US6963613B2 (en) | 2002-04-01 | 2005-11-08 | Broadcom Corporation | Method of communicating between modules in a decoding system |
US6983079B2 (en) | 2001-09-20 | 2006-01-03 | Seiko Epson Corporation | Reducing blocking and ringing artifacts in low-bit-rate coding |
US6996285B2 (en) * | 2000-09-28 | 2006-02-07 | Sony International (Europe) Gmbh | Quality rating function for a discrete decoded picture |
US7003174B2 (en) * | 2001-07-02 | 2006-02-21 | Corel Corporation | Removal of block encoding artifacts |
US7031392B2 (en) * | 2002-09-20 | 2006-04-18 | Seiko Epson Corporation | Method and apparatus for video deblocking |
US7120197B2 (en) | 2001-12-17 | 2006-10-10 | Microsoft Corporation | Motion compensation loop with filtering |
US7227901B2 (en) | 2002-11-21 | 2007-06-05 | Ub Video Inc. | Low-complexity deblocking filter |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5668698A (en) * | 1996-01-22 | 1997-09-16 | General Motors Corporation | Smart connector for an electrical device |
-
2002
- 2002-06-24 US US10/179,825 patent/US20030235250A1/en not_active Abandoned
-
2005
- 2005-07-27 US US11/191,131 patent/US7567618B2/en not_active Expired - Fee Related
- 2005-07-27 US US11/191,220 patent/US7567619B2/en not_active Expired - Fee Related
- 2005-07-27 US US11/191,204 patent/US7660351B2/en not_active Expired - Fee Related
Patent Citations (48)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0381067A2 (en) | 1989-01-31 | 1990-08-08 | Schlumberger Technologies, Inc. | A method for registration of CAD model to video images with added clutter |
US5142592A (en) | 1990-12-17 | 1992-08-25 | Moler Keith E | Method and apparatus for detection of parallel edges in image processing |
US5548662A (en) | 1993-02-08 | 1996-08-20 | Lg Electronics Inc. | Edge extracting method and apparatus using diffusion neural network |
US5512956A (en) * | 1994-02-04 | 1996-04-30 | At&T Corp. | Adaptive spatial-temporal postprocessing for low bit-rate coded image sequences |
US5446804A (en) | 1994-04-14 | 1995-08-29 | Hewlett-Packard Company | Magnifying digital image using edge mapping |
US5661824A (en) | 1994-04-14 | 1997-08-26 | Allebach; Jan P. | Magnifying digital image using mapping |
US5802213A (en) * | 1994-10-18 | 1998-09-01 | Intel Corporation | Encoding video signals using local quantization levels |
US5805221A (en) * | 1994-10-31 | 1998-09-08 | Daewoo Electronics Co., Ltd. | Video signal coding system employing segmentation technique |
US5684894A (en) | 1994-11-28 | 1997-11-04 | Eastman Kodak Company | Scale specific and robust line/edge encoding of images |
US5668598A (en) | 1995-03-27 | 1997-09-16 | International Business Machines Corporation | Motion video compression system with guaranteed bit production limits |
US5920356A (en) | 1995-06-06 | 1999-07-06 | Compressions Labs, Inc. | Coding parameter adaptive transform artifact reduction process |
US5852475A (en) | 1995-06-06 | 1998-12-22 | Compression Labs, Inc. | Transform artifact reduction process |
US5850294A (en) | 1995-12-18 | 1998-12-15 | Lucent Technologies Inc. | Method and apparatus for post-processing images |
US6014693A (en) | 1996-03-29 | 2000-01-11 | Mitsubishi Denki Kabushiki Kaisha | System for delivering compressed stored video data by adjusting the transfer bit rate to compensate for high network load |
US5995080A (en) | 1996-06-21 | 1999-11-30 | Digital Equipment Corporation | Method and apparatus for interleaving and de-interleaving YUV pixel data |
US5796875A (en) * | 1996-08-13 | 1998-08-18 | Sony Electronics, Inc. | Selective de-blocking filter for DCT compressed images |
US6104434A (en) | 1996-10-24 | 2000-08-15 | Fujitsu Limited | Video coding apparatus and decoding apparatus |
US5903673A (en) | 1997-03-14 | 1999-05-11 | Microsoft Corporation | Digital video signal encoder and encoding method |
US6504873B1 (en) * | 1997-06-13 | 2003-01-07 | Nokia Mobile Phones Ltd. | Filtering based on activities inside the video blocks and at their boundary |
US6281942B1 (en) | 1997-08-11 | 2001-08-28 | Microsoft Corporation | Spatial and temporal filtering mechanism for digital motion video signals |
US6040861A (en) | 1997-10-10 | 2000-03-21 | International Business Machines Corporation | Adaptive real-time encoding of video sequence employing image statistics |
US6539060B1 (en) | 1997-10-25 | 2003-03-25 | Samsung Electronics Co., Ltd. | Image data post-processing method for reducing quantization effect, apparatus therefor |
US6178205B1 (en) * | 1997-12-12 | 2001-01-23 | Vtel Corporation | Video postfiltering with motion-compensated temporal filtering and/or spatial-adaptive filtering |
US6181742B1 (en) | 1998-01-26 | 2001-01-30 | International Business Machines Corporation | Single pass target allocation for video encoding |
US6278735B1 (en) | 1998-03-19 | 2001-08-21 | International Business Machines Corporation | Real-time single pass variable bit rate control strategy and encoder |
US6611503B1 (en) | 1998-05-22 | 2003-08-26 | Tandberg Telecom As | Method and apparatus for multimedia conferencing with dynamic bandwidth allocation |
US6285801B1 (en) | 1998-05-29 | 2001-09-04 | Stmicroelectronics, Inc. | Non-linear adaptive image filter for filtering noise such as blocking artifacts |
US6320905B1 (en) * | 1998-07-08 | 2001-11-20 | Stream Machine Company | Postprocessing system for removing blocking artifacts in block-based codecs |
US6665346B1 (en) | 1998-08-01 | 2003-12-16 | Samsung Electronics Co., Ltd. | Loop-filtering method for image data and apparatus therefor |
US6898321B1 (en) * | 1998-10-09 | 2005-05-24 | Snell & Wilcox Limited | Method and apparatus for blocking effect reduction |
US6728414B1 (en) | 1998-11-25 | 2004-04-27 | Samsung Electronics Co., Ltd. | De-blocking method and apparatus |
US6690838B2 (en) | 1998-11-30 | 2004-02-10 | Equator Technologies, Inc. | Image processing circuit and method for reducing a difference between pixel values across an image boundary |
US6668095B2 (en) | 1998-12-03 | 2003-12-23 | Koninklijke Philips Electronics N.V. | Systems and methods for compressing and decompressing images |
US6373482B1 (en) | 1998-12-23 | 2002-04-16 | Microsoft Corporation | Method, system, and computer program product for modified blending between clip-map tiles |
US6449255B1 (en) | 1999-04-26 | 2002-09-10 | Cisco Technology, Inc. | Method and apparatus for managing packets using a real-time feedback signal |
US20010019634A1 (en) | 2000-01-21 | 2001-09-06 | Nokia Mobile Phones Ltd. | Method for filtering digital images, and a filtering device |
US6816166B2 (en) | 2000-02-25 | 2004-11-09 | International Business Machines Corporation | Image conversion method, image processing apparatus, and image display apparatus |
US6996285B2 (en) * | 2000-09-28 | 2006-02-07 | Sony International (Europe) Gmbh | Quality rating function for a discrete decoded picture |
US20020159096A1 (en) * | 2001-03-29 | 2002-10-31 | Sharp Laboratories Of America, Inc. | Adaptive image filtering based on a distance transform |
US20030035586A1 (en) * | 2001-05-18 | 2003-02-20 | Jim Chou | Decoding compressed image data |
US7003174B2 (en) * | 2001-07-02 | 2006-02-21 | Corel Corporation | Removal of block encoding artifacts |
US6983079B2 (en) | 2001-09-20 | 2006-01-03 | Seiko Epson Corporation | Reducing blocking and ringing artifacts in low-bit-rate coding |
US20030058944A1 (en) | 2001-09-24 | 2003-03-27 | Macinnis Alexander G. | Method and apparatus for performing deblocking filtering with interlace capability |
US7120197B2 (en) | 2001-12-17 | 2006-10-10 | Microsoft Corporation | Motion compensation loop with filtering |
US6963613B2 (en) | 2002-04-01 | 2005-11-08 | Broadcom Corporation | Method of communicating between modules in a decoding system |
US6950473B2 (en) | 2002-06-21 | 2005-09-27 | Seiko Epson Corporation | Hybrid technique for reducing blocking and ringing artifacts in low-bit-rate coding |
US7031392B2 (en) * | 2002-09-20 | 2006-04-18 | Seiko Epson Corporation | Method and apparatus for video deblocking |
US7227901B2 (en) | 2002-11-21 | 2007-06-05 | Ub Video Inc. | Low-complexity deblocking filter |
Non-Patent Citations (5)
Title |
---|
Al-Fahoum, Amjed S., "Combined Edge Crispness and Statistical Differencing for Deblocking JPEG Compressed Images", IEEE Transactions of Image Processing, vol. 10, No. 9, Sep. 2001, pp. 1288-1298. |
Chou, Jim et al., "A Simple Algorithm For Removing Blocking Artifacts in Block-Transform Coded Images", University of Illinois as Urbana-Champaign, Dept. of Electrical and Computer Engineering; Sep. 27, 1997, 10 pages. |
Chou, Jim, "A Simple Algorithm for Removing Blocking Artifacts in Block-Transform Coded Images", IEEE Signal Processing Letters, vol. 5, No. 2, Feb. 1998, pp. 33-35. |
Sung, Duek Kim, et al., "A Deblocking Filter with Two Separate Modes in Block-Based Video Coding", IEEE TRansactions on Circuits and Systems for Video Technology, vol. 9, No. 1, Feb. 1999, pp. 156-160. |
Wang, Qiu-Hong, "Reducation of Blocking Artifacts in Real Time Video Compression", Mini-Micro System, vol. 22, No. 6, Jun. 2001, 5 pages, Including English-language Abstract. |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100033633A1 (en) * | 2006-12-28 | 2010-02-11 | Gokce Dane | Detecting block artifacts in coded images and video |
US8879001B2 (en) * | 2006-12-28 | 2014-11-04 | Thomson Licensing | Detecting block artifacts in coded images and video |
Also Published As
Publication number | Publication date |
---|---|
US7660351B2 (en) | 2010-02-09 |
US20050259745A1 (en) | 2005-11-24 |
US7567618B2 (en) | 2009-07-28 |
US20050262531A1 (en) | 2005-11-24 |
US20030235250A1 (en) | 2003-12-25 |
US20050281340A1 (en) | 2005-12-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7567619B2 (en) | Video deblocking | |
US7620261B2 (en) | Edge adaptive filtering system for reducing artifacts and method | |
US7283588B2 (en) | Deblocking filter | |
US10027966B2 (en) | Apparatus and method for compressing pictures with ROI-dependent compression parameters | |
US7787541B2 (en) | Dynamic pre-filter control with subjective noise detector for video compression | |
US8831111B2 (en) | Decoding with embedded denoising | |
US7848408B2 (en) | Method and system for parameter generation for digital noise reduction based on bitstream properties | |
EP1564997A1 (en) | Encoding and decoding of video images based on a quantization with an adaptive dead-zone size | |
US9071844B2 (en) | Motion estimation with motion vector penalty | |
US20110299604A1 (en) | Method and apparatus for adaptive video sharpening | |
US7822125B2 (en) | Method for chroma deblocking | |
US20090080517A1 (en) | Method and Related Device for Reducing Blocking Artifacts in Video Streams | |
US8077773B2 (en) | Systems and methods for highly efficient video compression using selective retention of relevant visual detail | |
US20070076802A1 (en) | Video presentation at fractional speed factor using time domain interpolation | |
US6810082B1 (en) | Chroma based adaptive signal peaking | |
US20160205398A1 (en) | Apparatuses and methods for efficient random noise encoding | |
US20090060368A1 (en) | Method and System for an Adaptive HVS Filter | |
US20030031377A1 (en) | Apparatus and method for removing block artifacts, and displaying device having the same apparatus | |
US6040875A (en) | Method to compensate for a fade in a digital video input sequence | |
KR100803132B1 (en) | Method and apparatus for reduction MPEG noise using wavelet transform | |
KR101051331B1 (en) | Adaptive Fast Mode Decision Method Using Dynamic Threshold | |
Davies | A Modified Rate-Distortion Optimisation Strategy for Hybrid Wavelet Video Coding | |
Richardson et al. | Varying slice size to improve error tolerance of MPEG video | |
Basavaraju et al. | Modified pre and post processing methods for optimizing and improving the quality of VP8 video codec | |
Bodecek et al. | Image compression in digital video broadcasting |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
AS | Assignment |
Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICROSOFT CORPORATION;REEL/FRAME:034543/0001 Effective date: 20141014 |
|
FPAY | Fee payment |
Year of fee payment: 8 |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20210728 |