RU2575418C2 - Filter of central band adaptive shift for video coding - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: claimed video coder is configured for filtration with adaptive shift of counts. It can define the magnitude of the centre of the set of pixels proceeding from the magnitudes of pixels in the given set. Besides, it can divide the bands of pixel magnitudes into groups on the basis of said magnitude of the centre to define the shift of said bands on the basis of said groups.

EFFECT: higher efficiency of coding due to adaptive determination of groups of bands.

37 cl, 7 dwg

Description

This application claims priority to provisional application US No. 61/556,085, filed November 4, 2011, the contents of which are incorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates generally to video coding and, in particular, relates to filtering with adaptive sample offset (SAO) in a video coding process.

BACKGROUND

Digital video capabilities can be incorporated into a wide range of applications, including digital television sets, direct digital broadcasting systems, wireless broadcasting systems, personal digital assistants (PDAs), laptops or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite radiotelephones, so-called “smartphones”, devices for I'm holding newsgroups, video streaming devices, etc. Digital video devices implement various video compression techniques, which, for example, are described in standards defined as MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, Advanced Standard video coding (AVC), a high-performance video coding (HEVC) standard under development, as well as an extension of these standards. Video devices can transmit, receive, encode, decode and / or store digital video information more efficiently by implementing these video compression techniques.

According to video compression techniques, spatial (intra-frame, intra-) prediction and / or temporal (inter-frame, inter-) prediction is performed to reduce or eliminate the redundancy inherent in video sequences. For block-based video coding, a video slice (i.e., a video frame or part of a video frame) can be divided into video blocks, which can also be called treeblocks, coding units (CUs), and / or coding units. Video blocks in an intra-coded (I) image slice are encoded using spatial prediction as applied to reference samples in neighboring blocks of the same image. In video blocks in an inter-coded image slice (P or B), spatial prediction for reference samples in adjacent blocks of the same image or temporal prediction for reference samples of other reference images can be used. Images may be referred to as frames, and reference images may be referred to as reference frames.

As a result of spatial or temporal prediction, a predicted block is obtained for the block to be encoded. The residual data represents the pixel difference between the source block to be encoded and the predicted block. An inter-coded block is encoded in accordance with a motion vector that indicates a block of reference samples forming a predicted block and residual data indicating the difference between the coded block and the predicted block. The intra-encoded block is encoded in accordance with the intra-encoding mode and residual data. For further compression, the residual data can be converted from the pixel region to the transformation region, resulting in residual transformation coefficients, which can then be quantized. The quantized transform coefficients, originally arranged in a two-dimensional matrix, can be scanned to form a one-dimensional vector of transform coefficients. Then you can apply entropy coding to achieve even greater compression.

SUMMARY OF THE INVENTION

Here, in the general case, techniques related to adaptive sampling bias (SAO) filtering are described, and in particular, techniques for signaling the coefficients used for band-based SAO filtering are described in the encoded bitstream. This invention describes techniques for adaptively determining band groups based on a center value, and using certain groups to signal offset values for said bands.

In one example, a video encoding method includes determining a center value for a set of pixels, the center value being based on the values of said pixels; dividing the bands of pixel values into groups based on the center value; and determining bias values for said bands based on said groups.

In another example, a video coding apparatus includes a video encoder configured to determine a center value for a set of pixels, the center value being based on the values of said pixels; dividing the bands of pixel values into groups based on the center value; and determining bias values for said bands based on said groups.

In yet another example, instructions are stored in a computer-readable storage medium that initiate the determination by one or more central processors of a center value for a set of pixels, the center value being based on the values of said pixels; dividing by one or more processors the bands of pixel values into groups based on the center value; and determining by one or more processors the offset values for said bands based on said groups.

In the following example, a device for encoding video data includes means for determining a center value for a set of pixels, the center value being based on the values of said pixels; means for dividing the bands of pixel values into groups based on the center value; and means for determining bias values for said bands based on said groups.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example video coding and decoding system in which filtering techniques for adaptive sample biasing described in the present invention can be used;

FIG. 2 is an example of a classification of edge offsets for filtering with adaptive edge offset sampling;

Figures 3A and 3B are examples of band grouping for classifying band offsets for filtering with adaptive band offset samples;

FIG. 4 is a flowchart illustrating an example video encoder that can implement adaptive sample offset filtering techniques described in the present invention;

FIG. 5 is a flowchart illustrating an example video decoder that can implement the adaptive sample offset filtering techniques described in this invention;

FIG. 6 is a flowchart illustrating adaptive bias filtering techniques described herein.

DETAILED DESCRIPTION

Adaptive Sampling Offset (SAO) filtering is a type of loop filtering used in video coding. In general, adding offset values to pixels in a video frame (for example, in a reconstructed image) can in some cases improve coding without significantly increasing the bit-productive bit load required to store or transmit encoded video data. An increase in coding efficiency, potentially resulting from the use of SAO filtering, may lie, for example, in that the decoded image will have a closer resemblance to the original image. As explained in more detail below, SAO techniques allow you to apply different offset values for different pixels (or blocks of pixels) depending on the classification metric of a pixel (or block), such as a border metric, strip metric, or other types of metrics.

As explained in more detail below, in some configurations, the SAO filter unit may be configured to perform bias filtering of two types, commonly referred to herein as band offset filtering and edge offset filtering. The SAO filter unit sometimes also may not apply an offset, which in itself can be considered as filtering with a third type of offset. The type of offset filtering used by the SAO filter can be reported to the video decoder explicitly or implicitly. When applying filtering with edge offsets, pixels can be classified based on the information about the edge of the coding block, and the offset for pixels can be determined based on the classification of edges. As explained in more detail with reference to FIG. 2, there are usually four edge-based SAOs, where the pixel value is compared with two of its eight neighboring pixels. Which two pixels to use for comparison depends on the edge offset option used. The offset is added to the pixel value based on the difference in magnitude.

When using filtering with strip offset, the pixels can be assigned to different stripes depending on the pixel value, for example, the intensity value, and each strip has a corresponding offset. The bar includes a range of pixel values. For example, pixel values in the range from 0 to 255 can be divided into 32 equal bands (designated from 0 to 31), so that pixel values 0-7 will represent the first strip, pixel values 8-15 will represent the second strip, pixel values 16- 23 will represent the third lane and so on for all thirty-two lanes. These stripes can be used to determine which specific offset value to apply to a pixel or group of pixels. For example, if the pixel has a value of 10 (which is within the second strip, that is, in the range of values 8-15, in the above example), then the offset associated with the second strip can be added to the pixel value.

In order to signal and create offsets for different bands, these bands can be grouped into two or more groups. For filtering with strip shifting, pixels, for example, can be assigned to thirty-two bands (bands 0-31), as described above, and these bands can be grouped into two groups (for example, two groups of sixteen bands, one group of four bands and one group of twenty-eight bands, one group of eight bands and one group of twenty-four bands, or use other grouping options). These banding options can be used to determine the order in which the offset values are signaled for the specified bands in the encoded video bitstream, and / or they can be used to determine if a particular band has a non-zero offset value. The offset signaling for the indicated bands can be carried out using various coding techniques, according to which the current value signaling is transmitted as the difference between the current value and the previous value. The bit savings achieved as a result of using the indicated coding techniques, in some cases, may depend on the order of signaling about the mentioned values. For example, more significant bit savings can be achieved in cases where close offset values are transmitted as signaling in series. In some cases, it can be assumed that all bias values for a particular group, for example, the second group, are zero.

According to some techniques, these banding options are static. For example, according to one technique, sixteen middle bands (bands 8-23) always form one group of sixteen bands, while eight bands on the left (bands 0-7) and eight bands on the right (bands 24-31) form a second group of sixteen stripes. This invention describes techniques for dynamically determining band grouping options. As explained in more detail below, a video encoder or video decoder can dynamically determine a banding option by dynamically determining a center value based on pixel values. In some configurations, signaling indicating the center value can be provided in an encoded video bitstream, so that the video decoder does not have to determine the center value based on pixel values, but instead can determine this pixel value based on the signaling in an explicit form presented in the bitstream . The indicated center value can be used to determine the first group of bands, and the remaining bands can form a second group of bands. As will be clear from the examples below, the term “center value” as used in this invention usually refers to any value that can be used to align or approximate the center of a group of stripes to a specific pixel value. Thus, the term “center value” as used herein may refer to a specific pixel value at which the center of the band group is aligned, or may be some other value, for example, the initial pixel value or the final pixel value for a band or group of bands. In some cases, as opposed to identifying a pixel value, the center value can identify a particular strip as the first band of the group, the last band of the group, or represent another identification based on which the center of the group of bands can be aligned with a specific pixel value.

Dynamic determination of band grouping options in some cases improves signaling of offset coefficients for SAO filtering and thus can improve the quality of video coding as a whole. Such a dynamic determination of band grouping options can also be used to determine for which SAO band groups filtering would be most beneficial in terms of video coding quality, assuming other bands have a zero offset value.

In FIG. 1 is a flowchart showing an example of a video coding and decoding system in which adaptive SAO techniques described in the present invention can be used. As shown in FIG. 1, the system 10 includes a source device 12 that generates encoded video data to be further decoded by the destination device 14. The source device 12 and the destination device 14 may include any of a variety of devices, including a desktop computer, laptop (that is, “laptop”), tablet computer, television set-top box, handset, such as the so-called “smartphone”, the so-called “smartpad”, television receiver, camera, display device, digital media player, video game console l, streaming video or the like In some cases, source device 12 and destination device 14 may be equipped with wireless communications.

The destination device 14 may receive encoded video data to be decoded via communication line 16. Communication line 16 may comprise a medium or device of any type capable of transferring encoded video data from source device 12 to destination device 14. In one example, communication line 16 may include communication medium allowing source device 12 to transmit encoded video data directly to destination device 14 in real time. The encoded video data may be modulated in accordance with a communication standard, such as a wireless communication protocol, and may be transmitted to a destination device 14. The communication medium may comprise any wired or wireless communication medium, for example, a radio frequency (RF) spectrum or one or more physical lines transmission. A communication medium may form part of a packet network, such as a local area network, a regional network, or a wide area network, such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful for communicating between the source device 12 and the destination device 14.

Alternatively, the encoded data may be output from the output interface 22 to the storage device 32. Similarly, encoded data can be accessed from the storage device 32 through the input interface. Storage device 32 may include any storage medium from a wide variety of distributed or locally available storage media, such as a hard disk drive, Blu-ray discs, DVD discs, CD discs, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In yet another example, the storage device 32 may correspond to a file server or other intermediate storage device that can store the encoded video generated by the source device 12. The destination device 14 can access the stored video data in the storage device 32 using streaming or downloading video data. A file server can be any type of server capable of storing and storing encoded video data and transferring it to the destination device 14. Examples of file servers include a web server (for example, for a website), an FTP server, network attached storage devices (NAS) or local drive on disk. The destination device 14 may have access to the encoded video data through any standard data connection, including an Internet connection. Such a connection may include a wireless channel (for example, Wi-Fi connection), a wired connection (for example, DSL, cable modem, etc.), or a combination of both types of connections, which is suitable for access to encoded video data stored on a file server. The transmission of the encoded video data from the storage device 32 may be implemented as streaming, uploading, or a combination thereof.

The techniques disclosed herein are not necessarily limited to wireless applications or installations. These techniques can be applied to video coding to support any of a variety of multimedia applications, such as terrestrial television broadcasting, cable television, satellite television, streaming video, for example, via the Internet, digital video encoding for storage on a storage medium, digital video decoding, stored on a storage medium, or other applications. In some examples, system 10 may be configured to support unidirectional or bidirectional video transmission to support applications such as video streaming, video playback, video broadcasting and / or video telephony.

In the example of FIG. 1, source device 12 includes a video source 18, video encoder 20, and output interface 22. In some cases, output interface 22 may include a modulator / demodulator (modem) and / or transmitter. In the source device 12, the video source 18 may include a source such as a video capture device, for example, a video camera, a video archive containing previously captured video, a video feed interface for receiving video from a video content provider and / or computer graphics system for generating computer graphics data as the source video or a combination of the specified sources. For example, if the video source 18 is a video camera, then the source device 12 and the destination device 14 can form so-called video phones or camera phones. However, the techniques described in this invention can be applied to video coding in general and used for wireless and / or wired applications.

The currently captured, pre-captured or computer-generated video can be encoded by video encoder 20. The encoded video data can be transmitted directly to the destination device 14 via the output interface 22 of the source device 12. The encoded video data can also (or alternatively) be stored in a storage device 32 for future reference by the destination device 14 or other devices for decoding and / or reproduction.

The destination device 14 includes an input interface 28, a video decoder 30, and a display device 32. In some cases, the input interface 28 may include a receiver and / or modem. The input interface 28 of the destination device 14 receives the encoded video data via the communication line 16. The encoded video data transmitted through the communication line 16, or provided in the storage device 32, may include various syntax elements generated by the video encoder 20 for use by the video decoder, such as the video decoder 30, when decoding the video data. These syntactic elements may be included in encoded video data transmitted through a communication medium, stored on a storage medium or stored on a file server.

The display device 32 may be an integral part (or external) of the destination device 14. In some examples, the destination device 14 may include an integrated display device, and may also be configured to interact with an external display device. In other examples, destination device 14 may be a display device. In general, the display device 32 displays decoded video data to a user and may comprise any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

Video encoder 20 and video decoder 30 may operate according to a video compression standard, such as the high performance video coding (HEVC) standard currently being developed, and may conform to the HEVC test model (HM). The latest draft HEVC standard, called “HEVC Working Draft 8” or “WD8,” is described in JCTVC-H1003_d7, Bross et al., “High efficiency video coding (HEVC) text specification draft 8” by the Joint Video Coding Development Team (JCT-VC ) ITU-T standards SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, 10th Meeting: Stockholm, SE, July 11-20, 2012, which as of October 2, 2012 can be downloaded from the following link: http: // phenix.int-evry.fr/jct/doc_end_user/documents/10_Stockholm/wg11/JCTVC-J1003-v8.zip.

Alternatively, video encoder 20 and video decoder 30 may operate according to other commercial or industrial standards, such as ITU-T H.264, also called MPEG-4, part 10, AVC (Advanced Video Coding), or extensions to these standards. However, the techniques disclosed in this invention are not limited to any particular coding standard. Other examples of video compression standards include MPEG-2 and ITU-T H.263.

Although this is in FIG. 1 is not shown, in some aspects, video encoder 20 and video decoder 30 can each be integrated with an audio encoder and audio decoder and may include respective MUX-DEMUX units (multiplexer-demultiplexer) or other hardware and software for encoding both audio and video in general data stream or in separate data streams. In some examples, if this is the case, the MUX-DEMUX units may conform to the ITU H.223 Multiplexer Protocol or other protocols, such as User Datagram Protocol (UDP).

Video encoder 20 and video decoder 30 may each be implemented as any suitable encoder or decoder circuit, such as one or more microprocessors, DSP processors, ASIC integrated circuits, FPGA gate arrays, discrete logic circuits, software, hardware, firmware or any combination thereof. In the partial implementation of these techniques by software, the device can store software instructions on a suitable long-term computer-readable medium to execute these instructions in hardware using one or more processors that implement the techniques disclosed in the invention. Both video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, or each of them may be integrated in a combined encoder / decoder (“CODEC”) in a corresponding device.

JCT-VC is working to create the HEVC standard. The HEVC standardization work is based on an improved model of a video coding device called the HEVC standard test model (HM). The HM model has several additional features for video coding devices compared to existing devices that comply, for example, with the ITU-T H.264 / AVC standard. For example, while the H.264 standard provides nine intra-prediction coding modes, the HM model can provide up to thirty-three intra-prediction coding modes.

In general, the HM working model assumes that a video frame or video image can be divided into a sequence of treeblocks or the largest coding units (LCUs), which include luminance samples and color samples. The purpose of the treeblock is the same as the macroblock in the H.264 standard. A slice includes a series of consecutive treeblocks in coding order. A video frame or video image may be divided into one or more slices. Each treeblock can be divided into coding units (CUs) according to a quadtree. For example, the root node of a quad tree can be divided into four child nodes, and each child node, in turn, can be a parent node and can be divided into four other child nodes. The final undivided child node as the terminal node of the quad tree contains an encoding node, that is, an encoded video block. The syntax data associated with the encoded bitstream may determine the maximum number of possible tree block partitions, and may also determine the minimum size of the encoding nodes.

The CU includes coding unit and prediction units (PUs) and transform units (TUs) associated with the coding unit. The size of the CU unit corresponds to the size of the encoding node, and it should have a square shape. The CU block size can range from 8 × 8 pixels to a tree block size with a maximum size of 64 × 64 pixels or more. Each CU may contain one or more PUs and one or more TUs. The syntax data associated with the CU may describe, for example, the partitioning of the CU into one or more PUs. The partitioning modes may differ depending on whether the CU is skipped, as well as on which of the encoding modes: in direct encoding mode, intra prediction encoding or inter prediction. PU blocks can be divided into non-square parts. The syntax data associated with the CU may also describe, for example, splitting a CU into one or more TUs according to a quad tree. A TU may be square or non-square.

The HEVC standard allows conversion to be performed according to TUs, which may differ from one CU to another. The size of the TUs is typically based on the size of the PUs in a given CU determined for a split LCU, although this is not always the case. Typically, TUs are the same or smaller with PUs. In some examples, the residual samples corresponding to one CU can be further divided into smaller blocks using a quad tree structure known as a residual quad tree (RQT). The end nodes of the RQT tree may be called TUs. The pixel difference values associated with the TUs can be converted to create transform coefficients, followed by quantization.

In general, the PU unit includes data related to the prediction process. For example, when the PU block is encoded in intra-prediction mode, it may include data describing the intra-prediction mode for a given PU block. In another example, when the PU is coded in inter-prediction mode, it may include data defining a motion vector for a given PU. The data defining the motion vector for the PU can describe, for example, the horizontal component of the motion vector, the vertical component of the motion vector, the resolution for the motion vector (for example, accurate to a quarter of a pixel or accurate to one eighth of a pixel), a reference image, on which indicates a motion vector and / or a list of reference images (for example, list 0, list 1 or list C) for the motion vector.

In general, the TU is used in the transformation and quantization processes. A given CU having one or more PUs may also include one or more transform units (TUs). Following the prediction, video encoder 20 can calculate the residual values corresponding to this PU. Residual values contain pixel difference values that can be converted to transform coefficients, quantized and scanned using TUs to create serialized transform coefficients for entropy coding. In this description, the term “video block” is generally used to refer to a coding unit of a CU. In some specific cases, the term “video block” may also be used in the present description to refer to a tree block, that is, an LCU or CU, which includes an encoding unit and PUs and TUs.

A video sequence typically includes a series of video frames or video images. A group of images (GOP) generally comprises a series of one or more video images. The GOP may include syntax data in the header of the GOP, the header of one or more images, or elsewhere that describe the number of images included in the GOP. Each image slice may include slice syntax data that describes the encoding mode for the corresponding slice. Video encoder 20 typically operates with video blocks within separate video slices to encode video data. The video block may correspond to a coding unit in a CU. Video blocks may have fixed or variable sizes and may vary in size in accordance with a given coding standard.

For example, the HM model supports prediction for different PU sizes. Assume that the specific CU block size is 2Nx2N, the HM model supports intra prediction for PU sizes of 2Nx2N or NxN, and inter prediction for symmetric PU sizes of 2Nx2N, 2NxN, Nx2N or NxN. The HM model also supports asymmetric intra prediction for PU sizes of 2NxnU, 2NxnD, nLx2N, and nRx2N. With asymmetric splitting in one direction of the CU, there is no splitting, while in the other direction, splitting is performed in the ratio of 25% and 75%. The portion of the CU corresponding to 25% of the portion is indicated as “n”, followed by the designation “Up”, “Down”, “Left” or “Right”. Thus, for example, “2NxnU” refers to a 2Nx2N block CU that is horizontally divided into a 2Nx0.5N PU block from above and a 2Nx1.5N PU block from below.

In this description, the forms “NxN” and “N by N” can be used interchangeably to refer to the pixel dimensions of the video block in terms of vertical and horizontal sizes. For example, 16 × 16 pixels or 16 by 16 pixels. In general, a 16 × 16 block will have 16 pixels in the vertical direction (y = 16) and 16 pixels in the horizontal direction (x = 16). Similarly, an NxN block typically has N pixels in the vertical direction and N pixels in the horizontal direction, where N represents a non-negative integer value. Pixels in a block can be arranged in rows and columns. In addition, the blocks do not have to have the same number of pixels in the horizontal direction and in the vertical direction. For example, blocks may contain NxM pixels, where M is not necessarily N.

Following intra or inter prediction coding using the PU units of the CU, video encoder 20 may calculate the residual data for the TUs of the CU. PUs can contain pixel data in a spatial domain (also called a pixel domain), and TUs can contain coefficients in a transform domain after applying a transform, such as a discrete cosine transform (DCT), integer transform, wavelet transform, or a conceptually similar transform applied to residual video data. Residual data may correspond to pixel differences between the pixels of the unencoded image and the predicted values corresponding to the PUs. Video encoder 20 may form TUs including residual data for a CU, and then transform TUs to create transform coefficients for a given CU.

After any conversion to create conversion coefficients, video encoder 20 may quantize the conversion coefficients. Quantization generally refers to a process that quantizes transform coefficients in order to possibly reduce the amount of data used to represent these coefficients, which provides additional compression. The quantization process can reduce the bit depth associated with some or all of the mentioned coefficients. For example, an n-bit value can be rounded down to get an m-bit value during quantization, where n is greater than m.

In some examples, video encoder 20 may use a predetermined scan or “scan” order to scan the quantized transform coefficients to obtain a serialized vector that can be encoded using entropy coding. In other examples, video encoder 20 may perform adaptive scanning. After scanning the quantized transform coefficients to form a one-dimensional vector, video encoder 20 can encode this one-dimensional vector using entropy coding, for example, in accordance with context-sensitive adaptive coding with variable word length (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntactically oriented adaptive binary arithmetic coding (SBAC), probabilistic entropy coding with interval partitioning (PIPE), or a friend m entropy encoding method. Video encoder 20 may also perform entropy encoding of syntax elements associated with encoded video data for use by video decoder 30 when decoding video data.

To perform CABAC encoding, video encoder 20 may assign a context in the context model for the symbol to be transmitted. This context may relate, for example, to whether neighboring symbol values are nonzero. To perform CAVLC encoding, video encoder 20 may select a variable-length code for the character to be transmitted. VLC codewords can be constructed so that relatively shorter codes correspond to more probable characters, while relatively longer codes correspond to less probable characters. Thus, the use of VLC coding can provide economical use of bits, for example, by using codewords of the same length for each character to be transmitted. The definition of probability may be based on the context assigned to this symbol.

In general, adding offset values to pixels in a video frame may in some cases improve coding quality. For example, offset values can be used for the pixels of the reconstructed video block to compensate for changes in brightness, quantization errors, or, in a more general context, make the decoded data more similar to the original video data. SAO methods allow you to apply different offset values to different pixels (or blocks of pixels) depending on the values of the pixel (or block of pixels). The offset value used for the pixel can be determined based on the pixel value. For example, if a pixel has a value located in the first strip, then the offset associated with the first strip can be applied to this pixel. For example, if a pixel has a value located in the second strip, then the offset associated with the second strip can be applied to this pixel, and so on for all the stripes.

In an SAO implementation of one type, each section (which consists of a set of LCUs) can have an offset of one of three types (underlying the so-called "pixel classification"). These three types of offsets are: “no offset”, “offset type 0/1 based on strip classification”, and “offset type 0/1/2/3 based on edge classification”. Each type of offset for classifying the bands has 16 possible offset values, while each type based on the classification of edges has 4 possible offset values. If one of the indicated types of bias is selected for use for the aforementioned portion, an alarm may be performed in the encoded video bitstream containing information that indicates the corresponding type of bias and offset values.

In FIG. 2 is a conceptual diagram showing four possible edge classifications used in one implementation of SAO filtering. In the example of FIG. 2, an edge offset type classifies each pixel based on edge information. For each edge classification shown in FIG. 2, the edge type for the current pixel (i.e., the particular currently encoded pixel) is calculated by comparing the values of the current pixel (C) with the values of neighboring pixels (1 and 2). When the edge is shifted according to SAO classification “0” (SAO_EO_0), the current pixel is compared with the left and right neighboring pixels. When the edge is shifted according to SAO classification “1” (SAO_EO_1), the current pixel is compared with the upper and lower neighboring pixels. When the edge is shifted according to SAO classification “2” (SAO_EO_2), the current pixel is compared with the upper left and lower right neighboring pixels. When the edge is shifted according to SAO classification “3” (SAO_EO_3), the current pixel is compared with the lower left and upper right neighboring pixels.

Initially, it is believed that the edge of the current pixel is of type zero. If the value of the current pixel C is equal to the values of both left and right neighboring pixels (1 and 2), then the edge type (EdgeType) remains "zero". If the value of the current pixel is greater than the value of the neighboring pixel 1, then the edge type is increased by one. If the value of the current pixel C is less than the value of the neighboring pixel 1, then the edge type is reduced by one. Similarly, if the current value of pixel C is less than the value of the neighboring pixel C2, then the type of edge is increased by one, and if the value of the current pixel is less than the value of neighboring pixel 2, then the type of edge is reduced by 1.

The current pixel C can essentially have the following types of edges: -1, -1, 0, 1, or 2. The type of edge is -2 if the value of the current pixel C is less than both values of neighboring pixels 1 and 2. The type of edge is -1, if the value of the current pixel C is less than the value of one neighboring pixel, but equal to the value of another neighboring pixel. The edge type is 0 if the value of the current pixel C matches the values of both neighboring pixels, or if the value of the current pixel C is greater than the value of one neighboring pixel, but less than the other neighboring pixel. The edge type is 1 if the value of the current pixel C is greater than the value of one neighboring pixel, but equal to the value of another neighboring pixel. The edge type is 2 if the value of the current pixel C is greater than both neighboring pixel values 1 and 2. For each nonzero value of the edge type, four offset values are determined and reported in the encoded video bitstream for use by the decoder (i.e., coffset _-2 , coffset _-1 , coffset ₁ and coffset ₂ ).

In light of the above description, for each classification of edge offsets, edge type values can be calculated using the following pseudo-code:

EdgeType = 0;

if (C> pixel 1), then EdgeType = EdgeType + 1;

if (C <pixel 1), then EdgeType = EdgeType-1;

if (C> pixel 2), then EdgeType = EdgeType + 1;

if (C <pixel 2), then EdgeType = EdgeType-1

In FIG. 3A is a conceptual diagram showing exemplary bands based on intensity values. Assume, for example, that the range of pixel values is 0-255, although other ranges can also be used. In this example, the maximum value shown in FIG. 3A is 255, and each of the thirty-two bands shown in FIG. 3A, has a range of 8. The leftmost lane will have pixel values 0-7, the next lane will have pixel values 8-15, the next lane will have pixel values 16-23, and so on, until the rightmost lane is reached, which will have pixel values of 248-255. As for the strip offset, the pixels are classified into different bands based on the intensity (that is, to classify the strip offset, the pixels are assigned to one of thirty-two stripes). An offset is added to the pixel, depending on which strip the value of that pixel is in. For example, if a pixel has a value of 19, then its value falls into the third strip with a range of pixel values from 16 to 23. Thus, the offset associated with the third strip will be added to the pixel value 19.

To provide signaling of the offset values associated with each band, the bands can be divided into two or more groups. In some implementations, sixteen bands in the center (bands 8-23) are assigned to one group, and the remaining bands (bands 0-7 and 24-31) are assigned to the second group. For each group of bands 16 define the displacement values (i.e., boffset _0, ..., boffset ₁₅₎ are communicated in a coded video bitstream for use by the video decoder. In some implementations, it can be assumed that all offset values for a group, for example, a second group, are 0, and in this case there is no need to signal offset values for such a group by including the corresponding signaling information in the specified encoded video bitstream.

As shown in FIG. 3A, sixteen middle bands (bands 8-23) form the first group of bands, while eight bands to the left (bands 0-7) form the first portion of the second group of bands, and eight to the right of the bands (bands 24-31) form the second section of the second group of bands. According to current methods, such a grouping of bands is fixed. The present invention describes techniques for adaptively determining band groupings. For example, techniques for determining a center value for a set of pixels based on pixel values are described herein. The center value can be determined, for example, based on a certain center for a group of bands, and the center for a given group of bands can be determined based on the average value for a set of pixels, the median value for a set of pixels, or by determining the pixel that hit the peak of the histogram.

Streaks of pixel values can be divided into groups based on said center value. For example, the center value can be used to determine the center for one group of bands. If we again assume that there are thirty-two bands (bands 0-31) and two groups, then the first group of bands may include eight bands to the right of the center and eight bands to the left of the center. The remaining sixteen bands may belong to the second group. Thus, in contrast to the known methods, according to which one group always consisted of sixteen middle bands, the present invention discloses methods according to which the groupings of the bands can be adaptive based on the center value, which is determined based on the actual pixel values.

In FIG. 3B shows one of many possible groupings that can be used according to the techniques disclosed herein. In the example of FIG. 3B, the center is defined adjacent to lane 18. Accordingly, the first group of lanes includes eight lanes to the left of the center (lanes 11-18) and eight lanes to the right of the center (lanes 19-26). The second group of bands includes the remaining bands, which in this example include bands 0-4 and bands 27-31. In FIG. 3B presents one of many possible scenarios. The main purpose of FIG. 3B is a demonstration that, according to the techniques disclosed in the present invention, the center for determining the first group of bands can be adaptively determined, and it is not necessary that its location shown in FIG. 3A was fixed, or was fixed elsewhere. The techniques for determining the center value and adaptively determining band groupings are explained in more detail below.

In FIG. 4 is a flowchart illustrating an example of a video encoder capable of performing SAO filtering techniques disclosed in the present invention. Video encoder 20 may perform encoding of video blocks with intra and inter prediction in video slices. Intra-prediction coding is based on spatial prediction in order to reduce or eliminate spatial redundancy in a given video frame or video image. Inter prediction coding is based on temporal prediction to reduce or eliminate temporal redundancy in adjacent frames or images of a video sequence. The intra prediction mode (I mode) may refer to any of several spatial compression modes. Inter prediction modes such as unidirectional prediction mode (P mode) or bi-directional prediction (B mode) may refer to any of several compression modes on a temporary basis.

In the example of FIG. 4, video encoder 20 includes a partition module 35, a prediction module 41, a reference image memory 64, an adder 50, a transform module 52, a quantization module 54, and an entropy encoding module 56. The prediction module 41 includes a motion estimation module 42, a motion compensation module 44, and an intra prediction module 46. To restore the video block, video encoder 20 also includes an inverse quantization module 58, an inverse transform module 60, and an adder 62. There is also a deblocking filter 72 for filtering block boundaries to remove “blocking” artifacts from the reconstructed video. As shown in FIG. 4, video encoder 20 also includes additional loop filters including adaptive sampling bias (SAO) filter 74 and adaptive loop filter (ALF) 76. Although the deblocking filter 72, SAO filter 74, and ALF 76 are shown in FIG. 4 as inlet filters, in some configurations a deblocking filter 72, an SAO filter 74, and an ALF 76 can be implemented as post-loop filters.

As shown in FIG. 4, video encoder 20 receives video data, and the splitting module 35 breaks this data into video blocks. This splitting can also include splitting non-slices, tiles (mosaic tiles) or other larger blocks, as well as splitting video blocks, for example, in accordance with the structure of the quad tree, which includes the LCU and CU. Video encoder 20 generally comprises components that encode video blocks in a video slice to be encoded. A slice can be divided into many video blocks (and possibly into sets of video blocks, called “tiles” here). Prediction unit 41 may select one of a plurality of possible coding modes, for example, one of a plurality of intra prediction coding modes, or one of a plurality of inter prediction coding modes, for the current video block based on error results (e.g., coding rate and distortion level ) Prediction module 41 may feed the resulting block encoded with intra- or inter-prediction to adder 50 to form residual block baths and to adder 62 to reconstruct the encoded block for use as a reference image.

The intra prediction unit 46 in the prediction unit 41 may perform encoding of the current video block with intra prediction with respect to one or more neighboring blocks in the same frame or slice as the current block to be encoded to provide spatial compression. Motion estimation module 42 and motion compensation module 44 in prediction module 41 inter-predict the encoding of the current video block with respect to one or more predicted blocks in one or more reference images to provide temporal compression.

Motion estimation module 42 may be configured to determine an inter-prediction mode for a video slice according to a predetermined pattern for the video sequence. A predefined template can designate video slices in the sequence as predicted slices (P slices), bidirectional slices (B slices), or generalized P / B slices (GPB slices). Motion estimation module 42 and motion compensation module 44 can be combined with a high degree of integration, but they are shown here separately for conceptual purposes. The motion estimation performed by the motion estimation module 42 is a process of generating motion vectors during which the motion of the video blocks is evaluated. The motion vector, for example, may indicate the offset of the PU of the video block in the current video frame or image relative to the predicted block in the reference image.

A predicted block is a block that is determined based on the close match of the PU block in the video block to be encoded as a pixel difference, which can be determined using difference indicators such as the sum of the absolute differences (SAD), the sum of the quadratic differences (SSD), or other amount. In some examples, video encoder 20 may calculate values for fractional pixel positions of reference images stored in the reference image memory 64. For example, video encoder 20 may interpolate pixel positions with an accuracy of one quarter, pixel positions with an accuracy of one-eighth, or pixel positions of a reference image with a different fractional accuracy. Thus, the motion estimation module 42 can search for motion relative to integer pixel positions and fractional pixel positions and derive a motion vector accurate to a fraction of a pixel.

Motion estimation module 42 calculates a motion vector for the video block PU in the inter-prediction slice by comparing the position of this PU with the position of the predicted reference image block. The reference image can be selected from the first list of reference images (list 0) or the second list of reference images (list 1), each of which identifies one or more reference images stored in the memory 64 of the reference images. Motion estimation module 42 sends the calculated motion vector to entropy encoding module 56 and motion compensation module 44.

The motion compensation performed by the motion compensation module 44 may comprise extracting or generating a predicted block based on the motion vector determined by the motion estimation, possibly interpolating to within a fraction of a pixel. After obtaining the motion vector for the PU of the current video block, the motion compensation module 44 may determine the location of the predicted block indicated by the motion vector in one of the reference image lists. Video encoder 20 generates a residual video block by subtracting the pixel values of the predicted block from the pixel values of the current encoded video block, generating pixel difference values. The values of the pixel differences form the residual data for the block in question, and they may include difference components of luminance and color. The adder 50 represents that component or components that perform the specified subtraction operation. Motion compensation module 44 may also generate syntax elements associated with video blocks and a video slice for use by video decoder 30 when decoding video blocks of a video slice.

The intra prediction unit 46 may perform intra prediction on the current block as an alternative to the inter prediction performed by the motion estimation module 42 and the motion compensation module 44, as described above. In particular, intra prediction module 46 may determine the intra prediction mode for use in encoding the current block. In some examples, intra prediction module 46 may perform coding of the current block using various intra prediction modes, for example, during separate coding passes, and intra prediction module 46 (or mode selection module 40 in some examples) may select the desired intra mode Predictions from test modes. For example, intra prediction module 46 may calculate values for the “speed-distortion” pair using a single-count analysis of the dependence of distortion on speed for various tested intra-prediction modes and select an intra-prediction mode having the best “speed-distortion” characteristics among tested modes. Analysis of the dependence of distortion on speed generally determines the amount of distortion (or error) between the encoded block and the original unencoded block that was encoded to create this encoded block, as well as the bit rate (i.e., the number of bits) used to create this encoded block. The intra prediction unit 46 can calculate the corresponding ratios based on the values of the above distortions and velocities for the various encoded blocks to determine which intra prediction mode gives the best speed-distortion ratio for a given block.

In any case, after selecting the intra prediction mode for the block, the intra prediction module 46 may provide the entropy encoding module 56 with information indicating the selected intra prediction mode for the block. Entropy encoding unit 56 may encode information indicating the selected intra prediction mode in accordance with the techniques disclosed herein. Video encoder 20 may include configuration data in a transmitted bitstream, which may contain a plurality of tables with intra prediction mode indices and a plurality of tables with modified intra prediction mode indices (also called codeword mapping tables), determining encoding contexts for various blocks and indicating on the most probable intra-prediction mode, a table of intra-prediction mode indexes and a table of modified intra-prediction mode indices for their use For each of the mentioned contexts.

After the predicted block prediction block for the current video block is generated by the inter-prediction module 41, the video encoder 20 generates a residual video block by subtracting the predicted block from the current video block. The residual video data in the residual block may be included in one or more TUs and used in the transform unit 52. The transform unit 52 converts the residual video data to the residual transform coefficients using a transform such as a discrete cosine transform (DCT) or a conceptually similar transform. The transform unit 52 may convert the residual video data from the pixel region to a transform region, for example, a frequency domain.

Transform module 52 may send the resulting transform coefficients to quantization module 54. The quantization module 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can reduce the bit depth associated with some or all of the mentioned coefficients. The degree of quantization can be modified by adjusting the quantization parameter. In some examples, quantization module 54 may then scan a matrix including quantized transform coefficients. Alternatively, scanning may be performed by entropy coding unit 56.

Following quantization, entropy coding unit 56 performs entropy coding of quantized transform coefficients. For example, entropy encoding module 56 may perform context-sensitive adaptive variable-length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntactically oriented adaptive binary arithmetic coding (SBAC), probabilistic entropy coding with interval partitioning (PIPE) or use a different coding methodology or technique. After entropy encoding is performed by module 56, the encoded bitstream can be transmitted to video decoder 10 or archived for future transmission or retrieval by video decoder 30. Entropy encoding module 56 can also perform entropy encoding of motion vectors and other syntax elements for the encoded current video slice.

The inverse quantization unit 58 and the inverse transform unit 60 perform inverse quantization and inverse transform, respectively, to restore the residual block in the pixel region for later use as a reference block of the reference image. Motion compensation module 44 may calculate the reference block by adding a residual block to the predicted block of one of the reference pictures in one of the reference picture lists. Motion compensation module 44 may also apply one or more interpolation filters to the reconstructed residual block to calculate pixel values in fractions of a pixel for use in motion estimation. The adder 62 adds the reconstructed residual block with the predicted block created by the motion compensation module 44 to create a reference block for storing it in the memory 64 of the reference images.

Before storing in memory 64, filtering of the reconstructed residual block with one or more filters may be performed. If necessary, to filter the recovered residual blocks, you can also use the deblocking filter 72 to eliminate blocking artifacts. To smooth the transitions between pixels or otherwise improve the quality of the video, you can also use other loop filters (either at the input of the encoding circuit, or after the encoding circuit). One example of another filter is the SAO filter 74. The motion estimation module 42 and the motion compensation module 44 can use this reference block as a reference block for intra-predicting the block in the next video frame or video image.

The SAO filter 4 can determine the offset values for SAO filtering in such a way as to improve the quality of video coding. Improving the quality of video coding may include, for example, determining bias values that provide closer similarity between the reconstructed image and the original image. Video encoder 20 may, for example, encode video data using multiple passages with different offset values, and select to include in the encoded bitstream those offset values that give the best encoding quality, determined based on calculations of the aforementioned speed-distortion ratio.

In some SAO configurations, the filter 74 may be configured to apply two types of offset (e.g., strip offset and edge offset) as described above. SAO filter 74 also sometimes sometimes may not apply bias at all, which in itself can be considered a third type of bias. The type of offset applied by the SAO filter 74 may be reported to the video decoder, either explicitly or implicitly. By applying edge offset, pixels can be classified based on the edge information in accordance with FIG. 2, and the filter can be determined based on the classification of the edges. When applying SAO band offset, the filter 74 may classify pixels into different bands based on a pixel value, such as an intensity value, with each band having a corresponding offset.

In order to signal and generate offsets for different SAO bands, filter 74 may group these thirty-two bands into two or more groups, as shown as examples in FIGS. 3A and 3B. The SAO filter 74 may, for example, assign each of thirty-two bands to one of two groups (i.e., two groups of sixteen bands). Although here, as a rule, a grouping of bands into two groups of the same size is described, it is also possible to use more than two groups, as well as groups with unequal sizes. The present invention describes methods for determining such groups.

Each bar may span a range of pixel values. The pixel value may be, for example, a pixel intensity value, wherein the pixel value may describe any of the following values: RGB value, brightness value, color value, or any other type of pixel value. The techniques disclosed in this invention can be applied, for example, either to color samples of video data, or samples to brightness of video data, or to both color samples and brightness samples of video data.

In some implementations, the first group of bands can be centered on the center of the range of possible pixel values (i.e., the maximum value of / 2). Thus, for example, if the range of pixel values is 0-255, then the center of the first group of stripes can be between a value of 127 pixels and a value of 128 pixels, the first group covering values from 64 to 191. The first section of the second group can cover pixel values from 0 up to 63, and the second section of the second group can cover the pixel value from 192 to 255. The first group includes sixteen bands, and the second group includes sixteen bands (eight bands in the first part and eight bands in the second part). This grouping example is shown in FIG. 3A. Although the examples disclosed herein generally assume that all thirty-two bands are of the same size, in some implementations, bands of a different size may be used.

According to the SAO techniques disclosed herein, filter 74 may adaptively determine the center of the first group instead of using static banding. The SAO filter 74 may, for example, determine the center of the first group based on the average or median of the pixel values in a certain area of the frame. The SAO filter 74 may also determine the center of the first group based on the pixel at which the histogram peak hits. After the SAO is determined by the filter 74 of the center of the first group, the SAO filter 74 may then determine the second group. Assuming again that thirty-two bands are used, the first group may include about eight bands to the left of the center and about eight bands to the right of the center. The remaining bands may form a second group.

In one example, assuming that the maximum pixel value is 256, then the SAO filter 74 determines the center for the first group of bands based on any of the above techniques in a pixel with a value of 90. In this case, compared with the static grouping shown in FIG. 3A, the first group of strips can be shifted to the left, which leads to a decrease in the first section of the second group, while the second section of the second group increases. However, if the SAO filter 74 determines the center for the first group of bands at a value of 170, then the first group is shifted to the right compared to the static grouping of FIG. 3A, which leads to an increase in the first portion of the second group and a decrease in the second portion of the second group. The SAO filter 74 may determine the center according to any of the techniques described here for determining the mean, median, or peak of the histogram, or it may be determined in some other way.

After determining the center of the SAO, the filter 74 can determine the first group of bands. The SAO filter 74 can determine the first group of bands by identifying eight bands to the left of the center and eight bands to the right of the center. In some cases, a center defined this way may be inside the strip. If we again assume that there are thirty-two equal bands (bands 0-31) and assume that the range of pixel values is 0-255, then strip 12 has a range of values from 96 to 103. If, for example, SAO filter 74 determines the center of value 99, then this center falls into the range of pixels covered by strip 12. In this case, the SAO filter 74 can be configured to always include strip 12 as one of the eight stripes to the left of the center or always include strip 12 as one eight lanes on the right from the center. If the SAO filter 74 identifies band 12 as one of eight bands to the left of the center, then the first group of bands includes bands 5-20, while the second group includes bands 0-4 and 21-31. If the SAO filter 74 identifies band 12 as one of eight bands to the right of the center, then the first group of bands includes bands 6-21, while the second group includes bands 0-5 and bands 22-31. The SAO filter 74 may also be configured to determine whether to include lane 12 with lanes to the left of the center or lanes to the right of the center, depending on whether the center is in that lane. Since 99 is closer to 96 than 103, the SAO filter 74 can be configured to identify band 12 as one of the bands to the left of the center according to the fact that the center is defined at a value of 99. However, when determining the center at a value of 101, which is closer to 103 than to 96, the SAO filter 74 may be configured to identify band 12 as one of the bands to the right of the specified center.

Although the techniques disclosed herein are described by way of example, in which thirty-two bands and two groups of sixteen bands are used, these techniques are generally applicable to SAO schemes in which more or less bands and more or fewer groups are used. As one of many possible examples, the techniques disclosed here can be used to determine the first group of four bands and the second group of twenty-eight bands, and it is assumed here that all offsets for the second group are zero. In another example, sixteen bands, sixty-four bands, or some other number of bands may be used.

The SAO filter 74 may create offsets for the bands based on the specified groupings and report these offsets. For each group of bands (assuming that, as before, there are groups of sixteen bands), sixteen offset values can be determined, which are reported in the encoded bitstream for use by a video decoder. In one example, the number of bits needed to send a group of sixteen band offset values can be reduced by using correlations between the band offset values in a given group effectively. In another example, the strip offset values of a neighboring portion can be used to predict the strip offset values for a group of the current portion. A neighboring portion may be a portion causally determined by the current portion (i.e., this neighboring portion has already been encoded) Examples of neighboring portions include spatially adjacent portions, such as portions located above or to the left of the current parcel, or temporally adjacent portions, such as sections in previous encoded frames (e.g., co-located sections).

The techniques for finding the center can be applied to each LCU or to groups of LCUs. These techniques can, for example, be applied to areas of a frame defined in the same way as the above quad-tree partitioning. A frame can be divided into four areas, with each area being divided into four smaller areas, etc. The SAO filter 74 may determine the center values, as described above, based on all the pixel values in the LCU block group or another area of the frame, or it can determine the center values by pixel downsampling using not all available pixel values in the LCU or LCU group.

The video encoder 20 of FIG. 4 is an example of a video encoder configured to determine a center value for a set of pixels based on the values of these pixels; dividing bands of pixel values into groups based on the center value; and determining the offset values for the bands based on these groups. Video encoder 20 may determine the center for the first set of pixels by determining the average value for the set of pixels, determining the median value for the set of pixels, and / or determining the pixel that has hit the peak of the histogram. Video encoder 20 may determine the offset values for the bands based on these groups by generating the strip offset values for the strip of the first group based on the offset of the second strip of the first group. Video encoder 20 may determine the offset values for the bands based on the indicated groups by generating the strip offset values for the strip from the first group based on the strip offset determined for the adjacent section. Video encoder 20 may also report band offset values for a neighboring portion. Video encoder 20 may also inform the video decoder of the strip offset values based on these groups. In some implementations, video encoder 20 may generate a syntax element for inclusion in an encoded bitstream that indicates a specific center value. In some cases, this syntax element may be an explicit indication of a center defined for a group of bands, but this syntax element need not be an explicit indication of a particular center. For example, the center value can identify the first or last lane in a particular group to allow the video decoder to restore the same groupings of bands determined by the video encoder 20. In this case, these groupings of bands restored by the video decoder can be aligned to the same center as the groups defined by the video encoder, even if the specified center is not explicitly transmitted in the encoded bitstream.

In FIG. 5 is a flowchart showing an example of a video decoder 30 that can implement the described techniques. In the example of FIG. 5, video decoder 30 includes an entropy decoding unit 80, a prediction unit 81, an inverse quantization unit 86, an inverse transform unit 88, an adder 90, and a reference image memory 92. Prediction module 81 includes motion compensation module 82 and intra prediction module 84. In some examples, video decoder 30 may perform a decoding run, which is generally the reverse of the coding pass described with respect to video encoder 20 of FIG. four.

During the decoding process, video decoder 30 receives an encoded video bitstream representing the video blocks of the encoded video slice and the corresponding syntax elements from video encoder 20. Entropy decoding module 80 of video decoder 30 performs entropy decoding of the bit stream to generate quantized coefficients, motion vectors, and other syntax elements. Entropy decoding unit 80 directs these motion vectors and other syntax elements to prediction unit 81. Video decoder 30 may receive syntax elements at the video slice level and / or at the video block level.

When the video slice is encoded using intra prediction (I), the intra prediction module 84 of the prediction module 81 may generate prediction data for a video block of the current video slice based on signaling in the intra prediction mode, as well as based on data from previously decoded blocks of the current frame or image . When the video frame is encoded as an inter-prediction slice (i.e., B, P or GPB), the motion compensation module 82 of the prediction module 81 creates prediction blocks for the video block of the current video slice based on motion vectors and other syntax elements received from the entropy decoding module 80. Prediction blocks can be created from one of the reference images in one of the lists of reference images. Video decoder 30 may construct reference frame lists, list 0 and list 1, using standard construction techniques based on reference images stored in the reference image memory 92.

Motion compensation module 82 determines the prediction information for the video block of the current video slice by parsing motion vectors and other syntax elements and uses the prediction information to create prediction blocks for the current decoded video block. For example, motion compensation module 82 uses some of the syntax elements obtained to determine a prediction mode (e.g., intra or inter prediction) used to encode video blocks of a video slice, such as an inter prediction slice (e.g., B slice, P slice, or GPB slice ), structure information for one or more lists of reference images for a given slice, motion vectors for each video block of the slice with inter-coding, the status of inter-prediction for each video block of the slice, encoded th with inter-prediction, and other information to decode the video blocks in the current videoslayse.

Motion compensation module 82 may also perform interpolation based on interpolation filters. Motion compensation module 82 may use interpolation filters, as is done by video encoder 20 in the process of encoding video blocks to calculate interpolated values for fractional pixels of reference blocks. In this case, motion compensation module 82 may determine the interpolation filters used by video encoder 20 based on the syntax elements obtained, and use these interpolation filters to create prediction blocks.

The inverse quantization module 86 performs inverse quantization, i.e., dequantization of the quantized transform coefficients provided in the bitstream and decoded by the entropy decoding unit 80. The inverse quantization process may include the use of a quantization parameter calculated by video encoder 20 for each video block in the video slice to determine the degree of quantization by analogy with the degree of inverse quantization, which should be applied. The inverse transform unit 88 applies an inverse transform, such as an inverse DCT transform, an inverse integer transform, or a conceptually similar inverse transform process to transform coefficients to provide residual blocks in the pixel domain.

After the motion compensation module 82 has generated a prediction block for the current video block based on motion vectors and other syntax elements, the video decoder 30 forms a decoded video block by summing the residual blocks received from the inverse transform module 88 with the corresponding prediction blocks generated by the motion compensation module 82 . An adder 90 represents a component or components that perform a summing operation. The decoded video blocks formed by the adder 90 can then be filtered using a deblocking filter 93, an SAO filter 94, and an adaptive loop filter 95. The decoded video blocks in this frame or image are then stored in the reference image memory 92, where reference images are used for subsequent compensation movement. The decoded video is also stored in the reference image memory 92 for later presentation on a display device such as a display device 32 of FIG. one.

The SAO filter 94 may be configured to apply the same filtering (e.g., with edge offset and band offset) as the SAO filter 74 discussed above performs. Thus, as part of performing the SAO band offset, the filter 74 can determine the center value for the first group of bands, as described above with reference to FIG. 4. The center value can, for example, be determined on the basis of a metric such as the average pixel value, the median value of the pixel, or you can determine the pixel that hit the histogram peak for an LCU or a group of LCUs.

In some implementations of SAO, the filter 94 may receive, in a bitstream video stream, an indication of the center defined by the video encoder. This indication can be represented as the actual center value calculated by the video encoder, or it can be represented as the identification of the first strip for the first group, the indication of the last strip of the first section of the second group, the average value for the first group, or some other information that allows The SAO filter 94 determines the banding used by the video encoder. Based on the first group and the second group, determined based on the center value, the SAO filter 94 may generate offset values based on information received in the encoded video bitstream. For one of the groups, for example, for the second group, we can assume that all its displacement values are equal to zero.

For the SAU LCU block group, the filter 94 can determine the center value by determining the average pixel value for the LCU block group. The SAO filter 94 can determine the average pixel value by summing the values of all or some of the pixels in the LCU block group and dividing the resulting sum by the number of pixels whose values are included in the specified sum, where the resulting value will represent the average pixel value for the LCU block group. The filter SAO block 94 may then use the obtained average pixel value in the manner described above to determine the banding.

As an alternative to an LCU block group, the SAO filter 94 may determine the center value by determining the median value for the LCU block group. The SAO filter 94 may determine the median value by identifying a value at which half the pixel values of the LCU group of blocks exceeds this value, and half the pixel values for the LCU group of blocks smaller than the specified value. The filter SAO block 94 may then use the found pixel median value in the manner described above to determine the banding.

Alternatively for LCUs, the SAO filter 94 can determine the center value by locating the peak of the histogram for the LCU or group of LCUs. In general, a histogram can be considered a statistical representation of the distribution of pixel values for a group of LCUs. Thus, the SAO filter 94 can determine the peak of the histogram by identifying the pixel values or pixel values that appear most often in the group of LCUs. Then, the SAO filter unit 94 may use the found peak value in the histogram as described above to determine band groups.

You can implement three examples of how you can determine the center value (for example, by the average, median or peak of the histogram) by any of the filters (SAO filter 74 or SAO filter 94) or both filters, and these three possible methods for determining the center value are not the only ones. It is contemplated that the techniques disclosed in the invention, which generally relate to the dynamic determination of band groups for signaling band offset values, may be compatible with other specified techniques for determining the center value.

The video decoder 30 of FIG. 5 is an example of a video decoder configured to determine a center value for a set of pixels based on values of said pixels, divide bands of pixel values into groups based on a specified center value, and determine offset values for said bands on the basis of available groups. In some implementations, video decoder 30 may determine the center value by obtaining a syntax element indicating the center value. As described above, this syntax element may be, but not necessarily, an explicit indication of the center values. In some implementations, video decoder 30 may determine the center value by determining an average value for a set of pixels, determining a median value for a set of pixels, and / or locating a peak in a histogram. Video decoder 30 may determine the offset values for the bands based on the groups by generating the offset values for the strip of the first group based on the offset of the second strip of the first group. Video decoder 30 may determine the offset values for the bands based on the indicated groups by generating the group offset values for the strip from the first group based on the strip offset determined for the adjacent section. Video decoder 30 may also recover the offset values of the bands transmitted from the video encoder based on the groups.

Thus, the video decoder 30 of FIG. 5 is an example of a video decoder configured to determine a center value for a set of pixels based on pixel values for a given group; dividing the bands of pixel values into groups based on the specified center value; and determining bias values for said bands based on the available groups. Video decoder 30 can determine the center value by obtaining a syntax element indicating the center value, determine the center value by determining the average value for the set of pixels, determine the center value by determining the median value for the set of pixels, or by determining the center value based on the location of the histogram peak. The specified set of pixels may, for example, include a group of the largest coding units (LCUs). Video decoder 30 may determine the offset values for the bands based on the groups by generating the offset values for the strip of the first group based on the offset of the second strip of the first group. Video decoder 30 may determine the offset values for the bands based on the indicated groups by generating the group offset values for the strip from the first group based on the strip offset determined for the adjacent section.

In FIG. 6 is a flowchart illustrating SAO filtering techniques disclosed in the present invention. The techniques of FIG. 6 may be performed by a video encoder having a SAO filtering module, such as video encoder 20 or video decoder 30. The techniques of FIG. 6 are described below with reference to video decoder 30, although it should be understood that the techniques of FIG. 6 are applicable to a wide variety of video coding devices and is not limited to video decoders.

Video decoder 30 determines a center value for a set of pixels based on pixel values (610). Video decoder 30 may, for example, determine the center value by finding the average value for the set of pixels, determining the median value for the set of pixels, and determining the pixel that hits the peak of the histogram. Video decoder 30 divides the strip of pixel values into groups based on the center value (620). Video decoder 30 determines the offset values for the bands based on these groups (630). Video decoder 30 may determine the offset values for the bands based on these groups by generating the strip offset values for the strip of the first group based on the offset of the second strip of the first group. Video decoder 30 may also determine the offset values for the bands based on the indicated groups by generating an offset value for the strip of the first group based on the offset of the strip determined for the adjacent section.

In one or more examples, the described functions may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored or transmitted in the form of one or more instructions or code on a computer-readable medium and executed by a processing hardware unit. A computer-readable medium may include a computer-readable storage medium that corresponds to a material medium, such as a storage medium or a transmission medium, including any medium that allows for transferring a computer program from one place to another, for example, according to any communication protocol. In this aspect, a computer-readable medium may generally correspond to (1) a tangible computer-readable storage medium, which is long-term, or (2) a communication medium such as a signal or carrier wave. Storage media for storage of data can be any available storage media that can be accessed by one or more computers or one or more processors to extract instructions, code, and / or data structures to implement the techniques disclosed in this invention. A computer program product may include a computer-readable medium.

By way of example, but not limitation, said computer-readable storage medium may include RAM, ROM, electrically erasable programmable ROM (EEPROM), ROM on a compact disc (CD-ROM), or other storage device on optical disks, a magnetic disk storage device or other magnetic storage devices, flash memory, or any other medium that can be used to store the required program code in the form of instructions or data structures that can be accessed by a computer. Also, any connection will correctly be called a computer-readable storage medium. For example, if commands are transmitted from a Web site, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then a coaxial cable , fiber optic cable, twisted pair, DSL lines, or wireless technologies such as infrared, radio, and microwave, are included in the media definition. However, it should be understood that the terms “computer-readable storage medium” and “storage medium” do not include connections carrying waves, signals, or other temporary media, but refer to a tangible long-term storage medium. As used herein, the term “disc” refers to a compact disc (CD), a laser disc, an optical disc, a digital versatile disc (DVD), a floppy disc, and a Blu-ray disc, the optical discs reproducing data optically using lasers, while how magnetic disks reproduce data in a magnetic way. Combinations of the above should also be included in the concept of a computer-readable storage medium.

Commands may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), user programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Accordingly, the term “processor” as used herein may refer to any of the above structures or any other structure suitable for implementing the above methods. In addition, in some aspects, the functionality described herein may be provided within specialized hardware and / or software modules configured for encoding and decoding, or included in a combination codec. Also, the techniques disclosed herein may be fully implemented in one or more circuits or logic elements.

The techniques disclosed herein may be implemented in a wide variety of devices or installations, including a cordless handset, integrated circuit (IC), or a set of integrated circuits (e.g., microprocessor set). Various components, modules or blocks are described herein in such a way as to emphasize the functional aspects of devices configured to perform the techniques disclosed herein, and this does not require their implementation using various hardware blocks. Rather, on the contrary, as discussed above, the various units can be combined in a hardware codec unit or provided as a set of interacting hardware units, including one or more processors, as described above, in combination with suitable software and / or firmware.

Various examples have been described here. These and other examples are included in the scope of the following claims.

Claims (37)

1. A method of encoding video data, the method comprising:
determining a center value for a set of pixels, wherein the center value is based on the values of said pixels;
dividing the stripes of pixel values into two or more groups of stripes based on said center value, wherein each strip of stripes of pixel values contains a range of pixel values, and wherein each group of two or more groups of stripes contains one or more stripes;
determining offset values for said bands based on said two or more groups of bands. 2. The method according to claim 1, wherein the method is implemented by a video decoder, and moreover, determining the center value comprises receiving a syntax element, and furthermore dividing the pixel value bands into two or more groups of strips comprises determining said two or more groups of strips based on said syntax element, this method further comprises applying an adaptive sampling bias (SAO) filter based on said bias values. 3. The method according to claim 1, wherein the method is implemented by a video encoder, and wherein the method further comprises:
generating a syntax element for inclusion in the encoded bitstream, wherein the syntax element indicates the center value. 4. The method of claim 1, wherein determining the center value comprises determining an average value for said set of pixels. 5. The method of claim 1, wherein determining the center value comprises determining a median value for said set of pixels. 6. The method according to claim 1, wherein determining the center value comprises determining a pixel onto which the peak of the histogram falls. 7. The method of claim 1, wherein said set of pixels comprises a group of largest coding units (LCUs). 8. The method of claim 1, wherein determining the offset values for said bands based on said two or more groups of bands comprises generating an offset value for a strip from the first group based on the offset of the second strip from said first group. 9. The method of claim 1, wherein determining the offset values for said bands based on said two or more groups of bands comprises generating an offset value for a strip from the first group based on a strip offset determined for a neighboring portion. 10. A video coding device, wherein the video coding device comprises:
a video encoder configured to determine a center value for a set of pixels, wherein the center value is based on the values of said pixels; dividing the bands of pixel values into two or more groups of bands based on said center value, each strip of said pixel value bands containing a range of pixel values, and wherein
each group of said two or more groups of bands contains one or more bands; and determining bias values for said bands based on said two or groups of bands. 11. The video coding apparatus of claim 10, wherein the video encoder comprises a video decoder, and in which the video encoder is configured to determine a center value by receiving a syntax element;
dividing said bands of pixel values into two or more groups of bands by determining two or more groups of bands based on said syntax element; and
applying an adaptive sampling bias (SAO) filter based on said bias values. 12. The video coding apparatus according to claim 10, wherein the video encoder comprises a video encoder, and in which the video encoder is further configured to form a syntax element for inclusion in the encoded bitstream, the syntax element indicating said center value. 13. The video coding apparatus of claim 10, wherein the video encoder is configured to determine a center value based on an average value for said set of pixels. 14. The video coding apparatus of claim 10, wherein the video encoder is configured to determine a center value based on a median value for said set of pixels. 15. The video coding apparatus according to claim 10, wherein the video encoder is configured to determine a center value based on a pixel to which a histogram peak hits. 16. The video coding apparatus of claim 10, wherein said set of pixels comprises a group of the largest coding units (LCUs). 17. The video coding apparatus of claim 10, wherein the video encoder is configured to determine offset values for said bands based on said two or more groups of bands by generating an offset value for a strip from the first group based on an offset of the second strip from said first group. 18. The video coding apparatus of claim 10, wherein the video encoder is configured to determine offset values for said bands based on said two or more groups of bands by generating an offset value for a strip from the first group based on a strip offset determined for a neighboring portion. 19. The video coding device according to claim 10, wherein the video coding device comprises at least one of:
integrated circuit;
microprocessor; and
a wireless device that includes a video encoder. 20. A computer-readable storage medium that stores instructions executed with the ability to initiate execution by one or more processors:
determining a center value for a set of pixels, the center value being based on the values of said pixels;
dividing the stripes of pixel values into two or more groups of stripes based on said center value, wherein each strip of said stripes of pixel values contains a range of pixel values, and wherein each group of said two or more groups of stripes contains one or more stripes;
determining offset values for said bands based on said two or more groups of bands. 21. The computer-readable storage medium of claim 20, wherein said one or more processors comprise a video decoder, and in which said commands initiate execution by one or more processors:
determining the center value by receiving a syntax element;
dividing the bands of pixel values into two or more groups of bands by determining two or more groups of bands based on said syntax element; and
applying an adaptive sampling bias (SAO) filter based on said bias values. 22. A computer-readable storage medium according to claim 20, wherein one or more processors comprise a video decoder, and wherein the computer-readable storage medium stores additional instructions that initiate the formation by one or more processors of a syntax element for its inclusion in the encoded bitstream, the syntax being the element indicates the center value mentioned. 23. The computer-readable storage medium according to claim 20, wherein said commands initiate the determination by one or more processors of the center value by determining an average value for said set of pixels. 24. The computer-readable storage medium of claim 20, wherein said instructions initiate the determination by one or more processors of a center value by determining a median value for said set of pixels. 25. The computer-readable storage medium according to claim 20, wherein said commands initiate the determination by one or more processors of the center value by determining the pixel to which the histogram peak hits. 26. The computer-readable storage medium of claim 20, wherein said set of pixels comprises a group of largest coding units (LCUs). 27. The computer-readable storage medium according to claim 20, wherein said instructions initiate the determination by one or more processors of the offset values for said bands based on said two or more groups of bands by generating an offset value for a strip from the first group based on the offset of the second strip from mentioned first group. 28. The computer-readable storage medium according to claim 20, wherein said instructions initiate the determination by one or more processors of the offset values for said bands based on said two or more groups of bands by generating an offset value for a strip from the first group based on the strip offset determined for the neighboring plot. 29. A device for encoding video data, the device comprising:
means for determining a center value for a set of pixels, wherein the center value is based on the values of said pixels;
means for dividing the pixel value strips into two or more groups of strips based on said center value, wherein each strip of said pixel value strips contains a range of pixel values, and each group of said two or more stripe groups contains one or more strips; and
means for determining offset values for said bands based on said two or more groups of bands. 30. The device according to claim 29, wherein the device comprises a video decoder, and wherein the means for determining the center value comprises means for receiving a syntax element, and means for dividing the bands of pixel values into two or more groups of bands contains means for determining two or more groups of bands into the basis of said syntax element; and wherein the device further comprises means for applying an adaptive sampling bias (SAO) filter based on said bias values. 31. The device according to p. 29, wherein the device comprises a video encoder, and wherein the device further comprises:
means for generating a syntax element for incorporating it into the encoded bitstream, the syntax element indicating said center value. 32. The apparatus of claim 29, wherein the means for determining the center value comprises means for determining an average value for said set of pixels. 33. The device according to p. 29, in which the means for determining the value of the center contains means for determining the median value for said set of pixels. 34. The device according to p. 29, in which the means for determining the value of the center contains means for determining the pixel on which the peak of the histogram. 35. The device according to p. 29, in which said set of pixels contains a group of the largest coding units (LCUs). 36. The apparatus of claim 29, wherein the means for determining the offset values for said bands based on said two or more groups of bands comprises means for generating an offset value for a strip from the first group based on the offset of the second strip from said first group. 37. The apparatus of claim 29, wherein the means for determining the offset values for said bands based on said two or more groups of bands comprises means for generating an offset value for a strip from the first group based on a strip offset determined for a neighboring portion.

Images