JP6744505B2 - Coding video syntax elements using context trees - Google Patents

Description

Digital video streams may represent video using a series of frames or still images. Digital video may be used in a variety of applications including, for example, video conferencing, high definition video entertainment, video advertising, or sharing user-generated video. Digital video streams contain large amounts of data and can consume large amounts of computing or communication resources of a computing device for processing, transmitting, or storing video data. Various approaches have been proposed to reduce the amount of data in video streams, including compression and other coding techniques.

A method of encoding a current block of a video frame according to one implementation of the present disclosure includes a first set of syntax elements associated with a previously encoded block of a video frame, and a first set of syntax elements. Includes identifying first contextual information used to encode the element. The method further includes generating a context tree by separating the first set of syntax elements into data groups based on the first context information. The context tree includes multiple nodes that represent a data group. The plurality of nodes is associated with cost savings for the first set of syntax elements. Separating the first set of syntax elements includes applying a separation criterion to the values of the first context information to generate at least some of the plurality of nodes. The method further includes identifying a second set of syntax elements associated with the current block and second context information associated with the second set of syntax elements. The method further comprises and identifies one of the plurality of nodes based on a value of second context information associated with a syntax element of one of the second set of syntax elements. Each node represents a data group including the one syntax element. The method further includes encoding the one syntax element according to a probabilistic model associated with the identified node.

A method of decoding a coded block of a coded video frame according to an implementation of the disclosure includes a first set of syntax associated with a previously decoded block of a coded video frame. Includes identifying an element and first contextual information used to decode the first set of syntax elements. The method further includes generating a context tree by separating the first set of syntax elements into data groups based on the first context information. The context tree includes multiple nodes that represent a data group. The plurality of nodes is associated with cost savings for the first set of syntax elements. Separating the first set of syntax elements includes applying a separation criterion to the values of the first context information to generate at least some of the plurality of nodes. The method further includes identifying a second set of syntax elements associated with the encoded block and second context information associated with the second set of syntax elements. The method further comprises and identifies one of the plurality of nodes based on a value of second context information associated with a syntax element of one of the second set of syntax elements. Each node represents a data group including the one syntax element. The method further includes decoding the one syntax element according to a probabilistic model associated with the identified node.

An apparatus for decoding an encoded block of an encoded video frame, according to an implementation of the disclosure, comprises a processor configured to execute instructions stored on a non-transitory storage medium. .. The instructions include a first set of syntax elements associated with a previously decoded block of an encoded video frame, and a first context used to decode the first set of syntax elements. Contains instructions that identify information. The instructions further include instructions for generating a context tree by separating the first set of syntax elements into data groups based on the first context information. The context tree includes multiple nodes that represent a data group. The plurality of nodes is associated with cost savings for the first set of syntax elements. The instructions that separate the first set of syntax elements include instructions that apply a separation criterion to a value of the first context information to generate at least some of the plurality of nodes. The instructions further include instructions identifying a second set of syntax elements associated with the encoded block and second context information associated with the second set of syntax elements. The instructions further include instructions for identifying one of the nodes based on a value of second context information associated with a syntax element of one of the second set of syntax elements. The identified node represents a data group that contains one syntax element. The instructions further include instructions for decoding one syntax element according to a probabilistic model associated with the identified node.

An apparatus for decoding a current block of a video frame, according to one implementation of the disclosure, comprises a processor configured to execute instructions stored on a non-transitory storage medium. The instructions identify a first set of syntax elements associated with a previously coded block of a video frame, and first context information used to code the first set of syntax elements. Including instructions. The instructions further include instructions for generating a context tree by separating the first set of syntax elements into data groups based on the first context information. The context tree includes multiple nodes that represent a data group. The plurality of nodes is associated with cost savings for the first set of syntax elements. The instructions that separate the first set of syntax elements include instructions that apply a separation criterion to a value of the first context information to generate at least some of the plurality of nodes. The instructions further include instructions that identify a second set of syntax elements associated with the current block and second context information associated with the second set of syntax elements. The instructions further include instructions for identifying one of the nodes based on a value of second context information associated with a syntax element of one of the second set of syntax elements, the identified node Represents a data group containing the one syntax element. The instructions further include instructions encoding the one syntax element according to a probabilistic model associated with the identified node.

The cost reduction may be a cost reduction for coding the first set of syntax elements. This cost can be a computational cost. Computational cost may indicate, for example, the amount of computational resources used for coding. The cost to code a syntax element is the time it takes to code the syntax element, the space used to code the syntax element (eg, the size of computer memory), and the cost to code the syntax element. It may depend on or indicate one or more parameters selected from the group of parameters consisting of the number of operations (eg arithmetic operations) and the number of syntax elements to code.

These and other aspects of the disclosure are disclosed in the following detailed description, the appended claims and the accompanying drawings.

The description herein refers to the accompanying drawings, which are described below, and like reference numerals refer to like features throughout the several views.
FIG. 3 is a schematic diagram of a video encoding and decoding system. FIG. 6 is a block diagram of an example of a computing device that may implement a transmitter station or a receiver station. FIG. 3 is a diagram of an exemplary video stream that is encoded and then decoded. FIG. 3 is a block diagram of an encoder according to an implementation of the present disclosure. FIG. 6 is a block diagram of a decoder according to an implementation of the present disclosure. FIG. 6 is a flow chart diagram of a technique for coding syntax elements associated with blocks of a video frame. FIG. 6 is a flow chart diagram of a technique for generating a context tree for coding syntax elements associated with blocks of a video frame. FIG. 6 is a block diagram of a system for encoding syntax elements associated with a current block of a video frame. FIG. 7 is a block diagram of a system for decoding encoded syntax elements associated with encoded blocks of an encoded video frame. 10A shows an example of a first step for generating a context tree, FIG. 10B shows an example of a second step for generating a context tree, and FIG. 10C shows a first step for generating a context tree. FIG. 10D is a diagram showing an example of the fourth stage for generating the context tree, showing an example of the three stages. It is a figure which shows an example of a context tree.

Video compression schemes use bits that are encoded using techniques that divide each image or frame into smaller parts, such as blocks, that limit the encoding of syntax elements associated with each of these blocks. It may include generating a stream. The encoded bitstream can be decoded to regenerate the source image from the encoded syntax elements. For example, the video compression scheme may include transforming the prediction residual of the current block of the video stream into transform coefficients of the transform block. The transform coefficients are quantized and entropy coded into the coded bitstream. The decoder decodes or decompresses the encoded bitstream with the encoded transform coefficients, preparing the video stream for viewing or further processing. Syntax elements are elements of data that represent all or part of a video sequence to be encoded or decoded. For example, the syntax element may be a transform coefficient of a transform block, a motion vector used to generate a prediction residual, a value of a flag in a frame header, or other data associated with a video sequence.

The context information may be used in entropy coding and entropy decoding of syntax elements. Examples of the context information may refer to a luminance plane, a chrominance plane, a neighboring coefficient, a coefficient position, a transform size, or a combination thereof. The value of context information may refer to how the context information is used to encode or decode syntax elements. The encoder or decoder can predict the probability distribution of syntax elements based on the context information. That is, the contextual information is what the syntax element used to be for a block similar to the current block to be encoded or decoded (eg, based on proximity within a video frame, block size, etc.). It may be associated with a probabilistic model that indicates whether it was coded. By coding the syntax elements with the appropriate probabilistic model, the encoder or decoder can encode or decode the syntax elements with fewer bits, respectively.

However, determining the appropriate probabilistic model to use can be difficult when there are multiple syntax elements associated with the block to be encoded or decoded. One solution may include segregating the syntax elements associated with blocks into different groups based on all combinations of contextual information. Each group includes multiple syntax elements associated with one combination of contextual information. The syntax elements can then be coded with a probabilistic model associated with the group containing the maximum number of syntax elements.

However, this solution may have drawbacks. For example, as the number of contextual information available increases, the number of groups may grow exponentially. This can undesirably increase the cost of the entropy coding process. In another example, there may be multiple groups, but each group may contain only a few syntax elements. This excessive separation results in an inaccurate stochastic model estimate for the syntax elements, which causes the encoder or decoder to code the syntax elements with a suboptimal or improper stochastic model, respectively. Or it may be decrypted.

Implementations of the present disclosure include coding syntax elements associated with blocks of video frames using a context tree. A syntax element associated with a block of a video frame is a syntax element contained in the block, a syntax element contained in the header of the video frame containing the block, or otherwise related to the content or coding of the block, or It may be a syntax element used for that purpose. Context information for coding a previously coded syntax element is identified and a context tree is generated by separating the previously coded syntax element into data groups based on the context information. The context tree represents a group of data and contains nodes associated with cost savings for previously coded syntax elements. After the context tree has been generated using previously coded syntax elements and their associated context information, another set of syntax elements and associated context information may be identified. For example, the previously coded syntax element may be the syntax element associated with the first block of the video frame and the other set of syntax elements associated with the second block of the video frame. It may be a syntax element. A syntax element of one of the other sets of syntax elements identifies one of a plurality of nodes of a context tree, such as based on a value of context information associated with that one syntax element. Can be coded by The one syntax element may be coded according to a probabilistic model associated with the identified node. Therefore, the context tree can be used to process a further set of syntax elements, which lowers the cost of entropy coding or entropy decoding.

Further details of techniques for coding syntax elements with context trees are described herein with initial reference to the system in which they may be implemented. FIG. 1 is a schematic diagram of a video encoding and decoding system 100. The transmitting station 102 can be, for example, a computer having an internal configuration of hardware as shown in FIG. However, other implementations of transmitting station 102 are possible. For example, the processing of the transmitting station 102 can be distributed to a plurality of devices.

Network 104 may connect transmitting station 102 and receiving station 106 for encoding and decoding of video streams. Specifically, the video stream can be encoded at the transmitting station 102 and the encoded video stream can be decoded at the receiving station 106. The network 104 may be the Internet, for example. The network 104 may be a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), a mobile phone network, or any station that transfers a video stream from a transmitting station 102 to a receiving station 106 in this example. Other means are possible.

Receiving station 106 may, in one example, be a computer having an internal hardware configuration as described in FIG. However, other suitable implementations of receiving station 106 are possible. For example, the processing of the receiving station 106 can be distributed to a plurality of devices.

Other implementations of the video encoding and decoding system 100 are possible. For example, the implementation may omit the network 104. In another implementation, the video stream may be encoded and stored for later transmission to the receiving station 106 or any other device having memory. In one implementation, receiving station 106 receives the encoded video stream (eg, via network 104, a computer bus, and/or some communication path) and stores the video stream for later decoding. To do. In one implementation, a real-time transport protocol (RTP) is used to transmit the encoded video over the network 104. In another implementation, a transfer protocol other than RTP may be used, such as, for example, a Hypertext Transfer Protocol (HTTP) based video streaming protocol.

When used in a video conferencing system, for example, transmitting station 102 and/or receiving station 106 may include the ability to encode and decode video streams, as described below. For example, receiving station 106 receives the encoded video bitstream from the video conferencing server (eg, transmitting station 102) for decoding and viewing, and also its own video bitstream for decoding and decoding by other participants. It may be a video conference participant who encodes for viewing and sends to the video conference server.

FIG. 2 is a block diagram of an example of a computing device 200 that may implement a transmitter station or a receiver station. For example, computing device 200 may implement one or both of transmitting station 102 and receiving station 106 of FIG. The computing device 200 may be in the form of a computing system that includes multiple computing devices, or in the form of one computing device such as, for example, a mobile phone, tablet computer, laptop computer, notebook computer, desktop computer. You can

The processor 202 of the computing device 200 can be a conventional central processing unit. Alternatively, processor 202 may be any other type of device or devices capable of manipulating or processing information that currently exists or will be developed in the future. For example, the disclosed implementations may be implemented in a single processor as shown (eg, processor 202), but the use of multiple processors may provide advantages in speed and efficiency.

The memory 204 in the computing device 200 may be a read only memory (ROM) device or a random access memory (RAM) device in one implementation. However, any other suitable type of storage device can be used as the memory 204. The memory 204 may include code and data 206 that the processor 202 accesses using the bus 212. The memory 204 can further include an operating system 208 and application programs 210, which include at least one program that enables the processor 202 to perform the techniques described herein. For example, application program 210 can include applications 1-N, which further include video coding applications that perform the techniques described herein. Computing device 200 may also include secondary storage 214, which may be, for example, a memory card used with mobile computing devices. Since video communication sessions may include a significant amount of information, they may be stored wholly or partially in secondary storage 214 and loaded into memory 204 as needed for processing.

Computing device 200 may also include one or more output devices such as display 218. The display 218 may, in one example, be a touch sensitive display that combines the display with touch sensitive elements operable to sense touch input. The display 218 may be connected to the processor 202 via the bus 212. Other output devices that allow a user to program or otherwise use computing device 200 may be provided in addition to or in place of display 218. When the output device is or includes a display, the display may be a liquid crystal display (LCD), a cathode ray tube (CRT) display, or a light emitting diode (LED) display such as an organic LED (OLED) display. Can be implemented in.

The computing device 200 may detect an image, such as an image of a user operating the computing device 200, such as an imaging device 220, such as a camera, or any other imaging device 220 currently or in the future developed. It may include or be in communication with the imaging device 220. The imaging device 220 may be arranged so that it is oriented towards the user operating the computing device 200. In one example, the position and optical axis of the imaging device 220 can be configured such that the field of view includes an area directly adjacent to the display 218 from which the display 218 is visible.

The computing device 200 includes, for example, a voice sensing device 222, such as a microphone, or any other voice sensing device now or hereafter developed capable of sensing sound near the computing device 200, or You can communicate with it. The voice sensing device 222 is directed toward a user operating the computing device 200 and is adapted to receive, for example, voice, other utterances made by the user while the user is operating the computing device 200. Can be configured to.

2 illustrates that the processor 202 and memory 204 of computing device 200 are integrated into one unit, other configurations may be utilized. The operation of processor 202 may be distributed across multiple machines (individual machines may have one or more processors) that may be connected directly or via a local area network or other network. The memory 204 may be distributed across multiple machines, such as network-based memory, or may be memory in multiple machines performing operations of the computing device 200. Although shown herein as a single bus, the bus 212 of the computing device 200 may be composed of multiple buses. Further, the secondary storage 214 may be directly connected to other components of the computing device 200, or accessed via a network, and may be an integrated unit such as a memory card or multiple memory cards. It can include multiple units. Accordingly, computing device 200 may be implemented in a wide variety of configurations.

FIG. 3 is a diagram of an example of a video stream 300 that is encoded and subsequently decoded. Video stream 300 includes video sequence 302. At the next level, the video sequence 302 includes multiple adjacent frames 304. Although three frames are shown as contiguous frames 304, video sequence 302 may include any number of contiguous frames 304. Adjacent frames 304 can be further subdivided into individual frames, such as frame 306. At the next level, frame 306 may be divided into a series of planes or segments 308. The segment 308 can be, for example, a subset of frames that allows parallel processing. The segment 308 can be a subset of frames that can separate the video data into different colors. For example, frame 306 of color video data may include a luma plane and two chromaticity planes. The segment 308 can be sampled at different resolutions.

Whether or not frame 306 is divided into segments 308, frame 306 may be further subdivided into blocks 310 that may include data corresponding to, for example, 16×16 pixels in frame 306. Block 310 may be configured to include data from the pixel data of one or more segments 308. Block 310 may be 4x4 pixels, 8x8 pixels, 16x8 pixels, 8x16 pixels, 16x16 pixels, or any other suitable size such as larger. Unless otherwise stated, the terms block and macroblock are used interchangeably herein.

FIG. 4 is a block diagram of an encoder 400 according to implementations of the present disclosure. Encoder 400 may be implemented within transmitting station 102 as described above, such as by providing a computer software program stored in a memory such as memory 204. The computer software program may include machine instructions that, when executed by a processor such as the CPU 202, cause the transmitting station 102 to encode video data in the manner described in FIG. The encoder 400 may be implemented as dedicated hardware included in the transmitting station 102, for example. In one particularly desirable implementation, encoder 400 is a hardware encoder.

The encoder 400 uses the video stream 300 as an input to perform various functions in the forward path (indicated by solid connecting lines) to produce an intra-predicted/inter-predicted coded bitstream 420. It has a stage 402, a transform stage 404, a quantization stage 406, and an entropy coding stage 408. The encoder 400 also includes a reconstruction path (indicated by a dashed connecting line) that reconstructs a frame for encoding of future blocks. In FIG. 4, the encoder 400 has the following stages to perform various functions in the reconstruction pass: an inverse quantization stage 410, an inverse transform stage 412, a reconstruction stage 414, and a loop filtering stage 416. The video stream 300 may be encoded using other structural variations of the encoder 400.

When the video stream 300 is presented for encoding, each of the adjacent frames 304, such as frame 306, may be processed in blocks. In intra-prediction/inter-prediction stage 402, each block may be encoded using intra-frame prediction (also called intra-prediction) or inter-frame prediction (also called inter-prediction). In either case, predictive blocks may be formed. For intra prediction, the predictive block may be formed from previously coded and reconstructed samples in the current frame. For inter prediction, the prediction block may be formed from the samples in one or more previously constructed reference frames.

Continuing to refer to FIG. 4, the prediction block is then subtracted from the current block in the intra-prediction/inter-prediction stage 402 to produce a residual block (also called residual). The transform stage 404 transforms the residual into, for example, transform coefficients in the frequency domain using block-based transforms. The quantization stage 406 transforms the transform coefficient into a discrete quantum value called a quantized transform coefficient using the quantized value or the quantized level. For example, the transform coefficient may be divided by the quantized value and truncated.

The quantized transform coefficients are then entropy coded by entropy coding stage 408. For example, entropy encoding stage 408 identifies the context information for encoding syntax elements associated with the current block, and separates the context elements into data groups based on the context information to construct a context tree. Producing may be included. Implementations for identifying context information and generating context trees are described below with respect to Figures 6-11. The entropy coded coefficients are for decoding blocks (eg, may include syntax elements such as those used to indicate the type of prediction used, transform type, motion vector, and quantized value, etc.). Output along with other information used for the compressed bitstream 420. The compressed bitstream 420 may be formatted using various techniques such as variable length coding (VLC) or arithmetic coding. Compressed bitstream 420 is also referred to as an encoded video stream or an encoded video bitstream, and these terms are used interchangeably herein.

To ensure that encoder 400 and decoder 500 (discussed below) use the same reference frames to decode compressed bitstream 420, the reconstruction pass ( (Represented by dotted connecting lines) is used. The reconstruction pass inversely quantizes the transform coefficient quantized in the inverse quantization stage 410, and inversely transforms the inverse quantized transform coefficient in the inverse transform stage 412 to obtain a differential residual block (differential residual difference). Perform a function similar to that performed during the decoding process (discussed below), including generating a. In the reconstruction stage 414, the prediction block predicted in the intra prediction/inter prediction stage 402 is added to the differential residual to create a reconstructed block. Loop filtering stage 416 may be applied to the reconstructed block to reduce distortions such as blocking artifacts.

Other variations of encoder 400 can be used to encode compressed bitstream 420. In some implementations, a non-transform based encoder may directly quantize the residual signal without using the transform stage 404 for certain blocks or frames. In some implementations, the encoder may have a quantization stage 406 and an inverse quantization stage 410 combined in a common stage.

FIG. 5 is a block diagram of a decoder 500 according to an implementation of the present disclosure. Decoder 500 may be implemented at receiving station 106, for example, by providing a computer software program stored in memory 204. The computer software program may include machine instructions that, when executed by a processor, such as processor 202, cause receiving station 106 to decode video data in the manner described in FIG. Decoder 500 may be implemented in hardware included in transmitting station 102 or receiving station 106, for example.

Decoder 500, in one example, performs the various functions, similar to the reconstruction pass of encoder 400 described above, to generate the output video stream 516 from compressed bitstream 420, the following stages: That is, it includes an entropy decoding stage 502, an inverse quantization stage 504, an inverse transform stage 506, an intra prediction/inter prediction stage 508, a reconstruction stage 510, a loop filtering stage 512, and a deblocking filtering stage 514. Other structural variations of the decoder 500 may be used to decode the compressed bitstream 420.

When the compressed bitstream 420 is presented for decoding, the data elements in the compressed bitstream 420 are decoded by the entropy decoding stage 502 to produce a set of quantized transform coefficients. It For example, entropy decoding stage 502 identifies context information for decoding the encoded syntax elements associated with the encoded block, the encoded syntax elements based on the context information. It may include generating a context tree by separating the data groups. Implementations for identifying context information and generating context trees are described below with respect to Figures 6-11.

The dequantization stage 504 dequantizes the quantized transform coefficient (eg, by multiplying the quantized transform coefficient by a quantized value), and the inverse transform stage 506 degenerates the dequantized transform coefficient. Is inverse transformed to produce a differential residual that is the same as that produced by the inverse transform stage 412 in encoder 400. Using the decoded header information from the compressed bitstream 420, the decoder 500 uses the intra-prediction/inter-prediction stage 508 to generate the encoder 400, eg, in the intra-prediction/inter-prediction stage 402. Create the same prediction block that was done.

At the reconstruction stage 510, the prediction block is added to the differential residual to create a reconstruction block. Loop filtering stage 512 may be applied to the reconstructed block to reduce blocking artifacts. Other filtering can be applied to the reconstructed block. In this example, deblocking filtering stage 514 is applied to the reconstructed block to reduce blocking distortion and the result is output as output video stream 516. Output video stream 516 is also referred to as a decoded video stream, and the terms are used interchangeably herein. Other variations of the decoder 500 can be used to decode the compressed bitstream 420. In some implementations, the decoder 500 may generate the output video stream 516 without the deblocking filtering stage 514.

Next, a technique for generating a context tree and coding syntax elements using the context tree will be described with reference to FIGS. 6 to 7. FIG. 6 is a flowchart of a technique 600 for coding syntax elements associated with blocks of video frames using a context tree. FIG. 7 is a flowchart of a technique 700 for generating a context tree for coding syntax elements associated with blocks of video frames. One or both of techniques 600 and 700 may be implemented, for example, as a software program executed by a computing device such as transmitting station 102 or receiving station 106. For example, the software program may include machine-readable instructions stored in memory, such as memory 204 or secondary storage 214, that, when executed by a processor, such as processor 202, cause a computing device to perform technology 600 and/or technology 700. You can One or both of technology 600 or technology 700 may be implemented using dedicated hardware or firmware. As mentioned above, some computing devices may have multiple memories or processors, and the operations described in one or both of techniques 600 or 700 may result in multiple processors, memories, or It may be dispersed using both.

One or both of technique 600 or technique 700 may be performed by an encoder, eg, encoder 400 shown in FIG. 4, or a decoder, eg, decoder 500 shown in FIG. Accordingly, references within the following description of technique 600 and technique 700 refer to encoding the current block or decoding the encoded block, or encoding the current block or decoding the encoded block. Discussion may be included regarding the generation of the context tree used. All or a portion of technique 600 or technique 700 may be used to encode the current block or to decode the encoded block, such as "encode the current block", or References such as "decode an encoded block" can refer to applicable acts. For example, if technique 600 or technique 700 is used as part of a process for encoding a current block, references to “decode encoded block” and the like may be ignored. Similarly, if technique 600 or technique 700 is used as part of a process for decoding an encoded block, references to “encode the current block” and the like may be ignored.

Referring first to FIG. 6, a flowchart of a technique 600 for coding syntax elements associated with blocks of video frames using a context tree is shown. At 602, contextual information for coding a first set of syntax elements associated with a previously coded block of a video frame is identified. Identifying the context information may include determining possible values for the context information that may be used to encode or decode the first set of syntax elements. During the coding process, an encoder (eg, encoder 400 shown in FIG. 4) or a decoder (eg, decoder 500 shown in FIG. 5) may use a context within a defined set of context information. It can be configured to use the value of the information. The values of context information for each set can reflect different probabilities for coding syntax elements.

For example, there may be N sets of contextual information available for use by the encoder or decoder. The N sets of context information may be stored in a context buffer available to the encoder or decoder. The values of this set are default probabilities for coding syntax elements (eg, second set of syntax elements associated with blocks of a video frame to be coded), syntax elements (eg, first set). (Syntax element of ), the probability determined based on the previous coding, etc. can be reflected. Each set of contextual information may include different values for that set of contextual information. The value of the first context information of the first set may not be the same as the value of the same first context information of the second set.

Identifying context information may include generating, receiving, or otherwise identifying a context vector. The context vector contains some or all of the possible values of context information for coding syntax elements. The context vector may be represented as the variable ctx. The context vector index ctx[N] may refer to one possible value of the sets of context information. For example, the value stored in ctx[0] may refer to different values of the first context information. The context information contained in the context vector may be stored in the cache.

For example, generating a context vector includes defining a data structure (eg, array, object series, etc.) and storing values of context information from different sets of context information within an index of the data structure. obtain. In another example, identifying the context vector obtains a predefined data structure from a database or similar data store accessible by the encoder or decoder (eg, a cache where context information is stored). May be included. In yet another example, receiving the context vector may include receiving data indicating the context vector from a software or hardware component. For example, the server device may include software for sending data to the encoder or decoder. The data may include a context vector, or data that can be used to identify the context vector, such as in a database or similar data store.

At 604, a context tree is generated. The context tree may be a binary tree or a non-binary tree that includes nodes that represent the data used to code the first set of syntax elements. The nodes of the context tree can be leaf nodes or non-leaf nodes. Leaf nodes are nodes that represent a set or subset of a first set of syntax elements, which set or subset is referred to herein as a data group. A non-leaf node is a node that represents an expression used to separate data groups, which expression is referred to herein as a separation criterion. For example, if the context tree is a binary tree, non-leaf nodes are either both leaf nodes, both non-leaf nodes, or two child nodes containing one leaf node and one non-leaf node. It is a parent node that can have.

Generating the context tree may include separating the first set of syntax elements into data groups based on the identified context information (eg, the value of the set of context information in the context vector). A node of the context tree is created to represent the data group of the first set of syntax elements. Separating the first set of syntax elements may include applying separation criteria to the values of the context information to generate some of the plurality of nodes. The separation criteria that can be used to generate the context tree can be defined in a list, database, or other data store accessible to the encoder or decoder.

As explained below, the separation criterion is applied to the values of the context information, and in response to determining that using the separation criterion results in the greatest cost savings for the first set of syntax elements, 1 One or more nodes can be created. For example, candidate cost savings may be determined based on applying different separation criteria to different values of contextual information. The separation criterion that yields the greatest cost savings and the corresponding value of the context information are selected to generate one or more new nodes of the context tree.

A node may start as a leaf node and then change to a non-leaf node. For example, the first level context tree may include one node that is a leaf node that represents all of the first set of coded syntax elements. Separation criteria may be applied to the context information values to separate the first set of syntax elements. After the separation criterion is applied, the node becomes a non-leaf node representing the applied separation criterion, and the leaf node is created as a child node of the non-leaf node. Each of the child nodes represents a data group that includes a subset of all the first set of syntax elements. For example, if the context tree is a binary tree, applying a separation criterion to the value of the context information for a node in the first level context tree includes creating two child nodes of that node. The data group represented by each of these child nodes may include, for example, half of the first set of syntax elements. Alternatively, these data groups may include different amounts of the first set of syntax elements.

The nodes of the context tree are associated with cost savings for the first set of syntax elements (and, for example, subsequent sets of syntax elements coded with the context tree). The context tree is generated to determine the lowest cost to code the syntax elements associated with blocks of video frames. The cost of coding syntax elements may be the computational cost of coding syntax elements. Computational cost may indicate, for example, the amount of computational resources used for coding. For example, the cost to code a syntax element is the time to code the syntax element, the space used to code the syntax element, and the number of operations to code the syntax element (eg, , Arithmetic operation) may indicate one or more parameters selected from the group of parameters, or may be a function of some of them. The cost to code a syntax element may depend on the number of syntax elements to code. The number of syntax elements to code can be represented using a data group. The leaf nodes of the context tree represent different groups of data and thus different possible numbers of syntax elements. As such, the cost of entropy coding the syntax elements associated with blocks of a video frame may be determined for each leaf node. For example, the cost of entropy coding syntax elements based on the data groups represented by the leaf nodes of the context tree can be calculated using the following formula:

Where g _i is the i-th data group, e(g _i ) is the entropy cost function of data group g _i , and r(size(g _i )) is a positive value that decreases with the size of data group g _i. Is a size penalty function having an output of λ, and λ is a weight of the size penalty function r(size(g _i )). The size penalty function r(size(g _i )) may have, for example, a range and a domain R ⁺ →R ⁺ . The entropy cost function can be calculated using the following formula:

Here, n _i is the length of the data in the data group g _i , and p _i [k] is the probability for the syntax element of the data group g _i having the syntax value k. Each data group represented by a node in the context tree may be associated with a probabilistic model.

Generating the context tree can include determining potential cost savings from creating the nodes that represent the separated data groups of the first set of syntax elements. That is, when the separation criterion is applied to a leaf node, the leaf node becomes a non-leaf node (for example, a parent node) having two or more child nodes. The cost savings in determining which node to separate using the separation criterion is calculated by subtracting the total cost of the resulting child nodes from the cost of the parent node. The generated context tree may have one or more leaf nodes that may be non-leaf nodes (eg, parent nodes) using the separation criteria. Thus, generating the context tree is for entropy coding the first set of syntax elements when any of the leaf nodes of a given level or set of leaf nodes of the context tree are isolated. Determination of the highest cost savings of

Determining to separate the leaf nodes that yield the highest cost savings in a given set of leaf nodes in the context tree is to determine the data groups that represent the leaf nodes before and after the separation criterion is applied to those leaf nodes. It may include determining a cost reduction for entropy coding. The leaf nodes associated with the highest cost savings can be isolated to produce child nodes, but another consequence is that in some implementations the data groups represented by other nodes in the set of leaf nodes are not. .. Further implementations and examples for generating a context tree are described below with respect to FIG.

At 606, a second set of syntax elements and associated contextual information is identified. The second set of syntax elements may be associated with blocks placed in a raster scan or other scan order after the block containing the first set of syntax elements used to generate the context tree. .. Alternatively, the second set of syntax elements may be associated with a different video frame than the first set of syntax elements used to generate the context tree. The context information used to code the second set of syntax elements may be represented as a value of a second context vector. The second context vector may be the same or different than the context vector containing the values of the context information used to code the first set of context information.

At 608, one of the plurality of nodes of the context tree is identified based on a value of context information associated with one of the syntax elements of the second set of syntax elements. The identified node represents a data group that includes that syntax element of the second set of syntax elements. Identifying the node based on the value of the context information is coded previously (eg, used to generate the context tree), eg, for the value of the second context information associated with the syntax element. May include applying some of the separation criteria used to separate the syntax elements associated with the blocks.

For example, if the context tree is a binary tree, the context information values may be decomposed as true or false against some of the separation criteria. The first separation criterion used to separate the first node of the first tree level into the second and third nodes of the second tree level is the context information of the first index of the context vector. You can ask if the value is greater than 3. If not, the separation criterion used to separate the second node into the fourth and fifth nodes of the third tree level may be applied to the value of the context information. However, if so, the separation criteria used to separate the third node into the sixth and seventh nodes of the third tree level may be applied instead to the value of the context information. .. This process may be repeated until one node is identified based on the value of the contextual information and the separation criteria.

At 610, the syntax element of the second set of syntax elements is coded according to a probabilistic model associated with the identified node. The nodes of the context tree, or the data groups represented by those nodes, are each associated with a probabilistic model. The probabilistic model associated with a node or data group can reflect the probabilities for the syntax elements of that node or data group. In some implementations, the probabilistic model identifies (eg, includes) probabilities only for those syntax elements of the node or data group and determines probabilities for syntax elements that are not included in the data group. Not specified (eg, not included).

The probabilistic model can indicate the probability that a syntax element associated with a block of a video frame will have a particular value, or will be present in that block or frame. For example, the probabilistic model may include integer values that reflect different probabilities that may be associated with one or more of the syntax elements. A maximum value may be defined for a probability model so that a given probability for a syntax element can be expressed as a percentage derived by dividing an integer value by the maximum value. For example, the maximum value for the probabilistic model can be on a scale of 256. The probabilities for syntax elements may reflect the value 119. Therefore, the probabilistic model will show that there is a 119/256 probability associated with that syntax element.

Coding that syntax element of the second set of syntax elements according to the probabilistic model associated with the identified node is performed during the encoding process according to the probabilistic model of the second set of syntax elements. Of the syntax elements of, or decoding the syntax elements of the second set of syntax elements according to a probabilistic model. Probabilities associated with the identified probabilistic model are processed using entropy coding (eg, in entropy encoding stage 408 shown in FIG. 4 or entropy decoding stage 502 shown in FIG. 5). For example, arithmetic coding can be used to encode or decode syntax elements that have a probability of meeting a threshold so that the syntax element is not coded or decoded when its probability is too low.

Therefore, arithmetic coding may be performed to limit the total size of the bitstream or the total number of syntax elements encoded in the bitstream, such as minimizing the cost of sending the bitstream. it can. Therefore, during the encoding operation, and after the arithmetic coding is performed on the second set of syntax elements according to the identified probabilistic model, the second set of syntax elements is converted into an encoded bitstream. It is encoded by being compressed. Alternatively, during the decoding operation, and after the arithmetic coding is performed on the second set of syntax elements according to the identified probabilistic model, the second set of syntax elements is extracted from the encoded bitstream. It is decrypted by being decompressed.

In some implementations, the technique 600 updates the context tree with a second set of syntax elements, and then codes the second set of syntax elements with the updated context tree. Can be included. For example, the probabilistic model associated with the nodes of the context tree may be recomputed based on the second set of syntax elements and their associated contextual information, and so on. A second set of syntax elements may then be coded based on the recalculated stochastic model. For example, the recomputed cost savings are used to isolate a leaf node, eg, change that leaf node to a non-leaf node associated with the cost savings, thereby coding a second set of syntax elements. An estimated minimum cost to do is obtained.

In some implementations, the context tree may be generated based on the data received in the encoded bitstream. For example, if the technique 600 is performed to decode an encoded syntax element, the data indicative of the context tree may be encoded as transmitted from an encoder, relay device, or another computing device. It can be received at the decoder in a separate bitstream. The encoded bitstream includes encoded video frames, and the encoded video frames include encoded blocks to which the encoded syntax elements are associated. Thus, the context tree may be generated based on the information received from the encoder, for example. For example, the data used to generate the bitstream may include data indicating a separation criterion applied to each of the context information for decoding the encoded syntax elements.

In some implementations, the syntax elements associated with a block may be encoded or decoded with data stored in cache. For example, cached data may indicate that syntax elements are separated into data groups. Isolation may be used, for example, to generate a context tree, update a context tree, or confirm that the current context tree matches the indicated isolation.

In some implementations, the probabilistic model with the syntax elements coded accordingly may be updated. For example, the probabilistic model may be updated in response to encoding or decoding the final block of the video frame. Updating the probabilistic model may include counting the number of times a syntax element is associated with a block of video frames. For example, the number may be updated in response to each applicable block being encoded or decoded. The probabilistic model may be updated based on the total number obtained after the last block of the video frame has been encoded or decoded. For example, if the count is greater than a threshold, the probabilistic model can be updated to reflect that the syntax element is at a particular value or the probability of being present has increased. If the count is less than the threshold, the probabilistic model can be updated to reflect that the probability has decreased. The threshold may be, for example, the total number of its syntax elements in the previous video frame.

The updating of the probabilistic model can be done independently by each of the encoder and the decoder. For example, an encoder and a decoder may separately store probabilistic models that can be used to encode and decode syntax elements according to the techniques of this disclosure. Alternatively, the probabilistic model updates may be determined at the encoder and communicated to the decoder. For example, the encoder updates the probabilities associated with the probabilistic model after the video frame is encoded and synchronizes those updated probabilities with the decoder to decode the encoded video frame. Make it available.

In some implementations, identifying context information for coding syntax elements associated with a block selects one of a set of context information used to code the syntax elements. May be included. The encoder is configured to select one of the N sets available to it and to generate, receive, or otherwise identify a context vector based only on the possible values of the selected set. Can be done. For example, the encoder may select the set of context information that is most used to code syntax elements associated with other blocks of the same video frame. In another example, the encoder may select the set of contextual information used to code the syntax elements by the most recently encoded block of the video frame. The context vector does not include values for other sets of context information. As such, each index in the context vector reflects a single value of context information for the selected set.

In some implementations, the encoder or decoder does the type of context information within the set context information is suitable for coding the syntax elements associated with the block before determining its value? You can judge. For example, if one or more of the types of contextual information in the set are not associated with or provide meaningful information that can be used to encode or decode syntax elements associated with the block. Can be. The encoder or decoder may not determine a value for irrelevant type context information, or otherwise do not include or use irrelevant type context information in the context vector. In some implementations, the decoder may not be configured to generate the context vector. For example, the value of the context information used to encode the syntax element may be conveyed from the encoder to the decoder, such as by using the context vector. The decoder can decode the syntax element using the values contained in the context vector received from the encoder.

Referring now to FIG. 7, a flowchart of a technique 700 for generating a context tree for coding syntax elements associated with blocks of video frames is shown. Technique 700 may include one or more of the operations of technique 600 shown in FIG. 6, such as performed at 604. As such, technique 700 may be implemented as part of a technique, eg technique 600, for coding syntax elements associated with blocks of video frames. Alternatively, technique 700 may be performed separately from the technique for coding syntax elements. For example, the act of generating the context tree may be independent of the act of coding the syntax element. This may be the case, for example, if the context tree is generated with a sample set of syntax elements and is trained to update the context tree before the context tree itself is used to code the syntax elements. ..

At 702, the separation criterion to use for the first group of data is determined. The first data group may be a data group that includes all syntax elements for encoding or decoding with the generated context tree. The separation criterion can be one of a plurality of separation criteria that can be used to evaluate contextual information. For example, the encoder or decoder may have access to a list of separation criteria, which may be stored in, for example, a database or other data store. Each separation criterion in the list may include a representation that can return a number, a range of numbers, or a binary value, depending on the type of context tree. For example, if the context tree is a binary tree, each separation criterion may include expressions that, when applied to the value of context information, return as true or false.

Determining the separation criterion to apply to the first group of data includes applying different separation criteria to different values of context information for multiple syntax elements to encode or decode to determine a candidate cost reduction. Can include doing. Each candidate cost reduction may represent one separation criterion and the corresponding value of context information to which it applies. Multiple candidate cost savings can be compared to determine the highest cost savings of them. The separation criterion that provides the highest cost savings can be determined as the separation criterion to use for the first data group. For example, determining a candidate cost reduction may include performing a breadth-first search on nodes of the context tree that have already been generated to identify one or more candidate data groups to separate. Thus, different ones of the candidate cost savings are associated with different one or more candidate data groups.

At 704, the first data group is separated into a second data group and a third data group using the determined separation criteria. For example, if the context tree is a binary tree, then separating the first data group into a second data group and a third data group using the determined separation criteria is a synonym for the first data group. It may include determining which of the tax elements are decomposed as true when the separation criterion is used and which are decomposed as false instead. For example, the second data group includes the syntax elements of the first data group that are decomposed as true, and the third data group includes the syntax elements of the first data group that is decomposed as false. You can The first data group may be represented in the context tree with the first node. Thus, separating the first data group creates a second node representing the second data group in the context tree, creating a third node representing the third data group in the context tree. Can include doing.

At 706, a separation criterion to use for each of the second and third data groups is determined. Determining the separation criterion to use for the second data group is because applying different separation criteria to different values of contextual information for the first part of the syntax element for which the candidate cost reduction was determined. Determining the resulting set of candidate cost savings, such that the first data group is separated into second and third data groups. The separation criterion that yields the highest candidate cost savings is selected for the second data group. Since determining the separation criterion to use for the third data group also applies different separation criterion to different values of context information for the second part of the syntax element for which the candidate cost reduction was determined. Determining the resulting set of candidate cost savings, such that the first data group is separated into second and third data groups. The separation criterion that yields the highest candidate cost savings is selected for the third data group. The first portion and the second portion of the syntax elements subject to candidate cost reductions that result in the separation of the first data group may be different portions of those syntax elements. Alternatively, the first portion and the second portion may share some or all of the syntax elements subject to candidate cost reduction resulting in the separation of the first data group.

At 708, the highest cost savings resulting from using the separation criterion selected for the second data group is the best obtained by using the separation criterion selected for the third data group. Compared to cost savings. For example, the highest cost savings resulting from using the separation criterion selected for the second data group based on that comparison is to use the separation criterion selected for the third data group. A determination can be made that it is greater than the maximum cost savings caused by.

At 710, the highest cost savings incurred by using the segregation criteria selected for the second data group is the highest cost savings incurred by using the segregation criteria selected for the second data group. The second group of data may be separated in response to the determination of greater than. For example, the second data group may be separated into a fourth data group and a fifth data group using the separation criteria selected for the second data group. Separating the second data group may further include generating a node representing the fourth data group in the context tree, generating a node representing the fifth data group in the context tree.

At 712, a context tree is generated that includes nodes representing the first, second, third, fourth, and fifth data groups. Generating the context tree may include generating the nodes described above. For example, the node representing the first data group may not have been generated during or prior to separating the first data group into second and third data groups. Similarly, the node representing the fourth data group may not have been generated during or prior to the separation of the second data group into the fourth and fifth data groups. Alternatively, all of these nodes may be created at the same time, or near the same time, after the separation of the data groups is complete. Alternatively, generating the context tree may include associating previously generated nodes with the context tree (eg, during or before each separation of the data groups).

In some implementations, the technique 700 may reduce the total cost for entropy coding the second and third data groups before separating the first data group into the second and third data groups. It may include determining if the cost is less than the cost of entropy coding the one data group. For example, determining a separation criterion to apply to the value of context information includes calculating a cost of entropy coding each of the first data group, the second data group, and the third data group. obtain. A comparison may then be performed between the cost of entropy coding the first data group and the total cost of entropy coding the second and third data groups. For example, if it is determined that the total cost to entropy code the data groups represented by the nodes is not less than the cost to entropy code the data groups that are separated to produce those nodes, Those nodes may not be created.

In some implementations, comparing the cost of entropy coding the first data group with the total cost of entropy coding the second and third data groups is performed by the node It can be executed after being generated as a part. For example, a determination that the total cost to entropy code the data groups represented by the newly generated nodes is not less than the cost to entropy code the separated data groups to generate those nodes. If done, those newly created nodes may be removed from the context tree.

In some implementations, the data groups are separated into two or more data groups based on the amount of cost savings resulting from applying the separation criteria to the value of context information. For example, separation of data groups may be responsive to a determination that the highest candidate cost reduction meets the reduction threshold. The reduction threshold can be an integer, a floating point number, or other value that represents a minimum reduction in cost for coding syntax elements associated with a block. A data group may not be separated if its minimum reduction is not met by the application of a separation criterion to the value of context information. For example, the reduction threshold may be used to prevent computational resources from being consumed without improving coding efficiency of syntax elements associated with the block.

A system for coding syntax elements with a context tree is now described with reference to FIGS. 8-9. The systems shown in FIGS. 8-9 may include using all or a portion of one or both of the technique 600 shown in FIG. 6 or the technique 700 shown in FIG. 7. For example, a system may represent hardware and/or software components used to perform all or part of one or both of technology 600 or technology 700.

FIG. 8 is a block diagram of a system for encoding syntax elements associated with a current block of a video frame. A system for encoding syntax elements may be implemented by, for example, an encoder such as encoder 400 shown in FIG. The system includes context information 800 corresponding to previously coded syntax elements 802. Context information 800 may be one or more sets of context information defined for use by the encoder. Context information 800 may include context vectors generated, received, or otherwise identified based on the values of the set of context information. Syntax element 802 is a syntax element associated with the block to be encoded.

Context information 800 and previously coded syntax elements 802 are cached in context cache 804 and syntax element cache 806, respectively. The data stored in context cache 804 and syntax element cache 806 is then used as input to generate context tree 808. For example, the context vector contained in the context information 800 may be cached in the context cache 804 and subsequently received by a software component configured to generate the context tree 808. For example, the value of the context information contained in the context vector is used in conjunction with the separation criteria available to the encoder to base the lowest cost to encode one of the syntax elements in syntax element cache 806. Can generate a context tree 808. After the context tree 808 is generated, the probabilistic model 810 can be estimated for each leaf node in the context tree based on the data groups represented by those leaf nodes. The probabilistic model is based on an incoming syntax element (eg, the syntax element 802 used to generate the context tree 808 is associated with the second block of the video frame, such as when the second block of the video frame is associated with the first block). Syntax element associated with a block of. For example, previously coded syntax element 802 refers to a first set of syntax elements associated with a first block of a video frame, the incoming syntax elements being the current block to be encoded. May refer to a second set of syntax elements associated with a second block of a video frame, such as. To encode the incoming syntax element, context information (eg, of context information 800) corresponding to the incoming syntax element may be identified. The context information can then be applied to identify the node in the context tree 808 that has the lowest cost to encode the incoming syntax element. The probabilistic model 810 associated with that identified node of the context tree 808 may then be identified. Probabilistic model 810 may be one of a plurality of probabilistic models stored in a probability table of probabilities for different context information available to the encoder. Identifying the probabilistic model 810 may include, for example, querying the probability table with information associated with the identified node in the context tree 808.

Accordingly, the context information corresponding to the incoming syntax element to be encoded is passed to the context tree 808 to request or otherwise identify the probabilistic model 810. The probabilistic model 810 is then passed to the arithmetic coder 812. Arithmetic coder 812 may be a software module or similar component configured to perform arithmetic coding with the probabilities of probabilistic model 810. Incoming syntax elements are also passed to arithmetic coder 812. Arithmetic coder 812 then entropy codes the incoming syntax elements according to stochastic model 810. As a result of entropy coding by arithmetic coder 812, coded block 814 is generated. Encoded block 814 includes the incoming syntax elements in encoded form. Encoding block 814 may be included in an encoded bitstream, such as compressed bitstream 420 shown in FIG.

Context cache 804 and syntax element cache 806 may store syntax element 802 and corresponding encoded context information 800. Context cache 804 and syntax element cache 806 may be used to generate context tree 808. The context cache 804 and the syntax element cache 806 are pre-existing when a subsequent set of syntax elements is received and cached in the syntax element cache 806 or otherwise identified for encoding. It can be used to update the context tree 804. Syntax element cache 806 may be a cache for storing syntax elements 802, incoming syntax elements, and/or other sets of encoded syntax elements. For example, the subsequent set of syntax elements may be compared to the syntax elements stored in syntax element cache 806. If a match with the information stored in the syntax element cache 806 is determined, the context tree 808 can be used without modification.

FIG. 9 is a block diagram of a system for decoding coded syntax elements associated with coded blocks of a coded video frame. A system for decoding encoded syntax elements may be implemented by, for example, a decoder such as decoder 500 shown in FIG. Encoded bitstream 900 represents a video stream and includes encoded blocks of the video stream. For example, the encoded bitstream 900 may be the compressed bitstream 420 shown in FIG. The encoded block of encoded bitstream 900 may be, for example, encoded block 810 shown in FIG.

The system uses the context information 902 corresponding to the previously coded syntax element 904. Context information 902 may be one or more sets of context information defined for use by the decoder. Context information 902 and previously coded syntax elements 904 are cached in context cache 906 and syntax element cache 908, respectively. The data stored in context cache 906 and syntax element cache 908 may be used as input to generate context tree 910. For example, contextual information 902 may include a context vector that is generated, received, or otherwise identified based on the value of the set of contextual information. For example, the context vector may be received from the encoder. The context vector is cached in the context cache 906. Syntax element 904 is a previously coded syntax element that is used with context information 902 to generate context tree 910. The context tree 910 may be the context tree 808 shown in FIG. For example, context tree 910 may be generated in the same manner that context tree 808 was generated. In another example, context tree 808 may be communicated to a decoder for use in decoding syntax elements as context tree 910. The generated context tree 910 is then used to generate incoming syntax elements (eg, the syntax element 904 used to generate the context tree 910 is the first code of the encoded video frame). A probabilistic model for decoding the syntax elements associated with the second encoded block of the encoded video frame, such as those associated with encoded blocks. For example, previously coded syntax element 904 refers to a first set of syntax elements associated with a first block of a video frame, and the incoming syntax elements are coded to be decoded. A second set of syntax elements associated with a second block of a video frame, such as a block.

Probabilistic model 912 may pass context information corresponding to incoming syntax elements to be decoded to context tree 910 in the same manner as described above with respect to probabilistic model 810 shown in FIG. (By identifying the model 912), it is identified based on the context tree 910. Probabilistic model 912 is then passed to arithmetic coder 914 to decode the incoming syntax elements. For example, arithmetic coder 914 can be a software module or similar component configured to entropy decode incoming syntax elements using the probabilities of probabilistic model 912. Decoded block 916 is generated in response to arithmetic coder 914 entropy decoding the incoming syntax elements using the probabilities of probabilistic model 912. The decoded block 916 may then be output as part of a video stream, such as output video stream 516 shown in FIG.

Context cache 906 and syntax element cache 908 can store syntax element 904 and the corresponding decoded context information 902. Context cache 906 and syntax element cache 908 may be used to generate context tree 910. Context cache 906 and syntax element cache 908 are used when a subsequent set of encoded syntax elements is received and cached in syntax element cache 908, or otherwise identified for decoding. , Can be used to update an existing context tree 910. The syntax element cache 908 is a cache for storing syntax elements 904, incoming syntax elements, and/or other sets of encoded syntax elements before they are decoded. You may For example, the subsequent set of encoded syntax elements may be compared to the encoded syntax elements stored in syntax element cache 908. If a match with the information stored in the syntax element cache 908 is determined, the context tree 910 can be used without modification.

Referring now to FIGS. 10A-11, exemplary context tree generation is described. The three exemplary context trees generated as shown in FIGS. 10A-11 are for performing all or part of one or both of the technique 600 shown in FIG. 6 or the technique 700 shown in FIG. Can be generated by. Further, the exemplary context trees generated as shown in FIGS. 10A-11 may be generated using one or both of the systems shown in FIGS. 8-9. For example, the exemplary context tree may be generated by an encoder using the system shown in FIG. 8 or a decoder using the system shown in FIG. The encoder or decoder may generate a tree as part of a technique for coding syntax elements, eg, based on technique 600. Alternatively, the encoder or decoder may generate a tree, for example based on technique 700, independent of coding syntax elements.

FIG. 10A is a diagram showing an example of a first stage for generating a context tree. For example, FIG. 10A illustrates a first level generation of an exemplary context tree. Node 1000 is a leaf node that represents a data group that includes all of the syntax elements associated with the block to be encoded or decoded. The data groups represented by node 1000 may be separated into data groups represented by nodes 1002 and 1004 by applying separation criteria to the values of context information. As mentioned above, the particular isolation criteria that result in the isolation of the node 1000 and the value of the context information may be determined based on a candidate cost reduction determined based on the application of different isolation criteria to different values of context information. Nodes 1000 are separated by applying a separation criterion of greater than 3 to the value of context information located at the [0] index of the context vector.

FIG. 10B is a diagram showing an example of the second stage for generating the context tree. For example, FIG. 10B illustrates a second level generation of the exemplary context tree. Nodes 1002 and 1004 are created in response to the separation of the data groups represented by node 1000 in the first stage shown in FIG. 10A. The data group represented by node 1000 has a total cost for coding the syntax elements contained in the data groups represented by nodes 1002 and 1004, and the total cost of coding the syntax elements contained in the data group represented by node 1000. In response to determining that it is less than costly to code, the data groups represented by nodes 1002 and 1004 are separated.

Nodes 1002 and 1004 are leaf nodes. Node 1000 becomes a non-leaf node in response to the creation of nodes 1002 and 1004. Node 1002 represents a data group that includes some of the syntax elements associated with the block to be encoded or decoded. Node 1004 represents a data group containing the rest of these syntax elements. The data group represented by node 1002 can be separated into the data group represented by node 1006 and the data group represented by node 1008. The data group represented by node 1004 can be separated into the data group represented by node 1010 and the data group represented by node 1012.

Separation criteria are selected to separate the data groups represented by node 1002. The separation criterion includes the expression that the remainder by 2 is less than 1, and is applied to the value of the context information located at the [2] index of the context vector. The cost savings resulting from using that isolation criterion for node 1002 are 20. Different separation criteria are selected to separate the data groups represented by node 1004. The separation criterion includes the expression less than 2 and is applied to the value of the context information located at the [1] index of the context vector. The cost savings by using that isolation criterion for node 1004 is 15. Thus, the cost savings resulting from separating the data group represented by node 1002 into the data groups represented by nodes 1006 and 1008 is the data group represented by node 1004 represented by nodes 1010 and 1012. Greater than the cost savings resulting from the separation.

FIG. 10C is a diagram showing an example of the third stage for generating the context tree. For example, FIG. 10C illustrates a third level generation of an exemplary context tree. Nodes 1006 and 1008 are created in response to the separation of the data groups represented by node 1002 in the second stage shown in FIG. 10B. The data group represented by node 1002 has a total cost for coding the syntax elements contained in the data groups represented by nodes 1006 and 1008, and In response to determining that it is less than costly to code, it is separated into the data groups represented by nodes 1006 and 1008.

Nodes 1006 and 1008 are leaf nodes. Node 1002 becomes a non-leaf node in response to the generation of nodes 1006 and 1008. Node 1006 represents a data group that includes some of the syntax elements contained in the data group represented by node 1002. Node 1008 represents a data group that contains the rest of these syntax elements. The data group represented by node 1006 may be separated into the data group represented by node 1014 and the data group represented by node 1016. The data group represented by node 1008 may be separated into the data group represented by node 1018 and the data group represented by node 1020.

Separation criteria are selected to separate the data groups represented by node 1006. The separation criterion includes the expression that the remainder is equal to minus one and is applied to the value of the context information located at the [1] index of the context vector. The cost savings resulting from using that isolation criterion for node 1006 is 7. Different separation criteria are selected to separate the data groups represented by node 1008. The separation criterion includes the expression greater than 1 and is applied to the value of the context information located at the [0] index of the context vector. The cost savings by using that isolation criterion for node 1008 is 18. Therefore, the cost savings resulting from separating the data group represented by node 1008 into the data groups represented by nodes 1018 and 1020 are the data groups represented by leaf node 1006 and the data represented by nodes 1014 and 1018. The cost savings resulting from the separation into groups are greater than the cost savings resulting from separating the data group represented by leaf node 1004 into the data groups represented by nodes 1010 and 1012.

FIG. 10D is a diagram showing an example of the fourth stage for generating a context tree. For example, FIG. 10D illustrates a fourth level generation of the exemplary context tree. Nodes 1018 and 1020 are created in response to the separation of the data groups represented by node 1008 in the third stage shown in FIG. 10C. The data group represented by node 1008 has a total cost for coding the syntax elements contained in the data groups represented by nodes 1018 and 1020, and the total cost of coding the syntax elements contained in the data group represented by node 1008. In response to determining that it is less than costly to code, it is separated into the data groups represented by nodes 1018 and 1020.

Nodes 1018 and 1020 are leaf nodes. Node 1008 becomes a non-leaf node in response to the generation of nodes 1018 and 1020. Node 1018 represents a data group that includes some of the syntax elements contained in the data group represented by node 1008. Node 1020 represents a data group that contains the rest of these syntax elements. The data group represented by node 1018 may be separated into the data group represented by node 1022 and the data group represented by node 1024. The data group represented by node 1020 can be separated into the data group represented by node 1026 and the data group represented by node 1028.

Separation criteria are selected to separate the data groups represented by node 1018. The separation criterion includes the expression that the remainder is less than 1 and is applied to the value of the context information located at the [1] index of the context vector. The cost savings resulting from using that isolation criterion for node 1018 is 6. Different separation criteria are selected to separate the data groups represented by node 1020. The separation criterion includes the expression that the remainder by 2 is equal to 1 and is applied to the value of the context information located at the [2] index of the context vector. The cost savings by using that isolation criterion for node 1020 is three.

Thus, the cost savings resulting from separating the data group represented by node 1018 into the data groups represented by nodes 1022 and 1024 is the data group represented by node 1020 representing the data group represented by nodes 1026 and 1028. Greater than the cost savings resulting from the separation. However, the total cost for coding the syntax elements included in the data group represented by nodes 1022 and 1024 is significantly greater than the cost for coding the syntax elements included in the data group represented by node 1018. A determination may be made that it is not small. As a result of that determination, the data group represented by node 1018 is not separated and no node is created at the fifth level of the exemplary context tree.

FIG. 11 is a diagram showing an example of the context tree 1100. Context tree 1100 includes nodes 1000, 1002, 1004, 1006, 1008, 1010, 1012, 1018, and 1020 generated in the first, second, third, and fourth stages shown in FIGS. 10A-10D. It is a binary tree containing. For example, although not described above, nodes 1010 and 1012 have a total cost of coding the syntax elements included in the data groups represented by nodes 1010 and 1012 that the data groups represented by node 1004. It may be generated in response to determining that it is less than the cost to code the included syntax elements.

The context tree 1100 may be used for subsequent coding of syntax elements. Alternatively, the context tree 1100 may be trained before being used to code the syntax elements without further modification. For example, context tree 1100 may have been generated based on N syntax elements. Training the context tree 1100 with another N syntax elements and verifying that the coding efficiency of these N syntax elements is improved over the first N syntax elements. it can. The context tree 1100 can then be used for further coding of subsequent syntax elements without further modification.

The implementation of context tree 1100 shown in FIG. 11 may include additional features, fewer features, or different features than those shown and described. In some implementations, the context tree may be a non-binary tree. For example, a separation criterion applied to contextual information may return multiple branches in a context tree, such as when each branch corresponds to a range of possible values to meet the separation criterion. For example, the binary version of the separation criterion may ask if the value of the context information in the first index of the context vector is less than 3. The tree may include two branches based on its separation criterion, one branch leading to a node representing a data group whose context information value is less than 3 and the other branch the context information value. Is connected to a node that represents a data group of 3 or more.

However, the non-binary version of that separation criterion can ask what the value of the context information at the first index of the context vector is. For example, a first branch based on that separation criterion leads to a node representing a data group whose context information has a value of 0 to 1, while other such branches have a context information value of 2 to 3 and so on. Other such branches may lead to nodes representing data groups, and other such branches may lead to nodes representing data groups whose context information has a value of 4 to 5. In some implementations, there may be configurable or non-configurable limits on the maximum number of branches resulting from the application of non-binary separation criteria to the value of context information.

The encoding and decoding aspects described above illustrate some examples of encoding and decoding techniques. However, it is to be understood that encoding and decoding may refer to compression, decompression, conversion, or any other process or modification of data as those terms are used in the claims.

The word "example" is used herein to mean an example, instance, or illustration. Any aspect or design described herein as "examples" should not necessarily be construed as preferred or advantageous over other aspects or designs. Rather, use of the word "example" is intended to present concepts in a concrete way. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless specified otherwise, or explicitly indicated otherwise by context, the expression "X comprises A or B" means any natural inclusive permutation. Intended to. That is, "X includes A or B" is satisfied when X includes A, X includes B, or X includes both A and B. Further, the articles "a" and "an" used in the present application and the appended claims, unless otherwise indicated or clearly indicated by the context of the singular, refer to "one or more. Should be generally construed as meaning "." Furthermore, the use of the terms “implementation” or “one implementation” in this disclosure is not intended to mean the same embodiment or implementation, unless so stated.

Implementations of the transmitting station 102 and/or the receiving station 106 (as well as the algorithms, methods, instructions, etc., which the encoder 400 and decoder 500 include, and are stored in, and/or executed by) the hardware. , Software, or any combination thereof. Hardware includes, for example, computers, intellectual property (IP) cores, application specific integrated circuits (ASICs), programmable logic arrays, optical processors, programmable logic controllers, microcode, microcontrollers, servers, microprocessors, digital signal processors. , Or other suitable circuitry. In the claims, the term "processor" should be understood to include any of the foregoing hardware, either alone or in combination. The terms "signal" and "data" are used interchangeably. Moreover, some of the transmitting station 102 and the receiving station 106 do not necessarily have to be implemented in the same way.

Further, in one aspect, for example, transmitting station 102 or receiving station 106 comprises a computer program product that, when executed, executes any of the methods, algorithms, and/or instructions described herein. It may be implemented using a general purpose computer or a general purpose processor. Additionally or alternatively, a dedicated computer/processor may be utilized that may include, for example, other hardware for executing any of the methods, algorithms, or instructions described herein. ..

The transmitting station 102 and the receiving station 106 may be implemented on a computer in a video conferencing system, for example. Alternatively, transmitting station 102 may be implemented on a server and receiving station 106 may be implemented on a device such as a handheld communication device separate from the server. In this case, the transmitting station 102 may use the encoder 400 to encode the content into an encoded video signal and transmit the encoded video signal to the communication device. The communication device can decode the encoded video signal using the decoder 500. Alternatively, the communication device may decrypt content stored locally on the communication device, eg, content that was not transmitted by transmitting station 102. Other suitable transmit and receive implementations are available. For example, receiving station 106 may be a generally stationary personal computer rather than a portable communication device, and/or the device containing encoder 400 may include decoder 500.

Moreover, all or part of an implementation of the present disclosure can take the form of, for example, a computer program product accessible from a computer usable or computer readable medium. Computer-usable or computer-readable media can be any device that can tangibly contain, store, communicate, or transport a program, for example, for use by or in connection with any processor. The medium can be, for example, an electronic, magnetic, optical, electromagnetic, or semiconductor device. Other suitable media are also available.

The embodiments, implementations, and aspects described above are set forth to facilitate an understanding of the present disclosure and are not intended to limit the present disclosure. On the contrary, the present disclosure is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims, which scope includes all such modifications and equivalent arrangements. The broadest interpretation as permitted by law should be given.

Claims (20)

A method of decoding an encoded block of an encoded video frame, the method comprising:
Context tree,
Determining a first candidate cost reduction resulting from applying a separation criterion to context information about a first syntax element of a previously decoded block of the encoded video frame,
A first group of the first syntax elements according to a first separation criterion of the separation criteria resulting in the highest of the first candidate cost savings, and the first syntax. Separating the syntax elements into a second group of elements,
Determining a second candidate cost reduction resulting from applying the separation criterion to contextual information for the first group,
Determining a third candidate cost reduction resulting from applying the separation criterion to contextual information for the second group,
In response to determining that the highest candidate cost reduction of the second candidate cost reductions is greater than the highest candidate cost reduction of the third candidate cost reductions, the second candidate cost reductions A first subgroup of the first group and a second subgroup of the first group according to a second separation criterion of the separation criteria that results in the highest candidate cost reduction of Generate by separating into two subgroups,
Decoding a second syntax element of the encoded block according to a stochastic model identified using the context tree. The context tree is
A first node representing the first syntax element;
A second node representing the first group,
A third node representing the second group,
A fourth node representing the first subgroup,
A fifth node representing the second subgroup,
The first node and the second node are non-leaf nodes,
The method of claim 1, wherein the third node, the fourth node, and the fifth node are leaf nodes. The plurality of nodes of the context tree include the first node, the second node, the third node, the fourth node, and the fifth node, the method comprising:
Of the first syntax elements including the one syntax element of the second syntax elements based on context information associated with the one syntax element of the second syntax elements; Identifying one of the plurality of nodes representing a group or subgroup,
The method of claim 2, further comprising: identifying the probabilistic model using identified nodes. Identifying one node of the plurality of nodes representing the first group of syntax elements that includes the one syntax element of the second syntax elements,
Applying the first separation criterion or the second separation criterion to context information associated with the one syntax element of the second syntax elements. Method. Updating the context tree by recomputing cost savings for one or more nodes of a plurality of nodes based on the second syntax element;
The method according to any one of claims 2 to 4, further comprising: decoding the third syntax element of the second encoded block using the updated context tree. Separating syntax elements into the first group and the second group according to a first separation criterion of the separation criteria that results in the highest of the first candidate cost savings. Is
Separating syntax elements into the first group and the second group in response to determining that the highest candidate cost reduction of the first candidate cost reductions meets a reduction threshold. The method according to any one of claims 1 to 5, comprising: Determining a first candidate cost reduction resulting from applying a separation criterion to contextual information about the first syntax element is:
7. A method according to any one of claims 1 to 6, comprising performing a breadth-first search on a plurality of nodes of the context tree to identify a node representing the first syntax element. An apparatus for decoding an encoded block of an encoded video frame, comprising:
Memory and
A processor configured to execute a plurality of instructions stored in the memory, the processor comprising:
Instructions for determining a first candidate cost reduction resulting from applying a separation criterion to context information about a first syntax element of a previously decoded block of the encoded video frame.
A first group of the first syntax elements according to a first separation criterion of the separation criteria that results in a highest candidate cost savings of the first candidate cost savings; and a first syntax of the first syntax elements. An instruction to separate the syntax elements into a second group of elements,
Instructions for determining a second candidate cost reduction resulting from applying the separation criterion to contextual information for the first group;
Instructions for determining a third candidate cost reduction resulting from applying the separation criterion to contextual information for the second group;
An instruction to compare the highest candidate cost reduction of the second candidate cost reductions with the highest candidate cost reduction of the third candidate cost reductions to determine which is greater;
In response to determining that the highest candidate cost reduction of the second candidate cost reductions is greater than the highest candidate cost reduction of the third candidate cost reductions,
Said first group according to a second separation criterion of said separation criteria resulting in the highest of the second candidate cost savings, said first group being a first subgroup of said first group; An instruction that separates into a second subgroup of the first group;
A plurality of first syntax elements, the first group, the second group, the first subgroup of the first group, and a plurality of second subgroups of the first group. Instructions to generate a context tree containing nodes,
In response to determining that the highest candidate cost reduction of the third candidate cost reductions is greater than the highest candidate cost reduction of the second candidate cost reductions,
Said second group according to a third separation criterion of said separation criteria that results in the highest of the second candidate cost savings, said second group being a first subgroup of said second group; An instruction that separates into a second subgroup of the second group;
A plurality of first syntax elements, the first group, the second group, the first subgroup of the second group, and a plurality of second subgroups of the second group. Instructions to generate a context tree containing nodes,
And decoding instructions for decoding a second syntax element of the encoded block according to a probability model identified using the context tree. The plurality of instructions are
Of the first syntax elements including the one syntax element of the second syntax elements based on context information associated with the one syntax element of the second syntax elements; An instruction identifying the node representing the group or subgroup,
Instructions for identifying the probabilistic model using identified nodes. An instruction identifying a node representing a group or subgroup of the first syntax elements that includes the one syntax element of the second syntax elements,
The apparatus of claim 9, comprising instructions for applying a separation criterion of one of the separation criteria to context information associated with the one syntax element of the second syntax element. The cost for entropy decoding one syntax element of the second syntax elements is
An entropy cost function of the group or subgroup of first syntax elements that includes the one syntax element of the second syntax elements;
A size penalty function having a positive output that decreases with the size of the first group of syntax elements or subgroups;
The apparatus according to any one of claims 8 to 10, wherein the apparatus is calculated based on the weighting of the size penalty function. The entropy cost function is
The length of the data in the first syntax element group or subgroup, and
12. The apparatus of claim 11, wherein the syntax elements included in the first group or subgroups of syntax elements have a probability of having a specified value. A method of decoding an encoded block of an encoded video frame, the method comprising:
Decoding syntax elements of the encoded block according to a probability model identified using a context tree, the context tree comprising:
A first node representing a syntax element of a previously decoded block of the encoded video frame,
A second node representing the first group of syntax elements;
A third node representing the second group of syntax elements;
A fourth node representing a first subgroup of the first group of syntax elements;
A fifth node representing a second subgroup of the first group of syntax elements;
The syntax elements are separated into the first group and the second group using one separation criterion that results in the highest candidate cost savings of the first set of candidate cost savings. Yes, the first set of candidate cost reductions is determined by applying a plurality of separation criteria including the one separation criterion to context information of the syntax element,
The first group is responsive to a determination that the highest candidate cost reduction of the second set of candidate cost reductions is greater than the highest candidate cost reduction of the third set of candidate cost reductions. , The first subgroup and the second subgroup are separated, and the second set of candidate cost reductions is based on the plurality of separation criteria with respect to the context information of the first group. And the third set of candidate cost reductions is determined by applying the plurality of separation criteria to the context information of the second group. ,Method. 14. The method of claim 13, further comprising: generating the context tree based on data encoded in an encoded bitstream that includes the encoded video frames. The plurality of nodes of the context tree include the first node, the second node, the third node, the fourth node, and the fifth node, the method comprising:
Including the one syntax element of the syntax elements of the encoded block based on context information associated with the syntax element of one of the syntax elements of the encoded block. Identifying one of a plurality of nodes representing a group or subgroup of the syntax elements of the previously decoded block,
15. The method of claim 13 or 14, further comprising: identifying the probabilistic model using identified nodes. The one separation criterion is a first separation criterion and one of a plurality of nodes representing a first group of syntax elements including one of the second syntax elements. To identify
Applying the first or second separation criterion of the plurality of separation criteria to context information associated with the one syntax element of the syntax elements of the encoded block; 16. The method of claim 15, comprising: Updating the context tree by recalculating cost savings for one or more nodes of the plurality of nodes based on a second syntax element;
The method of claim 15, further comprising: decoding the syntax elements of the second encoded block using the updated context tree. Responsive to determining that the highest candidate cost reduction of the first set of candidate cost reductions meets a reduction threshold, the syntax elements of the previously decoded block are converted into the first candidate cost reduction. 18. The method according to any one of claims 13 to 17, further comprising separating into a group and the second group. The cost for entropy decoding one syntax element of the syntax elements of the encoded block is:
An entropy cost function for a group or subgroup of syntax elements of the previously decoded block that includes the one syntax element of the syntax elements of the encoded block;
A size penalty function having a positive output that decreases with the size of the group or subgroup of syntax elements of the previously decoded block;
The method according to any one of claims 13 to 18, calculated based on the weighting of the size penalty function. The entropy cost function is
The length of data in a group or subgroup of syntax elements of the previously decoded block,
20. The method of claim 19, wherein the syntax elements included in the group or subgroup of syntax elements of the previously decoded block are calculated based on the probability of having a specified value.