KR102600727B1 - Binary arithmetic coding using parameterized probability estimation finite state machines - Google Patents

Abstract

For at least one each bin of the bin stream, the decoder may determine the value of each bin based on the state for that respective bin, the interval for each bin, and the offset value. Additionally, the decoder determines one or more finite state machine (FSM) parameters for the next bin of the bin stream. The one or more FSM parameters for the next bin control how probability estimates for the next bin are calculated from the state for each bin. The decoder updates the next bin in the bin stream using a parameterized state updating function that takes as input the state for each bin, one or more FSM parameters for the next bin in the bin stream, and the value of each bin. determine the status of The decoder may debinarize the empty stream to form decoded syntax elements.

Description

Binary arithmetic coding using parameterized probability estimation finite state machines

This application claims the benefit of U.S. Provisional Patent Application No. 62/474,919, filed March 22, 2017, and U.S. Provisional Patent Application No. 62/475,609, filed March 23, 2017, each of which is filed in its entirety. The content is incorporated herein by reference.

technology field

This disclosure relates to video coding, such as video encoding and video decoding.

background

Digital video capabilities include digital televisions, digital direct broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital cameras, A wide range of devices, including digital recording devices, digital media players, video gaming devices, video game consoles, cellular or satellite wireless phones, so-called “smart phones,” video teleconferencing devices, video streaming devices, etc. can be integrated into Digital video devices support MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264/MPEG-4, Part 10, Advanced Video Coding (AVC), ITU-T H.265, High Efficiency Video Coding Implements video compression techniques, such as those described in the standards defined by the (HEVC) standard, and extensions of those standards. Video devices may transmit, receive, encode, decode, and/or store digital video information more efficiently by implementing such video compression techniques.

Video compression techniques may perform spatial (intra-picture) prediction and/or temporal (inter-picture) prediction to reduce or remove redundancy inherent in video sequences. For block-based video coding, a video slice (e.g., a video frame or portion of a video frame) may be partitioned into video blocks, such as coding tree blocks and coding blocks. Spatial or temporal prediction generates a predictive block for the block to be coded. Residual data represents pixel differences between the original block to be coded and the predictive block. For further compression, the residual data may be transformed from the pixel domain to the transform domain resulting in residual transform coefficients, which may then be quantized.

In general, this disclosure describes techniques related to arithmetic coding. As described herein, techniques of this disclosure may improve compression of video data, potentially by improving adaptive probability estimation.

In one example, the present disclosure describes a method of decoding video data, the method comprising determining a decoded syntax element by applying binary arithmetic decoding to an offset value included in a bitstream, comprising: Applying the binary arithmetic decoding generates an empty stream, wherein generating the empty stream includes: for each bin of at least one of the empty stream, a state for each bin, an interval for each bin, and determining the value of each bin based on the offset value; determining one or more finite state machine (FSM) parameters for the next bin of the bin stream, wherein the one or more FSM parameters for the next bin control how probability estimates for the next bin are calculated from the state for each bin; , determining the one or more FSM parameters, wherein the next bin in the bin stream follows each bin in the bin stream; and, the state for the next bin in the bin stream using a parameterized state updating function that takes as input the state for each bin, one or more FSM parameters for the next bin in the bin stream, and the value of each bin. generating the empty stream, including determining: and, determining the decoded syntax element, including debinarizing the empty stream to form the decoded syntax element; and reconstructing a picture of the video data based in part on the decoded syntax elements.

In another example, this disclosure describes a method of encoding video data, the method comprising: generating syntax elements based on the video data; determining an offset value, at least in part, by applying a binary arithmetic encoding to a syntax element, wherein applying the binary arithmetic encoding at least in part binarizes the syntax element; for at least one each bin of the bin stream, determining an interval for the next bin of the bin stream based on a state for each bin, an interval for each bin, and a value of each bin; Determining one or more finite state machine (FSM) parameters for the next bin of a bin stream, wherein the one or more FSM parameters for the next bin control how probability estimates for the next bin are calculated from the state for each bin. , determining the one or more FSM parameters; and, the state for the next bin in the bin stream using a parameterized state updating function that takes as input the state for each bin, one or more FSM parameters for the next bin in the bin stream, and the value of each bin. generating an empty stream by determining an offset value, wherein the offset value is equal to a value in the interval for the last bin of the empty stream; And, it includes outputting a bitstream including an offset value.

In another example, the present disclosure describes an apparatus for decoding video data, the apparatus comprising: one or more storage media configured to store video data; and, one or more processors, wherein the one or more processors determine a decoded syntax element by applying binary arithmetic decoding to an offset value included in the bitstream, wherein the one or more processors perform the binary arithmetic decoding. As part of applying, one or more processors: generating an empty stream, wherein, as part of generating an empty stream, one or more processors: for at least one each bin of the empty stream, state for each bin. , determine the value of each bin based on the interval and offset value for each bin; determining one or more finite state machine (FSM) parameters for the next bin of the bin stream, wherein the one or more FSM parameters for the next bin control how probability estimates for the next bin are calculated from the state for each bin; , the next bin in the bin stream determines the one or more FSM parameters that follow each bin in the bin stream; Then, the state for the next bin in the bin stream using a parameterized state updating function that takes as input the state for each bin, one or more FSM parameters for the next bin in the bin stream, and the value of each bin. generate the empty stream, and determine the decoded syntax element, configured to debinarize the empty stream to form a decoded syntax element; And, it is configured to reconstruct a picture of the video data based in part on the decoded syntax elements.

In another example, the present disclosure describes an apparatus for encoding video data, the apparatus comprising: one or more storage media configured to store video data; and one or more processing circuits coupled to one or more storage media, wherein the one or more processing circuits generate a syntax element based on video data; By applying the binary arithmetic encoding to the syntax element, determining the offset value, the one or more processors: As part of applying the binary arithmetic encoding, the one or more processors: binarize, at least in part, the syntax element; and, for at least one each bin of the bin stream, determining an interval for the next bin of the bin stream based on the state for each bin, the interval for each bin, and the value of each bin; Determining one or more finite state machine (FSM) parameters for the next bin of a bin stream, wherein the one or more FSM parameters for the next bin control how probability estimates for the next bin are calculated from the state for each bin. , determining the one or more FSM parameters; and, the state for the next bin in the bin stream using a parameterized state updating function that takes as input the state for each bin, one or more FSM parameters for the next bin in the bin stream, and the value of each bin. configured to generate the empty stream, wherein the offset value is equal to a value in the interval for the last bin of the empty stream; And, it is configured to output a bitstream including an offset value.

In another example, the present disclosure describes an apparatus for decoding video data, the apparatus comprising means for determining a decoded syntax element by applying binary arithmetic decoding to an offset value included in a bitstream. , applying the binary arithmetic decoding generates an empty stream, wherein generating the empty stream includes: for at least one each bin of the empty stream, a state for each bin, and an interval for each bin. , and determining the value of each bin based on the offset value; determining one or more finite state machine (FSM) parameters for the next bin of the bin stream, wherein the one or more FSM parameters for the next bin control how probability estimates for the next bin are calculated from the state for each bin; , determining the one or more FSM parameters, wherein the next bin in the bin stream follows each bin in the bin stream; and, the state for the next bin in the bin stream using a parameterized state updating function that takes as input the state for each bin, one or more FSM parameters for the next bin in the bin stream, and the value of each bin. generating the empty stream, including determining: and means for determining a decoded syntax element, comprising debinarizing an empty stream to form a decoded syntax element; and means for reconstructing a picture of the video data based in part on the decoded syntax elements.

In another example, this disclosure describes an apparatus for encoding video data, the apparatus comprising: means for generating syntax elements based on the video data; means for determining an offset value by, at least in part, applying a binary arithmetic encoding to a syntax element, wherein applying the binary arithmetic encoding comprises, at least in part, binarizing a syntax element; and, for at least one each bin of the bin stream, determining an interval for the next bin of the bin stream based on the state for each bin, the interval for each bin, and the value of each bin; Determining one or more finite state machine (FSM) parameters for the next bin of a bin stream, wherein the one or more FSM parameters for the next bin control how probability estimates for the next bin are calculated from the state for each bin. , determining the one or more FSM parameters; and, the state for the next bin in the bin stream using a parameterized state updating function that takes as input the state for each bin, one or more FSM parameters for the next bin in the bin stream, and the value of each bin. means for determining an offset value, comprising generating the empty stream by determining , wherein the offset value is equal to a value in the interval for the last bin of the empty stream; and means for outputting a bitstream including the offset value.

In another example, the present disclosure describes a computer-readable storage medium storing instructions that, when executed, cause one or more processors to apply binary arithmetic decoding to an offset value included in a bitstream. whereby, as part of determining a decoded syntax element, causing one or more processors to apply the binary arithmetic decoding, execution of the instructions causes the one or more processors to: generate an empty stream, As part of causing one or more processors to generate an empty stream, execution of the instructions causes the one or more processors to: for at least one each bin of the empty stream, a state for each bin, determine the value of each bin based on the interval for and the offset value; determining one or more finite state machine (FSM) parameters for the next bin of the bin stream, wherein the one or more FSM parameters for the next bin control how probability estimates for the next bin are calculated from the state for each bin; , the next bin in the bin stream determines the one or more FSM parameters that follow each bin in the bin stream; Then, the state for the next bin in the bin stream using a parameterized state updating function that takes as input the state for each bin, one or more FSM parameters for the next bin in the bin stream, and the value of each bin. determine: to generate the empty stream; and determine decoded syntax elements, thereby debinarizing the empty stream to form decoded syntax elements; Then, a picture of the video data is reconstructed based partially on the decoded syntax elements.

In another example, the present disclosure describes a computer-readable storage medium storing instructions that, when executed, cause one or more processors to generate a syntax element based on video data; As part of causing one or more processors to apply binary arithmetic encoding, by applying a binary arithmetic encoding to a syntax element, determining an offset value, execution of the instructions causes the one or more processors to: In part, cause the one or more processors to binarize syntax elements; and, for at least one each bin of the bin stream, determine an interval for the next bin of the bin stream based on the state for each bin, the interval for each bin, and the value of each bin; Determining one or more finite state machine (FSM) parameters for the next bin of a bin stream, wherein the one or more FSM parameters for the next bin control how probability estimates for the next bin are calculated from the state for each bin. , determine the one or more FSM parameters; and the state for the next bin in the bin stream using a parameterized state updating function that takes as input the state for each bin, one or more FSM parameters for the next bin in the bin stream, and the value of each bin. generate an empty stream, and determine an offset value, where the offset value is equal to the value in the interval for the last bin of the empty stream; Then, a bitstream including the offset value is output.

The details of one or more aspects of the disclosure are set forth in the accompanying drawings and the detailed description below. Other features, objectives, and advantages of the techniques described in this disclosure will be apparent from the description, drawings, and claims.

Brief description of the drawing
1 is a block diagram illustrating an example video encoding and decoding system that may utilize one or more techniques described in this disclosure.
2 is a block diagram illustrating an example video encoder that may implement one or more techniques described in this disclosure.
3 is a block diagram illustrating an example video decoder that may implement one or more techniques described in this disclosure.
4 is a block diagram of an exemplary general finite state machine.
Figure 5 is an example block diagram for context-based binary arithmetic encoding using many finite state machines (FSMs) for bin probability estimation.
Figure 6 is an example block diagram for context-based binary arithmetic decoding using multiple FSMs for bin probability estimation.
Figure 7 is an example block diagram for context-based binary arithmetic encoding considering a single selected context.
Figure 8 is an example block diagram for context-based binary arithmetic decoding considering a single selected context.
9 is an example block diagram for context-based binary arithmetic encoding, in accordance with one or more aspects of the present disclosure.
10 is an example block diagram for context-based binary arithmetic decoding, in accordance with one or more aspects of the present disclosure.
FIG. 11A is a block diagram showing that FSM parameters can be derived from neighboring blocks (eg, CTUs, CUs) in the same picture along scan order.
FIG. 11B is a block diagram showing that FSM parameters used in blocks of a current picture can be determined based on information associated with blocks in a previously coded picture.
Figure 12 is a block diagram of an example probability estimation filter.
Figure 13 is a block diagram of another example probability estimation filter.
Figure 14 shows an example of a probability estimation filter using cascade filters.
Figure 15 is a block diagram illustrating an example entropy encoding unit.
Figure 16 is a block diagram illustrating an example entropy decoding unit.
17 is a flow chart illustrating example operation of a video encoder, in accordance with one or more techniques of this disclosure.
18 is a flow chart illustrating example operation of a video decoder, in accordance with one or more techniques of this disclosure.

details

Arithmetic coding is often used in video coding to provide data compression. In a typical arithmetic coding process, a video encoder selects a coding context that is associated with initial probability estimates for a first binary symbol and a second binary symbol. The video encoder uses the value of the bin of the empty stream and probability estimates to determine the offset value. Additionally, the video encoder may use a state update function based on the bin value to update probability estimates. The video encoder may then use the next bin value of the bin stream and the updated probability estimates to update the offset value. This process of updating the probability estimates and offset value may continue until the video encoder reaches the end of the empty stream.

Conversely, a video decoder may receive a byte stream containing offset values. The video decoder selects the same coding context as selected by the video encoder and uses the probability estimates specified by the coding context to determine pairs of sub-intervals, each of which corresponds to a different bin value. do. If the offset value is the first sub-interval, the video decoder decodes the first bin value. If the offset value is the second sub-interval, the video decoder decodes the second bin value. The video decoder may then use a state update function to update probability estimates based on the decoded bin value. The video decoder uses the updated probability estimates to re-determine the sub-intervals, where the lower end of the updated lower sub-interval is the lower end of the previous sub-sub-interval if the offset value is within the previous sub-sub-interval. is equal to and is equal to the low end of the previous upper sub-interval if the offset value is within the previous upper sub-interval. The video decoder may continue this process until the video decoder reaches the end of the byte stream. The video decoder may de-binarize the resulting empty stream to determine the values of one or more syntax elements.

In the arithmetic encoding and arithmetic decoding processes described above, the video encoder and video decoder update probability estimates using a state update function. Video encoders and video decoders use the same state update function for all coding contexts when coding all types of syntax elements. However, as described in this disclosure, using different state update functions in different situations may result in improved coding efficiency and bit rate reduction. This, in turn, may result in improved picture quality and/or reduced bandwidth consumption.

In one example of this disclosure, a video encoder generates a syntax element based on video data and determines an offset value by applying binary arithmetic encoding to the syntax element. As part of applying binary arithmetic encoding, a video encoder creates an empty stream by binarizing syntax elements. Additionally, for each bin of at least one bin stream, the video encoder determines, based on the state for that respective bin, the interval for each bin, and the value of each bin, the video encoder for the next bin in the bin stream. Determine the spacing. The video encoder also determines one or more finite state machine (FSM) parameters for the next bin of the bin stream. The one or more FSM parameters for the next bin control how probability estimates for the next bin are calculated from the state for each bin. Additionally, the video encoder may use a parameterized state updating function to determine the state for the next bin of the bin stream. The parameterized state updating function takes as input the state for each bin, one or more FSM parameters for the next bin in the bin stream, and the value of each bin. In this example, the offset value is equal to the value in the interval for the last bin of the bin stream. A video encoder may output a bitstream including an offset value.

In another example of this disclosure, a video decoder may determine the decoded syntax element by applying binary arithmetic decoding equal to the offset value included in the bitstream. As part of applying binary arithmetic decoding, a video decoder may generate an empty stream. As part of generating an empty stream, for at least one each bin of the empty stream, the video decoder may determine the value of each bin. The video decoder may make this decision based on the state for each bin, the interval for each bin, and the offset value. Additionally, the video decoder may determine one or more finite FSM parameters for the next bin of the bin stream. The one or more FSM parameters for the next bin control how probability estimates for the next bin are calculated from the state for each bin. The next bean in the empty stream follows each bean in the empty stream. Additionally, the video decoder determines the state for the next bin of the bin stream using a parameterized state updating function. The parameterized state updating function takes as input the state for each bin, one or more FSM parameters for the next bin in the bin stream, and the value of each bin. A video decoder may debinarize the empty stream to form decoded syntax elements. Additionally, a video decoder may reconstruct a picture of video data based in part on decoded syntax elements.

1 is a block diagram illustrating an example video encoding and decoding system 10 that may utilize the techniques of this disclosure. As shown in FIG. 1 , system 10 includes a source device 12 that provides encoded video data to be later decoded by a destination device 14 . Source device 12 provides encoded video data to destination device 14 via computer-readable medium 16. Source device 12 and destination device 14 may include desktop computers, notebook (i.e., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called “smart” phones, tablet computers, televisions, cameras, etc. may include any of a wide range of devices, including display devices, digital media players, video gaming consoles, video streaming devices, and the like. In some cases, source device 12 and destination device 14 are equipped for wireless communication. Accordingly, source device 12 and destination device 14 may be wireless communication devices. The techniques described in this disclosure may be applied to wireless and/or wired applications. Source device 12 is an example video encoding device (i.e., a device for encoding video data). Destination device 14 is an example video decoding device (i.e., a device for decoding video data).

The illustrated system 10 of Figure 1 is just one example. Techniques for processing video data may be performed by any digital video encoding and/or decoding device. In some examples, the techniques may be performed by a video encoder/decoder, commonly referred to as a “CODEC”. Source device 12 and destination device 12 are examples of such coding devices where source device 14 generates coded video data for transmission to destination device 14. In some examples, source device 12 and destination device 14 operate in a substantially symmetric manner such that each of source device 12 and destination device 14 includes video encoding and decoding components. Therefore, system 10 may support one-way or two-way video transmission between source device 12 and destination device 14, for example, for video streaming, video playback, video broadcasting, or video calling.

In the example of FIG. 1 , source device 12 includes a video source 18, a storage medium 19 configured to store video data, a video encoder 20, and an output interface 22. Destination device 14 includes an input interface 26, a storage medium 28 configured to store encoded video data, a video decoder 30, and a display device 32. In other examples, source device 12 and destination device 14 include other components or arrangements. For example, source device 12 may receive video data from an external video source, such as an external camera. Likewise, destination device 14 may interface with an external display device rather than including an integrated display device.

Video source 18 is a source of video data. Video data may include a series of pictures. Video source 18 may include a video capture device, such as a video camera, a video archive containing previously captured video, and/or a video feed interface for receiving video data from a video content provider. It may also include . In some examples, video source 18 generates computer graphics-based data, or a combination of live video, archived video, and computer-generated video. Storage media 19 may be configured to store video data. In each case, captured, pre-captured, or computer-generated video may be encoded by video encoder 20.

Output interface 22 may output encoded video information to computer-readable medium 16. Output interface 22 may include various types of components or devices. For example, output interface 22 may include a wireless transmitter, a modem, a wired networking component (e.g., an Ethernet card), or other physical component. In examples where output interface 22 includes a wireless transmitter, output interface 22 is configured to transmit data, such as encoded video data, that is modulated according to a cellular communication standard such as 4G, 4G-LTE, LTE Advanced, 5G, etc. It may be configured. In some examples where output interface 22 includes a wireless transmitter, output interface 22 may be configured to modulate in accordance with other wireless standards, such as the IEEE 802.11 specification, IEEE 802.15 specification (e.g., ZigBee™), Bluetooth™ standard, etc. and may be configured to transmit data, such as encoded video data. In some examples, the circuitry of output interface 22 is integrated into the circuitry of other components of video encoder 20 and/or source device 12. For example, video encoder 20 and output interface 22 may be parts of a system on a chip (SoC). The SoC may also include other components such as a general-purpose microprocessor, graphics processing unit, etc.

Destination device 14 may receive encoded video data via computer-readable medium 16. Computer-readable medium 16 may include any type of medium or device capable of moving encoded video data from source device 12 to destination device 14. In some examples, computer-readable medium 16 includes a communication medium to enable source device 12 to transmit encoded video data directly to destination device 14 in real time. Communication media may include any wireless or wired communication medium, such as the radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. Communication media may include routers, switches, base stations, or any other equipment that may be useful to facilitate communication from source device 12 to destination device 14. Destination device 14 may include one or more data storage media configured to store encoded video data and decoded video data.

In some examples, output interface 22 may output data, such as encoded video data, to an intermediate device, such as a storage device. Similarly, input interface 26 of destination device 14 may receive encoded data from an intermediate device. The intermediate device may be a variety of distributed devices such as hard drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded video data. It may include any of a centralized or locally accessed data storage medium. In some examples, the intermediate device corresponds to a file server. Exemplary file servers include web servers, FTP servers, network attached storage (NAS) devices, or local disk drives.

Destination device 14 may access the encoded video data via any standard data connection, including an Internet connection. This may include a wireless channel (eg, Wi-Fi connection), a wired connection (eg, DSL, cable modem, etc.), or a combination of both suitable for accessing encoded video data stored on a file server. Transmission of encoded video data from the storage device may be a streaming transmission, a download transmission, or a combination thereof.

Computer-readable media 16 refers to temporary media such as wireless broadcasts or wired network transmissions, or storage media such as hard disks, flash drives, compact disks, digital video disks, Blu-ray disks, or other computer-readable media (i.e. , non-transitory storage media). In some examples, a network server (not shown) may receive encoded video data from source device 12 and provide the encoded video data to destination device 14, e.g., via network transmission. Similarly, a computing device in a media production facility, such as a disk stamping facility, may receive encoded video data from source device 12 and produce a disk containing the encoded video data. Accordingly, computer-readable medium 16 may, in various examples, be understood to include one or more computer-readable media in various forms.

Input interface 26 of destination device 14 receives data from computer-readable medium 16. Input interface 26 may include various types of components or devices. For example, input interface 26 may include a wireless receiver, a modem, a wired networking component (e.g., an Ethernet card), or other physical component. In examples where input interface 26 includes a wireless receiver, input interface 26 may be configured to receive data, such as a bitstream, modulated according to a cellular communication standard such as 4G, 4G-LTE, LTE Advanced, 5G, etc. It may be possible. In some examples where input interface 26 includes a wireless receiver, input interface 26 may be configured to modulate in accordance with other wireless standards, such as the IEEE 802.11 specification, IEEE 802.15 specification (e.g., ZigBee™), Bluetooth™ standard, etc. It may also be configured to receive data such as a bitstream. In some examples, the circuitry of input interface 26 may be integrated into the circuitry of other components of video decoder 30 and/or destination device 14. For example, video decoder 30 and input interface 26 may be parts of a SoC. The SoC may also include other components such as a general-purpose microprocessor, graphics processing unit, etc.

Storage medium 28 may be configured to store encoded video data, such as encoded video data (e.g., a bitstream) received by input interface 26. Display device 32 displays the decoded video data. Display device 32 may include any of a variety of display devices, such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, organic light emitting diode (OLED) display, or other type of display device. there is.

Video encoder 20 and video decoder 30 each include one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, and firmware. or any combination thereof. When the techniques are implemented in part in software, the device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the techniques of the present disclosure. there is. Each of video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder/decoder (CODEC) in the respective device. .

In some examples, video encoder 20 and video decoder 30 may operate in accordance with a video coding standard. For example, video encoder 20 and video decoder 30 may support ITU-T H.261, ISO/IEC MPEG-1, including Scalable Video Coding (SVC) and Multi-View Video Coding (MVC) extensions. Visual, ITU-T H.262 or ISO/IEC MPEG-2 Visual, ITU-T H.263, ISO/IEC MPEG-4 Visual and ITU-T H.264 (also known as ISO/IEC MPEG-4 AVC) , or may encode and decode video data according to other video coding standards or specifications. In some examples, video encoder 20 and video decoder 30 include High Efficiency Video Coding (HEVC), known as ITU-T H.265, its scope and screen content coding extensions, and its 3D video coding extension ( Encode and decode video data according to 3D-HEVC), its multiview extension (MV-HEVC), or its scalable extension (SHVC).

This disclosure may generally refer to “signaling” some information, such as syntax elements. The term “signaling” may generally refer to the communication of other data and/or syntax elements used to decode encoded video data. Such communication may occur in real time or near-real time. Alternatively, such communication may occur when encoding the syntax elements and storing them as a bitstream on a computer-readable storage medium (which may then be retrieved by a decoding device at any time after storage on the medium). Likewise, it may occur over a time range.

In HEVC and other video coding specifications, video data includes a series of pictures. Pictures may also be referred to as “frames.” A picture may include one or more sample arrays. Each individual sample array in a picture may include an array of samples for an individual color component. A picture may contain three sample arrays, denoted as SL, SCb, and SCr. S _L is a two-dimensional array (i.e. block) of luma samples. S _Cb is a two-dimensional array of Cb chroma samples. S _Cr is a two-dimensional array of Cr chroma samples. In other cases, the picture may be monochrome and may contain only an array of luma samples.

As part of encoding the video data, video encoder 20 may encode pictures of the video data. In other words, video encoder 20 may generate encoded representations of pictures of video data. An encoded representation of a picture may be referred to herein as a “coded picture” or “encoded picture.”

To create an encoded representation of a picture, video encoder 20 may encode blocks of the picture. Video encoder 20 may include encoded representations of video blocks in a bitstream. In some examples, to encode a block of a picture, video encoder 20 performs intra-prediction or inter-prediction to generate one or more predictive blocks. Additionally, video encoder 20 may generate residual data for the block. The residual block contains residual samples. Each residual sample may represent the difference between the generated predictive blocks and the corresponding sample of that block. Video encoder 20 may apply a transform to blocks of residual samples to generate transform coefficients. Video encoder 20 may also quantize the transform coefficients. In some examples, video encoder 20 may generate one or more syntax elements to indicate a transform coefficient. Video encoder 20 may entropy encode one or more of the syntax elements representing transform coefficients.

More specifically, when encoding video data according to HEVC or other video coding specifications to create an encoded representation of a picture, video encoder 20 divides each array of samples in the picture into coding tree blocks (CTBs). You can also partition and encode those CTBs. A CTB may be an NxN block of samples in a picture's sample array. In the HEVC main profile, the size of the CTB can range from 16x16 to 64x64, although technically an 8x8 CTB size may be supported.

A coding tree unit (CTU) of a picture may include one or more CTBs, and may include syntax structures used to encode samples of the one or more CTBs. As an example, each CTU may include a CTB of luma samples, two corresponding CTBs of chroma samples, and syntax structures used to encode the samples of the CTBs. For monochromatic pictures or pictures with three separate color planes, a CTU may include a single CTB, and the syntax structures used to encode samples of that CTB. A CTU may also be referred to as a “tree block” or “largest coding unit” (LCU). In this disclosure, “syntax structure” may be defined as zero or more syntax elements that exist together in a bitstream in a specified order. In some codecs, an encoded picture is an encoded representation that includes all CTUs of the picture.

To encode a CTU of a picture, video encoder 20 may partition the CTU's CTBs into one or more coding blocks. A coding block is an NxN block of samples. In some codecs, to encode a CTU of a picture, video encoder 20 recursively performs quad-tree partitioning on the coding tree blocks of the CTU, dividing the CTBs into coding blocks, thus, a so-called “coding tree.” units”. A coding unit (CU) may include one or more coding tree blocks and syntax structures used to encode samples of the one or more coding tree blocks. For example, a CU may be a luma It may include a coding block of luma samples of a picture with a sample array, a Cb sample array, and a Cr sample array, and two corresponding coding blocks of chroma samples, and syntax structures used to encode the samples of the coding blocks. In monochromatic pictures or pictures with three separate color planes, a CU may include a single coding block and the syntax structures used to code the samples of that coding block.

Additionally, video encoder 20 may encode CUs of a picture of video data. In some codecs, as part of encoding a CU, video encoder 20 may partition a coding block of the CU into one or more prediction blocks. A prediction block is a rectangular (ie, square or non-square) block of samples to which the same prediction is applied. A prediction unit (PU) of a CU may include one or more prediction blocks of the CU and a syntax structure used to predict the one or more prediction blocks. For example, a PU may include a prediction block of luma samples, two corresponding prediction blocks of chroma samples, and syntax structures used to predict the prediction blocks. For monochromatic pictures or pictures with three separate color planes, a PU may include a single prediction block, and the syntax structures used to predict the prediction block.

Video encoder 20 may generate predictive blocks (e.g., luma, Cb, and Cr predictive blocks) for predictive blocks (e.g., luma, Cb, and Cr predictive blocks) of a PU of a CU. It may be possible. Video encoder 20 may generate a predictive block using intra prediction or inter prediction. When video encoder 20 uses intra prediction to generate a predictive block, video encoder 20 may generate the predictive block based on decoded samples of a picture that includes a CU. When video encoder 20 uses inter prediction to generate a predictive block of a PU of a current picture, video encoder 20 generates a PU based on decoded samples of a reference picture (i.e., a picture other than the current picture). Predictive blocks can also be created. In HEVC, video encoder 20 generates a “prediction_unit” syntax structure within a “coding_unit” syntax structure for inter predicted PUs, but a “prediction_unit” syntax structure within a “coding_unit” syntax structure for intra predicted PUs. does not create Rather, in HEVC, syntax elements related to intra predicted PUs are included directly in the “coding_unit” syntax structure.

Video encoder 20 may generate one or more residual blocks for a CU. As an example, video encoder 20 may generate a luma residual block for a CU. Each sample in the CU's luma residual block represents the difference between a luma sample in one of the CU's predicted luma blocks and a corresponding sample in the CU's original luma coding block. Video encoder 20 may also generate a Cb residual block for the CU. Each sample in a Cb residual block of a CU may indicate a difference between a Cb sample in one of the CU's predictive Cb blocks and a corresponding sample in the CU's original Cb coding block. Video encoder 20 may also generate a Cr residual block for the CU. Each sample in a CU's Cr residual block may represent the difference between a Cr sample in one of the CU's predictive Cr blocks and a corresponding sample in the CU's original Cr coding block.

Additionally, video encoder 20 may decompose the residual blocks of the CU into one or more transform blocks. As an example, video encoder 20 may decompose the residual blocks of a CU into one or more transform blocks using quad-tree partitioning. A transform block is a rectangular (eg, square or non-square) block of samples to which the same transform is applied. A transform unit (TU) of a CU may include one or more transform blocks. For example, a TU may include transform blocks of luma samples, two corresponding transform blocks of chroma samples, and syntax structures used to transform the transform block samples. Accordingly, each TU of a CU may have a luma transform block, a Cb transform block, and a Cr transform block. The luma transform block of a TU may be a sub-block of the luma residual block of a CU. The Cb transform block may be a sub-block of the Cb residual block of the CU. The Cr transform block may be a sub-block of the Cr residual block of the CU. For monochromatic pictures or pictures with three separate color planes, a TU may contain a single transform block, and the syntax structures used to transform the samples of that transform block.

Video encoder 20 may apply one or more transforms to the transform block of a TU to generate a coefficient block for the TU. A coefficient block may be a two-dimensional array of transform coefficients. The conversion coefficient may be a scalar quantity. In some examples, one or more transforms transform a transform block from the pixel domain to the frequency domain. Accordingly, in these examples, the transform coefficient may be a scalar quantity that is assumed to be in the frequency domain. A transform coefficient level is an integer quantity that represents the value associated with a particular two-dimensional frequency index in the decoding process prior to scaling for calculation of transform coefficient values.

In some examples, video encoder 20 omits application of transforms to a transform block. In these examples, video encoder 20 may process residual sample values in the same way as transform coefficients. Accordingly, in examples where video encoder 20 omits application of transforms, the following discussion of transform coefficients and coefficient blocks may be applicable to transform blocks of residual samples.

After generating the coefficient block, video encoder 20 may quantize the coefficient block to possibly reduce the amount of data used to represent the coefficient block, potentially providing additional compression. Quantization generally refers to the process by which a range of values is compressed into a single value. For example, quantization may be done by dividing the value by a constant and then rounding to the nearest integer. After quantizing the coefficient block, video encoder 20 may quantize the transform coefficients of the coefficient block. Quantization may reduce the bit depth associated with some or all transform coefficients. For example, n-bit transform coefficients may be rounded down to m-bit transform coefficients during quantization, where n is greater than m. In some examples, video encoder 20 skips quantization.

Video encoder 20 may potentially generate syntax elements representing some or all of the quantized transform coefficients. Video encoder 20 may entropy encode one or more of the syntax elements representing the quantized transform coefficient. For example, video encoder 20 may perform context adaptive binary arithmetic coding (CABAC) on syntax elements indicating quantized transform coefficients. Accordingly, an encoded block (eg, an encoded CU) may include entropy encoded syntax elements representing quantized transform coefficients.

Video encoder 20 may output a bitstream containing encoded video data. In other words, video encoder 20 may output a bitstream containing an encoded representation of video data. An encoded representation of video data may include an encoded representation of pictures of the video data. For example, a bitstream may include a sequence of bits that form a representation of encoded pictures of video data and associated data. In some examples, a representation of an encoded picture may include encoded representations of blocks of the picture.

Video decoder 30 may receive a bitstream generated by video encoder 20. As mentioned above, a bitstream may include an encoded representation of video data. Video decoder 30 may decode the bitstream to reconstruct pictures of video data. As part of decoding the bitstream, video decoder 30 may obtain syntax elements from the bitstream. Video decoder 30 may reconstruct pictures of video data based at least in part on syntax elements obtained from the bitstream. The process for reconstructing pictures of video data may generally be reciprocal to the process performed by video encoder 20.

As an example, as part of decoding a picture of video data, video decoder 30 may use inter prediction or intra prediction to generate predictive blocks. Additionally, video decoder 30 may determine transform coefficients based on syntax elements obtained from the bitstream. In some examples, video decoder 30 inverse quantizes the determined transform coefficients. Inverse quantization maps map quantized values to reconstructed values. As an example, video decoder 30 may inverse quantize a value by determining the value to be multiplied by the quantization step size. Additionally, video decoder 30 may apply an inverse transform to the determined transform coefficients to determine values of residual samples. Video decoder 30 may reconstruct a block of a picture based on the residual samples and corresponding samples of the generated predictive blocks. As an example, video decoder 30 may add residual samples to corresponding samples of the generated predictive blocks to determine reconstructed samples of the block.

More specifically, in HEVC and other video coding specifications, video decoder 30 may use inter-prediction or intra-prediction to generate one or more predictive blocks for each PU of the current CU. Additionally, video decoder 30 may inverse quantize coefficient blocks of TUs of the current CU. Video decoder 30 may perform inverse transforms on coefficient blocks to reconstruct transform blocks of TUs of the current CU. Video decoder 30 may reconstruct a coding block of the current CU based on samples of predictive blocks of the PUs of the current CU and residual samples of transform blocks of the TUs of the current CU. In some examples, video decoder 30 may reconstruct coding blocks of a current CU by adding samples of predictive blocks for PUs of the current CU to corresponding decoded samples of transform blocks of TUs of the current CU. It may be possible. By reconstructing the coding blocks for each CU of the picture, video decoder 30 may reconstruct the picture.

A slice of a picture may contain an integer number of blocks of the picture. For example, in HEVC and other video coding specifications, a slice of a picture may contain an integer number of CTUs in the picture. The CTUs of a slice may be sequentially ordered in the same scan order as the raster scan order. For HEVC, a slice is defined as an integer number of CTUs contained in one independent slice segment and in all subsequent dependent slice segments (if any) that precede the next independent slice segment (if any) within the same access unit. do. Additionally, in HEVC, a slice segment is defined as an integer number of CTUs contained in a single NAL unit and sequentially ordered in a tile scan. A tile scan is a specific sequential ordering of CTBs positioning a picture in which the CTBs are sequentially ordered from tile to CTB raster scan, while the tiles in a picture are sequentially ordered with a raster scan of the tiles of the picture. A tile is a rectangular area of CTBs within a specific tile row and a specific tile row in a picture.

As mentioned above, video encoder 20 ?? Video decoder 30 may apply CABAC encoding and decoding to syntax elements as part of a video coding and compression scheme. To apply CABAC encoding to a syntax element, video encoder 30 may binarize the syntax element to form a series of one or more bits, referred to as “bins.” Video encoder 20 may also identify a coding context. Coding context may identify initial probabilities of bins having specific values. As an example, a coding context may indicate a probability of 0.7 to code a 0-valued bin and a probability of 0.3 to code a 1-valued bin. After identifying the coding context, video encoder 20 may split the interval into a lower sub-interval and an upper sub-interval. One of the sub-intervals may be associated with the value 0 and the other sub-interval may be associated with the value 1. The widths of sub-intervals may be proportional to the probabilities indicated for the values associated with the identified coding context. If a bin of a syntax element has a value associated with a lower sub-interval, the encoded value may be equal to the lower boundary of the lower sub-interval. If the same bin of the syntax element has a value associated with the upper sub-interval, the encoded value may be equal to the lower boundary of the upper sub-interval. To encode the next bin of the syntax element, video encoder 20 may repeat these steps at intervals that are sub-intervals associated with the value of the bit being encoded. When video encoder 20 repeats these steps for the next bin, video encoder 20 may use modified probabilities based on the actual values of the bins being encoded and the probabilities indicated by the identified coding context.

When video decoder 30 performs CABAC decoding on a syntax element, video decoder 30 may identify a coding context. Video decoder 30 may then split the interval into a lower sub-interval and an upper sub-interval. One of the sub-intervals may be associated with the value 0 and the other sub-interval may be associated with the value 1. The widths of sub-intervals may be proportional to the probabilities indicated for the values associated with the identified coding context. If the encoded value is within the lower sub-interval, video decoder 30 may decode the bin with the value associated with the lower sub-interval. If the encoded value is within the upper sub-interval, video decoder 30 may decode the bin with the value associated with the upper sub-interval. To decode the next bin of the syntax element, video decoder 30 may repeat these steps at intervals that are sub-intervals containing the encoded value. When video decoder 30 repeats these steps for the next bin, video decoder 30 may use modified probabilities based on the probabilities indicated by the bins being decoded and the identified coding context. Video decoder 30 may then debinarize the bins to recover the syntax elements.

Video encoder 20 may encode some bins using bypass CABAC coding. It may be computationally less expensive to perform bypass CABAC coding on a bean rather than performing regular CABAC coding on the bean. Additionally, performing bypass CABAC coding may allow for a higher degree of parallelization and throughput. Bins encoded using bypass CABAC coding may be referred to as “bypass bins.” Grouping the bypass bins together allows the Video Encoder (20) ?? The throughput of video decoder 30 may be increased. A bypass CABAC coding engine may be able to code several bins in a single cycle, whereas a regular CABAC coding engine may be able to code only a single bin in a cycle. These bypass CABAC coding engines may be simpler because the bypass CABAC coding engine may not select contexts and assume a probability of 1/2 for both symbols (0 and 1). As a result, in bypass CABAC coding, the intervals are directly split in half.

2 is a block diagram illustrating an example video encoder 20 that may implement the techniques of this disclosure. 2 is provided for illustrative purposes and should not be considered limiting of the techniques as broadly illustrated and described in this disclosure. The techniques of this disclosure may be applicable to various coding standards or methods.

The processing circuitry includes video encoder 20, where video encoder 20 is configured to perform one or more of the example techniques described in this disclosure. By way of example, video encoder 20 includes an integrated circuit, and the various units shown in FIG. 2 may be formed as hardware circuit blocks that are interconnected with a circuit bus. These hardware circuit blocks may be separate circuit blocks, or two or more of the units may be combined into a common hardware circuit block. A hardware circuit block is a combination of electrical components that form an arithmetic block such as an arithmetic logic unit (ALU), an elementary function unit (EFU), as well as logical blocks such as AND, OR, NAND, NOR, XOR, XNOR, and other similar logic blocks. It can also be formed as.

In some examples, one or more of the units illustrated in FIG. 2 may be a software unit executing on processing circuitry. In this example, the object code for these software units is stored in memory. The operating system may cause video encoder 20 to retrieve object code and execute the object code, which causes video encoder 20 to perform operations to implement example techniques. In some examples, the software unit may be firmware that runs when video encoder 20 starts up. Accordingly, video encoder 20 is an architectural component that has hardware that performs example techniques or that specializes in hardware and has software/firmware running on the hardware to perform example techniques.

In the example of FIG. 2 , video encoder 20 includes prediction processing unit 200, video data memory 201, residual generation unit 102, transform processing unit 204, quantization unit 206, and dequantization unit ( 208), an inverse transform processing unit 210, a reconstruction unit 212, a filter unit 214, a decoded picture buffer 216, and an entropy encoding unit 218. Prediction processing unit 200 includes an inter prediction processing unit 220 and an intra prediction processing unit 226. Inter prediction processing unit 220 may include a motion estimation unit and a motion compensation unit (not shown).

Video data memory 201 may be configured to store video data to be encoded by components of video encoder 20. Video data stored in video data memory 201 may be obtained from video source 18, for example. Decoded picture buffer 216 may be a reference picture memory that stores reference video data for use in encoding video data by video encoder 20, such as in intra or inter coding modes. Video data memory 201 and decoded picture buffer 216 may be dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. It may be formed by any of various memory devices such as. Video data memory 201 and decoded picture buffer 216 may be provided by the same memory device or separate memory devices. In various examples, video data memory 201 may be on-chip with or off-chip relative to other components of video encoder 20. Video data memory 201 may be the same as or be a part of storage medium 19 of FIG. 1 .

Video encoder 20 receives video data. Video encoder 20 may encode each CTU in a slice of a picture of video data. Each of the CTUs may be associated with luma coding tree blocks (CTBs) of the same size and the corresponding CTBs of the picture. As part of encoding a CTU, predictive processing unit 200 may perform partitioning, dividing the CTU's CTBs into progressively smaller blocks. Smaller pixel blocks may be coding blocks of CUs. For example, prediction processing unit 200 may partition CTBs associated with CTUs according to a tree structure.

Video encoder 20 may encode the CUs of a CTU to generate encoded representations of the CUs (i.e., coded CUs). As part of encoding a CU, prediction processing unit 200 may partition coding blocks associated with a CU among one or more PUs of the CU. Accordingly, each PU may be associated with a luma prediction block and corresponding chroma prediction blocks. Video encoder 20 and video decoder 30 may support PUs of various sizes. As indicated above, the size of a CU may indicate the size of a luma coding block of the CU and the size of a PU may indicate the size of a luma prediction block of the PU. Assuming that the size of a particular CU is 2Nx2N, video encoder 20 and video decoder 30 may use PU sizes of 2Nx2N or NxN for intra prediction, and symmetries of 2Nx2N, 2NxN, Nx2N, NxN, or similar for inter prediction. Multiple PU sizes may be supported. Video encoder 20 and video decoder 30 may also support asymmetric partitioning for PU sizes of 2NxnU, 2NxnD, nLx2N, and nRx2N for inter prediction.

Inter prediction processing unit 220 may generate predictive data for a PU. As part of generating predictive data for a PU, inter prediction processing unit 220 performs inter prediction for the PU. Predictive data for a PU may include predictive blocks of the PU and motion information for the PU. Inter prediction processing unit 220 may perform different operations on a PU of a CU depending on whether the PU is an I slice, a P slice, or a B slice. In an I slice, all PUs are intra predicted. Accordingly, if the PU is in an I slice, inter-prediction processing unit 220 does not perform inter prediction on the PU. Accordingly, for blocks encoded in I-mode, a predicted block is formed using spatial prediction from previously encoded neighboring blocks within the same frame. If the PU is in a P slice, inter-prediction processing unit 220 may use one-way inter prediction to generate a predictive block of the PU. If the PU is in a B slice, inter-prediction processing unit 220 may perform unidirectional or bidirectional inter prediction to generate a predictive block of the PU.

Intra prediction processing unit 226 may generate predictive data for a PU by performing intra prediction for the PU. Predictive data for a PU may include the PU's predictive blocks and various syntax elements. Intra-prediction processing unit 226 may perform intra prediction on PUs in I slices, P slices, and B slices.

To perform intra prediction for a PU, intra-prediction processing unit 226 may use multiple intra prediction modes to generate multiple sets of predictive data for the PU. Intra prediction processing unit 226 may generate a prediction block for a PU using samples from sample blocks of neighboring PUs. Neighboring PUs may be above, above and to the right, above and to the left, or to the left of a PU, assuming a left-to-right, top-to-bottom encoding order for the PUs, CUs, and CTUs. there is. Intra prediction processing unit 226 may use various numbers of intra prediction modes, for example, 33-way intra prediction modes. In some examples, the number of intra prediction modes may depend on the size of the region associated with the PU.

Prediction processing unit 200 generates a prediction for a PU of a CU among the prediction data generated by inter prediction processing unit 220 for a PU or the prediction data generated by intra prediction processing unit 226 for a PU. You can also select last name data. In some examples, prediction processing unit 200 selects predictive data for PUs of a CU based on rate/distortion metrics of sets of predictive data. Predictive blocks of selected predictive data may be referred to herein as selected predictive blocks.

Residual generation unit 102 generates coding blocks for a CU (e.g., luma, Cb, and Cr coding blocks) and selected predictive blocks for PUs of the CU (e.g., prediction luma, Cb, and Based on the Cr blocks), residual blocks (eg, luma, Cb, and Cr residual blocks) for the CU may be generated. In an example, residual generation unit 102 may generate a residual block of a CU such that each sample in the residual blocks has a value equal to the difference between a sample in a coding block of the CU and the corresponding selected predictive block of a PU of the CU. create them.

Transform processing unit 204 may partition the residual blocks of a CU into transform blocks of the TUs of the CU. As an example, transform processing unit 204 may perform quad tree partitioning, partitioning the residual blocks of a CU into transform blocks of TUs of the CU. Accordingly, a TU may be associated with a luma transform block and two chroma transform blocks. The sizes and positions of the luma and chroma transform blocks of the TUs of a CU may or may not be based on the sizes and positions of the prediction blocks of the PUs of the CU. A quad-tree structure, known as a “residual quad-tree” (RQT), may contain nodes associated with each region. TUs of a CU may correspond to leaf nodes of an RQT.

Transform processing unit 204 may generate transform coefficient blocks for each TU of a CU by applying one or more transforms to transform blocks of the TU. Transform processing unit 204 may apply various transforms to the transform block associated with a TU. For example, transform processing unit 204 may apply a discrete cosine transform (DCT), a directional transform, or a conceptually similar transform to the transform block. In some examples, transform processing unit 204 does not apply transforms to the transform block. In such examples, the transform block may be treated as a transform coefficient block.

Quantization unit 206 may quantize the transform coefficients in a coefficient block. Quantization unit 206 may quantize a coefficient block associated with a TU of a CU based on a quantization parameter (QP) value associated with the CU. Video encoder 20 may adjust the degree of quantization applied to coefficient blocks associated with a CU by adjusting the QP value associated with the CU. Quantization may result in loss of information. Therefore, quantized transform coefficients may have lower accuracy than the original ones.

Inverse quantization unit 208 and inverse transform processing unit 210 may apply inverse quantization and inverse transforms, respectively, to the coefficient block to reconstruct a residual block from the coefficient block. Reconstruction unit 212 may generate a reconstructed transform block associated with a TU by adding the reconstructed residual block to corresponding samples from one or more prediction blocks generated by prediction processing unit 200. By reconstructing the transform blocks for each TU of a CU in this way, video encoder 20 may reconstruct the coding blocks of the CU.

Filter unit 214 may perform one or more deblocking operations to reduce blocking artifacts in coding blocks associated with a CU. Decoded picture buffer 216 may store reconstructed coding blocks after filter unit 214 performs one or more deblocking operations on the reconstructed coding blocks. Inter prediction processing unit 220 may use a reference picture containing reconstructed coding blocks to perform inter prediction on PUs of other pictures. Additionally, intra-prediction processing unit 226 may use the reconstructed coding blocks in decoded picture buffer 216 to perform intra prediction on other PUs in the same picture as the CU.

Entropy encoding unit 218 may receive data from other functional components of video encoder 20. For example, entropy encoding unit 218 may receive coefficient blocks from quantization unit 206 and syntax elements from prediction processing unit 200. Entropy encoding unit 218 may perform one or more entropy encoding operations on data to generate entropy encoded data. For example, entropy encoding unit 218 may perform a CABAC operation or another type of entropy encoding operation on data. Video encoder 20 may output a bitstream containing entropy encoded data generated by entropy encoding unit 218. As an example, the bitstream may include data representing values of transform coefficients for a CU.

3 is a block diagram illustrating an example video decoder 30 configured to implement the techniques of this disclosure. 3 is provided for illustrative purposes and is not limiting to the techniques broadly illustrated and described in this disclosure. For purposes of explanation, this disclosure describes video decoder 30 in the context of HEVC coding. However, the techniques of this disclosure may be applicable to other coding standards or methods.

The processing circuitry includes video decoder 30, where video decoder 30 is configured to perform one or more of the example techniques described in this disclosure. By way of example, video decoder 30 includes an integrated circuit, and the various units shown in FIG. 3 may be formed as hardware circuit blocks that are interconnected with a circuit bus. These hardware circuit blocks may be separate circuit blocks, or two or more of the units may be combined into a common hardware circuit block. A hardware circuit block is a combination of electrical components that form an arithmetic block such as an arithmetic logic unit (ALU), an elementary function unit (EFU), as well as logical blocks such as AND, OR, NAND, NOR, XOR, XNOR, and other similar logic blocks. It can also be formed as.

In some examples, one or more of the units illustrated in FIG. 3 may be a software unit executing on processing circuitry. In this example, the object code for these software units is stored in memory. The operating system may cause video decoder 30 to retrieve object code and execute the object code, which causes video decoder 30 to perform operations to implement the example technique. In some examples, the software unit may be firmware that runs when video decoder 30 starts up. Accordingly, video decoder 30 is an architectural component that has hardware that performs example techniques or that specializes in hardware and has software/firmware running on the hardware to perform example techniques.

In the example of FIG. 3 , video decoder 30 includes an entropy decoding unit 300, a video data memory 301, a prediction processing unit 302, an inverse quantization unit 304, an inverse transform processing unit 306, and a reconstruction unit. 308, a filter unit 310, and a decoded picture buffer 312. Prediction processing unit 302 includes motion compensation unit 314 and intra prediction processing unit 316. In other examples, video decoder 30 may include more, fewer, or different functional components.

Video data memory 301 may store encoded video data, such as an encoded video bitstream, to be decoded by components of video decoder 30. Video data stored in video data memory 301 may be transmitted, for example, from a computer-readable medium 16, for example, from a local video source such as a camera, via wired or wireless network communication of video data, or from a physical It may also be obtained by accessing a data storage medium. Video data memory 301 may form a coded picture buffer (CPB) that stores encoded video data from an encoded video bitstream. Decoded picture buffer 312 is a reference picture that stores reference video data for use in decoding video data by video decoder 30, e.g., in intra- or inter-coding modes or for output. It could be memory. Video data memory 301 and decoded picture buffer 312 may be dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. It may be formed by any of various memory devices such as. Video data memory 301 and decoded picture buffer 312 may be provided by the same memory device or separate memory devices. In various examples, video data memory 301 may be on-chip with or off-chip relative to other components of video decoder 30. Video data memory 301 may be the same as or be a part of storage medium 28 of FIG. 1 .

Video data memory 301 receives and stores encoded video data (e.g., NAL units) of a bitstream. Entropy decoding unit 300 may receive encoded video data (e.g., NAL units) from video data memory 301 and parse the NAL units to obtain syntax elements. Entropy decoding unit 300 may entropy decode entropy encoded syntax elements in NAL units. Prediction processing unit 302, inverse quantization unit 304, inverse transform processing unit 306, reconstruction unit 308, and filter unit 310 decode video data based on syntax elements extracted from the bitstream. You can also create . Entropy decoding unit 300 may perform a generally opposite process to entropy encoding units.

In addition to obtaining syntax elements from the bitstream, video decoder 30 may perform a reconstruction operation on a non-partitioned CU. To perform a reconstruction operation on a CU, video decoder 30 may perform a reconstruction operation on each TU of the CU. By performing a reconstruction operation on each TU of a CU, video decoder 30 may reconstruct the residual blocks of the CU.

As part of performing a reconstruction operation on a TU of a CU, inverse quantization unit 304 may inverse quantize, or dequantize, the coefficient blocks associated with the TU. After inverse quantization unit 304 inverse quantizes the coefficient block, inverse transform processing unit 306 may apply one or more inverse transforms to the coefficient block to generate a residual block associated with a TU. For example, inverse transform processing unit 306 may apply an inverse DCT, inverse integer transform, inverse Karhunen-Loeve transform (KLT), inverse rotation transform, inverse directional transform, or other inverse transform to the coefficient block.

Inverse quantization unit 304 may perform certain techniques of this disclosure. For example, for at least one each quantization group of a plurality of quantization groups within a CTB of a CTU of a picture of video data, inverse quantization unit 304 may perform at least in part based on local quantization information signaled in the bitstream. , each quantization parameter for each quantization group may be derived. Additionally, in this example, inverse quantization unit 304 may inverse quantize at least one transform coefficient of a transform block of a TU of a CU of a CTU based on the respective quantization parameters for each quantization group. In this example, each quantization group consists of consecutive Cus or coding blocks in coding order such that the boundaries of each quantization group must be boundaries of Cus or coding blocks and the size of each quantization group is greater than or equal to a threshold. Defined as a group. Video decoder 30 (e.g., inverse transform processing unit 306, reconstruction unit 308, and filter unit 310) may reconstruct a coding block of a CU based on the inverse quantized transform coefficients of the transform block. It may be possible.

When a PU is encoded using intra prediction, intra prediction processing unit 316 may perform intra prediction to generate predictive blocks of the PU. Intra prediction processing unit 316 may use an intra prediction mode to generate prediction blocks of a PU based on spatially neighboring blocks samples. Intra prediction processing unit 316 may determine an intra prediction mode for a PU based on one or more syntax elements obtained from the bitstream.

If the PU is encoded using inter prediction, entropy decoding unit 300 may determine motion information for the PU. Motion compensation unit 314 may determine one or more reference blocks based on the motion information of the PU. Motion compensation unit 314 may generate prediction blocks (e.g., prediction luma, Cb, and Cr blocks) for a PU based on one or more reference blocks.

Reconstruction unit 308 reconstructs the transform blocks (e.g., luma, Cb, and Cr coding blocks) for TUs of the CU, as applicable, to reconstruct the coding blocks (e.g., luma, Cb, and Cr coding blocks) for the CU. , luma, Cb, and Cr transform blocks) and prediction blocks of PUs of a CU (e.g., luma, Cb, and Cr blocks), that is, either intra prediction data or inter prediction data may be used. For example, reconstruction unit 308 may use transform blocks (e.g., luma, Cb, and Cr transform blocks) to reconstruct coding blocks (e.g., luma, Cb, and Cr coding blocks) of a CU. Samples of may be added to corresponding samples of predictive blocks (e.g., luma, Cb, and Cr predictive blocks).

Filter unit 310 may perform a deblocking operation to reduce blocking artifacts associated with coding blocks of a CU. Video decoder 30 may store a coding block of a CU in decoded picture buffer 312. Decoded picture buffer 312 may provide reference pictures for subsequent motion compensation, intra prediction, and presentation on a display device, such as display device 32 of FIG. 1. As an example, video decoder 30 may perform intra-prediction or inter-prediction operations on PUs of different CUs, based on blocks in decoded picture buffer 312.

Arithmetic coding is a fundamental tool used in data compression. For example, “I. H. Witten, R. M. Neal, and J. G. Cleary, “Arithmetic coding for data compression,” Commun. ACM, vol. 30, no. 6, pp. 520-540, June 1987 (hereinafter “Reference 1”); A. Said, “Arithmetic Coding,” in “Lossless Compression Handbook,” K. Sayood, Ed., Academic Press, chapter 5, pp. 101-152, 2003 (hereinafter “Reference 2”); and A. Said, “Introduction to arithmetic coding - theory and practice,” Hewlett Packard Laboratories, Palo Alto, CA, USA, Technical Report HPL-2004-76, Apr. 2004, (http://www.hpl.hp.com/techreports/2004/HPL-2004-76.pdf) (hereinafter referred to as “Reference 3”).

Arithmetic coding was optional in the AVC/H.264 video compression standard. 「I. D. Marpe, H. Schwarz, and T. Wiegand, “Context-based adaptive binary arithmetic coding in the H.264/AVC video compression standard,” IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 620-636, Jul. 2003 (hereinafter “Reference 4”); See I. E. Richardson, The H.264 Advanced Video Compression Standard, 2nd ed., John Wiley and Sons Ltd., 2010 (hereinafter “Reference 5”).

Arithmetic coding has become the only entropy coding technique in video coding standards HEVC/H.265 and VP9. 「V. Sze and M. Budagavi, “High throughput CABAC entropy coding in HEVC,” IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 12, pp. 1778-1791, Dec. 2012 (hereinafter “Reference 6”); V. Sze and D. Marpe, “Entropy coding in HEVC,” in High Efficiency Video Coding (HEVC): Algorithms and Architectures, V. Sze, M. Budagavi, and G. J. Sullivan , Eds., chapter 8, pp. 209-274. Springer, 2014 (hereinafter “Reference 7”); M. Wien, High Efficiency Video Coding: Coding Tools and Specification, Springer-Verlag, 2015 (hereinafter “Reference 8”); D. Mukherjee, J. Bankoski, R. S. Bultje, A. Grange, J. Han, J. Koleszar, P. Wilkins, and Y. Xu, “The latest open-source video codec VP9 - an overview and preliminary results,” in Proc. 30th Picture Coding Symp., San Jose, CA, Dec. 2013 (hereafter, “References Refer to Document 9”)”.

Because of the superior compression efficiency of arithmetic coding, arithmetic coding is expected to remain the only entropy coding technique used in future video coding standards. However, one of the main problems with using entropy coding in real applications is that while the most effective methods are designed to be optimal for static data sources, real data from complex signals (such as video) are different from static ones. It is far. Current solutions use data classification and adaptive coding methods to solve this problem, and the techniques of this disclosure may increase the efficiency of adaptation techniques.

The techniques of this disclosure improve compression efficiency by taking advantage of the fact that even when data is finitely divided into many classes (coding contexts), there is still a lot of diversity in the statistics of the data assigned to each class. It can also be improved. Therefore, instead of using a single “universal” adaptation technique for all classes, the present disclosure changes the adaptation parameters for each class and, within each class, adjusts the expected or observed It is proposed to further change the adaptation parameters depending on the measured variations in probability values, or estimates.

Additionally, this disclosure describes example techniques for improving binary probability estimation that is an adaptation to binary arithmetic coding. The example techniques may enable better compression, lower complexity and stability when using low-accuracy implementations using discrete-time signal analysis to improve methods for estimating probabilities using recursive equations. These are special features that make it possible.

As described in references 1, 2, and 3, modern video coding standards adopt a strategy of decomposing entropy coding into modeling and actual coding. Accordingly, the binary arithmetic coding process used in modern video compression standards is split into three main stages. Video encoder 20 may perform the operations of these stages, and video decoder 30 may perform the inverse operations of these stages.

(a) Binarization: Each data element (or syntax element) to be coded is first decomposed into a sequence of binary data symbols (bins). Because binary symbol probabilities depend on the data element and binary symbol position in its decomposition, an empty context (or simply context) is assigned for each type of binary symbol, uniquely identifying the probability estimate to be used for its entropy coding. do.

(b) Adaptive probability estimation: Since all bins assigned to a given context are assumed to have similar, but not exactly identical, probabilities, the encoder and decoder are based on previously encoded or decoded bin values. to update their probability estimates.

(c) Arithmetic coding: The value of each binary symbol (0 or 1) is entropy coded using the estimated probability of its value defined by the bin's corresponding context.

Techniques of this disclosure may achieve better compression by improving adaptive probability estimation.

Exemplary techniques used in practice in binary probability estimation are presented in the following references:

F. T. Leighton 및 R. L. Rivest, “Estimating a probability using finite memory,” IEEE Trans. Inf. Theory, vol. 32, no. 6, pp. 733-742, Nov. 1986 (이하, "참조문헌 10”).

FT Leighton and RL Rivest, “Estimating a probability using finite memory,” IEEE Trans. Inf. Theory, vol. 32, no. 6, pp. 733-742, Nov. 1986 (hereinafter referred to as “Reference 10”).

WB Pennebaker and JL Mitchell, “Probability estimation for the Q-Coder,” IBM J. Res. Theory, vol. 32, no. 6, pp. 737-752, Nov. 1988 (hereinafter referred to as “Reference 11”).

PG Howard and JS Vitter, “Practical implementations of arithmetic coding,” in Image and Text Compression, JA Storer, Ed., chapter 4, pp. 85-112. Kluwer Academic Publishers, Norwell, MA, 1992 (hereinafter referred to as “Reference 12”).

E. Meron and M. Feder, “Finite-memory universal prediction of individual sequences,” IEEE Trans. Inf. Theory, vol. 50, no. 7, pp. 1506-1523, Nov. 2004 (hereinafter referred to as “Reference 13”).

E. Belyaev, M. Gilmutdinov, and A. Turlikov, “Binary arithmetic coding system with adaptive probability estimation by `virtual sliding window'," in Proc. IEEE Int. Symp. Consumer Electronics, St. Petersburg, Russia, June 2006 ( Hereinafter, “Reference 14”).

A. Alshin, E. Alshina, and J.-H. Park, “High precision probability estimation for CABAC,” in Proc . IEEE Visual Commun . Image Process. Conf . , Kuching, Malaysia, Nov. 2013 (hereinafter referred to as “Reference 15”).

A. V. Oppenheim and R. W. Schafer, Discrete-Time Signal Processing, Prentice-Hall, Inc., Upper Saddle River, NJ, 3rd ed., Aug. 2009 (hereinafter referred to as “Reference 16”).

SK Mitra, Digital Signal Processing: A Computer-based Approach, McGraw-Hill Publishing Co., New York, NY, 4th ed., 2010 (hereinafter referred to as “Reference 17”).

To achieve practical requirements of very low computational complexity, probability estimation is typically done using some type of finite-state-machine (FSM). For ease of explanation of the techniques of this disclosure, this disclosure does not cover details regarding the implementation of the FSMs of References 10-14, but it is useful to define appropriate terminology, and some examples are provided at the end of this section. presented.

Figure 4 is a block diagram of an exemplary generic FSM 400. More specifically, Figure 4 is a graphical representation of the sequence of state transitions, inputs, and outputs of a typical finite state machine. In Figure 4, sequences represents the states, inputs, and outputs of FSM 400, which are vectors with integer elements (the vector dimensions and their sets of allowed element values should be well-defined, but are not important for the present discussion). Each is indicated.

Following the above definitions, the arrows in the diagram of Figure 4 represent the state updating equation and the output equation, which are

(One)

, where T is the state updating function and P is the output function.

In probability estimation FSMs, the inputs are empty values and the outputs are empty probability estimates. The use of FSMs during entropy encoding and decoding is shown in Figures 5, 6, 7, and 8, where, by convention, the binary probability estimation FSM is called the coding context.

As shown in the examples of FIGS. 5 and 6, in practical video coding, an entropy coding stage may be implemented using a large number of coding contexts. Coding contexts are selected during encoding and decoding depending on the type (or class) of bin being encoded or decoded.

Figure 7 is an example block diagram for context-based binary arithmetic encoding considering a single selected context. In FIG. 7 , arithmetic encoding unit 700 includes arithmetic encoder 702, bit buffer 704, state determination unit 706, and probability estimation FSM unit 708. Arithmetic encoding unit 700 may, in some examples, receive an empty stream from binarization unit 712. Arithmetic encoding unit 700 and may form part of entropy encoding unit 218 of FIG. 2 . Binarization unit 712 encodes each data element (e.g., syntax element) into a sequence of binary data symbols (bins). A sequence of binary data symbols may be referred to as a “bin stream.” Additionally, arithmetic encoding unit 700 may receive a context reinitialization signal. As an example, arithmetic encoding unit 700 may receive a context reinitialization signal when arithmetic encoding unit 700 begins encoding a different type of binary symbol.

Additionally, in FIG. 7 , in response to receiving a context reinitialization signal, state determination unit 706 may reinitialize the state of the probability estimation FSM. In general, reinitialization refers to resetting the probability estimates to the initial probability estimates associated with the coding context. For example, based on the type of binary symbol to be encoded, state determination unit 706 may look up initial probability estimates in a predefined table. Predefined tables may be defined by video coding standards such as HEVC. State determination unit 706 may provide the determined initial probability estimates to empty probability estimation FSM unit 708. For the first bin of the bin stream, bin probability estimation FSM unit 708 provides initial probability estimates to arithmetic encoder 702. Additionally, bin probability estimation FSM unit 708 updates probability estimates based on the actual value of the first bin of the bin stream. For each subsequent bin until state determination unit 706 resets the probability estimates, bin probability estimation FSM unit 708 generates probability estimates according to a state updating function, e.g., as shown in equation (1). Update.

For each bin in the bin stream, arithmetic encoder 702 may use probability estimates provided by bin probability estimation FSM unit 708 to encode the bin as described elsewhere in this disclosure for CABAC. there is. Bit buffer 704 may store bins encoded by arithmetic encoder 702. In FIG. 7 , delay box 710 means that the bin probabilities generated by bin probability estimation FSM unit 708 are based on the bin preceding the bin currently being encoded by arithmetic encoder 702.

Figure 8 is an example block diagram for context-based binary arithmetic decoding considering a single selected context. In FIG. 8 , arithmetic decoding unit 800 includes an arithmetic decoder 802, a bit buffer 804, a state determination unit 806, and a probability estimation FSM unit 808. Arithmetic decoding unit 800 generates an empty stream that, in some examples, may be received by inverse binarization unit 812. Arithmetic decoding unit 800 and inverse binarization unit 812 may form part of entropy decoding 300 of FIG. 3 . Debinarization unit 812 converts the empty stream into a series of one or more syntax elements.

8, arithmetic decoding unit 800 receives a byte stream that may be parsed from a bitstream received by video decoder 30. Additionally, arithmetic decoding unit 800 may receive a context reinitialization signal. As an example, arithmetic decoding unit 800 may receive a context reinitialization signal when arithmetic decoding unit 800 begins encoding a different type of binary symbol. Additionally, in FIG. 8, in response to receiving a context reinitialization signal, state determination unit 806 may reinitialize the state of the probability estimation FSM. For example, based on the type of binary symbol to be encoded, state determination unit 806 may look up initial probability estimates in a predefined table. Predefined tables may be defined by video coding standards such as HEVC. The predefined table may be the same table used by state determination unit 706 (FIG. 7). State determination unit 806 may provide the determined initial probability estimates to empty probability estimation FSM unit 808. For the first bin of the bin stream, bin probability estimation FSM unit 808 provides initial probability estimates to arithmetic decoder 802. Additionally, bin probability estimation FSM unit 808 updates probability estimates based on the actual value of the first bin of the bin stream. For each subsequent bin until state determination unit 806 resets the probability estimates, bin probability estimation FSM unit 808 generates probability estimates according to a state updating function, e.g., as shown in equation (1). Update.

For each bin of the bin stream, arithmetic decoder 802 may use probability estimates provided by bin probability estimation FSM unit 808 to decode the bin as described elsewhere in this disclosure for CABAC. there is. Bit buffer 804 may store bins to be decoded by arithmetic decoder 802. In FIG. 8, delay box 810 means that the bin probabilities generated by bin probability estimation FSM unit 808 are based on the bin preceding the bin currently being decoded by arithmetic decoder 802.

Figures 7 and 8 are simplified diagrams considering an example case in which a single context is selected. Figures 7 and 8 also illustrate one feature that is always present in real applications, and that is the need to periodically resynchronize the encoder and decoder states using a shared table with data converted to encoder states. For example, in the HEVC standard, contexts determine how, for each context, compression-quality parameters (known as quantization steps, or quantization parameter (QP) values) (see references 7 and 8) are converted into FSM states. It is a table that defines conversion and is periodically reinitialized.

In the HEVC standard, FSM functions are implemented using only table look-up methods. See references 7 and 8. In a recent draft of the Joint Exploration Model (JEM) produced by ITU-T/MPEG JVET, the FSM was implemented using two types of discrete-time infinite impulse response (IIR) filters. at first:

of the form (see references 13 and 14), where: is an integer sequence of scaled probability estimates (FSM states and outputs), and b is a binary sequence of empty values (FSM inputs), and a is a positive integer allowing multiplication and division to be implemented by bit shifting.

The second form is “A. Alshin, E. Alshina, and J.-H. Park, “High precision probability estimation for CABAC,” in Proc. IEEE Visual Commun. Image Process. Conf., Kuching, Malaysia, Nov. 2013” (hereinafter referred to as “Reference 15”), and is defined by the following equations:

(2)

In this case, the probability estimation FSM inputs and outputs are still respectively sequences and , but the states are pairs is defined by The estimate here is, Since both are probability estimates that are updated and stored, they can also be called “two-track estimates.”

Because entropy coding was designed to be optimal for static data sources, its practical effectiveness depended on classifying data elements, and thus the statistics in each class were approximately static, almost “universal”. That is, it is possible to use probability estimation FSMs that are equally well adapted to all data elements. As shown in Figures 7 and 8, based on that assumption, the probability estimation FSMs are not changed, and only their states are periodically reinitialized.

This disclosure addresses the fact that “universally optimal” probability estimation FSMs for use in CABAC do not actually exist, because not only the probability change for each context, but also the rate and magnitude of the changes differ between contexts. . This disclosure describes a solution to this problem that takes advantage of the fact that variations occur depending on either context or estimated probability, but the best strategy is to use both to determine the optimal FSM. The techniques of this disclosure also cover the option of picking FSM parameters by evaluating changes in FSM states, such as measuring the difference between more than one estimate.

In accordance with one or more techniques of this disclosure, the conventional definition of probability estimation FSMs is modified, and the techniques of this disclosure also define an FSM parameter vector h, which can be used to vary the FSM responses. With this definition, equation (1) can be written as:

(3)

Here, T is a parameterized state updating function and P is a parameterized output function. In other words, the state updating and output equations may be redefined as shown in equation (3). These probability estimation FSMs may be referred to herein as “parameterized-context FSMs.”

With this definition, two factors can be identified that define the coding performance of a parameterized context FSM:

FSM states (): 이진 심볼들의 확률들을 계산하기 위해 직접 사용되는 수치적 또는 논리적 정보를 포함하고, 오직 이전의 표준드에서 재초기화에 의해 변경된 데이터이다;

FSM states ( ): Contains numerical or logical information used directly to calculate the probabilities of binary symbols, and is data that has only been changed by reinitialization from a previous standard;

FSM parameters ( ): defines state updating, and how probability estimates are computed from the states; The present invention improves compression by modifying these parameters during coding or reinitialization.

For example, the probability estimate in equation (2) is equivalent to the equations:

(4)

It can be changed to use positive integers ( a, b ) as parameters in . Since parameters a and b are used to determine the estimated probability for the next bin (i.e., p[ k + 1]), parameters a and b in equation (4) are to be considered as for the next bin. It may be possible. In equation (4), the parameters (a, b) may vary state transitions, but are not the output equation. This distinction is used because although it is mathematically possible to define FSM parameters as part of the FSM state, it represents a practical difference.

9 and 10 illustrate how the techniques of this disclosure are integrated in an arithmetic coding process. In the example of FIG. 9 , arithmetic encoding unit 900 includes arithmetic encoder 902, bit buffer 904, state determination unit 908, probability estimation FSM unit 912, and FSM parameter determination unit 912. do. Arithmetic encoding unit 900 may, in some examples, receive an empty stream from binarization unit 913. Arithmetic encoding unit 900 and binarization unit 913 may form part of entropy encoding unit 218 of FIG. 2 . Binarization unit 913, arithmetic encoder 902, bit buffer 904, state determination unit 908, and delay box 910 are similar to binarization unit 713, arithmetic encoder 702, and bit buffer in FIG. 704, state determination unit 706, and delay box 710 may operate in a similar manner.

In the example of FIG. 10 , arithmetic decoding unit 1000 includes arithmetic decoder 1002, bit buffer 1004, state determination unit 1006, probability estimation FSM unit 1008, and FSM parameter determination unit 1012. do. Arithmetic decoding unit 1000, in some examples, generates an empty stream that may be received by inverse binarization unit 1013. Arithmetic decoding unit 1000 and inverse binarization unit 1013 may form part of entropy decoding 300 of FIG. 3 . Debinarization unit 1013 converts the empty stream into a series of one or more syntax elements. Binarization unit 1013, arithmetic decoder 1002, bit buffer 1004, state determination unit 1006, and delay box 1010 are similar to inverse binarization unit 812, arithmetic decoder 802, and bit in FIG. Buffer 804, state determination unit 806, and delay box 810 may operate in a similar manner.

The main difference from FIGS. 7 and 8 (shown with dashed lines in FIGS. 9 and 10) is the FSM parameter determination units 912, 1012 included in FIGS. 9 and 10. FSM parameter determination units 912, 1012 determine FSM parameters (e.g., a and b in equation (4)). FSM parameter determination units 912, 1012 may determine FSM parameters during coding, in response to context reinitialization events, or in other situations. Accordingly, in FIGS. 9 and 10, parameterized context FSMs may be modified during coding or reinitialization by FSM parameter determination units 912, 1012 using probability values, quality factors, and other data. It may be possible. Data supplied to the FSM parameter determination units 912, 1012 may include or consist of reinitialization parameters, and also current states (eg, bin probabilities).

Accordingly, in accordance with one or more techniques of this disclosure, video encoder 20 may receive video data. Video data may include one or more pictures. Additionally, prediction processing unit 200, quantization unit 206, and potentially other components of video encoder 20 may generate syntax elements based on the video data. In this example, entropy encoding unit 218 may determine the offset value by applying binary arithmetic encoding to one of the generated syntax elements. As part of applying binary arithmetic encoding, entropy encoding unit 218 may generate an empty stream by binarizing one or more syntax elements. Additionally, for at least one respective bin of the empty stream (e.g., a specific bin of the empty stream, each bin of the empty stream, each bin of the empty stream other than the last bin of the empty stream, etc.), an entropy encoding unit 218 ) may determine the interval for the next bin of the bin stream based on the state for each bin, the interval for each bin, and the value of each bin. Additionally, entropy encoding unit 218 may determine one or more FSM parameters for the next bin of the bin stream. Entropy encoding unit 218 may also determine the state for the next bin of the bin stream based on the state for each bin, one or more FSM parameters for the next bin of the bin stream, and the value of each bin. there is. In this example, the offset value may be equal to the value in the interval for the last bin of the bin stream. Video encoder 20 may output a bitstream that includes the offset value.

Additionally, according to one or more techniques of this disclosure, entropy decoding unit 300 of video decoder 30 may determine a decoded syntax element by applying binary arithmetic decoding to an offset value included in the bitstream. As part of applying binary arithmetic decoding, entropy decoding unit 300 may generate an empty stream. As part of generating the empty stream, entropy decoding unit 300 may decode at least one respective bin of the empty stream (e.g., a specific bin of the empty stream, each bin of the empty stream, bins other than the last bin of the empty stream). For each bin of the stream, etc.), the value of each bin may be determined based on the state for each bin, the interval for each bin, and the offset value. Additionally, entropy decoding unit 300 may determine one or more FSM parameters for the next bin of the bin stream. The next bean in the empty stream follows each bean in the empty stream. Additionally, entropy decoding unit 300 determines the state for the next bin of the bin stream based on the state for each bin, one or more FSM parameters for the next bin of the bin stream, and the value of each bin. It may be possible. Entropy decoding unit 300 may debinarize the empty stream to form decoded syntax elements. Other components of video decoder 30 may reconstruct a picture of the video data based in part on the decoded syntax elements. In other words, video decoder 30 may use the decoded syntax elements in the process to reconstruct the picture.

In some examples, FSM parameter determination units 912, 1012 use different functions for different contexts. Illustratively, each context may use a different function in FSM parameter determination units 912, 1012. FSM parameter determination units 912, 1012 use a function on context to determine FSM parameters used by bin probability estimation FSM units 908, 1008. As an example, each of FSM parameter determination units 912, 1012 may access a predefined table that maps different contexts to different FSM parameters. In another example, different contexts may be associated with different predefined tables. In this example, each of FSM parameter determination units 912, 1012 accesses a predefined table associated with the current context and creates an entry in the predefined table associated with the current context based on one or more additional pieces of information. You can also find it. In this example, additional pieces of information may include information about the coding modes of neighboring blocks, information about the position of the last non-zero coefficient, etc. In some examples, a function to a context is a mapping from the context to predetermined values of FSM parameters. Accordingly, in some examples, each of FSM parameter determination units 912, 1012 is comprised of a table that maps contexts to predetermined values of FSM parameters.

In some examples, even for the same context, the FSM parameters (e.g., (a, b) for a 2-track arithmetic core) may also include slice types and/or quantization parameters, and/or coded mode information (e.g. , for example, any type of information that is coded as side information to the decoder in the slice header). For example, each of the FSM parameter determination units 912, 1012 may have a table mapping combinations of (factors (e.g., context, slice types, quantization parameters, coded mode information, etc.) to FSM parameters. It may be composed of:

FSM parameter determination units 912, 1012 may modify FSM parameters in various ways. In other words, FSM parameter determination units 912, 1012 may change which FSM parameters are used by bin probability estimation FSM unit 908 in response to various events and based on various types of information. Some possibilities are listed below, which may be applicable for each context.

In one example, FSM parameters are modified during reinitialization depending on state reinitialization parameters such as quantization step or QP value. For example, a video coder may be configured with a predefined table that maps values of state reinitialization parameters to FSM parameters. In this example, a video coder may use the table to find FSM parameters based on state reinitialization parameters.

In some examples, FSM parameters are modified according to estimated probability values during reinitialization, or at periodic intervals shorter than reinitialization (eg, for each CTU, after each bin, etc.). For example, a video coder may be configured with a predefined table that maps estimated probability values to FSM parameters. In this example, a video coder may use the table to find FSM parameters based on the estimated probability values. In this example, a video coder may use the table to find FSM parameters during context reinitialization. For example, a video coder may use probability estimates specified by the coding context to find FSM parameters during context reinitialization. Moreover, in some examples, bin probability estimation FSM units 908 or 1008 may determine probability estimates for bins that follow the first bin after context reinitialization. In such examples, a video coder may use the probability estimates determined by bin probability estimation FSM units 908 or 1008 to find FSM parameters in a predefined table.

In some examples, FSM parameters are modified based on measurement of past probability variation. A video coder may calculate a measure of past probability variation by summing the absolute differences between probabilities. In different examples, a video coder may estimate probabilities using different techniques. In one example, using an FSM defined by equation (4), a video coder may use an “estimation variation measure” to determine FSM parameters. In this example, the “estimated change measure” is the equation:

(5)

It may be defined according to . In the above equation, σ [ k +1] is the estimated variance measure for k+ 1, σ [ k ] is the estimated variance measure for bin k , and q1 _[ k ] and q2 _[ k ] are in equation (4) is defined, and c is a parameter (e.g., c may be a constant). In this example, a video coder may access a predefined table that maps different values of the estimated variance measure σ [ k ] to different sets of FSM parameters. Accordingly, in this example, a video coder may use the table to find FSM parameters based on the estimated variance measure. In some examples, a video coder may access different predefined tables for different coding contexts. In such examples, different entries in the predefined table for the coding context may map different values of σ [ k ] to values of FSM parameters. In some examples where the video coder accesses different predefined tables for different coding contexts, different entries in the table for the coding context may use different values of σ [ k ] and different combinations of additional information to the values of the FSM parameters. You can also map. In such examples, additional information may include slice types, quantization parameters, etc.

In some examples, a video coder may use any of the techniques as defined above to modify FSM parameters, but instead of using data generated from a previous reinitialization, the video coder may use data from other parts of the video sequence. You can also use data from For example, FIG. 11A is a block diagram showing that FSM parameters can be derived from neighboring blocks (e.g., CTUs, CUs) in the same picture along scan order. For example, in the example of FIG. 11A , FSM parameter determination units 912, 1012 may determine FSM parameters based at least in part on information from one or more neighboring blocks. The information may include prediction modes, quantization parameters, etc. In some examples, the information may include data derived from some measure of stochastic variability. For example, a single bit could be used, with 1 to indicate that significant changes in probability estimates were observed and 0 to indicate otherwise.

FIG. 11B is a block diagram showing that FSM parameters used in blocks of a current picture 1110 can be determined based on information associated with blocks in a previously coded picture 1112. Blocks in previously coded picture 1112 may have the same spatial location as blocks in current picture 1110. For example, in the example of FIG. 11B, FSM parameter determination units 912, 1012 may determine FSM parameters based at least in part on information from one or more blocks in a previously coded picture. The information may include prediction modes, quantization parameters, etc. In some examples, the information may include data derived from some measure of stochastic variability. For example, a single bit could be used, with 1 to indicate that significant changes in probability estimates were observed and 0 to indicate otherwise.

Binary probability estimation, i.e. the process of estimating the probability of a random binary data source, may involve the well-known Bernoulli (or binomial) attempts and, therefore, have been studied for decades. However, its use for entropy coding is still under development because, in practical applications, the estimation method must consider two conflicting objectives: (1) compression efficiency requires higher accuracy of probability estimates; and the ability to quickly change estimates while maintaining accuracy, which requires higher computational complexity. And, (2) because arithmetic coding speed can severely limit the throughput (in Mbits/sec) of compression and decompression systems, it is desirable for arithmetic coding to be performed with small computational complexity to increase throughput.

Because of the preference in practical video coding systems for low complexity, most methods of probability estimation for binary arithmetic coding have been used since the first practical implementations of arithmetic coding, as described in reference 11, as described in reference 10. As described above, it was based on FSM. The notation of the basics of these techniques is defined below, which may be used in video coding applications.

Binary random data source (i.e. ) a sequence of N symbols from , the true probabilities for symbol 1, that is, unknown sequence of Assuming that exists , the binary probability estimate is that the causality condition is The estimated probabilities that best approximate the true probabilities, provided that they can only depend on The problem is to find a sequence of .

In coding applications, variables and equations are modified to use only integer arithmetic. Usually, probabilities are scaled by a power of 2. For example, an integer scaling factor is used, and uppercase letters are used to indicate corresponding integer values, the scaled values of the probabilities and bins may be:

One particular type of probability estimation FSM that has been adopted, and has been "rediscovered" and renamed several times over the past few decades, is the adaptation parameter. , and recursive form:

(6)

has

In a real encoding application, video encoder 20 ?? Both video decoders 30 have the same initial probability estimate (usually from a shared fixed table). Start with , then each bean is the estimated probability are optimally encoded and decoded sequentially using , and each probability estimate is updated with equation (6) after each bin is encoded or decoded. Because this is a recursive equation, each probability value depends on all previously encoded or decoded bins.

Signal processing as described in references 16 and 17 shows that when equation (6) is used, the estimated probability values are dependent on previously coded bins, using exponentially decreasing weights. For that reason, this probability estimation technique is used in actual coding applications. It was called exponential aging in reference 12, proposed using the value of . The CABAC arithmetic coding method adopted in the AVC/H.264 and HEVC/H.265 video coding standards also provides [4, 6, 7] uses this approach, and one of the differences from previous implementations relates to its use of a finite state machine based on table look-up.

More recently, the same approach has been called exponentially decaying memory in references 13 and 14. Reference 14 called it a “virtual sliding window” technique, and it is

(7)

This is because is associated with a randomized algorithm for probability estimation, using a “sliding window” of W bins. Moreover, references 13 and 14 show that since high-complexity divisions can be replaced by efficient integer bit shifts,

(8)

We showed that it can be effectively implemented with integer arithmetic when W is a power of 2.

One practical problem with previous estimation formulas is that different window values W may be needed depending on the binary data of a given context. Either the probabilities of empty values change slowly, or the probabilities of empty values are very different (e.g. ) case, more compression is obtained with larger values of W , since they average over a large number of previous bin values. On the other hand, small values of W may be beneficial when probabilities change quickly and frequently.

One solution to this problem described in reference 15 is

(9)

Adaptation parameters of number M so that , and weights define , and in the same recursive form of equation (6), use several probability estimators,

(10)

Next, calculate the final estimate as a four-weighted average:

This approach has proven to be more efficient in video coding applications, and for that reason the current ITU/MPEG experimental video compression program uses the following three equations (using integer arithmetic) for probability estimation:

(11)

In this case, the probability estimation FSM uses Q ₁ and Q ₂ as state elements, since Q ₁ > 1/2 and Q ₂ < 1/2 can occur simultaneously or vice versa, the least likelihood symbol (least -probable-symbol; LPS) makes it difficult to use the usual technique of estimating only the probability.

Using the discrete-time signal processing described in references 16 and 17, and the definition of the z-transform, the result is:

(12)

By convention, this disclosure uses capital letters for z-transforms, but they can be recognized by using parentheses instead of brackets. For example, P [ k ] is used to denote the scaled integer version of p [ k ], while P ( z ) denotes the z-transform of p [ k ].

Using these definitions, equation (6) corresponds to:

(13)

or

(14)

This is a response where the probability estimate is applied to a sequence of empty values as shown in Figure 12.

(15)

This means that it is an output of infinite impulse response (IIR). By way of illustration, Figure 12 shows one example of probability estimation defined by equation (6).

The probability estimator of equation (10) corresponds to the parallel filter implementation shown in Figure 13, using two factors and equal weights. In Figure 13, the probability estimation filter defined by equation (10) includes two estimators and equal weights.

Accordingly, in some examples, video encoder 20 uses the example in FIGS. 12 and 13 to determine the bin probability (e.g., p[k]) of a particular bin (e.g., b[k]) used for arithmetic encoding. Operations of probability estimation filters may also be implemented. b[k] for video encoder 20 may be based on binarized syntax elements or more generally video data encoded into the bitstream. Similarly, video decoder 30 may use the probability estimation illustrated in FIGS. 12 and 13 to determine the bin probability (e.g., p[k]) of a particular bin (e.g., b[k]) used for arithmetic encoding. You can also implement the operations of filters. b[k] for video decoder 30 may be included in the bitstream from which video data is decoded.

Since entropy coding (e.g., arithmetic coding) is designed to be optimal for static data sources (e.g., non-moving content), its practical effectiveness depends on classifying data elements, so the statistics in each class are approximate. It is static. In practice, each class is represented by a coding context (or an empty context for binary alphabets), and even though each context corresponds to a different set of symbol probabilities, if the symbol probabilities change, those changes occur. It is assumed that the methods are sufficiently similar that a single adaptation method will be sufficient for all contexts.

However, in reality, in practical video coding, the data in each of the streams formed by classification also has distinct higher-order variations. For example, the classes correspond to the following cases:

데이터는 참으로 정적임;

Data is truly static;

Symbol probabilities vary frequently but by small amounts;

Symbol probabilities do not change often, but when they do, the probability values change very significantly.

Additionally, there may also be variability regarding how often data symbol probabilities change, the rate of change, and the average magnitude of the changes. In order to handle all different cases, it may be desirable to have good control of the estimation process. This disclosure describes how this is done by increasing the order of estimation and adjusting the parameters of the new estimators to optimally match the data source statistics. Additionally, the techniques described in this disclosure may solve the problem of FSM estimators that do not have a probability estimate as part of their state, making it difficult to use conventional techniques that only estimate the probability of the least likelihood symbol (LPS). .

To enable a greater degree of freedom in the predictors' responses, the techniques may use higher order filters. The IIR filter response is

. (16)

It is usually defined using a polynomial expression, such as:

However, in probability estimation problems, the parameters of the polynomial expression usually do not allow for minimal computational complexity. Additionally, best responses are known to be not numerically stable, which further complicates implementation.

This disclosure describes using the following product as a starting point from which operations used for probability estimation may be determined.

(17)

here, are the poles and zeros of H ( z ), respectively, and the constant γ is defined such that the condition H (1) = 1 is satisfied, i.e., the probability estimate is scaled appropriately.

The following steps are to take into account that the poles and zeros are expected to be approximately unity, and while it may be acceptable to have some integer products it may be useful to replace all divisions by bit shifts. Under such conditions, the filter may be defined using equation (18).

(18)

Here, to satisfy the condition H (1) = 1, it must be as follows.

(19)

Justice

(20)

can be written as:

(21)

In this cascade (product) form, it is possible to take advantage of the fact that individual filters can be arranged in any order.

With infinite accuracy, the order of the filters may be irrelevant, but the order of the filters may only matter when using finite accuracy and integer operations. For example, when parameter γ has factors different from 2, it may simply be moved to the first stage, and the integer sequence B [ k ] may be rescaled to avoid multiplications or divisions.

In some examples disclosed herein, a set of parameters are all small integers to minimize the complexity of arithmetic operations (which may be useful for custom hardware) and to simplify implementations.

In Figure 14 a diagram of the implementation is shown. For example, Figure 14 shows an example of a probability estimation filter using cascade filters. For example, Figure 14 shows a plurality of filters that each perform operations of a transfer function H _i (z) to determine respective bin probabilities of bins of an empty stream. As an example, the empty stream may be a binarized empty stream generated in video encoder 20 for encoding purposes. An empty stream may also be an empty stream that video decoder 30 generates for debinarization to generate actual video data (e.g., syntax elements).

For video decoder 30, a first filter of the plurality of filters receives values based on a bitstream (e.g., a bitstream generated by video encoder 20). The last filter of the plurality of filters outputs an empty probability. Each of the other filters receives values from its immediately preceding filter and determines values for its next filter based on the received values and the respective parameter values for each of the filters. The last filter receives values from its immediately preceding filter and determines the bin probability based on the received values and the parameter values for that last filter. As an example, the respective parameter values for each of the filters are _i , _i , _i , and It may be _i , but fewer or more parameter values are possible.

For video encoder 20, a first filter of the plurality of filters receives values based on an empty stream (generated by binarization of syntax elements). The last filter of the plurality of filters outputs an empty probability. Each of the other filters receives values from its immediately preceding filter and determines values for its next filter based on the received values and the respective parameter values for each of the filters. The last filter receives values from its immediately preceding filter and determines the bin probability based on the received values and the parameter values for that last filter. As an example, the respective parameter values for each of the filters are _i , _i , _i , and It may be _i , but fewer or more parameter values are possible.

A video coder (e.g., video encoder 20 or video decoder 30) may operate in a variety of ways. _i , _i , _i , and _i can also be determined. For example, a video coder can use many different types of information. _i , _i , _i , and You may also access one or more predefined tables that map to the values of _i . For example, a video coder _i , _i , _i , and You can also use a table associated with a specific context to find the values of _i . In these examples, the one or more tables may be determined based on empirical analysis of the video data.

In some examples, techniques of this disclosure may be implemented using entropy encoding unit 218 (FIG. 2). For example, video encoder 20 may receive video data. Video data may include one or more pictures. Additionally, prediction processing unit 200, quantization unit 206, and potentially other components of video encoder 20 may generate syntax elements based on the video data. In this example, entropy encoding unit 218 may receive a syntax element based on video data and apply binary arithmetic encoding to the syntax element. In some examples, applying binary arithmetic encoding may include generating an empty stream by binarizing a syntax element and determining an empty probability for the last one bin of the empty stream with a plurality of filters. The first filter of the plurality of filters receives values based on the empty stream, the last filter of the plurality of filters outputs the empty probability, each of the other filters receives values from its immediately preceding filter, and the received Determine the values for its next filter based on the respective parameter values for each of the values and filters, the last filter receiving values from its immediately preceding filter, the received values and the parameters for that last filter. Determine the bin probability based on the values. Applying binary arithmetic encoding may also include generating a bitstream based on at least one bin and bin probability. Entropy encoding unit 218 may output a bitstream.

Moreover, in some examples, the techniques of this disclosure may be implemented using entropy decoding unit 300 (FIG. 3). For example, entropy decoding unit 300 may determine a decoded syntax element by applying binary arithmetic decoding to the bitstream. Applying binary arithmetic decoding includes generating an empty stream, and generating the empty stream includes determining an empty probability for the last one bin of the empty stream with a plurality of filters. The first of the plurality of filters receives values based on the bitstream, the last filter of the plurality of filters outputs an empty probability, and each of the other filters receives values from its immediately preceding filter, and the received Determine the values for its next filter based on the respective parameter values for each of the values and filters, the last filter receiving values from its immediately preceding filter, the received values and the parameters for that last filter. Determine the bin probability based on the values. Entropy decoding unit 300 may debinarize the empty stream to form decoded syntax elements. Predictive processing unit 302 may reconstruct a picture of video data based in part on decoded syntax elements.

The order of operations may also be important when direct access to the probability estimate is useful for quickly determining the minimum likelihood symbol. For example, the following equations define a probability estimate FSM parameterized by positive integer constants a and b, where the probability sequence P [ k ] is part of the minimum FSM state.

(22)

A video coder may determine the values of FSM parameters a and b using any of the examples provided elsewhere in this disclosure. Moreover, in some examples, a video coder may use equation (22) instead of equation (4).

Figure 15 is a block diagram illustrating an example entropy encoding unit. For example, Figure 15 shows one example of entropy encoding unit 218 of Figure 2 in more detail. Entropy encoding unit 218 may include additional or fewer components, and the specific interconnections illustrated in FIG. 15 are for ease of understanding only and should not be considered limiting. Entropy encoding unit 218 may include a binarization circuit 1500 that receives video data (e.g., a syntax element) and performs a binarization process (e.g., a sequence of binary data symbols, also called bin values). Probability estimation circuit 1502 may receive bin values generated by binarization circuit 1500 and use example techniques described in this disclosure, such as using the cascade filtering technique illustrated and described with respect to FIG. 14. Thus, the probability for that bin can be determined. Arithmetic encoder circuit 1504 encodes a binary code to binarization circuit 1500 based on coding contexts (e.g., probabilities) generated by probability estimation circuit 1502 to generate a bitstream that video encoder 20 outputs. Binarized data generated by can also be entropy-encoded.

As an example, probability estimation circuitry 1502 may determine the bin probability for at least one bin of the bin stream with a plurality of filters (e.g., H _i (z)). A first filter (eg, H ₀ (z)) of the plurality of filters receives values based on the empty stream. The last filter of the plurality of filters (eg, H _F (z)) outputs an empty probability. Each of the other filters (e.g. H ₁ (z) to H _F-1 (z)) receives values from its immediately preceding filter, and sets the received values and the respective parameter values for each of the filters (e.g. , _i , _i , _i , and _i ) to determine the values for its next filter. The last filter receives values from its immediately preceding filter, and sets the received values and the parameter values for that last filter (e.g. _F , _F , _F , and _F ) determine the empty probability based on

The transfer function applied by the first filter to generate its values is Includes, where: contains parameters for the first filter, and z is a variable for the z-transform. The transfer function applied by each of the filters subsequent to the first filter is , where i represents the relative order of the filters, where: _i , _i , _i , and _i is the respective parameter values for the filters. _i , _i , _i , and Each one of _i may be an integer, and in some examples, _i , _i , _i , and Each of _i may be a relatively small integer.

Figure 16 is a block diagram illustrating an example entropy decoding unit. For example, Figure 16 shows one example of the entropy encoding unit 300 of Figure 3 in more detail. Entropy decoding unit 300 may include additional or fewer components, and the specific interconnections illustrated in FIG. 16 are for ease of understanding only and should not be considered limiting. Entropy decoding unit 300 is an arithmetic decoder circuit that receives a bitstream (e.g., encoded syntax elements) and performs arithmetic decoding based on coding contexts (e.g., probabilities) determined by probability estimation circuitry 1606. (1604) may also be included. As one example, probability estimation circuitry 1606 may perform some form of the operations illustrated in FIG. 14 or the inverse of those operations. Inverse binarization circuit 1600 receives the output from arithmetic decoder circuit 1604 and performs an inverse binarization process to generate video data used to reconstruct the pictures.

As an example, probability estimation circuit 1606 determines the bin probability for at least one bin of the bin stream with a plurality of filters (e.g., H _i (z)). A first filter (eg, H ₀ (z)) of the plurality of filters receives values based on the bitstream. The last filter of the plurality of filters (eg, H _F (z)) outputs an empty probability. Each of the other filters (e.g. H ₁ (z) to H _F-1 (z)) receives values from its immediately preceding filter, and sets the received values and the respective parameter values for each of the filters (e.g. , _i , _i , _i , and _i ) to determine the values for its next filter. The last filter receives values from its immediately preceding filter, and sets the received values and the parameter values for that last filter (e.g. _F , _F , _F , and _F ) determine the empty probability based on

The transfer function applied by the first filter to generate its values is Includes, where: contains parameters for the first filter, and z is a variable for the z-transform. The transfer function applied by each of the filters subsequent to the first filter is , where i represents the relative order of the filters, where: _i , _i , _i , and _i is the respective parameter values for the filters. _i , _i , _i , and Each one of _i may be an integer, and in some examples, _i , _i , _i , and Each of _i may be a relatively small integer.

The following numbered paragraphs describe specific examples according to the techniques of this disclosure.

Example 1. A method of decoding video data, the method comprising determining a decoded syntax element by applying binary arithmetic decoding to a bitstream, wherein applying the binary arithmetic decoding comprises determining a decoded syntax element: Generating the empty stream includes: determining an empty probability for at least one bin of the empty stream with a plurality of filters, wherein a first filter of the plurality of filters is based on the empty stream. Receives values, the last filter of the plurality of filters outputs an empty probability, and each of the other filters receives values from its immediately preceding filter, based on the received values and the respective parameter values for each of the filters. determines the values for its next filter, and the last filter receives values from its immediately preceding filter and determines the bin probability based on the received values and the parameter values for the last filter. to decide; and debinarizing the empty stream to form decoded syntax elements, wherein the method also includes reconstructing a picture of the video data based in part on the decoded syntax elements.

Example 2. The method of Example 1, wherein the plurality of filters are arranged in a cascade configuration.

Example 3. The method of Examples 1 and 2, wherein the transfer function applied by the first filter to generate its values is Includes, where: contains parameters for the first filter, and z is a variable for the z-transform.

Example 4. The method of any of Examples 1-3, wherein the transfer function applied by each of the filters subsequent to the first filter is , where i represents the relative order of the filters, where: _i , _i , _i , and _i is the respective parameter values for the filters.

Example 5. In the method of Example 4, _i , _i , _i , and Each one of _i is an integer.

Example 6. In the method of Example 4, _i , _i , _i , and Each one of _i is a relatively small integer.

Example 7. The method of any of Examples 1 to 6 is:

and determining a minimum likelihood symbol based at least in part on performing the operations of

Example 8. A method of encoding video data, the method comprising receiving a syntax element based on the video data; applying a binary arithmetic encoding to a syntax element, wherein applying the binary arithmetic encoding comprises: binarizing the syntax element, thereby producing an empty stream; Determining an empty probability for at least one bin of an empty stream with a plurality of filters, wherein a first filter of the plurality of filters receives values based on the empty stream and a last filter of the plurality of filters receives the empty probability. Each of the other filters receives values from its immediately preceding filter, determines values for its next filter based on the received values and the respective parameter values for each of the filters, and the last filter receives values from its immediately preceding filter and determines the bin probability based on the received values and parameter values for the last filter; and generating a bitstream based on the at least one bin and the bin probability, wherein the method also includes outputting the bitstream.

Example 9. The method of Example 8, wherein the plurality of filters are arranged in a cascade configuration.

Example 10. The method of Examples 8 and 9, wherein the transfer function applied by the first filter to generate its values is Includes, where: contains parameters for the first filter, and z is a variable for the z-transform.

Example 11. The method of any of Examples 8-10, wherein the transfer function applied by each of the filters subsequent to the first filter is , where i represents the relative order of the filters, where: _i , _i , _i , and _i is the respective parameter values for the filters.

Example 12. In the method of Example 11, _i , _i , _i , and Each one of _i is an integer.

Example 13. In the method of Example 11, _i , _i , _i , and Each one of _i is a relatively small integer.

Example 14. The method of any of Examples 8 to 13 is:

and determining a minimum likelihood symbol based at least in part on performing the operations of

FIG. 17 is a flow chart illustrating example operation of video encoder 20, in accordance with one or more techniques of this disclosure. The flowcharts of this disclosure are provided as examples. Other examples according to the techniques of this disclosure may involve more, fewer, or different actions. Moreover, in some examples, certain actions may be performed in different orders or in parallel.

In the example of FIG. 17 , video encoder 20 may generate a syntax element based on the video data (1700). For example, video encoder 20 may generate a syntax element indicating whether the residual value is greater than 1, a syntax element indicating whether the residual value is greater than 2, or another type of syntax element.

Additionally, video encoder 20 may determine the offset value, at least in part, by applying binary arithmetic encoding (e.g., CABAC encoding) to the syntax element (1702). As part of applying binary arithmetic encoding to a syntax element, video encoder 20 may generate an empty stream by at least partially binarizing the syntax element (1704). Video encoder 20 may binarize syntax elements in various ways. For example, video encoder 20 may binarize syntax elements using a Truncated Rice binarization process, a k-order Exp-Golomb binarization process, a fixed length binarization process, or another type of binarization process. In some examples, video encoder 20 uses different binarization processes for different types of syntax elements. In some examples, a bin stream may include bins created by binarizing multiple syntax elements.

Additionally, for each bin of at least one bin stream, video encoder 20 may determine an interval for the next bin of the bin stream (1706). In this example, video encoder 20 may determine the state for each bin, the interval for each bin, and the interval for the next bin based on the value of each bin. The state for each bin corresponds to an estimate of the probability of each bin being a first value and an estimate of the probability of each bin being a second, different value. If each bin is the first bin after reinitialization, the state for each bin is equal to the initial probability estimates associated with the coding context.

In some examples, video encoder 20 may perform the following actions on each individual bin of the bin stream as part of determining the interval for the next bin. In particular, video encoder 20 may split the interval for each bin into an interval associated with the first symbol and an interval associated with the second symbol, based on the state for each bin. Additionally, video encoder 20 determines the upper bound or lower bound of the interval for the next bin based on whether the value of each bin is equal to the first symbol or the second symbol. bound) can also be set. In response to determining that the value of each bin is equal to the first symbol, the upper limit of the interval for the next bin is set to the upper limit of the interval associated with the first symbol and the lower limit of the interval for the next bin is unchanged. No. In response to determining that the value of each bin is equal to the second symbol, the lower limit of the interval for the next bin is set to the lower limit of the interval associated with the second symbol and the upper limit of the interval for the next bin is unchanged. No.

For example, if each bin is the first bin after reinitialization, the interval for each bin is 0 to 1. In this example, if the state for the first bin indicates that symbol 0 has probability 0.6 and symbol 1 has probability 0.4, then video encoder 20 sets the interval for the first bin to be between 0 and 0.6 and 0.6. It can also be divided into intervals from 1 to 1. If the value of the first bin is 0, video encoder 20 may set the upper limit of the spacing for the second bin to 0.6 and may set the lower limit of the spacing for the second bin to 0. If the value of the first bin is 1, video encoder 20 may set the lower limit of the spacing for the second bin to 0.6 and may set the upper limit of the spacing for the second bin to 1. Subsequently, if the status for the second bin indicates that symbol 0 has probability 0.7 and symbol 1 has probability 0.3, and that the interval for the second bin is 0 to 0.6, video encoder 20 may The interval for may be divided into an interval of 0 to 0.42 and an interval of 0.42 to 0.6. If the value of the second bin is 0, video encoder 20 may set the upper limit of the spacing for the third bin to 0.42 and may set the lower limit of the spacing for the third bin to 0. If the value of the second bin is 1, video encoder 20 may set the lower limit of the spacing for the third bin to 0.42 and the upper limit of the spacing for the second bin to 0.6. In some examples, video encoder 20 may use integer values instead of values between 0 and 1.

Video encoder 20 may also determine one or more FSM parameters for the next bin of the bin stream (1708). One or more FSM parameters for the next bin control how probability estimates for the next bin are calculated from the state for each bin. In some cases, one or more FSM parameters for a first bin (eg, current bin) of an empty stream are different from one or more FSM parameters for a second bin (eg, next bin) of an empty stream.

Video encoder 20 may determine one or more FSM parameters according to any of the examples provided elsewhere in this disclosure. Illustratively, in one example, as part of determining one or more FSM parameters for the next bin of an bin stream, video encoder 20 may determine one or more FSM parameters for the next bin of an bin stream according to a state reinitialization parameter. You can also reinitialize them.

In some examples, video encoder 20 modifies the FSM parameters for the next bin of the bin stream according to the estimated probability values. In some examples, video encoder 20 modifies the FSM parameters for the next bin of the bin stream based on a measure of past probability variation. In some examples, video encoder 20 determines one or more FSM parameters for the next bin of the bin stream based on one or more neighboring blocks in the same frame or a previously decoded frame.

In some examples, video encoder 20 performs a process to determine one or more FSM parameters for each bin of the bin stream. In other examples, video encoder 20 performs a process to determine one or more FSM parameters only for specific bins of the bin stream. Illustratively, in one example, video encoder 20 may perform a process to determine one or more FSM parameters to be used in determining probability estimates for the second bin only after reinitialization and then reinitialization. You may continue to use the same one or more FSM parameters until the event. In some examples, video encoder 20 may perform a process to determine one or more FSM parameters at different times, such as block boundaries, slice boundaries, etc.

Video encoder 20 may also use a parameterized state updating function to determine the state for the next bin of the bin stream (1710). The parameterized state updating function takes as input the state for each bin, one or more FSM parameters for the next bin in the bin stream, and the value of each bin. For example, video encoder 20 may use equation (4) or equation (22) to determine the state for the next bin.

In the example of Figure 17, the offset value may be equal to the value in the interval for the last bin of the bin stream. Video encoder 20 may output a bitstream that includes the offset value (1712). By way of example, video encoder 20 may store or transmit a bitstream to computer-readable medium 16 (FIG. 1).

FIG. 18 is a flow chart illustrating example operation of video decoder 30, in accordance with one or more techniques of this disclosure. In the example of FIG. 18, video decoder 30 may receive a bitstream and determine one or more decoded syntax elements by applying binary arithmetic decoding to an offset value included in the bitstream (1800). As part of determining one or more decoded syntax elements, video decoder 30 may generate an empty stream. This empty stream may be part of a longer empty stream or an entire empty stream. A bitstream may contain an encoded representation of video data.

Additionally, as part of generating an empty stream, video decoder 30 may perform the following actions for at least one each bin of the empty stream. In particular, video decoder 30 may determine the value of each bin (1802). Video decoder 30 may determine the value of each bin based on the state for each bin, the interval for each bin, and the offset value. Illustratively, in one example, for each respective bin of the bin stream, video decoder 30 determines the interval for each bin based, at least in part, on the state for each bin to the first symbol. The value of each bin may be determined by dividing into an interval associated with and an interval associated with the second symbol. Additionally, video decoder 30 may determine the value of each bin based on whether the offset value is in the interval associated with the first symbol or the interval associated with the second symbol. In this example, in response to determining that the offset value is within an interval associated with a first symbol, the value of each bin is equal to the first symbol. Moreover, in this example, in response to determining that the offset value is within an interval associated with the second symbol, the value of each bin is equal to the second symbol.

For example, in one particular example, the interval associated with the first bin may range from 0 to 1. In this example, the state for the first bin indicates that symbol 0 has probability 0.6, and the state for the second bin indicates that symbol 1 has probability 0.4. Accordingly, in this example, video decoder 30 may split the interval associated with the first bin into an interval associated with symbol 0, ranging from 0 to 0.6, and an interval associated with symbol 1, ranging from 0.6 to 1. It may be possible. In this example, if the offset value is between 0 and 0.6, video decoder 30 determines that the value of the first bin is equal to 0. If the offset value is between 0.6 and 1, video decoder 30 determines that the value of the first bin is equal to 1. In this example, if the value of the first bin is 0, video decoder 30 may determine that the interval for the second bin is 0 to 0.6. If the value of bin 1 is 1, video decoder 30 may determine that the interval for bin 2 is between 0.6 and 1. Additionally, assuming that the value of the first bin is equal to 1 and the state for the second bin is that the second bin has a probability of 0.7 for symbol 0 and a probability of 0.3 for symbol 1, video decoder 30 can 1 may be divided into an interval from 0.6 to 0.88 corresponding to symbol 0 and an interval from 0.88 to 1 corresponding to symbol 0. Accordingly, if the offset value is between 0.6 and 0.88, video decoder 30 determines that the value of the second bin is 0. If the offset value is between 0.88 and 1, video decoder 30 determines that the value of the second bin is 1. Video decoder 30 may continue this process for each bin of the bin stream. In some examples, video decoder 30 may use integer values instead of values between 0 and 1.

In the example of FIG. 18, video decoder 30 may determine one or more FSM parameters for the next bin of the bin stream (1804). One or more FSM parameters for the next bin control how probability estimates for the next bin are calculated from the state for each bin. The next bean in the empty stream follows each of the above bins in the empty stream. In some cases, one or more FSM parameters for a first bin (eg, current bin) of an empty stream are different from one or more FSM parameters for a second bin (eg, next bin) of an empty stream.

Video decoder 30 may determine one or more FSM parameters according to any of the examples described elsewhere in this disclosure. For example, as part of determining one or more FSM parameters for the next bin, video decoder 30 may reinitialize the FSM parameters for the next bin of the bin stream according to the state reinitialization parameter. In some examples, as part of determining one or more FSM parameters for the next bin, video decoder 30 may modify the FSM parameters for the next bin of the bin stream according to the estimated probability values. In some examples, as part of determining one or more FSM parameters for the next bin, video decoder 30 modifies the FSM parameters for the next bin of the bin stream based on a measure of past probability variation. In some examples, as part of determining one or more FSM parameters for the next bin of the bin stream, video decoder 30 may determine the FSM parameters of the bin stream based on one or more neighboring blocks in the same frame or a previously decoded frame. One or more FSM parameters may be determined for the next bin.

In some examples, video decoder 30 performs a process to determine one or more FSM parameters for each bin of the bin stream. In other examples, video decoder 30 performs a process to determine one or more FSM parameters only for specific bins of the bin stream. Illustratively, in one example, video decoder 30 may perform a process to determine one or more FSM parameters to be used in determining probability estimates for the second bin only after reinitialization and then reinitialization. You may continue to use the same one or more FSM parameters until the event. In some examples, video decoder 30 may perform a process to determine one or more FSM parameters at different times, such as block boundaries, slice boundaries, etc.

Video decoder 30 may also determine the status for the next bin of the bin stream (1806). Video decoder 30 may use a parameterized state updating function that takes as input the state for each bin, one or more FSM parameters for the next bin in the bin stream, and the value of each bin. For example, video encoder 20 may use equation (4) or equation (22) to determine the state for the next bin.

Additionally, video decoder 30 may debinarize the empty stream to form one or more decoded syntax elements (1808). As mentioned elsewhere in this disclosure, syntax elements may be binarized using various processes, such as a Truncated Rice binarization process, a k-order Exp-Golomb binarization process, a fixed-length binarization process, or other types of binarization processes. . These processes map the values of syntax elements to binary codes. To debinarize the empty stream, video decoder 30 may find values corresponding to binary codes in the empty stream.

In the example of FIG. 18, video decoder 30 may reconstruct a picture of video data based in part on one or more decoded syntax elements (1810). For example, if the decoded syntax elements indicate whether there are remainder values for residual data, video decoder 30 may use the remainder values to make a decision about the values of the residual samples. In this example, video decoder 30 may use the residual samples and corresponding predictive samples to reconstruct sample values of the picture, as described elsewhere in this disclosure. In some examples, the decoded syntax elements may include syntax elements that indicate whether the blocks are encoded with intra-prediction or inter-prediction. In these examples, video decoder 30 may use such syntax elements to determine whether to reconstruct blocks using intra-prediction or inter-prediction.

Certain aspects of the present disclosure have been described with respect to extensions of the HEVC standard for purposes of illustration. However, the techniques described in this disclosure may be useful for other video coding processes, including other standard or proprietary video coding processes that have not yet been developed.

A video coder as described in this disclosure may refer to a video encoder or video decoder. Similarly, a video coding unit may refer to a video encoder or video decoder. Likewise, video coding may refer to video encoding or video decoding, as applicable. In this disclosure, the phrase “based on” may represent solely based, at least partially based, or based on in some way. This disclosure may use the terms “video unit,” “video block,” or “block” to refer to one or more sample blocks and syntax structures used to code samples of one or more sample blocks. Example types of video units may include CTU, CU, PU, transformation unit (TU), macroblock, macroblock partition, etc. In some contexts, discussion of PUs may be interchangeable with discussion of macroblocks or macroblock partitions. Example types of video blocks may include coding tree blocks, coding blocks, and other types of blocks of video data.

The techniques of this disclosure include over-the-air television broadcasts, cable television transmissions, satellite television transmissions, Internet streaming video transmissions, such as Dynamic Adaptive Streaming over HTTP (DASH), data It may be applied to video coding to support any of a variety of multimedia applications, such as digital video being encoded on a storage medium, decoding of digital video stored on a data storage medium, or other applications.

Depending on the example, any specific acts or events of the techniques described herein may be performed in a different sequence, and may be added, merged, or removed entirely (e.g., all acts or events described It should be recognized that these techniques are not essential for implementation. Moreover, in certain examples, acts or events may be performed concurrently, for example, through multi-threaded processing, interrupt processing, or multiple processors, rather than sequentially.

In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media refers to a computer-readable storage medium, such as a tangible medium, such as data storage media, or any device that facilitates transfer of a computer program from one place to another, e.g., according to a communication protocol. It may also include communication media including media. In this manner, computer-readable media may generally correspond to (1) non-transitory computer-readable storage media or (2) communication media, such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processing circuits to retrieve instructions, code and/or data structures for implementation of the techniques described in this disclosure. . A computer program product may include computer-readable media.

By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or in the form of instructions or data structures. It may be used to store desired program code and may include any other medium that can be accessed by a computer. Additionally, any connection is properly termed a computer-readable medium. For example, if the Software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave. Cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, wireless, and microwave are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but are instead directed to non-transitory tangible storage media. As used herein, disk and disk include compact disk (CD), laser disk, optical disk, digital versatile disk (DVD), floppy disk, and Blu-ray disk, where disk is While disks usually reproduce data magnetically, discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

The functionality described in this disclosure may be performed by fixed-function and/or programmable processing circuitry. By way of illustration, instructions may be executed by fixed function and/or programmable processing circuitry. Such processing circuitry may include one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic. It may also include one or more processors, such as circuitry. Accordingly, the term “processor,” as used herein, may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or integrated into a combined codec. Additionally, the techniques may be fully implemented in one or more circuits or logic elements. Processing circuitry may be coupled to other components in a variety of ways. For example, the processing circuitry may be coupled to other components through internal device interconnects, wired or wireless network connections, or other communication media.

The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chip set). Various components, modules, or units are described in this disclosure to highlight functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require implementation by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit or provided by a collection of interoperable hardware units, including one or more processors as described above, together with suitable software and/or firmware. It may be possible.

Various examples have been explained. These and other examples are within the scope of the following claims.

Claims (44)

A method for decoding video data, comprising:
The above method is,
determining a decoded syntax element by applying binary arithmetic decoding to an offset value included in the bitstream, wherein applying the binary arithmetic decoding comprises:
By creating an empty stream, creating the empty stream includes:
For at least one each bin of the bin stream,
determining a value for each bin based on the state for each bin, the interval for each bin, and the offset value;
Determining one or more finite state machine (FSM) parameters for the next bin of the bin stream, wherein the one or more FSM parameters for the next bin determine how the probability estimates for the next bin are derived from the state for each bin. determining the one or more FSM parameters that control whether the next bin of the bin stream follows the respective bin in the bin stream; and
update the bin stream using a parameterized state updating function that takes as input the state for each bin, the one or more FSM parameters for the next bin of the bin stream, and the value of each bin. Determining the state for the next bin of the bin stream includes:
The following expressions:

and determining a status for the next bin according to
Here, p[k+1] is the state for the next bin of the bin stream, p[k] is the state for each bin, b[k] is the value of each bin, and a is determining a state for the next bin of the bin stream, wherein b is a first parameter of the one or more FSM parameters for the next bin, and b is a second parameter of the one or more FSM parameters for the next bin.
generating the empty stream, including: and
debinarizing the empty stream to form the decoded syntax element
determining the decoded syntax element, comprising: and
A method of decoding video data, comprising reconstructing a picture of the video data based in part on the decoded syntax elements. According to claim 1,
Determining one or more FSM parameters for the next bin of the bin stream includes:
A method of decoding video data, comprising reinitializing the FSM parameters for the next bin of the bin stream according to a state reinitialization parameter. According to claim 1,
Determining one or more FSM parameters for the next bin of the bin stream includes:
A method for decoding video data, comprising modifying the FSM parameters for the next bin of the bin stream according to estimated probability values. According to claim 1,
Determining one or more FSM parameters for a next bin of the bin stream includes modifying the FSM parameters for the next bin based on a measure of past probability variation. According to claim 4,
The measure of the past probability fluctuation is,

It is calculated by summing the absolute differences between the estimated probabilities using the estimated variance measure for a particular bin defined as
where σ[k+1] is the estimated variance measure for the next bin of the bin stream, σ[k] is the estimated variance measure for the specific bin, and q ₁ [k] is the estimated variance measure for the specific bin. is a first probability estimate, q ₂ [k] is a second probability estimate for the specific bin, and c is a parameter. According to claim 1,
Determining one or more FSM parameters for the next bin of the bin stream includes:
A method for decoding video data, comprising determining the one or more FSM parameters for the next bin of the bin stream based on one or more neighboring blocks in the same frame or a previously decoded frame. delete According to claim 1,
For each individual bin of the bin stream, determining the value of each bin includes:
based on the state for each bin, dividing the interval for each bin into an interval associated with a first symbol and an interval associated with a second symbol; and
determining a value of each bin based on whether the offset value is in an interval associated with the first symbol or an interval associated with the second symbol,
In response to determining that the offset value is within an interval associated with the first symbol, the value of each bin is equal to the first symbol, and
In response to determining that the offset value is within an interval associated with the second symbol, the value of each bin is equal to the second symbol. According to claim 1,
A method for decoding video data, further comprising receiving the bitstream. According to claim 1,
The one or more FSM parameters for a first bin of the empty stream are different from the one or more FSM parameters for a second bin of the empty stream. A method of encoding video data, comprising:
The above method is,
generating a syntax element based on the video data;
determining an offset value, at least in part, by applying a binary arithmetic encoding to the syntax element, wherein applying the binary arithmetic encoding comprises:
At least in part,
Binarizing the syntax elements;
For at least one each bin of the bin stream,
determining an interval for the next bin of the bin stream based on the state for each bin, the interval for each bin, and the value of each bin;
Determining one or more finite state machine (FSM) parameters for the next bin of the bin stream, wherein the one or more FSM parameters for the next bin determine how the probability estimates for the next bin are the state for each bin. determining the one or more FSM parameters that control whether to be calculated from; and
update the bin stream using a parameterized state updating function that takes as input the state for each bin, the one or more FSM parameters for the next bin of the bin stream, and the value of each bin. Determining the state for the next bin of the bin stream includes:
The following expressions:

and determining a state for the next bin of the bin stream according to,
Here, p[k+1] is the state for the next bin of the bin stream, p[k] is the state for each bin, b[k] is the value of each bin, and a is is a first parameter of the one or more FSM parameters for the next bin of the bin stream, and b is a second parameter of the one or more FSM parameters for the next bin of the bin stream. determining the status of
generating the bin stream, wherein the offset value is equal to a value in the interval for the last bin of the bin stream; and
A method of encoding video data, comprising outputting a bitstream including the offset value. According to claim 11,
Determining one or more FSM parameters for the next bin of the bin stream includes reinitializing the FSM parameters for the next bin of the bin stream according to a state reinitialization parameter. According to claim 11,
Determining one or more FSM parameters for the next bin of the bin stream includes modifying the FSM parameters for the next bin of the bin stream according to estimated probability values. According to claim 11,
wherein determining one or more FSM parameters for the next bin of the bin stream includes modifying the FSM parameters for the next bin of the bin stream based on a measure of past probability variation. . According to claim 14,
The measure of the past probability fluctuation is,

It is calculated by summing the absolute differences between the estimated probabilities using the estimated variance measure for a particular bin defined as
where σ[k+1] is the estimated variance measure for the next bin of the bin stream, σ[k] is the estimated variance measure for the specific bin, and q ₁ [k] is the estimated variance measure for the specific bin. is a first probability estimate, q ₂ [k] is a second probability estimate for the specific bin, and c is a parameter. According to claim 11,
Determining the one or more FSM parameters for the next bin of the bin stream includes the one or more FSM parameters for the next bin of the bin stream based on one or more neighboring blocks in the same frame or a previously decoded frame. A method of encoding video data, including determining: delete According to claim 11,
For each bin of the bin stream, determining the interval to the next bin of the bin stream includes:
based on the state for each bin, dividing the interval for each bin into an interval associated with a first symbol and an interval associated with a second symbol; and
setting either an upper limit or a lower limit of the interval for the next bin based on whether the value of each bin is the same as the first symbol or the second symbol,
In response to determining that the value of each bin is equal to the first symbol, the upper limit of the interval for the next bin is set to the upper limit of the interval associated with the first symbol and the upper limit of the interval for the next bin is set to the upper limit of the interval associated with the first symbol. the lower limit of the interval is not changed, and
In response to determining that the value of each bin is equal to the second symbol, the lower limit of the interval for the next bin is set to the lower limit of the interval associated with the second symbol and the lower limit of the interval for the next bin is set to the lower limit of the interval associated with the second symbol. The method of encoding video data, wherein the upper limit of the interval is not changed. According to claim 11,
The method of encoding video data, wherein the one or more FSM parameters for a first bin of the empty stream are different from the one or more FSM parameters for a second bin of the empty stream. A device for decoding video data, comprising:
The device is,
One or more storage media configured to store video data; and
Contains one or more processors,
The one or more processors:
determining a decoded syntax element by applying binary arithmetic decoding to an offset value included in a bitstream, wherein, as part of applying the binary arithmetic decoding, the one or more processors:
Generating an empty stream, as part of generating the empty stream, the one or more processors:
For at least one each bin of the bin stream,
determine a value for each bin based on the state for each bin, the interval for each bin, and the offset value;
Determining one or more finite state machine (FSM) parameters for the next bin of the bin stream, wherein the one or more FSM parameters for the next bin determine how the probability estimates for the next bin are derived from the state for each bin. determine the one or more FSM parameters that control whether the next bin of the bin stream follows each bin in the bin stream; and
update the bin stream using a parameterized state updating function that takes as input the state for each bin, the one or more FSM parameters for the next bin of the bin stream, and the value of each bin. Determining a state for the next bin of, wherein the one or more processors:
The following expressions:

It is configured to determine the status for the next bin according to,
Here, p[k+1] is the state for the next bin of the bin stream, p[k] is the state for each bin, b[k] is the value of each bin, and a is wherein b is a first parameter of the one or more FSM parameters for the next bin, and b is a second parameter of the one or more FSM parameters for the next bin. Creates a stream,
debinarize the empty stream to form the decoded syntax elements.
determine which decoded syntax element is configured; and
An apparatus for decoding video data, configured to reconstruct a picture of the video data based in part on the decoded syntax elements. According to claim 20,
As part of determining one or more FSM parameters for the next bin of the bin stream, the one or more processors may reset the FSM parameters for the next bin of the bin stream according to a state reinitialization parameter. An apparatus for decoding video data, configured to initialize. According to claim 20,
As part of determining one or more FSM parameters for the next bin of the bin stream, the one or more processors modify the FSM parameters for the next bin of the bin stream according to estimated probability values. An apparatus for decoding video data, configured to: According to claim 20,
wherein, as part of determining the one or more FSM parameters for the next bin, the one or more processors are configured to modify the FSM parameters for the next bin based on a measure of historical probability variation. A device for decoding data. According to claim 23,
The one or more processors may be configured to:

It is configured to be calculated by summing the absolute differences between the probabilities estimated using the estimated variance measure for a specific bin defined as,
where σ[k+1] is the estimated variance measure for the next bin of the bin stream, σ[k] is the estimated variance measure for the specific bin, and q ₁ [k] is the estimated variance measure for the specific bin. is a first probability estimate, q ₂ [k] is a second probability estimate for the specific bin, and c is a parameter. According to claim 20,
As part of determining one or more FSM parameters for a next bin of the bin stream, the one or more processors may determine the FSM parameters based on one or more neighboring blocks in the same frame or a previously decoded frame. Apparatus for decoding video data, configured to determine the one or more FSM parameters for the next bin of a bin stream. delete According to claim 20,
For each respective bin of the bin stream, the one or more processors: As part of determining a value of the respective bin, the one or more processors:
Based on the state for each bin, divide the interval for each bin into an interval associated with a first symbol and an interval associated with a second symbol; and
determine the value of each bin based on whether the offset value is in an interval associated with the first symbol or an interval associated with the second symbol
It is composed,
In response to determining that the offset value is within an interval associated with the first symbol, the value of each bin is equal to the first symbol, and
In response to determining that the offset value is within an interval associated with the second symbol, the value of each bin is equal to the second symbol. According to claim 20,
The apparatus for decoding video data, wherein the one or more processors are further configured to receive the bitstream. According to claim 20,
The one or more FSM parameters for a first bin of the empty stream are different from the one or more FSM parameters for a second bin of the empty stream. According to claim 20,
The device is,
integrated circuit,
microprocessor, or
wireless communication device
A device for decoding video data, including. A device for encoding video data, comprising:
The device is,
One or more storage media configured to store video data; and
comprising one or more processing circuits coupled to the one or more storage media,
The one or more processing circuits,
generate syntax elements based on the video data;
By applying a binary arithmetic encoding to the syntax element, determining an offset value causes one or more processors to: As part of applying the binary arithmetic encoding, the one or more processors, at least in part,:
Binarizing the syntax elements; and
For at least one each bin of the bin stream,
determining an interval for the next bin of the bin stream based on the state for each bin, the interval for each bin, and the value of each bin;
Determining one or more finite state machine (FSM) parameters for the next bin of the bin stream, wherein the one or more FSM parameters for the next bin determine how the probability estimates for the next bin are the state for each bin. determining the one or more FSM parameters that control whether to be calculated from; and
update the bin stream using a parameterized state updating function that takes as input the state for each bin, the one or more FSM parameters for the next bin of the bin stream, and the value of each bin. As part of determining the state for the next bin of the bin stream, the one or more processors:
The following expressions:

configured to determine a state for the next bin of the bin stream according to
Here, p[k+1] is the state for the next bin of the bin stream, p[k] is the state for each bin, b[k] is the value of each bin, and a is is a first parameter of the one or more FSM parameters for the next bin of the bin stream, and b is a second parameter of the one or more FSM parameters for the next bin of the bin stream. determining the status of
configured to generate the empty stream by,
determine the offset value, wherein the offset value is equal to the value in the interval for the last bin of the bin stream; and
Apparatus for encoding video data, configured to output a bitstream including the offset value. According to claim 31,
As part of determining one or more FSM parameters for the next bin of the bin stream, the one or more processors may reset the FSM parameters for the next bin of the bin stream according to a state reinitialization parameter. An apparatus for encoding video data, configured to initialize. According to claim 31,
As part of determining one or more FSM parameters for the next bin of the bin stream, the one or more processors modify the FSM parameters for the next bin of the bin stream according to estimated probability values. A device for encoding video data, configured to: According to claim 31,
As part of determining one or more FSM parameters for a next bin of the bin stream, the one or more processors determine the FSM parameter for the next bin of the bin stream based on a measure of past probability variation. A device for encoding video data, configured to modify them. According to claim 34,
The one or more processors may be configured to:

It is configured to be calculated by summing the absolute differences between the probabilities estimated using the estimated variance measure for a specific bin defined as,
where σ[k+1] is the estimated variance measure for the next bin of the bin stream, σ[k] is the estimated variance measure for the specific bin, and q ₁ [k] is the estimated variance measure for the specific bin. is a first probability estimate, q ₂ [k] is a second probability estimate for the specific bin, and c is a parameter. According to claim 31,
As part of determining one or more FSM parameters for a next bin of the bin stream, the one or more processors may determine the FSM parameters based on one or more neighboring blocks in the same frame or a previously decoded frame. and determine the one or more FSM parameters for the next bin of a bin stream. delete According to claim 31,
For each bin of the bin stream, the one or more processors: As part of determining an interval for the next bin of the bin stream, the one or more processors:
Based on the state for each bin, divide the interval for each bin into an interval associated with a first symbol and an interval associated with a second symbol; and
configured to set either an upper limit or a lower limit of the interval for the next bin based on whether the value of each bin is the same as the first symbol or the second symbol,
In response to determining that the value of each bin is equal to the first symbol, the upper limit of the interval for the next bin is set to the upper limit of the interval associated with the first symbol and the upper limit of the interval for the next bin is set to the upper limit of the interval associated with the first symbol. The said lower limit of said interval is not changed,
In response to determining that the value of each bin is equal to the second symbol, the lower limit of the interval for the next bin is set to the lower limit of the interval associated with the second symbol and the lower limit of the interval for the next bin is set to the lower limit of the interval associated with the second symbol. The apparatus for encoding video data, wherein the upper limit of the interval is not changed. According to claim 31,
The one or more FSM parameters for a first bin of the empty stream are different from the one or more FSM parameters for a second bin of the empty stream. According to claim 31,
The device is,
integrated circuit,
microprocessor, or
wireless communication device
A device for encoding video data, including a. A device for decoding video data, comprising:
The device is,
Means for determining a decoded syntax element by applying binary arithmetic decoding to an offset value included in a bitstream, said applying binary arithmetic decoding comprising:
By creating an empty stream, creating the empty stream includes:
For at least one each bin of the bin stream,
determining a value for each bin based on the state for each bin, the interval for each bin, and the offset value;
Determining one or more finite state machine (FSM) parameters for the next bin of the bin stream, wherein the one or more FSM parameters for the next bin determine how the probability estimates for the next bin are derived from the state for each bin. determining the one or more FSM parameters that control whether the next bin of the bin stream follows the respective bin in the bin stream; and
update the bin stream using a parameterized state updating function that takes as input the state for each bin, the one or more FSM parameters for the next bin of the bin stream, and the value of each bin. Determining the state for the next bin of the bin stream includes:
The following expressions:

and determining a status for the next bin according to
Here, p[k+1] is the state for the next bin of the bin stream, p[k] is the state for each bin, b[k] is the value of each bin, and a is determining a state for the next bin of the bin stream, wherein b is a first parameter of the one or more FSM parameters for the next bin, and b is a second parameter of the one or more FSM parameters for the next bin.
generating the empty stream, including: and
debinarizing the empty stream to form the decoded syntax element
means for determining the decoded syntax element, comprising: and
Apparatus for decoding video data, comprising means for reconstructing a picture of the video data based in part on the decoded syntax elements. A device for encoding video data, comprising:
The device is,
means for generating syntax elements based on the video data;
means for determining an offset value, at least in part by applying a binary arithmetic encoding to the syntax element, wherein applying the binary arithmetic encoding comprises:
Binarizing the syntax elements; and
For at least one each bin of the bin stream,
determining an interval for the next bin of the bin stream based on the state for each bin, the interval for each bin, and the value of each bin;
Determining one or more finite state machine (FSM) parameters for the next bin of the bin stream, wherein the one or more FSM parameters for the next bin determine how the probability estimates for the next bin are the state for each bin. determining the one or more FSM parameters that control whether to be calculated from; and
update the bin stream using a parameterized state updating function that takes as input the state for each bin, the one or more FSM parameters for the next bin of the bin stream, and the value of each bin. Determining the state for the next bin of the bin stream includes:
The following expressions:

and determining a state for the next bin of the bin stream according to,
Here, p[k+1] is the state for the next bin of the bin stream, p[k] is the state for each bin, b[k] is the value of each bin, and a is is a first parameter of the one or more FSM parameters for the next bin of the bin stream, and b is a second parameter of the one or more FSM parameters for the next bin of the bin stream. determining the status of
and generating the empty stream by,
means for determining the offset value, wherein the offset value is equal to a value in the interval for the last bin of the bin stream; and
Apparatus for encoding video data, comprising means for outputting a bitstream containing the offset value. A computer-readable storage medium storing instructions that, when executed, cause one or more processors to:
determining a decoded syntax element by applying binary arithmetic decoding to an offset value included in a bitstream, wherein execution of the instructions as part of causing the one or more processors to apply the binary arithmetic decoding causing the one or more processors to:
Generating an empty stream causes the one or more processors to: As part of generating the empty stream, execution of the instructions causes the one or more processors to:
For at least one each bin of the bin stream,
determine a value for each bin based on the state for each bin, the interval for each bin, and the offset value;
Determining one or more finite state machine (FSM) parameters for the next bin of the bin stream, wherein the one or more FSM parameters for the next bin determine how the probability estimates for the next bin are derived from the state for each bin. determine whether the one or more FSM parameters are calculated and the next bin of the bin stream follows each bin in the bin stream; and
update the bin stream using a parameterized state updating function that takes as input the state for each bin, the one or more FSM parameters for the next bin of the bin stream, and the value of each bin. determining the state for the next bin of the bin stream, wherein determining the state for the next bin of the bin stream includes:
The following expressions:

and determining a status for the next bin according to
Here, p[k+1] is the state for the next bin of the bin stream, p[k] is the state for each bin, b[k] is the value of each bin, and a is determine a state for the next bin of the bin stream, wherein b is a first parameter of the one or more FSM parameters for the next bin, and b is a second parameter of the one or more FSM parameters for the next bin. cause to create an empty stream; and
determine the decoded syntax element, cause to debinarize the empty stream to form the decoded syntax element; and
A computer-readable storage medium that allows reconstructing a picture of video data based in part on the decoded syntax elements. A computer-readable storage medium storing instructions that, when executed, cause one or more processors to:
generate syntax elements based on video data;
By applying a binary arithmetic encoding to the syntax element, determining an offset value, as part of causing one or more processors to apply the binary arithmetic encoding, execution of the instructions causes the one or more processors to: , at least in part, causing the one or more processors to:
binarize the syntax elements; and
For at least one each bin of the bin stream,
determine an interval for the next bin of the bin stream based on the state for each bin, the interval for each bin, and the value of each bin;
Determining one or more finite state machine (FSM) parameters for the next bin of the bin stream, wherein the one or more FSM parameters for the next bin determine how the probability estimates for the next bin are the state for each bin. determine the one or more FSM parameters that control whether to calculate from; and
update the bin stream using a parameterized state updating function that takes as input the state for each bin, the one or more FSM parameters for the next bin of the bin stream, and the value of each bin. Determining the state for the next bin of the bin stream includes:
The following expressions:

and determining a state for the next bin of the bin stream according to,
Here, p[k+1] is the state for the next bin of the bin stream, p[k] is the state for each bin, b[k] is the value of each bin, and a is is a first parameter of the one or more FSM parameters for the next bin of the bin stream, and b is a second parameter of the one or more FSM parameters for the next bin of the bin stream. By making it possible to determine the state of affairs,
Create the empty stream,
the offset value determines that the offset value is equal to the value in the interval for the last bin of the bin stream; and
A computer-readable storage medium that outputs a bitstream including the offset value.